Skip to main content

Consultant

Warning message

  • This job post is currently not published and therefore cannot be searched for or applied to!
  • This job post has expired!
Occupational fields: 
Employment type: 
Region: 
Organization: 
Will be unpublished on: 
Fri, 15/06/2018 - 17:30

Terms of Reference Programme Final Evaluation

 

Project Title: Improving Health Outcomes for Women and Children in Sierra Leone through Community Engagement and Health Systems Strengthening.

 

Background Information

Welbodi Partnership received funding from Department of International Development (DFID) which provides significant funding to civil society organisations (CSOs) annually in line with its overall strategy to alleviate poverty and promote peace, stability and good governance. The Programme Partnership Arrangements (PPA) and UK Aid Direct are two of DFID's principal funding mechanisms and have provided £480 million to approximately 230 CSOs between 2011 and 2013.

We also received funding from Comic Relief which spends 40- 50% of its overall resource mobilization helping people in UK and the rest is spent internationally with a focus on supporting programmes in Africa, Asia, Latin America and beyond.

The current political climate and results-based agenda demand a rigorous assessment of the effectiveness of funds disbursed to ensure that they are managed to provide value for money.

Introduction

Welbodi Partnership is a UK registered charity organization that has been working in Sierra Leone since 2008 with a mission to "build the capacity of the health system in Sierra Leone to reduce the number of women and children who are sick, suffer or die unnecessarily." Our work is conceived, designed, led and implemented in collaboration with local partners. Our role is to coordinate funding, international expertise and local know-how to facilitate improvements in the delivery and quality of health care. We work to build relationships and consensus with leaders within the Government of Sierra Leone (GoSL), local health facilities, donors, non-governmental organizations (NGOs) and international partners.

The Welbodi Partnership Primary Care and Community Engagement Project aims to reduce maternal and child mortality in eight city sections in the east part of the Western Area - Urban (Freetown) by organizing and coordinating women's groups with the aim of improving preventive health practices and increase demand and uptake of maternal and neonatal health services within the project operational communities. The project also organized and oversee Quality Improvement Groups to discuss barriers to the delivery of high-quality antenatal, obstetric, and postpartum care to women and infants, and to develop strategies to overcome those barriers at selected health facilities as part of its efforts to strengthen health system within the project operational areas.

This is a four- year project funded by DFID and co funded by Comic Relief. The project commenced in 1st September 2014 and will end 31st August 2018. During the past years, the community project had established women volunteer groups led by women facilitators as team leaders to promote maternal and child health education within communities. Other efforts include engagement with community leaders and stakeholders who support the project intervention and assisted in creating linkages with health facilities in the project operational areas. The project therefore worked with health facility staff in health promotion activities, joint implementation of community events and others. To support the district in strengthening efforts on primary health care, the project had established quality improvement (QI) teams in seven PHUs whose members comprised health workers, community leaders and Welbodi Partnership volunteers. These QI teams have developed strategies to address barriers affecting delivery of quality health services at health facilities. These strategies were presented in form of projects or proposals and implemented in collaboration with the project and the DHMT. There is strong collaboration between the project and the district health management team (DHMT) to promote life -saving activities targeting pregnant women, lactating mothers and children under the age of five years.

The Objectives of the final Evaluation

The independent final evaluation has two key objectives:

  1. To independently verify (and supplement where necessary), grantees' record of achievement as reported through Annual Reports and defined in the project's log frame;
  2. To assess the extent to which the project performed well and was good value for money, which includes considering;

-          How well the project met its objectives;

-          How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome;

-          What has happened because of DFID and Comic Relief funding that wouldn't have otherwise happened; and

-          How well the project aligns with DFID's and Comic Relief's goals of supporting the delivery of the MDGs

The purpose & tasks of the Consultancy

The purpose of this consultancy is to lead and deliver a project evaluation that provides findings and conclusions that are based on robust and transparent, evidence; and where necessary supplements the grantee's own data with independent research on the utilization of both grants (DFID and Comic Relief). The independent evaluation should ensure quality of the assessment design, data collection and analysis and a final report.

The independent final evaluation reports that are submitted by grantees will be used to inform the Fund Manager's understanding of the grantee's performance at the project level and will also be used to inform the Evaluation Manager's assessment of performance at the UK Aid Direct and Comic Relief fund level. The independent final evaluation report needs to be a substantial document that (a) answers all the elements of the Terms of Reference (ToR); (b) provides findings and conclusions that are based on robust and transparent evidence; and (c) where necessary supplements the grantee's own data with independent research.

Tasks for this consultancy include but are not limited to the following: Planning

  1. Review project design document for orientation and results-based framework for both grants.
  2. Liaise with Project Manager to provide project annual reports submitted to donors including other relevant documents and information
  3. Communicate logistical requirements for field assessments and meetings with key informants, etc. for the final evaluation with the Project Manager
  4. Review and comment on suggested project assessment methodology with the Project Manager and Country Director.

Verification of grantee reporting

The first task of the final evaluation is to verify grantee achievement. The record of achievement will be presented in past Annual Reports and progress against the project log frames. This exercise could include verifying information that was collected by the grantee for reporting purposes and possibly supplementing this data with additional information collected through primary and secondary research. Verifying the results from the project log frames will begin to capture what the project has achieved. However, there will be other activities and results that occur outside of the log frame that may require examination in order to respond to the different evaluation questions. Verifying reporting will also necessarily include a review of the data and systems that were used to populate results.

Assessment of value for money

The final evaluation should assess the extent to which the delivery and results of the project are good value for money. Value for money can be defined in different ways, but at minimum the evaluation report should include an assessment against:

  • How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome;
  • What has happened because of DFID and Comic Relief funding that wouldn't have otherwise happened; and
Evaluation Questions
All evaluators are encouraged to structure their research questions according to the OECD-DAC criteria of relevance, effectiveness, efficiency, sustainability and impact.
Relevance
  • To what extent did the grantee support achievement towards the MDGs, specifically off­track MDGs?
  • To what extent did the project target and reach the poor and marginalised?
  • To what extent did the project mainstream gender equality in the design and delivery of activities (and or other relevant excluded groups)?
  • How well did the project respond to the needs of target beneficiaries, including how these needs evolved over time?

Effectiveness

  • To what extent are the results that are reported a fair and accurate record of achievement?
    • To what extent has the project delivered results that are value for money? To include but not limited to:

How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome; o What has happened because of DFID and Comic Relief funding that wouldn't have otherwise happened; and o To what extent has the project used learning to improve delivery?

  • What are the key drivers and barriers affecting the delivery of results for the project?

Efficiency

  • To what extent did the grantee deliver results on time and on budget against agreed plans?
    • To what extent did the project understand cost drivers and manage these in relation to performance requirements?

Sustainability

  • To what extent has the project leveraged additional resources (financial and in-kind) from other sources?
  • What effect has this had on the scale, delivery or sustainability of activities?
  • To what extent is there evidence that the benefits delivered by the project will be sustained after the project ends?

Impact

  • To what extent and how has the project build the capacity of civil society?
  • How many people are receiving support from the project that otherwise would not have received support?
  • To what extent and how has the project affected people in ways that were not originally intended?

Evaluation methods

The successful consultant or consulting to conduct the final evaluation and the grantee are jointly responsible for choosing the methods that are the most appropriate for demonstrating impact. Evaluation methods should be rigorous yet at all times proportionate and appropriate to the context of the project intervention. Where possible, the evaluator(s) are encouraged to triangulate data sources so that findings are as robust as possible.

Suggestion on different approaches to assessing impact

Although it is not strictly mandatory, the evaluator(s) are encouraged to apply a mixed-methods approach for assessing impact. This would combine qualitative data to provide an explanation of 'why' and 'how' the project has achieved the type and scale of results that are quantitatively observed.

Assessing impact through experimental or quasi-experimental approaches

To definitively attribute impact, then the establishment of a counter factual is required: e.g. what would have happened to beneficiaries in the absence of the intervention? Evaluators are encouraged to consider the extent to which approaches, such as measuring the difference between treatment and control groups, can be successful in capturing impact while also balancing concerns relating to proportionality. It is likely that experimental or quasi- experimental approaches will not be appropriate for a large number of grantees.

Assessing impact through contribution-based research

Contribution-based approaches are helpful for overcoming the attribution issue of proving cause and effect. A contribution-based approach should result in a 'plausible' account of the difference that DFID's and Comic Relief funding has made to the impact of grantees. Such an approach is typically informed by a wide range of evidence sources that are brought together to produce a 'plausible' assessment of the 'contribution' of grantees to higher level outcomes and impacts. It is likely that such an approach will be appropriate for a large majority of grantees

Further guidance

Additional guidance concerning the fundamental principles and techniques of impact and contribution research is presented in 'Broadening the Range of Designs and Methods of Impact Evaluations' available at:

https://www.gov.uk/government/uploads/system/uploads/attachment data/file/67427/design- method-impacteval. And Annex 8.1 of the Evaluation Manager's Evaluation Strategy for the UK Aid Direct and PPA which is also public available:

Indicative materials to review

Relevant to review may include review of the grantee's original application for funding; MOU with DFID and Comic relief for funding; updated versions of organisational project log frames; monitoring data; monitoring systems, annual reports and comments provided by the Fund Manager; organisational monitoring & evaluation strategy; studies undertaken by the grantee; financial information / information on resources spent; information on synergies / collaboration with DFID and Comic Relief country programmes and other actors; published material (e.g. to demonstrate sharing of learning with others); and additional relevant documents.

 

Profile of the Independent Evaluation provider

The Independent Evaluator should be a suitably-qualified and experienced consultant or consulting firm. The consultant profile should include:

  • An evaluation specialist with a minimum of seven years' experience in programme/project evaluation in an international development context.
  • Often a mixed approach that incorporates the technical skills of an evaluation specialist but includes some inputs from a sector specialist is most effective
  • Experience of results-based monitoring and evaluation;
  • Ability to design and plan the evaluation approaches and research methodologies, including quantitative and qualitative research methods. Where feasible and proportionate, the person or team should include skills and expertise required to design, plan and conduct impact evaluation, potentially using experimental or quasi-experimental techniques
  • Relevant subject matter knowledge and experience such as education sector, gender, non­state actors and economic growth to ensure the evaluation design and research methods are as relevant and meaningful as possible given the aims and objectives of the project and the context in which it is being delivered
  • Ability to manage a potentially large-scale and complex evaluation and research process, including interpreting baseline data and conducting a final evaluation;
  • Ability to design, manage and implement primary research in potentially challenging project environments, such as fragile and conflict affected states. This may include the design of surveys, in-depth interviews, focus group and other research;
  • Design and manage data and information systems capable of handling large datasets for monitoring and evaluation purposes and;
  • Consideration of the extent to which the evaluator or evaluation team has appropriate country
  • knowledge/experience. This includes language proficiency to conduct the research required or that resources be made available (e.g. translator or social gatekeeper) to enable the research to proceed smoothly and
  • While evaluation consultants may be nominated by the grantees they must not have a conflict of interest with the on-going activities of grantees

Field Work

  • Establish evaluation framework with the Welbodi Partnership management team
  • Ensure quality control of both the English and local language (if needed) versions of the questionnaire 
  • Manage quality control of data collection and analysis
  • Complete analysis of data set including measurement of all project indicators and project requirement
  • Lead the interpretation of findings
Deliverables:

In order to minimalize the risk of not being to access the required project staff and key stakeholders it is recommended that the reports should be completed before the project closes. The main body of the report (draft and final version) must be limited to 40 pages (this can include or exclude annexes). One of the annexes should consist of a table which summarises the findings according to the OECD-DAC criteria. The report should be written in English.

A full power- point presentation of key findings and impact in project operational areas to Welbodi Partnership and partners.

In order to minimalize the risk of not being to access the required project staff and key stakeholders it is recommended that the reports should be completed before the project closes. The main body of the report (draft and final version) must be limited to 40 pages (this can include or exclude annexes). One of the annexes should consist of a table which summarises the findings according to the OECD-DAC criteria. The report should be written in English.

A full power- point presentation of key findings and impact in project operational areas to Welbodi Partnership and partners.

To ensure consistency across evaluation reports, the following structure should be used for reporting:

  1. a.Executive Summary
  2. b.Introduction
  • Purpose of the evaluation
  • Organisation context
  • Logic and assumptions of the evaluation
  • Overview of UK Aid Direct and Comic Relief funded activities respectively
  1. c.Evaluation Methodology
  • Evaluation plan
  • Strengths and weaknesses of selected design and research methods
  • Summary of problems and issues encountered
  1. d.Findings
  • Overall Results
  • Assessment of accuracy of reported results
  • Relevance
  • Effectiveness
  • Efficiency
  • Sustainability
  • Impact
  1. e.Conclusions

Summary of achievements against evaluation questions

Summary of achievements against rationale for UK Aid Direct and Comic Relief funding

Overall impact and value for money of UK Aid Direct and Comic Relief funded activities

  1. f.Lessons learnt (where relevant)
  • Project level - management, design, implementation
  • Policy level
  • Sector level
  • UK Aid Direct and Comic Relief management
    1. gRecommendations
    2. h.Annexes (such as)

    Independent final evaluation terms of reference

    Evaluation research schedule

    Evaluation framework

    Data collection tools

    List of people consulted

    List of supporting documentary information

    Details of the evaluation team

    Grantee management response to report findings and recommendations

 
 

Timeframe

The total period for this work is estimated at 20 working days including preparation, field work and write up of report. Envisage distribution of the timeframe:



Week 1

Finalize discussions, questionnaires and sampling methodology with Project manager

Week 2

Field data collection

Week 3

Field data collection continuation

Week 4

Data analysis, preparation summary tables, Reporting, debriefing to Welbodi Partnership Management and partners

All deliverables are to be submitted by 31st August 2018
 

Interested consultants must submit a technical and financial proposal detailing the following:

  1. Their understanding and interpretation of the Terms of Reference
  2. Summary of relevant experience to the assignment
  3. Methodology to be used in the undertaking of this assignment
  4. Timeline for the completion of this study
  5. Daily rate in US dollars
  6. Attach key personnel's CVs and appropriate references
  7. Submission of proposals

    The proposal can be sent only by email to: jobs@welbodipartnership.org Indicate in the subject: Evaluation Community Programme Deadline: 15th June 2018

Evaluation and Award of Consultancy

Welbodi Partnership will evaluate the proposals and award the assignment based on technical and financial feasibility. Welbodi Partnership reserves the right to accept or reject any proposal received without giving reasons and is not bound to accept the lowest, the highest or any bidder. Only the successful applicant will be contacted.


 

***Please indicate in your application that you saw this advertisement on the Careers.sl Platform***