Terms of reference: End of project evaluation to analyse programming of the “Increasing Access to Quality Reproductive Health for Women and Youth in Yemen” project in MSIY.
Job Title: Consultant
Location: Yemen
Period: 4 Months (between May and August 2024)
Reporting to: Director of Operational Excellence, MSI Yemen
Budget Responsibilities (Y/N): No
Background
Yemeni women face barriers and disadvantages in nearly every aspect of their lives, including access to health services, economic opportunity, political participation, cultural norms and beliefs, physical safety and security, and control of finances. With an estimated 5.5 million women and girls of reproductive age, the decline in access to reproductive health services has severe impacts for health outcomes. In Yemen, less than half of births are assisted by skilled personnel. Every two hours, a woman loses her life during childbirth with a 75% mortality rate of preventable causes. Additionally, 1.5 million pregnant and breastfeeding women suffer severe malnutrition.
The health risks are compounded by various challenges, with 46% of Yemen's non-functioning or partially functioning facilities and insufficient health workers, electricity, medicines, supplies, and equipment shortage. Economic barriers hinder access to services and limited access due to insecurity, roadblocks, and poor socio-economic conditions. Yemen continues to have one of the highest maternal mortality rates in the Arab region (183/ 100,000) , a situation which is worsened by a lack of access to nutritious food, and a crumbling health system. Gender norms, and emerging safety issues, have further limited women’s freedom of movement and access to health care as a result.
About MSIY
Established in 1998, MSIY is focused on providing high-quality mother and child health services through a broad service package including Basic Emergency, obstetric and newborn health care (BEmONC), reproductive health (RH) care, safe delivery, obstetric ante- and post-natal care (ANC/ PNC); as well as other communicable diseases and general healthcare. MSIY does this through a network of clinics; provision of technical and quality support to private midwives and public sector healthcare professionals; and a specialised health contact centre.
About the “Increasing Access to Quality Reproductive Health for Women and Youth in Yemen” project.
Access to high quality mother and child health services including BEmONC and reproductive health services and information is core to this project where MSIY seek to increase access, broaden health choices for women and families, and raise awareness about the positive impacts of planning healthy families. The project aims to contribute to the reduction of maternal and infant mortality in Yemen through the following four result areas:
Objective 1: Better information and greater choice for people, including young people, about their health.
Objective 2: Improved access to mother and child health services including BEMONC and RH services
Objective 3: Strengthened public and private healthcare for RH service provision
Objective 4: Effective and efficient project management.
Objectives of this consultancy
To provide an independent evaluation of the project to assess relevance, effectiveness, efficiency, and sustainability of the project and make suggestions for future projects.
More specifically, the objectives of the End of Project Evaluation will be to assess:
- Progress made toward the achievements against the results framework from project inception (November 2020) to date
- Are the outputs delivered as planned?
- What have been the main challenges in the delivery of these outputs? What has the project done the overcome the main challenges encountered?
- Which activities are the most/least effective in contributing to the projects’ objectives and why?
- Relevance of the project strategies and design in the evolving context of changing socio-economic developments and realities in Yemen. Including evaluating the performance-based financing approach with public sector facilities.
- To what extent are the project activities and outputs aligned with the needs and demands of the intended beneficiaries?
- What have been the key achievements and challenges of the MSIY program at the different levels (organizational/MISY, national/institutional, local/beneficiaries)?
- What are the main factors that have facilitated or obstructed the achievement?
- What has been the nature and quality of interaction and collaboration at the national level, in particular with other partners and relevant government institutions (national and local level)?
- Are the established quality assurance mechanisms adequate?
- Are the established monitoring mechanisms adequate to identify fraud in the project?
- Did any cases of SEAH occur in the project and if so, did MSIY respond adequately?
- Has the project been implemented in a cost-efficient manner? Did the project make best use of available financial and knowledge inputs to achieve the desired results?
- Have there been any unintended changes within the program?
- Have there been any unforeseen risks occurred and what did the project do to mitigate these risks?
- What are the main lessons learned from implementation?
- Recommendations for possible (strategic) adjustments for future project design.
- Potential sustainability of the project. What steps have been taken to ensure project sustainability? Are there any contextual factors conducive to lasting outcomes?
- Recommendations for future projects
Scope of work/ methodology
The consultant will work in close consultation with MSIY team, specifically the team implementing the “Increasing Access to Quality Reproductive Health for Women and Youth in Yemen” project to:
Step 1: Inception meeting, desk review of key programme documents and key stakeholder interviews to understand the scope of the EPE. Review project theory (desk review and meetings) and stakeholder analysis.
The consultant will attend an inception meeting where s/he will be oriented on project objectives and key progresses made. At this stage of the EPE, the consultant will have the chance to meet with MSIY staff to be oriented with the project and define scope as well as design of the EPE. At this stage, MSIY will provide the consultant with key project documents for review. The documents could include agreement with donor, capacity assessment report, monitoring reports by MSIY staff etc. Documents should provide a sense of the intent of the project as well as what is occurring.
Step 2: Submission of Inception Report and finalization of methodology and evaluation design based on MSI Yemen feedback.
The inception report should include final evaluation questions, identified stakeholders for interviews and discussions, present the methodology of the EPE, and a detailed budget for the study (data collection, tools, printing, travel costs for site visit etc.) This will be finalised in agreement with MSIY.
Step 3: Data collection (mixed methods).
Interviews and focus group discussions, or questionnaire interview should focus on what stakeholders know and perceive to be true about the project. Data from project staff, documentation, and stakeholder interviews and group discussions will be used to determine plausibility of the project model. That is, data are analysed to determine the extent to which the project is properly implemented, sufficiently developed, and activities appropriate, resulting in meeting the desired project outcomes.
The consultant will also be provided secondary programme data such as routine MIS data on service delivery and other indicators in the results framework during the inception. The consultant will be required to prepare an analysis plan to examine progress against milestones which will provide insights whether the programme is on track and what questions to ask and what areas need improvement during the focus group discussions and interviews.
Step 4: Data analysis.
Collected data should be analysed. Analysis framework should be clearly explained in the report.
Step 5: Share preliminary findings and presentation draft report with MSIY, with a two-page brief.
The consultant will share preliminary findings and recommendations with MSIY at the end of the field visit and interviews with stakeholders. The consultant will draft the report and present the initial report to a group with representatives from MSIY and key stakeholders. The consultant will write a 4-page summary of the findings. Feedback will be taken into consideration and incorporated into the final report.
Step 6: Draw conclusions and make recommendations (analysis and report writing).
The consultant makes conclusions and recommendations. Conclusions and recommendations are drawn from the data. The consultant is encouraged to guard against validity threats, such as personal bias.
Step 7: Draft report.
The consultant finalises a draft report. MSIY will review the report as part of quality assurance and will share it with the reference group for their feedback.
Step 8: Finalisation of the EPE report.
The consultant will present the final draft EPE report to the stakeholders in a validation workshop. Recommendations of the EPE report will also be presented, and stakeholders will prioritise recommendations to draft action plans for the Management Response. MSIY will be responsible for finalising the Management Response and follow up of the Actions Points.
Step 9: Dissemination of EPE report and plan specific steps for utilisation of EPE data.
Based on the stakeholder analysis, MSIY will develop a dissemination plan from the beginning of the evaluation process. It will follow the principles of gender and human rights, particularly participation and inclusion of stakeholders.
Deliverables
1. Lead the whole evaluation process and manage it in a timely manner.
2. Inception report: The inception report should detail the consultant’s understanding of what is being evaluated and why, showing how each evaluation question will be answered by way of: proposed methods; proposed sources of data; and data collection procedures. The inception report should also include a proposed schedule of tasks, activities, deliverables, and budget. This inception report should also identify the sites visits and it should elaborate on the selection criteria for those sites selected.
3. Preliminary findings: The consultant shall share initial findings and recommendations with the MSIY programme team prior to the stakeholders’ consultation.
4. Draft report. The consultant shall share the first draft of the report for MSIY’s review and comments. This will include a four-page summary.
5. Facilitate and presentation of draft EPE report: A presentation will be done on the draft report and the draft report will be shared with MSIY for comments. In addition to validation of data, the consultant should draft practical recommendations in consultation with stakeholders.
6. Final EPE report: The final report will not exceed 40 pages (not including annex) in soft copy to be submitted to MSIY.
Skills and experience
The assignment will be undertaken by a qualified and experienced consultant. The consultant profile should include:
- Proven expertise and experience in leading a programme evaluation, preferably in humanitarian contexts.
- Background and experience in working on gender, reproductive health, or related fields.
- Holder of relevant academic qualifications such as a master’s degree in relevant discipline (e.g., Gender and Development, Social Sciences, Development studies, or related field).
- Fluent in both written and spoken English and Arabic.
- Experience in working in Yemen will be an advantage.
- Demonstrable experience and ability to produce a concise and high-quality report.
- Ability to work independently and remotely.
- Supports MSIY’s mission and values.
Timelines
The assignment should be undertaken between May and August 2024, and final report delivered by 28th August 2024.
1. Reporting and contractual arrangements
Include the names, contact details and contractual responsibilities of the Research Manager, or other country programme team member who will support the development and delivery of the study. This section should also include detail on the roles and responsibilities of any other people who will oversee the study, e.g. Regional Research Manager, collaborating agencies etc. Also state the rights and ownership of data and work, and extent to which documentation will be distributed.
2. Remuneration
The consultant will submit their own budget; however, you need to as a minimum indicate the payment schedule (i.e. What proportion of the payment will be paid at different stages of the research/analysis). In addition, if there are strict limits or conditions to your budget, you may choose to include some guidance here to help the consultant draw up a feasible budget. You could include a budget ceiling or range, as well as information such as:
- Costs to be covered and not covered
- How invoicing and payment will work
3. Intellectual Property Right (IPR)
The Intellectual property rights for all tools, data and reports emerging from this study is reserved with MSIY. The consultant may neither disclose, share nor publish in part or full anything related to the study without prior written permission.