Evaluator

Nature of the consultancy: 

This is an international, home-based consultancy for the final evaluation of the Project “Improving Access to healthcare for Refugees and People Displaced from Ukraine Benefiting of Temporary Protection in EU Member States".

Project Context and Scope: 

The International Organization for Migration (IOM), established in 1951, is the leading intergovernmental organization dedicated to promoting humane and orderly migration for the benefit of all. It does so by providing services and advice to governments and migrants in the frameworks of the IOM Strategic Plan 2024-2028’s three strategic objectives: Saving lives and protecting people on the move; Driving solutions to displacement; and Facilitating pathways for regular migration.

IOM works to improve the access to health care services for refugees and people displaced from Ukraine under the Temporary Protection Directive (TPD) activated by the Council of the EU following the onset of the war in Ukraine on 24 February 2022. Funded by the European Commission under the EU4Health Programme, the project is part of the broader response to Ukraine post-displacement crisis within the EU TPD framework. Through this project, IOM has been supporting Member States most affected by the Ukraine crisis to strengthen the capacity of their national health systems. The programme focuses on helping health systems cope with the additional inflow of displaced populations, improving access for patients with specific needs during a transitional period until safe return is possible, and facilitating navigation through the receiving health systems. 

The overall objective is to contribute to the reinforcement of EU health systems in need of support to enable access to services for displaced populations from Ukraine. This goal is to be achieved through the realization of the following interdependent outcomes:

• Strengthened capacity of health systems to cope with the health needs of displaced populations;
• Contribution to the fight against health inequalities in the Union, increasing the care for vulnerable populations; and 
• Increased health security in the EU by guaranteeing access to the necessary healthcare, prevention and treatment of diseases for all.

Ultimately, the programme aims to contribute to the resilient health systems and healthcare infrastructure, ensuring inclusive accessibility to healthcare and continuity of care for the most vulnerable populations, particularly refugees and displaced populations within the EU.

The programme is implemented across the 10 countries most affected by Ukraine post-crisis displacement, including Bulgaria, Czechia, Estonia, Hungary, Latvia, Lithuania, Poland, Republic of Moldova, Romania and Slovakia. Specific programme components are implemented in close coordination with the World Health Organization (WHO) and national health authorities. These include: the recruitment, training, and deployment of health mediators; the establishment of a health professional network and provision of vocational training for health workers; and Risk Communication and Community Engagement (RCCE) activities to promote health literacy and integration. Health promotion materials and campaigns are also developed in the languages of the targeted populations to ensure inclusivity and effectiveness. The project design includes a systematic approach to documenting its success and lessons learnt that can inform replication and/or scale up of the whole or part of the interventions in similar settings. 

IOM’s work on health systems resilience aligns with global frameworks such as the WHO Global Action Plan on Refugee and Migrant Health, the Global Compact for Migration, and the Sustainable Development Goals (notably SDG 3 – Good Health and Well-being, and SDG 10 – Reduced Inequalities).

Evaluation Objective:

This evaluation was selected based on its alignment with IOM strategic priorities, its high relevance to Member States due to its EU-wide implementation, and its potential to inform future scalable models for health system resilience in displacement settings. The evaluation aims to identify operational and policy-level lessons that could guide similar initiatives across other regions facing displacement-related health shocks. The overall objective of this evaluation is to assess the strategic relevance and effectiveness of the IOM work in strengthening resilience of health systems for displaced populations focusing on countries hosting Ukraine refugees.

The evaluation will specifically: 

• Assess the relevance of IOM’s EU4Health project in addressing the needs of displaced populations and national health systems.
• Evaluate the effectiveness and efficiency of IOM interventions and the coordination mechanisms between IOM, other UN agencies, with particular attention to WHO, and national health authorities in diverse humanitarian settings. 
• Analyse the impact of IOM interventions on health system resilience, policy development, and international health frameworks. 
• Identify lessons learned and best practices to inform future multi-stakeholder initiatives. 
• Assess the sustainability of the IOM interventions and their potential for institutionalization across different regions. 
• Examine the coherence of the IOM’s EU4Health Project with the broader global health security, humanitarian, and migration policies, and alignment with key national and international frameworks.

This evaluation is expected to generate evidence-based insights and recommendations that will inform future health system strengthening intervention strategies, programmatic improvements, partners engagement, and advocacy effort at national, region and global level. The final evaluation report will be presented to IOM and partner UN agencies, international partners, and policymakers. Findings will be used to enhance global health governance, inform future strategic planning, and support joint IOM-WHO advocacy for stronger health systems in emergency contexts. More specifically, the evaluation is expected to generate insights on what worked well and what did not work so well and highlight unmet needs and emerging opportunities through a country-level case studies selected among the 10 countries implementing the EU4Health project  for Ukrainian refugees. The findings and recommendations from the cased study should inform current and future partners’ decisions on potential further programme development.

Evaluation scope:

The evaluation will cover: 

• Geographic scope: IOM interventions in 10 countries implementing the EU4Health project namely, Bulgaria, Czechia, Estonia, Hungary, Latvia, Lithuania, Poland, Republic of Moldova, Romania, Slovakia. Specific case studies will be selected to generate deeper insights, potential good practices and lessons learnt. 
• Thematic scope: Health system strengthening, regional and national health governance, multi-stakeholder coordination, policy impact, and health service accessibility for displaced populations. 
• Period: project implementation period October 2023 to December 2025.

Evaluation criteria: 

The evaluation will be structured around OECD-DAC criteria: relevance, effectiveness, efficiency, coherence, impact, and sustainability.

Evaluation questions:

Relevance:

• How well did the  EU4Health project address the pressing health needs of refugees and people displaced from Ukraine in Europe?
• How well do the project aligns with internal (IOM Strategic Response Plan for Ukraine) national and international migration health frameworks (e.g. SDG 3)?
• To what extent has the EU4health project adapted to meet the emerging migration health challenges associated with population displacement?
• Have the target beneficiaries (displaced populations, refugees, and host communities) been adequately consulted in the design of health interventions?

Effectiveness:

• How sound is the project Theory of Change? To what extent did the project activities translate into planned short, medium- and long-term results? What were the major factors (internal and/or external) mediating the achievement of the results?
• How well did the IOM collaboration with WHO, national authorities and other actors improve the health and well-being of the Ukrainian refugees and people displaced from Ukraine?

Efficiency:

• To what extent did the investments in digital health technologies, telemedicine, or innovative health service delivery models improve the efficiency of services for displaced populations?
• How well were the resources (funds, expertise, and time) used to undertake activities and converted into results?

Coherence:

• How well did the project align internally with other IOM initiatives but also externally with other UN agencies’ efforts in the health and migration sectors (e.g., UNHCR, UNICEF, UNDP, World Bank)?
• How well did the project align with broader humanitarian, peace, and development nexus (HPDN) objectives?
• How effectively has the project aligned with/and contributed to the development/implementation of relevant regional and national health policies and frameworks in host countries?

Impact:

• To what extent has the IOM role, through this programme, influenced national or regional policy shift? What impact prospects (intended and unintended) has the EU4health project brought about at national, regional and global levels? What were the mechanisms (or intervention components) through which the observed impacts or changes come about?
• What are the intended/unintended positive and/or negative consequences of the project and the partnerships established by IOM on health systems strengthening and health security for displaced populations (including enhancing access to health services for displaced populations?
• How has this partnership contributed to long-term institutional capacity building in national health systems, particularly in host countries?
• What are some success stories and best practices emerging from the IOM’s EU4health project that could be replicated or scaled up globally?

Sustainability:

• To what extent are the activities, mechanisms, tools and services put in motion by IOM through the project sufficiently resilient to continue beyond the project life cycle? 
• How have local health authorities, national governments, and regional bodies been involved in the institutionalization of these health interventions?
• What strategies could enhance the scalability and sustainability of IOM’s EU4health project? 

Cross-cutting issues: 

• To what extent did the EU4Health project mainstream the needs of diverse groups including vulnerable women, children, older persons and persons with disabilities? Are there differences in health service access among these groups?
• Given the sensitive nature of health service delivery to vulnerable populations and IOM’s prominent role in this space, the evaluation will also consider potential reputational and operational risks associated with project implementation and stakeholder expectations.

Evaluation methodology: 

The evaluation is expected to follow a mixed methods approach. A detailed methodology for the evaluation is to be defined by the selected evaluation consultant(s) in the Inception Report. This section provides general methodological requirements to inform proposals from interested consultant(s) and should not be taken as a full description of the evaluation methodology. The proposed methodology can entail  a mix of methods , including but not limited to documentation review, structured and semi-structured interviews with key informants and focus group discussions (targeting IOM staff, partners and external stakeholders, such as beneficiaries, community representatives, government authorities, Member States etc.), surveys, and relevant other methods. 

Interviews for the collection of qualitative data can be carried out both in person and remotely (by phone, MS Teams, via email or through similar means). The data collection instruments should be annexed to the Inception Report for review by IOM and the RG and should include also accompanying interview protocols or moderation instructions in addition to questions or themes for discussion. The selected consultant(s) shall be responsible for the accurate transcription of interviews, following standards (e.g. intelligent verbatim) agreed upon during the inception phase. 

The proposed methodological design of the evaluation should prominently feature mixed methods to increase the credibility and validity of evaluation findings by allowing triangulation of data and findings, review by peers, and by methodically ensuring cross-comparison of findings from various data strands. Where the use of different methods and data is foreseen, it is essential that the proposed methodology elaborates in depth on:

• Purpose of methods mixing
• What data will be collected, from whom and why.
• Strengths and limitations of each source of data, including possible sources of bias, direction of bias and mitigation measures.
• How the results and findings from the analysis of different datasets or types of data will be compared and contrasted to increase validity.
• The specific sequence of data collection and analysis tasks, for each type of data and method intended to be used, showing clearly how results from one data strand or method shall inform the development of data collection instruments and procedures for data to be collected subsequently, as well as their analysis.

During the inception phase, IOM will provide indications or guidelines for the selection of the case studies, based on methodological criteria agreed with the selected consultant(s). The Inception report will elaborate on the analytical framework to be used for the case studies, which may replicate the same structure of the core evaluation or be based on specific criteria and questions. The potential methodological limitations of the evaluation should be elaborated upon in the Inception report, along with the related mitigation measures.

Case studies: a dedicated comparative in-depth assessments of selected EU4healthcountry-level interventions’ case studies will be undertaken to generate deeper insights on what worked well and what did not work well, in what contexts and identify lessons learnt and good practices.

Ethics, norms and standards for evaluation:

IOM abides by the Norms and Standards of the UN Evaluation Group (UNEG) and expects all evaluation stakeholders, including external Consultants and Service Providers, to be familiar with the UNEG Ethical guidelines for evaluation and the UNEG code of conduct for evaluation in the UN System. UNEG and CED policy and technical references are available in the IOM Evaluation Webpage. 

The selected consultant(s) will be required to comply with IOM Data Protection principles and guidelines throughout the evaluation process and beyond. All issues pertaining to confidentiality requirements and anonymization for any type of primary or secondary data used shall be closely coordinated with IOM. Requirements for the storage, organization and access to any data collected for this evaluation will be defined by IOM.

IOM is committed to gender and disability inclusion and CED supports IOM regional and country offices in the inclusion of such considerations into decentralized IOM evaluations. For the current evaluation, the selected consultant(s) will consider the gender and disability mainstreaming within the UN system and especially in evaluation, with specific reference to the UN-SWAP 3.0 framework and its technical guidance on the Evaluation indicator, as well as UNEG Guidance on Integrating Disability Inclusion in Evaluations. The selected consultant(s) are required to:

• Include a dedicated section of the Inception Report on how gender and disability considerations will be addressed in the evaluation as well as any specific methodological adjustment or hurdle that may be reasonably foreseen.
• During the implementation of the evaluation, promptly flag to IOM any challenge encountered or difficulty that may prevent adequate inclusion of gender and disability consideration in the final report of the evaluations.
• Upon submission of the first complete draft of the final report, provide a detailed self-assessment of the inclusion of gender and disability considerations into the evaluation for verification by IOM, and follow up on any issue preventing the evaluation from reaching a satisfactory rating.

All evaluation deliverables submitted by the selected consultant(s) are required to be copyedited and to comply with IOM House Style Manual and with the IOM Brand Guidelines. Prospective consultant(s) are recommended to budget for professional editing services, to ensure that the professionals they resort to are familiar with the above-mentioned IOM style references and guidelines.

Organizational Department / Unit to which the Consultant is contributing: 

Commissioned by: IOM Central Evaluation Division 

Managed by: IOM Sub-Regional Office Brussels

The evaluation will be conducted by external consultant(s) under the guidance and responsibility of IOM Sub-Regional Office Brussels. The main counterparts for the evaluation are the IOM, Country Offices, national implementing partners and national authorities in target countries. The counterparts will provide support during the conduct of evaluation and appoint members to the reference group (RG) for the evaluation. 

The RG will meet regularly (from the inception stage and during the conduct of the exercise) and share relevant documentation, identify internal and external structures, processes, policies, strategies, and programmatic approaches necessary for the successful conduct of the evaluation. The RG will provide technical feedback on the evaluation deliverables, i.e. terms of reference, the inception report, and the evaluation report. In collaboration with the IOM SRO Brussels the RG will propose the list of key informants to be interviewed inside and outside of IOM, which will be finalized in coordination with the consultant(s). The RG will also work with the consultant to finalize the relevant samples from which to collect the needed data, including survey sample, FGD participants etc.

Tangible and measurable outputs of the work assignment 

The consultant is expected to delivery the following six deliverables according to the following indicative schedule, between July and December 2025, which will be updated as needed to ensure a smooth evaluation process.

First deliverable: Inception Report
Including methodology, data collection protocol, evaluation matrix and draft tools as well as evaluation stakeholder mapping

Second deliverable: Interim Findings Report. 
Presentation of preliminary insights to identify any evidence gaps and red flags. Validation workshop with all key stakeholders

Third deliverable: Draft Evaluation Report  
Comprehensive assessment with findings, conclusions, and recommendations

Fourth deliverable: Final Evaluation Report  
Incorporating feedback and providing concrete and strategic recommendations for global policy and operational improvements

Fifth deliverable: Evaluation Brief that provides a succinct summary of key findings, recommendations, lessons learnt, if any, for high-level stakeholder audience.

Performance indicators for the evaluation of results:

Quality and timely submission of deliverables.

Education, Experience and/or skills required

Interested independent evaluation consultants are expected to meet the following qualifications:

• Master's degree in social science, international development, demography or Public Health or in Evaluation
• Expertise in global health, migration, and displacement issues.
• Proven experience in evaluating multi-country and multi-agency partnerships (preferably involving UN organizations).
• 7 years of experience conducting evaluations
• Strong qualitative and quantitative research skills.
• Excellent analytical and report-writing abilities.
• Proficiency in English is required and knowledge of Russian is an advantage.

Submission package

Interested candidates should submit their expression of interest by including the below as attachments:

1. Recent CV.
2. A technical proposal with description of the approach, methodology, activities, work plan with realistic timeframe in line with the ToR. 
3. An all-inclusive budget proposal in Euro that includes a detailed breakdown of costs per activity, personnel costs, and any other costs relating to the implementation of the tasks outlined in the ToR. An indicative cost can be included for potential travel to the field in participating countries. 
4. Two examples of recently completed evaluation reports
5. Three references.

Potential conflicts of interest should be declared, including having any relatives in the service of IOM.