1. Overview

The Right Here Right Now partnership, comprising Rutgers, ARROW, AMPF, CHOICE for youth and sexuality, Reproductive Health Uganda and RNW Media seek the services of an evaluator for the purpose of conducting an end-term evaluation of the Right Here Right Now programme according to the terms of reference set out herein. Key details:

Programme locations: Bangladesh, Benin, Burundi, Ethiopia, Indonesia, Kenya, Morocco, Nepal, Tunisia, Uganda, global.

Application deadline: 6 October 2024

Timeline:  1 November 2024 – December 2025

Budget range: A maximum budget of EUR 285,000 (including all taxes) is available. Out of this, approx. EUR 150,000 is expected to be spent in-country on country-specific evaluations. Please note that a cost-benefit analysis is part of the analysis of proposals.

Published: 4 september 2024

Last date to apply: 6 October 2024

The Right Here Right Now programme runs from 2020 to 2025. We are a strategic partnership consisting of six consortium members, one technical partner and nearly 50 country partners in ten countries in Africa and Asia. The end term evaluation (referred as ETE) is a key opportunity to gather evidence on our programme and to understand what works (or not) as well as lessons learned. The evidence gathered will be vital for programme partners to show the result of work, to inform future funding proposals (to the Dutch Ministry of Foreign Affairs (MOFA) or other funders) as well as the development and implementation of future programmes. Furthermore, we have seen from previous MOFA funding rounds how analysis of programme end evaluations is used to inform future policy decisions.

The partnership calls for an external, independent party to address these ETE objectives - as described in greater detail below. Further details about the programme, the application/selection process, and the timeline for the assignment are presented below.

2. Programme background

Right Here Right Now enables young people in all their diversity to enjoy their sexual and reproductive health and rights in societies where there is equity and equality between genders in all aspects of life. We place young people at the forefront, and strive towards young people, especially girls, young women and young LGBTQI+ people being empowered to make decisions about their sexuality, voice their needs and claim their rights. The partnership boldly tackles taboos. We advocate for the rights of marginalised groups because we can, based on our robust experience, and because few others do. We lobby governments to adopt and be accountable for inclusive human rights-based policies and laws which enable young people’s sexual and reproductive health and rights and gender justice. We call on governments to give young people access to comprehensive sexuality education, information and youth-friendly services – including safe abortion – so they can make choices free from stigma, discrimination and violence.

The Right Here Right Now theory of change works towards four mutually reinforcing long-term outcomes (LTOs). Our approach under each LTO (or pathway) is outlined below.

LTO 1: Empowered young people make decisions about their sexuality, voice their needs and claim their rights: Adolescents and young people need access to quality information and education adapted to their needs and preferences, to gain competencies (knowledge, attitudes, skills) on sexuality and claim their sexual and reproductive rights. We engage with youth agents and educators to provide comprehensive sexuality education to young people (in- and out-of-school, online and offline), while engaging caregivers, teachers and school management. We harness digital media platforms to connect young people to information directly – when how and where they need it. We engage and activate young people in all their diversity both on- and offline so they can claim their rights and stand up for others.

LTO 2: A critical mass reinforces positive norms and values regarding young people’s SRHR and gender justice: In order to advance acceptance of young people’s SRHR and their rights as active citizens, a critical mass - a sufficient number of mobilisers to create further support in society - is required that reinforces positive norms and values around gender and sexuality. We work with key influencers, (traditional and digital) media partners and young people in all their diversity through country-specific strategies and campaigns to strengthen knowledge, capacities and skills. In order to change harmful gender norms, we take a gender-transformative and intersectional approach.

LTO 3: Governments adopt, implement and account for human rights-based policies and laws that enable young people’s SRHR and gender justice: Realisation of the SRHR of young people in all their diversity depends on the adoption and implementation of policies and laws that ensure their rights and access to SRHR information, education and services. Therefore, together with young people, we build and influence the political will of decision makers, including policy makers, politicians and public-service providers at national and subnational levels. We work at local, national, regional and international level to hold decision makers to account for international SRHR and gender commitments.

LTO 4: Strengthened civil society for young people’s SRHR and gender justice: Our ambition under LTO4 is to contribute to (networks of) CSOs which are legitimate in representing young people in all their diversity, which embrace progressive values anchored in human rights and which are strategic in their alliances and ways of working at multiple levels for increased impact. Our work to achieve this includes mutual capacity strengthening, supporting meaningful and inclusive youth participation and addressing power dynamics. Inclusivity is a guiding principle of our programme. We aim for CSOs to be connected and well organised to seize opportunities and increase resilience when facing adversity, thereby avoiding fragmentation.

3. Objectives and scope

3.1 Objectives and overall guiding questions      

The end term evaluation of Right Here Right Now will focus on evaluation of programmatic aspects as well as collaboration between partners. ETE will be led by a successful applicant to address following key objectives, defined in careful consideration by RHRN partnership and programme partners in alignment with Ministry of Foreign Affairs Netherlands requirements[1]. Overall ETE will document the lessons of the programme and inform future work. The objectives are:

1. Evaluating the relevance and validity of the theory of change: Evaluating the relevance and validity of the theory of change: To evaluate the programmes’ theory of change, focusing on the relevance of its objectives and the validity of the assumptions underlying the ToC in different country contexts. While the primary focus will be on relevance and validity, this evaluation will also touch on effectiveness by providing insights into which mechanisms of change were effective, which were not, and why, to the extent that these insights help validate the assumptions of the ToC.

2. Evaluating the effectiveness of the programme: evaluate the results of the programme, which results were achieved, how results were achieved in different context and the contribution of the programme to these changes. The evaluation will also consider unintended or unexpected effects, the extent to which the programme reached the most marginalized rights holders, and the integration of crosscutting themes such as gender, youth, and climate. Following are more specific sub-objectives:

2.1. To assess the country programme progress towards the achievement of the 5-year targets at output and outcome level.

2.2. To assess the country programme strategies regarding:

2.2.1. Sustainability (what is the likelihood that the programme results will be sustainable, in terms of systems, institutions, policies, financing) and institutionalization (e.g. meaningful and inclusive youth participation).

2.2.2.  Operationalisation of programme principles (do we live up to our own principles?).

2.2.3.  Capacity strengthening (Is the programme sufficiently sensitive and responsive to capacity strengthening needs? Is the country capacity strengthening plan effective and likely to lead to sustained capacity improvements in the long-term?)

2.2.4.  Effectiveness of strategies against opposition to SRHR and gender, assess their contribution to partner resilience, and provide recommendations for future investments.

2.2.5. Unintended/unexpected effects and the programme’s success in reaching the most marginalized rights holders.

2.2.6. Integration of crosscutting themes such as gender, youth, and climate across all strategies and interventions.

2.3.   A mapping of funding in relation to budgets vs. expenditures at activity, thematic and geographical level, as well as an evaluation of financial resource management (e.g. appropriateness of budget planning, whether grant allocation has been appropriate, whether there were any bottlenecks in the system of financial disbursement between consortium, national lead partner and coalition partners.

2.4.  Additional country specific evaluation and/or learning objectives.

3.       Evaluating the partnership collaboration focusing on coherence, localization/leading from the South, partnership with the Ministry and the Embassies, lessons learned and good practices.

The End-Term Evaluation (ETE) will be guided by the technical requirements laid out in the IOB criteria from the Ministry of Foreign Affairs of the Netherlands and contextualized by OECD criteria (please review Annex 2). According to IOB knock-out criteria 1, the inception report, including the evaluation questions, must be first approved by the reference group. This group will also provide comments and advice on the Terms of Reference (ToR), the selection of evaluators, the detailed methodology in the inception report, and the draft evaluation report. These draft evaluation questions have been developed by the Program Monitoring Evaluation and Learning team to facilitate the ETE process. Evaluation questions will be reviewed and finalized by the Global consultant(s) for evaluation with full independence, taking into account the reference group's input and final approval. For more details on draft evaluation question please review annex 3 and annex 4 for key indicators per LTO.

3.2 Scope and approach

The ETE will look at programme implementation and processes (partnership, collaboration, governance, power balance) at both country and global level and the interlinkage between those levels.

The ETE reports will consist of a global consolidated evaluation report including sections for specific country reports. The consolidated report will include an analysis of key findings from across ten project countries as well as findings on the global level, and interlinkages between country and global levels. The successful candidates will be responsible for the quality of both the country and the consolidated reports and supporting data and analysis under supervision of the Reference Group[2].

ETE consultant is expected to work with the reference group and program team to understand and clarify scope of ETE during inception phase of the process. The ETE process will build on analysing existing data on outcomes achieved as well as collection of new data to answer evaluation questions. For example, this will include an external validation of a selection of previously harvested (and substantiated) outcomes (LTO3). This will independently inform the credibility of advocacy achievements documented over project span 2020-25. The ETE will focus on examining how RHRN activities may have contributed to observed changes, without assuming a direct causal relationship. This approach will allow for a balanced analysis, considering both evidence of contribution and scenarios where RHRN’s influence may be limited or indeterminate.  Similarly, to validate Theory of Change, ETE consultant will conduct a literature review on key assumptions underpinning ToC and validate them by collecting data from stakeholders and beneficiaries from countries and partnership. In addition to consulting program stakeholders, it is important to meet IOB quality criteria 17, which requires the evaluator to independently select and consult a sufficient number of independent sources. This includes gathering opinions from other experts or non-beneficiaries who can critically reflect on the intervention, as well as incorporating objective observations and validated secondary data. Last but not the least, ETE consultant will also collect data on collaboration between partners at country, regional and global level to assess coherence (check annexure 2 for more details on OECD criteria).

Users and uses: Considerations of the ultimate users/uses of the review should inform all evaluation decisions. We anticipate that the key audiences and end users of the report include:

• Members and partners of the partnership. The findings will be used by consortium members and in the programme countries to steer and adapt, and to identify successful strategies and lessons learned.

• Programme participants and stakeholders in each country. Engaging directly with these audiences ensures downward accountability and for learning, inspiration and motivation.

• The international field of SRHR programming, contributing to global learning and innovation concerning best practices, scalable interventions, and lessons learned.

• Furthermore, the evaluation report will be shared with the Ministry of Foreign Affairs of the Netherlands (the primary donor of the programme), relevant embassies in countries which were consulted as stakeholders and the Dutch development sector in particular.

4. Methods and approach

Paradigm for the evaluation: For the RHRN end-term evaluation, we will use the realist evaluation approach, one of the four approaches endorsed by the IOB. This method focuses on understanding what works, for whom, and under what circumstances, guided by the programme’s theory of change and the context at both country and global levels. The realist evaluation provides a framework for examining the context and mechanisms that enable a programme to succeed. It is especially useful for comprehending complex interventions and the influence of context on programmes.

The realist evaluation process involves revisiting the theory of change, mapping out assumptions about how the programme works (i.e., who is involved, what actions are taken, and what outcomes are expected), and then gathering data to verify these assumptions and understand the impact of the context. This approach offers a deeper understanding of the success factors and challenges that facilitate or hinder sustainable impact within and across all programme levels, while not aiming to capture all possible variables or provide a simple cause-and-effect analysis or broad generalizations. Unlike other approaches, the realist evaluation focuses on the nuanced interplay of context and mechanisms, rather than solely on outcomes, allowing for a more detailed exploration of how and why specific results are achieved in particular settings. Data will be collected around these assumptions to explore the ‘black box’ of initiatives, identifying which components, modules, or messages drive the most positive results and what factors may be limiting success.

Testing of assumptions (theory to inform the evaluation): During the inception phase of the end-term evaluation, each country coalition together with the Global consultant(s) and their respective national consultants, will develop their country specific set of assumptions that they want to test and evaluate in order to make the evaluation of their programme as contextualized/relevant, focused and useful as possible. Global consortium partners will work with the Global Consultant(s) to develop set of assumptions to test and evaluate based on consortium level work. Each country set will be derived from the following documents:

• the literature review on the assumptions of the global theory of change assumptions (to be conducted by the ETE consultant)

• Consultant can conduct a desk review of assumptions developed by the planning, monitoring, evaluation, and learning (PMEL) technical working group during the program. The assumptions developed by the PMEL technical working group is based on an iterative and formative process undertaken input from the programme staff.  The assumptions as developed during the programme proposal of the RHRN theory of change can be found in annex 1. Furthermore, a scoping study was commissioned by Rutgers to refine the RHRN Theory of Change (TOC) assumption on youth empowerment identifying various frameworks. The global consultant can review these documents as part of the desk review. The consultant is be expected to reconstruct ToC based on literature review and validate assumptions using primary data collected as part of ETE.

• The recommendations from the mid-term reviews should be considered and prioritized where relevant within the scope of the project, with prioritization and qualification based on the independent judgment of the ETE consultant and discussion with the Reference Group.

• The evaluation's effectiveness should be based on OECD DAC criteria, with the consultant expected to deliver 'Context-Mechanism-Outcome' (CMO) configurations on selected strategies or themes. Given the depth of realist evaluation (RE), it may be best to focus on selected/sampled strategies.

Data collection methods: A range of data collection methods can be included in a realist evaluation. For this end-term evaluation interviews, focus group discussions, outcome harvesting, and surveys. ETE global consultant is expected to work with national researchers/consultants however where necessary ETE global consultant can plan for country visits for quality assurance, trainings or necessary project management effort (however this should be built within the specified budget).  Outcome harvesting will not be used as a standalone method but will be coupled with more rigorous approaches such as contribution analysis to ensure a robust evaluation. There is also scoped to include additional methods as needed. This approach will contribute to future program development by providing insights into what works in different contexts and the ‘transferability’ of various program elements. Triangulation techniques will be systematically applied throughout the analysis process and validated across the different data collection method

Evaluation rigor: The end-term evaluation will look at programme implementation and results at both country and global (advocacy) level, as well as at processes (partnership, collaboration, governance, power balance) at in-country level, at consortium level and the interlinkage between those levels. As mentioned under section 3, the end-term evaluation needs to adhere to the requirements from the Ministry and the IOB which include – amongst others –, achievements and validation of the output and outcome indicators that are linked to the strengthening civil society and thematic result framework basket indicators, crosscutting themes (gender, youth, climate), and sustainability. Partnership collaboration needs to focus on: coherence, and localization/leading from the South. The partnership has a solid M&E framework in place which contains – amongst others – progress data on capacity strengthening activities; all available data needs to be validated by the consultants.

Evaluation principles: Joint learning is a key component of the end-term evaluation. Through participatory reflection, sense making, validation, and linking and learning meetings during the evaluation period, the end-term evaluation will contribute to strengthening the evidence of the RHRN programme. It is expected that the end-term evaluation will also provide input for internal reflection and stimulate innovation for future programme development and implementation. 

Role of Reference Group: A reference group has been set up for RHRN End Term Evaluation in line with IOB knock out criteria number 1. The reference group is composed of independent and programmatic experts who will review and approve following milestones during ETE process.

• Terms of reference for hiring the Global Consultant(s).

• Selected Global Consultant(s) to be contracted as evaluator.

• Inception report by the evaluator.

• Data collection tools.

• Draft evaluation report

• End evaluation report by the evaluator along with a slide deck and associated data.
  

[1]https://english.iob-evaluatie.nl/publications/guidelines/2022/04/22/evaluation-quality-criteria

[2] Reference Group: This group generally comprises the commissioner of the evaluation, a representative from the implementing organization evaluated and members with both thematic and evaluation experience, including at least one independent member. The role of the reference group is to assure evaluation quality and independence. It advises the commissioner on the ToR, the selection of evaluators, the elaborated methodology (inception report) and the draft evaluation report. Comments and advice from the reference group should be seriously considered by the evaluation team. https://english.iob-evaluatie.nl/publications/guidelines/2022/04/22/evaluation-quality-criteria

5. Timeline and deliverables

The selected consultants are expected to start their assignment in October or early November 2024. The deadline for the final report will be December 2025.

1. Hiring of national consultants: The Global consultant(s) will be required to hire, and line manage national research consultants in each program countries. If they have an existing team of researchers within project countries which can effectively organize and accomplish research process within the countries then they can propose that as part of the proposal. This is expected to be completed by start of December 2024. t Evaluator would have full independence on selection of national consultant however if they require support they can seek input from country partners by way of suggestions. The Global Consultant(s) is also expected to look for and give preference to youth researchers and ensure meaningful and inclusive youth participation in line with program principles. Where youth research consultants are unavailable due to limitations in terms of research capacity/competencies, national research consultant will be expected to work with national youth advisory boards to ensure inclusion and participation of target group in research design process.  

2. National Consultants are required to conduct a ETE planning workshop with country teams (2-3 days) before global planning workshop in the last week of Jan 2025.  This should include developing a country specific ETE plan that includes evaluation questions for the country. This activity should inform and support, global ETE consultant to develop a methodology for a global planning workshop and inception report.   

3. End-term evaluation planning workshop; develop a methodology for and (co-)facilitation of the end-term evaluation workshop with country and consortium representatives, and the national consultants and/or youth researchers (January 2025, the Netherlands). Since it will be towards the end of Inception phase, ETE consultant will be expected to conduct initial inception meetings with countries to develop draft ETE methodology. This will be a key opportunity to co-design work plans with partners and global consortium and finalise methodology of ETE. The workshop will be paid in full (flights, hotel, transportation, food (excluding diner) for country staff including national (country) consultants by Rutgers. However, ETE consultant is expected to pay for their own hotel costs, travel, flights and any other relevant cost (if applicable).  The programme for the workshop will be led by ETE consultant while developed jointly with representatives from the partnership.

4. Inception report (expected timeline: February 2025).

5. Development of the global and country specific tools (expected timeline: February 2025; tool development can also go partially in parallel with the writing of the inception report).

6. Collect and analyse data at global and national level (estimated period: March – Sep 2025)

7. Methodology for and (co-)facilitation of a validation process with the country coalitions and the global partnership (estimated period: Q3 2025).

8. Submission and presentation of draft report to the Reference Group (Q3 2025)

9. Incorporation of the feedback (Q3 2025) and submission of second draft version to the Reference Group (Q 3 2025)

10. Final report (estimated deadline for approving the end-term evaluation report by the reference group is mid-December 2025. It will be important for evaluators to explain in the bid proposal how would they ensure process from draft to final report is robust and smooth with sufficient room for feedback).

Detailed ETE Timeline:

6. Considerations

If you are interested in carrying out the RHRN ETE, we are eager to receive your technical proposal. We are keen to learn not only from the results of the ETE, but also from its process, at all levels of the partnership. We are eager to read in your proposal how you will enable this through your methodology. Please take note of the considerations below.

Collaboration and communication: The End Term review process will be a strong collaborative process between the successful applicant and various parties within the RHRN partnership, all working with unified goals. The final process will be designed and revised over the term of the assignment by all participating parties while meeting requirements by Ministry of Foreign Affairs 1.  The RHRN partnership will be eager to hold initial meetings with the selected applicant as soon as possible in the timeline, to clarify any questions related to the assignment, and to provide necessary background documentation, (digital) data, and other materials as needed. Regular updates on the process, for example on the country studies, are required. The applicant should address the communication process and collaboration between the different parties in their proposal.

Values: The RHRN programme has identified meaningful and inclusive youth participation, a gender-transformative approach, inclusivity, a human rights-based approach and safety and security as its core values. In addition, for PMEL, we value co-ownership and being utilisation focused. It is expected that these values are reflected in the proposal including in the evaluation methodology and broader process. Co-ownership includes ensuring ownership of findings and recommendations, as well as linking and learning between different parts of the partnership. The Global Consultant(s) will be expected to describe in the proposal as how these values would be integrated during evaluation plans.

Joint learning throughout: With regards to the principles of RHRN, inclusiveness, meaningful and inclusive youth participation and accountability, the end-term evaluation needs to include young feminist voices, young people in all their diversity as well as excluded and marginalized groups throughout the process. Preferably we select a feminist oriented consultancy agency that has a strong network of local feminist youth researchers. It is expected that the principle of co-ownership is fully embedded in the proposal methodology. With the selected consultants, the RHRN partnership must ensure that their different voices are included in the end-term evaluation design, including inclusive and accessible language.

Multilingual Team: Project operates in ten countries working with different languages therefore it will be important that the Global Consultant(s) team is able to engage meaningfully both in English and French at a minimum. There are four Francophone countries in the program, while six Anglophone countries in program. However, it will be ideal for national consultants to have an understanding of the predominant local/regional languages when being recruited. Ideally, we seek a consulting team that is gender-balanced and includes young people, in alignment with the program's vision and mission.

Project Management: RHRN is a large and complex programme and the management of the ETE will require both strong evaluation/research skills and strong project management/stakeholder management skills. Complex programs are characterized by their multiple components, diverse target behaviors, and varying levels of intervention delivery and reception. This complexity necessitates a comprehensive evaluation framework that considers the interactions between the program components and their contexts, aiming to understand how and under what circumstances the interventions achieve their outcomes (Skivington et al., 2021). The RHRN program works across ten countries to empower young people to advocate for their sexual and reproductive health and rights (SRHR), targeting individual, community, and policy levels. This complexity requires a flexible approach to evaluation, as the program must adapt to different cultural contexts and settings. For example, the program's efforts to provide comprehensive sexuality education and promote gender equality must consider the unique challenges and opportunities in each country. Additionally, the program's advocacy work involves engaging with a wide range of stakeholders, including young people, educators, policymakers, and community leaders, each with their own perspectives and needs. To effectively evaluate such a complex program, a realist informed evaluation approach is being planned, which focuses on understanding the interplay between the program components and their contexts. This approach helps identify which elements work, for whom, and under what circumstances, allowing for a deeper understanding of the factors that facilitate or hinder the program's success.

Inception Phase: For the upcoming evaluation, a three-month inception phase is being planned compared to the mid-term review (MTR), allowing consultants more time to understand the program before the planning workshop in January 2025. We will work with ETE consultant to have a meaningful inception phase allowing flexibility in designing an inception phase that works for them. Following key objectives have been laid out for inception phase:

• Program orientation would be crucial at this stage to develop a robust evaluation plan therefore Rutgers will support ETE consultant by providing information packages (both globally and nationally) including orientation meetings with steering committee members, program managers and country leadership teams. This phase should allow consultants to understand values of the program, governance structure and programmatic scope.

• We can discuss budget and scope during the inception phase. Consultants can revisit their draft budgets and approaches after gaining a full understanding of the program.

• We can also determine the need for realist training based on the consultants' familiarity with the realist approach. National consultants may need specific training in realist interviewing. The overall/global contractor must demonstrate proficiency with the realist approach to ensure they are well-equipped to meet the realist training requirements for data collection, including techniques such as realist interviewing. However, if you envision realist training for prospective national staff (researchers), please outline this in the bid proposal as it can be further discussed during the interview process. Rutgers staff previously found training with Justin Jagosh useful, and training materials are available which may be utilized if need for training is low. We may also consider scheduling training during the global meeting in January 2025.

7. The successful applicant

The RHRN partnership, through its lead agency Rutgers, wants to contract one (lead) organisation, responsible for the ETE deliverables and the process implemented. We also welcome applications that reflect a collaboration of multiple organisations for this assignment; research institutes and/or independent researchers/consultants can collaborate in order to conduct the end term evaluation. It is likely impossible for a single independent evaluator to accomplish this assignment, given the scope of travel and data collection required within the timeline.

For reasons of ownership and context specificity it is expected that the successful consultants will recruit, train and work with local/national consultants or local research teams in the programme countries. These research teams must have knowledge of the SRHR context of the countries and be able to carry out interviews in the local language and should have been jointly selected or in agreement with the RHRN team at global level and in each respective country.

Criteria for selection of global team of consultants:

Given the size of this evaluation, interested candidates will be required to submit in a small team. The consultant team must meet the following criteria and qualifications which will be assessed using a scoring method by interviewing panel and presented to reference group for final selection..

• Proven expertise in Realist Evaluation, ideally critical or feminist realism

• Experience with contribution analysis

• Proven experience in evaluation of lobbying and advocacy programs.

• Proven experience in evaluating complex multi-country programmes implemented by large partnerships.

• Proven experience in evaluating SRHR programmes.

• Preferably experience in Outcome Harvesting.

• Works from a Human Rights Based approach.

• Has a progressive stance on sexual and reproductive health and rights, specifically young people’s sexual rights and diversity.

• Is youth-friendly and has non-discriminatory attitudes towards working with young people. We are particularly interested in those who have worked with young people in the past or has experience in Meaningful and Inclusive Youth Participation (MIYP). More information can be found here.

• Demonstrated experience or commitment to inclusivity by effectively engaging young people in all their diversity, in the evaluation process.

• Strong writing skills with ability to present in a style that is accessible to readers.

• Strong facilitation skills.

• Proficiency in English and French.

• Geographic diversity of the team and its experience is a plus.

Criteria for the technical proposal

• Alignment with/understanding of the TOR in general.

• Feasibility of the proposal in terms of objectives and timeline.

• Strong proposal for participant selection/sampling of countries, data collection and analysis methodology.

• Level of attention for gender, meaningful and inclusive youth participation, inclusivity, and diversity, and how this is considered in the methodology.

• Clear and appropriate roles for different team members.

• Demonstrates how proposal adheres to the IOB quality criteria.

• Readability of the proposal.

• Whether the proposed total cost of the evaluation is realistic and achievable within the allocated budget

• Sufficient attention to time consideration for management of national consultants, how data from countries will contribute to the global level report, and how quality assurance will be done given the diverse consultants involved.

• Value for money (# of days proposed level of experience research team).

• The organization of joint learning within the ETE process and the division of labor between global and national consultants need to be clearly defined and proposed in the bid

• Although realist evaluation is major component of the overall evaluation, other questions, particularly those related to coherence, sustainability, cross cutting themes and capacity strengthening, can also be addressed using primary research data (as part of realist approach). Evaluators can propose creative approaches to data collection, analysis and triangulation that can address these additional evaluation questions as part of the broader evaluation approach.

• Demonstrates how would they meet data protection requirements to comply with GDPR

 The consultancy team should not comprise current or former staff (minimum of 5 years) of any of the members or partners of the consortium (including volunteers and board members), in order to protect the independence of the evaluation. Additionally, any consultant who has developed tools for one of the partners or was responsible for the past implementation of an intervention/programme/etc. should also be excluded.

8.How to apply

Applications for this role are open until 6 October 2024. Please send your application by responding to the link in Homerun application.

Interested parties should prepare a proposal of no more than ten pages in length, including the following:

• individual or institutional skills and background which make you suitable for this assignment.

• Your track record on the evaluation of complex programmes, with specific attention to SRHR, gender justice and RHRN’s theory of change and values, as described in this TOR. Shortlisted candidates will be requested to send two examples of recent evaluations or similar assignments with your application.

• Your understanding of each of the evaluation objectives and your suggested methodology for data collection and analysis to address draft evaluation questions.

• Your response to the considerations as highlighted in chapter 5 of this TOR (collaboration and communication, values and joint learning).

• A clear identification of the lead researcher and additional key personnel connected with your application, their proposed roles and time investments in the mid-term review, as well as your proposed approach to appointing local researchers.

• A statement describing why there are no conflicts of interest.

In addition to the proposal text, please submit supplementary materials including the following:

• An initial budget proposal, (with a maximum of EUR 285,000, of which approx. EUR 150,000 is spent in-country on country-specific evaluations). Please include the number of days you would spend on the assignment and daily fees (both senior and local researchers) and a clear indication of how the total proposed amount has been calculated. The budget should cover global and in-country costs. The proposal that will be selected will provide an appropriate balance between the quality and the costs of the evaluation.

• CVs, highlighting relevant experience(s) and personal profile, of the lead researcher and all named key personnel of the research team.

• If applicable: names, contact details, and CVs or country-specific research institutes or consultants that will be involved in the programme.

A first round of interviews is scheduled for 1st week of October. A second round of interviews is scheduled for 2nd Week of October. The successful candidates are expected to be ready to start the assignment from 1 November 2022.

9. Additional notes 

For enquiries, please contact Muhammad Sharjeel, senior PMEL adviser at m.sharjeel@rutgers.nl  

Note 1. This call for proposals has been distributed among the professional networks of the consortium, through our websites and social media, and also through several email listservs of evaluation and SRHR specialists.

Note 2. Rutgers is not a contracting authority within the meaning of the ‘Aanbestedingswet 2012’ (the Dutch Public Procurement Act). This tender procedure is a voluntary and private tender procedure. The Aanbestedingswet 2012 as well as the principles of procurement law are explicitly not applicable.

Applicants are hereby advised that Rutgers is not committed to any course of action as a result of its issuance of this Terms of Reference and/or its receipt of a proposal from the applicant or other forms in response to it. Rutgers reserves all rights and liberties regarding the tender procedure, including:

• the right to terminate the tender procedure at any moment in time.

• the right to reject any proposal.

• the right to engage negotiations with one or more parties (also third parties) regarding the contract or a part thereof.

• the right to award the contract or a part thereof.

• the right to award the contract to one or more parties.

• the right to make any decision subject to conditions.

• the right not to award the contract.

• the right to depart from or modify the proposed framework and/or any other procedure in relation to the Terms of Reference.

Annex 1: Theory of Change RHRN

Annex 2: Ministry requirements for Evaluation

IOB criteria

IOB has formulated 26 quality criteria. Providing a clear explanation and practical examples, this document shows when a criterium has been well met, sufficiently met or not met (found here). Criteria 1-9 aid in formulating the Terms of Reference (ToR) for the evaluation, addressing the quality control of the evaluation, project or programme background, the evaluation’s objectives and scope, and the evaluation questions. The evaluator should critically assess the Theory of Change (ToC) presented in the ToR (criteria 4* and 5*) and may reconstruct it in the inception report. The ToR should include a preliminary methodology, which can be assessed using criteria 10-17 below. Criteria 10-17 are used to judge a technical proposal and review and comment on an inception report, focusing on methodological quality. The commissioner may request prospective evaluators to propose their methodology in their tender. The appointed evaluator is then expected to develop the full methodology in the inception report, with the possibility of adjustments as necessary. Knock-out criteria in this section include 10*, 11*, 13*, 14*, 15*, and 17*. Criteria 18-24 assess the quality of the draft report, assuming that the ToR and inception report have already been evaluated. They focus on the quality of the methodology used and the conclusions drawn. At this stage, while it is often not possible to adjust the evaluation implementation, it remains possible to enhance the analysis or reformulate conclusions based on certain methodological limitations. Knock-out criteria in this section include 20*, 21*, 22*, and 23*. Criteria 25 and 26 assess the final evaluation report, assuming the draft report has already been evaluated. When assessing the overall quality, IOB recommends that at least 23 of the 26 evaluation criteria be scored as ‘adequate’ or ‘good’. Additionally, there are 13 knock-out criteria. If an evaluation scores ‘inadequate’ on one of these 13 criteria, the evaluation as a whole should be regarded as inadequate. The knock-out criteria are: 2*, 4*, 5*, 10*, 11*, 13*, 14*, 15*, 17*, 20*, 21*, 22*, and 23*. In IOB guidance document, the knock-out criteria have been asterisked (*).

Please note that a revised set of criteria will be released by Ministry of Foreign Affairs in September 2024.

OECD DAC evaluation criteria 2023[1]

As per requirements of MFA[2], ETE will meet three key OECD criteria: effectiveness, coherence, and sustainability requirements as laid out in OECD criteria3.

Program Effectiveness: To assess effectiveness, we will assess whether the programme has achieved its intended outcomes, the process by which these were achieved, and the factors that influenced this process. It also includes evaluating any unintended effects, whether positive or negative, to provide a comprehensive understanding of the programme's impact. We will assess results at various levels of the results chain (outputs, outcomes, and impacts) to understand if the stated objectives were met.

Partnership Coherence: To assess coherence, we will examine how well the partnership fits within the context and levels of RHRN project. It assesses both internal coherence (synergies and interlinkages between the partnership and program activities) and external coherence (alignment with other SRHR partnerships and consistency with international norms and standards). In the context of partnerships, coherence involves analysing the extent to which the programme collaborated effectively with other stakeholders and other interventions to avoid duplication of efforts and to create synergies. This also includes understanding how well the partnership adds value to the program activities.

Sustainability: Lastly, sustainability assesses whether the benefits of the intervention will last over the long term, considering financial, economic, social, and environmental dimensions. This criterion will look at whether the systems and capacities built by the intervention can sustain the benefits after the programme ends. We will consider if the programme included strategies for sustainability, such as building local ownership, ensuring financial sustainability, and creating resilience to opposition to SRHR. This involves examining if the intervention anticipated and planned for the continuation of benefits and identifying any barriers that might hinder sustainability.

Annex 3: Draft Evaluation Questions

Annex 4: Indicators to be measured in the RHRN End Term Evaluation

The following sets out the RHRN programme indicators which are to be measured in the ETE. Information is provided on the quantitative and qualitative measures for each indicator, availability of existing data, and notes on possible approaches. Note that the final approach will be identified in collaboration with the successful candidate and proposals on alternative approaches are welcome. Note that not all indicators need to be measured in all countries.

Annex 5:  Realist Perspective

The "Right Here Right Now" (RHRN) project represents a transformative youth initiative across ten different social contexts. It builds a youth movement that lobbies and advocates to influence the sexual and reproductive health and rights (SRHR) for young people across those societies. It is crucial in evaluation to ground our thinking and approach in a philosophy that reflects the complexity and depth of the social phenomena we are studying. To this end, we draw upon the principles of critical realist ontological emergence. It is a philosophical perspective that draws upon stratified reality and the emergent properties that arise due to interaction of project entities with the open social systems. Critical realism particularly focuses on ontological emergence, which can offer valuable insights into studying these realities beyond what is observable. This perspective acknowledges allows to assess different levels of realities (empirical, actual, real) each with its own unique properties which can generate multiple mechanisms leading to outcomes.

In the context of RHRN project, this means studying emerging changes in support of sexual rights of young people influenced by interaction of project with broader social, cultural, and institutional structures. Ontological emergence emphasizes that new opportunities and behaviors can emerge from these interactions. This translates to actions by youth agents influencing broader social support and create different partnerships (youth-youth or youth adult) to influence SRHR policies that affect their rights. Furthermore, this interplay to generate outcomes cannot be fully explained by any single pathway alone instead by multiple iterative pathways. Therefore, this approach can capture the richness and complexity of these interactions. In addition, this philosophical depth allows to underscore not only the empirical and observable aspects of change but also the underlying generative mechanisms and causal powers that drive social change.

Integrating feminist principles into this framework further enhances our approach by emphasizing the need to address power imbalances and gender inequities by integrating lived experiences of young people and marginalized groups in the process. Feminist theory aligns well with critical realism in its commitment to unveiling the deep-seated structures that perpetuate inequality and foresees transformative change within the agents of change. This alignment with feminist principle will ensure that our evaluation not only examines the structural and emergent properties of SRHR movements but also centers the voices and experiences of those most affected by these issues.

Evaluating the RHRN project through the lens of (critical realist) ontological emergence and feminist principles involves several key steps. First, we will analyze the project's outcomes at multiple levels, from individual experiences and community practices to institutional changes and global influences. Second, we will identify and assess the emergent properties resulting from the project's interventions. This will assess new patterns of behaviors in youth (empowered action or sexual empowerment) and social actors (power sharing, support, partnerships, meaningful youth participation), shifts in social norms (public support for marginalized groups), and changes in institutional policies (contribution to policy changes) that arise from the project's efforts. Third, by exploring the mechanisms at play we can understand how different elements of the project interact to produce outcomes. This involves examining both the intended and unintended consequences of the project's activities. Finally, our evaluation will maintain a non-reductionist stance, recognizing that SRHR issues cannot be fully understood by looking at individual components in isolation. Instead, we will consider the dynamic interactions between various factors to provide a holistic assessment.

This approach not only aligns with the project's comprehensive vision but also enhances our ability to generate meaningful insights that can inform future SRHR initiatives globally. In doing so, we commit to a philosophically rigorous and practically relevant evaluation that honors the multifaceted nature of social reality, the emergent properties that define it, and the feminist commitment to equity and justice.

[1]https://web-archive.oecd.org/temp/2024-05-13/81829-daccriteriaforevaluatingdevelopmentassistance.htm

[2]https://english.iob-evaluatie.nl/publications/guidelines/2022/04/22/evaluation-quality-criteria2.