Call for Evaluators: “INDEPENDENT FINAL EVALUATION FOR THE PROJECT: YOUTH ENVIRONMENTAL IMPACT (YENI)”

Job Description

TERMS OF REFERENCE (TOR)

INDEPENDENT FINAL EVALUATION OF THE PROJECT:

“YOUTH ENVIRONMENTAL IMPACT (YENI)”

BACKGROUND INFORMATION

Caritas Switzerland (CACH) is an independent Swiss NGO, engaged in Switzerland and more than 20 countries worldwide, including Kosovo. Our mission is to prevent, combat and sustainably eradicate poverty. To achieve this goal, CACH employs a policy of equity: we support inclusive communities with opportunities for everyone regardless of their ethnic background, group affiliation, or gender. CACH has been working in Kosovo since 1999, focusing on social inclusion, rural development and rehabilitation. Based on the Strategy 2021-2025, our central themes are Climate Change, Income Generation and Migration.

Overall, the CACH Country Programme for Kosovo is based on CACH’s International Cooperation Strategy and it defines country-specific outcomes. Within the theme of Climate, CACH in Kosovo is committed to climate justice and promoting the inclusion of women and members of minority groups.

The Western Balkans, including Kosovo, is one of the regions in Europe that is most heavily affected by the impact of climate change. This trend is projected to continue, with estimates of temperature increases of 1.7 – 4.0°C and even exceeding 5.0°C by the end of the century, depending on the global effort to reduce greenhouse gas emissions. Kosovo is also prone to a wide variety of natural hazards – including floods, landslides, droughts, earthquakes and wildfires – that could pose serious damage to the economy, fiscal balance and well-being of vulnerable populations. Many of these climate-related hazards are expected to magnify with future climate change. These climate and disaster risks can seriously affect productive sectors of the economy such as agriculture, infrastructure, energy, water resources, communities and households. Kosovo’s agriculture sector makes up 11 percent of the Gross Domestic Product, but it has exhibited low growth and has been affected by weather conditions and climate change.

Kosovo’s Constitution of 2008 recognizes the protection of the environment as one of the main values (art. 7) and “nature, biodiversity, environment and national inheritance as everyone’s responsibility” (art. 14). The Government of Kosovo has adopted the energy strategy, which will have a huge impact on Kosovo’s economy. A Climate Change Strategy is in place and the first climate change law was adopted by the Kosovo Assembly. If fully implemented, this could provide a sound basis for tackling climate change.

The YENI project goal is to improve climate action and environmental protection in Kosovar municipalities through the empowerment and active engagement of youth. By putting a long-term emphasis on youth via schools and climate education in the local context, this project aims to support Kosovar local authorities in enhancing their ability to address the country’s climate change vulnerabilities. To contribute to this overall goal, the project activities are focusing on tackling existing barriers to make behavioural change constructively possible. Therefore, the project will facilitate the strengthening of the role of youth as agents of climate change in the Kosovo municipalities.

The YENI project aims to motivate youth for climate change engagement and vision, as well as complementing skills, enabling them to become active citizens and contribute to climate change mitigation or adaptation. Therefore, the project will facilitate the strengthening of the role of youth as agents of climate change in the targeted municipalities.

YENI project is in line with the Sustainable Development Goals (SDGs), specifically, SDG 4 (Quality Education), SDG 13 (Climate Action) andSDG 7 (Affordable and Clean Energy) which emphasize the special importance of environmental education, awareness-raising, energy conservation on topics related to climate change and its associated impacts. Education is a crucial component of climate change mitigation and adaptation. Increasing environmental awareness and education on the climate crisis can contribute to the effective reduction of greenhouse gas (GHG) emissions and to adapting to a changing climate in both the short and long term.

1. PURPOSE OF THE EVALUATION

The final evaluation should explore YENI’s work and achievements and support to assess overall progress. The evaluation will analyze the whole scope of the project and its results framework, which comprises all elements of the intervention logic. Through the assessment of the performance, achievements and lessons learned to date, the review will contribute to both – learning and accountability. This evaluation will adhere to the OECD/DAC[1] and the SEVAL[2] standards for evaluation. For all practical purposes, this TOR defines the overall scope of this evaluation.

The main purpose of this final independent evaluation is to provide an independent assessment of the overall progress, through an analysis of the relevance, effectiveness efficiency, impact and sustainability of the project. The evaluation should give furthermore a brief outlook on the potential future impact of the project.

The specific objectives of the final evaluation are the following:

1. Assess the relevance of the project’s design, regarding country needs and how the project is perceived and valued by the target groups.
2. Evaluate the relevance of the YENI project to address the environmental and climate change issues for youth in the 17 targeted Kosovar municipalities. It will also assess the relevance of the stakeholders targeted by the intervention and the methodologies and approaches to do so.
3. Identify the contributions of the project to the Sustainable Development Goals 13 – Climate action and SDG 4 – Education and its synergy with the overall CACH objectives and strategy.
4. Assess the effectiveness of the YENI project at achieving its objectives and provide clear insights about what has and has not worked so far and why, including unexpected results and factors affecting project implementation (positively and negatively).
5. Assess the efficiency in terms of value for money of the delivery of the YENI outputs.
6. Assess and give a general outlook on the realized impact of the project.
7. Assess the sustainability of the project: Will the project’s benefits last beyond its completion? What is the possibility that project outputs and benefits will continue and be sustainable after the project is completed?
8. Identify lessons and provide a set of actionable recommendations for potential future phases of the project.

Recommendations emerging from the final evaluation should be strongly linked to the findings of the evaluation and should provide clear guidance to stakeholders on how they can address them. It is expected that the findings and recommendations of this final review will help to identify any needed action that was lacking and can be used for the for potential future phase of the project.

2. BRIEF DESCRIPTION OF THE PROJECT INTERVENTION

The YENI project is designed to help the Kosovar local authorities to increase their capacity to address the country’s climate change vulnerabilities by focusing long-term on youth via schools and education in the local context. To accomplish this overall goal, the project activities are focusing on tackling existing barriers to make behavioural change constructively possible. Therefore, the project will facilitate the strengthening of the role of youth as agents of climate change in the municipalities.

The overall objective of the project is:

• Improved climate action and environmental protection in Kosovar municipalities through mobilization and active engagement of youth.

This shall be accomplished by educating and mobilizing youth from 60 high schools across Kosovo on climate change and environmental issues. The topic will be strengthened by establishing and strengthening Eco Clubs and utilizing existing student councils in the field of climate and environmental protection. As a result, mobilized youth shall actively participate in developing Local Environmental Action Plans (LEAPs) within their municipalities.

The three outcomes of the project are as follows:

• Youth of secondary schools is actively undertaking sustainable measures and is recognized as a partner for climate action by the local and national authorities.
• Improved climate action and environmental protection in Kosovar municipalities through active engagement of youth.
• Municipal Education Directorates, Student Councils, Eco Clubs and pupils succeed with their Climate Change initiatives in their municipalities.

CACH, respectively the YENI project, aims to provide youth with a climate engagement and vision, as well as complementing skills, to enable them to become active citizens and make their contribution to mitigating or adapting to climate change (CC). Eco Clubs or similar youth mechanisms are not established in all upper secondary schools in Kosovo. However, most schools have established Student Councils, nevertheless, only a small part deals with environmental protection. Therefore, with climate change education and environmental skills development young Kosovar girls and boys can play an active role in protecting and improving the environment. They can change their lifestyle and how it affects the environment. Moreover, they can build social and individual capacities and attitudes toward climate change. They can make their homes, schools and youth organizations more environmentally friendly by adopting climate-friendly practices, recycling different materials as well as preserving resources such as water and electricity. Engaging youth in environmental protection not only directly impacts changing youth behaviour and attitudes but possibly influences their parents, relatives and families. Eco Clubs in schools encourage students to engage in meaningful environmental activities and projects. It is a forum for students to reach out and influence their parents and neighborhood communities to promote responsible environmental and climate-friendly behavior, enabling students to go beyond the confines of a syllabus or curriculum to investigate environmental concepts and actions. While everyone, everywhere emphasizes the importance of “learning to live sustainably”, the environment and concrete behavior change remains a secondary concern in the formal education system. Creating awareness and sensitivity among students toward environmental issues and their related problems is only a start for that. Imparting knowledge and nudge on behavior change helps individuals and social groups gain a variety of experiences and acquire a basic understanding of the environment and its associated problems, but also helps to adapt the behavior of the next generation. Building attitudes to help individuals and social groups acquire a set of values and feelings of concern for the environment and the motivation for actively participating in environmental protection. Eco Clubs are one way to encourage interest and involvement and can be extremely rewarding, especially when students are given the responsibility to make their own decisions about the changes they want to see and the opportunity to improve their school environment.

An additional outcome envisaged is reserved for the inclusion and contribution of school youth to the development of the LEAPs, in which the suggestions of the youth shall be incorporated. Furthermore, this document must be harmonized with other local-level documents to create synergies of actions with other actors that are active at the municipal level. Lastly, in this process, the Student Councils and Eco Clubs shall harmonize their efforts with other youth and environmental NGOs and CSO groups to increase advocacy and lobbying efforts and to have a bigger impact on local authorities. Advocacy and lobbying are essential so that youth is not left behind in the conversation, especially young people in the upper secondary schools in Kosovo which are the key actors in the sustainable development process. Youth advocacy is a key factor in the development and communication of climate change issues and environmental protection.

3. SCOPE & FOCUS OF EVALUATION / EVALUATIVE QUESTIONS

3.1. SCOPE OF THE EVALUATION

The final evaluation will analyze all planned outputs and outcomes under the project within this period (please see Table 1), with particular attention to synergies between the components and contribution to climate change and environmental issues and youth empowernemt in Kosovo.

The evaluation will discuss, throughout the evaluation questions, how the project addressed its main issue (i.e., climate action, youth empowerment and environmental protection) and the CACH cross-cutting themes including conflict sensitivity, gender, partner support and participation. The evaluation should help to understand how and why the project has obtained or not the specific results from output to potential impacts, how changes in the context have influenced progress.

Furthermore, the evaluation process will assess the whole project of the YENI project activities conducted and implemented during the period 2022 – 2024 to provide final recommendations based on the findings from the field.

The evaluation experts/service provider is not allowed to share any information and/or evaluation results without prior notification to the CACH/YENI project staff. The service provider will sign a copy of the Code of Conduct document and attach it as Annex to the service agreement.

Table 1: Scope & focus of the evaluation.

Regionally the following target municipalities are part of the YENI project:

Table 2: Geographical coverage

3.2 EVALUATIVE QUESTIONS & FOCUS

Concerning the DAC criteria (Relevance, Effectiveness, Efficiency, Impact, Sustainability), the following questions are of special interest:

3.2.1 Relevance

• How appropriate and relevant is the YENI project towards the alignment with the national and international policies on environmental and climate as: National Climate Strategy National, Youth Strategy, Agenda 2030 and Sustainable Development Goals (4 – Education, 7 – Affordable and Clean Energy, and 13 – Climate Action) as well as the Local Environmental Action Plans (LEAPs) of 17 target municipalities?
• How consistent are the activities and outputs with the intended effects?
• Is the project relevant to the identified needs of beneficiaries (target group of project)?
• What is CACH/YENI project’s position towards cooperation with relevant stakeholders/actors in climate education and youth empowerment?
• To what extent is the project fit for purpose to:

1. Encouraging young people to take sustainable action on climate change issues by helping local and national authorities acknowledge them as partners in the fight against climate change?
2. Increase youth knowledge and engagement in environmental protection and climate action in Kosovar municipalities?
3. Developing awareness among highschool students regarding the field of climate change and environmental protection?
4. The development of new climate change materials for the high school level?
5. Ensuring that the students, student councils, Eco Clubs, and MEDs in each municipality are successful in their climate change projects?
6. Youth Empowerment in the development of the School Environmental Plans and youth components for the municipalities LEAPs?

• Has there been any change since the project was formulated that might have affected its relevance? If so, what are these changes and to what extent has the project managed to adapt to ensure it remains relevant?

3.2.2 Effectiveness

• To what extent has the YENI project delivered on its outputs and outcomes at Kosovo regional level? Were there any positive or negative unintended consequences or unexpected effects? In particular:

a. How effective has YENI been so far in developing methodologies to raise awareness on climate change and environmental protection matters? How effective has YENI been in collecting the data that feed into these methodologies?
b. How effective has YENI been in engaging with key decision makers in the country to mainstream climate change in policy and decision making? What were the main markers of changes among decision/policy makers that demonstrate YENI was on its way to tackle existing barriers to make behavioral change constructively possible and strengthen of the role of youth as agents of climate change in the municipalities?

• For all the above questions, what are the factors influencing positively and negatively the effectiveness of the project?

3.2.3 Efficiency

• To what extent are the YENI outputs in balance with the level of effort, time and resources spent?
• To what extent do spending and project delivery progress according to the planned schedule?
• How much has the project management adapted to any changing condition to improve project implementation efficiency?
• To what extent has the project built on existing agreements, initiatives, data sources, synergies and complementarities with other projects, partnerships, etc. and avoided duplication of similar activities by other groups and initiatives?
• Which alternative approaches could have led to equivalent results at lower costs?

3.2.4 Impact

• To what extent has a project made a serious contribution to the overall goal of the project?

“Improved climate action and environmental protection in Kosovar municipalities through mobilization and active engagement of youth.”

The evaluators are requested to give a general and forward-looking assessment of the impact the project had already within its three years, and an outlook into the future with what has been implemented, and how it will further impact after the end.

3.2.5 Sustainability

• To what extent are the benefits of the project likely to be sustained after the completion of this project?
• What is the likelihood of continuation and sustainability of project outcomes and benefits after completion of the project?
• How effective were the exit strategies, and approaches to phase out assistance provided by the project including contributing factors and constraints?
• What are the key factors that will require attention in order to improve prospects of sustainability of Project outcomes and the potential for replication of the approach?
• How were capacities strengthened at the individual and organizational level (including contributing factors and constraints)?
• Describe the main lessons that have emerged.
• What are the recommendations for similar support in the future?

4. EVALUATION METHODS AND PROCESS

The independent final evaluation will comply with the OECD/DAC and the SEVAL standards for evaluation. The evaluation will apply a mixed methods approach, engaging with key stakeholders of the project at all levels during the design, fieldwork, validation and reporting stages.

The evaluator(s) is expected to develop an evaluation framework based on the suggested key evaluation questions above but may suggest additional questions or modifications. The inception report will be prepared as the first deliverable of the evaluation and will include an evaluation matrix[3] presenting how the key issues will be addressed, the data sources and the data collection methods that will be used for the final evaluation and a set of criteria to rate the strength of the evidence collected.

All data collection tools are to be included as annexes to the final evaluation report. The link between evaluation questions, data collection, analysis, findings and conclusions must be made and set out transparently in the presentation of the evaluation findings. Conclusion and recommendations should be underpinned by a strong set of evidence.

The evaluation will seek the views of the range of stakeholders who have been engaged in the project[4] to conclude whether the project is on track and expected to realize its set objectives.

To collect the data for analysis, the evaluation will make use of the techniques listed below (but not limited to). The data from these sources will be triangulated[5] to increase the validity and rigour of the evaluation findings.

• Review of relevant documentation from the project (documents to be shared once the evaluator has been selected).
• Desk review: project design and strategy documents, activity documents, communications, research and publications.
• Key informant interviews and focus groups: with project staff, tripartite constituents, beneficiaries of the project and potentially other stakeholders and partners.
• Field in-depth interviews: the evaluation team is expected to meet project beneficiaries’ men and women to undertake more in-depth reviews of the project work and results.

The evaluator must indicate the selection criteria for individuals to interview.

The selection of the field project visit locations should be based on criteria to be defined by the evaluator.

• Presentation of the preliminary findings to the key stakeholders in a workshop.

At the end of the fieldwork the evaluation team will present preliminary findings to the project’s key stakeholders in a workshop to discuss and refine the findings and fill information gaps.

The methodology should include examining the interventions, specifically in the light of the project’s log frame, logical connection between levels of results, its coherence with external factors, their alignment with CACH’s strategic objectives, and Agenda 2030 – SDG (13 – Climate action, 4 – Education, SDG 7 Clean and affordable energy) and related targets. The data and information should be collected, presented and analyzed with appropriate gender, ethnicity and where feasible in terms of disability disaggregation, even if the project design did not take those into account. A more detailed methodology for the assignment will be elaborated by the evaluator based on this TOR, in consultation with the CACH/YENI team and key stakeholders in the inception report.

5. DELIVERABLES

5.1. An inception report (not more than 30 pages excluding the annexes) – upon the review of available documents and an initial discussion with the project management, the inception report will be developed, including:

• Conceptual framework that will be used to undertake the evaluation.
• Own methodology proposed.
• Data required to answer the evaluation questions, data sources by specific evaluation questions (emphasizing triangulation as much as possible) data collection methods, and sampling.
• Selection criteria for locations to be visited.
• Selection criteria for individuals for interviews (as much as possible should include men and women).
• Detail the work plan for the evaluation, indicating the phases in the evaluation, their key deliverables and milestones.
• Set out the list of key stakeholders to be interviewed and the tools to be used for interviews and discussions.
• Set out the agenda for the stakeholder’s workshop.
• Set out an outline for the final evaluation report.
• Interview and focus group guides.
• Evaluation question matrix – attached as Annex III to this TOR.
  5.2 Preliminary Findings to be shared with the key stakeholders – After completion of data collection or before sharing the draft report, the evaluator should present preliminary debriefing and findings to the key stakeholders. The evaluator will set the agenda for the half-day to one-day meeting. The presentation should provide a brief review of key results for each evaluation criterion. The workshop will be technically organized by the evaluation team with the logistic support of the YENI project.

The inception report should be approved by the Senior Project Manager and the Ministry of Regional Development coordinator before proceeding with the fieldwork. More detailed instructions can be found in Annex II – Writing the inception report.

5.3 First draft of Evaluation Report – The report should be no longer than 50 pages excluding annexes. The Project Manager, CACH Head Office the Country Director in Kosovo and YENI coordinator from the MRD will review the first draft of the Final Review (MTR) report, to ensure that it meets the required quality standards and covers all agreed components and contents of the MTR. Detailed comments and feedback on the draft report will be provided to evaluation consultant(s) and discussion may be held to provide clarifications as necessary. The draft review report will also be shared with the MRD for additional feedback and input. The evaluator(s) should submit a comprehensive draft report consisting of major findings and recommendations for the future phase.

5.4 The final version of the evaluation report will be produced by the evaluation consultant(s) based on feedback received on the draft report, incorporating comments received from CACH/YENI, the MRD . The report should be no longer than 50 pages excluding annexes and executive summary. The final draft report should be submitted within the given timeline with enough detail and quality.

The draft and final version of the evaluation report in English (maximum 50 pages) will be developed under the following structure (Annex IV – Preparing the evaluation report).

Table 3: Structure of report

6. SCHEDULE

This assignment is anticipated to take place between 02.09.2023 – 11.11.2024. The total duration of the final evaluation process is estimated to be 50 working days for both national evaluators/experts or a consultancy company.

Table 4: Evaluation Timetable and Schedule

Table 5: Proposed workdays for the evaluation team

EVALUATION TEAM / QUALIFICATIONS

The evaluation team should consist of two national evaluation specialists/consultants, who will work jointly to achieve the expected results. Both national evaluation specialists will work in close coordination with the YENI Project Manager and Ministry of Regional Development coordinator. The project team will provide administrative and logistical support as needed.

Evaluation Manager: the evaluation will be accompanied by Mr Bardh Xërxa – YENI Project Manager, who is responsible for completing the following specific tasks:

• Brief the evaluator on CACH evaluation policies, procedures and sharing of relevant documents for desk review.
• Initial coordination with the project team on the development of the field mission schedule and the preliminary results workshop. and feedback to the inception report.
• Circulate the first draft of the evaluation report for comments by key stakeholders and CACH to report draft.
• Ensure the final version of the evaluation report addresses MRD comments (or an explanation why any has not been addressed) and meets CACH requirements, approvals and payment.

The National Evaluators should have the following qualifications:

• Education:
• Master’s degree in one of the following relevant fields: Environmental Economics, Education, Development Studies, Public Finance, Public Management, Management or other related field.
• Academic/professional background in M&E, RBM or programme management is an asset.
• Work Experience:
• At least 5 years overall professional experience as an evaluator.
• At least 3 years of relevant professional work experience in climate change, environmental management, youth, education, or performance/results-based budgeting.
• Track record in conducting project evaluations in the field of environment, climate and youth.
• Experience working with international development partners, especially in climate change, youth, environmental management or public financial management.
• Experience in working with OECD/DAC or the SEVAL standards.
• Proven experience with logical framework approaches and other strategic planning approaches, M&E methods and approaches, information analysis and report writing.
• Extensive knowledge of, and experience in applying, qualitative and quantitative research methodologies.
• Functional competencies:
• Strong interpersonal and communication skills.
• Strong analytical, reporting and writing abilities.
• Excellent speaking and presentation skills.
• Openness to change and ability to receive and integrate feedback.
• Language requirements:
• Excellent spoken and written Albanian and English language skills required. Other local languages, in particular Serbian, are considered an asset.

7. BUDGET

This assignment foresees a total of 50 days for the two (2) evaluators or a consultancy company, to cover the whole evaluation process.

The available maximum budget foreseen for the evaluation process is EUR 15,000.00.

The following costs will be covered by evaluators consultancy company (i.e., they need to be included in the offer, in addition to daily fees of each evaluator):

• Transportation cost in Kosovo.
• Half a day workshop and necessary translation costs.

CACH/YENI project will not pay any additional costs/expenses that may arise from the evaluation process except the financial amount presented and agreed on in the service agreement with the evaluator(s).

8. SPECIFICATIONS FOR THE SUBMISSION OF OFFERS APPLICATION PROCESS

Interested candidates/companies are requested to submit an electronic copy of their expression of interest/proposal by 23rd August 2024 with the subject REF: “Final Evaluation YENI” to kosovo@caritas.ch and with bxerxa@caritas.ch in CC.

Applicants must submit see more in: Annex I bidding instructions.

ANNEXES

• Annex I: Bidding Instructions.
• Annex II: Writing the inception report.
• Annex III: Evaluation question matrix (EQM).
• Annex IV: Preparing the evaluation report.
• Annex V: General conditions for procurement.
• Annex VI: Exclusion criteria.

[1] https://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm

[2] https://www.seval.ch/en/standards-competences/standards/

[3] ANNEX III – Evaluation matrix for the inception report.

[4] Indicative list and contact of stakeholders who have been engaged in the project – to be shared once the evaluator has been selected.

[5] Triangulation – looking for at least three sources or applying at least three different methods. https://rise.articulate.com/share/oPu-uFZAE0-X9S6EnnyBJ_WrjrYiTCFh#/lessons/j3AG eK2ILpYzzRcvfbFwc6dew39ThVw