Mid-Term Evaluation for the Education in Emergency Project in Galkacyo South District
Evaluation of the Project: Education in Emergency Project for Most Vulnerable Internally Displaced Persons (IDPs) in Galkacyo South District
Project code: K-SOM-2021-4070
Commissioned by: Diakonie Katastrophenhilfe -DKH through the local implementing partner Centre for Peace and Democracy (CPD).
- DKH and CPD Overview.
Diakonie Katastrophenhilfe (hereafter DKH) is a humanitarian assistance agency in Germany with headquarters in Berlin. It has regional offices in Asia, Africa, and Latin America, as well as country offices in our priority countries, which are responsible for the coordination of aid programs. DKH’s operating model is structured around implementing projects exclusively through a network of local partner organizations. As such, DKH’s core business is partner support and partner strengthening.
On average, DKH supports around 150 projects annually across the globe. Besides humanitarian assistance, DKH also focuses on Disaster Risk Reduction (DRR), preparedness measures as well as rehabilitation and reconstruction. In recent years, DKH has accelerated its commitment and investment in strengthening its cash and voucher assistance (CVA) approach to ensure context-appropriate and dignified responses for affected families.
DKH’s projects are funded through donations, a percentage of the annual income of sister agency ‘Bread for the World’, collaborations with other aid agencies such as Caritas International, as well as institutional donor funds such as the German Federal Foreign Office (GFFO) and the Federal Ministry for Economic Cooperation and Development (BMZ). DKH is also a recognized cooperation partner of the ECHO, the Office for Humanitarian Aid of the European Commission, as well as of EuronAid, a network of European non-governmental organizations specialized in food aid and food security.
In Somalia, DKH has been running intensive relief aid programs for more than twenty years. In recent years, DKH and its local partner organizations implemented also transitional aid programs. In close cooperation with its local partner organizations, DKH works across several sectors including WASH, Shelter, Camp Management & Non-Food Items, Education, Food Security, Livelihoods with integrated Cash Voucher Assistance, and livelihoods and vocational training. wherever feasible.
Centre for Peace and Democracy (CPD), a local Non-Governmental Organization working in Somalia has been implementing the “Education in Emergency (EiE) Project in Galkacyo South District” with funding from the DKH somalia program. Therefore South Galkacyo is one of the IDP hosting Districts in Somalia with an estimated 45 IDP Camps (population of IDPs estimated at 9,364 Households (55,528 people) according to CCCM Report for Galkacyo South.
- The project Background.
The Education in Emergency (EIE) project supports around 1,200 (50%F) school-aged children from at least 800 vulnerable IDP households located in various IDP settlements in Galkacyo South District. the basic approach of the project is designed to address the children, girls, and boys, in need of education, directly through an individual selection of the most vulnerable children that are forcefully displaced from their families or separated from their families and live now in foster households.
Education provides immediate physical, psychosocial, and cognitive protection during crises. Out-of-school children are more vulnerable and exposed to protection risks like; GBV, early marriages, early pregnancy, child labor, and forced recruitment. Opposed to out-of-school children, children with a certain level of education are less vulnerable to illicit recruitment, unwanted pregnancy, and early marriage. Educated youths, girls and boys, have better chances to find employment and social integration.
The project supported school-aged children including children living with disability (50% girls, 50% boys) in ages between 6 and 13 years for primary school learning level (~95% of beneficiaries), and ages 14 to 18 for secondary school level education (whereby in single cases those age limits may vary due to the generally late enrollment of children in school) selected from vulnerable IDP households.
The project focused on children for new enrolments for small children, girls and boys, that are not enrolled in any primary school and re-enrolment for those that have dropped out of school due to their background of displacement and related vulnerability and poverty constraints, those ones usually in a higher age and in higher classes, and are willing to attend school, as well as retention in classes for vulnerable secondary school girls and boys that already have dropped out or that are in danger to drop out shortly before their final exams due to vulnerability constraints of their households and their inability to pay school fees and other ancillary costs attached to their education
- Purpose of the mid-term Evaluation
The purpose of this mid-term evaluation is to review the project progress, achievements, and challenges, inform the DKH, and the project partners of key lessons learned, and propose recommendations for any improvement or adjustment needed for the remaining project implementation period. Particularly, the mid-term evaluation will focus on the following aspects:
- To inform DKH/CPD and other project stakeholders on the progress in implementation of the EIE project, i.e. the delivery of planned Objectives and Results achieved as well as progress towards achievements of the respective objectives while highlighting enabling factors and challenges encountered.
- Drawing key findings and lessons learned and proposing improvement and adjustment needed to ensure that the project remains relevant to the needs of the target groups and contributes to knowledge development.
- To identify the factors (positive and negative) that have impacted the achievement of the objectives and expected results within the project time frame.
- To analyze what is working well for the project, the main bottlenecks, and what needs to be done in the future for similar projects.
- To inform resource allocation for a future round of proposals, as well as to inform decisions on scaling-up, replication, and /or continuation.
- Project Objectives
The Overall project objective: Enhanced resilience of crisis affected IDP boys and girls, especially out of school children in South Galkacyo district in Galmudug State, Somalia .
Specific objective: Improved enrollment and retention of qualitative education in emergencies for girls and boys.
The Specific Objective indicators
- 75 % of targeted school-aged children are constantly in schools in the IDP camps and attend classes regularly
- 75 % of new and current enrolled learners (50% F and 50% M) are retained and complete at least one school term during the project period.
Result 1. Ensure emergency and crisis-affected children have access to a safe and protective learning environment
Indicators for Result 1
- 1200 children (600 M/ 600 F) reached with Education in Emergency assistance.
- 25 classrooms constructed or rehabilitated in 10 IDP schools.
- 11 child-friendly and inclusive WASH facilities are available to emergency-affected children and youth in 6 of the 10 targeted schools.
Result 2. Ensure targeted most vulnerable children, respective HHs, and teachers’ basic needs are adequately addressed as a key barrier to enrolment and retention in education.
Indicators for Result 2
- 30 teachers (21 M/ 9 F) benefitting from vouchers for school fees to pay emergency incentives
- 1.200 learners-beneficiaries (600 M/ 600 F) with access to food rations through monthly food vouchers.
Result 3. Strengthen the capacity to deliver effective and coordinated education in emergencies preparedness and response within the education system.
Indicators for Result 3
- 30 teachers (21 M; 9 F) trained in different modules according to the teacher in Crisis Training Module.
- 70 Community Education Committee members (50 M, 20F) trained in school safety, various TiCC modules, and hygiene promotion in schools, as well as provided advocacy campaigns on the importance of education, especially for girls in Galkayo South.
- Scope of the mid-term Evaluation
The mid-term evaluation will cover the assessment of the key activities implemented during the duration of the project in the timeframe between November 1st, 2021, and October 31st, 2022. The evaluation will be guided by the indicative questions presented below and generate corresponding findings and recommendations aiming at improving project implementation and guiding the future strategic planning of education in emergency interventions.
The project is designed to achieve the following 3 results mentioned above and the indicators to measure the achievements of the EIE project as implemented at mid-point:-
- Approach and Methodology
The proposed Mid-Term evaluation seeks to aim at quality evaluation that provides credible and useful evidence to strengthen accountability for development results or contribute to learning processes, or both. These Standards aim to improve quality and ultimately reinforce the contribution of evaluation to improving development outcomes.
The evaluation will be expected to be highly participatory and shall involve engagement with the EIE project stakeholders involved in the project. These shall include the Galmudug State Ministry of Education, Galkayo South District education department, Community Education Committees (CECs), IDP Camp Management Committees, Parents of the learners, teachers, and participating EIE project schools’ management.
The identified EIE stakeholders will be involved in providing their perception of the various EIE project activities, identifying issues and challenges that need to be addressed, and evaluating questions to be answered. The approach shall also be required to incorporate the analysis of records to analyze the actual performance of some of the parameters of EIE indicators as designed by the project in terms of enrolment, retention, and improvement in teaching skills class performance based on administered tests.
The consultant shall be required to develop a viable evaluation methodology that is based on the Education in Emergency evaluation parameters. The methodology and tools used will be required to capture, analyze and rate the following: – I). Improvement in access to education for the emergency-affected populations targeted IDPs children, II). Provision of learning space including improvement of the learning environment through the construction of temporary learning spaces (TLS), III). Provision of Vouchers (conditional and flexible) for Education (V4E) to cater to their primary and/or secondary education that will be redeemable by the school administration in EIE project selected schools to cover teacher incentives, teaching materials, school maintenance costs, and basic supply of clean water to the learners for drinking and handwashing.
The evaluation will be required to describe the sources of information used (documents, respondents, administrative data, literature, etc.) in sufficient detail so that the adequacy of the information can be assessed. The evaluation methodology shall be required to explain the selection of case studies or any samples used in the evaluation and their usefulness. The methodology used shall also develop viable tools and processes to help capture data and information that will contribute to analyzing the achievement of the processes used in delivering the project and also the degree to which the indicators have been achieved. The methodology shall provide the probable limitations regarding the representativeness of the samples identified and alternative/mitigation measures to be adopted. The evaluation methodology proposed by the consultant shall be capable of undertaking cross-validation of the information sources and critically assessing the validity and reliability of the data. The consultant shall also be required to provide and include a complete list of interviewees and other information sources consulted, to the extent that this does not conflict with the privacy and confidentiality of participants.
The consultant will be required to select/develop and adapt the data collection tools that will be relevant to use in the specific context of this assignment as provided in the scope of work. The sample tools will be required to conform to the approach recommended by the Somalia Education Cluster relevant to the assessment of needs in emergencies. An appropriate and strategic mixed sampling method should be indicated and its potential to provide the data and information required indicated/justified.
The suggested methodology will include:
- a)An in-depth desk study analysis and mapping of all project documents, relevant material, and resources, including:
- The project proposal
- The project log frame
- Vulnerability assessment report
- Education in the emergencies assessment report
- Project narrative progress reports generated up to the mid-point of project implementation.
- Project financial reports.
- b)Questionnaires and surveys were addressed to appropriate groups of stakeholders. Semi-structured Interviews, focus group discussions, and meetings (in person, via skype and telephone) with relevant DKH & CPD staff, donor representatives, and various stakeholders; – to gather substantial anecdotal evidence on the effectiveness, efficiency, relevance, and timeliness of the project activities implementation and delivery.
- c)Selected field visits to meet with the local stakeholders, beneficiaries, and partners involved in the implementation of the EiE project, including interviews and focused group discussions with direct beneficiaries to determine the relevance and effectiveness of the project.
- d)Inception Workshops to evaluate the evaluation, methodology, design, and sample.
- e)A mid-term evaluation presentation to present and discuss findings and recommendations.
- Key Guiding Questions
The consultant will be required to develop a methodology that is in line with the guidelines of the OECD evaluation criteria of relevance, effectiveness, efficiency, coherence and sustainability, and impact.
Based on the EIE project’s design and implementation as undertaken so far, the following key questions have been developed to guide the development of the assignment approach, and methodology and inform the final findings.
- To what extent are the projects, activities, and outputs aligned with the project goals as captured in the proposal?
- Was the project designed in a way to meet the educational needs of targeted beneficiaries?
- Has the project successfully prioritized the most vulnerable IDP children as envisioned in the project design?
- Has the project succeeded in creating a safe and protective learning environment for the learners enrolled in schools by the project? If not, what needs to be done?
- Has the project been successful in ensuring at least 75% of the enrolled beneficiaries are consistently attending school? If not, what needs to be done?
- Has the project ensured that at least 75 % of new and current enrolled learners (50% F and 50% M) are retained and complete at least one school term during the period covered by the evaluation?
- Has the project improved teachers’ and community education committees’ capacity to support quality education access in emergencies for IDP children?
- How successful has the project been in ensuring the inclusion of girls compared to boys?
- What is the perception of the community and relevant stakeholders towards this project and the impact it has on the community?
- What has been the impact of external factors such as COVID-19, climate change, and conflict on the implementation of the project against the activity plan?
- b)Effectiveness and Efficiency
- What have been the key outputs of the projects?
- Is output delivery on schedule? Are there any delays? What have been the main challenges in the delivery of these outputs?
- Which activities and use of the produced output were the most/least effective in contributing to the projects’ objectives and why? What are the common quality aspects and challenges?
- What are the main factors that have facilitated or obstructed the achievement of outcomes?
- Are adequate monitoring tools and mechanisms in place, and functional? Do they allow communication and exchange of lessons learned?
- Have the existing monitoring and evaluation mechanisms contributed to the achievement of expected results throughout the project implementation and how have these been effective? How could these work better for future interventions in terms of Education in Emergencies support?
- Has the project been implemented in a cost-efficient manner?
- What are the main factors that facilitate or threaten the (financial and institutional) sustainability of the outcomes of the projects and activities?
- What are the main challenges in safeguarding the sustainability of the project?
- What are the main lessons learned from the project that could be documented? And to what extent could the project be scaled up or replicated in other regions?
- How has the work impacted the wider, non-material dimensions of well-being?
- Expected Deliverables
The external evaluation consultant/team will be required to deliver the following key deliverables.
- a)Inception report: The inception report will serve as an agreement between parties on how the evaluation will be conducted. Items to address:
- Appropriate validated draft data collection tools (e.g. methodological guidelines, group interview questions, Key Informant Interview questions guide)
- Raw data in any of the following statistical packages (STATA, SPSS, cSPro) and also transcribed qualitative scripts
- Understanding of the issues and questions raised in the ToR
- Data sources; how to assess the questions in the ToR
- Research methodology, including suggested sample and size
- Schedule of activities and traveling (timeline)
- Detailed budget
- b)Facilitate a workshop for the presentation and validation of findings and recommendations to DKH and CPD.
- c)(Draft and Final) Evaluation report of max. 25 pages (excluding annexes) to be structured as follows:
- Executive Summary
- Summary Evaluation report highlighting the cross-cutting key findings, lessons learned, and recommendations, including
- Evaluation purpose
- Evaluation methodology
- Main findings (presented in terms of achievements based on the process and impact indicators and their alignment with the project objectives & challenges encountered during the implementation of the various activities.)
- Stories of change and quotes from respondents
- Lessons Learned
- Conclusions and Recommendations
- Regarding future steps/activities / follow up
- Regarding improvements in project design
- Regarding approaches and methodologies
Recommendations should be based on evidence and analytically sound. They should be targeted to the relevant audience, and they should be relevant and feasible)
- Relevant maps and photographs of the evaluation areas where necessary (Photos should be Geo-tagged)
- Bibliography of consulted secondary sources
- Finalized data collection tools
- List of interviewees with accompanying informed consent forms
Skills and Qualifications
- Required Expertise and Qualification
The consultant should have the following qualifications, skills, and experience:
- University degree in Education, Development Studies, Social Sciences, or equivalent qualification. Ideally, the persons will have experience in conducting Education evaluations and assessments. Education background, at least a Bachelor’s/Master’s degree in education or equivalent is desired
- Extensive experience in conducting surveys and knowledge of sampling methodologies
- Experience in community-based development approaches / participatory methods.
- Excellent communication and reporting skills
- Team Leader experience.
- Computer literate with skills in data analysis software SPSS.
- Good command of both written and spoken English.
- Familiarity with the Somalia context and education sector in particular
- Familiarity, ability, and willingness to travel within Somalia and under armed escort.
- Extensive experience in teacher training particularly in crisis contexts
- Time frame
The evaluation is expected to start in December 2022 with an inception phase followed by intensive data collection (desk review, interviews, and surveys), analysis, and report writing. A virtual meeting for presentation and discussion of preliminary findings should be conducted on 26 December 2022, and the final revised evaluation report should be delivered by 31 December 2022.
- Selection criteria
The selection for the suitable consultant will be based on scoring on awarding set marks for experience in undertaking similar assignments in education, especially in the complex humanitarian emergencies in Somalia, production of past consultancy assignments and references, and ability to interpret the consultancy assignment and add value to the methodology proposed and required outcomes among other parameters. The consultant shall be selected based on meeting the requirements provided for the assignments as indicated in the sections above.
- Required Application Documents
The consultant shall be required to provide the following
- Technical and Financial Proposal (in USD) with clear breakdown including VAT in USD.
- Updated CVs with 3 referees at least.
- Registration and Tax compliance certificates from the Federal Government of Somalia
How to apply
Your electronic offer comprising of a technical proposal and a financial proposal in one PDF document in English shall be sent to the following email address no later than 2nd December 2022, Mid-Night: firstname.lastname@example.org. For questions on the application process please contact the same email address (email@example.com ).
Incomplete applications will be considered inadmissible for further review by the evaluation selection panel.
Please forward the expression of interest, in English and marked “Consultancy for Mid-Term Evaluation for Project K-SOM-2021-4070” no later than 2nd December 2022.
- Address mogadishu