Terms of Reference
Country: Somalia
Location : Bardera, Luuq and Elwak districts of Gedo region, Adaado, Abudwak and Balanbale districts in Galgaduud region and Afmadow and Badhade districts of Lower Juba region
Starting date: March 3rd 2014 (indicative and subject to the prevailing security/access context)
Duration of the field mission: 27 days
Time dedicated to the desk review and to the writing of the report: 7 days
Under responsibility of: M&E Coordinator
HQ follow-up: Desk Programs Manager
Funding Sources for the evaluation: ECHO/HF/BUD/2013/91021
Presentation of Solidarites International
Solidarites International (SI) is an international non-governmental organization which provides humanitarian assistance to population affected by natural disaster or man-made crisis.
For more than 30 years, SI has concentrated its action on meeting three vital needs: Water, Food and Shelter. In the Horn of Africa, SI is currently implementing Food Security, Livelihoods support as well as Water and Sanitation projects in Somalia and Kenya.
Brief presentation of SI in the country
SI has had an implementing presence in South and Central Somalia since 2007, having opened programs in Gedo and Lower Juba regions in 2007, followed by the opening of the Galgaduud program in late 2009.
SI continues to implement projects with its own staff in all 3 program areas. SI is one of the few International NGO operating directly in Gedo and Lower Juba regions since 2007.
The main focus of SI's program has been to address the immediate urgent needs of IDPs (conflict and drop-out pastoralist) and local host populations, while continuously working to strengthen community coping mechanisms and resilience to natural disasters (drought).
Main activities include: WASH – rehabilitation and construction of water points, mainly water catchments, birkads, shallow wells and boreholes’ support through fuel and spare parts; hygiene promotion, distribution of ceramic filters and contingency stocks, as well as sanitation – construction of latrines, waste management-, Food security and livelihoods’ support through food vouchers distribution; milking cans distribution; farming and livestock support through deworming, vaccination and training of community animal health workers (CAHWs).
SI activities are complemented by a partnership with SADO a local organization in Gedo (Bardera) and Galgaduud (Balanbale) regions.
SI and SADO partnership provides the required synergy in terms of capacity building and improved access to locations with clan conflict dynamics respectively.
SI is currently implementing 4 projects in Somalia in WASH and food security in the three regions of intervention, including the current ECHO.
Targeted beneficiaries of SI’s program include protracted and recent IDPs, drought affected pastoralists, riverine and agro-pastoral farmers and marginalized clan communities.
Presentation of the project/ programme to be evaluated
The project has started in May 2013 for 11 months in the three intervention areas and is a continuation of previous project co-funded by ECHO with similar objectives and activities in WASH and Food Security.
The main objective of the project is to contribute to an improvement in resilience to drought and food security crisis among drought and conflict affected populations in South and Central Somalia.
More specifically it aims at providing lifesaving WASH and Food Security interventions for targeted drought and conflict affected households, while improving the resilience of their livelihoods strategies for future shocks.
Result 1: The targeted households have access to improved WASH services and have adopted key safe hygiene practices
Result 2: Improved food security during emergency periods, complemented by improved disaster resilience and access to livelihoods recovery assistance
Activities under Result 1.
- AWD/Cholera prevention and response: Pre-positioning of contingency stock of PUR/bleach/chlorine/buckets/soap for 3,400 vulnerable host and IDP households
- Provision of household ceramic water filters and associated O&M training for riverine, pastoral, agro-pastoral and IDP population
- Construction of 7 birkads of 500m3 capacity for pastoral and agro-pastoral populations
- Rehabilitation/reconstruction of 7 water catchments (6,000m3) and construction of 7 water catchment fed infiltration wells for agro-pastoral and pastoral areas (Using cash for work approach for excavation of catchment).
- Rehabilitation/reconstruction of 12 existing shallow wells for pastoral, agro-pastoral and IDP populations
- Formation and training, or refresher training, of water user committees and technicians for all new/rehabilitated water infrastructures
- Construction of 150 household latrines in specific areas where there is high risk of AWD/Cholera outbreak due to open defecation for vulnerable host and IDP population
- Disposal and management of solid waste in IDP encampments and urban centre through supporting 28 women and youth groups in solid waste management
- Hygiene promotion through campaigns/community sensitization training sessions and PHAST training 1.10WaSH emergency response capacity: Water vouchers for 1,800 households and fuel subsidy for 10 boreholes.
- Provision of 20-litre water containers to 650 vulnerable riverine population
Activities under Result 2.
- Support for beekeeping production, targeting 300 vulnerable riverine households
- Engagement of 700 households with limited purchasing power in Cash For Work activities (linked to rehabilitation of water catchments (activity 4 under Result 1)
- Provide emergency food vouchers to 1,000 households vulnerable to food insecurity during the peak hunger gap
Monitoring
Activities and indicators are being monitored by the field teams supported by the M&E coordinator through the update of the Project Implementation Plan (PIP) on a monthly basis.
This EXCEL format report presents the project work plan per week/per month, including M&E activities: surveys, questionnaires and other sources of verification to measure result indicators as well as process indicators.
The PIP provides SI teams (both on the field and at coordination level) with quantitative information on project progress, complemented by a qualitative analysis based on a checklist of “critical incidents” based on the twelve COMPAS quality criteria1. (A monitoring plan had also been developed to monitor process, activities and result indicators through a range of tools such as Post Distribution Monitoring surveys; pre and post KAP surveys, monitoring forms as well as direct observation from the field staff.
Purposes of the evaluation
The purpose of this evaluation is to draw lessons from the experiences and challenges the programme team has built in the course of the project.
This evaluation will cover all the dynamics of the project including evaluating each and every activity as per the proposal and former projects’ evaluations of similar activities in the intervention areas.
It will help SI in validating the relevance and impact of proposed activities and reorient or modify if necessary future programming. In particular, lessons learned on specific activities described below are critical.
The project will be shared with the donor and local partner and is meant to be used internally by SI.
Scope and focus
The evaluation will focus on the operation’s approach, the implementation process and the performance of the project.
The project should be evaluated through the following criteria: relevance, coverage, effectiveness, efficiency, sustainability, impact, coherence, accountability and any other relevant criteria the evaluation team if deemed appropriate.
Recommendations for improving each of the criteria are made.
A suggested action plan corresponding to each recommendation is included in the evaluation report. Recommendations for the strategic orientation of the next project are made.
Relevance, effectiveness and sustainability criteria will be particularly focused during this evaluation.
The evaluation must give answers to the following questions:
- Did expected results fulfill the needs? (relevance) In particular for food and water vouchers: assess the relevance of these activities according to the needs and specific context in terms of access and availability, as well as other factors such as security, risk of diversion etc. Analyze the best approach and make recommendation for further programming (e.g. in-kind vouchers to be exchanged locally; cash for work or unconditional cash vouchers or cash transfers approach) (relevance)
- Did the activities implemented allowed to reach the expected results and specific objective of the project? (impact)
- Does the program cover the initially targeted population? (coverage)
- To what extent has targeting been appropriate, taking into consideration both exclusion and inclusion errors? In particular, to what extent gender has been taken into consideration for analyzing the needs and targeting specific groups? In particular for Cash for work: how was the targeting process conducted and how most vulnerable households (including female headed, pregnant women, lactating mothers, children, elderly and people with disabilities) have actually benefited from the cash distributed? (relevance and coverage)
- Are the project activities timely implemented as planned? ( effectiveness of work plan implementation)
- Are the monitoring tools adapted to the context and do they allow information to be delivered on time? (Effectiveness and efficiency). This aspect should also include an analysis of the comparative advantage of different monitoring tools / methods in place and whether they allow for triangulation of information and identification of problems.
- Are beneficiary feedback mechanisms in place, functional and adequate? What are the feedback and complaints mechanism in place in the targeted locations? What is the role of Village committees / elders in managing and following up on feedback and complaints? More generally, what is the composition and selection process of village committees, as well as the relevance to the needs of specific vulnerable groups (representation)?
- What are the main recommendations that can be made to improve local accountability towards beneficiaries? (accountability)
- How are the resources being utilized in the course of project implementation so far? Are the Human Resources and Logistics mean adapted and sufficient? Are the roles and responsibilities of the team members clear and efficient (efficiency)
- Are results of activities sustainable and to what extend? How successful were the training activities?
- What is the level of dependency of the beneficiaries to SI support and what could be done in future programming, given the context in Somalia, to decrease the level of dependency and increase ownership and sustainability (e.g. through conditional aid)? (sustainability)
- What negative or positive midterm influence (socioeconomic, environmental, security…etc.) of the project is already foreseen – at household level, community, village, district? (impact)
- Has the project been adapted appropriately to changing needs or context (flexibility)
- Can the project be seen as complementary to other actions on the ground (either by SI or other stakeholders)? (coherence)
The following specific activities should be evaluated according to the following considerations:
- Water filters: What is the willingness of local suppliers to purchase and sell ceramic water filters and to what price? What are the stocking capacities? What is the local communities’ willingness to purchase water filters?
- Performance and sustainability of water users committees: especially on the use of fuel subsidy, spare parts and management of water points: how are the fees collected and used for the maintenance of the water points? What is the impact of trainings and support on WUCs? What is the composition of the WUCs in the different areas of intervention? What specific recommendations could be drawn to improve management of water points by WUCs
- Cash for work: analyze the impact of Cash for work activities on local economies and how monitoring mechanism could be adapted to capture this impact
- Hygiene promotion: evaluate the hygiene promotion approach (methods, tools, messages content and local perception/ understanding) and the impact on beneficiaries' attitudes and practices, identify key gaps and make recommendations
The evaluation should also assess the appreciation of the program by the beneficiaries as well as their participation at various levels of the project management cycle.
Finally, the evaluation should assess how the implementation of the program is respectful of the ethic of humanitarian practice vis-à-vis the SPHERE standards and the Code of conduct for the Red Cross Movement and for NGO during emergency intervention.
Evaluation process and methods
- The evaluation methods should be clearly outlined in the report and their appropriateness, relative to the evaluation's primary purpose, focus and users, should be explained pointing out the strengths and weaknesses of the methods. A description of the overall flow of the evaluation process (i.e. sequence of the key stages) should be given in the evaluation report. The evaluation approach and the methods used to collect and analyze data should also be described. The nature (e.g., external or mixed) and make-up of the team (e.g. sectorial expertise, local knowledge, gender balance) and its appropriateness for the evaluation should be outlined.
- During field mission, following stakeholders should be consulted and interviewed and the list of consultations provided in the evaluation report: authorities, beneficiaries (balanced between men, women, girls and boys) of each activity and livelihood groups, including most marginalized; WUCs, Village committees; elders; SADO and SI staff; youth and women groups (waste management).
- The evaluation report should outline the sources of biases that might affect the evaluation and how these have been addressed.
- The evaluation report should also present the key constraints to carrying out the evaluation (e.g., lack of baseline data, lack of access to key information sources, use of translators), and the effect of these constraints.
- Whenever secondary sources will be referred to, the evaluator should indicate the level of reliability of the given information.
- After the field work, the evaluation team will present and discuss with the project team the preliminary findings and the proposed recommendations.
- A first draft of the evaluation report should be shared with the coordination team of Solidarites International before a final version is validated.
Procedures and logistics
- The evaluation team must have full access to all of intervention areas in Somalia. Access to Bardera district is considered an asset.
- The evaluation team must comply with Solidarites International rules and procedures related to security and relations with the media.
- The evaluation team must respect the ethic and the deontology related to evaluation practice.
- Logistics, movement and security would be provided and organised by Solidarites International team
Deliverables
The evaluation report should include at least:
- One narrative report (max 40 pages) including an executive summary (2 pages maximum).
- A separate table summarizing the main findings and the lessons learned, including corrective measures for further similar projects.
- A separate table showing the different recommendations and tips for their implementation (who will be in charge of implementing this recommendations, when? dead line? necessary means? who will be in charge of checking that the recommendations are being implemented and when? etc.).
- Detailed plan of visits and interviews conducted in the field
- Relevant maps and pictures of the assessed zone and programme.
- A Power point presentation of the main findings and recommendations of the evaluation must be submitted to SI in order to facilitate dissemination of the results of the evaluation to stakeholders.
Documents of reference
- Format of the evaluation report
- Planning of the evaluation
- Proposal of the project – former proposals if necessary and the NCE request
- Last Interim project report
- Current organizational chart
- Last Activity Progress Update of the programme
- Existing Monitoring and Evaluation tools/framework
- PDM and preKAP reports
- Relevant maps
- Beneficiaries database per activity and location
- Security guidelines
Note:
All internal documents provided by SI to the consultant will remain SI’s property and are confidential.
Sharing individual and identifiable data can constitute a risk for the beneficiaries and is a violation of their privacy.
As such, the consultant must not share files containing this type of information with any third party.
To ensure the protection of program participants the consultant must also agree to destroy any file containing personal data upon completion of the mission.
Qualifications/Experience Required
- University degree in Project management, Water and Sanitation, Food Security or related field
- Minimum 5 years of proven experience on humanitarian programming
- Proven experience of project evaluation or assessment tasks in similar reduced access context
- (Somalia)
- Strong methodology and writing capacity
- Somali speaker would be an added advantage
- Note: The field work for this evaluation will require qualified consultants to access the 3 zones of intervention.
- Field work should be done in all 3 sites at the same time. Applicants are therefore requested to present the profile (CV) of each consultant to be involved in the field work. The access and capacity of the field consultants will be one of the primary selection criteria for this consultancy.
How to apply
Please send your proposal, highlighting the following:
- A brief introduction of bidding firm or person, including the relevant CVs
- Your understanding of the Terms of Reference
- Proposed methodology and approach (evaluation criteria).
- Proposed work plan
- Confirmation of the availability of each proposed consultant (in case on an evaluation team)
- Itemized financial proposal
Proposals, including all relevant supporting information (CV, technical and financial proposal, work plan and 3 contact references), should be sent to info@solidarites-kenya-som.org before 11pm on 21th February 2014.
Please indicate the consultancy you are applying for in the title of your email.
Only short-listed applications will be contacted.