Terms of Reference for Evaluator of Oxfam’s Disaster Risk Reduction Programme

“Supporting community resilience in the South Caucasus” The purpose and scope of the evaluation In July 2014 Oxfam GB and local partner organisations started the third phase of a participatory multi-hazard disaster risk reduction programme in Armenia and Georgia, funded for 15 months by the European Commission, Department of Humanitarian Aid and Civil Protection. The ‘Supporting Community Resilience in the South Caucasus’ programme is due to end in September 2015 and an external evaluation is sought to determine the impact of this programme to date and to make recommendations to inform future programming for the Oxfam team and partners, as well as the donor, ECHO. The evaluation itself will focus on a number of key questions in the following areas: overall impact of the programme, impact on gender equality, sustainability as well as effectiveness of coordination with government and non-government actors. What the programme aimed to achieve The specific objective of this programme is to increase resilience and reduce vulnerability of local communities and relevant national institutions by supporting strategies that enable them to prepare for, mitigate and respond to natural disasters in the South Caucasus region, through 3 main results:

Increased capacity of local communities (especially women) to identify and address the needs of most vulnerable populations (including women, men, children, disabled and elderly) in the wake of disasters

Vulnerable communities face less risk to potential disasters through planning, mitigation and adaptation

Leading agencies and government authorities demonstrate increased awareness and acceptance of gendered community-based DRR

Questions to be addressed by the evaluation Overall Impact

To what extent were the project results and indicators achieved during the project?

Are the actual project outputs (e.g. the small-scale mitigations, new technologies, the community rescue teams trained, DRR centres, green laboratories, resilient communities, village disaster management, local disaster management plans, etc.) present within a community at the time of evaluation, and to what extent are they actually utilized (or could be meaningfully utilized) by the project beneficiaries?

To what extent can the preparedness-related project outputs potentially be used in actual emergency situations (e.g. whether a rescue team already “rescued” people, whether it would be physically available during an emergency, whether it could – or had the means to - assemble in time, whether the community was ever evacuated in time during drills and simulation exercises etc)?

What were the unintended positive/negative impacts of the programme? Gender and inclusion of the most vulnerable groups

What impact has the programme had on gender equality and women’s realities? (e.g. increased decision making, control over resources/assets, leadership roles within community management structures etc)

What steps were undertaken to ensure the active participation of women within the programme and what steps could be taken in the future to improve this practice?

Did the programme meet specific gender indicators and targets outlined within the project document?

How were vulnerable groups (including children and the elderly) involved in the project? What impact has the programme had on reducing vulnerabilities of these age groups?

Sustainability, institutionalisation and replication of Oxfam and partners’ interventions;

To what extent are changes made/achieved likely to be sustained?1

Are there examples of replication of developed models?

Do communities’ (or the final beneficiaries’) genuinely have ownership of the project outputs? Will there be motivation to continue and develop the project activities after Oxfam and partners’ withdraw?

1 distinguishing between the following types of sustainability o financial sustainability, i.e. ensuring that the “project activities are financed after the EC funding ends” o institutional sustainability, i.e. ensuring the “local “ownership” of the results of the action”; o local sustainability, i.e. ensuring that the “links and coordination with the relevant local authorities” are maintained; o policy-level sustainability i.e. ensuring that the action has a “structural impact” e.g. improved legislation, following codes of conduct,

methods and methodologies adopted etc.

How cost-effective was the programme?

How successfully did the project maximise government, community and other agency resources?

How successfully did the programme leverage additional sources of funding? Examples of good practices and lessons learned;

o Do any of the activities/projects and if so which “show results, innovative qualities and potential for replication”?

o Do any of the activities/projects show results or innovative qualities in relation to gender equality or women’s active participation in the project?

o To what degree have Oxfam and partners learned from this experience as an organisation and shared the learning with partners and other actors? What are the lessons learned?

Advocacy and coordination with the Government, amongst the other actors and with donors;

To what extent does Oxfam’s projects fit into and strengthen the existing local government or wider national or regional emergency response and DRR government systems, institutions or plans?

To what extent has the programme worked with the government (participated in design of activities, utilised government resources, worked to support the capacity development of the government structures)?

To what extent the level of coordination amongst partners and other NGOs contributed to useful exchange of information, sharing of resources, prevention of duplication and did this impact on the beneficiary communities?

To which extent donors are as well coordinated? Resilience and integration of programmes:

How successfully did the project connect with other Oxfam projects and how could a one-programme approach on resilience encompassing several Oxfam projects be designed?

Regional End of Programme Workshop & Evaluation Process The evaluator will be requested to actively participate in the Regional End of Programme Workshop to be held in Armenia at the beginning of September. During this workshop country teams will present successes and challenges faced during the implementation of the project. The evaluator will also conduct evaluation field trips in Georgia (Adjara, Tbilisi, Mtskheta) and Armenia (Vayots Dzor, Tavush, Yerevan), conduct a review meeting in Tbilisi and will develop the evaluation report. The evaluator will be asked to do a desk work prior to the Regional Learning Review and field visits. Documentation review/ Existing data to be used in evaluation

Review of programme proposal documents

Baseline/endline report findings

Monitoring review reports

Frontline national reports and related case studies collected (in Georgia and Armenia)

Best practice write ups for DIPECHO

Review of project materials

Feedback/results of activities / Photos

External evaluation and final report from the second phase programme Overview of programme

Briefings and overview of programme with Programme Coordinator

Briefings with each country team including partners on key progress and challenges: review of data collected including media work, photographs plus overview of mitigation and adaptation work including process and documentation as outlined in log-frame. Country specific indicators and activities will have to be taken into account when evaluating the programme.

Field based data collection

Interviews with national, and local government (6 interviews)

Interviews with other actors (DIPECHO partners, ECHO local representative plus one other per country)

Focus group discussions with beneficiaries (in 6 villages- 3 per country)

Visit to mitigation and adaptation projects (in 6 villages) Restrictions to consider

Some members of communities (particularly in Georgia) leave for upper mountains for cattle-herding Key outputs Executive summary and full report in English including key recommendations (max 22 pages plus annexes).

Suggested contents page Executive summary x 2 pages Introduction and methodology x2 pages Overall Impact x 6 pages Gender x 1.5 page Sustainability of Oxfam and partners’ interventions x 1.5 page How cost-effective was the programme? x 1 page Examples of good practice x 2 pages Coordination with the government and amongst the other actors x 2 pages Resilience and integration of programmes x 2 pages Conclusions and recommendations for the next stage of the programme x 2 pages Additional responsibilities As a part of Oxfam’s commitment to the capacity building of staff, the evaluator is requested to allow for the shadowing of their work in the field and in all data collection. In addition, the evaluator will be requested to allow for for regular Q&A sessions by shadowee. Where the evaluator sees appropriate, they may use the shadowee as a resource to undertake relevant data collection or analysis tasks. Relevant skills sets and knowledge the consultant should have Please note that as per donor requirement the evaluator must be an external candidate and not an

existing Oxfam member of staff Consultant should have permission to travel in the region (Georgia and Armenia) – visa or non-visa travel right, and be based in one of the countries.

Essential - Skills & Experience

Knowledge and proven experience in managing and leading evaluation assignments for a range of major aid agencies or NGOs in particular evaluating complex multi-country or large donor programmes e.g. EC, ECHO

Strong data collection, analysis and writing skills

Broad knowledge of development issues and national policy and practice in DRR (in particular community based DRR)

Familiar with the resilience concept

Good understanding of gender equality issues

Proven experience of working in DRR and/or humanitarian response programmes

Fluency in English

Desirable - Skills & Experience

Experience or knowledge of the South Caucasus

Management experience of community based disaster risk reduction programmes

Spoken and written Russian

Essential - Competencies

Ability to work with significant levels of autonomy

Proven analytical skills and ability to work with others with a dynamic and proactive attitude

Recruitment procedure Candidates are asked to submit a copy of their CV, a cover letter outlining how they meet the key competencies as well as a writing sample (ideally a previous evaluation authored by the candidate) and a financial proposal for the assignment with a lump sum covering a fee and per diems. In addition, Oxfam will provide travel and accommodation in both countries. Assignment dates 31 August - 23 September 2015 Timeline and deliverables Work will be carried out during the following times and reports shared with Oxfam on end dates: Preparatory work, desk analysis of delivered documentation & materials: 31 August – 1 September 2015 Participation in Regional End of Programme Workshop in Armenia, Field visits and Interviews in Armenia (Vayots Dzor, Yerevan and Tavush) and Georgia (Adjara, Mtskheta and Tbilisi): 2-15 September 2015 Summary review meeting in Tbilisi: 16 September 2015 Report development and delivery of the evaluation report: 23 September 2015 Application deadline: 17 August 2015