evaluation department - aidstream › files › documents › sciaf-ukam... · web viewhowever, the...

SCIAF UK Aid Match Joint Evaluation TOR v6 November 2015

Terms of Reference

Evaluating effectiveness, relevance and sustainability of livelihoods and advocacy interventions on women’s empowerment in DR Congo, Malawi and

Rwanda

Introduction

Scottish Catholic International Aid Fund (SCIAF) is the official international development and humanitarian agency of the Scottish Catholic Bishops Conference. SCIAF is working over 16 countries across Africa, Asia and Latin America.

On 1 November 2015 SCIAF will begin a three-year livelihoods, women empowerment and climate change resilience project in three African countries (i.e. DR Congo, Malawi and Rwanda) with support from the UK Aid Match of Department for International Development (DFID). The programme covers 2,000 small-scale farming households in 25 villages with the help of four implementing partners. Within this approach SCIAF will aim to integrate its cross-cutting issues of disability, inclusion, HIV/AIDS, and climate change and the environment.

As part of the programme, DFID and SCIAF have agreed to undertake a joint monitoring and evaluation process that will focus on: assessing the extent to which the intervention successfully and sustainably empowers women and positively changes gender relations within livelihoods, families and communities, and contributes to improved, holistic human development for individuals and families.

A related output will be a tested framework, method and tool for the design, monitoring and evaluation of livelihoods interventions based upon the Integral Human Development framework (IHD)1.

The evaluation will be co-implemented by SCIAF and four implementing partner organisations with technical support and evaluation expertise from a consultant. DFID will provide additional funding for the evaluation, alongside the UK Aid Match funding for the project itself, and engage in the overall governance of the evaluation.

The proposed evaluation is an integral part of the programme. It will draw on and support the routine, regular monitoring work of the programme and undertake additional in depth studies and analysis.

The tools and methodologies should draw on participatory approaches, and incorporate community definitions of ‘well-being’ and ‘gender empowerment’ to SCIAF’s Integral Human Development framework (IHD).

The evaluation will take place at the end of the programme. However, the evaluation ‘process’ will take place throughout the life of the programme (i.e. 3 years – 1 November 2015 to 31 October 2018). The IHD framework will be a fundamental part of the evaluation, and will influence the design of the programme, and the data collected for monitoring and evaluation.

SCIAF and its partners will be fully engaged throughout, with the expectation that capacities for monitoring, evaluation and learning will be significantly developed

1 Within this approach SCIAF will aim to integrate its cross-cutting issues of disability, inclusion, HIV/AIDS and climate change and the environment.

of 17


through the process as a result of interacting with the consultant and through internal learning and reflection processes.

Programme Background SCIAF will implement a 3-year livelihood, women’s empowerment and climate change resilience programme from 1st November 2015 to 31st October 2018 in three African countries (DR Congo, Malawi and Rwanda) with support from the UK Aid Match of DFID. The programme will support 2,000 small-scale farming households in 25 villages with the help of four implementing partners to increase their agricultural productivity, diversify income sources, and develop structures needed for influencing decision-makers and building community-based resilience to climate change. Income security and more control of resources will contribute to empowerment of women and greater gender equality in the target communities.

The programme is covering seven villages of Walungu territories in South Kivu Province of Eastern DR Congo, 12 villages of Ntcheu and Nsanje districts in Southern Malawi and six villages of Bugesera district of Eastern Rwanda.

The programme aims to address absolute poverty compounded by food insecurity and vulnerability to adverse weather events in the target areas. DR Congo is currently 186th, Malawi 170th and Rwanda 167th on the Human Development Index ranking with Multidimensional Poverty Indexes of 74%, 66.7% and 69% respectively. Agriculture is the main source of income and employment – 70% DR Congo and 80% Malawi and Rwanda. In terms of food security, DR Congo ranks 107, Malawi 99 and Rwanda 97 on the Global Food Security Index and their gender equality ranking on the Gender Inequality Index is 109, 124 and 76 respectively. Rural poverty is closely linked to agricultural income and lack of capacity/opportunities to diversify income sources and manage disasters. Small landholdings (around 1 ha) with unstainable farming techniques and dependence on rain-fed agriculture leading to low productivity are a common feature of the target areas. A Stakeholder Steering Committee (SSC) will be formed to contribute to decision-making, assist in targeting beneficiaries, ensure shared understanding of programme activities, baseline information and ultimately provide a channel of accountability to the community. The SSC, led by SCIAF Programme Officer, will facilitate skills sharing, learning and communication and develop relationships. The programme approach includes:

1) Beneficiaries invited for training in agriculture, livestock management, climate change adaption, gender, forming savings and credit groups and development/management of micro business to improve technical knowledge base.

2) Promotion of seed bank and kitchen gardens, supply of basic farming equipment and small livestock for income diversification.

3) Development of irrigation facilities and formation of water users’ committees to reduce rain-fed dependence.

4) Development of community-based disaster mitigation strategies including early warning system and recovery plan to reduce impact of climate change.

5) Promotion of women’s leadership by training in organising and negotiation skills, record keeping and information sharing about women’s rights and Sexual and Gender Based Violence (SGBV) laws for promotion of gender equality and

6) Capacity building of target groups by implementing partners to help them to access government services and formation of farmer associations.

of 17


Anticipated impact – Reduction in poverty levels in target areas of DR Congo, Malawi and Rwanda

Outcome – 2,000 small-scale farming households of 25 villages of DR Congo , Malawi and Rwanda have increased food security, greater gender equality and increased community resilience to climate change.

Outputs 1) Increased agricultural productivity through increased adoption of sustainable farming practices.Output 2) Target communities become less vulnerable to the impacts of climate change by developing and implementing community-based disaster adaption and disaster mitigation strategies. Output 3) Women are skilled and confident, with more control over resources and decision making as a result of changing community attitude and promoting gender equity.Output 4) Male and female small-scale farmers are able to influence state and non-state actors who impact on their livelihoods and are better able to access services through farmers’ associations.

Please refer to the appendix for the programme logical framework.

The direct programme beneficiaries are 2,000 (female 1,560 + 440 male), small-scale farmers comprising widows, single women and women/child-headed households, women survivors of sexual and gender based violence, disabled, people living with HIV and AIDS and ex-combatants. 17,750 (female 11,149 + 6,601 male) family members of the direct beneficiaries are indirect beneficiaries of this programme.

90% of beneficiaries are women and from other marginalised groups and the programme is designed to maintain this rate of participation.

Objectives of the Evaluation

The specific objectives of the evaluation are to:1. assess the extent to which the project has achieved its objectives, and 2. contribute to a better understanding of the factors that:

a. Promote and strengthen livelihoods opportunities for women living in varied contexts;

b. Reduce gender and social inequalities; andc. Enhance women’s (and men’s) integral human development.

Outputs of the evaluation:1. An evaluation report that answers the evaluation questions outlined in this TOR;2. Monitoring and evaluation tool(s) for Integral human Development are designed,

developed, tested and finalised;3. Increased capacity of SCIAF and the four partners to design, implement, monitor

and evaluate livelihoods interventions;4. Increased capacity to use digital data collection and analysis tools amongst

SCIAF and partners;5. One promising practice booklet on the design, monitoring and evaluation of

projects promoting Integral Human Development for women and men.

Questions for the Evaluation

of 17


The evaluation process will focus on specific questions: Relevance How relevant is the programme intervention for women in the different country

and livelihoods contexts?o To what extent does the programme respond to the needs, interests and

capabilities of individual households?o Is the programme targeted at the right people?

Effectiveness How effective are the livelihood activities implemented by women at making a

contribution to raising them out of poverty and low levels of integral human development?

o What are the best entry points to working with women in SCIAF’s livelihoods programmes (groups, individuals, female headed households)?

o What contextual factors support or hinder livelihoods work with women?o To what extent has the programme been successful in responding to the

factors affecting women’s access to extension services? o What mix of livelihoods support is appropriate for women in target areas? o How effectively do government extension services work for women?

How effectively does the programme develop the leadership capacity of women at different levels?2

How effectively does the programme promote gender equality in the context of families and communities?

How capable, willing and effective are we at learning and adapting to findings?

Integral Human Development How can we effectively measure integral human development for both men and

women in different country context?

Role of the Consultant

Note the days here are indicative.

The consultant will:1. Provide advisory and review services in the design of the monitoring system, to

ensure it provides robust information related to the evaluation questions and the IHD framework (10 days);

2. Develop M&E tools for IHD framework, including possible multi-variate indicator and data capture and analysis tools (10 days);

3. Provide technical support to the design and implementation of the baseline survey, and associated gender analysis (5 days);

4. Provide training on the implementation of the monitoring system to SCIAF staff and support them to train partner staff (10 days);

5. Provide technical support to SCIAF and partners in the design and facilitation of annual reviews by SCIAF (12 days);

6. Review annual reports for quality and review the adequacy of the ongoing monitoring system (6 days);

7. Conduct a training session for SCIAF headquarters staff on lessons learned (timing to be determined) (2 days);

2 Social (family and village/community), institutional (bank, government officials/department and market) and political

of 17


8. Carry out a final evaluation of the project (30 days);9. Write a promising practice booklet on lessons learned for publication (10 days).

The consultant may also be asked to attend specific meetings with internal and external stakeholders (e.g. Caritas Internationalis and CIDSE network members), interested civil society and academic organisations as well as appropriate decision-makers in programme countries and United Kingdom as per agreed dissemination plan in order to raise awareness of evaluation findings. Such events would be coordinated by the SCIAF Programme Manager and agreed on a case-by-case basis with the evaluator. The above outputs can be negotiated at the inception phase of the project.

Overarching Theory of Change that frames the interventions: Integral Human Development

Integral Human Development (IHD) is a holistic approach to development based in Catholic Social Teaching (CST). It relates to an understanding of poverty as multi-dimensional and consequently of human development (or the end of poverty) as multi-dimensional. It includes an element of relationships with family and community, and thus gender relations.

This project will test an IHD framework for the design, monitoring and evaluation of livelihoods projects. This is a new initiative for SCIAF as we seek to integrate IHD into our programmes. The evaluation will be based on the IHD framework and will evaluate to what extent women (and men) have improved their livelihoods and reduced poverty as a result of the programme.

Considerable work has been done by agencies such as Catholic Relief Service (CRS), networks such as Cooperation Internationale pour le Développement et la Solidarité (CIDSE), and academic institutions such as the Australian Catholic University to develop and operationalise integral human development in programme work. This evaluation would assess the extent to which the methods and tools trailed in this programme can be adapted.3 Figure 1 below sets out the CRS framework as an example. It is important to note that SCIAF is drawing on Caritas Australia’s work as well as the work of CRS and the CRS model is shown only as an indicator of the kinds of issues to be considered in the model. The Advisory Group began work in mid-2015 and will produce a first draft framework in early 2016.

3 For example CRS manual – “Integral Human Development and Livelihoods Programming”

of 17


Figure 1: The CRS Integral Human Development conceptual framework A diagram of the IHD conceptual framework is shown above in Figure 1. A key purpose of the IHD conceptual framework is to help CRS and our partners become more effective in assisting the people we serve to improve their overall well-being through improved livelihood Outcomes. The primary livelihood outcome sought is Integral Human Development. That is, people are able to lead full and productive lives, meeting their basic physical needs and living their lives in an atmosphere of peace, social justice and human dignity. Livelihood Outcomes are the result of the livelihood Strategies people use. For most people, livelihood Strategies are based primarily on their many different Assets — these are the resources that they have to work with and build on. Systems and Structures are the institutions, rules and social norms that we work within. They affect how different Assets can be used, and in some cases who has access to specific Assets. Individuals or communities with a lot of Assets may also be able to change some of the rules. So there are important interactions between Systems and Structures and Assets. CRS and partner Strategies should also take into account risks that threaten lives and livelihoods. These are listed as Shocks, Cycles and Trends. The IHD conceptual framework depicts these basic components and the interactions between them: households develop livelihood Strategies based on available Assets, and they use these Assets within the surrounding external context (Systems and Structures). Finally, they take into account the major sources of risks to lives, livelihoods and overall well-being as they develop their Strategies.

Outcomes

Shocks, Cycles & Trends

Strategies

Assets

Spiritual & Human

Physical Social

Natural Political

Financial

Structures and Systems

(Institutions; value systems; power structures; social, economic, religious and

political systems and beliefs.)

Influence

Access

Feedback = opportunities & constraints

of 17


It is anticipated that the evaluator will provide support to SCIAF and partners through the programme, as well as conducting a formal, independent evaluation at the end of the programme.

The evaluation process will be based on the IHD framework. An Advisory Group formed of individual SCIAF staff and Board members with expertise in IHD will be formed to advise on this process in Quarter 3 of 2015, and it will be developed as part of the elaboration of the SCIAF Strategic Plan 2016-2020.

While the Advisory Group will provide strategic direction on the framework the Consultant will be expected to provide a technical review of the framework from a monitoring and evaluation perspective and identify requirements for the MEL system to deliver robust information for the evaluation. A review of the theory of change, the log-frame and the MEL plan will be required. Training in implementation of the MEL system may also be required.

In addition to IDH the evaluation should be informed by the OECD-DAC evaluation criteria, and the BOND evidence principles. The IHD framework will be used to assess the different aspects of livelihoods which should be considered by the evaluation, and how these relate to the external policies and processes, household livelihood strategies, and shocks and trends. This will guide a judgement on the programme’ relevance and effectiveness and the BOND evidence principles will be used to judge the robustness of data used in the evaluation.

The IHD measurement may require a multi-variable measure for IHD, implemented through a participatory approach that can be used for the review of programme designs. The consultant will be expected to advise on the suitability of such a tool, and how it could be used. SCIAF is also interested to see how the Most Significant Change approach and longitudinal case studies could be incorporated into the evaluation system. The consultant should also advise on the feasibility of this.

It is anticipated that the consultant will review the logical framework to ensure it is compatible with the IHD model (The joint evaluation plan has been developed after the programme approval so the IHD indicators are not included in the programme logical framework but it will be included during the programme first year – 1 November 2015 to 31 March 2016 in consultation with four implementing partners.)

Once finalised the theory of change, log-frame and MEL plan will be used to underpin programme implementation. The consultant will oversee a phased approach that progressively builds evidence on the relevance and effectiveness of the intervention. Throughout these phases it is intended that a lead evaluator helps to frame, analyse, synthesise and quality review products produced by SCIAF and its four implementing partners. SCIAF will collect data and implement the tools. The consultant would focus on system definition and support data analysis; their data collection role would be limited in order to reduce costs. The programme will collect data to monitor against milestones e.g. annual household survey, gender analysis, risks and vulnerability assessments, annual yield survey and village development plan. The datasets will be both qualitative and quantitative and gender disaggregated. During first six months of preparatory phase, SCIAF will explore possibility to have or don’t have comparison/control groups for this programme.

The consultant will work closely with the SCIAF Programme Manager, Programme Officer and the SCIAF MEL Lead based in Glasgow, Scotland and two local

of 17


coordinators based in Malawi and Rwanda; and the Advisory Group. The four implementing partners will hire target village based data collectors (male + female) to support all M&E activities. A job description will be developed for the data collectors. Data collectors – ten each for DR Congo Malawi and Rwanda = total 40 – 1 data collector for 50 beneficiaries.

The two annual self-reviews in year 2 and 3 will take place under the guidance of SCIAF with inputs from the evaluation consultant. Learnings, findings and changes (if any) will take place during the annual review meetings in Rwanda. The consultant will conduct the end of programme evaluation will focus on a specific case study.

In the final phase of the programme the consultant will conduct a full, independent evaluation of the programme, using the evaluation framework set out at the beginning of the programme.

Key tasks Phase 1 (preparatory phase – 1 June 2015 to 31 March 2016) Advisory Group established in June 2015. Identification of a consultant for SCIAF UK Aid Match joint evaluation project by

end of January 2016. Identification of two local coordinators with active involvement of the

implementing partners by end of November 2015. Formation of Advisory Group and development of an IHD framework. Development of baseline survey and gender analysis tools, training materials and

plan by December 2015. Identification of key elements of IHP that will be introduced across the

programme (including multi-variable measure + household wellbeing in terms of women empowerment), which would be tested throughout the evaluation by end of March 2016.

Support baseline survey and gender analysis carry out and production of reports (partner wise and consolidated) by end of March 2016.

Development of monitoring tools by end of March 2016. Support identification of key elements (IHP tools including multi-variable

measure) of IHD that will be affected across the project, which would be tested throughout the evaluation from end of March 2016.

Propose design of monitoring and evaluation premised on the results of the baseline and understanding of the theory of change by end of March 2016.

Consultant, implementing partners, SCIAF will be involved in all above activities. DFID and SCIAF will review phase one in May 2016.

DFID and SCIAF will develop agenda for all review meetings at least one month in advance.

Phase 2 (monitoring and implementation phase – 1 April 2016 to 31 October 2017) Review implementation of the monitoring system in first year annual review and

report by end of April 2016. Support quality review of monitoring data during second year 6 month stage data

collection and analysis by end of October 2016. Feed analysis into second year annual review and report by end of April 2017. Third Year 6 month stage data collection and analysis by end of October 2017. Revise Theory of Change and monitoring and evaluation tools by May 2017.

During this period the SCIAF Programme Officer will continue to make field visits in all target areas and an annual audit will be carried out by the external accountant.

of 17


Phase 3 (evaluation and information dissemination phase – 1 April to 31 October 2018) Conduct an evaluation of cumulative outcomes achieved across the programme

in relation to evaluation question by April 2018. In depth case study in sub-set of countries/intervention themes by June 2018

related to gender issues. Develop materials to support the use of evaluation findings in SCIAF and beyond

– production of promising practice booklet by June 2018 and information dissemination phase from July to August 2018.

The final phase of the evaluation will need to provide a broad appraisal of programme outcomes across the countries as well as joint evaluation process, based on a synthesis of all available monitoring and panel data and an end-line study in each of the three countries (specifically addressing the first of the evaluation questions given above). This will be supplemented by a more in-depth evaluation that will be in the form of a case study.

The evaluation coordination committee in consultation with programme teams and the evaluator will define the case study of interest based upon the experience of implementation. It is expected that the case study will cover three countries and further unpack an intervention or set of interventions that show particular promise to contribute to development impact. The case study seeks to move from annual review and monitoring processes to generate a high quality evaluation report on the gender issues in relation to programme design. The case study is expected prioritised on the following criteria:

1. Likelihood of a contribution to development impact – Based upon the monitoring and review information an area of particular success will be identified. This could be in one output area or span a number of outputs.

2. Feasibility of drawing out additional learning – The available review and monitoring information will provide an indication of success, but also raise further questions around the intervention.

3. Salience of the findings – The case will be targeted at an intervention area that is of interest to SCIAF related to women’s livelihoods.

4. Data available in three countries – The case will be focused on a unit of analysis that is of interest in three countries. This is to help understand how change takes place within context.

The case study will document positive (successful) and negative (unsuccessful or any other) impact on the individual and on the community including government role.

DFID will have unlimited access to the material produced by the joint evaluation.

Sampling

The programme is implemented across three countries: Malawi, Rwanda and DR Congo. Within those countries a total of four organisations (the implementing partners) will be involved in implementation:

Malawi: CADECOM DedzaCADECOM Chikwawa

Rwanda: CDJP Kigali

DR Congo: CDJP Bukavu

of 17


The evaluation process (including baseline and development of the monitoring system) will involve samples from all four implementing partners. The final numbers will be agreed upon with the consultant, four implementing partners, SCIAF and DFID.

Gender issues to be addressed

Gender is central to the evaluation. As set out in the evaluation questions it will seek to improve our understanding of how livelihoods interventions can promote women’s empowerment, and foster holistic human development for both men and women.

Target audience/s

Target audiences in the evaluation include:

Target Audience Nature of Engagement Mechanism for Engagement

Women and men in the programme area.

Learning about gender relations including decision making and integral human development; monitoring and evaluating the programme.

Engagement in the reviews; information sharing by the implementing partner programme teams.

SCIAF staff and the implementing partner organisations involved in the programme.

Learning and adaptation of the programme as it evolves;

Future programme and programme design, monitoring and implementation influenced by learning from the evaluation.

Participation in the monitoring and evaluation;

Participation in the evaluation.

Implementation of monitoring tools.

Catholic Church in Scotland, Malawi, Rwanda and Burundi.

Learning from the programmes about gender and livelihoods and integral human development.

Participation in the review meetings and evaluation.

DFID staff concerned with the specific questions within the evaluation.

Learning from the evaluation during and at the end of the programme.

Review of the evaluation reports.

SCIAF as an organisation and four implementing partners.

Future project and programme design and implementation influenced by learning from the evaluation.

Briefings and sharing of evaluation findings in internal learning processes;

IHD method and tool used for design, monitoring and evaluation of future projects.

The CIDSE and Caritas and Internationalisé

Learning from the evaluation during and at the end of the programme.

Evaluation report shared within CIDSE;

of 17


networks. Integral Human Development method and tool shared with CIDSE.

Local government officials especially the extension workers of DR Congo, Malawi and Rwanda.

Supporting and training farmers in increasing food production and accessing government programmes.

Regular visit to farmers (monthly or quarterly) and training.

Management of the Evaluation

SCIAF has overall responsibility for the evaluation management, monitoring achievement of outputs against the performance indicators in the logical framework; and also for providing or organising technical support and the implementing partner capacity building and coordinating reporting and evaluation.

SCIAF will provide guidance and mentoring support to the implementing partners in information/data collection, tools development, developing beneficiary selection criteria and will play a key role in the baseline survey, village development plan and gender analysis and in developing further actions based on the findings.

The four implementing partners will be responsible for direct programme/evaluation implementation on the ground and grassroots-level monitoring, government relations, reporting to SCIAF and data collection.

Overall management of the evaluation will be by the SCIAF Programme Manager responsible for the UK Aid Match programme, with support from the Programme Officer and the Design, Monitoring and Evaluation (DME) Lead based in Glasgow.

A Programme Officer has been recruited to solely manage this programme. This Programme Officer will work with two in-country Coordinators; will have a day-to-day role in the coordination of the work, support to the national partners, ensuring quality data collection and data analysis and monitoring and evaluation of the programme activities.

A Programme Management and Monitoring Committee will be established made up of SCIAF and the implementing partner staff to oversee the programme. They will be review quarterly reports to be prepared by the Programme Officer and local Coordinators. The Programme Management and Monitoring Committee consist of 11 people (male 7 and female 4). See organisational chart below. The quarterly skype meeting will be organise for report review and programme related matters. The members will be graduates, experienced programme managers and have community organisation, livelihoods, agricultural, financial and M&E skills. The programme and joint evaluation project will be monitored as part of a comprehensive, documented, monitoring, evaluation and learning (MEL) plan. The MEL plan will be developed jointly by SCIAF, DFID and four implementing partners in consultation with a consultant within six months of the programme.

The joint evaluation programme is based on the following key principles – stakeholder participation, learning, accountability and value for money.

of 17


SCIAF and its four implementing partners will carry out annual reviews as the programme progress and the joint evaluation in consultation with the consultant and DFID at the end of each year. The findings (data/information) will be incorporated into the programme and make revision if require. This will contribute to learning and improving attainment of intended outcomes for SCIAF and its four implementing partners.

The joint evaluation process aims to support SCIAF and its four implementing partners become more accountable to their constituents – beneficiaries, communities, governments and donor (UK Aid Match). The findings and outcomes of annual deliberation will be shared with the constituents in open and meaningful ways to promote better relationship and accountability. Their feedback mechanism will be developed with the help of the consultant and advisory group. This will build confidence and capacity in constituents especially among the beneficiaries and communities to hold others more powerful to account is the essential cornerstone of social change.

The SCIAF Programme Officer will be responsible for the day-to-day management of the programme along with four implementing partners. She will work with two local coordinators (in-country appointments) and report to the Programme Manager. She will be involved in the programme-related M&E work.

The 40 data collectors will be identified with the active involvement of the implementing partners, community leaders and consultant. The data collectors (M/F) will be from the target areas, literate and responsible for data/information collection of the programme (as per the log-frame and the joint evaluation programme ToR). The in-depth data collection training will be given to them and training syllabus, roles and responsibilities and selection process will be determined prior to the selection with the help of consultant. SCIAF will be responsible for the quality of data collection and close monitoring of this group. The consultant will finalise design of the system and provide support for training of the data collectors. Together with the SCIAF DME Lead and the Coordinators, there will be regularly random selection of collected data to test and ensure quality of data. Currently SCIAF is managing such data collectors in one of its livelihoods programme in Rwanda and DR Congo supported by DFID. The four implementing partners are responsible to provide all logistical support including travel to data collectors, local coordinators and SCIAF/DFID/Consultant as and when they will travel to respective countries.

The implementing partner staff responsible for M&E and report writing will work with the data collectors and collect raw data. This collected raw data will be shared with the local coordinators and basic data analysis will take place at field level. The consultant and SCIAF DME Lead will provide guidelines for basic data analysis to the implementing partner staff and local coordinators in consultation with the SCIAF Programme Manager/Officer. The consultant will conduct advance level of analysis and will be in regular contact with the local coordinators and the implementing partner staff through Skype, telephone and email. The findings will be shared with the implementing partners and SSC on regular bases. As much as possible the programme will encourage local analysis and strengthen capacity to do this.

Potential Risks and challenges

Risks and challenges Mitigation planWorking in two francophone countries so difficulty in effective communication.

Appointment of staff – Scotland and in-country – who can speak French and

of 17


English effectively.Difficulty in managing four implementing partners in three countries.

Appointment of exclusive Programme Officer and two local coordinators.

Identification of appropriately experienced consultant for evaluation.

Timely and widely circulated advertisement to identify proper consultant.

Negative impact of Burundi and DR Congo presidential election on the Great Lakes Region and programme – conflict and insecurity.

Close watch on implementing and local situation and developing appropriate mitigation strategy for programme implementation.

Beneficiaries and government officials are not cooperative and supportive.

Sensitisation campaign for beneficiaries and government officials on programme.

Natural disaster – long dry spell and/or flood – in target areas.

Programme is developing community based disaster mitigation plan.

Stakeholders will not cooperate and support in the evaluation process.

The programme will develop agreements with stakeholders to participate in the evaluation process.

Governance and Co-ordination with DFID

The SCIAF Programme Manager is responsible for this joint evaluation and the implementation of any recommendations from it. He will work with the SCIAF DME Lead based in Glasgow. The two local coordinators based in Rwanda and Malawi will be responsible to the Programme Manager. The Programme Manager is the contact point in SCIAF for the evaluator.

The Evaluation Coordination Committee consists of the SCIAF Programme Manager, DME Lead, Head of International Programmes, and a representative of DFID. The SCIAF Programme Officer and International Financial Accountant will be invited as and when require. The evaluation coordination committee will meet twice a year. The main evaluation products will be reviewed by DFID quality assurance mechanisms. But at all significant points in the evaluation (e.g. ToR for baseline, evaluation, etc.) the evaluation coordination committee will meet and discuss and approve products. Terms and conditions of this committee will be developed by 31 May 2016. The Programme Manager will act as link between the programme and evaluation. SCIAF will report to DFID on an annual basis.

Digital Data Collection

SCIAF is piloting a range of digital data collection methods across its programmes. SCIAF have significant capacity in programme-based used of digital data collection and support for the implementing partners in the use of smart phone and tablet –based data collection. SCIAF currently have pilots using a range of approaches in Malawi, Ethiopia, South Sudan and Nicaragua and El Salvador using digital data collection for baselines and regular monitoring and evaluation, and are experimenting with web-based database for implementing partners to enter monitoring data for reporting purposes.

These offer potential gains in terms of efficiency of data collection and aggregation and data quality, reporting and making data available to all the implementing partners in a more equitable fashion.

of 17


SCIAF plan to test these tools in the UK Aid Match programme this will be a significant test for the data collection learning and builds on SCIAF experience to date. A key lesson SCIAF have learned to date is that local support is needed and the training of the local coordinators in this technology and their support for the implementing partners will be key in successful use of the technology.

Programme Timeframe1. Advisory Group established in June 2015.2. Identification of two local coordinators with active involvement of the

implementing partners by end of October 2015.3. Programme start-up workshop in Cyangugu, Rwanda for all four implementing

partners in November 2015.4. Identification of 40 data collectors by four implementing partners by end of

December 2015. The implementing partners will provide orientation to data collectors after the programme start-up workshop by 15 January 2016.

5. Baseline survey and gender analysis report (implementing partner wise and consolidated) by end of March 2016.

6. Support four implementing partners to develop village development plans based on the baseline survey and gender analysis by end of June 2016.

7. Programme implementation and ongoing monitoring from 1 November 2015 to 31 October 2018.

8. Final case study to be presented by July 2018.

Person Specification for the Consultant

The Consultant(s) should have the following experience and qualifications:1. Educated to degree level in social sciences, agriculture, evaluation

methodologies, or similar (essential);2. Proven track-record of leading similar evaluations, 5 years minimum (essential);3. Experience of working in sub-Saharan Africa (essential);4. Awareness of and appreciation of gender issues in livelihoods and Sub-Saharan

Africa (essential);5. Experience of using participatory methodologies in monitoring and evaluation

(essential);6. Fluent in English and French languages (essential); 7. Experience of quantitative and qualitative data approaches and an ability to use

both approaches, collecting and analysing information in a robust way (essential);8. Ability to carry out value for money and efficiency analysis of projects (desirable);9. Experience of multivariate indicators, case studies and most significant change

approach (desirable);10. Where a team of consultants is employed, a mix of male and female would be

(desirable);11. Education to Post-Graduate level in related subject area (desirable).

of 17


Organisational Chart

SCIAF In-Country Coordinator

SCIAF Design, M&E Tech Officer

CADECOM DedzaSecretary

CADECOM ChikwawaSecretary

CDJP KigaliProject

Coordinator

CDJP BukavuPartner Programme

Officer

SCIAF Programme Officer

SCIAF International Finance Accountant

SCIAF In-Country Coordinator

of 17


Timeframe

Please refer evaluation approach section for detail information. The basic timeframe for the evaluation will be as follows:

UK Aid Match Programme MEL Cycle (major events only)

of 17

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12

Annual ReviewAnnual Report and Revised Log-frame

On-going Monitoring – Quarterly Reporting

Annual ReviewAnnual Report and Revised Log-frameEnd of Programme Evaluation

Baseline Data Collection + gender analysis + Programme start-up

workshopVillage devt plans


Budget

Request to DFID: £117,256

Note this budget is additional to the budgeted activities in the agreed programme. Also note that significant part of the cost relates to the work of two local Coordinators to be employed to support the programme in the region. These were not foreseen in the original budget. The budget includes VAT and is for guidance. The budget is attached herewith for reference.

Particular Amount (£)

Honorarium to data collectors (40 data collectors x £16 ($25) per data collector/month x 36 months). 23,040

Consultancy support for baseline survey/report; gender analysis village development plan; training, development of IHD assessment and selected monitoring tools and theory of change; preparation of final report and ‘promising practice’. Includes consultancy fee, travel (DR Congo, Malawi and Rwanda), and associated costs.

55,000

Training on digital data collection/analysis (local Coordinators, Programme Officer, 40 data collectors and four implementing partner staff + two trainers = 46 persons) in various locations/day. Five days training - £58 per person.

13,340

Annual review meetings (year two and three) in Rwanda/Malawi for three days for nine persons (eight implementing partner staff + two local Coordinator + PO). £7,700 per year x two years.

15,400

DME Lead/PM and PO travel for programme visits and annual reviews (twice for two people [two already included in programme budget for each]).

5,476

IHD programme implementation and M&E practice booklets. Translation, graphic design and printing (English/French/Spanish). 5,000

Total cost 117,256

Bids are requested up to £55,000 for the consultant (first choice will be implementing/local consultant) to deliver.

Provide advisory and review services in the formation and implementation of the monitoring system, particularly to build information related to the evaluation questions.

Train people on monitoring system. Support the conduct of annual reviews by SCIAF. Undertake a case study evaluation process towards the end of the programme. Write promising practice booklets.

Appendix – the programme logical framework. Please refer sheet 1 for more information on joint evaluation.

of 17

evaluation department - aidstream › files › documents › sciaf-ukam... · web viewhowever, the...

Documents