Tsunami Evaluation Tsunami Evaluation Coalition:Coalition:

Progress to DateProgress to Date

A L N A P. . . . L E A R N I N G A C C O U N T A B I L I T Y Q U A L I T Y

I N H U M A N I T A R I A N A C T I O N

Background Background

OCHA, WHO, ALNAP Secretariat OCHA, WHO, ALNAP Secretariat ++DANIDA, USAID DANIDA, USAID

Level 1 Purpose:Level 1 Purpose:To promote a sector wide approach to evaluations of tsunami To promote a sector wide approach to evaluations of tsunami response in order to optimise sector wide learningresponse in order to optimise sector wide learning

AimsAims1.1. To To enhance coordinationenhance coordination through the establishment of an through the establishment of an

‘evaluation coalition’‘evaluation coalition’2.2. To To increase comparabilityincrease comparability & & reduce unnecessary duplicationreduce unnecessary duplication 3.3. To To strengthen policy levelstrengthen policy level as well as operational evaluation and as well as operational evaluation and

learninglearning4.4. To To increase and strengthen learning opportunitiesincrease and strengthen learning opportunities

Level 2 Purpose:Level 2 Purpose:To develop and test procedures for future timely To develop and test procedures for future timely

establishment establishment of an evaluation coordination mechanism (coalition) that of an evaluation coordination mechanism (coalition) that

could could facilitate such an approachfacilitate such an approach

Concept PaperConcept Paper (Geneva Feb 2005)(Geneva Feb 2005)

Developing a Sectorwide Approach for Developing a Sectorwide Approach for Evaluation Tsunami AssistanceEvaluation Tsunami Assistance

1. Map1. Map

(ALNAP Full & Observer Members = 500 +; handout)(ALNAP Full & Observer Members = 500 +; handout)

2. Options2. Options for way forwardfor way forward

(Lessons from previous joint evaluations)(Lessons from previous joint evaluations)

OPTION 1OPTION 1::

INFORMAL FRAMEWORKINFORMAL FRAMEWORK

Key concept:Key concept:

Harmonisation & rationalisationHarmonisation & rationalisation

Key features:Key features:

Sharing ToR; avoiding duplication; sharingSharing ToR; avoiding duplication; sharing

evaluative reports evaluative reports

OPTION 2OPTION 2::‘‘COMMON EVALUATION FRAMEWORK’COMMON EVALUATION FRAMEWORK’

Key concepts:Key concepts:

Comparability for increased lesson Comparability for increased lesson learninglearning

Improved quality Improved quality

Key features:Key features:

CEF (common issues & questions)CEF (common issues & questions)

Active facilitation by OCHA & ALNAP Active facilitation by OCHA & ALNAP Secretariat Secretariat

OPTION 3OPTION 3::

FORMAL FRAMEWORKFORMAL FRAMEWORK

Key concepts:Key concepts:

Comparability for increased lesson Comparability for increased lesson learninglearning

Greater robustness & rigour of final Greater robustness & rigour of final evaluationsevaluations

Key features:Key features:

CEFCEF

Core Management Group (CMG)Core Management Group (CMG)

OPTION 4OPTION 4::

SYSTEMWIDE EVALUATIONSYSTEMWIDE EVALUATION

Key concepts:Key concepts:

Optimised coordination & sectorwide Optimised coordination & sectorwide learning learning

Key features (eg):Key features (eg):

SC & Project Mgmt CommitteeSC & Project Mgmt Committee

Core ‘Study Teams’Core ‘Study Teams’

One overall ToR for evaluationOne overall ToR for evaluation

Regional ‘Expert Panel’ Regional ‘Expert Panel’

INTEGRATED FRAMEWORKINTEGRATED FRAMEWORK

OPTION 1: OPTION 1: Concepts & featuresConcepts & features

OPTION 2: OPTION 2: CEFCEF

OPTION 3: OPTION 3: CMGCMG

OPTION 4: OPTION 4: eg, thematic study teams; regional eg, thematic study teams; regional expert panel expert panel

Meeting AgreementsMeeting Agreements

1.1. To constitute an ‘evaluation coalition’To constitute an ‘evaluation coalition’

2.2. ALNAP Secretariat: coordinating ALNAP Secretariat: coordinating mechanismmechanism

3.3. End of Year Synthesis ReportEnd of Year Synthesis Report

4.4. Longer term StudyLonger term Study

1. To constitute an ‘evaluation coalition’1. To constitute an ‘evaluation coalition’ (Core Management Group & four Guiding Principles)(Core Management Group & four Guiding Principles)

SharingSharingLesson learningLesson learningAccountabilityAccountabilityMutual support & transparencyMutual support & transparency: such that : such that potentially negative findings are seen to present potentially negative findings are seen to present opportunities for improved policy & practiceopportunities for improved policy & practice

2. ALNAP Secretariat: coordinating 2. ALNAP Secretariat: coordinating mechanismmechanism

• Evaluation mapEvaluation map• Online Forum Online Forum • End-of-Year Synthesis Report (RHA 2005)End-of-Year Synthesis Report (RHA 2005)

3. End of Year Synthesis Report3. End of Year Synthesis Report (emergency & initial recovery)(emergency & initial recovery)

1.1. Findings & learning from individual evaluation Findings & learning from individual evaluation reports (coalition) reports (coalition)

2.2. Multi-agency, ‘thematic’ evaluations on cross-Multi-agency, ‘thematic’ evaluations on cross-cutting issues of priority interest cutting issues of priority interest

• CoordinationCoordination• Needs assessmentNeeds assessment• Local / regional capacitiesLocal / regional capacities• Funding / fundraising (incl role of media)Funding / fundraising (incl role of media)• Civil-military axisCivil-military axis• OthersOthers

4. Longer term Study4. Longer term Study

Synthesis Report

Individual agency evaluations (own mgmt and/or sectoral)

Multi-agency thematic evaluations

Individual agency evaluations (thematic)

Common Framework/s

Core Management Group

ALNAP Secretariat / Coalition Coordinator

Online Forum

Evaluation Map

Longer term Study

E – V – A – L – U – A – T – I – O – N C – O – A – L – I – T – I – O – N

Evaluation Coalition:Evaluation Coalition:

Current Structure & ActivitiesCurrent Structure & Activities

How Jointly is ‘Joint’?How Jointly is ‘Joint’?

DAC Evaluation Network Study: 11 typologiesDAC Evaluation Network Study: 11 typologies

Decentralised multi-Decentralised multi-partner:partner:

Joint activities focus on Joint activities focus on establishing common establishing common frameworks. Implementation is frameworks. Implementation is decentralised – ie, devolved to decentralised – ie, devolved to different partners in the different partners in the evaluations (evaluations (IDP Synthesis StudyIDP Synthesis Study))

Stratified multi-partner:Stratified multi-partner: Partners enter on the Partners enter on the understanding of some understanding of some stratification of their respective stratification of their respective roles roles (eg, lead agency + others in (eg, lead agency + others in group)group)

Agencies Involved to DateAgencies Involved to Date

Donor Donor UN Agencies UN Agencies Intl NGOsIntl NGOs

DANIDADANIDA FAO FAO (DEC)(DEC)

USAIDUSAID OCHAOCHA (IWG)(IWG)

SIDASIDA UNDPUNDP

CIDACIDA UNICEFUNICEF

MFA-NetherlandsMFA-Netherlands (UNFPA)(UNFPA)

DC-Ireland DC-Ireland

Total Evaluation Activity to DateTotal Evaluation Activity to Date

Concrete plans:Concrete plans: 3535Total activities:Total activities: 4444Evaluations:Evaluations: 2626 (12 ‘traditional’; 7 ‘joint’; 6 real-(12 ‘traditional’; 7 ‘joint’; 6 real-

time)time)Reviews:Reviews: 1111 (5 ‘learning reviews’; 2 ‘after (5 ‘learning reviews’; 2 ‘after

action’)action’)Other:Other: 88 (various including surveys, research (various including surveys, research

projects & projects & monitoring visits)monitoring visits)

Joint:Joint: Thematic evaluations + IWG Thematic evaluations + IWG (World Vision Intl, Oxfam GB, Care)(World Vision Intl, Oxfam GB, Care)

WHO, SEARO + HAC coordinated by WHO, SEARO + HAC coordinated by OCHAOCHA

Geographical emphasis: Indonesia, India, Thailand, Geographical emphasis: Indonesia, India, Thailand, Sri LankaSri Lanka

Next StepsNext Steps

• CMG: 14th April CMG: 14th April • Writing of CFs & ToR: April – MayWriting of CFs & ToR: April – May• ALNAP Biannual: June ALNAP Biannual: June • Evaluation research: July – early OctEvaluation research: July – early Oct• Synthesis: Dec/JanSynthesis: Dec/Jan• Publication: Jan/Feb (later for RHA)Publication: Jan/Feb (later for RHA)