evaluation of country programmes between finland and nepal, nicaragua and tanzania, 7 february 2012
DESCRIPTION
Evaluation reports: Nepal http://formin.finland.fi/public/default.aspx?contentid=240704 Nicaragua http://formin.finland.fi/public/default.aspx?contentid=239705 Tanzania http://formin.finland.fi/public/default.aspx?contentid=241146TRANSCRIPT
Evaluation of Country Programmes Between Finland and Nepal,
Nicaragua and Tanzania
Oral presentationFebruary 7, 2012
Julian CALDECOTT, Team Leader,Julian CALDECOTT, Team Leader,Michael HAWKES, Nepal Country expertMichael HAWKES, Nepal Country expert
Fred van SLUIJS, Nicaragua Country expertFred van SLUIJS, Nicaragua Country expertArto VALJAS, Tanzania Country expertArto VALJAS, Tanzania Country expert
Bernadeta KILLIAN, Tanzania national expertBernadeta KILLIAN, Tanzania national expert
Introduction to the evaluationIntroduction to the evaluationJulian CALDECOTTJulian CALDECOTT
AimsAims
FocusFocus on 2002-2011: 1998, 2001, 2004, 2007 policy on 2002-2011: 1998, 2001, 2004, 2007 policy frameworks.frameworks.
InfluencesInfluences: policy, actors, events: policy, actors, events MechanismsMechanisms: programming, dialogue, management: programming, dialogue, management Aid effectivenessAid effectiveness, Finnish added value, cross-, Finnish added value, cross-
cutting themescutting themes StrengthsStrengths, weaknesses, lessons learned., weaknesses, lessons learned.
2
MethodsMethods
A team A team consisting of a Team leader, 3 International consisting of a Team leader, 3 International country experts, 3 National experts and a country experts, 3 National experts and a ResearcherResearcher
Interviews Interviews (300 people) in Finland and field(300 people) in Finland and field Literature: Literature: reports, archives, publicationsreports, archives, publications Feedback Feedback from embassies and expertsfrom embassies and experts.. SelectedSelected field visits field visits 14 criteria 14 criteria (Relevance, Efficiency, Strategic
effectiveness, Impact, Sustainability, Coordination, Complementarity, Compatibility, Connectedness, Coherence, FAV, Partner satisfaction, Programming logic, Replicability).
3
ReportsReports
3 Country reports and a Policy brief3 Country reports and a Policy brief Context: Context: history, governance, needs, donorshistory, governance, needs, donors Evolution: Evolution: sectors, projects, circumstancessectors, projects, circumstances Performance: Performance: the 14 criteriathe 14 criteria Highlights: Highlights: influences, policy-into-practice, aid influences, policy-into-practice, aid
effectiveness, FAV, etc.effectiveness, FAV, etc. Lessons learnt and key recommendations.Lessons learnt and key recommendations.
4
Nepal country evaluationNepal country evaluationMike HAWKESMike HAWKES
The evaluation period encompasses the The evaluation period encompasses the introduction of the 2004 and 2007 Finnish introduction of the 2004 and 2007 Finnish development policies and the climax, resolution development policies and the climax, resolution and aftermath of the decade-long armed conflict in and aftermath of the decade-long armed conflict in NepalNepal
5
Volume and sources of aidVolume and sources of aid
2009: overall ODA almost US$ 500 million. 2009: overall ODA almost US$ 500 million. (nearly 3.5 per cent from Finland).(nearly 3.5 per cent from Finland).
Top bilateral donors (94% total): UK, Top bilateral donors (94% total): UK, United States, Germany, Norway, Denmark, United States, Germany, Norway, Denmark, EC, Japan, Switzerland and Finland.EC, Japan, Switzerland and Finland.
These figures exclude aid from India and These figures exclude aid from India and China.China.
7
8
9
Spatial data capture analysis and Spatial data capture analysis and interpretationinterpretation
Topographic mapping. Topographic mapping. Forest Resource Assessment.Forest Resource Assessment.
10
Water Sanitation and HygieneWater Sanitation and Hygiene
Consistent investment.Consistent investment. Focus on genuine needs. Focus on genuine needs. Concerns.Concerns.
11
EducationEducation
Consistent support to SWAp.Consistent support to SWAp.
Modest financial but major Modest financial but major intellectual contribution.intellectual contribution.
Nepal enthusiastic.Nepal enthusiastic.
12
ForestryForestry
Withdrew 1999.Withdrew 1999. Reengaged 2007. Reengaged 2007. Forest Resource Assessment.Forest Resource Assessment. IFAD leasehold forestry project. IFAD leasehold forestry project. Multi-Stakeholder Forestry Multi-Stakeholder Forestry
Programme with UK and Programme with UK and Switzerland.Switzerland.
13
Environment and Climate changeEnvironment and Climate change
Environmental Administration and Environmental Administration and ManagementManagement
Solid Waste ManagementSolid Waste Management Natural disastersNatural disasters Regional Flood Information Regional Flood Information
SystemSystem
14
Peace-building, governance and human Peace-building, governance and human rightsrights
Basic Operating GuidelinesBasic Operating Guidelines The Nepal Peace Trust FundThe Nepal Peace Trust Fund Office of the High Commissioner for Office of the High Commissioner for
Human Rights Human Rights The National Human Rights Commission The National Human Rights Commission UN Security Council Resolutions 1325 UN Security Council Resolutions 1325
of 2000 and 1820 of 2008 on Women, of 2000 and 1820 of 2008 on Women, Peace and Security Peace and Security
15
Country programme scores for Country programme scores for evaluation criteriaevaluation criteria
Criterion Notes and scores (where ‘a’ = very good, ‘b’ = good, ‘c’ = some problems, and ‘d’ = serious deficiencies)
Relevance Overall score (b). Problems in strategic relevance.Efficiency (a).Strategic effectiveness
(a).
Impact (a).Sustainability Overall score (c). Problems in environmental
sustainability.Coordination (a).Complementarity
Overall score (b).
Compatibility (a).Connectedness (c). Problems in internal politics, external
relations, climate change.Coherence (a).Finnish added value
(a).
Partner satisfaction
(a).
Programming logic
(b).
Replicability (b).
16
Key findingsKey findings – past programming – past programming
Very strong positive influence by:Very strong positive influence by: Embassy-government relations.Embassy-government relations. Policy dialogue and country Policy dialogue and country
consultations.consultations. Embassy participation in multi-donor Embassy participation in multi-donor
mechanisms.mechanisms. Good results in peace building and Good results in peace building and
constitution building.constitution building. Mainstreaming of cross-cutting themes Mainstreaming of cross-cutting themes
was effective and strongly influential.was effective and strongly influential.
17
Key findingsKey findings – future programming – future programming
Rapid growth and Rapid growth and fragmentation.fragmentation.
Limited strategic ambition on Limited strategic ambition on climate proofing.climate proofing.
18
Nicaragua country evaluationNicaragua country evaluationFred van SLUIJSFred van SLUIJS
Introduction Context: history, governance, needs, donor activity.Evolution: sectors, interventions, circumstances.Performance: the 14 criteria, research questions, lessons learnedHighlights: political landscape, global issues, good governance
19
Context Regions and peoples Politics and polarization (1979
FSLN, 1990-2006 Conservative Restoration, 2006 FLSN)
Donor landscape: old friends, new friends.
20
21
Evolution International influences (SWAp, SAP,
PRSP, Paris Declaration) Timeline (1989 project policy, 2001
operationalization, 2004 policy streamlining, 2005 decentralization pilot.
Architecture (four pronged approach) Challenges of transition 2008-2011
22
PerformanceApplying the evaluation
criteriaResearch questions
23
Highlights and issues Nicaragua as a special case Re-balancing of powers (ALBA, China) Global issues (e.g. climate change,
security): old problems, new challenges.
The meaning of good governance. Values in development cooperation:
presence in difficult times.
24
Lessons learned
Good practices (e.g. sustainable institutionalisation, donor coordination)
Bad practices (e.g. vulnerable basket funds, weak dialogue)
25
Tanzania country evaluationTanzania country evaluationArto VALJAS and Bernadeta KILLIANArto VALJAS and Bernadeta KILLIAN
Development ContextDevelopment ContextCountry overview Country overview Political contextPolitical contextEconomic growthEconomic growthDevelopment strategiesDevelopment strategiesDual development approachDual development approach
26
Donor community Donor community
Donor darlingDonor darling Front runner in coordination and Front runner in coordination and
harmonisation harmonisation Front runner in aid effectiveness Front runner in aid effectiveness Donor response to the dual Donor response to the dual
development approachdevelopment approach
27
Development Results Development Results
MDGs Income poverty Public consumption in health
and education High growth areas FDI and trade (in particular
from/with China)
28
Evolution of the country Evolution of the country programmeprogramme
2005-09 period of growth in volume2005-09 period of growth in volume 2010 unplanned downturn in 2010 unplanned downturn in
volumevolume Areas of cooperation: Areas of cooperation:
Beginning Beginning MidwayMidway EndEnd
29
Main areas of cooperation Main areas of cooperation
GBSGBS Local government reformLocal government reform Forestry/land useForestry/land use
Including targeted projects Including targeted projects SMOLE and LIMASSMOLE and LIMAS
30
Other areas of cooperationOther areas of cooperation
New areas of cooperation New areas of cooperation NGO support and the LCF NGO support and the LCF Regional programmes Regional programmes Economic cooperationEconomic cooperation
31
Influence of policy until 2007 Influence of policy until 2007
Match between policy, Match between policy, programme and contextprogramme and context
ConsequencesConsequences
32
Influence of policy 2008-2011 Influence of policy 2008-2011
Changes in contextChanges in context Ministerial involvementMinisterial involvement The 2008 Country Assistance The 2008 Country Assistance
PlanPlan Country consultations and the Country consultations and the
2009-2011 Development 2009-2011 Development Cooperation PlanCooperation Plan
33
Influence of FAV and PDInfluence of FAV and PD
Different Different definitions/interpretations of FAVdefinitions/interpretations of FAV
FAV influence strong after 2007FAV influence strong after 2007 Finnish PD performance above Finnish PD performance above
averageaverage Difference between PD rhetoric Difference between PD rhetoric
and practiceand practice
34
Evaluation criteria and Evaluation criteria and questionsquestions
Variations in relevanceVariations in relevance Variations in sustainabilityVariations in sustainability Cross-cutting themesCross-cutting themes Coordination Coordination CompatibilityCompatibility
35
Strengths & WeaknessesStrengths & Weaknesses Strength: Reservoir of trust Strength: Reservoir of trust Strength: Influence amongst Strength: Influence amongst
donors donors Weakness: fragmentation & Weakness: fragmentation &
incoherenceincoherence Best practice: NAFORMABest practice: NAFORMA
36
Worst practices Worst practices
Absence of comprehensive Absence of comprehensive reportingreporting
Quality of dialogue in Mtwara/LindiQuality of dialogue in Mtwara/Lindi Decision-making process on SMOLE Decision-making process on SMOLE Decision-making on SDIDecision-making on SDI DRPS strategic value and DRPS strategic value and
compatibilitycompatibility
37
ConclusionsConclusions Consequences of ambiguous Consequences of ambiguous
policy and application of slippery policy and application of slippery conceptsconcepts
Need for consolidationNeed for consolidation Timeframe for consolidationTimeframe for consolidation
38
RecommendationsRecommendations Agreeing principles for Agreeing principles for
consolidationconsolidation CCTs and logical frameworks CCTs and logical frameworks Guidance from HelsinkiGuidance from Helsinki Reconsideration of modalities Reconsideration of modalities
and sectorsand sectors
39
Overview of findingsOverview of findingsJulian CALDECOTTJulian CALDECOTT
InteractionsInteractions with Governmentwith GovernmentNepal Nicaragua Tanzania
In the field during armed conflict, 1996-2006 (WASH, environment, education).
Development of SWAps, debt relief & GBS, 2002-2006 (2004 policy influence: strong).
Dialogue mainly through multi-donor channels, and around SWAps and GBS.
Dialogue and programming suspended, 2005-2006 (2004 policy influence: weak).
Friction with government from 2007, governance SWAp and GBS closed (2007 policy influence: weak).
National capacity constraints lead donors back to projects, but weaken dialogue with government.
Excellent relations since 2007, focused on government priorities (2007 policy influence: strong).
Rural development and health/gender SWAps continue as effective, pro-poor and constructive.
From 2007 Finnish bilateral activities multiply, contributing to good high-level relations.
40
Interactions with other DonorsInteractions with other Donors
All 3 countries:All 3 countries: Low-cost access to multi-donor Low-cost access to multi-donor
forums.forums. Embassy staff influence forums. Embassy staff influence forums. Challenge from new donors, Challenge from new donors,
uncoordinated government-to-uncoordinated government-to-government support and investment.government support and investment.
41
Interactions with civil societyInteractions with civil society
MFA support for NGOs & embassy MFA support for NGOs & embassy (LCF) support for local CSOs, but:(LCF) support for local CSOs, but:LCF focus on fewer and larger LCF focus on fewer and larger CSOs in Nepal and Tanzania.CSOs in Nepal and Tanzania.Closure of LCF in 2008 in Closure of LCF in 2008 in Nicaragua, funds diverted to other Nicaragua, funds diverted to other modalities that in effect weakened modalities that in effect weakened NGOs.NGOs.
42
Implementing the Paris DeclarationImplementing the Paris DeclarationNepal Nicaragua Tanzania
Donors resist commitment to programmatic approach due to fears over corruption.
GBS and all SWAps were in line with the Declaration.
GBS and all SWAps were in line with the Declaration.
Education sector SWAp is a success.
Problems after 2007 because of conflict over government efforts to own decentralisation, human rights, and civil society interventions.
Leaves education SWAp, forestry SWAp fails, weak consensus around good governance and anti-corruption measures linked to GBS.
Increasing attention to investments with shared responsibility (e.g. WASH and forestry sectors).
Sectoral budget support largely replaced GBS and the share paid through government increased after 2007.
Proliferation of project modalities by donors including Finland.
43
Promoting Cross-cutting ThemesPromoting Cross-cutting ThemesNepal Nicaragua Tanzania
Peace-building, governance and human rights mainstreamed in the country programme.
Gender and health-related CCTs fully integrated within the health/gender SWAp.
CCTs in multilateral and bilateral dialogue, in bilateral programmes, and key objectives of LCF.
GESI embedded in education SWAp and in environmental and WASH interventions.
Since 2007, mainstreamed support for democratic accountability, rule of law, human rights & gender equity.
Results on governance and anti-corruption mixed (lack of interest by government); other CCTs largely neglected.
LCF used to support CSOs that work on caste exclusion and human rights among women and children.
LCF promoted rights of vulnerable groups; last grants supported CSOs on cultural values and special needs of youth and women.
LCF is where the CCTs are best taken into account, and where paying attention to them contributes most to success.
44
Putting policy into practicePutting policy into practiceAll 3 countries :All 3 countries : Joint programming by embassy and MFA.Joint programming by embassy and MFA. Little influence of 2008 CAP.Little influence of 2008 CAP. Diverse unplanned influences on choices.Diverse unplanned influences on choices. Embassy staff adapt & use opportunities.Embassy staff adapt & use opportunities. Weak guidance on CCTs, Paris Weak guidance on CCTs, Paris
Declaration and climate change.Declaration and climate change. FAV: special interests, technical fields FAV: special interests, technical fields
and ways of working.and ways of working.
45
StrengthsStrengthsNepal Nicaragua Tanzania
Finnish role in donor coordination
Finnish role in donor coordination
Finnish role in donor coordination
Small financial contributions to obtain maximum leverage and influence in multiple forums.
The health/gender and rural development SWAps.
Finnish influence in the donor community with embassy personnel as chairs or co-chairs of the Development Partner Group and its working groups.
GESI strategies in the WASH and environment sector projects.
Keeping ways to explore new ideas through innovative and strategic projects.
Partnership with FAO has great potential in DeNRM, forestry and climate change.
46
WeaknessesWeaknessesNepal Nicaragua Tanzania
Ambition on strategic issues
Ambition on strategic issues
Ambition on strategic issues
Opinions differ, but no failure or worst practice was found.
Pause in consultations before and after 2009 session, when continual dialogue was needed.
Disordered programming process after 2007.
Refocusing of the governance programme towards a rights agenda, while creating aid dependency and political isolation among NGOs.
Weak quality of dialogue and decision-making around several interventions.
Willingness for the embassy to be considered hostile to government.
Opaque means by which certain funding decisions were made.
47
Key influencesKey influences Opportunities seen by embassies (all).Opportunities seen by embassies (all). Government needs (Nepal).Government needs (Nepal). Paris Declaration (Tanzania).Paris Declaration (Tanzania). Policies (2007 Nepal, 2004 Nicaragua).Policies (2007 Nepal, 2004 Nicaragua). Government plans (Tanzania).Government plans (Tanzania). Links with particular locations (all).Links with particular locations (all). Changed circumstances (Nepal, Changed circumstances (Nepal,
Nicaragua).Nicaragua). Personal decisions (MFA Tanzania, Personal decisions (MFA Tanzania,
embassy Nicaragua, government embassy Nicaragua, government Tanzania).Tanzania).
48
Lessons from strengthsLessons from strengths
Make small contributions to multi-donor Make small contributions to multi-donor enterprises and earn maximum influence.enterprises and earn maximum influence.
Use well-formulated strategies to Use well-formulated strategies to promote participation, inclusion and promote participation, inclusion and equity.equity.
Participate with government and donors Participate with government and donors in serious, long-term sectoral in serious, long-term sectoral programmes.programmes.
Preserve a complementary role for Preserve a complementary role for innovative and strategic experiments and innovative and strategic experiments and partnerships.partnerships.
49
Lessons from weaknessesLessons from weaknesses Actively build agreement with government Actively build agreement with government
that strategic issues should be addressed.that strategic issues should be addressed. Maintain active dialogue with government Maintain active dialogue with government
especially at difficult times.especially at difficult times. Understand consequences of supporting Understand consequences of supporting
CSOs that compete with government.CSOs that compete with government. Maintain at all times the role of critical Maintain at all times the role of critical
friend, partner and counsellor of friend, partner and counsellor of government.government.
Base programming on rigorous, Base programming on rigorous, transparent analysis and collective transparent analysis and collective decision making.decision making.
50
Thank you!Thank you!
51