multilateral_aid_review_full_linked

8/7/2019 multilateral_aid_review_full_linked

1/232

Multilateral Aid Review

Ensuring maximum value for money forUK aid through multilateral organisations March 2011


2/232

Organisations/Funds

included in the

Multilateral Aid ReviewThe African Development Fund (AfDF)

The Asian Development Fund (AsDF)The Caribbean Development Bank

(CDB)Central Emergency Response Fund

(CERF)The Climate Investment Funds (CIFs)The Development Programmes of the

Commonwealth Secretariat(CommSec)

European Commission Budget(ECion Budget)

European Development Fund (EDF)European Bank for Reconstruction

and Development (EBRD)Food and Agriculture Organisation (FAO)The Education for All - Fast-track

Initiative (FTI)The Global Alliance for Vaccines and

Immunisation (GAVI)Global Environment Facility (GEF)The Global Fund to Fight AIDS, TB

and Malaria (GFATM)Global Facility for Disaster Reduction

and Recovery (GFDRR)European Commission Humanitarian

Aid and Civil Protection (ECHO)Inter-American Development Bank

(IADB)

International Committee of the RedCross (ICRC)

International DevelopmentAssociation (IDA)

International Fund for AgriculturalDevelopment (IFAD)

International Federation of Red Crossand Red Crescent Societies (IFRC)

International Finance Corporation (IFC)

International Labour Organisation (ILO)International Organisation for

Migration (IOM)Ofce of the High Commissioner for

Human Rights (OHCHR)United Nations Peacebuilding Fund (PBF)The Private Infrastructure

Development Group (PIDG)

United Nations Human SettlementsProgramme (UN-HABITAT)

Joint United Nations Programme onHIV/AIDS (UNAIDS)

United Nations DevelopmentProgramme (UNDP)

United Nations Educational, Scienticand Cultural Organisation (UNESCO)

United Nations EnvironmentProgramme (UNEP)

Expanded Delivering as One FundingWindow for the achievement of theMDGs (EFW)

United Nations Population Fund(UNFPA)

United Nations High Commission forRefugees (UNHCR)

United Nations Childrens Fund(UNICEF)

United Nations IndustrialDevelopment Organization (UNIDO)

United Nations Development Fundfor Women (UNIFEM)

The United Nations InternationalStrategy for Disaster Reduction(UNISDR)

UNITAIDUnited Nations Ofce for the

Coordination of Humanitarian Affairs(OCHA)

World Food Programme (WFP)World Health Organisation (WHO)


3/232

Table of ContentsForeword by the Secretary of State iExecutive summary iiiChapter 1 The challenge and case for multilateralism 1 Chapter 2 Analytical framework and methodology 7Chapter 3 Thematic ndings from the review 17Chapter 4 Value for money for UK aid across the

multilateral organisations 72Annex 1 Methodology and assessment framework 103

Appendix 1 Assessment Framework forthe Multilateral Aid Review Assessments 112Appendix 2 Methodology Note: Construction ofFocus on Poor Countries Component 130

Annex 2 Multilateral Aid Review consultation 140Annex 3 Country Visits 143Annex 4 Partner country views 149Annex 5 External reviewers commentary 159

Annex 6 One page summaries of all 43 assessments 162Annex 7 Terms of reference for the review 207

Appendix 1 Organisations/Funds includedin the Multilateral Aid Review 211Appendix 2 Terms of Reference for Country Reviews 213Appendix 3 Extract from the Coalition Agreement 215Appendix 4 Assessment Criteria (update since original ToRs) 217

Annex 8 Terms of reference for the external reviewers 219


4/232

Tabls

Table 1 Selected results from some multilateral organisations 3

Table 2 The Multilateral Aid Review Assessment Framework 10

Table 3 Construction of composite indices 13

Table 4 Descriptors for indices for contribution to UK developmentobjectives and organisational strengths 14

Table 5 Value for money assessments and index scores 15

Table 6 Groupings of multilateral organisations 18

Table 7 Strengths and weaknesses of multilateral groupings 21

Table 8 Grouping the components into composite indices 73

Table 9 Descriptors for indices for contribution to UK developmentobjectives and organisational strengths 73

Table 10 Numbers of multilateral organisations with differentcombinations of scores against the two indices 77

Table 11 Value for money assessment of multilateral organisations 78

Table 12 Value for money for UK aid funding of individualmultilateral organisations 80

Charts

Chart 1 Global core aid funding through multilateral organisations 5

Chart 2 Multilateral core aid funding commitments as a shareof all aid commitments for DAC countries 2009 6

Chart 3 UK core multilateral aid funding 2010/11 6

Chart 4 Strengths and weaknesses across the multilateral organisations 20

Chart 5 Critical roles in meeting international developmentand humanitarian objectives 23

Chart 6 Critical roles in meeting UK development andhumanitarian objectives 24

Chart 7 Focus on poor countries 27

Chart 8 Contribution to results 31

Chart 9 Strategic and performance management 35Chart 10 Financial resource management 41

Chart 11 Cost and value consciousness 44

Chart 12 Partnership behaviour across the multilateral organisations 49

Chart 13 Transparency and accountability 55

Chart 14 Working in fragile contexts 60

Chart 15 Promoting gender equality 64

Chart 16 Attention to climate change and environmental sustainability 68

Chart 17 Contribution to UK development objectives 75

Chart 18 Organisational strengths 76

Chart 19 Value for money of the multilateral organisations for UK aid 79


5/232

Bxs

Box 1 Pillars of the system 25Box 2 Assessing focus on poor countries for multilateral

organisations that support development objectives 28Box 3 Global funds giving aid where it will have the most impact 29Box 4 Unblocking constraints to investment in infrastructure 31Box 5 European Commission innovating for results 33Box 6 UN making a difference on the ground 34Box 7 Management for development results in Asia and the Pacic 36Box 8 Evaluation at the heart of the Global Funds model 37Box 9 Managing for development results to improve

performance in the UN 39Box 10 Multilateral development banks allocating resources

to maximise impact 41Box 11 Strong nancial accountability within the UN 42Box 12 Scrutinising decisions to ensure value 45Box 13 Relocating UN back ofce functions to lower cost locations 47Box 14 Making cost savings in UN country operations 48Box 15 Why partnership behaviour matters 49Box 16 Europes strong partnership through the

European Development Fund (EDF) 50Box 17 Partnership working in the UN 52Box 18 Regional development banks: trusted partners

within their regions 53Box 19 The International Aid Transparency Initiative 54Box 20 IDA strengthening transparency and accountability 57Box 21 Accountability to local people and communities 58Box 22 Humanitarian aid working in the most difcult

places in the world 61Box 23 Long-term commitment to Afghanistan 62Box 24 Responding exibly and creatively to the needs of fragile states 63Box 25 Using gender-disaggregated data to change our thinking 65Box 26 Making a difference to womens lives 66Box 27 Transforming the climate nance landscape 69Box 28 EBRD transforming itself to tackle climate change 70Box 29 UN reducing its carbon footprint 71


6/232

Frr

Secretary of State forInternational Development

Rt Hon Andrew Mitchell MP

The decision taken last year by this Coalition Government

to provide 0.7 per cent of Gross National Income as aidfrom 2013 provides a once in a generation opportunityto transform the lives of millions of poor people aroundthe world.

Yet this commitment made in a time of tough choices also brings a responsibility to extract the greatest possiblevalue for money from our development budget. This is ourduty both to the British taxpayer, on whose behalf wework, and to the people we are helping to pull themselvesout of poverty.

This applies not only to the aid budget that Britain investsdirectly through our work in and with developingcountries, but also to the money we give through themultilateral organisations that work on our behalf.

That is why one of my rst acts as International Development Secretary last yearwas to instigate a review of our contributions to multilateral organisations.

This review has taken a long, hard look at the value for money offered byforty-three of the different multilateral bodies through which Britain has, untilnow, invested our aid. Rigorous and robust, it provides for the rst time a

comprehensive overview of the strengths and weaknesses of each multilateralorganisation. It critically examines both the organisations aims and objectivesand how effective they are in achieving them and delivering results.

Because this review is intended to guide future British aid investment, it alsoassesses the value for money provided by each organisation against the aimsthe Coalition Government has set out for our own development efforts.

Readers familiar with the multilateral system will be unsurprised to nd a verymixed picture emerge from this report.

I was delighted to nd that so many multilateral organisations offer good or very

good value for money for UK aid including UNICEF, the Global Alliance forVaccines and Immunisations and the Global Fund to Fight AIDS, Tuberculosis andMalaria. I look forward to working with and through these organisations to makea real difference to the lives of millions of poor people.

However I was extremely concerned that a small group of organisations have beenassessed as having serious weaknesses. I will be taking a very tough approach tothese organisations. We will not tolerate waste, inefciency or a failure to focuson poverty reduction.

i


7/232

FoRewoRd

The review also gives us an exceptional opportunity to work with others to shapean international reform agenda based on a rigorous comparison of the strengthsand weaknesses of the multilateral organisations.

It identies a number of areas where improvements in performance could lead toa step change in the ability of the multilateral system to deliver for poor people.Reforms will be specic to each organisation, but across the board we will bepressing for more accountability, a greater focus on results, better management ofresources and a focus on the critical areas for development such as supportingwomen and girls, and doing more in countries affected by conict.

This review provides all of us with a determined interest in furthering the causeof development with an invaluable resource. The multilateral system has uniquereach, relevance and respect in helping to tackle global poverty. Now, armed withthe evidence provided by this review, we can help to steer the reforms needed to

ensure that we build on the strengths of the multilateral system, and address itsweaknesses. Together, we can make a world of difference.

ii


8/232

UK Multilatral Ai RviExecutive Summary1. The Multilateral Aid Review was commissioned to assess the value for money for

UK aid of funding through multilateral organisations. Forty-three organisationswere assessed. Nine were deemed to offer very good value for money, sixteen tooffer good value for money, nine to offer adequate value for money, and nine tooffer poor value for money for UK aid.

2. The organisations covered by the review ranged from multilateral developmentbanks to UN specialised agencies, and from development nance institutions tohumanitarian organisations. All were assessed against the same set of criteria,

interpreted exibly to t with their different circumstances, but always grounded inthe best available evidence. Together the criteria capture the value for money forUK aid of the whole of each organisation. The methodology was independentlyvalidated and quality assured by two of the UKs leading development experts1.

3. The assessment framework included criteria which relate directly to the focus andimpact of an organisation on the UKs development and humanitarian objectives such as whether or not they are playing a critical role in line with their mandate,what this means in terms of results achieved on the ground, their focus on girlsand women, their ability to work in fragile states, their attention to climatechange and environmental sustainability, and their focus on poor countries. Thesecriteria were grouped together into an index called Contribution to UKdevelopment objectives.

4. The framework also included criteria which relate to the organisations behavioursand values that will drive the very best performance such as transparency,whether or not cost and value consciousness and ambition for results are drivingforces in the organisation, whether there are sound management and accountabilitysystems, whether the organisations work well in partnership with others andwhether or not nancial resource management systems and instruments helpto maximise impact. These were grouped together into an index calledOrganisational strengths.

5. Value for money for UK aid was assessed on the basis of performance againstboth indices. So, for example, organisations with a strong overall performanceagainst both indices were judged to offer very good value for money for UK aid,while those with a weak or unsatisfactory performance against both indices weredeemed to offer poor value for money. Chart 1 shows what this means for theorganisations covered by the review.

6. As important as this assessment of value for money was, the review was alsosignicant as the rst attempt by the UK to systematically examine strengths and

Dr Alison Evans, Director of the Overseas Development Institute, and Professor Lawrence Haddad, Director of

the Institute of Development Studies and President of the UK and Irelands Development Studies Association

1

iii


9/232

eeCUTe SUMMARy

weaknesses across the multilateral system. This was noteworthy for two reasons.First, it provides the evidence basis for the UK to work with the multilateralorganisations and other partners to develop a focused reform agenda which, if

implemented, will make a real difference to the lives of poor people. Second, byidentifying areas of weakness, as well as strength, across the whole multilateralsystem, it gives added impetus to reform efforts, as well as a sound basis formonitoring progress.

alu fr mn f th multilatral rganisatins fr UK ai

OrganisationalStrengths

Contribution to UK Development Objectives1

4

3.5

3

2.5

2

1.51.5 2 2.5 3 3.5 4

ISDRCommSec

UNESCO UNHABITATILO FAO

IOMUNIFEMUNIDO

UNEPUNAIDSOHCR EFW

UNFPAEC Budget

GEF

EBRD

IFCCIFsGFDRR

UNDPPBFFTI WFP UNHCR

UNICEFECHO

ICRCEDFGFATMAsDB

GAVIPIDG

IDAIFADAfDB

UNITAIDIFR OCHA

CERF

IADBCDB

WHO

PoorAdequateVery GoodGood

Note:1

includes humanitarian objectives

7. The review conrmed that the multilateral system is a critical complement to whatthe UK government can do alone. Together the multilateral organisations mobiliselarge-scale funding, bring specialist expertise, support innovation, play pivotalleadership roles with other donors, have the mandates and legitimacy to help todeal with conict situations, and provide a platform for action in every country inthe world. Multilateral organisations particularly those assessed as offering verygood value for money for UK aid such as UNICEF, the GAVI Alliance and the

iv


10/232

eeCUTe SUMMARy

Global Fund to Fight AIDS, Tuberculosis and Malaria are saving poor peopleslives and increasing their life chances.

8. But the review also found that the system is complex and fragmented, withoverlapping mandates and co-ordination problems. There was not enoughevidence of multilaterals consistently delivering results on the ground, particularlyin fragile states. Too many organisations lack a clear strategic direction, andsystems to get the right staff in post at the right time, and ensure thatmanagement and staff are focused on achieving results and held to account forthis. Most multilaterals are not paying sufcient attention to driving down costsor achieving value for money. Most multilaterals are not concentrating enough ongender issues. There is still much room for improvement for the multilaterals as agroup on transparency and accountability. And poor partnership workingbetween multilateral organisations is undermining the effectiveness of the system.

9. The report includes summaries of the results of the assessments as well as theorganisation by organisation results. A separate policy paper sets out DFIDsresponse to the ndings of the review.

v


11/232

Chaptr 1

The challenge and the case for multilateralism

Overview1. When the UK Coalition Government came to power in 2010, it pledged to

increase aid volumes to reach the international commitment to spend 0.7% ofnational income on aid from 2013. The UK is the rst G20 country to set out adetailed plan to make this happen, and to propose legislation to make thecommitment binding.

2. Just as the UK Coalition Government is resolved to stand by its responsibility tohelp the worlds poorest people, it is determined to ensure that UK aid is directedto where it can make the greatest difference. The government thereforeestablished three major reviews of UK aid: the Bilateral Aid Review, theMultilateral Aid Review, and the Humanitarian Emergency Response Review. TheBilateral and Multilateral Aid Review ndings are published on 1st March 2011;the Humanitarian Emergency Response Review is due to report at the end ofMarch 2011.

3. This report presents the ndings of the Multilateral Aid Review (MAR). The reviewfocuses on the UKs core aid funding through the multilateral organisations,

including the World Bank, the United Nations (UN) and the EuropeanCommission2. It aims to assess the value for money that different multilateralorganisations offer for UK aid. Its purpose is to ensure that decisions about UKfunding through these organisations are based on evidence about the value formoney they offer and the results they achieve. Terms of reference for the revieware attached at Annex 1.

4. The remainder of this chapter provides more detail on the context for theMultilateral Aid Review. The next chapter sets out our analytical framework andmethodology. Chapter 3 summarises the thematic ndings from the review. Thenal chapter reports on value for money across the multilateral organisations and

summarises the ndings for the individual multilateral organisations.

The Multilateral Organisations5. The multilateral organisations were established by international law to enable

national governments to work together on particular issues. The MAR focuses onthose multilateral organisations that have an important development and/or

The review refers to the EU where it talks about shared development policies with the EU Member Statesor broader external policies, and the European Commission where the focus is on the Commissions work asa donor and implementer. It distinguishes between the European Commission budget instruments (ECion

Budget or EC Budget in graphs) and the European Development Fund, or EDF

1

2


12/232

CATeR 1.

humanitarian role. We refer to them here as the multilateral organisations,or the multilateral system. In addition to the organisations referred to above,they include the regional development banks, development nance institutions,

the global funds, and the humanitarian organisations, many of which are alsopart of the UN or European Commission.

6. The multilateral organisations are an essential part of the international system fordevelopment and humanitarian aid. They have a global presence and thelegitimacy to work even in politically sensitive contexts where nationalgovernments are not welcome. They provide specialist technical expertise, anddeliver aid on a large scale. They offer a wide range of aid instruments to meetthe needs of all countries. They have the legitimacy to lead and co-ordinatedevelopment and humanitarian assistance. They broker international agreementsand monitor adherence to them. They develop and share knowledge about what

works, and why. Some are at the cutting-edge of innovation, leading thedevelopment of new initiatives to deliver development and humanitarianassistance more effectively.

7. Box 1 shows a few of the development and humanitarian results that themultilateral system has contributed to over the past few years. It highlights boththe breadth of the areas in which the multilateral organisations work, and thescale of their engagement.

8. The multilateral organisations are a critically important complement to theprogrammes of bilateral (national government) donors such as the UK Departmentfor International Development (DFID). First, they enable bilateral donors to support

development and humanitarian objectives in a much wider range of countries,including some where bilateral donors are not welcome. Second, the scale of theiroperations enables them to deliver specialist technical advice, other knowledgeservices such as research, and nancial products such as grants and loans, at alower cost. Third, their leadership and co-ordination function reduces transactionscosts for both donors and developing countries. Fourth, their role in brokeringinternational agreements and monitoring adherence to them raises standardsacross the whole of the international system.

9. This is not to say that all multilateral organisations are equally critical to theachievement of international development and humanitarian objectives, or that

they are performing equally well. The multilateral system is complex andfragmented, with overlapping mandates and co-ordination problems. While someorganisations are praised for delivering exciting programmes, for acting effectivelyunder difcult and sometimes dangerous circumstances, and for respondingquickly to needs on the ground, others are criticised for being slow to act, forfailing to deliver for the poor, and for being wasteful of their resources. Laterchapters of this report discuss our ndings on the performance and value formoney for UK aid of the multilateral organisations.

2


13/232

Te CAee Ad TeCASe FoR MUTATeRASM

Tabl 1 Slct rsults frm sm multilatral rganisatins

Multilatral Rsults

European assisted 24 million people through food security related socialCommissioni transfers (between 2002 and 2009)

enrolled 9 million new pupils in primary education (since 2004) helped protect 1.5 million km2 of forests and conserve 1.1 million

km2 of protected areas connected more than 31 million households to improved

drinking water and 9.3 million to sanitation facilities (since 2004) conducted 58 election observation missions (in the last ve years)

United Nations with its partners, procured and distributed 42 million insecticide-Childrens Fund treated nets for malaria prevention in 2009(UNICEF)ii distributed over 574 million vitamin A capsules in 74 countries

in 2009 immunised more than 140 million children against measles in 2009

World Health following the launch of the WHO DOTS strategy for TB controlOrganisation in 1995, 41 million people were successfully treated and over 6

million lives saved, including 2 million among women and childreniii

provided technical support to 74 countries to provide skilled carefor pregnant women and newbornsiv

United NationsDevelopmentProgramme

helps between 30 and 40 countries each year with electoral support created an estimated 95,700 jobs under a cash for work schemev

in Haiti

International Over the last ten yearsDevelopmentAssociationvi ofthe World Bank

immunised 310 million children provided 113 million people with an improved water source provided basic health, nutrition and population services to

47 million people improved the quality of teaching and facilities for

105 million children given 26 million people access to an all-season road

Asian Between 2005 and 2008DevelopmentFundvii improved nearly 25,000km of roads, beneting over

133 million people

newly connected 834,000 households to electricity helped over 2 million people with micronance

African Between 2006 and 2008DevelopmentFundviii provided over 16 million people with a new electricity

connection provided over 41 million people with improved access

to transport newly enrolled over 11 million students granted 433,000 micro-credits

3


14/232

Tabl 1 Slct rsults frm sm multilatral rganisatins continuedMultilatral Rsults

CATeR 1.

Global Fund toght AIDS, TBand Malariaix

Global Alliancefor VaccinesandImmunisationx

World FoodProgrammexi

InternationalFederation ofRed Cross andRed CrescentSocieties

InternationalCommittee ofthe Red Crossxiv

Since 2004

distributed 160 million insecticide treated nets treated 7.7 million new TB cases treated 3 million HIV positive people with anti-retroviral drugs

Since 2000

prevented 3.4 million deaths vaccinated 213 million children

provided food for 101.8 million people in 75 countries in 2009 delivered 4.6 million metric tons of food to the hungry in 2009 feeds more than 90 million people in more than 70 countries

every year

reached some 44.8 million people worldwide through IFRCsupported emergency response operationsxii

provided assistance to 1.5 million people in Haiti half of the 2.9million affected by the earthquakexiii

In 2009

1,063 people were reunited with their families 5.8 million people beneted from ICRC-supported health

care facilitiesmore than 4,068,000 people received aid in the form of

food and 4,488,000 in the form of essential household andhygiene items

i Source: EU contribution to Millennium Development Goals; Some key results from European Commissionprogrammes, 2010

ii Source: 2009 EDAR & 2009 Report on Regular Resourcesiii Global Tuberculosis Control Report 2010, WHOiv WHO Performance Assessment Report 2008/09.v UNDP In Action 2009/10vi IDA 16 Deputies Report, Delivering Development Results, December 2010vii Asian Development Bank 2008 Development Effectiveness Review. The Asian Development Fund is the

concessional arm of the Asian Development Bank.viii African Development Bank Mid Term Review, Progress in implementing the ADF-11 Results

Measurement Framework, October 2009. The African Development Fund is the concessional arm of theAfrican Development Bank.

ix GFATM websitex GAVI websitexi World Food Programme websitexii Source: IFRC-DFID Institutional Strategy III (2007-10) Annual Summary Review 2009xiii IFRC Progress Report, Haiti: One Year Onxiv ICRC Annual Report, 2009

4


15/232

EC IDA EDFWFPGF

ATM

OtherU

NAfDB

UNHC

REC

HOUN

DPAsDB

ICRC IOM

UNICEF

GEF

UNFPA

UNHC

RCE

RFIFRCGAVIIBR

DIFA

DWHO

UNITA

ID

FTI*

OTHE

R

Te CAee Ad TeCASe FoR MUTATeRASM

10. Governments across the world contribute core aid funding to the multilateralorganisations through membership fees, capital contributions and additionalun-earmarked voluntary contributions. Chart 1 shows how global core aid

funding was distributed across the multilateral system in 2009. 35% wentthrough the European Commission, 28% through a variety of UN organisations,22% through the multilateral development banks, and 10% through the globalfunds. Some multilateral organisations provide non-concessional resources as wellas aid funding. This is an important source of nance for some developingcountries, but was not a major focus for the Multilateral Aid Review. A list of themultilateral organisations covered by the MAR, together with their acronyms, canbe found at the beginning of this report.

Chart 1 lbal cr ai funing thrugh multilatral rganisatins

(m)

7,0006,0005,0004,0003,0002,0001,000

0

* Reported amounts for FTI are based on pledges only

UK Support to the Multilateral Organisations11. The UK has traditionally been a strong supporter of the multilateral system,

although not unusually so. Chart 2 shows how our multilateral core aid funding

compared with that of other rich countries in 2009.

5


16/232

Italy

Austria

Greece

Portu

gal

Germ

any

France

Belgium

Finlan

d

Japan

Luxembourg

Sweden

Denm

ark

UnitedKingdom

Irelan

d

Spain

Korea

NewZealan

d

Netherlan

ds

Switzerlan

d

Norway

Canada

Australia

UnitedStates

Chart 2 Multilatral cr ai funing cmmitmnts as a shar f all ai cmmitmntsfr dAC cuntris 2009

CATeR 1.

Multilateral percentage of ODA

Commitmentsin2009US$(millions)

MultilateralAidas%ofTotalODA

40,000

35,000

30,000

25,000

20,000

15,000

10,000

5,000

0

80%

70%

60%

50%

40%

30%

20%

10%

0%

United Kingdom29%

Other ODA ($) Multilateral ODA ($) Multilateral percentageof ODA

Note: Figures for commitments in 2009 were used from the DAC database (using DACgures as of 17/02/2011).

12. The Department for International Development, DFID, will disburse about3.7 billion of UK aid as core funding through the multilateral organisationsin 2010/11. Because of our EU treaty obligations, about a fth of this willgo through the European Commission budget instruments. Of the remainder,about 30% will go through the World Bank, 25% through the global funds,15% through the European Development Fund, and 9% through the UN.Chart 3 shows the distribution of UK multilateral core aid funding this year.

Chart 3 UK cr multilatral ai funing 2010/11

UN, 266m

EuropeanCommission,

1,189m

Humanitarian, 208m

Other, 130m

Global funds, 741m

Internationalfinancialinstitutions,1,129m

6


17/232

Chaptr 2

Analytical framework and methodology

Overview1. As explained in the previous chapter, the UK Coalition Government wanted to

assess the value for money offered by all the multilateral organisations thatreceive signicant core aid funding. Our rst task was to determine whichcontributions to include.

2. We needed to establish a comprehensive set of assessments to guide ourdecision-making into the future. We therefore decided to cover, rst, all themultilateral organisations which regularly receive 1 million or more of core aidfunding from the Department for International Development (DFID). There werethirty-ve of these. Second, we included all multi-donor trust funds which receivefully exible funding from DFID headquarters. There were four of these. Third, weincluded three international nancial institutions3 which operate from their owncapital, but where we are engaged in nancial decisions on issues such as the useof net income or the provision of new capital. Finally, and looking towards thefuture, we added in one humanitarian organisation which does not receiveregular multilateral core aid funding, but which might be a candidate for such

funding in the future. In order to ensure that our assessments were comparable,we excluded central funding allocated for specic projects, as the best means ofachieving very narrow and specic objectives. In total, we covered sixteen UNbodies, ten humanitarian organisations (some of which are also part of the UNfamily, and one of which is the humanitarian arm of the European Commission),six global funds, ve multilateral development banks (generally focusing on theconcessional funds)4, three development nance institutions or facilitiessupporting the private sector, two European funding vehicles, and theCommonwealth Secretariat.

3. Our second task was to dene what we mean by value for money. We wanted tocapture the value for money for UK aid of the whole of each organisation. Wetherefore took a broad view, with a denition that reaches from control of coststhrough to delivery of outcomes, and includes system-wide impacts as well asmore focused contributions. Figure 1 gives an illustration of this. Inputs here are

3 The international nancial institutions include both the multilateral development banks and developmentnance institutions such as International Finance Corporation (IFC).

4 In the case of the World Bank and the African and Asian Banks we concentrated entirely on the concessionalfunds. In volume terms, these are where the key decisions for UK aid funding are taken. We also provide,or expect to seek parliamentary approval for providing, new capital to these and other non-concessionalbank operations, partly as a result of the impact of the nancial crisis on their balance sheets. Our assessmentof the value for money of these decisions draws on the ndings of the MAR. Many of the organisationalissues are exactly the same as for the concessional funds, although some of the policies are specic to

non-concessional lending.

7


18/232

CATeR 2.

goods and services, such as textbooks and training for teachers. Outputs areconcrete deliverables, such as schools with adequate staff and teaching materials.And outcomes are the difference that all this makes to peoples lives, such as

better educated children with an increased chance of escaping poverty.

Figur 1 alu fr Mn

Value for Money

Costs () Inputs OutputsQualitativeQuantitative

Outcomes

Efficiency EffectivenessEconomy

4. The further we go up the chain, from costs, to inputs, to outputs and then tooutcomes, the more difcult it becomes to measure the things that we areinterested in. But it is still important to try to do this, because this is what matters:not the number of teachers trained, but the difference this makes to poorchildrens life chances. Our approach has therefore been to measure what we can,and look at proxy measures for the rest. We developed a set of criteria whichcover the whole of the value for money chain, from cost and value consciousness

at one end, to focus on poor countries and ambition to tackle challengingobjectives at the other. Then we drew on the best available evidence to formjudgements of the performance of each individual multilateral organisationagainst these criteria. We explain how we did this later in this chapter.

5. In order to ensure the rigour of the framework and assessment process, weinvited two of the UKs leading development experts to be external reviewers.Dr Alison Evans, Director of the Overseas Development Institute, and ProfessorLawrence Haddad, Director of the Institute of Development Studies and Presidentof the UK and Irelands Development Studies Association, agreed to take on thisrole. We are extremely grateful to them both for their advice and robust challengethroughout the process of the review. Annex 8 gives their Terms of Reference.

6. The rest of this chapter: sets out the assessment framework and explains how itwas developed; describes our sources of evidence and methodology for makingthe assessments; and explains our use of indices to summarise our view of thevalue for money offered by the multilateral organisations.

The Assessment Framework7. The assessment framework is based on both development and organisational

theory. Development theory says that organisations that are working in poorcountries, trying to do things that matter, and following the principles set out in

8


19/232

AAyTCA FRAMewoRKAd MeTodooy

the Paris Declaration on Aid Effectiveness5, are more likely to be making a realdifference for poor people and communities. Organisational theory says thatorganisations that have certain characteristics, such as strong strategic

management and a culture of seeking to minimise costs, are more likely to deliverresults and offer value for money. Our draft assessment criteria drew on both setsof theories, so it included both, for example, promotion of gender equality andnancial resource management.6

8. Having developed draft criteria for the assessment, we consulted widely on them,with multilateral organisations, civil society, and our external reviewers. Annex 1discusses how the framework developed through the consultation process. Table2 below sets out the nal framework, showing how we grouped the criteriatogether into ten components.

Tabl 2 Th Multilatral Ai Rvi Assssmnt Framrk

CoMoeT CRTeRA

1. Critical role inmeetingdevelopmentobjectives

Important role in delivering key international developmentgoals or humanitarian objectives,with country level evidence of this

Important role in delivering UK development or humanitarianpriorities, with country level evidence of this

2. Attention tocross-cutting issues

Performs well in fragile contexts Promotes gender equality Ensures its activities are low carbon, climate resilient and

environmentally sustainable

3. Focus on poorcountries

Allocates resources to countries that need it mostor prioritises areas of greatest humanitarian need

Allocates resources to countries where it will be best used

4. Contributionto results

Objectives are challenging e.g. strives to reach thevery poorest

Strives for results at country level Demonstrates delivery against objectives Contributes to development or humanitarian results

5 The Paris Declaration, endorsed in 2005, is an international agreement which commits signatories to continue toincrease efforts in harmonisation, alignment and managing aid for results, with a set of monitorable actionsand indicators.

6 Our conceptual framework builds on the work of the Multilateral Organisation Performance AssessmentNetwork, MOPAN, which uses a Balanced Scorecard approach to assessing organisational effectiveness. We also

used MOPAN data wherever available to inform our judgements.

9


20/232

CATeR 2.

Tabl 2 Th Multilatral Ai Rvi Assssmnt Framrk continuedCoMoeT CRTeRA

5. Strategic/performancemanagement

Has a clear mandate, and strategy and implementation plansto deliver it

Governing body is effective at holding management toaccount

Leadership is effectiveMeasures results Has an effective evaluation function Governing body and management use results and evaluation

evidence to improve decision making Has good HR policies and practices

6. Financial

resourcemanagement

Allocates aid transparently

Funding is predictable Pro-actively manages poorly performing projects and

programmes Ensures nancial accountability Instruments are appropriate

7. Cost and valueconsciousness

Challenges and supports partners to think about value formoney

Rates of return and cost effectiveness issues are importantfactors in decision-making

Achieves economy in purchase of programme inputs Controls administrative costs

8. Partnershipbehaviour

Works effectively in partnership with others Implements social safeguard policies including incorporating

beneciary voice Has exibility which enables a country-led approach Follows Paris/Accra principles in its approach to aid delivery Provides an effective leadership and co-ordination role in

humanitarian settings

9. Transparencyand accountability

Has a comprehensive and open disclosure policy Promotes transparency and accountability in partners/

recipients Routinely publishes project documentation and project data Signatory of IATI and shows commitment to implementation Governing structures include effective partner country

representation Partner country stakeholders have right of redress and

complaint

10. Likelihood ofpositive change

Governing body and management continuously strive forimprovement

Evidence of progress against reform objectives in the pastOpportunities to promote reform are anticipated

10


21/232

11


Sources of evidence9. Most academic assessments of donor effectiveness are based on a set of

quantitative indicators, such as disbursement delays, or adherence to Pariscommitments. This approach has the advantage of being very clear andtransparent. But it also has disadvantages. First, there are no data sets whichcover all of the multilateral organisations. This limits their usefulness for exercisessuch as this, which need to be comprehensive. Second, some of the criteria thatwe were concerned about are not very amenable to this approach. For example, itwould be extremely difcult to develop a set of measurable quantitative indicatorswhich effectively capture the answer to the question, Does the multilateralorganisation challenge and support partners to think about value for money?.

10. We therefore took a different approach. We used quantitative data whereverpossible, from other assessments such as the Publish What You Fund AidTransparency Assessment, and from surveys such as the Paris survey on aideffectiveness, the Multilateral Organisation Performance Assessment Network(MOPAN), and the Heavily-Indebted Poor Countries Capacity Building Project(HIPC CBP) Partner Country Evaluations of Multilateral Institutions. But we alsodrew on other, more qualitative, sources of evidence.

11. First, we very strongly wanted the review to be grounded in the experience ofdeveloping countries. So we visited ten countries to gather evidence onmultilateral performance. We covered some multilaterals in all countries, andmost multilaterals in some countries. But the coverage was not complete: somemultilaterals were not covered at all, and some were only covered in one or twocountries. It is therefore important to stress that these visits were only one sourceof evidence about multilateral organisation performance in-country. Other sourcesinclude the surveys referred to above, external evaluations, and reporting fromthe multilateral organisations themselves. Annex 3 gives more detail on thecountry visits.

12. Second, we were keen to hear the views of our developing country partners.These are available through MOPAN for some multilaterals, but there are nocomprehensive sources of data on partner country views of multilateraleffectiveness. We therefore undertook two consultation exercises. The rst was ashort facilitated workshop with 24 senior representatives from African partnercountry governments and civil society. The second was a series of interviews ofofcials from 14 developing country governments by staff from UK Embassies andDFID ofces. In both cases, we asked respondents how we should prioritise aidfunding, as well as inviting more general comments about the performance ofthe multilateral organisations. As with the country visits, these exercises wereimpressionistic rather than systematic. But they were not carried out in isolation:they also draw on earlier stakeholder studies commissioned by DFID. Annex 4provides more detail on the partner country consultations.

13. Third, we also drew on evidence from a wide variety of stakeholders. Themultilateral organisations themselves were extremely helpful in giving us written


22/232

CATeR 2.

submissions with sign-posting to relevant documents. UK civil society also kindlyprovided us with written submissions and participated in two round tablediscussions (civil society in developing countries was consulted through the

consultation exercises and the country visits). And staff from across the UKgovernment, particularly, but not only, DFID, were also closely involved.

Using the framework14. A common approach to assessment frameworks such as this is to dene a set of

criteria for each component, count how many of these criteria are met by eachorganisation, and to what extent, and then use a formula to arrive at a nalscore. We considered this approach, but rejected it. The main reason for this isthat we needed a framework which would apply equally to a large multilateraldevelopment bank and a small normative agency, a humanitarian organisation

and a global fund with no in-country delivery capacity. In other words, we neededa exible tool, not a straightjacket.

15. We therefore took a different approach. We developed guidance whichelaborates on the criteria set out in table 2, posing detailed questions about, forexample, how aid is allocated, or what project documentation is made publiclyavailable. Any multilateral organisation that meets all of the criteria against aparticular component will be displaying best practice in that area. However,because we set extremely high standards, we allowed organisations to receive astrong score if they fully met the criteria that are most important and relevant fortheir work, but only partially met some of the other criteria.

16. We made evidence-based assessments of performance against each of the criteriain the framework. Crucially, we also used our best judgement to draw theseassessments together into a single evidence-based assessment of performanceagainst the component as a whole. In doing this, we always came back to the keyunderlying issue. For example, the component for strategic and performancemanagement asks for assessments against six criteria. But underlying all of theseis the question, Is the organisation led and managed in a way that will bestenable it to meet its development or humanitarian objectives. All componentswere scored on a scale of 1 to 4, where 1 was unsatisfactory, 2 was weak, 3 wassatisfactory, and 4 was strong.

17. The exibility we created in the assessment framework and process was adeliberate choice, intended both to ensure that the framework was applicable toall of the very different organisations we were assessing, and to ensure that wecould make use of all available evidence. But there is a ne line between exibilityand imprecision. In order to stay on the right side of this line, we instituted arigorous and lengthy process of quality assurance.

18. Quality assurance took place at a variety of different levels within DFID, drawingon the knowledge and experience of our policy and country-based staff as wellas that of staff working on the multilateral organisations. Ofcials from other UKgovernment departments with experience of the multilateral system were also

12


23/232


closely involved. Our external reviewers provided advice and challenge throughout. Annex 1 describes the quality assurance process.

Composite indices19. Having arrived at assessments of performance against each component, we then

clustered them together into two composite indices. First, those that relate to theorganisational values and behaviours that support lasting progress towardsdevelopment and humanitarian objectives. Second, those that are more directlytied to the expected impact on those objectives. Table 3 shows how these indiceswere constructed.

Tabl 3 Cnstructin f cmpsit inics

de CoMoeTS

Contribution toUK developmentobjectives *

Average of: critical role in meeting international development and

humanitarian objectives critical role in meeting UK development and humanitarian

objectives Average of:

adaptation to fragile contexts promotion of gender equality; and ensures activities are low carbon, climate resilient and

environmentally sustainable

Focus on poor countries Contribution to results

Organisationalstrengths

Cost and value consciousness Partnership behaviour Strategic/performance management Financial resources management Transparency and accountability

* includes humanitarian objectives

20. We debated how to treat the component on contribution to results. It includessome criteria which assess actual contributions globally and in-country. Theset well with the language of contribution to UK development objectives.But because organisations vary in their ambition, and more particularly in theirreporting of results, it also includes some proxy measures for delivery, such aswhether or not there is evidence of management striving for results, which arebetter described as organisational strengths. In the end we concluded that theunderlying question for this component is about whether or not organisations arein practice making a difference to development and humanitarian objectives. Wetherefore placed it in the index on contribution to UK development objectives.But it is important to be aware that this means that this index does include someelements that are about organisational behaviours. By and large, though, we can

13


24/232

CATeR 2.

characterise the indices as follows: the index on contribution to UK developmentobjectives primarily assesses what organisations do, and where, while the indexon organisational strengths primarily assesses how well they are organised and

managed to do it.21. In the absence of any strong theoretical basis for a different weighting scheme,

we followed the practice that is usually adopted in the literature of not givingadditional weight to any one component over another. That is, we treated all thecomponents as being of equal importance. There were two exceptions to thisrule. First, we averaged out the assessments of criticality to international andUK development and humanitarian objectives to give one overall assessment ofcriticality. Second, and to prevent the assessments of performance against thecross-cutting issues of attention to fragile contexts, gender and environmentand climate change considerations from dominating one of the indices, we

averaged these out to form one overall assessment of performance againstcross-cutting objectives.

22. For the purposes of summarising our ndings, we grouped scores against thecomposite indices as follows:

Tabl 4 dscriptrs fr inics fr cntributin t UK vlpmnt bjctivs anrganisatinal strngths

nx scr gn dscriptrs Unrling cmpnnt scrs

3 and above Strong At least one strong score.

Satisfactory on almost all others.

2.5 2.99 Satisfactory Mainly satisfactory with some weak scores.Any strong scores are more than balanced outby weak or unsatisfactory scores.

2 2.49 Weak Mainly weak with some satisfactory scores.Any unsatisfactory scores are balanced out bysatisfactory scores.

Less than 2 Unsatisfactory At least one unsatisfactory score.Weak on almost all others.

23. We considered the possibility of constructing a single index of value for moneyfor the multilateral organisations, but decided against it. Neither development nororganisational theory are well enough advanced for us to be condent that wewould be able to construct a truly meaningful single index of overall effectiveness.Again, this approach of using multiple rather than single indices seems to beemerging as best practice in the literature on aid effectiveness7.

7 Birdsall, N; H. Kharas and A. Mahgoub (2010) A Quality of Development Assistance Index Centre for GlobalDevelopment and the Wolfensohn Centre for Development at the Brookings Institution; Publish What YouFund (2010) Aid Transparency Assessment Global Campaign for Aid Transparency; HIPC CPB Partner Country

Evaluations of Multilateral Institutions (September 2010), Development Finance International.

14


25/232

15

Value for Money24. The purpose of the Multilateral Aid Review is to assess the value for money of

delivering UK development assistance to poor people through the multilateralorganisations. So, although we were reluctant to construct a single index of valuefor money, we did need to nd an approach to assessing overall value for moneyin which we could be condent.

25. As discussed earlier, the criteria we set in the assessment framework and guidancewere based on development and organisational theories about the behavioursand characteristics that we would expect to deliver results and to offer value formoney. The standards we set were very demanding, both in theory and, as itturned out, in practice. The next chapter goes through them in detail, as well asproviding a thematic overview of the results of the assessments. There were very

few strong scores, even though we know from our country visits and our broaderexperience of working with the multilateral organisations that many of themare in fact performing well. We therefore took the view that any multilateralorganisations receiving mainly strong or satisfactory scores against the components,and therefore strong or satisfactory scores against the combined indices, offergood value for money for UK development assistance.

26. However, we wanted to go further than this, and to identify those multilateralorganisations that offer very good value for money for UK developmentassistance. Given the high standards we set, we believe that any organisationswith strong scores on both indices offer very good value for money. We alsoargue that there can be some trade-offs between the indices so that, for example,an organisation which is particularly strong at contributing to UK developmentobjectives may offer very good value for money even if it is only satisfactory onthe other index, so long as there are more strengths than weaknesses overall.

27. Multilateral organisations which do not offer good or very good value for moneymay still offer adequate value for money, in the same way that an investmentproject may offer a rate of return that is just acceptable given the cost of capital.However, we judge that any multilateral organisation with a weak or unsatisfactoryscore against both indices offers poor value for money for UK aid funding.The table below summarises how we translated index scores into value formoney assessments.

Tabl 5 alu fr mn assssmnts an inx scrs

alu fr mnassssmnt

Unrling scrs

3 and above Average score across both indices is strong

2.5 2.99 Average score across both indices is satisfactory

2 2.49 One satisfactory and one weak score against the indices

Less than 2 Scores on both indices are weak or unsatisfactory



26/232

CATeR 2.

28. Organisations close to the dividing line between good and very good value formoney, good and adequate value for money, or adequate and poor value formoney, will in practice have similar levels of performance. This is true of many

similar decision rules.29. The results of the assessments against both the components and the composite

indices are shown in the next two chapters. Chapter 3 takes a thematic approach,looking across the components. Chapter 4 summarises the results for theindividual multilateral organisations, looking both at their scores against thecomposite indices and their overall value for money. Summaries of theassessments for each organisation can be found in Annex 6.

16


27/232

Chaptr 3

Thematic ndings from the review

Overview1. The Multilateral Aid Review provides a rich picture of the strengths and

weaknesses of the multilateral organisations. It is clear that the multilateralsystem forms an essential part of the international architecture for sustainablepoverty reduction. Many multilateral organisations are valued partners fordeveloping country governments, civil society and the private sector. Together

they mobilise large-scale funding, bring specialist expertise, support innovation,play pivotal leadership roles with other donors, have the mandates and legitimacyto help to deal with conict situations, and provide a platform for action in everycountry in the world. Many are a critically important complement to theoperations of bilateral donors such as DFID.

2. This does not, however, mean that every multilateral organisation is performingas well as it should, or making an equal contribution to development andhumanitarian objectives. We found considerable variability in performance.This variability exists at many levels. First, the multilateral organisations as agroup perform better in some parts of the assessment framework than in others.

Second, different kinds of multilateral organisations, such as the multilateraldevelopment banks or the global funds, perform better in some frameworkareas than in others. Third, individual multilateral organisations have their ownparticular strengths and weaknesses.

3. The next sections explain our approach to analysing the assessments, and thendiscuss strengths and weaknesses across the multilateral system.

Analysing the data4. In order to understand better why organisations received the scores they did

on our assessments, we placed them into seven groups. First, those multilateraldevelopment banks where we primarily assessed the concessional funds. Second,development nance institutions and funds supporting private sector development.Third, global funds for health, education and climate change adaptation andmitigation. Fourth, humanitarian organisations, including UN and Europeanhumanitarian organisations. Fifth, other UN organisations. Sixth, the EuropeanCommission, consisting of the European Development Fund and the otherdevelopment instruments (hereafter the European Commission budget instruments),but excluding ECHO. Two organisations did not t into these categories: theInter-American Development Bank, where non-concessional and concessionalnance were equally covered in our assessment, and the Commonwealth

Secretariat. We grouped these together as Other. Although the other

17


28/232

CATeR 3.

organisations are shown in the charts and tables in this chapter, we do not giveany other analysis of their performance here. Table 6 below shows how manyorganisations fall into each group8. A full list of the organisations covered by the

review is given at the beginning of this report.

Tabl 6 rupings f multilatral rganisatins

Multilatral rganisatins . frganisatins

Multilateral developmentbanks, with a focus on theconcessional funds

AfDF, AsDF, CDBi, IDA 4

Development nanceinstitutions and funds

supporting private sectordevelopment

EBRD, IFC, PIDG 3

Global funds for health,education and climatechange

CIFs, FTI, GAVI, GEF, GFATM, UNITAID 6

Humanitarian organisations CERF, ECHO, GFDRR, ICRC, IFRC, IOM,ISDR, OCHA, UNHCR, WFP

10

UN organisations exc.humanitarian

EFW, FAO, HABITAT, IFAD, ILO, OHCHR,PBF, UNAIDS, UNDP, UNEP, UNESCO,UNFPA, UNICEF, UNIDO, UNIFEM, WHO

16

European Commission exc.humanitarian

European Commission budgetinstruments, EDF

2

Other CommSec, IADB 2

Total 43

i We primarily focused on the concessional funding window of the Caribbean Development Bank, theSpecial Development Fund.

8 We could have arranged these groupings differently. In particular, we could have clustered the UN humanitarianorganisations together with the other UN organisations, and the European Commissions humanitarian arm,ECHO, with the other European Commission budget instruments. The reason we did not do this was because itwould lose some important commonalities between the organisations. However, where relevant, the analysisbelow does consider UN humanitarian organisations together with the other UN organisations, and ECHO

together with the other European Commission budget instruments.

18


29/232

19

TeMATC FdS FRoM Te Reew

5. We considered whether or not to weight the assessments to take account ofdifferences in scale across the multilateral system. We decided against this for tworeasons. First, the discrepancy between the largest and the smallest organisation

in the review is just too big. If we weighted the assessments by annualdisbursements, we would generate a weighted average which reects theperformance of only a handful of organisations, losing the richness of our data.Second, the Multilateral Aid Review was intended to assess the value for moneyof each of the multilateral organisations which receive core UK aid funding, andnot to assess the value of the multilateral system as a whole. We have thereforeabstracted from questions of scale, and taken simple averages only.

Strengths and weaknesses across the

multilateral organisations6. As a group, the multilateral organisations highest scores are for their critical rolein meeting international and UK development and humanitarian objectives, theirpartnership behaviour, and their focus on poor countries. But there is not enoughevidence of multilaterals consistently delivering results on the ground, particularlyin fragile states. This is partly because of weaknesses in strategic and performancemanagement, including human resource management, in many multilaterals.Most multilaterals are not sufciently focused on driving down costs or achievingvalue for money. Most multilaterals are not paying enough attention to genderissues. And there is still much room for improvement for the multilaterals as agroup on transparency and accountability.

7. These points are illustrated by chart 4, which gives average scores on eachcomponent across the multilateral system together with the numbers oforganisations in each scoring band. This shows that, for example, the multilateralorganisations as a group averaged a score of 3.2 (where 3 is satisfactory and 4 isstrong) in terms of their critical role in meeting UK development and humanitarianobjectives. Of the forty-three organisations we assessed, fteen were strong onthis criterion, twenty-one were satisfactory, seven were weak, and none wereunsatisfactory. At the other end of the scale, the average score on strategic andperformance management was 2.3 (where 2 is weak and 3 is satisfactory). Fourof the organisations were strong, ten were satisfactory, twenty-six were weak,

and three were unsatisfactory.


30/232

Critic

alro

lefo

rUK

aidobj

ectiv

es

Critic

alro

lefo

r int

erna

tiona

l aid o

bjec

tives

Partn

ersh

ipbeh

avio

ur

Clim

ate

chan

ge/envi

ronm

ent

Focu

son

poorc

ount

ries

Finan

cialr

esou

rces

mana

gem

ent

Fragile

cont

exts

Cost

and

valu

econsc

ious

ness

Cont

ribut

ionto

resu

lts

Gender

equ

ality

Tran

spar

ency

and

accou

ntab

ility

Stra

tegi

c/per

form

ance

mana

gem

ent

43

Chart 4 Strngths an aknsss acrss th multilatral rganisatins

CATeR 3.

numberofmultilateralorganisations

average score100 3.2

3.12.9 2.8 2.8

2.6 2.5 2.5 2.52.4 2.4 2.3

8060

240

120

0 0averagescore

8. This overall picture of strengths and weaknesses masks some of the variabilityacross the multilateral system. For example, the multilateral organisationsgenerally good showing on partnership behaviour is driven primarily by strongperformance by the UN and the European Commission. Equally, their generallypoor showing on cost and value consciousness is largely due to weakperformance by the UN and the humanitarian organisations, and in spite ofstrong performance by the global funds. Table 7 sets out the strengths andweaknesses of the different multilateral groupings.

9. The rest of this chapter examines how the multilateral organisations performagainst the components of the assessment framework. The charts use thefollowing abbreviations:

ECion European Commission, excluding humanitarianGF Global fundsH Humanitarian organisationsMDB Concessional funds of the multilateral development banksPSD Private sector development organisationsUN UN organisations, excluding humanitarian

20


31/232


Tabl 7 Strngths an aknsss f iffrnt multilatral grupings

Strngths (scr >3) waknsss (scr


32/232

CATeR 3.

Fragile contexts Gender equality

Climate change and environmental sustainability

Critical role in meeting development and

humanitarian objectives11. Instead of looking at whether or not the multilateral organisations have mandates

which mean that they are, in theory, important for meeting development andhumanitarian objectives, we asked whether or not multilateral organisations are,in practice, critical to achieving these objectives. We looked for evidence ofmultilateral organisations playing a leading role at the global or national level in,

for example, co-ordinating the development or humanitarian system, providinglarge-scale nance or specialist expertise, lling policy and knowledge gaps, ormeeting humanitarian needs.

12. In doing this assessment we looked rst at international development andhumanitarian objectives, including the Millennium Development Goals (MDGs),and second at the UKs specic development and humanitarian objectives, as setout in DFIDs Business Plan (link to Business Plan). They include a focus onparticular issues, such as wealth creation, governance and security, direct deliveryof the MDGs, humanitarian relief, climate change and environmental sustainability,on low income countries, and on some institutions, such as the Commonwealth,

where the UK has especially strong ties. The effectiveness of the multilateralorganisations in dealing with some of these issues is also explored in more detaillater in this chapter.

13. Chart 5 below shows how the different groups of multilateral organisationswere assessed in terms of their role in meeting international development andhumanitarian objectives. Eighteen of the forty-three multilateral organisations aresatisfactory on this component, and fteen of them are strong.

22
http://www.dfid.gov.uk/About-DFID/Finance-and-performance/DFID-Business-plan-2011---2015/http://www.dfid.gov.uk/About-DFID/Finance-and-performance/DFID-Business-plan-2011---2015/


33/232


Chart 5 Critical rls in mting intrnatinal vlpmnt an humanitarianbjctivs

average score

4

number of organisations%ofscoresthatarestrongorsatisfactory

Strong

Satisfactory

Weak

Unsatisfactory

E Cion H GF MDB PSD UN Other

1 6 2 2 4

1 2 4 1 3 6 1

2 1 6 1

100%50%

0

50% 100%

3.5 3.43.3 3.3

3.02.9

2.5 3210

averagescore

14. Chart 6 below shows how the different groups of multilateral organisationsfared against our assessment in respect of UK development and humanitarianobjectives. Five multilateral organisations that we deem to be only weakly criticalfor the achievement of international objectives are satisfactory in terms of theUKs particular priorities. In some cases this is because they play a key role incountries or regions which are especially important to the UK. In others it isbecause their objectives are so close to the heart of the UKs development agendathat, even though their role is very narrow, it is still critical for the achievementof UK priorities. Equally, two international nancial institutions which play asatisfactory role in meeting international development objectives operate inregions which are not priorities for UK aid, and are therefore seen as only weaklycritical for meeting UK development objectives.

23


34/232

Chart 6 Critical rls in mting UK vlpmnt an humanitarian bjctivs

CATeR 3.

4.0 average score


Strong

Satisfactory

Weak

Unsatisfactory

MDB ECion H GF UN PSD Other

4 1 5 1 4

1 4 5 8 2 1

1 4 1 1

100%50%

050%

100%

3.5 3.4 3.23.0

2.72.5

43210

averagescore

15. Overall, fteen of the forty-three multilateral organisations were judged to bestrong in performing a role which is critical to meeting UK development andhumanitarian objectives, and twenty-one were deemed to be satisfactory. This isa very strong endorsement of the central importance of the multilateral systemfor meeting UK development and humanitarian objectives.

16. There are many reasons why the multilateral organisations are so important to thesuccess of the international aid effort. The most common nding was that theyplay a vital leadership and co-ordination role. Examples included convening

international gatherings, leading donor co-ordination mechanisms, facilitatingstakeholder dialogue, building a shared knowledge base, establishing commonstandards, and providing intellectual leadership. Other critically important aspectsof multilateral organisation behaviour include: the sheer scale of their activities,their specialist technical expertise, the range of instruments they offer, theirunique legitimacy, which enables them to work in every country and tacklepolitically difcult issues, such as electoral reform or population dynamics, andtheir focus on poor countries. Box 2 illustrates some of these.

17. Only ve multilateral organisations were judged to be weakly critical to bothUK and international development and humanitarian objectives. All of them are

24


35/232


small. All of them are specialised UN agencies, programmes or strategies. All havemandates to work in niche areas, but lack the strategic focus and scale to be keyplayers in practice.

Bx 1 illars f th sstm

The World Bank, the United Nations and the European Commission are sometimesreferred to as the three pillars of the international development and humanitariansystem. Table 1 in chapter 1 illustrated the difference they make. The examples belowshow what makes them so important.

dA pvrt ructin at scalThe International Development Association, IDA, is the concessional funding windowof the World Bank. Its comparative advantage is the breadth and quality of itstechnical expertise, the scale of its concessional nancing, and its global reach. IDA

helps 79 of the poorest countries in the world on a wide range of priorities in nationaldevelopment plans, focusing on poverty reduction and the achievement of the MDGs.IDA is, for example, the single largest source of donor funds for basic social services inthe poorest countriesi. Through its size and reach, IDA can have a transformationaleffect that individual national donors cannot match.

wo spcialising in halthThe World Health Organisation WHO, is a UN specialised agency. Its comparativeadvantage is its global legitimacy in health matters, its convening power, and itsauthority to lead and co-ordinate others. WHOs work covers areas such as maternaland child health, malaria, nutrition, drinking water quality and sanitation. It setsstandards and guidelines and establishes conventions, regulations and codes. As anexample, the WHOs Global Malaria Programme provides the latest internationally

agreed recommendations on how to effectively prevent, diagnose and treat malaria atthe lowest cost, while its World Malaria Report tracks progress in 106 malaria endemiccountries. On the ground, WHO helps to improve health systems, including emergencypreparedness and health surveillance, and it leads the Global Health Cluster inhumanitarian situations.

25


36/232

Bx 1 illars f th sstm continuedeCo mting humanitarian n

ECHO is the humanitarian arm of the European Commission. It is the second biggesthumanitarian donor in the world. Its comparative advantage is its huge eld presenceacross the world and in most fragile states, and its size, including the ability to releaseup to c3m in the 72 hours following an emergency. This makes it a signicantcontributor to humanitarian outcomes and allows it to focus on forgotten crises,lling a gap in the system and supporting a more consistent approach to fundingaccording to need. In 2009, ECHO funded relief for approximately 150 millionbeneciaries in more than 70 countries.

i IDA16 Replenishment announcement Q&A and IDA website

CATeR 3.

Focus on poor countries18. This component looks at how well the multilateral organisations target their aid

spending on countries with high levels of need and effectiveness, where aid islikely to have the greatest impact.

19. We assessed need and effectiveness in different ways for different kinds oforganisations. For multilateral organisations with a focus on humanitarian need orconict, climate change and environmental sustainability or disaster risk reductionobjectives, we analysed need in terms of the incidence of the particular problemthey are mandated to address. Where possible, we also looked at their effectiveness

by considering how well allocation decisions are driven by evidence of countryand/or situation specic contexts. For all other organisations, we generated aquantitative index of need and effectiveness, where need is derived from numbersof poor people, human development indicators, and fragility, and effectiveness isbased on the strength of their institutional and policy environment. We thencompared each organisations country by country aid spend with this index. Themethodology and ndings are explained in Box 2, and with more detail in Annex 1.

20. Chart 7 below shows how well focused the multilateral organisations are on thecountries where, according to our assessment of need and effectiveness, they arelikely to have the most impact. We call this focus on poor countries.

26


37/232


Chart 7 Fcus n pr cuntris

average score4


Strong

Satisfactory

Weak

Unsatisfactory

GF MDB H UN ECion PSD Other

3 2 4 2 1

3 1 4 4 1

8 1 1

1 2 1 2

100%50%

050%

100%

3.53.0 3.0

2.62.5

2.3

1.0

3210

averagescore

27


38/232

Bx 2 Assssing fcus n pr cuntris fr multilatral rganisatins that supprtvlpmnt bjctivs

We used indices of need (numbers of poor people, human development indicators andfragility) and effectiveness (strength of institutional and policy environment, usingCPIAi) to construct a country ranking for all low income and lower middle incomecountries. The higher up the ranking, the more likely it is that aid to that country willcontribute to the UKs poverty reduction objectives.

We looked at how the multilateral development organisations allocate their corecountry based aid the ofcial development assistance that they disburse at thecountry level, excluding their global policy and normative work, and any resources notfocused on development concerns and compared this with our country ranking.Organisations that give a large share of their aid to countries high up in the rankingare, in purely geographic terms, more likely to be contributing to poverty reduction.We describe them as having a good focus on poor countries (FoPC).

The chart below gives the FoPC score for the multilateral development organisations,and the share of aid going to countries in different parts of the ranking. It shows that,for example, GAVI gives over 40% of its aid to the 5% of countries that offer thehighest impact on UK poverty reduction objectives, while the CommonwealthSecretariat gives less than 10% of its country-based aid to this group, and almost 40%to upper middle income and high income countries. The FoPC scores were bandedtogether to generate scores for focus on poor countries on a range of 1 to 4.

ProportionofSpendacross

Quartiles(%)andFoPCScore

GA

VI

UNITAID

AsianDev.Fu

nd

PID

G

UNICEF

AfricanDev.Fu

nd

EDF

IFC

Comms

ecFTI

UNDP

UNFPA

GlobalFu

nd

IFA

D

EFW

EC DCI

UNHABITAT,UNIFE

MFA

O

ID

A

UNSpA

gs

IDBSp.Fu

nd

EBRD

CaribbeanDev.Ba

nk

100

80

60

40

20

0

Top 5% Most need Effective countries

Next 5% Most need Effective countries

Rest of 1st Quartile

All of 1st Quartile

Mos Ranked according to Geographic Fit Score

2nd Quartile

3rd Quartile

4th Quartile

i CPIA (Country Policy and Institutional Assessment) is a World Bank index of the quality of a countrysinstitutional and policy framework.

CATeR 3.

28


39/232


21. Twenty-ve of the forty-three multilateral organisations have a strong orsatisfactory focus on the countries where need and effectiveness assessmentssuggest that aid will have the most impact. Six are unsatisfactory. It is very striking

that the newest organisations the global funds have the best focus on thesecountries. This is discussed further in Box 3.

22. Where multilaterals do not have a good focus on the countries where need andeffectiveness assessments suggest that aid will have the most impact, this isoften largely a result of deliberate choices made in the past by the internationalcommunity. For example, most of the multilaterals with a weak focus on thesecountries belong to the UN system. They tend to be organisations which aremandated to have a global eld presence, including in upper middle incomecountries. Equally, those of the international nancial institutions with anunsatisfactory focus on these countries were established to work in regions

which have pockets of poverty, but where most countries are not low income.Finally, for geo-political reasons, the European Commission budget developmentinstruments have a strong focus on middle-income countries, particularly in theregions around the EU.

Bx 3 lbal funs giving ai hr it ill hav th mst impact

The four health and education global funds that we assessed are all in the top tenmultilateral development organisations in terms of their focus on poor countries. Thisis a striking nding, since their demand-led models appear to give them less controlthan some other organisations over where they allocate their resources. On average:

98% of the global funds country funding goes to low and lower-middle-incomecountries; 72% goes to the 25% of these countries that are most likely to deliverpoverty reduction according to our model (most of which are low income), and 26%goes to the 5% of these countries where we judge that aid has the greatest potentialimpact. On average, the equivalent gures for the other development institutions weassessed were 88%, 44%, and 16%.

29


40/232

CATeR 3.

23. The chart in Box 2 shows whether or not organisations are concentrating oncountries where our assessment of need and effectiveness suggests that aid willhave the most impact, but does not explain why they may not be. While sixteen

multilateral organisations are not well focused on these countries, this is partly aconsequence of decisions taken by the international community. Nevertheless,this is a valid question to ask in the context of an assessment of value for moneyfor UK aid funding. It is important that the UKs resources are targeted on theorganisations and countries that are most likely to deliver against the UKsdevelopment and humanitarian objectives.

Contribution to results24. This component assesses whether or not multilateral organisations are setting and

meeting challenging objectives and making a consistently positive contribution to

development or humanitarian results. We were looking for ambitious targets, forexample in innovation or outreach to the poorest, for demonstration of achievementagainst targets at the country level9, and for evidence of management resolvingproblems and striving for a real impact.

25. The requirement to not only achieve results, but also to clearly measure them,made it difcult for multilaterals to score well on this component. Despite this,we found many examples of multilaterals making an important contribution todevelopment or humanitarian objectives, including all of the private sectordevelopment organisations, most of the humanitarian organisations and globalfunds, half of the concessional funds of the multilateral development banks, and

the European Development Fund. Some other UN organisations are having animpact at country level, but many of them nd it difcult to demonstrate thisclearly.

9 The quality of corporate results frameworks and reporting is assessed later under strategic and performance

management. This component looks only for evidence of management for results at the country level.

30


41/232


Chart 8 Cntributin t rsults

average score4


Strong

Satisfactory

Weak

Unsatisfactory

PSD H GF MDB ECion Other UN

1 3

2 5 4 2 1 1 2

1 2 2 1 1 12

1 2

100%50%

050%

100%

3.33.0

2.72.5 2.5 2.5

2.03210

averagescore

26. This section gives some examples of the impact that multilateral organisationshave at country level; more can be found in Table 1 in Chapter 1.

27. The private sector development organisations are making a good contribution toresults. The best of them are innovative organisations which set themselveschallenging objectives and are able to demonstrate delivery against them, butthey tend not to have a strong poverty focus.

Bx 4 Unblcking cnstraints t invstmnt in infrastructur

The Private Infrastructure Development Group (PIDG) aims to mobilise privateinfrastructure investment in poor countries in order to foster economic growth andreduce poverty. PIDG supported projects have attracted $10.5bn of private investment,with every $1 of donor funding delivering some $27 of investment in infrastructure indeveloping countries. Projects that have already reached nancial close are projectedto provide new or improved services to over 50m people and directly generate over160,000 long term jobs.

31


42/232

CATeR 3.

28. We are condent that most of the humanitarian organisations are making a realdifference to the lives of some of the poorest and most vulnerable people undervery difcult circumstances.

29. All of the global funds are able to show a demonstrable impact globally and/orat country level, despite some surprising weaknesses in results reporting. Mostof them have challenging objectives and are often innovative in their approach.However, the global funds are generally dependent on other agencies todeliver their projects and programmes, and they can nd it difcult to resolveproblems in implementation at country level when these occur. It is not clearthat management in some global funds is doing its utmost to sort out thesecountry-level problems.

30. Many of the multilateral development banks are also able to demonstrate deliveryof outputs against challenging objectives, although they nd it more difcult to

demonstrate contribution to outcomes, and their objectives are not always wellpoverty-focused. Many of them played an important role in responding to thenancial crisis. Some of them are innovating and taking risks to deliver criticalprojects in fragile environments, yet IDA in particular struggles with a businessmodel that is not appropriate for fragile states.

31. Most of the multilateral development banks have a good management focuson results. However, this is not always matched by internal incentives. Staff andmanagers tend to be rewarded for getting projects and loans approved, ratherthan for delivering outputs, such as rehabilitated roads, or outcomes, such asshorter journey times to markets, schools and health facilities, higher incomes

and healthier and better educated populations.32. The European Commission suffers from weaknesses in its results reporting.

Nonetheless, in most cases we were able to nd indirect evidence of impact.The European Development Fund fared reasonably well against our criteria.Objectives appear ambitious, there is good evidence of innovation, such as theMDG contracts (see Box 5), external assessments indicate reasonable performance,and there are many examples of contributions to outputs and even outcomes atthe country level. It was more difcult to nd evidence of contributions todevelopment objectives for the European Commission budget instruments.

32


43/232


Bx 5 eurpan Cmmissin innvating fr rsults

The MDG Contracts offer long term, predictable budget support with a variable

performance tranche to low income countries, mainly in sub-Saharan Africa. The aimis to accelerate real improvements in reaching the MDGs by 2015. The MDG Contractsclearly link EDF funding to results.

In Rwanda, primary school completion rates increased from 52% to 75.6% and 80%for girls between 2006 and 2009/10, and, in Zambia, the proportion of HIV pregnantwomen receiving anti-retroviral treatment increased from 40% in 2007 to 66% in 2009.The European Commission has also earmarked c1 billion of EDF funds to tackle themost off-track MDGs in Africa, the Caribbean and Pacic countries. The new countryowned MDG Initiative will provide extra funds to partner countries strategic plansand budgets and help to address urgent challenges, including around maternal andchild mortality and access to water.

33. We found evidence of some UN organisations having an impact on developmentresults at the global level, or making a real difference at country level. We metstaff in UN organisations who were passionate in their ambition and drive toachieve their objectives. Box 6 gives an example drawn from our countr

multilateral_aid_review_full_linked

Documents