evaluation of the start fund design & build phase

EvaLuaTIon oF thE Start fund

dESign&buildphaSEA U T H O R : a N dy f e aT h e r S T o N e | F U N D A N A Ly S I S : Ta S N e e m m o w j e e | j a N u a r y 2 0 1 5

UK aid branding guidance June 2014

d o N o r S | pa r T N e r S

The STarT NeTwork haS aN ambiTiouS viSioN which emphaSizeS The imporTaNce of embraciNg diverSiTy iN humaNiTariaN reSpoNSe, deceNTraliSiNg power from The global To The local aNd leveragiNg The poTeNTial of workiNg collaboraTively

03 | Evaluation of thE Start fund dESign & build phaSE | January 2015

1 | iNTroducTioN

The Start Fund is a multi-donor pooled funding mechanism that consists of a network of international non-governmental organisations (NGOs) and provides small-scale funding (up to 300k per organisation over a 45 day window) to frontline NGOs. It seeks to complement the existing humanitarian financing system by focusing on initial rapid response to low-profile, underfunded emergencies. The first phase of the Start Fund, April to December 2014, represents a design and build period where the Fund is open for allocations to crisis alerts, whilst simultaneously the infrastructure and Fund management processes are being tested and refined.

PurPose of the evaluation and methodologyThe main objective of the evaluation is to assess the performance of the Start Fund during the 9-month design period to draw lessons on areas for improvement in order to inform the on-going development of the Start Fund. The overall purpose of the evaluation is for learning, especially to support the scale-up of the Fund. The evaluation used a mix of methods, both qualitative and quantitative, to answer the evaluation questions which included interviews with the Start Team, Start Network members and external informants, a document review and analysis of Start Fund data and key performance indicators, two country case study visits (to Nairobi to discuss the two Somalia activations and to Bangladesh to discuss the floods activation). The evaluator also participated in the Start Network Assembly, Donor Forum and Allocation Committee strategy meeting.

2 | evaluaTioN fiNdiNgS

relevanceIn the first nine months a Start Fund Alert was raised 19 times, and the Fund has been activated on 12 occasions. While a review of the activations shows that the Fund had broad relevance across different types of emergencies in a range of countries, it found that there was greatest consensus on responses to medium-sized emergencies (natural and man-made) and least consensus on responses to slow onset crises. The Fund failed to identify or respond to a localised or small-scale crisis (such as mud-slides, fires or building collapse). The Fund demonstrated its ability to activate in an anticipatory capacity which has the potential to further increase its speed of response.

Given the swift activation of the fund, it offers a good fit with some of the UN pooled funds (ERF, CHF and CERF), which take longer to activate but often disburse larger funding envelopes to a broader range of organisations for longer project timeframes. The focus of the Start Fund on medium-sized humanitarian emergencies is

also a good fit with both the CERF (which tends to focus on larger crises although is also activated for spikes in chronic crises) and with bilateral funding instruments such as DFIDs RRF, which disburses larger grants to international NGO partners responding to large crises.

effectivenessWith only a small number of minor exceptions, the Start Fund has consistently met or exceeded its KPIs. For a fund that has only been established for nine months and which has set itself an ambitious target for its activation that exceeds the aspirations of other pooled funds, the Start Team and members have performed well in consistently meeting their performance targets. The agility of the Start Fund in responding to crises has filled a gap that many would have considered beyond the reaches of a membership-based, peer-led, global, pooled fund. Knowing that timely response to humanitarian needs can play a significant role in saving lives and supporting livelihoods, the evidence suggests that the Start Fund is positioned to have impact at acute stages of a crisis.

The process of alert, activation, allocation and project selection are peer-managed and have largely proved themselves to be fit for purpose and rigorously implemented. There is scope to better prepare country-level staff for the decentralised project selection process, a need to simplify and strengthen the project assessment criteria and to improve quality assurance. However, the 12 activations have been successful in meeting timeliness standards across a range of interventions. The fact that a learning component is hard-wired into the Start Fund offers an opportunity for members to participate in an exciting learning agenda.

For such a young entity, the governance functions are well-developed and appear to work effectively, albeit with some space for improvements on closing the feedback loop between the different Fund management, selection and learning committees and with a need for greater clarity about how the Donor Forum can be best used to support the strategic direction of the Start Fund. One area where progress is required is in articulating a risk management and control framework, which will be necessary to achieve the Funds ambitions of growth and independence.

collaboration, decentralisation and diversityThe Start Network has an ambitious vision which emphasizes the importance of embracing diversity in humanitarian response, decentralising power from the global to the local and leveraging the potential of working collaboratively. In mobilising the 19 members and their network of 6,700 partners to respond in a timely way, the Start Fund has shown some measure of success in role-modelling the changes it is seeking in the wider humanitarian community.

execuTive Summary


The use of peer review mechanisms, the decentralisation of key decision-making processes and a focus of members on accountability to crisis-affected populations are all consistent with these values. However, there are important decisions to be made about how to diversify the Start Funds membership and how far to stretch decentralisation. The location of the Fund in the UK and the comparatively large number of UK-based members has led to misperceptions about it being a British rather than global fund an issue that needs to be addressed if the network is to achieve its aspirations to change the humanitarian system.

sustainabilitySome of the most significant challenges for the Start Fund are related to its future direction and how it will seek to balance the need to expand its donor base while increasing the diversity of its members. While it has been grappling with these strategic challenges, the humanitarian financing landscape has been changing with a growing number of NGO pooled funds which have the potential to both complement and compete with the Start Fund.

Despite the challenges, the commitment that has been made by DFID which includes a significantly larger contribution for the coming year will already push the Start Team and members into unchartered territory, and although plans have been put in place to maintain the Start Funds impressive record, there will continue to be the need to learn and adapt as new challenges arise.

3 | coNcluSioN aNd recommeNdaTioNS

The members of the Start Network and Team have proved worthy custodians of the funding entrusted to them by Irish Aid and DFID. While the fund size has been modest in comparison with other pooled funds, there is ample evidence of diligent fund oversight and management which has delivered some impressive results both in terms of what can be achieved through collaborative action and in terms of responding to humanitarian needs. That is not to say that proof of concept will guarantee the future success of the Fund. The evaluation has highlighted a number of issues both strategic and operational that will be important to address to strengthen Fund processes and outcomes in the future. Looking forwards, the evaluation has highlighted a number of challenges ahead, and while it can offer recommendations for how the Fund can position itself to navigate these, it will only be through galvanising the collective support of the membership and being able to breathe life into its vision that it will stand the greatest likelihood of long-term success.

1 | STreNgTheNiNg aNTicipaTioN [ref: p11] Anticipation is an area that the Start Fund has the potential to strengthen given the operational reach of its 19-members and the resources that already exist in the Fund. By participating in the Early Warning Early Action Group and by integrating its analysis into its decision-making processes, the Fund could improve the potential for early action as well as strengthening its complementarity with the Central Emergency Response Fund.

2 | coordiNaTiNg wiTh oTher Ngo-led pooled fuNdS [ref. p13] The similarities between the Canadian Humanitarian Assistance Fund and the Start fund suggests that there would be added value in establishing more formal coordination mechanisms between the two funds which should include at a minimum, notification of alerts and activations. With the possibility of similar funds being established in the future, and given the emphasis placed by the Start Fund on collaboration, there is scope for the Start Team to role-model this.

3 | evaluaTiNg projecT ouTpuTS aNd ouTcomeS [ref. p16] Given the focus of this evaluation on process and governance issues, there is a keen interest from the members to look in more detail at project outputs and outcomes. Despite the methodological complexities, by commissioning a field-focused evaluation which prioritises project review and which focuses on interviews with field staff, partners and crisis-affected communities, it should be possible to build up a more detailed picture of changes that can be attributed to the disbursement of timely Start Funding. Furthermore, if a range of different activations were selected (e.g. rapid/slow onset/spikes etc.), then it may be possible to strengthen the evidence base for the types of activation where the Fund can have greatest effect.

4 | meaSuremeNT of fuNd leveragiNg [ref. p17] The key Performance indicator on fund leveraging is poorly articulated and exaggerates the

Pr

ior

ity r

eco

mm

end

atio

ns

The memberS of The STarT NeTwork aNd Team have proved worThy cuSTodiaNS of The fuNdiNg eNTruSTed To Them by iriSh aid aNd dfid


funding that is raised as a consequence of the Start Funding received by members. If fund leveraging is considered to be an important metric, then members should only report funds that were raised as a direct consequence of the Start Funding. An accompanying question which would assist in strengthening the evidence base for the 45-day activation period would be whether members received follow-on funding for the specific Start Funded intervention.

5 | delegaTiNg allocaTioN proceSSeS To The STarT Team [ref. p19] With a view to a future Start Fund with more members and a larger fund size that is likely to be activated more frequently, some concern was raised about the mechanism to trigger a delegated process. One means of strengthening this would be to more clearly outline the dimensions and complexity of a crisis in the alert note. Requiring the alerting agency to outline the crisis against a pre-determined set of criteria (which could include the number of people/refugees/displaced persons affected, the geographic area affected, the political complexity of the crisis, security/risk rating) would simplify the identification of crises which may lend themselves to Start Team oversight.

6 | maiNTaiNiNg The iNdepeNdeNce of The STarT fuNd [ref. p20] It is important that the Start Network communicates and defends the independence of the Fund, ensuring that activations are consistent with the commitment of its members to the humanitarian imperative.

7 | QueSTioNiNg The relevaNce of The STarT fuNd for reSpoNSe To Slow oNSeT criSeS [ref. p20] The evaluation has found that the use of the Fund to respond to slow onset crises has been contentious among the members and some of the results have been disappointing. The Allocation Committee should either make changes to the process for responding to these crises to allow additional time for analysis and project

development to strengthen the relevance of the interventions or it should concede that these response types are not best served by the Start Fund.

8 | STreNgTheNiNg The imparTialiTy of The projecT SelecTioN proceSS [ref. p22] To strengthen the impartiality of the project selection process, it is recommended that the questions under each of the criteria are re-formulated and re-phrased so they are less open-ended and lend themselves more easily to applying a score. While it is understood that project selection cannot and should not be a quantitative exercise, given that scoring is an important part of the project selection process, it is in the interests of transparency and accuracy that the process provides the greatest assurance of consistent usage across all of the members.

9 | STreNgTheNiNg QualiTy aSSuraNce iN The projecT SelecTioN proceSS [ref. p22] The project application template and peer review scoring process should be revised so that members are required to articulate relevant standards and to routinely interrogate these as part of the project selection process. This should be complemented with a field-based review that is triggered after a specific time period or number of activations. This would provide evidence to support the assertion that the peer review process is sufficiently robust to drive up programme quality.

10 | cloSiNg The gap iN The goverNaNce feedback loop [ref. p24] The Allocation Committee (which is also the mandated fund management body), Project Selection Committee and Learning and Evaluation Committee require a regular structured dialogue in order to close the feedback loop to maximise the potential of strengthening the Start Fund. Given that the Allocation Committee meets four times over the course of the year, it is recommended that at least one of these meetings is given over to a formal review of learning about the Fund process.

11 | clarifyiNg The role of The doNor forum [ref. p28] Given that Start Fund members and donors have a common interest in having a well-managed and well-resourced Fund and have a shared motivation to drive forward the ambitious strategy for growth, there would be considerable benefit to more clearly articulating a terms of reference for the Donor Forum. This should clarify its purpose and function, describe its links to other governance bodies, outline its membership and establish a standing agenda for its meetings. Included in the agenda should be an annual discussion about progress made towards achieving the vision, a biannual review of the performance and discussion about risk management. It is also recommended that at a minimum representation from each of the Board, Assembly and Allocation Committee (given its current Start Fund management role) is formalised.

12 | eSTabliShiNg a riSk maNagemeNT aNd coNTrol framework [ref. p29] In outlining key risks and articulating a summary risk mitigation plan, the Start Fund has begun to identify some of the risk management infrastructure that will need to be put in place as the Fund prepares to expand. In making progress in this, it will be important to negotiate a framework that will meet donor needs for risk management but that is also consistent with the Start Networks vision of having a diverse membership which includes international and national NGOs.

13 | agreeiNg The fuTure of The fuNd [ref. p33 & 34] It is recommended that the Start Network produce a three-year strategy which includes a road map and tactics for the Fund that are consistent with the Networks Declaration of Intent and which provides clarity over the vexed issue of vison-led versus donor-led growth. This would offer a powerful tool to build consensus and galvanise support of the membership and Donor Forum.


coNTeNTS

execuTive Summary 03Table of coNTeNTS 06liST of abbreviaTioNS 07ackNowledgemeNTS 07

1 | iNTroducTioN aNd purpoSe of The evaluaTioN 081.1 introduction 081.2 Purpose of the evaluation 081.3 methodology 081.4 limitations 091.5 structure of the report 09

2 | backgrouNd To The STarT NeTwork aNd fuNd 102.1 the start network 102.2 the start fund 102.3 an analysis of relevant funding instruments 13

3 | relevaNce 153.1 classifying crises understanding the role of the start fund in responding to crises 153.2 an analysis of the complementarity of the start fund with other humanitarian funds 17

4 | effecTiveNeSS 194.1 analysis performance metrics and project results 194.2 analysis of process effectiveness 224.3 the potential for learning 274.4 start team and network member resourcing 294.5 donor engagement and participation in the strategic direction of the fund 304.6 risk management and control framework 31

5 | The coNTribuTioN of The fuNd To advaNciNg collaboraTioN, deceNTraliSaTioN aNd diverSiTy 325.1 collaboration 325.2 decentralisation 335.3 diversity 35

6 | The fuTure SuSTaiNabiliTy of The fuNd 366.1 the challenge of achieving a sustainable start fund 366.2 fit for the future? the implications of fund growth on the management model 36

7 | coNcluSioN 387.1 Priority recommendations 38

aNNexeSannex 1: Summary terms of reference for the evaluation 40annex 2: evaluation participants 45annex 3: re-organised evaluation framework 46annex 4: start fund timeframe diagram and process 47annex 5: Proposed modifications to the project selection criteria 48annex 6: comparison of pooled fund risk management, quality assurance and control 50


abbreviaTioNS

ac allocation committeeacapS assessment capacities Projectacf action against hungercbha consortium of british humanitarian agenciescerf central emergency response fundchaf canadian humanitarian assistance fundchf common humanitarian funddac development assistance committeedec disasters emergency committeedecaf disasters emergencies committees accountability frameworkdfid department for international developmentdi development initiativesdrc democratic republic of congoecho european commission humanitarian aid and civil Protection departmenterf emergency response fundfSl food security & livelihoodsfTS financial tracking systemhc humanitarian coordinatorhcT humanitarian country teamiaSc inter-agency standing committeeidp internally displaced Personimc international medical corpsirc international rescue committeejNa Joint needs assessmentkpi Key Performance indicatorl&ec learning and evaluation committeeNgo non-governmental organisationocha office for the coordination of humanitarian affairsoecd organisation for economic cooperation and developmentpSc Project selection committeerapid responding to Pakistans internally displaced (fund)rr rapid response (cerf funding Window)rrf rapid response facilityrTe real time evaluationufe under-funded emergencies (cerf funding Window)uN united nationsuSaid united states agency for international developmentwaSh Water, sanitation and hygiene

ackNowledgemeNTS

The Start Team have been exceptionally helpful in supporting requests for documentation and information and have also been extremely candid in their participation in the evaluation. My special thanks are due to Matt Kletzing, who has been extremely diligent in supporting the evaluation process. I would also like to thank Tasneem Mowjee who undertook the analysis of humanitarian funding instruments. Muslim Aid was particularly helpful in providing logistics and administrative support for the case study trip to Bangladesh.


1.1 | introduction

The Start Fund is a multi-donor pooled funding mechanism that consists of a network of International NGOs and provides small-scale funding (up to 300k per organisation over a 45 day window) to frontline NGOs. It seeks to complement the existing humanitarian financing system by focusing on initial rapid response to low-profile, underfunded, emergencies. The first phase of the Start Fund, April to December 2014, represents a design and build period where the Fund is open for allocations to crisis alerts, whilst simultaneously the infrastructure and Fund management processes are being tested and refined.

1.2 | PurPose of the evaluation

The main objective of the evaluation is to assess the performance of the Start Fund during the nine month design period to draw lessons on areas for improvement in order to inform its on-going development. The evaluation asks questions about the Funds performance and covers its processes and some programmatic issues. It also assesses the extent to which the Fund has been successful in advancing the Start Networks vision of shifting the power. The evidence generated will be used by the Start Team, the Start Fund committees and the Start Network Board to improve the Start Fund. The overall purpose of the evaluation is for learning, especially to support the scale-up of the Fund (the full terms of reference is reproduced in annex 1).

1.3 | methodology

The evaluation used a mix of methods, both qualitative and quantitative, to answer the evaluation questions. These included the following approaches.

1 | iNTroducTioN aNd purpoSe of The evaluaTioN

An inception report was prepared which outlined in detail the approach to be used and specific methodologies for the evaluation. It identified key informants and the support required by the evaluator throughout the process. It served as an evaluation guide and accountability tool between the evaluation team and the evaluation manager.

A set of guided conversations were conducted to feed back on the performance of the Fund, to identify and analyse specific issues and to provide a briefing for the field work. External stakeholders were identified and interviews were conducted both prior to and after the field work. Interviews were also conducted with Start Network members in selected countries that have received Start Fund allocations in a range of contexts. A set of questions was developed to guide the interviews which were modified to meet the needs of different stakeholder groups (evaluation participants are listed in annex 2).

A document review and analysis was undertaken which included literature on Start Network, Start Fund policy and guidance documents, humanitarian financing evaluations and policy documents and external reports on humanitarian anticipation and response. Key performance indicators (KPIs) for the Start Fund were analysed against performance targets. A comparative analysis was also conducted of measures of pooled fund effectiveness from a range of global and country-based humanitarian pooled funds.

Two case study countries (Kenya for the two Somalia activations and Bangladesh) were visited as part of the review to offer an opportunity to gather field experience of the Start Fund from its members and external informants to strengthen the evidence base of the evaluation. The field work did not seek to assess the outputs or outcomes of Start Funded projects, which would have gone beyond the ToR of the evaluation. Rather they sought to assess the relevance of the Fund, analyse its effectiveness, assess collaboration and partnership and elicit opinions on the future of the Start Fund.

The evaluator participated in the Start Network Assembly, Donor Forum and Allocation Committee strategy meeting. The purpose of this was twofold: to strengthen the evaluation findings and to provide real-time feedback on some of the early analysis.

incePtion rePort

intervieWs With the start team, netWorK members and external informants

Performance and oPerational document revieW and data analysis

fieldWorK

ParticiPation in start fund meetings

ACTIvITy DETAILS


1.4 | limitations

The ToR is extremely broad and there was understandable interest by members in participating in the evaluation. As a consequence the interview phase of the evaluation lasted longer than was anticipated. To compound this, the allocation of days for the exercise was significantly reduced which has limited the time available for the evaluation team to fulfil the terms of reference. The ToR for the evaluation outlined a large number of questions, and it was necessary to re-organise them according to evaluation criteria (the re-organised evaluation framework is reproduced in annex 3). While the ToR outlined an evaluation of the first six months of the Start Fund, where relevant data was available, the evaluation expanded this to include the first nine months.

1.5 | structure of the rePort

Section 1 of this report provides an overview of the purpose of the evaluation and the methodology. Section 2 introduces the Start Network and Fund and locates it in the wider humanitarian financing landscape. Section 3 seeks to determine the relevance of the Start Fund to humanitarian response and assess its added value and complementarity with other funds. Section 4 assesses the effectiveness and performance of the Start Fund in its first nine months. Section 5 analyses the contribution of the Start Fund to advancing the vision of the Network. Section 6 assesses the extent to which evidence from the design and build phase supports future scale-up and discusses the implications of a scale-up on the Start Fund Team and wider Network. Section 7 concludes the evaluation and outlines primary and secondary recommendations.

Recommendations have been highlighted in bold where they occur in the report and the priority recommendations have been extracted, summarised and presented in the final section of the report and the Executive Summary.


2.1 | the start netWorK

The Start Network is a consortium of 19 NGOs working together to strengthen the humanitarian aid system. The consortium works in three areas: Start Fund (financing for emergency response the subject of this evaluation); Start Build (strengthening civil society capacity); and Start Beta (creating platforms for partnerships and learning).

The members of the Start Network promote a way of working that seeks to enable international and local humanitarian actors to coexist. The vision of the Network is of a self-organising system where the agencies best placed to respond to a crisis are empowered to do so. To realise this vision, the members aim to work together to catalyse a humanitarian sector that is more diverse, decentralised and collaborative (figure 1 provides an explanation of these terms).

2.2 | the start fund

What is the start fund?The Start Fund is a multi-donor pooled funding mechanism that aims to draw on traditional humanitarian bilateral donors, newly emerging humanitarian donors, major private philanthropic organisations and contributions by Start Network members. Its creation arose out of a common desire from its members to promote early action and reduces suffering, mortality and morbidity2 and is based on the rationale that enabling NGOs to manage their own emergency response fund will enable the best placed organisations to respond to more emergencies, more rapidly, which will strengthen their responsive capacity to provide life-saving humanitarian assistance to reduce the suffering, mortality and morbidity of affected populations.3

The Fund provides small scale funding to the Networks 19 members and their partners (up to 300k per organisation over a 45 day window to their members and partner organisations), with a focus on initial rapid response to low-profile, underfunded, emergencies. It releases funds to responders within 72 hours, and work starts within seven days. It evolved from the Emergency Response Fund which was established and managed by the Consortium of British Humanitarian Agencies (the CBHA) and has benefitted considerably from the lessons learned during its tenure.4

2 | backgrouNd To The STarT NeTwork aNd fuNd

f i g u r e 1

STarT NeTwork declaraTioN of iNTeNT our collecTive viSioN1

we collaboraTe becauSe The chaNge ThaT iS demaNded of uS caNNoT be achieved by aNy SiNgle orgaNiSaTioN aloNe. TogeTher, STarT NeTwork ageNcieS caN TraNSform criSiS reSpoNSe. we will Shape a SySTem ThaT iS:

diverSe: The humanitarian system must increase its diversity and tolerance of alternative approaches. We aim for a humanitarian ecosystem that contains organisations of different sizes, types, cultures and modes of response, in a state of continual experimentation and growth.

deceNTraliSed: We aim to shift the centre of humanitarian gravity, so that decision making and leadership take place at the front line and affected people are empowered to improve their lives. Everyone has a contribution to make to reduce the risk of crises, whether it is at a global or local level, but we need to ensure that local ownership and capacity driven humanitarian response.

collaboraTive: Crises in the future will demand humanitarian response that involves many more people and organisations than today. We will need to do different things, and work together in new ways. Relationships across boundaries national, cultural, organisational will be key in rising to this challenge. We will make collaboration central to our action and not allow competition between our agencies to interfere with our common objectives.

1 See Start Network Declaration of Intent, 14 February 2014, p1. 2 Start Fund outcome statement articulated in the DFID six month log frame.3 Start Network (2014) Start Fund Handbook, 25th March 2014, p4.

4 For an independent evaluation of the CBHAs Emergency Response Fund, see http://www.start-network.org/wp-content/uploads/2014/01/October_27_Desk_Review_of_CBHA_ERF_final.pdf


To date the Start Fund has secured funding of up to 30,511,177 GBP over three years, with contributions from two national government donors: Ireland (Irish Aid) and the United Kingdom (DFID). (During the nine-month design and build phase, 2,504,054 GBP was available for disbursement.) The Fund seeks to complement existing global humanitarian pooled funds by focusing on rapid response to under-served and small to medium sized emergencies (see figure 2 for a description of the baseline and anticipated reach of the Fund).

hoW does the start fund WorK?6

Any member of the extended network can raise an alert though the 19 members about an emergency. The Allocation Committee (AC) decides whether to hold a teleconference within 24 hours of the alert. On the basis of a member survey, a context analysis provided by the Assessment Capacities Project (ACAPS) and a funding analysis provided by Development Initiatives (DI), the AC holds a teleconference to discuss activating the Start Fund. Qualifying agencies then have 24 hours to submit a full application to the Start Team. A country-based Project Selection Committee (PSC) agrees which applications to support within 24 hours of receipt, using criteria based on OECD/DAC evaluation criteria7 (where this is not possible, or not desirable, a UK-based PSC takes its place). The Start Team will initiate the transfer of the awarded funds within 24 hours of the decision. Committee decisions are mostly made by peer review which is intended to be impartial and to avoid undue bureaucracy. This is done to enable the Network to make collective decisions and uphold the quality of the process.

5 Ibid. 6 This section is adapted from the Start Fund Handbook, 25th March, 2014.7 In 1991 the Organisation of Economic Cooperation and Development/Development Assistance

Committee established several principles of evaluation to guide member states. These principles have subsequently been developed into five specific criteria which are widely used in evaluation.

f i g u r e 2

STarT fuNd baSeliNe aNd aNTicipaTed reach5

baSeliNe: The current financial architecture for humanitarian crises leaves a gap for NGOs. Local and national NGOs especially struggle to access international sources of funding. Large multilateral pooled funds and public fundraising efforts, such as agencies individual appeals and the efforts of the Disasters Emergency Committee, often leave a large number of small to medium scale disasters unaddressed, neglected or forgotten. Some pooled funds are directly accessible by NGOs, and money from the CERF reaches NGOs through sub-contracts, but the speed of funding is slow, and this inhibits NGO capacity to respond to crises of all sizes and scales. This situation could be improved by: l funding for under-served emergencies (whether they are small, cyclical or neglected/forgotten) l funding is needs-based and allocated through peer-review to the agency that is best placed to respondl direct, fast funding for ngos (within 72 hours)

reach: Based on the absorptive capacity of the global Network and the fact that emergencies are unpredictable, it is estimated that the Start Fund (with 20 million capacity) will be able to respond to approximately 40 crises per year, to save the lives, alleviate suffering and protect the dignity of 10-15 million affected people currently under-supported disasters. The Start Network currently consists of 19 international NGOs with formal implementation and accountability relationships with close to 7,000 partner organisations in 200 countries.


f i g u r e 3

alerTS, acTivaTioNS aNd implemeNTiNg ageNcieS for The STarT fuNd iN iTS firST NiNe moNThS8

South Sudan l action against hungerl Christian aid l plan l tearfund

Somalia 1l international rescue Committeel muslim aid

Somalia 2l relief internationall Christian aid

bangladEShl Carel Christian aid l islamic reliefl oxfam

YEmEnl international medical Corpsl Save the Children

CamErooninternational medical Corps l

plan l

turKEYCafod l

Concern Worldwide l

paKiStan

China

afghaniStan

nEpall handicap international l plan

mYanmarl Christian aid

nigEriaSave the Children l

drCCafod l

Christian aid ltearfund l

india

KEnYa

Fund activation

Implementing agencies

alert not activated

As a guide, not more than 50% of the available funds can be used in any one emergency. The maximum amount that can be requested by any member for a given emergency is 300,000. Start Fund grants are available in 72 hours, and the work must start in seven days and be completed in 45 days. Reports are due within 60 days of the start of the response. The Start Fund is accompanied by a Learning Framework that incentivises improved performance by offering a grant of 1% of the funded amount for learning activities. This aspect of the Start Funds work and monitoring and evaluation of the Funds Key Performance Indicators (KPIs) is overseen by the Learning and Evaluation Committee (L&EC), which meets to peer review the results of every activation by assessing performance at the project, crisis response and process or system levels.

What has been achieved in the first nine months?The first phase of the Start Fund from April to September 2014 represented a design and build period where the Fund was open for allocations to crisis alerts, whilst simultaneously the infrastructure and Fund management processes were being tested and refined. In its first nine months, the Start Fund has been alerted 19 times and has funded 29 projects in 12 emergencies and has spent a total of 2,106,121 to reach 1,320,017 disaster-affected people (see figure 3).9 49% of the funding applications received were awarded, with funding disbursed to 17 of the 19 member organisations, and 84% of the available funds were disbursed.

9 The 20th alert came on December 17, after writing this report, and has not been included in the calculations. For this alert, the Fund was activated in less than 48 hours for crisis response to isolated

communities in Columbia and awarded 27,469 GBP to one project that is expected to reach 3,600 people.

iN iTS firST NiNe moNThS, The STarT fuNd haS beeN alerTed 19 TimeS, fuNded 29 projecTS iN 12 emer-geNcieS aNd haS SpeNT 2,106,121 To reach 1,320,017 diSaSTer-affecTed people


2.3 | an analysis of relevant funding instruments

In recent years, the humanitarian financing landscape has become increasingly complex with a growing diversity of funds established which target different constellations of organisations and types of emergencies. Included in this are UN-managed and NGO-led pooled funds, umbrella humanitarian organisations which raise funds for member responses to large crises, and donor-managed rapid response funds disbursed through NGOs. While multi-agency funds can be quite different from each other, many have some areas of commonality with the Start Fund which make them of interest to the evaluation. The section below seeks to briefly describe each of these types of funds to draw out the commonalities to provide a basis for comparative analysis in the evaluation (see below for an introduction to each of the funds and figure 4 for a comparison of each against standardised criteria).

un-managed and ngo-led Pooled fundsPooled funds have been used as a means to facilitate coordinated response for many years although they have gained greater visibility with the implementation of humanitarian reform after 2005. They are generally considered to be more flexible and are often more quickly activated than bilateral funding. While the proportion of humanitarian assistance that is channeled through pooled funds has remained relatively constant (around 5% between 2009 and 2014), the volume of humanitarian assistance has increased from $824 in 2009 to over $1 billion in 2013.10

There are three main types of UN-managed pooled funds that are in common use: Common Humanitarian Funds (CHFs), Emergency Response Funds (ERFs) and the Central Emergency Response Fund (CERF). The CERF is a global pooled fund while the CHF and ERF are country-based pooled funds.

CHFs are country-based pooled funds that provide early and predictable funding to NGOs and UN agencies for their response to critical humanitarian needs. CHFs enable Humanitarian Country Teams (HCT) to allocate resources where they are most needed, and to fund priority life-saving projects as identified in humanitarian action plan. CHFs are currently present in six countries including Afghanistan, the Democratic Republic of the Congo, Somalia, South Sudan, Sudan and the Central African Republic. CHF funding decisions involve cluster leads and other key humanitarian partners in a comprehensive prioritisation and allocation process. CHFs are managed by the Humanitarian Coordinator (HC), with support provided by OCHA.

ERFs provide NGOs and UN agencies with rapid and flexible funding to address critical gaps in humanitarian emergencies. ERFs, also known as Humanitarian Response Funds in some countries, are usually established to meet unforeseen needs not

included in humanitarian action plan. ERFs predominantly fund NGOs and may support local NGO capacity-building. They are relatively small in comparison with other UN pooled funds (less than US$10 million per year) and provide small to medium size grants. Funding decisions are taken by the HC and the HCT after a consultation and prioritization process. HCs oversee the ERFs, with support provided by OCHA. In February 2013, there were 13 active ERFs in Afghanistan, Colombia, Democratic Republic of the Congo, Ethiopia, Haiti, Indonesia, Kenya, Myanmar, Pakistan, State of Palestine, Syria, yemen and Zimbabwe.

The CERF provides initial funding for life-saving assistance at the onset of humanitarian crises, and critical support for poorly funded, essential humanitarian response operations. Each year, CERF allocates approximately US$400 million. It has three objectives: to promote early and coordinated action and response to save lives, to enhance response to time-critical requirements, and to strengthen humanitarian response in under-funded crises. Similar to the country-based pooled funds, CERF funding decisions are made based on a prioritization process facilitated by the clusters and allocation decisions are made by the HC and often include the participation of the HCT.

NGO-led pooled funds, to which the Start Fund is a new addition, are far less common than UN-managed pooled funds; one of the better known examples is the Responding to Pakistans Internally Displaced (RAPID) fund.11 Established in 2009 through an agreement between USAID and Concern Worldwide, in its first four years, RAPID provided over 29 million to local, national and international NGOs.

umbrella humanitarian organisationsUmbrella organisations are formed at the national level and support joint fundraising for specific emergencies, which are often large-scale. While they are different to pooled funds in several important aspects, particularly in that the funds they raise are earmarked for specific crises, they also share some similarities, including an emphasis on inter-agency collaboration around a common crisis and the need for a common mechanism for funds to be allocated, managed and reported on.

In the United Kingdom, the Disasters Emergency Committee (DEC) is formed of 13 humanitarian aid agencies that fund raise jointly in times of disaster such as flood, earthquake or famine in poorer countries around the world. Appeals are reserved for major disasters and emergencies that cannot be dealt with by the usual coping mechanisms within affected countries and where DEC member agencies are in a position to respond quickly and effectively. Since its launch in 1963, the DEC has run 64 appeals and raised more than 1.1 billion.12 There are similar umbrella organisations in other parts of the world, most notably Canadas Humanitarian Coalition.13

10Global Humanitarian Assistance (2014) Global Humanitarian Assistance Report, 2014, Development Initiatives, p.60. 11 For more information about the RAPID fund, see https:

//www.concern.net/where-we-work/asia/pakistan/rapid-fund12 http://www.dec.org.uk/about-dec13 http://humanitariancoalition.ca/about-us


donor-managed raPid resPonse facilitiesA relatively recent humanitarian funding phenomenon which also seeks to galvanise collective humanitarian action are donor-managed rapid response funds. Membership typically comprises of pre-qualified organisations that are either headquartered in the country, or which have offices established there. DFIDs Rapid Response Facility (RRF) can mobilise in 72 hours following a rapid onset disaster, a spike in a chronic humanitarian emergency, or other disasters. Members of the RRF are all international NGOs with a proven record of response which have been through a robust pre-qualification process. The RRF is activated only for large-scale emergencies.

With a different configuration of stakeholders and mandate Canadas

Department of Foreign Affairs, Trade and Development has established a partnership with the Humanitarian Coalition to create the Canadian Humanitarian Assistance Fund (CHAF). The purpose of the CHAF is to facilitate a timely and effective response to localized disasters that have not received global attention. It seeks to enhance the capacity for early response by leveraging the established local presence maintained by the member agencies.

At the time of the evaluation the proposed creation of similar humanitarian funds by the governments of Netherlands and the United States that will be disbursed through their networks of national NGOs suggests that this may be becoming a wider humanitarian financing trend.

f i g u r e 4

compariSoN of The characTeriSTicS of humaNiTariaN pooled fuNdS

14 Thompson, E., Horekens, J., Mander, N., Conoir, Y., Arsenault, M., Wallis, T. and Cherkaoui, N. (2013) The Global Evaluation of Emergency Response Funds (ERFs), March 2013, UN OCHA, New York, p17.

15 Channel Research (2011) 5-Year Evaluation of the Central Emergency Response Fund, Final Synthe-sis Report, 11 August 2011, p66.16 Miller, P. & Ullah Khan, N. (2014), Report of Evaluation of RAPID Fund I, p20.

CRITERIA START ERF CHF CERF RAPID CHAF DEC DFID RRF

tyPes of emergencies

new rapid & slow onset crises, spikes in chronic crises, under-funded emergencies

small/ medium crises, early response and spikes in chronic crises

unforeseen needs and gaps in humanitarian response

Priority life-saving needs in protracted crises

large-scale rapid onset crises or spikes in chronic crises

urgent relief and early recovery needs

small & under-funded rapid onset, spikes in chronic crises, gaps in response

large-scale disasters with media visibility

countries of oPeration

global (45 countries in 2013)

global country-based (13 countries)

country-based (6 countries)

globalPakistan global global (2 appeals in 2013)

mobilised for emergencies Which received start funding

ncameroon, myanmar, nepal, nigeria, s. leone, s. sudan, somalia, yemen

13 activations in 12 countries

myanmar, yemen

somalia, south sudan

sierra leone, south sudan

no bdesh, cameroon, nepal, turkey

sierra leone ebola response

timelines3.5 - 7 weeks for first submission to disbursement (ufe & rr window)15

funding awarded in 72 hours and project start in 7-days

4 - 8 weeks from identification of problem to project approval14

tbc (global evaluation currently in process)

funding should be awarded within 72 hours

10 - 30 days for approval of proposals16

activation phase should not exceed 13 days

dec Phase 1 plans have to be submitted a month from the date of the appeal

size of fund $421m in 2014 and $478m in 2013

2.25m for 9 month pilot phase

usually less than $4m p.a. although may exceed this during a crisis

usually $60-120m p.a. although less in 2014

not specified2009 - 2013 $29m

can$2m from dfat-d for 18 month pilot

depends on public response to appeal. 27m for sierra leone ebola (as of 27/11/2014) response

decision - maKing Processes

united nations country team or hct with support from clusters

membership-based peer review for allocations and project selection

hct with support from clusters

hct with support from clusters

dfidcountry-based by concern Worldwide on behalf of ofda

dfatd in coordination with the humanitarian coalition

dec trustees supported by the secretariat.

eligible reciPients

un agencies and iom

ingos mainly ingo and nngo although un is eligible for some erfs

un, ingo and nngo

implementing partners (ingos)

ingo and nngo

humanitarian coalition members (ingos)

13 ingo members

fund management

ocha- cerf secretariat on behalf of erc

ngo member ocha on behalf of the hc

ocha on behalf of the hc (undP finance mgt)

dfidconcern Worldwide on behalf of ofda

humanitarian coalition secretariat in coordination with dfat-d

dec secretariat on behalf of dec board

fund mobilisation

annual reporting includes funds leveraged

funds leverage an explicit KPi


3 | relevaNce

3.1 | classifying crises - understanding the role of the start fund in resPonding to crises

In the first nine months a Start Fund Alert has been raised on 19 occasions, and the Fund has been activated on 12 occasions. During the design & build phase, a policy of piloting different response types was pursued in order to better understand its appropriateness to different types of crises. These response types included the following:l rapid onset natural or man-made disasters;l early responses to slow onset or cyclical disasters;l spikes in chronic disasters.

It is also important to note the type of crisis for which the Start Fund has not been activated which includes large scale natural and man-made disasters for which other funding mechanisms, including DFIDs Rapid Response Facility (RRF) or the UN CERF, are considered to be more appropriate.

While the design and build phase has begun to provide an evidence base for the types of responses that are considered the best fit for the Start Fund, because trialling diverse responses was an anticipated outcome of the pilot process, it meant that the AC proactively sought diverse contexts (see figure 5 below).

f i g u r e 5

aNalySiS of STarT fuNd alerTS aNd acTivaTioNS

alerT: Floods in Afghanistan, Internally Displaced Person (IDP) crisis in Pakistan (Alert.PAK2), earthquake in China, floods in Bangladesh, floods in Nepal, Syrian refugees in Turkey, cyclone in India and floods in Somalia fuNded: Floods in Bangladesh, Nepal and Somalia, and Syrian refugees in Turkey

raPid onset disasters (natural & man-made)

sloW onset/food insecurity crises

anticiPatory resPonses to natural and man-made disasters

sPiKes in chronic crises*

sPiKes in ongoing crises*

CHARACTERISTIC # ALERT # FUNDED NOTES

alerT: All for food insecurity in South Sudan, Somalia and Kenya fuNded: Kenya not funded because the Start Fund would have a limited impact & crisis already receiving donor attention

alerT: South Sudan (in advance of rains and funding), Somalia (in advance of peak hunger and funding), Kenya (in advance of peak hunger), Sierra Leone (in advance of the spread of Ebola and significant funding), India (in advance of cyclone Hudhud making landfall) fuNded: Kenya not funded (see above) and India not funded due to an uncertainty about the potential needs

alerT: IDPs in Myanmar, Pakistan (Alert.Pak1) and Democratic Republic of the Congo (DRC) (Alert.DRC1 & Alert.DRC2) fuNded: Myanmar and DRC (Alert.DRC2)

alerT: IDPs in Yemen, cholera in Cameroon and Nigeria fuNded: All alerts received funding

alerT: Floods in Afghanistan, earthquake in China, floods in Bangladesh, floods in Nepal, cyclone in India, floods in Somalia fuNded: Floods in Bangladesh, Nepal & Somalia. Did not fund Afghanistan because it was considered to be a cyclical crisis, China because government response sufficient & cyclone in India because of uncertainty about potential needs

alerT: IDPs in Myanmar, Pakistan (Alert.PAK1 & Alert.PAK2), Yemen, DRC (Alert.DRC1 & Alert.DRC2), Syria refugees in Turkey, Ebola in Sierra Leone & cholera in Cameroon & Nigeria fuNded: Myanmar, Ebola in Sierra Leone, Yemen, cholera in Cameroon & Nigeria, Syria IDPs in Turkey, DRC (Alert.DRC2)

natural disasters

man-made disasters (conflict, disease)

*crises ongoing for > 24 months

8 4

3 2

5 3

4 2

3 3

6 3

10 7

Drawing on an analysis of the Start Funds activations in the first nine months, this section seeks to determine its relevance to supporting the humanitarian response of its members and assesses its added value and complementarity with other funds that are part of the broader humanitarian financing landscape.


A review of the AC meeting minutes for each of the activations reveals the greatest level of consensus for response to rapid onset crises, particularly to natural disasters. Where there was greatest uncertainty in allocating funding was in response to slow onset/food insecurity crises where the members off the AC tended to be divided over the added value of the small grant size and the short intervention period.

Responses to crises that can be classified as anticipatory have also tended to elicit a mixed response with three such responses funded and two not receiving funding. While it is difficult to clearly determine the reason for this, a lack of a shared analysis about the alert (Kenya), disagreement about the relevance of the Fund to the context (Somalia although an activation did follow) and uncertainties about whether funding for a humanitarian response would be required (India) all played a part in complicating decision-making about the respective allocations. However, for the pre-positioning of relief items in advance of the rains in South Sudan, and support for the Ebola outbreak in Sierra Leone in advance of the global focus on addressing the crisis, there was far greater consensus on the relevance of the Start Fund.

One type of emergency for which the Fund has not been activated has been for small emergencies that happen at a local-level such as mud-slides, fires or building collapse. These types of emergencies rarely capture the attention of the media

but may cumulatively account for significant suffering and loss of life. Typically they require a presence on the ground and an ability to move swiftly which is most suited to national partner response rather than international organisations. Given the mandate of the Start Fund and the potential that exists for it to be activated quickly, it could play a role in responding to these crises, but it has not been called on to do so. It is possible that with the support from the capacity building component of the Start Network, Start Build, and as members partner agencies become more familiar with the Fund (or become members of the Network themselves), that these sorts of responses may become more viable. as the fund is replenished in 2015 activations in support of a small-scale emergency would assist in building the evidence base for the relevance of the fund for small-scale emergencies.17

analysing the evidence base for decision-maKing and strengthening anticiPationAs it grows in size, greater importance will be attached to the Start Fund being able to provide robust evidence of high quality needs-based decision-making. This is currently achieved through the analysis of its members and the external inputs received by ACAPS and DI, which provide objective analysis to support decisions about the activation of the Fund. Interviews suggest that these tools provide high quality information which was considered by members to support objective analysis.

f i g u r e 6

compariSoN of STarT fuNd alerTS wiTh global vulNerabiliTy iNdiceS aNd acapS briefiNg NoTe guidaNce

17 The Colombia activation which occurred shortly after the evaluation took place was in response to a small-scale emergency and will start to fill the gap in evidence.

START FUND ALERT ACAPS START FUND BRIEFING DECISION TAKEN(in date order) NOTE CLASSIFICATION By THE AC

NEED FOR ASSISTANCE (0 LOW, 4 HIGH)

ExPECTED IMPACT (0 LOW, 4 HIGH)

TOTAL (NEED + IMPACT SCORE)

south sudan 0 1 2 3 4 0 1 2 3 4 7 activated myanmar 0 1 2 3 4 0 1 2 3 4 4 activated afghanistan 0 1 2 3 4 0 1 2 3 4 5 not activated PaKistan 0 1 2 3 4 0 1 2 3 4 not activated somalia 0 1 2 3 4 0 1 2 3 4 5 activated sierra leone 0 1 2 3 4 0 1 2 3 4 6 activated PaKistan 0 1 2 3 4 0 1 2 3 4 5 not activated yemen 0 1 2 3 4 0 1 2 3 4 6 activated cameroon 0 1 2 3 4 0 1 2 3 4 4 activated bangladesh 0 1 2 3 4 0 1 2 3 4 4 activated drc 0 1 2 3 4 0 1 2 3 4 6 not activated china 0 1 2 3 4 0 1 2 3 4 3 not activated nePal 0 1 2 3 4 0 1 2 3 4 3 activated turKey 0 1 2 3 4 0 1 2 3 4 6 activated india 0 1 2 3 4 0 1 2 3 4 3 not activated Kenya 0 1 2 3 4 0 1 2 3 4 4 not activated drc 0 1 2 3 4 0 1 2 3 4 6 activated nigeria 0 1 2 3 4 0 1 2 3 4 6 activated somalia 0 1 2 3 4 0 1 2 3 4 4 activated

AC DECISION ON FUND ACTIvATION


A review of the ratings given in the ACAPS briefing notes for international assistance and expected impact (see figure 6) highlights some disparities between the ACAPS steer and the decisions made by the AC. While the ACAPS classification and the decision taken by the AC for the China and India alert were consistent, some of the alerts which were rated most highly by ACAPS, were rejected by the AC (e.g. for Afghanistan, Pakistan and DRC). While such inconsistencies are not surprising as the decisions made by the AC also draw from its members operational presence in the crisis-affected area as well as the DI funding analysis, there would be value in the Start Team documenting the reasons for the inconsistencies.

An area where the Start Fund is seeking to strengthen its analysis is in anticipating countries and/or crises that are most likely to require a Start Fund response. Anticipation offers the obvious benefits of giving greater time for organisations to refine their analysis and prepare to respond. One means of strengthening anticipation is the IASC Early Warning Early Action group which collaborates to produce bi-annual Early Warning briefing notes. The briefing notes are primarily aimed at the IASC Emergency Directors Group and the CERF secretariat to influence the triggering of collective resources for early action. anticipation is an area that the start fund has the potential to strengthen given the operational reach of its 19-members and the resources that already exist in the fund. by participating in the early Warning early action group and by integrating its analysis into its decision-making processes, the fund could improve the potential for early action as well as strengthening its complementarity with the cerf.

3.2 | an analysis of the comPlementarity of the start fund With other humanitarian funds

The growth in global humanitarian need has placed humanitarian financing mechanisms under significant strain. While the increase in the scale and diversity of pooled funds has gone some way to strengthening both the timeliness and needs-base of humanitarian response, there remain important gaps in the financing architecture. Two of the most important of these are in the provision of timely funding immediately after a crisis and the provision of funding directly to local and national NGOs, which often provide the majority of operational capacity in the first days and weeks of humanitarian response. From the perspective of those affected by crises, these two gaps are among the most important to fill.

At present, with its membership restricted to 19 INGOs, the key contribution of the Start Fund is in promoting early response to humanitarian crises with its 72 hour activation, an unprecedented speed for a pooled fund. While at this time it does not directly fund national or local NGOs, it does pass funding through to these organisations within the same timeframe and requires project commencement within seven days of award. Given the

swift activation of the fund, it offers a good fit with some of the UN pooled funds (ERF, CHF and CERF), which take longer to activate but which often disburse larger funding envelopes to a broader range of organisations over longer periods of time. The focus of the Start Fund on small-to-medium humanitarian emergencies also potentially positions it as a good fit with funds which focus on larger emergencies such as the CERF and DFIDs RRF. In order to assess the extent to which the Start Fund complements other funds, an analysis was undertaken of four emergencies, Cameroon, Nepal, South Sudan and yemen, that are broadly representative of the geographic and humanitarian profile of the Start Fund activations (figure 7).

Assuming that the reported data is correct, for the four emergencies analysed, the Start Fund performed well both in terms of the timeliness of its response to the crisis event and its response time in comparison to other funds.

l In response to flooding in Nepal, the fund was activated within eight days and;l For the upsurge in the conflict in Yemen, activation occurred within two weeks; l The cholera in Cameroon reached epidemic proportions in May but was considered to have gone beyond the capacity of the national health authority in early July with Start Fund activation occurring at the end of the month in anticipation of a further escalation due to the rains;l The South Sudan activation occurred in advance of the rains which start in June, two months after the activation occurred.

While the uneven use of FTS by donors to report humanitarian funding can offer only an incomplete picture of what funding was received for each of the four emergencies, the information available suggests that the Start Fund is a potentially good fit with the mosaic of existing instruments largely on account of its speed of activation. In each of the four crises, it complemented funding from at least one other UN pooled fund (albeit disbursed to different organisations and often in different locations) with bilateral, multilateral donors and foundations offering larger grants but often arriving later than the Start Fund allocations were made.

It is important to acknowledge that its comparatively small size means that the Start Fund has not been a significant partner to most of the other funds described, but the evidence suggests that in targeting emergencies that are often not so visible or by seeking to respond in a timely way to spikes in crises, the Start Fund has found a relevant niche. The South Sudan and Cameroon activations also provide evidence of the value of the fund when triggered in anticipation of an increase in humanitarian need. The analysis of pooled funds and donor responses to the four crises provides evidence not only of the agility of the Fund, but also its prominent position in supporting initial response and in so doing complementing existing humanitarian funding instruments.

The pooled fund that has most in common with the Start Fund is the CHAF which was activated in two of the four crises (and which


18 The Response Plan for each crisis articulates humanitarian needs that are far broader than the specific subset that the Start Fund was targeting, but the figures do provide an overall view of the proportion of humanitarian needs that received funding.19 OCHA FTS, 11/12/2014. 20 OCHAs Financial Tracking Service (FTS) records all reported humanitarian aid contributions

with a special focus on strategic response plans and appeals. See http://fts.unocha.org/pageloader.aspx?page=home 21 Despite the limited likelihood of double funding, it is noteworthy that Plan International received funding from both the Start Fund and the CHAF in Cameroon, however, they were earmarked for different crises.

f i g u r e 7

aNalySiS of humaNiTariaN fuNdiNg To four criSeS which received STarT fuNdiNg iN 2014

in total has been activated in six of the Start fund activations and also in one crisis where the fund was alerted but not activated). While the focus of the CHAF on funding a single NGO member for each of its activations means that the likelihood of overlap with the Start Fund is slim21 the similarities between the chaf and the start fund suggests that there would be added value

in establishing more formal coordination mechanisms between the two funds which should include at a minimum, notification of alerts and activations. With the possibility of similar funds being established in the future, and given the emphasis placed by the start fund on collaboration, there is scope for the start team to role-model this.

requested: $126m received: $55m % funded: 43

2014 resPonse Plan funding18,19

start fund (Eligibility: 19 INGOs)

erf (Eligibility: INGO, NNGO, sometimes UN)

cerf (Eligibility: UN)

chaf

FUND/COUNTRy CAMEROON NEPAL SOUTH SUDAN yEMEN

No response plan requested: $658 received: $322 % funded: 49

requested: $596 received: $323 % funded: 54

Cholera outbreak in nine health districts since 18 March but officially declared an epidemic in May. A spike in need occurred in early July. allocation decision: 30 July (140k)

Localized flash floods affecting a total of 28,656 families in six districts since 1 Sept. allocation decision: 9 Sept (100k)

Acute food insecurity in South Sudan and the need to preposition relief supplies before the rains in June. allocation decision: 2 April (664k)

Spike in fighting in Amran, North Yemen, creating a need for affected communities since 4th July. allocation decision: 15 July (156k)

No ERF No ERF No ERF ERF allocations in July including in Sanaa, which was the focus for Start Funding ($1.3m)

chf (Eligibility: INGO, NNGO and UN)

No CHF No CHF CHF funding released in Jan, March, May & July. No funding to Start members ($7.5m)

No CHF

Cameroon selected for UFE window because Strategic Response Plan 10% funded & unmet needs of refugees. Project approvals in September and October ($4.5m)

RR window allocation in response to the floods. Approval dates in October ($1.8m)

RR window allocation which focused on funding for nutrition and health interventions. Project approvals in April ($14.9m)

Allocation from first round of funding from UFE window to 8 UN agencies so not directly related to the crisis which received Start Funding ($13.9m)

Allocation for Child Health & Nutrition Response for CAR Refugees in East Cameroon to Plan International in July (CAD$402k)

WASH & Emergency Food Security & Livelihoods intervention to Oxfam in September (CAD$300k)

No CHAF funding No CHAF funding

additional ProJect funding recorded on fts20

Gates Foundation, August ($900k)

Danish funding, end-August ($107k) ECHO funding, end-August and early-Sep ($329k) Irish Aid funding, early September ($33k) Gates Foundation funding, early-Sep ($400k) Danish funding, early-Sep ($52k) Swedish funding, early-Sep ($172k) Luxembourg funding, mid-Oct ($142k)

OFDA funding, March ($801k)Private funding, mid-March ($102k) Finland funding, mid-March ($32k)Czech funding, mid-March ($100k)DFID funding, mid-March ($13.8m)Norwegian funding, end-March ($1.96m)OFDA funding, April ($5+4+5m) Denmark funding, May ($6.4m)

DFID funding, early June ($1.9m) ECHO funding, mid-June ($4.08m) US funding, end-June ($727k + $361k) US funding, end-June ($1.16m) US funding, end-June ($700k) DFID funding, early-July ($54k) DFID funding, early-July ($4.55m) DFID funding, early-July ($913k) US funding, end-July ($24.1m)


4 | effecTiveNeSS

This section seeks to assess the effectiveness and performance of the Start Fund in its first nine months and focuses on process management, information, coordination, quality assurance, fund leveraging and learning.

f i g u r e 8

explaNaTory NoTe oN The efficieNcy criTerioN

efficiency measures how economically inputs (usually financial, human, technical and material resources) were converted to outputs. assessment of efficiency tends to start with financial data and usually involves comparisons with alternatives.22

In seeking to determine the efficiency of the Start Fund in converting inputs (funding) to outputs (attainment of KPIs and project results but also less tangible outcomes including progress made towards the vision), the evaluation has failed to find comparable data sets. While the financial data to support comparisons is available by the Start Fund, the same data is not available has not been made available in other pooled fund evaluations (RAPID, ERF) which precludes comparisons. A second more complex challenge is in finding a suitable comparator for the Start Fund as there is no single fund that is a good fit. Of the potential comparators, some have far larger budgets (CERF or CHFs), are country-based rather than global (RAPID, ERF, CHF), or target a more diverse group of members including perceived higher risk organisations (RAPID, CHF and ERF). While paucity of financial data for pooled funds means that quantitative comparisons are not possible, there are a number of other metrics and qualitative data that can be compared. In order to provide some analysis of how the Start Fund compares with other pooled funds, the evaluation will analyse a range of humanitarian funds against a set of performance criteria. It will also highlight efforts that have been taken by the Start Fund and its members to use the resources it has in the most efficient way (functioning in the best possible manner with the least waste of time and effort) and make recommendations about how these efforts can be strengthened in the future.

4.1 | analysing Performance metrics and ProJect results

an assessment of Key Performance indicatorsUsing the project report forms, the Start Team has maintained a database which has tracked each of its key performance metrics over the nine month life of the Start Fund. An analysis of this data is provided below (see figure 9).

The most significant finding from the analysis is that with only a small number of minor exceptions the Start Fund has consistently met or exceeded its KPIs. For a fund that has only been established for nine months and which has set itself an ambitious target for its activation that exceeds the aspirations of any other pooled fund, the Start Team and members have performed well in consistently meeting their performance targets. The agility of the Start Fund in responding to crises has filled an

important gap in the humanitarian financing landscape that many would have considered beyond the reaches of a membership-based, peer-led, global, pooled fund. That this has been achieved from the launch of the fund, when it was little known outside of member headquarters, is all the more impressive. While some members admitted that the swift activation strained some of their systems, they viewed this tension positively and felt that the Fund had spurred them on to respond more quickly than they would have otherwise done.

The quality of the data that has been collected on the Start Funds metrics is worthy of note and suggests that the resourcing of performance monitoring has been a wise investment. Knowing that timely response to humanitarian needs can play a significant role in saving lives and supporting livelihoods, the evidence suggests that the Start Fund is well-positioned to have impact.

22 Beck T (2006) Evaluating humanitarian action using the OECD-DAC criteria: An ALNAP guide for humanitarian agencies, ALNAP, p44.


23 This does not include a one-week deferment period.

f i g u r e 9

aNalySiS of STarT fuNd key performaNce iNdicaTorS

COUNTRy TyPE OF TyPE OF ALERT TO TOTAL ACTIvATION TO APPLICATION ALERT TO AWARD TO PROjECT ALERT CRISIS AC MTG FUNDS APPLICATION TO PCS AWARD START-UP COMPLETED (hrs.) ALLOCATED DEADLINE (hrs.) (hrs.) (days) (days)

Slow onset

Early response to slow onset crisis

22.83 425k 24 21 74.5 ACF: 0 Christian Aid: 7 Tearfund: 3

45southsudan

Slow onset

Spike in chronic crisis

22.66 200k 24 4.25 54.5 Christian Aid: 5 45myanmar

Slow onset

Early response to slow onset crisis

24.25 500k 24 20 71.5 IRC: 5Muslim Aid: 1

45somalia

Rapid onset

Rapid onset crisis

25.75 350k 24 18 69 ActionAid: 2 Christian Aid: 6 Concern: 1 Save: 0

45sierra leone

Slow onset


24.83 200k 24 16.5 67 IMC: 3 Save: 7

45yemen

Slow onset


24.33 140k 24 19 70.5 IMC: 1,Plan: 1

45cameroon

Rapidonset

Rapid onset crisis

22.3323 200k 24 26.66 65.5 CARE: 3Christian Aid: 3,Islamic Relief: 5

45bangladesh

Rapid onset

Rapid onset crisis

23.75 100k 24 20.25 71 Handicap: 3Plan: 4

45nePal

Rapidonset

Rapid onset crisis

24.66 200k 24 22 70 CAFOD: 5, Concern: 4

45turKey

Slow onset


24 200k 24 21 70 TBC 45dem. reP. congo

Rapid onset

Rapid onset crisis

24 200k 24 21 70 TBC TBCsomalia

Rapid onset

Spike in chronic crisiss

24 125k 24 24 72 TBC TBCnigeria

TARGET 24 24 24 72 7 45

AvERAGE 24.09 236K 24 19.47 68.79 45


analysis of ProJect resultsWhile an assessment of project outputs and outcomes is outside of the scope of the evaluation, project monitoring provides a breakdown of the numbers of people reached and the sectors of intervention from the Start Funds 29 projects (see figure 10). As the Start Fund is not linked to Clusters in the same way that the UN-managed pooled funds are, it is not possible to determine the extent to which the sectors that have been prioritised by the Start Fund members for a given emergency are consistent with those that have been prioritised by the humanitarian coordination architecture.

in the feedback to the draft evaluation report, there was some concern that the evaluation did not deal in any detail with project outputs and outcomes. While this was not part of the tor and hence was outside the scope of the exercise, it will be important in the future to address this gap. experience in trying to evaluate the impact of other pooled funds has highlighted the complexities of this task as pooled funding is generally co-mingled with other funding which makes it difficult to attribute change to a particular source. given the 45-day activation period for the start fund, the challenges of attribution will be even more pronounced. however, by commissioning a field-focused evaluation which prioritises project review and interviews with field staff, partners and crisis-affected communities, it should be possible to build up a more detailed picture of changes that can be attributed to the disbursement of timely start funding. furthermore, if a range of different activations were selected (e.g. rapid/slow onset/spikes etc.), then it may be possible to strengthen the evidence base for the types of activation where the fund can have greatest effect.

identifying good Practice in data disaggregationThe Start Fund has established a reporting mechanism that disaggregates beneficiary numbers by gender and by age group (under 18, 18-49 years, and over 50 years) which should be considered good practice. There was some concern raised about the viability of routinely collecting this data which in some cases required an additional investment in time and resources. By routinely disaggregating data in this way, there is significant potential to strengthen collective understanding of the differential impact of crises on peoples lives. There have been requests to strengthen this by including an analysis of disability in both the project application and report form and in making a decision on this it will be important to strike a balance between what is required to strengthen needs-based decision-making and what is feasible in the context of responding rapidly to crises.

an analysis of funding continuity and leveragingA critical question for the viability of the Start Fund model is the extent to which its rapid activation and execution fits with follow-on funding from other donors. While it is difficult to determine this with any degree of rigour due to the vagaries of donor reporting and limitations in the systems that are used to collect funding data, information collected during the fieldwork suggests a mixed picture.

l The response scenario that appears to best support funding continuity is rapid onset interventions where a 45 day injection of funding was often followed by funding from other pooled funds, bilateral or multilateral donors. As an example, for the Bangladesh activation, all of the agencies that were Start-funded received follow-on funding from either ECHO or DFID within the 45 day activation period (see figure 11) although it is important to note that many other organisations that were not Start Funded received similar funding from the same donors.l A more problematic example was the Somalia activation in which one member received no follow-on funding and so the intervention ceased after 45 days while the second received funding significantly later than the 45-day period. In the context of a protracted slow onset crisis, such a short timeframe is not an effective use of resources.

f i g u r e 1 0

STarT fuNd projecTS fuNded, people reached aNd SecTorS of iNTerveNTioN24

SECtorS of intErvEntion

# of projects with activities covering the given sector

16 8 5 11WaSh Shelter nutrition health

4protection

11fSl

3Education

pEoplE rEaChEd

MyanmarSomaliaSierra Leone

yemenCameroon

29 projECtS

ExpECtEd

to datE

4

2

2

1

2

34,377 2,176 27,829 596,405 37,482 456,29770,792 5,79413,152

59,25215,20336,354 602,84763,992513,947172,244 7,96713,152

422

South Sudan

Individuals (adjusted total)

1,320,017

activities(cumulative total)

1,648,781

4

BangladeshnepalTurkey

DrCnigeriaSomalia

10,65535,09829,950

27,25561,76874,800

12

3

24 Adapted from Start Fund (2014) The Start Fund story after 9 months, the Start Network p3.


one of the metrics routinely collected in the project report form is the quantity of funding that has been leveraged which is defined as funding that the member receiving during or just after the 45 day project. feedback collected during the evaluation suggests that donor funding is disbursed for numerous reasons which includes historical organisational performance, participation in preferred humanitarian consortia and the quality of donor proposals in addition to the prior presence of an organisation in the affected area which may be attributed to the start fund. as a result of this there is a likelihood of considerable over-reporting against this KPi. With a view to strengthening the evidence base for the 45 day intervention period and given the problems associated with the existing approach to gathering data on fund leveraging, there would be value in routinely asking members whether there was follow-on

funding for the specific start-funded intervention (rather than a more general question about fund leveraging).

4.2 | analysis of Process effectiveness

The process of raising alerts, activating the fund, allocating funding, selecting projects and reporting on results has benefitted considerably from the lessons learned during the management of the Emergency Response Fund by the Consortium of British Humanitarian Agencies (CBHA), which was the precursor to the Start Fund. While the systems have since evolved, the evaluation received considerable feedback about issues related to effectiveness. Each of the steps outlined in figure 12 will be reviewed, and recommendations will be made where there is scope to strengthen them.

f i g u r e 1 1

fuNd leveragiNg by STarT-fuNded memberS iN The baNgladeSh acTivaTioN

17 Evaluation of the Start Fund design & build phase

A critical question for the viability of the Start Fund model is the extent to which its rapid activation and execution fits with follow-on funding from other donors. While it is difficult to determine this with any degree of rigour due to the vagaries of donor reporting and limitations in the systems that are used to collect funding data, information collected during the fieldwork suggests a mixed picture. The response scenario that appears to best support funding continuity is rapid onset

interventions where a 45 day injection of funding was often followed by funding from other pooled funds, bilateral or multilateral donors. As an example, for the Bangladesh activation, all of the agencies that were Start-funded received follow-on funding from either ECHO or DFID within the 45 day activation period (see figure 12) although it is important to note that many other organisations that were not Start Funded received similar funding from the same donors.

A more problematic example was the Somalia activation in which one member received no follow-on funding and so the intervention ceased after 45 days while the second received funding significantly later than the 45-day period. In the context of a protracted slow onset crisis, such a short timeframe is not an effective use of resources.

One of the metrics routinely collected in the project report form is the quantity of funding that has been leveraged which is defined as funding that the member receiving during or just after the 45 day project. Feedback collected during the evaluation suggests that donor funding is disbursed for numerous reasons which includes historical organisational performance, participation in preferred humanitarian consortia and the quality of donor proposals in addition to the prior presence of an organisation in the affected area which may be attributed to the Start Fund. As a result of this there is a likelihood of considerable over-reporting against this KPI. With a view to strengthening the evidence base for the 45 day intervention period and given the problems associated with the existing approach to gathering data on fund leveraging, there would be value in routinely asking members whether there was follow-on funding for the specific Start-funded intervention (rather than a more general question about fund leveraging). Figure 12: Fund leveraging by Start-funded members in the Bangladesh activation

4.2 Analysis of process effectiveness The process of raising alerts, activating the fund, allocating funding, selecting projects and reporting on results has benefitted considerably from the lessons learned during the management of the Emergency Response Fund by the Consortium of British Humanitarian Agencies (CBHA), which was

dayS afTer baNgladeSh floodS commeNced

hu

ma

NiT

ar

iaN

Need

f i g u r e 1 2

STarT fuNd Timeframe diagram aNd proceSS25

18 Evaluation of the Start Fund design & build phase

the precursor to the Start Fund. While the systems have since evolved, the evaluation received considerable feedback about issues related to effectiveness. Each of the steps outlined in figure 13 will be reviewed, and recommendations will be made where there is scope to strengthen them. Figure 13: Start Fund timeframe diagram and process25

Step 1: Crisis alert

In the event of a crisis, the Start Network member agencies report a crisis to the Start Team. This is done by sending an alert note. The Start Team requests an assessment of the crisis from ACAPs and a funding analysis from Development Initiatives. The members inform the Start Team within 24 hours of the alert whether they would apply for funding should the Start Fund be activated. If at least 1 agency is potentially able to carry out a response and fewer than half of the membership objects to a telecon, the Start Team organises a teleconference (or face-to-face meeting) with the AC, and shares the ACAPS assessment with them.

Alerts have been raised in a variety of different forms by members independently and collectively. While there is no obligation for a member to inform others about an alert, given the need for a majority of AC members to activate the Fund, collaboration in raising alerts can be helpful in strengthening analysis and in anticipating crises and prompting early action. The use of an online platform for analysis and alerts: There is interest from within the membership to strengthen collective anticipation by the use of an online platform to share analysis. A secondary benefit of such a system would be to offer an opportunity for members who work exclusively through partners and do not have a presence in-country to participate in raising an alert. This issue was raised by a member who was concerned that the lack of an operational presence meant that it missed the opportunity of alerting its partners and may have played a part in its failure to receive funding. An online platform would also permit the Start Team to monitor up-and-coming alerts which would assist with anticipating resource needs are in place to manage and facilitate the Fund activation processes. While members should retain the right to raise an alert without consultation, investment in tools to strengthen collaboration in early warning and response have much to offer the Start Fund, particularly as it seeks to expand its membership. The establishment of an online platform by the Start Fund would offer the benefits of pooling information and analysis, notifying members of the potential for an alert and strengthening anticipation and is strongly recommended. Strengthening decision-making for the delegation of the allocation process to the Start Team: Once an alert note has been circulated, members are requested to fill in a questionnaire to elicit additional information about the crisis, to canvas opinions about whether to activate the Start Fund and to inform a decision about whether the Start Team oversees the process on behalf of the AC. It is anticipated that the Start Team may oversee smaller activations and during the pilot phase this occurred on one occasion. 25 Adapted from Start Network (2014) Start Fund Handbook, p9. The diagram is replicated in full in annex 4.

25 Adapted from Start Network (2014) Start Fund Handbook, p9. The diagram is replicated in full in annex 4.


STep 1: criSiS alerT

In the event of a crisis, the Start Network member agencies report a crisis to the Start Team. This is done by sending an alert note. The Start Team requests an assessment of the crisis from ACAPs and a funding analysis from Development Initiatives. The members inform the Start Team within 24 hours of the alert whether they would apply for funding should the Start Fund be activated. If at least 1 agency is potentially able to carry out a response and fewer than half of the membership objects to a telecon, the Start Team organises a teleconference (or face-to-face meeting) with the AC, and shares the ACAPS assessment with them.

Alerts have been raised in a variety of different forms by members independently and collectively. While there is no obligation for a member to inform others about an alert, given the need for a majority of AC members to activate the Fund, collaboration in raising alerts can be helpful in strengthening analysis and in anticipating crises and prompting early action.

the use of an online Platform for analysis & alerts: There is interest from within the membership to strengthen collective anticipation by the use of an online platform to share analysis. A secondary benefit of such a system would be to offer an opportunity for members who work exclusively through partners and do not have a presence in-country to participate in raising an alert. This issue was raised by a member who was concerned that the lack of an operational presence meant that it missed the opportunity of alerting its partners and may have played a part in its failure to receive funding. An online platform would

evaluation of the start fund design & build phase

Documents

start fund alert

fund data

ta s n e e

fund management processes

fund design build phase

h e r s t o n e f u

d o n o r s pa r t n

n dy f e