world bank document · 2016. 7. 12. · report no: icr2517 . implementation completion and results...

Document of The World Bank

Report No: ICR2517

IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF093358)

ON

A GRANT

IN THE AMOUNT OF US$ 50.0 MILLION

TO THE

REPUBLIC OF INDONESIA

FOR THE

ACEH – ECONOMIC DEVELOPMENT FINANCING FACILITY PROJECT

May 28, 2013

Financial and Private Sector Development Indonesia Country Department East Asia and Pacific Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective May 1, 2013)

Currency Unit = Indonesian Rupiah (IDR or Rp) US$ 1.00 = IDR 9,722.50

FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS AAA Actionaid Australia ADF Aceh Development Fund BAPPEDA Badan Perencanaan Pembangunan Daerah (of Government of Aceh) BAPPENAS State Ministry for Development and Planning (of GOI) BPKP Financial and Development Supervisory Board (Badan Pengawasan Keuangan

dan Pembangunan CCA Canadian Cooperative Association CCR Caritas Czech Republic DIPA Daftar Isian Pelaksanaan Anggaran or Government of Indonesia annual budget

allocation document EDFF Economic Development Financing Facility (The Project) FMS Financial Management Specialist GOI Government of Indonesia IFR Interim Financial Report IOM International Organization for Migration IRW Islamic Relief Worldwide KPDT Ministry for Disadvantaged Areas (of GOI) KPI Key Performance Indicator MDF Multi-Donor Trust Fund for Aceh and North Sumatra MIA Muslim Aid Indonesia MOF Ministry of Finance MoU Memorandum of Understanding NCB National Competitive Bidding NGO Non-Governmental Organization PMC Project Management Consultant PMU Project Management Unit RPJMD Medium Term Development Plan of the Government of Aceh Satker Budget Executing Agency SA SIE

Special Account Sub-project Implementing Entity

UNDP United Nations Development Program

Vice President: Axel van Trotsenburg Country Director: Stefan Koeberle

Sector Director: T. Tunc Uyanik Project Team Leader: Alexandra Drees-Gross

ICR Team Leader: Nancy Chen

COUNTRY: INDONESIA

Aceh – Economic Development Financing Facility Project

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 5 3. Assessment of Outcomes .......................................................................................... 13 4. Assessment of Risk to Development Outcome ......................................................... 19 5. Assessment of Bank and Borrower Performance ..................................................... 20 6. Lessons Learned ....................................................................................................... 23 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 26 Annex 1. Project Costs and Financing .......................................................................... 27 Annex 2. Outputs by Component ................................................................................. 28 Annex 3. Economic and Financial Analysis ................................................................. 32 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 34 Annex 5. Beneficiary Survey Results ........................................................................... 36 Annex 6. Stakeholder Workshop Report and Results ................................................... 37 Annex 7. Summary of Borrower’s ICR and/or Comments on Draft ICR .................... 38 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 42 Annex 9. List of Supporting Documents ...................................................................... 45 Annex 10. Analysis of EDFF Results ........................................................................... 46 Annex 11. Summary Performance of SIEs, by SIE ...................................................... 49 Annex 12. Analysis of Critical Risks at Appraisal and Risk Outcomes ....................... 53

MAP …………………………………………………………………………………56

A. Basic Information

Country: Indonesia Project Name: Aceh Economic Development Financing Facility

Project ID: P109024 L/C/TF Number(s): TF-93358 ICR Date: 05/01/2013 ICR Type: Core ICR

Lending Instrument: SIL Grantee: REPUBLIC OF INDONESIA

Original Total Commitment:

USD 50.00M Disbursed Amount: USD 44.46M

Revised Amount: USD 50.00M Environmental Category: B Implementing Agencies: Ministry for The Development of Disadvantaged Areas (KPDT) Government of Aceh Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 09/07/2007 Effectiveness: 03/30/2009

Appraisal: 05/15/2008 Restructuring(s): 12/15/2011 06/26/2012

Approval: 12/16/2008 Mid-term Review: 03/15/2011 Closing: 06/30/2012 11/30/2012 C. Ratings Summary

C.1 Performance Rating by ICR

Outcomes: Moderately Unsatisfactory Risk to Development Outcome: High Bank Performance: Moderately Unsatisfactory Grantee Performance: Moderately Unsatisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)

Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory Government: Moderately Satisfactory

Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies:

Moderately Unsatisfactory

Overall Bank

Performance: Moderately Unsatisfactory

Overall Borrower

Performance: Moderately Unsatisfactory

C.3 Quality at Entry and Implementation Performance Indicators Implementation

Performance Indicators

QAG Assessments

(if any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Satisfactory

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing) Animal production 25 25 General agriculture, fishing and forestry sector 45 45 General industry and trade sector 20 20 Public administration- Agriculture, fishing and forestry 5 5 Public administration- Industry and trade 5 5

Theme Code (as % of total Bank financing) Export development and competitiveness 20 20 Other financial and private sector development 20 20 Rural markets 20 20 Rural non-farm income generation 20 20 Rural services and infrastructure 20 20 E. Bank Staff

Positions At ICR At Approval

Vice President: Axel van Trotsenburg James W. Adams Country Director: Stefan G. Koeberle Joachim von Amsberg Sector Manager: Hormoz Aghdaey T. Tunc Uyanik Project Team Leader: Alexandra L. Drees-Gross Thomas A. Rose ICR Team Leader: Nancy Chen ICR Primary Author: Lloyd R. Kenward

F. Results Framework Analysis Project Development Objectives (from Project Grant Agreement) The objective of the Economic Development Financing Facility is to promote post-tsunami economic recovery and foster sustainable equitable long-term economic development in Aceh in line with Recipient’s own plans for economic development. Revised Project Development Objectives (as approved by original approving authority) N/A

(a) PDO Indicator(s)

Indicator Baseline Value

Original Target

Values (from

approval

documents)

Formally

Revised

Target

Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 1 : Employment generated in enterprises supported by the project Value quantitative or Qualitative)

0 26381.00 34163.00 25000.00

Date achieved 09/30/2011 03/30/2011 06/30/2012 Comments (incl. % achievement)

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target

Values (from

approval

documents)

Formally

Revised

Target Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 1 : Private Sector Support: Enterprise supported Value (quantitative or Qualitative)

0 265 350 318

Date achieved 12/31/2010 09/30/2011 06/30/2012 09/30/2012 Comments (incl. % achievement)

Enterprises area supported in cocoa, coffee, patchouli, rice, emping, fisheries, and livestock.

Indicator 2 : Private Sector Support: Farmers and fisherman communities supported Value (quantitative or Qualitative)

0 26381.00 25000.00 34163.00


The project directly supports both marine and aquaculture fishermen; livestock, rice, patchouli, cocoa, and coffee farmers. Indirectly, the project supports more than 100,000 beneficiaries.

Indicator 3 : Public Infrastructure: Infrastructure built Value (quantitative or Qualitative)

0.00 14.00 250.00 203.00


Completed infrastructure includes learning centers, nurseries, jetty, hatchery, zooplankton facility, warehouse for warehouse receipts system.

Indicator 4 : Public Infrastructure: Infrastructure built being used by intended beneficiaries Value (quantitative or Qualitative)

0.00 100.00 80.00 100.00

Date achieved 12/31/2010 09/30/2011 06/30/2012 09/27/2012 Comments All infrastructure built to-date is being used by the intended beneficiaries.

(incl. % achievement) Indicator 5 : Effective Project Management: Capacity of implementing agencies built Value (quantitative or Qualitative)

10.00 13.00 10.00 13.00


12 PMU and 1 KPDT staff involved in implementation of the project full time. KPDT also has 4 part time consultants and 1 part time staff.

Indicator 6 : Effective Project Management: Sub-projects successfully implemented Value (quantitative or Qualitative)

0.00 37.50 75.00 50.00


Swisscontact, CCA, CCR, ADF, and IOM sub-projects have broadly been implemented successfully, according to most recent monitoring reports.

G. Ratings of Project Performance in ISRs

No. Date ISR

Archived DO IP

Actual

Disbursements

(USD millions)

1 10/29/2009 Unsatisfactory Unsatisfactory 5.00

2 09/17/2010 Moderately Unsatisfactory

Moderately Unsatisfactory 18.98





5 04/16/2012 Moderately Satisfactory Moderately Unsatisfactory 44.46

6 11/03/2012 Moderately Satisfactory Moderately Unsatisfactory 44.46

H. Restructuring (if any)

Restructuring

Date(s)

Board

Approved

PDO Change

ISR Ratings at

Restructuring

Amount

Disbursed at

Restructuring

in USD

millions

Reason for Restructuring &

Key Changes Made DO IP

12/15/2011 MU MU 44.46 06/26/2012 MS MU 44.46

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal

1. On December 26 2004, a major tsunami devastated the coastal regions of North-Western Sumatra, Indonesia. According to Indonesia’s National Disaster Relief Coordination Agency, 126,915 people died in the tsunami and another 37,063 were missing; the UN estimated that 655,000 people were homeless and took shelter in scattered refugee camps across the province. Bappenas’ (State Ministry for Development and Planning) preliminary damage assessment estimated that 80,000 small- and medium-sized enterprises were destroyed, representing a loss of employment for 140,000 people. Almost all casualties and damage were within the province of Aceh. 2. The Governments of Indonesia (GoI) and of Aceh (GoA) responded quickly with a reconstruction program that was one of the largest such efforts in the developing world. In support the international community mobilized large amounts of financing, including a Multi-Donor Trust Fund for Aceh and Nias (MDF) which, after extension, ran through end-2012. The initiative mobilized some US$650 million from 15 donors in support of the government’s rehabilitation and reconstruction blueprint. To move beyond rehabilitation and reconstruction, the Aceh Economic Development Financing Facility (EDFF) was designed to fill a gap up to that point, namely to assist in sustainable long-term economic recovery and development. The project, funded from MDF resources in the amount of US$50 million plus a government contribution of US$2.85 million, began in January 2009. 3. Local Economic Context. The tsunami followed many years of civil unrest in Aceh. Taken altogether, these factors contributed significantly to high unemployment and poverty in Aceh, with a large share of the population subsisting just above the poverty line. In the year prior to the tsunami, 2004, the economy had contracted by 10 percent, owing mainly to a decline in manufacturing and the oil and gas industry, which accounted for some 28 percent of GDP at project appraisal. In the year after the tsunami, the economy contracted by a further 10 percent, reflecting the impact of the tsunami on the manufacturing and agriculture sectors. In 2006, the economy expanded modestly (by 1.6 percent, owing mainly to the reconstruction effort) before contracting again in 2007 and 2008, by 2.4 percent and 5.2 percent respectively. At project appraisal, a strong recovery of the agriculture and manufacturing sectors was critically vital to avoiding a continuing structural decline as reconstruction activity wound down. A special note in this regard concerned known gas reserves, which were projected to decline to near zero by 2014. 4. The Reconstruction Program. Following the tsunami, the GoI and GoA quickly committed to rebuilding Aceh, with a reconstruction program estimated at approximately US$7.7 billion, spread over the period 2005-09. Looking beyond reconstruction, both governments identified economic growth as an essential element of recovery. At the same time, GoA highlighted a series of factors hampering Aceh’s longer-term economic development (e.g., weak government, damaged infrastructure, and low investment) in the agriculture, manufacturing and services sectors.

2

5. Rationale for Bank Involvement. As presented in the Project Appraisal Document (PAD), the World Bank was well-positioned to act as Partner Agency under the EDFF’s MDF. The Bank had extensive prior engagement in Aceh, and the project built upon the direction set out in the Indonesia Country Assistance Strategy FY2004-07 (especially the role of communities and local governments in shaping their own development objectives). The project was also consistent with the Country Partnership Strategy (CPS) 2009-12 for Indonesia, including reliance upon and strengthening government systems.

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved)

6. Objectives. The project’s higher-level Project Development Objective (PDO

1) was to

promote post-tsunami economic recovery and foster sustainable equitable long-term economic

development in Aceh in line with the Recipient’s own plans for economic development. To achieve this PDO, the project would finance sub-projects that contribute to the following objectives, identified by GoA as key for sustainable economic development in Aceh:

Development of job-creating, market-driven enterprises engaged in value-added processing and manufacturing, especially in agriculture and fisheries;

Sustainable improvement of production quality and value in agriculture, fisheries and estate crops that contributes to alleviation of poverty;

Increase in international trade, especially direct exports; and Increase in domestic and foreign investment in Aceh

7. The project also focused on capacity building of the Development and Planning Agency of Aceh (Bappeda), which housed EDFF’s Project Management Unit (PMU). This capacity building was seen as particularly important in light of the large amount of financing available to the provincial government through the recently created Special Autonomy Fund. KPDT (the State Ministry for Disadvantaged Areas, EDFF’s main counterpart in Jakarta) was also expected to benefit from skills development, although this was officially outside the mandate of MDF. 8. Key Performance Indicators

2 (KPIs). The following key performance indicators

(KPIs) were to be used to assess progress in achieving the PDO:

Number of business constraints that were successfully alleviated in Aceh; Employment generated in enterprises supported by the project; Number of users of public infrastructure financed by the project; and Success rate of sub-projects financed by the project.3

1 The PDO in the PAD is slightly different from that of the Grant Agreement but has the same meaning. 2 Please note that the ISR results framework was not fully complete, as only one of the three KPIs were reported in the ISR, although it was monitored and the PMU had report on it. 3 The PAD is contradictory as to whether this is a KPI or an Intermediate Outcome Indicator. In the PAD (p. 5), it is listed as a KPI, but in the Results Framework and Monitoring (Annex 3), it is an Intermediate Outcome. The Grant Agreement (Annex 2 to Schedule 2) is consistent with the PAD’s Annex 3.

3

9. Intermediate Outcomes. Moving down the hierarchy of the EDFF’s Results Framework, the PAD identified three Intermediate Outcomes (also known as ‘Results Areas’):

i) Improving the Business Environment: a better enabling environment for private sector development and investment;

ii) Private Sector Support: to improve the productivity of the private sector, farmers and fishermen; and

iii) Public Infrastructure: financing of economic infrastructure necessary for business development and job creation

10. For the PDO and each Intermediate Outcome, quantitative results indicators were defined and would be linked to Outputs once the sub-projects were identified. An analysis of EDFF Results is summarized in the Aceh EDFF Final Report, and reproduced as Annex 10. Please also refer to Section 1.3 for more discussion. 11. As designed, EDFF was innovative in that it delegated project implementation and budget to the province as a way to help build capacity to implement similar projects at the local level. Also, it was the first time that economic growth sub-projects were being implemented through sub-grants from the government to international and local NGOs. This approach resulted in enormous implementation challenges, indeed far more difficult than anticipated at design.

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and

reasons/justification

12. At the time of project approval, the sub-projects were not yet identified. Consequently, it was impossible to develop specific output indicators at that stage. Accordingly, the PAD (p. 47) explicitly notes the likely need to revisit the Results Framework once the sub-projects had been selected. As a result, in early 2011 the Government and Bank project team identified specific monitoring indicators and reported back to the MDF Technical Working Group on the Results Framework. While the overall objectives were not revised, one PDO-level outcome indicator, the number of business constraints alleviated, was rendered less relevant due to a lack of sub-project proposals in this area. 13. The additional intermediate indicators mainly refined Results Area #2 (Private Sector Support), especially sustainable improvement of production quality and value in agriculture, fisheries and estate crops. The great majority were output indicators. Also, the training indicators are totally quantitative, that is, taking the quality and targeting of the training for granted. 14. As for the values of performance targets, only one (a 75 percent success rate for sub-projects) was specified in the PAD. All others (see Annex 10) were chosen in the course of sub-project selection. This provided a great deal of flexibility during implementation.

4

1.4 Main Beneficiaries

15. According to the PAD (p. 5 and p. 75), the beneficiaries of the sub-projects would be the people, businesses and communities of Aceh. Further details were to be defined in the sub-projects. 16. Beneficiary details are well-documented in the Aceh EDFF Final Report (Chapter 5 and Annex 14) and they vary widely by Intermediate Outcome/Results Area. In general, beneficiaries of outputs were primary commodity producers (especially cocoa, coffee and fisheries); small and medium-sized enterprises; forums and traders associations; cooperatives; and regional governments. Beneficiaries totaled 36,568 persons (versus a target of 36,706) and were spread across 962 desas, 142 sub-districts and 17 districts. With regard to capacity development (generally interpreted in project documentation as one aspect of project management), the main beneficiaries were the PMU, Bappeda, Satker and to a lesser extent the KPDT staff in the EDFF Secretariat. The main strategy was learning-by-doing, because the scope for formal skills and knowledge transfer was limited by the small number of counterpart staff and their turnover, especially at the EDFF Secretariat in KPDT.

1.5 Original Components (as approved)

17. The project had two main components. Component One of US$44.5 million financed the sub-projects that were to be identified, except in general terms (Section 2.1). Component Two (US$5.5 million) financed project management and capacity building, including an international consultancy to assist the project management. The main beneficiary was expected to be Bappeda Aceh (the provincial-level implementing agency), although some support for KPDT was envisaged as well.

1.6 Revised Components

18. There were no major changes to project components. However, as the sub-projects took shape, the emphasis in Component One shifted noticeably from ‘Improving the Business Environment’ (Results Area #1) in favor of ‘Private Sector Support’ (Results Area #2).

1.7 Other significant changes

19. Extension and Reallocation of Cost Categories. The project faced extensive start-up and implementation delays, including those relating to sub-project selection and implementation, procurement and financial management. As a result, in December 2011 the project closing date was extended from June 30, 2012 to November 30, 2012 in order to provide additional time to complete project activities and help to achieve the PDO. Along with this extension, the Ministry of Finance also requested a minor reallocation of cost categories to provide additional resources for the incremental operating cost and consultant services categories (Table 1). This reallocation was considered necessary during the extension period to provide the government with sufficient resources to achieve the project objectives.

5

Table 1. Reallocation of Cost Categories Category Original Grant

Allocation

(in US$)

Revised Grant

Allocation

(in US$)

Percentage of

Expenditures to be

Financed

(inclusive of taxes) Sub-project Grants 44,500,000 43,400,000 100% Consultant Services 5,255,000 6,100,000 100% Incremental Operating Costs 245,000 500,000 100% Total 50,000,000 50,000,000

20. NGOs Dominate. As envisaged in the PAD, the sub-project implementing entities (SIEs) could be non-governmental organizations (NGOs) (local or international), private sector organizations or international aid agencies. In the event, 4 only NGOs were selected. Specifically, seven out of eight were international NGOs (four partnering with local institutions) and one was a local NGO. These international NGOs brought significant community-level implementation expertise and strong governance, especially as regards transparent procurement processes. However they lacked experience in dealing with the World Bank’s and GoI’s processes and regulations. This had significant ramifications and created major implementation delays as these firms had to gain knowledge and familiarity with the project procedures. Although administrative and financial management capacity assessments were conducted as part of sub-project selection, the difficulties in adapting each organization’s systems to the project procedures were significantly underestimated. This can be considered one of the main factors negatively affecting implementation. 21. Narrower Scope in Shorter Timeframe. As described in Section 2, the project got off to a very slow start, which effectively cut the implementation period roughly in half. This reduced the scope and scale of some sub-projects and required some activities to be dropped (see the “Main Problem” column of Annex 11).

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry

22. Sound Contextual Analysis. The background analysis for this project was overall well done, drawing upon the Bank’s already extensive experience in post-tsunami Aceh. The PAD provides a clear statement of the then-current economic and institutional (GoI and GoA) situations, including the state of reconstruction efforts and the outlook for key sectors of the economy. The only obvious deficiency in this area concerns the limited discussion on the state

4 In explaining this outcome, the careful wording in the PAD is notable, namely, private sector organizations (italics added) implied non-profit organizations, not businesses; in the local context, inviting businesses could have opened the door to the possibility of nepotism and/or financial irregularities. Also, among international aid agencies, the most likely applicants would have been United Nations agencies, which would have had trouble reconciling their own operating guidelines with those of the Bank. For their part, local NGOs generally found the application procedures to be onerous. The best outcome would have been a strong partnership between an international and local NGO.

6

of the NGO community. In light of NGOs’ important role in the project–and, ultimately, their highly mixed performance as SIEs–more discussion was warranted as to their likely capacity as managers of World Bank and GoI projects.5 23. Strong government ownership and local government involvement. The decision to work through government systems for the project had important benefits and resulted in a high degree of ownership throughout implementation. The decision to select sub-projects in a number of sectors (as opposed to just one or two) and work in as many districts as possible also resulted in a strong sense of ownership of the project across the province and in many communities and helped achieve broad support for it at critical times.

24. General Design Issues. Discussions on the project began in mid-2006, with the intention of proceeding on an emergency basis to fill a gap in post-tsunami assistance, namely the lack of attention to longer-term growth and development. However, design issues quickly emerged. For example, GoI appears to have had doubts as to how much financial authority should be delegated to GoA. Likewise, differences arose between GoA’s wish (one project for each of the 17 districts in Aceh) and the mandate of MDF (which emphasized regions directly impacted by the tsunami). These issues caused delays and forced compromises, like a smaller number of sub-projects, but with cross-district coverage. Also, the process at this stage entailed a time-consuming, two-layer review and approval process, by the Bank and MDF. The latter entailed resolving the diverse views of a large consortium of donors in an already complex operation. Indeed, there were serious discussions of dropping the project at this stage. But after due consideration, the decision of the Steering Committee was to proceed, in the belief that the remaining time was adequate for good outcomes, despite the risks. This stage of the project consumed 1½ years. 25. As designed, the project had three key characteristics: it was innovative (Section 1.2); it had a great deal of built-in flexibility, because the sub-projects were still to be selected; and it was ambitious, particularly in light of the short project lifetime (3½ years).6 Being innovative was useful under the circumstances and the extent of the built-in flexibility demonstrated valuable foresight. However the ambitious number 7 and complexity of the envisaged sub-projects made for a risky project even after risk mitigation measures. See Sections 3.1 and 4 for more discussion of the downstream implications. 26. Specific Design Weaknesses. There were five important issues in this area.

5 Please refer to Annex 12 for more details. 6 The PAD readily acknowledged these issues, but mainly saw them as an evaluation issue (p. 54). 7 For instance, the PAD (Table 9, p. 64) mentioned a total of 11-16 large sub-projects and 8-12 small ones. In the event, implementing 8 sub-projects proved difficult. In commenting on this issue, the Bank’s design team believe that this interpretation of the number of projects cited in Table 9 of the PAD is misleading because the total number was not intended to be additive across Results Areas. Apparently, their intention was to have 6 or 7 large projects (to achieve economies of scale) plus a few smaller ones (to accommodate innovative, but risky undertakings, probably to help local NGOs get started). In retrospect, even this would have been challenging. A better strategy would have been to start smaller (say 3-4 sub-projects); work out the implementation arrangements better; and then scale-up in a successor project (Section 6).

7

1. A complex project structure that made for slow decision-making and diffused

responsibilities. The project structure included national, regional and local governments, which sometimes had competing interests; several (ultimately 8) diverse sub-projects and SIEs; and 15 aid donors with the Bank serving as Partner Agency for the MDF. Individually, there were good reasons for these complexities,8 but collectively they were burdensome.

2. The project was ‘on-budget and on-treasury.’ Notwithstanding an early agreement to the

contrary, 9 SIEs had to comply with GoI as well as World Bank processes and regulations. Many of the SIEs were well-established with a good track record in finance and procurement, using their own systems. Nevertheless they had to adjust to new World Bank and government policies, which required a major effort by all parties. The difficulties were seriously underestimated at design.

3. Project design included a process for choosing the sub-projects. Unfortunately, the initial

nine months or so were consumed by the call for and selection of sub-project proposals,10

some of which were hundreds of pages long. In addition, after the long selection process, certain time-consuming administration requirements from government and World Bank further delayed field implementation. Taken together, these reduced the initial 36 month timeline to around 18 months. The selection process did not allow enough time to look at

an adequately wide range of criteria, for example, field visits and possibly a review of partnership arrangements. Also, the selection committee excluded national consultants and experts from the provincial and district departments of local government who could have given quality inputs.

4. Capacity development at Bappeda Aceh (and, to a lesser extent, KPDT) was important

for this project (p. 4 of the PAD) and Component #2 was dedicated to these activities. But the PAD design did not ensure success in this regard as an Intermediate Outcome/Results Area, nor were there any KPIs. As designed,11 this important function would have had secondary priority for any results-oriented PMC.

5. The fixed closing date for the project limited sustainability. From inception, EDFF was

constrained by the pre-determined closing date for MDF; there was no possibility for an extension beyond this point. Given the project complexity and implementation

8 All these agencies were important at one stage or another of the project, and it would have been counter-productive to try to bypass any of them. 9 As an early attempt at compromise, an agreement was reached that SIEs could use their own internal systems, as long as they corresponded with regulations of the World Bank and GoI. Both World Bank and GoI rules would apply, and in the case of contradictions, the Bank’s would apply. Unfortunately, BPKP Aceh (the local arm of GoI’s Supreme Audit Agency) never fully bought into this agreement and there was no mechanism for resolving conflicts between the two systems. 10 For most of this time, the PMU was just getting off the ground; the PMC was barely operational; and all counterpart staff in project implementation were newly-recruited. Still, Bank staff were reluctant to get involved in the substance of the selection process, because it was considered very important that GoA learn from, and have ownership of, the selection process. 11 This oversight was corrected during redesign of the Results Framework during implementation, a good example of the project’s flexibility.

8

difficulties, many sub-projects had to curtail their planned activities in one way or another. This greatly reduced potential for sustainability.

27. Quality at Entry. There was no Quality at Entry assessment of this project.

2.2 Implementation

28. Despite initial implementation delays, the project achieved important results on the ground in a remarkably short implementation period of time. Several of the NGOs, particularly in the cocoa, coffee, patchouli and agriculture sectors, managed to reach nearly 35,000 farmers through various interventions. Largely, the NGOs implemented a model that combined international experience with SME development programs with local partners and cooperative groups. Training and technical assistance activities focused broadly on developing improved agricultural practices, marketing and business development skills, and establishing linkages with markets. While there were shortcomings in the design, EDFF was a notably ambitious and bold project that has shaped some key sectors in Aceh (i.e. cocoa) through knowledge transfer and capacity building and demonstrated a potential of an NGO partnership model for the Government to replicate in order to meet economic development goals in disadvantaged communities. 29. Broad distribution of subprojects across the province. As mentioned above, the broad distribution of EDFF sub-project across the province – which was a strategic decision by the Government in sub-project selection – was a success factor and generated important support in hundreds of communities and broadly shared the benefits of the project. 30. Governance issues. Governance issues in project management and sub-project implementation had a significant impact on the overall EDFF project results. These issues at times created a very challenging operating environment for the NGOs, and affected implementation of the sub-projects by delaying procurement or implementation of key activities.

31. Uneven Implementation Capacity. The various sub-projects covered a great deal of ground (Annex 2), and they were implemented by a diverse group of NGOs (Annex 2). Not surprisingly, wide differences in capacity became obvious during implementation. As highlights, one SIE turned in a stellar performance (Section 3.2), whereas the others were spread out, right to the opposite end of the performance spectrum (Annex 11). In general, most SIEs delivered their main outputs, despite challenges and delays (see following paragraphs). Shortfalls during implementation were often due to the limited time left for consolidation of gains and steps to help with sustainability (Section 4).

32. Several sub-projects had a number of notable implementation difficulties, the great majority of which could be categorized as ‘not subject to their control.’12 First, several SIEs experienced difficulties with their local partners, in some cases leading to a split in the partnership. Second, the sub-projects relied in substantial part upon cooperatives as the

12 Appendix F to “Implementation Completion and Results Report Guidelines,” OPCS, dated August 2006, last updated 10/05/2011, p. 51.

9

institution-of-choice for sustainable development. As a conceptual device, this approach was questionable, considering the checkered history of cooperatives in Indonesia. Third, most of the SIEs opted for newly formed cooperatives or groups, 13 instead of attempting to reform existing entities. This may have been a mistake because start-up proved to be time-consuming, and this cut further into the available implementation period. Fourth, most SIEs were reluctant to team up with private sector entities, whereas the best results were achieved in sub-projects where the private sector was involved (e.g. Swisscontact, IOM and CCR; see Annex 11).

33. Procurement Issues.14 The eight sub-projects averaged 63 procurements, which

encompassed a wide range of procurement methods. This presented a challenging, time-consuming task for all during implementation because:

SIE procurement officers had no experience with applying World Bank procurement guidelines. This seriously delayed the processing of documents.

Several SIEs lacked expertise in areas like technical design and specifications, advertisements and contracting, and while these are project management issues they in turn impacted on the timeliness of the procurement process.

The procurement experts of the PMC: (1) did not prepare applicable sample documents for the NGOs; (2) did not provide sufficient training to the NGOs before their sub-projects were implemented; and (3) were not available for some periods of the project implementation.

A lack of familiarity with the World Bank procurement guidelines and how they differ from that of the SIE led to significant confusion, frustration and duplication of effort.

The Bank’s preference for National Competitive Bidding over flexible procurement mechanisms, like community contracting for certain non-typical goods (e.g. livestock), was problematic in the local context. It created resentments vis-a-vis some SIEs who had experience, expertise and confidence in these approaches. It also undermined local ownership of the investments, thereby reducing sustainability. Also, the Bank’s last ex-post review raised some questions about the true level of competitiveness in the bidding process of some of the packages of non-typical goods, and the concerned local SIEs acknowledged their weakness in identifying governance issues in procurement and they pledged to focus and build their capacity for addressing such issues in the future.

34. Procurement Rating. The procurement rating for this project is “Moderately Unsatisfactory.” 35. Financial Management and Audit. Issues in this area were similar in nature and magnitude to those concerning procurement just discussed.

13 Documentation (Aceh EDFF Final Report, p.23) indicates that SIEs wanted to avoid existing cooperatives in Aceh, which are generally considered weak and often seen as social institutions depending on government aid, not self-supporting, business-oriented organizations. It remains unclear as to why the SIEs believed that newly-established cooperatives would ultimately function any better than existing ones. 14 By and large, these factors (and those under Financial Management and Audit) were outside the control of government, the SIEs and the Bank; they were indicative of the project’s complex operating environment.

10

Key tasks originally allocated to the KPDT Secretariat (e.g., the compilation of the monthly financial reports) had to be reallocated to the PMU because KPDT lacked capacity. Also, the budget executing agency’s (Satker’s) requests relating to those financial reports required the PMU to verify all supporting documents submitted by the SIEs. PMU was unprepared for these developments, which required 7 additional staff.

Once the SIEs were operational, there was insufficient supervision of the SIEs by the PMU/PMC.

Delays were occasioned by the government’s demanding budgetary management (i.e., DIPA) system.15 In principle, this factor was ‘subject to the government’s control.’

Not all parties bought into the early agreement that World Bank financial management guidelines would be operative (see footnote 10). Many GoI/BPKP findings (which contradicted Bank guidelines) could never be resolved and several SIEs had to repay considerable amounts of money.

Follow-up to Bank audit findings sometimes proved challenging due to insufficient exit meetings prior to report finalization.

Many of the same considerations apply to the process of asset transfer, done under time pressures at the end of the project. At this point, governance issues and transparency of process were particularly important issues.

36. Financial Management Rating. The financial management rating for this project is “Moderately Unsatisfactory.” 37. Excessively Short Implementation Period. These many start-up and implementation problems seriously compressed the SIEs’ implementation period, from 36 months to 15-20 months, depending upon the start-up date of the sub-project. As mentioned, the implementation period was eventually extended by 5 months. 38. Intensive Implementation Support. The project required intensive implementation support from the grant of US$ 5 million for a PMC to manage the project and from MDF funds to finance the Bank team’s implementation support. Effectively, this project was more akin to 8 small projects than one mid-sized project and implementation and administrative requirements reflected this (Section 3.3).

15 Complexities of the DIPA process were known beforehand, but Bank staff had assurances from Bappenas that the process could be more flexible for grant financing. Also, the Ministry of Finance was open to the possibility of providing financing out-of-cycle with the DIPAs. However this flexibility proved inadequate, and there may have been a problem with KPDT’s capacity to prepare the DIPAs.

11

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

39. M&E took place at several levels in this project, including at the PMU/PMC, Bappeda Aceh, the local government and KPDT. For the purposes of this ICR, the main focus is on the project’s Results Monitoring Framework, which as updated after selection of the sub-projects, would form the basis for Evaluation, according to the PAD (p.54). It is important to note that the design to update the result framework after sub-project selection was agreed upon explicitly during project design and the MDF Technical Working Group endorsed the updated indicators.

40. M&E Design. The PAD’s M&E design had some minor technical problems,16 but by and large was satisfactory for that stage. Review of the Results Monitoring Framework, after identification of the sub-projects, led to only modest higher-level changes (Section 1.3). In its details, the Framework’s most notable feature was its heavy reliance upon output (not outcome) indicators. This is largely due to the difficulties of measuring outcomes from a short implementation period, which was already down to roughly 1½ years at this stage. 41. The M&E Unit of the PMU/PMC, which was responsible for design, coordination and implementation of the M&E system, including field verification visits and coaching the SIEs. Their main reporting tools were a Quarterly Physical and Financial Progress table (QPF) and District Commodity Progress report (RCD). The former reported progress against the work plan, including financial and physical progress.17 The latter was a table of output indicators by sub-project/SIE, by district and by commodity that, in principle, are linked to key EDFF results indicators.18 42. In another notable step, the PAD’s Results Framework was expanded to include sub-project outputs and reworked into a simple, coherent presentation. In examining this Framework’s results chain, the linkages from inputs to outputs to outcomes are clear. Namely, the project’s inputs were directed at successful completion of the sub-projects, which were selected to support economic development in the agriculture (food crops, non-food crops and fisheries) and manufacturing sectors. These sectors had already been identified as priorities in Aceh’s regional economic development plan, and the sub-project selection process included representation from the central government; the Governor of Aceh’s office; the Aceh Bappeda; and a member of the academic community in Aceh. This is fully in line with the PDO, as well as GoI’s own plans (Section 1.2). Outputs and outcomes are reviewed in Annexes 2 and 3, respectively. 43. M&E Implementation & Utilization. Implementation of the M&E system was slow at the outset as it was initially difficult to agree upon reporting formats. In addition, the SIEs’ logframes and existing reports could not be easily linked to the EDFF Results Framework (which motivated development of the RCD report). However once the QPF and RCD reports were operational, they seem to have worked well for monitoring outputs, despite technical problems

16 These concerned some limited, but notable, confusion between Outputs and Outcomes at the Intermediate Outcome Indicator level for the project (Table 7 in the PAD). 17 Physical progress (in %) calculates cumulative outputs of activities, weighting activities by budget. 18 The M&E unit also collected detailed beneficiaries lists from each SIE.

12

with the QPF;19 the RCD reports are the basis for the great bulk of the project’s evaluation data. No information is available on the reports’ cost to the PMC; SIEs found the formats burdensome, partly because Bappeda was anxious to link it into their provincial system. 44. The PMC’s M&E system has yielded an impressive volume of data on, among others, SIE outputs, beneficiaries and outcomes. In reviewing this system, a few important issues came to light:

The ratings system for sub-projects is potentially controversial. In particular, if the rating

is based primarily on Results,20 only 3 of the sub-projects were rated as ‘Satisfactory’ or above (column 3 of Annex 11). Two others were rated as ‘Moderately Satisfactory.” On this basis, one Intermediate-level performance indicator (and KPI) is not met.

However, if performance also takes into account financial issues, administrative

management and reporting, two SIEs would go up in the ratings and one would go down. On this basis, the Intermediate-level performance indicator is met.21

This ICR relies upon the results-oriented rating, which is more consistent with ICR Guidelines.22

In examining the final Physical Progress Report (Annex 23 of the Aceh EDFF Final Report), this review spotted sufficient errors in the calculations to undermine confidence in the accuracy of the final numbers.23

2.4 Safeguard and Fiduciary Compliance

45. Environmental Safeguards. There were no important issues in this area. Implementation of the sub-projects’ environmental safeguards demonstrates that all requirements were met with little or no negative environmental impact. Indeed, the sub-projects are credited with promoting good environmental practices through awareness-building and environmentally friendly practices supported by the project (e.g., for cocoa, coffee and patchouli). In general the project promoted best environmental management practices that are in line with the environmental safeguards principles that comply with both Bank and Indonesian national requirements. 46. The only notable point concerns the PAD’s classification of all proposed sub-projects as “Category B” projects (which requires an Environmental Management Plan). Subsequently, most sub-projects were reclassified as “Category C” projects, implying that no further environmental actions were required, beyond signing a Statement of Environmental

19 In particular, the methodology biases measurement in favor of achieving objectives. Also, it assumes a stable and consistent statement of planned outputs. In practice, the SIEs regularly revised their budgets and planned activities. 20 Source: Annex 6, p.147 of the Aceh EDFF Final Report. 21 Source: See preceding footnote. 22 “Implementation Completion and Results Report Guidelines,” OPCS, dated August 2006, last updated 10/05/2011, p. 8 (unnumbered). 23 Namely, six errors for one SIE report, chosen at random, from Annex 5 of the Aceh EDFF Final Report.

13

Commitment. Ultimately the reclassification saved the SIE’s a lot of unnecessary work, but it also caused confusion and wasted effort at the outset.

2.5 Post-completion Operation/Next Phase

47. As mentioned, EDFF was always constrained by a fixed endpoint due to the pre-determined closing date (end-2012) for MDF. There was no possibility for an extension beyond this point and no follow-on project is in store. 48. Three sub-projects (ADF, IRW and Swisscontact) were able to line up some follow-on financial support. However, a comprehensive solution that would address the need, for example, for continued training for the EDFF-supported cooperatives was not found. These considerations underscore the issue of sustainability, which is discussed in Section 4.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation

49. Relevance of Project Design Objective. Appropriately, the project’s PDO is carefully worded, while aiming high. In particular, the phrase “To promote post-tsunami economic recovery and foster sustainable” (italics added) is descriptively accurate, without over-commitment. This is important because, while EDFF’s resources were sizable, they were small relative to Aceh’s economic development needs and challenges. In addition the PDO is adequately clear and suitable as a follow-on operation to post-tsunami rehabilitation and reconstruction. Furthermore it is consistent with the direction set out in the Bank’s key country strategy documents (Section 1.1). 50. Two other aspects of the PDO are especially notable because they would have important, and largely unforeseen, implications during implementation. These were the emphasis on equitable long-term development in line with GoI’s own plans for economic development. In the local context, these almost guaranteed that cooperatives would play a major role in the project. This, in conjunction with a 3½ year project lifetime, set the stage for issues of sustainability (Section 4).

51. The clarity of the PDO was diluted somewhat by the supplementary importance attached to capacity building at the main GoI counterparts. Capacity building at GoI counterparts was inevitable, but it added an institutional development dimension to the project, whose difficulty looks to have been seriously underestimated.

52. Continuing Relevance of the PDO. The PDO remained highly relevant throughout the project’s lifetime. As tsunami reconstruction wound down, the GoA remained (and remains) highly focused on the challenges relating to building the foundations for sustained economic development in Aceh.

53. Relevance of Design. As mentioned in Section 2, EDFF’s project design was innovative, institutionally complex, very ambitious and flexible.

14

Innovative. According to the PAD, the project broke new ground for GoI insofar as this

was the first time that economic growth sub-projects were being implemented through sub-grants from the government to private international and local entities, including NGOs. Unfortunately, although the NGOs provided to bring in important knowledge and expertise in certain areas, the non-piloted implementation arrangements proved much more difficult than anticipated.

But Overly Complex. As discussed, the project was too complex for such a short timeframe, and there was no possibility for extension beyond December 2012. In light of the innovative and untested implementation arrangements, the project should have been a ‘pilot project’.

And Too Ambitious. Notwithstanding this complexity, the design envisaged significant outcomes in 4 wide-ranging ‘Results Areas’ by 8 diverse SIEs in a short period, 3½ years. The number of sub-projects should have been significantly smaller.

But Built-in Flexibility Helped Save the Project. Of necessity, many design details had to be delayed until the sub-projects were selected, but this also provided the project with enough flexibility to adjust as circumstances unfolded in the field. This turned out to be very important during implementation. Indeed, this flexibility–combined with an extension and a major push towards the end of the project–contributed to considerable results during the project’s final stage.

54. Risk Assessment at Design. At appraisal, the project was understood to be a risky undertaking. Indeed, overall risk was categorized as ‘Substantial’ after mitigation measures. Seven ‘High’ risks were identified, most of them implementation risks at the sub-project level. After mitigation measures, all were reduced to a ‘Significant’ risk. As the project developed, most of these high risks eventuated, and the risk mitigation measures proved inadequate and ineffective (see Annex 12 for analysis). 55. Relevance of Implementation. Initial implementation was dedicated to the sub-project application and selection process, which faced serious delays as discussed (Section 2.1). The next stage was dominated by issues of procurement and financial management, some of which did not get resolved during the project’s lifetime (Section 2.2).

56. By the time the SIEs actually got their operations off the ground, the implementation period had been cut roughly in half. Consequently, the envisaged scale and scope of some sub-projects had to be cut back; several physical investments were cancelled and many training activities to promote sustainability were cancelled or shortened. With only 12 months left before the envisaged project closing date (June 2012), SIE disbursements were only 16.2 percent of target.

57. The mid-term review (conducted in February 2011) concluded, among others, that without substantial support of all stakeholders involved it would be hard to achieve the EDFF objectives on a timely basis because of World Bank and GOI process requirements, which were beyond the capacity of the SIEs. Sustainability of EDFF activities was assessed as risky and options were advanced for increasing the chances for sustainability. Ultimately, project closing

15

had to be extended by about 5 months,24 which allowed for the completion of some critical activities and investments, but still left project management on a very demanding schedule.

58. Despite these many problems, the SIEs produced an impressive number of outputs, under pressure mainly during the last 12 months of the project. The summary of QPF indicators captures this conveniently, notwithstanding their technical problems (Section 2.3). In total, the SIEs achieved 92 percent of their physical and 85 percent of their financial targets as set out in sub-project proposals and budgets, and some specific successes are particularly notable (see the concluding paragraph in Section 3.2).

59. Limited Private Sector Involvement. A striking characteristic of project implementation is the limited role of the private sector. The NGOs who ran the SIEs had a strong preference for the cooperative empowerment model rather than direct engagement with the broader private sector, which sometimes made it difficult, for example, to move upward in the value chain (see AAA, ADF, CCA and IRW in Annex 11).

3.2 Achievement of Project Development Objectives

60. The Results Monitoring Framework provided for evaluation at two levels, namely, at the higher, PDO level and the intermediate, ‘Results Area’ level. In principle, both are highly quantitative and the latter includes a large number (21) of performance indicators. 61. PDO-level Indicators. As designed in the PAD, there were 3 outcome indicators at the PDO level,25 but one quickly proved less relevant owing to a lack of sub-project proposals in this area. The PMC did continue to report against this PDO-level indicator as it considered appropriate. In its final report, it noted, a conference and public-private forum (see Annex 10, Results Area #1, column 4); improved market linkages, basically for cocoa; and improved availability of certain equipment and inputs (see Annex 6 to the Aceh EDFF Final Report). Making good use of the flexibility built into project design, it was rolled into the intermediate level indicators for the relevant ‘Results Area’ (i.e., Improving the Business Environment). In the circumstances, this was sensible.

62. All Outcome Indicators at this level shared an important characteristic, namely, that the PAD did not include quantified baseline values or final completion targets for measuring the impact of the project (see Annex 3 for further considerations in this regard). That function was explicitly delegated to the PMC and SIEs (p. 52 and 54 in the PAD). In the event, one extensive baseline study was done (for cocoa production; see Annex 2) and other sources were used to set quantitative targets for the individual sub-projects, in line with PAD suggestions (p. 54). This was a reasonable approach, because there was little analytical basis on which to define an overall target at design. Still, it is awkward to judge project outcomes without a well-defined overall target.

24 In January 2012, an extension was approved for 4 months for the sub-projects and 5 months for the PMC. In August 2012, 5 sub-projects were extended for 1 more month. By end-November, all activities/investments in the eight sub-projects had been finalized and handed over to the beneficiaries. 25 See Footnote 3.

16

63. Performance against the two remaining PDO-level indicators is reported in the Aceh EDFF Final Report, together with a great deal of underlying details (summarized in Annex 10). That source reports the equivalent of 4,400 fulltime jobs created and a total of 17,178 users supported by the infrastructure financed by the project. These are large numbers for a mid-sized and short project. They are equivalent to almost 1/4 of 1 percent of total employment in Aceh in 2011 and more than one-third of 1 percent of total population in 2010, respectively. See Section 3.3 and Annex 3 for further discussion.

64. Intermediate-level Indicators. On intermediate-level indicators, the overall performance is positive, bearing in mind that the indicators are predominantly output-oriented, not outcome-oriented (Section 1.3). In particular, the great majority of these indicators were exceeded. Looking more closely at performance by ‘Results Area’ (for details, see Annex 11):

1. ‘Improving the Business Environment’: Successful performance in this area, against a relatively ‘soft’ outcome indicator and two output indicators.

2. ‘Private Sector Support’: Exceeded performance criteria for 9 of 10 indicators, almost all of which are output-related. Some particularly impressive productivity increases are reported for some commodities (e.g., cocoa and coffee).26 Some others are particularly disappointing (e.g., the success rate for artificial insemination of cattle and fingerling production).

3. ‘Public Infrastructure’: Mixed result. Exceeded performance criteria for infrastructure built and used, but sustainability below target.

4. ‘Success Rate of Sub-projects’: Ambiguous result. All SIEs improved capacity in EDFF’s financial management, procurement and safeguards, 27 but only 5 SIEs were rated ‘Satisfactory’ or above, versus a results-related target of 6 (see Section 2.3). Indicators of capacity development suggest success with SIEs, but government is less clear; Bappeda Aceh exceeded the performance criteria, but KPDT was well below target. Indeed the Aceh EDFF Final Report (p. 85) downplays the governments as beneficiaries and the government’s draft evaluation report makes no mention of the government as a beneficiary.

65. Other Relevant Results. EDFF’s best performer, Swisscontact, warrants special commentary. It was engaged in cocoa production and marketing (Annex 11) and it accounted for about 17 percent of project budget, but more than 40% of the project’s achievements. These included training one out of three cocoa farmers in all of Aceh and reshaping GoA’s agricultural

26 The performance criterion was ‘% increase in the value of smallholder production’ (italics added), but productivity increases are reported. Productivity is the more relevant measure. 27 Note that some of the SIEs already had good systems in place, and improved capacity in EDFF’s financial processes is no assurance that their general capacity has increased further. Improved capacity in EDFF’s processes is really only useful for NGOs that may want to work on another World Bank project. Most are more likely to benefit from improved capacity with GoI systems, not EDFF’s.

17

strategies and engagement with the private sector (e.g., Nestle, Kraft and Archer Daniel Midlands). Also, Swisscontact has entered into a MoU with KPDT and a number of other private sector partners and donors, to expand its cocoa program to 5 other provinces, increasing its outreach to 60,000 cocoa farmers. Wider engagement with the private sector (including with foreign companies like Nestle, Kraft and ADM) is especially important in this context, because GoA’s development objectives include increased foreign and private investment in Aceh (Section 1.2). 66. Among the other sub-projects, CCR (engaged in cultivation and processing of nilam; Annex 2) reported major increases in land tillage and yields (Annex 11). However, overall outcomes were frustrated by falling commodity prices (a factor outside the SIE’s control), which reduced plantings. In the case of IOM (Annex 11), its matching grants to SMEs (for construction, transportation and equipment) attracted some Rp4 billion (US$400,000) in new private investment into Aceh’s coffee sector.

3.3 Efficiency

67. PAD Criterion Met. As detailed in Annex 3, the requisite gains in GDP (necessary to justify the Bank’s minimum, 10-12 percent, economic rate of return) were comfortably exceeded, despite the delay as discussed earlier. Also, growth in the targeted sector, agriculture, exceeded the target by a small margin. By sub-sector, growth was robust where the project was operating, namely in food crops, non-food crops, and to a lesser extent, fisheries. 68. Analytical Cross-checks are Ambiguous. Cross-checks to substantiate causality and attribution yield ambiguous results (Annex 3). For example, the number of new jobs (and number of project beneficiaries) claimed by the PMC is surprisingly large. By contrast, the project cost per new job (4,400 jobs for about US$47 million) is roughly US$10,750, which seems expensive. 69. Large Overheads. The project administrative and management costs of this project were high, reflecting the complexity of project design and the weak institutional capacity. The direct project overheads including transfers to the consultancy company, KPDT and Bappeda were almost exactly 10 percent of the project budget. 28 In addition, the SIEs charged administrative overhead rates, which ranged from 2-10 percent, depending upon the SIE.29 Taken together, these would run overhead costs up to about 15 percent of the project budget, which is on the high side. However, this still excludes the Bank’s large implementation support costs which, in this case, were covered by MDF. 70. As for the 2-10 percent charged by the SIEs, this is a surprisingly wide range; 2 percent would be a bargain, whereas 10 percent would be on the high side. By way of comparison, the current recognized norm among Bank staff seems to be 5 percent.

28 The Aceh EDFF Final Report, p. viii. 29 The Aceh AEDFF Final Report, p. 12.

18

3.4 Justification of Overall Outcome Rating

Rating: Overall Outcome Rating: Moderately Unsatisfactory

71. The rating of Moderately Unsatisfactory is based upon the discussion in previous sections, especially: the overall relevance of the project PDO; the project design which had important strengths, particularly in its flexibility; and the achievements of the PDO- and intermediate-level outcome indicators. Also, the international NGOs brought implementation and project management expertise to bear on the project. However there were also some important weaknesses that limited results as follows:

Weak risk mitigation measures at design; Overly complex institutional arrangements, especially among the counterparts and SIEs; Underestimation of the support SIEs would need in order to conform with WB systems

of procurement and financial management; Underestimation of the process and administrative demands that would be placed upon

Bank staff (including procurement and FM staff) during implementation; The short, 3 ½ year project lifetime, which was seriously compressed due to start-up and

implementation delays; Design did not anticipate the implications of cooperatives playing a major role in the

project; and, Unanticipated complications stemming from BPKP oversight occasioned by on-budget

transfers.

3.5 Overarching Themes, Other Outcomes and Impacts

(a) Poverty Impacts, Gender Aspects, and Social Development

72. Poverty Impact. This project was designed and implemented with a view to creating jobs and reducing poverty in Aceh (PAD, p. i). In the project’s early stages, the beneficiaries of the sub-projects SIEs were selected based upon poverty analysis, to include the most marginalized communities. In implementation, SIEs worked successfully with highly marginalized communities, like women, widows and ex-combatants. In judging overall impact, the project’s results indicators (Annex 10) are defined more narrowly than ‘poverty impact,’ but virtually all of them imply success in poverty alleviation. Especially notable, considering the size of the project, are estimated employment gains (of 4,400 jobs) and total project beneficiaries (36,568 persons). See Annex 3 for more details. 73. Gender. One of the project’s strengths was the attention to developing gender-informed activities in all of the sub-projects. Further, the monitoring system required gender disaggregated monitoring indictors for all the sub-projects. This included gender breakdown of project beneficiaries by District and by SIE, and training by institution. In aggregate, roughly one-third of total project beneficiaries were women.

19

(b) Institutional Change/Strengthening

74. As noted in Sections 1.2 and 3.1, capacity building at the Aceh Bappeda (and to a lesser extent KPDT) was an important focus of the PAD, despite no direct mention in the PDO or Intermediate Objectives. Quantitative targets for institutional strengthening were added in due course, although the complexities look to have been seriously underestimated and the Aceh EDFF Final Report openly acknowledges that capacity building had second priority almost from the outset. In the event, the quantitative target for Bappeda Aceh was exceeded, but the full potential for capacity building among staff at that institution has not been realized; KPDT’s target was missed by a significant margin (Annex 10, Results Area #4). As a further consideration in this regard, several of the NGOs are now well-positioned for future projects operating within the GoI and/or World Bank regulatory frameworks, owing to their experience as SIEs. (c) Other Unintended Outcomes and Impacts (positive or negative)

75. Unintended Positive Outcomes. The government’s draft ICR notes encouraging behavioral changes among some coffee, cocoa and nilam farmers. Prior to the project, many farmers did not maintain their trees properly, being skeptical of the need for pruning, grafting, and other maintenance activities. As a result of the interventions, they are now convinced of the resulting benefits; the trees are now well maintained, and the farmers appreciate group efforts. A similar observation was made by one SIE, namely that local cattle farmers are starting to use their waste land to grow grass for fodder of cattle. This increases land productivity; provides alternative fodder during the dry season; and generates income. 76. Unintended Negative Outcomes. The government notes that the beneficiaries for most sub-projects were based upon poverty analysis, with a view to including the most marginalized communities. In some cases, this led to resentment on the part of some who were not selected and who did not benefit from the project.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops (optional for Core ICR, required for ILI, details in annexes)

N.A.

4. Assessment of Risk to Development Outcome

Rating: Rating of Risk to Development Outcome: Significant 77. It is likely that many of the outcomes of the project will not be sustained due to the short project timeframe and the lack of follow-up support at the community level. Three SIEs have lined up some follow-on support, but only one (Swisscontact) has obtained sufficient funding to sustain the interventions and replicate the project to new districts and regions.30 Outcomes are at particular risk in the following areas:

30 KPDT has entered into a separate agreement with Swisscontact to develop smallholder-based economic development projects in various other parts of Indonesia.

20

Cooperatives. Documentation repeatedly notes that very few, if any, of the 60

supported cooperatives were operating on a self-sustaining basis by the end of the project.31 As reported by the PMC, the cooperatives would require several additional years of support to be expected to operate on a sustainable basis.

Infrastructure Maintenance. This is one of the few intermediate-level performance targets that were not met. Only 70% of the project’s infrastructure has plans for sustainability.

Weak Areas in Results Area #2 (Private Sector Support). Sustainability of results in the Bireun livestock project is at risk, and the project requires more support from government and its NGO. In fisheries, the fish hatchery project is not yet capable of carrying on alone, but did secure more financing for a follow-on project. Even among successful commodities, like the coffee and cocoa projects, there are question marks, like the Warehouse Receipt System for coffee producers; at project-end, it was under-utilized and high-cost, and it required government financing to cover its management overheads. For the cocoa value chain project, intentions to improve downstream market linkages got bogged down in cooperative issues.

Replicability. The EDFF experience shows that the NGO model of sub-project implementation has important strengths in transferring knowledge and bringing in effective methodologies for SME development in key sectors relevant to Aceh. The implementation arrangements would need to be substantially streamlined and simplified to be replicable in other disadvantaged areas.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance

(a) Bank Performance in Ensuring Quality at Entry

Rating: Rating of Bank’s Performance at Entry: Moderately Unsatisfactory 78. As discussed, there were several weaknesses in the PAD’s design and risk assessment, and most of these should have been foreseen more clearly during design, especially in light of the Bank’s experience in Aceh. The ambitious number and complexity of the envisaged sub-projects made for a risky project even after risk mitigation measures. The impact was to slow implementation, reduce the scope and extent of SIE activities and to limit outcomes. In particular, much more time should have been allowed for implementation, or the PDO should have been less ambitious in the number and range of activities to be covered by the SIEs. There are also issues concerning the decision at design to delegate the setting of targets to the PMC and SIEs. However, on balance the decision was reasonable because of the weak analytical basis for any target-setting at that point. Please refer to paragraph 23 for more detail.

31 The government, in its (draft) Evaluation Report, goes further. It asserts that very few of the 150 supported coops and other groups (italics added) were ready by project-end to continue on their own. In its discussion of ways to address the sustainability issues, all options entail further support in one form or another.

21

79. There was no Quality of Entry rating for this project. (b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Rating of Bank’s Implementation Support: Moderately Satisfactory 80. Positive Areas. The complexity and flexibility provided at entry required close project supervision and oversight during implementation. The Bank responded proactively through provision of intensive implementation support, particularly in the areas of financial management and procurement. For example:

Substantial off- and on-site support and capacity building provided to PMU/PMC, including for procurement, financial management, environmental and social safeguards. There had been a number of financial management and procurement support missions in addition to regular project supervision mission. These missions aimed to build capacity to address the weaknesses of the SIEs on fiduciary aspects. For example, the Bank team helped reviewed working capital activities under the revolving funds, and assisted the transfer of revolving funds to the cooperatives under the project.

Bank oversight of procurement, financial management and audits appears to have performed well in emphasizing the importance of transparency and mitigation of fraud and corruption risks; and,

Ultimately, the Bank successfully advocated for a project extension. The extension, combined with a major push near project end, allowed completion of the main project interventions.

81. Negatives. There were other areas in which the Bank’s response could have been more effective, namely:

Bank decisions were not always consistent across SIEs or widely understood (on procurement, revolving funds, etc.), especially early in the project’s lifetime;

Financial Management Assessments (FMAs) of the SIEs were not conducted early enough and thoroughly enough to be effective; and

Bank procurement systems could have been more flexible in adapting to local conditions, e.g., allowing community contracting for non-typical goods.

82. There is also a question as to whether the Bank missed an opportunity (Section 1.3) to restructure the PDO and the intermediate indicators, essentially to scale-back expectations as implementation difficulties began to set in. Strictly speaking, this is probably correct, at least as far as the intermediate indicators are concerned. However the issue was fully discussed with the relevant MDF Technical Working Group and the revisions reflected the consensus at that time. Consequently, this ICR takes the view that this does not reflect poorly on the quality of the Bank’s implementation support. (c) Justification of Rating for Overall Bank Performance Rating: Rating of Bank’s Overall Performance: Moderately Unsatisfactory

22

83. Based upon the factors discussed above, the Bank’s overall performance is rated as Moderately Unsatisfactory.

5.2 Borrower Performance (a) Government Performance Rating: Rating of Governments’ Overall Performance: Moderately Unsatisfactory

84. Positive Factors. Governments at all levels displayed strong ownership and commitment to achieving the project’s development objective. Both Bappeda and KPDT played key roles during project implementation, and Bappenas intervened at certain times to resolve implementation issues. Also, there was compliance with fiduciary covenants; adequate beneficiary/stakeholder consultations; and effective engagement through the SIEs. In general, governments were responsive to donor questions and in facilitating donor visits. 85. Negative Factors. Readiness for implementation was highly questionable at the outset of the project. There were long delays due to unavailability of funding (the DIPA process); a lengthy process was needed to provide a legal basis for international NGOs to implement SIEs; there was limited institutional capacity to implement fiduciary requirements; there was heavy reliance on consultants; and key staff for the Secretariat and PMU was not appointed on a timely basis. 86. During implementation, governments stood ready to help resolve issues, but actual resolution was sometimes slow, aggravated by ineffective coordination among between implementing agencies. The DIPA process delayed implementation and slow resolution of issues strained relations between Bappeda/PMU and the SIEs. This lasted throughout the project. 87. At project end, the governments’ transition arrangements to support sustainability of the investments remained uncertain. (b) Implementing Agency or Agencies Performance Rating: Rating of KPDT’s, BAPPPEDA Aceh’s and SIEs Overall Performance: Moderately Unsatisfactory

88. Many of the comments immediately above apply directly to KPDT and Bappeda (because they were the main counterpart agencies) and those comments will not be repeated here. This section only adds some institution-specific commentary. 89. KPDT was highly committed to the project development objective and was proactive in monitoring and resolving implementation issues within its mandate. KPDT also took the lead in coordinating with Bappeda/PMU. However, some tasks assigned to KPDT had to be reallocated to the PMU because KPDT lacked the necessary capacity.

23

90. Bappeda’s commitment to the development objective was initially strong and it remains so for the Government of Aceh. Also, Interim Financial Reports (IFRs) were always provided on a timely basis and in good condition. However burdensome administrative requirements overwhelmed the PMU’s capacity and resulted in implementation delays at the sub-project level.

91. Regarding capacity development at these institutions, KPDT’s quantitative target was missed by a significant margin (see Annex 10). By contrast Bappeda’s target was exceeded (Annex 10), but the full potential for capacity building among Bappeda Aceh staff has not been realized.

92. Performance was marked by highly uneven results across the SIEs (Annex 11). In general, the NGOs brought good international practices, especially methodologies for SME development and a focus on governance. Also, some SIEs already had strong internal management systems and adapted well to the demands of the project; others could not. The only indicator defined in the PAD was in this area and, viewed from a results perspective, it was not achieved (5 SIEs rated ‘Satisfactory’ or above, versus a target of 6; Sections 2.3 and 3.2).

93. The SIEs were highly motivated to launch their activities and, by design, they actively engaged with project stakeholders. However, their administrative capacity (i.e., especially procurement and financial management) was often inadequate. This caused significant delays. Also, some project components turned out to be too ambitious for project duration. In implementation, problem resolution often suffered from burdensome oversight requirements that distracted the SIEs from core activities. This lasted throughout the project and diluted results.

94. By project-end, some SIEs had secured financing to continue operating, at least for a year or so. This is a helpful step, but highlights the issue of sustainability (c) Justification of Rating for Overall Borrower Performance Rating: Rating of Overall Borrower Performance: Moderately Unsatisfactory

95. Based upon the discussion in the two previous sub-sections, the overall borrower performance is rated as Moderately Unsatisfactory.

6. Lessons Learned 96. There are a large number of lessons learned from this project. They are especially applicable for any effort to replicate this project in Indonesia. 97. Post-disaster and/or Post-conflict Operations. These are a valuable type of operation and highly likely to recur. The EDFF experience indicates that planning for these activities should start at a relatively early stage, as an integral part of reconstruction activities.

98. GoI-NGO Collaboration. This project demonstrated that GoI collaboration with non-state actors is viable strategy, but it needs a more effective implementation model. In particular

24

it requires careful screening and selection of the NGOs; a long-term perspective; a willingness to allow the NGOs to apply their own approaches; and strong coordination with government.

99. More Time Needed or More Limited Objectives. The project’s compressed implementation period limited the project’s scope, activities, outputs and outcomes. A project lifetime of 5-7 years looks more appropriate than the 3 1/2 years provided for in the PAD. The risk of lengthy delays is considered high sustainability at risk for overly ambitious projects. This is particularly the case when working with counterparts and implementing entities that are likely to require considerable capacity development.

100. Complexity. In a complex setting with such weak institutional capacity, implementation arrangements should be simpler than EDFF’s. It would have been more effective to work with 3-4 larger projects, rather than 8 smaller ones, because most are going to need intensive support. Effectively, a small pilot project would be desirable, to learn from the experience and scale-up with a follow-on operation, if appropriate.

101. Replication. While the project procedures and implementation arrangements were too complex in their totality, EDFF offered a number of features that could be replicable for the government. Foremost, the model of the local government working through NGOs was groundbreaking in Aceh and demonstrated to have a high potential to bring in new and innovative methodologies and approaches for supporting SMEs through livelihood projects. In addition, the project also generated some good practices at the farm level to be replicated. Most notable is how to disseminate technical skills that increase productivity, quality and quantity of key crops. In particular, Farmer Field Schools (FFSs) focused on training all farmers in cocoa rehabilitation, supported by provision of training, materials and equipment. This approach of directly training all farmer beneficiaries was found to be superior to training-of-trainers, who did not always pass on the techniques that they had learned.

102. Procurement. The procurement capacity constraints of NGOs require particular attention, including steps for mitigating governance risks in procurement. These should be implemented before effectiveness of the project. During implementation, the performance of the PMC should be closely monitored so as to ensure that the designed assistance is provided. Also, procurement arrangements (particularly for non-typical goods) should be carefully proposed in the procurement plan so as to be suitable for the local conditions. 103. Dual Oversight Systems. Operating a project ‘on-budget and on-treasury’ looks likely to entail financial oversight from GoI/BPKP. Some binding, up-front agreement is needed–among all parties–on (relatively simple) operative financial management guidelines and on the means for resolving unexpected differences of view. This is particularly important in the context for the Bank’s commitment to working within local government systems.

104. The Model Approach. EDFF’s most successful approach combined the following ingredients: intensive capacity building for a large cross-section of farmers and SMEs; low-key investment support to enable the farmers and SMEs to conform to market requirements; and excellent networking with larger buyers to provide strong, market-driven financial incentives for the farmers and SMEs.

25

105. Sub-Project Selection. This is likely to be a difficult, complicated process, but it can be made easier by:

Commencing capacity building as soon as possible in the agency responsible for sub-project selection. This needs to happen prior to undertaking the selection process, possibly even before project effectiveness;

Allowing more time for wider dissemination of project information (i.e., ‘socialization’) prior to the call for project proposals;

Sufficient narrow, technical skills in the selection committee to cover the envisaged range of proposals;

Local experts on the selection committee; A wide range of selection criteria, including for example, financial management capacity

and expertise in procurement; and On-site visits and interviews to confirm applicants’ qualifications, especially those who

have not worked with GoI and/or WB previously.

26

7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors

(a) Grantee/Implementing agencies N.A. (b) Cofinanciers/Donors N.A. (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) N.A.

27

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate

(USD million)

Actual/Latest

Estimate (USD

million)

Percentage of

Appraisal

Component 1–Financing of Sub-Projects 44.50 37.26

2.76* 89%

Component 2–Support for the Supervision of the EDFF, Planning, Design, Selection of Sub-Projects, Supervision and Implementation Oversight

5.50 4.45 80.9%

Total Baseline Cost 50.00 44.47 88.9% Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00

Total Project Costs 50.00 44.47 88.9% Total Financing Required 50.00 44.47 88.9%

*This amount is an unaccounted for residual in MDF transfers, and appeared to be held in the Special Account at project closure, awaiting finalization of transfers and reimbursements to SIEs after Nov. 27, 2012. See the text discussion on p. 9 of the Aceh EDFF Final Report.

(b) Financing

Source of Funds Type of

Cofinancing

Appraisal

Estimate

(USD million)

Actual/Latest

Estimate

(USD million)

Percentage of

Appraisal

GoI 2.85 1.16** 40.7% MDF 50.00 44.47 88.9%

28

Annex 2. Outputs by Component

Component/sub-component Main Outputs/ Outcomes

Component I: Financing of Sub-projects Actionaid Australia with

Yayasan KEUMANG

#1: Increased Cocoa production

-Baseline studies in 3 districts. -Established demonstration project. -Training for 30 govt. extension workers & 4,500 farmers. -Provision of seedlings, fertilizer, tools & traps.

#2: Coops established/strengthened -Established 7 new primary coops and 1 secondary coop. -Training for 108 coop members; study tour for 9 coop officials. -Training for 20 warehouse staff assistants in processing cocoa.

#3: Improve Market Linkages Along the Value Chain and Promote Aceh Cacao -Some training for govt. and SIE staff.

Aceh Development Fund

with

#1: Ice block factory

Annisa, BIMA &

UNSYIAH -Coop established; factory operational & rented out.

#2: Iodized salt factory -2 coops established; factory operational. #3: Fish processing factory

-3 factories operational in traditional drying methods with greater efficiency & hygiene.

#4: Liquid smoke factory - Construction almost complete at project-end.

Canadian Cooperative

Association with PASKA

#1: Capacity building for cooperatives & SMEs -Training for marine fishermen & coops; provided working capital, offices & trucks. -Training in freshwater aquaculture for coops & fish farmers; provided loans & nursery. -Training for coops & farmers in rice intensification; provided demo plots, storage facilities & working capital. -Training & provision of machinery for emping production & marketing.

#2: Develop central market facility for KOPEMAS -Established marketing facility, including building, equipment & recruitment of

personnel. Caritas Czech Republic #1: Improved agricultural & processing technology for nilam (an aromatic herb) -Farmer Field Schools formed; provided training to 1,929 farmers (66% of target).

-51 demo plots established across 4 districts. -34 new distillation kettles installed and some 1,250 beneficiaries trained.

#2: Strengthened Cooperatives -Training for 355 cooperative members and 34 board members. -Facilitated an MoU with Bank BRI and distribution of revolving fund to coops.

#3: Improved marketing linkages -Facilitated relationships with buyers, including by workshops, buyer visits &

29

expos. International Organisation

for Migration #1: Capacity building for farmers & coops -50 ag extension workers trained in organization, good ag practices and management. -Community nurseries established in 50 locations & key input materials provided. -1,821 coffee farmers trained in land preparation, coffee production, pest & disease management, post-harvest management, quality testing and marketing. -10 coops received equipment & training in admin, finance, coffee production, cooperative management, quality testing and Fair Trade standards. -Warehouse Receipt System set-up for credit to farmers. -Build & strengthen the Arabica Coffee Network.

#2: Capacity building for coffee businesses - 78 SMEs received matching grants for storage facilities, equipment & machinery. - Over 200 SMEs and cooperatives trained in business plan development, accounting, business ethics and environmental issues.

#3: Strengthened links with buyers & exporters -Research nursery established with the Gajah Putih University. -Established an online Arabica trading platform.

Islamic Relief Worldwide

with Aceh Ocean Coral

#1: Introducing grouper marine culture -Rehabilitated the govt. grouper hatchery.

#2: Improved gear for fishermen -Provided 970 fishermen with improved nets, fishing gear and boat repairs.

#3: Improved Infrastructure -Built a jetty and 116 fish cages; rehabilitated 2 ice factories; and strengthened 2 cooperatives.

#4: Improved Market Access -Training & equipment for 59 women to improve use of leftover catch. -Disseminated info and organized meetings with local govt. officials, related agencies & private sector.

Muslim Aid Indonesia #1: Capacity building of farmers & govt. officials

-Training for 1,968 persons in livestock production and health management, fodder production and management, cattle reproduction, market access, group management and revolving funds, composting and biogas production.

#2: Introduction of new technologies -279 artificial inseminations of cattle along with supply of medicine and concentrate (50% of target). -Biogas production facilities installed and ready to operate.

#3: Provision of new barns & equipment -36 cattle barns constructed; 21 drilling wells constructed; 15 ring wells completed. -1,080 bulls and 360 heifers delivered (100% of target). -98.8 hectares of grazing area established. -36 composting units were established.

Swisscontact #1: Cocoa forum & master plans

-Facilitated establishment of the Aceh Cocoa Forum to promote transparent and

30

fair cocoa trade in Aceh. -Completed 5 District Cocoa Master Plans (DCMP); being used by District govts.

#2: Cocoa field work -5 District Cocoa Clinics completed. These comprise nurseries; a cleaning center; composting; some clonal gardens, a side grafting demo plot; and post‐harvest equipment. -Training of Trainers for 403 farmers and government extension workers in cultivation, seedling growing, garden maintenance, side grafting, top grafting, sanitation, composting, etc. -Rehabilitation of 1,212,935 cocoa trees in 1,516 hectares (76% of target). -112 smallholder cocoa enterprises organized and supported (187% of target); trained in farm organization and management; marketing; and composting and seedling business. Also supported with 112 nurseries, compost choppers, and other equipment.

#3: Market access & finance - One day events to provide for discussion on marketing and access to finance/credit with banks, SMEs and SCEs as well as GOI representatives. 141 people participated, comprising cocoa farmers, traders, govt. reps & Bank staff.

Component II: Project

Management and TA for

Bappeda and KPDT

Finance Unit #1: Guidelines for financial management, internal and external audits and

reporting #2: Training of SIE financial staff #3: Visits to SIEs to assess and assist staff and procedures #4: Review and processing of requests for budget revisions #5: Review and processing of advance requests #6: Monitoring, review, (re-)verification and processing of monthly reports #7: Other activities as deemed necessary including follow up of BPKP findings,

World Bank visits, External Audit reports, etc. Procurement Unit #1: One-day training workshop on procurement, attended by all procurement

officers from 8 SIEs #2: Assistance and guidance on procurement to all SIEs until WB NOL obtained #3: Assistance and guidance to all SIEs on bidding documents and Bid Evaluation

Reports on Shopping for NCB packages Environment Unit #1: Ensure that SIE activities have no negative environmental impacts

#2: Ensure that land uses are in line with the Land Acquisition Protocol of the EDFF Project

#3: Ensure that the EDFF Project shall not cause negative impacts to the lives and livelihood of vulnerable groups or communities

#4: Ensure implementation of SIE activities in line with the World Bank Anti-Corruption Standards

M&E Unit #1: Design, implementation & maintenance of a system for analysis of progress, performance and outcomes at sub-project and project levels #2: Field verification of progress reported by the SIEs in all 17 districts #3: PMU Technical Monitoring & Evaluation Manual #4: Operation of Management Info System and Geographical Information System

31

Training & Communication #1: Capacity building by the PMC for the PMU and GoA -12 workshops on, among others, procurement, financial management safeguards

and project management. #2: Capacity building by the PMC for the SIEs -Three workshops on M&E, procurement and credit & tax issues. #3: Ensuring due attention to gender issues and visibility of EDFF

32

Annex 3. Economic and Financial Analysis (including assumptions in the analysis)

1. This Annex revisits the PAD’s Economic and Financial Analysis (Annex 10 of the PAD) with the objective of assessing the higher-level outcomes of EDFF against the PAD’s pre-determined standard of economic returns. 2. The PAD convincingly argued that it was not possible to do standard cost-benefit analysis for EDFF, because the sub-projects and their activities were unidentified at that stage. Instead, in quantitative terms, it looked at the increase in Aceh’s economy that would be required to achieve the minimum Economic Rate of Return (ERR) that the World Bank requires in its investment projects, i.e. 10-12%. In qualitative terms, the PAD also noted that analysis would focus on the soundness of the approach undertaken, to support the strategies of the provincial authorities to promote sustainable economic growth in the province and to improve the capacity of Bappeda and other relevant entities to select, implement and monitor sub-projects that support economic development in Aceh. 3. Regarding the quantitative analysis of incremental growth necessary to achieve the Bank’s 10-12% threshold, the PAD concluded that an increase in GDP growth of .03% during the project period and a sustained boost of .01% per year thereafter, through 2020 would be sufficient. This implies average GDP growth of 3.27% from 2009-2012. On a sectoral basis, the PAD postulated that maintaining agricultural growth at 5%, would raise overall GDP growth to 3.6%, well above the rate necessary for the project to be economically viable. Table 3. GDP Growth in Aceh, with Selected Components

(2000 prices, in %)

2006 2007 2008 2009 2010

2011

Avg.

2010-

11

Total GDP 1.6 -2.4 -5.2 -5.5 2.8 5.0 3.9 o/w: Agriculture 1.5 3.6 0.8 2.6 5.0 5.5 5.3 o/w: Farm Food Crops 1.1 4.7 -2.9 3.7 7.9 6.9 7.4 Farm Non-food

Crops 0.3 2.6 4.1 3.6 3.1 4.6 3.8

Fisheries 2.7 4.5 2.5 2.9 3.8 3.5 3.7 4. During the first official year of the project, 2009, Aceh’s economy continued to contract, by 5.5%. But as described in the main text, virtually nothing was accomplished on the ground by the project during that year. Consequently, analysis here focuses on the period 2010-11, which is an exceedingly short period from which to draw conclusions. 5. Average GDP growth during the short, 2010-11 period was 3.9%, well above the required GDP growth rate of 3.27%, foreseen as necessary by the PAD. Likewise, growth in the agricultural sector was 5.3%, a little above the 5% mentioned in the PAD as necessary for economic viability. Looking at sub-sectoral activity, good growth ensued in the project’s three

33

main sub-sectors, namely Food Crops, Non-Food Crops and possibly Fisheries (see the accompanying Table). 6. There is, of course, a major issue of attribution here, because achieving growth of 3.9% can hardly be attributed solely to EDFF. Further crosschecks are necessary, to ensure that EDFF has made a worthwhile contribution to this stronger growth. 7. Three simple crosschecks on efficiency are considered here. The first is the estimate of 4,400 jobs created by EDFF. This is equivalent to almost 1/4 of 1% of total employment in Aceh in 2011,32 which is surprisingly large for an expected gain in GDP of roughly 1/10th of 1% over the project lifetime. It implies an employment elasticity of about 2.5 relative to the postulated, minimum change in GDP. 8. The second crosscheck is the estimated number of total EDFF beneficiaries, namely 36,568 persons. This is equivalent to more than ¾ of 1% of total population in Aceh in 2010. Again, this is surprisingly high, but in line with the estimate of ‘jobs created,’ considering the ‘beneficiaries’ is a much broader concept than ‘new jobs.’ 9. As for the third cross-check on efficiency, the project cost per job is roughly estimated at USD10,700,33 which looks very expensive, especially considering sustainability issues. 10. Turning to the qualitative issues mentioned in the PAD, the record is mixed. On the positive side, capacity building at Bappeda has been successful in project selection and in managing projects like EDFF. However even if BAPEDDA can manage a project like EDFF, it is unclear whether these new skills are easily transferred to GoI projects and systems, and there is scant evidence of a shift towards a more results-oriented management style. Also, the commitment of KPDT and, to a lesser extent, Bappeda to capacity building was often questionable during EDFF. For example, permanently allocated resources were normally insufficient and only few officials participated in training sessions and monitoring visits. Likewise, the uptake of tools introduced by EDFF (like the Management Information System and impact monitoring) has been limited. In this context, it is notable that the government’s evaluation report makes no mention of benefits from the TA provided.

32 Source: Table 3.2.4 of ‘Statistik Indonesia 2012,’ the Indonesian National Bureau of Statistics. 33 That is, US$52.85 million less US$5.5 million (which went unspent) divided by 4,400.

34

Annex 4. Grant Preparation and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/

Specialty

Lending/Grant Preparation Said Fauzan Baabud Operations Officer EASFP Team member John Victor Bottini Consultant MNSUR Team member

Gillian M. Brown Senior Gender Specialist EASSO - HIS Team member

William Peter Mako Lead Private Sector Development Specialist MNSF1 Team member

Sylvia Njotomihardjo Program Assistant EACIF Team member – Logistics support

Thomas A. Rose Advisor FFSFP Task Team Leader (up to Dec. 2008)

Imad Saleh Lead Procurement Specialist EASR1 Procurement review

Andrew Daniel Sembel Environmental Specialist EASIS Environmental review

Unggul Suprayitno Sr. Financial Management Specialist EASFM FM review

Supervision/ICR Alexandra L. Drees-Gross Senior Financial Specialist FFSAB Task Team Leader Said Fauzan Baabud Operations Officer EASFP Team member

Enrique Blanco Armas Senior Economist AFTP1 Task Team Leader (up to Oct 2010)

Zhentu Liu Senior Procurement Specialist EASR2 Procurement review Enggar Prasetyaningsih Procurement Analyst EASR1 Procurement review Sri Hanizar Consultant EASR1 Procurement review Christina I. Donna Financial Management Specialist EASFM FM review I Gusti Ngurah Wijaya Kusuma Financial Management Analyst EASFM FM review

Vicki Diane Peterson Consultant EASFP Team member Dhonke Ridhong Kafi Consultant EASIS Team member Mr. Purwanto Anti-corruption EACIF Team member Virza S. Sasmitawidjaja Consultant EASIS Environmental

review Fahmi Wibawa Consultant EASFP Team member Lloyd R. Kenward Consultant EASIS ICR main author Nancy Chen Sr. Financial Specialist EASFP ICR Thang-Long Ton Economist EASPR ICR

35

Jiyoung Song Operations Analyst EASFP ICR

Michael Z. Figueroa Information Assistant EASFP ICR Logistics support

(b) Staff Time and Cost

Stage of Project Cycle

Staff Time and Cost

No. of staff weeks USD Thousands (including

travel and consultant costs)

Lending

FY08 23 40.30 FY09 53 222.98

Total: 76 263.28 Supervision/ICR

FY10 66 222.78 FY11 105 410.11 FY12 55 212.95 FY13 18 169.03

Total: 244 1,014.87

36

Annex 5. Beneficiary Survey Results (if any) N.A.

37

Annex 6. Stakeholder Workshop Report and Results (if any)

N.A.

38

Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR 1. As part of Aceh Rehabilitation and Reconstruction Blueprint, The EDFF main objective was to promote post-tsunami economic recovery and foster sustainable equitable long-term Aceh economic development. The Aceh economy was driven by the oil and gas fields in the East Coast, now in decline and with very little linkages to the rest of the economy. As tsunami reconstruction winds down, the construction boom linked to the reconstruction effort was coming to an end and with it the high growth rates experienced by certain sectors of the economy (e.g. construction, trade and transport) during the reconstruction period. This natural economic slowdown compounded by the negative impact of the global financial crisis, together with the fact that Aceh’s economy has not entirely recovered from the negative impact of the 30-year conflict. The reconstruction agency, the national and provincial authorities and many donors recognized the need to focus on putting Aceh’s economy on a more sustained and longer-term growth path with an effort to create more jobs.

2. The EDFF Project was designed to help in overcoming these constraints and to contribute to increased productivity and quality of produce in the key economic sectors, as well as to foster an improvement of all aspects of the value chain. This was done through the provision of financial support, economic infrastructure development and capacity building support to farmers/fishermen and their organizations as well as to other crucial players in the value chain of the targeted commodities.

3. The EDFF comprised two main components. Component one supported sub-projects that address critical issues affecting economic development in Aceh by contributing to building a more competitive and supportive business environment necessary to create broad based private sector job opportunities and growth in Aceh, targeting the poor and other vulnerable groups. Component Two was allocated for funding project management and capacity building for Government of Aceh (Project Management Unit) and the KPDT.

4. According to Grant Agreement between The World Bank and Government of Indonesia Project was designed for 36 month for implementation. However, effective implementation period reduced to 20 month because project preparation took more than expected in the project design. Since EDFF implementation mechanism was unique, Preparing SOP, sub project selection, revising proposal and Preparation sub-project implementation documents were time consuming.

5. The overall financial achievement in respect of the 50 million US$ EDFF Project budget is 44.46 million, which includes the consultant costs and incremental project cost, resulting in approximately US$ 5.5 million unspent. US $ 37.26 was transferred to Sub Project and US$ 4.45 was provided to the consultancy company. Most sub project managed to complete their activities and to reach between 77% and 100% of their targets. Overall, the Sub Project achieved around 92% of their physical and 85% of their financial targets as per approved sub-project proposal and budget. 6. With respect to the key result areas the following Intermediate outcomes were achieved:

39

Improving the business environment

the cocoa sector development plans guide the cocoa development planning in 5 districts; the forums and traders associations are currently operational and performing as expected; the revolving funds are currently being used and repayment rates are generally

satisfactory; matching grants provided proved to be very effective and attracted additional investment

from the private sector of approximately IDR 4 billion

Private sector support/employment generated

increased productivity due to adopted Best Agricultural Practices; higher prices because of improved quality for all commodities and value addition through

improved processing; improved marketing through established linkages between cooperatives/farmer groups

and markets.

Infrastructure development

7. Nearly all (93%) of the 586 constructed units are currently being utilized. The users are Project beneficiaries. Field monitoring findings indicate that the availability of the infrastructure has assisted people in carrying out economic activities and becoming more efficient in their businesses. It has also created additional labor to operate the various structures. The entire infrastructure that has been built belongs to the community through the established institutions supervised by the government. However, there are also some infrastructure works that have not been utilized yet. Because most construction works have been completed relatively recently, it is too early to assess the real effects and sustainability of these infrastructures.

Effective management of the Project

8. Because the kind of the project was first experience for government, ToR of consultant service was changed along the project implementation time line as needed. As a result of intensive coaching for sub projects, performance in financial management, procurement and safeguards improved in line with AEDFF procedures.

9. To some extend the rigidity government and World Bank administration regulations were detrimental the objective oriented management that is considered of crucial importance for the success of a unique project like EDFF. Also the uptake of tools developed, such as the Management Information System and methods for systematic performance and impact monitoring has been slow. Most Sub Implementing Entities has little knowledge and experience applying of the Government of Indonesia (GoI) and the World Bank regulations and procedures. As a result, this consumed a lot of time and resources for couching in order to comply with SOP.

10. In addition, the selected NGOs should have adequate expertise and management capacity to implement the livelihood project activities. Clearly, it was found out that NGOs which had

40

expertise and management capacity adequately were more successful in implementing and achieving project goals than those which did not have.

Achievement of Project Development Objective (PDO)

11. The Project contributed in various ways to the economic recovery and sustainable equitable long-term economic development in Aceh. Statistics shows that starting from year 2010 to 2012 Aceh economic is back to experience positive growth after 3 years contracted. In 2010 to 2012 with oil and gas Aceh economic growth reached to 2.36, 5.02 and 6.02 percent respectively. The economic growth coupled with poverty alleviation reduced poverty to 18 percent in 2012. 12. The support to primary producers has resulted in increased productivity and improved quality of Aceh’s main commodities. Farmers receive higher commodity prices and the collectors and wholesalers are also benefiting from higher quality products and more consistent supply from farmers. Some big traders are now aware of the higher quality of products from farmers.

13. The Project has contributed to employment creation through the establishment of new formal jobs, as well as through the introduction of more labor intensive agricultural production methods.

14. Through improved linkages to the market and value chain support for related small enterprises & businesses, more efficient marketing and higher prices through value–added processing for certain commodities can be observed. Some private sector investment is taking place, for example in the cocoa sector and for ice production. International export is still limited but for some commodities indications for increased regional trade to other parts of Indonesia can be found. Due to short project implementation and delays in the start up the overall effects on the economy are maybe still limited, but the foundations for economic development have been clearly laid. 15. The main concern is with the sustainability of the AEDFF investments that are handed over to the end beneficiaries, especially (pre-) cooperatives. It is very clear that due to the mentioned time constraints, the SIEs were unable to build sufficient capacity to guarantee this sustainability. Government support to continue the development in these key commodity sectors is mandatory to increase the chance for sustainability. Lessons learnt

16. Administrative burden such as finance and procurement management is the major constraint for achieving project goals. It was also observed that World Bank sometimes went too much into detail. The result has been that contracted parties were unable to finish their sub-projects timely, affecting the quantity, and at times the quality of planned activities. Procurement has been an enormous challenge. A major design flaw was the omission of community contracting and other flexible procurement mechanisms. This particularly affected the procurement of large quantities of life material for agricultural livelihood activities, such as

41

seedlings and breeding stock, as well as simple constructions, such as stables and floating fish cages. The large number of packages and variety of procurement methods, ranging from shopping to international tenders, proved extremely challenging for many SIEs, which delayed the implementation process as procurement took much longer than originally planned. Therefore when it comes to agricultural livelihood activities, there should be some flexibility in the procurement process for purchasing those kinds of materials.

17. In the early stages of the project implementation it was assumed that the guidelines for financial management had been clear and complete. But yet it was found out that during the audit by Badan Pemeriksa Keuangan (BPKP) there were several aspects that were not clear and could be interpreted in different ways by SIEs and BPKP. Many of the findings of BPKP went against this principle, but there was no clear coordination mechanism in place to resolve the differences and hold all government institutions to the original agreement. As a result many of the BPKP findings could not be resolved and several SIEs have to pay back considerable amounts of money. So it is hugely important to have clear and complete most needed regulations at the start of the project implementation.

18. A large number of institutions were involved in the monitoring and auditing of the sub-projects, however, without much communication, coordination and consistency in methodology and conclusions. This was not only disturbing for the SIEs and beneficiaries but it also put a lot of pressure on the PMU and Satker to provide the required information and data in various formats. 19. The AEDFF Project is a unique innovative concept/model that needs time for trial and error to be successful. However, given the ambitious set of objectives, the time available simply was insufficient, even with the 5 months extension. This clearly affected impact & sustainability.

42

Annex 8. Comments of MDF Secretariat (Cofinanciers) and Other

Partners/Stakeholders 1. These comments were prepared by the Secretariat of the Multi Donor Fund for Aceh and Nias (MDF) and do not necessarily represent the views of the MDF Steering Committee or donors. The MDF provided a grant of US$ 50 million for the Aceh Economic Development Financing Facility (EDFF), one of the MDF’s 23 projects for post-disaster recovery in Aceh and Nias.

2. The Multi Donor Fund for Aceh and Nias (MDF) was established to support the implementation of the Government of Indonesia’s (GOI’s) rehabilitation and reconstruction program after the December 2004 tsunami and subsequent March 2005 earthquake. At the request of the government, the World Bank serves as the trustee to administer the MDF which is in turn governed by a Steering Committee comprised of donors, GOI, and civil society representatives, with the United Nations and international non-governmental organizations participating as observers. The donors contributing to the MDF are: the European Union, the Netherlands, the United Kingdom, the World Bank, Sweden, Canada, Denmark, Norway, Germany, the Asian Development Bank, the United States of America, Belgium, Finland, New Zealand and Ireland. The MDF pools about US$655 million in grant resources provided by these 15 donors, an amount equivalent to about 10% of the overall reconstruction efforts. The MDF provided grants for implementation of 23 projects in five outcome areas: recovery of communities, recovery of large infrastructure and transport, strengthening governance and capacity building, supporting sustainable management of the environment, economic development and livelihoods, and providing support to enhance the overall reconstruction process. The Aceh EDFF project was the major vehicle in Aceh for achieving the MDF’s outcome area of supporting economic development and livelihoods.

3. The EDFF was a key element in the MDF’s third phase of activities, supporting a transition to economic development. Disasters often cause interruptions in development, and in many cases can reverse or set back economic growth. In addition to the devastation brought about by the tsunami and its direct impact on livelihoods, in Aceh the post-disaster reconstruction challenges were compounded by nearly 30 years of armed conflict that had stifled economic growth and interrupted development. Despite Aceh’s rich natural resources, the economy was under-performing and poverty rates were higher than the national average. Growth in key commodities had lagged as farmers had been afraid to go to their fields and both government and the private sector had been unable to provide extension and outreach. The post-tsunami recovery provided an opportunity not only to rebuild homes and infrastructure, but to help put Aceh back on its feet economically, which was deemed vital to preserving the fragile peace that was made several months after the tsunami. The MDF Steering Committee made supporting Aceh’s economic recovery a full priority of the reconstruction and provided its full support to the EDFF project. The $50 million allocated to the EDFF made it the fourth largest project in the MDF’s portfolio.

4. Immediately after the disaster Aceh experienced a reconstruction boom as people and resources poured in to rebuild. The phased approach to reconstruction adopted by the

43

Government of Indonesia and the MDF took this reconstruction boom into account. The EDFF was designed to rejuvenate Aceh’s traditional key economic sectors and to help facilitate a return to work in these sectors once the jobs created by the construction boom had wound down.

5. The innovative project design of EDFF was intended to both strengthen key economic sectors and to build local capacity to support economic growth. As a result, the EDFF project had a high degree of local government control and ownership, more than perhaps any other MDF project. EDFF introduced an innovative project implementation modality through which international and local NGOs executed programs in conjunction with the local government through an on-budget mechanism. The NGOs added value by bringing their expertise in implementing successful livelihood programs to this partnership. At the national government level, the Ministry for Disadvantaged Areas as the main implementing agency worked closely with the provincial government planning agency, Bappeda, to support sub-project implementing entities (SIEs), the NGOs, to ensure that all program activities were aligned with local government program priorities in all selected districts. This design is in keeping with the spirit of the Jakarta Commitment, an agreement that donors and development partners’ programs should be in line with government priorities and use country systems, wherever possible. The project also provided technical assistance and support to the provincial Bappeda, from program planning and design to monitoring and supervision. The Project Management Unit (PMU) was managed by provincial Bappeda team and supported by a professional management consultant. These new institutional arrangements took time to put in place and required joint stakeholder efforts at times to resolve issues and unblock bottlenecks, but they also built partnerships between government and NGOs and institutional capacity that will be of long-term benefit to the government of Aceh in planning for future economic development. Moreover, EDFF offered huge lessons and most importantly direct and real experiences for provincial government managing the Bank program implemented by various NGOs.

6. However, significant delays in project start up and implementation due to issues in setting the innovative institutional arrangements in place meant that the sub-project implementing entities were left with a much shorter time to implement their projects than had originally been planned. The MDF’s closing date was non-negotiable as donors had clearly outlined the time limits in their agreements and further extension of the project was simply not possible. The MDF Steering Committee adjusted the implementation period to the maximum amount of time possible, to November 2012, in order to give SIE’s as much time as possible to reach their targets. Under these circumstances, the NGOs, for the most part, performed exceptionally well. Despite the truncated implementation time, most of the projects were able to meet their objectives, and complete activities by the closing date.

7. The results achieved by the project within this timeframe are impressive in terms of strengthened organizations such as cooperatives and farmers groups, new technical skills taught and connecting producers to markets, increased production and incomes in a variety of agricultural sectors including coffee, cocoa, livestock, fisheries, rice, and food processing. There are significant risks for the sustainability of some of these results since the sub-projects often did not have time to support capacity building to the extent originally intended, but nevertheless outcomes from the projects’ efforts are generally good. Aceh’s cocoa and coffee sectors, in particular, are much stronger now as a result of the EDFF’s efforts, and are on strong footing for

44

continued development. In other sectors cases of local successes are evident and a range of good practices have been developed and disseminated among farmers and other small producers that will have lasting impact beyond the project.

8. The results across sub-projects varied with the capacity of the NGOs selected, and with their partnership arrangements. Lessons learned from this experience is that many very capable, accountable and experienced NGOs, both local and international, may not have the financial management and procurement capacity to meet the government and World Bank requirements in these areas. Thus future project designs would need to take this into account and not expect that NGOs can immediately begin implementing projects to these standards. Strong partnerships and clearly defined roles, especially between local and international NGO partners, are also important factors for success in this approach. Despite prior agreements, government financial requirements often took precedence over World Bank requirements, resulting in financial exposure for the implementing NGOs.

9. Due to many of these challenges resulting from the EDFF’s innovative institutional arrangements, the project was at times rated as a problem project by the World Bank and ratings on both Bank and borrower performance have not been satisfactory. In terms of results achieved, however, especially given the short implementation timeframe, the MDF Secretariat and donors have considered the project to be a success. This was in large part due to the Bank’s strong supervision of the project, made possible through additional supervision resources which were provided for all MDF projects given the implementation challenges they faced.

10. The motto of the overall Aceh and Nias recovery effort was “build back better.” Post-disaster situations provide a window for doing things differently that can often lead to positive outcomes. While there are lessons to be learned from some of the challenges the EDFF project faced and how it addressed them in terms of project design, from the perspective of the MDF Secretariat, avoiding innovation in post-disaster context should not be one of these lessons. Innovations are often critically important in complex situations where standard approaches don’t work; the lesson perhaps is that multiple innovations need to be managed carefully so that the challenges in totality do not overwhelm the system.

11. Nevertheless, sustainability remains a risk for the EDFF, especially in terms of local governments’ capacity to continue providing support to Nias’ farmers beyond the project’s end. One of the lessons learned by the MDF has been that capacity building takes time and is often in conflict with the need for urgency in a post-disaster context. However, given the long preparation time and truncated period available for implementation before the end of the MDF’s mandate, the EDFF completed its activities and met its objectives, making an important contribution in helping Aceh transition from the post-disaster and post-conflict context to a path of longer-term sustainable development. The role of the Bank’s implementation support was critical to success, as was the focus and attention of the national executing agency, the Ministry for Disadvantaged Areas, for the project.

45

Annex 9. List of Supporting Documents 1. Project supervision Aides Memoir 2. Mid-Term Review report 3. Sub-project Proposals 4. Aceh EDFF Final Report 5. PMU/PMC Draft Evaluation Report 6. Sub-project Grant Agreements

Annex 10: Analysis of EDFF Results

46

Project Development Objective (PDO): To promote post-tsunami economic recovery and foster sustainable, equitable long-term

development in Aceh in line with GoI’s own plans for economic development.

PDO-Level Outcome Indicators Target Estimated Results

Comments

34

Indicator #1

Number of business constraints successfully alleviated in Aceh

Unspecified Not available

Indicator #2 Employment generated in enterprises supported

by the project. Unspecified 1,172,954 man-days per year or roughly 4,400 full-time jobs

Indicator #3 Number of users of public infrastructure financed

by the project. Unspecified 17,178 users Intermediate Level Indicators/Results Areas

Results Area

#1 Improving the Business Environment

Indicator i) Number of sector development plans developed in consultation with the private sector and adopted by local governments

5 Cocoa sector development plans completed & used in 5 districts Soft Indicator; post-PAD

Indicator ii) Number of public-private sector forums established or strengthened 4 4 Forums plus 1 conference

Output; post-PAD

Output; post-PAD Indicator iii) Number of recipients & value of micro-credits. Unspecified 109 institutions and Rp8.1 trillion Results Area

#2 Private Sector Support

Indicator i)

Average % increase in value of smallholder production of project beneficiaries. 10

Major productivity increases reported by commodity in Annex 23 of the Aceh EDFF Final Report

Post-PAD

Indicator ii) Employment created in supported enterprises Unspecified 1,172,954 man-days/year Output Indicator iii) Enterprises supported.

350 381

Output

Indicator iv) Farmers & fishermen supported

25,000 36,568 beneficiaries

Output Indicator v) Primary producers linked to markets

12,500 24,132 producers

Output; post-PAD

34 The PAD (p.47) explicitly notes the likely need to revisit the Results Framework (RF) once sub-projects had been selected. The RF was updated in 2011 in consultation with the MDF Technical Working Group.


47




Comments

34

Indicator vi) Number of producers trained in improved technology or marketing 25,000 36,568 beneficiaries

Training Output; post-PAD

Indicator

vii)

Number of SMEs trained in improved processing or marketing 110 380 SMEs, coops and groups

Training Output; post-PAD

Indicator

viii)

Number of producer groups established or strengthened 430 799 groups

Output; post-PAD

Indicator ix) Number of cooperatives established or strengthened 65 60

Output; post-PAD

Indicator x) Number of household members indirectly benefiting from project 100,000 109,007 members

Output; post-PAD

Results

Area #3 Public Infrastructure

Indicator i) Infrastructure built 250 586

Includes: cocoa clinics; rice and emping (a rice product) processing facilities; ice plant; drying units; cattle barns; wells; compost & biogas units; warehouse rehabilitation; coffee research center, fish hatchery; and a jetty.

Indicator ii) Infrastructure built being used by intended beneficiaries. 80% 93%

Indicator

iii) Infrastructure built having plans for sustainability 80% 70%

Post-PAD Results

Area #4 Success rate of sub-projects financed by the project

Indicator i) % of SIEs with demonstrated improved capacity in financial management, procurement and safeguards.

80% 100% Post-PAD

Indicator ii) Sub-projects successfully completed 75% (6) 62.5% (5): 1 Highly Satisfactory

2 Satisfactory 2 Moderately Satisfactory 2 Moderately

Unsatisfactory 1 Unsatisfactory

PDO-level Indicator (KPI) in text of PAD


48




Comments

34

Indicator iii) No. of SIE staff on project implementation full-time Unspecified 787 Output; post-PAD Indicator iv) No. of KPDT staff on project implementation full-time 5 3 part time Output

Indicator v) No. of Bappeda Aceh staff on project implementation full-time 5 7 permanent Output

Annex 11: Summary Performance of SIEs, by SIE

49

(SIEs in alphabetical order)

Name of SIE Main Activity

Results-

oriented Funds Main Achievements Main Problems

Rating

Actionaid Australia (AAA) Cocoa production & marketing MU 1/ Rp64.6b Quality & quantity of cocoa

production increased Conflict between partners; 6-9 months lost.

w/ local partner:

(USD7.0m) 10 cooperatives

established/strengthened Cocoa factory cancelled.

Yayasan KEUMANG

Training & inputs for farmers Value-added chain activities scaled back, consolidated with other SIE or cancelled.

Recovery in coops near project-end, but too late.

Delays due to procurement issues. Aceh Development Fund

(ADF)

Fish, salt & related industries MU 1/ Rp41.5b Ice Block factory & coop operating Procurement problems; slow approval of

NoL.

w/ local partners: Annisa,BIMA &

UNSYIAH

(USD4.5m) Salt factory coops established; factories to start-up in 12/2012

Printing & packaging unit for salt & fish cancelled.

Improved efficiency & hygiene of traditional fish drying methods

Fish processing scaled back due to time constrains & viability concerns.

Coops established for ice block, salt & fish operations Limited success with coops.

Liquid smoke factory almost complete at project end

Very limited outputs in marketing & export strengthening due slow salt & fish progress.

Canadian Cooperative

Association (CCA)

Fish, rice & emping (a rice product) MS 2/ Rp54.8b

Freshwater fisheries productivity up; costs down; more local sourcing. Two coops profitable

Long procurement process; limited local human resources. Burdensome monitoring & oversight.

w/ local partner: PASKA (USD6.0m) Rice productivity up & costs down in demo plots

Limited technical skills, including for value-chain analysis; too many commodities.

Emping production & revenues way up Coops not sustainable at project end.

Training & development of central market facility for KOPEMAS

Started-up in Apr 2011, but very slow progress. Not sustainable at end of project.


50



Results-


Rating Caritas Czech Republic

(CCR)

Nilam (an herb) cultivation, S Rp43.7b Land tillage increased 80%; yields

up 60% Commodity prices down, so reduced plantings.

no local partner processing & marketing (USD4.7m) Product quality up & qualified for

GI certification Reduced plantings & lack of kettles meant a major production shortfall.

Training coops in business skills & leadership

Procurement delays impacted all aspects of project.

Marketing & buyer workshops BPKP’s audit finds contradictory & unclear.

International Organisation

for Migration (IOM) Coffee cultivation, MS Rp34.0b

TA for farmers increased product quality. Nursery produces seedlings of better quality than govt.-supplied seedlings.

Burdensome procurement, environmental and social safeguards, monitoring and reporting systems.

no local partner production & marketing (USD3.7m) Training of coops & certification

with Fair Trade US. Slow decision-making processes.

Warehouse Receipt System operating; provides financing to farmers.

Warehouse Receipt System underutilized & does not cover its operating costs.

Attempts to establish auction market.

Coffee businesses skeptical of auction market.

TA & matching grants to SMEs, which attracted new private investment.

Islamic Relief Worldwide

(IRW) Marine fisheries HU Rp46.9b Grouper hatchery established Very low fingerling survival rate.

w/ local partner:

(USD5.1) Floating fish cages operating.

Fish cages plagued with problems. Similar platform project cancelled, and simpler devices substituted.

Aceh Ocean Coral

TA & equipment for good fisheries practices Efforts to strengthen coops unsuccessful.

Boat jetty built; ice factories rehab & upgraded.

Original plans for fish processing scaled back; some ice upgradings incomplete at project end.

Conflicts between IRW & AOC. Neither had requisite technical skills.


51



Results-


Rating

Muslim Aid Indonesia

(MIA)

Livestock (cattle) production & marketing S Rp33.2b

Provision of livestock, materials & equipment

Simple project, focusing on ex-combatants. Delays due to procurement & quality of livestock.

(USD3.6m)

Introduction of new technologies, e.g., biogas

Very low success in artificial insemination & bulls being sold at low prices.

TA to cattle farmers & govt. staff Operations not sustainable at project end.

Swisscontact Cocoa production & HS Rp65.4b

5 cocoa master plans completed & used. Coffee forum operating w/ SMS price info

Complex (transport, storage and timing) grafting arrangements may not be sustainable.

no local partner marketing

(USD7.1m) TA through District Cocoa Clinics

Farm rehabilitation yielded major increases in productivity & quality.

TA in seedling & compost production gives smallholders additional income.

MoU with KPDT and other private Sector Partners and Donors, to expand its cocoa program to 5 other provinces.

1 / Rating raised to MS, if financial issues, admin. management and reporting are included as performance criteria. 2/ Rating lowered to US, if financial issues, admin. management and reporting are included as performance criteria.

53

Annex 12. Analysis of Critical Risks at Appraisal and Risk Outcomes

Risk Risk

Rating

(before/after mitigation)

Mitigation measure Outcome

At Project Level Late budget releases cause delay

H/M Dialogue with the MOF Delays due to DIPAs & late release of funds were a continuing problem.

Inadequate capacity of the implementing agencies (Bappeda and KPDT)

S/M TA consultants will provide the required capacity and will help the GoA to establish PMU staffed by experienced technical, administrative, procurement and financial management staff as well as KPDT at the central government level.

Risk fully eventuated. TA consultants over-burdened.

Fiduciary Risks H/S Involvement of key government counterparts in the design of the project to increase ownership and apply local knowledge to mitigation measures. Operations Manual that sets clear rules understood by all stakeholders. Use of on-budget mechanism, allowing the local government to use its own institutions to supervise the allocation and use of funds. Inclusion of an agreed upon anti-corruption plan throughout the live of the project.

Ineffective mitigation measure. BPKP did buy key provisions of the Operations Manual. On-budget mechanism was not a mitigation measure. It created problems, because of requisite oversight from BPKP.

At Sub-Project Level Technical/design: delays in sub-project implementation, overly optimistic estimates of outcomes, underestimation of time to implement

M/M Most entities implementing sub-projects are likely to have already gained experience in implementing subprojects in Aceh, which should make their implementation plans more realistic. The team preparing the EDFF project is well aware of the need to monitor the projected outcomes and

Risk fully eventuated. Mitigation measure was unrealistic. Overly complex structure made the mitigation measure ineffective.

54

timelines, and will liaise with entities implementing sub-projects accordingly, managing also expectations from the government and donors’ sides.

Ability or willingness of entities implementing subprojects to implement according to World Bank policies and guidelines

H/S Many entities have approached the World Bank enquiring about EDFF financing, therefore the interest is high. PMU will have expertise to assist entities to implement according to World Bank guidelines. Initial discussions with some entities that have different policies and guidelines took place, indicating the willingness of these entities to adopt World Bank guidelines for the purpose of implementing activities financed through EDFF

Risk fully eventuated, and severity of the problem seriously underestimated. High interest and willingness to adopt WB guidelines were ineffective as a mitigation measure.

Weak procurement, environment and collusive practices in past projects

H/S To ensure transparency in the procurement process, enhanced procurement procedures are included in the Anti-Corruption Action Plan. Procurement training activities and a capacity assessment will also be conducted with consultant’s assistance if needed. Early on, PMU will rely on international consultants’ expertise on procurement issues to assess the procurement capacity of entities implementing subprojects as well as monitoring procurement processes. The procurement ability of the sub-project proponent will be considered as one of the sub-project selection criteria.

Procurement issues were a serious problem. Extent of required training was seriously underestimated. Very demanding of consultants. Unrealistic mitigation measure.

Sub-project contracts will not be executed to standards and costs as designed

S/M Most entities implementing sub-projects are likely to have entities gained experience in implementing subprojects in Aceh. Many entities have learned important lessons over the last three years both in terms of realistic planning (timelines, costs as well as outputs) and how to improve

Extent of problem seriously underestimated. Ineffective mitigation measure.

55

their ability to implement projects. In addition, improved bid documents and tight control by PMU are expected.

All sub-projects financed by EDFF will not be fully implemented by the time the project closes in June 2012

S/N Selection of sub-projects will be undertaken early in implementation and will strive to identify projects with the greatest likelihood of completion within the timeframe.

Unrealistic mitigation measure.

Lack of sustainability of sub-projects financed by EDFF

S/M Selection evaluation will require sub-projects to address sustainability of investments (e.g. through Commitment of local governments to provide funds for operations and maintenance as necessary or design exit strategy for the implementing party).

Ineffective mitigation measure.

Risk Rating - H (High Risk), S (Substantial Risk), M (Modest Risk), N (Negligible or Low Risk)

Kutanibong

Ceulala

Sibigo

Lasikin

Lewak

NasreheuInor

Kandang

Rundeng

P. BaleeTi. Nibong

Bakungan

Lhoknga

Lamno

Jantho

Keumala

Geumpang

Lutung

Jeuram

Lamie

Peureulak

Uwak

Lhok Sukon

Pangkalan-brandan

MeureuduTaupingraya

Cot Girek

Peunaron

Meulaboh

Takengon

Calang

Seulimeum

Sigli

Sabang

Bireun

Blang Pidie

Tapaktuan

Karang Baru

Blangkejeren

Singkil

Kutacane

Sinabang

Lhokseumawe

Langsa

Banda Aceh

I N D I A N O C E A N

S t r a i t o f Ma l a c c a

We

Breueh

Lasia

Babi Bangkaru

Tuangku

Simeulue

Nias

Pulau-pulauKokos

95°E 96°E

95°E 96°E

97°E 98°E

97°E

6°N 6°N

5°N 5°N

4°N

3°N

2°N

1°N 1°N

Area ofMap

I N D I A N O C E A N

75˚E 90˚E 105˚E 165˚E

0˚

15˚S

A U S T R A L I A

VANUATU

SOLOMONISLANDS

PAPUANEW GUINEA

NAURU

SINGAPORE

MALDIVES FEDERATED STATES OF MICRONESIAM A L A Y S I A

TIMOR-LESTE

BRUNEI

PALAU

SRI LANKA

INDIA

PHILIPPINESCAMBODIA

MARSHALLISLANDS

THAILANDVIETNAM

MYANMAR

Guam (U.S.)

NorthernMariana

Islands (U.S.)

New Caledonia (Fr.)

I N D O N E S I A

INDONESIARECONSTRUCTION OF ACEH LANDADMINISTRATION SYSTEM PROJECT

NATIONAL ROADS

PROVINCIAL ROADS

RAILROADS

MAIN CITIES AND TOWNS

KABUPATEN CAPITALS

PROVINCE CAPITAL

KABUPATEN BOUNDARIES

PROVINCE BOUNDARY

Th is map was produced by the Map Des ign Uni t o f The Wor ld Bank. The boundar ies , co lo rs , denominat ions and any other in format ionshown on th is map do not imply, on the par t o f The Wor ld BankGroup, any judgment on the lega l s tatus of any te r r i to r y, o r anyendorsement or acceptance of such boundar ies .

0 10 20 30 40

0 10 20 30 40 50 Miles

50 Kilometers

IBRD 35289

MA

RCH

2010