i. basic project datapcrv_… · the gum arabic project covered 11 localities in five geographical...

23
Project Completion Report Validation Revitalizing The Sudan Gum Arabic Production and Marketing Project Republic of The Sudan Date of validation by IOE: June 2017 I. Basic project data Approval (US$ '000) Actual (US$ '000) Region Near East , North Africa and Europe Total project costs 10,880 9,665 18 Country Republic of The Sudan IFAD loan and percentage of total 3,000 2,863 00 Loan number DSF-8043-SD Borrower 380 0 00 Type of project (subsector) Multi-Donor Trust Fund 7,000 6,802 18 Financing type Grant Beneficiaries 500 0 00 Lending terms * Date of approval 15/09/2009 Date of loan signature 03/11/2009 Date of effectiveness 03/11/2009 Other sources: Loan amendments Number of beneficiaries 198,895 direct and indirect beneficiaries in total and 236 GAPAs 23,895 direct and 175,000 indirect Loan closure extensions No cost extension approved on 07/05/2012 Country programme managers Rasha Omar Loan closing date 30/06/2015 Regional director(s) Mid-term review No MTR was conducted Project completion report reviewer Philippa Morgan IFAD loan disbursement at project completion (%) 2,863,272.00 (95.44%) Project completion report quality control panel Prashanth Kotturi Michael Carbon Date of the project completion report Source: World Bank Implementation Completion Report. * The grant was provided under the IFAD Debt Sustainability Framework, which classifies The Sudan as a “red light” country, therefore, making it eligible for 100 per cent grant financing. The Republic of The Sudan agreed to sign a “Subsidiary Agreement” with the Forestry National Cooperation and to be subjected to annual audits undertaken by the World Bank. .

Upload: others

Post on 19-Apr-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

Project Completion Report Validation

Revitalizing The Sudan Gum Arabic Production and Marketing Project

Republic of The Sudan

Date of validation by IOE: June 2017

I. Basic project data

Approval (US$ '000) Actual (US$ '000)

Region

Near East , North Africa and Europe Total project costs 10,880 9,665 18

Country Republic of The Sudan

IFAD loan and percentage of total 3,000 2,863 00

Loan number DSF-8043-SD Borrower 380 0 00

Type of project (subsector)

Multi-Donor Trust Fund 7,000 6,802 18

Financing type Grant Beneficiaries 500 0 00

Lending terms*

Date of approval 15/09/2009

Date of loan signature 03/11/2009

Date of effectiveness 03/11/2009 Other sources:

Loan amendments

Number of beneficiaries

198,895 direct and indirect beneficiaries in

total and 236 GAPAs 23,895 direct and

175,000 indirect

Loan closure extensions

No cost extension

approved on 07/05/2012

Country programme managers Rasha Omar Loan closing date 30/06/2015

Regional director(s) Mid-term review

No MTR was conducted

Project completion report reviewer

Philippa Morgan

IFAD loan disbursement at project completion (%) 2,863,272.00 (95.44%)

Project completion report quality control panel

Prashanth Kotturi

Michael Carbon

Date of the project completion report

Source: World Bank Implementation Completion Report.

* The grant was provided under the IFAD Debt Sustainability Framework, which classifies The Sudan as a “red light” country, therefore, making it eligible for 100 per cent grant financing. The Republic of The Sudan agreed to sign a “Subsidiary Agreement” with the Forestry National Cooperation and to be subjected to annual audits undertaken by the World Bank.

.

Page 2: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

2

II. Project outline 1. Introduction. Revitalizing The Sudan Gum Arabic Production and Marketing

Project (henceforth “the Gum Arabic Project”) in the Republic of The Sudan was

approved by IFAD Executive Board on 15 September 2009. The IFAD loan

agreement was signed on 3 November 2009 and became effective on the same

date. The Gum Arabic Project was extended by one year past the original closing

date of 30 June 2014 to 30 June 2015. The project was executed in six years and

the grant was closed on 30 December 2016.

2. Project area. The Gum Arabic Project covered 11 localities in five geographical

states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

Nile. Gum Arabic has long been a strategic agricultural commodity for Sudan and

the country has historically been the world’s leading producer; the majority of gum

production is sourced from smallholders located within isolated rural communities

in the gum belt. However, in 2009, the sector experienced a steep decline in

production. Gum Arabic farmers suffered from a series of challenges in the sector

including erratic rainfall, degraded soils, low use of productivity-enhancing

technologies, limited knowledge and poor access to rural financial services.

Increased conflict among pastoralists and the secession of the South resulting in a

decrease of available agricultural land also led to a decline in production. At an

average rural poverty rate of 57.6 per cent1, one million poor Sudanese households

rely on revenue generated from gum arabic production2, representing 50 per cent

of the total cash income for households. Between 2007 and 2011, the Government

of Sudan put in place the Agriculture Revitalization Plan which allocated SDG 10

billion for the agriculture sector. Furthermore, in June 2009, the President issued a

decree declaring the liberalization of the production, processing, marketing and

export of gum arabic3. A combination of these factors resulted in a niche for policy

reform in the gum arabic sector that could benefit and facilitate higher yields to

Sudanese gum arabic producers.

3. Project goal, objectives and components. The overall objective of the project

was to “increase the production and income of small-scale gum producers in

selected areas of the gum belt through improved performance of production and

marketing systems”. The project was comprised of three components:

Component 1 – Gum Arabic sector reform and support (US$1.57M at

design; US$0.79 at completion): the main objective was to create equal

opportunity and competition among all market actors through Gum Arabic

sector reform and support. Key activities under this component included

studies to identify barriers to domestic trade and investments, an analysis of

the world market and demand structures, and the identification of new market

opportunities. The project was to establish a gum arabic market information

system to address the lack of market information available to gum arabic

producers, to enable them to negotiate fair prices for their gum. In addition,

matching grants were allocated to support capacity development in

marketing, product promotion and research.

Component 2 – Institutional and capacity-building support to Gum

Arabic Producers Associations (GAPAs) (US$6.15M at design; US$5.59M

at completion) with the objective of improving incomes through investments

in capacity-building, productive infrastructure and inputs. Specific activities

under this component included capacity building activities to strengthen the

management and skills of GAPAs in savings, credit, marketing, quality control

and production. Revolving savings and credit schemes were set up to support

newly established GAPAs.

1 COSOP Mid-term Review, P.13

2 President’s Report, P.9

3 President’s Report, P.9

Page 3: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

3

Component 3 - Project management and supervision (US$2.28M at

design; US$3.29M at completion), including M&E activities and related

baseline studies, mid-term review and final project assessments. The project

provided training and logistical support to the Forest National Cooperation

(FNC), the Project Coordination Unit (PCU) and the Locality Project

Coordination Units under this component.

4. Target group. The Project selected small-scale gum arabic producers who were

organised into GAPAs located in the rain fed gum arabic belt. Eight localities were

selected under phase one of implementation with a further three localities to be

included during phase two under the IFAD grant. The localities were determined

based on the presence of productive gum arabic trees and gum arabic producer’s

associations and the geographic proximity between localities to reduce

implementation costs. GAPAs containing women members and seasonal labourers

were prioritized in line with IFAD country programme strategy to increase access of

poor rural women and men to markets and microfinance4. In total, 23.895 direct

beneficiaries (17.080 who participated in microfinance, 5.315 in matching grants

and 1.500 in training and capacity-building) representing 236 GAPAs were directly

supported5.

5. Financing. The project was financed jointly through an IFAD grant, fully

administered by the World Bank’s Multi-Donor Trust Fund (MDTF)6, the

Government of Sudan and beneficiaries for a total planned amount of US$10.88

million.

6. A two-phase implementation was employed with MDTF funding earmarked for

phase one to the tune of US$7.00M at appraisal and US$6.996M at closure. IFAD’s

contribution was set at US$3.00M at appraisal with a total disbursed amount of

US$2.84M at project end scheduled for disbursement during phase 27. The actual

total amount disbursed at closure was US$9.665M, which excluded the planned

contribution of the Government of Sudan, in the form of taxes, and beneficiaries

amounting to a combined total of US$880K. On 21 January 2009, an official

communication stipulated that the project would not receive counterpart

contribution from the Government8. Table 1 below summarizes the planned,

revised and total disbursed amount by donor.

Table 1 Project costs in US$ Million equivalent

Donor Planned US$

million Percentage of

total (%)

Revised at Restructuring

2011

Percentage of total

Disbursed Disbursement

rate (%)

IFAD 3.00 27.57 3.00 30 2.86 95.44

MDTF 7.00 64.33 7.00 70 6.80 97.17

Government 0.38 3.50 0 0 0 0

Beneficiaries 0.50 4.60 0 0 0 0

Total 10.88 100 10.00 100 9.67 96.65

Source: World Bank Implementation Completion and Results Report, June 20, 2016.

7. Financing for individual components was undertaken jointly by the MDTF and IFAD

grant. Allocation of funds for the three components at project end was not in

proportion to the original funding allocation at project design. In part justified by

4 World Bank Implementation Completion Report, P.1

5 Technical Assistance Report for Preparing the Implementation Completion and Results Report, P.6

6 The project was co-financed by IFAD and the World Bank administered Multi-Donor Trust Fund for North Sudan.

7 World Bank Implementation Completion Report, P.V

8 World Bank Final Project Proposal Gum Arabic, P.23

Page 4: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

4

the extended project implementation period, the final expenditures under

component 1 were approximately 50 per cent below appraisal estimates while

component 3 overspent with expenditures, reaching 144 per cent.

Table 2 Total Component costs in US$ Million equivalent

Component Estimated cost at design

Percentage of total

Revised at Restructuring

2011

Percentage of total

Expenditure Disbursement

rate (%)

Sector Reform and Support 1.70 17 1.70 17 0.79 46

Institutional and capacity building to GAPAs

5.52 55 5.52 55 5.59 101

Project management and supervision 2.77 28 2.77 28 3.29 119

Total 10.00 100 10.00 100 9.67 97

Source: World Bank Implementation Completion and Results Report, June 20, 2016.

8. Changes and developments during implementation. There were three main

restructuring events during June 2011, June 2012 and November 2012 that

affected project implementation.

9. New conflict in 2010 within the three localities previously selected under the IFAD

grant in phase two led to the halt of activities and the loss of some project assets

in the Blue Nile and South Kordofan States. While activities were restarted in these

areas, uninterrupted implementation remained an issue throughout the project

lifespan.

10. Both MDTF and IFAD experienced delays, with the monetary disbursement for the

latter and grant extension after phase one completion for the former, which

extended original project closure dates for MDTF from December 2011 to June

2012, December 2012 and finally May 2013, whereas the disbursement of IFAD

grant was completed in June 2015. Initial disbursement of IFAD funds to the

project were significantly delayed and contributed to the final extension of the

project closing date9. Consequently, the project was extended beyond the original

period of four years to six years. No justification was provided in the project

documents clarifying the reason for the delay in the disbursement of IFAD funds.

11. The results framework faced a major overhaul as a result of recommendations

stated by the Implementation Supervision/Support Mission (ISM) in May 2010.

Formally adopted in the restructuring of 2011, the changes were based on the

need to better capture outputs and performance and affected intermediate level

results and certain Project Development Objective (PDO) level indicators. Notably,

the PDO remained constant throughout the project, whereas intermediate level

indicators and results faced significant changes. Intermediate Results (IR)

indicators were amended to better capture the functioning of the gum arabic

market and capacity-building for GAPAs and related institutions in the Locality

Implementation Units (LIU) and Project Steering Committee (PSC), while indicators

relating to policy reform, functioning of microcredit / revolving funds and project

management were later dropped.

12. The original log frame stated in the President’s report underwent substantial

changes from appraisal to project end. While the PDO remained the same, two of

the original PDO level indicators were downgraded to the IR level. The original

result relating to environmental degradation was dropped from the final log frame.

All indicators relating to project management and supervision were dropped from

9 World Bank Implementation Completion Report, P.7

Page 5: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

5

the log frame in favour of the two original PDO indicators (see annex IV for a

comparative analysis of the log frame at appraisal and at project end).

13. As the project had already undergone an impact assessment in 2011, and due

partially to the delayed decision on the extension, the original envisioned Mid-term

Review (MTR) was replaced by implementation closure related assessments10.

14. Project implementation. The implementation was carried out over two phases

with IFAD grant covering phase 2. Phase one of the project provided support

through analytical studies in sector knowledge11 to facilitate policy reform

initiatives and training and capacity-building for GAPAs in seven localities, while

supervision and project management was undertaken by the implementing agency

(FNC) - who were also tasked with the production of a functioning M&E system,

including a baseline survey.

15. While the Ministry of Foreign Trade endorsed the development of the Gum Arabic

Marketing Information System, the project was tasked with providing equipment,

training and technical assistance to support the initiative. In parallel, the project

established a revolving credit scheme to support GAPAs in the target areas.

16. Phase 2 extended support to GAPAs in additional locations and monitored policy

sector reform. The operational guidance and oversight was provided by the Ministry

of Agriculture and stakeholder organisations through a PSC at the regional level. A

technical committee, chaired by the FNC General Director, was established to

provide technical oversight to the PSC. At the federal level, the PCU, consisting of a

project coordinator, credit and marketing, procurement and financial management

specialists, and M&E officer housed at the FNC, supervised eight LIUs, comprised of

coordinators, forestry extension officers and financial management officers, and

coordinated with the Gum Arabic Commodity Council to facilitate sector reform

initiatives. At the locality level, GAPA’s leaders were responsible for organising

training activities for their members and mobilising members’ contributions to

finance. No state level PCU was created.

17. Intervention logic. The PDO was clear and focused at design and remained stable

throughout the projects lifespan. If links between project inputs, outputs and

outcomes where not clearly defined in the results framework, sufficient details

were provided under the project description. The project used a dual focus

approach on sector reform, combining it with interventions designed to address

issues around production, marketing and access to finance12 through two

components. The third component supported project management and supervision

of activities. The activities under each component were relevant with the exception

of some activities under component 1, relating to government-adopted policy

reforms which were implemented prior to project commencement. The activities

relating to improved production of gum arabic were also unclear.

18. Delivery of outputs. The project reached the majority of stated performance

targets. Component one delivered four main outputs: Studies on policy reform;

market promotion, research and development grants; market information systems

and promotion of a media grant. The two revised indicators under this component

were 100 per cent achieved and 40 per cent underachieved respectively.

19. Component two comprised of training and capacity-building to GAPAs, micro

finance and matching grants to support GAPA infrastructure and investments in

FNC physical structure. Targets for indicators under this component were partially

met with five out of ten overachieving and four underachieving. All targets relating

10

World Bank Implementation Completion Report, P.8 11

Implementation of the proposed sector reform initiatives, based on the recommendations of the analytical studies during phase one, opened up the market and attracted new market participants. 12

Independent Evaluation Group (IEG) Implementation Completion Report Review, P.4

Page 6: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

6

to training were overachieved. Indicators relating to matching grants, with the

exception of infrastructure support, rated well below stated targets.

20. Despite an initial delay with trainings in phase 2, due in part to the renewed

conflict in the three IFAD localities, which were previously under the control of

Sudan People’s Liberation Movement, overall indicators related to trainings in this

category were overachieved. By project end, 236 GAPAs were supported, of which

129 received seasonal and micro-finance support, 56 received matching grants and

51 received training13. Of those GAPAs trained, 470 individual GAPAs members

received training on financial management, 134 per cent achieved; 642 received

business management training, 107 per cent achievement rate; and 280 received

agro-forestry training, 122 per cent achieved. Seasonal credit and micro-finance

grants were provided to 110 selected GAPAs, which subsequently overachieved by

117 per cent at project end. Performance of matching grants (MG) fared less well

with only one of the three performance indicators reaching target. MGs for research

and technology and media promotion both underperformed, while MGs for

infrastructure support overachieved by 187 percent. While the use of matching

grants was relevant to the PDO, the FNC had limited experience in administering

micro credit to gum arabic producers.

21. Key outputs for component 3, which included support to project management and

supervision, were overachieved. The project had tasked the Ministry of Foreign

Trade with the establishment of a market information system; the basic structure

of the system was in place by 2013, whereas the market information system was

not fully functional by project close and subsequently affected achievement of the

number of text messages communicated to producers.

22. The implementation of the management information system faced a series of

challenges that resulted in under performance in the second indicator. Although the

basic structure of the market information system was in place by 2015, the system

was not fully functional as no price, production or other data had been collected

and disseminated through the system.

23. The Gum Arabic Project was financed through a joint MDTF and IFAD grant;

however, the administration was fully undertaken by the World Bank.

Consequently, there was no IFAD project completion report developed. The World

Bank finalised an Implementation Completion Report (ICR) in 2016 followed up by

an ICR Review of the ICR by an independent consultant. There was no MTR

undertaken during the project, therefore targets where not adjusted at midterm. A

restructuring paper with recommendations for results and indicator adjustments

was submitted and subsequently implemented in 2011. The review of findings is

based primarily on the Banks ICR and ISM reports undertaken during the lifespan

of the project.

Relevance

24. Policy Relevance. The Gum Arabic Project is in line with the Government of

Sudan Agriculture Revitalization Plan launched in April 2008 and the Expenditure

Priorities and Funding Needs, which places agricultural development as the engine

of growth and rural poverty reduction at its core14. Furthermore, the Government

of National Unity Minister of Finance and National Economy committed to abolish

the export monopoly under the Joint Assessment Mission Framework for Sustained

Peace, Development and Poverty Eradication15, which called for the removal of

concessions for the raw gum export and provision of financial and technical support

to producers to enable them to benefit from liberalized trade and to participate in

the export of processed gum arabic.

13

Independent Evaluation Group (IEG) Implementation Completion Report Review, P.5 14

COSOP 2009, P.6 15

President’s Report, P.9

Page 7: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

7

25. The project was directly linked to three of the five priority focus areas of the MDTF:

Supporting state-locality-level investment programmes with high visibility in war-

affected zones; making rural development pro-poor with support to micro/small

enterprises and laying the groundwork for good governance by opening up the

private sector. The project contribution to consolidation of the initial set of policy

reform in the gum arabic sector were followed by a longer term dialogue on policy

and institutional changes to improve market access and expand processing and

export opportunities16.

26. Additionally, The Sudan qualified for 100 per cent grant financing under IFAD’s

DEBT Sustainability Framework approved in April 2007, making the programme

highly relevant and responsive to support economic growth and poverty reduction

among poor and conflict affected households17. Furthermore, the project fell within

the scope of IFADs objectives to facilitate economic growth and market

development, enhanced access to scalable and sustainable rural financial systems,

increased farming and livestock productivity and income in a sustainable way

through research, extension and improved technologies and mainstream

knowledge management, partnerships and participatory dialogue for better national

policies to improve livelihood of rural communities18.

27. Relevance to local needs. Gum arabic production is an important source of cash

for producers and can contribute up to 50 per cent of total household cash

earnings19 during the year. The project objective to increase income for producers

remains relevant to livelihood priorities of producers. Producers were traditionally

heavily reliant on sheil, an informal lending system, which led to low prices for

producers. The programme supported the government made to promote

microfinance institutions as an alternative source of financing. Given the more

active role envisioned for GAPAs in development, the project responded through

direct support to policy reform to address constraints and bottle necks and to

create an enabling environment for small-scale farmers in order to benefit from the

liberalization of the gum arabic sector. A key focus being capacity-building through

training to address technical shortfalls and knowledge gaps and the provision of

matching grants as well as the availability of micro finance aimed at contributing to

the programme's relevance.

28. Relevance of design. The design of the project was highly relevant as evidenced

by the stability of the PDO throughout the lifespan of the project. The three core

areas of support were accurately represented through three IR components

including support to policy reform, institutional and capacity-building and project

supervision. Changes to the IR components included the revision and removal of

some results and performance indicators as recommended in 2011. The design

could have benefited from specific gender results and corresponding indicators to

better highlight the achievement of support to female headed GAPAs. The design

included a focused project development objective, but did not provide clear links

between project inputs, outputs and expected outcomes. The project log frame

underwent substantial changes from appraisal to project end. No clear justification

was provided for the restructuring of the results framework. Activities to support

improved production of gum arabic were unclear, and the ICR Review noted that

attribution of activities under component 1 were questionable given the

government-adopted policy reforms that were enacted prior to the projects

approval.20

29. Relevance of Targeting. The project specifically targeted gum arabic producers,

who were already organized into GAPAs situated and operating in the gum belt. An

16

World Bank Implementation Completion Report, P.38 17

World Bank Implementation Completion Report, P.10 18

COSOP 2009, P.18 19

World Bank Implementation Completion Report, P.10 20

Independent Evaluation Group (IEG) Implementation Completion report Review, P.4

Page 8: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

8

emphasis was placed on those GAPAs that contained women, seasonal workers and

poor members, which ensured a pro poor approach was taken during beneficiary

selection. Targeting criteria took into account the following assumptions:

beneficiaries had less than 20 feddans of gum trees; track record of using the sheil

system during gum tapping and collection season; established track record of

repayment; women headed households; resource poor households with proposals

linked to livelihoods improvement initiatives. Targeting was in line with IFAD 2006

targeting policy.

30. The projects relevance is rated at satisfactory (5), which is in line with the

Programme Management Department (PMD) rating.

Effectiveness

31. The information below was reported on the outcome level indicators as stated on

the World Bank’s log frame (see annex IV). In many cases, output indicators were

used as proxy indicators to measure higher level results. For example, number of

female beneficiaries, which was an output level indicator, was used to measure the

PDO. As no MTR or final evaluation report were undertaken, the effectiveness of

the project was evaluated through the ICR, which included the project log frame

and the ICR Review.

32. Programme outcomes were measured through two of the three components and

three outcome performance indicators. Two of the indicators were geared to assess

income while the third was more of an output/ intermediate indicator that tracked

the percentage of Gum Arabic Producer Associations with effective savings and

credit systems. There was no outcome indicator to assess the effect of the project

on the increase in production of gum arabic.

Outcome 1: Implementation of the gum arabic sector reform, creating equal opportunity

and competition among all market actors

33. Studies on policy reform were undertaken in order to meet the project's

justification for phase 2 and helped the project to identify a further set of policy

reforms in the sector and create an enabling environment for equal opportunity

and competition among market actors. These included the studies on Gum Arabic

Value Chain Analysis (2011), World Demand Structure and Market for Sudanese

Gum Arabic (2012) and Gum Arabic Auction Markets and Market Information

Systems (2011).

34. Policy reform in the gum arabic sector has positively contributed to its revitalization

as reflected in the percentage of Free on Board (FOB) price received by producers.

The ICR Review showed that according with the 2014 baseline survey the project

beneficiaries received higher prices for their gum arabic produce compared with

non-beneficiaries, furthermore, project beneficiaries received 71 per cent of the

FOB price compared with the control group, who received 59 per cent21. Moreover,

technical and analytical studies financed by the Project on Auction Markets and

Value Chain have assisted the Gum Arabic Commodity Council to identify

opportunities for opening new markets. To this extent, the project achieved the

outcome of improving the broader gum arabic sector by increasing knowledge of

local and international market opportunities.

35. Both the 2007 Policy Note and FPP noted that the producers received 10-15 per

cent of the FOB price under the pre-liberalized system. Project surveys indicated

project beneficiaries received a higher FOB price in 2014. The percentage change

of the FOB price received by small scale gum producers after project intervention

was 108 per cent achieved and percentage change in domestic use compared to

export was 120 per cent achieved. The percentage change in the number of new

entrants to the market, however, was only 40 per cent achieved.

21

Independent Evaluation Group (IEG) Implementation Completion Report Review, P.6

Page 9: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

9

36. It was noted, however, in the Country Strategic Opportunities Program (COSOP)

MTR Report 2012, that although producers of gum arabic had organized and

obtained higher output prices, this achievement was attributed mostly to the new

enabling environment rather than a direct consequence of the project activities –

supporting producers so that they can organize to obtain higher output prices22.

Significant reform took place in the sector prior to project approval as a result of a

Presidential Directive issued in June 2009. This represented a major liberalization

of the sector and while a few additional sector reforms were identified by the

project, they cannot be directly attributed to the project.

Outcome 2: Strengthening GAPA organization and capacity in order to improve their

bargaining position

37. Improved gum arabic production was a key outcome of the project. The project

produced 2.7 million seedlings as a result of investments in tree nursery

rehabilitation in the FNC stations. The results from the new trees will contribute to

the future sustainability and production of gum arabic in the targeted localities

across the gum arabic belt.

38. The ICR Review stated that total gum arabic production for the 40 GAPAs under

survey increased from 546 tons in 2011 to 1,623 tons in 201423. Furthermore, the

same study concluded that the delivered project extension officers improved

production technologies and educated the small producers consequently increasing

the crop productivity as well as adoption of new techniques for gum arabic

cleaning, grading and packing.

39. Strengthening the GAPA organizations was a key outcome of the project. This was

attained through a series of trainings, designed to build capacity of producer’s

associations, including improved production techniques, credit, financial

management, marketing, business/organizational skills and gender mainstreaming.

In total, 189 GAPAs attended the trainings. In order to promote a culture of

learning and continuance of knowledge, Training of Trainers trainings were

delivered to Forestry Extension Officers and locality coordinators on community

organizational skills, credit-marketing and gender mainstreaming with the aim of

replicating the trainings after project closure.

40. Outcomes generated as a result of the support included improving incomes through

investments in capacity building, productive infrastructure and inputs24. The ICR

analysis on the net increase in incomes among direct project beneficiaries, after

discounting the general rise in income for all gum arabic producers, shows the

increase is approximately 21 per cent in real terms.25

41. The project provided training for GAPAs on better collective marketing techniques

and the provision of matching grants to better improve marketing among small-

scale producers. GAPAs supported by the project benefited from efforts in collective

marketing to obtain better prices and avoid low prices in the peak marketing

season. Improved marketing among small-scale producers targeted by the project

was evidenced through a 25 per cent change, 100 per cent achievement of stated

target, in quantities of gum arabic marketed by GAPAs at project end.

42. Project investments in savings and credit systems allowed individual gum arabic

producers to access credit and reduce reliance on other traditional lending schemes

such as Sheil, which typically resulted in low prices to gum arabic producers.

Matching grants (MG) for marketing were provided for: (i) infrastructure support;

(ii) research and technology; and (iii) media promotion and tracked through IR

indicators. While the project over-achieved targets for MG infrastructure support;

22

COSOP MTR, P.22 23

Independent Evaluation Group (IEG) Implementation Completion Report Review, P.5 24

World Bank Implementation Completion Report, P.3 25

World Bank Implementation Completion Report P.23

Page 10: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

10

MGs for research and technology were only moderately successful, 67 per cent

achievement. MGs for media promotion were, however, less successful with only 10

percent provided by project end. MGs for infrastructure support allowed farmers to

make investments in storage and transport thus enabling them to sell in bulk when

prices were highest26.

43. Previously, it was noted that a shortage of work force, mostly in the form of able

bodied youth, had contributed to the low levels of production of gum arabic. Youths

would typically abandon working in the gum arabic sector for gold mining, which

provides more stable income. Following project completion, beneficiaries attributed

the return of youth to the villages to the increase in available work closer to home.

44. The project’s main objectives regarding policy reforms, investments in institutional

capacity building, producer organization development, credit facilities, productive

investments and restocking of gum Arabic trees were met to a large extent. Certain

outcomes, including the ability of beneficiaries to access up-to-date and robust

market information, were only partially met. The original framework included

matching grants to provide alternative financial sources to small-scale farmers.

While these grants mostly met the stated targets, the lack of experience in

managing and supplying micro finance by the FNC resulted in mixed achievements.

These outcomes were, however, not critical in achieving the project objectives.

45. Considering the above, the PCRV rates effectiveness satisfactory (5), same as PMD.

Efficiency

46. At project design, the original total commitment of the IFAD grant stood at US$3

million. Actual expenditure at end of project amounted to US$2,863,272.00,

equalling 95.44 per cent grant disbursement. According to the World Banks ICR27,

the internal rate of return for the Gum Arabic Project is estimated at 7 per cent,

calculated based on estimated increase in revenue generated by gum sales among

project beneficiaries using assumptions on the average production levels, number

of direct beneficiaries and average premium price realized by project beneficiaries.

This excludes indirect beneficiaries, who may have benefited from the broader

policy reform attributed to the project. As such, it was noted that due to the

fundamental flaws in estimation and the exclusion of indirect beneficiaries, an

additional calculation was applied to a narrower set of costs associated with

provision of services to direct beneficiaries. The internal rate of return increased to

21 per cent.

47. The project faced higher than average operational costs with component 3, project

management and supervision consuming 32.87 per cent of project budget. The

total combined MDTF and IFAD funding noted 46 per cent under spending for

component 1 from project appraisal while component 3 expenditures were 19 per

cent above appraisal estimates. When broken down by IFAD grant, component 1

spent 100.56 per cent of allocated funding, a drop in spending for component 2

amounted to spending of 65 per cent, while component 3 overspent by 154 per

cent.

48. IFAD first disbursement was transferred in April 2013 following a discrepancy in

application and closing dates between World Bank and IFAD which subsequently

slowed planned activities in phase 2 originally scheduled for completion in

December 2013. A no-cost extension was formally awarded to the project

extending IFADs original project close to June 201528.

49. In line with the PMD rating, efficiency is rated at moderately satisfactory (4), due

to the over expenditure on project management and supervision that consumed

roughly one third of overall project funding.

26

World Bank Implementation Completion Report, P.27 27

World Bank Implementation Completion Report, P.18 28

Sudan Amendment Financing Agreement, P.2

Page 11: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

11

Rural poverty impact

50. Household income. The project design was primarily poverty focused as

evidenced by the PDO to “increase the production and income of small-scale gum

producers in selected areas of the gum belt through improved performance of

production and marketing systems”. The project hypothesized that increased

income, improved gum arabic production and improved marketing among small-

scale producers targeted by the project would positively affect rural poverty rates

amongst small-scale producers. The results from an analysis of the baseline survey

(2011) and the end line survey (2014) in the impact assessment29 showed that the

original target of a 20 per cent increase in income level of beneficiary households

from gum arabic production after the project intervention was met30. The

assessment utilized a control group for comparison and noted that both project and

non-project beneficiaries benefited from the liberalization of the sector which

resulted in a gradual increase in producer prices during project implementation. As

a result, both groups registered an increase in income. The net increase in income

among direct project beneficiaries, discounting the general rise in income for all

gum arabic producers, was approximately 21 per cent in real terms.

51. Agricultural productivity. Gum arabic production increased from 546 tons in

2011 to 1,623 tons in 2014 or by 197 per cent31 due in part to an increased

utilization of tree resources and access to inputs and finance to mobilize labour.

However, the increase in gum production within the targeted project localities could

only be due to harvesting existing trees, predating the project, as gum trees

require an average of five years before reaching maturity. Thus, the effect of new

gum trees planted will be measurable only beyond the project period. Furthermore,

there was no clear picture of what elements of Gum Arabic production improved.

For example, better tree management or improved harvesting techniques.

However, the ICR did state that improvement in Gum Arabic production could be

attributed to improved production technologies as a result from trainings to small

scale producers by project support extension officers. Matching grants provided

much needed income to employ seasonal labourers during harvesting, and

investments in tractors and water points facilitated transportation and water to the

field. Project impact on food security has not been monitored or reported upon in

the project documents reviewed.

52. Institutions and policies. The programme supported decentralization of the

management of gum arabic production to the FNC through training and capacity

building, which has created a more enabling environment for policy sector reform.

Furthermore, the studies completed in phase 1 contributed significantly to the

reform agenda in the second phase of project implementation. However, the

second-generation reforms faced lack of consensus around policy reform relating to

the collection of remaining taxes and fees, the structure of the Gum Arabic Auction

Market, export licensing arrangements and access to finance under Islamic banking

principals32. However, the project did finance studies specifically to analyse and

consolidate these issues into a concrete reform agenda.

53. The overall rural poverty impact is rated at satisfactory (5), which is equal to the

PMD rating.

54. Sustainability of benefits. The project developed a formal exit strategy ensuring

that the FNC would continue to operate and maintain training centres and nurseries

financed through the project. It noted that GAPAs would be responsible for

operations and maintenance of commodities such as water points, tractors and

29

Results from the impact assessment were reported through the Banks’s ICR, therefore data was quoted from the Bank’s ICR as a secondary data source. 30

World Bank Implementation Completion Report P.11 31

Independent Evaluation Group (IEG) Implementation Completion Report Review, P.5 32

World Bank Implementation Completion Report, P.18

Page 12: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

12

pumps using the training received as part of the matching grants. Support to

sustainability will also be provided through the Gum Arabic Board.

55. The establishment of the Gum Arabic Board in 2009 by the Government of Sudan

has helped to smooth the way for sustainability of the sector. The liberalization of

the sector, and more specifically sector policy reform, has resulted in a rise in

private sector development and new entrants in the form of exports and basic

marketing and processing.

56. As a clear indication of government commitment, the FNC has proactively proposed

a Gum Arabic Commodity Exchange and Warehouse Receipt System project to

further develop the gum arabic value chain and support higher level producer

organization development.

57. The project noted a “triple bottom line”33 in benefits – social, environmental and

financial benefits in that it focused on regenerating and expanding the stock of

gum arabic trees, investing in community infrastructure, improving sustainable

harvest methods, and increasing profits from gum arabic sales. These outcome

results contributed directly to the sustainability of project benefits.

58. On this basis, sustainability is scored at satisfactory (5), which is equal to the PMD

rating.

A. Other performance criteria

59. Innovation. While project activities were not innovative themselves, the overall

project design emphasised innovation through new farming techniques. IFADs

contribution to phase 2 of the project, direct support and capacity-building to

GAPAs further contributed to the innovation of the reform by assisting small-scale

farmers to benefit from the liberalization of the sector through adapting farming

techniques and utilizing new marketing arrangements.

60. Innovation is rated at moderately satisfactory (4), which is in line with the PMD

rating.

61. Scaling up. The project employed a dual scaling up approach through policy

reform and its enforcement, and developing the capacity of the FNC to provide

technical and marketing support to GAPAs.

62. Given the high interest in policy reform in gum arabic sector and the

decentralisation of the management of implementation of the reform to the FNC,

scaling up is rated at satisfactory (5), which is in line with the PMD rating.

63. Gender equality and women’s empowerment. No specific project results were

dedicated to gender equality and women’s empowerment. According to IFAD's

Regional Division for Near East, North Africa and Europe, the project consistently

adhered to gender sensitivity, gender equality and women economic empowerment

principles which were integrated into project planning, execution, monitoring and

reporting. Through the participatory development approach, women were heavily

engaged in the decision making of the GAPAs, thus ensuring women members had

equal access to the disseminated knowledge, technology and finance.34 However,

IOE was unable to validate (or to rescind) these assertions, as reporting on gender

is extremely scarce in the project documentation available to IOE, including the

ICR.

64. Direct project beneficiaries consisted for 24 per cent of women, which is just one

point below the target of 25 per cent. Information collected during the lifespan of

the project indicates that the project contributed to an increase in the share of

female membership in GAPAs over time. In some cases, project-supported GAPAs

experienced an increase in female membership from levels below 20 per cent to 25

33

World Bank Implementation Completion Report, P.37 34

Source: Near East, North Africa and Europe Division of IFAD comments received on the draft PCRV report.

Page 13: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

13

per cent by project end.35 In addition, ten all-female GAPAs were created by the

project (out of a total of 236 supported GAPAs) to address women's specific needs.

Finally, the Project Completion Report (PCR) indicates gender-specific effects in the

production patterns of gum Arabic whereby women were entrusted with the

protection of hashab trees. This was an interesting unintended consequence of the

project.

65. As such, gender equality and women’s empowerment is rated as moderately

satisfactory (4), in line with the PMD rating.

66. Environment and natural resources management. The Gum Arabic Project

was rated category C at appraisal by the World Bank as it was not expected to

generate significant environment and social impacts. However, it was noted, in a

wider review of safeguards in 2012, that the project actually operated under

category B of the Environmental and Social Management Framework and the

decision to not officially reclassify the project resulted in a missed opportunity to

strengthen environmental protection and social sustainability for the future. It

should be noted that in IFAD President's Report, the project was classified under

category B.36

67. The ESFM for Natural Resources Projects in The Sudan specifically included a clause

for mitigation measures that could be used should any of the matching grants be

found to have substantial environmental impacts. An external safeguard review

found no major issues resulting in the need to trigger the clause.

68. A key outcome result for addressing environmental degradation issues in target

areas was included in the original results framework of the President's Report but

subsequently dropped from the final results framework following the 2011

restructuring paper recommendations.

69. The project contributed significantly to the management of environment and

natural resources despite a lack of consensus in rating the project Environmental

and Social Management Framework category level. The focus on regenerating and

expanding the stock of gum arabic trees resulted in the production of 2.7 million

seedlings and significant investments in tree nursery rehabilitation at FNC stations.

On that basis, environment and natural resources management is rated

satisfactory (5), which is equal to the PMD rating.

70. Adaptation to climate change. The project did not have a specific climate

change focus, however, as the World Bank noted in the ICR, gum arabic trees play

an important secondary role in agro ecological systems by contributing to restoring

soil fertility, providing dune or wind breaks to reduce soil erosion. A key unintended

outcome of the project positively contributes to environmental and climate change

through carbon sequestration37.

71. Adaption to climate change is rated as moderately satisfactory (4), same as the

PMD rating, due to the missed opportunity to include a specific focus on climate

change within the project design.

B. Overall project achievement

72. At the time of appraisal, The Sudan was implementing the provisions of the

Comprehensive Peace Accord signed in 2005. The Gum Arabic Project sought to

support the consolidation of peace and pro-poor growth within The Sudan through

the implementation of the gum arabic sector reform, creating equal opportunity

and competition among all market actors and strengthening GAPA organization and

capacity to improve their bargaining position. The policy reform in the gum arabic

sector positively contributed to a 50 per cent increase in FOB price received by

35

World Bank Implementation Completion Report, P.20 36

President's Report, P.13 37

World Bank Implementation Completion Report, P.21

Page 14: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

14

producers in 2011, a 10 per cent rise prior to the policy reform, and a 250 per cent

increase in gum arabic exports in the same year. The project supported and

strengthened GAPAs through direct trainings and facilitated exchanges between

GAPAs to share knowledge and promote sustainability for the establishment of new

GAPAs in the future, exceeding original targets. Direct support to the FNC to build

long-term capacity contributed to their ability to support GAPAs.

73. The project provided continued capacity-building to create a competent staff at all

levels of management, financial and procurement aspects. Increased knowledge

and communication of products and market opportunities has led to recognition of

the importance of a market information system to ensure efficiency and

transparency in the sector. Beneficiaries placed high importance on the need for

up-to-date, robust market information. A fully functioning market information

system is essential to create more sustainable opportunities for scaling up

production. Overall, project achievement is rated satisfactory (5), same as PMD.

C. Performance of partners

74. IFAD. IFAD’s grant covered activities in phase 2 under the administration of the

World Bank. As such, the below review includes a review of both IFAD contribution

and the World Bank administration. IFAD performed well during project design by

contributing programme documents and engaging with the World Bank.

Supervision visits were undertaken jointly between IFAD and the World Bank,

however, it was noted that although a revision of the M&E framework was

undertaken and implemented following the 2012 restructuring paper, further

revisions to the framework were suggested in 2013 that did not result in

amendments that could have better contributed to measuring the projects

successes. Delays in the initial first payment tranche by IFAD contributed to the

extension of the project and decreased the duration of IFAD’s funding from original

appraisal of 24 months to 18 months. The MTR, which was originally scheduled to

take place in September 2011 was not carried out, in part as it was deemed too

close to the original project closure date38. Justification provided in the ICR for the

cancellation of the MTR was the institutional restructuring process that took place

in 2011.

75. A criticism noted by the ICR related to the frequent turnover of the task team

leader position, which “may have led to a loss of institutional memory and affected

the extent of the Bank’s engagement on policy dialogue over the course of

implementation”.39

76. Performance of IFAD during the Gum Arabic Project was difficult to rate because

the project was exclusively administered through the World Bank, however, IFAD

participated in all ISM missions and contributed significantly to project design.

Slight delays occurred in the first grant disbursement, therefore, IFAD performance

is rated at moderately satisfactory (4), in line with the PMD rating.

77. Government. Project implementation was supervised and managed through the

FNC and Ministry of Agriculture and Forestry. Major challenges faced by the FNC

during implementation were its: (i) inability to establish a fully functioning market

information system; and (ii) lack of experience in supplying matching grants and

micro finance. M&E activities were incorporated under component 3 (project

supervision and management) and included the production of monthly and

quarterly reports prepared by LIUs and submitted to central coordinating units.

Quarterly reports were then prepared by the PCU and submitted to the Bank.

78. As stated in the Bank's ICR, rigorous monitoring, evaluation and learning was

adopted throughout the project and included implementation performance

monitoring, outcome evaluation and a learning/knowledge management

38

Independent Evaluation Group (IEG) Implementation Completion Report Review, P.3 39

Independent Evaluation Group (IEG) Implementation Completion Report Review, P.8

Page 15: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

15

component. A baseline survey was undertaken in the project’s first year, an impact

assessment and final impact assessment based on household surveys targeting,

were also completed. Quarterly technical progress reporting was consistent

throughout the project’s lifespan. None of these documents are available at IFAD,

however, so none could be consulted for this PCRV.

79. In the original project proposal, the President's Report allocated US$380,000 in the

form of taxes to the government’s contribution40; however, the project did not

receive any counterpart contribution and was wholly financed through the MDTF

and IFAD grant. It was noted that the Government took on part of the cost of the

PCU in the final year of the project; however, the amount of Government resources

was not quantified.

80. While both implementing agencies faced challenges throughout the lifespan of the

project, the PCRV rates government performance moderately satisfactory (4),

equal to the PMD rating.

III. Assessment of PCR quality 81. The below findings are representative of the World Bank's ICR and not the IFAD

PCR. As previously mentioned, the Gum Arabic Project was wholly administered

through the World Bank and therefore, project completion reports and reviews

were undertaken by the Bank and external consultants.

82. Scope. All chapters including relevant annexes were adequately covered in the

World Bank’s ICR as per the Bank’s guidelines. The President's Report noted that

as the project was administered by the World Bank, associated sections of the IFAD

General Conditions for Agriculture Development Financing would be omitted in

favour of World Bank policies and procedures41. On this basis, scope is rated

satisfactory (5).

83. Quality. The Bank undertook a beneficiary survey and stakeholder workshop as

part of project closure. The workshop captured and incorporated data on

beneficiaries, LIU and PCU perspectives. Additionally, challenges, constraints and

successes faced by the project during implementation were included in the final

ICR. Although an MTR was not undertaken at mid-term, an impact assessment in

2011, baseline assessments, economic and financial analysis supported ICR

findings. The Bank and IFAD undertook six joint supervision visits throughout the

project and reported candid ratings of project shortcomings and constraints.

84. The ICR utilized robust data collected throughout the lifespan of the project which

contributed to the production of a high quality report. On this basis, quality is rated

satisfactory (5).

85. Lessons. The Bank’s ICR draws important lessons from project design and project

implementation and adequately reports lessons learned based on explicit

evaluation findings at project completion. Given the rigorous and comprehensive

list of lessons captured at project completion, Lessons is rated satisfactory (5).

86. Candour. The ICR reports candid findings supported by data captured during the

multiple ISM visits undertaken during the project lifespan. Both negative and

positive findings were captured throughout the ICR and the narrative was

sufficiently objective. Findings were, for the most part, coherent with the narrative

and reflected sound judgement and analysis. Candour is rated satisfactory (5).

IV. Lessons learned 87. Market information systems are better located with organizations directly

controlled by beneficiaries. Having the market information systems anchored

40

President's Report, P.12 41

President's Report 2009, P.10

Page 16: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

16

within a line ministry has proved challenging in other country contexts and appears

to have faced similar challenges under the project. Lessons from other countries

have shown market information systems should be hosted and managed by the

direct beneficiaries who need information and have interest to share information.

The need for information should be determined by producers and private operators

and they could create their own associations to manage the system. The Ministry of

Foreign Trade does not have the same needs and interest to access quick and

reliable market information.

88. Creating a logical results chain in a post conflict setting and uncertain policy

environment requires continued attention and iterative processes. As a project that

was processed under OP8.042 that also targeted sector reforms, the project faced

an initial challenge in articulating a results chain that accurately identified outputs,

intermediate outcome and results. Revisions to the results framework were

required as the impact of sector reforms was better understood and the specific

investments in GAPA capacity building were finalized. An iterative process of

refining results can address gaps in knowledge in uncertain policy environments

and/or post conflict settings where ex ante analysis may be limited by time,

available information and levels of client or stakeholder engagement.

89. Illiteracy had adverse effects on some of the GAPAs ability to function. In

some instances, the formation and management of the GAPA executive committees

was tasked to educated individuals in the village who were not gum arabic farmers

but teachers or government workers. This led, in one case, to embezzlement and

misappropriation of funds that was resolved by a court law. Ensuring the literacy of

the executive committees of GAPAs should be prioritized through the inclusion of

long term literacy training as part of the broader capacity building plan.

90. Rural credit interventions should be managed by institutions with

sufficient experience and capacity. The FNC had limited prior experience in

managing matching grants and revolving credit schemes. Given the prevalence of

use of traditional financing methods (Sheil), GAPAs were initially hesitant to use

the microfinance system at the locality levels. Furthermore, failure of the GAPAs to

understand that the Matching Grant and Revolving Credit were to be used as seed

money for initial investments that would foster income generating activities in the

future, led to multiple further requests for money from the project. In order for a

functioning microfinance system to be implemented in The Sudan, further

investments in training of the FNC to facilitate and take the lead is required.

91. Collective marketing is proven to be a more effective means of improving

market access by small scale producers. Small scale Gum Arabic producers

face multiple marketing constraints and often have limited direct links to the

markets. Furthermore, there is evidence to suggest that improving market access

for small scale farmers will lead to increased income and food security. Efforts

should be made to harmonize gum arabic production and marketing across the

gum arabic belt states. Further emphasis should be placed on studies that address

marketing constraints and trainings covering marketing strategies.

42

Projects supported by the World Bank’s Investment Project Financing are governed by OPs (Operating Procedures). The Bank has classified this project under OP8.0: Rapid Response to Crises and Emergencies.

Page 17: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

Annex I

17

Definition and rating of the evaluation criteria used by IOE

Criteria Definition * Mandatory

To be rated

Rural poverty impact Impact is defined as the changes that have occurred or are expected to occur in the lives of the rural poor (whether positive or negative, direct or indirect, intended or unintended) as a result of development interventions.

X Yes

Four impact domains

Household income and net assets: Household income provides a means of assessing the flow of economic benefits accruing to an individual or group, whereas assets relate to a stock of accumulated items of economic value. The analysis must include an assessment of trends in equality over time.

No

Human and social capital and empowerment: Human and social capital and empowerment include an assessment of the changes that have occurred in the empowerment of individuals, the quality of grass-roots organizations and institutions, the poor’s individual and collective capacity, and in particular, the extent to which specific groups such as youth are included or excluded from the development process.

No

Food security and agricultural productivity: Changes in food security relate to availability, stability, affordability and access to food and stability of access, whereas changes in agricultural productivity are measured in terms of yields; nutrition relates to the nutritional value of food and child malnutrition.

No

Institutions and policies: The criterion relating to institutions and policies

is designed to assess changes in the quality and performance of institutions, policies and the regulatory framework that influence the lives of the poor.

No

Project performance Project performance is an average of the ratings for relevance, effectiveness, efficiency and sustainability of benefits. X Yes

Relevance The extent to which the objectives of a development intervention are consistent with beneficiaries’ requirements, country needs, institutional priorities and partner and donor policies. It also entails an assessment of project design and coherence in achieving its objectives. An assessment should also be made of whether objectives and design address inequality, for example, by assessing the relevance of targeting strategies adopted.

X Yes

Effectiveness The extent to which the development intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance.

X

Yes

Efficiency

Sustainability of benefits

A measure of how economically resources/inputs (funds, expertise, time, etc.) are converted into results.

The likely continuation of net benefits from a development intervention beyond the phase of external funding support. It also includes an assessment of the likelihood that actual and anticipated results will be resilient to risks beyond the project’s life.

X

X

Yes

Yes

Other performance criteria

Gender equality and women’s empowerment

Innovation

Scaling up

The extent to which IFAD interventions have contributed to better gender equality and women’s empowerment, for example, in terms of women’s access to and ownership of assets, resources and services; participation in decision making; work load balance and impact on women’s incomes, nutrition and livelihoods.

The extent to which IFAD development interventions have introduced innovative approaches to rural poverty reduction.

The extent to which IFAD development interventions have been (or are likely to be) scaled up by government authorities, donor organizations, the private sector and others agencies.

X

X

X

Yes

Yes

Yes

Environment and natural resources management

The extent to which IFAD development interventions contribute to resilient livelihoods and ecosystems. The focus is on the use and management of the natural environment, including natural resources defined as raw materials used for socio-economic and cultural purposes, and ecosystems and biodiversity - with the goods and services they provide.

X Yes

Adaptation to climate change

The contribution of the project to reducing the negative impacts of climate change through dedicated adaptation or risk reduction measures.

X Yes

Page 18: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

Annex I

18

Criteria Definition * Mandatory To be rated

Overall project achievement

This provides an overarching assessment of the intervention, drawing upon the analysis and ratings for rural poverty impact, relevance, effectiveness, efficiency, sustainability of benefits, gender equality and women’s empowerment, innovation, scaling up, as well as environment and natural resources management, and adaptation to climate change.

X Yes

Performance of partners

IFAD

Government

This criterion assesses the contribution of partners to project design, execution, monitoring and reporting, supervision and implementation support, and evaluation. The performance of each partner will be assessed on an individual basis with a view to the partner’s expected role and responsibility in the project life cycle.

X

X

Yes

Yes

* These definitions build on the Organisation for Economic Co-operation and Development/Development Assistance Committee (OECD/DAC) Glossary of Key Terms in Evaluation and Results-Based Management; the Methodological Framework for Project Evaluation agreed with the Evaluation Committee in September 2003; the first edition of the Evaluation Manual discussed with the Evaluation Committee in December 2008; and further discussions with the Evaluation Committee in November 2010 on IOE’s evaluation criteria and key questions.

Page 19: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

Annex II

19

Rating comparisona

Criteria

Programme Management

Department (PMD) rating

IOE Project Completion Report Validation (PCRV)

rating

Net rating disconnect

(PCRV-PMD)

Rural poverty impact 5 5 0

Project performance

Relevance 5 5 0

Effectiveness 5 5 0

Efficiency 4 4 0

Sustainability of benefits 5 5 0

Project performanceb 4.75 4.75 0

Other performance criteria

Gender equality and women's empowerment 4 4 0

Innovation 4 4 0

Scaling up 5 5 0

Environment and natural resources management 5 5 0

Adaptation to climate change 4 4 0

Overall project achievementc 5 5 0

Performance of partnersd

IFAD 4 4 0

Government 4 4 0

Average net disconnect 0

a Rating scale: 1 = highly unsatisfactory; 2 = unsatisfactory; 3 = moderately unsatisfactory; 4 = moderately satisfactory; 5 =

satisfactory; 6 = highly satisfactory; n.p. = not provided; n.a. = not applicable. b Arithmetic average of ratings for relevance, effectiveness, efficiency and sustainability of benefits. c This is not an average of ratings of individual evaluation criteria but an overarching assessment of the project, drawing upon

the rating for relevance, effectiveness, efficiency, sustainability of benefits, rural poverty impact, gender, innovation, scaling up, environment and natural resources management, and adaptation to climate change. d The rating for partners’ performance is not a component of the overall project achievement rating.

Ratings of the project completion report quality

PMD rating IOE PCRV rating Net disconnect

Candour - 5 NA

Lessons - 5 NA

Quality (methods, data, participatory process) - 5 NA

Scope - 5 NA

Overall rating of the project completion report 5 NA

Rating scale: 1 = highly unsatisfactory; 2 = unsatisfactory; 3 = moderately unsatisfactory; 4 = moderately satisfactory; 5 = satisfactory; 6 = highly satisfactory; n.p. = not provided; n.a. = not applicable.

Page 20: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

Annex III

20

Abbreviations and Acronyms

COSOP Country Strategic Opportunities Program

FNC Forestry National Cooperation

FOB Free On Board

GAPA Gum Arabic Producers Association

ICR Implementation Completion Report

IR Intermediate Result

ISM Implementation Supervision/Support Mission

LIU Locality Implementation Unit

MDTF Multi-donor Trust Fund

MG Matching Grant

MTR Mid Term Review

PCR Project Completion Report

PCRV Project Completion Review Validation

PCU Project Coordination Unit

PDO Project Development Objective

PMD Programme Management Department

PSC Project Steering Committee

Page 21: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

21

The Sudan Gum Arabic Project - Log Frame at Appraisal and Project End

President Report Log Frame at Appraisal World Bank ICR Log Frame at Completion

Project Development Objective

Increase the production and income of small scale gum producers in selected areas of the gum belt through improved performance of the production and marketing systems.

Indicators

Household asset index (RIMS);

Prevalence of malnutrition in children under 5 (RIMS);

Percentage change in income level of beneficiary households from Gum Arabic production after the project intervention;

Percentage of the FOB price received by small scale gum producers after project intervention.

Project Development Objective

Same as President Report.

Indicators

Percentage change in income level of beneficiary households from Gum Arabic production after the project intervention;

Direct project beneficiaries;

Direct project beneficiaries of which are female.

Intermediate Result Component 1 Gum Arabic Sector Reform and Support

Equal opportunity and competition to all market actors.

Indicators

Implementation of the agreed-on sector reform packages within two years of project effectiveness and open up the market for potential market actors;

Value Chain, International Market structure and demand analysis, standards and quality of products at Auction markets, and socioeconomic studies discussed; finalized and shared with major stakeholders;

Amount of Marketing Capacity Improvement and Product Promotion Matching Grant disbursed.

Intermediate Result Component 1

Same as President Report.

Indicators

Percentage change in quantities of Gum Arabic marketed by GAPAs;

Percentage change in number of new entrants to the market (exporters).

Annex IV

Page 22: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

22

Intermediate Result Component 2 Support to GAPAs

Organized and empowered GAPAs;

Market conditions improved;

Environmental degradation issues in target areas addressed.

Indicators

Number and % of GAPAs in target localities directly selling their product to processors and exporters;

Number and % of GAPAs who received improved production, agro forestry and quality maintenance trainings;

% change in production of Gum Arabic by GAPAs after project intervention;

New GAPAs formed and registered with significant representation of women and poor producers;

% of women and poor GAPA members receiving revolving funds;

% change in Community Capability Index (CCI);

% of GAPA members borrowing money from (sheil) local traders to finance production and marketing costs;

% repayment rate of GAPAs loans from the revolving credit system.

Intermediate Result Component 2

Same as President Report, however, the IR result describing environmental degradation was removed.

Indicators

% targeted GAPAs with effective saving and credit systems;

% of women targeted in GAPAs;

Number of Hashab seedlings produced by project nurseries;

Number of GAPA members who received training on business management;

Number of Agro forestry training sessions for GAPAs conducted by extension officers;

Number of GAPA members who received training on financial management;

Number of in-country and exchange visits of GAPAs conducted to share experience;

Number of Matching Grants for Marketing: Media Promotion;

Number of Matching Grants for Marketing: Research and Technology;

Number of Matching Grants for Marketing: Infrastructure support.

Intermediate Result Component 3 Project Management

The project is well managed and coordinated at the National, State and Locality levels through the existing FNC institutional structure.

Indicators

The Project Steering Committee is fully operational with regularly scheduled meetings;

Project Monitoring and Learning System established, sharing forums at different levels of project management.

Intermediate Result Component 3

Better management and coordination of the sector at national, state and locality levels.

Indicators

% of the FOB price received by small scale gum producers after project intervention;

% change in domestic use compared to export;

Number of visitors to the dynamic website of MoFT;

Number of text messages communicated to producers.

Page 23: I. Basic project dataPCRV_… · The Gum Arabic Project covered 11 localities in five geographical states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue

Annex V

23

Bibliography

Forest National Corporation. 2015. Revitalizing the Sudan Gum Arabic Production and

Marketing Project, Technical Assistance Report for Preparing the Implementation

Completion Results Report, P.6

Independent Evaluation Group (IEG), Sudan Gum Arabic Export Marketing Project,

Implementation Completion Report (ICR) Review. P.3, P.5, P.8

International Fund for Agricultural Development (IFAD). 2009. President’s Report -

Proposed grant to the Republic of The Sudan for the Revitalizing The Sudan Gum

Arabic Production and Marketing Project. P.9, P.11, P.12, P.13.

______. 2009. Republic of the Sudan Country Strategic Opportunities Programme

(COSOP). P.6, P.18

______. 2012. Results-based Country Strategic Opportunities Programme (RB-COSOP)

Mid-term Review Report. P.13, P.22

______. 2013. Sudan Amendment Financing Agreement. P.2

World Bank. 2016. Implementation Completion Report for Revitalizing The Sudan Gum

Arabic Production and Marketing Project. P.1, P.3, P.V, P.7, P.8, P.10, P.11, P.18,

P.20, P.21

______. 2009. Final Project Proposal on Revitalizing The Sudan Gum Arabic Production

and Marketing Project. P.23