i. basic project datapcrv_… · the gum arabic project covered 11 localities in five geographical...
TRANSCRIPT
Project Completion Report Validation
Revitalizing The Sudan Gum Arabic Production and Marketing Project
Republic of The Sudan
Date of validation by IOE: June 2017
I. Basic project data
Approval (US$ '000) Actual (US$ '000)
Region
Near East , North Africa and Europe Total project costs 10,880 9,665 18
Country Republic of The Sudan
IFAD loan and percentage of total 3,000 2,863 00
Loan number DSF-8043-SD Borrower 380 0 00
Type of project (subsector)
Multi-Donor Trust Fund 7,000 6,802 18
Financing type Grant Beneficiaries 500 0 00
Lending terms*
Date of approval 15/09/2009
Date of loan signature 03/11/2009
Date of effectiveness 03/11/2009 Other sources:
Loan amendments
Number of beneficiaries
198,895 direct and indirect beneficiaries in
total and 236 GAPAs 23,895 direct and
175,000 indirect
Loan closure extensions
No cost extension
approved on 07/05/2012
Country programme managers Rasha Omar Loan closing date 30/06/2015
Regional director(s) Mid-term review
No MTR was conducted
Project completion report reviewer
Philippa Morgan
IFAD loan disbursement at project completion (%) 2,863,272.00 (95.44%)
Project completion report quality control panel
Prashanth Kotturi
Michael Carbon
Date of the project completion report
Source: World Bank Implementation Completion Report.
* The grant was provided under the IFAD Debt Sustainability Framework, which classifies The Sudan as a “red light” country, therefore, making it eligible for 100 per cent grant financing. The Republic of The Sudan agreed to sign a “Subsidiary Agreement” with the Forestry National Cooperation and to be subjected to annual audits undertaken by the World Bank.
.
2
II. Project outline 1. Introduction. Revitalizing The Sudan Gum Arabic Production and Marketing
Project (henceforth “the Gum Arabic Project”) in the Republic of The Sudan was
approved by IFAD Executive Board on 15 September 2009. The IFAD loan
agreement was signed on 3 November 2009 and became effective on the same
date. The Gum Arabic Project was extended by one year past the original closing
date of 30 June 2014 to 30 June 2015. The project was executed in six years and
the grant was closed on 30 December 2016.
2. Project area. The Gum Arabic Project covered 11 localities in five geographical
states along the gum belt: South and North Kordofan, Sinnar, White Nile and Blue
Nile. Gum Arabic has long been a strategic agricultural commodity for Sudan and
the country has historically been the world’s leading producer; the majority of gum
production is sourced from smallholders located within isolated rural communities
in the gum belt. However, in 2009, the sector experienced a steep decline in
production. Gum Arabic farmers suffered from a series of challenges in the sector
including erratic rainfall, degraded soils, low use of productivity-enhancing
technologies, limited knowledge and poor access to rural financial services.
Increased conflict among pastoralists and the secession of the South resulting in a
decrease of available agricultural land also led to a decline in production. At an
average rural poverty rate of 57.6 per cent1, one million poor Sudanese households
rely on revenue generated from gum arabic production2, representing 50 per cent
of the total cash income for households. Between 2007 and 2011, the Government
of Sudan put in place the Agriculture Revitalization Plan which allocated SDG 10
billion for the agriculture sector. Furthermore, in June 2009, the President issued a
decree declaring the liberalization of the production, processing, marketing and
export of gum arabic3. A combination of these factors resulted in a niche for policy
reform in the gum arabic sector that could benefit and facilitate higher yields to
Sudanese gum arabic producers.
3. Project goal, objectives and components. The overall objective of the project
was to “increase the production and income of small-scale gum producers in
selected areas of the gum belt through improved performance of production and
marketing systems”. The project was comprised of three components:
Component 1 – Gum Arabic sector reform and support (US$1.57M at
design; US$0.79 at completion): the main objective was to create equal
opportunity and competition among all market actors through Gum Arabic
sector reform and support. Key activities under this component included
studies to identify barriers to domestic trade and investments, an analysis of
the world market and demand structures, and the identification of new market
opportunities. The project was to establish a gum arabic market information
system to address the lack of market information available to gum arabic
producers, to enable them to negotiate fair prices for their gum. In addition,
matching grants were allocated to support capacity development in
marketing, product promotion and research.
Component 2 – Institutional and capacity-building support to Gum
Arabic Producers Associations (GAPAs) (US$6.15M at design; US$5.59M
at completion) with the objective of improving incomes through investments
in capacity-building, productive infrastructure and inputs. Specific activities
under this component included capacity building activities to strengthen the
management and skills of GAPAs in savings, credit, marketing, quality control
and production. Revolving savings and credit schemes were set up to support
newly established GAPAs.
1 COSOP Mid-term Review, P.13
2 President’s Report, P.9
3 President’s Report, P.9
3
Component 3 - Project management and supervision (US$2.28M at
design; US$3.29M at completion), including M&E activities and related
baseline studies, mid-term review and final project assessments. The project
provided training and logistical support to the Forest National Cooperation
(FNC), the Project Coordination Unit (PCU) and the Locality Project
Coordination Units under this component.
4. Target group. The Project selected small-scale gum arabic producers who were
organised into GAPAs located in the rain fed gum arabic belt. Eight localities were
selected under phase one of implementation with a further three localities to be
included during phase two under the IFAD grant. The localities were determined
based on the presence of productive gum arabic trees and gum arabic producer’s
associations and the geographic proximity between localities to reduce
implementation costs. GAPAs containing women members and seasonal labourers
were prioritized in line with IFAD country programme strategy to increase access of
poor rural women and men to markets and microfinance4. In total, 23.895 direct
beneficiaries (17.080 who participated in microfinance, 5.315 in matching grants
and 1.500 in training and capacity-building) representing 236 GAPAs were directly
supported5.
5. Financing. The project was financed jointly through an IFAD grant, fully
administered by the World Bank’s Multi-Donor Trust Fund (MDTF)6, the
Government of Sudan and beneficiaries for a total planned amount of US$10.88
million.
6. A two-phase implementation was employed with MDTF funding earmarked for
phase one to the tune of US$7.00M at appraisal and US$6.996M at closure. IFAD’s
contribution was set at US$3.00M at appraisal with a total disbursed amount of
US$2.84M at project end scheduled for disbursement during phase 27. The actual
total amount disbursed at closure was US$9.665M, which excluded the planned
contribution of the Government of Sudan, in the form of taxes, and beneficiaries
amounting to a combined total of US$880K. On 21 January 2009, an official
communication stipulated that the project would not receive counterpart
contribution from the Government8. Table 1 below summarizes the planned,
revised and total disbursed amount by donor.
Table 1 Project costs in US$ Million equivalent
Donor Planned US$
million Percentage of
total (%)
Revised at Restructuring
2011
Percentage of total
Disbursed Disbursement
rate (%)
IFAD 3.00 27.57 3.00 30 2.86 95.44
MDTF 7.00 64.33 7.00 70 6.80 97.17
Government 0.38 3.50 0 0 0 0
Beneficiaries 0.50 4.60 0 0 0 0
Total 10.88 100 10.00 100 9.67 96.65
Source: World Bank Implementation Completion and Results Report, June 20, 2016.
7. Financing for individual components was undertaken jointly by the MDTF and IFAD
grant. Allocation of funds for the three components at project end was not in
proportion to the original funding allocation at project design. In part justified by
4 World Bank Implementation Completion Report, P.1
5 Technical Assistance Report for Preparing the Implementation Completion and Results Report, P.6
6 The project was co-financed by IFAD and the World Bank administered Multi-Donor Trust Fund for North Sudan.
7 World Bank Implementation Completion Report, P.V
8 World Bank Final Project Proposal Gum Arabic, P.23
4
the extended project implementation period, the final expenditures under
component 1 were approximately 50 per cent below appraisal estimates while
component 3 overspent with expenditures, reaching 144 per cent.
Table 2 Total Component costs in US$ Million equivalent
Component Estimated cost at design
Percentage of total
Revised at Restructuring
2011
Percentage of total
Expenditure Disbursement
rate (%)
Sector Reform and Support 1.70 17 1.70 17 0.79 46
Institutional and capacity building to GAPAs
5.52 55 5.52 55 5.59 101
Project management and supervision 2.77 28 2.77 28 3.29 119
Total 10.00 100 10.00 100 9.67 97
Source: World Bank Implementation Completion and Results Report, June 20, 2016.
8. Changes and developments during implementation. There were three main
restructuring events during June 2011, June 2012 and November 2012 that
affected project implementation.
9. New conflict in 2010 within the three localities previously selected under the IFAD
grant in phase two led to the halt of activities and the loss of some project assets
in the Blue Nile and South Kordofan States. While activities were restarted in these
areas, uninterrupted implementation remained an issue throughout the project
lifespan.
10. Both MDTF and IFAD experienced delays, with the monetary disbursement for the
latter and grant extension after phase one completion for the former, which
extended original project closure dates for MDTF from December 2011 to June
2012, December 2012 and finally May 2013, whereas the disbursement of IFAD
grant was completed in June 2015. Initial disbursement of IFAD funds to the
project were significantly delayed and contributed to the final extension of the
project closing date9. Consequently, the project was extended beyond the original
period of four years to six years. No justification was provided in the project
documents clarifying the reason for the delay in the disbursement of IFAD funds.
11. The results framework faced a major overhaul as a result of recommendations
stated by the Implementation Supervision/Support Mission (ISM) in May 2010.
Formally adopted in the restructuring of 2011, the changes were based on the
need to better capture outputs and performance and affected intermediate level
results and certain Project Development Objective (PDO) level indicators. Notably,
the PDO remained constant throughout the project, whereas intermediate level
indicators and results faced significant changes. Intermediate Results (IR)
indicators were amended to better capture the functioning of the gum arabic
market and capacity-building for GAPAs and related institutions in the Locality
Implementation Units (LIU) and Project Steering Committee (PSC), while indicators
relating to policy reform, functioning of microcredit / revolving funds and project
management were later dropped.
12. The original log frame stated in the President’s report underwent substantial
changes from appraisal to project end. While the PDO remained the same, two of
the original PDO level indicators were downgraded to the IR level. The original
result relating to environmental degradation was dropped from the final log frame.
All indicators relating to project management and supervision were dropped from
9 World Bank Implementation Completion Report, P.7
5
the log frame in favour of the two original PDO indicators (see annex IV for a
comparative analysis of the log frame at appraisal and at project end).
13. As the project had already undergone an impact assessment in 2011, and due
partially to the delayed decision on the extension, the original envisioned Mid-term
Review (MTR) was replaced by implementation closure related assessments10.
14. Project implementation. The implementation was carried out over two phases
with IFAD grant covering phase 2. Phase one of the project provided support
through analytical studies in sector knowledge11 to facilitate policy reform
initiatives and training and capacity-building for GAPAs in seven localities, while
supervision and project management was undertaken by the implementing agency
(FNC) - who were also tasked with the production of a functioning M&E system,
including a baseline survey.
15. While the Ministry of Foreign Trade endorsed the development of the Gum Arabic
Marketing Information System, the project was tasked with providing equipment,
training and technical assistance to support the initiative. In parallel, the project
established a revolving credit scheme to support GAPAs in the target areas.
16. Phase 2 extended support to GAPAs in additional locations and monitored policy
sector reform. The operational guidance and oversight was provided by the Ministry
of Agriculture and stakeholder organisations through a PSC at the regional level. A
technical committee, chaired by the FNC General Director, was established to
provide technical oversight to the PSC. At the federal level, the PCU, consisting of a
project coordinator, credit and marketing, procurement and financial management
specialists, and M&E officer housed at the FNC, supervised eight LIUs, comprised of
coordinators, forestry extension officers and financial management officers, and
coordinated with the Gum Arabic Commodity Council to facilitate sector reform
initiatives. At the locality level, GAPA’s leaders were responsible for organising
training activities for their members and mobilising members’ contributions to
finance. No state level PCU was created.
17. Intervention logic. The PDO was clear and focused at design and remained stable
throughout the projects lifespan. If links between project inputs, outputs and
outcomes where not clearly defined in the results framework, sufficient details
were provided under the project description. The project used a dual focus
approach on sector reform, combining it with interventions designed to address
issues around production, marketing and access to finance12 through two
components. The third component supported project management and supervision
of activities. The activities under each component were relevant with the exception
of some activities under component 1, relating to government-adopted policy
reforms which were implemented prior to project commencement. The activities
relating to improved production of gum arabic were also unclear.
18. Delivery of outputs. The project reached the majority of stated performance
targets. Component one delivered four main outputs: Studies on policy reform;
market promotion, research and development grants; market information systems
and promotion of a media grant. The two revised indicators under this component
were 100 per cent achieved and 40 per cent underachieved respectively.
19. Component two comprised of training and capacity-building to GAPAs, micro
finance and matching grants to support GAPA infrastructure and investments in
FNC physical structure. Targets for indicators under this component were partially
met with five out of ten overachieving and four underachieving. All targets relating
10
World Bank Implementation Completion Report, P.8 11
Implementation of the proposed sector reform initiatives, based on the recommendations of the analytical studies during phase one, opened up the market and attracted new market participants. 12
Independent Evaluation Group (IEG) Implementation Completion Report Review, P.4
6
to training were overachieved. Indicators relating to matching grants, with the
exception of infrastructure support, rated well below stated targets.
20. Despite an initial delay with trainings in phase 2, due in part to the renewed
conflict in the three IFAD localities, which were previously under the control of
Sudan People’s Liberation Movement, overall indicators related to trainings in this
category were overachieved. By project end, 236 GAPAs were supported, of which
129 received seasonal and micro-finance support, 56 received matching grants and
51 received training13. Of those GAPAs trained, 470 individual GAPAs members
received training on financial management, 134 per cent achieved; 642 received
business management training, 107 per cent achievement rate; and 280 received
agro-forestry training, 122 per cent achieved. Seasonal credit and micro-finance
grants were provided to 110 selected GAPAs, which subsequently overachieved by
117 per cent at project end. Performance of matching grants (MG) fared less well
with only one of the three performance indicators reaching target. MGs for research
and technology and media promotion both underperformed, while MGs for
infrastructure support overachieved by 187 percent. While the use of matching
grants was relevant to the PDO, the FNC had limited experience in administering
micro credit to gum arabic producers.
21. Key outputs for component 3, which included support to project management and
supervision, were overachieved. The project had tasked the Ministry of Foreign
Trade with the establishment of a market information system; the basic structure
of the system was in place by 2013, whereas the market information system was
not fully functional by project close and subsequently affected achievement of the
number of text messages communicated to producers.
22. The implementation of the management information system faced a series of
challenges that resulted in under performance in the second indicator. Although the
basic structure of the market information system was in place by 2015, the system
was not fully functional as no price, production or other data had been collected
and disseminated through the system.
23. The Gum Arabic Project was financed through a joint MDTF and IFAD grant;
however, the administration was fully undertaken by the World Bank.
Consequently, there was no IFAD project completion report developed. The World
Bank finalised an Implementation Completion Report (ICR) in 2016 followed up by
an ICR Review of the ICR by an independent consultant. There was no MTR
undertaken during the project, therefore targets where not adjusted at midterm. A
restructuring paper with recommendations for results and indicator adjustments
was submitted and subsequently implemented in 2011. The review of findings is
based primarily on the Banks ICR and ISM reports undertaken during the lifespan
of the project.
Relevance
24. Policy Relevance. The Gum Arabic Project is in line with the Government of
Sudan Agriculture Revitalization Plan launched in April 2008 and the Expenditure
Priorities and Funding Needs, which places agricultural development as the engine
of growth and rural poverty reduction at its core14. Furthermore, the Government
of National Unity Minister of Finance and National Economy committed to abolish
the export monopoly under the Joint Assessment Mission Framework for Sustained
Peace, Development and Poverty Eradication15, which called for the removal of
concessions for the raw gum export and provision of financial and technical support
to producers to enable them to benefit from liberalized trade and to participate in
the export of processed gum arabic.
13
Independent Evaluation Group (IEG) Implementation Completion Report Review, P.5 14
COSOP 2009, P.6 15
President’s Report, P.9
7
25. The project was directly linked to three of the five priority focus areas of the MDTF:
Supporting state-locality-level investment programmes with high visibility in war-
affected zones; making rural development pro-poor with support to micro/small
enterprises and laying the groundwork for good governance by opening up the
private sector. The project contribution to consolidation of the initial set of policy
reform in the gum arabic sector were followed by a longer term dialogue on policy
and institutional changes to improve market access and expand processing and
export opportunities16.
26. Additionally, The Sudan qualified for 100 per cent grant financing under IFAD’s
DEBT Sustainability Framework approved in April 2007, making the programme
highly relevant and responsive to support economic growth and poverty reduction
among poor and conflict affected households17. Furthermore, the project fell within
the scope of IFADs objectives to facilitate economic growth and market
development, enhanced access to scalable and sustainable rural financial systems,
increased farming and livestock productivity and income in a sustainable way
through research, extension and improved technologies and mainstream
knowledge management, partnerships and participatory dialogue for better national
policies to improve livelihood of rural communities18.
27. Relevance to local needs. Gum arabic production is an important source of cash
for producers and can contribute up to 50 per cent of total household cash
earnings19 during the year. The project objective to increase income for producers
remains relevant to livelihood priorities of producers. Producers were traditionally
heavily reliant on sheil, an informal lending system, which led to low prices for
producers. The programme supported the government made to promote
microfinance institutions as an alternative source of financing. Given the more
active role envisioned for GAPAs in development, the project responded through
direct support to policy reform to address constraints and bottle necks and to
create an enabling environment for small-scale farmers in order to benefit from the
liberalization of the gum arabic sector. A key focus being capacity-building through
training to address technical shortfalls and knowledge gaps and the provision of
matching grants as well as the availability of micro finance aimed at contributing to
the programme's relevance.
28. Relevance of design. The design of the project was highly relevant as evidenced
by the stability of the PDO throughout the lifespan of the project. The three core
areas of support were accurately represented through three IR components
including support to policy reform, institutional and capacity-building and project
supervision. Changes to the IR components included the revision and removal of
some results and performance indicators as recommended in 2011. The design
could have benefited from specific gender results and corresponding indicators to
better highlight the achievement of support to female headed GAPAs. The design
included a focused project development objective, but did not provide clear links
between project inputs, outputs and expected outcomes. The project log frame
underwent substantial changes from appraisal to project end. No clear justification
was provided for the restructuring of the results framework. Activities to support
improved production of gum arabic were unclear, and the ICR Review noted that
attribution of activities under component 1 were questionable given the
government-adopted policy reforms that were enacted prior to the projects
approval.20
29. Relevance of Targeting. The project specifically targeted gum arabic producers,
who were already organized into GAPAs situated and operating in the gum belt. An
16
World Bank Implementation Completion Report, P.38 17
World Bank Implementation Completion Report, P.10 18
COSOP 2009, P.18 19
World Bank Implementation Completion Report, P.10 20
Independent Evaluation Group (IEG) Implementation Completion report Review, P.4
8
emphasis was placed on those GAPAs that contained women, seasonal workers and
poor members, which ensured a pro poor approach was taken during beneficiary
selection. Targeting criteria took into account the following assumptions:
beneficiaries had less than 20 feddans of gum trees; track record of using the sheil
system during gum tapping and collection season; established track record of
repayment; women headed households; resource poor households with proposals
linked to livelihoods improvement initiatives. Targeting was in line with IFAD 2006
targeting policy.
30. The projects relevance is rated at satisfactory (5), which is in line with the
Programme Management Department (PMD) rating.
Effectiveness
31. The information below was reported on the outcome level indicators as stated on
the World Bank’s log frame (see annex IV). In many cases, output indicators were
used as proxy indicators to measure higher level results. For example, number of
female beneficiaries, which was an output level indicator, was used to measure the
PDO. As no MTR or final evaluation report were undertaken, the effectiveness of
the project was evaluated through the ICR, which included the project log frame
and the ICR Review.
32. Programme outcomes were measured through two of the three components and
three outcome performance indicators. Two of the indicators were geared to assess
income while the third was more of an output/ intermediate indicator that tracked
the percentage of Gum Arabic Producer Associations with effective savings and
credit systems. There was no outcome indicator to assess the effect of the project
on the increase in production of gum arabic.
Outcome 1: Implementation of the gum arabic sector reform, creating equal opportunity
and competition among all market actors
33. Studies on policy reform were undertaken in order to meet the project's
justification for phase 2 and helped the project to identify a further set of policy
reforms in the sector and create an enabling environment for equal opportunity
and competition among market actors. These included the studies on Gum Arabic
Value Chain Analysis (2011), World Demand Structure and Market for Sudanese
Gum Arabic (2012) and Gum Arabic Auction Markets and Market Information
Systems (2011).
34. Policy reform in the gum arabic sector has positively contributed to its revitalization
as reflected in the percentage of Free on Board (FOB) price received by producers.
The ICR Review showed that according with the 2014 baseline survey the project
beneficiaries received higher prices for their gum arabic produce compared with
non-beneficiaries, furthermore, project beneficiaries received 71 per cent of the
FOB price compared with the control group, who received 59 per cent21. Moreover,
technical and analytical studies financed by the Project on Auction Markets and
Value Chain have assisted the Gum Arabic Commodity Council to identify
opportunities for opening new markets. To this extent, the project achieved the
outcome of improving the broader gum arabic sector by increasing knowledge of
local and international market opportunities.
35. Both the 2007 Policy Note and FPP noted that the producers received 10-15 per
cent of the FOB price under the pre-liberalized system. Project surveys indicated
project beneficiaries received a higher FOB price in 2014. The percentage change
of the FOB price received by small scale gum producers after project intervention
was 108 per cent achieved and percentage change in domestic use compared to
export was 120 per cent achieved. The percentage change in the number of new
entrants to the market, however, was only 40 per cent achieved.
21
Independent Evaluation Group (IEG) Implementation Completion Report Review, P.6
9
36. It was noted, however, in the Country Strategic Opportunities Program (COSOP)
MTR Report 2012, that although producers of gum arabic had organized and
obtained higher output prices, this achievement was attributed mostly to the new
enabling environment rather than a direct consequence of the project activities –
supporting producers so that they can organize to obtain higher output prices22.
Significant reform took place in the sector prior to project approval as a result of a
Presidential Directive issued in June 2009. This represented a major liberalization
of the sector and while a few additional sector reforms were identified by the
project, they cannot be directly attributed to the project.
Outcome 2: Strengthening GAPA organization and capacity in order to improve their
bargaining position
37. Improved gum arabic production was a key outcome of the project. The project
produced 2.7 million seedlings as a result of investments in tree nursery
rehabilitation in the FNC stations. The results from the new trees will contribute to
the future sustainability and production of gum arabic in the targeted localities
across the gum arabic belt.
38. The ICR Review stated that total gum arabic production for the 40 GAPAs under
survey increased from 546 tons in 2011 to 1,623 tons in 201423. Furthermore, the
same study concluded that the delivered project extension officers improved
production technologies and educated the small producers consequently increasing
the crop productivity as well as adoption of new techniques for gum arabic
cleaning, grading and packing.
39. Strengthening the GAPA organizations was a key outcome of the project. This was
attained through a series of trainings, designed to build capacity of producer’s
associations, including improved production techniques, credit, financial
management, marketing, business/organizational skills and gender mainstreaming.
In total, 189 GAPAs attended the trainings. In order to promote a culture of
learning and continuance of knowledge, Training of Trainers trainings were
delivered to Forestry Extension Officers and locality coordinators on community
organizational skills, credit-marketing and gender mainstreaming with the aim of
replicating the trainings after project closure.
40. Outcomes generated as a result of the support included improving incomes through
investments in capacity building, productive infrastructure and inputs24. The ICR
analysis on the net increase in incomes among direct project beneficiaries, after
discounting the general rise in income for all gum arabic producers, shows the
increase is approximately 21 per cent in real terms.25
41. The project provided training for GAPAs on better collective marketing techniques
and the provision of matching grants to better improve marketing among small-
scale producers. GAPAs supported by the project benefited from efforts in collective
marketing to obtain better prices and avoid low prices in the peak marketing
season. Improved marketing among small-scale producers targeted by the project
was evidenced through a 25 per cent change, 100 per cent achievement of stated
target, in quantities of gum arabic marketed by GAPAs at project end.
42. Project investments in savings and credit systems allowed individual gum arabic
producers to access credit and reduce reliance on other traditional lending schemes
such as Sheil, which typically resulted in low prices to gum arabic producers.
Matching grants (MG) for marketing were provided for: (i) infrastructure support;
(ii) research and technology; and (iii) media promotion and tracked through IR
indicators. While the project over-achieved targets for MG infrastructure support;
22
COSOP MTR, P.22 23
Independent Evaluation Group (IEG) Implementation Completion Report Review, P.5 24
World Bank Implementation Completion Report, P.3 25
World Bank Implementation Completion Report P.23
10
MGs for research and technology were only moderately successful, 67 per cent
achievement. MGs for media promotion were, however, less successful with only 10
percent provided by project end. MGs for infrastructure support allowed farmers to
make investments in storage and transport thus enabling them to sell in bulk when
prices were highest26.
43. Previously, it was noted that a shortage of work force, mostly in the form of able
bodied youth, had contributed to the low levels of production of gum arabic. Youths
would typically abandon working in the gum arabic sector for gold mining, which
provides more stable income. Following project completion, beneficiaries attributed
the return of youth to the villages to the increase in available work closer to home.
44. The project’s main objectives regarding policy reforms, investments in institutional
capacity building, producer organization development, credit facilities, productive
investments and restocking of gum Arabic trees were met to a large extent. Certain
outcomes, including the ability of beneficiaries to access up-to-date and robust
market information, were only partially met. The original framework included
matching grants to provide alternative financial sources to small-scale farmers.
While these grants mostly met the stated targets, the lack of experience in
managing and supplying micro finance by the FNC resulted in mixed achievements.
These outcomes were, however, not critical in achieving the project objectives.
45. Considering the above, the PCRV rates effectiveness satisfactory (5), same as PMD.
Efficiency
46. At project design, the original total commitment of the IFAD grant stood at US$3
million. Actual expenditure at end of project amounted to US$2,863,272.00,
equalling 95.44 per cent grant disbursement. According to the World Banks ICR27,
the internal rate of return for the Gum Arabic Project is estimated at 7 per cent,
calculated based on estimated increase in revenue generated by gum sales among
project beneficiaries using assumptions on the average production levels, number
of direct beneficiaries and average premium price realized by project beneficiaries.
This excludes indirect beneficiaries, who may have benefited from the broader
policy reform attributed to the project. As such, it was noted that due to the
fundamental flaws in estimation and the exclusion of indirect beneficiaries, an
additional calculation was applied to a narrower set of costs associated with
provision of services to direct beneficiaries. The internal rate of return increased to
21 per cent.
47. The project faced higher than average operational costs with component 3, project
management and supervision consuming 32.87 per cent of project budget. The
total combined MDTF and IFAD funding noted 46 per cent under spending for
component 1 from project appraisal while component 3 expenditures were 19 per
cent above appraisal estimates. When broken down by IFAD grant, component 1
spent 100.56 per cent of allocated funding, a drop in spending for component 2
amounted to spending of 65 per cent, while component 3 overspent by 154 per
cent.
48. IFAD first disbursement was transferred in April 2013 following a discrepancy in
application and closing dates between World Bank and IFAD which subsequently
slowed planned activities in phase 2 originally scheduled for completion in
December 2013. A no-cost extension was formally awarded to the project
extending IFADs original project close to June 201528.
49. In line with the PMD rating, efficiency is rated at moderately satisfactory (4), due
to the over expenditure on project management and supervision that consumed
roughly one third of overall project funding.
26
World Bank Implementation Completion Report, P.27 27
World Bank Implementation Completion Report, P.18 28
Sudan Amendment Financing Agreement, P.2
11
Rural poverty impact
50. Household income. The project design was primarily poverty focused as
evidenced by the PDO to “increase the production and income of small-scale gum
producers in selected areas of the gum belt through improved performance of
production and marketing systems”. The project hypothesized that increased
income, improved gum arabic production and improved marketing among small-
scale producers targeted by the project would positively affect rural poverty rates
amongst small-scale producers. The results from an analysis of the baseline survey
(2011) and the end line survey (2014) in the impact assessment29 showed that the
original target of a 20 per cent increase in income level of beneficiary households
from gum arabic production after the project intervention was met30. The
assessment utilized a control group for comparison and noted that both project and
non-project beneficiaries benefited from the liberalization of the sector which
resulted in a gradual increase in producer prices during project implementation. As
a result, both groups registered an increase in income. The net increase in income
among direct project beneficiaries, discounting the general rise in income for all
gum arabic producers, was approximately 21 per cent in real terms.
51. Agricultural productivity. Gum arabic production increased from 546 tons in
2011 to 1,623 tons in 2014 or by 197 per cent31 due in part to an increased
utilization of tree resources and access to inputs and finance to mobilize labour.
However, the increase in gum production within the targeted project localities could
only be due to harvesting existing trees, predating the project, as gum trees
require an average of five years before reaching maturity. Thus, the effect of new
gum trees planted will be measurable only beyond the project period. Furthermore,
there was no clear picture of what elements of Gum Arabic production improved.
For example, better tree management or improved harvesting techniques.
However, the ICR did state that improvement in Gum Arabic production could be
attributed to improved production technologies as a result from trainings to small
scale producers by project support extension officers. Matching grants provided
much needed income to employ seasonal labourers during harvesting, and
investments in tractors and water points facilitated transportation and water to the
field. Project impact on food security has not been monitored or reported upon in
the project documents reviewed.
52. Institutions and policies. The programme supported decentralization of the
management of gum arabic production to the FNC through training and capacity
building, which has created a more enabling environment for policy sector reform.
Furthermore, the studies completed in phase 1 contributed significantly to the
reform agenda in the second phase of project implementation. However, the
second-generation reforms faced lack of consensus around policy reform relating to
the collection of remaining taxes and fees, the structure of the Gum Arabic Auction
Market, export licensing arrangements and access to finance under Islamic banking
principals32. However, the project did finance studies specifically to analyse and
consolidate these issues into a concrete reform agenda.
53. The overall rural poverty impact is rated at satisfactory (5), which is equal to the
PMD rating.
54. Sustainability of benefits. The project developed a formal exit strategy ensuring
that the FNC would continue to operate and maintain training centres and nurseries
financed through the project. It noted that GAPAs would be responsible for
operations and maintenance of commodities such as water points, tractors and
29
Results from the impact assessment were reported through the Banks’s ICR, therefore data was quoted from the Bank’s ICR as a secondary data source. 30
World Bank Implementation Completion Report P.11 31
Independent Evaluation Group (IEG) Implementation Completion Report Review, P.5 32
World Bank Implementation Completion Report, P.18
12
pumps using the training received as part of the matching grants. Support to
sustainability will also be provided through the Gum Arabic Board.
55. The establishment of the Gum Arabic Board in 2009 by the Government of Sudan
has helped to smooth the way for sustainability of the sector. The liberalization of
the sector, and more specifically sector policy reform, has resulted in a rise in
private sector development and new entrants in the form of exports and basic
marketing and processing.
56. As a clear indication of government commitment, the FNC has proactively proposed
a Gum Arabic Commodity Exchange and Warehouse Receipt System project to
further develop the gum arabic value chain and support higher level producer
organization development.
57. The project noted a “triple bottom line”33 in benefits – social, environmental and
financial benefits in that it focused on regenerating and expanding the stock of
gum arabic trees, investing in community infrastructure, improving sustainable
harvest methods, and increasing profits from gum arabic sales. These outcome
results contributed directly to the sustainability of project benefits.
58. On this basis, sustainability is scored at satisfactory (5), which is equal to the PMD
rating.
A. Other performance criteria
59. Innovation. While project activities were not innovative themselves, the overall
project design emphasised innovation through new farming techniques. IFADs
contribution to phase 2 of the project, direct support and capacity-building to
GAPAs further contributed to the innovation of the reform by assisting small-scale
farmers to benefit from the liberalization of the sector through adapting farming
techniques and utilizing new marketing arrangements.
60. Innovation is rated at moderately satisfactory (4), which is in line with the PMD
rating.
61. Scaling up. The project employed a dual scaling up approach through policy
reform and its enforcement, and developing the capacity of the FNC to provide
technical and marketing support to GAPAs.
62. Given the high interest in policy reform in gum arabic sector and the
decentralisation of the management of implementation of the reform to the FNC,
scaling up is rated at satisfactory (5), which is in line with the PMD rating.
63. Gender equality and women’s empowerment. No specific project results were
dedicated to gender equality and women’s empowerment. According to IFAD's
Regional Division for Near East, North Africa and Europe, the project consistently
adhered to gender sensitivity, gender equality and women economic empowerment
principles which were integrated into project planning, execution, monitoring and
reporting. Through the participatory development approach, women were heavily
engaged in the decision making of the GAPAs, thus ensuring women members had
equal access to the disseminated knowledge, technology and finance.34 However,
IOE was unable to validate (or to rescind) these assertions, as reporting on gender
is extremely scarce in the project documentation available to IOE, including the
ICR.
64. Direct project beneficiaries consisted for 24 per cent of women, which is just one
point below the target of 25 per cent. Information collected during the lifespan of
the project indicates that the project contributed to an increase in the share of
female membership in GAPAs over time. In some cases, project-supported GAPAs
experienced an increase in female membership from levels below 20 per cent to 25
33
World Bank Implementation Completion Report, P.37 34
Source: Near East, North Africa and Europe Division of IFAD comments received on the draft PCRV report.
13
per cent by project end.35 In addition, ten all-female GAPAs were created by the
project (out of a total of 236 supported GAPAs) to address women's specific needs.
Finally, the Project Completion Report (PCR) indicates gender-specific effects in the
production patterns of gum Arabic whereby women were entrusted with the
protection of hashab trees. This was an interesting unintended consequence of the
project.
65. As such, gender equality and women’s empowerment is rated as moderately
satisfactory (4), in line with the PMD rating.
66. Environment and natural resources management. The Gum Arabic Project
was rated category C at appraisal by the World Bank as it was not expected to
generate significant environment and social impacts. However, it was noted, in a
wider review of safeguards in 2012, that the project actually operated under
category B of the Environmental and Social Management Framework and the
decision to not officially reclassify the project resulted in a missed opportunity to
strengthen environmental protection and social sustainability for the future. It
should be noted that in IFAD President's Report, the project was classified under
category B.36
67. The ESFM for Natural Resources Projects in The Sudan specifically included a clause
for mitigation measures that could be used should any of the matching grants be
found to have substantial environmental impacts. An external safeguard review
found no major issues resulting in the need to trigger the clause.
68. A key outcome result for addressing environmental degradation issues in target
areas was included in the original results framework of the President's Report but
subsequently dropped from the final results framework following the 2011
restructuring paper recommendations.
69. The project contributed significantly to the management of environment and
natural resources despite a lack of consensus in rating the project Environmental
and Social Management Framework category level. The focus on regenerating and
expanding the stock of gum arabic trees resulted in the production of 2.7 million
seedlings and significant investments in tree nursery rehabilitation at FNC stations.
On that basis, environment and natural resources management is rated
satisfactory (5), which is equal to the PMD rating.
70. Adaptation to climate change. The project did not have a specific climate
change focus, however, as the World Bank noted in the ICR, gum arabic trees play
an important secondary role in agro ecological systems by contributing to restoring
soil fertility, providing dune or wind breaks to reduce soil erosion. A key unintended
outcome of the project positively contributes to environmental and climate change
through carbon sequestration37.
71. Adaption to climate change is rated as moderately satisfactory (4), same as the
PMD rating, due to the missed opportunity to include a specific focus on climate
change within the project design.
B. Overall project achievement
72. At the time of appraisal, The Sudan was implementing the provisions of the
Comprehensive Peace Accord signed in 2005. The Gum Arabic Project sought to
support the consolidation of peace and pro-poor growth within The Sudan through
the implementation of the gum arabic sector reform, creating equal opportunity
and competition among all market actors and strengthening GAPA organization and
capacity to improve their bargaining position. The policy reform in the gum arabic
sector positively contributed to a 50 per cent increase in FOB price received by
35
World Bank Implementation Completion Report, P.20 36
President's Report, P.13 37
World Bank Implementation Completion Report, P.21
14
producers in 2011, a 10 per cent rise prior to the policy reform, and a 250 per cent
increase in gum arabic exports in the same year. The project supported and
strengthened GAPAs through direct trainings and facilitated exchanges between
GAPAs to share knowledge and promote sustainability for the establishment of new
GAPAs in the future, exceeding original targets. Direct support to the FNC to build
long-term capacity contributed to their ability to support GAPAs.
73. The project provided continued capacity-building to create a competent staff at all
levels of management, financial and procurement aspects. Increased knowledge
and communication of products and market opportunities has led to recognition of
the importance of a market information system to ensure efficiency and
transparency in the sector. Beneficiaries placed high importance on the need for
up-to-date, robust market information. A fully functioning market information
system is essential to create more sustainable opportunities for scaling up
production. Overall, project achievement is rated satisfactory (5), same as PMD.
C. Performance of partners
74. IFAD. IFAD’s grant covered activities in phase 2 under the administration of the
World Bank. As such, the below review includes a review of both IFAD contribution
and the World Bank administration. IFAD performed well during project design by
contributing programme documents and engaging with the World Bank.
Supervision visits were undertaken jointly between IFAD and the World Bank,
however, it was noted that although a revision of the M&E framework was
undertaken and implemented following the 2012 restructuring paper, further
revisions to the framework were suggested in 2013 that did not result in
amendments that could have better contributed to measuring the projects
successes. Delays in the initial first payment tranche by IFAD contributed to the
extension of the project and decreased the duration of IFAD’s funding from original
appraisal of 24 months to 18 months. The MTR, which was originally scheduled to
take place in September 2011 was not carried out, in part as it was deemed too
close to the original project closure date38. Justification provided in the ICR for the
cancellation of the MTR was the institutional restructuring process that took place
in 2011.
75. A criticism noted by the ICR related to the frequent turnover of the task team
leader position, which “may have led to a loss of institutional memory and affected
the extent of the Bank’s engagement on policy dialogue over the course of
implementation”.39
76. Performance of IFAD during the Gum Arabic Project was difficult to rate because
the project was exclusively administered through the World Bank, however, IFAD
participated in all ISM missions and contributed significantly to project design.
Slight delays occurred in the first grant disbursement, therefore, IFAD performance
is rated at moderately satisfactory (4), in line with the PMD rating.
77. Government. Project implementation was supervised and managed through the
FNC and Ministry of Agriculture and Forestry. Major challenges faced by the FNC
during implementation were its: (i) inability to establish a fully functioning market
information system; and (ii) lack of experience in supplying matching grants and
micro finance. M&E activities were incorporated under component 3 (project
supervision and management) and included the production of monthly and
quarterly reports prepared by LIUs and submitted to central coordinating units.
Quarterly reports were then prepared by the PCU and submitted to the Bank.
78. As stated in the Bank's ICR, rigorous monitoring, evaluation and learning was
adopted throughout the project and included implementation performance
monitoring, outcome evaluation and a learning/knowledge management
38
Independent Evaluation Group (IEG) Implementation Completion Report Review, P.3 39
Independent Evaluation Group (IEG) Implementation Completion Report Review, P.8
15
component. A baseline survey was undertaken in the project’s first year, an impact
assessment and final impact assessment based on household surveys targeting,
were also completed. Quarterly technical progress reporting was consistent
throughout the project’s lifespan. None of these documents are available at IFAD,
however, so none could be consulted for this PCRV.
79. In the original project proposal, the President's Report allocated US$380,000 in the
form of taxes to the government’s contribution40; however, the project did not
receive any counterpart contribution and was wholly financed through the MDTF
and IFAD grant. It was noted that the Government took on part of the cost of the
PCU in the final year of the project; however, the amount of Government resources
was not quantified.
80. While both implementing agencies faced challenges throughout the lifespan of the
project, the PCRV rates government performance moderately satisfactory (4),
equal to the PMD rating.
III. Assessment of PCR quality 81. The below findings are representative of the World Bank's ICR and not the IFAD
PCR. As previously mentioned, the Gum Arabic Project was wholly administered
through the World Bank and therefore, project completion reports and reviews
were undertaken by the Bank and external consultants.
82. Scope. All chapters including relevant annexes were adequately covered in the
World Bank’s ICR as per the Bank’s guidelines. The President's Report noted that
as the project was administered by the World Bank, associated sections of the IFAD
General Conditions for Agriculture Development Financing would be omitted in
favour of World Bank policies and procedures41. On this basis, scope is rated
satisfactory (5).
83. Quality. The Bank undertook a beneficiary survey and stakeholder workshop as
part of project closure. The workshop captured and incorporated data on
beneficiaries, LIU and PCU perspectives. Additionally, challenges, constraints and
successes faced by the project during implementation were included in the final
ICR. Although an MTR was not undertaken at mid-term, an impact assessment in
2011, baseline assessments, economic and financial analysis supported ICR
findings. The Bank and IFAD undertook six joint supervision visits throughout the
project and reported candid ratings of project shortcomings and constraints.
84. The ICR utilized robust data collected throughout the lifespan of the project which
contributed to the production of a high quality report. On this basis, quality is rated
satisfactory (5).
85. Lessons. The Bank’s ICR draws important lessons from project design and project
implementation and adequately reports lessons learned based on explicit
evaluation findings at project completion. Given the rigorous and comprehensive
list of lessons captured at project completion, Lessons is rated satisfactory (5).
86. Candour. The ICR reports candid findings supported by data captured during the
multiple ISM visits undertaken during the project lifespan. Both negative and
positive findings were captured throughout the ICR and the narrative was
sufficiently objective. Findings were, for the most part, coherent with the narrative
and reflected sound judgement and analysis. Candour is rated satisfactory (5).
IV. Lessons learned 87. Market information systems are better located with organizations directly
controlled by beneficiaries. Having the market information systems anchored
40
President's Report, P.12 41
President's Report 2009, P.10
16
within a line ministry has proved challenging in other country contexts and appears
to have faced similar challenges under the project. Lessons from other countries
have shown market information systems should be hosted and managed by the
direct beneficiaries who need information and have interest to share information.
The need for information should be determined by producers and private operators
and they could create their own associations to manage the system. The Ministry of
Foreign Trade does not have the same needs and interest to access quick and
reliable market information.
88. Creating a logical results chain in a post conflict setting and uncertain policy
environment requires continued attention and iterative processes. As a project that
was processed under OP8.042 that also targeted sector reforms, the project faced
an initial challenge in articulating a results chain that accurately identified outputs,
intermediate outcome and results. Revisions to the results framework were
required as the impact of sector reforms was better understood and the specific
investments in GAPA capacity building were finalized. An iterative process of
refining results can address gaps in knowledge in uncertain policy environments
and/or post conflict settings where ex ante analysis may be limited by time,
available information and levels of client or stakeholder engagement.
89. Illiteracy had adverse effects on some of the GAPAs ability to function. In
some instances, the formation and management of the GAPA executive committees
was tasked to educated individuals in the village who were not gum arabic farmers
but teachers or government workers. This led, in one case, to embezzlement and
misappropriation of funds that was resolved by a court law. Ensuring the literacy of
the executive committees of GAPAs should be prioritized through the inclusion of
long term literacy training as part of the broader capacity building plan.
90. Rural credit interventions should be managed by institutions with
sufficient experience and capacity. The FNC had limited prior experience in
managing matching grants and revolving credit schemes. Given the prevalence of
use of traditional financing methods (Sheil), GAPAs were initially hesitant to use
the microfinance system at the locality levels. Furthermore, failure of the GAPAs to
understand that the Matching Grant and Revolving Credit were to be used as seed
money for initial investments that would foster income generating activities in the
future, led to multiple further requests for money from the project. In order for a
functioning microfinance system to be implemented in The Sudan, further
investments in training of the FNC to facilitate and take the lead is required.
91. Collective marketing is proven to be a more effective means of improving
market access by small scale producers. Small scale Gum Arabic producers
face multiple marketing constraints and often have limited direct links to the
markets. Furthermore, there is evidence to suggest that improving market access
for small scale farmers will lead to increased income and food security. Efforts
should be made to harmonize gum arabic production and marketing across the
gum arabic belt states. Further emphasis should be placed on studies that address
marketing constraints and trainings covering marketing strategies.
42
Projects supported by the World Bank’s Investment Project Financing are governed by OPs (Operating Procedures). The Bank has classified this project under OP8.0: Rapid Response to Crises and Emergencies.
Annex I
17
Definition and rating of the evaluation criteria used by IOE
Criteria Definition * Mandatory
To be rated
Rural poverty impact Impact is defined as the changes that have occurred or are expected to occur in the lives of the rural poor (whether positive or negative, direct or indirect, intended or unintended) as a result of development interventions.
X Yes
Four impact domains
Household income and net assets: Household income provides a means of assessing the flow of economic benefits accruing to an individual or group, whereas assets relate to a stock of accumulated items of economic value. The analysis must include an assessment of trends in equality over time.
No
Human and social capital and empowerment: Human and social capital and empowerment include an assessment of the changes that have occurred in the empowerment of individuals, the quality of grass-roots organizations and institutions, the poor’s individual and collective capacity, and in particular, the extent to which specific groups such as youth are included or excluded from the development process.
No
Food security and agricultural productivity: Changes in food security relate to availability, stability, affordability and access to food and stability of access, whereas changes in agricultural productivity are measured in terms of yields; nutrition relates to the nutritional value of food and child malnutrition.
No
Institutions and policies: The criterion relating to institutions and policies
is designed to assess changes in the quality and performance of institutions, policies and the regulatory framework that influence the lives of the poor.
No
Project performance Project performance is an average of the ratings for relevance, effectiveness, efficiency and sustainability of benefits. X Yes
Relevance The extent to which the objectives of a development intervention are consistent with beneficiaries’ requirements, country needs, institutional priorities and partner and donor policies. It also entails an assessment of project design and coherence in achieving its objectives. An assessment should also be made of whether objectives and design address inequality, for example, by assessing the relevance of targeting strategies adopted.
X Yes
Effectiveness The extent to which the development intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance.
X
Yes
Efficiency
Sustainability of benefits
A measure of how economically resources/inputs (funds, expertise, time, etc.) are converted into results.
The likely continuation of net benefits from a development intervention beyond the phase of external funding support. It also includes an assessment of the likelihood that actual and anticipated results will be resilient to risks beyond the project’s life.
X
X
Yes
Yes
Other performance criteria
Gender equality and women’s empowerment
Innovation
Scaling up
The extent to which IFAD interventions have contributed to better gender equality and women’s empowerment, for example, in terms of women’s access to and ownership of assets, resources and services; participation in decision making; work load balance and impact on women’s incomes, nutrition and livelihoods.
The extent to which IFAD development interventions have introduced innovative approaches to rural poverty reduction.
The extent to which IFAD development interventions have been (or are likely to be) scaled up by government authorities, donor organizations, the private sector and others agencies.
X
X
X
Yes
Yes
Yes
Environment and natural resources management
The extent to which IFAD development interventions contribute to resilient livelihoods and ecosystems. The focus is on the use and management of the natural environment, including natural resources defined as raw materials used for socio-economic and cultural purposes, and ecosystems and biodiversity - with the goods and services they provide.
X Yes
Adaptation to climate change
The contribution of the project to reducing the negative impacts of climate change through dedicated adaptation or risk reduction measures.
X Yes
Annex I
18
Criteria Definition * Mandatory To be rated
Overall project achievement
This provides an overarching assessment of the intervention, drawing upon the analysis and ratings for rural poverty impact, relevance, effectiveness, efficiency, sustainability of benefits, gender equality and women’s empowerment, innovation, scaling up, as well as environment and natural resources management, and adaptation to climate change.
X Yes
Performance of partners
IFAD
Government
This criterion assesses the contribution of partners to project design, execution, monitoring and reporting, supervision and implementation support, and evaluation. The performance of each partner will be assessed on an individual basis with a view to the partner’s expected role and responsibility in the project life cycle.
X
X
Yes
Yes
* These definitions build on the Organisation for Economic Co-operation and Development/Development Assistance Committee (OECD/DAC) Glossary of Key Terms in Evaluation and Results-Based Management; the Methodological Framework for Project Evaluation agreed with the Evaluation Committee in September 2003; the first edition of the Evaluation Manual discussed with the Evaluation Committee in December 2008; and further discussions with the Evaluation Committee in November 2010 on IOE’s evaluation criteria and key questions.
Annex II
19
Rating comparisona
Criteria
Programme Management
Department (PMD) rating
IOE Project Completion Report Validation (PCRV)
rating
Net rating disconnect
(PCRV-PMD)
Rural poverty impact 5 5 0
Project performance
Relevance 5 5 0
Effectiveness 5 5 0
Efficiency 4 4 0
Sustainability of benefits 5 5 0
Project performanceb 4.75 4.75 0
Other performance criteria
Gender equality and women's empowerment 4 4 0
Innovation 4 4 0
Scaling up 5 5 0
Environment and natural resources management 5 5 0
Adaptation to climate change 4 4 0
Overall project achievementc 5 5 0
Performance of partnersd
IFAD 4 4 0
Government 4 4 0
Average net disconnect 0
a Rating scale: 1 = highly unsatisfactory; 2 = unsatisfactory; 3 = moderately unsatisfactory; 4 = moderately satisfactory; 5 =
satisfactory; 6 = highly satisfactory; n.p. = not provided; n.a. = not applicable. b Arithmetic average of ratings for relevance, effectiveness, efficiency and sustainability of benefits. c This is not an average of ratings of individual evaluation criteria but an overarching assessment of the project, drawing upon
the rating for relevance, effectiveness, efficiency, sustainability of benefits, rural poverty impact, gender, innovation, scaling up, environment and natural resources management, and adaptation to climate change. d The rating for partners’ performance is not a component of the overall project achievement rating.
Ratings of the project completion report quality
PMD rating IOE PCRV rating Net disconnect
Candour - 5 NA
Lessons - 5 NA
Quality (methods, data, participatory process) - 5 NA
Scope - 5 NA
Overall rating of the project completion report 5 NA
Rating scale: 1 = highly unsatisfactory; 2 = unsatisfactory; 3 = moderately unsatisfactory; 4 = moderately satisfactory; 5 = satisfactory; 6 = highly satisfactory; n.p. = not provided; n.a. = not applicable.
Annex III
20
Abbreviations and Acronyms
COSOP Country Strategic Opportunities Program
FNC Forestry National Cooperation
FOB Free On Board
GAPA Gum Arabic Producers Association
ICR Implementation Completion Report
IR Intermediate Result
ISM Implementation Supervision/Support Mission
LIU Locality Implementation Unit
MDTF Multi-donor Trust Fund
MG Matching Grant
MTR Mid Term Review
PCR Project Completion Report
PCRV Project Completion Review Validation
PCU Project Coordination Unit
PDO Project Development Objective
PMD Programme Management Department
PSC Project Steering Committee
21
The Sudan Gum Arabic Project - Log Frame at Appraisal and Project End
President Report Log Frame at Appraisal World Bank ICR Log Frame at Completion
Project Development Objective
Increase the production and income of small scale gum producers in selected areas of the gum belt through improved performance of the production and marketing systems.
Indicators
Household asset index (RIMS);
Prevalence of malnutrition in children under 5 (RIMS);
Percentage change in income level of beneficiary households from Gum Arabic production after the project intervention;
Percentage of the FOB price received by small scale gum producers after project intervention.
Project Development Objective
Same as President Report.
Indicators
Percentage change in income level of beneficiary households from Gum Arabic production after the project intervention;
Direct project beneficiaries;
Direct project beneficiaries of which are female.
Intermediate Result Component 1 Gum Arabic Sector Reform and Support
Equal opportunity and competition to all market actors.
Indicators
Implementation of the agreed-on sector reform packages within two years of project effectiveness and open up the market for potential market actors;
Value Chain, International Market structure and demand analysis, standards and quality of products at Auction markets, and socioeconomic studies discussed; finalized and shared with major stakeholders;
Amount of Marketing Capacity Improvement and Product Promotion Matching Grant disbursed.
Intermediate Result Component 1
Same as President Report.
Indicators
Percentage change in quantities of Gum Arabic marketed by GAPAs;
Percentage change in number of new entrants to the market (exporters).
Annex IV
22
Intermediate Result Component 2 Support to GAPAs
Organized and empowered GAPAs;
Market conditions improved;
Environmental degradation issues in target areas addressed.
Indicators
Number and % of GAPAs in target localities directly selling their product to processors and exporters;
Number and % of GAPAs who received improved production, agro forestry and quality maintenance trainings;
% change in production of Gum Arabic by GAPAs after project intervention;
New GAPAs formed and registered with significant representation of women and poor producers;
% of women and poor GAPA members receiving revolving funds;
% change in Community Capability Index (CCI);
% of GAPA members borrowing money from (sheil) local traders to finance production and marketing costs;
% repayment rate of GAPAs loans from the revolving credit system.
Intermediate Result Component 2
Same as President Report, however, the IR result describing environmental degradation was removed.
Indicators
% targeted GAPAs with effective saving and credit systems;
% of women targeted in GAPAs;
Number of Hashab seedlings produced by project nurseries;
Number of GAPA members who received training on business management;
Number of Agro forestry training sessions for GAPAs conducted by extension officers;
Number of GAPA members who received training on financial management;
Number of in-country and exchange visits of GAPAs conducted to share experience;
Number of Matching Grants for Marketing: Media Promotion;
Number of Matching Grants for Marketing: Research and Technology;
Number of Matching Grants for Marketing: Infrastructure support.
Intermediate Result Component 3 Project Management
The project is well managed and coordinated at the National, State and Locality levels through the existing FNC institutional structure.
Indicators
The Project Steering Committee is fully operational with regularly scheduled meetings;
Project Monitoring and Learning System established, sharing forums at different levels of project management.
Intermediate Result Component 3
Better management and coordination of the sector at national, state and locality levels.
Indicators
% of the FOB price received by small scale gum producers after project intervention;
% change in domestic use compared to export;
Number of visitors to the dynamic website of MoFT;
Number of text messages communicated to producers.
Annex V
23
Bibliography
Forest National Corporation. 2015. Revitalizing the Sudan Gum Arabic Production and
Marketing Project, Technical Assistance Report for Preparing the Implementation
Completion Results Report, P.6
Independent Evaluation Group (IEG), Sudan Gum Arabic Export Marketing Project,
Implementation Completion Report (ICR) Review. P.3, P.5, P.8
International Fund for Agricultural Development (IFAD). 2009. President’s Report -
Proposed grant to the Republic of The Sudan for the Revitalizing The Sudan Gum
Arabic Production and Marketing Project. P.9, P.11, P.12, P.13.
______. 2009. Republic of the Sudan Country Strategic Opportunities Programme
(COSOP). P.6, P.18
______. 2012. Results-based Country Strategic Opportunities Programme (RB-COSOP)
Mid-term Review Report. P.13, P.22
______. 2013. Sudan Amendment Financing Agreement. P.2
World Bank. 2016. Implementation Completion Report for Revitalizing The Sudan Gum
Arabic Production and Marketing Project. P.1, P.3, P.V, P.7, P.8, P.10, P.11, P.18,
P.20, P.21
______. 2009. Final Project Proposal on Revitalizing The Sudan Gum Arabic Production
and Marketing Project. P.23