DIME – FRAGILE STATESDUBAI, MAY 31 – JUNE 4

Sudan Community Development Fund:Preliminary Slice I Impact Evaluation Results and Needs for Future Evaluations

Abdulgadir Turkawi, Krishna Pidatala, Tei Fujiwara, Ryan Sheely

Community Development Fund (CDF) National program that uses CDD approach to rapidly provide basic social infrastructure and services to war-affected and underdeveloped areas of North Sudan – i.e. In 4 states out of 15 states

Two Phases Slice-1 (2006 -2008) - $25 million Slice-2 (2008 -2011) - $50 million

CDF – Project Background

Project Background (2) Slice I (2006-2008) - US$ 25 million

4 states out of 15 states; 10 neediest localities within these 4 states; 20 communities per locality = 200 total communities

Slice II (2008-2011) - US$ 50 million 4 states out of 15 states; 6 more

communities added to the original 10 Slice-I localities; 19 new localities added in the 4 states in Slice-II to increase coverage within the states.

Total Slice-I communities = 260 Total Slice-II communities = 380

Implementation Overview Baseline 1 – May 2007 (3 treatment

communities per locality; 2 control communities per locality; 27 random Households in each community )

Follow-up Survey/Baseline Survey 2 – June 2008

Follow-up survey for Slice1 - panel survey for same households as in baseline 1 for Slice1 ;

Baseline survey for Slice2 - 4 treatment communities per each new locality; 2 control communities per each new locality; 24 random Households in each community

Going Forward – Final survey expected in 2011

Evaluation Strategy and Survey Design Targeting – based upon poverty &

population assessments; 20 lowest ranked communities in each locality were selected to receive the CDF program. 21st and 22nd lowest communities in each locality picked as the control group

Treatment & Control groups – selected communities with similar characteristics

Cluster Random Sampling – Households in Treatment and Control groups were selected randomly

Selection of Treatment Communities

Total of 20 Communities per Locality Needed to Ensure that there was at least one

treatment Community per Administrative Unit Within each Administrative Unit, communities

were ranked based on poverty, population, availability and condition of Basic Services and Population

Number of Communities Chosen for Treatment Per Administrative Unit - based on above ranking (Poverty, population, availability & condition of basic infrastructure & services)

Selection of Control Group

To construct a control group, all communities within each locality were ranked based on the number and condition of basic services and population

The 21st and 22nd communities on the list were selected as a control group These were the communities that were

not selected that were most similar to the selected communities

Survey Methodology Community and Household Questionnaires 50 Communities Chosen from all 10

Localities – 5 from each locality 3 Treatment Communities Randomly

Chosen From Each Locality 2 Control Communities From Each Locality

Selection of Households 27 Households Randomly Selected from

Each of the 5 Communities Sampling Frame-Household Lists where

available, “Spin the Pen” method where not available

Evaluation QuestionsThe Slice 1 Baseline and Follow-Up

Surveys were designed to assess the overall effectiveness of the project at meeting its objectives:

Measurements : Access to Education, Health, and Water? Good Governance? Participation and Social Capital?

Results of Slice I Impact Evaluation

Results Estimated Using Difference-in-Difference Approach

Education Gains in enrollment, reduction in

dropouts▪ Decline in female dropouts

Increases in number of classrooms, toilets, benches, and teachers dormitories

Treatment communities 34% more satisfied with education after intervention, compared to control group

Results of Slice I Impact Evaluation Health

Fewer statistically significant increases in health center functionality

Increase in frequency of health center visits Satisfaction with health facilities significantly

increased Water

Fewer statistically significant increases in water quality

Increased consumption of water Increased Number of Pump sets Increased Satisfaction with Access to Water

Results of Slice I Impact EvaluationGovernance

Increase in Reported Rates of Leader Compliance with Community Needs and Leader Responsiveness

Decrease in Ease of Changing Leader Participation and Social Capital

Increase in Community’s Ability to Solve Development Problems

No significant increase in Participation in Community Activities or Meetings

Evaluation Challenges & Lessons Learned

Sample attrition – possibility that some households could have moved by 2011

Data Management – difficulty in matching of some households from baseline & follow-up surveys

Gender sensitivity & participation – 1st baseline survey did not have any female respondents. Addressed this shortcoming in 2nd baseline survey for Slice-II.

Survey questionnaires too long – Need to condense follow-up questionnaires. Interview takes 1 hour 15 minutes.

Looking forward – Impact Evaluation

2011 - Final survey expected to be undertaken

Impact Evaluation – Need for continued support from DIME for facilitation & technical expertise to the project

Phase-II – Dependent upon the Referendum in January 2011 and the political landscape there after. We expect a more rigorous IE design for the next phase/project. Evaluation design - possible sub treatment

interventions Survey design and management Sampling

Questions & Feedback Needed:Slice II Follow-Up Survey

Budget constraints – For the final survey, should we reduce – (a) the number of households per community ,or (b) the number of communities per locality ?

Preparing for Future Impact Evaluations

Build Local capacity – involve local counterparts in IE technical design & analysis (as far as possible)

Gender Sensitivity/Participation – IE expert on team to be a woman (based upon past experience)

Focus on Project/Program – Develop project/program questions to be answered by IE

National Statistics Bureau – Look at possibility to involve them in some way to build their capacity (most projects ignore them)

DIME Support– need continued support from them. DIME to provide technical expertise, oversee IE work, ensure quality of work, etc.

Questions For Future EvaluationsWhat is the effect of installation of

solar electricity on health, education, and security outcomes?

What is effect of social accountability mechanisms on Infrastructure Functionality?

Due to implementation in progress, may not be able to evaluate until Phase II

Will look for opportunities for evaluations in Slice II

THANK YOU