Evaluating Impact:
Turning Promises into Evidence
Shafiullah Rasikh, Qazi Azmat Isa, Ghizaal Haress, Naila Ahmed, Wali Ibrahimi (Moderator Abdu)
Kathmandu, Nepal
February 2010
Afghanistan Rural Enterprise
Development Program (AREDP)
2
KEY ISSUES:
75% of population live in rural areas where agriculture is the primary activity;
High unemployment rates and unskilled labor force;
Shortage of business planning, management, and marketing know-how; and access to formal credit
PROJECT DESCRIPTION
PDO:
To improve employment opportunities and income of rural men and women; and sustainability of targeted local enterprises
Program Components:
(a) Community-led Enterprise Development;
(b) Small and Medium Enterprise Development;
3
4
Initiate savings
groups and monitor
internal lending
Initiate enterprise
groups and develop
enterprise plans
Identify SMEs and
Initiate financial
literacy with SMEs
Identify Partial
Guarantee
Administrator
Situation analysis
and value chain
assessment
Inventory and
contract of BDS
providers
Discussion with banks
to identify support
need
Form and register
Village Savings and
Loan Associations
BDS delivery and
VSLA lending to
enterprise groups
Commercial bank
lending and BDS
delivery to SMEs
Innovation
awards
Seed grant
to mature
VSLAs
Human Resources;
Financial Resources
-Livelihood grants to the comm unities
- Partial Guarantee Funds for SMEs
Community mobilization
30% of CDCs in a province organized into groups (35 % female)
Well-governed community groups;
Unemployment decreased
Net revenues increased for SMEs
Incomes increased for the households
Women Empowered
Increased private sector investment;
Improved productivity and economic stability;
Delivery of technical support and facilitation
70% of groups federate to VSLAs,
80% of VSLAs receive grants and linked to MFIs;
Improved savings & inter-loaning and access to formal credit by community groups
Business Development Services to SMEs
750 SMEs supported by the project;
Employment increased throughSMEs
SMEs accessing more financial resources
High net revenues to SMEs 5
2. Results Chain
Inputs Activities Outputs Outcomes Long-term
Outcomes
6
3. Primary Research Questions
Q1: Did AREDP have an impact on increasing incomes of participating households?
Q2: Did AREDP increase women participation in enterprise activities?
Q3: Did the Business Development Services under AREDP have an impact on increasing net revenues of local enterprises?
7
4. Outcome Indicators
Number of Well-governed community groups
Percentage of groups demonstrating improved savings & access to formal credit;
At least 30% increase in employment through SMEs
SMEs accessing loans from financial institutions and have at least 50% higher net revenues
5. Identification Strategy/Method
Selection of Communities: STAGE 1: NSP covers 75% of Afghanistan with est. 22,000 CDCs;
STAGE 2: Roll-out: YR1 7 Provinces; YR2 Contingent on funding and performance to roll-out to all 34 Provinces;
Selection criteria: security, better entrepreneurial opportunities;
STAGE 3: Districts and villages will be selected.
Selection criteria: accessibility, better performing CDCs, and demand for project;
STAGE 4: Households self select into Program:
Selection criteria: Interest to participate, ability to save.
STAGE 5: AREDP community groups form as savings or enterprise groups.
Selection of SME’s: Baseline survey of 1708 SMEs in 33 Provinces (2008);
8
Identification Strategy/Method
9
7 selected provinces
27 remaining provinces
Potential counterfactual:
At each stage, there are potential members of communities who would have been benefited from the Program but have been left out due to several reasons.
Selected Districts
Remaining Districts
Selected Villages
Remaining Villages
FUNDING
Treatment Potential Control Groups
e.g. Accessibility
e.g. CDC performance
Criteria
5. Identification Strategy/Method
Random Sample of non-participants will be drawn from potential control groups;
Impact Evaluation Method
With respect to the Community groups: Q1 and Q2: Matching
With respect to the SMEs: Q3: Difference in difference
10
11
6. Sample and data
Sample selection and Size
Universe: Avg. 1000 CDCs per Province under NSP; 7000 CDCs are our universe from which AREDP will select 30% (2000 CDCs);
Size: Consultation with the Central Statistics Office to determine a nationally representative sample size;
Data Collection Method Quantitative methods: Design questionnaires to capture
Household, Community and SME data;
Qualitative methods: To capture process related information: stakeholders include elected reps. Focus group discussions,
12
7. Time Frame/Work Plan
Timeframe:
Baseline: 2010
Mid-term review: 2012
Final survey: 2015
Process monitoring (throughout)
Work plan (e.g. activity: survey implementation and data analysis)
Consultative process to include variety of stakeholders;
Design of survey instruments
Pre-testing survey instruments
Contracting of survey firm
Survey equipment, tools
Full survey implementation
Post survey (data cleaning, verification)
Analysis to answer Q1, Q2 and Q3
Report writing
Dissemination
13
8. Sources of Financing
Approx. 2% of project cost which is in the project budget. Approx. $1.5 million
Other possible funding.