mdg-relevant j-pal rcts review

WHAT DO RANDOMIZED EVALUATIONS TELL US ABOUT MDG-RELEVANT POLICY INTERVENTIONS?

A REVIEW OF J-PAL EVALUATIONS OF GOVERNANCE AND SOCIAL DEVELOPMENT INTERVENTIONS

DRAFT15 JUNE 2010Andy Sumner

EXECUTIVE SUMMARY

Randomised Evaluations (REs) are a form of impact evaluation that allows research to isolate the impacts of a specific intervention from other factors. A major source of such research is the Abdul Latif Jameel Poverty Action Lab also know as J-PAL. J-PAL is a network of researchers around the world united by their use of Randomized Evaluations to answer questions related to interventions for poverty reduction. This paper reviews the 100+ J-PAL REs relavant to the social and governance concerns of the MDGs.

The body of studies say three key things of particular note:

1. Not surprising – here are numerous social development interventions that are successful (ie de-worming tablets increase school attendance for example);

2. Surprising - given heated policy debates - user fees for public goods and services, even very small user fees, lead to very drastic and unequivocal falls in service usage amongst the poor;

3. New - an emerging area for REs is in accountability interventions particularly mechanisms around reaching the poor with public services such as report cards for clinics and other accountability mechanisms.

It is worth noting that REs are not without critique:

1. The most important being that they’re generally expensive as a form of evaluation; 2. REs generally can tell you if but not exactly why an intervention worked;

3. REs can only answer limited amount of generally very micro questions (and thus there is a concern methods start to dictate the policy/research questions).

Table 1. Summary of J-Pal Randomised Evaluations relevant to the MDGs by MDG

Table 2. Full list of MDG-relevant RCTs conducted by J-PAL

Table 3. J-PAL REs yet to report with relevance to MDGs listed on J-Pal website (May 2010).

1

Table 1. Summary of J-Pal Randomised Evaluations relevant to the MDGs by MDG

MDG Evidence of successful interventions on poverty reduction

No evidence of success

1 Fertilizers for farmers (Kenya); Provision of mechanisms to save harvest income for future fertilizer purchase (Kenya); Community monitoring to improve health worker performance (Uganda); Community-level iron fortification program (India); Improving the supply of infrastructure for immunization, and improving supply and simultaneously increasing demand through the use of modest, non-cash incentives (India); Delivery of iron and Vitamin A supplementation and de-worming drugs to 2-6 year old children (India)

Access to credit and training course for farmers (Kenya)

2 Merit-based scholarships for girls (Kenya); Cash incentives for teachers (Kenya); Provision to schools with funding to hire a local contract teacher (Kenya); Primary School De-worming Project (Kenya); Operational tools and training for district administrators (Madagascar); Village Education Committees (India); Monetary incentive attached to teacher attendance (India); Remedial education intervention (extra tutoring) (India); Computer Assisted Learning (India); Delivery of iron and Vitamin A supplementation and de-worming drugs to 2-6 year old children (India); Teacher performance pay program (India); PACES- vouchers to enable children to enroll in school (Colombia)

Provision of additional official government textbooks (Kenya)

3 Merit-based scholarships for girls (Kenya); Cash incentives for teachers (Kenya); Provision to schools with funding to hire a local contract teacher (Kenya); Operational tools and training for district administrators (Madagascar); Vouchers for appointments with family planning nurse and access to contraception (Zambia); Village Education Committees (India); Randomized selection of some village chiefs (pradhans) positions to be reserved for women (India); Remedial education intervention (extra tutoring) (India); Computer Assisted Learning (India); Delivery of iron and Vitamin A supplementation and de-worming drugs to 2-6 year old children (India); Teacher performance pay program (India); Commitment savings product called a SEED (Philippines); PACES- vouchers to enable children to enroll in school (Colombia)

Provision of additional official government textbooks (Kenya); Mandated representation of women in village councils (India); Multiple pre-election voter education campaigns (India); Provision of sanitary products to girls (Nepal)

4 Free provision of ITNs (Kenya); Vouchers for Cost-sharing of ITNs; Subsidized interest-free

2

long-lasting ITNs (Kenya); Water chlorination supply to households (Kenya); Improve water quality at the source (springs) (Kenya); Community monitoring to improve health worker performance (Uganda); Incentives program to improve Assistant Nurse Midwife (ANM) attendance (India); Community-level iron fortification program (India); Improving the supply of infrastructure for immunization, and improving supply and simultaneously increasing demand through the use of modest, non-cash incentives (India); Delivery of iron and Vitamin A supplementation and de-worming drugs to 2-6 year old children (India); Voluntary commitment savings program to stop smoking (Philippines)

loan to install a water connection at home (Morocco); Paying for Clorin (chlorine to ensure clean water) ( Zambia)

5 HIV infections in school (Kenya); Training teachers on how to teach the HIV curriculum (Kenya); HIV testing and counseling (Malawi); Vouchers for appointments with family planning nurse and access to contraception (Zambia); Water chlorination supply to households (Kenya); Community monitoring to improve health worker performance (Uganda); Incentives program to improve Assistant Nurse Midwife (ANM) attendance (India); Community-level iron fortification program (India); Voluntary commitment savings program to stop smoking (Philippines)

Subsidized interest-free loan to install a water connection at home (Morocco); Paying for Clorin (chlorine to ensure clean water) ( Zambia)

6 Free provision of ITNs (Kenya); Vouchers for long-lasting ITNs (Kenya); information on HIV infections in school (Kenya); Training teachers on how to teach the HIV curriculum (Kenya); HIV testing and counseling (Malawi); Primary School De-worming Project (Kenya); Community monitoring to improve health worker performance (Uganda); Vouchers for appointments with family planning nurse and access to contraception (Zambia); Water chlorination supply to households (Kenya); Improve water quality at the source (springs) (Kenya); Improving the supply of infrastructure for immunization, and improving supply and simultaneously increasing demand through the use of modest, non-cash incentives (India); Delivery of iron and Vitamin A supplementation and de-worming drugs to 2-6 year old children (India); Voluntary commitment savings program to stop smoking (Philippines)

Cost-sharing of ITNs; fee for de-worming drugs (Kenya); paying for long-lasting ITNs (Kenya): Subsidized interest-free loan to install a water connection at home (Morocco); Paying for Clorin (chlorine to ensure clean water) ( Zambia)

7 Water chlorination supply to households (Kenya); Improve water quality at the source (springs) (Kenya)

Subsidized interest-free loan to install a water connection at home (Morocco); Paying for Clorin (chlorine to ensure clean water)( Zambia);

3

Provision of sanitary products to girls (Nepal)Most MDGs

Small consumer loans to households ot marginal applicants (South Africa); Police training and increasing the duration of job postings (India); Proxy means testing (Indonesia); Incentives to open a formal savings account (Indonesia); Auditing of funds for public projects (Indonesia); Deposit-collecting program (Philippines); Loans to marginally creditworthy applicants (Philippines); Commitment savings product called a SEED (Philippines); Offer of credit through village banks (Peru); business training to a group lending program (Peru)

Introduction of microcredit to a new market (India)

4

Table 2. Full list of MDG-relevant RCTs conducted by J-PAL

Full reference Relevant MDGs Policy intervention tested

Findings

AfricaKenya Dupas & Cohen.

‘Free Distribution or Cost Sharing? Evidence from a Malaria Prevention Experiment in Kenya’. 2006

MDG6 (HIV/AIDS, malaria, and other diseases)MDG4 (Infant mortality)

Cost-sharing (patients contributing to cost of drugs) v free Insecticide Treated Nets (effect on usage and demand of Insecticide Treated Nets of cost-sharing v free provision)

No evidence was found to suggest that cost-sharing (patients contributing to cost of drugs) increases ITN usage; Cost-sharing does considerably dampen demand- results suggest that free distribution of ITNs is both more efficient and more cost-effective than cost-sharing. These results imply that demand for ITNs is 75% lower at the cost-sharing price prevailing in Kenya at the time of the study ($0.75) than it is under a free distribution scheme

Miguel & Kremer. ‘The Illusion of Sustainability: Comparing Free Provision of Deworming Drugs and "Sustainable" Approaches in Kenya’. 1997-2001


Sustainable provision of de-worming drugs (cost-sharing (patients contributing to cost of drugs); health education and verbal commitment; social learning)

The introduction of a small fee for deworming drugs led to an 80% reduction in treatment rates, consistent with the hypothesis that people have low private valuation for de-worming.

Dupas, P. ‘HIV/AIDS Prevention Through Relative Risk Information for Teenage Girls in Kenya’. 2004-2005

MDG6 (HIV/AIDS, malaria, and other diseases)MDG5 (Maternal mortality)

Ensuring the adoption of safer sexual behavior among youth: Information on the distribution of HIV infections by age and gender-“Relative Risk Information

As a result of this intervention, the incidence of childbearing was reduced by 28% (from 5.4 percent of girls getting pregnant within a year, to 3.9 percent).

5

Campaign” in 71 schools randomly selected among 328 primary schools involved in another HIV intervention evaluation

This suggests that the intervention reduced the likelihood that girls engage in unsafe sex.

Miguel et al. ‘Incentives to Learn: A Merit-Based Girls' Scholarship Program in Kenya’. 2001-2002

MDG2 (Primary education)MDG3 (Gender equality)

The Girls’ Scholarship Program (GSP)- Out of a set of 127 schools, 64 were randomly invited to participate in a program which gave merit-based scholarships to 6th grade girls

The program raised test scores by 0.19 standard deviations for girls enrolled in schools eligible for the scholarship. While the program impact on school participation is nearly zero among girls in some areas, the impact in Busia is positive at 3.2 percentage points; greater teaching effort was directed to the class as a whole; Anecdotal evidence from teacher interviews suggests greater parental monitoring occurred in Busia as a result of the program

Miguel et al. ‘Source Dispensers and Home Delivery of Chlorine in Kenya’. 2004-2008

MDG6 (HIV/AIDS, malaria, and other diseases)MDG2 (Primary education)MDG3 (Gender equality)MDG7 (Water and sanitation)

Water chlorination: examine the impact of factors including price, persuasion, promotion and the chlorination products themselves with a two-phase study

Most households have a low willingness to pay for chlorine, despite its well known benefits. After receiving a free 7-month supply, chlorine was detected in 58% of households, much more than the 2% starting level; Investments in marketing campaigns and coupon schemes proved to be

6

ineffective strategies to encourage point-of-use chlorination. Chlorine was detected in 40% of households visited by a promoter, compared to only 4% in those who weren’t visited

Dupas, P. ‘The Role of Exposure, Social Networks and Marketing Messages in Households' Willingness to Pay for Malaria Prevention in Kenya’. 2007-2008


Vouchers for long-lasting ITN (LLIN)

The demand for malaria-preventing bed nets in Western Kenya is relatively price sensitive; an increase in price from $0 to $1 leads to a drop of 35 percentage points in take up, and an increase from $1 to $2 leads to a further drop of 25 percentage points. Gaining access to a free or highly subsidized LLIN in the first year increased households’ reported, as well as observed, willingness to pay for a second LLIN; Neither of the two framing options (health or financial) had any impact at all on LLIN take up

Miguel et al. ‘Cleaning Springs in Kenya’. 2005-2006

MDG6 (HIV/AIDS, malaria, and other diseases)MDG4 (Infant mortality)MDG7 (Water and sanitation)

Improve water quality at the source (springs). Identified 200 springs in the Busia district of Kenya, and persuaded each local community to contribute 10% of the costs of the

The simple infrastructure investment of “spring cleaning” significantly reduced both water contamination and the incidence of diarrhea. There was 66% less E-coli

7

improvement project, usually in labor.

contamination in treated springs than in untreated ones, and an average of 24% less contamination in users’ home water supplies among households who collected water from multiple springs and those who only used protected springs. Diarrheal incidence in children under 3 years old fell by 4.7 percentage points, or 25%, though there was no significant effect on children ages 5 to 12.

Duflo et al. ‘Teacher Training for HIV/AIDS Prevention Education in Primary School Classrooms in Kenya’. 2003-2006


Teachers from 164 randomly selected schools were trained on HIV/AIDS and on how to teach the HIV curriculum. The curriculum covers facts about the disease, and encourages abstinence until marriage and faithfulness afterwards

Increased chances of teachers teaching about HIV in the classroom; students had greater knowledge about the disease and also reported more tolerant attitudes toward those with AIDS; intervention did not reduce childbearing rates among girls, suggesting that it did not decrease the likelihood that girls engage in unsafe sex.

Kremer et al. ‘Teacher Incentives Based on Students' Test Scores in Kenya’. 1996-2000


Effects of a teacher incentives program (e.g. cash incentives) on both teacher behavior and student test scores in Kenya

Student test scores increased significantly in treatment schools. However, evidence suggests that this improvement did not necessarily occur through the

8

intended channel of regular classroom teaching; Test score improvements dropped off after the program was completed. Prior to the program, incentive schools were slightly less likely to offer test preparations, but after the introduction of the program, treatment schools were 4.2 percentage points more likely to conduct prep sessions in the first year and 7.4 percentage points more likely in the second

Kremer et al. ‘Textbooks and Test Scores in Kenya’. 1995-2000


Provision of additional official government textbooks

After one school year there was no evidence that the textbook provision increased average test scores, or that it reduced either grade repetition or dropout rates. Textbooks increased progression to secondary school for eighth graders but did not reduce grade repetition or raise attendance in lower grades

Duflo et al. ‘Rates of Return to Fertilizer: Evidence from Field Experiments in Kenya’. 2000-2005

MDG1 (Nutrition) Researchers set out to experimentally measure the returns to fertilizer among area farmers

All fertilizer treatments led to increases in yield, though in different amounts. Interventions A, B, and C led to yield increases of 28%, 48% and 63%

9

respectively, relative to comparison plots. Intervention D, the Ministry of Agriculture recommended package, led to an average 91% increase in yield relative to comparison plots. This evidence demonstrates that fertilizer use can have substantial returns, even in the absence of any changes in other farming practices on real-world farms

Duflo et al. ‘Nudging Farmers to Use Fertilizer: Experimental Evidence from Kenya’. 2000-2005

MDG1 (Nutrition) Savings and Fertilizer Initiative (SAFI): researchers designed an intervention to test if providing mechanisms to save harvest income for future fertilizer purchase could be effective in increasing usage

The SAFI program was very popular. The basic SAFI was offered in two seasons. In the first season, the program increased usage by 14 percentage points, on a base of 23 percentage points. In the second season, the increase was even bigger, increasing usage by 18 percentage points. Overall, the results suggest that offering farmers small, time-limited discounts on fertilizer may substantially increase usage without inducing overuse among farmers who are already using fertilizer, at relatively low cost.

10

Karlan et al. ‘Finding Missing Markets: An Agricultural Brokerage Intervention in Kenya’. 2003-2005

MDG1 (Nutrition) Access to credit and training course- Researchers conducted a randomized study with DrumNet, a Kenyan NGO, to evaluate whether a package of services could help small farmers overcome barriers to adopting, financing and marketing export crops.

One year after the program began, treatment individuals were 19.2 percentage points more likely to be growing an export crop, but there were no significant gains in income. While credit might have made exporting easier to some clients, access to credit had no effect on income gains compared to no-credit groups

Robinson, J. ‘Limited Insurance Within the Household in Kenya’. 2006

MDG1 (Nutrition) Intra-household risk-sharing (i.e. sharing risk among couples) arrangements such as loans and gifts

Overall, the study suggests significant barriers to efficient risk sharing among these couples. Since this study evaluated relatively small shocks, the failure of intra-household risk sharing is likely to be even more pronounced for bigger shocks such as a poor harvest or major illness. In weeks in which they receive the shock, men increase their expenditures on privately consumed items by 21%

Kremer et al. ‘Flipcharts, School Inputs & Retrospective vs. Prospective Analyses in Kenya’. 1996-1998


Provision of flipcharts to each of 89 randomly selected Kenyan primary schools

The comparison between retrospective and prospective estimates of the effect of flipcharts in Kenyan primary schools finds that randomized prospective

11

estimates are much smaller than retrospective estimates; policy should reflect prospective study findings as retrospective studies may produce misleading results.

Duflo et al. ‘Peer Effects, Pupil-Teacher Ratios, and Teacher Incentives in Kenya’. 2005-2007


Provision to 140 schools with funding to hire a local contract teacher to address classroom overcrowding

Providing school committees with funds to hire an extra teacher on a short-term contract had a generally positive effect on learning, as measured by test scores. Contract teachers were present in school more than their civil-service counterparts, and their students scored higher on exams.

Miguel & Kremer. ‘Primary School Deworming in Kenya’. 1997-2001

MDG2 (Primary education)MDG6 (HIV/AIDS, malaria, and other diseases)

Primary School De-worming Project (PSDP). Schools with worm prevalence over 50% were mass treated with de-worming drugs every six months. In addition to medicine, treatment schools received regular public health lectures, wall charts on worm prevention, and training for one designated teacher.

De-worming reduced serious worm infections by half amongst children in the treatment groups; De-worming increased school participation by at least 7 percentage points, which equates to a one-quarter reduction in school absenteeism; The entire community and those living up to 6 kilometers away from treatment schools benefited from “spillovers” of the de-worming treatment.

12

Madagascar Duflo et al. ‘Primary Education Management in Madagascar’. 2005-2007


District administrators in treatment districts received operational tools and training that included forms for supervision visits to schools, and procurement sheets for school supplies and grants (district-level intervention).

Top-down approach: The interventions targeted at the district and sub-district level had minimal effects on the administrator’s behaviors, and the schools and students under their responsibility; Bottom-up: The interventions at the school level led to significantly improved teacher behavior: Test scores were 0.1 standard deviations higher than those in the comparison group two years after the implementation of the program. Additionally, student attendance increased by 4.3 percentage points compared to the comparison group average of 87%, though teacher attendance and communication with parents did not improve.

Malawi Thornton, R. ‘The Demand for and Impact of Learning HIV Status in Malawi’. 1998-2004


Uptake of HIV testing and counseling (impact of monetary incentives and distance and impact on sexual behaviour)

On average, respondents who received any cash-value voucher were twice as likely to go to the VCT center to obtain their HIV test results as those who received no cash incentive; Receiving an HIV positive diagnosis

13

significantly increased the likelihood of purchasing condoms among those who were sexually active. While only 34% of those who received no monetary incentive attended clinics to learn their results, monetary incentives were highly effective at increasing result-seeking behavior

Morocco Duflo et al. ‘Household Water Connections in Tangier, Morocco’. 2007-2008

MDG6 (HIV/AIDS, malaria, and other diseases)MDG4 (Infant mortality)MDG5 (Maternal mortality)MDG7 (Water and sanitation)

Subsidized interest-free loan to install a water connection at home

Since the participating households already had access to the water grid through free public taps, no improvements in the quality of water consumed by households have been found. Despite significant improvement in water quantity, no change in the incidence of waterborne illnesses was found. Time saving was found to occur, as well as bypassing a source of tension between households.

South Africa Karlan et al. ‘Small Individual Loans and Mental Health in South Africa’. 2004-2006

Most MDGs (indirect) Small consumer loans to households (Marginal applicants- those who may have been turned down for credit beforehand)

Expanding access to credit is found to significantly increase well-being of borrowers. Economic self-sufficiency (employment and income) was higher for treated

14

applicants than for those in the comparison group 6-12 months after treatment. Twenty-six percent of treated households report an improvement in food consumption.

Uganda Svensson & Bjorkman. ‘Community-Based Monitoring of Primary Healthcare Providers in Uganda’. 2004

MDG6 (HIV/AIDS, malaria, and other diseases)MDG4 (Infant mortality)MDG5 (Maternal mortality)

Community monitoring to improve health worker performance (and the impact this might have on health utilization and outcomes)

In the intervention group, utilization of general outpatient services was higher (20%), more people came for child birth deliveries (58%) and more patients sought prenatal care (19%). More people sought family planning services (22%) and immunizations increased for all age groups, especially newborns. Better health facilities at centres, reduced waiting times, improved health outcomes (e.g. increase in infant weight): the overall effect of this intervention was significantly positive, bringing great hope to the possibility of community led and sustained monitoring efforts

Zambia Field et al. ‘Contraceptive Adoption, Fertility, and the Family in Zambia’. 2006

MDG3 (Gender equality)MDG6 (HIV/AIDS)

Vouchers for appointments with family planning nurse and access to contraception (vouchers given in presence of

Take up of the voucher was high at 47%, indicating that women valued the substantial reduction in the time cost of an

15

husbands) appointment associated with the voucher. However, male knowledge of the voucher led to a substantial reduction in use of these services, suggesting that policies or technologies that shift relative control of contraceptive methods from men to women may significantly increase contraceptive use and reduce average fertility in some contexts

Ashraf et al. ‘Can Higher Prices Stimulate Product Use? Evidence from a Randomized Experiment in Zambia’. 2006

MDG7 (Water and sanitation)MDG4 (Infant mortality)MDG5 (Maternal mortality)

Door-to-door sale of Clorin (chlorine to ensure clean water) at a randomly chosen, below-market “offer price” to about 1,260 households with poor access to piped water or chlorine home-delivery (Researchers sought to find whether higher prices can help target those who would most use the product)

Researchers found that fewer people bought Clorin (chlorine to ensure clean water) as the price rose; for every 1% increase in price there was a 0.67% decrease in quantity demanded. However higher prices did appear to screen out those who would not have used the product in any event, and a higher willingness to pay was associated with greater propensity to use. Overall, the use of chlorine does decrease with higher prices due to dampened demand, but this decline is partially offset by better targeting of the product to families who are

16

likely to use it. South AsiaIndia Duflo et al. ‘The

Impact of Women Policy Makers on Public Goods in India’. 2000-2002

MDG3 (Gender equality) Researchers studied the policy consequences of mandated representation by determining whether there was any difference in the provision of social services between male and female led village councils

There was no evidence that the policy impact of the reservations is driven by features other than the gender of the president. It is because women’s own preferences are more aligned to the general preferences of women that they end up serving them better.

Banerjee et al. ‘Measuring the Impact of Microfinance in Hyderabad, India’. 2005-2008

MDG1 (Nutrition)MDG2 (Primary education)MDG3 (Gender equality)MDG4 (Infant mortality)MDG5 (Maternal mortality) (related)

Introduction of microcredit to a new market

No evidence was found to suggest that microcredit empowers women or improves health or educational outcomes. Additionally, 30% of loans were reportedly used to repay an existing loan, 15% to buy a durable good for household use, and 15% to smooth household consumption.

Banerjee et al. ‘Incentives for Nurses in the Public Health Care System in Udaipur, India’. 2005-2007

MDG4 (Infant mortality)MDG5 (Maternal mortality)

Incentives program to improve Assistant Nurse Midwife (ANM) attendance at rural subcenters serving 135 villages in Udaipur District

These results show that, like other public service providers, nurses are responsive to properly administered incentives; Even when the incentive was effective at increasing presence, it did not increase the rate at which patients came to the sub-centers, which in any case is very

17

low. At any given time between 0.46 and 0.9 clients were being seen in the center, fewer even than the 2-3 clients observed two years earlier.

Banerjee et al. ‘Campaigns to Influence Voting Behavior in Uttar Pradesh, India’. 2007

MDG3 (Gender equality) Multiple pre-election voter education campaigns (PEVACs)

The likelihood that an individual would vote for the party which represented their caste decreased from 57% to 52% in villages which received this campaign. Voting patterns made it clear that the poor do have distinct preferences for representatives who focus on issues that are important for them

Banerjee et al. ‘Reducing Anemia Through Iron Fortification of Grain in Udaipur, India’. 2002-2009

MDG6 (HIV/AIDS, malaria, and other diseases)MDG1 (Nutrition)MDG4 (Infant mortality)MDG5 (Maternal mortality)

Community-level iron fortification program (an option for iron supplementation for families who do not buy commercially processed grain)

A midline survey shows a significant difference in anemia rate between treatment and comparison villages, of about 7 percentage points, but by end-line, when the uptake had fallen, there was no difference in anemia between treatment and comparison groups. The program was effective in reducing anemia as long as take up was high enough, but ineffective when take up was low; Monthly health monitoring reveals a similar pattern for

18

weakness and fatigue: symptoms were reduced when take up was sufficiently high, but by end-line the program had no effect on health.

Banerjee et al. ‘Can Informational Campaigns Raise Awareness and Local Participation in Primary Education in India?’ 2005-2006

MDG2 (Primary education)

3 interventions to determine if more information and encouragement to use the channels available to them would cause VECs (Village Education Committees) and community members to demand and receive better services

Those who could read only letters at baseline were 3.5% more likely to read at least paragraphs or words, and 3.3% more likely to read stories if they were in an Intervention 3 village. These changes in average literacy across the village came despite the fact that only 8% of children, including 13% of those who could not recognize letters prior to the intervention, attended the classes. If we assume that all the improvement in the villages came from the reading classes then children who attended the classes must have seen very big improvements in reading

Banerjee et al. ‘Improving Immunization Rates Through Regular Camps and Incentives in India’. 2004-2007

MDG1 (Nutrition)MDG6 (HIV/AIDS, malaria, and other diseases)MDG4 (Infant mortality)

Improving the supply of infrastructure for immunization, and improving supply and simultaneously increasing demand through the use of modest, non-cash incentives

Study results indicate that offering families in resource-poor settings small, non-financial incentives in addition to reliable services and education is more effective than providing services

19

and education alone. It is also more cost-effective—more children utilize the same immunization facilities, lowering the cost per child immunized, even considering the cost of the incentives.

Duflo et al. ‘Perceptions of Female Leaders in India’. 2006-2007

MDG3 (Gender equality) Randomized selection of some village chiefs (pradhans) positions to be reserved for women

Mandated Exposure to a female leader does not affect villagers’ stated taste for male leaders. Men rated male leaders 1.44 points higher than female leaders. Women exhibited a smaller, though significant, bias in favor of male leaders, ranking them 0.56 points above female leaders. Political reservation led to significant electoral gains for women.

Duflo et al. ‘Encouraging Teacher Attendance through Monitoring with Cameras in Rural Udaipur, India’. 2003-2006


Monetary incentive attached to teacher attendance

Over the 30 months of the study, teachers at program schools had an absence rate of 21%, compared to 44% at baseline and 42% in the comparison schools. Two and a half years into the program, children from the treatment schools were also 62% more likely to transfer to a formal primary school, which requires passing a competency test.

20

The program resulted in an immediate and long lasting improvement in teacher attendance rates in treatment schools; Increasing teacher attendance improves educational outcomes (test scores)

Banerjee et al. ‘Balsakhi Remedial Tutoring in Vadodara and Mumbai, India’. 2001-2004


Balsakhi Program, a remedial education intervention (extra tutoring)

The number of students in the bottom third of program classes who passed basic competency tests increased by nearly 8%, while those in the top third who passed increased by 4%. The program had substantial positive impacts on children’s academic achievement. Scores on tests administered after the program showed that in both cities in both years, the program improved overall test scores

Banerjee et al. ‘Computer-Assisted Learning Project with Pratham in India’. 2001-2004


Pratham, an education-oriented NGO, designed a program that supplemented classroom instruction with Computer Assisted Learning (CAL)

Considerable unmet demand for loans. The CAL program was shown to be highly effective in raising students' skill levels in math (though not in languages), but was less cost-effective than the tutor-based Balsakhi program. n the first year, math scores increased approximately 0.36 standard deviations,

21

a substantial achievement when compared to other education interventions.

Miguel et al. ‘Balwadi Deworming in India’. 2001-2003

MDG6 (HIV/AIDS, malaria, and other diseases)MDG4 (Infant mortality)MDG1 (Nutrition)MDG2 (Primary education)MDG3 (Gender equality)

Pratham’s preschool nutrition and health project in the slums of Delhi, India. The program delivered a package consisting of iron and Vitamin A supplementation and de-worming drugs to 2-6 year old children through an existing preschool network.

Large gains in child weight—roughly 0.5 kg on average—were found in the treatment schools relative to comparison schools over the two-year study period; Average preschool participation rates increased sharply by 5.8 percentage points among treated children, reducing preschool absenteeism by roughly one fifth.

Muralidharan & Sundararaman. ‘Teacher Performance Pay in Andhra Pradesh, India’. 2004-2007


Teacher performance pay program implemented in the Indian state of Andhra Pradesh (AP). Two types of teacher performance pay (group bonuses based on school performance, and individual bonuses based on teacher performance)

Teacher pay based on student performance is found to be highly effective at improving student learning. After two years of the program, students in incentive schools performed on average 0.28 and 0.16 standard deviations higher than those in comparison schools in math and language tests respectively. At the end of two years, the average treatment effect was a 0.27 standard deviation increase in test scores in the individual incentive schools compared to

22

0.16 standard deviations in the group incentive schools.

Bertrand et al. ‘Obtaining a Driver's Licence in India: An Experimental Approach to Studying Corruption’. 2004-2006

Governance (related to most MDGs)

International Finance Corporation (IFC) followed 822 driver’s license candidates, collecting data on whether they obtained licenses, as well as detailed micro data on the specific procedures, time, and expenditures involved

Close to 71% of license getters in the comparison group did not take the licensing exam, and 62% were unqualified to drive (according to the independent test). Overall, these results support the view that corruption does not merely reflect transfers from citizens to bureaucrats but distorts allocation of resources.

Banerjee et al. ‘Police Performance and Public Perception in Rajasthan, India’. 2006-2008

Governance (related to most MDGs)

Rajasthan Police initiated an intervention with J-PAL researchers which aimed to enhance police performance, improve public opinion, and gather objective information about crime rates and performance

Evidence from public opinion surveys revealed implementing police training and increasing the duration of job postings can significantly improve the public’s opinion of the police. Increasing the number of trained officers from 0% to 100% raises the probability that crime victims were satisfied with police investigation by 31 percentage points

Duflo et al. ‘Efficiency and Rent Seeking in Local Government’. 2000-2003

MDG2 (Primary education)MDG3 (Gender equality)MDG7 (Water and sanitation)(indirect)

Survey of all almost all village councils in Birbhum, totaling 161 councils and resource map

The provision of irrigation and education services was lower in lower-caste areas relative to higher-caste ones, drinking water and informal education

23

centers were provided equally among groups, and lower-caste communities saw a larger allocation of sanitation services. Village heads from the Scheduled Castes built on average 10% more infrastructure projects in lower-caste areas relative to the amount built in lower-caste areas of communities which did not have a reserved village head

Banerjee et al. ‘Providing Health Insurance through Microfinance Networks in Rural Karnataka, India’. 2007

MDG1 (Nutrition)MDG4 (Infant mortality)MDG5 (Maternal mortality)MDG6 (HIV/AIDS, malaria, and other diseases)MDG7 (Water and sanitation)

SKS Microfinance's pilot health insurance program

The data suggest that SKS households, despite being members of a microcredit organization, face considerable financial risk from health shocks. Less than 1% of households have accident or health insurance, but they face frequent and serious health shocks.

Nepal Thornton & Oster. ‘Menstruation and Education in Nepal’. 2006-2008

MDG3 (Gender quality)MDG7 (Water and sanitation)

Effect of provision of sanitary products on school attendance, attainment, self-esteem and health among adolescent girls

Girls not allocated menstrual cups in the initial randomization were 2.6 percentage points less likely to be in school on days they were menstruating. This falls well below the 10-20% estimates made by policy makers. Similarly, the allocation of a

24

menstrual cup has no effect on test scores, and this is not due to low adoption of the cup: 60% of treatment girls report using the cup six months into the study.

South east AsiaIndonesia Banerjee et al.

‘Effectively Targeting Anti-Poverty Programs in Indonesia’. 2008-2009

Most MDGs (indirect) Proxy means testing (PMT) (approach that focuses on the household level, identifies potential beneficiaries by a single or set of indicators that are highly correlated with (low) expenditures) and community-based methods of targeting

The PMT method outperformed both the community and hybrid treatment in terms of the overall mis-target rates

Cole et al. ‘Financial Literacy, Access to Finance and the Effect of Being Banked in Indonesia’. 2007

Most MDGs (indirect) The Access to Finance survey was followed by a financial literacy experiment, which was only undertaken in Java, Indonesia, designed to directly test the role and relative importance of financial literacy and prices in determining demand for banking services

Results show a strong relationship between financial literacy and banking behaviour. Specifically, an increase in the incentive from US$3 to $14 increases the share of households that open a formal savings account from 3.5% to 12.7%, an almost three-fold increase.

Olken, B. ‘Combating Corruption in Community Development in Indonesia’. 2003-2004

Most MDGs (indirect) A randomized evaluation in 608 Indonesian villages in East Java and Central Java: auditing of funds for public projects

The evidence suggests that increasing the probability of external audits substantially reduced missing funds in the project; The invitations increased the number of people

25

participating in the accountability meetings by about 40%

Olken, B. ‘Direct Democracy and Local Public Goods in Indonesia’. 2005-2006

MDG3 (Gender equality) Randomized evaluation in 48 villages, all of which were preparing to apply for infrastructure projects (assessing participation and governance)

Direct elections on women’s projects, on the other hand, did result in projects that were located in poorer areas of the villages. Villagers in treatment locations were 22 percentage points more likely to correctly identify the type and location of the general village project, and 29 percentage points more likely to know these things about the woman’s project.

Philippines Ashraf, N. ‘Spousal Control and Intra-Household Decision Making in the Philippines’. 2005-2006

Most MDGs (indirect) A field study in which couples attempted to make decisions with varying degrees of privacy and ability to bargain

Evidence shows that the setting of intra-household decision-making has significant impact on final outcomes; These results suggest that spouses with different information about household income may create incentives for hiding funds. In the private setting, 51% of men preferred to directly deposit money into their own account rather than putting it into their spouse’s account (23%) or committing it to consumption (25%). However, when information about

26

the outcomes was made public, 60% of men chose to commit their money to consumption rather than putting the money into their own account (21%) or into their spouse’s account (19%).

Karlan et al. ‘Deposit Collectors in the Philippines’. 2001-2004

Most MDGs (indirect) Deposit-collecting program offered by the Green Bank of Caraga

The gender difference suggests that intra-household decision making factors play a strong role in the take-up of deposit-collection services; The deposit-collection service resulted in a substantial increase in savings for those offered the service. he deposit-collection service increased savings by about 25% after 10 months. The average person made 3.85 deposits over the 10 month period, and the average deposit amounted to 497 pesos

Karlan et al. ‘Measuring the Impact of Microcredit and Interest Rate Sensitivity in the Philippines’. 2007-2009

Most MDGs (indirect) Loans to marginally creditworthy applicants (those who may have been refused credit beforehand)

Additionally, borrowing households substituted away from labor for their businesses and into education for their children, choosing to invest in their family’s human capital, rather than in capital specific to their businesses

Karlan et al. MDG4 (Infant mortality) Committed Action to Results suggest that

27

‘CARES Commitment Savings for Smoking Cessation in the Philippines’. 2006-2007

MDG5 (Maternal mortality)MDG6 (HIV/AIDS, malaria, and other diseases)

Reduce and End Smoking (“CARES”) is a voluntary commitment savings program.

those who signed a CARES commitment increase their probability of test passage and a lasting quit spell by several fold. These results were far above the reductions in smoking associated with the cue card treatment. ndividuals who were offered a CARES contract were 3 to 6 percentage points more likely to pass a urine test for nicotine after six months than those in the comparison group, a substantial effect considering that only 8 to 12 percent of comparison individuals passed the test.

Karlan et al. ‘Commitment Savings Products in the Philippines’. 2000-2003

MDG 3 (Gender equality)Most MDGs (indirect)

Commitment savings product called a SEED (Save, Earn, Enjoy Deposits) account. The SEED account provides individuals with a commitment to restrict access to their savings, thus potentially helping with either self-control or family-control issues

The SEED product leads to more decision making power for women in the household, and likewise an increase in purchases of female-oriented durable goods. Findings suggest that the 81% increase in savings after one year did not crowd out savings held outside of the participating bank, but that the longer-term impact over two and a half years was only a

28

33% increase, which is no longer statistically significant.

Latin AmericaColombia Kremer et al.

‘Vouchers for Private Schooling in Colombia’. 1998-2004


Programa de Ampliación de Cobertura de la Educación Secundaria (PACES), one of the largest voucher programs to date (vouchers to enable children to enroll in school)

Preliminary results showed no significant differences in school enrollment between lottery winners and losers three years after application. But lottery winners were 15 percentage points more likely to have attended private school and to attend university; While test scores suggest vouchers had more positive effects than simply reducing grade repetition, no strong conclusions were able to be drawn about the long-term effects of private schooling on test scores. Lottery winners were more likely to take the university entrance exam, a good predictor of high school graduation, since 90% of all high school graduates take the exam. Within the sample of voucher applicants, about 28 percent of students took the ICFES exam, and vouchers raised the exam registration rates about 7 percentage points.

29

Peru Karlan et al. ‘Valuing Trust in Shantytowns in Peru’. 2005

Most MDGs (indirect) In collaboration with PRISMA, a Peruvian NGO offering credit through village banks, researchers designed and implemented a new loan product and administered surveys to 9,000 shantytown households

These findings underscore the prediction on which the program was founded, namely that borrowers with close social relationships to their sponsors allow the bank to be more certain that this new client will repay their loan

Karlan & Valdivia. ‘Business Education for Microcredit Clients in Peru’. 2002-2005

Most MDGs (indirect) Impact of adding business training to a group lending program

The impact on business outcomes was significantly positive though the impact on institutional outcomes was less impressive. For clients in the treatment group, sales in the month prior to the follow up surveys were 15% higher than in the comparison group, and returns were an average 26% higher in "bad months" when they would have expected downward fluctuations in their sales

Source: J-PAL Summaries at: http://www.povertyactionlab.org/search/apachesolr_search?view=grid&filters=type:evaluation

30

Table 3. J-PAL REs yet to report with relevance to MDGs listed on J-Pal website (May 2010).

Africa Asia Latin AmericaBeninKarlan, D. ‘Microfinance and Health Protection Initiative, Including Malaria Education, in Benin’EthiopiaKarlan & Goldberg. ‘Targeting the Ultra-Poor in Ethiopia’GhanaDuflo et al. ‘Return to Secondary Schooling in Ghana’Fischer et al. ‘Willingness and Ability to Pay for the Kosim Clean Water Filter in Ghana’Karlan, D. ‘Returns to Medium and Small Enterprise Management Consulting in Ghana’KenyaKremer et al. ‘The Impact of Distributing School Uniforms on Children's Education in Kenya’. 2001-2004Kremer et al. ‘Decentralization: A Cautionary Tale - Public Finance in Kenya’. 1995-2000Kremer & Gugerty. ‘The Rockefeller Effect: Looking at Organizations of the Disadvantaged in Kenya’. 2002-2003Kremer & Vermeersch. ‘School Meals, Educational Achievement and School Finance in Kenya’Robinson & Dupas. ‘Savings Accounts for Village Micro-Entrepreneurs in Kenya’. 2006-2007MoroccoCrépon et al. ‘Impact of Rural Microcredit in Morocco’. 2006-2009Duflo et al. ‘Conditional Cash Transfers for Education in Morocco’

AfghanistanLinden & Burde. ‘Village Based Schools in Afghanistan’. 2007-2008BangladeshField & Glennerster. ‘Empowering Girls in Rural Bangladesh’. 2007Rasul et al. ‘Targeting the Ultra-Poor in Bangladesh’IndiaBanerjee et al. ‘Helping the Ultra-Poor Use Microcredit in Murshidabad, India’. 2006Banerjee et al. ‘Read India: Helping Primary School Students in India Acquire Basic Reading and Math Skills’. 2008-2010Bertrand & Mullainathan. ‘Misclassification in the Targeted Public Distribution System in India’. 2007Chattopadhyay et al. ‘Rainfall Insurance in Gujarat, India’Chattopadhyay et al. ‘Futures Prices and Risk Hedging in Gujarat, India’Duflo et al. ‘Cooking Stoves, Indoor Air Pollution, and Respiratory Health in India’. 2006Field et al. ‘Business Training for Women in Ahmedabad, India’Field & Pande. ‘Microfinance Repayment Schedules in West Bengal, India’. 2006Hanna et al. ‘Teacher and Student Motivation, Family Participation, and Student Achievement in Rural Udaipur, India’. 2003Karlan et al. ‘Debt Traps for Micro-Entrepreneurs in Chennai, India’. 2007-2008Linden, L. ‘Computer-Assisted Learning

BoliviaKarlan et al. ‘Text Message Reminders for Savings and Loan Repayment, in Bolivia and Philippines’Karlan, D. ‘Hope versus Fear in Health Credit Marketing in Bolivia’Karlan, D. ‘Lock Boxes and a Bridge to Formal Banking in Bolivia’Karlan & Gine. ‘Peeling Back the Layers of Group Liability in Bolivia’ColombiaLinden et al. ‘Designing Conditional Cash Transfers in Education: Colombia's Subsidios Program’Linden & Barrera-Osorio. ‘Spanish Computer-Assisted Learning Program in Colombia’. 2006-2008Dominican RepublicSchoar & Fischer. ‘Small and Medium Enterprise Financing and Mentoring Services in Emerging Markets in the Dominican Republic’El SalvadorMartínez et al. ‘Responses to Degree of Control over Remittances in El Salvador’. 2007-2008MexicoKarlan et al. ‘Interest Rate Sensitivity Among Village Banking Clients in Mexico’. 2006-2008Karlan et al. ‘Psychological Responses to Microfinance Loan Recovery in Peru’. 2006-2008No author. ‘Inviting Husbands to Female Solidarity Groups in Mexico’. No author. ‘Emergency Savings Accounts for Remittance Receivers in Mexico’

32

NigerHuillery et al. Decentralizing Education Expenditures: Primary School Community Grants in Niger’. 2008-2009Sierra LeoneMiguel & Glennerster. ‘The Role of Information and Radios on Political Knowledge and Participation in Sierra Leone’Miguel et al. ‘Community Driven Development in Sierra Leone’South AfricaKarlan et al. ‘Marketing Effects in a Consumer Credit Market in South Africa (Pricing the Effect of Psychological "Features")’. 2003-2006Karlan & Zinman. ‘Interest Rates, Loan Maturity and Demand for Microfinance Loans in South Africa’. 2003Karlan & Zinman. ‘Estimating the Impact on the Lender's Bottom Line and Borrowers' Household Welfare of Expanding the Supply of Consumer Credit to the Working Poor in South Africa’UgandaKarlan & Linden. ‘Micro-Savings in Primary Schools as a Means to Decrease Student Drop-Out Rates in Uganda’Svensson & Bjorkman. ‘Promoting Community Health in Uganda’Uganda & TanzaniaRasul et al. ‘Human Capital, Financial Capital, and the Economic Empowerment of Female Adolescents in Uganda and Tanzania’ZambiaAshraf et al. ‘The Role of Incentives in the Distribution of Public Goods in Zambia’

Project with Gyan Shala in India’. 2004-2006Linden et al. ‘How to Teach English in India: Testing the Relative Productivity of Pratham Instruction Methods’. 2005-2007Linden et al. ‘Teaching Pre-Schoolers to Read: A Randomized Evaluation of the Pratham Shishuvachan Program in India’. 2005-2007NepalLinden & Edmonds. ‘Interventions Aimed at Child Laborers in the Nepalese Carpet Industry’PakistanKarlan & Pariente. ‘Graduating the Poorest Into Microfinance: Linking with Financial Services on the Model of BRAC’s TUP in Pakistan’. 2008Linden et al. ‘Private School Incentive Program in Pakistan’IndonesiaOlken et al. ‘Project Generasi: Conditional Community Block Grants in Indonesia’. 2007PhilippinesKarlan et al. ‘Health Insurance and Asymmetric Information in the Philippines’Karlan et al. ‘Impact of Health Insurance in the Philippines’. 2007Karlan et al. ‘Psychology of Savings: Evaluation of Commitment Savings Program - FICO Bank, Philippines’Karlan & Gine. ‘Group versus Individual Liability in the Philippines’. 2004-2009Karlan & Mullainathan. ‘Debt Trap, Financial Literacy, and Savings in Philippines’No author. ‘Determinants of Delinquency in the Philippines’

NicaraguaField & Thornton. ‘Extending Health Insurance to the Informal Sector Through Microfinance in Nicaragua’. 2007-2009PeruKarlan, D. ‘Barriers to the Demand for Condoms in Peru’Karlan, D. ‘Interest Rates and Client Incentives for Repayment in Puno and San Martin, Peru’Karlan & Thuysbaert. ‘Targeting the Ultra-Poor in Peru’. 2009Karlan & Zinman. ‘Using Encouragement to Overcome Psychological Barriers to Saving in Peru’. 2005No author. ‘Impact of Rural Credit in Peru

33

mdg-relevant j-pal rcts review

Documents