using non-experimental methods to evaluate a conditional cash transfer program: the case of bolsa...
Post on 02-Jan-2016
213 Views
Preview:
TRANSCRIPT
Using non-experimental methods to evaluate a
Conditional Cash Transfer program:
The case of Bolsa Alimentação
• International Food Policy Research Institute– Pedro Olinto
• Bolsa Alimentação (Ministry of Health, Brazil)– Eduardo Nilson
• London School of Hygiene & Tropical Medicine– Saul Morris
• Emory University– Rafael Flores
• Instituto Materno-Infantil de Pernambuco– Ana Claudia Figueró
• Instituto de Estudos do Trabalho e Sociedade– Alinne Veiga
The program• Bolsa Alimentação is a Brazilian federal
government program designed to reduce nutritional deficiencies and infant mortality
• Beneficiaries– (i) pregnant women,– (ii) mothers breastfeeding a child up to 6 mo – (iii) children of age six months up to seven years
• Beneficiary selection:– Municipal quota based on projected number
malnourished children– Municipal authorities select beneficiary households
• R$15 (US$6.25) /beneficiary /month (up to R$45)
• Mother signs “Charter of responsibilities”
Evaluation design: main features
• Non-experimental• Ex-post only comparison of beneficiary
and non-beneficiary households• Non-beneficiary households were
registered by local authorities as beneficiaries, but excluded due to:– Data transmission error– “Special characters” in the name– Discrepancy family records Bolsa Escola
Evaluation design: main features
• Evaluation limited to municipalities– In the Northeast region of the country– With at least 40 excluded households– Where transfers had been made for 6
months
• Fieldwork undertaken exactly 6 mo after first transfer in each of the 4 survey municipalities
Pairing of beneficiaries and excluded
• Pairing separate for each class of beneficiary– Pregnant women– “Lactating mothers” (up to 6 mo post-partum)– Children
• For each “would-be beneficiary”, the most similar similar actual beneficiary was identified, based on– Municipality of residence*– Sex*– Socio-economic status (based on reported income;
rent; water, electricity & gas bills, family size)– Age
Pairing (2)
• Pairing algorithm: “nearest neighbor” with calipers– Pairwise distance calculated based on Euclidean
distances– Relatively importance of age and SES chosen
subjectively– No replacement of beneficiaries– “Excludeds” matched in a random order– Two beneficiaries per “excluded”, to maximize
powerExcluded Benef.1 Benef.2 Total
Pregnant 25 25 25 75Lactating 18 18 18 54Child 463 463 457 1383
Total 506 506 500 1512
Problems with the pairing
• Turned out that program registers had not recorded all the young children in the hhs– Expected no. under-7s: 1.3 per hh– Revealed no. under-7s: 1.9 per hh
• Individual-level pairing not so good for household-level analyses, since same beneficiary hh paired to more than one excluded hh
Bolsa Alimentação beneficiary households
Excluded
households
P-value
Educational level of women aged 15-49:
Incomplete primary
79.0%
(646/818)
79.3%
(252/318)
0.92
Flooring material:EarthFloor tilesOther (mostly cement)
8.0% (57/716)10.8% (77/716)
81.3% (582/716)
9.6% (27/282)
11.0% (31/282)79.4%
(224/282)
0.70Water source: public network 50.5%
(360/713)50.4%
(142/282)0.97
Telephone 9.2% (66/717) 11.0% (31/282) 0.39
Family size, mean (s.d.):Total
0.0 – 6.9 y7.0 – 13.9 y
5.4 (2.1)1.9 (0.9)0.8 (1.0)
5.8 (2.3)1.8 (0.9)1.1 (1.1)
0.0030.02
<0.001Bolsa Escola beneficiary 26.8%
(192/717)55.7%
(157/282)<0.001
Duration of exposure to Bolsa Alimentação at the time of the survey, months, mean (s.d.):
5.9 (0.3)
N/A
N/A
Socio-economic and demographic characteristics of Bolsa Alimentação beneficiary and excluded
households in the evaluation sample
Marginal propensity to consume food
out of Bolsa Alimentação transfers
• OLS model:– R$0.70 for every R$1.00 transferred
• IV model (controlling for possible endogeneity of number of beneficiaries):– R$0.57 for every R$1.00 transferred
Impact on dietary diversity
Dependent variables:
-1 -2 -3 -4
Dummy = 1 if beneficiary of BA 1.901 1.879 0.089 0.088[0.585]**[0.587]**[0.024]**[0.024]**
Dummy = 1 if beneficiary of BE -0.4 -0.9 0.001 -0.03[0.692][0.549] [0.029] [0.022]
Log of household population -0.25 -0.013[0.893] [0.037]
Share of members aging 0 to less than 7 -4.21 -0.198[2.466] [0.099]*
Share of members aging 7 to less than 15 -3.68 -0.217[2.733] [0.112]
Share of members aging 15 to less than 19 -6.19 -0.295[3.206] [0.138]*
Share of members aging 61 and up 3.518 0.152[3.730] [0.137]
Constant 29.5 27 3.357 3.232[1.678]**[0.556]**[0.068]**[0.024]**
Observations 1005 1005 1005 1005Robust standard errors in brackets* significant at 5%; ** significant at 1%
# of food items ln(# food items)
Weight-for-age Z-score
Age group: Bolsa Alimentacao Difference
Benef’y Excl’ded Unadjusted Adjusted
0.0 - 23.9 momean -0.68 -0.40 mean -0.28 -0.25
s.d. 1.11 1.15 s.e. 0.13 0.13
n 315 103 signif P=0.030 P=0.058
24.0 - 47.9 momean -0.75 -0.65 mean -0.10 -0.11
s.d. 1.06 1.02 s.e. 0.10 0.10
n 430 141 signif P=0.33 P=0.29
48.0 - 83.9 momean -0.77 -0.74 mean -0.03 -0.08
s.d. 1.04 0.90 s.e. 0.08 0.08
n 596 238 signif P=0.68 P=0.31
Totalmean -0.74 -0.64 mean -0.10 -0.13
s.d. 1.06 1.00 s.e. 0.06 0.06
n 1,341 482 signif P=0.068 P=0.024
Longitudinal analysisFractional Polynomial (0 1)
Co
mp
on
en
t+re
sid
ua
l fo
r w
eig
ht
age0 12 24 36 48 60
2
20.6
Associations between length of exposure to Bolsa Alimentação and weight gain in children aged 0–36 months. Panel analysis based on routinely recorded weight data:
random effects model.------------------------------------------------------------------------------ WEIGHT | Coef. Std. Err. z P>|z| [95% Conf. Interval]-------------+---------------------------------------------------------------- AGE | .0835284 .0050511 16.54 0.000 .0736286 .0934283 ln(AGE) | 1.941647 .0445841 43.55 0.000 1.854264 2.029031 MALE | .5057401 .0886546 5.70 0.000 .3319804 .6794998 BA_BASELINE | -.1923509 .1034153 -1.86 0.063 -.3950412 .0103395 BA_DURATION | -.0305449 .0074744 -4.09 0.000 -.0451945 -.0158953 BE | -.0201923 .0475438 -0.42 0.671 -.1133765 .0729919 _cons | 3.187013 .1141902 27.91 0.000 2.963205 3.410822-------------+---------------------------------------------------------------- sigma_u | 1.0507184 sigma_e | .61632495 rho | .74400885 (fraction of variance due to u_i)------------------------------------------------------------------------------
Differential weight gain associated with each month of exposure to Bolsa
Alimentação, estimated for children of ages 6, 12, 18, 24, 30, and 36 mo, in a
random effects model that allows program effect to interact both with age
and the natural log of age------------------------------------------------------------------------------ weight | Coef. Std. Err. z P>|z| [95% Conf. Interval]-------------+---------------------------------------------------------------- 6 months | -.0324806 .0137262 -2.37 0.018 -.0593834 -.0055778------------------------------------------------------------------------------ 12 months | -.0457275 .0095092 -4.81 0.000 -.0643653 -.0270897------------------------------------------------------------------------------ 18 months | -.0402928 .0084215 -4.78 0.000 -.0567987 -.0237869------------------------------------------------------------------------------ 24 months | -.0272094 .0088983 -3.06 0.002 -.0446498 -.009769------------------------------------------------------------------------------ 30 months | -.009935 .0122145 -0.81 0.416 -.0338749 .0140049------------------------------------------------------------------------------ 36 months | .0099903 .0176865 0.56 0.572 -.0246747 .0446553------------------------------------------------------------------------------
Lessons learnt: methodology
• Very detailed understanding of beneficiary selection process resulted in robust non-experimental identification
• Success critically dependent on getting into the field at the right time
• Individual matching ensured that beneficiary and non-beneficiary groups were very similar
• Routinely collected weight data permitted longitudinal analysis, quality much better than expected
• However, lack of conventional baseline allows controversy over results to rage indefinitely!
Lessons learnt: results
• Strikingly high proportion of transfer spent on food, suggesting good targeting
• Diversity of family diet improved markedly, as in similar programs
• Vaccination coverage not improved because already excellent
• Weight gain effect probably illustrates dangers of linking targeting and/or program graduation to nutritional status outcomes
• For nutrition, too many beneficiaries over 2 yo, damage likely to be irreversible
top related