Beyond Cash: assessingexternality and behavior

effects on non-experimentalcash transfers

Fabio Veras, Clarissa Teixeira, Elydia Silva, GuilhermeHirata, Joana Costa and Tatiana Brito

Introduction• Conditional Cash Transfer Programmes (CCTs)

– In general, its objectives are defined as:• to alleviate poverty in the short-run (transfers);• to break the intergeneration transmission of

poverty (conditionalities and social workers);

– Transfers depend on actions required from thefamilies:

• school attendance• health check-ups (and updating immunization

cards).– social worker visits/complementary programmes

Tekoporã: a pilot CCT inParaguay• Targeting:

– Geographical: poorest districts according to the IPG;– Categorical: families with children or pregnant woman;– Means-testing: poorest families according to the Index

of living conditions (ICV) – non-monetary <=40 points• Household information collected through a census

(Ficha Hogar).– Transfer: Basic benefit 60,000 Guaranies (US$12).

Extra 30,000 Guaranies (US$ 6) per child up to a limit of4. (Min: 90,000 (US$18) and Max: 180,000 Guaranies(US$36)).

• Pilot started in 2005 in 5 out of 7 pre-selected districts.• Program has been scaled up. In 2009 15/66 selected

districts

Literature ReviewExternality:• Bobonis&Finan(2009), Lavile&Cataneo (2009)– peer effect• Angelucci&Di Giorgi (2006) – inter-household transfers• Bobba (2008) – externality and coverageIncome and substitution effect:• Hoddinott&Skoufias (2004), Maluccio&Flores (2005) – motivation• Rubacalva et al (2004), Handa et al (2009) – disentangling income

and substitution effectConditionality effect:• Schady&Araujo (2008), Hodinott&Brauw (2007) – conditionality

knowledge/enforcement

Hypotheses• Both beneficiary and non-beneficiary households can be

affected by existence of a social programme and thepresence of other beneficiaries in their community

• Externalities can heighten (lessen) the programme’spotential impact when it is on the same (opposite)direction as the intended effect

• There is a substitution effect that changes the wayhouseholds make their decisions, beyond the expectedchanges due to the increased income.

Objectives• Evaluate Tekopora’s impact on education and

health care outcomes taking into account thepossible existence of indirect effect due toexternality and assess the non-cash componentsrole.

– In order to do so we propose to disentangle the effecton the treated (ATT) into two components:

a) participation effect (APT) – programme’s direct effect;b) externality effect (AET) – programme’s indirect effect

due to general equilibrium and social interaction.

Illustrating Externality

Objectives

– and further decompose these components into:• Income effect - allows families to consume more

goods and services, including health care andschooling;

• Substitution effect - changes the way householdsspend their income, beyond the expected changesdue to the increased income due to non-cashcomponents;

• and unobservable effects.

Theoretical illustration

Sample design• Baseline sample drawn from program record (Ficha

Hogar and payroll);• Follow up info field between January and April 2007

= Ficha hogar + consumption + school attendance +visits to health centre;

• Sample: 1,093 households (6,404)– 316 (29%) treated,– 430 (39%) within district control - overlooked– 347 (32%) between district control

Incorporating externality:multiple treatment effect

approach

• Average Participation Effect on the Treated (APT) isdefined as:

• Average Externality Effect on the Treated (AET) isdefined as:

• Identification assumption: additivity

• Then, we may write the ATT effect as the sum of botheffects:

( ) ( )[ ]1|0,01,0 ===!=== iiiiiiip TTDYTDYE"

( ) ( )[ ]1|0,00,1 ===!===iiiiiiieTTDYTDYE"

( ) ( )[ ]1|0,01,1 ===!===+= iiiiiiipe TTDYTDYE"""

Di = household is in the programme districtTi = treated household

( ) ( ) ( ) ( )0,00,11,01,1 ==!==+=====iiiiiiiiiiiiTDYTDYTDYTDY

Quasi-Experimental Evaluation• Conditional Independence Assumption:• Combination of two methods to estimate ATT (Hirano e

Imbens, 2002):– Regression (single difference or difference in

differences):

– Adjusting (weighting) on observable variables

( )[ ] ( )[ ] ,1|1| 32

10

iiiiiiiii

iieipi

DTXEXTTXEX

XDTY

!""

"##"

+$=%&+$=%&+

&+$+$+=

( )( ) ( )

( )( ) ( )( ) ( )

ii

ii

i

iiiiiii

XeXp

DXp

Xe

DTXpTXDT

ˆˆ1

1ˆ

ˆ

1ˆ,,ˆ

!!

!"+

"!"+=#

( ) [ ]iiii XTDPXp |1,1 === ( ) [ ]iiiiXTDPXe |0,1 ===

Quasi-Experimental EvaluationNeed to match observations;

– turn the comparison groups comparable to thetreated;

– Using a multinomial propensity score;

0

.02

.04

.06

.08

0 20 40 60 80

ICV

treatment

within-community control

between-community control

without PS weighting

0

.02

.04

.06

.08

0 20 40 60 80

ICV

with PS weighting

Measuring income andbehavioural change effects

• J-M-P decomposition (1993):– Suppose that– If and are counterfactual functions:

– For comparison purpose we also estimate the marginalincome effect:

( )( )TDiTDiTDTDiTDTDi WFWgY ,,,,

1

,,,,,, |, !"=

( )..,g ().Fu

TDi

g

TDi

W

TDiTDi YYYY,,,,,,,,

++=

( ) ( )iTDi

W

TDi FWgY !1,,,,

"+=

( ) ( )[ ] ( ) ( )[ ]( ) ( ) and , ,,,,,

1

,,

1

,,,,,

TDiTDiTD

iTDiiTDiTD

g

TDi

WgWg

FWgFWgY

!=

+!+= !! ""

( ) ( )[ ] ( ) ( )[ ]( ) ( ). 11

,

1

,,,

1

,,,,,,

iiTD

iTDiTDiTDTDiTD

u

TDi

FF

FWgFWgY

!!

!!""

""

"=

+"+=

( ) ( )1

, , , 0, 0 , 0, 0

MW

i D T i D T i D TY g W F !"

= = = == +

Theoretical illustration

( )[ ]( ) ( )[ ] ( ) ( )[ ]( ) ( )

0,0,1,1,

1

0,0,

1

1,1,

0,0,1,1,1

ii

iiii

i

W

i

W

i

W

ATT

WgWg

FWgFWg

TYYE

!=

+!+=

=!=!! ""

#

( )[ ]( ) ( )[ ] ( ) ( )[ ]( ) ( )[ ] W

ATTii

iiii

i

g

i

g

i

g

ATT

WgWg

WgWgWgWg

TYYE

!

!

""=

"""=

="=

0,0,0,01,1,1,1

0,0,0,0,0,01,1,1,1,1,1

0,0,1,1,1

Results - Consumption

Estimated APT, AET and ATT on Household ConsumptionAPT AET ATT

log per capita consumption 0.211 -0.075 *** -0.298 -0.081 *** -0.087 -0.055

log pc food consumption 0.148 -0.075 ** -0.270 -0.079 *** -0.122 -0.061 **

saving rate 0.158 -0.074 ** 0.150 -0.088 * 0.307 -0.073 ***

share of food expenditures -0.039 -0.015 ** 0.022 -0.016 -0.017 -0.014

share of child clothing exp. 0.015 -0.004 *** 0.003 -0.003 0.018 -0.003 ***

share of adult clothing exp. 0.007 -0.002 *** -0.010 -0.003 *** -0.003 -0.003

Note: *** significant at 1%, ** significant at 5%, * significant at 10%.

Standard errors, between parentheses, calculated using 1,000 bootstrap resampling.

Results - ConsumptionEstimated Decomposition of the ATT, APT and AET on Household Consumption

APT MIE IE SE UElog per capita consumption 0.131 -0.036 *** 0.124 -0.039 *** 0.070 -0.069 0.018 -0.029log pc food consumption 0.136 -0.037 *** 0.132 -0.038 *** 0.004 -0.070 0.013 -0.030saving rate 0.179 -0.045 *** 0.154 -0.038 *** 0.030 -0.070 -0.026 -0.031share of food expenditures 0.002 -0.004 0.001 -0.003 -0.033 -0.016 ** -0.007 -0.008share of child clothing exp. 0.000 -0.001 0.000 -0.001 0.013 -0.004 *** 0.002 -0.002share of adult clothing exp. 0.001 -0.001 0.000 -0.001 0.006 -0.002 *** 0.001 -0.001

AET MIE IE SE UElog per capita consumption -0.049 -0.035 -0.049 -0.039 -0.226 -0.077 *** -0.023 -0.033log pc food consumption -0.053 -0.037 -0.053 -0.039 -0.197 -0.076 *** -0.020 -0.033saving rate -0.074 -0.056 -0.057 -0.049 0.185 -0.073 ** 0.022 -0.032share of food expenditures 0.000 -0.003 -0.001 -0.002 0.021 -0.017 0.002 -0.010share of child clothing exp. 0.000 -0.001 0.000 0.000 0.003 -0.004 0.000 -0.002share of adult clothing exp. 0.000 -0.001 0.000 -0.001 -0.010 -0.003 *** 0.000 -0.001

ATT MIE IE SE UElog per capita consumption 0.082 -0.028 *** 0.074 -0.029 *** -0.156 -0.050 *** -0.005 -0.022log pc food consumption 0.084 -0.029 *** 0.079 -0.029 *** -0.193 -0.055 *** -0.007 -0.021saving rate 0.105 -0.045 ** 0.096 -0.041 ** 0.215 -0.054 *** -0.004 -0.027share of food expenditures 0.002 -0.003 0.000 -0.002 -0.012 -0.015 -0.006 -0.007share of child clothing exp. 0.000 0.000 0.000 -0.001 0.016 -0.004 *** 0.002 -0.001share of adult clothing exp. 0.001 -0.001 0.000 -0.001 -0.004 -0.003 0.001 -0.001Note: *** significant at 1%, ** significant at 5%, * significant at 10%.Standard errors, between parentheses calculated using 1,000 bootstrap resampling.MIE = Marginal Income Effect, IE = Income Effect, SE = Substitution Effect, UE = Unexplained Effect.

Results – EducationImpact decomposition over school attendanceDifference in

DifferencesSingle

DifferenceCoef. Std. Coef. Std.

ATT 0.033 0.030 0.069 0.021***MIE -0.005 0.003 0.001 0.008IE -0.004 0.004 -0.003 0.005SE 0.057 0.014*** 0.054 0.021**UE -0.020 0.030 0.018 0.010*APT 0.019 0.074 0.050 0.039MIE -0.002 0.005 0.001 0.011IE -0.002 0.005 -0.001 0.004SE 0.065 0.056 0.072 0.060UE -0.045 0.055 -0.021 0.029AET 0.014 0.079 0.019 0.041MIE -0.003 0.003 0.000 0.007IE -0.003 0.004 -0.002 0.005SE -0.008 0.057 -0.018 0.061UE 0.025 0.058 0.039 0.034Source: Own calculation based on the Evaluation Survey.Note: Significant different from treated group at *10%, **5%and ***1%.

Impact decomposition over school progressionDifference inDifferences Single Difference

Coef. Std. Coef. Std.ATT 0.050 0.036 0.076 0.026***MIE -0.002 0.004 0.022 0.016IE -0.006 0.005 -0.001 0.009SE 0.043 0.017** 0.070 0.027***UE 0.013 0.034 0.006 0.013APT 0.055 0.078 0.057 0.057MIE 0.002 0.007 0.019 0.016IE -0.002 0.007 0.004 0.007SE 0.097 0.070 0.094 0.074UE -0.040 0.054 -0.040 0.027AET -0.005 0.082 0.019 0.058MIE -0.005 0.005 0.003 0.016IE -0.004 0.005 -0.005 0.009SE -0.054 0.069 -0.023 0.075UE 0.053 0.060 0.046 0.031Source: Own calculation based on the Evaluation Survey.Note: Significant different from treated group at *10%, **5% and***1%.

Results –HealthImpact decomposition over number of visits to child height and

weight control (at least 3 in the last 12 months)Coef. Std.

ATT 0.163 0.053***MIE 0.011 0.015IE 0.015 0.011SE 0.148 0.055***UE 0.000 0.021APT 0.176 0.072**MIE 0.017 0.014IE 0.014 0.017SE 0.195 0.074***UE -0.034 0.036AET -0.014 0.079MIE -0.005 0.017IE 0.000 0.017SE -0.047 0.079UE 0.033 0.034

Source: Own calculation based on the Evaluation Survey.Note: Significant different from treated group at *10%, **5% and ***1%.

Conditionality knowledge

AverageHas received at least 1 visit per month from social workers 86%Is aware of programme conditionalities 92%Is aware of school attendance conditionality 83%Is aware of visits to child height and weight controlconditionality 67%Is aware of vaccination conditionality 58%Source: Own calculation based on the Evaluation Survey.

Results - HeterogeneityImpact decomposition over schoolattendance

Singledifference Social Worker Conditionality

Coef.Std.Err. Coef. Std. Err. Coef. Std. Err.

ATT 0.069 0.021*** 0.025 0.027 0.040 0.047

MIE 0.001 0.008 -0.009 0.014 0.003 0.018

IE -0.003 0.005 -0.007 0.008 -0.007 0.012

SE 0.054 0.021** -0.007 0.008 -0.003 0.009

UE 0.018 0.010* 0.039 0.026 0.049 0.043Source: Own calculation based on the Evaluation Survey.Note: Significant different from treated group at *10%, **5% and ***1%.Impact decomposition over number of visits to child height and weight control (at least 3 in the last 12

months)

Single difference Social Worker Conditionality

Coef. Std. Err. Coef. Std. Err. Coef. Std. Err.

ATT 0.163 0.053*** -0.074 0.088 0.152 0.118

MIE 0.011 0.015 0.034 0.021* 0.059 0.039

IE 0.015 0.011 0.020 0.017 0.039 0.027

SE 0.148 0.055*** 0.010 0.026 0.052 0.054

UE 0.000 0.021 -0.104 0.088 0.062 0.117

Source: Own calculation based on the Evaluation Survey.

Note: Significant different from treated group at *10%, **5% and ***1%.

Conclusion• There is no externality on education or health.• ATT effect on education and health it is due to

substitution effect.• Heterogeneity show no significant change for those

unaware of conditionality or those who have not receivedsocial worker visits. The substitution effect must comethrough other factors related to the Tekoporã program.