Estimating the impact of a policy reform on welfare participation:

The 2001 extension to the Minimum Income Guarantee for UK pensioners

Steve PudneyFrancesca Zantomio

Institute for Social & Economic Research, University of Essex

Ruth HancockDepartment of Health & Human Sciences, University of

Essex

13 May 2006(2)

Outline • Introduction

• The MIG system and the 2001 reform

• Data and entitlements simulation

• Statistical Analysis

• Results

• Conclusions

13 May 2006(3)

“Non take-up”

• People are observed NOT to claim means-tested welfare benefits to which they are entitled.

• Take-up rate measures:

by caseload: # claimants / # entitled benefit units

by proportion of claimed government expenditure

• The phenomenon seem to be particulary severe for pensioners in the UK

13 May 2006(4)

UK Pensioners Take-up

• 2 million pensioners living in low income households

• Government commitment to ensure a decent income in retirement

• Welfare benefits (incl. state pension) make up the entire income of 15 % of pensioners

• Still between a quarter and a third of entitled pensioners do not claim income support

• Policy relevance

13 May 2006(5)

“Non take-up” rationale

Qualitative research (Costigan et al.,1999; CAG, 2002) suggests that welfare participation involves claiming costs due to:

Information and transaction costs

• ignorance of the existence of the program• insufficient knowledge of the administrative procedures or

entitlement criteria• language difficulties• troubles in filling the forms and gathering required

information and documentation,• time, hassle…

Intangible social stigma costs- distatste for welfare income- self-humiliating feeling• fear of stigmatisation from others• fear of losing independence

13 May 2006(6)

“Take-up” literature

Social sciences literatures exploring various aspects of take-up behaviour (Kerr,1982; Hirsh and Rank 1999; Kayser and Frick,2001; Castranove et al., 2001)

Economic literature (Moffit,1983; Blundell et al., 1988; Duclos, 1995; Pudney et al., 2006; Hernandez et al.,2006)

• Claiming as utility maximizing choice: expected benefit vs claiming cost

• Entitlement amount acts as positive incentive to take-up

• Econometric approach:entitlemnts simulation and parametric modelling of take-up probability, estimation of stigma cost

13 May 2006(7)

Our approach

Use of a policy change to pensioners income support system to identify the impact of variation in entitlement on the take-up behaviour

Non parametric analysis (misspecification)

Comparison with traditional parametric approach

Family Resorces Survey data( samples of pensioners interviewed before and after the reform

came into force)

13 May 2006(8) The Minimum Income Guarantee (MIG) Income support for pensioners (60+), income and asset-

tested

– brings income up to a minimum level, B = G(X) – means– withdrawn at 100% as income rises

• MIG in 2000/1: – assets between £3,000 & £8,000 converted to notional income– no eligibility if assets > £8,000– system of age-related additions

• Reformed system in 2001/2:– real increase in guarantee level– abolition of age additions– asset limits raised to (£6,000, £12,000)

• Every eligible pensioner gained; some became newly-eligible

13 May 2006(9)

Allowances and Premiums

(£ per week)

single couple

Basic allowance 52.2 81.95

Pensioner Premium

26.25 40.00

Enhanced Pensioner Premium

28.65 43.4

Higher Pensioner Premium

33.85 49.10

Capital limit 3,000-8,000

PRE-REFORM

Allowances and Premiums

(£ per week)

single couple

Basic allowance 53.05(52.21)

83.25(81.93)

Pensioner Premium

39.10(38.48)

57.30(56.39)

Enhanced PensionerPremium.

39.10(38.48)

57.30(56.39)

Higher Pensioner Premuim

39.10(38.48)

57.30(56.39)

Capital limit 6,000-12,000

POST-REFORM

13 May 2006(10)

MIG recipients, entitled non recipients and caseload take-up ratesCouple Single

MaleSingle Female All

Number of Recipients (thousands)

1999/2000 240 240 900 1390

2000/1 260 250 920 1430

2001/2 280 270 960 1520

Range of Entitled non Recipients

1999/2000 90-170 60-170 220-460 390-770

2000/1 110-170 80-140 230-380 450-670

2001/2 170-260 90-160 310-480 600-870

Caseload Take-up Range

1999/2000 59-72 59-79 66-80 64-78

2000/1 60-69 65-76 70-80 68-76

2001/2 52-62 64-75 67-75 63-72

(Ranges are 95% confidence interval to reflect sampling errors); source: DWP (2004)

13 May 2006(11)

The evaluation problem • Rubin-type evaluation methods require an experimental

implementation and a comparison group outside the experiments

• Limitations:– assume a limited number of policy options

– a non-universalist approach to policy that allows small-scale experiments

• In the UK experiments are rare because these conditions aren’t fulfilled– lots of policy options are considered in advance of reform

– there is a centralised or universalist approach to policy

– policy is designed with a very short time horizon – by the time an experiment would be designed, implemented & evaluated, government has lost interest

• The dominant approach remains “behavioural” modelling, which allows generalisation from past experience to hypothetical new policy environments

• Raises issues of:– specification error

– other modelling problems, such as measurement error

13 May 2006(12)

Our evaluation problem• want to identify the average treatment effect of the reform

on the take-up of the treated

• there can be no “control” group since:– all before-reform eligible people were affected– non-eligible people have no take-up behaviour to study

• for any pensioner we observe the take-up behaviour only under one benefit regime (either before or after the reform)

“Selection on observables” approach:

Pre-reform interviewed pensionersare are matched in terms of pre-reform and post reform entitlements (and demographic

characteristics) to post-reform interviewed pensioners and used as conterfactual

13 May 2006(13)

Family Resources Survey Data

• Annual cross-section surveys for Department for Work & Pensions (DWP)

• 2 years’ data, for t = 0 (FY 2000/1) & 1 (FY 2001/2)• Pensioner units at least 5 years over state retirement

age, in single unit hholds, without mortgages/earned income

• Extensive data cleaning of benefits data to minimize the scope for measurement error

• Benefit entitlement is simulated under the pre-reform rules and post-reform rules

• Final sample of entitled pensioners contains n0 = 845 in 2000/1 & n1 = 756 in 2001/2

13 May 2006(14)

Notation• Entitlement: Bt

r = amount of simulated MIG entitlement in circumstances of year t = 0, 1 under benefit system r = 0, 1

• Take-up outcome: Ttr = binary indicator of take-up status

that would be observed in circumstances of year t = 0, 1 under benefit system r = 0, 1. Tt

t is only meaningful if true Bt

t > 0• Survey response: Rt = binary indicator of survey response

that would be encountered in year t if selected for interviewing

• We observe:{Xt , Tt

t, Bt0, Bt

1 | Rt = 1} for t = 0, 1• Three population groups:

– People ineligible before & after reform (Bt0 = 0 , Bt

1 = 0)– People ineligible before & eligible after (Bt

0 = 0 , Bt1 > 0)

– People eligible before & after (Bt0 > 0 , Bt

1 > 0)• Our concern is with response to change in generosity of the

benefit system, so we concentrate on group 3

13 May 2006(15)

Change intake-up rate

Empirical take-up rates, pre- and post-reform(standard errors in parentheses)

Population groupPre-reform take-up rate(FRS 2000/1)

Post-reform take-up rate(FRS 2001/2)

Mean entitlement(£ per week)(FRS 2000/1)

Mean entitlement(£ per week)(FRS 2001/2)

Change in mean

entitlement(£ per week)

Single disabledn2000/1 = 189; n2001/2 =189

.577(.036)

.651(.034)

.074(.050)

44.68(1.739)

48.21 (1.858)

3.53(2.545)

Couple, at least one disabledn2000/1 =57; n2001/2 =34

.579(.066)

.618(.085)

.039(.108)

37.23(4.742)

41.37 (5.345)

4.14(7.145)

Single aged below 70n2000/1 =66; n2001/2 =38

.864(.043)

.868 (.055)

0.004(.070)

16.47(1.749)

32.31(3.689)

15.84(4.083)

Single aged 70-74n2000/1 =106; n2001/2 =97

.632(.047)

.835(.038)

0.203(.060)

16.14(1.663 )

29.42(2.105)

13.28(2.683)

Single aged 75-79n2000/1 =116; n2001/2 =119

.690(.043)

.731(.041)

0.041(.059)

14.58(1.396)

28.60(2.050)

14.02(2.480)

Single aged 80 or aboven2000/1 =215; n2001/2 =189

.637(.033)

.582(.036)

-0.055(.049)

18.56(1.251)

18.87(1.001)

0.31(1.602)

Couple at least one aged above 74n2000/1 =57; n2001/2 =45

.491(.067)

.311(.070)

-0.181(.097)

19.58(3.930)

31.23(4.244)

11.65(5.784)

Couple both below 74, one below 68n2000/1 =18; n2001/2 =21

.444(.120)

.476(.112)

0.032(.164)

48.69(10.485)

52.07(8.928)

3.38(13.771)

Couple both below 74, one above 68n2000/1 =21; n2001/2 =24

.381(.108)

.708(.095)

0.327(.144)

35.62(11.786)

47.71(7.022)

12.09(13.719)

All groupsn2000/1 =845; n2001/2 =756

.624(.017)

.656(.017)

.032(.024)

25.78(.911)

33.35(.942)

7.57(1.310)

13 May 2006(16)

Definitions & assumptionsCrude change in take-up rate

Causal impact of reform:

Assumptions: A1 conditionally ignorable non-responce

A1 Rt Ttt | Wt = w, for t = 0, 1 and all w, where Wt

= (Xt , Bt0, Bt

1)

A2 no counfunding macro level changes, conditional on Wt

A2 E(Ttr | Wt = w) independent of t for r = 0, 1 and all

w

Under these assumptions:= causal effect + effect of change in distribution of covariates

13 May 2006(17)

Matching approach

• Matching on variables, not PSM: “selection” into year 0 & year 1 samples is random so no confounding except for non-response

• stratification on Xt (sex, age, marital status, disability)

St k 1

9

Stk , t 0,1

• Nearest neighbour algorithm, distance minimization:

Di, ji Di, j j S1k

For each individual i within stratum S0k the matched individualj i S1k, satisfies

• Mahalanobis distance measure on (Bt0 , Bt

1)

• matching with replacement

• caliper options: reject matches breaching D i,j i

13 May 2006(18)

Matching approachStratum k estimator for change in take-up

k

1

n 0k iM0k

T1j i

1 T0i0

overall estimator of the reform-induced change

k 1

9

k k

Mkt is the set of nkt individuals in stratum k in year t, for whom a match can be found

where k is the relative size of stratum k in the baseline year t

13 May 2006(19)

Impact with 2000/1 characteristics:

*0

Impact with 2001/2 characteristics:

*1

Matching estimates with and without demographic stratification and caliper options(standard errors in parentheses)

caliper: none Є = 0.05

Є = 0.025

Є = 0.01

none Є = 0.05

Є = 0.025

Є = 0.01

2 strata: singles, couples.053

(.038).048

(.047).037

(.048).040

(.048).088

(.036).086

(.044).084

(.044).095

(.045)

3 strata: singles by gender, couples

.046(.039)

.036(.054)

.031(.055)

.027(.049)

.088(.037)

.086(.052)

.086(.052)

.087(.080)

9 strata: demographic groups(marital status, age, disability)

.035(.035)

.019(.046)

.021(.046)

.031(.057)

.086(.036)

.060(.043)

.067(.045)

.056(.055)

No stratification.092

(.040).090

(.042).085

(.042).082

(.043).094

(.037).099

(.038).091

(.038).092

(.039)

Results

13 May 2006(20)

Estimate of impact with 2000/1 characteristics: 0

Estimate of impact with 2001/2 characteristics: 1

Matching estimates by increase in entitlement, with and without stratification(standard errors in parentheses)

Size of increase in entitlement

Number of cases

Take-up rate2000/01

Take-up rate2001/02

(matched)

no stratification

2 strata 3 strata 9 strata

< £10 per week 542 0.638 0.6640.026

(0.051).007

(.065)-.013(.075)

.013(.093)

£10-15 per week 223 0.668 0.7710.103

(0.158).023

(.080).025

(.086).040

(.106)

>£15 per week 80 0.400 0.7370.337

(0.161)0.300

(0.145)0.303

(0.147)0.137

(0.106)

Size of increase in entitlement

Number of cases

Take-up2000/01

(matched)

Take-up2001/02

no stratification

2 strata 3 strata 9 strata

< £10 per week 484 0.566 0.6200.054

(0.048).045

(.058).017

(.068).047

(.088)

£10-15 per week 192 0.417 0.7710.354

(0.157).094

(.142).145

(.149).118

(.116)

>£15 per week 80 0.237 0.6000.362

(0.127)0.296

(0.097)0.290

(0.099)0.241

(0.101)

13 May 2006(21)

0%

20%

40%

60%

80%

100%

0.00

00

0.00

40

0.00

80

0.01

20

0.01

60

0.02

00

0.02

40

0.02

80

0.03

20

0.03

60

0.04

00

0.04

40

0.04

80

0.05

20

Distance threshold

Matching qualityPercentage of cases matched with D(i, j) < threshold

13 May 2006(22)

Parametric approach

PrT i 1|x i x i

0a 1

n 0 iS 0

x i1 A T0i

0

0p 1

n 0 iS 0

x i1 A x i

0 B

Probit model

Predicted change in the take-up rate (eg When i is sampled in 2000/1)

13 May 2006(23)

0ˆ a

0ˆ p

1ˆ a

1ˆ p

0ˆ a

0ˆ p

1ˆ a

1ˆ p

Parametric results: Predicted change in take-up rate (standard errors in parentheses)

Size of increase in entitlement

all < £10 per week £10-15 per week >£15 per week

Separate probits for each year, whole sample, full covariates set

.052(.018)

.015(.022)

.084(.032)

.212(.066)

.052(.002)

.038(.002)

.072(.004)

.104(.011)

n 839 540 221 78

.084(.018)

.048(.024)

.125(.025)

.138(.061)

.084(.004)

.058(.004)

.140(.015)

.123(.013)

n 753 408 267 78

Separate probits for each year, whole sample; same regressors as stratification variables in non parametric analysis

.069(.016)

.036(.020)

.098(.030)

.216(.056)

.069(.002)

.052(.002)

.096(.005)

.118(.010)

n 842 542 122 78

.100(.017)

.066(.024)

.139(.025)

.142(.058)

.100(.004)

.066(.003)

.135(.007)

.154(.015)

n 753 408 267 78

13 May 2006(24)

Conclusions• Even without a real or natural experiment or a control

group, it is worth trying to use nonparametric matching methods as a complement to parametric simulation models

• The take-up of MIG was significantly increased by the reform

• The effect was particularly large for those with largest potential gains from claiming

• Overall, a conventional probit take-up model for the 2001 MIG reform for UK pensioners works OK compared to the nonparametric results; but there is some evidence of probit under-prediction of the response to reform by people facing large gains – with potentially important welfare implications

• Next…

13 May 2006(25)

13 May 2006(26)

Pre-reform period

Publicity effect

Reform effect

Post-reform period

Publicity campaign

Time

Tak

e-up

pr

ope

nsit

y

2000 MIG publicity campaign

13 May 2006(27)

Balance of covariates with pair matching, no caliper, no stratification

Matching with 2000/1 characteristics

VariablesMean

treatedMean control

Mean matched control

Std % biasBefore

matching

Std % biasafter matching

%Reduction in Absolute Bias

Entitlement before reform 25.78 24.30 25.50 5.7 1.1 80.8

Entitlement after reform 34.74 33.35 34.57 5.3 0.7 87.4

Matching with 2001/2 characteristics

VariablesMean

treatedMean control

Mean matched control

Std % biasbefore

matching

Std % biasafter matching

%Reduction in Absolute Bias

Entitlement before reform 24.30 25.78 24.17 -5.7 0.5 90.9

Entitlement after reform 33.36 34.74 33.08 -5.3 1.1 79.9

13 May 2006(28) Take-up Probit estimates for single pensioners full covariates set

Sampled in 2000/1 Sampled in 2001/2

Regressor CoefficientStandard

errorCoefficient

Standard error

Owner -.444 .109 -.691 .118

Female .081 .154 -.175 .171

Black -.595 .448 -2.501 .644

Disabled .064 .118 .255 .132

Years worked -.001 .003 -.001 .003

Age -.086 .158 -.086 .169

Age2 .001 .001 .001 .001

Education -.032 .036 -.001 .019

ln entitlement .246 .055 .387 .091

Net income -.009 .001 -.011 .001

Constant 5.255 6.232 5.003 6.721

n 688 632

LR 2(10) 80.78 124.54

Pseudo R2 .0905 .1585

Log likelihood -405.789 -330.648

Test for parameter stability 2(10)=14.04

13 May 2006(29)Take-up Probit estimates for pensioners couples full covariates set

Sampled in 2000/2001 Sampled in 2001/2002

Covariate CoefficientStandard

errorCoefficient

Standard error

Owner -.560 .230 -.941 .256

Disabled head .256 .259 .561 .294

Disabled spouse .238 .246 .516 .276

Years worked head -.009 .009 .002 .010

Years worked spouse .010 .008 -.010 .008

Head’s education .087 .074 .014 .025

Spouse’s education -.040 .055 .061 .085

ln entitlement .356 .099 .406 .217

Net income -.001 .002 -.002 .003

Constant -1.399 1.337 -2.301 1.797

n 151 121

LR 2(9) 30.36 35.55

Pseudo R2 .1451 .2119

Log likelihood -89.455 -66.091

Test for parameter stability 2(9)=13.18

13 May 2006(30) Take-up probit estimates for pensioners stratification variables in non parametric analysis used as covariates

Sampled in 2000/2001 Sampled in 2001/2002

Covariate CoefficientStandard

errorCoefficient

Standard error

Single disabled -.532 .139 -.207 .153

Couple, at least one disabled -.312 .193 -.191 .247

Single aged below 70 .827 .216 .724 .278

Single aged 70-74 .065 .156 .607 .181

Single aged 75-79 .242 .153 .266 .157

Couple at least one aged over 74 -.380 .202 -.859 .227

Couple both below 74, one below 68 -.736 .338 -.558 .305

Couple both below 74, one over 68 -.473 .266 -.2597 .286

ln entitlement .356 .051 .430 .085

Constant -.555 .158 -.986 .255

n 842 753

LR 2(10) 82.38 81.20

Pseudo R2 0.0739 .0838

Log likelihood -515.975 -444.050

Chow test for parameters stability 2(10) = 16.5

13 May 2006(31)

D(i, j) = (P0i − P1j)’V −1 (P0i − P1j)