propensity score calipers and the overlap condition · 9/18/2017 propensity score calipers and the...

9/18/2017 Propensity score calipers and the overlap condition

https://benthestatistician.github.io/PISE/#1 1/19

Propensity score calipers and theoverlap conditionBen Hansen, UMich Statistics ([email protected]). Project site - github.com/benthestatistician/PISE (code/manuscript), benthestatistician.github.io/PISE (slides)



Outline1. Problem: determining the region of overlap

2. Propensity score caliper matching

3. PPSE calipers

4. Applications & discussion

/



Overlap & bias in comparisons w/ 1 covariateThe quasi-experimental setup: , ;

, .· X Y

Z ∈ {0, 1} Yc

·

Overlap?

Balance - i.e., is ?

·

Observations well outside region ofoverlap should be excluded.

Wasteful to exclude members of because their is just

outside .

May distort research question, too.

Close/far distinctions presuppose amodel.

-

-{i : = 1}Zi x

range({ : = 0})xj Zj

-

-

· ≈x̄t x̄c

The question presupposes overlap.(OW mean comparisons can bemisleading.)

A topic for another day...

-

-

/



Statistical assumptions for general esX

Now, multiple es: , not .

Strong ignorability conditions:

No unmeasured confounding says, there are natural experiments in each "cell" of .

Problem: if multiple es, many observations will be all alone in their cells.

Rosenbaum & Rubin's (1983) solution is match on , the PS.

Remaining problem: In a cell s.t. , there's no data to estimate , even as . (Whether or not you match or otherwise use PSes.)

The second problem leads people to reduce analytic sample to region of common supporton (the method of "strict overlap").

· X X X

·

No unmeasured confounding: .

Overlap: .

- ⊥ Z|XYc

- Pr(Z = 1|X) < 1

· X

· X

· (Z = 1|X = x)Prˆ

· x0 Pr(Z = 1|X = ) = 1x0

Pr( ∈ ⋅|X = )Yc x0 n ↑ ∞

·PSˆ

/



Example 1: violence & public infrastructure inMedellín

Medellin, Colombia (population 2 million)

As of early 2000s, 60\% poverty rate, 20\%unemployment, homicide 185 per 100K

High residential segregation, w/concentrated poverty in surrounding hills.

2004-2006: gondolas connect some butnot all to city center.

Cerda et al (2012, Am J. Epideomiol.) studye�ects on neighborhood violence,propensity matching treatmentneighborhoods to control.

·

·

·

·

·

/



Propensity scores in the Medellin studySmall matching problem: , (neighborhoods).

s and s not too far apart, but PS matchingbrought them closer.

Figure shows PS we matched on - the from alogistic regression (Cerda et al 2012, Am J Epi).

Region of strict overlap contains only 6/25 s and4/23 s!

Yet Cerda et al full-matched all 48 neighborhoods.

· = 25nt = 23nc

· x̄t x̄c

· Xβˆ

· t

c

·

/





3. PPSE calipers


/



Matching within PS calipers (R.& R.)For matching, useful to specify a tolerance or caliper.

Absent unmeasured confounding, matches with small PS di�erences mimic paired randomassignment.

In e�ect, caliper adjudicates "closeness" to region of common support.

·

·

·

R. & R. (1985, Amer. Statist.), Rubin & Thomas (2000,

JASA) recommend .

(Not the estimated probability, theestimated index function. I.e., logits of estimatedprobabilities.)

The other widely used method (e.g. Austin & Lee2009, Lunt 2013), besides requiring strict overlap.

·

( )/4sp PSˆ

· PSˆ PSˆ

·

/



Room for improvement (in caliper = ).25sp

In the Medellin study, caliper still excludes 14/25 treatment neighborhoods.

At the same time, in that study it's tenable ( ) that all PSes are the same!

· .25 ( )sp PSˆ

· p = .10

· Moral: in small studies, may betoo strict.

In large studies, PS may be estimable w/

much more precision than . Atsame time, more potential controls.

Moral: in large studies, may betoo loose.

· .25 ( )sp PSˆ

·

.25 ( )sp PSˆ

· .25 ( )sp PSˆ

/





3. PPSE calipers


/



Calipers as insurance that PS differences tend to0

Pairings (denoted " ") should satisfy as .

Can this be accomplished with a requirement of form , since

?

( won't do. We need . In itself, even this won't be quite enough.)

Scanning pairs , .

Expect .

Even w/ bounded, uniformly in , and , this is .We'll need to assume .

In that case, a that's will do the trick.

E.g., a measuring average size of .

· i ∼ j |( − )β| ↓ 0supi∼j x ⃗ i x ⃗ j n ↑ ∞

· |( − ) | ≤x ⃗ i x ⃗ j βˆ wn

|( − )β| ≤ |( − )( − )| + |( − ) |x ⃗ i x ⃗ j x ⃗ i x ⃗ j βˆ βˆ x ⃗ i x ⃗ j βˆ

· = /4wn sp 0wn →P

· i, j ≤ n |( − )( − β)| ≤ | − | − βx ⃗ i x ⃗ j βˆ x ⃗ i x ⃗ j |2 βˆ |2

· |( − )( − β)| ≈ ( | − )| − βsupi,j x ⃗ i x ⃗ j βˆ supi,j x ⃗ i x ⃗ j |2 βˆ |2

· | |xij i, j n O( ) ([p/n ) = (p/ )p1/2 OP ]1/2 OP n1/2

p/ ↓ 0n1/2

· wn (p/ )OP n1/2

· wn |( − )( − β)|x ⃗ i x ⃗ j βˆ

/



Sampling variability of paired PS differencesIntuition: Even if we had matched perfectly on the PS, there would still be matcheddi�erences in . Let's estimate the size of these di�erences & use result to de�ne caliper.

To limit model sensitivity, work on index scale.

If , is the error of estimation of paired PS di�erence .

Considering all possible pairs , the expected MS estimation error of paired PSdi�erences can be expressed as a Frobenius inner product:

·PSˆ

caliper as . So, paired PS di�erences .

excludes treatment subjects whose PSes are distinguishably di�erent from all controls'.

- ↓ 0 n ↑ ∞ →β ⇒βˆ ↓ 0

-

·

· d = −x1 x2 d( − β)βˆ dβ

· ( )n2 r

( ( − β) /( ) = 2⟨ , {( − β)( − β } .Eβ ∑r

dr βˆ )2 n

2S

(x)Eβ βˆ βˆ )

′⟩F

/



Sampling variability of paired PS diffs (ii)But all possible pairs include many that are quite di�erent on the PS. We wanted tocharacterize among pairs s.t. .

Since there may be few or no such pairs, instead "residualize" all pairs for the true PS,leaving remaining di�erences in place. For , de�ne residual of

regression on ; . By construction, for all .

Rather than the mean-square of PS-di�erence estimation errors, ,

consider the mean-square PS-di�erence estimation error, net of di�erences in the true PS:

where , .

Plugging in and a extracted from the regression �t gives

the "propensity-paired standard error" (PPSE).

·E( ( − β)dr βˆ )2 r β ≈ 0dr

·i = 1,… , p =d⊥(i) d(i)

dβ = ( … )d⊥ d⊥1 d⊥2 d⊥p β = 0d

⊥

r r

· E ( ( − β)( )n2−1

∑r dr βˆ )2

( ( − β) = 2⟨ , ,( )n

2

−1

Eβ ∑r

d⊥

r βˆ )2 S⊥ Cβˆ⟩F

= Cov( ) = Cov( )S⊥ 1

2d⊥

x⊥xβ = e(x|1, xβ)x⊥xβ

· = e(x|1, x e(x|1, x ))Sˆ⊥ 1

n−1βˆ)′ βˆ Cˆ

βˆ

/





3. PPSE calipers


/



Example 2: Real or cosmetic reform in policepractices?Vagrancy arrests in the 60s and 70s

Before 1960, arrests for vagrancy were relatively common in U.S.; by 1980, relatively rare.

Increasingly forbidden by laws, court rulings, administrative decisions.

Did pretextual arrests decline, a civil rights victory, or were they simply shifted to othercategories (displacement)?

Using administrative records assembled by R. Golubo�, D. Thacher and I identi�ed 121 (localagency, year) instances, of at-risk agency-years, that experienced a sharp,anomalaous decline in vagrancy arrests.

and we matched them to otherwise similar agencies in terms of year, region of U.S.,population, prior use of vagrancy arrests and a propensity score.

·

·

·

·n = 5600

·

/



Example 3: Vascular closure devices vs manualclosure following percutaneous coronaryintervention

Once this stent has been threaded up intoyour heart, the hole in your femoral arteryneeds to be closed.

VCDs are more comfortable than manualclosure -- are they as safe?

Gurm et al (2013, Ann Intern Med) useddata from a 32-hospital collaborative inMichgan to conduct a propensity-matchedstudy.

Large matching problem: K; K.

·

·

·

· = 31nt

= 54nc

/



ApplicationsFor the Medellin example, 2.5 PPSEs works out to 4 logits ( ).

No neighborhoods excluded. Recall that the alternatives excluded at least 12!

In the vagrancy study, 2.5 PPSEs = 3.8 logits ( ).

In the VCD example 2.5 PPSEs= 0.15 logits ( ).

Larger sample smaller PPSE.

· 3.9sp

·

· .5sp

· .3sp

· ⇒

/



Multiple PS calipers in the vagrancy study (Ex2)

/



Discussion

Recommendations:

Getting consistency out of PPSE caliper requires . If , still interpretable as anSE of sorts.

No need to restrict yourself to a single index score.

With a saturated PS model, observed information may be poorly conditioned. The method ofPPSE estimation as described here requires some elaboration (see code on github).

Likeness of observations ("like to like") is interpreted in terms of PS variables. If too strict ortoo loose, then adjust PS variables.

· /n ↓ 0p2 p/n ↓ 0

·

·

·

I like calipers of 2.5 PPSEs

I'm using sandwich estimates of , w/ some special sauce to limit numerical instability.

Our optmatch R package (Fredrickson et al, 2016) does optimal pair and full matching (Gu &Rosenbaum, 1993; Hansen & Klopfer, 2006; Stuart & Green, 2008), readily accommodatingPS calipers.

·

· Cov( )βˆ

·

/

propensity score calipers and the overlap condition · 9/18/2017 propensity score calipers and the...

Documents