Download - Fundamentals of Program Impact Evaluation
![Page 1: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/1.jpg)
Peter M. Lance, PhDMEASURE Evaluation University of North Carolina at Chapel Hill
MARCH 31, 2016
Fundamentals of Program Impact Evaluation
![Page 2: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/2.jpg)
Global, five-year, $180M cooperative agreement
Strategic objective:
To strengthen health information systems – the capacity to gather, interpret, and use data – so countries can make better decisions and sustain good health outcomes over time.
Project overview
![Page 3: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/3.jpg)
Improved country capacity to manage health information systems, resources, and staff
Strengthened collection, analysis, and use of routine health data
Methods, tools, and approaches improved and applied to address health information challenges and gapsIncreased capacity for rigorous evaluation
Phase IV Results Framework
![Page 4: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/4.jpg)
Global footprint (more than 25 countries)
![Page 5: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/5.jpg)
![Page 6: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/6.jpg)
How Do We Know If A Program Made A Difference? A Brief Helicopter Tour of Methods for Estimating Program
Impact
![Page 7: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/7.jpg)
• The Program Impact Evaluation Challenge
• Randomization
• Selection on observables
• Within estimators
• Instrumental variables
![Page 8: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/8.jpg)
• The Program Impact Evaluation Challenge
• Randomization
• Selection on observables
• Within estimators
• Instrumental variables
![Page 9: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/9.jpg)
• The Program Impact Evaluation Challenge
• Randomization
• Selection on observables
• Within estimators
• Instrumental variables
![Page 10: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/10.jpg)
• The Program Impact Evaluation Challenge
• Randomization
• Selection on observables
• Within estimators
• Instrumental variables
![Page 11: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/11.jpg)
• The Program Impact Evaluation Challenge
• Randomization
• Selection on observables
• Within estimators
• Instrumental variables
![Page 12: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/12.jpg)
• The Program Impact Evaluation Challenge
• Randomization
• Selection on observables
• Within estimators
• Instrumental variables
![Page 13: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/13.jpg)
• The Program Impact Evaluation Challenge
• Randomization
• Selection on observables
• Within estimators
• Instrumental variables
![Page 14: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/14.jpg)
Newton’s “Laws” of Motion
![Page 15: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/15.jpg)
![Page 16: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/16.jpg)
![Page 17: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/17.jpg)
Did the program make a difference?
![Page 18: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/18.jpg)
Did the program cause a change in an outcome of interest Y ?
(Causality)
![Page 19: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/19.jpg)
Our outcome of Interest
What happens if an individual does not participate in a program
What happens if that individual does participate in a program
Potential Outcomes
:
:
:
![Page 20: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/20.jpg)
Our outcome of interest
What happens if an individual does not participate in a program
What happens if that individual does participate in a program
Potential Outcomes
:
:
:
![Page 21: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/21.jpg)
Our outcome of interest
What happens if an individual does not participate in a program
What happens if that individual does participate in a program
Potential Outcomes
:
:
:
![Page 22: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/22.jpg)
Our outcome of interest
What happens if an individual does not participate in a program
What happens if that individual does participate in a program
Potential Outcomes
:
:
:
![Page 23: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/23.jpg)
What happens if
the individual participates
{Causal} Program Impact
Program Impact
What happens if
the individual does not
participate
![Page 24: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/24.jpg)
What happens if
the individual participates
{Causal} Program Impact
Program Impact
What happens if
the individual does not
participate
![Page 25: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/25.jpg)
What happens if
the individual participates
{Causal} Program Impact
Program Impact
What happens if
the individual does not
participate
![Page 26: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/26.jpg)
What happens if
the individual participates
{Causal} Program Impact
Program Impact
What happens if
the individual does not
participate
![Page 27: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/27.jpg)
What happens if
the individual participates
{Causal} Program Impact
Program Impact
What happens if
the individual does not
participate
![Page 28: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/28.jpg)
𝑃 𝑖={1 if individual 𝑖participates ¿0 if individual 𝑖does not participate
Program Participation
![Page 29: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/29.jpg)
𝑌 𝑖=𝑃 𝑖∙𝑌 𝑖1+(1−𝑃 𝑖 ) ∙𝑌 𝑖
0
Observed Outcome
![Page 30: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/30.jpg)
𝑌 𝑖=𝑃 𝑖∙𝑌 𝑖1+(1−𝑃 𝑖 ) ∙𝑌 𝑖
0
Observed Outcome
𝑃 𝑖=1
![Page 31: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/31.jpg)
𝑌 𝑖=1∙𝑌 𝑖1+ (1−1 ) ∙𝑌 𝑖
0
Observed Outcome
𝑃 𝑖=1
![Page 32: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/32.jpg)
𝑌 𝑖=𝑌 𝑖1+0 ∙𝑌 𝑖
0
Observed Outcome
𝑃 𝑖=1
![Page 33: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/33.jpg)
𝑌 𝑖=𝑌 𝑖1
Observed Outcome
𝑃 𝑖=1
![Page 34: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/34.jpg)
𝑌 𝑖=𝑃 𝑖∙𝑌 𝑖1+(1−𝑃 𝑖 ) ∙𝑌 𝑖
0
Observed Outcome
![Page 35: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/35.jpg)
𝑌 𝑖=𝑃 𝑖∙𝑌 𝑖1+(1−𝑃 𝑖 ) ∙𝑌 𝑖
0
Observed Outcome
𝑃 𝑖=0
![Page 36: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/36.jpg)
𝑌 𝑖=0 ∙𝑌 𝑖1+(1−0 ) ∙𝑌 𝑖
0
Observed Outcome
𝑃 𝑖=0
![Page 37: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/37.jpg)
𝑌 𝑖=𝑌 𝑖0
Observed Outcome
𝑃 𝑖=0
![Page 38: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/38.jpg)
{𝑌 𝑖1 ,𝑌 𝑖
0 }
Observed Outcome
![Page 39: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/39.jpg)
{𝑌 𝑖1 ,𝑌 𝑖
0 }
Observed Outcome
![Page 40: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/40.jpg)
{𝑌 𝑖1 ,𝑌 𝑖
0 }
Observed Outcome
![Page 41: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/41.jpg)
{𝑌 𝑖1 ,𝑌 𝑖
0 }
Observed Outcome
![Page 42: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/42.jpg)
{𝑌 𝑖1 ,𝑌 𝑖
0 }
Observed Outcome
Fundamental Identification Problem
of Program Impact Evaluation
![Page 43: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/43.jpg)
{𝑌 𝑖1 ,𝑌 𝑖
0 }
Observed Outcome
Fundamental Identification Problem
of Program Impact Evaluation
![Page 44: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/44.jpg)
Individual Population
![Page 45: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/45.jpg)
Individual Population Hi. They call me
individual i
![Page 46: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/46.jpg)
Individual Population ?!?
![Page 47: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/47.jpg)
![Page 48: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/48.jpg)
{𝑌 𝑖1 ,𝑌 𝑖
0 }
![Page 49: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/49.jpg)
{𝑌 𝑖1 ,𝑌 𝑖
0 }
![Page 50: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/50.jpg)
An expected value for a random variable is the average value from a large number of repetitions of the experiment that random variable represents
An expected value is the true average of a random variable across a population
Expected Value
![Page 51: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/51.jpg)
An expected value for a random variable is the average value from a large number of repetitions of the experiment that random variable represents
An expected value is the true average of a random variable across a population
Expected Value
![Page 52: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/52.jpg)
An expected value is the true average of a random variable across a population
Expected Value
![Page 53: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/53.jpg)
Expectations: Properties
![Page 54: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/54.jpg)
Expectations: Properties
![Page 55: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/55.jpg)
Expectations: Properties
![Page 56: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/56.jpg)
Expectations: Properties
![Page 57: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/57.jpg)
Expectations: Properties
![Page 58: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/58.jpg)
Expectations: Properties
![Page 59: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/59.jpg)
Expectations: Properties
![Page 60: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/60.jpg)
Expectations: Properties
![Page 61: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/61.jpg)
Expectations: Properties
![Page 62: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/62.jpg)
Expectations: Properties
![Page 63: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/63.jpg)
Average Treatment Effect (ATE)
Average Effect of Treatment on the Treated (ATT)
Hi there
Individual Impact
![Page 64: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/64.jpg)
𝑌 𝑖1−𝑌 𝑖
0
![Page 65: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/65.jpg)
𝐸 (𝑌 𝑖1−𝑌 𝑖
0 )
![Page 66: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/66.jpg)
Average Treatment Effect (ATE)
Average Effect of Treatment on the Treated (ATT)
Treatment Effects
![Page 67: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/67.jpg)
![Page 68: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/68.jpg)
![Page 69: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/69.jpg)
Suppose that we have a sample of individuals….
…but for each individual we observe either or …
…but not both
So how do we estimate??
![Page 70: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/70.jpg)
Suppose that we have a sample of individuals….
…but for each individual we observe either or …
…but not both
So how do we estimate??
![Page 71: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/71.jpg)
Remember, however, a key property of expectations:
…but this means that in principle we could estimate and
separately
So how do we estimate??
![Page 72: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/72.jpg)
Remember, however, a key property of expectations:
…but this means that in principle we could estimate and
separately
So how do we estimate??
![Page 73: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/73.jpg)
For instance, suppose that in our sample we have:
participants()
and
non-participants()
(hence )
So how do we estimate??
![Page 74: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/74.jpg)
Then an estimator of is
calculated with the participants out of the sample of individuals
So how do we estimate??
![Page 75: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/75.jpg)
Then an estimator of is
calculated with the participants out of the sample of individuals
So how do we estimate??
![Page 76: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/76.jpg)
Then an estimator of is
calculated with the participants out of the sample of individuals
So how do we estimate??
![Page 77: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/77.jpg)
Then an estimator of is
calculated with the participants out of the sample of individuals
So how do we estimate??
![Page 78: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/78.jpg)
Then an estimator of is
calculated with the participants out of the sample of individuals
So how do we estimate??
![Page 79: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/79.jpg)
Then an estimator of is
calculated with the participants out of the sample of individuals
So how do we estimate??
![Page 80: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/80.jpg)
Similarly, an estimator of is
calculated with the non-participants out of the sample of individuals
So how do we estimate??
![Page 81: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/81.jpg)
So then an estimate of
is
So how do we estimate??
![Page 82: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/82.jpg)
But is it a good estimate??
![Page 83: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/83.jpg)
![Page 84: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/84.jpg)
![Page 85: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/85.jpg)
![Page 86: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/86.jpg)
So we have two samples of size
By random chance, between the two samples we almost surely have
1. A different precise mix of individuals
2. A different number of participants () and non-participants ()
3. Different estimates and of and : 𝑌 1=
∑𝑗=1
𝑛𝑃
𝑌 𝑗
𝑛𝑃 =∑𝑗=1
𝑛𝑃
𝑌 𝑗1
𝑛𝑃
𝑌 0=∑𝑘=1
𝑛𝑁
𝑌 𝑘
𝑛𝑁 =∑𝑘=1
𝑛𝑁
𝑌 𝑘0
𝑛𝑁
![Page 87: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/87.jpg)
So we have two samples of size
By random chance, between the two samples we almost surely have
1. A different precise mix of individuals
2. A different number of participants () and non-participants ()
3. Different estimates and of and : 𝑌 1=
∑𝑗=1
𝑛𝑃
𝑌 𝑗
𝑛𝑃 =∑𝑗=1
𝑛𝑃
𝑌 𝑗1
𝑛𝑃
𝑌 0=∑𝑘=1
𝑛𝑁
𝑌 𝑘
𝑛𝑁 =∑𝑘=1
𝑛𝑁
𝑌 𝑘0
𝑛𝑁
![Page 88: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/88.jpg)
So we have two samples of size
By random chance, between the two samples we almost surely have
1. A different precise mix of individuals
2. A different number of participants () and non-participants ()
3. Different estimates and of and : 𝑌 1=
∑𝑗=1
𝑛𝑃
𝑌 𝑗
𝑛𝑃 =∑𝑗=1
𝑛𝑃
𝑌 𝑗1
𝑛𝑃
𝑌 0=∑𝑘=1
𝑛𝑁
𝑌 𝑘
𝑛𝑁 =∑𝑘=1
𝑛𝑁
𝑌 𝑘0
𝑛𝑁
![Page 89: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/89.jpg)
So we have two samples of size
By random chance, between the two samples we almost surely have
1. A different precise mix of individuals
2. A different number of participants () and non-participants ()
3. Different estimates and of and : 𝑌 1=
∑𝑗=1
𝑛𝑃
𝑌 𝑗
𝑛𝑃 =∑𝑗=1
𝑛𝑃
𝑌 𝑗1
𝑛𝑃
𝑌 0=∑𝑘=1
𝑛𝑁
𝑌 𝑘
𝑛𝑁 =∑𝑘=1
𝑛𝑁
𝑌 𝑘0
𝑛𝑁
![Page 90: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/90.jpg)
So we have two samples of size
By random chance, between the two samples we almost surely have
1. A different precise mix of individuals
2. A different number of participants () and non-participants ()
3. Different estimates and of and : 𝑌 1=
∑𝑗=1
𝑛𝑃
𝑌 𝑗
𝑛𝑃 =∑𝑗=1
𝑛𝑃
𝑌 𝑗1
𝑛𝑃
𝑌 0=∑𝑘=1
𝑛𝑁
𝑌 𝑘
𝑛𝑁 =∑𝑘=1
𝑛𝑁
𝑌 𝑘0
𝑛𝑁
![Page 91: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/91.jpg)
So we have two samples of size
By random chance, between the two samples we almost surely have
1. A different precise mix of individuals
2. A different number of participants () and non-participants ()
3. Different estimates and of and : �̂� 𝟏=
∑𝒋=𝟏
𝒏𝑷
𝒀 𝒋
𝒏𝑷 =∑𝒋=𝟏
𝒏𝑷
𝒀 𝒋𝟏
𝒏𝑷
𝑌 0=∑𝑘=1
𝑛𝑁
𝑌 𝑘
𝑛𝑁 =∑𝑘=1
𝑛𝑁
𝑌 𝑘0
𝑛𝑁
�̂� 𝟏 𝑬 (𝒀𝟏 )
![Page 92: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/92.jpg)
![Page 93: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/93.jpg)
![Page 94: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/94.jpg)
![Page 95: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/95.jpg)
![Page 96: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/96.jpg)
![Page 97: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/97.jpg)
![Page 98: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/98.jpg)
![Page 99: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/99.jpg)
![Page 100: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/100.jpg)
![Page 101: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/101.jpg)
![Page 102: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/102.jpg)
�̂� 𝟏=∑𝒋=𝟏
𝒏𝑷
𝒀 𝒋
𝒏𝑷 =∑𝒋=𝟏
𝒏𝑷
𝒀 𝒋𝟏
𝒏𝑷
![Page 103: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/103.jpg)
�̂� 𝟏=∑𝒋=𝟏
𝒏𝑷
𝒀 𝒋
𝒏𝑷 =∑𝒋=𝟏
𝒏𝑷
𝒀 𝒋𝟏
𝒏𝑷
![Page 104: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/104.jpg)
![Page 105: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/105.jpg)
![Page 106: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/106.jpg)
![Page 107: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/107.jpg)
![Page 108: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/108.jpg)
![Page 109: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/109.jpg)
�̂� 𝟏=∑𝒋=𝟏
𝒏𝑷
𝒀 𝒋
𝒏𝑷 =∑𝒋=𝟏
𝒏𝑷
𝒀 𝒋𝟏
𝒏𝑷
𝑬 (�̂� 𝟏)=𝑬 (𝒀𝟏 )
![Page 110: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/110.jpg)
�̂� 𝟏=∑𝒋=𝟏
𝒏𝑷
𝒀 𝒋
𝒏𝑷 =∑𝒋=𝟏
𝒏𝑷
𝒀 𝒋𝟏
𝒏𝑷
𝑬 (�̂� 𝟏)=𝑬 (𝒀𝟏 )
![Page 111: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/111.jpg)
![Page 112: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/112.jpg)
![Page 113: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/113.jpg)
𝑬 (�̂� 𝟏)=𝑬 (∑𝒋=𝟏𝒏𝑷
𝒀 𝒋𝟏
𝒏𝑷 )𝑬 (�̂� 𝟏)=𝑬 (𝒀𝟏 )
𝑬 (�̂� 𝟏)=𝒏𝑷 ∙𝑬 (∑𝒋=𝟏𝒏𝑷
𝑬 (𝒀 𝒋𝟏 ))
𝑬 (�̂� 𝟏)= 𝟏𝒏𝑷 ∙𝑬 (∑𝒋=𝟏
𝒏𝑷
𝑬 (𝒀 𝒋𝟏))
𝑬 (�̂� 𝟏)= 𝟏𝒏𝑷 ∙𝒏
𝑷 ∙𝑬 (𝒀 𝒋𝟏 )
𝑬 (�̂� 𝟏)=𝑬 (𝒀 𝒋𝟏)
1ST Rule:
2nd Rule:
![Page 114: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/114.jpg)
𝑬 (�̂� 𝟏)=𝑬 (∑𝒋=𝟏𝒏𝑷
𝒀 𝒋𝟏
𝒏𝑷 )𝑬 (�̂� 𝟏)=𝑬 (𝒀𝟏 )
𝑬 (�̂� 𝟏)=𝒏𝑷 ∙𝑬 (∑𝒋=𝟏𝒏𝑷
𝑬 (𝒀 𝒋𝟏 ))
𝑬 (�̂� 𝟏)= 𝟏𝒏𝑷 ∙𝑬 (∑𝒋=𝟏
𝒏𝑷
𝑬 (𝒀 𝒋𝟏))
𝑬 (�̂� 𝟏)= 𝟏𝒏𝑷 ∙𝒏
𝑷 ∙𝑬 (𝒀 𝒋𝟏 )
𝑬 (�̂� 𝟏)=𝑬 (𝒀 𝒋𝟏)
1ST Rule:
2nd Rule:
![Page 115: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/115.jpg)
𝑬 (�̂� 𝟏)=𝑬 (∑𝒋=𝟏𝒏𝑷
𝒀 𝒋𝟏
𝒏𝑷 )𝑬 (�̂� 𝟏)=𝑬 (𝒀𝟏 )
1ST Rule: 1ST Rule:
2nd Rule:
![Page 116: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/116.jpg)
𝑬 (�̂� 𝟏)=𝑬 (∑𝒋=𝟏𝒏𝑷
𝒀 𝒋𝟏
𝒏𝑷 )𝑬 (�̂� 𝟏)=𝑬 (𝒀𝟏 )
𝑬 (�̂� 𝟏)= 𝟏𝒏𝑷 ∙(∑𝒋=𝟏
𝒏𝑷
𝑬 (𝒀 𝒋𝟏 ))
1ST Rule: 1ST Rule:
2nd Rule:
![Page 117: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/117.jpg)
𝑬 (�̂� 𝟏)=𝑬 (∑𝒋=𝟏𝒏𝑷
𝒀 𝒋𝟏
𝒏𝑷 )𝑬 (�̂� 𝟏)=𝑬 (𝒀𝟏 )
𝑬 (�̂� 𝟏)= 𝟏𝒏𝑷 ∙(∑𝒋=𝟏
𝒏𝑷
𝑬 (𝒀 𝒋𝟏 ))
𝑬 (�̂� 𝟏)= 𝟏𝒏𝑷 ∙𝒏
𝑷 ∙𝑬 (𝒀 𝒋𝟏 )
𝑬 (�̂� 𝟏)=𝑬 (𝒀 𝒋𝟏)
1ST Rule:
![Page 118: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/118.jpg)
𝑬 (�̂� 𝟏)=𝑬 (∑𝒋=𝟏𝒏𝑷
𝒀 𝒋𝟏
𝒏𝑷 )𝑬 (�̂� 𝟏)=𝑬 (𝒀𝟏 )
𝑬 (�̂� 𝟏)= 𝟏𝒏𝑷 ∙(∑𝒋=𝟏
𝒏𝑷
𝑬 (𝒀 𝒋𝟏 ))
𝑬 (�̂� 𝟏)= 𝟏𝒏𝑷 ∙𝒏
𝑷 ∙𝑬 (𝒀 𝒋𝟏 )1ST Rule:
![Page 119: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/119.jpg)
𝑬 (�̂� 𝟏)=𝑬 (∑𝒋=𝟏𝒏𝑷
𝒀 𝒋𝟏
𝒏𝑷 )𝑬 (�̂� 𝟏)=𝑬 (𝒀𝟏 )
𝑬 (�̂� 𝟏)= 𝟏𝒏𝑷 ∙𝑬 (∑𝒋=𝟏
𝒏𝑷
𝑬 (𝒀 𝒋𝟏))
𝑬 (�̂� 𝟏)= 𝟏𝒏𝑷 ∙𝒏
𝑷 ∙𝑬 (𝒀 𝒋𝟏 )
𝑬 (�̂� 𝟏)=𝑬 (𝒀 𝒋𝟏)
![Page 120: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/120.jpg)
𝑬 (�̂� 𝟏)=𝑬 (𝒀 𝒋𝟏)
𝑬 (�̂� 𝟏)=𝑬 (𝒀𝟏 )
𝑬 (𝒀 𝒋𝟏 )=𝑬 (𝒀𝟏 )
![Page 121: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/121.jpg)
𝑬 (�̂� 𝟏)=𝑬 (𝒀 𝒋𝟏)
𝑬 (�̂� 𝟏)=𝑬 (𝒀𝟏 )
𝑬 (𝒀 𝒋𝟏 )=𝑬 (𝒀𝟏 )
![Page 122: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/122.jpg)
𝑬 (�̂� 𝟏)=𝑬 (𝒀 𝒋𝟏)
𝑬 (�̂� 𝟏)=𝑬 (𝒀𝟏 )
𝑬 (𝒀 𝒋𝟏 )=𝑬 (𝒀𝟏 )
![Page 123: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/123.jpg)
𝑬 (𝒀 𝒋𝟏 )=𝑬 (𝒀𝟏 )
![Page 124: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/124.jpg)
![Page 125: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/125.jpg)
![Page 126: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/126.jpg)
𝑬 (𝒀𝟏 )
![Page 127: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/127.jpg)
𝑃=0𝑃=0
𝑃=0
𝑃=1𝑃=1
𝑃=1
𝑃=0
𝑃=1𝑃=1
𝑃=1
𝑃=0
𝑃=1
𝑃=1
𝑃=0
𝑃=0
𝑃=1
𝑃=1
𝒀 𝟏
![Page 128: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/128.jpg)
𝑃=0𝑃=0
𝑃=0
𝑃=1𝑃=1
𝑃=1
𝑃=0
𝑃=1𝑃=1
𝑃=1
𝑃=0
𝑃=1
𝑃=1
𝑃=0
𝑃=0
𝑃=1
𝑃=0
𝒀 𝟏
![Page 129: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/129.jpg)
𝑃=1𝑃=1
𝑃=1
𝑃=1𝑃=1
𝑃=1
𝑃=1
𝑃=1
𝑃=1
𝒀 𝟏
![Page 130: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/130.jpg)
Z W
“Z Causes W”
𝑬 (𝑾∨𝒁 )≠𝑬 (𝑾 )
![Page 131: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/131.jpg)
Z W
“Z causes W”
𝑬 (𝑾∨𝒁 )≠𝑬 (𝑾 )
![Page 132: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/132.jpg)
Z W
“Z causes W”
𝑬 (𝑾∨𝒁 )≠𝑬 (𝑾 )
![Page 133: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/133.jpg)
X Y1
![Page 134: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/134.jpg)
X
Y
P
![Page 135: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/135.jpg)
X
Y
P
0
![Page 136: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/136.jpg)
X
Y
P
![Page 137: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/137.jpg)
X
Y
P
![Page 138: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/138.jpg)
X Y1
![Page 139: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/139.jpg)
X Y1
![Page 140: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/140.jpg)
X Y1
![Page 141: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/141.jpg)
X
Y
P
![Page 142: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/142.jpg)
𝑃=1𝑃=1
𝑃=1
𝑃=1𝑃=1
𝑃=1
𝑃=1
𝑃=1
𝑃=1
![Page 143: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/143.jpg)
![Page 144: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/144.jpg)
![Page 145: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/145.jpg)
Selection Bias
![Page 146: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/146.jpg)
The estimator
of
would be biased if some individuals occurred only among participants or non-participants
Or more often among one of the two groups
![Page 147: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/147.jpg)
X
Y
P
![Page 148: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/148.jpg)
X
Y
P
![Page 149: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/149.jpg)
Sir Austin Bradford Hill
![Page 150: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/150.jpg)
Strength: How strong is the relationship?Consistency: How consistently is link found?Specificity: How specific is the setting or disease?Temporality: Does the cause precede the effect?Gradient: Does more cause lead to more
effect?Analogy: Do similar “causes” have similar effect?Coherence: Are field and laboratory findings similar?Experiment: Was variation in the cause random?Plausibility: Does theory agree?
Bradford Hill Criteria
![Page 151: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/151.jpg)
Strength: How strong is the relationship?Consistency: How consistently is link found?Specificity: How specific is the setting or disease?Temporality: Does the cause precede the effect?Gradient: Does more cause lead to more
effect?Analogy: Do similar “causes” have similar effect?Coherence: Are field and laboratory findings similar?Experiment: Was variation in the cause random?Plausibility: Does theory agree?
Bradford Hill Criteria
![Page 152: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/152.jpg)
Strength: How strong is the relationship?Consistency: How consistently is link found?Specificity: How specific is the setting or disease?Temporality: Does the cause precede the effect?Gradient: Does more cause lead to more
effect?Analogy: Do similar “causes” have similar effect?Coherence: Are field and laboratory findings similar?Experiment: Was variation in the cause random?Plausibility: Does theory agree?
Bradford Hill Criteria
![Page 153: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/153.jpg)
Strength: How strong is the relationship?Consistency: How consistently is link found?Specificity: How specific is the setting or disease?Temporality: Does the cause precede the effect?Gradient: Does more cause lead to more
effect?Analogy: Do similar “causes” have similar effect?Coherence: Are field and laboratory findings similar?Experiment: Was variation in the cause random?Plausibility: Does theory agree?
Bradford Hill Criteria
![Page 154: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/154.jpg)
Strength: How strong is the relationship?Consistency: How consistently is link found?Specificity: How specific is the setting or disease?Temporality: Does the cause precede the effect?Gradient: Does more cause lead to more
effect?Analogy: Do similar “causes” have similar effect?Coherence: Are field and laboratory findings similar?Experiment: Was variation in the cause random?Plausibility: Does theory agree?
Bradford Hill Criteria
![Page 155: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/155.jpg)
Strength: How strong is the relationship?Consistency: How consistently is link found?Specificity: How specific is the setting or disease?Temporality: Does the cause precede the effect?Gradient: Does more cause lead to more
effect?Analogy: Do similar “causes” have similar effect?Coherence: Are field and laboratory findings similar?Experiment: Was variation in the cause random?Plausibility: Does theory agree?
Bradford Hill Criteria
![Page 156: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/156.jpg)
Strength: How strong is the relationship?Consistency: How consistently is link found?Specificity: How specific is the setting or disease?Temporality: Does the cause precede the effect?Gradient: Does more cause lead to more
effect?Analogy: Do similar “causes” have similar effect?Coherence: Are field and laboratory findings similar?Experiment: Was variation in the cause random?Plausibility: Does theory agree?
Bradford Hill Criteria
![Page 157: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/157.jpg)
Strength: How strong is the relationship?Consistency: How consistently is link found?Specificity: How specific is the setting or disease?Temporality: Does the cause precede the effect?Gradient: Does more cause lead to more
effect?Analogy: Do similar “causes” have similar effect?Coherence: Are field and laboratory findings similar?Experiment: Was variation in the cause random?Plausibility: Does theory agree?
Bradford Hill Criteria
![Page 158: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/158.jpg)
Strength: How strong is the relationship?Consistency: How consistently is link found?Specificity: How specific is the setting or disease?Temporality: Does the cause precede the effect?Gradient: Does more cause lead to more
effect?Analogy: Do similar “causes” have similar effect?Coherence: Are field and laboratory findings similar?Experiment: Was variation in the cause random?Plausibility: Does theory agree?
Bradford Hill Criteria
![Page 159: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/159.jpg)
Strength: How strong is the relationship?Consistency: How consistently is link found?Specificity: How specific is the setting or disease?Temporality: Does the cause precede the effect?Gradient: Does more cause lead to more
effect?Analogy: Do similar “causes” have similar effect?Coherence: Are field and laboratory findings similar?Experiment: Was variation in the cause random?Plausibility: Does theory agree?
Bradford Hill Criteria
![Page 160: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/160.jpg)
Strength: How strong is the relationship?Consistency: How consistently is link found?Specificity: How specific is the setting or disease?Temporality: Does the cause precede the effect?Gradient: Does more cause lead to more
effect?Analogy: Do similar “causes” have similar effect?Coherence: Are field and laboratory findings similar?Experiment: Was variation in the cause random?Plausibility: Does theory agree?
Bradford Hill Criteria
![Page 161: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/161.jpg)
We are presented with data in the form of a sample:
Causality: Our Approach
,
![Page 162: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/162.jpg)
We are presented with data in the form of a sample:
Causality: Our Approach
,
Assumptions
ModelE(Y1-Y0),
E(Y1-Y0|P=1),Etc.
![Page 163: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/163.jpg)
We are presented with data in the form of a sample:
Causality: Our Approach
,
Assumptions
ModelE(Y1-Y0),
E(Y1-Y0|P=1),Etc.
![Page 164: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/164.jpg)
Conclusion
![Page 165: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/165.jpg)
Links:
The manual:
http://www.measureevaluation.org/resources/publications/ms-14-87-en
The webinar introducing the manual:
http://www.measureevaluation.org/resources/webinars/methods-for-program-impact-evaluation
My email:[email protected]
![Page 166: Fundamentals of Program Impact Evaluation](https://reader038.vdocument.in/reader038/viewer/2022110219/5871fd191a28ab5c348b7cf7/html5/thumbnails/166.jpg)
MEASURE Evaluation is funded by the U.S. Agency for International Development (USAID) under terms of Cooperative Agreement AID-OAA-L-14-00004 andimplemented by the Carolina Population Center, University of North Carolina at Chapel Hill in partnership with ICF International, John Snow, Inc., Management Sciences for Health, Palladium Group, and Tulane University. The views expressed in this presentation do not necessarily reflect the views of USAID or the United States government.www.measureevaluation.org