a program evaluation strategy the causal pathway
TRANSCRIPT
![Page 1: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/1.jpg)
A Program Evaluation Strategy
The Causal Pathway
![Page 2: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/2.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Evaluation Strategy
• Our Evaluation Model starts in the initation phase of a program, not upon its conclusion.
• We start by developing a “Causal Pathway” or roadmap, which identifies the steps necessary to achieve the desired goal.
![Page 3: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/3.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Causal Pathway Framework
A roadmap of a project that shows:
![Page 4: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/4.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Causal Pathway Framework
Impact
ImpactChange in the health, social,economic status of thepopulation
![Page 5: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/5.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Causal Pathway Framework
ImpactEffect
Effect:Change in the knowledge,attitudes, skills, behaviorof the population of interest
![Page 6: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/6.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Causal Pathway Framework
ImpactEffectOutputs
OutputsProducts and services that must be in place for theeffects and impact to occur.
![Page 7: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/7.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Causal Pathway Framework
ImpactEffectOutputsActivities
ActivitiesTechnical and support tasksrequired to produce theoutputs
![Page 8: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/8.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Causal Pathway Framework
ImpactEffectOutputsActivitiesInputs
InputsResources needed to support the activities
![Page 9: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/9.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Evaluation Strategy
• This Evaluation model starts by helping to design a program which has the highest probability of achieving success
• We use this model to plan the program starting with the desired Goal or Impact, and planning all intermediate steps.
![Page 10: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/10.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Causal Pathway Framework
ImpactEffectOutputsActivitiesInputs
Design Direction
![Page 11: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/11.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Evaluation Strategy
• Given that we now have a program with built-in logical steps toward success, we can then use this Causal Pathway tool to evaluate the progress and success of the program.
![Page 12: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/12.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Evaluation Strategy
• Whereas in the planning phase, we worked from the desired Impact back to necessary Inputs, in the Evaluation Phase we monitor the completion of each step from Inputs to Activities to Outputs to Effects to Impact.
![Page 13: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/13.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Causal Pathway Framework
ImpactEffectOutputsActivitiesInputs
Monitoring and Evaluation Direction
![Page 14: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/14.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Evaluation Strategy
• One of the benefits of using this same tool in Evaluation is that even if there has not been enough time to realize a change in Impact, we can still determine how far along the logical pathway our program has advanced.
![Page 15: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/15.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Causal Pathway FrameworkSay for instance that we observe no change in Impact:
ImpactEffectOutputsActivitiesInputs
No Impact
Monitoring and Evaluation Direction
![Page 16: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/16.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Evaluation Strategy
• We can still determine how far along the logical pathway our program has advanced as an indication of success.
• We can also use this process to Diagnose where things are not working
![Page 17: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/17.jpg)
Center for Population and Family HealthJoseph L. Mailman School of Public Health of Columbia University
Causal Pathway Framework
ImpactEffectOutputsActivitiesInputs
Were inputsavailable?
Were activities performed?
Were outputsproduced?
Were effectsobserved?
No Impact
![Page 18: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/18.jpg)
BDI LOGIC MODEL
![Page 19: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/19.jpg)
“If you don’t know where you want to go, how do you know if you’ve gotten there?”
THE RABBIT, Alice in Wonderland
![Page 20: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/20.jpg)
BDI Logic Model
Health GoalBehaviorDeterminantsIntervention
![Page 21: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/21.jpg)
What is resiliency?
![Page 22: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/22.jpg)
Risk & Protective Factors
![Page 23: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/23.jpg)
Risk Factors: conditions that increase the likelihood of
outcomes that comprise health, quality of life, or life itself
![Page 24: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/24.jpg)
Risk Factors Occur in 5 major life domains:
family
school
community
peer groups
individual
![Page 25: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/25.jpg)
The more risk factors, the greater the risk
![Page 26: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/26.jpg)
Common risk factors predict diverse behavioral problems:
• substance abuse
• delinquency
• violent behavior
• risky sex teen pregnancy
• school behavior problems dropping out
![Page 27: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/27.jpg)
If the impact of any given risk factor can be reduced, it is likely to reduce several risk
behaviors
![Page 28: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/28.jpg)
Risk Behavior: behavior that can compromise adolescent psychosocial development
Examples?
![Page 29: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/29.jpg)
Risk Taking Behavior: behaviors that entail a
conscious awareness of the risk/danger in a deliberate
thrill-seeking attemptExamples?
![Page 30: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/30.jpg)
Protective Factors: factors that buffer youth against the
adverse effects of risk factors to which they are exposed
and diminish the likelihood of negative health and social outcomes
![Page 31: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/31.jpg)
Protective Factors occur in many domains:
• individual (e.g. resilient temperament, social & cognitive skills)
• connections to family, friends, school, other adults
• opportunities for involvement (e.g. church, youth clubs in communities)
![Page 32: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/32.jpg)
Many adolescent risk behaviors serve a purpose and are goal-
directed
![Page 33: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/33.jpg)
Drinking, smoking grass, smoking cigarettes
• feel relaxed
• feel socially accepted
• feel independent
• feel grown up
• feel excitement bucking the system
![Page 34: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/34.jpg)
Magnitude of Risk?
• intensity of involvement
• # of risk behaviors
• age of onset
• presence of protective factors
![Page 35: A Program Evaluation Strategy The Causal Pathway](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d0a5503460f949dd42c/html5/thumbnails/35.jpg)
Implications for Prevention
• comprehensive, multiple domains & multiple behaviors
• target risk and protective factors
• address larger social context in which behavior embedded