arizona advocacy
DESCRIPTION
Slides from Evaluating Advocacy PresentationTRANSCRIPT
![Page 1: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/1.jpg)
Evaluating Advocacy:Dilemmas, Tactics, and Methods
Julia CoffmanCenter for Evaluation Innovation
October 7, 2011
![Page 2: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/2.jpg)
Three Questions
How is evaluating advocacy different?
What can we measure about advocacy?
How can we measure it?
![Page 3: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/3.jpg)
How is evaluating advocacy different?
![Page 4: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/4.jpg)
Challenge: The policy environment is complex, and that makes attribution hard.
![Page 5: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/5.jpg)
What is my uniquecontribution?
Solution: If accountability is the purpose, demonstration of contribution
is expected, not attribution.
![Page 6: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/6.jpg)
Shifting Politics Shifting Economics New Partners
Challenge: Advocacy strategies shift in response to the environment.
![Page 7: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/7.jpg)
EV
AL
UA
TIO
N
ADVOCACY
Solution: Advocacy is a good opportunity to integrate or embed evaluation for learning
(not just accountability).
![Page 8: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/8.jpg)
Challenge: Timeframes can be unpredictable.
Advocacy Timeframe
1 Year
Goal
Reporting Timeframe
1 Year 1 Year
10 Years5 Years1 Year
![Page 9: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/9.jpg)
Goal
…progress…
Solution: Assess progress, not just the end result.
![Page 10: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/10.jpg)
What can we measure about influencing?
![Page 11: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/11.jpg)
Measure meaningful things. Don’t just count what is easy to quantify.
![Page 12: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/12.jpg)
Have realistic expectations.
What are other
influencers doing?
What’s the opposition
doing? What are you doing and
who are you targeting?
What’s the political context?
Where is your issue in the policy
process?
![Page 13: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/13.jpg)
Policy Influence
…progress…
Measure the changes made along the way, not just the end result.
INTERIM OUTCOMES
![Page 14: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/14.jpg)
Interim outcomes are the changes you expect as you work toward your goal.
Think about the changes you will see in your audiences.
![Page 15: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/15.jpg)
AUDIENCES
DECISION MAKERS
OU
TCO
MES
ACTI
ON
WIL
LAW
AREN
ESS
PUBLIC INFLUENCERS
WHO will change as a result of your work?
HOW will they change as a result of your work?
Use the framework to think about interim outcomes.
![Page 16: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/16.jpg)
Where are your audiences?How far do you need to move them?
AUDIENCES
DECISION MAKERS
ACTI
ON
WIL
LAW
AREN
ESS
PUBLIC INFLUENCERS
Increase quality of child care
Child care providers
Parents of young
children
INTE
RIM
OU
TCO
MES
Legislators
![Page 17: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/17.jpg)
Increased media coverage
Increased collaboration
among advocates
Increased willingness to act
Increased knowledge
Changed attitudes or beliefs
New and active advocatesIncreased salience
Reframing of the issue
New donors
Increased or diversified funding
Increased issue visibility or recognition
New and active high-profile champions
Activities and
Outputs
Policy Goals
Interim OutcomesAwareness
Will
Action
Increased capacity to act
Increased personal or collective
efficacy
![Page 18: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/18.jpg)
How can we measure it?
![Page 19: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/19.jpg)
Interviews
Surveys
Focus Groups
Polling
Traditional Evaluation Methods
![Page 20: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/20.jpg)
System Mapping
Network Mapping
Advocacy Capacity Assessment
Critical Incident Timelines
Intense Period Debriefs
Media Scorecards
Research Panels
Crowdsourcing
Snapshot Surveys
Intercept Interviews
360-Degree Critical Incident Debriefs
Policy Maker Ratings
Media Tracking
ECCO Analysis
Policy Tracking
Non-Traditional Methods
Charting and Mapping
Survey and Interview
Debriefing and Rating
Media/Messaging Tracking
Bellwether Methodology
Champion Tracking
![Page 21: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/21.jpg)
Bellwether Methodology
► Policymakers
► Administrators
► Other Advocates
► Funders
► Business
► Media
Bellwethers are:
“Influentials” in the public and private sectors whose positions require that they are politically informed and that they track a broad range of policy issues
Developed by Harvard Family Research Project
![Page 22: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/22.jpg)
Bellwether Methodology
What are the top 3 Issues on the children’s (ages 0-18)
policy agenda?2005
(n=40)2008
(n=44)
1. K-12 education2. Health care3. Preschool
1. K-12 education2. Health care3. Preschool
![Page 23: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/23.jpg)
Yes 84%
No 14%
Don't Know 2%
2008(n=44)
Can preschool help address the achievement gap?
How would you address the achievement gap?
2008 FULL SAMPLE (n=72) %
1. K-12 reform 67%
2. Education funding/resources 47%
3. Nonschool supports (health care, family involvement,after school)
32%
4. Supplemental school programs (ELL, literacy programs)
29%
5. Preschool/early care and education 21%
6. Other 13%
Bellwether Methodology
![Page 24: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/24.jpg)
Very likely 9%
Somewhat likely 20%
Not very likely 50%
Not at all likely 20%
2008(n=44)
What is the likelihood that California will
increase preschool investments in the
next 3 years?
Bellwether Methodology
![Page 25: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/25.jpg)
Policymaker Ratings
Support Influence Confidence
Ratings are completed for a whole governing body or defined group of policymakers
Developed by Harvard Family Research Project
![Page 26: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/26.jpg)
Policymaker Ratings
35
71
15
3
35
55
23
7
0
10
20
30
40
50
60
70
80
Not at all supportive
Somewhat supportive
Supportive Extremely supportive
Year 1
Year 2
Developed by Harvard Family Research Project
![Page 27: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/27.jpg)
Policymaker Ratings
Alameda
AlpineAmador
Butte
Calaveras
Colusa
Costa
Del Norte
El Dorado
Fresno
Glenn
Humboldt
Imperial
Inyo
Kern
Kings
Lake
Lassen
Los Angeles
Madera
Marin
Mariposa
Mendocino
Merced
Modoc
Mono
Monterey
Napa
Nevada
Orange
Placer
Plumas
Riverside
Sacramento
Benito
San Bernardino
San Diego
San FranciscoJoaquin
Obispo
Mateo
Barbara
Santa ClaraSanta
Shasta
Sierra
Siskiyou
Solano
Sonoma
Sutter
Tehama
Trinity
Tulare
Tuolumne
Ventura
Yolo
Yuba
Contra
San
San
SanLuis
San
Santa
Cruz
Stanislaus
Low Support
Medium Support
High Support
= Increase in Support
* Hypothetical Data
Developed by Harvard Family Research Project
![Page 28: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/28.jpg)
Policymaker Ratings
Low Influence High Influence
Low Support
High Support
68 legislators 57% of the legislature
17 legislators 14% of the legislature
13 legislators 11% of the legislature
80% Democrat20% Republican
100% Assembly
85% Democrats15% Republican
75% Assembly25% Senate
50% Assembly50% Senate
67% Assembly33% Senate
49% Democrat51% Republican
64% Democrat36% Republican
22 legislators 18% of the legislatureLo Med Hi
Confidence Rating
70%5% 25%
Lo Med Hi
Confidence Rating
60% 40%
Lo Med Hi
Confidence Rating
100%
Lo Med Hi
Confidence Rating
40% 60%
Developed by Harvard Family Research Project
![Page 29: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/29.jpg)
Wrap Up
Stay anchored in what you can reasonably change.
It’s okay to prioritize and focus on what is most important to assess.
Be creative—advocacy evaluation is an emerging field.
![Page 30: Arizona advocacy](https://reader034.vdocument.in/reader034/viewer/2022051817/547a3b7cb4af9f13618b46b5/html5/thumbnails/30.jpg)
www.evaluationinnovation.org
Center for Evaluation Innovation