building support through effective communication strategies
DESCRIPTION
Building Support through Effective Communication Strategies . Building Support through Effective Communication Strategies . Robert Kight Chief , Division of WIA Adult Services and the Workforce System, ETA Steven Baker - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/1.jpg)
Building Support through Effective Communication
Strategies
![Page 2: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/2.jpg)
Building Support through Effective Communication Strategies Robert KightChief, Division of WIA Adult Services and the Workforce System, ETASteven BakerVice President of Marketing and Communications, Jobs for the FutureJacob KlermanPrincipal Associate/Scientist, Abt Associates
2
![Page 3: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/3.jpg)
Here’s What to Expect
• Cover the basics of good story telling
• Framing your project’s story • Develop a working draft• Leveraging evaluation results
3
![Page 4: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/4.jpg)
How to use your story
4
• Internal communications• Talking points/speeches• Presentations• Funding proposals/reports• Articles/publications• Web/social media• Press releases/media coverage• Community engagement
![Page 5: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/5.jpg)
Fundamentals of a good story
5
•It is clear and understandable, no matter how technical the work may be
•It is relatable; the story connects the work to people
•It is compelling; the outcomes/benefits are easily understood
•It is motivating; your audience wants you to succeed and they want to help you
![Page 6: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/6.jpg)
Framing your story
6
First decide:• Who are your primary audiences?
• Program participants, businesses, funders, community leaders, policy makers
• What do they care about?• Populations, efficiencies, outcomes
• What will you want them to do?• Give direct support• Advocate on your behalf
![Page 7: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/7.jpg)
Drafting your story
7
• What are the problem(s) you’re trying to solve? Frame in terms of both systems and people
• What is your solution? How will it help? Avoid jargon; keep it simple
• How is your solution unique? What’s innovative? Don’t forget to take credit; you’re the hero!
• What are the intended outcomes? What will be different? Better?
Who will benefit? Why should people care?
![Page 8: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/8.jpg)
Using Evaluation Results
Jacob Alex KlermanWIF Grantees ConferenceWashington DC, March 2014
![Page 9: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/9.jpg)
Abt Associates | pg 9
Outline
WIF as the “Vanguard”
Impact Results
Earlier Results
An Exercise
![Page 10: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/10.jpg)
Abt Associates | pg 10
Outline
WIF as the “Vanguard”
Impact Results
Earlier Results
An Exercise
![Page 11: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/11.jpg)
Abt Associates | pg 11
WIF Grantees are the “Vanguard” Until recently, social policy in general and workforce
strategies in particular have been set and funds allocated based on “plausibility”– And, rigorous impact evaluation suggests that impacts have often
been mediocre
Increasingly (Obama Administration’s Evidence Agenda, ED’s i3, CNCS’s SIF), we see a move towards a new and better strategy: “Evidence Based Policy”– Pilot, rigorously (impact) evaluate, replicate– Broad scale rollout only after success at earlier steps– Leading to better programs and (any and) larger impacts
WIF is a key component of the strategy
![Page 12: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/12.jpg)
Abt Associates | pg 12
Perhaps “Bleeding Edge” … Being part of the “vanguard” is not easy
We are moving towards “evidence based policy” exactly because, when evaluated, many programs will not be found to be effective– So participating in WIF—and being evaluated—is the “right thing”– But, the results are not always pleasant
The rest of this talk considers how to deal with that reality
![Page 13: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/13.jpg)
Abt Associates | pg 13
Turn Burden into Selling Point In your materials, note that you are “doing the right thing”
– Stepping through the tiers of evidence
And, along the way, showing that:– Your program can actually implemented
(i.e., a successful pilot; see third part of talk)– You are incremental tweaking your program
(again, see third part of talk)– You can work constructive with an evaluator– You have good internal data systems that can support an evaluation– And, we hope building increasingly high quality evidence of impact
(see the next section of the talk)
… which are valued by “Evidence Based Policy” funders
![Page 14: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/14.jpg)
Abt Associates | pg 14
Outline
WIF as the “Vanguard”
Impact Results
Earlier Results
An Exercise
![Page 15: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/15.jpg)
Abt Associates | pg 15
Talking about Results;Especially if they are “Good” Evaluator should
produce a detailed report– Background, methods,
data, results
Someone (evaluator or grantee) should produce a brief “Executive Summary”– Standard format is one
page, two sides– See handout with example
![Page 16: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/16.jpg)
Abt Associates | pg 16
Talking about Results;Especially if they are “Good” Evaluator should
produce a detailed report– Background, methods,
data, results
Someone (evaluator or grantee) should produce a brief “Executive Summary”– Standard format is one
page, two sides– See handout with example
Executive SummaryStandard Outline
Overview: One paragraph program and findings
Background Program Design Methods Implementation findings Impact findings, with 1-2
figures Discussion References
You really need to do this, whether results are “good” or “bad”
![Page 17: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/17.jpg)
Abt Associates | pg 17
Strength of Methodology “Evidence Based Policy” is associated with “tiered evidence”
– Programs move up the “tiers of evidence”– Pilot->pre/post->QED->random assignment->replication– And only then to broad program roll-out
Claim credit for working through the tiers, but, don’t over-state the strength of the evidence– Random assignment provides the strongest evidence of impact– Pre/post and QED tend to over-state impact– They should, therefore, be taken (primarily) as evidence that moving
to the next tier of evidence is appropriate
Exercise 1: Grantees and evaluators should discuss the evaluation’s methodology and how they will describe it and its strength
![Page 18: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/18.jpg)
Abt Associates | pg 18
Consider Precision Carefully
A nearly ideal example: SEIS/Sectoral Employment Impact Study (Maguire, et al. 2010)– Random assignment evaluation found clear evidence of
larger impacts– Impact on total earnings of 24 months since randomization
Why do we (“the policy community”) say that?
![Page 19: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/19.jpg)
Abt Associates | pg 19
Consider Precision CarefullyCase 1/SEIS
Impact $4,509 This is a very large impact;18% of control
Standard Error $1,286 Relative to impact this is a small number
T-statistic 3.5 = Impact / Standard Error
P-value p<0.01 **** If no impact, this would happen less than 1% of the time
95% Confidence Interval $1,806 - $9,785 Range of plausible (95% of time)
impacts does not include zero
Language “Strong Evidence” When p<0.95
![Page 20: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/20.jpg)
Abt Associates | pg 20
Several Very Different CasesCase 1/SEIS Case 2 Case 3 Case 4
Impact $4,509 $4,509 $2,000 $200
Standard Error $1,286 $2,652 $1,286 $129
T-statistic 3.51 1.70 1.56 1.56P-value p<0.01 p<0.10 p>0.10 p>0.10
95% Confidence Interval
$1,806 – $9,785
-$689 –$9,707
-$521 –$4,521
-$52 –$452
Language “Strong evidence”
“Limited evidence”
“No evidence”
“No evidence”
![Page 21: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/21.jpg)
Abt Associates | pg 21
Several Very Different CasesCase 1/SEIS Case 2 Case 3 Case 4
Impact $4,509 $4,509 $2,000 $200
Standard Error $1,286 $2,652 $1,286 $129
T-statistic 3.51 1.70 1.56 1.56P-value p<0.01 p<0.10 p>0.10 p>0.10
95% Confidence Interval
$1,806 – $9,785
-$689 –$9,707
-$521 –$4,521
-$52 –$452
Language “Strong evidence”
“Limited evidence”
“No evidence”
“No evidence”
Just Can’tTell
No Impact
Very different cases; “Just Can’t Tell” is more common
![Page 22: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/22.jpg)
Abt Associates | pg 22
Outline
WIF as the “Vanguard”
Impact Results
Earlier Results
An Exercise
![Page 23: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/23.jpg)
Abt Associates | pg 23
Logic Model for Training Welders
Inputs OutputsOutcomes
Short-Term Long-Term
• Staff• Space• Materials• Partnerships
• Sessions held
• Teaching with fidelity
• Internships
• Attend sessions• Graduate• Pass external
exam• Employed in
target industry
• Earnings• Transfer
program participation
• Taxes paid
![Page 24: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/24.jpg)
Abt Associates | pg 24
So Far, Discussed Impact
Inputs OutputsOutcomes
Short-Term Long-Term
• Staff• Space• Materials• Partnerships
• Sessions held
• Teaching with fidelity
• Interships
• Attend sessions• Graduate• Pass external
exam• Employed in
target industry
• Earnings• Transfer
program participation
• Taxes paid
“Impact”: Treatment vs. Control for “Long-Term Outcomes”– Not revealed until “later”– Not useful for short-term adjustments
![Page 25: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/25.jpg)
Abt Associates | pg 25
Use Logic Model to Refine Program
Inputs OutputsOutcomes
Short-Term Long-Term
• Staff• Space• Materials• Partnerships
• Sessions held
• Teaching with fidelity
• Internships
• Attend sessions• Graduate• Pass external
exam• Employed in
target industry
• Earnings• Transfer
program participation
• Taxes paid
Many “early steps” (inputs, activities/outputs, and short-term outcomes) can be observed early on, in your treatment group, at low cost
![Page 26: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/26.jpg)
Abt Associates | pg 26
Learning from Early Results A program’s Logic Model describes necessary—but perhaps
not sufficient—early steps to achieve meaningful impact
You are currently implementing the program; you can check those early steps– Epstein and Klerman (2012) note that you will often find that the
early steps are not achieved
If your program is not achieving early steps, you can adjust the program design and implementation– Now, during this implementation (before impact results are known)– For the next implementation (in response to impact results)
![Page 27: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/27.jpg)
Abt Associates | pg 27
Some Early Steps to Check1. Secured partnerships?
Recruited and retained the right staff?2. Attracted target number of participants? 3. Do Enrolled participants complete (enough of) the
program?4. Is the program implemented with fidelity?5. Do participants show progress on pre/post measures of
the program’s short-term outcomes?6. Do participants pass external exams?7. Do participants find employment in targeted industry?
Program details matter; examples have caveats
![Page 28: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/28.jpg)
Abt Associates | pg 28
An Exercise
WIF as the “Vanguard”
Impact Results
Earlier Results
An Exercise
![Page 29: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/29.jpg)
Abt Associates | pg 29
Exercise 2: Motivation
Epstein and Klerman (2012) argue that:– Even w/o an impact evaluation, we can establish that some
(many?) programs are unlikely to show impact– Because they don’t “succeed” in the earlier steps of their
own logic model
This exercise attempts to help you to be ready to use that insight constructively: – In this grant cycle, and in future grant cycles
![Page 30: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/30.jpg)
Abt Associates | pg 30
Exercise 2: Part 1
Who: Grantee and evaluator, together
Materials: A copy of your program’s “Logic Model”– Ideally, a figure/graphic and the narrative discussing it
Part 1: Walk through your logic model, identifying each of the verifiable early steps– Inputs acquired– Activities and outputs produced– Outcomes achieved—in the treatment group, during or at
the end of treatment
![Page 31: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/31.jpg)
Abt Associates | pg 31
Exercise 2: Parts 2-5 Part 2: For each step, define “success”
– Often defining success will require a quantitative standard(e.g., number of trainees enrolled, percent of classes actually attended)
Part 3: Establish how and how early you can (and will) easily measure “success” for each earlier step
Part 4: If not “successful” at an early step of your logic model, discuss how you will “adjust” your program– During this grant period or in the next grant period
Step 5: If we identify common issues, we can try to arrange technical assistance
![Page 32: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/32.jpg)
Abt Associates | pg 32
For More on these Ideas
Wholey, Joseph 1994. ‘‘Assessing the Feasibility and Likely Usefulness of Evaluation.’’ In Handbook of Practical Program Evaluation, edited by H. P. Hatry, J. S. Wholey, & K. E. Newcomer. San Francisco, CA: Jossey-Bass.
Epstein, Diana and J.A. Klerman. 2012. “When is a Program Ready for Rigorous Impact Evaluation?” Evaluation Review. 36(5): 373-399.
Abt Associates Policy Brief: When is a Social Program Ready for Rigorous Impact Evaluation?
![Page 33: Building Support through Effective Communication Strategies](https://reader035.vdocument.in/reader035/viewer/2022070423/56816767550346895ddc4ce9/html5/thumbnails/33.jpg)
Abt Associates | pg 33
Outline
WIF as the “Vanguard”
Impact Results
Earlier Results
An Exercise