building the evidence base of americorps programs:return
TRANSCRIPT
![Page 1: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/1.jpg)
AmeriCorps Off ice of Research and Evaluat ion
December 2020
Building the Evidence Base of AmeriCorps Programs:Return on Investment Studies
![Page 2: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/2.jpg)
WelcomeReturn on Investment Studies
2
Dr. Lily ZandniapourResearch and Evaluation Manager
Office of Research and Evaluation, AmeriCorps
![Page 3: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/3.jpg)
Coordinates with AmeriCorps Programs to:
• Inform the development and implementation of programs and initiatives with evidence;
• Measure the performance and results of programs and initiatives through research and evaluation;
• Build knowledge about the effectiveness of national service and volunteering as solutions to community needs using scientific methods;
• Encourage a culture of learning and continuous improvement within the agency and within the nonprofit organizations AmeriCorps funds; and,
• Shape key agency policy and program decisions using credible data.
What does the Office of Research and Evaluation (ORE) do?
3
![Page 4: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/4.jpg)
Examples of Questions:
• Are there short-term or intermediate term outcomes from the programs or program models that can be measured and attributed to the program?
• Can the outcomes be monetized to calculate a financial return?• Can the costs of the program per participant or otherwise be
estimated?• Can we estimate the Return on Investment (ROI) for the programs we
implement or support?
Interest in Return on Investment (ROI)
4
![Page 5: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/5.jpg)
• ORE partnered with ICF to conduct a series of ROI Analyses
• ICF conducts feasibility studies and produces ROI studies
• They will present findings on four studies conducted in 2019-2020
• ORE intends to commission more ROI studies in the coming years and support the capacity of our programs and partners to increase their readiness for ROI analyses
Return on Investment Analyses
5
![Page 6: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/6.jpg)
• Sign up for ORE newsletter!
• Email us at [email protected]!
• Check out Evidence Exchange (AmeriCorps Evidence Library) on our website for ROI Reports!
• Participate in our future webinars and Research Summits!
How do I stay informed?
6
![Page 7: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/7.jpg)
Dr. Janet PershingVice President
Housing & Community Development, ICF
SpeakerReturn on Investment Studies
7
![Page 8: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/8.jpg)
• Define return on investment (ROI) and review ROI research on national service programs
• Review an ROI case study: Community Technology Empowerment Program
• Consider how to build the foundation for an ROI study• Consider how to find ROI-ready programs
Agenda
![Page 9: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/9.jpg)
Defining Return on Investment (ROI) and Reviewing ROI Research on National Service Programs
![Page 10: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/10.jpg)
Total BenefitsTotal Costs
ROI=
About Return on Investment
![Page 11: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/11.jpg)
• Columbia University (2013): The Economic Value of National Service (Voices for National Service & The Aspen Institute)
• ICF (2020): AmeriCorps and Senior Corps, Quantifying the Impacts (Voices for National Service)
• AmeriCorps ROI Studies Initiative
ROI Research on National Service
![Page 12: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/12.jpg)
2020 ROI Studies – Selected Programs
![Page 13: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/13.jpg)
The four AmeriCorps ROI Studies Initiative studies produced to date:• Reflect diversity within AmeriCorps’ program portfolio.• Use conservative methodologies.• Use short-term, medium-term, and long-term scenarios
to estimate the duration of program effects.• Show positive ROI results across all scenarios.
General Results
![Page 14: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/14.jpg)
Case Study: Community Technology Empowerment Program
![Page 15: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/15.jpg)
• Implemented by the St. Paul Neighborhood Network (SPNN)
• Addresses the digital divide in the Minneapolis–Saint Paul metropolitan area
• AmeriCorps members:• Teach informal classes.• Provide one-on-one mentoring and training in computer and
technology labs.• Certify participants in technology skills.
About Community Technology Empowerment Project (CTEP)
![Page 16: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/16.jpg)
• 10.3 percent of participants who were unemployed before starting the CTEP program indicated that CTEP helped them become employed.
• $30,763 was the average annualized earnings for CTEP program participants who were unemployed pre-program and employed post-program.
CTEP Evaluation Results
Source: Backman, D. (2018). 2018 Evaluation Report: Impact of the Northstar Assessment & Related Computer Skills Programming on Employment in CTEP Programs. Retrieved from https://www.nationalservice.gov/sites/default/files/evidenceexchange/SaintPaul_19AC208840_Report_Revised_508%20%281%29.pdf
![Page 17: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/17.jpg)
CTEP Beneficiaries
• Increased earningsCTEP Program Participants
• Stipends, living allowances, and education awards• Increased earnings post-serviceAmeriCorps Members
• Tax revenue from increased earnings• Reduced spending on corrections, public assistance,
social insurance.
Federal, State, and Local Government
![Page 18: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/18.jpg)
CTEP ROI Results
ROI CalculationROI Scenario
Short-Term Medium-Term Long-Term
Total Benefits per Federal Dollar $4.58 $33.35 $53.20
Total Benefits per Funder Dollar $2.95 $21.48 $34.26
Federal Government Benefits per Federal Dollar $1.30 $7.39 $11.60
![Page 19: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/19.jpg)
CTEP ROI Results – Medium-Term Scenario
ROI Calculation Medium-Term ROI Estimate Results Statement
Total Benefits per Federal Dollar $33.35The ROI analysis estimated that every dollar of federal government investment generates $33.35 in benefits.
Total Benefits per Funder DollarFor AmeriCorps-supported programs, funders can include:• Federal, state, and local government agencies• Foundations and other philanthropies • Private donors• Community organizations
$21.48 The ROI analysis estimated that every dollar of funder investment generates $21.48 in benefits.
Federal Government Benefits per Federal Dollar $7.39
The ROI analysis estimated that every dollar of investment by the federal government generates $7.39 in benefits to the federal government alone.
![Page 20: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/20.jpg)
Building a Foundation for an ROI Study
![Page 21: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/21.jpg)
• Step 1: Use a Logic Model.• Identify outcome metrics.• Identify costs and cost metrics.• Identify benefits.
• Step 2: Do a feasibility study.• Step 3: Compile and collect
data.• Step 4: Estimate and monetize
benefits and costs.• Step 5: Calculate ROI.
Foundational Steps
![Page 22: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/22.jpg)
Step 1: Use a Logic Model (CTEP Example)
![Page 23: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/23.jpg)
• Includes an analysis plan• Includes a list of required data and data collection
plan• Serves as a check on feasibility before making
investments
Step 2: Do a Feasibility Study
![Page 24: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/24.jpg)
• May include data already in hand from operations and from past evaluations
• May include data that needs to be collected• May include public datasets
Policies about participant privacy still apply!
Step 3: Compile and Collect Data
![Page 25: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/25.jpg)
• Leverage rigorous research that connects data to outcomes and estimates the value of outcomes.
• Adjust dollar values for different benefits for a common year.
Step 4: Estimate and Monetize Benefits and Costs
![Page 26: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/26.jpg)
• An “All In” ROI calculation includes all captured benefits and all captured costs.
• Also explore multiple ROI calculations that are:• Restricted to a particular group of stakeholders.• Restricted to a particular funding source.
• Sample ROI calculation:
Step 5: Calculate ROI
Total Benefits per Federal Dollar
Benefits to Participants + Benefits to AmeriCorps Members + Benefits to the Government______________________________________________________________________________________
Federal Funding
=
![Page 27: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/27.jpg)
Finding ROI-Ready Programs
![Page 28: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/28.jpg)
• Help build evidence of the impact of national service by:
• Encouraging grantees to conduct rigorous evaluations that measure near-, medium-, and long-term outcomes and to capture program costs.
• Suggesting programs that may be ROI-ready to the AmeriCorps Office of Research and Evaluation.
Finding ROI-Ready Programs
![Page 29: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/29.jpg)
• For more information or to suggest a program for an ROI study, contact Lily Zandniapour ([email protected]), AmeriCorps Office of Research and Evaluation.
Questions? Discussion?
![Page 30: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/30.jpg)
Dr. Janet PershingVice President
Housing & Community Development, ICF
SpeakerReturn on Investment Studies
30
![Page 31: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/31.jpg)
Closing RemarksReturn on Investment Studies
31
Dr. Lily ZandniapourResearch and Evaluation Manager
Office of Research and Evaluation, AmeriCorps
![Page 32: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/32.jpg)
• Sign up for ORE newsletter!
• Email us at [email protected]!
• Check out Evidence Exchange (AmeriCorps Evidence Library) on our website for ROI Reports!
• Participate in our future webinars and Research Summits!
How do I stay informed?
32
![Page 33: Building the Evidence Base of AmeriCorps Programs:Return](https://reader034.vdocument.in/reader034/viewer/2022050420/62711723072cfd5be73b6962/html5/thumbnails/33.jpg)
Closing RemarksReturn on Investment Studies
33
Dr. Lily ZandniapourResearch and Evaluation Manager
Office of Research and Evaluation, AmeriCorps