2016 AmeriCorps State:Grant Development

October 15-30, 2015

Marisa Petreccia, Kate Pisano & Nancy Stetter

Session Overview

• Planning & Preparation• 2016 Requirements• Narratives• Theory of Change• Evidence• Performance Measures• Resources

2

Planning & Preparation• Take the Self-Paced Tutorials through National

Service Knowledge Network– Performance Measure Module– Theory of Change– Evidence– High Quality Performance Measures

• Review 2016 SRI RFP and materials• Review NOFO and Grant Application

Instructions• Build in time for academic research

3

2016 RequirementsNew for 2016:– Phase 1 eGrants submission – 20 member slot minimum, not MSYs• Can be a combination of slot types to reach the

minimum

– SRI’s TA structure• After November 16, SRI staff will be in quality review

mode

– SRI Board Presentations• December 15th

4

2016 Requirements (cont.)• Evaluation Plan requirements & FAQs

– http://www.nationalservice.gov/resources/evaluation/cncs-evaluation-policies

5

If you are competing for: You are required to:

First AC grant Not required to submit an evaluation plan with the application or complete an evaluation during the initial grant period. You are encouraged to begin the evaluation planning process.

Second AC grant Required to submit an evaluation plan with the application and complete the evaluation during the next three-year grant period.

Third AC grant Required to submit a report on the evaluation conducted during the previous grant period with the application, and submit an evaluation plan for the next grant period with the application.

AC grant beyond third award Required to submit an evaluation plan for the next grant period, along with a report that details findings from evaluations conducted in previous grant periods with the application. Evaluation plans should build on previous evaluation efforts.

Formula grantee applying for competitive funds

Required to submit an evaluation plan for the next grant period

Narratives

Narrative Sections• Get right to the point! Limited space to write– Specific need and specific AC intervention– Avoid extraneous socio-economic issues

• Must respond to each point in RFP– SRI review sheets will follow RFP sections/fields

• Need to explain dosage (frequency, intensity and duration) of AC interventions

• AC resource vs. staff (or existing programming)• Quantify Logic Model outcomes

7

Narrative Sections (cont’d)• Narrative Page Limits– 15 pages PDF printed out – 18 pages for Multi-focus intermediaries, School

Turnaround, and Next Gen – Within page limits, include: Facesheet, Executive

Summary, Program Design, Org Capacity, Cost Effectiveness and Budget Adequacy

• Logic Model Page Limits– No more than 3 pages printed

• Print out in eGrants BEFORE submitting to SRI– Select “View/print”

8

Theory of Change & Logic Model

Definitions & Purpose• Theory of Change– Conceptual understanding of how you believe your

program will create change• Logic Model– The detailed representation of your program and

theory of change• Purpose– Internal planning process, external development and

engagement, grant application components

10

Theory of Change: Elements

11

Community Need:

A negative condition exists

Specific Intervention:

If AC members do this…

Intended Outcome:…Then this will

happen

11

EVIDENCE: needed to support Intervention and Outcome

STATISTICS: needed to support Community Need

Theory of Change (cont.)

12

Community Need

Specific Intervention

Intended Outcome

1 23

• When developing your TOC be able to answer:• Does the outcome address the need?• Next check to affirm that the selected intervention can

be expected to result in the intended outcome

Documenting Need• Data documenting need/problem should answer

the following questions:– SCOPE: Who and how many are directly affected? How

severe is this?– SIGNIFICANCE: What makes this a compelling need? Is

it likely to become worse? What will happen if we do nothing?

– CAUSE(S): Why does the need exist? How is it perpetuated?

• Goal: to find and use data as close to the source as possible

13

Intervention: Design & DosageDescribe the design and dosage of your intervention (service activity):• Design (who does what with whom?)• Dosage– Frequency (how many sessions a week?)– Intensity (length of each session)– Duration (how many total weeks of sessions?)

14

Testing your TOCBe able to ask if your Theory of Change is:• Plausible: Does the logic of the model seem

correct: “if we do these things, will we get the results we expect?”

• Feasible: Are resources sufficient to implement the chosen intervention?

• Meaningful: Are intended outcomes important? Is the magnitude of expected change worth the effort?

15

Logic Model Components

16

Inputs Activities Outputs Outcomes

Resources available for

program activities

Tools, actions, events,

processes of program

Direct product of program

activities, often quantified

Changes in population

served due to program

Evidence

Evidence: Sources

18

Evidence Continuum

19

• Preliminary– Outcome results from performance measurement – Does not show causality

• Moderate– Results from studies/evaluations using “quasi-experimental

design,” i.e. comparison groups– Shows causality

• Strong– Results from studies/evaluations using “experimental

design,” i.e. control groups– Strongest way to show causality

Evidence Continuum: Example

20

Preliminary Moderate Strong

Performance measurement shows that 75% of girls age 14-16 participating in the Shape Up program made more healthy food choices.

A 2005 impact evaluation by internal evaluators (using a quasi-experimental design of girls participating in a similar program to Shape Up) found that after 12 weeks, the girls in the program made 50% more healthy food choices than the comparison group.

A 2010 impact evaluation of the GEMS program by University of MN using experimental design/random assignment found after 12 weeks, the girls in the experimental group made 50% more healthy food choices than control group girls

Assessing Evidence

21

Considerations:• Similar: Cites comparable intervention with similar

beneficiaries and results• Significant: Findings show that the program had a positive

and statistically significant effect on beneficiaries• Up-to-date: Recently published or most recent available• High Quality: Use well-implemented and appropriate

research methodologies given the research questions of interest

• Reputable: Source with no stake in outcome and published in a peer reviewed journal or by credible organization

Evidence Resources

22

• Evidence Exchange:http://www.nationalservice.gov/impact-our-nation/research-and-reports/evidence-exchange

• Google Scholar: https://scholar.google.com/ • Evidence Checklist (online Knowledge

Network)

High Quality Performance Measures

Performance Measures• Required by CNCS (FAQ, E.1.)– AmeriCorps programs must have an aligned

output and outcome that reflects the program‘s primary service activity and aligns with the program’s theory of change.

• How many PMs? (FAQ, E. 14)– You can have additional PMs for other significant

components of your program. PMs reflecting non-primary activities should not be included in your application and not reported to CNCS.

24

Performance Measures (cont.)

• Just as with your Theory of Change, there needs to be alignment among outputs and outcomes

• Set smart goals and realistic targets– Outcomes should be:• Meaningful• Ambitious yet realistic• Reflect the type of change (attitude, knowledge,

behavior or condition) you want to measure

25

Performance Measures (cont.)

• Program Time Frame– Organizational resources to collect and analyze

data– Collecting relevant data that ties back to Theory of

Change– Within the annual grant period

• Scope of AmeriCorps intervention– Being able to measure those who are directly

served by the AmeriCorps program

26

Performance Measures (cont.)• Alignment– Intervention produces output– Output leads to the outcome– Output and outcome measure the same intervention and

beneficiaries

27

Intervention Output Outcome

75 100

PM Resources• Performance Measure Instructions– Detailed instructions for each individual PM, definitions,

and descriptions of data collection methods

• Core Curriculum– http://www.nationalservice.gov/resources/performance-

measurement/training-resources#overview

28

Resources

Resources• 2016 AmeriCorps State Grant page• National Service Knowledge Network – http://www.nationalservice.gov/resources– Performance Measure Core Curriculum

• AC Performance Measures Instruction Packet (including specific info on each PM)

• SRI Office: 331-2298 – Marisa or Kate for programmatic– Nancy for fiscal

30