show me the outcomes - united states

Show Me the Outcomes! Evaluating and Proving Your Impact on the Community

Kathryn Engelhardt-Cronk Founder/CEO/President Community TechKnowledge, Inc. [email protected]

Why Are You Here Today?

  How am I going to continue to raise funds to keep my nonprofit running?

  How am I going to prove that our services are a success?

  How do I keep my staff focused on what works?

  How can I save money, while increasing our capacity to serve?

  How do I keep my board and other volunteers informed of our organization’s progress

  How do I engage and empower our consumers in their progress

You Will Need Technology Tools to Support Your New Outcomes/Evaluation Plan

Solution to Your “Pain”- #1

Benefits of Technology in Implementing an Outcomes Model

1.  Provide a simple, completely secure and straightforward, easy and efficient, unique data entry at the point of service


1.  Simple, completely secure and straightforward, easy and efficient, unique data entry at the point of service

2.  Provide sophisticated, visually clear and attractive, informative reports in a variety of color-coded, graphic formats



2.  Sophisticated, visually clear and attractive, informative reports in a variety of color-coded, graphic formats

3.  Allow you to provide consumers, policy makers, staff and funders with timely critical information re: your mission, progress & success



2.  Sophisticated, visually clear and attractive, informative reports in a variety of color-coded, graphic formats

3.  Consumers, policy makers, staff and funders receive critical information re: your mission, progress & success

4.  You won’t have to be Archimedes to track your data

You (or me) Tracking Outcomes Without Technology

You Reporting Outcomes WITH Technology

Permanent Senior Access Green Arts Access Increased Pre-School Decrease

Housing to Health Care Strategy for School Voter Readiness School

Education Age Children Registration Drop Out %

Develop an Outcomes/Evaluation Plan!

Solution to Your “Pain” - #2

1.  To define program/project success

At Least 5 Good Reasons …

1.  To define program/project success 2.  To understand the components and process

of success



of success 3.  To prove program/project success



of success 3.  To prove program/project success 4.  To make informed decisions about future

programming and resource allocation


1.  To define program/project success 2.  To understand the components and

process of success 3.  To prove program/project success 4.  To make informed decisions about

future programming and resource allocation

5.  To qualify for more and larger grants and for other funding


In a Perfect World …

Outcome/Evaluation models are implemented as a part of a total community-wide Impact Strategy

(Where YOU, the Service Provider, Gain Focus and Begin Planning)

Impact Area/s

Outcomes

Indicators

Milestones Target Achievement

Impact Strategy/Statement

5 Step Impact/Outcomes Framework

Step 1: What is an Impact Strategy Statement?

MANY stakeholders develop a community-wide, long-term strategy that includes the relevant universe and that results in a quality of life-based statement.

In Other Words…

Most service provider organizations DO NOT create or sustain community-wide Impact Strategies.

Sample Impact Strategy Statements

Promote community-wide wellness

Build vibrant and tolerant democracies whose governments are accountable to their citizens.

(Where YOU, the Service Provider, Gain Focus and Begin Planning)

Step 1

Impact Area/s

Outcomes

Indicators



This is where strategic thinking begins to focus on the specific needs of a community or special demographic

Tip #1. Allows Service Providers and Funders to begin to understand where their organizations and programs “fit” within the strategy and how they can make a measureable difference.

Step 2: Impact Areas

Sample Impact Areas


Impact Area: Health

Impact Area: Housing

The Open Society Institute works to build vibrant and tolerant democracies whose governments are accountable to their citizens.

Step 2

Outcomes

Indicators



Housing

…where your work begins

Step 3: Outcomes

But first, let’s create a fictitious organization that needs an Outcomes/Evaluation Plan.

The Nonprofit: “Life Skills Housing Access”

Program for which funding

is needed: Homeless Housing

Services

Mission: “To provide effective and enhanced housing options that meet the needs of all community members.”

Outcomes Definition

Outcomes are benefits or changes for individuals or populations during or after participating in program activities and/or influenced by a program's outputs.

(a.k.a. Objectives; Strategic Themes)

Outcome Attributes

 Can be measured

Outcome Attributes

 Can be measured  Are directly related to Impact Areas

Outcome Attributes

 Can be measured  Are directly related to Impact Areas  Acts to define performance goals of

Impact Strategy

Outcome Attributes

 Can be measured  Are directly related to Impact Areas  Acts to define performance goals of

Impact Strategy  Requires periodic review

Tip #2. Individual or multiple Outcome/s can be collaboratively imposed by funders OR developed by the service provider or a communicating consortium of providers.

“Permanent Housing Access for Homeless Individuals in Our Community”

Sample Outcome

Outcomes Statement

Step 3 Permanent Housing For Our Homeless

Indicators



Housing

 Specific, measure-able performance goals, consisting of individual outputs, inputs and milestones/targets.

Step 4: Indicators

What Are Indicators?, con’d.…

 Specific, measure-able performance goals, consisting of individual outputs, inputs and milestones/targets.

 These Indicators are tracked to measure how well a program is achieving an Outcome.

Indicator Attributes

 Definable, quantifiable, measure-able



 Metrics are drawn upon analysis of available national /regional information



 Metrics are drawn upon analysis of available national /regional information

 Aligns Outcomes with your services’ relevance

Two (2) Sample Indicators

Indicator #1: 80% of program Participants Will Successfully Acquire Permanent Housing

Two (2) Sample Indicators

Indicator #1: 80% of program Participants Will Successfully Acquire Permanent Housing

Indicator #2: 85% of Enrollees Will Report a Better Understanding of Available Housing Options

Permanent Housing for Our Homeless

Milestones Target

Achievements


Housing

80% of program Participants Will Successfully Acquire Permanent

Housing Step 4

Indicator/s Statement

Step 5: The Evaluation Plan

Now comes the hardest (and the most satisfying) component of Outcomes Modeling.

The Definition:

Strategies and objectives for a program's, client’s/consumer’s or group of clients’/consumers’ level of achievement and how you will prove success.

What is Performance Evaluation?

a.k.a Service or Consumer Goals; Results

Performance Evaluation / Measurement Attributes

  Definition of target population/s


  Definition of target population/s   Identification of service/program inputs, and

outputs and goals that tie to each Indicator



outputs and goals that tie to each Indicator   Expectations defined that directly correlate

with analysis of what constitutes successful performance



outputs and goals that tie to each Indicator   Expectations defined that directly correlate

with analysis of what constitutes successful performance

  Progress is monitored via defined methodologies to measure Indicator fulfillment

Permanent Housing for Our Homeless

Milestones Target

Achievements: Evaluation Plan


Housing

80% of program Participants Will Successfully Acquire Permanent

Housing Step 4

Unique Performance Milestones

  Target Population: 50 males and females, aged 18+ who are currently homeless or living in temporary quarters

Homeless Housing Services: Performance Measurement Example …


  Service Description: Counseling and one-to-one support to prepare and place homeless individuals for permanent housing access

Plan Description


  Service Description: Counseling and one-to-one support to prepare and place homeless individuals for permanent housing access

  Inputs: Three full-time staff counselors; Life Skills Housing Funds - total $20,000; resource lists of local AA and NA groups; bus vouchers for transportation to groups: total $500; furniture vouchers for each successfully placed participant: total $4,500

Plan Description

Performance Measurement Example: Outputs

  50 homeless individuals recruited


  50 homeless individuals recruited   Each applicant receives $10 in bus vouchers


  50 homeless individuals recruited   Each applicant receives $10 in bus vouchers   Each participant attends three AA or NA

meetings prior to apt application (150 meetings)




  Each participant applies for, and views, at least three available apartments (150 views)





  Each participant applies for Life Skills Housing Fund assistance of $500






  Each successful participant receives Re-Use Network voucher for furniture assistance – each participant $100






  Each successful participant receives Re-Use Network voucher for furniture assistance – each participant $100

  Each applicant receives moving assistance

Demographic/Output Data Collection

Client Demographics

Client Demographics: Age

Output Tracking Form

Sample Program Output Report

Performance Measurement Example – Program Goals

Consumer/Program Goals  40 of all 50 of program enrollees

successfully placed in permanent housing for at least 60 days

Performance Measurement Example – Program Goals

Consumer/Program Goals  40 of all 50 of program enrollees

successfully placed in permanent housing for at least 60 days

 47 of 50 enrollees self-report that they understand more about planning for and finding housing

The Completed Impact Strategy Outcomes Framework

Basic Needs/Housing

Permanent Housing Access for Homeless

Homeless Housing

Evaluation Plan


80% of Participants Will Successfully Acquire Permanent Housing

Consumer Program Goals

60 Day Follow Up--by Program

Evaluation Methodology:  Enrollees self-report AA/NA Attendance

Methodology Examples

Evaluation Methodology:  Enrollees self-report AA/NA Attendance  Counselors document Life Skills Housing Fund

application



application  Counselors document 3 apartment applications

and viewing




and viewing  Counselors document apartment acceptance

and furniture voucher is provided





and furniture voucher is provided  Move-in date and event are documented by

counselor






counselor  Counselor schedules 60-day follow-up visit to

assess housing permanence






counselor  Counselor schedules 60-day follow-up visit to

assess housing permanence  Counselor contacts both unsuccessful enrollees

and successful enrollees to request self-report on knowledge gained and assesses interest in renewed attempt


Consumer Measurements

Attendance Measurement

Program Measurements

Individual Milestone Tracking to Achieve Indicator/Outcome

Individual Breaching Milestones

Unmet

Aggregated Milestone Tracking to Achieve Program Indicator/Outcome

show me the outcomes - united states

Self Improvement