2016 cause conference farron levy practical techniques for measuring social impact

Practical Techniques for Impact Measurement

May 19, 2016

Susan Pollara 617.305.7501 spollara@trueimpact.com

Farron Levy flevy@trueimpact.com 617.903.0210

Agenda

9:20 – 10:00

Introduction & Overview

Principles of Impact Measurement: - What to measure - How to measure - When to measure - How much to claim

10:15 – 10:30 Break

10:30 – 11:00 Workshopping: Applying Principles

11:00 – 12:00 Reporting Out / Discussion

About Us – Measurement Experience

Education, Income, Health Civic Engagement

(100+ companies)

Health impacts of clinical research Safe water access

$100MM early childhood education

Envt’l Innovation Grants, Diversity, Green Teams

EarthSmart Outreach (global CO2 reduction) Global Health Fellows

College access, arts & culture, etc. Global volunteerism

US portfolio & $100MM pro bono program

Full $60MM corporate philanthropic portfolio

Domestic violence, teen safe driving, etc.

Nutrition, Military, Pub Broadcasting; Partners

$13MM Literacy Portfolio Civics, arts, environment, STEM education

Youth financial education Youth internet safety

Measurement is a Means to an End

Prove Value

→ Validate success

→ Build/retain support among internal/external stakeholders

→ If they’re not asking now – they will be soon

Improve Value

→ Identify what’s working (replicate) and what’s not (intervene)

→ Steer investment to highest social ROI interventions

1. What to measure

2. How to measure

3. When to measure

4. How much to claim

Overview

Principles of Practical Measurement

What to Measure: Can Measure ≠ Should Measure

INPUTS OUTPUTS OUTCOMES

Resources invested (time, materials, cash)

Services delivered (and to whom)

Resulting improvement in targeted social condition

(e.g., literacy, hunger, homelessness)

Manager Focus

“Investor” Focus: Social Impact

(End Outcome)

What to Measure: Core Investor Metrics

1. Social Impact. How much are we solving the social problem we care about? (End Outcome) – X inner-city adults gain living-wage employment – X low-income children attain healthy weight

2. (Contextual) Cost Per Outcome. How efficiently are we creating that social value? – Option A: $300 = 1 child gains reading proficiency – Option B: $200 = 1 child gains reading proficiency

o Types of services delivered to whom

o Resulting changes in skills, knowledge, resources, attitudes

o Resulting change in behavior

o Resulting state of being

Reach Preparation Mobilization Condition (Social Impact)

Outcomes

Interim Outcomes End Outcome

− Receive career training, mentoring

− Gain industry-specific skills; interview tactics

− Actively job hunting; secure offer

− Gain living-wage employment

− Participate in healthy eating workshop

− Gain food preparation skills; health insights

− Eating healthy − Improve health (e.g., reduce BMI)

Outcomes

2. (Contextual) Cost Per Outcome. How efficiently are we creating that social value? – Option A: $1,500 = 1 adult gains employment – Option B: $1,200 = 1 adult gains employment

Equivalent? Account for differences in cost, difficulty, priorities, etc.

2. (Contextual) Cost Per Outcome. How efficiently are we creating that social value? – Option A: $1,500 = 1 adult gains employment – Option B: $1,200 = 1 adult gains employment

How to Measure: Ideal ≠ Practical

How to Measure: Best Available Data

Reach Preparation Mobilization Condition

100 people trained

?? people learned skills

?? people changed behavior

?? people achieved success

Data Sources & Assumptions

Tracking

− Randomized controlled trial (scientific confirmation)

− Compare program participants with similar population not receiving services

− Systematic outcome tracking

Estimation

− Sampling, piloting

− Previous results

− External studies

Speculation

− Logical assumptions

100 people trained

90 people learned skills

?? people changed behavior

?? people achieved success

Tracking

Estimation

Speculation

100 people trained

90 people learned skills

68 people changed behavior

51 people achieved success

Tracking

Estimation

Speculation

90% 75% 75%

When to Measure: Later

Your Focus Measurement

When to Measure: Now

Integrate into initial planning Avoid foreseeable failures

Embed in existing systems Highest quality data / least effort

Integrate into grant applications Increase alignment, promote accountability

The first – and most crucial – measurement step: forecasting impact (i.e., who, what, how much, when)

How much to claim: Contribution vs. Attribution

Project Impact

100 adults gain employment

Portion Funded X 75%

Funder’s Claim

Funded 75 adults to gain employment

How much to claim: A 3-Tier Claim Structure

Social Impact. How much are we solving the social problem we care about? (End Outcomes)

– X inner-city adults gain living-wage employment – X low-income children attain healthy weight

Share. An allocation of a program’s impact equal to the portion funded.

Project Impact

Funder’s Claim

Yes No

Instrumental to ensuring program implementation (a “foundational” investor)?

Enabled 100 adults to gain employment

Program. The total impact resulting from the program the funder enabled.

Project Impact

Funder’s Claim

Yes No

Instrumental to ensuring program implementation (a “foundational” investor)?

Enabled 100 adults to gain employment

Yes No

Program inspired or enabled other similar programs?

Catalyzed 1,000 adults to gain employment

Catalyzed. Impacts from program enabled by a funder-enabled program.

Program. The total impact resulting from the program the funder enabled.

Program Location Cost Gain Employment (# beneficiaries)

$ per Outcome

Intervention A Locale 1 $75,000 179 $419

Intervention B Locale 4 $20,000 11 $1,818

Intervention C Locale 5 $5,000 57 $88

Grand Total $280,000 757 $370

Social Objective: Helping inner-city adults gain living-wage employment

Proving & Improving Value

$ per Outcome

Grand Total $280,000 757 $370

Proving Value

We helped 757 people gain employment

$ per Outcome

Grand Total $280,000 757 $370

Improving Value

Intervention A /Locale 2 is 2-4x as cost effective as the others. →What best practices can we learn?

Proving Value

$ per Outcome

Grand Total $280,000 757 $370

Improving Value

Intervention A /Locale 2 is 2-4x as cost effective as the others. →What best practices can we learn?

Intervention C is the most cost-effective strategy at $88-$91/person. →Worth additional investment?

Proving Value

Measurement Principles

1. What to measure Investor metrics (outcome / cost per outcome)

2. How to measure Use best available data

3. When to measure Now

4. How much to claim 3-Tier Structure (funded, enabled, catalyzed)

Measurement Principles

Workshopping: What to Measure – Core Investor Metrics

− Receive career training, mentoring

− Gain industry-specific skills; interview tactics

− Actively job hunting; secure offer

− Gain living-wage employment

− Participate in healthy eating workshop

− Gain food preparation skills; health insights

− Eating healthy − Improve health (e.g., reduce BMI)

Outcomes

Program:

Types of services delivered to whom

Resulting change in skills, knowledge, resources,

and/or attitudes

Resulting change in behavior

Resulting state of being

Success Criteria (what to measure)

How to Measure

Outcomes

Program:

Outcomes

Types of services delivered to whom

Resulting change in skills, knowledge, resources,

and/or attitudes

Resulting change in behavior

Resulting state of being

Success Criteria

How to Measure

ent • Volunteer

engagement • Education • Service projects

• Inspired and prepared to become environmental steward

• Take environmental action • Increase patronage of

parks, natural resources • Evangelize others

• Environmental improvement

• Increased enjoyment of parks, natural resources

• Education • Service projects

• Inspired and prepared to become civically engaged

• Take civic action • Evangelize others • Social improvement

• Arts Education • Arts Access

• Learn concepts (art-related or otherwise)

• Value the arts

• Arts engagement (as a consumer, patron, and/or practitioner)

• Enjoyment/fulfillment • Increased arts support • Improved education

tc.) • Competitions

• Education • Service projects • Training

• Gain skills/confidence • Awareness/appreciation for

career paths

• Pursue career paths • Apply for jobs • Interview better • Stay in school

• Academic achievement • Employment (STEM,

other)

o Funder’s claim (social impact)

o Profile Content

Individual Program Profiles

Universal Reporting Tool for Social Investments

Investment

Impact Receipt®

Grants + Volunteerism + In-kind Donation Donor

Prove Value Improve ROI Save Time

• Quantitative outcomes • Rollups • Allocates “claim” • Translates cash, in-kind

• Benchmarking • Best practices

identification

• Single report to update • Automatic reporting • Integrates with

existing systems

Grantee

Thank you!

Farron Levy flevy@trueimpact.com 617.903.0210

2016 cause conference farron levy practical techniques for measuring social impact

Marketing

levy global warming fraser salmon - david a. levy

santiago levy

general rough integration, levy rough paths and a levy

thomas levy

levy flight

mayonesa, levy

howard levy

el paso county · levy = 4.279 $91.78 water districts levy...

a notice of levy is not a levy. right to property/release of...

william levy

do enlarged fiscal deficits cause inflation: national...

10 branding insights from mark levy of levy innovation

clement levy

persianliteratur levy

academic language function€¦ · [academic] english, the...

levy levy prospect theory mgmt sci 2002

appprenticeship levy

national audit office: the climate change levy and climate...

levy intro

levy presentation