the outcomes toolbox presentation by bob penna

The Outcomes Toolbox: What You Need to Know to

Use Outcomes Robert M. Penna, Ph.D.

Author: The Nonprofit Outcomes Toolbox MNA Conference 2012 September 19, 2012 Helena, Montana

2

Today’s topics: An introduction to Outcomes:

What is an “outcome”? How is an “outcomes approach” different? What is the language of outcomes? What are the characteristics of a good outcome? Crafting an outcomes statement

An introduction to “outcomes tools”;; the view from 30,000 feet Tools for planning Tools for tracking and management Tools for reporting Tools for learning and assessment

Using some tools Building your “Outcomes Tree” Understanding and Assessing Capacity Reporting

3

Our targeted outcome:

That within one week you will do something different at work because of something you learned today

4

A quick bit of history……

5

So, think of it this way….

Outcomes: the 3rd Stage of Management

• Stage 1—Agriculture and Early Industrial Economy: Management of Workers

• Stage 2—1920–1970: Management ofWork

• Stage 3—The Age of Outcomes: Management of the Product of Work. . . The Results

6

An introduction to “Outcomes”

What is an “outcome”?

7

What makes an “outcomes approach” different?

8

The Problem Approach

9

The Activity Approach

10

The Process Approach…

11

The Vision Approach…

12

The Outcomes Approach

13

Compare this……

14

To this…. What’s different??

15

Even better…..

16

Three Changes in Thinking

1. From Funder to Investor 2. From Activity to Effectiveness 3. From Services to Change

The BACKS Measures

17

The “Language” of Outcomes

نتائج

Processus השפעת

Ingresso

Επιπτώσεις

Väljund

18

The Characteristics of a Good Outcome Are some outcomes “better” than others? What does a “good” outcome look like? Positive Improvement Meaningful Sustainable Bound in Time Bound in Number Narrowly Focused and “Doable” (with a stretch!) Measurable

Clearly Definable Discernable…can be “evinced” Quantifiable…according to an agreed-upon scale!

Verifiable

19

The “Outcomes Statement”

What is it? Why Use it?

20

Practice

Your turn……..

21

Take a break!

22

An Introduction to “Outcomes Tools”: a view from 30,000 feet….. What are “outcome tools”? 4 sets:

Tools for Planning Tools for Tracking & Managing Tools for Reporting Tools for Learning & Assessment

23

Tools for Planning

You Get What You Measure A Common Outcomes Framework

24

Tools for Tracking and Managing

The Logic Model RBA TOP GTO TRI Outcomes Management Results Mapping The Balanced Scorecard

25

Snapshot: The Logic Model

32

26

A Detailed Logic Model

33

27

But it gets tricky when….

28

Snapshot: Results-Based Accountability

42

29

Snapshot: TOP

Targeting Outcomes of Programs (TOP) Model

KASA:K = KnowledgeA = AttitudeS = SkillA = Aspiration

Program Development(Goals/Objectives)

SEEC:S = SocialE = EconomicE = EnvironmentalC = Conditions

SEECPractices

KASAReactions

ParticipationActivities

Resources

Program Performance(Evaluation Measures)

ResourcesActivities

ParticipationReactions

KASAPractices

SEEC

Source: Claude Bennett & Kay Rockwell, 1994

30

Snapshot: GTO The Getting To OutcomesTM

Process

Choose which problem(s) to

focus on.

Choose which problem(s) to

focus on.

1

Identify goals,target population,

and desired outcomes.

Identify goals,target population,

and desired outcomes.

2

Find existingprograms andbest practices

worth copying.

Find existingprograms andbest practices

worth copying.

3Modify the program

or best practicesto fit your needs.

Modify the programor best practicesto fit your needs.

4

Assess capacity(staff, financing,

etc.) to implementthe program.

Assess capacity(staff, financing,

etc.) to implementthe program.

5

Make a planfor getting started: who, what, when, where, and how.

Make a planfor getting started: who, what, when, where, and how.

6

Steps 1-6PLANNING

DELIVERING PROGRAMS

Evaluate planning and

implementation.How did it go?

Evaluate planning and

implementation.How did it go?

7

Evaluate program’s success

in achievingdesired results.

Evaluate program’s success

in achievingdesired results.

8Make a plan for

ContinuousQuality

Improvement.

Make a plan forContinuous

QualityImprovement.

9

Consider how to keep the program

going if it issuccessful.

Consider how to keep the program

going if it issuccessful.

10Steps 7-10

EVALUATING AND IMPROVING

Choose which problem(s) to

focus on.

Choose which problem(s) to

focus on.

1Choose which problem(s) to

focus on.

Choose which problem(s) to

focus on.

1

Identify goals,target population,

and desired outcomes.

Identify goals,target population,

and desired outcomes.

2Identify goals,

target population,and desired outcomes.

Identify goals,target population,

and desired outcomes.

2

Find existingprograms andbest practices

worth copying.

Find existingprograms andbest practices

worth copying.

3Find existingprograms andbest practices

worth copying.

Find existingprograms andbest practices

worth copying.

3Modify the program

or best practicesto fit your needs.

Modify the programor best practicesto fit your needs.

4

Modify the programor best practicesto fit your needs.

Modify the programor best practicesto fit your needs.

4

Assess capacity(staff, financing,

etc.) to implementthe program.

Assess capacity(staff, financing,

etc.) to implementthe program.

5Assess capacity

(staff, financing,etc.) to implement

the program.

Assess capacity(staff, financing,

etc.) to implementthe program.

5

Make a planfor getting started: who, what, when, where, and how.

Make a planfor getting started: who, what, when, where, and how.

6Make a plan

for getting started: who, what, when, where, and how.

Make a planfor getting started: who, what, when, where, and how.

6

Steps 1-6PLANNING

DELIVERING PROGRAMS

Evaluate planning and

implementation.How did it go?

Evaluate planning and

implementation.How did it go?

7Evaluate

planning and implementation.How did it go?

Evaluate planning and

implementation.How did it go?

7

Evaluate program’s success

in achievingdesired results.

Evaluate program’s success

in achievingdesired results.

8Evaluate

program’s successin achieving

desired results.

Evaluate program’s success

in achievingdesired results.

8Make a plan for

ContinuousQuality

Improvement.

Make a plan forContinuous

QualityImprovement.

9Make a plan for

ContinuousQuality

Improvement.

Make a plan forContinuous

QualityImprovement.

9

Consider how to keep the program

going if it issuccessful.

Consider how to keep the program

going if it issuccessful.

10Consider how to keep the program

going if it issuccessful.

Consider how to keep the program

going if it issuccessful.

10Steps 7-10

EVALUATING AND IMPROVING

31

Snapshot: TRI/Outcomes Management

32

Snapshot: Results Mapping

MLS7

MLS6

ACT5 > LEV5

MLS4

ACT3 > LEV3

ACT2 > LEV2

ACT1 > LEV1 38

33

Snapshot: A Balanced Scorecard

34

Tools for Reporting

Scales & Ladders The STORY format

35

Snapshot: Scales & Ladders

36

Telling your story

First off….begin by thinking of it as a story….

37

A Quick Test: These

38

As opposed to these……

050

100

1stQtr

3rdQtr

EastWestNorth

HQ

Boston Seattle Dover

A 82 45 54 62

B 37 97 17 12

C 10 83 7 7

39

Tools for Learning and Assessment

1. Lessons Learned 2. Six Sigma 3. SERVQUAL 4. Value Engineering 5. FMEA 6. TRIZ 7. CAT 8. GTO

40

LUNCH!!!

41

Building Your Outcomes Tree

1. Outcome 2. Performance Targets 3. Milestones

42

You try it….

43

Take a break!

44

Understanding and Assessing Capacity

What is “capacity”? Three levels

Structural Functional Implementation

Two Tools CAT GTO

45

CAT Essentially a self-scoring grid/matrix format…can be created in Excel

ASPIRATIONS

Mission1 Clear need for increased capacity

2. Basic level of capacity in place

3. Moderate level of capacity in place

4. high level of capacity in place

Some expression of organization’s reason for existence that reflects its values and purpose, but may lack clarity; held by only a few; lacks broad agreement or rarely referred to.

Clear expression of organization’s reason for existence which reflects its values and purpose; held by many within organization and often referred to.

Clear expression of organization’s reason for existence which describes an enduring reality that reflects its values and purpose; broadly held within organization and frequently referred to.

VPP/SVP Capacity Tool

No written mission, or only a limited expression of the organization’s reason for existence; lacks clarity or specificity; either held by very few in organization or rarely referred to

46

Snapshot: GTO The Getting To OutcomesTM

Process

Choose which problem(s) to

focus on.

Choose which problem(s) to

focus on.

1

Identify goals,target population,

and desired outcomes.

Identify goals,target population,

and desired outcomes.

2

Find existingprograms andbest practices

worth copying.

Find existingprograms andbest practices

worth copying.

3Modify the program

or best practicesto fit your needs.

Modify the programor best practicesto fit your needs.

4

Assess capacity(staff, financing,

etc.) to implementthe program.

Assess capacity(staff, financing,

etc.) to implementthe program.

5

Make a planfor getting started: who, what, when, where, and how.

Make a planfor getting started: who, what, when, where, and how.

6

Steps 1-6PLANNING

DELIVERING PROGRAMS

Evaluate planning and

implementation.How did it go?

Evaluate planning and

implementation.How did it go?

7

Evaluate program’s success

in achievingdesired results.

Evaluate program’s success

in achievingdesired results.

8Make a plan for

ContinuousQuality

Improvement.

Make a plan forContinuous

QualityImprovement.

9

Consider how to keep the program

going if it issuccessful.

Consider how to keep the program

going if it issuccessful.

10Steps 7-10

EVALUATING AND IMPROVING

Choose which problem(s) to

focus on.

Choose which problem(s) to

focus on.

1Choose which problem(s) to

focus on.

Choose which problem(s) to

focus on.

1

Identify goals,target population,

and desired outcomes.

Identify goals,target population,

and desired outcomes.

2Identify goals,

target population,and desired outcomes.

Identify goals,target population,

and desired outcomes.

2

Find existingprograms andbest practices

worth copying.

Find existingprograms andbest practices

worth copying.

3Find existingprograms andbest practices

worth copying.

Find existingprograms andbest practices

worth copying.

3Modify the program

or best practicesto fit your needs.

Modify the programor best practicesto fit your needs.

4

Modify the programor best practicesto fit your needs.

Modify the programor best practicesto fit your needs.

4

Assess capacity(staff, financing,

etc.) to implementthe program.

Assess capacity(staff, financing,

etc.) to implementthe program.

5Assess capacity

(staff, financing,etc.) to implement

the program.

Assess capacity(staff, financing,

etc.) to implementthe program.

5

Make a planfor getting started: who, what, when, where, and how.

Make a planfor getting started: who, what, when, where, and how.

6Make a plan

for getting started: who, what, when, where, and how.

Make a planfor getting started: who, what, when, where, and how.

6

Steps 1-6PLANNING

DELIVERING PROGRAMS

Evaluate planning and

implementation.How did it go?

Evaluate planning and

implementation.How did it go?

7Evaluate

planning and implementation.How did it go?

Evaluate planning and

implementation.How did it go?

7

Evaluate program’s success

in achievingdesired results.

Evaluate program’s success

in achievingdesired results.

8Evaluate

program’s successin achieving

desired results.

Evaluate program’s success

in achievingdesired results.

8Make a plan for

ContinuousQuality

Improvement.

Make a plan forContinuous

QualityImprovement.

9Make a plan for

ContinuousQuality

Improvement.

Make a plan forContinuous

QualityImprovement.

9

Consider how to keep the program

going if it issuccessful.

Consider how to keep the program

going if it issuccessful.

10Consider how to keep the program

going if it issuccessful.

Consider how to keep the program

going if it issuccessful.

10Steps 7-10

EVALUATING AND IMPROVING

47

GTO capacity question

CAPACITY TOOL: Program specific staff capacities (full version) Capacity assessment item Requirements Is current capacity

sufficient? (Y/N) Plan to enhance the capacity

Staff Training needed

Staffing level required

Staff qualifications (e.g. minimum degree, certification or training needed; years of experience)

48

Capacity grid for Recruiting potential trainees Milestone Challenge Unready Marginally Ready Fairly Ready Fully Ready

Identifying and contacting 125 unemployed adults in the community who will be potential candidates for job training and placement program

Not sure how we'll do it

Some idea of how we'll do it, but few details worked out

Pretty good idea of how we'll do it; only a couple of details still need to be resolved

Firm idea of how we'll do it; details all worked out

1 Communicating with potential candidates; fielding and answering their questions

Not sure how we'll do it

Some idea of how we'll do it, but few details worked out

Pretty good idea of how we'll do it; only a couple of details still need to be resolved

Firm idea of how we'll do it; details all worked out

2 Holding orientation session for 100 potential candidates

Not sure where or how we'll do it

Some idea of where and how we'll do it, but few details worked out

Pretty good idea of how we'll do it; only a couple of details still need to be resolved

Firm idea of where and how we'll do it; details all worked out

3 Processing enrollment of 75 potential candidates

Not sure where or how we'll do it

Some idea of where and how we'll do it, but few details worked out

Pretty good idea of where & how we'll do it; only a couple of details still need to be resolved

Firm idea of where and how we'll do it; details all worked out

7 Identifying prospective local employers who might be willing to hire program graduates

Not sure how we'll do it

Some idea of how we'll do it, but few details worked out

Pretty good idea of how we'll do it; only a couple of details still need to be resolved

Firm idea of how we'll do it; details all worked out

49

Reporting: Using Scales & Ladders

Next Steps: Action Planning Come up with a specific action step to be started next week

63

51

Wrap-up, feedback, and Questions

52

Thank you for your time and

participation…..

Contact: Dr. Robert Penna PO Box 38081 Stuyvesant Station Albany, NY 12203-8081 518-438-7502 drbob@outcomestoolbox.com Website: www.outcomestoolbox.com