logic model pp presentation_final_for sharing

Introduction to Logic Models

for Program, Project, and

Evaluation Design

Session Overview

O Introduction to logic modeling

O Purposes of logic modeling

O Different types of models

O Logic Model, Theory of Change,

Theory of Action

O Developing a logic model

Systematic Inquiry Cycle

Planning and Design

Protecting Participant

Rights

Developing/ Modifying

Instruments

Data Collection

and Processing

Data Analysis

and Reporting

What is a Logic Model?

A program logic model is a picture of how your

program works – the theory and assumptions

underlying the program. ...This model provides

a road map of your program, highlighting how

it is expected to work, what activities need to

come before others, and how desired

outcomes are achieved (p. 35).

W.K. Kellogg Foundation Evaluation

Handbook (1998)

History of Logic Modeling

O 35+ year history

O Clear identification of goals (outcomes)

O First attempt to standardize a depiction of

program components so that activities align

with outcomes

O United Way provided the first widely-

accepted model

Key Features of Logic

Models O Systematic

O Visual

O Sequential but not necessarily linear

O Typically have 5 components

O Shows “What” and “How”

O May show “Why”

Basic Logic Model

W.K. Kellogg Foundation (January 2004)

Basic Logic Model

Components PLANNED WORK

O Resources: human, financial, organizational,

community resources a program has

available to do the work (i.e., Inputs)

O Activities: processes, tools, events,

technology, and actions that are an

intentional part of program implementation.

Basic Logic Model

Components INTENDED RESULTS

O Outputs: direct products of program activities

O Outcomes: specific changes in program

participants’ behavior, knowledge, skills, status

Short-term 1 to 3 years

Longer-term 4 to 6 years

O Impact: fundamental intended or unintended

change as a result of program (7 to 10 years)

Logic Model Logic

•.

•.

Certain resources are needed to

operate your program

If . . . Then . . .

you have access to them, you can use them to accomplish your

planned activities.

you accomplish your planned

activities,

you will deliver the amount of

product/service that you intended.

you accomplish your planned

activities to the extent intended,

your participants will benefit in specific

ways.

these benefits are achieved, certain changes might occur under

specified conditions.

Why Use Logic Modeling?

1. Planning and Design

2. Implementation and

Management

3. Evaluation and Communication


Planning and Design


Rights


Instruments

Data Collection

and Processing

Data Analysis

and Reporting

Determine purpose(s),

resources, activities, and

anticipated outcomes

Using Logic Models for

Planning and Design

Logic Model

Outcomes and

Impact

Resources and

Activities

Purpose and

Questions

Planning and Design O Provides a road map

O Describes sequence of events

O Connects the need for the program with

desired results

Most of the value in a logic model is in the

process of creating, validating, and modifying

the model …The clarity of thinking that occurs

from building the model is critical to the

overall success of the program (p. 43).



Planning and Design


Rights


Instruments

Data Collection

and Processing

Data Analysis

and Reporting

Manage, monitor and modify

activities to improve progress

and final outcomes

Implementation and

Management O Helps visualize and understand how investments

contribute to achieving intended program goals

O Monitors incremental progress toward outcomes

O Facilitates program improvements

Program logic modeling is a learning and

management tool that can be used throughout a

program’s life . . . result(ing) in effective

programming and offering greater learning

opportunities (p. 1)



Planning and Design


Rights


Instruments

Data Collection

and Processing

Data Analysis

and Reporting

Document outcomes and

share knowledge with

others

Evaluation and

Communication O Documentation of outcomes

O Shared knowledge about what works and why

O Improves likelihood of sustaining a program over

time through continuation of funding and/or

stakeholder buy-in.

A program is a theory and an evaluation is its test. .

. to organize the evaluation to provide a responsible

test, the evaluator needs to understand the

theoretical premises on which the program is

based (p. 55).

Carol Weiss (1998)

How Logic Models Better Position

Programs For Success

Program Elements

Criteria for Program Success

Benefits of Program Logic Models

Planning and Design Program goals and objectives, and

important side effects are well defined

ahead of time.

Finds “gaps” in the theory or logic of

a program and work to resolve them.

Program goals and objectives are

both plausible and possible. Builds a shared understanding of

what the program is all about and

how the parts work together.

Program Implementation

and Management Relevant, credible, and useful

performance data can be obtained. Focuses attention of management on

the most important connections

between action and results.

Evaluation, Communication,

and Marketing

The intended users of the evaluation

results have agreed on how they will

use the information.

Provides a way to involve and engage

stakeholders in the design, processes,

and use of evaluation.

W.K. Kellogg Foundation (January 2004) p.6

Sidebar 1 O There is no “Best” Logic Model - Choose the

model that fits your program best and provides

information you need in the format that is

most helpful.

O Practice using logic models - Experiment with

logic model design to determine what works

best for your program.

O Don’t worry if your model doesn’t look like

others but DO worry if you cannot explain your

model to others

More Key Features of Logic

Models

OFlexible

OShow areas of strength,

weakness, and gaps

ODynamic

OPromotes strategic

management

Forms of Logic Modeling

O Typically logic models use table and flow

chart formats with text and arrows or a

graphic representation of program ideas.

O A logic model does not have to be linear.

Concept maps or other visuals may

better describe complex program

concepts.

Three Approaches to Logic

Modeling O Theory approach model (conceptual)

O Outcome approach model

O Activities approach model (applied)

O A program may use all three types of

logic models for different purposes.

Theory Approach Models

O Emphasize the theory of change

influencing design and plan for the

program

O Explain how and why your program will

work

O May have additional parts (e.g., describe

problem or issue addressed by program,

connect strategies to activities)

Theory Model


Theory and Practice

Why use a Theory-based

Approach? O Demonstrates that program

implementers have deep understanding

of program aspects

O Allows consideration of context and

other mutable variables

O Funders like, and often, require them

O Facilitates program improvement and

evaluation

Keeping it Simple

O Theories of Change (ToC) and Theories

of Action (ToA) are simply extensions of

Logic Modeling

O ToC and ToA emphasize different

components of a logic model

O Both ToC and ToA represent theory-

based programs and are often used

together to represent complex initiatives

How ToC and ToA Extend

Logic Models Logic Models

Theories of Change

and Theories of Action

O 1970s

O Representation

O List of Components

O Descriptive – WHAT

and HOW

O 1990s

O Justification

O Pathway of Change

O Explanatory – HOW

and WHY

How are ToC and ToA

Different Theory of Change Theory of Action

O Represents an

understanding of

how a social,

political, economic,

cultural change

happens

O Complex

O Contextual

O Outcomes-based

O Explains how a

particular initiative

contributes to the

change process, and

is informed by the

broader ToC

O Sequential

O Specific

O Activities-based

Theory of Change Model

Theory of Action Model

Program Theory –

ToC and ToA Model

Outcomes Approach Models

O Focus on during early aspects of

program planning

O More clearly connect resources

and/or activities with desired results

O Subdivide outcomes and impact that

may result from a given set of

activities over time (i.e., short-term,

long-term, impact)

Outcomes Model


Why use an Outcomes-

based Approach? O Demonstrates that program implementers

have understanding of anticipated results

O Explicitly considers time as a variable

O Stakeholders like them

O Facilitates program summative evaluation

O Can use when program theory is

underdeveloped – but not UNdeveloped

Activities Approach Models

O Focus on program implementation

process

O Clearly link activities to each other to

map implementation

O Provide more detail about steps and

sequence of program implementation

Activities Model

W.K. Kellogg

Foundation

(January 2004)

Why use an Activities-based

Approach? O Demonstrates that program implementers

have understanding of relationships among

program activities

O Allows for studying interactions of activities

O Facilitates program monitoring, management

O Participants like them because they show

what you will actually do.

O Facilitates program formative or process

evaluation

Sidebar 2 O Logic Model, Theory of

Change, Theory of

Action are often used

interchangeably and are

not clearly defined in the

literature

O Funders’ often request

“logic model” when they

expect a representation

or model of theory

Basic Logic Model

Another Example

And Another . . .

Building a Logic Model

Conceptualize Program

Describe Assumptions

Add Program Components

What are your

intended

outcomes?

What do you know

(or assume) to be

true?

How will you

produce

outcomes?

Basic Logic Model Template

Resources

Activities

Outputs

Outcomes

Impact

In order to

accomplish

our set of

activities we

will need the

following:

In order to

address our

problem or

asset we will

conduct the

following

activities:

We expect

that once

completed or

under way

these

activities

will produce

the following

evidence of

service

delivery:

We expect that

if completed or

ongoing these

activities will

lead to the

following

changes in 1–

3 then 4–6

years:

We expect

that if

completed

these

activities

will lead

to the

following

changes in

7–10 years:

Sidebar 3 O Grantees usually know what they

want to do – “activity focused”

O Make explicit connections

between program activities and

related “best practice” literature

– “why will your approach work?”

O Practitioner wisdom also counts --

but not as much as it used to


Resources

Activities

Outputs

Outcomes

Impact

In order to

accomplish

our set of

activities we

will need the

following:

In order to

address our

problem or

asset we will

conduct the

following

activities:

We expect

that once

completed or

under way

these

activities

will produce

the following

evidence of

service

delivery:

We expect that

if completed or

ongoing these

activities will

lead to the

following

changes in 1–

3 then 4–6

years:

We expect

that if

completed

these

activities

will lead

to the

following

changes in

7–10 years:

Outputs or Outcomes?

Outputs Outcomes

O Direct results of

program activities

(e.g., # of meetings,

students enrolled)

O Results you expect to

achieve over time as a

result of implementing

activities

Conducting an activity is not the same as achieving

results from accomplishing that activity.

Outcomes

SMART GOALS

O Specific

O Measurable

O Action-oriented

O Realistic

O Timed

Outcomes are specific

changes in attitudes,

behaviors, knowledge,

skills, status, or level of

functioning expected to

result from program

activities, most often at

an individual or group

level.


Resources

Activities

Outputs

Outcomes

Impact

In order to

accomplish

our set of

activities we

will need the

following:

In order to

address our

problem or

asset we will

conduct the

following

activities:

We expect

that once

completed or

under way

these

activities

will produce

the following

evidence of

service

delivery:

We expect that

if completed or

ongoing these

activities will

lead to the

following

changes in 1–

3 then 4–6

years:

We expect

that if

completed

these

activities

will lead

to the

following

changes in

7–10 years:

Impacts

O Long-term

(7-10 years)

O Improved conditions

O Increased capacity

O Policy changes

Impacts are collective

organizational,

community, and/or

system level changes

expected to result from

program activities.


Resources

Activities

Outputs

Outcomes

Impact

In order to

accomplish

our set of

activities we

will need the

following:

In order to

address our

problem or

asset we will

conduct the

following

activities:

We expect

that once

completed or

under way

these

activities

will produce

the following

evidence of

service

delivery:

We expect that

if completed or

ongoing these

activities will

lead to the

following

changes in 1–

3 then 4–6

years:

We expect

that if

completed

these

activities

will lead

to the

following

changes in

7–10 years:

Resources – Available and

Limiting Resources and/or

barriers, which

potentially enable

or limit program

effectiveness.

O Enabling: funding, existing

organizations,

collaborating partners,

staff, time, equipment,

supplies

O Limiting: attitudes, lack of

resources, policies, laws,

regulations, geography

Activities

Activities are

processes, tools,

techniques, events,

technology, and

actions used to

bring about desired

results.

O Products: promotional

materials, educational

curricula

O Services: education,

training

O Infrastructure: structure,

relationships, capacity


Resources

Activities

Outputs

Outcomes

Impact

In order to

accomplish

our set of

activities we

will need the

following:

In order to

address our

problem or

asset we will

conduct the

following

activities:

We expect

that once

completed or

under way

these

activities

will produce

the following

evidence of

service

delivery:

We expect that

if completed or

ongoing these

activities will

lead to the

following

changes in 1–

3 then 4–6

years:

We expect

that if

completed

these

activities

will lead

to the

following

changes in

7–10 years:

Outputs

O Size and/or scope of

services/products

delivered

O Number of classes

taught, meetings

held, or materials

produced

O Participation rates

and demography

Outputs are the direct

results of program

activities. They indicate

if a program was

delivered to the

intended audiences as

planned.

References O W.K. Kellogg Foundation. (January 2004).

Using Logic Models to Bring Together

Planning, Evaluation, and Action: Logic Model

Development Guide. Battle Creek, Michigan.

O Chen, H. T. (1990). Theory driven evaluations.

Newbury Park, CA: Sage.

O Weiss, C. H. (1998). Evaluation: Methods for

studying programs and policies. (2nd Ed).

Upper Saddle River, NJ: Prentice Hall.

Resources

Resources for Logic Model Development

O The Evaluation Forum (1999). Outcomes for

success.

O Targeting Outcomes of Programs.

http://deal.unl.edu/TOP/synopsis.htm

O Innovation Network, Inc. electronic logic model

development tool http://www.inetwork.org

Resources Related Planning and Evaluation Resources

O W.K. Kellogg Foundation, http://www.wkkf.org

O Taking Stock. http://www.horizon-research.org

O The Evaluation Forum (1994). A field guide to

outcome-based program evaluation.

O Rossi, P. H., Freeman, H. E., & Lipsey, M. W.

(1999). Evaluation: A systematic approach.

Thousand Oaks, CA: Sage.

O Patton, M. Q. (1997). Utilization-focused

evaluation: The new century text. Thousand Oaks,

CA: Sage.

Contact & Location Information

Discovery Center for Evaluation,

Research, and Professional Learning

210 E. Spring St.

408 McGuffey Hall

Phone: (513) 529-1686

Email: [email protected]

Updates to

Discovery Center Website

coming soon.

logic model pp presentation_final_for sharing

Education