introduction to logic models

—Tools for Research Project, Program, and Evaluation Design—

Introduction to Logic Models

June 27, 2016

Discovery/EHS Professional Development

1

Presentation Overview

》Section 1, Explain:

• What is systematic inquiry?

• What is logic modeling?

• What are the purpose(s) of using a logic model?

》Section 2, Review:

• Different types of logic models (e.g., Basic, Theory,

Activities, Outcomes).

• Tips/Reasons for using each type of model.

》Section 3, Discuss:

• How can I create/design a logic model?

• Break-down and examples of logic model components.

Section 1: Systematic Inquiry & Logic Models

》Explain the idea behind “systematic-inquiry”

》Connect “big-picture” idea of systematic inquiry

to logic modeling.

》Discuss the benefit of thinking about the whole

process/cycle of research (*outcomes first)

before designing a research project or program

plan.

Systematic Inquiry Cycle

"Systematic implies [...] a thoughtful process

of asking critical questions [a.k.a, "inquiry"],

collecting appropriate information, and then

analyzing and interpreting the information

for a specific use and purpose." – Program Development and Evaluation

University of Wisconsin-Extension, Cooperative Extension, 2008.

http://www.uwex.edu/ces/4h/evaluation/documents/glossary.pdf

Planning and Design

Protecting Participant

Rights

Developing and

Modifying Instruments

Data Collection

and Processing

Data Analysis

and Reporting

http://www.uwex.edu/ces/4h/evaluation/documents/glossary.pdf

What is a Logic Model?

“A program logic model is a picture of how your program works – the theory and assumptions underlying the program. ...This model provides a road map of your program, highlighting how it is expected to work, what activities need to come before others, and how desired outcomes are achieved.” - W.K. Kellogg Foundation Evaluation Handbook (1998, p. 35)

“…a valuable tool for discovering and displaying the links between activities and outcomes.” - New Directions for Evaluation (2008, p. 17)

Key Features of Logic Models

》Systematic – Visual – Dynamic - Flexible

》Sequential but not always linear

》Shows the “What” and “How” of your project/program

》May show the “Why” behind your project/program

》Shows area of strengths, weaknesses, gaps, and

limitations

》Promotes strategic/programmatic management

》Typically has 5 components (e.g., Resources/Inputs,

Activities, Outputs, Outcomes, Impacts)

Basic Logic Model Outline

W.K. Kellogg Foundation (January 2004)

Basic Logic Model Components

》PLANNED WORK

• Resources/Inputs: human, financial, organizational, or

community resources a program has available to do the work

• Activities: processes, tools, events, technology, and actions

that are an intentional part of program implementation

Basic Logic Model Components

》 INTENDED RESULTS

• Outputs: direct products of program activities

• Outcomes: specific changes in program participants’

behavior, knowledge, skills, status

• Short-term: 1-3 years; Long-term: 4-6 years

• Impact: fundamental intended or unintended change as a

result of program (outcomes 7-10 years)

Basic Logic Model Example

Put on Your Dancing Shoes

Go Dancing w/Friends

Your Dance Moves get Posted on YouTube

YouTube Video goes

Viral

You become Famous/

Overnight Sensation

Logic Model “Logic”

Certain resources are needed to operate your project/program:

If . . . Then . . .

you have access to them, you can use them to accomplish your

planned activities.

you accomplish your planned

activities,

you will deliver the amount of

product/service that you intended.

you accomplish your planned

activities to the extent intended,

your participants will benefit in specific

ways.

these benefits are achieved, certain changes might occur under

specified conditions.

Why Use Logic Modeling?

》Planning and Design

》 Implementation and Management

》Evaluation and Communication

How Logic Models Better Position Research/Programs for Success

Program ElementsCriteria for Program

Success

Benefits of Program

Logic Models

Planning and DesignProgram goals and objectives, and important side effects are well defined ahead of time.

Finds “gaps” in the theory orlogic of a program and work toresolve them.

Program goals and objectivesare both plausible and possible.

Builds a shared understandingof what the program is allabout and how the parts worktogether.

Program Implementation and Management

Relevant, credible, and usefulperformance data can beobtained.

Focuses attention ofmanagement on the mostimportant connections between action and results.

Evaluation, Communication, and Marketing

The intended users of theevaluation results have agreed onhow they will use theinformation.

Provides a way to involve andengage stakeholders in thedesign, processes, and use ofevaluation.

W.K. Kellogg Foundation (January 2004) p.6


Planning and Design


Rights

Developing and Modifying

Instruments

Data Collection

and Processing

Data Analysis and

Reporting

Determine purpose(s),

resources, activities, and

anticipated outcomes;

ensure participants’

safety/rights and

credibility for your work.

Planning and Design

》 Provides a road map for ideas/plan/design

》 Describes sequence of events

》 Connects the need for the program with desired results

》 Allows you to see missing links or limitations

“Most of the value in a logic model is in the process of creating, validating, and modifying the model …The clarity of thinking that occurs from building the model is critical to the overall success of the program.” - W.K. Kellogg Foundation (2004, p. 43)


Planning and Design


Rights


Instruments

Data Collection

and Processing

Data Analysis and

Reporting

Manage, monitor, and

modify activities to

improve progress and

final outcomes.

Implementation and Management

》Helps visualize and understand how investments/inputs contribute to achieving intended projects or program goals

》Monitors incremental progress toward outcomes

》Facilitates project or program improvements

“Program logic modeling is a learning and management tool that can be used throughout a program’s life . . . result(ing) in effective programming and offering greater learning opportunities.” - W.K. Kellogg Foundation (2004, p. 4)


Planning and Design


Rights


Instruments

Data Collection

and Processing

Data Analysis and

Reporting

Document outcomes and

share knowledge with

others.

Evaluation and Communication

》Documentation of outcomes

》Shared knowledge about what works and why

》 Improves likelihood of sustaining a program over time should you use your work for future academic endeavors (e.g., publication, funding)

“A program is a theory and an evaluation is its test. . . to organize the evaluation to provide a responsible test, the evaluator needs to understand the theoretical premises on which the program is based.” - Carol Weiss (1998, p. 55)

Section 2: Logic Model Types/Tips/Uses

》Review 3 different types of logic models:

• Theory-Approach Model

• Outcomes-Approach Model

• Activities-Approach Model

》Review reasons and tips for using each type of

logic model.

》Review additional examples of logic models.

How to create a “logic model”…

Chris Lysy (http://freshspectrum.com/)

Logic Model Tips/Types

》There is no “Best” Logic Model - Choose the model that fits your research project or program best and provides information you need in the format that is most helpful.

》Don’t worry if your model doesn’t look like others but DO worry if you cannot explain your model to others.

》Typically logic models use table and flow chart formats with text and arrows or a graphic representation of program ideas.

》A logic model does not have to be linear. Concept maps or other visuals may better describe complex program concepts.

More Logic Model Tips

Chris Lysy (freshspectrum.com)

》Models do not need to be drawn to scale.

Three Approaches to Logic Modeling

》Theory Approach Model (conceptual)

》Outcome Approach Model

》Activities Approach Model (applied)

Theory Approach Models

》Emphasize the theory of [logic] influencing design and

plan for the research project/program

》Explain how and why your project/program will work

》May have additional parts (e.g., describes problem or

issue addressed by project/program, connects

strategies to activities)

Theory Approach Model Example



Why use a Theory-based Approach?

》Demonstrates that project/program implementers have

deep understanding of project/program aspects/ideas

》Allows consideration of context (to see “Big Picture”)

》Theories of Change (ToC) and Theories of Action

(ToA) are simply extensions of Logic Modeling – and

help explain the rationale or logic behind why certain

entities are implementing a project/program.

》Facilitates project/program improvement and

evaluation

Outcomes Approach Models

》Still emphasizes the theory of change influencing

design and plan for the project/program…

》…But focuses during early stages of project/program

planning on connecting resources and/or activities with

desired results (a.k.a. outcomes)

》Subdivides outcomes and impact that may result from

a given/distinct set of activities over time (i.e., short-

term, long-term, impact)

Outcomes Approach Model Example


Why use an Outcomes-based Approach?


understanding of anticipated results

》Can use when project/program theory is

underdeveloped – but not UNdeveloped

》Facilitates project/program summative evaluation

Activities Approach Models

》Focus on project/program implementation process

emphasizing higher detail on “nuts-and-bolts” of

project/program plan.

》Clearly link activities to each other to map

implementation

》Provides more detail about steps and sequence of

project/program implementation

Activities Approach Model Example


Why use a Activities-based Approach?


understanding of relationships among project/program

activities

》Allows for studying interactions of activities

》Facilitates program monitoring/management

》Participants like them because they show what you

will actually do

》Facilitates program formative or process evaluation

Basic Logic Model (More Examples)

Outcomes Approach Model (More Examples)

Section 3: Logic Model Break-Down & Development

》Break-down of basic logic model components:

• *Remember to think “in order of design” – Outcomes, Impact,

Resources/Inputs, Activities, and Outputs.

》Review of logic model component examples.

• *If time allows – Review examples of student research/projects.

》Offer additional logic model resources.

Review of Basic Logic Model Components

Resources

InputsActivities Outputs Outcomes Impact

In order to

accomplish our

set of activities

we will need the

following…

In order to

address our

problem or

asset we will

conduct the

following

Activities…

We expect

that once

completed or

under way

these

activities

will produce

the following

evidence of

service

delivery…

We expect that

if completed or

ongoing these

activities will

lead to the

following

changes in 1–3

then 4–6

years…

We expect

that if

completed

these

activities will

lead

to the

following

changes in

7–10 years…

Break-down of Logic Model Components

Resources


In order to

accomplish our

set of activities

we will need the

following…

In order to

address our

problem or

asset we will

conduct the

following

Activities…

We expect

that once

completed or

under way

these

activities

will produce

the following

evidence of

service

delivery…

We expect that

if completed or

ongoing these

activities will

lead to the

following

changes in 1–3

then 4–6

years…

We expect

that if

completed

these

activities will

lead

to the

following

changes in

7–10 years…

Outcomes

Outcomes Example(s)

Outcomes are specific changes in

attitudes, behaviors, knowledge,

skills, status, or level of functioning

expected to result from program

activities, most often at an

individual or group level (Short-

term: 1-3 years; Long-term: 4-6

years).

Earlier Example: Dancing YouTube

video goes viral.

Another Example: Teachers create

more inquiry-based STEM units for

their classrooms, and have

increased comfort levels creating

inquiry-based STEM units (medium-

term), and a deeper understanding

of how to teach inquiry-based

STEM (long-term).

》 Sometimes “reverse-thinking” is required – What results do I expect…(i.e., think about outcomes first)

》 SMART GOALS: Specific, Measurable, Action-oriented, Realistic, and Timed

Differences between Outcomes vs. Outputs

Outputs Outcomes

Direct results of program activities

(e.g., # of meetings, students

enrolled)

Results you expect to achieve over

time as a result of implementing

activities.

》 Remember the differences between components of a logic model - *Conducting an activity is not the same as achieving results from accomplishing that activity.

Think About Basic Logic Model Components

Resources


In order to

accomplish our

set of activities

we will need the

following:

In order to

address our

problem or

asset we will

conduct the

following

activities:

We expect

that once

completed or

under way

these

activities

will produce

the following

evidence of

service

delivery:

We expect that

if completed or

ongoing these

activities will

lead to the

following

changes in 1–3

then 4–6 years:

We expect

that if

completed

these

activities will

lead

to the

following

changes in

7–10 years:

Impacts

Impacts Example(s)

Impacts are collective

organizational, community, and/or

system-level changes expected to

result from program activities.

Earlier Example: You become

famous via overnight sensation.

Another Example: Students have

better critical thinking skills and

choose STEM career pathways at a

higher rate.

》 Long-Term: 7 – 10 years, such as improved conditions, increased capacity, or policy changes.

Think About Basic Logic Model Components

Resources


In order to

accomplish our

set of activities

we will need the

following:

In order to

address our

problem or

asset we will

conduct the

following

activities:

We expect

that once

completed or

under way

these

activities

will produce

the following

evidence of

service

delivery:

We expect that

if completed or

ongoing these

activities will

lead to the

following

changes in 1–3

then 4–6 years:

We expect

that if

completed

these

activities will

lead

to the

following

changes in

7–10 years:

Resources/Inputs: Available and Limiting

Enabling/Limiting Resources Example(s)

Enabling: Funding, existing

organizations, collaborating

partners, staff, time, equipment,

supplies.

Limiting: Attitudes, lack of

resources, policies, laws,

regulations, geography.

Earlier Example: Put on your

dancing shoes.

Another Example: New PD

curriculum developed; faculty

trained to teach workshop; space in

McGuffey Hall provided for PD

workshops.

》 Resources and/or barriers, which potentially enable or limit program effectiveness.

Activities

Activities Example(s)

Activities are processes, tools,

techniques, events, technology,

and actions used to bring about

desired results

• Products: promotional materials,

educational curricula

• Services: education, training

• Infrastructure: organizational

structure, relationships, capacity

Earlier Example: You go dancing

with your friend.

Another Example: Teachers

participate in a 2-week PD

workshop and learn to create

inquiry-based STEM units as part of

their curricula.

Break-down of Logic Model Components

Resources


In order to

accomplish our

set of activities

we will need the

following…

In order to

address our

problem or

asset we will

conduct the

following

Activities…

We expect

that once

completed or

under way

these

activities

will produce

the following

evidence of

service

delivery…

We expect that

if completed or

ongoing these

activities will

lead to the

following

changes in 1–3

then 4–6

years…

We expect

that if

completed

these

activities will

lead

to the

following

changes in

7–10 years…

Outputs

Outputs Example(s)

Outputs are the direct results of

program activities. They indicate if

a program was delivered to the

intended audiences as planned.

• Size and/or scope of

services/products delivered

• Number of classes taught,

meetings held, or materials

products

• Participation rates and

demography

Earlier Example: Your dance moves

get posted on YouTube.

Another Example: 24 new inquiry-

based science units are created as

a result of teacher PD workshop.

Presentation References

》 W. K. Kellogg Foundation. (January 2004). Using Logic Models to

Bring Together Planning, Evaluation, and Action: Logic Model

Development Guide. Battle Creek, Michigan.

》 Weiss, C. H. (1998). Evaluation: Methods for studying programs

and policies. (2nd Ed). Upper Saddle River, NJ: Prentice Hall.

》 Hendricks, M., Plantz, M. C., & Pritchard, K. J. (2008). Measuring

outcomes of United Way–funded programs: Expectations and

reality. In J. G. Carman & K. A. Fredericks (Eds.), Nonprofits and

evaluation. New Directions for Evaluation, 119, 13–35.

》 Chris Lysy graphics: http://freshspectrum.com

Helpful Resources

Logic Models Development

》The Evaluation Forum (1999). Outcomes for success.

》Targeting Outcomes of Programs.

http://deal.unl.edu/TOP/synopsis.htm

》 Innovation Network, Inc. electronic logic model

development tool http://www.inetwork.org

Planning and Design

》W.K. Kellogg Foundation, http://www.wkkf.org

》Taking Stock. http://www.horizon-research.org

http://deal.unl.edu/TOP/synopsis.htm

http://www.inetwork.org/

http://www.wkkf.org/

For more information:

》 Emily Ryan

• [email protected]

Discovery Center for

Evaluation, Research, and

Professional Learning

210 E. Spring St.

408 McGuffey Hall

(513) 529-1686

[email protected]

》 Yue Li


》 Kristen Morio


mailto:[email protected]



introduction to logic models

Education