using a logic model to target your evaluation amy d’andrade san jose state university school of...
TRANSCRIPT
![Page 1: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/1.jpg)
Using a Logic Model to Target your
EvaluationAmy D’Andrade
San Jose State UniversitySchool of Social Work
November 20, 2014
![Page 2: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/2.jpg)
– Intensive coaching with 3 focus counties: Madera, Los Angeles, Tulare
– Peer cluster on-line evaluation coaching webinars
– Available by email to both groups:
Evaluation Coaching Support
![Page 3: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/3.jpg)
Plan for the webinar
1. Quick review: Evaluation, our philosophy, logic models
2. Follow-up with last webinar’s “homework”
3. Focus county update and lessons learned
4. Next Steps: Targeting your evaluation
![Page 4: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/4.jpg)
1. Quick review: Evaluation, our philosophy, logic models
![Page 5: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/5.jpg)
Why evaluate?
–Develop clarity re. program’s effectiveness
–Improve practice–Data for future funding/support
![Page 6: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/6.jpg)
Steps in Evaluation
ID team
Develop logic model
Gather dataAnalyze data
Adjust practice
![Page 7: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/7.jpg)
Linkages Evaluation Philosophy
• Partnership of shared expertise• Sustainable evaluation
![Page 8: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/8.jpg)
What is a Logic Model?
“.. a simple diagram that communicates the rationale for why a program will work.”
- Linkages Toolkit
![Page 9: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/9.jpg)
PROGRAM ACTIVITIES
LONG TERM OUTCOME
![Page 10: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/10.jpg)
INPUTS OUTPUTS OUTCOMES
Program resources
Activities Participation/ Short Medium
What we invest
What we do
Who we reach
What results we obtain
Long-term
![Page 11: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/11.jpg)
Summary
1. What ultimately are we hoping to change or improve? Attitudes, knowledge, skills, status, functioning
2. What are our activities? Processes, techniques, events, actions of the program
3. What evidence of service delivery will these activities produce?Quantify classes, sessions, participants
4. IF this is done, THEN what changes or is produced?
5. Repeat #4 until final outcome is reached.
![Page 12: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/12.jpg)
2. Follow-up with last webinar’s“homework”
![Page 13: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/13.jpg)
What’s next?
• Small group counties – Do attempt to draft, or we can do together
• Large group counties – Give it a try! And if you submit to me, I will review and give suggestions/feedback!
• NEXT WEBINAR – discussion about process. – How was the experience?
– Are you finding value in the product?
– If you didn’t do it, why not? What challenges got in the way?
![Page 14: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/14.jpg)
Brief Discussion
– How was the experience?
– Are you finding value in the product?
– If you didn’t do it, why not? What challenges got in the way?
![Page 15: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/15.jpg)
3. Focus county update and lessons learned
![Page 16: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/16.jpg)
Tulare Logic Model
![Page 17: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/17.jpg)
INPUTS ACTIVITIES OUTCOMES
Linkages Team
DPSS staff
CWS staff
Service providers
Families
CFPIC
Training for CWS staff on Linkages
Identification of shared families
Development of coordinated case
plan
DPSS staff attends TDMs
Regular communication between DPSS
and CWS workers
Case plans and casework
draw on knowledge
and resources of
both divisions
Services offered not duplicative
Services offered
provide for more family
needs
Greater service
availability/ fewer wait-
lists
Families better able to
complete case plan
requirements
Families more likely to
receive needed supports
Increased case
closure CWS / DPSS
Decreased re-entry CWS /
renewal DPSS
All eligible families receive
Linkages services
[To Linkages activities
1,2,3]
![Page 18: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/18.jpg)
Common Issues and Lessons Learned
• Co-creating a logic model takes time• Challenge to distinguish between process
and activity• Decisions about fundamental activities,
other outcomes not clear-cut• Importance of identification of shared cases• Meeting platform issues• Employee turnover/retention
![Page 19: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/19.jpg)
4. Next Steps: Targeting your evaluation
![Page 20: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/20.jpg)
Using Logic Models throughout program processes
Clarifying
Communicating
1. Planning
2. Implementation
3. Evaluation
![Page 21: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/21.jpg)
Two Approaches to Evaluation
Process Evaluation:Using empirical data to assess the delivery of programs– Extent: The number and quality of
program components delivered– Scope: The number and type of
recipients reached
![Page 22: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/22.jpg)
Two Approaches to Evaluation
Outcome Evaluation:Using empirical data to assess the impacts or outcomes of programs
![Page 23: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/23.jpg)
Are you confident implementation of your
Linkages program is full/strong/complete?
![Page 24: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/24.jpg)
INPUTS ACTIVITIES OUTCOMES
Linkages Team
DPSS staff
CWS staff
Service providers
Families
CFPIC
Training for CWS staff on Linkages
Identification of shared families
Development of coordinated case
plan
DPSS staff attends TDMs
Regular communication between DPSS
and CWS workers
Case plans and casework
draw on knowledge
and resources of
both divisions
Services offered not duplicative
Services offered
provide for more family
needs
Greater service
availability/ fewer wait-
lists
Families better able to
complete case plan
requirements
Families more likely to
receive needed supports
Increased case
closure CWS / DPSS
Decreased re-entry CWS /
renewal DPSS
All eligible families receive
Linkages services
[To Linkages activities
1,2,3]
![Page 25: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/25.jpg)
INPUTS ACTIVITIES
Linkages Team
DPSS staff
CWS staff
Service providers
Families
CFPIC
Training for CWS staff on Linkages
Identification of shared families
Development of coordinated case
plan
DPSS staff attends TDMs
Regular communication between DPSS
and CWS workers
80% of shared cases have a coordinated case plan developed outlining
both DPSS and CWS services
DPSS staff is present at 80% of TDMs in shared
cases
DPSS and CWS workers connect at least monthly on 80% of shared cases
80% of CWS staff received training annually
Identification of shared families occurs within a
week of CWS case opening; Linkages list matches IT list at 80%
![Page 26: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/26.jpg)
IF NO:Impact Evaluation Process Evaluation
Confident implementation of your Linkages program is full/strong/complete?
![Page 27: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/27.jpg)
INPUTS ACTIVITIES OUTCOMES
Linkages Team
DPSS staff
CWS staff
Service providers
Families
CFPIC
Training for CWS staff on Linkages
Identification of shared families
Development of coordinated case
plan
DPSS staff attends TDMs
Regular communication between DPSS
and CWS workers
Case plans and casework
draw on knowledge
and resources of
both divisions
Services offered not duplicative
Services offered
provide for more family
needs
Greater service
availability/ fewer wait-
lists
Families better able to
complete case plan
requirements
Families more likely to
receive needed supports
Increased case
closure CWS / DPSS
Decreased re-entry CWS /
renewal DPSS
All eligible families receive
Linkages services
[To Linkages activities
1,2,3]
![Page 28: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/28.jpg)
IF YES:Impact Evaluation Process Evaluation
Confident implementation of your Linkages program is full/strong/complete?
![Page 29: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/29.jpg)
IF NOT SURE:Impact Evaluation Process Evaluation
Confident implementation of your Linkages program is full/strong/complete?
![Page 30: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/30.jpg)
What’s next?
• Focus counties: – Finalize logic models
– Identify target of evaluation
• Peer cluster group counties: – Create time and space for evaluation?
– Draft and submit logic models?
– Discuss need for evaluation time with administration?
![Page 31: Using a Logic Model to Target your Evaluation Amy D’Andrade San Jose State University School of Social Work November 20, 2014](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e445503460f94b39199/html5/thumbnails/31.jpg)
Questions?