expanding possibilities program evaluation
TRANSCRIPT
- 1. Expanding Possibilities
Program Evaluation Plan for
2009- 2011
Presented by Rob Rice, Ph.D., LMHC
September 24th, 2009 - 2. Rob Rice, Ph.D., LMHC
Ph.D., University of Rochester, April 2009
Director for Easter Seals New York, 2004-2009
Currently a full time faculty member in the St. John Fisher College Masters Program in Mental Health Counseling
Excited about the Expanding Possibilities Project! - 3. Program Evaluation Model for Expanding Possibilities
Martin, L. & Kettner, P. (2009). Measuring the Performance of Human Service Programs, 2nd Ed. SAGE Publishing.
Basic Systems Model
Expanded Systems Model
Inputs
Process (treatment plan or service delivery methodology)
Output
Quality (e.g., responsiveness, strength-based, etc.)
Outcomes (results)
Feedback - 4. Output Performance Measures
Based on the following 2 grant priorities:
The percentage of relevant school personnel, community based organizations, and families served by the grant trained to make appropriate referrals.
Provide technical assistance and consultation to each school, community based organizations, and families participating in the program. - 5. Output Measure #1
Units of Service: One completed training of a schools personnel
Completion of Service: 1 year
Mid-grant objective: 50% of the relevant school mental health staff is trained to make appropriate referrals to mental health services
End of grant objective: 95% of the relevant school mental health staff is trained to make appropriate referrals to mental health services.
Method of Evaluation: The director will collect and report data to the program evaluator on a monthly basis - 6. Output Measure #2
Units of Service: # of schools/community based organizations that receive technical assistance
Completion of Service: 1 year
Mid-grant objective: 18 of the 27 schools and five community based organizations have received technical assistance and consultation from the program.
End of grant objective: all 27 schools and at least 8 community based organizations
Method of Evaluation: The director will collect and report data to the program evaluator on a monthly basis - 7. Quality Performance Measures
Based on the following grant priority:
All services will be provided in a linguistically appropriate and culturally competent method. - 8. Quality Measures
Quality Dimension: Cultural Competence
Mid grant objective: The cultural competence of the program will increase by 8%
End of grant objective: The cultural competence of the program will increase by 15%
Method of Evaluation: Likert based survey will be provided to families, schools, and mental health agencies
This survey will also endeavor to capture valuable data related to other quality dimensions important to the program (e.g., strength-based, solution-focused) - 9. Outcome Performance Measures
Based on the following 3 grant priorities:
The percentage of schools served by the grant that have comprehensive, detailed linkage protocols in place.
Improve linkages with all mental health community based organizations and government entities in Seneca and Tompkins County.
The programs effectiveness will be evaluated as to the effectiveness of the program in increasing student access to mental health services and recommendations will be made to the Secretary of Education about the sustainability of the program. - 10. Outcome Measure #1
Linkage Protocols
Objective: All 27 TST BOCES and component schools will have linkage protocols in place.
Method of Evaluation: The existence of linkage protocols will be verified with the principle of each school.
Relevant school staff will receive a Likert based survey intended to measure dimensions such as collaboration, accessibility , ease, responsiveness, effectiveness, sustainability, leadership, policies, resources, political climate, connectedness, community understanding, stigma, and other indicators.
The increasein reported results will indicate the improvement in the system. - 11. Outcome Measure #2
Student Access
Method of Evaluation: referrals for mental health services will be tracked by the school staff appointed to serve as the point person for the Expanding Possibilities Program.This data will be communicated to the Program Evaluator monthly.
The independent evaluator will provide a baseline, first year, and second year evaluation of the program. - 12. Exactly What We Need From You
Schools must identify appropriate staff to complete Likertbased survey intended to measure dimensions such as collaboration, accessibility , ease, responsiveness, effectiveness, sustainability, leadership, policies, resources, political climate, connectedness, community understanding, stigma, and other indicators.At a recent meeting, the Expanding Possibilities Team suggested using the newly formed linkage committees for this purpose.(Outcome Measure #1)TIMEFRAME: 12/09, 9/10, 9/11
School point people must begin tracking referrals for mental health services and communicate this data to the Program Evaluator monthly (Outcome Measure #2) TIMEFRAME: begin reporting 11/09 - 13. How to Reach Me
Cell (585) 727-4811
Office (585) 385-7318
E-mail: [email protected]