Planning for Performance Planning for Performance Measurement: Measurement:
Reviewing the Performance Reviewing the Performance MeasuresMeasures
Presenters:
Gregory Thompson, OJJDPHeidi Hsia, OJJDP
Marcia Cohen, DSG, Inc.
Reporting RequirementsReporting Requirements
Output: All mandatory (bold) and two non-mandatory
output measures.
Outcome: All mandatory (bold) and two non-mandatory
measures.*
*Except Core Requirements and JJ System Improvement require only one non-mandatory measure.
Key to Logic Models and Grid ChartsKey to Logic Models and Grid Charts
Bold = Mandatory measure
* = Mandatory only for intervention programs
** = Mandatory only for prevention programs
Output Objective CategoriesOutput Objective Categories
Six types of output objectives:• Increase organizational/system capacity• Improve planning and development• Improve program quality• Improve program activities• Improve program efficiency • Improve the monitoring of compliance
Outcome Objective CategoriesOutcome Objective Categories
Nine types of outcome objectives:• Reduce delinquency• Increase accountability• Improve system effectiveness• Increase prosocial behaviors• Increase program support• Increase compliance with Core Requirements• Reduce disproportionate minority contact• Improve school climate• Improve the management of the State Juvenile Justice and Delinquency Prevention program
Prevention and Intervention Prevention and Intervention Program Areas Program Areas
Mandatory Mandatory OutputOutput Measures for Measures for Prevention and InterventionPrevention and Intervention Programs Programs
Title V or FG $ allocated for services:
• The amount of Formula Grants and Title V funds in whole dollars that are allocated for delinquency prevention services during the reporting period. Program records are the preferred data source.
Mandatory Mandatory OutputOutput Measures for Measures for Prevention Prevention
and Interventionand Intervention Programs (cont’d) Programs (cont’d)
Number of program youths served:
• An unduplicated count of the number of youths served by the program during the reporting period. Number served should be determined by subtracting out youths who dropped out/completed program/or were discharged/ terminated during the reporting period and adding in new admissions during the reporting. Include all youths who received at least one program service and met the program’s minimum criteria for participation. Program records are the preferred data source.
Mandatory Mandatory OutcomeOutcome Measures Measures for for PreventionPrevention** Programs** Programs
Number and percent of program youths exhibiting desired change in targeted behaviors: • Substance use • School attendance • Antisocial behavior • Family relationships • Pregnancies
Note: Double ** denotes mandatory for prevention programs
Mandatory Mandatory OutcomeOutcome Measures Measures for for InterventionIntervention* Programs* Programs
Number and percent of program youths who offend or reoffend:
• The number of program youths who were rearrested or seen at juvenile court for a new delinquent offense. Appropriate for any youth-serving program.
Note: Single * denotes mandatory for intervention programs
Mandatory Mandatory OutcomeOutcome Measures for Measures for Prevention and InterventionPrevention and Intervention Programs Programs
Number and percent of youth completing program requirements:
• The number and percent of program youth who have successfully fulfilled all program obligations and requirements. Program obligations will vary by program but should be a predefined list of requirements or obligations that clients must meet prior to program completion. Program records are the preferred data source.
Example: Example: Probation Program Area Probation Program Area
Outcome Measure DefinitionsShort Term:Occurs during the program or by the end of the programLong Term:Occurs 6 months to 1 year after program completion
Key = System-level indicator
= Individual-level indicator = Objectives
Bold= Mandatory measure
Performance measures should report on those activities funded by Title II (Formula Grants) funds.
Long-Term Outcome ObjectivesReduce delinquencyIncrease accountability
Short-Term Outcome Objectives
Reduce delinquencyImprove prosocial behaviorsIncrease accountabilityIncrease program support
#/% of program staff with increased knowledge of program area
#/% of program youths exhibiting desired change in targeted behaviors (e.g., substance use, school attendance, antisocial behavior, family relationships, social competencies)
#/% of program youths completing program requirements
#/% of program youths charged with formal probation violation
#/% of program youths who offend or reoffend
#/% of program youths committed to correctional facility
OUTCOME MEASURES
SHORT-TERM
#/% of program youths committed to correctional facility
#/% of program youths who offend or reoffend
LONG-TERM
Delinquency
To improve JJ systems by increasing compliance with the Core Requirements and increasing the availability and types of prevention and intervention programs
Sample Logic Model: PA 24—Probation
PROBLEM
Goals
INTERVENTION PROGRAM
(Deficiency or lack of adequate probation
programs)
To support both State and local
intervention efforts and JJ
system improvements
SUBPROBLEM(S)
Objectives
Output ObjectivesIncrease organizational capacityImprove planning & developmentImprove program qualityImprove program activitiesImprove program efficiency
OUTPUT MEASURES
Conduct PlanningActivities
andImplement Program(s)
Monitor Program(s)
ACTIVITIES
FG $ allocated for services
# of planning activities conducted
# of MOUs developed
# of program slots available
# of FTEs funded with FG $
#/% of program staff trained
# of hours of program staff training provided
Use of best practice model (Y/N)
# of program materials developed
# of program/agency policies or procedures, created, amended, or rescinded
# of program youths served
Average length of stay in program
# of service hours completed
PROGRAM AREA 24. ProbationPROGRAM AREA 24. ProbationOUTPUTOUTPUT PERFORMANCE MEASURES PERFORMANCE MEASURES
Sample Grid Chart: PA 24—ProbationOutput Performance Measures
Sample Grid Chart: PA 24—ProbationOutcome Performance Measures
Example: Example: Delinquency Prevention Delinquency Prevention
Program Area Program Area
Outcome Measure DefinitionsShort Term:Occurs during the program or by the end of the programLong Term:Occurs 6 months to 1 year after program completion
Key = System-level indicator
= Individual-level indicator = Objectives
Bold= Mandatory measure
Performance measures should report on those activities funded by Title II (Formula Grants) funds.
Delinquency
To improve JJ systems by increasing compliance with the Core Requirements and increasing the availability and types of prevention and intervention programs
Sample Logic Model: PA 09—Delinquency Prevention
PROBLEM
Goals
PREVENTION PROGRAM
(Deficiency or lack of adequate delinquency prevention programs)
To support both State and local
intervention efforts and JJ
system improvements
SUBPROBLEM(S)
Objectives
Output ObjectivesIncrease organizational capacityImprove planning & developmentImprove program qualityImprove program activitiesImprove program efficiency
OUTPUT MEASURES
Conduct PlanningActivities
andImplement Program(s)
Monitor Program(s)
ACTIVITIES
FG $ allocated for services
# of MOUs developed
# of planning activities conducted
# of program slots available
# of FTEs funded with FG or TV $
#/% of program staff trained
# of hours of program staff training provided
Use of best practice model (Y/N)
# of program materials developed
# of program/agency policies or procedures, created, amended, or rescinded
# of program youths served
#/% of parents served
Average length of stay in program
# of service hours completed
Long-Term Outcome ObjectivesImprove prosocial behaviors
Short-Term Outcome Objectives
Improve prosocial behaviorsIncrease program supportIncrease accountability
#/% of program staff with increased knowledge of program area
#/% of program youths exhibiting desired change in targeted behaviors (e.g., substance use, school attendance, antisocial behavior, family relationships, pregnancies)
#/% of program youths completing program requirements
#/% of program families satisfied with program
OUTCOME MEASURES
SHORT-TERM
#/% of program youths exhibiting desired change in targeted behaviors (e.g., substance use, school attendance, antisocial behavior, family relationships, pregnancies)
LONG-TERM
#/% of youths satisfied with program
Sample Grid Chart: PA 09—Delinquency PreventionOutput Performance Measures
Sample Grid Chart: PA 09—Delinquency PreventionOutcomes Performance Measures
Core Requirements Core Requirements Program Areas Program Areas
Mandatory Mandatory OutputOutput Measures for Measures forCore RequirementsCore Requirements Programs Programs
FG funds allocated:
• The amount of Formula Grants money in whole dollars that is allocated for the separation requirement during the reporting period. Program records are the preferred data source.
Mandatory Mandatory OutputOutput Measures for Measures for Core RequirementsCore Requirements Programs (cont’d) Programs (cont’d)
Number of programs implemented:
• The number of programs created and/or implemented during the reporting period. Program records are the preferred data source.
Mandatory Mandatory OutcomeOutcome Measure for Measure for Core RequirementsCore Requirements Programs Programs
Change in the number of violations
The change in the number of violations of the sight and sound requirement from the previous reporting period compared with the current reporting period. Sight and sound compliance is determined according to the definition in the OJJDP Guidance Manual for Monitoring Facilities Under the JJDP Act of 2002. The Annual Compliance Monitoring Report is the preferred data source.
Example: Example: Separation Program AreaSeparation Program Area
Outcome Measure DefinitionsReported Annually.
Key = System-level indicator
= Individual-level indicator = Objectives
Bold= Mandatory measure
Performance measures should report on those activities funded by Title II (Formula Grants) funds.
Delinquency
To improve JJ systems by increasing compliance with the Core Requirements and increasing the availability and types of prevention and intervention programs.
Sample Logic Model: PA 28—Separation
PROBLEM
Goals
Outcome ObjectivesIncrease compliance with core requirementsIncrease program support
OUTCOME MEASURES
#/% of staff with increased knowledge of program area (e.g., separation regulations)
Change in the number of violations of sight and sound separation
OUTPUT MEASURESACTIVITIES
Output Objectives•Improve monitoring of compliance•Increase organizational capacity•Improve planning & development
Conduct Planning and
Oversight Activities
Implement Processes
and/or Programs
FG $ allocated for separation
# of MOUs developed
Needs assessment completed (Y/N)
#/% of programs implemented
# of transportation plans developed
# of shelter beds contracted
#/% of staff trained
# of hours of staff training provided
# of materials developed
# of program policies/procedures created, amended, or rescinded
Monitor Processes
and/or Programs
#/% of site visits conducted
# of facilities receiving TA
CORE REQUIREMENT(Ensuring/monitoring compliance with the
Separation Core Requirement)
To support both State and local prevention and
intervention efforts and JJ
system improvements
SUBPROBLEM(S)
Objectives
Sample Grid Chart: PA 28—SeparationOutput Performance Measures
Sample Grid Chart: PA 28—SeparationOutcome Performance Measures
Example: Example: Compliance Monitoring Compliance Monitoring
Program AreaProgram Area
Outcome Measure DefinitionsReported Annually.
Key = System-level indicator
= Individual-level indicator = Objectives
Bold= Mandatory measure
Performance measures should report on those activities funded by Title II (Formula Grants) funds.
Delinquency
To improve JJ systems by increasing compliance with the Core Requirements and increasing the availability and types of prevention and intervention programs
Sample Logic Model: PA 06—Compliance Monitoring
PROBLEM
Goals
CORE REQUIREMENT(Ensuring/monitoring
compliance with Section 223 (a)(14) of the JJDP Act of 2002)
To support both State and local prevention and
intervention efforts and JJ
system improvements
SUBPROBLEM(S)
Objectives
Output Objectives•Improve monitoring of compliance•Increase organizational capacity•Improve planning & development
OUTPUT MEASURES
Conduct Planning and
Oversight Activities
ACTIVITIES
Funds allocated to adhere to Section 223 (a)(14) of the JJDP Act of 2002
# of activities that address compliance with Section 223 (a)(14) of the JJDP Act of 2002
# of MOUs developed
#/% of program staff trained
# of hours of program staff training provided
# of program materials developed
# of program policies/procedures created, amended, or rescinded
# of facilities receiving technical assistance
Outcome ObjectivesImprove the management of the State JJDP ProgramIncrease program support
OUTCOME MEASURES
#/% of staff with increased knowledge of program area (e.g., compliance monitoring)
Submission of complete Annual Monitoring Report to OJJDP
Sample Grid Chart: PA 06—Compliance MonitoringOutput Performance Measures
Sample Grid Chart: PA 06—Compliance MonitoringOutcome Performance Measures
Outcome Measure DefinitionsShort-Term:Occurs during the program or by the end of the programLong-Term:Occurs 6 months to 1 year after program completion
Key = system-level indicator
= individual-level indicator = objectives
Bold= mandatory measure
Performance Measures should report on those activities funded by Title II (Formula Grants) funds.
Delinquency
To improve JJ systems by increasing compliance with the Core Requirements and increasing the availability and types of prevention and intervention programs
Sample Logic Model:PA 10 – Disproportionate Minority Contact
PROBLEM
Goals
Output ObjectivesImprove monitoring of complianceIncrease system capacityImprove program qualityImprove planning and development
OUTPUT MEASURES
Conduct PlanningActivities
&Implement Program(s)
and Processes
Monitor Program(s)
and/or Programs
ACTIVITIES
FG $ allocated for DMC at the State and local levels
# of FTEs funded with FG or TV $
# of activities conducted
# of programs implemented
#/% of program staff trained
# of hours of program staff training provided
# of materials developed
# of program/agency policies or procedures, created, amended, or rescinded
# of objective decision-making tools developed
# of program youth served
Average length of stay in program
# of service hours completed
#/% of site/TA visits conducted
Outcome ObjectivesReduce DMCImprove system effectivenessIncrease program supportReduce delinquency
Number of minority staff hired
#/% of recommendations implemented from assessment studies
OUTCOME MEASURES
Short-Term
% of contact points reporting reduction in diproportionality at state level
Long-Term
Number of agencies with improved data collection systems
#/% of program staff with increased knowledge of program area
#/% of program youth who offend or reoffend
#/% of program youth exhibiting a desired change in targeted behaviors (e.g., substance use, school attendance, family relationships, and antisocial behavior)
#/% of program youth completing program
#/% of program youth satisfied with program
#/% of program families satisfied with program
% of contact points reporting reduction in diproportionality at local level
* #/% of program youth who offend or reoffend
#/% of state agencies reporting improved data collection systems
# of local agencies reporting improved data collection systems
SUBPROBLEM(S)
CORE REQUIREMENT(Ensuring compliance
with the DMC Core Requirement)
To support both State and local prevention and
intervention efforts and JJ system
improvements to reduce DMC
Objectives
INTERVENTION PROGRAM
(Deficiency or lack of adequate intervention programs to reduce
DMC)
PREVENTION PROGRAM
(Deficiency or lack of adequate delinquency prevention programs)
Sample Grid Chart: PA 10—Disproportionate Minority ContactOutput Performance Measures
Sample Grid Chart: PA 10—Disproportionate Minority ContactOutcome Performance Measures
Juvenile Justice System Juvenile Justice System Improvement Program AreasImprovement Program Areas
Mandatory Mandatory OutputOutput Measure for Measure for JJ System ImprovementJJ System Improvement Programs Programs
Number of grants funded with Formula Grants funds:
• The number of grants funded with Formula Grants funds during the reporting period.
Mandatory* Mandatory* OutputOutput Measures for Measures for JJ System ImprovementJJ System Improvement Programs Programs
* If applicable (if not applicable, select another non-mandatory measure)
• Number of programs implemented• Number of program youth served• Number of subgrants awarded• Number of grant applications reviewed
and commented upon
Mandatory* Mandatory* OutcomeOutcome Measures for Measures for JJ System ImprovementJJ System Improvement Programs Programs
* If applicable (if not applicable, select another non-mandatory measure)
• Number/percent of youth completing program • Number/percent of youth exhibiting desired
changed in targeted behaviors • Average length of time between initial appearance
and disposition• Number of programs funded directly in line with 3-
year Plan• Number of Plan recommendations implemented • Number of allegations of civil rights violations filed• Number/percent of court hearings attended by an
advocate
Example: Example: State Advisory Group (SAG) State Advisory Group (SAG)
Program AreaProgram Area
Mandatory Mandatory OutputOutput Measures for Measures for SAGSAG Program Area Program Area
Number of grants funded with Formula Grants funds:
• The number of grants funded with Formula Grants funds during the reporting period.
Mandatory Mandatory OutputOutput Measures for Measures for (SAG) Program Area(SAG) Program Area
Number of grant applications reviewed and commented on:
• The number of grant applications reviewed and commented on to guide the development of juvenile justice programming in the State.
Mandatory Mandatory OutcomeOutcome Measure for Measure forSAGSAG Program Area Program Area
Number and percent of Plan recommendations implemented:
• Number and percent of SAG recommendations for the State Plan implemented during the reporting period. The preferred data source is
committee meeting minutes.
Outcome Measure DefinitionsReported Annually.
Key = System-level indicator
= Individual-level indicator = Objectives
Bold= Mandatory measure
Performance measures should report on those activities funded by Title II (Formula Grants) funds.
Delinquency
To improve JJ systems by increasing compliance with the Core Requirements and increasing the availability and types of prevention and intervention programs
Sample Logic Model: PA 31—State Advisory Group (SAG) Allocation
PROBLEM
Goals
JUVENILE JUSTICE SYSTEMS
IMPROVEMENT(Need for effective
SAG activities)
To support both State and local prevention and
intervention efforts and JJ
system improvements
SUBPROBLEM(S)
Objectives
Output Objectives•Improve planning and development•Improve program efficiency•Improve program quality
OUTPUT MEASURES
Conduct Planning and
Oversight Activities
ACTIVITIES
# of grants funded with Formula Grants funds
# of SAG committee and subcommittee meetings held
Annual Report submitted to the Governor
#/% of SAG members trained
# of grant applications commented on
#/% of activities/meetings that involve youth
# of units of local government consulted during implementation of the 3-Year Plan
#/% of programs using best practice model
% of SAG allocation used
# of SAG members contributing to plan or plan update
Outcome ObjectivesIncrease compliance with Core RequirementsIncrease program support
#/% of SAG members who show increased knowledge of the program areas (for which they have oversight)
OUTCOME MEASURES
# of FG-funded programs sustained after 3 years
#/% Plan recommendations implemented
Sample Grid Chart: PA 31—State Advisory Group Output Performance Measures
Sample Grid Chart: PA 31—State Advisory Group Outcome Performance Measures
Formula Grants Formula Grants
Performance Measurement Performance Measurement Implementation TimelineImplementation Timeline
(First year) (First year)
April 1, 2005 Reporting period begins
Sept. 30, 2005 Reporting period ends
Dec. 31, 2005 Data report due to OJJDP
Formula Grants Formula Grants
Performance Measurement Performance Measurement Implementation TimelineImplementation Timeline
(Second and Subsequent Years) (Second and Subsequent Years)
Oct. 1, 2005 Reporting period begins
Sept. 30, 2006 Reporting period ends
Dec. 31, 2006 Data report due to OJJDP
Title V Performance MeasurementTitle V Performance MeasurementImplementation Timeline Implementation Timeline
(First year)(First year)
January 1, 2005 Reporting period begins
Sept. 30, 2005 Reporting period ends
Nov. 30, 2005 Data reports due to OJJDP
March 31, 2006 FY 2005 OJJDP Report to Congress due
Title V Performance MeasurementTitle V Performance MeasurementImplementation TimelineImplementation Timeline
(Second and Subsequent Years)(Second and Subsequent Years)
October 1, 2005 Reporting period begins
Sept. 30, 2006 Reporting period ends
Nov. 30, 2006 Data reports due to OJJDP
March 31, 2007 FY 2006 OJJDP Report to Congress due
Performance MeasurementPerformance MeasurementTools and SupportTools and Support
• Web site:
www.dsgonline.com/performance_measures.htmwww.dsgonline.com/performance_measures.htm
• Training
• Technical assistance