521: the quality service review process
DESCRIPTION
521: The Quality Service Review Process. Day 1 - Agenda. Training Overview Overview of the QSR Process Overview of the Indicators Roll-Up Sheet The QSR Interviews Child/Youth and Family Status Indicators. Day 2 – Agenda. Practice Performance Indicators - PowerPoint PPT PresentationTRANSCRIPT
521: The Quality Service Review Process
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Day 1 - Agenda
• Training Overview• Overview of the QSR Process• Overview of the Indicators• Roll-Up Sheet• The QSR Interviews• Child/Youth and Family Status Indicators
2
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Day 2 – Agenda
• Practice Performance Indicators• Feedback Session with the
Caseworker/Supervisor• Case-Specific Team Debriefing• The QSR Written Case Review Summary• Closing and Evaluation
3
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Learning Objectives
• Identify the components of the QSR Process;• Recognize the QSR Indicators;• Apply the interview techniques relative to the
protocol;• Assign reliable status and performance
indicator scores;• Demonstrate a Caseworker/Supervisor
Feedback Session;• Identify the steps to prepare for the Case-
Specific Debriefing; and• Recognize the components of the QSR Written
Case Review Summary.4
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
PA CQI
Positive Change
QSR
QSR’s Connection to CQI
5
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
DAPIMTM and “Learning by Doing”
Implement
Work products – team activities, action plans, charters for working teams, communication efforts
Plan
Work products- quick wins, mid- and long-term improvement plans, communication and capacity plans
Define
Work products – baseline surveys, alignment notes, initial feelings, ground rules, defined areas for improvements, desired future state
Performance & Capacity
Monitor
Work products – monitoring quick wins and other changes, evaluations and measures of progress and impact
Assess
Work products – findings: strengths and gaps, root causes and general remedies
6
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Purposes of the QSR Process
• To appraise the current status of the focus child/youth and family and the performance of key practice areas;
• To assure that the actions of a team of child welfare professionals are guided by clear and comprehensive policies and are rooted in Pennsylvania’s Practice Model/Standards;
• To improve the quality and usefulness of information needed to evaluate the safety, permanence and well-being of children and youth;
7
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Purposes of the QSR Process (continued)
• To assure the quality and effectiveness of services for children, youth and families by regularly reviewing programs and practices;
• To support the efforts of child welfare agencies to maintain an environment that encourages learning and program improvement; and
• To stimulate actions and next steps, to build on strengths and support efforts to improve outcomes.
8
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
What is the QSR?
The QSR is designed to conduct an in depth review of front line practice in specific locations and points in time.
It is used for:
• Appraising the current status of a child/youth in key life areas;
• Appraising the current status of the parent/caregiver; and
• Appraising the performance of key practices for the same child/youth/family.
9
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
How is the QSR Different From Other Quality Assurance Methods?Traditional Review Were visits timely?
Is the child’s IEP inthe file?
Was there a referralto mental health?
Quality Service Review
Are children currentlysafe?
Is the child progressing well academically?
Is the child functioning well emotionally?
10
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
How is the QSR Different From Other Quality Assurance Methods? (continued)
Traditional Review
Is there a current plan in the file?
Did the parents sign the plan?
Is there a written assessment?
Quality Service Review
Is the plan likely to lead to permanence?
Do the parents feel that the plan is their own?
Does the assessment accurately identify underlying needs?
11
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Skills and Abilities
Discuss with your partner the following questions:
• Which three challenges or needs are most important for you to overcome in order to become a successful QSR Reviewer?
• On a scale of 1-10, with one being a minimal challenge or need and 10 being a major challenge or need, how would you rate your three identified challenges or needs?
• What will it take for you to decrease your rating 1-2 points?
12
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
The QSR Process
1. Review Team Pairings2. Sample Selection3. Focus Groups and Stakeholder Interviews4. Case Reviews5. Feedback Session with the
Caseworker/Supervisor6. Case-Specific Debriefing7. QSR Written Case Review Summary8. Exit Conference (optional for reviewers)9. Final Report10.Next Steps Meeting11.County Improvement Plan
13
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
What Happens Prior to the Onsite Review?• Identification and pairing of internal and
external reviewers;
• Random sample of a county’s cases are selected and stratified by a service area;
• Family permission obtained to participate in the review and consents/releases obtained; and
• Key interviews in each case are identified and appointments are scheduled.
14
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Focus Groups and Stakeholder Interviews
Purpose:– Obtain feedback specific to systemic
issues rather than case-specific information; and
– Provide a forum for open dialogue about
challenges and solutions within the system.
15
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Focus Groups and Stakeholder Interviews
Invited to…
Focus Groups:• Caseworkers, Supervisors, Managers, Service
Providers, and Resource Parents
Stakeholder Interviews:• Judges, Attorneys, Police, Probation Officers,
County Commissioners, etc.
16
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Exit Conference – What Happens?
• A few reviewers will be asked to present the highlights from their case review.
• The Local and State Site Leads deliver the aggregate findings from all the cases reviewed along with trends discovered from the focus groups and stakeholder interviews.
• Key strengths and areas needing improvement are identified.
17
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Final Report – What Happens?
• Final Report which includes analysis of demographic information, quantitative analysis of indicator scores, and information from the County Specific Data Packet is shared.
• The county will use the data and analysis to develop a County Improvement Plan.
18
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Next Steps Meeting – What Happens?
• Approximately 6-8 weeks after the onsite review;
• Led by the Local and State Site Leads;
• Lasts approximately two hours;
• Discuss the recommendations for next steps in working toward an action plan for improving frontline practice and system performance; and
• Begins the next step in the CQI process.19
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
What is Your Role?Each pairing of reviewers will:
• Conduct case reviews:
Case file review - overview of the case record;
Interviews - meet with key players involved in the case;
Roll-Up Sheet - demographic information and scoring;
• Provide feedback to the caseworker (and supervisor) - immediate feedback including suggestions and potential next steps for the case;
• Present information gathered from case review and feedback on the family story at the Case-Specific Team Debriefing;
• Complete a QSR Written Case Review Summary; and
• Reviewers may be invited to attend the Exit Conference.
20
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Child/Youth and Family Status Indicators
• Safety: Exposure to Threats of Harm;• Safety: Risk to Self/Others;• Stability;• Living Arrangement;• Permanency;• Physical Health;• Emotional Well-Being;• Early Learning and Development;• Academic Status;• Pathway to Independence; and• Parent or Caregiver Functioning.
21
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Practice Performance Indicators• Engagement Efforts;• Role and Voice;• Teaming;• Cultural Awareness and Responsiveness;• Assessment and Understanding;• Long-Term View;• Child/Youth and Family Planning Process;• Planning for Transitions and Life Adjustments;• Efforts to Timely Permanence;• Intervention Adequacy and Resource Availability;• Maintaining Family Relationships; and• Tracking and Adjusting.
22
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
What Time Period is Considered in Scoring?• Child/Youth and Family Status Indicators
look at the past 30 days– Exceptions are: Stability and Academic Status
• Practice Performance Status Indicators look at the past 90 days– Exceptions are: Planning for Transitions and Life
Adjustments and Efforts to Timely Permanence
NOTE: Recent performance improvements have some influence on scores
23
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
How are Cases Scored?
Each indicator is scored on a scale of 1 – 6• 6 – Optimal• 5 – Substantial Acceptable Range• 4 – Fair• 3 – Marginal• 2 – Poor Unacceptable Range• 1 – Adverse
Maintenance Zone: 6-5Refinement Zone: 4-3
Improvement Zone: 2-1
24
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Distinction Between 3 and 4
The Groundhog Rule
If this case were frozen in time as it is
today, would performance or status be
considered acceptable or unacceptable?
25
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
The Central Scoring RulesRules Include:
• Concentrate on the Focus Measure and Core Concepts in each indicator.
• Read each indicator’s instructions: – Some indicators score results, others score effort or intent.– Not Applicable rating criteria are identified as appropriate.– Score each sub-category within each indicator, as appropriate.
• Stay within the time-based observation windows associated with each indicator.
• Rate indicators based on events that have occurred or conditions that were present within the time-based observation window.
26
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Roll-Up Sheet Purpose
Purpose
• Gather Data and Monitor Practice Model Implementation:
– Demographic information– Case information– Indicator scores
27
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Roll-Up Sheet Completion
• Basic case information is filled out by the caseworker prior to onsite review.
• Remaining information is gathered through case file review and interviews.
• Reviewers will submit a validated Roll-Up Sheet electronically to Site Leads on a flash drive on the final day of onsite review.
28
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Roll-Up Sheet Components
1. General Review Information2. Child/Youth’s Demographic Information3. Child/Youth’s Current Placement4. Child/Youth’s Educational Information5. Case Information6. Child/Youth and Family Status Domain7. Practice Performance Status Domain
29
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Planning QSR Interviews
• Know the case history;
• Plan relevant questions for each respondent;
• Have accurate directions and be punctual; and
• Organize the interview to help the respondent participate.
30
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Prior to the Interview
• Review the interview schedule and begin preparation by identifying potential questions;
• Identify the lead interviewer and notetaker for each interview;
• Remember that you are a reviewer and not a caseworker during the QSR process; and
• Learn something about the interests of the child/youth to better engage him/her.
31
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
During the Interview• Take notes to document the interview;• Take every opportunity to recognize family strengths;• Listen more than talk;• Use the list of QSR questions to keep you on track;• When needed, ask questions on the Roll-Up Sheet;
32
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
During the Interview (continued)
• Always ask about the child’s safety (if safety threats are identified, call site lead immediately);
• Be patient with family stories;• Gather information, do not identify
solutions to the case’s or family’s concerns;
• Develop the relationship before inquiring about sensitive family issues; and
• Avoid leading questions.
33
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
After the Interview
• Do not share information gathered during one interview to the next interview;
• Resist reaching conclusions prior to the completion of all interviews;
• Contact your site lead or next interview if you are running a few minutes behind;
• Note areas needing confirmation or follow-up; and
• Contact your site lead if interviews lead to other key people in the child/youth’s life that you feel would be important to talk to regarding the case review.
34
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
The QSR Interview – Exploring Questions for Providers
For clinicians - What are your goals for this child/youth/family?
What are the child/youth/family’s goals? What are the strengths of this child/youth/family? What worries you most about this case? Tell me what you are working on now. Who do you see as members of this family’s team? What’s the next big hurdle for this child/youth/family? What will success look like? On a scale of 1-10, how likely do you think success will
be achieved? Ask follow-up questions. Where do you see this case in six months?
35
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Interviewing Practice – Sara’s Case
• Both reviewers read the handout, Facts about the Fisher Family.
• Reviewer playing the role of the mother reads the handout, Interview with Cynthia Fisher.
• Reviewer playing the role of reviewer considers, What do you want to learn in the interview with Sara’s mother?
• Conduct the interview.
36
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Child/Youth and Family Indicators• With your partner, review Handouts #6-10.
• Summarize the indicator, score the indicator and identify two good questions for each indicator you are assigned.
Group 1: 5 and 1aGroup 2: 2 and 1bGroup 3: 4 and 5Group 4: 3 and 6Group 5: 7b and 4Group 6: 6 and 9
• Be prepared to join other teams in a group discussion about the scoring experience.
37
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Child/Youth and Family Indicators• With your partner, review Handouts #6-10.
• Summarize the indicator, score the indicator and identify two good questions for each indicator you are assigned.
Group 1: 1a and 9Group 2: 2 and 1bGroup 3: 4 and 5Group 4: 3 and 6Group 5: 7b and 4
Be prepared to join other teams in a group discussion about the scoring experience.
38
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Practice Performance Indicators• With your partner, review Handouts #6-10.
• Summarize the indicator, score the indicator and identify two good questions for each indicator you are assigned.
Group 1: 1a and 5Group 2: 9 and 11Group 3: 2 and 10Group 4: 3 and 7Group 5: 4 and 8Group 6: 1b and 6
• Be prepared to join other teams in a group discussion about the scoring experience.
39
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Practice Performance Indicators• With your partner, review Handouts #6-10.
• Summarize the indicator, score the indicator and identify two good questions for each indicator you are assigned.
Group 1: 1a and 5Group 2: 6, 9, and 11Group 3: 2 and 10Group 4: 1b, 3, and 7Group 5: 4 and 8
• Be prepared to join other teams in a group discussion about the scoring experience.
40
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Caseworker/Supervisor Feedback Session• Thank the caseworker and supervisor for their time.• Review the QSR purpose. • Outline the strengths of the case.• Remind caseworker of the high QSR standards. • Very briefly summarize family story – ensure facts are
correct.• Share new information, if applicable.• Ask caseworker for ideas for next steps.• Discuss the strengths and challenges of the
indicators.• Share reviewer ideas for next steps.• Thank the caseworker/supervisor and emphasize that
they had a meaningful role in making a difference for children, their families and the system.
41
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Feedback Session Practice• Please complete Handout #13, Notes for Providing
Feedback to Caseworkers.• When completed, one partner will be the caseworker; the
other partner, the reviewer.• Practice a feedback session providing the caseworker
feedback on Sara’s case. • Compare your notes with your partner from Handout
#13, did you identify similar:– Strengths/Accomplishments;– Practice Challenges/Limiting Conditions;– Opportunities for Advancing Practice/ Improving Local
Conditions (Next Steps).
42
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Case-Specific Team Debriefing
• Inter-rater reliability check for consistency between case description and scores.
• Identification of themes for feedback to the county and written report.
• Preliminary communication about review findings to leadership.
43
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
Case-Specific Team Debriefing Outline
Case Presentation Outline – 15 minutes• Case Summary (2 minutes)• Child/Youth and Family Status Indicators –
Strengths and Challenges (4 minutes)• Practice Performance Indicators – Strengths
and Challenges (4 minutes)• Next Steps (2 minutes)• Group Questions/Feedback (3 minutes)
44
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
The QSR Written Case Review Summary - Purpose
• Explain the “story” behind the data;
• Provide feedback to the caseworker and local leadership; and
• Be utilized as an effective training tool within the county agency.
45
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
The QSR Written Case Review Summary- Additional Guidelines• Should not tell the case story/history; but rather include
information you gathered to support the ratings of the indicators;
• Written about the ratings and why the factors contributing to the indicators are either favorable or unfavorable;
• Practical and helpful recommendations; and • Make a note of your due date:
Three day Review – Due prior to the conclusion of the 3rd day
Two day Review – Due approximately 7-10 days following the onsite QSR
NOTE: There will be an editing process done by the Site Lead Team, so you may be asked to edit your Written Case Review Summary and/or provide more supportive information. 46
521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center
CQI Continues…
Remember QSR is NOT a one-time event. It’s part of the bigger CQI process that leads to
improved outcomes of Safety, Permanence and Well-Being for children, youth and families.
47