monitoring and increasing staff fidelity:...
TRANSCRIPT
A N N A B R A D Y & R Y A N P A S K I N S
MONITORING AND INCREASING STAFF FIDELITY: TOOLS & TIPS
What is Treatment Integrity?
Why It Matters?
What does it
look like?
Practice
Useful Tips
OBJECTIVES
1. Learn how to use the Performance-Diagnostic Checklist-Human Services (PDC-HS) as well as other fidelity measures to assess staff fidelity
1. Learn to use the PDC-HS to determine possible interventions to increase fidelity
1. Learn key components to developing their own fidelity checklists for academic and behavioral interventions
1. Learn the importance of treatment fidelity
LINKS TO TIERS OF SUPPORT
• For students with IEPs, 504, and/or BIPS
• But, tx fidelity applies to all types of educational programming.
DEFINITIONS
• Treatment adherence, treatment compliance, procedural reliability,
Procedural or Treatment: Having to do with procedures, methodology, or treatments
• “faithfulness”, “obedience”, “constancy”, “accuracy”, “exactness”, “precision”
Fidelity: “The degree of exactness with which something is copied or reproduced”
• “coherence”, “unity”, “ethics”, “honor”, “togetherness”
Integrity: “The quality of being honest and having strong moral principles”, or “the state of being whole and undivided”
TREATMENT INTEGRITY
“The extent to which procedures are implemented according to the behavior plan” (Vollmer et al., 2008)
“The reliability of the administration of an intervention” (Hinckley & Douglas, 2013)
“Treatments or interventions are delivered as prescribed or intended” (Gresham, 2009)
WHY DOES IT MATTER?
But…..“We’re not conducting research here, why does it matter?”Life changing decisions based on the assumption the data
reported are valid, and the procedures are being implemented correctly
EXAMPLE
EXAMPLE
LIFE CHANGING DECISIONS BASED UPON OUR INTERVENTION OUTCOMES
• Placement decisions• More restrictive interventions or settings• Changes to psychotropic medication• Using restrictive or labor intensive staffing
WHAT DOES IT LOOK LIKE?MONITORING TREATMENT FIDELITY
1. Observe intervention implementation
2. Score correct and incorrect implementation
3. Provide an evaluation or summary to therapist
4. Use the data
BENEFITS OF MONITORING? FEEDBACK OPPORTUNITIES
Positive feedback for correct implementation
Corrective feedback for incorrect implementation
BENEFITS OF MONITORING
• Ensure high ratio of positive feedback, but corrective feedback is also important
• Long term feedback from trends over time• Provide reinforcement for long term results• Identify problems with intervention
• Definitions, unclear procedures, etc.
WHAT WE ARE LOOKING FOR?
Errors
Correct Implementation
TYPES OF ERRORS
• Omission: Missed opportunity• Do not provide appropriate response when called for
• Commission: Incorrect action• Provide response at inappropriate time
ERROR OF COMMISSION EXAMPLE
• Procedure: Intervention-Extinction for Attention Maintained Self-Biting
• Procedures: • Therapist: Ignore all instances of self-biting• Following self-biting: Do not touch, took at, or talk to the
client
ERRORS OF COMMISSION
• Video…
ERRORS OF OMISSION
• Procedure: Assessment- Control condition of an FA• Procedures:
• Therapist: Provide praise and attention at least every 10 s, do not ask any questions or give any instructions, ignore instances of problem behavior
ERRORS OF OMISSION
• Video…
MEASURING INTEGRITY
Percentage of opportunities for correct implementationScore each component of an intervention
individuallyConduct brief monitoring sessions (e.g. 10 min
observation)Calculate (# Correct Responses) / (# Correct +
Incorrect)= _______*100 = Correct %
TREATMENT FIDELITY MONITORING: CREATING DATA SHEETS
Record useful information: date, therapist, student, who observed, duration of observationInclude all elements of the interventionWhat should the therapist do?What should the therapist not do?
Provide easy calculation tool on the sheetCan include space for notes, jot down positive
feedback and praise, track something interesting, etc.
EXAMPLE TREATMENT INTEGRITY SHEETS
PRACTICE CREATING TREATMENT FIDELITY SHEET
Look at example procedures on
board
Use blank template and fill in relevant
therapist procedures to track
BLANK TEMPLATE
Therapist provides attention every 30 s
EXAMPLE PROCEDURES
Tangible Condition of Functional AnalysisTherapist Procedures:The therapist will allow pre-session access to the toys. The
session will begin by the therapist removing the preferred item. The therapist will prevent the client from touching preferred items after that point. During the session the therapist will provide attention whenever solicited by the client, or provide free attention at least every 30 s. If the client engages in target problem behavior, the therapist will provide the high preferred item for 15 s. At the end of 15 s, the therapist will remove the preferred item again. The therapist will ignore all other responses exhibited by the client (e.g. non-targeted problem behavior).
CHECK IN
PRACTICE USING YOUR DATA SHEET
• Video…
FACTORS INFLUENCING ERRORS
Training of procedures
Complexity of procedures
Lack of generalization
Implementer ‘drift’
Competing contingencies
TRAINING
Do’s• Ensure clear procedures are outlined and plan elements
are operationalized• Ensure adequate trainings including opportunity to
practice skills• Check for understanding• Have therapist practice procedures while trainer provides
corrective feedback (i.e. competency based training)• Train to a pre-determined criteria (using fidelity checklist)
Don’ts• Therapist never demonstrated accurate implementation• Therapist just handed protocol to read and implement• Assume, “they’ve done it before they can do it again”
COMPLEXITY
Do’s•Ensure instructions clear, detailed, manageable•Use language therapists will understand•Use friendly formatting (e.g., bullets, titles, headings,
sections, etc.)•Create the protocol and fidelity document
simultaneously•Check in with therapists and edit protocol if needed
Don’ts•Write narrative protocol with pages and pages of
text•Use jargon and complicated terms in procedures•Create complicated unmanageable procedures
GENERALIZATION
Do’s• Train multiple examples• Train in different settings
Don’ts• Train in only one setting or
context• Train using limited examples
and skills
THERAPIST DRIFT
Do’s• Conduct frequent treatment fidelity
monitoring• Schedule monitoring on a regular basis
from the onset of treatment• Conduct booster training sessions
Don’ts• Support therapists modifying
procedures or drifting from original plan
• Assume initial competence will maintain without support and monitoring
COMPETING CONTINGENCIES
Do’s•Praise and reinforce accuracy of
implementation regardless of behavior outcomes
•Be aware of environmental barriers and attempt to solve them
Don’ts•Emphasize priority of behavior
change over priority of accurate implementation
•Don’t praise effects of intervention
PROCESS
Create clear intervention procedures
Prior to implementation
, conduct competency
based trainings for therapists
Create comprehensive
fidelity monitoring data
sheet
Plan and schedule regular
treatment fidelity
monitoring
Provide immediate praise and corrective feedback
Provide praise and incentives for long term
treatment fidelity
TREATMENT FIDELITY DATA
Use your data to inform decision-makingProvide feedback to therapistsInformation regarding which therapists need more
trainingInformation regarding which interventions are most
difficultExamine fidelity and behavior data together for
patterns (e.g., low fidelity may result in more problem behavior)
PDC-HS
PDC-HS
PDC-HS
PDC-HS
TAKE AWAY
Our interventions influence student lives and decision making
Interventions must be implemented accurately
We must monitor implementation to ensure continued implementation fidelity
REFERENCES
• Carr, J. E., Wilder, D. A., Majdalany, L., Mathisen, D., & Strain, L. A. (2013). An assessment-based solution to a human-service employee performance problem. An initial evaluation of the Performance Diagnostic Checklist-Human Services. Behavior Analysis in Practice, 6, 16-32.
• Gresham, F. M. (2009). Evolution of the treatment integrity concept: Current status and future directions. School Psychology Review, 38(4), 533-541.
• Hinckley, J. J., & Douglas, N. F. (2013). Treatment fidelity: Its importance and reported frequency in aphasia treatment studies. American Journal of Speech-Language Pathology, 22(2), S279-S284.
• Vollmer, T. R., Sloman, K. N., & Pipkin, C. S. P. (2008). Practical implications of data reliability and treatment integrity monitoring. Behavior Analysis in Practice, 1(2), 4.
REFERENCES
