revising implementation drivers to improve wraparound...
TRANSCRIPT
Revising Implementation Drivers
to improve wraparound fidelity & outcomes
Rosalyn M Bertram PhD
Leia Charnin B.A.
University of Missouri-Kansas City
Pam Schaffer PhD Systems of Hope Al Duchnowski PhD U. South Florida
Wraparound’s value-based principles
Family voice/choice Collaboration
Natural supports Team-based
Individualized Strengths-based
Culturally competent Community-based
Persistence Outcome-based
Wraparound A collaborative model of practice
Defined solely by values = more difficult to apply & support
Model Fidelity
Wraparound is not a form of case management
Wraparound is not a form of family advocacy
Wraparound is:
A collaborative, strengths-based model of practice in which
families act as equal decision-making partners,
guiding team development & assessment,
designing interventions,
evaluating interventions,
and
learning from their outcomes.
Wraparound implementation literature
A National Wraparound Initiative (NWI) review of 21 years of wraparound literature used NIRN frameworks to identify less-examined intervention & implementation components.
These included:
• Target population, theory base, & theory of change
• Organization culture/facilitative administration, decision support data systems, staff selection, training, coaching,
and use of purveyors
Bertram, Suter, Bruns, & O’Rourke (2011), Implementation research and wraparound literature: Building a research agenda. Journal of Child and Family Studies, 20 (6), 713-726.
NIRN Implementation Drivers
Systems of Hope: Participatory evaluation
To support grant sustainability, in year 4 of a 6 year SAMHSA Children’s Mental Health Initiative in Houston TX,
we evaluated wraparound implementation.
Director, supervisors, family members & consultant (n=8) examined previous year implementation patterns
• Results guided revisions to implementation drivers
• Transformed bureaucratic organization culture to a
culture of mutual support & accountability for achieving improved outcomes more efficiently with fidelity.
Evaluation methodology Within & between case analysis of all cases opened the
previous year (n=31).
Youth behaviors of concern, family stories, team actions, care plans & case progress notes were examined in relation to:
• Frequency & focus of family contact
• Timeliness of team development
• Team composition & structure
• Depth & breadth of team assessments
• Focus & means of team interventions
• Frequency & nature of care plan revisions
Demographic Characteristics:
Children Served
20.3
48.8
17.716
38.7
43.6
0
10
20
30
40
50
60
African
American
Caucasian Hispanic/Latino
Perc
enta
ge
National (n = 10371)
SOH (n = 183)
Presenting Problems
Reported
30
56
47
35 33
44
7267
49
59
0
10
20
30
40
50
60
70
80
Adjustment Conduct /
Delinquency
Hyperactivity
& Attention
School
Performance
Depression
Perc
enta
ge
National (n = 9731)
SOH (n = 176)
Findings Six (6) cases reflected wraparound implementation as
described in NWI monographs.
• Teams were well-composed, engaging extended family, friends & other natural supports in contextual assessments & multi-system interventions.
• They achieved goals & revised plans in a step-by-step, timely manner that addressed problem behaviors.
• Most were facilitated by staff members with previous wraparound experience in other settings
Constraining implementation patterns
In twenty-five (25) cases implementation patterns resembled a case management approach that constrained effectiveness, efficiency, fidelity, & grant sustainability.
Limited team composition
• Teams were primarily composed of female caregiver & youth, limiting assessment , design of interventions & use of strengths.
• When family conflict contributed to youth behavior, fathers, step-parents, grandparents & siblings were often not team participants, & those conflicts were seldom addressed in plans.
• Juvenile probation & child protective services were often not represented on teams when the family was legally engaged by these systems.
Constraining Implementation Patterns
Assessments & interventions
• Usually addressed basic needs, but did not clearly address behaviors of concern.
• Goals & interventions were not behaviorally specific & were
infrequently revised.
• Care plans used remarkably similar formal services.
Strengths-based interventions
• Were not behaviorally focused & seldom applied what two or more people did well as basis for the intervention.
• Often described a desire to accomplish broadly stated goals or a desire to obtain a specific service as the basis for change.
Examining Implementation Drivers
Competency drivers: constraining factors Staff selection
• Many changes in grant leadership = different hiring criteria
• No model pertinent criteria for hiring
• Job descriptions resembled case manager positions
• Position responsibilities not clearly differentiated
• Supervisors had no experience coaching staff development
Training
• Changes in grant leadership = selection of differing purveyors
• Value-based training did not inform behavioral assessment
• If hired after a training event, staff did not receive training
• No specific training for supervisors
Competency drivers: constraining factors Coaching
• Seldom regularly scheduled
• Usually focused on risk containment (most problematic case), administrative concerns, & staff member’s emotional or physical response to workload challenges.
Performance assessment
• Data system did not provide model pertinent information
• No staff baseline of model pertinent knowledge/skills
• Wraparound Fidelity Index & required grant measures of youth/family behavioral change were not timely & only provided aggregate assessment across grant
Organization drivers: constraining factors Decision support data system
• Reflected host agency priorities (child protective services)
• Not model pertinent
• Not re-purposed to support wraparound coaching
Facilitative administration
• Care coordinator caseloads were too large for frequent family contact & team activity (N=16-20)
• Care coordinators & parent partners worked together on the same case, but were coached by different supervisors.
• Position responsibilities of care coordinator & parent partner were not clearly differentiated & salaries were similar
Theory-based revisions to wraparound implementation:
The relationship between
core intervention components &
implementation drivers
NIRN Core Intervention Components
• Model Definition (Key elements, participants, activities & phases)
• Theory Bases (Anchor these elements & activities)
• Population Characteristics (behaviors, socio-economics, ages, culture, gender, ethnicity etc)
• Theory of Change how key elements/participants/activities contribute to what changes in population
• Alternative models (why they were rejected)
NIRN Implementation Drivers
Adjustments made by facilitative administration
• Large caseloads compromised family engagement, team composition, assessment & planning, so caseloads were reduced to <8 per care coordinator.
• To avert inefficient, conflicting coaching, care coordinators & parent partners assigned to same supervisor
• Roles & responsibilities of the supervisor, care coordinator & parent partner positions were differentiated & rewritten.
Adjustments made by adaptive leadership
Revisions to Implementation Drivers
Anchor in 2 theory-bases
Ecological systems theory: Family organization & development
• Clarify behavior of concern: Duration, frequency, intensity & context
• Explore events affecting family composition
Team composition & development theory
(Eno-Heineman, 1997; Bertram & Bertram 2004; Walker 2008; Bertram, 2008)
• Differentiate core & extended team
• Team structure via agreements on goals & rules of operation
Ecological systems theory: Assessment & interventions (Henggeler, et al., 2009)
• Fit circle assessment of contributing factors to family accomplishments
• Fit circle assessment of contributing factors to problem/unmet needs
• Specific, strengths-based, step-by-step interventions to change or eliminate factors contributing to well-identified problem.
Revisions to data systems & coaching
From ad hoc risk containment to systematic, data driven staff development.
• Evaluation team reviewed NWI model descriptions to identify wraparound knowledge, skills & aptitude . Supervisors repeat process with staff, establishing proficiencies baseline & individual plans for staff development
• For each case, new model pertinent data submitted weekly to supervisors
• Data shared with implementation consultant & directors
• Bi-weekly Skype consultation & coaching reviews implementation patterns
• Based on implementation consultation, supervisors adjust focus/format of coaching from learning groups to individual, to observations of family engagement or team meetings.
• Manual for coaching through these data developed & distributed
Revisions to training, data systems, & coaching
• Training developed by participatory evaluation team
• Theory bases & new data forms introduced
• Revised coaching formats introduced
20/20 hindsight?
• Given previous staff selection it was too much change in too few days
Our Family Story
Family Composition Name, age, gender, role (mother, step-father, etc.), ethnicity
Events that affected family composition Identify who, what, when, where, duration (Use of timelines helps!)
Describe behaviors of concern at referral Explore with whom, when, where, frequency, intensity, & context
(Use of timelines & fit circles helps!)
Explore & describe family/youth achievements Explore with whom, when, where, & what or who contributed
May be discovered in achievement fit circles drawn from points in timeline
Informal supports Who, with whom, what, duration, in what manner
May be discovered in achievement fit circles drawn from points in timeline
Formal service providers Who, what service, frequency, focus, duration, outcomes
Team Composition Data Form
Date Developed or Revised ___________ Family/Youth ___________________
Care Coordinator ___________________ Parent Partner ___________________
Core Team Members
Those who best know the family situation.
Those who may have significant influence over resources or decisions.
REMINDER When someone new joins the core team, we review & adjust our agreements on goals & rules, and we share & expand our assessment.
Extended Team Members
Those engaged in specific interventions. They provide information & service.
They are not team decision-makers. Team Composition
Name Role Contact Info On Off
Team Structure Data Form
Overall goals based upon behaviors of concern:
– What will situation be at home, school, or in community when team succeeds?
Rules of operation needed to achieve goals
Rules are based upon relationships & responsibilities in the team, and specifically support achieving team goals.
– What information do we need?
– How do we share this information? When? With whom?
– How do we make decisions, especially when we disagree?
– How will we resolve conflicts in our team?
Achievement Fit Circle
Better school behavior & grades
result in no suspensions
Mom & grandma develop chore chart for each boy. Rewards & consequences are clear & consistent
Teacher & youth agree on signal if he feels unfairly treated. Rewards if no outburst.
Mom works 2 jobs but use of chore chart & school chart has improved quality of family time
Mom has new boyfriend but chore chart & teacher chart of behavior & homework help her monitor
Youth engaged in church music group
Improved social skills with mentor & developed pro-social friends.
Family caregivers are more consistent in monitor, guidance & discipline.
Strengths Assessment Data Form • Strengths are behaviors, what 2 or more people do together
• Strengths are not personality characteristics, hopes or desires
• What family members do & what others do ---- family achievements
• These contributing factors are meaningful, useful strengths for interventions
• Develop achievement fit circles frequently with family & team
• Transfer identified strengths to this form
• Note when team uses them as basis for interventions.
Family/Youth _____ Care Coordinator ____Parent Partner ___ Revisions date _______
Date Identified Constraints Date Targeted in Intervention
Youth Family Peers School Community
Problem Fit Circle
Youth needs to improve school
behavior & academics so there are no more
suspensions
Explosive when he feels teacher treats him unfairly.
Talks back when grandma tries to help mom by directing him to clean his twin brother’s messes
Mom works 2 jobs = little time or patience for monitoring, guiding behavior & academic performance
Mom has new boyfriend she met at work. This draws her attention away from youth performance.
Peers pick on him daily, talk about his family.
Family conflicts spill into school setting. Mom favors him. Grandmother favors twin brother.
Poor social skills, has few pro-social friends.
Constraints Assessment Data Form
• Patterns of interaction in family or between family members & others often contribute to behaviors of concern
• We eliminate or diminish a contributing factor with strengths-based interventions designed by the family & team
• Regularly develop problem fit circles with family & team
• Transfer contributing factors as they are identified & when interventions target them to this form.
Date Identified Constraints Date Targeted in
Intervention
Youth Family Peers School Community
Smart Interventions Data Form
Behavior of Concern
Targeted contributing factors
Specific strengths used in intervention
Intervention Date and Evaluation of Intervention & Outcomes
Center of problem fit circle
Surround problem fit circles & in constraints data form
From strengths fit circles & in strengths data form
Who does what with whom, when, & in what manner? Who evaluates intervention & how frequently?
Outcomes?
Implemented as intended?
If not, what constrained? Transfer lessons to data forms
Date___ Family/Youth______Care Coordinator ____ Parent Partner ___
Systematic data-informed coaching develops a culture of support & accountability
Evaluation of revised wraparound implementation
Implementation patterns emerging after one year
Improved team development, assessments & interventions
• Expanded & differentiated team composition emerging
• With specific goals & rule agreements & use of fit circles, team assessment was becoming more robust.
(confirmation of both theory bases)
• Families embrace fit circles & focus on parenting style
• Strengths of 2 or more becoming basis for interventions
(theory bases’ activities supporting value-based principles)
• Greater complexity in design of interventions (E.g., Use of parental rewards & consequences integrated with school interventions)
• More effective interventions have duration of <6 weeks
Patterns when less fidelity, efficiency, or effectiveness
• Staff in multi-funded positions are shaped by different expectations
They adjust to implementation revisions more slowly
• Over-reliance on use of professional services vs. natural supports
Constraints in team development, assessments, & interventions
Confirmation of theory bases
• If goals & rules were not well specified, or when team composition was limited, assessment & planning resembled case management
• Male caregivers or extended family not well-engaged
• Singular strengths-base of interventions (usually youth strength)
• Hesitance to focus on changing parenting style
• Interventions last >6weeks & are less effective
Emerging trends in coaching staff development Confirmation of theory bases
Staff move from youth focus to family/community focus via coaching
As they do, assessment becomes more robust & interventions are more step-by-step toward transition
Confirmation of theory bases
Staff & team use of fit circle assessment ---> improved interventions
• Strengths of 2 or more people become basis for interventions
• More complex design of interventions
(e.g; school and home interventions that complement each other)
• Interventions of shorter duration move family to transition
After 18 months: Intervention component lessons
Model definition & dual theory bases
• Constrained by previous staff selection
• Some staff resisted revisions by using value-based principle of family voice/choice to remain in their “comfort zone”
• However, revisions to training, coaching, & data forms were integrated & anchored in two theory bases
• Performance assessment via data review in Skype consultation with administrators & supervisors ----> coaching consistency & adaptations
Resistance gradually diminished as both supervisors & staff experienced data as support for improved competence & confidence.
Competency & leadership driver lessons
Coaching & adaptive leadership
• At first, the manual for coaching through revised case data forms was read once & not consistently applied.
• Although we revised implementation via participatory evaluation, supervisors initially required individual coaching as did many staff.
Data system & performance assessment
• Comparison of frequency/type of contacts with weekly submission of model pertinent information (revised data forms) more efficiently identified staff difficulties or strengths.
• Isomorphs between consultant & supervisor translated to similar coaching patterns between supervisor & staff.
Organization culture lessons Changing from bureaucratic culture to a culture of multi-level support &
accountability is an uneven process
Weekly case data was initially understood as bureaucratic requirement
Language is important: “documentation” vs. “current case data so we can help you be more confident & effective”
Organization culture changes s-l-o-w-l-y…
No single implementation driver changes organization culture
However, simultaneous theory-based revisions to training, coaching, & data forms, integrated via adaptive leadership of administrators, consultant, & supervisors changed organization culture & climate
Demographics, presenting problems & outcomes
Demographic Characteristics:
Children Served
20.3
48.8
17.7
43.6
16
38.7
0
10
20
30
40
50
60
African
American
Caucasian Hispanic/Latino
Perc
enta
ge
National (n = 10371)
SOH (n = 183)
Presenting Problems
Reported
30
5647
35 33
44
7267
4959
0
10
20
30
40
50
60
70
80
Adjustmen
t
Cond
uct /
Delinqu
ency
Hyper
activ
ity &
Atte
ntion
Scho
ol Perform
ance
Depress
ion
Perc
enta
ge
National (n = 9731)
SOH (n = 176)
Social Problems: Child Behavioral Checklist (CBCL 6-18)
Caregiver Strain *p ≤ 0.05
6259
7074
0
10
20
30
40
50
60
70
80
*Baseline to Exit (n = 54) *Baseline to Follow-Up (n =
20)
Care
giv
er
Str
ain
Quest
ionnaire S
core
Caregiver Strain Questionnaire
WFI-4 Scores by Role * p ≤ 0.05
74
83
62
70
8184 84 82 84 87
7783
75 7378
0
10
20
30
40
50
60
70
80
90
100
*Combined
Respondents
Wraparound
Facilitator
*Caregivers *Youth Team
Member
Fid
elity
Perc
enta
ge
SOH 2010
SOH 2011
NationalMean
Pre and Post Implementation WFI-4 Scores by Principle *p ≤ 0.05
83
72
64
85
71
83
74
67
78
63
92
83
71
8882
0
10
20
30
40
50
60
70
80
90
100
*Family
Voice &
Choice
Team Based Natural
Supports
Collaboration*Community
Based
Fid
elity
Perc
enta
ge
SOH 2010
SOH 2011
National Mean
Pre and Post Implementation WFI-4 Scores by Principle *p ≤ 0.05
92
66
83 82
66
98
75
9187
77
91
69
83 82
67
0
10
20
30
40
50
60
70
80
90
100
*Culturally
Competent
*Individualized Strengths
Based
Persistence Outcome
Based
Fid
elity
Perc
enta
ge
SOH 2010
SOH 2011
National Mean
National (n = 134) SOH (n = 44)
Actions Intake 6 Months Intake 6 Months
Suspended 32.8% 26.1% 54.5% 31.8%
Expelled 0.0% 0.0% 4.5% 0.0%
Neither Suspended Nor Expelled
64.9% 69.4% 38.6% 63.6%
School Disciplinary Actions 2011
Implications Fidelity is more difficult to achieve when implementation & consultation focus
solely through wraparound’s value-based principles
Two theory bases should be engaged in training, coaching, & data support
(Malysiak, 1997, 1998; Bertram & Bertram, 2004; Walker 2008;
Bertram, Suter, Bruns & O’Rourke, 2011)
Limitations of staff selection may be compensated for via integrated training, coaching, & data support anchored in theory bases
Integrated theory-based revisions to training, coaching & data support improve wraparound fidelity, efficiency, & outcomes
Facilitative administration in adaptive leadership with consultant & staff is key
Over 18 months, these persistent adjustments improved fidelity, efficiency & outcomes while changing organization culture & climate
Thanks to:
Randy Joiner LMSW-AP
Executive Director Systems of Hope 2010-2011
DePelchin Children's Center Evaluation Team
Stacey Clettenberg, PhD. Director of Research
Raquel Runge, PhD, Lead Evaluator and Project Manager
Michael Santana, MPA, Project Manager
Jason Lau, BA, Senior Evaluator
Michelle Ibarra, MA, Evaluator
Aisha Dickerson, MSPH Evaluator
Eileene Chappelle, BA, Lead Family Evaluation Partner
Julie Bourne, Family Member