a phase iv meta analytic study of the replicability of motivational enhancement therapy/ cognitive...
Post on 17-Jan-2016
216 Views
Preview:
TRANSCRIPT
A Phase IV Meta Analytic Study of the Replicability of Motivational Enhancement
Therapy/ Cognitive Behavior Therapy for 5 sessions (MET/CBT5) in 36 sites
Michael L. Dennis, Ph.D., Melissa Ives, M.S.W. Chestnut Health Systems, Bloomington, IL
and
Randy Muck, M.Ed.Substance Abuse and Mental Health Services Administration (SAMHSA)
Center for Substance Abuse Treatment (CSAT), Rockville, MD
Joint Meeting on Adolescent Treatment Effectiveness, March 25, 2008, Washington, DC
Background
• In 1997 the third wave of cannabis use was the largest and youngest cohort to date, double the number of adolescents presenting to publicly funded treatment
• There were no publicly available manual guided evidenced based practices targeting this population
• The Cannabis Youth Treatment (CYT) experiments (n=600) were designed to manualize and field test five promising intervention for short term outpatient treatment of adolescent with cannabis (and other) substance use disorders
• Adapted from earlier studies with adult alcohol and cannabis users, Motivational Enhancement Therapy/ Cognitive Behavior Therapy for 5 sessions (MET/CBT5) was the briefest, one of the least expensive, similar in clinical outcomes, and hence one of the most cost-effective approaches evaluated (Dennis et al 2004; French et al 2003).
Effective Adolescent Treatment (EAT)
• From 2003 to 2008 SAMHSA’s Center for Substance Abuse Treatment (CSAT) conducted a phase IV (i.e., post randomization) replication of MET/CBT5 in 36 sites.
• All sites received standardized training, quality assurance and monitoring on their implementation of MET/CBT5, as well as the collection of data with the Global Appraisal of Individual Needs (GAIN) to facilitate comparison with the original CYT study in terms of implementation and outcome.
• The objectives of this presentation are to examine the extent to which the EAT sites were able to: 1. Replicate the implementation of MET/CBT5,2. Address gaps identified in the original study related to
continuing care, family involvement, and mental health services,
3. Replicate or improve on the outcomes.
Location of Sites
AK
AL
ARAZ
CA CO
CT
DC
DE
FL
GA
HI
IA
ID
IL IN
KS KY
LA
MA
MD
ME
MI
MN
MO
MS
MT
NC
ND
NE
NH
NJ
NM
NV
NY
OH
OK
OR
PARI
SC
SD
TN
TX
UTVA
VTWA
WI
WV
WY
CYT: 4 Sites
EAT: 36 Sites
Sample Selection
• The Target Population Inclusion Criteria for including cases from the EAT data set were adolescents who: – Were assigned to MET/CBT in Outpatient and– Reported lifetime abuse or dependence symptoms and– Reported substance use in the last 90 days they were in the
community and– Who were due for follow-up – 4702 of 6150 (76%) meet all inclusion criteria,
• For logistical reasons, and additional 828 (17.6% of 4702) cases were excluded because they did not have a follow-up or were missing the dependent variable at follow-up.
• The final sample size for EAT was 3874 Adolescents with an average time to their last follow-up of 8.0 months.
• This group was compared using GAIN data to a cohort of 202 Adolescents from CYT that met the same criteria with an average time to their last follow-up of 11.4 months.
Demographics
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Male*
AA
White
Hisp
Mixed
Other
10-14
15-17
18-22
Ra
ce G
rou
ps*
Ag
e*
CYT MET/CBT5Outpatient (n=202)
EAT MET/CBT5Outpatient (n=3874)
EAT more likely to be Hispanic (Mixed was not an
option in CYT)
* P<.05
Clinical Characteristics
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Alcohol
Marijuana
Amphetamines
Cocaine, Opioids, Other
Neither
Internalizing only
Both
Externalizing only
None
Unofficial
Arrest/Police Contact
Court/probation
Correctional institution
Pri
ma
ryS
ub
sta
nc
e*
Co
mo
rbid
ity
De
lin
qu
en
cy
Le
ve
l*
CYT MET/CBT5Outpatient (n=202)
EAT MET/CBT5Outpatient (n=3874)
EAT Less Likely to be Presenting for
Cannabis
* P<.05
Comparing CYT and EAT
• In order to compare CYT and EAT, we want to look at both the central tendency (median) and the range.
• In the next several slides we have done this using Tukey Box Plots like the one shown here.
Criteria
Median
Middle 50%
“Range”
In most cases we have scaled the response set relative to the average for MET/CBT5 in CYT
-2.00
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
2.00
2.50
3.00
Records Data on Initiation, Engagement and Continuing Care
-2.00
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
2.00
2.50
CYT EAT CYT EAT CYT EAT
Initiation Engagement Continuing Care
-2.00
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
2.00
2.50
(started within 2 weeks) (6+ weeks & 4+ sessions) (any Tx 91-180 days post intake)
Adolescent Reports on Treatment Received
-2.00
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
2.00
CYT EAT CYT EAT CYT EAT CYT EAT
DirectServices
FamilyServices
ExternalServices
All Services
-2.00
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
2.00
Adolescent Reports on GAIN Treatment Indices
-1.00
-0.50
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
CYT EAT CYT EAT CYT EAT
Substance Abuse Mental Health Physical Health
-1.00
-0.50
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
Comparison of Outcomes
• Evaluate the Increase in the Days of Abstinence from Intake to Last Observation.
• For all time periods days abstinent are adjusted by subtracting any days in a controlled environment during the period (average is less than 5 days).
• Change scores are calculated as last observation minus intake.
• The large sample sizes involved make even trivial differences statistically significant. Thus this comparison focuses as much on clinical significance by using effect sizes.
Within and Between Group Effect Size Calculations
• Effect sizes for within group change reported as: Cohen’s d = (MLast -MIntake )/ Std Dev.Intake
where small=.2, moderate=.4 and large=.8
• Effect sizes for group (G) differences in change scores (Last-Intake) are reported relative to the grand mean (GM) as:
Cohen’s f = (ABS(G(change) –GM(change)))/ Std Dev.(GM Change)
where small=.1, moderate=.2 and large=.4
CYT vs. EAT (f=.09)
0
10
20
30
40
50
60
70
80
90
Intake Last Followup
Day
s of
abs
tine
nce
- ad
just
ed .
CYT MET/CBT5(d=0.26)
EAT MET/CBT5(d=0.47)
EAT Sample size is so large the 95% CI
do not show here
* P<.05**
*
Range of Change Effect Sizes (d) by Site
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
4 CYT Sites (f=0.39)(median within site d=0.29)
36 EAT Sites (f=0.21)(median within site d=0.49)
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
Coh
en’s
d
Change in Days Abstinent by Amount of Continuing Care (f=.15)*
0
10
20
30
40
50
60
70
80
90
Intake* Last Follow-up
Below CYT(<95% CI)(d=0.39)
Similar to CYT(w/in 95% CI)(d=0.55)
Above CYT(>95% CI) (d=0.53)
* P<.05
Change in Days Abstinent by Amount of Family Services (f=.06)
0
10
20
30
40
50
60
70
80
90
Intake* Last Follow-up
Below CYT(<95% CI)(d=0.44)
Similar to CYT(w/in 95% CI)(d=0.53)
Above CYT(> 95% CI) (d=0.52)
* P<.05
Change in Days Abstinent by Amount of Mental Health Treatment (f=.05)
0
10
20
30
40
50
60
70
80
90
Intake* Last Follow-up*
Below CYT (< 95% CI) (d=0.48)
Similar to CYT(w/in 95% CI)(d=0.52)
Above CYT(> 95% CI)(d=0.42)
* P<.05
Change in Days Abstinent by Baseline Days of Abstinence (f=1.63)*
0
10
20
30
40
50
60
70
80
90
Intake* Last Follow-up*
0 days (d=N/A)1-12 days (d=13.16)13-44 days (d=3.45)45+ days (d=0.11)
* P<.05
Change in Days Abstinent by Baseline Days In Controlled Environment (f=.68)*
0
10
20
30
40
50
60
70
80
90
Intake* Last Follow-up*
0 days (d=0.41)1-12 days (d=0.54)13-44 days (d=0.96)45+ days (d=2.98)
* P<.05
Change in Days Abstinent by Gender (f=.11)*
0
10
20
30
40
50
60
70
80
90
Intake Last Follow-up*
Female (d=0.52)Male (d=0.44)
* P<.05
Change in Days Abstinent by Race/Ethnicity (f=.22)*
0
10
20
30
40
50
60
70
80
90
Intake* Last Follow-up
African American (d=0.45)White (d=0.42)Hispanic (d=0.64)Mixed (d=0.46)Other (d=0.23)
Race/Ethnicity:
* P<.05
Change in Days Abstinent by Age Group (f=.15)*
0
10
20
30
40
50
60
70
80
90
Intake* Last Follow-up*
10-15 (d=0.36)15-17 (d=0.5)18-22 (d=0.44)
Age:
* P<.05
Change in Days Abstinent by Most Severe Substance\a (f=.35)*
0
10
20
30
40
50
60
70
80
90
Intake* Last Follow-up*
Alcohol (d=0.2)
Marijuana (d=0.54)
Amphetamines (d=0.61)
Cocaine, Opioids, Other (d=0.56)
* P<.05
\abased on in descending order on most symptoms, frequency, recency, and preference
Change in Days Abstinent by Pattern of Psychiatric Comorbidity (f=.19)*
0
10
20
30
40
50
60
70
80
90
Intake* Last Follow-up*
Neither (d=0.41)
Internalizing Dis.Only (d=0.55)
Both Internalizing& ExternalizingDisorders (d=0.57)
ExternalizingDis. Only (d=0.42)
* P<.05
Adjusted Days Abstinent by Delinquency (f=.20)*
0
10
20
30
40
50
60
70
80
90
Intake* Last Follow-up*
None (d=0.46)
Unofficial delinquency (d=0.55)
Arrest/Police contact (d=0.45)
Court/Probation/Parole(d=0.41)
Correctional Institution (d=0.70)
* P<.05
Limitations
• Primarily relied on adolescent self report (plus some records on implementation). It would have been useful to have collateral or urine reports.
• First cut only examined days of abstinence, it is possible that increases in family treatment and mental health services impact other variables.
• Could have used other ways of adjusting for time in a controlled environment
• Some individual level variables are highly confounded with site (e.g., being Hispanic). Ideally we should (and will) combine site and individual level predictors in a multi-level model.
Conclusions
• On average, the EAT cohort did better than CYT on initiation and continuing care, then equal or slightly lower on most other measures.
• The big difference however was that the sites varied more in their implementation, with some actually providing much more than CYT and others much less.
• On average EAT sites had larger effect sizes than CYT, but in both cohorts there were large variations by site.
• The provision of additional continuing care was associated with larger effects; Increasing family services and mental health treatment was not associated with increased changes in the days abstinent.
• Several individual characteristics were associated with more change, in particular - few baseline days of abstinence, more days in a controlled environment, being Hispanic, having a drug problem, having internalizing disorders, and coming from a correctional institution.
This presentation was supported by the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Substance Abuse Treatment (CSAT) under contracts 207-98-7047, 277-00-6500, 270-2003-00006, and 270-07-0191 using data provided by the following CSAT grantees:
(CYT: TI-11320, TI-11317, TI-11321, TI-11323, TI-11324, EAT: TI-15413, TI-15415, TI-15421, TI-15433, TI-15438, TI-15446, TI-15447, TI-15458, TI-15461, TI-15466, TI-15467, TI-15469, TI-15475, TI-15478, TI-15479, TI-15481, TI-15483, TI-15485, TI-15486, TI-15489, TI-15511, TI-15514, TI-15524, TI-15527, TI-15545, TI-15562, TI-15577, TI-15586, TI-15670, TI-15671, TI-15672, TI-15674, TI-15677, TI-15678, TI-15682, TI-15686). Any opinions about these data are those of the authors and do not reflect official positions of the government or individual grantees. Thanks to Rod Funk, Mark Lipsey, Barth Riley, Michelle White and Ken Winters for their suggestions. Suggestions, comments, and questions can be sent to Dr. Michael Dennis, Chestnut Health Systems, 720 West Chestnut, Bloomington, IL 61701, mdennis@chestnut.org .
Acknowledgements
top related