Toolkits

Deni Carise, Ph.D.Tom McLellan, Ph.D.Adam Brooks, Ph.D.Robert Forman, Ph.D

Developing a Clinician Resource for Evidence-Based Treatment Delivery

Supported by NIDA Grant (R21DA-015977)

The Research-Practice Gap

• 25 years of heavy research investment in:

–Medication Development

–Behavioral Therapies

• Numerous treatments have:–Demonstrated efficacy in clinical trials

–Demonstrated effectiveness in dissemination studies

The Research-Practice Gap• Research shows very low rates of adoption of

these evidence-based practices

• Possible factors leading to non-adoption:

– Financial

– Complexity of interventions

– Provider organizational and policy concerns

The Research-Practice Gap• We interviewed counselors and program

directors in the Delaware Valley Research Practice Collaborative (PA, DE, NJ)

• Initial Focus Groups revealed two barriers as greatest contributor to the “gap”:– Finance– Training/Supervision

New therapies won’t be adopted if:

– the treatments cannot be supported by managers and funding agencies

– the training and supervision burdens of the treatments are overwhelming

The Research-Practice Gap

The Financial Barrier• It is a challenge to implement evidence-based treatment practices within the severely challenged infrastructure*

•Addiction treatment system has experienced a 20-year period of declining funds:**

*McLellan, Carise and Kleber, 2003.**Mark, Levit, et. al. 2007.

The Financial Barrier• General healthcare funding

declined 12% between 1988 and 1998*

• During the same period, addiction treatment funding declined 75%*

*Galanter, Keller et. al. 2000.

Financial Barrier ConsequencesProgram closures or re-organizations

• Increased counselor and management turn-over*

• In a survey of 450 treatment programs:

• 32% faced the threat of closure

• 5% actually closed during the year of the survey**

*Gallon, Gabriel et. a. 2003; McLellan, Carise et al. 2003. Knudsen, Ducharme et. al. 2004. **Roman, Blum 1997.

Financial Barrier Outcomes

Increased Reliance on Group Therapy

• Reimbursement for individual therapy is under-funded or not funded

• Offered as primary treatment modality

Group Psychotherapy

• Can be as effective as individual therapy*

• Are a top priority for bridging the gap

• Evidence-based approaches have often not been adapted for group treatment

*Weiss, Jaffee et al. 2004.

Supervision Barrier

•Research shows that clinical supervisors:

• spend less time mentoring and training• spend more time addressing human

resource or regulatory compliance issues• often have ongoing case loads of their

own

Training Barrier• Of 400 national substance abuse treatment centers, 20% had no staff training budget*

Decreased funding

++ Increased licensing,

accrediting, and funding compliance

requirements

Decreased clinical training

andIncreased attention to administrative issues

*Johnson 2000.

Training Demands of EBPs• Research on Training EBPs shows:

– Manuals (if read) are useful for learning interventions, but not enough to change counselor practice*– To be proficient in conducting new interventions, training needs to be ongoing:

•Requires more time than a standard 2-3 day workshop•Requires ongoing, expert supervision

*McCarty, Fuller et al. 2007

The Big Problem•We have complex treatments

– Content heavy

– Require intensive training

– Proven efficacy when delivered correctly

•We have a resource starved environment

•We have heavy turn-over in the field

Looking to Other Fields

• The field of education has experienced similar problems– Minimal resources

– Stressed workforce

– Complicated interventions (lessons) with little time to prepare

Possible Solution

Education field Toolkits -

An evidence-based approach to addressing the challenge of upgrading

skills in a resource restricted environment

Possible Solution

•Teachers use packaged lesson plans to assist teachers in conveying complex concepts

•They can be taught effectively, require little supervision and have been constructed, tested and refined through a scientific process

Possible Solution: Examples

Possible Solution: Examples

Possible Solution: Examples

Existing Treatment Curricula

• Currently, there are some existing prepackaged curricula for substance treatment

•Vary to the degree that they are informed by evidence based content

• Few, if any, have been rigorously tested to assess their impact on treatment or outcomes

Testing a Curriculum Sample

• We conducted significant background work to determine provider interest in a curriculum-based approach

• We developed and pilot-tested a single-session curriculum “Toolkit”

Surveying Interest

•21-item survey designed for the Clinical Trials Network projects called: “What Do You Need?”

•Administered in 18 community-based treatment programs in the Delaware Valley Practice Research Collaborative (PA, NJ, DE)

Surveying Interest

•269 treatment program staff completed the survey

•195 employees identified themselves as “treatment providers”

–Other options: researcher, managed care or faith-based organizations, support staff, or consumers were not included in the analyses (n=74)

Surveying Interest

Self-report survey included:

•Demographic questions: Age, gender, ethnicity, education, certification, years in the field

•Forced choice of preferred Training Topics (n=26)

Sample Demographics

Overall demographics:

•Age (Mean=42; SD=11)

•Years of Experience (Mean=9; SD=7)

•Gender: Female = 57%

•Ethnicity: Caucasian = 71%, African American = 22%, Hispanic/Latino = 3%, Other = 4%

Analyses 1•Which Topics were most frequently endorsed?•Was endorsement related to counselor background?

–Certification was defined as having any of the following certifications:

•Certified Drug/Alcohol Counselor (PCACB, PCB, etc)•ASAM Certification or APA Qualified in Addiction Medicine/Psychiatry•Licensed Health Professional•Ordained Clergy

–Experience was defined as having been in the field for at least 6 years (Range: 0-35 yrs)

Analyses 2Years of Experience and Certification were examined,

creating 4 groups:

Note: Participants without both Experience and Certification data were omitted from analysis (n=25; 14%)

Uncertified Certified

Inexperienced (n=34, 17%) (n=51, 26%)

Experienced (n=61, 31%) (n=24, 12%)

Results 1Percent Endorsed: Training Topic

62 -Increasing Client Motivation 17 -Obtaining Project Funding 49 -Relapse Prevention Techniques 17 -Finding Treatment Information 47 -Co-Occurring Disorders 15 -Harm Reduction Strategies 47 -Group Therapy Techniques 15 -Applying Research in Practice 45 -Spirituality & Recovery 14 -Age Specific Treatment 27 -12-Step Oriented Approaches 14 -Sexuality Related Treatment 25 -Ethnicity & Diversity 12 -Documentation Software 25 -Biological Basis of Addiction 11 -Tobacco Addiction 21 -Addictions Medications 11 -Use of Incentives & Rewards 19 -Sexual Addiction 11 -Internet Addiction Resources 18 -Gender Specific Treatment 9 -Research & Evaluation Methods 18 -Alternative Therapies 5 -Using Research Databases

4 -Statistics & Research FindingsMost frequently endorsedResearch-relatedTopics analyzed

Results 2Training Topic Preferences by Experience and

Certification:

Note: 1 = Main Effect (ME) for Experience, 2 = ME for Certification, 3 = Interaction, at p < .05Note: Participants can choose more than one categoryBinary Logistic Regression

% Inexperienced & Uncertified

(n=34)

% Inexperienced & Certified

(n=51)

% Experienced & Uncertified

(n=61)

% Experienced

& Certified (n=24)

Increasing Client Motivation 68 65 57 58Relapse Prevention 56 59 36 50Co-Occurring Disorders 1 44 31 56 58Group Therapy Techniques 3 59 49 33 58

Results 3Training Topic Preferences by Experience and

Certification:

Note: 1 = Main Effect (ME) for Experience, 2 = ME for Certification, 3 = Interaction, at p < .05Note: Participants can choose more than one categoryBinary Logistic Regression

% Inexperienced & Uncertified

(n=34)

% Inexperienced & Certified

(n=51)

% Experienced & Uncertified

(n=61)

% Experienced & Certified (n=24)

Spirituality & Recovery 32 49 48 5012-Step Oriented Approaches 1 27 39 20 17

Addiction Medications 1,2 35 18 23 4Applying Research in Clinical Practice

21 10 13 25

Results 4

•Out of the 26 possible Training Topics, the majority of counselors chose the same 5, making up 45% of the responses

•These “top 5” topics were most related to the treatment counselors were already providing

Survey: “Staff Beliefs about Addiction Treatment and Clinical Trials”

NIDA Clinical Trials Network (CTN) – DE Valley22-items assessing 317 counselors’:• Beliefs about addiction treatment• Willingness to try new treatment approaches

– Endorsed by 80% of surveyed counselors– Willingness to try research-based innovations– Endorsed by 82% of surveyed counselors

129 counselors were asked about their clinic’s videotape usage for group sessions:

• 100% used videotapes with patients

• 97% agreed videotapes were “a useful aid in educating patients about addiction and recovery”

Survey: Videotape Use in Addiction Treatment Settings

• 96% wanted “a library of up-to-date, scientifically accurate videotapes...”

• 63% agreed only showing “brief sections of videotapes with a discussion following them” was best

• 58% were dissatisfied with their clinic’s current education tools

Survey: Videotape Use in Addiction Treatment Settings

Toolkit Development• We incorporated provider feedback into

the development of a single “Toolkit”

• Goal was to:– assess provider satisfaction

– assess client satisfaction

– assess continued provider use of the Toolkit

Step 1 – Choosing Content

Examples: Relapse preventionHIV risk reductionBiology of Addiction MedicationsMotivational Interviewing

Toolkit Prototype – MI Component - Decisional Balance

Developing the Toolkit

Step 2: Designing the Toolkit

• Multiple formats (Digital & Hand-Outs)

• Created Toolkit prototype

Developing the Toolkit

“Toolbox”

Clinician Guide

DVD

Worksheet

Wallet Cards

Components:

Toolkit Components

DVD

Wallet Card

Worksheet Clinician’s Guide

Toolkit Video

Good Things and Not So Good Things about Recovery

Counselors

- 26 Counselors from 6 clinics

- Experienced Counselors (> 10 years)Procedures

- Counselors consented & oriented

- Use the Toolkit in Group & Complete Clinician survey

- One month later - Asked to use 2nd time

- Next 3 months: Record use of Toolkit components

Methods - Counselors

Group Members/PatientsN=230

Procedures- Consented prior to group- Attended Toolkit group - Completed Patient

Satisfaction Survey

Methods - Patients

• Will clinicians be able to use the Toolkit after only a brief orientation?

• Will clinicians continue to use it, even when not required?

• How will patients evaluate the toolkit group?

Research Questions

N=26 (counselors)

Will Clinicians Be Able To Use It After Only a Brief Orientation?

  Mean (SD)

Was the Toolkit orientation adequate? 4.5 0.8

Were you satisfied with the Toolkit group? 4.2 0.7

Was the Toolkit "User-Friendly?" 4.4 0.7

Do you think other Clinicians would like to use the Toolkit? 4.4 0.7

     

Scale:    

1=Not at all, 2=Somewhat, 3=Moderately, 4=Quite a Bit, 5=Very Much    

Counselor Ratings

0

2

4

6

8

10

12

14

Not at all Somewhat Moderately Quite a bit Very Much

CounselorSatisfactionIt's user friendly

Recommend it toother clinicians

Patient Satisfaction

0

20

40

60

80

100

Not At All A Little Moderately Quite a Bit Very Much

Was this session helpful?

Did you benefit fromattending the group?

• 96% of counselors have reused at least one component of the Toolkit

• On at least one occasion, 63% of clinicians used all 5 components of the Toolkit as directed

• The most frequently and broadly used were the core components:

• Counselor Guide & Patient Worksheet

Will Clinicians Continue To Use It After the Study Ends?

Over a 3-month tracking period:

• Counselor Guide: 83% used it again (M = 2.5 times)

• Patient Worksheet: 96% used it again (M = 3.2 times)

Will Clinicians Continue To Use It After the Study Ends?

Toolkit component

Clip-

boardDVD

Work-

sheetGuide

Wallet Cards

Mean (sd) 5 (9) 1 (1) 3 (3) 3 (3) 2 (2)

Range 1 - 42 1 - 3 1 - 12 1 - 12 1 - 9

Times Used Toolkit Components

Preliminary results in community-based settings support the Toolkit’s:

• feasibility• acceptability • sustainability

Conclusions

Future Directions

• Will curriculum impact clinician behavior in treatment?

• Will curriculum beneficially impact client outcomes?

Translating a Complete Treatment

• We will translate a full treatment intervention into a Toolkit format

• Starting with 6 Toolkits on Relapse Prevention Techniques– Content and didactic heavy– Popularly selected by providers– Evidence Based – Useful in group treatment

Testing a Complete Curriculum

In a larger sample, we will randomly assign sites to receive:

6 training manualsOR

6 training manuals +6 Toolkits Curriculum Supplements

Future Directions, Cont’d

• Toolkits for other evidence-based treatments

(12-step, etc)

• Toolkits to address comorbidity, HIV prevention, etc

• Arming clinicians with enough evidence-based content to fill 50-75% of group sessions

Future Directions

Your Thoughts?