class 13 and 14 jacobson et al (1996) apa (2006)evidence based practice in psychology 1
TRANSCRIPT
2
Experimental Between Group Designs
1. Post-Test Only Control
2. Pre-Test -- Post-Test Control3. Solomon Four Group (combination of 1 and 2 above)
Factorial Design more than one independent variable; interactions gender x treatment
Dependent Sample Design (Matching)
2
3
Experimental Between Group Designs
Post-Test Only Control Pre-Test -- Post-Test Control Solomon Four Group
Factorial Design (Treatment X Therapist) Dependent Sample Design (Matching)
# of previous episodes and severity depression, presence of dysthimia, gender, marital stauts
3
Types Outcome Studies Kazdin (chap 18)
1. Treatment Package Strategy
2. Dismantling Strategy
3. Constructive Strategy
4. Parametric Strategy (structural components)
5. Common factors Control Group
6. Comparative Outcome Strategy
7. Client and Therapist Variation Strategy Moderation Designs
Types Outcome Studies Kazdin (chap 18)
1. Treatment Package Strategy2. Dismantling Strategy
What are the active ingredients ?1. Constructive Strategy 2. Parametric Strategy (structural components)3. Common factors Control Group4. Comparative Outcome Strategy5. Client and Therapist Variation Strategy
Moderation Designs
Research FocusHow treatment effects change:
identify change mechanisms
VS.
How well treatment works
6
Change Mechanisms Activation Hypotheses ( BA) (activation)
Change behaviors- become active and access sources of reinforcement- occurs early in therapy
Coping Skills Hypotheses (AT) Learn effective coping strategies
(coping + activation)
Beck Hypotheses (CT) Change cognitive structures or core schemas
(cognitive schemas + activation + coping) 7
Chapter 7 8
Treatment Groups BA Condition: Behavioral Activation Hypotheses:
Behavioral Activation
AT Condition: Coping Skills Hypotheses Behavioral Activation Coping Skills - Automatic Thoughts
CT Condition: Cognitive Therapy Hypotheses Behavioral Activation Automatic Thoughts Core Schemas
8
Chapter 7 9
Mechanisms of Change Construct Measure
Behavioral Activation
Dysfunctional Thinking
Cognitive Schemas
Pleasant Events Schedule
Automatic Thoughts Q.
Expanded Attributional Styles
How did the authors examine if the mechanisms of change worked in therapy as predicted by theory?
9
How Treatment Effects Change
To what extent treatment groups differed in post-test measures of each change mechanism
Measure Ttreat. G Behavioral activation – Pleasant Event SCd BA Coping skills- - AT (Dist Thinking) AT Core schemas - Exp Attrb Style CT
To what extent was there change from pre-to post- treatment in each change mechanism
10
How Well Treatments Work Which of the three treatment conditions
yielded better outcomes regarding Depression at termination and at 6 month follow up? HRSD BDI Recovered Improved Rates
11
Research Questions Analyses
Treatment Outcomegroup differences Depression
Type of Analyses
CVsIVSDVs
Mechanism of Change:Treatment groups differed in post-test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas)
Type of Analyses
CVsIVSDVs
Mechanism of Change:Treatment groups change from pre-to post- treatment in each change mechanism (BA – AT-CS)
Type and # of Analyses
12
Research Questions Analyses
Treatment Outcomegroup differences in Depression
3 x 4 MANCOVACVs Pre-test BDI HRSDIVs Treatments: CT, AT, BA Therapists (n=4) Treatment X TherapistDVs Post-Test BDI HRSD
Mechanism of Change:Treatment groups differed in post-test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas)
Mechanism of Change:Treatment groups change from pre-to post- treatment in each change mechanism (BA – AT-CS)
13
How Well Treatments Work Which of the three treatment conditions
yielded better Depression outcomes at termination and at 6 month follow up?3 X 4 MANCOVA
IVS
3 Treatments (CT, AT, BA)
4 Therapists
Treatment X Therapist (Interaction)
DVs - Depression
HRSD – Clinician
BDI - Self- Report
14
Which group to use in outcome analyses
Group SessionsIntent-to treat Total sample
n = 149
Dropouts (not included in MANCOVA analyses; p. 299)
1<Sessions <12 n = 12
Completers At least 12/20 sessions n = 137
Maximum Completers 20 sessions n = 129
15
Results MANCOVAsTreatment Outcome: Post- Test Depression
Main Effect Treatment ?
Main Effect Therapist ?
Treatment X Therapist Interaction ?
Results p. 299 -- ANOVAs Table 3
16
Results MANCOVAs Treatment Outcome: Post- Test BID & HRSD Treatment Group N= 149 137 129
Main Effect Treatment NS NS NS
Main Effect Therapist NS NS NS
Treatment X Therapist NS NS NS
Results p. 299 -- ANOVAs Table 3
17
19
ANCOVA Post-TestMain Effects for Therapists
Post-Test Measures
Therapists BDI HRSD
TH-1 BDIT1 HRSDT1
TH-2 BDIT2 HRSDT2
TH-3 BDIT3 HRSDT3
TH-4 BDIT4 HRSDT4
19
20
ANCOVA - Post-Test
Effects Therapist X Treatment Interaction
TherapistBDI
Post-test Scores by Treatment
HRSDPost-test Scores by
Treatment
BA AT CT BA AT CT
TH-1 BDIT1 BDIT1 BDIT1 HRSDT1 HRSDT1 HRSDT1
TH-2 BDIT2 BDIT2 BDIT2 HRSDT2 HRSDT2 HRSD T2
TH-3 BDIT3 BDIT3 BDIT3 HRSDT3 HRSDT3 HRSDT3
TH-4 BDIT4 BDIT4 BDIT4 HRSDT4 HRSDT4 HRSD T4
20
Outcome: Follow-up 6 months Overall Impact of Therapy
ANCOVAS IV:Treatments DVs: Follow-Up HRSD BDI Covariate: Pretest Score
Changes in Follow-up Time ANCOVAS IV: Treatments DVs: Follow-Up HRSD BDI Covariate: Post-test Score
21
Clinical Significance
ImprovedNo major depression at post-test
RecoveredNo major depression and BDI<8
22
Clinical Significance: Mean Improvement/Recovered Rates across Treatments – Post -Test (p.299)
Group Improved Recovered
Intent-to treat Total sample (149)
62.3% 51.5%
Completers 12/20 N=37
58.3% 58.3%
Maximum CompletersN = 129
66.0% 54.5%
DropoutsN=12
16.7% 5.6%
24
Chapter 7 26
Mechanisms of Change Construct Measure
Behavioral Activation
Dysfunctional Thinking
Cognitive Schemas
Pleasant Events Schedule
Automatic Thoughts Q.
Expanded Attributional Styles
How did the authors examine to what extent the mechanisms of change worked in therapy as predicted by theory?
26
How Treatment Effects Change
To what extent treatment groups differed in post-test measures of each change mechanism
Measure Ttreat. Grps Behavioral activation – Pleasant Event SCd BA Coping skills- - AT (Dist Thinking) AT Core schemas - Exp Attrb Style CT
To what extent was there change from pre-to post- treatment in each change mechanism
27
Research Questions Analyses
Treatment Outcomegroup differences Depression
3 x 4 MANCOVACVs Pre-test BDI HRSDIVs Treatments: CT, AT, BA Therapists (n=4) Treatment X TherapistDVs Post-Test BDI HRSD
Mechanism of Change:Treatment groups differed in post-test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas)
Mechanism of Change:Treatment groups change from pre-to post- treatment in each change mechanism (BA – AT-CS)
28
Research Questions Analyses
Treatment Outcomegroup differences Depression
3 x 4 MANCOVACVs Pre-test BDI HRSDIVs Treatments: CT, AT, BA Therapists Treatment X TherapistDVs Post-Test BDI HRSD
Mechanism of Change:Treatment groups differed in post-test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas)
3 ANCOVAS (one CV & DV per analysis)
CV Pre-test: E Attrb Q, ATQ, PES IV Treatments: (CT, AT, BA)DV Post-test: E Attrb Q, ATQ, PES
Mechanism of Change:Treatment groups change from pre-to post- treatment in each change mechanism (BA – AT-CS)
29
Research Questions Analyses
Treatment Outcomegroup differences in Depression
3 x 4 MANCOVACVs Pre-test BDI HRSDIVs Treatments: CT, AT, BA Therapists Treatment X TherapistDVs Post-Test BDI HRSD
Mechanism of Change:Treatment groups differed in post-test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas)
3 ANCOVAS (one CV & DV per analysis)
CV Pre-test: E Attrb Q, ATQ, PES IV Treatments: (CT, AT, BA)DV Post-test: E Attrb Q, ATQ, PES
Mechanism of Change:Treatment groups change from pre-to post- treatment in each change mechanism (BA – AT-CS)
3 Paired T Tests per Treatment Pre-post differences in each mechanism of change for each treatment : E Attrb Q, ATQ, PES
30
Paired T Tests: Pre-test/Post-test Change
Treatment
Pleasant Events Schedule
Automatic Thoughts Questionnaires
Expanded Attributional Styles
Behavioral Activation PESch** ATQ EAS
Automatic Thoughts PESch** ATQ** EAS
Cognitive Therapy PESch** ATQ** EAS**
Clients in all conditions significantly improved on the three measures (p. 301)
31
Residual change score in Mechanism of Change from Pre-to Mid-T and RCS in Depression from Mid-T to Post-T (p.301)
32
Threats to Statistical Conclusion Validity Jacobson et al. (1996)
Are the observed relations among variables accurate?
1. Power
3. Unreliability of Treatment Implementation
33
Threats to Statistical Conclusion Validity
Are the observed relations among variables accurate?
1. Power
•N=149 and 3 groups/about 50 participants per group. Large N for an Exp study (+)•Outcome measures are well-known – high internal reliability (+) •No information is given about alphas with study sample Power analyses not reported
3. Unreliability of Treatment Implementation
•Therapists were experienced in CT and trained for study (+)•Training followed a manual prepared for each treatment group (+)•Therapy tapes were listened on an on-going basis and therapists flagged if they deviated (+) •Analyses of adherence based on 27 taped sessions showed that treatments were distinct and consistent with their respective protocols (+)
34
Threats to Statistical Conclusion Validity
Are the observed relations among variables accurate?
4. Extraneous Variance in the Experimental Setting
5. Heterogeneity of Participants
35
Threats to Statistical Conclusion Validity
Are the observed relations among variables accurate?
4. Extraneous Variance in the Experimental Setting
•Do not appear to be any – The same four therapists administered all treatments•However procedures and settings are not described in detail
5. Heterogeneity of Participants
•Study had many exclusion criteria including co-morbidity, taking psychotropic medication, suicidal… p. 296. (+)•However, don’t know #s in pool of volunteers from which the 152 accepted patients were taken from•72% were women (+)•No info regarding race/ethnicity, SES (-)
36
Threats to Internal Validity
Can we conclude that there is a causal relation between the IV and the DV?
1. Selection to Treat. Groups
2. History
3. Attrition
4. Repeated Testing Effects
5. Reaction to Control Group Assignment
37
Threats to Internal Validity
Can we conclude that there is a causal relation between the IV and the DV?
1. Selection to Treat. Groups
•Used Randomization after matching for episodes dep., dysthimia, severity of depression, gender , marital status (+)
2. History •Therapy appeared to occur for everyone at once – but this is not addressed
3. Attrition •Small -only 15 out of 152 - 8% -Attrition during acute treatment was comparable across treatment conditions – p.296 right (+)
4. Repeated Testing Effects
•Five administration of measures – at pre-test; post –test and 6, 12, 18 month follow-up (-)
5. Reaction to Control Group Assignment
•No placebo or no-treatment control group (+)•Every one received treatment (+). •Maybe some realized not receiving whole treatment (?) 38
Threats to Construct Validity
To what extent variables capture desired constructs
1.Mono-Operation Bias
2. Mono-Method Bias
3. Experimenter Expectancies
39
Threats to Construct Validity
To what extent variables capture desired constructs
1.Mono-Operation Bias
•Used two measures to assess outcome: Depression (+)
2. Mono-Method Bias
•HRSD is interview based and BDI is self-report (+)
3. Experimenter Expectancies
•Potential risk (-)•Should have worked in favor of the CT condition that included all aspects of CBT-- therapists and researchers were aligned with CBT
40
Threats to External Validity
Can we generalize observed relations across persons, settings and times
1.Person-Units
2. Outcome Measures
4. Settings
41
Threats to External Validity
Can we generalize observed relations across persons, settings and times
1.Person-Units
•Highly selected sample – p. 296 (-)•Exclusionary criteria (-)•Primarily female = 73% (-)•No info regarding race/ethnicity, SES (-)
2. Outcome Measures
•They used interview based and self report measures of Depression +•Examined outcome using clinical significance index +
4. Settings
•Empirical Question…..
42
Empirically Supported Therapies - EST
1993 APA Division 12Identify Efficacious treatments for
specific disorders FDA Criteria: Specificity
Specific ingredients vs. placebo effects
Randomized double-blind, placebo control design
43
44
What components of treatment are responsible for therapy effects?
Specific Ingredients unique to each therapy approach
Common Factors --- Placebo underlie most approaches
Scientific advances that lead to theRandomized Placebo Control Group Design
Germany: 1850 Wundt’s experimental method in psychology – 1st psychology lab
Britain:1800 Galton & Pearson – use of normal
curve in assessment of distribution mental abilities
France: Physicians start comparing treatments across groups of patients: treatment groups
45
Scientific advances that lead to theRandomized Placebo Control Group Design
1930s Fisher- ANOVA statistics - randomization
1930- 1950 Placebo treatment - physiochemical ingredients vs. patients’ expectations, hopes, beliefs
Mesmerized…. Placebo
46
Well-established treatments Division 12 Task Force Criteria
I. At least two good between-group design experiments must demonstrate efficacy in one or more of the following ways:
A. Superiority to pill or psychotherapy placebo, or to
other treatment (wait list control is not enough)
B. Equivalence to already established treatment with
adequate sample sizes
48
ORWell-established treatments (2)
II. A large series of single-case design experiments must demonstrate efficacy with
A. Use of good experimental design and
B. Comparison of intervention to another treatment
49
And,Well-established treatments (3)
III. Experiments must be conducted with treatment manuals or equivalent clear description of treatment
IV. Characteristics of samples must be specified
V. Effects must be demonstrated by at least two different investigators or teams
50
Probably efficacious treatments
I. Two experiments must show that the treatment is superior to waiting-list control group
ORII. One or more experiments must meet well-established criteria IA or IB, III, and IV
above, but V is not met (2 investigator teams)OR A small series of single-case design experiments must meet well-established-
treatment criteria
51
Treatment: Efficacious or Probably Efficacious ?
Family Intervention (Smith et al.) Anorexia in adolescent and young adult
women Superior cognitive behavioral intervention
(well established treatment) in 2 randomized clinical studies
Both studies conducted by Smith et al.
52
53
Evidence Based Psych. Practice
Best available research Results from randomized Controlled Clinical Trials
and other types of empirical studies
Clinical Expertise
Client Characteristics
Concerns EVT movement:
Brief, straightforward manualized treatments: Cognitive Behavioral
Emphasis on specific effects/ignore common factors
Lack of applicability to wide range of clients; comorbidity, males, race, ethnicity, social class
Mandates to use EVT- restriction to choice of treatment
54
APA Task Force Acceptable Research Designs Clinical observation Qualitative studies Systematic case studies Single-case experimental designs Public health and ethnographic research Process–outcome studies Studies in naturalistic settings Randomized Controlled Clinical Trials Meta-analysis
55
APA Task Force Research Designs
Clinical observation Qualitative studies Systematic case studies Single-case experimental designs Public health and ethnographic research Process–outcome studies Studies in naturalistic settings Randomized Controlled Clinical Trials Meta-analysis
56
Evidenced Based Practice Empirically Supported Tmts.
Effectiveness/Clinical utility: Efficacy;
Main Focus:
Main Focus:
Knowledge from clinical expertise
Knowledge from clinical expertise
Clients’ individual differences
Clients’ individual differences:
Therapists’ individual differences:
Therapists’ individual differences; 57
Evidenced Based Practice Empirically Supported Tmts.
Effectiveness: does it work; generalizability; feasibility
Efficacy; does treatment cause outcome
Main Focus: Client
Main Focus: Intervention
Knowledge from clinical expertise is directly applied
Knowledge fromclinical expertiseIndirect- to generate hypotheses
Clients’ individual differencesCentral
Clients’ individual differences: Nuisance
Therapists’ individual differences: Central
Therapists’ individual differences; Nuisance
58
61
Components Shared by Approaches to Psychotherapy (Wampold, 2000)
1. Emotionally charged, confiding relationship
2. In a healing setting
3. Rationale, conceptual scheme, or myth to explains patient’s symptoms
4. There is a ritual or procedure that engages client and therapist based on the rationale (Frank
& Frank, 1991).
62
Common Elements in Therapeutic Rituals and Procedures (1/2)
5. The relationship helps combat clients’ feelings of alienation
6. The process of therapy provides hope for improvement
7. Therapist provides new learning experiences
63
Common Elements in Therapeutic Rituals and Procedures (2/2)
8. Client’s emotions are aroused as a result of therapy and the clients expects to improve
9. Therapist enhances client’s sense of mastery and self-efficacy
10. Therapist provides opportunities for practice
64
Wampold concluded that: Specific theoretical ingredients are not
differentially related to outcome
Specific ingredients are necessary to construct a coherent treatment In which therapists have faith, and that provides a convincing rational for clients
Therefore, knowledge of theoretical approaches is necessary to build coherent interventions and treatments