1

Service Models:What should be adhered

to?

Meta-regression of Intensive case management studies

Tom BurnsUniversity of Oxford

“When the facts change, I change my opinion. What, sir, do you do?”

John Maynard Keynes, economist

2

Special problems researching community care interventions

• Complexity– (what is working and what just a passenger?)

• Pioneer effect– Good people make anything work better

• Sustainability– Is it worth the effort when the research is over?

• What are you comparing it to?

• ACT literature as an example3

“Exact method based on inexact data can lead to the most remarkable

mistakes.”

General Psychopathology - Volume 1, Karl Jaspers,

Page 24, John Hopkins University Press: 1913

5

Case Management vs Standard CareHospital admissions

6

ACT vs Standard Care Hospital Admissions

7

Results and dilemmas

• ACT mandated by UK government

• 300 teams established nationally

• But:

• No European study has replicated the reduced hospitalisation

• Furore over UK700 study

Poor model fidelity

• Assertive Community Treatment (ACT) is distinct from, and superior to, other forms of case management (Max Marshall)

• The failure of UK studies of Assertive Outreach to demonstrate reduced hospital care reflect poor service implementation (‘Model Fidelity’) Max Marshall et al.

• We know this not to be the explanation– Fiander et al, (2003), BJPsych, 182, 248-245

9

Attempting to answer the question empirically:

Going beyond definitions

10

How Meta-regression maximises data from the trials

• Skewed data included

• Data without SDs included where these can be imputed by statistical means

• Contacted trialists for missing information

• Used Independent Patient Data

• Split multi-centre trials

11

Inclusion criteria• All randomised control trials (Cochrane

Randomisation Category A or B) of intensive case management versus low intensity case management, standard care, or some combination of the two

• Intensive case management was defined as case management with a caseload of 20 or less

• Excluded if a majority of subjects were >65 yrs or not suffering from severe mental illness

12

Trials identified• 42 included trials with 7817 participants• 9 trials were multi-centre

– 8 disaggregated into a further 23 eligible trials with fidelity data for each

• Individual patient data obtained for 2084 participants in 5 trials– UK700 (n=708, 4 centres)– Rosenheck et al (n=873, 10 centres)– Drake et al (n=223, 7 centres)– Marshall et al (n=80, 1 centre)– McDonel et al (n=200, 2 centres)

13

Meta-regression used to test for impact on variation of:

• Date of study – Earlier studies more reduction?

• Size of study– Smaller studies bigger effect size as evidence

of publication bias

• Baseline hospitalisation rates– Higher rates permits greater reduction

• Model fidelity– Higher model fidelity greater reduction

14

Meta-regression used to test for impact on variation of:

• Date of study – Earlier studies more reduction? No

• Size of study– Smaller studies bigger effect size as evidence of

publication bias No

• Baseline hospitalisation rates– Higher rates permits greater reduction Yes

• Model fidelity– Higher model fidelity greater reduction Yes

15

Impact of current bed usage

16Copyright ©2007 BMJ Publishing Group Ltd.

Burns, T. et al. BMJ 2007;335:336

Metaregression of Intensive Case management studiesBaseline hospital use v mean days per month in hospital.

Negative treatment effect indicates reduction relative to control

17Copyright ©2007 BMJ Publishing Group Ltd.

Burns, T. et al. BMJ 2007;335:336

Metaregression of Intensive Case management studiesControl group mean v mean days per month in hospital.

Negative treatment effect indicates reduction relative to control

18

Impact of model fidelity (ACT)

19

IFACT scale (McGrew et al 1995)

• Expert consensus:– 20 experts rated importance of 73

program features

• 14 item scale tested in 18 “ACT” programs

• Items specified three domains– membership, – structure & organisation – care practices

20

Meta-regression of Fidelity v Reduction in IP days

-20

24

6m

ean

diffe

renc

e

2 4 6 8 10 12total fidelity score

mean difference Fitted values

21

Separating the IFACT Domains

22

M-R of Team staffing v Reduction in IP days

-20

24

6m

ean

diffe

rence

0 1 2 3 4staffing

mean difference Fitted values

23

M-R of Team organisation v Reduction in IP days

-20

24

6m

ea

n d

iffere

nce

0 2 4 6 8team organisation

mean difference Fitted values

24

Conclusions

• High staffing levels in ACT are not associated with reducing hospitalisation

• ‘Organisational’ elements are associated with reducing hospitalisation

• These organisational elements appear to be present in standard CMHTs

• Can we be more precise about them?– Which are they?

25

A more positive message..

What does work

26

Testing for characteristics of home-based care using cluster analysis

and regression

Core service componentsFigure 1: Associations between service components & hospitalisation: regression analysis

Wright C, Catty J, Watt H, Burns T. A systematic review of home treatment services. Classification and sustainability. Soc Psychiatry Psychiatr Epidemiol 2004;39:789-96

Regularly visiting at home Responsible for

health and social care

High % of contacts at home

Multidisciplinary teams

Psychiatrist integrated in team

Smaller caseloads

A Guide to current CMHT practices

• Generic CMHTs• Assertive Outreach

teams• Crisis Resolution /

Home Treatment teams

• Early onset teams• Specialist and

international perspectives

29

Treatment as usual studies

• The Dodo Bird society:– ‘Dedicated to making Treatment as

Usual studies history’

• Burns T, Priebe S. Mental health care systems and their characteristics: a proposal. Acta Psychiatrica Scandinavica 1996 December;94(6):381-5.

• Burns T ‘End of the line for TAU studies’ BJPsych, 2009

30

Treatment as usual studies

• The danger of being restricted to your hypothesis in interpretation

• Killaspy follow up study (REACT) confirmed earlier REACT study that ACT has equivalent outcomes to CMHTs

• Conclusion ACT is ‘not superior’• Is that right?• Conclusion is CMHTs superior!

Assessing Community Research

• Ensure that the comparator is relevant to you

• Ensure it is well described

• Check for sustainability (pioneer effect)

• Be sceptical about ‘ complex packages’– Studies should try to isolate key ingredients

• If it looks too good to be true it probably is

31

32