Early intervention in psychosis: a health economic evaluation using the net benefit approach in a real world setting

Authors:

Caragh Behan, Brendan Kennelly, Eric Roche, Laoise Renwick, Sarah Masterson, John Lyne, Brian O’

Donoghue, John Waddington, Catherine McDonough, Paul McCrone, Mary Clarke

Abstract

Background

Early intervention in psychosis (EIP) is a complex intervention, usually delivered in a specialist stand-

alone setting, which aims to improve outcomes in people with psychosis. Economic evaluation is a

useful framework to guide the evaluation of an intervention by using a metric which evaluates the

joint costs and effects.

Aims

To evaluate whether there is a net benefit to the health sector and society when EIP is delivered in a

real world setting.

Method

Two contemporaneous incidence-based cohorts presenting with a first episode psychosis aged 18-65

were evaluated.

Results

From the health sector perspective, the probability that EIP was cost-effective was 0.77 and the

incremental net benefit (INB) of EIP was €2,465 (95% CI -€4,418 to €9,347) when society placed a

value of €6,000, the cost of an in-patient relapse, on preventing a relapse requiring in-patient

admission or home care. Following adjustment for covariates, the probability that EI was cost-

effective was 1 and the INB to the health sector was €3,105 (95% CI - €8,453 to €14,663). From the

societal perspective, the adjusted probability that EIP was cost-effective was 1, and the INB was

€19,928 (95% CI -€2,075 to €41,931).

Conclusion

EIP has a modest INB from the health sector perspective but a large INB from the societal

perspective. The choice of outcome measure and perspective of the study are critical when

presenting an economic evaluation of a complex intervention such as EIP to policymakers and

service planners.

Early intervention in psychosis: a health economic evaluation using the net benefit approach in a real world setting

Introduction

Evidence on the cost-effectiveness of Early Intervention in psychosis (EIP) comes from a

heterogeneous group of studies, which show that EIP primarily achieves savings through a reduction

in in-patient admissions (1-4). The majority of evidence for the costs and effects of EIP come from

specialist stand-alone centres with a youth-oriented approach delivering EIP to a population aged

16-35 and compared to standard treatment as usual (TAU). Critics argue that TAU has evolved since

these studies were performed, and current community mental health (CMH) care, delivering more

sophisticated treatments such as home based treatment and assertive outreach treatment can

deliver effective EIP. (5, 6)

Interventions shown to have an effect in a trial setting may lose efficacy in real world settings where

delivery is constrained by loss of fidelity and problems with sustainability. Economic evaluation can

facilitate the examination of whether a complex intervention translates into the local context,

thereby generating information relevant for service planning and policy makers. Cost-effectiveness

evaluations typically report Incremental Cost-Effectiveness Ratios (ICERs). ICERs involve a ratio and

there are difficulties in interpreting the value of the ICER and in generating a measure of uncertainty,

and they are not amenable to regression analysis. (7, 8) Reformulating the cost-effectiveness

question to generate a linear net benefit (NB) facilitates interpretation of the results of the

evaluation, and allows use of regression techniques to adjust for socio-demographic and clinical

differences between the treatment as usual (TAU) and intervention groups.

While EIP is part of the core mental health service in countries such as Canada, the UK and Australia,

other countries have been trying to implement EIP in health systems that face significant challenges,

both in terms of financing and sustaining services. In Ireland, a number of health service reforms

have presented challenges for service delivery. EIP is one of three National Clinical Programmes in

mental health; presently coverage of the population is only 10%. Other challenges include strict

delineation between Child and Adolescent Mental Health Services (CAMHS) and General Adult

Mental Health Services (MHS), with no models of care for EIP which cross this divide.

Aim

The aim of this study was to conduct a cost-effectiveness evaluation of EIP in a real world setting in

comparison to best practice TAU using the NB framework. This study is presented according to the

CHEERS guidelines for economic evaluations.(9)

Methods

The study sample consisted of two incidence-based cohorts presenting to five catchment area

services in Ireland between 2010 and 2012. Data were collected at first presentation and at one year

follow up.followed-up at one year. Individuals presenting to the catchment area services with a first

episode of psychosis aged between 18 and 65 were included. Inclusion criteria were being in their

first episode of psychosis (FEP) and aged between 18 and 65. Exclusion criteria were intellectual

disability, being on antipsychotic medication for more than 30 days prior to study inception, and

having a diagnosis of psychosis secondary to a general medical condition. The authors assert that all

procedures contributing to this work comply with the ethical standards of the relevant national and

institutional committees on human experimentation and with the Helsinki Declaration of 1975, as

revised in 2008. Ethical approval for the study was obtained from the local ethics committees of the

relevant services (ID 406 St John of God Provincial Ethics Committee).

One cohort presented to an EIP service which operated as a specialist hub delivering EI to three

Community Mental Health Team (CMHT) services. The EI service offered an early detection and

phase specific intervention strategy. Anyone referred to the EI service was offered a rapid

assessment within 72 hours of referral to establish the presence of psychosis. Where possible, the

assessor also interviewed a family member. Care during any in-patient admission and medication

management remained the responsibility of the CMHT. The TAU cohort presented to a best practice

CMHT service with a home based treatment team and an assertive outreach team. Patients

presenting to the TAU service received a structured diagnostic interview (SCID-IV) and assessment by

a research registrar following presentation (on average within 41 days) but otherwise received

standard care. Following the assessment, the participant was offered one or all of three phase

specific interventions. Cognitive Behavioural Therapy for psychosis (CBTp) was delivered in group

format over 12 sessions. Family education and intervention was delivered in group format over 6

sessions. A psychosocial intervention was delivered in individual sessions for as many as were

required. Over the period of the study the psychosocial intervention was not consistently offered

due to resource reasons. A follow-up assessment was conducted at one year.

In both cohorts, each participant was assessed using a structured diagnostic interview (SCID-IV) to

establish a diagnosis of psychosis. (10) Information on health service and resource use was collected

using the Client Socio-demographic service and receipt inventory (CSSRI) for the one year period.

(11, 12) The CSSRI was modified for use in the Irish context. This information was verified and

supplemented using medical records and contact with the primary care service. Unit costs were

derived from previously published national cost of illness studies, other national studies which

reported unit cost data, personal contact with the individual services and personal contact with the

finance department of the Health Service Executive (HSE) in Ireland. Where national cost data were

not obtainable, we used data from the PSSRU in the UK and converted the costs to euro using PPP.

(13). For opportunity costs, hours of lost productivity were valued according to the Human Capital

Approach (HCA). (8) The average industrial wage for the time period of the study as published by the

CSO was used to value lost productivity. (14) We used the proxy good method to value informal care

and applied the hourly wage of the person who would replace that form of care, including carers and

childcare services. (8) Costs were reported in Euro for the year 2012.

The reference case was from the perspective of the health sector. Costs accruing to the health sector

relating to the one year period following presentation with the FEP were collected. These included

mental health in-patient admission costs, general medical admission costs related to the FEP, home-

based treatment costs, CMHT? service costs, costs from primary care, and external mental health

resources related to the FEP including counselling, medication and investigation costs. Secondary

analysis was from the societal perspective. The cost of lost productivity was collected by applying the

average industrial wage to the number of days lost from employment secondary to the FEP. The one

year follow-up period allowed sufficient time for the outcome to occur and there was no

requirement to apply a discount rate to the costs.

The primary outcome was defined as a relapse of psychosis which was sufficiently severe to require

admission to hospital or to the home-based treatment team (HBT). Information on relapse was

collected from the hospital electronic records and from the CMH written medical records.

Information to determine lost productivity and employment status at one year was collected from

clinical interview and/or medical records review and documented using the CSSRI.

Data were analysed in Microsoft excel 2010 and Stata 13.0. Univariate analysis of outcome data

were carried out using chi squared tests, parametric data were analysed using students t-tests and

non-parametric data were analysed using Mann-Whitney U tests. Multivariate analyses of outcome

data were carried out using logistic regression. As cost data are usually highly skewed, cost data

were analysed using a generalized linear model with a gamma family and a log link. Cost and

outcome data were adjusted for socio-demographic and baseline clinical characteristics for the

multivariate analyses. The NB statistic was generated using the equation NB=λ.E-C, where NB is the

net benefit, E is the effectiveness (i.e. avoidance of a relapse requiring admission or HBT) and C are

the service costs. λ is a theoretical, but unknown, value placed on the outcome by society. Cost and

effectiveness data were bootstrapped to 1000 replications using sampling with replacement to

generate 95% confidence estimates. The proportion of these replications that were greater than

zero indicated the probability that EI was more cost-effective than TAU. The probabilities were used

to generate cost-effectiveness acceptability curves (CEAC). As there is no guidance for choosing the λ

values when the outcome is not a quality adjusted life year (QALY), a range of values of willingness

to pay were plotted. Sensitivity analyses of the costs and outcomes were conducted and a secondary

analysis included the societal perspective using the value of lost productivity.

Results

Of 307 people presenting to services who fulfilled the inclusion criteria, 270 were eligible for follow-

up at one year, 212 were assessed at one year, and 201 people included in the cost-effectiveness

analysis. Reasons for non-inclusion in the cost-effectiveness analysis (6%) were incomplete CSSRIs or

attendance at private in-patient services. Table 1 shows the socio-demographic and clinical

characteristics of the sample. There were no statistically significant differences in baseline

characteristics as regards gender, marital status, living independently at presentation, the

proportion born in Ireland and the proportion with English as their first language. The TAU cohort

were younger at presentation (28 years v 33 years, Z =-2.646, p=0.008). The TAU cohort had a

significantly higher proportion in employment at baseline (47% v 27%, χ2 =7.823, df1, p=0.005). The

majority of the EI cohort lived in urban areas (98% v 39%, χ2 =87.34, df1, p<0.001). The TAU cohort

were living in areas with higher levels of deprivation (decile 9 v. decile 4, z=5.554, p<0.001).

Table 1 Baseline characteristics of the study sample

Categorical variables n (%)

Total(201)

TAU(77)

EI(124)

Statistic p value

GenderMale 113 (56) 48 (62) 65 (52) 1.898 0.168Never married 123 (61) 50 (65) 73 (59) 0.736 0.391Living independently

135 (67) 82 (68) 53 (67) 0.008 0.930

EducationFinished high school equivalent

136 (68) 48 (64) 88 (71) 1.049 0.306

Employed 70 (35) 36 (47) 34 (27) 7.823 0.005Urban 151 (75) 30 (39) 121 (98) 87.347 <0.001Irish born 153 (76) 59 (77) 94 (76) 0.017 0.895SSD 112 (56) 41 (53) 71 (57) 0.309 0.578Under 35 at presentation

127 (63) 56 (73) 71 (57) 4.887 0.027

Continuous variables Median (IQR)Age at presentation

32 (18) 28 (15) 33 (16) -2.646 0.008

Deprivation Index

7 (7) 9 (4) 4 (8) 5.454 <0.001

SF Index 9 (3) 8 (5) 9 (2) -4.351 <0.001GAF 30 (10) 27 (9) 30 (13.5) -2.781 0.005

at baselineTAU: Treatment as usual; EI: Early intervention; IQR: Inter quartile range; SF: Social fragmentation Index; GAF: Global assessment of functioning scale; SSD: schizophrenia spectrum disorder

Cost-effectiveness results

The data were initially evaluated using the standard ICER (Table 2). From the health sector

perspective, the intervention dominated. The intervention cost €1,681 (SE €3,247) less and more

relapses were avoided (0.10 (SE 0.06)). The unadjusted ICER was €17,078 saved per relapse avoided.

The bootstrapped estimates of the ICER were plotted on the cost-effectiveness plane and show that

the intervention dominated in 63% of replications. Following adjustment for socio-demographic and

clinical characteristics, the intervention dominated in 95% of replications (see figure 1). From the

societal perspective, the intervention dominated in 74% of replications and following adjustment for

socio-demographic and clinical characteristics, the intervention dominated in 95% of replications.

The unadjusted ICER was €25,543 saved per relapse avoided from the societal perspective.

Table 2 Incremental cost-effectiveness ratio

Mean (SE) TAU (77) EI (124) Difference † 95% CI (N) †Health sector perspectiveCost € 23,862 (2,835) 22,181 (1,857) -1,681 (3,247) -4,721 to 8,083Effect (relapse avoided)

0.74 (0.50) 0.84 (0.03) 0.10 (0.06) -0.21 to 0.02

ICER health sector -17,078

Societal perspectiveCost € 25,554 (2,823) 22,707 (1,863) -2,846 (3,246) -3,768 to 9,018Effect (relapse avoided)

0.74 (0.50) 0.84 (0.03) 0.10 (0.06) -0.21 to 0.02

ICER societal -25,543† Bootstrapped to 1000 replications; CI: Confidence Interval; N: normal based; ICER: Incremental cost-effectiveness ratio; SE: standard error; costs rounded up, all other figures rounded to 2 decimal places

Figure 2 Cost-effectiveness plane

€0

€50,000

€100,0000

0.5

1

Cost-effectiveness ac-ceptability curve

Unadjusted probabilityAdjusted probability

Willingness to pay- ceiling thresholdProb

abili

ty E

I cos

t-eff

ectiv

e

30%2%

66%3%

-150

00-1

0000

-500

00

5000

1000

0D

iffer

ence

in c

osts

(in

eur

os)

-.1 0 .1 .2 .3Difference in relapse

Threshold line 95% confidence ellipse

Cost-effectiveness plane (unadjusted)

33%1%

63%3%-100

00-5

000

050

0010

000

Diff

eren

ce in

cos

ts (

in e

uros

)

-.1 0 .1 .2 .3Difference in relapse

Threshold line 95% confidence ellipse

Unadjusted cost-effectiveness plane (health sector)

2%0%

95%3%

-300

00-2

0000

-100

000

1000

0D

iffer

ence

in c

osts

(in

eur

os)

-.2 0 .2 .4Difference in relapse

Threshold line 95% confidence ellipse

Adjusted cost-effectiveness plane (Health sector )

22%1%

74%3%

-150

00-1

0000

-500

00

5000

Diff

eren

ce in

cos

ts (

in e

uros

)

-.1 0 .1 .2 .3Difference in relapse

Threshold line 95% confidence ellipse

Unadjusted cost-effectiveness plane (societal perspective)

1%0%

95%4%

-300

00-2

0000

-100

000

1000

0D

iffer

ence

in c

osts

(in

eur

os)

-.1 0 .1 .2 .3Difference in relapse

Threshold line 95% confidence ellipse

Adjusted cost-effectiveness plane (societal perspective)

Using the NB framework, implementing the intervention resulted in an incremental net benefit (INB)

to the health sector of €1,796 (SE €3,376). This fell to €1,200 (SE €5,410) following adjustment for

socio-demographic and clinical characteristics, even when society placed no value on avoiding a

relapse requiring admission or home based treatment. When a value of €6,000, the approximate

cost of an in-patient relapse in the literature, was placed on avoiding such a relapse, implementing

the intervention resulted in an INB of approximately €2,465 (SE €3,389) to the health sector, and

€3,105 (SE €5,890) following adjustment for socio-demographic and clinical characteristics. The

standard errors of the mean were large and the 95% confidence intervals were wide reflecting the

degree of uncertainty around the cost data. When the value of λ was €0, the probability that the

intervention was cost-effective was 0.71 in the unadjusted model, and following adjustment for

socio-demographic and clinical characteristics, the probability that EI was cost-effective fell to 0.59.

From the societal perspective, implementing the intervention resulted in an INB of €34,694 (SE

€8,994). This fell to €17,604 (SE €10,933) following adjustment for socio-demographic and clinical

characteristics, even when society placed no value on avoiding a relapse requiring admission or HBT.

When λ was valued at €6,000, implementing the intervention resulted in an INB to society of

approximately €35,363 (SE €8,081), and €19,928 (SE €11,212) following adjustment for socio-

demographic and clinical characteristics. The probability that the intervention was cost-effective

was 1, before and after adjustment for socio-demographic and clinical characteristics. Figure 2 shows

the probabilities that the intervention was cost-effective for a range of values of willingness to pay

(λ).

Figure 2 Cost-effectiveness acceptability curves

0

20

40

60

80

100

Pro

babi

lity

that

ear

ly in

terv

entio

n is

cos

t-effe

ctiv

e (%

)

0 20000 40000 60000 80000 100000

Willingness to pay (in € )

0

20

40

60

80

100

Pro

babi

lity

that

ear

ly in

terv

entio

n is

cos

t-effe

ctiv

e (%

)

0 20000 40000 60000 80000 100000

Willingness to pay (in € )

Unadjusted Adjusted

Health sector perspectiveCost-effectiveness acceptability curves

0

20

40

60

80

100

Pro

babi

lity

that

ear

ly in

terv

entio

n is

cos

t-effe

ctiv

e (%

)

0 20000 40000 60000 80000 100000Willingness to pay (in €)

0

20

40

60

80

100

Pro

babi

lity

that

ear

ly in

terv

entio

n is

cos

t-effe

ctiv

e (%

)

0 20000 40000 60000 80000 100000

Willingness to pay (in €)

Unadjusted Adjusted

Societal perspectiveCost effectiveness acceptability curves

Sensitivity analyses were conducted to test the assumptions in the model and the data. Varying the proportion of relapses to 25% (the minimum relapse rate aimed for in the IRIS guidelines (15)) and adjusting for baseline costs had no effect on the result. Neither did restricting the analysis to those seen by clinical interview in comparison to those whose data was extracted from medical records.

Subgroup analyses were performed to evaluate for the effect of heterogeneity on the outcome. Limiting the intervention to people with schizophrenia spectrum disorder (SSD) increased the cost-

effectiveness of the intervention. Limiting the intervention to the under 35s decreased the cost-effectiveness of the intervention. Including people presenting with psychosis secondary to a medical condition decreased the cost-effectiveness of the intervention (see Table 3).

Table 3 Subgroup analyses

When Lambda (λ ) = €6K Reference case (n=201)

Under 35s(n=128)

SSD(n=120)

Including GMC(n=205)

Health sector perspectiveUnadjusted Δ INB € All mean (SE) †95% CI †

2,465 (3,389)-4,418 to 9,347

-864 (3,889)-8,497 to 6,768

7,642 (5,433)-3,017 to 18,302

970 (3,618)-6,130 to 8,071

Probability CE health sector 0.77 0.64 0.93 0.60Adjusted ††Δ INB € All mean (SE) †95% CI †

3,105 (5,890)-8,453 to 14,663

231 (7,754)-14,985 to 15,447

6,899 (8,712)-10,197 to 23, 995

721 (5,966)-10,985 to 13,125

Probability CE health sector 1 0.987 0.998 0.99

Societal perspectiveUnadjusted Δ INB € All mean (SE) †95% CI †

35,363 (8,081)17,582 to 53,144

34,882 (10,962)12,683 to 57,134

49,611 (11,459)22,444 to 76,777

33,582 (8,028)-15,897 to 51,267

Probability CE societal 1 1 1 1

Adjusted Δ INB € All mean (SE) †95% CI †

19,928 (11,212)-2,075 to 41,931

22,509 (15,603)-8,109 to 53,127

21,977 (17,188)-11,752 to 55,705

16,779 (10,977)-4,762 to 38,319

Probability CE societal 1 1 0.996 1Δ : difference in means; CE: cost-effective; CI: confidence interval; EI: early intervention; GMC: general medical INB: incremental net benefit; SE: standard error; SSD: schizophrenia spectrum disorder; €, Euro; † bootstrapped to 1000 replications

†† adjusted for age, gender, marital status, employment at baseline, diagnosis, GAF at baseline, the use of drugs, the presence of depression, catchment area and Social Fragmentation Index decile; set seed was the same as the unadjusted model to facilitate replication; cost data rounded up: all other figures rounded to 3 decimal places

Discussion

The evidence for the cost-effectiveness of EI in psychosis comes from a range of published studies

with heterogeneous methods and outcomes. The majority of studies use a historical TAU control (1,

16-21), three use decision analytic modelling (22-24), one uses data linkage (25) and two studies

evaluate the cost-effectiveness of EI in comparison to TAU in a RCT. (3, 4) In those studies where

patient-level data were used, the settings were primarily specialist EI services delivering EI to a

younger cohort with a maximum age of 40-45 from the health sector perspective. The published

studies showed that EI results in cost savings primarily through a reduction in in-patient admissions.

This study adds to the literature on the cost-effectiveness of EIP by considering both the health

sector and societal perspectives using patient-level data. Use of the NB approach, which takes into

account the joint effect of costs and effects upon each other, facilitated the evaluation of

uncertainty (7). This is particularly useful where the design of the study is not a RCT. Traditionally the

ICER has been used in economic evaluation. The ICER approach examines costs and effects in

isolation and does not handle the estimation of uncertainty well. (7, 8) To date, there has only been

one other study examining the cost-effectiveness of EI using the NB regression approach. That study

evaluated a specialist EI service in comparison to more traditional TAU community model, and

showed a high likelihood that EI was cost-effective if outcomes such as quality of life and vocation

were taken into account. (4)

Examination of the societal perspective by including the impact of EI on productivity showed that

delivering EI had a large INB in all populations and a probability of 1 of being cost-effective. This

highlights the need to look beyond the health sector at the wider benefits of mental health

interventions. In this case, lost productivity referred only to days of employment lost due to illness,

and did not include days of education, or any other activities that require role replacement such as

that of the carer.

This study further evaluated whether cost savings made in specialist centres translate to a real world

setting. Most countries have a rigid boundary between child and adult mental health services. This

study examines whether EI is cost-effective in a setting where there is no transitional youth model of

EIP, and EI is delivered through the community mental health setting to an adult population over 18,

albeit one with a specialist dedicated EI hub delivering the interventions. EI services have

traditionally been advocated for younger people, and yet there are a substantial proportion of

people who become psychotic or present for care for the first time over the age of 45. This study

included the older population (18-65) in the intervention and analyses. This is a particularly relevant

question from a policy perspective. Recent economic problems and policy changes have led to the

consideration of extending EI to this population in the UK and the ‘Early Intervention Access and

Waiting Time Standard’ published in 2016 specifically states that the target population should be

people aged 14-65. (26)

The NB approach was useful in this context, as it facilitated evaluation of sub-groups in which an

intervention is potentially more or less cost-effective. In this study, the subgroups were identified a

priori for policy reasons. Internationally, many EIP services restrict by age (usually 15-35) and by

diagnosis (usually functional psychosis). Delivering EI to the functional psychosis SSD sub-group was

highly cost-effective from both a health sector and a societal perspective. This was most likely due

to the fact that the proportion of relapses was higher in this subgroup in the TAU cohort (0.18 v.

0.10). As the NB is affected more by costs when the effect size approaches zero, a bigger effect size

in this sub-group demonstrated a different INB profile.

In contrast, the evaluation of the 18-35 subgroup in this setting, revealed that EI was less likely to be

cost-effective from the health sector perspective. The INB favoured traditional CMHT until the

willingness to pay for the intervention was over €15,000 per relapse avoided. By contrast, there was

an INB from the societal perspective even when the value of λ or willingness to pay was €0. There

are policy implications to this finding, as delivering EI in a setting with no youth model of mental

health, such as in countries like Ireland with a strict boundary between CAMS and adult MHS; may

have no benefit from the perspective of the health payer. The youth model of mental health delivers

EI across this boundary, usually across an age range of 14-25 years. This makes sense from a

theoretical perspective, as this range of ages is the peak time for the potential onset of mental

health disorders in the developing adolescent and young adult brain. Therefore, when examining the

cost-effectiveness of delivering EI to a youth sub-group attending an adult EI service, the data

suggest that EI is not cost-effective.

The final subgroup analysis tested the effect of including people with organic psychosis, or psychosis

secondary to a general medical condition (GMC). Internationally, research on FEP and services

delivering EI typically excludes people presenting with organic psychosis. A case can be made that

people with psychosis secondary to a GMC will still present to EI services, and the costs of treating

them can still be incurred, as in a real world setting it is not always immediately apparent that the

cause of psychosis is medical rather than functional. These cases can generate high costs and the

effect of EI is uncertain, as the psychosis is often resolved by treating the underlying medical

condition. Health economic analyses are usually concerned with the mean value, as this allows

policy makers, service planners and decision makers to consider the total cost and effect of an

intervention, rather than the median which is not subject to interpretation in a meaningful way. In

this sample, despite the total number presenting with an organic psychosis being small - seven over

three years of whom four were followed up and were eligible for inclusion as a sensitivity analysis of

this study- there was a substantial impact on the results of the analysis. The marked change in the

INB by excluding four cases from the analysis also illustrates the degree of uncertainty associated

with cost data, and this uncertainty should be taken into account when presenting this information

to service planners and policy makers.

There are a number of strengths to this study. Previous studies of EI have compared EI to older

models of TAU. This TAU cohort received best practice community mental health care, including

HBT and assertive community outreach, all models of care designed to deliver acute care in the

community where that is appropriate. This study also used robust methodology for the case finding

and evaluation of people presenting with a FEP. While one of the two private hospitals in Ireland

which may have admitted people with a FEP from any of the five catchment areas was not

contacted, the other private hospital was situated in one of the catchment areas covered by the

study. Otherwise, both samples are epidemiological samples of FEP presenting to community

mental health services in the five catchment areas included in the study. Each person attending the

EI service received a comprehensive diagnostic interview and assessment by trained assessors with

good inter-rater reliability. Of those presenting to the TAU catchment area, 80% were assessed at

baseline with a comprehensive diagnostic interview and assessment by trained assessors, also with

good inter-rater reliability.

The patient-level direct and indirect costs included in this study were collected in a structured,

standardised manner. As a year is a long time for patient recall with the CSSRI, the information

provided by patients was supplemented by corroborating it with information on resource use from

medical records and by contacting primary care practices. Evidence shows that using patient recall

alone underestimates resource use and therefore costs. (27)

With regard to potential limitations, the study design is not as robust as that of an RCT and there are

potential sources of observed and unobserved bias. While not an RCT, the use of two comparative

incidence-based cohorts in a well-defined population with regression analysis to control for

observed population differences, can facilitate the evaluation of complex interventions in a real

world setting and may have more generalisability. (28) There was potential sample bias as there

were observed and potentially unobserved differences between the two cohorts. The TAU cohort

was younger, and from a predominantly rural and more deprived setting with higher levels of

unemployment. The EI cohort was a mixture of individuals from a predominantly affluent area with

a relatively older population and a smaller proportion from deprived areas and rural areas. Using

the NB framework to conduct the cost-effectiveness analysis allowed adjusting for differences in the

socio-demographic and catchment area level characteristics in the analysis. Propensity score

matching can be used to simulate the conditions of an RCT design. (29) However, the optimal

conditions for use of matching usually require more observations in the control group than the

treatment group. In this case there were more observations in the treatment group than the control

group and this excluded a large number of people from the analysis as they did not have a match.

Therefore the propensity score was used as a covariate in the initial analysis; however as this did not

yield any extra information in comparison to including the covariates themselves, the score was not

ultimately included in the final model. Rather, the NB approach was used, facilitating the use of

regression to control for observed differences between the groups. There are also unobserved

differences between the two groups that are not included. The unit cost data in the study are from a

variety of sources as there are no published national sources of unit cost data in Ireland. While

considerable time and resources were devoted to generating the unit costs used in this study, there

are still some unit costs lacking and some were incomplete, possibly leading to an underestimate of

the cost information. This is particularly relevant in the intervention cost which did not include

capital costs or non-contact costs. The lack of published national unit costs limits the ability to

compare studies of this type as the cost data may differ by study. The primary outcome used for this

study was limited to one that could be reliably extracted from the clinical records at one year, a

limitation of the study which was imposed by the pragmatic difficulties of doing research in a real

world setting. The gold standard outcome measure of choice in economic evaluations is the quality

adjusted life year (QALY), which has accepted values of willingness to pay per QALY improvement in

different health settings internationally. (8) The primary outcome used in this study has some

limitations. Relapse requiring admission or HBT does not have a defined societal threshold ratio or

value of WTP. However; relapse has a significant impact on outcome. Relapse limits recovery and is

distressing for the participant and their family and carers, and is a significant predictor of costs.

Between 30 and 70% of people with FEP will relapse. (30) There is a significant cost difference

between those who relapse and require admission and those who do not require admission, one

study finding a cost difference of £6,000 between those who did not relapse requiring admission

(£2,000) and those who did (£8,000), and we used this figure as a benchmark for willingness to pay

to avoid relapse. (31) While relapse is a pertinent outcome, this is also a potential limitation of the

study, as in-patient admission and HBT are both components of cost.

There was potential selection bias in the one year follow-up data. Due to differences in the ethical

approval from each ethics committee, the tracing procedures in each cohort were different. Not all

of the eligible EI sample was followed up at one year, while almost all the TAU sample had follow-up

at one year either by clinical interview or by using medical records. The impact of this potential

selection bias was tested by examining the total sample in the reference case analysis, and then by

re-conducting the analysis only in those followed up by clinical interview in both samples, to test the

assumptions. There were no statistically significant different socio-demographic or clinical

characteristics between those followed up by clinical interview and those followed up by clinical

record. Specifically, there was no statistically significant difference in the primary outcome measure.

There were some differences in the cost data, and therefore in the NB statistic in the repeat analysis.

The probability that EI was cost-effective shifted down and to the right; however the trajectory of

the NB statistic remained the same. As previously alluded to, the literature suggests that patient

reports of health service resource use are often higher than resource use taken from medical

records, so this suggests that the cost data in the overall sample is an underestimate of the costs in

the TAU group rather than an over-estimate.

Conclusion

This study adds to the evidence base on the economic evaluation of EI in psychosis. Use of patient-

level data from a mix of urban and rural settings, evaluation of an intervention taking place in a real

world setting and extensive consideration of the context in which the study took place provide

additional insights into how EI services make their impact. Previous research has found that EI

makes its cost savings by reducing inpatient admissions. Modelling studies have shown that the

societal impact of EI is larger than its impact on the health sector through the effects EI has on

employment and education. (24) This study has provided evidence using patient-level data that EI,

delivered in a real world setting, in a mental health system which has no youth oriented specialist EI

service, can still provide a modest INB to the health sector even when the value of preventing a

relapse requiring admission is unknown, and shows that EI has a large INB and is extremely likely to

be cost-effective when a societal perspective is taken. As mental health interventions will often

impact on outcomes outside the health service such as employment, housing and education, policy

makers and service planners should be aware of this, and consider alternate sources of funding

mental health interventions as benefits accrue beyond the health service.

Source of funding:

This study was funded by a Health Research Board (HRB) Grant HPF-2011-042. There was no other

source of funding and no conflict of interests to report. The PI of the HRB grant also worked in the EI

service.

References

1. Mihalopoulos C, McGorry PD, Carter RC. Is phase-specific, community-oriented treatment of early psychosis an economically viable method of improving outcome? Acta Psychiatr Scand. 1999; 100(1): 47-55.2. Mihalopoulos C, Harris M, Henry L, Harrigan S, McGorry P. Is early intervention in psychosis cost-effective over the long term? Schizophr Bull. 2009; 35(5): 909-18.3. Hastrup LH, Kronborg C, Bertelsen M, Jeppesen P, Jorgensen P, Petersen L, et al. Cost-effectiveness of early intervention in first-episode psychosis: economic evaluation of a randomised controlled trial (the OPUS study). Br J Psychiatry. 2013; 202(1): 35-41.4. McCrone P, Craig TK, Power P, Garety PA. Cost-effectiveness of an early intervention service for people with psychosis. Br J Psychiatry. 2010; 196(5): 377-82.5. Castle DJ. The truth, and nothing but the truth, about early intervention in psychosis. Aust N Z J Psychiatry. 2012; 46(1): 10-3.6. Bosanac P, Patton GC, Castle DJ. Early intervention in psychotic disorders: faith before facts? Psychol Med. 2010; 40(3): 353-8.7. Hoch JS, Briggs AH, Willan AR. Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost-effectiveness analysis. Health Econ. 2002; 11(5): 415-30.8. Drummond MF, Sculpher MJ, Claxton K, L. SG, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. Oxford University Press, 2015.9. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. BMJ. 2013; 346: f1049.10. First MB, Spitzer RL, Gibbon M, Williams JB. Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Non-Patient Edition (SCID-I/NP). In: Biometrics Research (ed NYSP Institute): 2002.11. Knapp M, Beecham J. Costing mental health services. Psychol Med. 1990; 20(4): 893-908.12. Chisholm D, Knapp MR, Knudsen HC, Amaddeo F, Gaite L, van Wijngaarden B. Client Socio-Demographic and Service Receipt Inventory--European Version: development of an instrument for international research. EPSILON Study 5. European Psychiatric Services: Inputs Linked to Outcome Domains and Needs. Br J Psychiatry Suppl. 2000; (39): s28-33.13. Curtis L. Unit Costs of Health and Social Care 2012. (ed PRSSU). Personal Social Services Research Unit, 2012.14. Office CS. Statistical Yearbook of Ireland. Central Statistics Office, 2012.15. IRIS. IRIS Guidelines Update. In: IRIS (2012) IRIS Guidelines Update IRIS Initiative Ltd wwwiris-initiativeorguk. IRIS Initiative Ltd., 2012.16. Mihalopoulos C, Harris M, Henry L, Harrigan S, McGorry P. Is Early Intervention in Psychosis Cost-Effective Over the Long Term? Schizophrenia Bulletin. 2009; 35(5): 909-18.17. Cullberg J, Mattsson M, Levander S, Holmqvist R, Tomsmark L, Elingfors C, et al. Treatment costs and clinical outcome for first episode schizophrenia patients: a 3-year follow-up of the Swedish "Parachute Project" and two comparison groups. Acta Psychiatr Scand. 2006; 114(4): 274-81.18. Goldberg K, Norman R, Hoch JS, Hoch J, Schmitz N, Windell D, et al. Impact of a specialized early intervention service for psychotic disorders on patient characteristics, service use, and hospital costs in a defined catchment area. Can J Psychiatry. 2006; 51(14): 895-903.19. Wong KK, Chan SKW, Lam MML, Hui CLM, Hung SF, Tay M, et al. Cost-effectiveness of an early assessment service for young people with early psychosis in Hong Kong. Australian and New Zealand Journal of Psychiatry. 2011; 45(8): 673-80.

20. Behan C, Cullinan J, Kennelly B, Turner N, Owens E, Lau A, et al. Estimating the Cost and Effect of Early Intervention on In-Patient Admission in First Episode Psychosis. J Ment Health Policy Econ. 2015; 18(2): 57-62.21. Cocchi A, Mapelli Vi, Meneghelli A, Preti A. Cost-effectiveness of treating first-episode psychosis: five-year follow-up results from an Italian early intervention programme. Early Intervention in Psychiatry. 2011; 5(3): 203-11.22. Serretti A, Mandelli L, Bajo E, Cevenini N, Papili P, Mori E, et al. The socio-economical burden of schizophrenia: A simulation of cost-offset of early intervention program in Italy. European Psychiatry. 2009; 24(1): 11-6.23. McCrone P, Knapp M, Dhanasiri S. Economic impact of services for first-episode psychosis: a decision model approach. Early Interv Psychiatry. 2009; 3(4): 266-73.24. Park AL, McCrone P, Knapp M. Early intervention for first episode psychosis: Broadening the ‐scope of economic estimates. Early Intervention in Psychiatry. 2014.25. Tsiachristas A, Thomas T, Leal J, Lennox BR. Economic impact of early intervention in psychosis services: results from a longitudinal retrospective controlled study in England. BMJ Open. 2016; 6(10): e012611.26. NHS England tNCCfMHatNIfHaCE. Implementing the Early Intervention in Psychosis Access and Waiting Time Standard: Guidance. (ed tNCCfMHatNIfHaCE NHS England). NHS England Publications Gateway, 2016.27. Gillespie P, O'Shea E, Smith SM, Cupples ME, Murphy AW. A comparison of medical records and patient questionnaires as sources for the estimation of costs within research studies and the implications for economic evaluation. Fam Pract. 2016.28. Lamont T, Barber N, Pury J, Fulop N, Garfield-Birkbeck S, Lilford R, et al. New approaches to evaluating complex health and care systems. BMJ. 2016; 352: i154.29. Heckman JJ, Ichimura H, Todd P. Matching as an Econometric Evaluation Estimator. The Review of Economic Studies. 1998; 65(2): 261-94.30. Addington D, Addington MDJ, Patten S. Relapse rates in an early psychosis treatment serviceActa Psychiatrica Scandinavica Volume 115, Issue 2. In: Acta Psychiatrica Scandinavica: 126-312007.31. Almond S, Knapp M, Francois C, Toumi M, Brugha T. Relapse in schizophrenia: costs, clinical outcomes and quality of life. Br J Psychiatry. 2004; 184: 346-51.