€¦ · web viewemail address: [email protected]. abstract. considerable research has been...

33
Dynamic Prediction of Transition to Psychosis Using Joint Modelling 1,2 H. P. Yuen, MSc* 3,4 A. Mackinnon, PhD 1,2 J. Hartmann, PhD 1,2 G. P. Amminger, MD, PhD (Habil) 1,2 C. Markulev, M Psych Clin 1,2 S. Lavoie, PhD 1 M. R. Sch äfer, MD 1,2,5 A. Polari, MD, FRANZCP 6 N. Mossaheb, MD 7 M. Schlögelhofer, MA 8 S. Smesny, MD, PhD 9 I. B. Hickie MD 10 G. Berger, MD 11 E. Y. H. Chen, MD 12 L. de Haan, MD, PhD 12 D. H. Nieman, PhD 13 M. Nordentoft, MD, PhD 14 A. Riecher-Rössler, MD, PhD 15 S. Verma, MD 1,16 A. Thompson, MD, MBBS 1,17 A. R. Yung, MD, FRANZCP 1,2 P.D. McGorry, MD, PhD 1,2 B. Nelson, PhD 1 Orygen, The National Centre of Excellence in Youth Mental Health, Melbourne, Australia 2 Centre for Youth Mental Health, The University of Melbourne, Australia 3 Centre for Mental Health, Melbourne School of Population and Global Health, The University of Melbourne, Australia. 4 Black Dog Institute and University of New South Wales, New South Wales, Australia. 5 Orygen Youth Health, Melbourne, Australia. 6 Department of Psychiatry and Psychotherapy, Clinical Division of Social Psychiatry, Medical University of Vienna, Austria 7 Department of Child and Adolescent Psychiatry, Medical University of Vienna, Austria 8 University Hospital Jena, Germany 1

Upload: others

Post on 04-Nov-2019

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Transition to Psychosis Using Joint Modelling

1,2H. P. Yuen, MSc*3,4A. Mackinnon, PhD1,2J. Hartmann, PhD1,2G. P. Amminger, MD, PhD (Habil)1,2C. Markulev, M Psych Clin1,2S. Lavoie, PhD1M. R. Schäfer, MD1,2,5A. Polari, MD, FRANZCP6N. Mossaheb, MD7M. Schlögelhofer, MA8S. Smesny, MD, PhD9I. B. Hickie MD10G. Berger, MD11E. Y. H. Chen, MD12L. de Haan, MD, PhD12D. H. Nieman, PhD13M. Nordentoft, MD, PhD14A. Riecher-Rössler, MD, PhD15S. Verma, MD1,16A. Thompson, MD, MBBS1,17A. R. Yung, MD, FRANZCP1,2P.D. McGorry, MD, PhD1,2B. Nelson, PhD1Orygen, The National Centre of Excellence in Youth Mental Health, Melbourne, Australia2Centre for Youth Mental Health, The University of Melbourne, Australia3Centre for Mental Health, Melbourne School of Population and Global Health, The University of Melbourne, Australia.4Black Dog Institute and University of New South Wales, New South Wales, Australia.5Orygen Youth Health, Melbourne, Australia.6Department of Psychiatry and Psychotherapy, Clinical Division of Social Psychiatry, Medical University of Vienna, Austria7Department of Child and Adolescent Psychiatry, Medical University of Vienna, Austria8University Hospital Jena, Germany9Brain and Mind Research Institute, University of Sydney, Australia10Child and Adolescent Psychiatric Service of the Canton of Zurich, Zurich, Switzerland11Department of Psychiatry, University of Hong Kong, Hong Kong12Academic Medical Center, Amsterdam, the Netherlands13Mental Health Centre Copenhagen, Mental Health Services in the Capital Region, Copenhagen University Hospital, Denmark14Psychiatric University Clinics Basel, Basel, Switzerland15Department of Psychosis, Institute of Mental Health, Singapore, Singapore16Division of Mental Health and Wellbeing, Warwick Medical School, University of Warwick, Coventry, England, and North Warwickshire Early Intervention in Psychosis Service, Coventry and Warwickshire NHS Partnership Trust, England17Institute of Brain, Behaviour and Mental Health, University of Manchester, Manchester, UK, and Greater Manchester West NHS Mental Health Foundation Trust, Manchester, England

1

Page 2: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

*Corresponding author:Hok Pan YuenOrygen, The National Centre of Excellence in Youth Mental Health,Locked Bag 10, ParkvilleVictoria 3052Australia+61 3 9342 2975Email address: [email protected]

2

Page 3: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

Abstract

Considerable research has been conducted seeking risk factors and constructing prediction

models for transition to psychosis in individuals at ultra-high risk (UHR). Nearly all such

research has only employed baseline predictors, i.e. data collected at the baseline time point,

even though longitudinal data on relevant measures such as psychopathology have often been

collected at various time points. Dynamic prediction, which is the updating of prediction at a

post-baseline assessment using baseline and longitudinal data accumulated up to that

assessment, has not been utilized in the UHR context. This study explored the use of dynamic

prediction and determined if it could enhance the prediction of frank psychosis onset in UHR

individuals. An emerging statistical methodology called joint modelling was used to

implement the dynamic prediction. Data from the NEURAPRO study (n=304 UHR

individuals), an intervention study with transition to psychosis study as the primary outcome,

were used to investigate dynamic predictors. Compared with the conventional approach of

using only baseline predictors, dynamic prediction using joint modelling showed significantly

better sensitivity, specificity and likelihood ratios. As dynamic prediction can provide an up-

to-date prediction for each individual at each new assessment post entry, it can be a useful

tool to help clinicians adjust their prognostic judgements based on the unfolding clinical

symptomatology of the patients. This study has shown that a dynamic approach to psychosis

prediction using joint modelling has the potential to aid clinicians in making decisions about

the provision of timely and personalized treatment to patients concerned.

Keywords: UHR, Transition to psychosis, Dynamic prediction, Joint modelling

Abstract word count: 247

Text body count: 4046

3

Page 4: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

HPY received support through an Australian Government Research Training Program

Scholarship. JH was supported by a University of Melbourne McKenzie Fellowship. BN was

supported by an NHMRC Senior Research Fellowship.

4

Page 5: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

1. Introduction

1.1 Background

Strategies for identifying people at ultra-high risk (UHR) of developing psychosis

(Klosterkötter, 2001; Schultze-Lutter et al., 2007; Schultze Lutter, 2009; Yung et al., 2004;

Yung, 1998; Yung et al., 1996) (otherwise termed ‘clinical high risk’) established over the

past twenty years have been an important step in providing early intervention to prevent or

mitigate the devastating effects of psychotic disorders. Upon receiving treatment, some of

these individuals will nevertheless progress to a psychotic disorder. Therefore much research

has focussed on identifying risk factors for transition to psychosis in UHR individuals.

Recent developments towards this goal include the use of machine learning (Koutsouleris et

al., 2015; Mechelli et al., 2016), Bayesian probabilistic models (Clark et al., 2016) and the

construction of risk calculators (Cannon et al., 2016; Fusar-Poli et al., 2017). These

developments, like nearly all past UHR research in the context of seeking risk factors for

transition, make use of baseline predictor variables, typically variables assessed at service

entry (Cannon et al., 2008; Nelson et al., 2013; Nieman et al., 2014; Ruhrmann et al., 2010;

Yung et al., 2004). Baseline predictor variables include fixed predictors whose values are

constant (such as gender) and the baseline values of measures (whose values can change over

time). However, the increasingly recognised changeable nature of psychopathology (Nelson

et al., 2017c; van Os, 2013) suggests that using only baseline information may be under-

utilizing the available information and limiting predictive ability. Hence interest has emerged

to investigate whether the changeable nature of psychopathology can be used to improve

prediction of the onset of psychosis (Nelson et al., 2017c; Yuen and Mackinnon, 2016; Yuen

et al., 2018). This is the idea behind dynamic prediction, which is to update prediction

continuously over time as more and more information about changes in patients’ conditions is

obtained. The aim of this paper is to apply dynamic prediction to a dataset on UHR

5

Page 6: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

individuals and to determine its potential usefulness over prediction based on baseline

predictors only.

1.2 Dynamic Prediction

Studies of psychosis prediction usually require follow-up of participants over time in order

to ascertain which individuals transition to psychosis and when they do so. The longitudinal

nature of these studies provides the opportunity to capture the changing characteristics of

psychopathology by conducting multiple assessments on various symptom measures over the

study period. Measures assessed repeatedly over time can be used to provide dynamic

prediction of the onset of psychosis and are called time-dependent predictors (TDPs) (or

covariates in some contexts). Specifically, dynamic prediction would be applied as follows.

Based on an existing dataset, appropriate fixed predictors and TDPs are used to construct a

dynamic prediction model. For a new patient, after the baseline assessments have been

completed, the patient’s data on the relevant fixed predictors and TDPs are input into the

dynamic prediction model to provide a risk estimate for the patient. At this point, only

baseline data is available and so only baseline data is used to estimate the risk. Based on the

risk estimate and any other relevant information, clinicians can make a decision about

indicated treatments. Assuming that the patient has not transitioned and is reassessed on a

subsequent occasion, say one month later, the patients’ data on the relevant fixed predictors

and TDPs are entered into the prediction model again to generate an updated risk estimate.

For this second round, the data of the TDPs used come from both the baseline and second

assessments. The updated risk estimate gives an indication of the current risk and how much

improvement or deterioration has occurred. The risk estimate can continue to be updated with

more longitudinal data of the TDPs obtained from further subsequent assessments until the

patient is discharged or transition occurs. Updated risk estimates can be used to make

decisions regarding modifying treatment type or intensity.

6

Page 7: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

In the past, a lack of suitable statistical methodology and software has impeded the use of

TDPs. Over the past two decades, statistical methodology that can combine the time-to-

transition aspect and the longitudinal aspect of TDPs in one model has emerged. The

methodology is called joint modelling (Chi and Ibrahim, 2006; Rizopoulos, 2012; Tang and

Tang, 2015; Tang et al., 2014). Corresponding software for implementing joint modelling has

also become available in mainstream statistical software packages (Gould et al., 2015; Yuen

and Mackinnon, 2016). With the advent of joint modelling and the corresponding software,

dynamic prediction has been examined in various areas of medicine (Ediebah et al., 2015;

Gueorguieva et al., 2012; Guler et al., 2014; Levine et al., 2015; Li et al., 2017; Mauguen et

al., 2015; Njagi et al., 2013; Pilla Reddy et al., 2012; Sweeting, 2017) but not in psychosis

prediction in UHR research. Our previous work has described the joint modelling

methodology and its use in the context of transition to psychosis (Yuen et al., 2018). In this

paper, we sought to determine whether dynamic prediction through joint modelling could

enhance the prediction of transition to psychosis among UHR individuals.

2. Methods

2.1 The Data

The data came from the NEURAPRO Study, which was a multi-centre placebo-controlled

randomized trial of the effect of omega-3 polyunsaturated fatty acids on transition rate and

other outcomes of UHR individuals. The intervention period was for 6 months. The sample

size was 304. There were in total 40 known transitions (see Supplement Section 1 for

definition of transition), 30 of which occurred within one year from baseline. No significant

difference was found between the two trial treatments in terms of transition rate and

symptomatic and functional outcomes. Details about the methodology of the study are

provided in Markulev et al (Markulev et al., 2017) and the results of the trial are reported in

McGorry et al (McGorry et al., 2017).

7

Page 8: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

Study assessments were conducted monthly during the first 6 months and then at months 9

and 12. In particular, these repeated assessments involved the following psychopathology

measures: Brief Psychiatric Rating Scale (BPRS) (McGorry et al., 1988), Montgomery

Asberg Depression Rating Scale (MADRS) (Montgomery and Asberg, 1979), Young Mania

Rating Scale (YMRS) (Young et al., 1978) and Scale for the Assessment of Negative

Symptoms (SANS) (Andreasen, 1982). These measurements were used as TDPs in

developing the dynamic prediction model.

In addition to TDPs, fixed predictors were also considered. They included: gender, age at

entry, duration between symptom onset and treatment/acceptance at UHR service, years of

education, ethnicity (Caucasian vs. non-Caucasian), migrant status (Yes=participant

immigrant or both parents immigrants and No=otherwise), recruiting site (Melbourne, Vienna

and other sites (Sydney, Basel, Zurich, Jena, Copenhagen, Hong Kong, Singapore and

Amsterdam)), functioning (Social and Occupational Functioning Assessment Scale (Goldman

et al., 1992) and Global Functioning: Social and Role Scales (Auther et al., 2006; Niendam et

al., 2006)) and Comprehensive Assessment of the At-Risk Mental State (Yung et al., 2005)

positive symptoms. The summary statistics of these variables and the baseline values of the

TDPs are presented in Table 1.

2.2 Joint Modelling

In joint modelling, the trajectories of the TDPs over time are estimated using linear mixed-

effects modelling (Fitzmaurice et al., 2011). The estimated trajectories are then incorporated

into the Cox regression framework (Lee and Wang, 2013) to enable the estimation of the

effects of TDPs on the risk of the event outcome (Rizopoulos, 2012), which is transition in

our context. The usual assumption underlying joint modelling is that the risk of transition

depends on the current values of the TDPs at any given time point. This is called the basic

joint model (Rizopoulos, 2012), which was used in this paper. Further information about joint

8

Page 9: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

modelling can be obtained from Rizopoulos (Rizopoulos, 2012) and Yuen et al (Yuen and

Mackinnon, 2016; Yuen et al., 2018).

The JM R package Version 1.4-0 (Rizopoulos, 2010; Rizopoulos, 2012) was used to

implement joint modelling. This package allows various specifications for the baseline hazard

of the Cox regression aspect. For the purpose of this paper, the baseline hazard was specified

as a piecewise-constant function, i.e. it was assumed to have different constant values at

different intervals over the timeframe concerned. Such a specification allows some degree of

flexibility because the baseline hazard is not assumed to be associated with any specific

probability distribution. The JM package also provides estimates for the survival probability,

which, in the present context, is the probability of remaining non-psychotic. The survival

probability can be estimated from a given joint model at a specified time for a particular

individual with particular values for the predictors concerned up to that point.

2.3 Analysis

The steps that were used in developing a dynamic prediction model of transition using

joint modelling were as follows. The level of significance used was 0.05.

1. An appropriate variable selection method was firstly used to determine which baseline

variables were significant predictors of transition. As different variable selection methods

could produce different results, both stepwise regression and least absolute shrinkage and

selection operator (LASSO) were used to provide a consistency check. Baseline variables

in this step included the fixed predictors mentioned above and baseline values of the

TDPs. Longitudinal information from the TDPs was not used in this step.

2. Joint modelling was then applied to determine the significance of each TDP individually

adjusting for the significant baseline predictors found in Step 1. The rationale underlying

this approach is that, since a prediction model based on only baseline data is simpler than

one based on longitudinal data, the latter would only be used if it could be shown that the

9

Page 10: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

longitudinal data enhances prediction significantly after adjusting for the contribution of

the significant baseline predictors.

3. Established joint modelling software accommodates only a single TDP. Accordingly, the

most significant TDP together with the significant baseline predictors found in step 1

were used to produce a prediction model using joint modelling. We refer to this as

JM_model.

In order to provide a benchmark for the joint model, a Cox regression prediction model using

only the significant baseline predictors found in step 1 was also included. We refer to this as

BL_model. The performance of the prediction models was then evaluated using the

procedure outlined below.

Prediction of transition within one year after entry was carried out using each of the

above two models. The timeframe of one year was used because the data only allowed the

estimation of the trajectories of the TDPs within this period. Prediction was carried out as

follows. For BL_model, the one-year survival probability for each individual was estimated

from the model. If the estimated survival probability was less than a specified cut-off value,

then the predicted one-year transition status was ‘yes’; otherwise it was ‘no’. The possible

cut-off values for the survival probability considered were 0, 0.01, 0.02, …, 1, i.e. the entire

probability range with increments of 0.01. For each cut-off probability, the predicted and

observed transition status for each individual were used to compute sensitivity and specificity

values.

For the joint model, the prediction of transition for each individual was determined by

estimating the one-year survival probability from the model incrementally. Specifically, for

each individual, a first estimation of the survival probability was made using the baseline

score of the TDP. If the estimated probability was less than a specified cut-off value, the

individual’s predicted one-year transition status was set as ‘yes’ and estimation stopped.

10

Page 11: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

Otherwise, a second estimation of the survival probability was made using the next available

follow-up score of the TDP together with its baseline score. Two conditions were considered

at this step. The first condition was whether the estimated survival probability fell below a

specified cut-off value. The second condition was whether the change in the estimated

survival probability compared with the estimated baseline survival probability was less than a

specified cut-off value. This second condition could indicate that insufficient improvement or

even deterioration had occurred in the patient’s condition. If either of these conditions were

satisfied, then the predicted one-year transition status was set as ‘yes’ and the estimation

stopped. Otherwise, the next follow-up score of the TDP together with all previous scores

were used in the next step to perform the prediction. This process continued until the

predicted transition status was set as ‘yes’ or until the scores were exhausted. Each of the two

conditions above had separate cut-off values. The entire probability range of 0 to 1 with

increments of 0.01 were used as possible cut-off values for the first condition. The second

condition pertained to change in probability and visual inspection of the estimated survival

probability values indicated that a reasonable range for the possible cut-offs was 0.1 to 0.1

with increments of 0.01. Sensitivity and specificity were computed for each combination of

the two sets of cut-off values.

The performance of the prediction models was compared using receiver operating

characteristic (ROC) curves. The area under the ROC curve (AUC) for each model was

computed using the trapezoidal rule. The significance of the difference between AUCs was

evaluated by estimating the standard deviation of the difference between the two AUCs using

500 bootstrap draws. The AUC difference divided by the standard deviation was then

compared to the normal distribution to provide a statistical test (Efron and Tibshirani, 1993).

Positive and negative likelihood ratios (Guyatt et al., 2015) were also used to compare the

prediction models.

11

Page 12: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

3. Results

The significant baseline predictors identified using stepwise Cox regression (step 1 above)

were baseline BPRS total (p=2×10-7), ethnicity (p=0.002) and migrant status (p=0.033). The

same variables were chosen using LASSO. Accordingly, BL_model was a Cox regression

prediction model consisting of these three variables. See supplement 2 for the significance of

the other baseline predictors after adjusting for these three chosen variables.

With the exception of BPRS total, the significance of each TDP was evaluated using joint

modelling after adjusting for baseline BPRS total, ethnicity and migrant status (step 2 above).

For the TDP BPRS total, the adjustment was done using ethnicity and migrant status only as

baseline BPRS total is part of the data of this TDP. The following TDPs were found to be

significant: BPRS total (p=4×10-8) and BPRS Psychotic subscale (p=0.001). See Table 2 for

the significance of the other TDPs. As BPRS total was the most significant TDP, JM_model

consisted of ethnicity, migrant status and BPRS total.

Figure 1 shows the ROC curves for the two models. It can be seen that the curve of

JM_model is completely above that of BL_model, i.e. it exhibits better prediction results. The

AUC of the joint model was significantly greater than that of the baseline model (p-

value=0.019). Figure 2 shows the likelihood ratios of the two models. Larger positive

likelihood ratios (LR+) and smaller negative likelihood ratios (LR–) indicate better

predictions. In the present context, a larger LR+ means a prediction of transition occurs more

frequently in transitioned cases than non-transitioned cases, whereas a smaller LR– means no

prediction of transition occurs less frequently in transitioned cases than non-transitioned

cases. As a general rule, LR+ should be greater than 2 and LR– should be less than 0.5 in

order to be practically useful: this desirable region is shown as the shaded region in Figure 2.

It can be seen that both models have points falling into the desirable region. However, all the

12

Page 13: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

points of BL_model in this region are surpassed by the points of the joint model in the sense

that the latter have better LR+ and/or LR- values.

To provide further insight into the advantage of joint modelling, the point in the ROC

curve closest to the ideal point (i.e. 100% true positive rate and 0% false positive rate) was

evaluated for JM_model and also for BL_model. For this point, the number of predicted

transitions were 97 and 88 respectively for JM_model and BL_model. The corresponding

sensitivity and specificity values were 82.8% and 72.4% respectively for JM_model and

69.0% and 73.8% respectively for BL_model. In this particular scenario, specificity was

about the same between the two models but JM_model showed a higher sensitivity, implying

that JM_model was able to detect more transitioned cases. Although the additional number of

transitions detected by JM_model in this scenario was 4, which was modest, it corresponded

to a 14 point gain in sensitivity with little loss in specificity. The TDP involved in JM_model

was BPRS total. Figure 3 shows the trajectories of BPRS total for these 4 cases. In this figure,

the highlighted point indicates the assessment point at which JM_model yielded a prediction

that transition would occur within the year. For all these cases, there was an early increase in

BPRS total, i.e. an early deterioration in terms of general psychopathology and transition was

correctly predicted at a post-baseline assessment.

4. Discussion and conclusion

4.1 Discussion

To date, psychosis prediction studies have relied on baseline assessment of relevant

variables. It has been argued that changes in measures over time should be taken into account

in predictive modelling, giving rise to dynamic prediction (Nelson et al., 2017c). Joint

modelling is a statistical tool that can incorporate the dynamic characteristics of

psychopathology into analysis of psychosis transition in the form of TDPs. Our previous

work showed that the effect of a TDP may be underestimated if only its baseline values are

13

Page 14: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

used (Yuen and Mackinnon, 2016; Yuen et al., 2018). The current paper has further shown

that the dynamic approach of prediction through joint modelling can provide a more powerful

means of prediction than the traditional static approach.

The model BL_model was developed using only baseline data. The model JM_model was

developed using joint modelling and allowed the incorporation of longitudinal data. The

results indicate that the joint model provided better sensitivity, specificity and likelihood

ratios than the baseline model. In the current dataset, those true positives detected by the joint

model at a post-baseline time point were identified quite early (within about two months from

entry). This shows that all the true positives detected by the joint model were identified either

at the baseline assessment or at a post-baseline assessment well before the actual time of

transition. The clinical implication of this result is that dynamic prediction may allow

sufficient time to implement an appropriate and targeted intervention to prevent the

occurrence of transition.

In order for dynamic prediction to be superior to prediction based on baseline data only,

two conditions need to be satisfied: (1) dynamic prediction must have better predictive ability

than baseline prediction and (2) dynamic prediction needs to identify true positive cases at a

time well before the actual time of transition so as to allow enough time to implement

appropriate interventions. Both conditions were satisfied in the analysis of the NEURAPRO

data. It is interesting to see that the best predictor turned out to be BPRS total, which is a

measure of general psychopathology. This result seems to suggest that an overall

psychopathology score may function as a stronger predictor than variables which are more

disorder-specific such as negative symptoms (SANS). This suggests the importance from a

clinical point of view of paying attention to persistence or increase in general

psychopathology as indicative of potential psychosis risk, and not to confine clinical attention

to symptom domains that are considered to be more central to psychotic disorders such as

14

Page 15: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

positive and negative psychotic symptoms. However, we note that there is inconsistency with

the findings of one of our previous studies, the Pace400 study (Nelson et al., 2013). In the

Pace400 study, we analysed predictors for transition to psychosis within a sample of about

400 UHR individuals and BPRS total did not appear to be a significant predictor. While it is

true that only fixed predictors were used in the Pace400 analysis and no dynamic prediction

was involved, BPRS total was a significant predictor even in the baseline model of the

current data. This inconsistency speaks to the importance of continuing to search for reliable

and replicable clinical predictors in this population (Nelson et al., 2017b). Another possible

reason for the inconsistency is that there were not enough transitions in the current data to

tease out the effects of a multiplicity of predictors. If there had been more transitions, it is

possible that some of the TDPs may have remained significant after adjusting for the fixed

predictors. Further research assembling large representative UHR samples with longitudinal

data should be a priority in future exploration of dynamic prediction models.

It is possible to construct a risk calculator for transition using joint modelling in a similar

way as others have done using baseline predictors (Cannon et al., 2016; Fusar-Poli et al.,

2017). The difference is that the latter provides a single risk estimate based on baseline

measures whereas a joint model can provide updated estimates at each assessment point. A

joint model could also function as an alert system. An alert can be triggered when the risk

estimate is too high and also when the improvement in the risk estimate over time is too little.

For example, an alert might be triggered if the risk estimate is above 80% and also when the

improvement in risk is less than 2%. In this scenario, if a patient has a risk estimate of 75% at

both baseline and a subsequent assessment, then an alert would be triggered even though the

estimate at both time points are below 80%. The rationale for this is that we would expect

improvement over time in response to treatment. If the patient does not show enough

improvement or even deteriorates (in terms of risk), then it would be clinically useful to

15

Page 16: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

provide an alert even if the risk remains below the specified risk threshold. This is a

difference between dynamic and static prediction in that the former can aid decisions using

change in risk whereas the latter cannot.

4.2 Limitations

The NEURAPRO study was a randomized trial comparing fish oil with placebo in

transition rate. Since no difference was found between the two treatment groups, they were

ignored in our analysis. However, as placebo is not a normal treatment in practice, its

presence could be regarded as a limitation in the development of prediction models.

Currently, established joint modelling software packages, including the JM package used

in this paper, are restricted in allowing the incorporation of only one TDP in a model.

Multiple TDPs need to be combined into a single predictor in order for the software packages

to be utilized. A pragmatic way to combine TDPs is simply to sum them on each occasion of

measurement either without or with standardization (using corresponding baseline means and

SDs). This has been used in other applications of joint modelling (Battes et al., 2015).

However, this is a crude way of utilizing multiple TDPs and the resulting composite variable

may not always be meaningful. For the current data, the TDPs that were found to be

significant individually were BPRS total and BPRS Psychotic subscale. Since one is the

subscale of the other, it is debatable whether it is meaningful to sum them and we decided to

use only BPRS total (the clearly more powerful one) in the joint model. As joint modelling is

an active research area, the limitation of fitting only one TDP may well be overcome in the

near future. In fact, a package for the R environment called joineRML (Hickey et al., 2018)

was recently released. It is designed to accommodate multiple TDPs in one model. However,

it is a user-developed package which has been released for a short time only. So it is

advisable not to employ it for general use until extended exploration of its performance has

been conducted.

16

Page 17: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

4.3 Conclusion

In this paper, we have demonstrated the potential usefulness of joint modelling in

psychosis prediction and provided empirical support for the value of dynamic prediction in

psychosis research. Further research such as expanding this approach beyond the basic joint

model and incorporating other potential predictors such as neurocognitive variables and

biomarkers would be worthwhile.{Rizopoulos, 2012 #1573} Clinical decision making and

prognostic judgements are generally ‘adaptive’ in nature, i.e. they react to and are updated

based on the unfolding clinical symptomatology of the patient, rather than relying solely on

the profile of the patient’s first clinical presentation (Nelson et al., 2017a). We have shown

that a dynamic approach to psychosis prediction using joint modelling may provide an

empirically-based and rigorous guide for clinical decision making in regard to modifying

preventive intervention in response to the evolution of the patient’s clinical profile.

17

Page 18: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

References

Andreasen, N.C., 1982. Negative symptoms in schizophrenia. Definition and reliability. Archives of general psychiatry 39(7), 784-788.

Auther, A.M., Smith, C.W., Cornblatt, B., 2006. Global Functioning: Social Scale (GF: Social). Zucker-Hillside Hospital, Glen Oaks, NY.

Battes, L.C., Caliskan, K., Rizopoulos, D., Constantinescu, A.A., Robertus, J.L., Akkerhuis, M., Manintveld, O.C., Boersma, E., Kardys, I., 2015. Repeated measurements of NT-pro-B-type natriuretic peptide, troponin T or C-reactive protein do not predict future allograft rejection in heart transplant recipients. Transplantation 99(3), 580-585.

Cannon, T.D., Cadenhead, K., Cornblatt, B., Woods, S.W., Addington, J., Walker, E., Seidman, L.J., Perkins, D., Tsuang, M., McGlashan, T., Heinssen, R., 2008. Prediction of psychosis in youth at high clinical risk: a multisite longitudinal study in North America. Archives of general psychiatry 65(1), 28-37.

Cannon, T.D., Yu, C., Addington, J., Bearden, C.E., Cadenhead, K.S., Cornblatt, B.A., Heinssen, R., Jeffries, C.D., Mathalon, D.H., McGlashan, T.H., Perkins, D.O., Seidman, L.J., Tsuang, M.T., Walker, E.F., Woods, S.W., Kattan, M.W., 2016. An individualized risk calculator for research in prodromal psychosis. American Journal of Psychiatry 173(10), 980-988.

Chi, Y.Y., Ibrahim, J.G., 2006. Joint models for multivariate longitudinal and multivariate survival data. Biometrics 62(2), 432-445.

Clark, S.R., Baune, B.T., Schubert, K.O., Lavoie, S., Smesny, S., Rice, S.M., Schäfer, M.R., Benninger, F., Feucht, M., Klier, C.M., McGorry, P.D., Amminger, G.P., 2016. Prediction of transition from ultra-high risk to first-episode psychosis using a probabilistic model combining history, clinical assessment and fatty-acid biomarkers. Translational psychiatry 6(9).

Ediebah, D.E., Galindo-Garre, F., Uitdehaag, B.M.J., Ringash, J., Reijneveld, J.C., Dirven, L., Zikos, E., Coens, C., van den Bent, M.J., Bottomley, A., Taphoorn, M.J.B., 2015. Joint modeling of longitudinal health-related quality of life data and survival. Quality of Life Research 24(4), 795-804.

Efron, B., Tibshirani, R., 1993. An introduction to the bootstrap. Chapman & Hall, New York.

Fitzmaurice, G.M., Laird, N.M., Ware, J.H., 2011. Applied longitudinal analysis, 2nd ed. Wiley, Hoboken, N.J.

Fusar-Poli, P., Rutigliano, G., Stahl, D., Davies, C., Bonoldi, I., Reilly, T., McGuire, P., 2017. Development and validation of a clinically based risk calculator for the transdiagnostic prediction of psychosis. JAMA psychiatry 74(5), 493-500.

Goldman, H.H., Skodol, A.E., Lave, T.R., 1992. Revising axis V for DSM-IV: A review of measures of social functioning. American Journal of Psychiatry 149(9), 1148-1156.

Gould, A.L., Boye, M.E., Crowther, M.J., Ibrahim, J.G., Quartey, G., Micallef, S., Bois, F.Y., 2015. Joint modeling of survival and longitudinal non-survival data: current methods and issues. Report of the DIA Bayesian joint modeling working group. Statistics in medicine 34(14), 2181-2195.

Gueorguieva, R., Rosenheck, R., Lin, H., 2012. Joint modelling of longitudinal outcome and interval-censored competing risk dropout in a schizophrenia clinical trial. Journal of the Royal Statistical Society. Series A: Statistics in Society 175(2), 417-433.

Guler, I., Calaza-Díaz, L., Faes, C., Cadarso-Suárez, C., Giraldez, E., Gude, F., 2014. Joint modelling for longitudinal and time-to-event data: Application to liver transplantation data, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 580-593.

18

Page 19: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

Guyatt, G., Rennie, D., Meade, M., Cook, D., 2015. Users' guides to the medical literature. Essentials of evidence-based clinical practice, Third edition. ed. McGraw-Hill Education Medical, New York.

Hickey, G.L., Philipson, P., Jorgensen, A., Kolamunnage-Dona, R., 2018. joineRML: Joint Modelling of multivariate longitudinal data and time-to-event outcome. R package version 0.4.1.

Klosterkötter, J., 2001. Diagnosing schizophrenia in the initial prodromal phase. Archives of general psychiatry 58(2), 158-164.

Koutsouleris, N., Riecher-Rössler, A., Meisenzahl, E.M., Smieskova, R., Studerus, E., Kambeitz-Ilankovic, L., Von Saldern, S., Cabral, C., Reiser, M., Falkai, P., Borgwardt, S., 2015. Detecting the Psychosis Prodrome Across High-Risk Populations Using Neuroanatomical Biomarkers. Schizophrenia bulletin 41(2), 471-482.

Lee, E.T., Wang, J.W., 2013. Statistical methods for survival data analysis, 4th ed. Wiley, Hoboken, New Jersey.

Levine, S.Z., Goldberg, Y., Samara, M., Davis, J.M., Leucht, S., 2015. Joint modeling of dropout and outcome in three pivotal clinical trials of schizophrenia. Schizophrenia research 164(1-3), 122-126.

Li, K., Furr-Stimming, E., Paulsen, J.S., Luo, S., 2017. Dynamic prediction of motor diagnosis in Huntington's disease using a joint modeling approach. Journal of Huntington's Disease 6(2), 127-137.

Markulev, C., McGorry, P.D., Nelson, B., Yuen, H.P., Schaefer, M., Yung, A.R., Thompson, A., Berger, G., Mossaheb, N., Schlögelhofer, M., Smesny, S., de Haan, L., Riecher-Rössler, A., Nordentoft, M., Chen, E.Y.H., Verma, S., Hickie, I., Amminger, G.P., 2017. NEURAPRO-E study protocol: a multicentre randomized controlled trial of omega-3 fatty acids and cognitive-behavioural case management for patients at ultra high risk of schizophrenia and other psychotic disorders. Early intervention in psychiatry 11(5), 418-428.

Mauguen, A., Rachet, B., Mathoulin-Pélissier, S., Lawrence, G.M., Siesling, S., MacGrogan, G., Laurent, A., Rondeau, V., 2015. Validation of death prediction after breast cancer relapses using joint models. BMC Medical Research Methodology 15(1).

McGorry, P.D., Goodwin, R.J., Stuart, G.W., 1988. The development, use, and reliability of the brief psychiatric rating scale (nursing modification)--an assessment procedure for the nursing team in clinical and research settings. Comprehensive psychiatry 29(6), 575-587.

McGorry, P.D., Nelson, B., Markulev, C., Yuen, H.P., Schäfer, M.R., Mossaheb, N., Schlögelhofer, M., Smesny, S., Hickie, I.B., Berger, G.E., Chen, E.Y.H., De Haan, L., Nieman, D.H., Nordentoft, M., Riecher-Rössler, A., Verma, S., Thompson, A., Yung, A.R., Paul Amminger, G., 2017. Effect of ω-3 polyunsaturated fatty acids in young people at ultrahigh risk for psychotic disorders: The NEURAPRO randomized clinical trial. JAMA psychiatry 74(1), 19-27.

Mechelli, A., Lin, A., Wood, S., McGorry, P., Amminger, P., Tognin, S., McGuire, P., Young, J., Nelson, B., Yung, A., 2016. Using clinical information to make individualized prognostic predictions in people at ultra high risk for psychosis. Schizophrenia research.

Montgomery, S.A., Asberg, M., 1979. A new depression scale designed to be sensitive to change. British Journal of Psychiatry 134(4), 382-389.

Nelson, B., Amminger, G.P., Yuen, H.P., Wallis, N., J. Kerr, M., Dixon, L., Carter, C., Loewy, R., Niendam, T.A., Shumway, M., Morris, S., Blasioli, J., McGorry, P.D., 2017a. Staged Treatment in Early Psychosis: A sequential multiple assignment randomised trial of interventions for ultra high risk of psychosis patients. Early intervention in psychiatry.

Nelson, B., McGorry, P., Hartmann, J., 2017b. A moving target: how risk for mental disorder can be modeled in dynamic rather than static terms., in: McGorry, P., Hickie, I. (Eds.),

19

Page 20: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

Clinical staging in psychiatry: making diagnosis meaningful for clinical care and research. Cambridge University Press., Cambridge. In press.

Nelson, B., McGorry, P.D., Wichers, M., Wigman, J.T.W., Hartmann, J.A., 2017c. Moving From Static to Dynamic Models of the Onset of Mental Disorder: A Review. JAMA psychiatry 74(5), 528-534.

Nelson, B., Yuen, H.P., Wood, S.J., et al., 2013. Long-term follow-up of a group at ultra high risk (“prodromal”) for psychosis: The pace 400 study. JAMA psychiatry 70(8), 793-802.

Nieman, D.H., Ruhrmann, S., Dragt, S., Soen, F., van-Tricht, M.J., Koelman, J.H.T.M., Bour, L.J., Velthorst, E., Becker, H.E., Weiser, M., Linszen, D.H., de Haan, L., 2014. Psychosis Prediction: Stratification of Risk Estimation With Information-Processing and Premorbid Functioning Variables. Schizophrenia bulletin 40(6), 1482-1490.

Niendam, T.A., Bearden, C.E., Johnson, J.K., Cannon, T.D., 2006. Global Functioning: Role Scale (GF: Role). University of California, Los Angeles, CA.

Njagi, E.N., Rizopoulos, D., Molenberghs, G., Dendale, P., Willekens, K., 2013. A joint survival-longitudinal modelling approach for the dynamic prediction of rehospitalization in telemonitored chronic heart failure patients. Statistical Modelling 13(3), 179-198.

Pilla Reddy, V., Kozielska, M., Johnson, M., Suleiman, A.A., Vermeulen, A., Liu, J., De Greef, R., Groothuis, G.M.M., Danhof, M., Proost, J.H., 2012. Modelling and simulation of the positive and negative syndrome scale (PANSS) time course and dropout hazard in placebo Arms of schizophrenia clinical trials. Clinical Pharmacokinetics 51(4), 261-275.

Rizopoulos, D., 2010. JM: An R package for the joint modelling of longitudinal and time-to-event data. J Stat Softw 35(9), 1-33.

Rizopoulos, D., 2012. Joint models for longitudinal and time-to-event data : with applications in R. CRC Press, Boca Raton.

Ruhrmann, S., Schultze-Lutter, F., Salokangas, R.K., Heinimaa, M., Linszen, D., Dingemans, P., Birchwood, M., Patterson, P., Juckel, G., Heinz, A., Morrison, A., Lewis, S., von Reventlow, H.G., Klosterkotter, J., 2010. Prediction of psychosis in adolescents and young adults at high risk: results from the prospective European prediction of psychosis study. Archives of general psychiatry 67(3), 241-251.

Schultze-Lutter, F., Klosterkötter, J., Picker, H., Steinmeyer, E., Ruhrmann, S., 2007. Predicting first-episode psychosis by basic symptom criteria. Clinical Neuropsychiatry 4(1), 11-22.

Schultze Lutter, F., 2009. Subjective Symptoms of Schizophrenia in Research and the Clinic: The Basic Symptom Concept. Schizophrenia bulletin 35(1), 5-8.

Sweeting, M.J., 2017. Using predictions from a joint model for longitudinal and survival data to inform the optimal time of intervention in an abdominal aortic aneurysm screening programme. Biometrical J 59(6), 1247-1260.

Tang, A.M., Tang, N.S., 2015. Semiparametric Bayesian inference on skew-normal joint modeling of multivariate longitudinal and survival data. Statistics in medicine 34(5), 824-843.

Tang, N.-S., Tang, A.-M., Pan, D.-D., 2014. Semiparametric Bayesian joint models of multivariate longitudinal and survival data. Comput Stat Data An 77, 113-129.

van Os, J., 2013. The dynamics of subthreshold psychopathology: implications for diagnosis and treatment. The American journal of psychiatry 170(7), 695-698.

Young, R.C., Biggs, J.T., Ziegler, V.E., Meyer, D.A., 1978. A rating scale for mania: Reliability, validity and sensitivity. British Journal of Psychiatry 133(11), 429-435.

Yuen, H.P., Mackinnon, A., 2016. Performance of joint modelling of time-to-event data with time-dependent predictors: an assessment based on transition to psychosis data. PeerJ 4, e2582.

20

Page 21: €¦ · Web viewEmail address: hokpan.yuen@orygen.org.au. Abstract. Considerable research has been conducted seeking risk factors and constructing prediction models for transition

Dynamic Prediction of Psychosis

Yuen, H.P., Mackinnon, A., Nelson, B., 2018. A new method for analysing transition to psychosis: Joint modelling of time-to-event outcome with time-dependent predictors. International Journal of Methods in Psychiatric Research 27(1).

Yung, A., Phillips, L., Yuen, H., McGorry, P., 2004. Risk factors for psychosis in an ultra high-risk group: psychopathology and clinical features. Schizophrenia research 67(2-3), 131-142.

Yung, A.R., 1998. Prediction of psychosis: A step towards indicated prevention of schizophrenia. British journal of psychiatry 172(JUNE SUPPL. 33), 14-20.

Yung, A.R., McGorry, P.D., McFarlane, C.A., Jackson, H.J., Patton, G.C., Rakkar, A., 1996. Monitoring and care of young people at incipient risk of psychosis. Schizophrenia bulletin 22(2), 283-303.

Yung, A.R., Yuen, H.P., McGorry, P.D., Phillips, L.J., Kelly, D., Dell'Olio, M., Francey, S.M., Cosgrave, E.M., Killackey, E., Stanford, C., Godfrey, K., Buckby, J., 2005. Mapping the onset of psychosis: The Comprehensive Assessment of At-Risk Mental States. Australian and New Zealand Journal of Psychiatry 39(11-12), 964-971.

21