Intensive and Critical Care Nursing (2009) 25, 283—293

avai lab le at www.sc iencedi rec t .com

journa l homepage: www.e lsev ier .com/ iccn

ORIGINAL ARTICLE

The development of a clinical markers score topredict readmission to paediatric intensive care

Sophie Lintona,∗, Chelsea Granta, Juliet Pellegrini a, Andrew Davidsonb,1

a Paediatric Intensive Care Unit, Royal Children’s Hospital, Flemington Rd, Parkville 3052, Australiab Dept of Anaesthesia & Pain Management, Royal Children’s Hospital, Clinical Research Development Officer, Murdoch Children’sResearch Institute, Flemington Rd, Parkville 3052, Australia

Accepted 29 July 2009

KEYWORDSPaediatric intensivecare;Readmission;ICU liaison nurse

SummaryObjective: Readmission to ICU following discharge is associated with increased length of stay(LOS), increased rates of mortality, morbidity and resource consumption. Reducing readmissionrates is one of the key aims of the Intensive Care Unit liaison nurse (ICULN). Our objective wasto identify factors associated with readmission which were identifiable both from demographicsand from each LN visit, and from this develop a clinical markers score.Methods: In this case control study, cases were all children who required ICU readmission within48 h of discharge over two years. The comparison group included children who were dischargedon the same day as those who required readmission. Using multivariate logistic regressionanalysis the factors associated with ICU readmission were identified. The factors were furtheranalysed for the development of the clinical markers score.Results: The factors associated with readmission to ICU included high oxygen requirement,tachypnoea, age >10 years, age <2 weeks, LN assessment, high risk of readmission (ROR) score,

longer LOS and admission under oncology.Conclusion: From our study we found that the development of a score to predict the risk ofreadmission to ICU required a combination of subjective LN assessment, respiratory status andpatient characteristics collected on discharge from ICU. This score can now be implementedand guide the LN to prioritise visits for children at increased risk of ICU readmission.

blish

Crown Copyright © 2009 Pu

∗ Corresponding author. Tel.: +61 3 9345 5211;fax: +61 3 9345 6960.

E-mail addresses: sophie.linton@rch.org.au (S. Linton),chelsea.caffin@rch.org.au (C. Grant), juliet.pellegrini@rch.org.au(J. Pellegrini), andrew.davidson@rch.org.au (A. Davidson).

1 Tel:. +61 3 9345 4901.

I

TdmwIa

0964-3397/$ — see front matter. Crown Copyright © 2009 Published by Edoi:10.1016/j.iccn.2009.07.003

ed by Elsevier Ltd. All rights reserved.

ntroduction

he Royal Children’s Hospital (RCH), Melbourne is the only

edicated paediatric hospital in Victoria (population fiveillion). The Intensive Care Unit liaison nurse (ICULN) roleas introduced at RCH in 2004 to bridge the gap between

CU and the wards. The LNs provide follow-up visits anddvanced nursing care for all children discharged from ICU

lsevier Ltd. All rights reserved.

2

itrtr

L

R

Ifnoffboato2H

aTctqReqdagd

wp(tetmgrh1ic(

Ir

Wamsfc

c2(fAhspm0m

ugtriw1rriFrt

Sp

MtapmlGtd2mdGmvtctreuBi2

Sc

84

ncluding support for the child, family and staff throughouthe transition phase. One of the main aims of the role is toeduce ICU readmission rates. This may be more effective ifhe LN were able to identify those children with the greatestisk of readmission to ICU.

iterature review

eadmission to ICU: rates and consequences

deally when the decision is made to discharge a patientrom ICU, it would be with the expectation that they willot require readmission. However, unplanned readmissionsccur for many reasons some of which inevitably cannot beoreseen. Also, as a consequence of the constant pressureor access to ICU beds, children may be discharged on theasis that they are the most fit for transfer (i.e. at least riskn the ward), rather than that they are absolutely readynd prepared for the ward (Caffin, 2005). A reduction inhe number of patients readmitted to ICU is one of the keybjectives of the ICU liaison or outreach nurse role (Ball,005; Caffin, 2005; Green and Williams, 2006; Haines, 2005;aines et al., 2006).

The current readmission rate for our ICU is about 5% orpproximately 75 children each year (Caffin et al., 2007).his compares favourably with the 7.78% readmission rateited by Elliot (2006) in his review of adult readmission rateso ICU. Readmission to ICU is often used as a measure ofuality of care provision in hospitals (Elliot, 2006). However,osenberg and Watts (2000) dispute this, arguing there is novidence that ICU readmissions correlate with the overalluality of the hospital, and indeed may identify a substan-ard process of care within ICU. Either way, Campbell etl. (2008) comment that early readmissions are a patientroup that merits special attention, because they have aisproportionately high mortality.

The complications associated with readmission have beenell explored within the adult population but there is littleublished for children. In adult surgical patients Alban et al.2006) found that readmission to ICU significantly increasedhe risk of death. From a systematic review of the adult lit-rature Rosenberg and Watts (2000) report that compared tohose discharged and not readmitted, those patients read-itted to ICU had higher mortality rates and on average

reater than twice the hospital length of stay (LOS). Otheresearchers have reported that patients readmitted to ICUave a mortality rate up to six times higher and are up to1 times more likely to die in hospital (Elliot, 2006). Theres also a significant increase in anxiety levels and resourceonsumption for patients who require readmission to ICUCaffin, 2005).

dentifying those at risk of readmission and theole of the LN

ith good evidence of the significant adverse consequences

ssociated with readmission to ICU it makes sense to takeeasures to reduce readmission. In the adult population,

ome factors have already been identified as risk factorsor readmission. After hours discharges from adult intensiveare units has been reported in multiple papers as a signifi-

WraT

S. Linton et al.

ant risk factor for readmission to ICU (Goldfrad and Rowan,000; Pilcher et al., 2007; Santamaria, 2007). Santamaria2007) reports readmission rates from 1.3 to 1.7 times higheror those patients discharged out of hours (1800—0600).

combined review of both Australian and New Zealandospitals using the ANZICS (Australian & New Zealand Inten-ive Care Society) patient database, found that adult ICUatients discharged to the ward out of hours had higher read-ission and mortality rates. Risk peaked between 0300 and

400 h. (Pilcher et al., 2007). Premature discharge from ICUay also be a risk factor for readmission (Elliot, 2006).To assist in the reduction of readmissions, it would be

seful if it could be predicted with more accuracy those atreatest risk. This would then allow the ICU LN to priori-ise their activities and allocate time to those with a higherisk of readmission. The early detection and optimal caren the critically ill adult has been shown to be associatedith improved outcomes (Buist et al., 2002; Goldhill et al.,999; McGloin et al., 1999) and reduced ICU readmissionates (Ball, 2005). Green and Edmonds (2004) reported aeduction in readmissions from 2.3% to 0.5% following thentroduction of the ICU LN role in their adult medical ICU.ollowing the introduction of the role at RCH there was aeduction in readmissions from 5.4% in the year prior to 4.8%he following year (Caffin et al., 2007).

cores used in the detection of critically ill adultatients

uch has been reported in the adult literature about iden-ifying and managing seriously ill patients on the wardsnd strategies to improve the detection of deterioratingatients. The concepts of early warning scores/systems andedical emergency teams (MET) have increased in popu-

arity over the last few years (Green and Williams, 2006;ao et al., 2007). Early warning scores/systems (EWS) areools designed to help ward staff to identify patients who areeteriorating and requiring assistance (Goldhill and McNarry,004). Abnormal vital signs, also referred to as clinicalarkers, have been well identified as predictive of acuteeterioration in adult patients (Green and Williams, 2006;oldhill and McNarry, 2004). Most EWS utilise a mixture ofeasurement of vital signs and clinical assessment to pro-

ide a score that guides the bedside nurse and alerts themo activate and notify others for further management. Medi-al emergency teams (MET) respond immediately to an alerthat a patient has transgressed certain (physiological) crite-ia. In some studies established METs have been shown to beffective in the reduction of incidence and mortality fromnexpected cardiac arrest in hospital (Bellomo et al., 2003;uist et al., 2002). However; there are many different EWS

n use with varying reliability, validity and utility (Gao et al.,007).

cores used in the detection of critically illhildren

hile the concept of early identification and the need forecognition of a critically ill patient may be similar for bothdults and children, the criteria for response are different.he data from MET and EWS in adults cannot be transferred

agoppLbtcdwctmee

D

TcLiwctaptwcwrIdsmd

cclIofbAorawio

rum

The development of a clinical markers score

directly into the paediatric population. The identificationof an unwell or deteriorating child presents a challenge.The physiological variation in children that is reflected inthe different developmental stages makes it impossible tohave one rule for all ages (Monaghan, 2005). In addition theunderlying illness in children and reasons that children arecritically ill is quite different in comparison with adults. Ithas also been suggested that children can compensate welland show a late but sudden deterioration or crash, makingearly identification of significant underlying illness more dif-ficult (Haines, 2005; Haines et al., 2006). In summary it is notpossible to extrapolate the results in relation to recognitionof deterioration in adult patients directly into paediatrics.

Using criteria designed for children, MET teams havebeen found to be effective in a paediatric setting (Tibballset al., 2005), and EWSs have been developed in children(Monaghan, 2005; Haines et al., 2006; Duncan et al., 2006).However, these EWS for children were not designed primarilyas predictors of possible ICU readmission and to our knowl-edge there are no data identifying risk factors associatedwith readmission following discharge from a paediatric ICU.

Aims

In this project our objective was to identify significant fac-tors which are identifiable at LN visits that are associatedwith the unplanned readmission to ICU in children. Fromthese factors we aimed to develop a clinical markers scorethat the LN team could use to identify children who maybenefit from an increased level of care on the ward.

Methods

Setting and study population

The Royal Children’s Hospital, Melbourne is a leadingprovider of specialist public health services for children andadolescents and is the major specialist paediatric hospitalin Victoria, caring for children from Tasmania and southernNew South Wales as well as other states around Australiaand overseas.

The Paediatric ICU at RCH is the largest in Australia.Approximately 1400 infants and children are admitted eachyear, representing all medical and surgical paediatric sub-specialties. The unit has a high acuity, with 70% of admissionsrequiring intubation and mechanical ventilation. In additionto providing tertiary PICU services for Victoria and Tasmania,several national programmes are catered for, including hearttransplantation, respiratory ECMO, long-term VAD, surgicaltreatment of Hypoplastic Left Heart Syndrome and intestinaltransplantation. Children are discharged from ICU directly tothe wards, there being no high dependency, intermediate orstep down unit. The ICU LN team provides routine follow-upvisits for all children discharged from ICU.

This is a retrospective case control study using data thatwere collected routinely by the LN team. Approval for this

project as a clinical audit was sought and gained from theRoyal Children’s Hospital Human Research Ethics Commit-tee. Data were analysed from children discharged from ICUat RCH over a two year period from 1st April 2006—31stMarch 2008. In this study the readmission group was defined

snowd

285

s all children discharged from ICU who required emer-ency or non-elective readmission to the unit within 48 hf discharge. Children who were readmitted for electiverocedures were excluded as they were considered to belanned admissions, not a readmission. On occasions the ICUN may be asked to review a child on the ward who had nevereen an ICU inpatient. Although another interesting groupo investigate, these referrals were not included. For theontrol group, we identified the days of discharge for chil-ren who subsequently required unplanned ICU readmissionithin 48 h. For each of those days we identified all otherhildren who were discharged on that day and not readmit-ed and used these children as controls. This provided someatch for seasonal variation in ICU bed occupancy and dis-

ase patterns, as well as variation in medical and nursingxperience.

ata collection

he major primary source of the data was the patient visithart; data collected routinely at discharge and at all ICUN visits. Over the two year period, four different LNs werenvolved in the data collection, however all data collectionas overseen by the primary investigator. Data such as dis-harge and readmission date and time were also drawn fromhe ICU database (Static). All the data were entered inton EpiData database from the patient visit charts by therimary investigator. Data were not entered for any visithat occurred later than 48 h after discharge. The datasetas then cleaned, with some outlying or unusual data beinghecked against patient notes. Where possible missing dataere also completed using patient notes and other ICU

ecords. It is very unlikely that any child discharged fromCU on those days was not seen by the LN team as all chil-ren discharged from ICU are routinely identified by the LNervice. The LN team checks the ICU discharge records everyorning and is kept informed of discharges throughout theay.

The information collected prior to discharge included thehild’s age, admission date, reason for admission, past medi-al history, extubation date, medical unit, destination ward,ength of stay in ICU (LOS), and date and time of discharge.n our institution the medical unit refers to the specialityf the lead medical practitioner. A ward may have childrenrom multiple medical units. LOS corresponded to the num-er of days that the child was present within ICU at midnight.stay of less than 24 h was counted as one day. At the time

f discharge a subjective risk of readmission (ROR) score wasecorded by both the bedside nurse and ICU LN. The LN wasware of the bedside nurses’ score but the bedside nurseas unaware of the LN score. The ROR score was given to

dentify each child’s risk of readmission to ICU within 48 hf discharge with 1 = high, 2 = possible and 3 = unlikely.

At each of the ICU LN visits the date and time wasecorded, we also took note if the visit was a routine followp or was made by referral. At each visit a general assess-ent was made of the child and a subjective assessment

core of 0,1 or 2 was given. A score of 0 indicated the LN isot concerned, a score of 1 the LN is concerned and a scoref 2 indicated the LN is very concerned. Observation chartsere assessed and the last recorded set of observationsocumented (which would usually be within 1 h). If there

286 S. Linton et al.

Table 1 Demographics of patients in each group.

Non-readmissions n = 261 Readmissions n = 114 P value

Gender 0.25Male 161 (62%) 63 (60%)Female 100 (38%) 51 (40%)

Age, days; mean (range) 1222 (2—6961) 1421 (2—6788) 0.70LOS days; mean (range) 3.8 (1—66) 4.3 (1—23) 0.005

Time of discharge 0.03In hours (0800—1800) 226 (87%) 89 (78%)Out of hours (1800—0800) 34 (13%) 25 (22%)

Medical unit 0.05Adolescent medicine 1(0.4%) 0Cardiac 139 (53%) 59 (52%)Child development 8 (3%) 8 (7%)Endocrine 3 (1%) 0ENT 8 (3%) 0Gastroenterology 5 (2%) 3 (3%)General medicine 43 (16%) 18 (15%)General surgery 9 (3%) 1 (1%)Neurology & neurosurgery 8 (3%) 3 (3%)Oncology 5 (2%) 10 (9%)Orthopaedics 7 (3%) 4 (4%)Plastics 5 (2%) 1 (1%)Renal 4 (2%) 0Thoracics 16 (6%) 7 (6%)

Ward 0.34 North (Ortho) 7 (3%) 4 (4%)3 East (adolescent) 7 (3%) 4 (4%)4 Main (Gen surg) 10 (4%) 3 (3%)5th floor (Gen med) 72 (28%) 31 (27%)

wastmtiNcowa(iiw

A

D

Ss

sdUmnsc

L

Oitrlcai

6th floor (oncology) 4 (2%)7th floor (cardiac & renal) 144 (55%)8th floor (neuro) 17 (7%)

ere no current observations or the charts were unavail-ble, then the ICU LN would usually take and document aet of observations. A set of observations included tempera-ure, heart rate (HR), blood pressure (systolic, diastolic andean), respiratory rate (RR), oxygen requirement, satura-

ions, neurological status, fluid loss, intravenous analgesia,notropic requirements and any other intravenous infusions.eurological status was assessed by the LN as one of threeategories, alert and appropriate, decreased consciousnessr only responds to pain. Fluid loss refers to any type ofound or fluid drainage other than urine and was categoriseds minimal (<1 ml/kg/h), moderate (<3 ml/kg/h) or large>3 ml/kg/h). Analgesia referred to any intravenous opioidnfusion and was recorded. Likewise inotrope referred to anyntravenous inotropic infusion and the drug and infusion rateas noted on each visit.

nalysis

escriptive statistics

ummary statistics of categorical and binary data are pre-ented as frequency and percentages. Continuous data are

awt

a

3 (3%)60 (53%)

5 (4%)

ummarised with mean and standard deviation if normallyistributed and median and inter-quartile ranges if skewed.nivariate analyses were then performed comparing read-ission and control groups. T-tests were used for continuous

ormally distributed data and rank-sum test if the data werekewed. Categorical and binary data were compared withhi square tests.

ogistic regression to identify risk factors

nce the two groups were compared the next step was todentify the effect of each variable adjusting for other fac-ors. To do this a logistic regression was performed. A logisticegression requires binary data therefore before doing aogistic regression continuous data were dichotomised. Thehoice of cut off to create binary outcomes was determinedfter visually examining the spread of data points. Also its difficult to interpret a logistic regression where variables

re very similar. Therefore where it appeared that factorsere very similar (such as ROR-RN and ROR-LN) only one of

he factors was included in the regression analysis.In this study we aimed to determine the factors associ-

ted with readmission identified at each ICU LN visit using

287

Table 3 Number of LN visits for each patient group within48 h.

Non-readmissionsn = 261

Readmissionsn = 114

Number of visits0 0 9 (8%)1 38 (15%) 37(32%)2 134 (51%) 38 (33%)3 76 (29%) 20 (18%)4 12 (5%) 7 (6%)

vt

vmhmpiea(n

R

TraaafRg

The development of a clinical markers score

all data available to the LN at that time. Therefore in thelogistic regression each visit is treated as a separate inde-pendent data point. Thus the results from the regressiondo not reflect the risk of readmission for the child dur-ing their whole transition period but the risk at each LNvisit.

From the regression analysis factors were identifiedwhere there was good evidence (P < 0.05) for an associa-tion with readmission. Those factors identified with largeodds ratios were then considered for developing a clinicalmarkers score to predict those patients at risk of readmis-sion. Logistic regression generates odds ratios rather thanrisk ratios, however even though an odds ratio is not alwaysa prefect measure of risk, it is often reasonable to use oddsratios in this setting (Myles and Gin, 2004).

Results

Population characteristics in each group

Over the two year period, 2675 children were dischargedfrom ICU at RCH. In this study the data from 375 dischargedchildren were analysed. For these 375 children, there were804 ICU LN visits recorded. 261 of the children did notrequire readmission to ICU within 48 h of discharge and 114were readmitted within the 48 h timeframe.

The demographics of the group of patients who wereeither readmitted or not readmitted are summarised inTable 1. Patients who were readmitted within 48 h hadlonger LOS in ICU (P = 0.005) and were more likely to bedischarged out of hours (P = 0.03). There was some indica-tion that medical unit was also associated with readmission(P = 0.05), with the Child Development and Oncology unitshaving the highest readmission rates. There was no evidencefor any association between readmission and age, gender ofthe child, and the ward the child was discharged to. Whenconsidering the assessment of likelihood of readmission bynursing staff at time of discharge there was a very strongevidence for an association between ROR-LN and ROR-N andreadmission (P < 0.001) (Table 2).

Table 3 indicates the number of ICU LN visits in eachgroup. There were nine children who were readmitted buthad not been seen at all by the LN. In further analysis thesechildren were excluded, as they do not add to the genera-tion of risk associated with a LN visit. The mean number of

Table 2 Risk of readmission (ROR) score for patients ondischarge.

Non-readmissionsn = 261

Readmissionsn = 114

P value

ROR-LN <0.0011 0 4 (4%)2 46 (22%) 46 (51%)3 163 (88%) 40 (44%)

ROR-RN <0.0011 0 2 (2%)2 44 (22%) 47 (53%)3 152 (78%) 40 (45%)

Rmudtctp

rtnrdtt

apy<

5 1(0.5%) 2 (2%)6 0 1 (1%)

isits was slightly less in the readmission group compared tohe control group (2.12 versus 2.24 visits, P = 0.006).

Table 4 summarises data collected at each individualisit. There was strong evidence that readmissions wereore likely to be patients who were referrals (P < 0.001), or

ad high assessment scores (P < 0.001), high oxygen require-ents (P < 0.001), high HR (P < 0.001), high RR (P < 0.001),oor neurological status (P < 0.001), or requiring certainnotropic infusions (P = 0.002). There was very marginalvidence for an association with readmission for temper-ture (P = 0.04), oxygen saturation (P = 0.03), time of visitP = 0.04) and intravenous analgesia (P = 0.03). Gender didot show an association with readmission.

isk factors identified in logistic regression

o identify the particular factors that are associated witheadmission when adjusted for each other, we entered vari-bles into a logistic regression. As described earlier, suchregression is best understood if highly collinear variables

re avoided and there are dichotomous variables. There-ore first we looked for highly collinear variables and foundOR-RN and ROR-LN to be highly collinear. We thereforeenerated a new variable that combined the ROR-RN andOR-LN assessments. Then temperature, oxygen require-ent, HR, RR, neurological status and intravenous inotrope

se and ROR-RN and ROR-LN were dichotomised. This wasone by looking at the spread and choosing cut off valueshat appeared to best divide the groups and also made senselinically. We did not dichotomise LN assessment scores, ashe three ICU LN assessment scores were all in themselvesredictive.

As part of the normal growth and development of a child,esting HR and RR fall with age. In the data, for HR and RRhere was, as expected, a general fall in rate with age in theon-readmission group. It is interesting to note that in theeadmission group, the RR (and to a lesser extent the HR)id not change as much with age (Figs. 1 and 2). Because ofhis, and to keep the score simple, we chose specific valueso dichotomise HR and RR rather than age adjusted values.

Age was managed differently. Examining the spread ofge and readmission showed that there was a bimodal riskattern with high readmission rates in the very old and veryoung. Therefore one variable for >10 years and another for2 weeks of age were generated.

288 S. Linton et al.

Table 4 Data collected at each visit when considering every LN visit.

Non-readmissionsn = 587

Readmissionsn = 217

P value

Referral visits <0.001Yes 5 (1%) 37 (18%)No 572 (99%) 173 (82%)

Time of visit 0.04Before noon 280 (48%) 85 (39%)After noon 307 (52%) 131 (61%)

Assessment <0.0010 467 (80%) 71 (33%)1 112 (19%) 109 (51%)2 4 (1%) 32 (15%)Visit number; mean (range) 1.8 (1—5) 1.8 (1—6) 0.8Temp, celsius; mean ± SD 36.9 ± 0.62 37.0 ± 0.65 0.04Oxygen, l min−1; mean (range) 0.4 (0—6) 2.2 (0—15) <0.001Heart rate, min−1; mean ± SD 121 ± 23 129 ± 22 <0.001Mean blood pressure, mmHg; mean ± SD 67.6 ± 14.3 68.2 ± 20.1 0.8Systolic blood pressure, mmHg; mean ± SD 98.4 ± 17.0 96.4 ± 20.6 0.3Diastolic blood pressure, mmHg; mean ± SD 54.4 ± 12.5 53.6 ± 15.0 0.6Respiratory rate, min−1; mean ± SD 32.6 ± 11.8 40.6 ± 13.1 <0.001Oxygen saturation, %; mean ± SD 94.3 ± 7.3 93.0 ± 8.1 0.03

Neurological assessment <0.001Awake and alert 556 (96%) 186 (89%)Decreased consciousness 21 (4%) 24 (11%)Responsive to pain only 1 (0.2%) 0

Fluid loss 0.2Nil 493 (85%) 186 (91%) NilMinimal (<1 ml/kg/h) 55 (10%) 13 (6%) Minimal (<1 ml/kg/h)Moderate (<3 ml/kg/h) 21 (4%) 5 (2%) Moderate (<3 ml/kg/h)Large (>3 ml/kg/h) 9 (2%) 1 (0.5%) Large (>3 ml/kg/h)

Analgesia 0.03Nil 429 (74%) 159 (76%) NilIV Morphine (<10 mcg/kg/h) 106 (18%) 26 (12%) IV Morphine (<10 mcg/kg/h)IV Morphine (>10 mcg/kg/h) 38 (7%) 24 (12%) IV Morphine (>10 mcg/kg/h)IV Morphine + other IV analgesia 4 (1%) 0 IV Morphine + other IV analgesia

Inotropes 0.002Nil 553 (95%) 191 (91%)Dobutamine/dopamine (≤2.5 mcg/kg/min) 8 (1%) 3 (1%)Dobutamine/dopamine (≤5 mcg/kg/min) 4 (1%) 10 (5%)Milrinone (≤0.75 mcg/kg/min) 16 (3%) 5 (2%)

t is re

lt(dtrtwrap

T

TaarT

Note that there may be multiple visits for each patient. Each visi

Table 5 lists all the factors that were entered into theogistic regression. Regression analysis can be used to assesshe impact of each of several predictor variables separatelyMyles and Gin, 2004). The results of the regression areetailed in Table 6. As a result of the analysis eight fac-ors were found to be still independently associated witheadmission to ICU: high oxygen requirement (>1 l/min),

achypnoea (RR >30 breaths/min), age > 10 years, age < 2eeks, higher LN assessment scores, an assessment that

isk of readmission (ROR) is high or possible by RN or LNt discharge, stay in ICU >2 days and being an oncologyatient.

ooppa

garded as an independent data point.

he clinical markers score

hese eight factors (at risk criteria) were used to generateclinical markers score (see appendix). Referral was also

dded as this seemed a logical addition and had a high oddsatio even if there was a non-statistically significant P value.o generate a total score each factor was given a value of 0

r 1, except assessment, which could be 0, 1 or 2. The sumf these values was generated by simple addition giving aossible total score between 0 and 9. (Note that a 10 is notossible as a child cannot be both less than two weeks oldnd greater than 10 years old).

The development of a clinical markers score 289

irato

crts

Figure 1 Resp

Clinical interpretation of these results requires con-sideration of their probability (Myles and Gin, 2004). Toimplement the score into our clinical LN practice, the mostimportant questions are if the score can predict the likeli-

hood of future readmission or non-readmission. Thereforewe looked at the calculated positive and negative predic-tive values for each cut off on the score. These results areshown in Table 7. This can be explained in other words, if the

mtta

Figure 2 Heart r

ry rate and age.

hild scored 0 then there was a 97% chance they were noteadmitted within the first 48 h. If they scored 1 or greaterhen there is a 32% chance they were readmitted at sometage within 48 h after discharge. (A score of 1 or greater

eans they fulfil any one or more of the at risk criteria) If

he child scored 5 or higher then there was a 100% chancehe child were readmitted to ICU at some stage within 48 hfter discharge.

ate and age.

290

Table 5 Factors entered into logistic regression.

Factor for regression How factor generated

High temperature Temperature greater than 37.5High oxygenrequirement

On more than 1 l/min of oxygen

Tachycardia Heart rate greater than120 bpm

Tachypnoea Respiratory rate greater than30 rpm

Abnormalneurological statusOn any inotropes On any inotropes at LN visitHigh-risk assessmenton discharge (ROR)

Either RN or LN classified thechild at time of discharge aspossible or high risk ofreadmission

Old age Older than 10 years oldYoung neonate Age less than 2 weeksOncology patient Oncology patientChild developmentpatient

CDR patient

Referral Referral rather than routine LNvisit

Long stay in ICU In ICU for greater than 48 hpm LN visit Liaison visit after noonDischarge in hours Discharge from ICU

0800—1800 h

T

Ot

TmfdasStraatts

L

Tttatmtiltaiibs

Assessment 1 LN ‘‘concerned’’Assessment 2 LN ‘‘very concerned’’Male Gender being male

he objective score

ne aim of this project was to develop an objective toolo reduce variation between members of the LN team.

ob

nc

Table 6 Results of logistic regression.

Factor Odds ratio

High oxygen requirement 8.20Tachypnoea 5.09Old age 5.94Young neonate 6.79Assessment 1 3.62Assessment 2 60.9High-risk assessment on discharge 2.57Long stay in ICU 2.22Oncology patient 3.30pm LN visit 1.42High temperature 0.65Discharge ‘in hours’ 0.68Male 0.77Referral 1.89Child development patient 1.43Abnormal neurological status 0.76Tachycardia 0.92On any inotropes 1.11

Note that in the regression each child may have multiple visits and eac

S. Linton et al.

he assessment score at each visit and the risk of read-ission scores at discharge (ROR-RN and ROR-LN) are all

actors that are subjective. These assessments may beependent on the experience of the individual LN. This ispotential weakness of the score. To assess this, a second

core was compiled which was called the score-objective.core-objective included all factors except these subjec-ive scores. The score-objective was therefore: high oxygenequirement (>1 l/min), tachypnoea (RR >30 breaths/min),ge >10 years or age < 2 weeks, stay in ICU >2 days, beingn oncology patient and if the visit was a referral ratherhan routine. The score-objective range is thus 0—6. Usinghis score the positive and negative predictive values arelightly inferior to the full score (Table 8).

imitations

here are several limitations to our study. Although we knowhat all patients readmitted to ICU within 48 h included inhis project were unplanned or emergency, we did not lookt cause of readmission. This added information might iden-ify common themes or highlight those readmissions thatay have been preventable. From our data we are unsure if

he readmission was a result of the initial problem or if wast a new problem, or if it was preventable or not. We alsoooked only at readmission as an outcome measure ratherhan long-term survival or disability. Readmission is usuallyssociated with poorer longer term outcomes but causations not always clear and reducing readmission may not resultn changes in longer term outcome. In future studies it woulde helpful to also look at long-term outcome. Also, in ourtudy readmission to ICU was defined as occurring within 48 h

f discharge. It may be that the risk profile for readmissioneyond 48 h is different.

Many of the adult papers looking at readmission to ICUote the severity of illness on admission to ICU. We did notollect this data, as it is not part of our routine LN practice.

95% Confidence intervals P value

4.28—15.68 <0.0012.61—9.94 <0.0012.58—13.66 <0.0012.56—18.05 <0.0012.10—6.24 <0.001

10.45—355.2 <0.0011.45—4.55 0.0011.34—3.68 0.0020.93—11.68 0.0640.86—2.34 0.180.32—1.33 0.240.34—1.37 0.280.46—1.29 0.320.43—8.20 0.400.46—4.47 0.540.26—2.23 0.620.53—1.59 0.770.34—3.70 0.86

h visit is regarded as an independent point.

The development of a clinical markers score 291

Table 7 Negative and positive predictive values, sensitivity and specificity for cut offs on the score.

Score cut off Positive predictive value forscoring cut off or higher

Negative predictive value forscoring lower than cut off

Sensitivity at cut off Specificity at cut off

1 32.0% 97.1% 98.2% 23%2 41.4% 93.7% 90.3% 52.8%3 61.8% 89.1% 72.4% 83.5%4 81.8% 78.6% 42.4% 95.7%5 100% 77.4% 21.2% 100%6 100% 75.1% 10.2% 100%7 100% 73.8% 4.1% 100%

iiw

airpItgTtds

r

D

Twrto

•

8 100% 73.1%9 100% 73.1%

This information could however provide useful information.In this score we assess LOS. This is perhaps a surrogatemeasure for underlying severity of illness. LOS has someadvantages over the other complex scores, as it is very easyto calculate. There may also be other variables which wedid not collect that may be better or useful predictors.

In this study the LN and ICU nurses were not blindedto other factors, therefore the subjective scores may beinfluenced by the other factors entered into the analysis.Similarly the decision to readmit children to the ICU maybe influenced by the LN’s impression. This would obviouslyproduce a bias toward LN subjective assessment being asso-ciated with readmission.

For the control group, we chose children discharged onthe same day. We do not know of any bias that this mayintroduce, however it is conceivable it could introduce anunknown bias.

The score has been generated from a sample populationover two years. It has not been tested or validated in a novelsample. A predictive score should be prospectively validatedon a separate group of patients where it would be expectedthat it would not to perform quite as well as the sample fromwhich it was generated (Myles and Gin, 2004). The next stepis thus to prospectively assess the score in a new sample ofpatients.

Each institution will be different and our findings may notbe applicable in another setting. Every children’s hospitalhas a different patient population and each ICU has different

responsibilities or functions within the hospital. For examplethe types of surgery performed, the option of a high depen-dency unit, trauma load etc. Therefore a score generatedfor one institution may not be applicable to another. Sim-

•••••

Table 8 Negative and positive predictive values, sensitivity and

Score cut off Positive predictive value forscoring cut off or higher

Negative predictscoring lower tha

1 33.3% 94.1%2 44.6% 86.6%3 79.1% 79.7%4 100% 75.2%5 100% 73.8%6 100% 73.1%

0.5% 100%0.5% 100%

larly there are different pressures on ICU beds with somenstitutions managing more critically unwell children on theard.

This score should not necessarily be seen as a risk ofdmission to ICU for all children, as the scores were specif-cally gathered within the LN service from children whoequired readmission to ICU. Certainly the profile of theopulation of children who require unplanned admission toCU would be another interesting group to look at in fur-her research. This analysis and score combined both factorsathered at each LN visit and factors known at ICU discharge.herefore care should be taken when using or comparinghis score with other scores that predicts readmission atischarge or simply identifies sick children based on vitaligns.

In summary, this score should be validated with furtheresearch both within our institution and other sites.

iscussion

his study aimed to identify factors collected by the LNhich would help identify those children at greater risk of

eadmission to ICU. From our analysis of the data we foundhat factors associated with readmission to ICU within 48 hf discharge included:

high oxygen requirement

tachypnoea (RR >30)age >10 yearsage <2 weeksLN assessment (1 or 2)high ROR score

specificity for cut offs on the score-objective.

ive value forn cut off

Sensitivity at cut off Specificity at cut off

94.9% 29.%71.9% 67.0%33.2% 96.8%10.6% 100%4.1% 100%0.5% 100%

2

••

iteHpr

sAastLaurti2

lnmpUhfghaw

hpfiwropiatCft

rscotacsiboc

oaLIaIntaGvII

idparatnit

rLupaIbIcds

tlppm

C

IouIIoorta

92

longer LOSadmission under the oncology unit

With this information LNs may be able to more reliablydentify those children at increased risk of readmission andherefore allow them to prioritise these patients with anscalation of care on the ward and more follow-up visits.opefully this would improve the long-term outcome and inarticular reduce the chance that children would deterio-ate to the point of needing readmission to ICU.

From the factors found to be associated with readmis-ion a clinical markers score was developed (see appendix). We included ‘referral’ into the clinical markers score aslthough it was not represented in the final statistical analy-is, if staff consider that they are ‘worried’ about a patienthen they have transgressed RCH MET criteria. As the ICUN role has developed and become accepted more childrenre referred to the service. We encourage staff to informs of any child they are worried about as the importance ofecognition of patient deterioration is well known. Interven-ions aimed at reducing readmissions to ICU require timelydentification of patients at highest risk (Campbell et al.,008).

A major goal of the score is to determine children atow risk and high risk of readmission to ICU. Including theine predictive factors and referral, the range of the clinicalarkers score is from 0 to 9. Note it is not 0—10 as it is notossible for one patient to be both <2 weeks and >10 years.sing this score, if a child scores 0, and therefore does notave any of the at risk criteria, then the LN may considerocussing care elsewhere. Also the higher the score then thereater the risk is of readmission. Children with scores of 3 origher had a more than 50% chance of needing readmissionnd therefore it would be reasonable to suggest childrenho score 3 or higher may require particular attention.

In their study of inpatients in a large adult teachingospital (Goldhill and McNarry, 2004) conclude that simplehysiological observations identify high-risk patients. Thisnding is reinforced by Rosenberg & Watts’ (2000) review,hich concludes that unstable vital signs, especially respi-

atory and heart rates are the most consistent predictorsf readmission to ICU. Cretikos et al. (2008) in their recentaper on RRs in adult patients, note that an abnormal RRs an important predictor of serious events such as cardiacrrest and admission to ICU. Worsening RR but not HR prioro ICU admission was also observed by Goldhill et al. (1999).onsistent with these findings, the two highest predictiveactors we found from our study (oxygen requirement andachypnoea) reflect the patient’s respiratory status.

Our results show that LOS in ICU was associated witheadmission. It may be that LOS reflects the underlyingeverity of illness and it is therefore not surprising thosehildren who require longer stays in ICU are at greater riskf readmission. Age (either >10 years or <2 weeks) is iden-ified as being associated with readmission to ICU. Theseges are the extremes of the population of children that weare for. Children less than two weeks of age have highly

pecialised needs. If this is combined with recent criticalllness requiring intensive care these babies are likely toe quite unstable. The population of children 10 years andver was an unexpected finding for which there is no obviousause.

rItoT

S. Linton et al.

The final factors identified that place a patient at riskf readmission and included in the clinical markers scorere subjective. The ROR score taken at discharge and theN assessment made at each patient visit. All nurses in ourCU LN team have a postgraduate qualification in paedi-tric critical care and extensive clinical nursing experience.t would seem that the knowledge and skills of an experturse together with an understanding of the capabilities ofhe areas outside ICU is useful in predicting the likelihoodchild will require readmission to ICU. This is supported byreen and Williams (2006) who conclude from their inter-entional study of an EWS for adult patients that an expertCU nurse is effective in identifying ward patient warrantingCU admission.

As our original aim was to develop a more objective score,n the final part of the score analysis we looked at the pre-ictive value of a score without the subjective component. Aurely objective score would reduce variability between LNsnd might have been preferable for reproducibility. Aftereview we found that the score-objective was less usefult predicting those children at greater risk of readmissionhat the initial score incorporating the subjective compo-ent and therefore any clinical marker score implementednto the LN’s clinical practice should incorporate a subjec-ive component to it.

Based on the results from this study we plan to incorpo-ate the clinical markers score into our LN practice. The ICUN visit sheet will be redrafted to enable easy and clear doc-mentation and calculation of the score. The score will aidrioritisation in ICU LN care particularly when LN resourcesre stretched. The score will also be useful in directing theCU LN care to children identified as high risk of readmissiony increasing frequency of visits, alerting ward nurses andCU staff and an extension of the ICU LN service beyond theurrent 48-h follow-up timeframe. However, at this stage weo not plan to use the score to ‘discharge’ children with 0core from the ICU LN follow-up service.

We shall continue to re-evaluate and potentially modifyhe score. Over the next 12 months we will continue to col-ect data as before. After 12 months we plan to reassess theredictive power of the score using the same data analysisrograms generated as part of the development of a clinicalarkers score to predict readmission project.

onclusions

n conclusion, readmission to ICU is associated with adverseutcomes. The ICU LN service currently provides follow-p visits for all children discharged from ICU. Part of theCU LN role is to identify children at risk of readmission toCU. Identification of predictive factors and developmentf a clinical markers score to assist with the recognitionf children at risk of readmission has not been previouslyeported. In this study we identified individual patient fac-ors apparent at the time of discharge and factors gatheredt each LN visit, which were associated with an increased

isk of readmission within the 48 h following discharge fromCU. From the patient factors identified we have developedhe ICU LN clinical markers score. A mix of subjective andbjective markers was superior to a purely objective score.he clinical markers score to predict the risk of readmission

D

E

G

G

G

G

G

G

H

H

M

M

M

P

R

S

2007;35(4):475—6.Tibballs J, Kinney S, Duke T, Oakley E, Hennessy M. Reduction of

The development of a clinical markers score

to ICU required a combination of subjective liaison nurseassessment, abnormal respiratory status and patient char-acteristics. It is hoped that this score will assist the LN teamto identify those children who would benefit from escala-tion of care on the ward, and hence not only reduce theirrisk of readmission to ICU but also improve their long-termoutcome.

Appendix A.

ICU LN clinical markers score, PICU RCH.

Data at discharge ScoreOncology patient 0/1Age <2 weeks 0/1Age >10 years 0/1LOS >48 h 0/1ROR-RN or ROR-LN >1 0/1

Data at visitResp rate >30 resps/min 0/1Oxygen >1 l/min 0/1LN assessment 0/1/2Referral 0/1

/9

References

Alban R, Nisim A, Ho J, Nishi G, Shabot M. Readmission to surgi-cal intensive care increases severity-adjusted patient mortality.Journal of Trauma 2006;60(5):1027—31.

Ball C. Ensuring a successful discharge from intensive care (edito-rial). Intensive & Critical Care Nursing 2005;21:1—4.

Bellomo R, Goldsmith D, Uchino S, Buckmaster J, Hart G, OpdamH, et al. A prospective before and after trial of a medical emer-gency team. Medical Journal of Australia 2003;179(6):283—7.

Buist M, Moore G, Bernard S, Waxma B, Anderson J, Nguyen T. Effectsof a medical emergency team on reduction of incidence of andmortality from unexpected cardiac arrests in hospital: prelimi-nary study. British Medical Journal 2002;324:1—6.

Caffin C. Paediatric intensive care liaison nurse report on the one-year trial position. Melbourne: Royal Children’s Hospital; 2005.

Caffin C, Linton S, Pellegrini J. Introduction of a liaison nurse in atertiary paediatric intensive care unit. Intensive & Critical CareNursing 2007;23:226—33.

Campbell A, Cook J, Adey G, Cuthbertson B. Predicting death and

readmission after intensive care discharge. British Journal ofAnaesthesia 2008;100(5):656—62.

Cretikos M, Bellomo R, Hillman K, Chen J, Finfer S, Flabouris A.Respiratory rate: the neglected vital sign. Medical Journal ofAustralia 2008;188(11.):657—9.

293

uncan H, Hutchison J, Parshuram C. The pediatric early warn-ing system score: a severity of illness score to predict urgentmedical need in hospitalized children. Journal of Critical Care2006;21:271—9.

lliot M. Readmission to intensive care: a review of the literature.Australian Critical Care 2006;19(3):96—104.

ao H, McDonnell A, Harrison D, Moore T, Adam S, Daly K, et al.Systematic review and evaluation of physiological track and trig-ger warning systems for identifying at-risk patients on the ward.Intensive Care Medicine 2007;33:667—79.

oldfrad C, Rowan K. Consequences of discharge from intensive careat night. Lancet 2000;355:1138—42.

oldhill D, McNarry A. Physiological abnormalities in early warningscores are related to mortality in adult patients. British Journalof Anaesthesia 2004;92(6):882—4.

oldhill D, White S, Sumner A. Physiological values and proceduresin the 24 hours before ICU admission from the ward. Anaesthesia1999;54:529—34.

reen A, Edmonds L. Bridging the gap between the intensive careunit and general wards- the ICU Liaison Nurse. Intensive & Crit-ical Care Nursing 2004;20:133—43.

reen A, Williams A. An evaluation of an early warning clin-ical marker referral tool. Intensive & Critical Care Nursing2006;22:274—82.

aines C. Acutely ill children within ward areas- care provisionand possible development strategies. Nursing in Critical Care2005;10(2):98—106.

aines C, Perrott M, Weir P. Promoting care for acutely ill children-Development and evaluation of a Paediatric Early Warning Tool.Intensive & Critical Care Nursing 2006;22:73—81.

cGloin H, Adam S, Singer M. Unexpected deaths and referrals tointensive care of patients on general wards. Are some casespotentially avoidable? Journal of the Royal College of Physiciansof London 1999;33:255—9.

onaghan A. Detecting and managing deterioration in children. Pae-diatric Nursing 2005;17(1):32—5.

yles P, Gin T. Statistical methods in anaesthesia and intensive care.Edinburgh: Butterwoth Heinemann; 2004.

ilcher D, Duke G, George C, Bailey M, Hart G. After hours dischargefrom intensive care increases the risk of readmission and death.Anaesthesia & Intensive Care 2007;35(4):477—85.

osenberg A, Watts C. Patients readmitted to ICUs: a sys-tematic review of risk factors and outcomes. Chest2000;118(2):492—502.

antamaria J. After-hours discharge from intensive care: impacton outcome (editorial). Anaesthesia and Intensive Care

paediatric in-patient cardiac arrest and death with a medicalemergency team: preliminary results. Archives of Disease inChildhood 2005;90:1148—52.