The effects of time and experience on nurses’ risk assessment

decisions: a signal detection analysis

C, Thompson1, L, Dalgleish2, T, Bucknall3, C, Estabrookes4, R, De Vos5, A, Hutchinson4, K, Fraser4, J, Binnekade5, G, Barrett6,

J, Saunders6

1University of York, UK;2University of Stirling, UK; 3 Deakin University, Australia; 4University of Alberta, Canada; 5University of Amsterdam, Netherlands; 6Bradford Hospitals NHS Trust, UK

Backgroundo 60% of cardiac arrests preventable1 o 50% of arrests have documented but

not-acted-on changes in “basic” data: heart rate, BP, urine output, conscious level etc. 2

o Nurses key link in preventing “failure to rescue”o 98% of calls to METs nurse-initiated3

o Transforming changes in status to MET call in only 2.8% of cases4

1Hodgetts et al 2002; 2Goldhill 2001; 3Cioffi 2000; 4Daffurn et al 1994

Background o Expertise and experience often

“confused”1

o “epidemiological” benefits of experience not easily seen in individual judgements and decisions2

o Intuitive judgement is modus operandi for nurses3

o Time pressure4 and irreducible uncertainty5 important clinical contexts

1Anders Ericsson 2007; 2Aiken et al. 2003; 3Thompson et al. 2005; 4Thompson 2001, 2004, Bucknall 2000; 5Eddy 1994

questions

o Does “generic” clinical experience improve the ability to detect the need to take action?

o Does “specialist” clinical experience improve the ability to detect the need to take action?

o How does time pressure impact on nurses’ decision making performance?

methodsSignal detection analysis1

  risk No risk

Yes TP+ FP-

no FN- TN+

1Stanislaw & Todorov 1999 Calculation of signal detection theory Measures, Behaviour research measures, instruments and computers 31(1), 137-149

methods

Thompson C, Dalgleish L et al. The effects of time pressure and experience on nurses' risk assessment decisions: a signal detection analysis. Nursing Research, 2008; 57(12): 302-311

methodso 50 clinical

scenarios via power point in wards/units

Methodso “Signal”

o MEWs (Modified Early Warning Score) clinical prediction rule1

o MEWS ≥5 = “at risk”o Thus 18 “signals” and 32 “no signals” from 50 scenarioso Scenario values randomly selected from 1 years MEWs

assessments in 1 UK acute Trust (n=1350)

o Time pressure = 10 seconds and a visual cue (clock symbol).o Time pressure = 26 scenarios; no time pressure = 24. o Cases mixed randomly to prevent primacy and recency effects

o Judgement = “would you intervene by contacting a senior nurse or doctor?” o nb: as per protocol in each site

1Subbe et al. 2001

analysiso N and proportions of hits and false

alarms calculatedo SDT indices d’ and ln(β) calculated1

o Experience made ordinalo 2 x mixed model ANOVA with d’ and ln(β)

as dependents and clinical experience (between subjects 4 levels) and time pressure (within subjects 2 levels)

o Country as a factor in all analysis

o Separate analysis looked at critical care experience and time pressure

1Stanislaw & Todorov 1999

participantso 245 acute or critical care nurses

o UK 95; Netherlands 50; Australia 50; Canada 50

o Sampled randomly in UK; convenience elsewhere

o Mean years registered 11.6 (SD 8.8)o Mean years in current specialty 8.8 (SD 6.7)o Mean age 34 years (SD 8.1) o 64% had more than a year’s critical care

experienceo Graduates:

o UK 6%; Canada 77%; Netherlands 40%; Australia 100%

o nb: assessing critical event risk was a common judgement for all the nurses

Results: time pressure

results: experience under pressureo All nurses performed better with no time

pressureo No significant interaction between

experience and time pressure on the d’ (signal detection ability) measure.

discussiono More time = greater accuracy and less

unwarranted (costly) interventiono Less time = more “failure to rescue” (14%

to 32%)o Dangers of spreading expertise too thinly

(critical care, METs, rapid response)o Variation in performance ?due to variations

in organisational contexto “Good enough” fast-and-frugal heuristics

used by nurses may (in the absence of feedback) may not be quite as good when analysed systematically.

conclusiono Time pressure masks nursing expertiseo Quantity of clinical experience ≠

expertiseo Quality of clinical experience =

expertiseo Nurses need to be taught the value of

clinical information, combating cognitive caution: clinical epidemiological ways of thinking

o We need to know more about the “signals” and “noise” that surrounds nursing judgement calls and decisions

Reference and contactThompson C et al. The effects of time pressure and experience on nurses' risk assessment decisions: a signal detection analysis. Nursing Research, 2008; 57(12): 302-311

Dr Carl Thompson Centre for Evidence Based NursingDepartment of Health SciencesArea 2, Seebohm Rowntree BuildingUniversity of YorkYork YO10 5DDUnited Kingdome: cat4@york.ac.uk t: +44 1904 321350