THE SURGICAL SAFETY CHECKLIST

Dr Jacqueline Hannam

Associate Professor Simon Mitchell

Department of AnaesthesiologyUniversity of Auckland

Complications after inpatient operations occur in up to 25% of patients

Reported crude mortality rate after major surgery is 0.5–5%

In industrialized countries, nearly half of all adverse events in hospitalized patients are related to surgical care, and at least half of these considered preventable

WHO SSC implemented at ACH in 2008 as part of the global pilot study (NEJM 2009;360:491-9)

Overall study outcome;Mortality pre vs post = 1.5 vs 0.8%Complications pre vs post = 11% vs 7%

The checklist at Auckland

Benjamin A BMJ 2008;336:1241-1245

Van Klei et al

What to audit? 1. Is it used?

Increased odds of complication or death if information sharing was omitted or poor either intra-operatively or at patient

handover.

What to audit? 2. Is it used correctly?

Compliance =

“verbal communication of that item by the checklist administrator or other OR team member during SSC

administration”

Engagement =

“Engagement was rated according to the number of OR teams engaged. At least one team member had to be

engaged in SSC administration for the team to be considered engaged, and engagement was defined as listening or contributing to SSC administration with

cessation of other activities and conversations.”

Definitions

Retrospective note review:

Possibly prone to inaccuracies and may not represent actual checklist compliance

Cannot measure team engagement with process

Direct observation:

Time consuming and potentially costly, but preferred option

Method of measurement?

• 41 SSC domains in two OR suites

• Observer rated compliance with domains and domain items using previous definitions

• Observer-rated compliance compared with self-reported compliance (boxes checked on SSC form in patient notes)

• Accuracy in documentation investigated using logistic regression analysis with adjustment for confounding factors (operating suite and SSC domain)

Accuracy of retrospective note review

Checking the checkers: an audit of self-reporting checklist use

• Domain compliance recording accurate in 75% of domains

• Item compliance recording was accurate for 51% of items • Likelihood of accurate self-reporting greater for items that

were observed as being completed than for those that were not (OR 22.85, 95% CI 10.78-48.42, p<0.0001)

• Example - item compliance reported as 86% at Suite 1 (68% true positive + 18% false positive) versus 82% of items at Suite 2 (15% + 67%).

At face value these seem comparable BUT item compliance as rated by observers was 68% at Suite 1 and just 15% at Suite 2!

Checking the checkers - results

Tendency to report compliance favourably when items are NOT completed

Reliance on self-reported rates of compliance with SSC is likely to grossly over-estimate true compliance

Big variation in observed compliance levels between OR suites within the same hospital, not detectable by retrospective note review

Inaccurate representation of checklist practices within centres could hinder attempts to identify areas where improvements could be made.

Checking the checkers - results

Audit 1:Direct observation100 cases at ACH

Compliance with administration of

checklist domainsand domain items

TeamEngagement

Primary outcomes Secondary outcomes

2 months, 2010 – 2011 (2 years post checklist roll out)

Domain complianceSign In: 99%Time out: 94 %Sign Out: 2 %

Item complianceMean (range)Sign In: 56% (27 – 100)Time out: 69% (33 – 100)Sign Out: NA

Team engagement

Entire staff

3 teams 2 teams 1 team

Sign inn=99

0% 3% 52% 45%

Time outn=94

15% 38% 35% 12%

Sign outn=2

0% 0% 50% 50%

Key findings of audit

• Sign In and Time Out are completed most of the time; including items that intuitively seem most important

• Sign out not being done!!

• Staff engagement sub-optimal

• A drift in administration standards appeared to have occurred since the study

Interventions?

• Present findings to OR staff

• Feedback and discussions with group

“Sign Out is not linked to a specific event in patient management”

• Link Sign Out to the end of the swab & instrument count

…re-audit to assess the intervention

Simultaneous audit of 100 cases at a study and a non-study hospital, 2011

Compliance with administration of

checklist domainsand domain items

TeamEngagement

Primary outcomes Secondary outcomes

Compliance with domain administration

Hospital 1

Study hospital

Hospital 2

Sign in 96% 31%

Time out 99% 48%

Sign out 22% 9%

Compliance with checklist item administration

Hospital 1

Study hospital

Hospital 2

Sign in 59% (3-100) 69% (3-100)

Time out 78% (15-100) 59% (2-98)

Sign out 74% (9-100) 44% (11-89)

Team engagement

All staff 3 teams 2 teams 1 team

Hospital 1 2 1 2 1 2 1 2

Sign in 0% 0% 1% 0% 39% 75% 60% 25%

Time out

0% 0% 14% 40% 47% 58% 38% 2%

Sign out 0% 0% 9% 0% 36% 33% 55% 67%

Led by anaesthetist at hospital 2

Led by surgeon at hospital 2

Key Points• A hospital which rolled out the checklist

independent of a study protocol exhibited poor compliance– This hospital may be more typical of

mainstream New Zealand– Implementation (or re-implementation)

strategies potentially make a significant difference

• A further drift in engagement at Hospital 1, but Sign Out had improved

• Senior team members leading domains gets better engagement

Next step – sustain improvements

• Attitude– Analysis and addressing of inappropriate beliefs

• Motivation– Education

• Addressing the more difficult quality issues

• Team behaviours

• Leadership– Involve all 3 professional groups– Change in OR checklist leadership

Current status – trialling OR team leadership

Do

mai

ns

Do

main

items