Check it Out! Can a post-procedure checklist improve quality in interventional radiology (IR)?

Alec Kherlopian BA, Jennifer Broder MD, Osagie Ebekozien* MBBS, MPH, Alex Norbash MD, MHCM, Glenn Barest MD Department of Radiology, Boston Medical Center and Boston University School of Medicine, Boston University School of Public Health, Harvard School of Public Health*, Boston, MA

5%

To develop a post-procedure checklist for Interventional Radiology and determine whether it can improve communication and reduce errors.

Aim

The post-procedure checklist project was initiated in response to the discovery of post-procedure errors such as missing samples and concerns

regarding the need for improved communication in the division of Interventional Radiology.

Context/Problem

Given the proven success of checklists in reducing errors in other medical and non-medical venues (1,2), it was determined that the division would develop and test a post-procedure checklist which would draw attention

to and require discussion of the most important steps of patient care after a procedure was completed. The checklist would be used collectively by

the entire IR team including physicians, nurses, and radiology technologists.

This was a two-phase project.

Phase I- Development of the post-procedure checklist: Including a literature review, process mapping to identify important points of

intervention, and writing of the checklist followed by a short trial with multiple revisions.

Phase II- Development and implementation of an evaluation tool:

Including a literature review, formulation of the evaluation tool (3-5), and testing with a pre- and post-intervention study design.

This poster focuses on the measures and results for Phase II.

Intervention

Planning Committee: Radiology Chairman, Radiology Quality Director, Interventional Radiology Section Head, Radiology Quality Fellow and

MD/MPH Student Intern

Project participants: IR attending physicians (9), IR nurses (11), IR technologists (4), radiology fellows and residents (6)

Multi-disciplinary Team

z

Lessons Learned

A standardized evaluation tool adapted from a literature review can be effectively modified to assess the value of a checklist. Use of the checklist improved patient care by increasing compliance with timely post-procedure communications. Checklist compliance rate is affected by how and when it is implemented, for instance:

• If the checklist is completed during suturing (not after) there are higher compliance rates.

• Nurses physically handing the checklist to the physicians contributes to higher compliance rates.

The skills of MD/MPH student interns are invaluable to the healthcare quality improvement process.

Results Project Design

Development of Checklist Process mapping to identify points which are error-prone Drafting (and redrafting) of checklist Evaluation and feedback by checklist expert

Development of Evaluation Tool Literature review of checklist evaluation tools Drafting, trial, and redrafting of evaluation tool

Implementation of Evaluation Tool Pre/Post-intervention study design Observation of IR teams during and after procedures both without and with use of the checklist Refinement of the checklist based on results

Development of Plan for Formal Implementation Review of results Project leaders decide on how to best integrate checklist into regular operations of the IR section Plan to reassess regularly after implementation

Frequency of Communication Failures, including the following types: Ambiguity Critical knowledge gap Lack of communication

Rate of compliance with IR division expectations of care, including the following:

Brief post-procedure note writing in Electronic Health Record (EHR) Radiation dose recorded in EHR

Rate of compliance with checklist items See Figure 1- Post-Procedure Checklist

IR Team Checklist User Opinions Gathered team opinions on checklist usability and value Employed survey using Likert scale

Measures Tracked

Figure 2. Survey Results: Checklist usability and value

Checklist survey question Mean Likert score, mean + SD, (IQR)*

The checklist was easy to use

4.4 + 0.72 (4-5)

This checklist has the potential to prevent errors in post-procedure patient care.

4.4 + 0.66 (4-5)

During this procedure, this checklist helped team members identify and resolve problems or ambiguities regarding post-procedure patient care.

3.5 + 1.2 (2-5)

If use of this checklist were optional, I would opt to use it regularly.

3.5 + 0.90 (3-4)

If I or my family member were going to be a patient in an IR procedure, I would want this checklist to be used.

4.0 + 0.98 (3-5)

On average, respondents thought the checklist was easy to use and has the potential to prevent errors.

*1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree; 23 responses per question; IQR = interquartile range

Figure 3. Changes in compliance score in the post-intervention phase

Nurse reminded resident to use checklist, left it at the nurse’s station, and then left the procedure room

Nurse left checklist next to window, and did not physically hand it to the fellow

Nurse left the procedure room without reminding the resident or fellow to use the checklist

.

Lack of nursing involvement may lead to poor compliance.

The Post-Procedure Checklist

Figure 1.

Figure 4. Association between checklist use and compliance of writing brief post-procedure note within one hour

Acknowledgements: We are grateful to Dr. Raj Vilvendhan and the entire Boston Medical Center IR Team for their enthusiastic participation and to Wayne W. LaMorte, MD, PhD, MPH for his assistance with statistical analysis. References: 1) Gawande, A. The Checklist Manifesto. New York: Metropolitan Books, 2009. 2) Haynes, AB, et al. A surgical safety checklist to reduce morbidity and mortality in a global populations. NEJM 2009 Jan 29;360:491-9. checklists for the operating room: development and pilot

testing. JACS 213(2) 212-217. 4) Lingard, L et al. A theory based instrument to evaluation of team communication in the operating room. Qual and Safe Health Care. 2006 Dec 15;(6) 422-26. 4) Spence, J et al. Student observed surgical safety practices across an urban regional health authority. BMJ Qual Safe 2011 Jul; 20(7):580-6.5)3) Ziewacz, et al. Crisis Lingard L, et al. Evaluation of a preoperative checklist and team briefing among surgeons, nurses and anesthesiologists to reduce failures in communication. Arch Surg. 2008 Jan; 143(1);12-7.

Among inpatient and outpatient cases, physicians were 3.25 times more likely to write a brief post-procedure note within one hour post-intervention compared to pre-intervention (p=0.0069, 95% CI (0.017, 0.68))

Among inpatient cases only, physicians were 2.67 times more likely to write a brief post-procedure note within one hour post-intervention compared to pre-intervention (p=0.0697, 90% CI (0.0119, 0.866))

0

2

4

6

8

10

12

14

16

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Co

mp

lian

ce s

core

Case #

Phase of study pre-intervention post-intervention pre-intervention post-intervention

x3.25 x2.67

Includes inpatient and outpatient procedures Includes inpatient cases only