deep dive: measurement for learning

DEEP DIVE: Measurement for Learning

June 15th, 2010Checklist Planning Group & Guests

Apr 20, 2023 1

Checklist Action Series Hosts

Apr 20, 2023 2

Leanne Couves, Improvement Associates Ltd. & Quality Moderator for Checklist Action Series DEEP DIVE

Chantal Bellerose, Quebec Node SIA Safer Healthcare Now! & French Liaison for Checklist Action Series

Tanis Rollefstad, Western Node SIA Safer Healthcare Now! & Technical Host for Checklist Action Series

Angela Thiessen, Western Node Administrative & Technical Support for Checklist Action Series

Marlies van Dijk, Western Node Leader Safer Healthcare Now! and Moderator for Checklist Action Series DEEP DIVE

Objectives for Today’s Call

• Identify your purpose for doing the checklist

• Choose which measures to start• Explore detailed definitions for key

measures, including data collection plans • Using data for learning

Apr 20, 2023 4

Interacting in WebEx

Apr 20, 2023 4

Be prepared to use: - Pointer Tool - Raise Hand - Chat “to all participants”

What ideas have you tried?

• Measuring compliance with Y/N?

• Measured more than compliance?

• Measuring OR culture?

• Started measuring other items?

• Haven’t started measuring.

Use the pointer

More on What Can Be Measured

Apr 20, 2023 6

Marlies vanDijkWestern Node Leader, SHN!

Leanne CouvesImprovement Associates

Examples of Purpose Statements

• Anticipate problems• Prevent adverse events and/or

outcomes• Prevent harm to patients• Improve culture, teamwork and

communicationUse Pointer to indicate which resonate with

your organization

POTENTIAL MEASURESMarlies van Dijk

• Extended length of anesthesia due to delays

Percent Surgical Cases with Checklist Done Percent Cases Abx Given within 60 Min

Staff Satisfaction Number of Good Catches

Data Display for Checklist Family of Measures

0

25

50

75

100

Wee

k 1

Wee

k 2

Wee

k 3

Wee

k 4

Wee

k 5

Wee

k 6

Wee

k 7

Wee

k 8

Wee

k 9

% Cases with Checklist Done at all 3 Phases

0%

25%

50%

75%

100%

Jan

-10 Fe

b-

10 Mar

-10 Apr

-10

May

-10 Ju

n-

10 Jul-

10A

ug-

10 Sep

-10 Oct

-10

Nov

-10 Dec

-10

% Cases with Abx Given Within 60 Min

0%

25%

50%

75%

100%

Jan

Feb

Mar

Apr

May Ju

n

Jul

Aug

Sep

t

% Staff Rating Ease of Speaking Up in a Case

0

5

10

15

20

25

Jan

Feb

Mar

Ap

r

May Jun

Jul

Aug

Sep

t

Axi

s T

itle

Number of Good Catches

Average Waiting Times: All Primary Care

Clinics in VHA System

0

20

40

60

80

100

Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June Jul Aug Sep Oct Nov Dec Jan

2 0

0 0

2 0

0 1

2

0

Overall Outcome Measure

Measures using Small Multiples:

Overall System and 4 sites

0

10

20

30

40

50

60

70

80

90

100

Apr Jun Aug Oct Dec Feb Apr Jun Aug Oct Dec

OUR SITE

0

10

20

30

40

50

60

70

80

90

100

Apr Jun Aug Oct Dec Feb Apr Jun Aug Oct Dec

SITE2

0

10

20

30

40

50

60

70

80

90

100

Apr Jun Aug Oct Dec Feb Apr Jun Aug Oct Dec

SITE3

0

10

20

30

40

50

60

70

80

90

100

Apr Jun Aug Oct Dec Feb Apr Jun Aug Oct Dec

SITE4

These graphs are called small multiples. They are designed for a quick visual comparisons of the data from each site The graphs are all presented on the same scale (both x and y axis)

Meaningful Measures followed by a Quebec Hospital

Chantal Bellerose

SIA Quebec Node

More on HOW to MeasureSmall Group Discussions

Breakout RoomsLed by Facilitators

Breakout Exercise Talk Through Measurement Plan

Sample Measurement Sheet*

*Adapted from Susan Macknak

http://www.uth.tmc.edu/schools/med/imed/patient_safety/questionnaires/

BREAKOUT SESSION SUMMARY & DEBRIEF

Use text tool on next slide to record one idea from breakouts

How Much Data is Enough?

Marlies van DijkWestern Node Leader, SHN!

Data Collection Methods

• Observation (direct or indirect)• Informal conversations• Surveys and questionnaires (written, phone, one-

on-one interviews)• Focus groups / group interviews• Direct experience “trading places”• Quantitative measurements (manual or

electronic)

Sampling

• Random (from random number generator)

• Systematic random e.g. every X (random number from generator) of Y

• Judgment (requires process knowledge)

– By time of day – By location– By patient population e.g. first 5 cases of the day, Mondays and Thursday’s

Based on “The Data Guide” by Sandra Murray and Lloyd Provost

# 1 Goal for implementingthe Checklist

#2 Measures to support your goal

# 3 Present to the OR teams and other

stakeholders

Checklist Sampling Ideas

Adverse Events Prevented

• Review incidents in the Glitch book weekly and categorize them

CommunicationEquipmentMedications/BloodWrong siteLab work/chart issues

• Staff Room Display

Length of anesthesia reduced with Checklist• Hips and Knee procedures only• Assign observational data

collection• Every Thursday when Dr. X is

doing Hips • Circulating Nurse will notate no.

of minutes delayed on white board as they arise

• Assign responsibility for someone to capture data after each case

• Review monthly

Checklist Sampling IdeasTeam work and communication

• Ask 5 questions on team work and communication (5 min to fill out)

• Capture discipline (nursing, surgery, anesthesia)

• Capture them during breaks and lunch

• Aim to get 10 people per discipline• 10 nurses, 10 surgeons and 10

anesthesiologist, 10 others (RT)• Collate and feedback to staff

Compliance with Checklist• Choose method (observational or

audit form)Audit:• Yes/No tool to each of the 3 stages of

the checklist• Choose surgical procedure (s) where

checklist is being used• Sample 2 charts a day (random or

specific procedures)• Sample 5 first charts once a week

before they go down to health records

• Collate and feedback to staff

Open Mike on Measurement

Poll Questions

Relevance Check

Next Steps

ACTION Items

Between now and June 23rd• Apply what you’ve learned today – start

your data collection plan• Download session materials and see

examples of data collection tools – Sign up on the SSSL Community of Practice

Advice to a “Newbie”

Retrieved from: http://www.google.ca/imgres?imgurl=http://www.mlahanas.de/Physics/Bios/images/AlbertEinstein.jpg&imgrefurl=http://www.mlahanas.de/Physics/Bios/AlbertEinstein.html&h=625&w=640&sz=27&tbnid=e5bsDYQz3HTO3M:&tbnh=134&tbnw=137&prev=/images%3Fq%3DEinstein&usg=__8X0EQ9DZKTZqOWv1jOsfee4tKS8=&sa=X&ei=lRMQTNHOHIWANrOg7NsM&ved=0CC0Q9QEwBA

It’s not as simple as it looks!

Thank You

• You• Guest speakers• Faculty• Facilitators• Message from

Chantal

Apr 20, 2023 33

Questions?

Tanis Rollefstad, Safety & Improvement AdvisorPhone: 306.693.0780Email:tanis.rollefstad@hqca.ca

Leanne Couves, Improvement Associates Ltd.Phone: 780.446.9940 cellEmail: lcouves@telus.net

Apr 20, 2023 34

French Speaking Teams

Chantal Bellerose Dt.P.,M.Sc (cand).Conseillère en matière de sécurité et

d'amélioration | Safety and Improvement Advisor SHN

Campagne québécoise: Ensemble, améliorons la prestation sécuritaire des soins de santé! |

Phone: 514 340 8222 #6705 et #4901‐ ‐Email: cbellerose@jgh.mcgill.ca

Apr 20, 2023 35

KEY THINGS TO CONSIDER WHEN IMPLEMENTING THE CHECKLIST:

HIGHLIGHTS

“See you” on June 23rd 0900-1030 PDT; 1000-1130 MDT; 1100-1230 CDT; 1200-

1330 EDT; 1300-1430 ADT; 1330-1500 NDT

Apr 20, 2023 35