assigning a team-based pager for on-call physicians ... · paging may offer improved communication....
TRANSCRIPT
77February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
Agreat deal of a physicianrsquos time is spent communicating Forexample in one study 24 of hospitalistsrsquo time is spent in
communication activities1 and in another study ICU staff spentthe majority of their time communicating2 Yet there are fewstudies on effective ways to improve communication within hos-pitals3 The growing complexity of hospitalized patients hasmade communication among members of multidisciplinaryteams caring for patients increasingly important Errors in com-munication are a major contributor to defects in quality andharm to patients4ndash10 Beyond patient care issues it is estimatedthat ineffective communication causes $12 billion of waste an-nually in hospitals equating to an annual loss of $4 million fora typical 500-bed hospital11
Pagers are commonly used to facilitate communication inhealth care settings Residents are paged an average of 57 plusmn3times on a day when they are on call9 and even when they arenot 12 plusmn3 times A study at a tertiary care academic teachinghospital revealed that 14 of pages were sent to an incorrectphysician who was unavailable at the time of the page Of thosepages 47 were deemed to be either urgent requiring a timelyresponse or emergencies requiring immediate attention12 Onestudy demonstrated perceptions of significantly improved pa-tient care by nursing staff after switching from a numeric to al-phanumeric paging system13 However many alphanumericpager messages have insufficient information10 Two-way textpaging may offer improved communication14 Another study atan academic medical center demonstrated that implementationof a standardized team-based paging system reduced incorrectpages by 1115
Delays in paging response can be attributed to multiplecauses A prior multi-institution study revealed that the majorityof pages occur when interns are engaged in direct patient careMost of the pages were nonurgent with only a fraction of pagesrequiring a prompt response in the judgment of the recipient16
As a result many nursing pages may be triaged by house staffand responses delayed by competing priorities Residency work-
hour regulations may also contribute to nursing communicationerrors Hospitalized patients at academic medical centers are fre-quently covered by a variety of shift-based schedules which in-creases handoffs17 Consequently an individual patient may becovered by multiple physicians throughout the course of a dayThis potentially increases the risk of paging an incorrect physi-cian who is no longer available for patient care and it signifi-cantly complicates nursing communication as a delayed pagingresponse can initially be indistinguishable from a page to thewrong physician
This improvement project was initiated at our institution be-cause an average of five paging incident reports per monthdemonstrated delayed patient care that was attributed to diffi-culty contacting the correct physician In some instances com-munication difficulty was followed by adverse events andescalation of patient care These data were supported by a surveyof 78 resident physicians who reported a large volume of pagesfrom nurses each day a significant proportion of which were inerror Our initial problem assessment indicated that a nursersquos fa-miliarity with physician teams was the primary driver in know-ing whom to contact The findings showed that if anldquooff-servicerdquo or ldquounfamiliarrdquo patient was housed on the nursingunit the confidence in knowing whom to contact with questionsdecreased dramatically Prior to this project 83 of incident re-ports concerning an inability to contact the correct physician oc-curred on units where nurses were less familiar with thephysician or service This unfortunately happens when the hos-pital has high occupancy and patients cannot always be admit-ted at least initially to the unit usually designated for a specialty
In our local care environment call schedules specific to ser -vices teams types of activity day of the week and time of daywere the primary tool for nurses knowing whom to page Physi-cians placed high value on these call schedules as they definedtheir duty hours However the schedules occasionally containederrors which in turn led to paging errors There was no standardapproach used to maintain these schedules
Case Study in Brief
Assigning a Team-Based Pager for On-Call Physicians Reduces Paging Errors in a Large Academic Hospital
Lisa Shieh MD PhD Jeffrey Chi MD Carol Kulik RN MSN Arash Momeni MD Andrew Shelton MD Cynthia DePorte RN MSN Joseph Hopkins MD MMM
Copyright 2014 copy The Joint Commission
78 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
To address this problem we implemented a model of team-based pagers similar to that previously used by Wong et al atanother academic medical center15 The planned primary out-come of this improvement project was to improve the certaintyof nurses knowing which physician to page
MethodsDEVELOPING THE GHOST-PAGER MODEL
To address the deficiencies of paging system elements in the localcare environment factors likely to influence change were con-sidered Initial thought was given to improving and standardiz-ing call schedules However significant barriers made thisunlikely to succeed Nearly every service had differing needs re-lated to the nature of its specialty many different tools were inuse for constructing schedules and an array of types of rolesvarying from chief residents to clerical personnel were assignedthe responsibility for the schedules These patterns were deeplyentrenched and judged to be very difficult to standardize An al-ternative of a single pager number for a team or a specific consultservice had been used successfully in a few situations A multi-disciplinary team was formed to confirm the feasibility of thismodel and implement it for the entire hospital The team in-cluded physicians nurses residents and a physician-informati-cist
The main building blocks of the new model were assignmentof a treatment team to each patient and a dedicated pager num-ber (that is one that never changes) which we referred to as theldquoghost-pagerrdquo (as suggested by the lead author [LS]) numberto each team The treatment team was assigned to each patientby the resident (1) admitting the patient (2) writing postoper-ative orders or (3) accepting a patient from another service Astandard team list was made available in the electronic medicalrecord (EMR) to facilitate the assignments Reports were printedon units twice a day listing any patient without a treatment teamassignment Nurses contacted teams asking that the patient beassigned or the unit clerk completed the missing assignmentswhen possible The percentage of patients with a treatment teamassignment was tracked weekly A separate report listed the at-tending physician of patients lacking a treatment team assign-ment and reminders were sent to these physiciansApproximately 15 of Stanford Medical Center patients arecared for by community physicians who do not have house staffcoverage These physicians were not targeted for this interven-tion as difficulties paging them or their correct cross-coveringphysicians were uncommon
Each treatment team was linked to a ghost-pager number(Figure 1 page 79) These pagers do not exist physically but
function electronically like a pager Each time the responsibilityfor covering a teamrsquos patients changes the physician responsibleforwarded this ghost pager to his or her own personal pager orcell phone This link was made by any of three methods (1) call-ing the paging operator (2) entering a series of numbers on anytouch-tone phone or (3) using a Web-based system Coverageof ghost pagers was assessed periodically to determine if physi-cians were using them The new model was implemented duringa six-week period in JunendashJuly 2010
Several enhancements of the EMR were made to facilitatenurses finding the correct ghost pager for the treatment teamThis number was embedded in the name of the treatment teamthat appears on the nursersquos list of patients and in the end-of-shift sign-out nursing notes Whiteboards on units were modi-fied to list the ghost-pager number of the team rather than thename of the individual physician covering a patient A directoryof treatment teams and ghost-pager numbers was also distributedelectronically throughout the hospital
Education of nurses and physicians regarding the new systemwas carried out through presentations at meetings posterspocket cards and reminders on workstation screens Physicianchampions who were identified for each specialty providedstakeholder input during the design of the new system andhelped disseminate information about it
ASSESSING THE EFFECTIVENESS OF THE NEW MODEL
The effectiveness of the new model was assessed using a be-fore-and-after prospective assessment of nursesrsquo knowledge ofwhom to page as measured by nursesrsquo level of certainty regardingwhom to page Before we initiated this model in June 2010nurses from all shifts on selected units in the hospital were sur-veyed by e-mail to determine how much difficulty they experi-enced in knowing which physician to page when they neededmedical input in the care of a patient Each respondent was askedto identify the medical or surgical service most commonly pagedand then to rate the level of certainty of which specific physicianto page on the basis of a 4-point scale ldquonearly alwaysrdquo ldquousuallyrdquoldquosometimesrdquo and ldquorarelyrdquo This process was then repeated forthe second- and third-most common service that the nurse typ-ically needed to page The survey was conducted during a three-week period in February 2009 and again in February 2011 One reminder message was sent halfway through each survey period to increase the response rate Pearsonrsquos chi-square test wasused to assess whether certainty over whom to page increasedfrom the 2009 to the 2011 time period The threshold level oftype I error (alpha) for determining statistical significance was p lt 05
Copyright 2014 copy The Joint Commission
79February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
ResultsADOPTION OF THE GHOST-PAGER MODEL
Adoption of the ghost-pager model was rapid and uniform andcompliance with treatment team assignment was very highAmong patients of faculty physicians 64 (315491) had anassigned treatment team within the first week of the rollout Thisvalue grew to greater than 80 within a month and has re-mained in the 92ndash93 range since then
NURSE SURVEY RESPONSES
There were 561 responses to the nurse survey in 2009 and916 responses in 2011 Before implementation of this model73 of the nurses responding to the survey indicated theyldquonearly alwaysrdquo or ldquousuallyrdquo knew whom to page This level ofcertainty varied considerably among services ranging from 33for neurology to 91 for neurosurgery Overall certainty for allservices rose to 87 after the model was implemented (range75 ndash100) Eight of the services had gt 90 certainty includ-ing three with a certainty rate of 100 (Figure 2 page 80)
The improvement for the whole hospital was highly signifi-cant (p lt 0001) The improvements for medicine-cardiology (p lt 0001) medicine-gastrointestinal (p lt 03) medicine (p lt0001) oncology (p lt 01) and trauma (p lt 04) were all statis-tically significant
QUALITATIVE RESULTS
Qualitative results were also positive Incident reports of in-ability to contact a physician using the ghost-pager number havebeen uncommon Nurses in general have expressed wide supportfor the model as have physicians Some services have begunusing ghost pagers to help sort calls within teams such as ldquoad-mitting residentrdquo or for additional consulting services Pagingoperators liked the new model as the increased work of morecalls to forward a pager were offset by reduction in the burdenof trying to find the correct physician often a frustrating processpunctuated by repeated calls from nurses upset that the physicianhad not called back Residents now can directly control handoffsor last-minute changes in coverage The model has remained in-
Stanford University Hospital Paging System
Figure 1 The paging model requires residents to assign the treatment team to each patient and to forward the ghost pager (GP) to their personal pager wheneverthey are responsible for the patient or on call Nurses page physicians using the ghost-pager number which that never changes for a team Ghost-pager numbersare embedded in the electronic medical record (EMR) nursing work flow (Available in color in online article)
Copyright 2014 copy The Joint Commission
80 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
tact without changes since it was implemented Respiratory ther-apists radiology technologists and some consulting services alsohave begun using some aspects of the model For example oto-laryngology uses two ghost pagers designated as ldquoConsults-Adultsrdquo and ldquoConsults-Pediatricrdquo for receiving and sortingrequests for consultation and the ICU consult team uses a ded-icated ghost pager as a point of contact
DiscussionEffective timely communication is essential to provide outstand-ing patient care Over the years there have been repeated con-cerns about improving nurse-to-physician communication18ndash25
In academic medical centers with complex teams of physiciansthe most fundamental step in communicating is finding the cor-rect physician responsible for the patient Our results suggest
that the use of a dedicated team-based pager number that is al-ways assigned to a team greatly improved the likelihood of anursersquos knowing whom to page This was particularly true forservices with more complicated team structures or whose pa-tients tend to be more dispersed The results confirm the find-ings reported by Wong et al15 who showed that a team pagerreduced the number of after-hours paging errors on a generalmedicine service in an academic hospital Our study extends thisfinding to multiple other medical and surgical specialties and toall three nursing shifts
The application of new communication technologies is oftenthe focus of communication improvement initiatives1426ndash32
However simply imposing these technologies on existing workflows may not improve communication273132 A technology maybe seen as an improvement for physicians but not nurses31 or
Percentage of Nurses Responding ldquoNearly Alwaysrdquo or ldquoUsuallyrdquo in Response to the Question ldquoI know which resident or pager number to page when I need physician input in
care of my patientsrdquo 2009 and 2011
Figure 2 Nursesrsquo responses show their level of certainty about which physician to page in 2009 and again in 2011 after the implementation of the new pagingmodel CT cardiothoracic GI gastrointestinal Probability (p) levels for chi-square tests are shown an asterisk indicates statistical significance (Available incolor in online article)
Copyright 2014 copy The Joint Commission
81February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
vice versa27 Furthermore one technology may be useful forsome types of communication but less so for others In onestudy the use of e-mail via smartphones was favored for convey-ing nonurgent information but other modes were preferred forurgent matters In addition team members disagreed about appropriate use of communication modes32 There is the possi-bility of making matters worse as Quan et al found when aWeb-based communication system increased interruptions ofresidents by 23333 The authors cautioned that ldquothe interplayof technology with existing clinical workflow culture and socialinteractions may create other unintended consequencesrdquo33(p 137)
The model evaluated in this study did not employ new tech-nologies depending only on those that were already being usedThe focus was on reducing variation and errors in the work flowof paging The results suggest significant potential for improvingpaging even without additional technological tools Further-more the work-flow improvements would have been neededeven if new communication technologies had been added
The new model implemented in this study did take advantageof functionality in the EMR system not previously employedThe hospitalrsquos informatics team was essential to move the processforward Embedding the ghost-pager number into the treatmentteam name eliminated any additional order entry steps for physi-cians assigning themselves to a patient The treatment team andghost-pager number information were also automatically pulledinto the nursing notes for additional quick reference A line wasadded in the RN sign-out note to enable nurses to communicatenonurgent issues to physicians for follow-up to reduce some ofthe unnecessary night-shift paging
The new model represented a significant change in traditionalpractice Stories drawn from paging incidents information gath-ered to characterize the problem and the data gathered in theinitial survey of nurses established the need for improving pagingeffectiveness From the onset of this project physicians re-sponded with genuine concern when reports of delays in patientcare were brought to their attention The congruency of datafrom all these sources was sufficiently compelling to establish amultidisciplinary team to critically evaluate paging effectivenessstrategies and implement the new model The potential to im-prove trainee experience with pages may have helped drive theircompliance with the new model as well
LIMITATIONS
This study has several limitations The results are based onnursesrsquo perceptions rather than objective measures of reducedpaging errors or of improved patient outcomes This was not arandomized control trial of the model The evaluators were not
blinded as to the intervention It is possible that the changes innursesrsquo perceptions were biased by the attention focused on themodel by the campaign conducted to increase adoption Never-theless the results suggest that this model can be helpful in re-ducing errors in communication due to paging the incorrectphysician Although improvements in communication in generalare expected to enhance quality and patient safety the impact ofthis model on improved patient outcomes is important to assessin further studies In addition more study is needed on whetherthe changes implemented have improved resident experiencewith the number and self-perceived appropriateness of pagingparticularly at night
SummaryAs complexity of care of hospitalized patients has increased theneed for communication and collaboration among members ofthe team caring for the patient has become increasingly impor-tant This often takes the form of a nursersquos need to contact a pa-tientrsquos physician to discuss some aspect of care and modifytreatment plans Errors in communication delay care and canpose risk to patients This report describes the successful imple-mentation of a standardized team-based paging system at an ac-ademic center Results showed a substantial improvement innursesrsquo perceptions of knowing how to contact the correct physi-cian when discussion of the patientrsquos care is needed This im-provement was found across multiple medical and surgicalspecialties and was particularly effective for services with thegreatest communication problems The authors thank Isabella Chu MPH for her assistance in reviewing and preparing
the manuscript and Pooja Loftus MS for providing statistical analysis
J
Lisa Shieh MD PhD is Clinical Associate Professor of Medicine Di-
vision of General Medical Disciplines Stanford School of Medicine
Stanford California and Unit Based Medical Director and Quality
Medical Director for Medicine Stanford Hospital and Clinics Jeffrey
Chi MD is Clinical Assistant Professor of Medicine Division of Gen-
eral Medical Disciplines Stanford School of Medicine and Unit Based
Medical Director Stanford Hospital and Clinics Carole Kulik RN
MSN is Director of Patient Care Practice and Education Nursing Ad-
ministration Stanford Hospital and Clinics Arash Momeni MD is
Chief Resident in Plastic Surgery Stanford Hospital and Clinics An-
drew Shelton MD is Clinical Associate Professor in Surgery Divi-
sion of Colorectal Surgery Stanford School of Medicine and Medical
Informatics Medical Director Stanford Hospital and Clinics Cynthia
DePorte RN MSN is Director of Cancer Services Stanford Hospital
and Clinics Joseph Hopkins MD MMM is Clinical Professor of
Medicine Division of General Medical Disciplines Stanford School of
Medicine and Senior Medical Director for Quality Stanford Hospital
and Clinics Please address correspondence to Joseph Hopkins
joehstanfordedu
Copyright 2014 copy The Joint Commission
82 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
References1 OrsquoLeary KJ Liebovitz DM Baker DW How hospitalists spend their timeInsights on efficiency and safety J Hosp Med 20061(2)88ndash932 Alvarez G Coiera E Interruptive communication patterns in the intensivecare unit ward round Int J Med Inform 200574(10)791ndash7963 Wu RC et al Effects of clinical communication interventions in hospitalsA systematic review of information and communication technology adoptionsfor improved communication between clinicians Int J Med Inform201281(11)723ndash7324 Dwyer K Communication breakdowns combine to cause system failure IntJ Qual Health Care 200113(3)2655 Dwyer K Flawed communication systems result in patient harm Int J QualHealth Care 200214(1)77 Erratum in Int J Qual Health Care 200315(1)996 Rollins G Medical errors poor communication undermine quality of carepatient satisfaction Rep Med Guidel Outcomes Res 2002 May 1713(10)5ndash77 Southwick LM Communication misadventures and medical errors Jt CommJ Qual Improv 200228(8)461ndash462 Comment on Weeks WB et al The or-ganizational costs of preventable medical errors Jt Comm J Qual Improv200127(10)533ndash539 Author reply 462ndash4638 Alvarez G Coiera E Interdisciplinary communication An uncharted sourceof medical error J Crit Care 200621(3)236ndash242 discussion 2429 Patel SP et al Resident workload pager communications and quality ofcare World J Surg 201034(11)2524ndash252910 Espino S Cox D Kaplan B Alphanumeric paging A potential source ofproblems in patient care and communication J Surg Educ 201168(6)447ndash45111 Agarwal R Sands DZ Schneider JD Quantifying the economic impact ofcommunication inefficiencies in US hospitals J Healthc Manag201055(4)265ndash281 discussion 281ndash28212 Wong BM et al Frequency and clinical importance of pages sent to thewrong physician Arch Intern Med 2009 Jun 8169(11)1072ndash107313 Nguyen TC et al Alphanumeric paging in an academic hospital settingAm J Surg 2006191(4)561ndash56514 Ighani F et al A comparison of two-way text versus conventional pagingsystems in an academic ophthalmology department J Med Syst201034(4)677ndash684
15 Wong BM et al Getting the message A quality improvement initiative toreduce pages sent to the wrong physician BMJ Qual Saf 201221(10)855ndash86216 Katz MH Schroeder SA The sounds of the hospital Paging patterns inthree teaching hospitals N Engl J Med 1988 Dec 15319(24)1585ndash158917 Myers JS Bellini LM Resident handoffs Appreciating them as a criticalcompetency J Gen Intern Med 201227(3)270ndash272 Comment on Helms ASet al Use of an appreciative inquiry approach to improve resident sign-out inan era of multiple shift changes J Gen Intern Med 201227(3)287ndash29018 Burns K Nurse-physician rounds A collaborative approach to improvingcommunication efficiencies and perception of care Medsurg Nurs201120(4)194ndash19919 Crawford CL Omery A Seago JA The challenges of nurse-physician com-munication A review of the evidence J Nurs Adm 201242(12)548ndash55020 Manojlovich M et al Developing and testing a tool to measure nursephysi-cian communication in the intensive care unit J Patient Saf 20117(2)80ndash8421 Nair DM et al Frequency of nurse-physician collaborative behaviors in anacute care hospital J Interprof Care 201226(2)115ndash12022 OrsquoLeary KJ et al Patterns of nurse-physician communication and agree-ment on the plan of care Qual Saf Health Care 201019(3)195ndash19923 Robinson FP et al Perceptions of effective and ineffective nurse-physiciancommunication in hospitals Nurs Forum 201045(3)206ndash21624 Rosenthal L Enhancing communication between night shift RNs and hos-pitalists An opportunity for performance improvement J Nurs Adm201343(2)59ndash6125 Tschannen D et al Implications of nurse-physician relations Report of asuccessful intervention Nurs Econ 201129(3)127ndash13526 Hanada E et al Advantages of low output mobile communication systemsin hospitals J Med Syst 200024(2)53ndash5927 Blair K Orr M Insights from an iBleep trial A report on lessons learnedHealth Care and Informatics Review Online 201115(1)3ndash1228 Horn G Thorel P Next-generation communication at Advocate HealthCare Alcatel Telecommunications Review 2006(1)10ndash1429 Etchells E et al Real-time clinical alerting Effect of an automated pagingsystem on response time to critical laboratory valuesmdashA randomised controlledtrial Qual Saf Health Care 201019(2)99ndash10230 Smith CN et al Understanding interprofessional communication A con-tent analysis of email communications between doctors and nurses Appl ClinInform 2012 Feb 13(1)38ndash5131 Wu RC et al The use of smartphones for clinical communication on in-ternal medicine wards J Hosp Med 20105(9)553ndash55932 Lo V et al The use of smartphones in general and internal medicine unitsA boon or a bane to the promotion of interprofessional collaboration J InterprofCare 201226(4)276ndash28233 Quan SD et al Itrsquos not about pager replacement An in-depth look at theinterprofessional nature of communication in healthcare J Hosp Med20138(3)137ndash143
Online-Only Content
See the online version of this article for
Figure 1 Stanford University Hospital Paging System (color version)
Figure 2 Percentage of Nurses Responding ldquoNearly Alwaysrdquo or
ldquoUsuallyrdquo in Response to the Question ldquoI know which
resident or pager number to page when I need physician
input in care of my patientsrdquo 2009 and 2011
(color version)
8
Copyright 2014 copy The Joint Commission
AP1February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
Stanford University Hospital Paging System
Online-Only Content8
Figure 1 The paging model requires residents to assign the treatment team to each patient and to forward the ghost pager (GP) to their personal pagerwhenever they are responsible for the patient or on call Nurses page physicians using the ghost-pager number which never changes for a team Ghost-pagernumbers are embedded in the electronic medical record (EMR) nursing work flow
Copyright 2014 copy The Joint Commission
AP2 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
71
75
40
80
67
65
33
91
73 76 80
82
72
74
75
96
100
93 100
85
80
97
100
79 85
94
77
91
0
10
20
30
40
50
60
70
80
90
100
CT
Sur
gery
(n =
11)
Med
icin
e - C
ardi
olog
y(n
= 1
12)
Med
icin
e - G
I(n
= 1
5)
Med
icin
e - H
emat
olog
y(n
= 5
5)
Med
icin
e - I
nfec
t Dis
ease
(n =
5)
Med
icin
e(n
= 2
12)
Neu
rolo
gy(n
= 1
6)
Neu
rosu
rger
y(n
= 6
5)
Onc
olog
y(n
= 8
7)
Orth
oped
ics
(n =
31)
Pain
( n =
28)
Sur
gica
l Spe
cial
ties
(n =
62)
Tran
spla
nt -
Sol
id O
rgan
(n =
31)
Trau
ma
(n =
81)
Kno
win
g w
ho to
pag
e
2009 2011
p = 90 p = 23 p = 36 p = 06 p = 32 p = 89 p = 75 p = 14 p = 77p lt 00 p lt 00 p lt 01 p lt 04p lt 03
Percentage of Nurses Responding ldquoNearly Alwaysrdquo or ldquoUsuallyrdquo in Response to the Question ldquoI know which resident or pager number to page when I need physician input in care of my patientsrdquo 2009 and 2011
Online-Only Content8
Figure 2 Nursesrsquo responses show their level of certainty about which physician to page in 2009 and again in 2011 after the implementation of the newpaging model CT cardiothoracic GI gastrointestinal Probability (p) levels for chi-square tests are shown an asterisk indicates statistical significance
Copyright 2014 copy The Joint Commission
78 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
To address this problem we implemented a model of team-based pagers similar to that previously used by Wong et al atanother academic medical center15 The planned primary out-come of this improvement project was to improve the certaintyof nurses knowing which physician to page
MethodsDEVELOPING THE GHOST-PAGER MODEL
To address the deficiencies of paging system elements in the localcare environment factors likely to influence change were con-sidered Initial thought was given to improving and standardiz-ing call schedules However significant barriers made thisunlikely to succeed Nearly every service had differing needs re-lated to the nature of its specialty many different tools were inuse for constructing schedules and an array of types of rolesvarying from chief residents to clerical personnel were assignedthe responsibility for the schedules These patterns were deeplyentrenched and judged to be very difficult to standardize An al-ternative of a single pager number for a team or a specific consultservice had been used successfully in a few situations A multi-disciplinary team was formed to confirm the feasibility of thismodel and implement it for the entire hospital The team in-cluded physicians nurses residents and a physician-informati-cist
The main building blocks of the new model were assignmentof a treatment team to each patient and a dedicated pager num-ber (that is one that never changes) which we referred to as theldquoghost-pagerrdquo (as suggested by the lead author [LS]) numberto each team The treatment team was assigned to each patientby the resident (1) admitting the patient (2) writing postoper-ative orders or (3) accepting a patient from another service Astandard team list was made available in the electronic medicalrecord (EMR) to facilitate the assignments Reports were printedon units twice a day listing any patient without a treatment teamassignment Nurses contacted teams asking that the patient beassigned or the unit clerk completed the missing assignmentswhen possible The percentage of patients with a treatment teamassignment was tracked weekly A separate report listed the at-tending physician of patients lacking a treatment team assign-ment and reminders were sent to these physiciansApproximately 15 of Stanford Medical Center patients arecared for by community physicians who do not have house staffcoverage These physicians were not targeted for this interven-tion as difficulties paging them or their correct cross-coveringphysicians were uncommon
Each treatment team was linked to a ghost-pager number(Figure 1 page 79) These pagers do not exist physically but
function electronically like a pager Each time the responsibilityfor covering a teamrsquos patients changes the physician responsibleforwarded this ghost pager to his or her own personal pager orcell phone This link was made by any of three methods (1) call-ing the paging operator (2) entering a series of numbers on anytouch-tone phone or (3) using a Web-based system Coverageof ghost pagers was assessed periodically to determine if physi-cians were using them The new model was implemented duringa six-week period in JunendashJuly 2010
Several enhancements of the EMR were made to facilitatenurses finding the correct ghost pager for the treatment teamThis number was embedded in the name of the treatment teamthat appears on the nursersquos list of patients and in the end-of-shift sign-out nursing notes Whiteboards on units were modi-fied to list the ghost-pager number of the team rather than thename of the individual physician covering a patient A directoryof treatment teams and ghost-pager numbers was also distributedelectronically throughout the hospital
Education of nurses and physicians regarding the new systemwas carried out through presentations at meetings posterspocket cards and reminders on workstation screens Physicianchampions who were identified for each specialty providedstakeholder input during the design of the new system andhelped disseminate information about it
ASSESSING THE EFFECTIVENESS OF THE NEW MODEL
The effectiveness of the new model was assessed using a be-fore-and-after prospective assessment of nursesrsquo knowledge ofwhom to page as measured by nursesrsquo level of certainty regardingwhom to page Before we initiated this model in June 2010nurses from all shifts on selected units in the hospital were sur-veyed by e-mail to determine how much difficulty they experi-enced in knowing which physician to page when they neededmedical input in the care of a patient Each respondent was askedto identify the medical or surgical service most commonly pagedand then to rate the level of certainty of which specific physicianto page on the basis of a 4-point scale ldquonearly alwaysrdquo ldquousuallyrdquoldquosometimesrdquo and ldquorarelyrdquo This process was then repeated forthe second- and third-most common service that the nurse typ-ically needed to page The survey was conducted during a three-week period in February 2009 and again in February 2011 One reminder message was sent halfway through each survey period to increase the response rate Pearsonrsquos chi-square test wasused to assess whether certainty over whom to page increasedfrom the 2009 to the 2011 time period The threshold level oftype I error (alpha) for determining statistical significance was p lt 05
Copyright 2014 copy The Joint Commission
79February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
ResultsADOPTION OF THE GHOST-PAGER MODEL
Adoption of the ghost-pager model was rapid and uniform andcompliance with treatment team assignment was very highAmong patients of faculty physicians 64 (315491) had anassigned treatment team within the first week of the rollout Thisvalue grew to greater than 80 within a month and has re-mained in the 92ndash93 range since then
NURSE SURVEY RESPONSES
There were 561 responses to the nurse survey in 2009 and916 responses in 2011 Before implementation of this model73 of the nurses responding to the survey indicated theyldquonearly alwaysrdquo or ldquousuallyrdquo knew whom to page This level ofcertainty varied considerably among services ranging from 33for neurology to 91 for neurosurgery Overall certainty for allservices rose to 87 after the model was implemented (range75 ndash100) Eight of the services had gt 90 certainty includ-ing three with a certainty rate of 100 (Figure 2 page 80)
The improvement for the whole hospital was highly signifi-cant (p lt 0001) The improvements for medicine-cardiology (p lt 0001) medicine-gastrointestinal (p lt 03) medicine (p lt0001) oncology (p lt 01) and trauma (p lt 04) were all statis-tically significant
QUALITATIVE RESULTS
Qualitative results were also positive Incident reports of in-ability to contact a physician using the ghost-pager number havebeen uncommon Nurses in general have expressed wide supportfor the model as have physicians Some services have begunusing ghost pagers to help sort calls within teams such as ldquoad-mitting residentrdquo or for additional consulting services Pagingoperators liked the new model as the increased work of morecalls to forward a pager were offset by reduction in the burdenof trying to find the correct physician often a frustrating processpunctuated by repeated calls from nurses upset that the physicianhad not called back Residents now can directly control handoffsor last-minute changes in coverage The model has remained in-
Stanford University Hospital Paging System
Figure 1 The paging model requires residents to assign the treatment team to each patient and to forward the ghost pager (GP) to their personal pager wheneverthey are responsible for the patient or on call Nurses page physicians using the ghost-pager number which that never changes for a team Ghost-pager numbersare embedded in the electronic medical record (EMR) nursing work flow (Available in color in online article)
Copyright 2014 copy The Joint Commission
80 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
tact without changes since it was implemented Respiratory ther-apists radiology technologists and some consulting services alsohave begun using some aspects of the model For example oto-laryngology uses two ghost pagers designated as ldquoConsults-Adultsrdquo and ldquoConsults-Pediatricrdquo for receiving and sortingrequests for consultation and the ICU consult team uses a ded-icated ghost pager as a point of contact
DiscussionEffective timely communication is essential to provide outstand-ing patient care Over the years there have been repeated con-cerns about improving nurse-to-physician communication18ndash25
In academic medical centers with complex teams of physiciansthe most fundamental step in communicating is finding the cor-rect physician responsible for the patient Our results suggest
that the use of a dedicated team-based pager number that is al-ways assigned to a team greatly improved the likelihood of anursersquos knowing whom to page This was particularly true forservices with more complicated team structures or whose pa-tients tend to be more dispersed The results confirm the find-ings reported by Wong et al15 who showed that a team pagerreduced the number of after-hours paging errors on a generalmedicine service in an academic hospital Our study extends thisfinding to multiple other medical and surgical specialties and toall three nursing shifts
The application of new communication technologies is oftenthe focus of communication improvement initiatives1426ndash32
However simply imposing these technologies on existing workflows may not improve communication273132 A technology maybe seen as an improvement for physicians but not nurses31 or
Percentage of Nurses Responding ldquoNearly Alwaysrdquo or ldquoUsuallyrdquo in Response to the Question ldquoI know which resident or pager number to page when I need physician input in
care of my patientsrdquo 2009 and 2011
Figure 2 Nursesrsquo responses show their level of certainty about which physician to page in 2009 and again in 2011 after the implementation of the new pagingmodel CT cardiothoracic GI gastrointestinal Probability (p) levels for chi-square tests are shown an asterisk indicates statistical significance (Available incolor in online article)
Copyright 2014 copy The Joint Commission
81February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
vice versa27 Furthermore one technology may be useful forsome types of communication but less so for others In onestudy the use of e-mail via smartphones was favored for convey-ing nonurgent information but other modes were preferred forurgent matters In addition team members disagreed about appropriate use of communication modes32 There is the possi-bility of making matters worse as Quan et al found when aWeb-based communication system increased interruptions ofresidents by 23333 The authors cautioned that ldquothe interplayof technology with existing clinical workflow culture and socialinteractions may create other unintended consequencesrdquo33(p 137)
The model evaluated in this study did not employ new tech-nologies depending only on those that were already being usedThe focus was on reducing variation and errors in the work flowof paging The results suggest significant potential for improvingpaging even without additional technological tools Further-more the work-flow improvements would have been neededeven if new communication technologies had been added
The new model implemented in this study did take advantageof functionality in the EMR system not previously employedThe hospitalrsquos informatics team was essential to move the processforward Embedding the ghost-pager number into the treatmentteam name eliminated any additional order entry steps for physi-cians assigning themselves to a patient The treatment team andghost-pager number information were also automatically pulledinto the nursing notes for additional quick reference A line wasadded in the RN sign-out note to enable nurses to communicatenonurgent issues to physicians for follow-up to reduce some ofthe unnecessary night-shift paging
The new model represented a significant change in traditionalpractice Stories drawn from paging incidents information gath-ered to characterize the problem and the data gathered in theinitial survey of nurses established the need for improving pagingeffectiveness From the onset of this project physicians re-sponded with genuine concern when reports of delays in patientcare were brought to their attention The congruency of datafrom all these sources was sufficiently compelling to establish amultidisciplinary team to critically evaluate paging effectivenessstrategies and implement the new model The potential to im-prove trainee experience with pages may have helped drive theircompliance with the new model as well
LIMITATIONS
This study has several limitations The results are based onnursesrsquo perceptions rather than objective measures of reducedpaging errors or of improved patient outcomes This was not arandomized control trial of the model The evaluators were not
blinded as to the intervention It is possible that the changes innursesrsquo perceptions were biased by the attention focused on themodel by the campaign conducted to increase adoption Never-theless the results suggest that this model can be helpful in re-ducing errors in communication due to paging the incorrectphysician Although improvements in communication in generalare expected to enhance quality and patient safety the impact ofthis model on improved patient outcomes is important to assessin further studies In addition more study is needed on whetherthe changes implemented have improved resident experiencewith the number and self-perceived appropriateness of pagingparticularly at night
SummaryAs complexity of care of hospitalized patients has increased theneed for communication and collaboration among members ofthe team caring for the patient has become increasingly impor-tant This often takes the form of a nursersquos need to contact a pa-tientrsquos physician to discuss some aspect of care and modifytreatment plans Errors in communication delay care and canpose risk to patients This report describes the successful imple-mentation of a standardized team-based paging system at an ac-ademic center Results showed a substantial improvement innursesrsquo perceptions of knowing how to contact the correct physi-cian when discussion of the patientrsquos care is needed This im-provement was found across multiple medical and surgicalspecialties and was particularly effective for services with thegreatest communication problems The authors thank Isabella Chu MPH for her assistance in reviewing and preparing
the manuscript and Pooja Loftus MS for providing statistical analysis
J
Lisa Shieh MD PhD is Clinical Associate Professor of Medicine Di-
vision of General Medical Disciplines Stanford School of Medicine
Stanford California and Unit Based Medical Director and Quality
Medical Director for Medicine Stanford Hospital and Clinics Jeffrey
Chi MD is Clinical Assistant Professor of Medicine Division of Gen-
eral Medical Disciplines Stanford School of Medicine and Unit Based
Medical Director Stanford Hospital and Clinics Carole Kulik RN
MSN is Director of Patient Care Practice and Education Nursing Ad-
ministration Stanford Hospital and Clinics Arash Momeni MD is
Chief Resident in Plastic Surgery Stanford Hospital and Clinics An-
drew Shelton MD is Clinical Associate Professor in Surgery Divi-
sion of Colorectal Surgery Stanford School of Medicine and Medical
Informatics Medical Director Stanford Hospital and Clinics Cynthia
DePorte RN MSN is Director of Cancer Services Stanford Hospital
and Clinics Joseph Hopkins MD MMM is Clinical Professor of
Medicine Division of General Medical Disciplines Stanford School of
Medicine and Senior Medical Director for Quality Stanford Hospital
and Clinics Please address correspondence to Joseph Hopkins
joehstanfordedu
Copyright 2014 copy The Joint Commission
82 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
References1 OrsquoLeary KJ Liebovitz DM Baker DW How hospitalists spend their timeInsights on efficiency and safety J Hosp Med 20061(2)88ndash932 Alvarez G Coiera E Interruptive communication patterns in the intensivecare unit ward round Int J Med Inform 200574(10)791ndash7963 Wu RC et al Effects of clinical communication interventions in hospitalsA systematic review of information and communication technology adoptionsfor improved communication between clinicians Int J Med Inform201281(11)723ndash7324 Dwyer K Communication breakdowns combine to cause system failure IntJ Qual Health Care 200113(3)2655 Dwyer K Flawed communication systems result in patient harm Int J QualHealth Care 200214(1)77 Erratum in Int J Qual Health Care 200315(1)996 Rollins G Medical errors poor communication undermine quality of carepatient satisfaction Rep Med Guidel Outcomes Res 2002 May 1713(10)5ndash77 Southwick LM Communication misadventures and medical errors Jt CommJ Qual Improv 200228(8)461ndash462 Comment on Weeks WB et al The or-ganizational costs of preventable medical errors Jt Comm J Qual Improv200127(10)533ndash539 Author reply 462ndash4638 Alvarez G Coiera E Interdisciplinary communication An uncharted sourceof medical error J Crit Care 200621(3)236ndash242 discussion 2429 Patel SP et al Resident workload pager communications and quality ofcare World J Surg 201034(11)2524ndash252910 Espino S Cox D Kaplan B Alphanumeric paging A potential source ofproblems in patient care and communication J Surg Educ 201168(6)447ndash45111 Agarwal R Sands DZ Schneider JD Quantifying the economic impact ofcommunication inefficiencies in US hospitals J Healthc Manag201055(4)265ndash281 discussion 281ndash28212 Wong BM et al Frequency and clinical importance of pages sent to thewrong physician Arch Intern Med 2009 Jun 8169(11)1072ndash107313 Nguyen TC et al Alphanumeric paging in an academic hospital settingAm J Surg 2006191(4)561ndash56514 Ighani F et al A comparison of two-way text versus conventional pagingsystems in an academic ophthalmology department J Med Syst201034(4)677ndash684
15 Wong BM et al Getting the message A quality improvement initiative toreduce pages sent to the wrong physician BMJ Qual Saf 201221(10)855ndash86216 Katz MH Schroeder SA The sounds of the hospital Paging patterns inthree teaching hospitals N Engl J Med 1988 Dec 15319(24)1585ndash158917 Myers JS Bellini LM Resident handoffs Appreciating them as a criticalcompetency J Gen Intern Med 201227(3)270ndash272 Comment on Helms ASet al Use of an appreciative inquiry approach to improve resident sign-out inan era of multiple shift changes J Gen Intern Med 201227(3)287ndash29018 Burns K Nurse-physician rounds A collaborative approach to improvingcommunication efficiencies and perception of care Medsurg Nurs201120(4)194ndash19919 Crawford CL Omery A Seago JA The challenges of nurse-physician com-munication A review of the evidence J Nurs Adm 201242(12)548ndash55020 Manojlovich M et al Developing and testing a tool to measure nursephysi-cian communication in the intensive care unit J Patient Saf 20117(2)80ndash8421 Nair DM et al Frequency of nurse-physician collaborative behaviors in anacute care hospital J Interprof Care 201226(2)115ndash12022 OrsquoLeary KJ et al Patterns of nurse-physician communication and agree-ment on the plan of care Qual Saf Health Care 201019(3)195ndash19923 Robinson FP et al Perceptions of effective and ineffective nurse-physiciancommunication in hospitals Nurs Forum 201045(3)206ndash21624 Rosenthal L Enhancing communication between night shift RNs and hos-pitalists An opportunity for performance improvement J Nurs Adm201343(2)59ndash6125 Tschannen D et al Implications of nurse-physician relations Report of asuccessful intervention Nurs Econ 201129(3)127ndash13526 Hanada E et al Advantages of low output mobile communication systemsin hospitals J Med Syst 200024(2)53ndash5927 Blair K Orr M Insights from an iBleep trial A report on lessons learnedHealth Care and Informatics Review Online 201115(1)3ndash1228 Horn G Thorel P Next-generation communication at Advocate HealthCare Alcatel Telecommunications Review 2006(1)10ndash1429 Etchells E et al Real-time clinical alerting Effect of an automated pagingsystem on response time to critical laboratory valuesmdashA randomised controlledtrial Qual Saf Health Care 201019(2)99ndash10230 Smith CN et al Understanding interprofessional communication A con-tent analysis of email communications between doctors and nurses Appl ClinInform 2012 Feb 13(1)38ndash5131 Wu RC et al The use of smartphones for clinical communication on in-ternal medicine wards J Hosp Med 20105(9)553ndash55932 Lo V et al The use of smartphones in general and internal medicine unitsA boon or a bane to the promotion of interprofessional collaboration J InterprofCare 201226(4)276ndash28233 Quan SD et al Itrsquos not about pager replacement An in-depth look at theinterprofessional nature of communication in healthcare J Hosp Med20138(3)137ndash143
Online-Only Content
See the online version of this article for
Figure 1 Stanford University Hospital Paging System (color version)
Figure 2 Percentage of Nurses Responding ldquoNearly Alwaysrdquo or
ldquoUsuallyrdquo in Response to the Question ldquoI know which
resident or pager number to page when I need physician
input in care of my patientsrdquo 2009 and 2011
(color version)
8
Copyright 2014 copy The Joint Commission
AP1February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
Stanford University Hospital Paging System
Online-Only Content8
Figure 1 The paging model requires residents to assign the treatment team to each patient and to forward the ghost pager (GP) to their personal pagerwhenever they are responsible for the patient or on call Nurses page physicians using the ghost-pager number which never changes for a team Ghost-pagernumbers are embedded in the electronic medical record (EMR) nursing work flow
Copyright 2014 copy The Joint Commission
AP2 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
71
75
40
80
67
65
33
91
73 76 80
82
72
74
75
96
100
93 100
85
80
97
100
79 85
94
77
91
0
10
20
30
40
50
60
70
80
90
100
CT
Sur
gery
(n =
11)
Med
icin
e - C
ardi
olog
y(n
= 1
12)
Med
icin
e - G
I(n
= 1
5)
Med
icin
e - H
emat
olog
y(n
= 5
5)
Med
icin
e - I
nfec
t Dis
ease
(n =
5)
Med
icin
e(n
= 2
12)
Neu
rolo
gy(n
= 1
6)
Neu
rosu
rger
y(n
= 6
5)
Onc
olog
y(n
= 8
7)
Orth
oped
ics
(n =
31)
Pain
( n =
28)
Sur
gica
l Spe
cial
ties
(n =
62)
Tran
spla
nt -
Sol
id O
rgan
(n =
31)
Trau
ma
(n =
81)
Kno
win
g w
ho to
pag
e
2009 2011
p = 90 p = 23 p = 36 p = 06 p = 32 p = 89 p = 75 p = 14 p = 77p lt 00 p lt 00 p lt 01 p lt 04p lt 03
Percentage of Nurses Responding ldquoNearly Alwaysrdquo or ldquoUsuallyrdquo in Response to the Question ldquoI know which resident or pager number to page when I need physician input in care of my patientsrdquo 2009 and 2011
Online-Only Content8
Figure 2 Nursesrsquo responses show their level of certainty about which physician to page in 2009 and again in 2011 after the implementation of the newpaging model CT cardiothoracic GI gastrointestinal Probability (p) levels for chi-square tests are shown an asterisk indicates statistical significance
Copyright 2014 copy The Joint Commission
79February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
ResultsADOPTION OF THE GHOST-PAGER MODEL
Adoption of the ghost-pager model was rapid and uniform andcompliance with treatment team assignment was very highAmong patients of faculty physicians 64 (315491) had anassigned treatment team within the first week of the rollout Thisvalue grew to greater than 80 within a month and has re-mained in the 92ndash93 range since then
NURSE SURVEY RESPONSES
There were 561 responses to the nurse survey in 2009 and916 responses in 2011 Before implementation of this model73 of the nurses responding to the survey indicated theyldquonearly alwaysrdquo or ldquousuallyrdquo knew whom to page This level ofcertainty varied considerably among services ranging from 33for neurology to 91 for neurosurgery Overall certainty for allservices rose to 87 after the model was implemented (range75 ndash100) Eight of the services had gt 90 certainty includ-ing three with a certainty rate of 100 (Figure 2 page 80)
The improvement for the whole hospital was highly signifi-cant (p lt 0001) The improvements for medicine-cardiology (p lt 0001) medicine-gastrointestinal (p lt 03) medicine (p lt0001) oncology (p lt 01) and trauma (p lt 04) were all statis-tically significant
QUALITATIVE RESULTS
Qualitative results were also positive Incident reports of in-ability to contact a physician using the ghost-pager number havebeen uncommon Nurses in general have expressed wide supportfor the model as have physicians Some services have begunusing ghost pagers to help sort calls within teams such as ldquoad-mitting residentrdquo or for additional consulting services Pagingoperators liked the new model as the increased work of morecalls to forward a pager were offset by reduction in the burdenof trying to find the correct physician often a frustrating processpunctuated by repeated calls from nurses upset that the physicianhad not called back Residents now can directly control handoffsor last-minute changes in coverage The model has remained in-
Stanford University Hospital Paging System
Figure 1 The paging model requires residents to assign the treatment team to each patient and to forward the ghost pager (GP) to their personal pager wheneverthey are responsible for the patient or on call Nurses page physicians using the ghost-pager number which that never changes for a team Ghost-pager numbersare embedded in the electronic medical record (EMR) nursing work flow (Available in color in online article)
Copyright 2014 copy The Joint Commission
80 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
tact without changes since it was implemented Respiratory ther-apists radiology technologists and some consulting services alsohave begun using some aspects of the model For example oto-laryngology uses two ghost pagers designated as ldquoConsults-Adultsrdquo and ldquoConsults-Pediatricrdquo for receiving and sortingrequests for consultation and the ICU consult team uses a ded-icated ghost pager as a point of contact
DiscussionEffective timely communication is essential to provide outstand-ing patient care Over the years there have been repeated con-cerns about improving nurse-to-physician communication18ndash25
In academic medical centers with complex teams of physiciansthe most fundamental step in communicating is finding the cor-rect physician responsible for the patient Our results suggest
that the use of a dedicated team-based pager number that is al-ways assigned to a team greatly improved the likelihood of anursersquos knowing whom to page This was particularly true forservices with more complicated team structures or whose pa-tients tend to be more dispersed The results confirm the find-ings reported by Wong et al15 who showed that a team pagerreduced the number of after-hours paging errors on a generalmedicine service in an academic hospital Our study extends thisfinding to multiple other medical and surgical specialties and toall three nursing shifts
The application of new communication technologies is oftenthe focus of communication improvement initiatives1426ndash32
However simply imposing these technologies on existing workflows may not improve communication273132 A technology maybe seen as an improvement for physicians but not nurses31 or
Percentage of Nurses Responding ldquoNearly Alwaysrdquo or ldquoUsuallyrdquo in Response to the Question ldquoI know which resident or pager number to page when I need physician input in
care of my patientsrdquo 2009 and 2011
Figure 2 Nursesrsquo responses show their level of certainty about which physician to page in 2009 and again in 2011 after the implementation of the new pagingmodel CT cardiothoracic GI gastrointestinal Probability (p) levels for chi-square tests are shown an asterisk indicates statistical significance (Available incolor in online article)
Copyright 2014 copy The Joint Commission
81February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
vice versa27 Furthermore one technology may be useful forsome types of communication but less so for others In onestudy the use of e-mail via smartphones was favored for convey-ing nonurgent information but other modes were preferred forurgent matters In addition team members disagreed about appropriate use of communication modes32 There is the possi-bility of making matters worse as Quan et al found when aWeb-based communication system increased interruptions ofresidents by 23333 The authors cautioned that ldquothe interplayof technology with existing clinical workflow culture and socialinteractions may create other unintended consequencesrdquo33(p 137)
The model evaluated in this study did not employ new tech-nologies depending only on those that were already being usedThe focus was on reducing variation and errors in the work flowof paging The results suggest significant potential for improvingpaging even without additional technological tools Further-more the work-flow improvements would have been neededeven if new communication technologies had been added
The new model implemented in this study did take advantageof functionality in the EMR system not previously employedThe hospitalrsquos informatics team was essential to move the processforward Embedding the ghost-pager number into the treatmentteam name eliminated any additional order entry steps for physi-cians assigning themselves to a patient The treatment team andghost-pager number information were also automatically pulledinto the nursing notes for additional quick reference A line wasadded in the RN sign-out note to enable nurses to communicatenonurgent issues to physicians for follow-up to reduce some ofthe unnecessary night-shift paging
The new model represented a significant change in traditionalpractice Stories drawn from paging incidents information gath-ered to characterize the problem and the data gathered in theinitial survey of nurses established the need for improving pagingeffectiveness From the onset of this project physicians re-sponded with genuine concern when reports of delays in patientcare were brought to their attention The congruency of datafrom all these sources was sufficiently compelling to establish amultidisciplinary team to critically evaluate paging effectivenessstrategies and implement the new model The potential to im-prove trainee experience with pages may have helped drive theircompliance with the new model as well
LIMITATIONS
This study has several limitations The results are based onnursesrsquo perceptions rather than objective measures of reducedpaging errors or of improved patient outcomes This was not arandomized control trial of the model The evaluators were not
blinded as to the intervention It is possible that the changes innursesrsquo perceptions were biased by the attention focused on themodel by the campaign conducted to increase adoption Never-theless the results suggest that this model can be helpful in re-ducing errors in communication due to paging the incorrectphysician Although improvements in communication in generalare expected to enhance quality and patient safety the impact ofthis model on improved patient outcomes is important to assessin further studies In addition more study is needed on whetherthe changes implemented have improved resident experiencewith the number and self-perceived appropriateness of pagingparticularly at night
SummaryAs complexity of care of hospitalized patients has increased theneed for communication and collaboration among members ofthe team caring for the patient has become increasingly impor-tant This often takes the form of a nursersquos need to contact a pa-tientrsquos physician to discuss some aspect of care and modifytreatment plans Errors in communication delay care and canpose risk to patients This report describes the successful imple-mentation of a standardized team-based paging system at an ac-ademic center Results showed a substantial improvement innursesrsquo perceptions of knowing how to contact the correct physi-cian when discussion of the patientrsquos care is needed This im-provement was found across multiple medical and surgicalspecialties and was particularly effective for services with thegreatest communication problems The authors thank Isabella Chu MPH for her assistance in reviewing and preparing
the manuscript and Pooja Loftus MS for providing statistical analysis
J
Lisa Shieh MD PhD is Clinical Associate Professor of Medicine Di-
vision of General Medical Disciplines Stanford School of Medicine
Stanford California and Unit Based Medical Director and Quality
Medical Director for Medicine Stanford Hospital and Clinics Jeffrey
Chi MD is Clinical Assistant Professor of Medicine Division of Gen-
eral Medical Disciplines Stanford School of Medicine and Unit Based
Medical Director Stanford Hospital and Clinics Carole Kulik RN
MSN is Director of Patient Care Practice and Education Nursing Ad-
ministration Stanford Hospital and Clinics Arash Momeni MD is
Chief Resident in Plastic Surgery Stanford Hospital and Clinics An-
drew Shelton MD is Clinical Associate Professor in Surgery Divi-
sion of Colorectal Surgery Stanford School of Medicine and Medical
Informatics Medical Director Stanford Hospital and Clinics Cynthia
DePorte RN MSN is Director of Cancer Services Stanford Hospital
and Clinics Joseph Hopkins MD MMM is Clinical Professor of
Medicine Division of General Medical Disciplines Stanford School of
Medicine and Senior Medical Director for Quality Stanford Hospital
and Clinics Please address correspondence to Joseph Hopkins
joehstanfordedu
Copyright 2014 copy The Joint Commission
82 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
References1 OrsquoLeary KJ Liebovitz DM Baker DW How hospitalists spend their timeInsights on efficiency and safety J Hosp Med 20061(2)88ndash932 Alvarez G Coiera E Interruptive communication patterns in the intensivecare unit ward round Int J Med Inform 200574(10)791ndash7963 Wu RC et al Effects of clinical communication interventions in hospitalsA systematic review of information and communication technology adoptionsfor improved communication between clinicians Int J Med Inform201281(11)723ndash7324 Dwyer K Communication breakdowns combine to cause system failure IntJ Qual Health Care 200113(3)2655 Dwyer K Flawed communication systems result in patient harm Int J QualHealth Care 200214(1)77 Erratum in Int J Qual Health Care 200315(1)996 Rollins G Medical errors poor communication undermine quality of carepatient satisfaction Rep Med Guidel Outcomes Res 2002 May 1713(10)5ndash77 Southwick LM Communication misadventures and medical errors Jt CommJ Qual Improv 200228(8)461ndash462 Comment on Weeks WB et al The or-ganizational costs of preventable medical errors Jt Comm J Qual Improv200127(10)533ndash539 Author reply 462ndash4638 Alvarez G Coiera E Interdisciplinary communication An uncharted sourceof medical error J Crit Care 200621(3)236ndash242 discussion 2429 Patel SP et al Resident workload pager communications and quality ofcare World J Surg 201034(11)2524ndash252910 Espino S Cox D Kaplan B Alphanumeric paging A potential source ofproblems in patient care and communication J Surg Educ 201168(6)447ndash45111 Agarwal R Sands DZ Schneider JD Quantifying the economic impact ofcommunication inefficiencies in US hospitals J Healthc Manag201055(4)265ndash281 discussion 281ndash28212 Wong BM et al Frequency and clinical importance of pages sent to thewrong physician Arch Intern Med 2009 Jun 8169(11)1072ndash107313 Nguyen TC et al Alphanumeric paging in an academic hospital settingAm J Surg 2006191(4)561ndash56514 Ighani F et al A comparison of two-way text versus conventional pagingsystems in an academic ophthalmology department J Med Syst201034(4)677ndash684
15 Wong BM et al Getting the message A quality improvement initiative toreduce pages sent to the wrong physician BMJ Qual Saf 201221(10)855ndash86216 Katz MH Schroeder SA The sounds of the hospital Paging patterns inthree teaching hospitals N Engl J Med 1988 Dec 15319(24)1585ndash158917 Myers JS Bellini LM Resident handoffs Appreciating them as a criticalcompetency J Gen Intern Med 201227(3)270ndash272 Comment on Helms ASet al Use of an appreciative inquiry approach to improve resident sign-out inan era of multiple shift changes J Gen Intern Med 201227(3)287ndash29018 Burns K Nurse-physician rounds A collaborative approach to improvingcommunication efficiencies and perception of care Medsurg Nurs201120(4)194ndash19919 Crawford CL Omery A Seago JA The challenges of nurse-physician com-munication A review of the evidence J Nurs Adm 201242(12)548ndash55020 Manojlovich M et al Developing and testing a tool to measure nursephysi-cian communication in the intensive care unit J Patient Saf 20117(2)80ndash8421 Nair DM et al Frequency of nurse-physician collaborative behaviors in anacute care hospital J Interprof Care 201226(2)115ndash12022 OrsquoLeary KJ et al Patterns of nurse-physician communication and agree-ment on the plan of care Qual Saf Health Care 201019(3)195ndash19923 Robinson FP et al Perceptions of effective and ineffective nurse-physiciancommunication in hospitals Nurs Forum 201045(3)206ndash21624 Rosenthal L Enhancing communication between night shift RNs and hos-pitalists An opportunity for performance improvement J Nurs Adm201343(2)59ndash6125 Tschannen D et al Implications of nurse-physician relations Report of asuccessful intervention Nurs Econ 201129(3)127ndash13526 Hanada E et al Advantages of low output mobile communication systemsin hospitals J Med Syst 200024(2)53ndash5927 Blair K Orr M Insights from an iBleep trial A report on lessons learnedHealth Care and Informatics Review Online 201115(1)3ndash1228 Horn G Thorel P Next-generation communication at Advocate HealthCare Alcatel Telecommunications Review 2006(1)10ndash1429 Etchells E et al Real-time clinical alerting Effect of an automated pagingsystem on response time to critical laboratory valuesmdashA randomised controlledtrial Qual Saf Health Care 201019(2)99ndash10230 Smith CN et al Understanding interprofessional communication A con-tent analysis of email communications between doctors and nurses Appl ClinInform 2012 Feb 13(1)38ndash5131 Wu RC et al The use of smartphones for clinical communication on in-ternal medicine wards J Hosp Med 20105(9)553ndash55932 Lo V et al The use of smartphones in general and internal medicine unitsA boon or a bane to the promotion of interprofessional collaboration J InterprofCare 201226(4)276ndash28233 Quan SD et al Itrsquos not about pager replacement An in-depth look at theinterprofessional nature of communication in healthcare J Hosp Med20138(3)137ndash143
Online-Only Content
See the online version of this article for
Figure 1 Stanford University Hospital Paging System (color version)
Figure 2 Percentage of Nurses Responding ldquoNearly Alwaysrdquo or
ldquoUsuallyrdquo in Response to the Question ldquoI know which
resident or pager number to page when I need physician
input in care of my patientsrdquo 2009 and 2011
(color version)
8
Copyright 2014 copy The Joint Commission
AP1February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
Stanford University Hospital Paging System
Online-Only Content8
Figure 1 The paging model requires residents to assign the treatment team to each patient and to forward the ghost pager (GP) to their personal pagerwhenever they are responsible for the patient or on call Nurses page physicians using the ghost-pager number which never changes for a team Ghost-pagernumbers are embedded in the electronic medical record (EMR) nursing work flow
Copyright 2014 copy The Joint Commission
AP2 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
71
75
40
80
67
65
33
91
73 76 80
82
72
74
75
96
100
93 100
85
80
97
100
79 85
94
77
91
0
10
20
30
40
50
60
70
80
90
100
CT
Sur
gery
(n =
11)
Med
icin
e - C
ardi
olog
y(n
= 1
12)
Med
icin
e - G
I(n
= 1
5)
Med
icin
e - H
emat
olog
y(n
= 5
5)
Med
icin
e - I
nfec
t Dis
ease
(n =
5)
Med
icin
e(n
= 2
12)
Neu
rolo
gy(n
= 1
6)
Neu
rosu
rger
y(n
= 6
5)
Onc
olog
y(n
= 8
7)
Orth
oped
ics
(n =
31)
Pain
( n =
28)
Sur
gica
l Spe
cial
ties
(n =
62)
Tran
spla
nt -
Sol
id O
rgan
(n =
31)
Trau
ma
(n =
81)
Kno
win
g w
ho to
pag
e
2009 2011
p = 90 p = 23 p = 36 p = 06 p = 32 p = 89 p = 75 p = 14 p = 77p lt 00 p lt 00 p lt 01 p lt 04p lt 03
Percentage of Nurses Responding ldquoNearly Alwaysrdquo or ldquoUsuallyrdquo in Response to the Question ldquoI know which resident or pager number to page when I need physician input in care of my patientsrdquo 2009 and 2011
Online-Only Content8
Figure 2 Nursesrsquo responses show their level of certainty about which physician to page in 2009 and again in 2011 after the implementation of the newpaging model CT cardiothoracic GI gastrointestinal Probability (p) levels for chi-square tests are shown an asterisk indicates statistical significance
Copyright 2014 copy The Joint Commission
80 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
tact without changes since it was implemented Respiratory ther-apists radiology technologists and some consulting services alsohave begun using some aspects of the model For example oto-laryngology uses two ghost pagers designated as ldquoConsults-Adultsrdquo and ldquoConsults-Pediatricrdquo for receiving and sortingrequests for consultation and the ICU consult team uses a ded-icated ghost pager as a point of contact
DiscussionEffective timely communication is essential to provide outstand-ing patient care Over the years there have been repeated con-cerns about improving nurse-to-physician communication18ndash25
In academic medical centers with complex teams of physiciansthe most fundamental step in communicating is finding the cor-rect physician responsible for the patient Our results suggest
that the use of a dedicated team-based pager number that is al-ways assigned to a team greatly improved the likelihood of anursersquos knowing whom to page This was particularly true forservices with more complicated team structures or whose pa-tients tend to be more dispersed The results confirm the find-ings reported by Wong et al15 who showed that a team pagerreduced the number of after-hours paging errors on a generalmedicine service in an academic hospital Our study extends thisfinding to multiple other medical and surgical specialties and toall three nursing shifts
The application of new communication technologies is oftenthe focus of communication improvement initiatives1426ndash32
However simply imposing these technologies on existing workflows may not improve communication273132 A technology maybe seen as an improvement for physicians but not nurses31 or
Percentage of Nurses Responding ldquoNearly Alwaysrdquo or ldquoUsuallyrdquo in Response to the Question ldquoI know which resident or pager number to page when I need physician input in
care of my patientsrdquo 2009 and 2011
Figure 2 Nursesrsquo responses show their level of certainty about which physician to page in 2009 and again in 2011 after the implementation of the new pagingmodel CT cardiothoracic GI gastrointestinal Probability (p) levels for chi-square tests are shown an asterisk indicates statistical significance (Available incolor in online article)
Copyright 2014 copy The Joint Commission
81February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
vice versa27 Furthermore one technology may be useful forsome types of communication but less so for others In onestudy the use of e-mail via smartphones was favored for convey-ing nonurgent information but other modes were preferred forurgent matters In addition team members disagreed about appropriate use of communication modes32 There is the possi-bility of making matters worse as Quan et al found when aWeb-based communication system increased interruptions ofresidents by 23333 The authors cautioned that ldquothe interplayof technology with existing clinical workflow culture and socialinteractions may create other unintended consequencesrdquo33(p 137)
The model evaluated in this study did not employ new tech-nologies depending only on those that were already being usedThe focus was on reducing variation and errors in the work flowof paging The results suggest significant potential for improvingpaging even without additional technological tools Further-more the work-flow improvements would have been neededeven if new communication technologies had been added
The new model implemented in this study did take advantageof functionality in the EMR system not previously employedThe hospitalrsquos informatics team was essential to move the processforward Embedding the ghost-pager number into the treatmentteam name eliminated any additional order entry steps for physi-cians assigning themselves to a patient The treatment team andghost-pager number information were also automatically pulledinto the nursing notes for additional quick reference A line wasadded in the RN sign-out note to enable nurses to communicatenonurgent issues to physicians for follow-up to reduce some ofthe unnecessary night-shift paging
The new model represented a significant change in traditionalpractice Stories drawn from paging incidents information gath-ered to characterize the problem and the data gathered in theinitial survey of nurses established the need for improving pagingeffectiveness From the onset of this project physicians re-sponded with genuine concern when reports of delays in patientcare were brought to their attention The congruency of datafrom all these sources was sufficiently compelling to establish amultidisciplinary team to critically evaluate paging effectivenessstrategies and implement the new model The potential to im-prove trainee experience with pages may have helped drive theircompliance with the new model as well
LIMITATIONS
This study has several limitations The results are based onnursesrsquo perceptions rather than objective measures of reducedpaging errors or of improved patient outcomes This was not arandomized control trial of the model The evaluators were not
blinded as to the intervention It is possible that the changes innursesrsquo perceptions were biased by the attention focused on themodel by the campaign conducted to increase adoption Never-theless the results suggest that this model can be helpful in re-ducing errors in communication due to paging the incorrectphysician Although improvements in communication in generalare expected to enhance quality and patient safety the impact ofthis model on improved patient outcomes is important to assessin further studies In addition more study is needed on whetherthe changes implemented have improved resident experiencewith the number and self-perceived appropriateness of pagingparticularly at night
SummaryAs complexity of care of hospitalized patients has increased theneed for communication and collaboration among members ofthe team caring for the patient has become increasingly impor-tant This often takes the form of a nursersquos need to contact a pa-tientrsquos physician to discuss some aspect of care and modifytreatment plans Errors in communication delay care and canpose risk to patients This report describes the successful imple-mentation of a standardized team-based paging system at an ac-ademic center Results showed a substantial improvement innursesrsquo perceptions of knowing how to contact the correct physi-cian when discussion of the patientrsquos care is needed This im-provement was found across multiple medical and surgicalspecialties and was particularly effective for services with thegreatest communication problems The authors thank Isabella Chu MPH for her assistance in reviewing and preparing
the manuscript and Pooja Loftus MS for providing statistical analysis
J
Lisa Shieh MD PhD is Clinical Associate Professor of Medicine Di-
vision of General Medical Disciplines Stanford School of Medicine
Stanford California and Unit Based Medical Director and Quality
Medical Director for Medicine Stanford Hospital and Clinics Jeffrey
Chi MD is Clinical Assistant Professor of Medicine Division of Gen-
eral Medical Disciplines Stanford School of Medicine and Unit Based
Medical Director Stanford Hospital and Clinics Carole Kulik RN
MSN is Director of Patient Care Practice and Education Nursing Ad-
ministration Stanford Hospital and Clinics Arash Momeni MD is
Chief Resident in Plastic Surgery Stanford Hospital and Clinics An-
drew Shelton MD is Clinical Associate Professor in Surgery Divi-
sion of Colorectal Surgery Stanford School of Medicine and Medical
Informatics Medical Director Stanford Hospital and Clinics Cynthia
DePorte RN MSN is Director of Cancer Services Stanford Hospital
and Clinics Joseph Hopkins MD MMM is Clinical Professor of
Medicine Division of General Medical Disciplines Stanford School of
Medicine and Senior Medical Director for Quality Stanford Hospital
and Clinics Please address correspondence to Joseph Hopkins
joehstanfordedu
Copyright 2014 copy The Joint Commission
82 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
References1 OrsquoLeary KJ Liebovitz DM Baker DW How hospitalists spend their timeInsights on efficiency and safety J Hosp Med 20061(2)88ndash932 Alvarez G Coiera E Interruptive communication patterns in the intensivecare unit ward round Int J Med Inform 200574(10)791ndash7963 Wu RC et al Effects of clinical communication interventions in hospitalsA systematic review of information and communication technology adoptionsfor improved communication between clinicians Int J Med Inform201281(11)723ndash7324 Dwyer K Communication breakdowns combine to cause system failure IntJ Qual Health Care 200113(3)2655 Dwyer K Flawed communication systems result in patient harm Int J QualHealth Care 200214(1)77 Erratum in Int J Qual Health Care 200315(1)996 Rollins G Medical errors poor communication undermine quality of carepatient satisfaction Rep Med Guidel Outcomes Res 2002 May 1713(10)5ndash77 Southwick LM Communication misadventures and medical errors Jt CommJ Qual Improv 200228(8)461ndash462 Comment on Weeks WB et al The or-ganizational costs of preventable medical errors Jt Comm J Qual Improv200127(10)533ndash539 Author reply 462ndash4638 Alvarez G Coiera E Interdisciplinary communication An uncharted sourceof medical error J Crit Care 200621(3)236ndash242 discussion 2429 Patel SP et al Resident workload pager communications and quality ofcare World J Surg 201034(11)2524ndash252910 Espino S Cox D Kaplan B Alphanumeric paging A potential source ofproblems in patient care and communication J Surg Educ 201168(6)447ndash45111 Agarwal R Sands DZ Schneider JD Quantifying the economic impact ofcommunication inefficiencies in US hospitals J Healthc Manag201055(4)265ndash281 discussion 281ndash28212 Wong BM et al Frequency and clinical importance of pages sent to thewrong physician Arch Intern Med 2009 Jun 8169(11)1072ndash107313 Nguyen TC et al Alphanumeric paging in an academic hospital settingAm J Surg 2006191(4)561ndash56514 Ighani F et al A comparison of two-way text versus conventional pagingsystems in an academic ophthalmology department J Med Syst201034(4)677ndash684
15 Wong BM et al Getting the message A quality improvement initiative toreduce pages sent to the wrong physician BMJ Qual Saf 201221(10)855ndash86216 Katz MH Schroeder SA The sounds of the hospital Paging patterns inthree teaching hospitals N Engl J Med 1988 Dec 15319(24)1585ndash158917 Myers JS Bellini LM Resident handoffs Appreciating them as a criticalcompetency J Gen Intern Med 201227(3)270ndash272 Comment on Helms ASet al Use of an appreciative inquiry approach to improve resident sign-out inan era of multiple shift changes J Gen Intern Med 201227(3)287ndash29018 Burns K Nurse-physician rounds A collaborative approach to improvingcommunication efficiencies and perception of care Medsurg Nurs201120(4)194ndash19919 Crawford CL Omery A Seago JA The challenges of nurse-physician com-munication A review of the evidence J Nurs Adm 201242(12)548ndash55020 Manojlovich M et al Developing and testing a tool to measure nursephysi-cian communication in the intensive care unit J Patient Saf 20117(2)80ndash8421 Nair DM et al Frequency of nurse-physician collaborative behaviors in anacute care hospital J Interprof Care 201226(2)115ndash12022 OrsquoLeary KJ et al Patterns of nurse-physician communication and agree-ment on the plan of care Qual Saf Health Care 201019(3)195ndash19923 Robinson FP et al Perceptions of effective and ineffective nurse-physiciancommunication in hospitals Nurs Forum 201045(3)206ndash21624 Rosenthal L Enhancing communication between night shift RNs and hos-pitalists An opportunity for performance improvement J Nurs Adm201343(2)59ndash6125 Tschannen D et al Implications of nurse-physician relations Report of asuccessful intervention Nurs Econ 201129(3)127ndash13526 Hanada E et al Advantages of low output mobile communication systemsin hospitals J Med Syst 200024(2)53ndash5927 Blair K Orr M Insights from an iBleep trial A report on lessons learnedHealth Care and Informatics Review Online 201115(1)3ndash1228 Horn G Thorel P Next-generation communication at Advocate HealthCare Alcatel Telecommunications Review 2006(1)10ndash1429 Etchells E et al Real-time clinical alerting Effect of an automated pagingsystem on response time to critical laboratory valuesmdashA randomised controlledtrial Qual Saf Health Care 201019(2)99ndash10230 Smith CN et al Understanding interprofessional communication A con-tent analysis of email communications between doctors and nurses Appl ClinInform 2012 Feb 13(1)38ndash5131 Wu RC et al The use of smartphones for clinical communication on in-ternal medicine wards J Hosp Med 20105(9)553ndash55932 Lo V et al The use of smartphones in general and internal medicine unitsA boon or a bane to the promotion of interprofessional collaboration J InterprofCare 201226(4)276ndash28233 Quan SD et al Itrsquos not about pager replacement An in-depth look at theinterprofessional nature of communication in healthcare J Hosp Med20138(3)137ndash143
Online-Only Content
See the online version of this article for
Figure 1 Stanford University Hospital Paging System (color version)
Figure 2 Percentage of Nurses Responding ldquoNearly Alwaysrdquo or
ldquoUsuallyrdquo in Response to the Question ldquoI know which
resident or pager number to page when I need physician
input in care of my patientsrdquo 2009 and 2011
(color version)
8
Copyright 2014 copy The Joint Commission
AP1February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
Stanford University Hospital Paging System
Online-Only Content8
Figure 1 The paging model requires residents to assign the treatment team to each patient and to forward the ghost pager (GP) to their personal pagerwhenever they are responsible for the patient or on call Nurses page physicians using the ghost-pager number which never changes for a team Ghost-pagernumbers are embedded in the electronic medical record (EMR) nursing work flow
Copyright 2014 copy The Joint Commission
AP2 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
71
75
40
80
67
65
33
91
73 76 80
82
72
74
75
96
100
93 100
85
80
97
100
79 85
94
77
91
0
10
20
30
40
50
60
70
80
90
100
CT
Sur
gery
(n =
11)
Med
icin
e - C
ardi
olog
y(n
= 1
12)
Med
icin
e - G
I(n
= 1
5)
Med
icin
e - H
emat
olog
y(n
= 5
5)
Med
icin
e - I
nfec
t Dis
ease
(n =
5)
Med
icin
e(n
= 2
12)
Neu
rolo
gy(n
= 1
6)
Neu
rosu
rger
y(n
= 6
5)
Onc
olog
y(n
= 8
7)
Orth
oped
ics
(n =
31)
Pain
( n =
28)
Sur
gica
l Spe
cial
ties
(n =
62)
Tran
spla
nt -
Sol
id O
rgan
(n =
31)
Trau
ma
(n =
81)
Kno
win
g w
ho to
pag
e
2009 2011
p = 90 p = 23 p = 36 p = 06 p = 32 p = 89 p = 75 p = 14 p = 77p lt 00 p lt 00 p lt 01 p lt 04p lt 03
Percentage of Nurses Responding ldquoNearly Alwaysrdquo or ldquoUsuallyrdquo in Response to the Question ldquoI know which resident or pager number to page when I need physician input in care of my patientsrdquo 2009 and 2011
Online-Only Content8
Figure 2 Nursesrsquo responses show their level of certainty about which physician to page in 2009 and again in 2011 after the implementation of the newpaging model CT cardiothoracic GI gastrointestinal Probability (p) levels for chi-square tests are shown an asterisk indicates statistical significance
Copyright 2014 copy The Joint Commission
81February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
vice versa27 Furthermore one technology may be useful forsome types of communication but less so for others In onestudy the use of e-mail via smartphones was favored for convey-ing nonurgent information but other modes were preferred forurgent matters In addition team members disagreed about appropriate use of communication modes32 There is the possi-bility of making matters worse as Quan et al found when aWeb-based communication system increased interruptions ofresidents by 23333 The authors cautioned that ldquothe interplayof technology with existing clinical workflow culture and socialinteractions may create other unintended consequencesrdquo33(p 137)
The model evaluated in this study did not employ new tech-nologies depending only on those that were already being usedThe focus was on reducing variation and errors in the work flowof paging The results suggest significant potential for improvingpaging even without additional technological tools Further-more the work-flow improvements would have been neededeven if new communication technologies had been added
The new model implemented in this study did take advantageof functionality in the EMR system not previously employedThe hospitalrsquos informatics team was essential to move the processforward Embedding the ghost-pager number into the treatmentteam name eliminated any additional order entry steps for physi-cians assigning themselves to a patient The treatment team andghost-pager number information were also automatically pulledinto the nursing notes for additional quick reference A line wasadded in the RN sign-out note to enable nurses to communicatenonurgent issues to physicians for follow-up to reduce some ofthe unnecessary night-shift paging
The new model represented a significant change in traditionalpractice Stories drawn from paging incidents information gath-ered to characterize the problem and the data gathered in theinitial survey of nurses established the need for improving pagingeffectiveness From the onset of this project physicians re-sponded with genuine concern when reports of delays in patientcare were brought to their attention The congruency of datafrom all these sources was sufficiently compelling to establish amultidisciplinary team to critically evaluate paging effectivenessstrategies and implement the new model The potential to im-prove trainee experience with pages may have helped drive theircompliance with the new model as well
LIMITATIONS
This study has several limitations The results are based onnursesrsquo perceptions rather than objective measures of reducedpaging errors or of improved patient outcomes This was not arandomized control trial of the model The evaluators were not
blinded as to the intervention It is possible that the changes innursesrsquo perceptions were biased by the attention focused on themodel by the campaign conducted to increase adoption Never-theless the results suggest that this model can be helpful in re-ducing errors in communication due to paging the incorrectphysician Although improvements in communication in generalare expected to enhance quality and patient safety the impact ofthis model on improved patient outcomes is important to assessin further studies In addition more study is needed on whetherthe changes implemented have improved resident experiencewith the number and self-perceived appropriateness of pagingparticularly at night
SummaryAs complexity of care of hospitalized patients has increased theneed for communication and collaboration among members ofthe team caring for the patient has become increasingly impor-tant This often takes the form of a nursersquos need to contact a pa-tientrsquos physician to discuss some aspect of care and modifytreatment plans Errors in communication delay care and canpose risk to patients This report describes the successful imple-mentation of a standardized team-based paging system at an ac-ademic center Results showed a substantial improvement innursesrsquo perceptions of knowing how to contact the correct physi-cian when discussion of the patientrsquos care is needed This im-provement was found across multiple medical and surgicalspecialties and was particularly effective for services with thegreatest communication problems The authors thank Isabella Chu MPH for her assistance in reviewing and preparing
the manuscript and Pooja Loftus MS for providing statistical analysis
J
Lisa Shieh MD PhD is Clinical Associate Professor of Medicine Di-
vision of General Medical Disciplines Stanford School of Medicine
Stanford California and Unit Based Medical Director and Quality
Medical Director for Medicine Stanford Hospital and Clinics Jeffrey
Chi MD is Clinical Assistant Professor of Medicine Division of Gen-
eral Medical Disciplines Stanford School of Medicine and Unit Based
Medical Director Stanford Hospital and Clinics Carole Kulik RN
MSN is Director of Patient Care Practice and Education Nursing Ad-
ministration Stanford Hospital and Clinics Arash Momeni MD is
Chief Resident in Plastic Surgery Stanford Hospital and Clinics An-
drew Shelton MD is Clinical Associate Professor in Surgery Divi-
sion of Colorectal Surgery Stanford School of Medicine and Medical
Informatics Medical Director Stanford Hospital and Clinics Cynthia
DePorte RN MSN is Director of Cancer Services Stanford Hospital
and Clinics Joseph Hopkins MD MMM is Clinical Professor of
Medicine Division of General Medical Disciplines Stanford School of
Medicine and Senior Medical Director for Quality Stanford Hospital
and Clinics Please address correspondence to Joseph Hopkins
joehstanfordedu
Copyright 2014 copy The Joint Commission
82 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
References1 OrsquoLeary KJ Liebovitz DM Baker DW How hospitalists spend their timeInsights on efficiency and safety J Hosp Med 20061(2)88ndash932 Alvarez G Coiera E Interruptive communication patterns in the intensivecare unit ward round Int J Med Inform 200574(10)791ndash7963 Wu RC et al Effects of clinical communication interventions in hospitalsA systematic review of information and communication technology adoptionsfor improved communication between clinicians Int J Med Inform201281(11)723ndash7324 Dwyer K Communication breakdowns combine to cause system failure IntJ Qual Health Care 200113(3)2655 Dwyer K Flawed communication systems result in patient harm Int J QualHealth Care 200214(1)77 Erratum in Int J Qual Health Care 200315(1)996 Rollins G Medical errors poor communication undermine quality of carepatient satisfaction Rep Med Guidel Outcomes Res 2002 May 1713(10)5ndash77 Southwick LM Communication misadventures and medical errors Jt CommJ Qual Improv 200228(8)461ndash462 Comment on Weeks WB et al The or-ganizational costs of preventable medical errors Jt Comm J Qual Improv200127(10)533ndash539 Author reply 462ndash4638 Alvarez G Coiera E Interdisciplinary communication An uncharted sourceof medical error J Crit Care 200621(3)236ndash242 discussion 2429 Patel SP et al Resident workload pager communications and quality ofcare World J Surg 201034(11)2524ndash252910 Espino S Cox D Kaplan B Alphanumeric paging A potential source ofproblems in patient care and communication J Surg Educ 201168(6)447ndash45111 Agarwal R Sands DZ Schneider JD Quantifying the economic impact ofcommunication inefficiencies in US hospitals J Healthc Manag201055(4)265ndash281 discussion 281ndash28212 Wong BM et al Frequency and clinical importance of pages sent to thewrong physician Arch Intern Med 2009 Jun 8169(11)1072ndash107313 Nguyen TC et al Alphanumeric paging in an academic hospital settingAm J Surg 2006191(4)561ndash56514 Ighani F et al A comparison of two-way text versus conventional pagingsystems in an academic ophthalmology department J Med Syst201034(4)677ndash684
15 Wong BM et al Getting the message A quality improvement initiative toreduce pages sent to the wrong physician BMJ Qual Saf 201221(10)855ndash86216 Katz MH Schroeder SA The sounds of the hospital Paging patterns inthree teaching hospitals N Engl J Med 1988 Dec 15319(24)1585ndash158917 Myers JS Bellini LM Resident handoffs Appreciating them as a criticalcompetency J Gen Intern Med 201227(3)270ndash272 Comment on Helms ASet al Use of an appreciative inquiry approach to improve resident sign-out inan era of multiple shift changes J Gen Intern Med 201227(3)287ndash29018 Burns K Nurse-physician rounds A collaborative approach to improvingcommunication efficiencies and perception of care Medsurg Nurs201120(4)194ndash19919 Crawford CL Omery A Seago JA The challenges of nurse-physician com-munication A review of the evidence J Nurs Adm 201242(12)548ndash55020 Manojlovich M et al Developing and testing a tool to measure nursephysi-cian communication in the intensive care unit J Patient Saf 20117(2)80ndash8421 Nair DM et al Frequency of nurse-physician collaborative behaviors in anacute care hospital J Interprof Care 201226(2)115ndash12022 OrsquoLeary KJ et al Patterns of nurse-physician communication and agree-ment on the plan of care Qual Saf Health Care 201019(3)195ndash19923 Robinson FP et al Perceptions of effective and ineffective nurse-physiciancommunication in hospitals Nurs Forum 201045(3)206ndash21624 Rosenthal L Enhancing communication between night shift RNs and hos-pitalists An opportunity for performance improvement J Nurs Adm201343(2)59ndash6125 Tschannen D et al Implications of nurse-physician relations Report of asuccessful intervention Nurs Econ 201129(3)127ndash13526 Hanada E et al Advantages of low output mobile communication systemsin hospitals J Med Syst 200024(2)53ndash5927 Blair K Orr M Insights from an iBleep trial A report on lessons learnedHealth Care and Informatics Review Online 201115(1)3ndash1228 Horn G Thorel P Next-generation communication at Advocate HealthCare Alcatel Telecommunications Review 2006(1)10ndash1429 Etchells E et al Real-time clinical alerting Effect of an automated pagingsystem on response time to critical laboratory valuesmdashA randomised controlledtrial Qual Saf Health Care 201019(2)99ndash10230 Smith CN et al Understanding interprofessional communication A con-tent analysis of email communications between doctors and nurses Appl ClinInform 2012 Feb 13(1)38ndash5131 Wu RC et al The use of smartphones for clinical communication on in-ternal medicine wards J Hosp Med 20105(9)553ndash55932 Lo V et al The use of smartphones in general and internal medicine unitsA boon or a bane to the promotion of interprofessional collaboration J InterprofCare 201226(4)276ndash28233 Quan SD et al Itrsquos not about pager replacement An in-depth look at theinterprofessional nature of communication in healthcare J Hosp Med20138(3)137ndash143
Online-Only Content
See the online version of this article for
Figure 1 Stanford University Hospital Paging System (color version)
Figure 2 Percentage of Nurses Responding ldquoNearly Alwaysrdquo or
ldquoUsuallyrdquo in Response to the Question ldquoI know which
resident or pager number to page when I need physician
input in care of my patientsrdquo 2009 and 2011
(color version)
8
Copyright 2014 copy The Joint Commission
AP1February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
Stanford University Hospital Paging System
Online-Only Content8
Figure 1 The paging model requires residents to assign the treatment team to each patient and to forward the ghost pager (GP) to their personal pagerwhenever they are responsible for the patient or on call Nurses page physicians using the ghost-pager number which never changes for a team Ghost-pagernumbers are embedded in the electronic medical record (EMR) nursing work flow
Copyright 2014 copy The Joint Commission
AP2 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
71
75
40
80
67
65
33
91
73 76 80
82
72
74
75
96
100
93 100
85
80
97
100
79 85
94
77
91
0
10
20
30
40
50
60
70
80
90
100
CT
Sur
gery
(n =
11)
Med
icin
e - C
ardi
olog
y(n
= 1
12)
Med
icin
e - G
I(n
= 1
5)
Med
icin
e - H
emat
olog
y(n
= 5
5)
Med
icin
e - I
nfec
t Dis
ease
(n =
5)
Med
icin
e(n
= 2
12)
Neu
rolo
gy(n
= 1
6)
Neu
rosu
rger
y(n
= 6
5)
Onc
olog
y(n
= 8
7)
Orth
oped
ics
(n =
31)
Pain
( n =
28)
Sur
gica
l Spe
cial
ties
(n =
62)
Tran
spla
nt -
Sol
id O
rgan
(n =
31)
Trau
ma
(n =
81)
Kno
win
g w
ho to
pag
e
2009 2011
p = 90 p = 23 p = 36 p = 06 p = 32 p = 89 p = 75 p = 14 p = 77p lt 00 p lt 00 p lt 01 p lt 04p lt 03
Percentage of Nurses Responding ldquoNearly Alwaysrdquo or ldquoUsuallyrdquo in Response to the Question ldquoI know which resident or pager number to page when I need physician input in care of my patientsrdquo 2009 and 2011
Online-Only Content8
Figure 2 Nursesrsquo responses show their level of certainty about which physician to page in 2009 and again in 2011 after the implementation of the newpaging model CT cardiothoracic GI gastrointestinal Probability (p) levels for chi-square tests are shown an asterisk indicates statistical significance
Copyright 2014 copy The Joint Commission
82 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
References1 OrsquoLeary KJ Liebovitz DM Baker DW How hospitalists spend their timeInsights on efficiency and safety J Hosp Med 20061(2)88ndash932 Alvarez G Coiera E Interruptive communication patterns in the intensivecare unit ward round Int J Med Inform 200574(10)791ndash7963 Wu RC et al Effects of clinical communication interventions in hospitalsA systematic review of information and communication technology adoptionsfor improved communication between clinicians Int J Med Inform201281(11)723ndash7324 Dwyer K Communication breakdowns combine to cause system failure IntJ Qual Health Care 200113(3)2655 Dwyer K Flawed communication systems result in patient harm Int J QualHealth Care 200214(1)77 Erratum in Int J Qual Health Care 200315(1)996 Rollins G Medical errors poor communication undermine quality of carepatient satisfaction Rep Med Guidel Outcomes Res 2002 May 1713(10)5ndash77 Southwick LM Communication misadventures and medical errors Jt CommJ Qual Improv 200228(8)461ndash462 Comment on Weeks WB et al The or-ganizational costs of preventable medical errors Jt Comm J Qual Improv200127(10)533ndash539 Author reply 462ndash4638 Alvarez G Coiera E Interdisciplinary communication An uncharted sourceof medical error J Crit Care 200621(3)236ndash242 discussion 2429 Patel SP et al Resident workload pager communications and quality ofcare World J Surg 201034(11)2524ndash252910 Espino S Cox D Kaplan B Alphanumeric paging A potential source ofproblems in patient care and communication J Surg Educ 201168(6)447ndash45111 Agarwal R Sands DZ Schneider JD Quantifying the economic impact ofcommunication inefficiencies in US hospitals J Healthc Manag201055(4)265ndash281 discussion 281ndash28212 Wong BM et al Frequency and clinical importance of pages sent to thewrong physician Arch Intern Med 2009 Jun 8169(11)1072ndash107313 Nguyen TC et al Alphanumeric paging in an academic hospital settingAm J Surg 2006191(4)561ndash56514 Ighani F et al A comparison of two-way text versus conventional pagingsystems in an academic ophthalmology department J Med Syst201034(4)677ndash684
15 Wong BM et al Getting the message A quality improvement initiative toreduce pages sent to the wrong physician BMJ Qual Saf 201221(10)855ndash86216 Katz MH Schroeder SA The sounds of the hospital Paging patterns inthree teaching hospitals N Engl J Med 1988 Dec 15319(24)1585ndash158917 Myers JS Bellini LM Resident handoffs Appreciating them as a criticalcompetency J Gen Intern Med 201227(3)270ndash272 Comment on Helms ASet al Use of an appreciative inquiry approach to improve resident sign-out inan era of multiple shift changes J Gen Intern Med 201227(3)287ndash29018 Burns K Nurse-physician rounds A collaborative approach to improvingcommunication efficiencies and perception of care Medsurg Nurs201120(4)194ndash19919 Crawford CL Omery A Seago JA The challenges of nurse-physician com-munication A review of the evidence J Nurs Adm 201242(12)548ndash55020 Manojlovich M et al Developing and testing a tool to measure nursephysi-cian communication in the intensive care unit J Patient Saf 20117(2)80ndash8421 Nair DM et al Frequency of nurse-physician collaborative behaviors in anacute care hospital J Interprof Care 201226(2)115ndash12022 OrsquoLeary KJ et al Patterns of nurse-physician communication and agree-ment on the plan of care Qual Saf Health Care 201019(3)195ndash19923 Robinson FP et al Perceptions of effective and ineffective nurse-physiciancommunication in hospitals Nurs Forum 201045(3)206ndash21624 Rosenthal L Enhancing communication between night shift RNs and hos-pitalists An opportunity for performance improvement J Nurs Adm201343(2)59ndash6125 Tschannen D et al Implications of nurse-physician relations Report of asuccessful intervention Nurs Econ 201129(3)127ndash13526 Hanada E et al Advantages of low output mobile communication systemsin hospitals J Med Syst 200024(2)53ndash5927 Blair K Orr M Insights from an iBleep trial A report on lessons learnedHealth Care and Informatics Review Online 201115(1)3ndash1228 Horn G Thorel P Next-generation communication at Advocate HealthCare Alcatel Telecommunications Review 2006(1)10ndash1429 Etchells E et al Real-time clinical alerting Effect of an automated pagingsystem on response time to critical laboratory valuesmdashA randomised controlledtrial Qual Saf Health Care 201019(2)99ndash10230 Smith CN et al Understanding interprofessional communication A con-tent analysis of email communications between doctors and nurses Appl ClinInform 2012 Feb 13(1)38ndash5131 Wu RC et al The use of smartphones for clinical communication on in-ternal medicine wards J Hosp Med 20105(9)553ndash55932 Lo V et al The use of smartphones in general and internal medicine unitsA boon or a bane to the promotion of interprofessional collaboration J InterprofCare 201226(4)276ndash28233 Quan SD et al Itrsquos not about pager replacement An in-depth look at theinterprofessional nature of communication in healthcare J Hosp Med20138(3)137ndash143
Online-Only Content
See the online version of this article for
Figure 1 Stanford University Hospital Paging System (color version)
Figure 2 Percentage of Nurses Responding ldquoNearly Alwaysrdquo or
ldquoUsuallyrdquo in Response to the Question ldquoI know which
resident or pager number to page when I need physician
input in care of my patientsrdquo 2009 and 2011
(color version)
8
Copyright 2014 copy The Joint Commission
AP1February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
Stanford University Hospital Paging System
Online-Only Content8
Figure 1 The paging model requires residents to assign the treatment team to each patient and to forward the ghost pager (GP) to their personal pagerwhenever they are responsible for the patient or on call Nurses page physicians using the ghost-pager number which never changes for a team Ghost-pagernumbers are embedded in the electronic medical record (EMR) nursing work flow
Copyright 2014 copy The Joint Commission
AP2 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
71
75
40
80
67
65
33
91
73 76 80
82
72
74
75
96
100
93 100
85
80
97
100
79 85
94
77
91
0
10
20
30
40
50
60
70
80
90
100
CT
Sur
gery
(n =
11)
Med
icin
e - C
ardi
olog
y(n
= 1
12)
Med
icin
e - G
I(n
= 1
5)
Med
icin
e - H
emat
olog
y(n
= 5
5)
Med
icin
e - I
nfec
t Dis
ease
(n =
5)
Med
icin
e(n
= 2
12)
Neu
rolo
gy(n
= 1
6)
Neu
rosu
rger
y(n
= 6
5)
Onc
olog
y(n
= 8
7)
Orth
oped
ics
(n =
31)
Pain
( n =
28)
Sur
gica
l Spe
cial
ties
(n =
62)
Tran
spla
nt -
Sol
id O
rgan
(n =
31)
Trau
ma
(n =
81)
Kno
win
g w
ho to
pag
e
2009 2011
p = 90 p = 23 p = 36 p = 06 p = 32 p = 89 p = 75 p = 14 p = 77p lt 00 p lt 00 p lt 01 p lt 04p lt 03
Percentage of Nurses Responding ldquoNearly Alwaysrdquo or ldquoUsuallyrdquo in Response to the Question ldquoI know which resident or pager number to page when I need physician input in care of my patientsrdquo 2009 and 2011
Online-Only Content8
Figure 2 Nursesrsquo responses show their level of certainty about which physician to page in 2009 and again in 2011 after the implementation of the newpaging model CT cardiothoracic GI gastrointestinal Probability (p) levels for chi-square tests are shown an asterisk indicates statistical significance
Copyright 2014 copy The Joint Commission
AP1February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
Stanford University Hospital Paging System
Online-Only Content8
Figure 1 The paging model requires residents to assign the treatment team to each patient and to forward the ghost pager (GP) to their personal pagerwhenever they are responsible for the patient or on call Nurses page physicians using the ghost-pager number which never changes for a team Ghost-pagernumbers are embedded in the electronic medical record (EMR) nursing work flow
Copyright 2014 copy The Joint Commission
AP2 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
71
75
40
80
67
65
33
91
73 76 80
82
72
74
75
96
100
93 100
85
80
97
100
79 85
94
77
91
0
10
20
30
40
50
60
70
80
90
100
CT
Sur
gery
(n =
11)
Med
icin
e - C
ardi
olog
y(n
= 1
12)
Med
icin
e - G
I(n
= 1
5)
Med
icin
e - H
emat
olog
y(n
= 5
5)
Med
icin
e - I
nfec
t Dis
ease
(n =
5)
Med
icin
e(n
= 2
12)
Neu
rolo
gy(n
= 1
6)
Neu
rosu
rger
y(n
= 6
5)
Onc
olog
y(n
= 8
7)
Orth
oped
ics
(n =
31)
Pain
( n =
28)
Sur
gica
l Spe
cial
ties
(n =
62)
Tran
spla
nt -
Sol
id O
rgan
(n =
31)
Trau
ma
(n =
81)
Kno
win
g w
ho to
pag
e
2009 2011
p = 90 p = 23 p = 36 p = 06 p = 32 p = 89 p = 75 p = 14 p = 77p lt 00 p lt 00 p lt 01 p lt 04p lt 03
Percentage of Nurses Responding ldquoNearly Alwaysrdquo or ldquoUsuallyrdquo in Response to the Question ldquoI know which resident or pager number to page when I need physician input in care of my patientsrdquo 2009 and 2011
Online-Only Content8
Figure 2 Nursesrsquo responses show their level of certainty about which physician to page in 2009 and again in 2011 after the implementation of the newpaging model CT cardiothoracic GI gastrointestinal Probability (p) levels for chi-square tests are shown an asterisk indicates statistical significance
Copyright 2014 copy The Joint Commission
AP2 February 2014 Volume 40 Number 2
The Joint Commission Journal on Quality and Patient Safety
71
75
40
80
67
65
33
91
73 76 80
82
72
74
75
96
100
93 100
85
80
97
100
79 85
94
77
91
0
10
20
30
40
50
60
70
80
90
100
CT
Sur
gery
(n =
11)
Med
icin
e - C
ardi
olog
y(n
= 1
12)
Med
icin
e - G
I(n
= 1
5)
Med
icin
e - H
emat
olog
y(n
= 5
5)
Med
icin
e - I
nfec
t Dis
ease
(n =
5)
Med
icin
e(n
= 2
12)
Neu
rolo
gy(n
= 1
6)
Neu
rosu
rger
y(n
= 6
5)
Onc
olog
y(n
= 8
7)
Orth
oped
ics
(n =
31)
Pain
( n =
28)
Sur
gica
l Spe
cial
ties
(n =
62)
Tran
spla
nt -
Sol
id O
rgan
(n =
31)
Trau
ma
(n =
81)
Kno
win
g w
ho to
pag
e
2009 2011
p = 90 p = 23 p = 36 p = 06 p = 32 p = 89 p = 75 p = 14 p = 77p lt 00 p lt 00 p lt 01 p lt 04p lt 03
Percentage of Nurses Responding ldquoNearly Alwaysrdquo or ldquoUsuallyrdquo in Response to the Question ldquoI know which resident or pager number to page when I need physician input in care of my patientsrdquo 2009 and 2011
Online-Only Content8
Figure 2 Nursesrsquo responses show their level of certainty about which physician to page in 2009 and again in 2011 after the implementation of the newpaging model CT cardiothoracic GI gastrointestinal Probability (p) levels for chi-square tests are shown an asterisk indicates statistical significance
Copyright 2014 copy The Joint Commission