electronic interface for emergency department management of asthma: a randomized control trial of...

ORIGINAL RESEARCH

Electronic interface for emergency departmentmanagement of asthma: A randomized controltrial of clinician performanceBen Taylor,1 Michael Dinh,2 Raymond Kwok,2 David Dinh,3 Matthew Chu4 and Eric Tang5

1Emergency Department, St Thomas Hospital, London, UK; and 2Emergency Department, Royal PrinceAlfred Hospital, 3OsTechnology, 4Emergency Department, Canterbury Hospital, 5Clinical School,Concord Hospital, Faculty of Medicine, Sydney University, Sydney, New South Wales, Australia

Abstract

Objectives: To evaluate the effectiveness of an integrated and dynamic electronic decision supportsystem for management of acute asthma in the ED.

Methods: A randomized trial was conducted comparing clinician performance using this electronicinterface compared with paper documentation in a simulation scenario. The outcomes weredocumentation of asthma-related information and consultation times.

Results: Use of this electronic interface was associated with significantly higher rates of documen-tation in 7 out of 10 variables, including provision of written short-term asthma manage-ment plans. After adjustment for participant seniority, there was no significant differencein consultation times.

Conclusion: In a simulation trial, use of this electronic interface was associated with improvements inclinical and discharge documentation. Further studies are required to test this prototype inclinical practice.

Key words: asthma, electronic interface, emergency department.

Introduction

The implementation of electronic health records withdecision support functionalities is regarded as animportant goal for future sustainable health systems.1

The potential benefits of real-time data collection,quality control and process redesign have been

advocated, but there are few evaluation studies in theED setting.

Our group has previously reported on the develop-ment of a prototype application for management ofacute asthma in the ED.2 This system incorporated deci-sion support, order entry and results reporting anddocument management into a seamless interface. This

Correspondence: Dr Michael Dinh, Emergency Department, Royal Prince Alfred Hospital, Missenden Road, Camperdown, NSW 2050,Australia. Email: [email protected]

Ben Taylor, MBChB, Registrar; Michael Dinh, MB BS, FACEM, Staff Specialist; Raymond Kwok, MB BS, Registrar; David Dinh, BApp. Fin, LLB,Managing Director; Matthew Chu, FACEM, Director of Emergency Department; Eric Tang, MB BS, Medical Student.

doi: 10.1111/j.1742-6723.2007.01040.xEmergency Medicine Australasia (2008) 20, 38–44

© 2007 The AuthorsJournal compilation © 2007 Australasian College for Emergency Medicine and Australasian Society for Emergency Medicine

mailto:[email protected]

was called asthma clinical assessment form and elec-tronic decision support (ACAFE). System developmentwas motivated by the desire to address key process-related problems specific to the ED. This included:• Multiple and disparate data entry points resulting in

duplication of documentation tasks, for example,clinical history, management plan and dischargesummary entries

• Inconsistent use of clinical pathways in the ED result-ing in variability in documentation quality andpatient management

• Lack of real-time quality control mechanisms.The objective of the present study was to compare

this system with paper documentation in a randomizedcontrol trial of scenario-based clinician performance.We aimed to examine whether there was a difference inclinical documentation and consultation times usingACAFE compared with paper-based consultations.

Method

Setting

This was a simulation trial conducted at three inner cityhospital ED (Royal Prince Alfred Hospital, The Canter-bury Hospital and Concord Hospital, Sydney, Austra-lia). All departments were accredited for emergencymedicine training by the Australasian College for Emer-gency Medicine.

Design

The present study was a randomized control trial of EDasthma documentation using either electronic interface(EI) or paper documentation (PD), using a simulatedpatient and standard scenario.

Study population

All ED doctors during the present study period wereinvited to participate. This included emergency physi-cians, emergency medicine trainees (registrars), residentmedical officers (including interns) and senior residentmedical officers. Participants were given a short patientvignette just prior to the simulated consultation. Clini-cians were not informed about study objectives or out-comes. The present study was conducted towards theend of resident rotations, so that all doctors were famil-iar with respective departments.

Study period

The present study was conducted over a 4 month periodfrom December 2006 to March 2007.

ACAFE interface

Asthma clinical assessment form and electronic deci-sion support was a 1 year collaborative innovationproject between emergency physicians and health infor-mation specialists.2 The objective was to develop sus-tainable solutions to clinical guideline compliance andquality control in the ED. The result was a computer-ized online point of care system designed by emergencyphysicians that seamlessly integrated all data entrypoints for a given patient encounter. This includedtriage and registration, clinical documentation, treat-ment orders, order entry and discharge documentation.The system integrated asthma management pathwaysbased on current guidelines into clinical and dischargedocumentation. Thus ACAFE was an example ofdynamic and integrated electronic decision supportsystem (EDSS). Key features were:• Decision support for asthma severity and suggested

management plans based on National AsthmaCouncil 2002 guidelines3

• Automated document generation for discharge plans,asthma management plans, smoking cessation adviceand performance summaries (waiting times, consulta-tion and admission times)

• Field alert modules, for example, allergy alerts,smoking cessation advice, severe asthma alerts

• Real-time reporting of performance variation.The system was designed for use at triage and point

of care. Clinical assessment forms were structured intointuitive history, examination and diagnosis formats.Data entry in specific clinical fields prompted decisionsupports and generated automated discharge or admis-sion summaries and management plan suggestionsincluding treatment orders and investigations. Thesecould then be confirmed by the clinician or altered.The system was server-based (Windows Server 2003Edition) and accessed via local hospital intranet.ACAFE was also designed to comply with currentHealth Level 7 standards relating to semantic and func-tional interoperability and National E Health TransitionAuthority of Australia standards for system develop-ment and data security. Other technical specificationsand hardware requirements of the system can beviewed at: http://www.ostechnology.com.au/acafe-clinical-decision-support.

Electronic interface for management of asthma

39© 2007 The AuthorsJournal compilation © 2007 Australasian College for Emergency Medicine and Australasian Society for Emergency Medicine

http://www.ostechnology.com.au/acafe-clinical-decision-support



A live online test version can be accessed at: http://acafe.msdrg.org (user name and password ‘doctor’).

Intervention

After obtaining written consent, participants were ran-domized to either PD or EI groups. Group assignmentswere documented in sealed envelopes, randomlyselected and provided to participants. Paper documen-tation consisted of paper-based clinical records, treat-ment order sheets and discharge documentation(including asthma clinical pathways, asthma plans,quit-smoking advice sheets). The electronic interfaceconsisted of ACAFE interface installed on desktopor laptop computers available in the consultationroom. Those in EI group were given a brief standard2 min introduction to the system, including basic func-tions of the program prior to commencement of thescenario.

Case scenario

The scenarios took place in respective departments in adesignated consultation room with all necessary equip-ment and documentation forms provided (clinical notes,discharge letters, clinical pathways, asthma manage-ment plans and smoking cessation brochures). Partici-pants were given the following vignette:

A 29 year old male presents today with shortness ofbreath and wheeze. He has a history of asthma. He wasassigned a triage category 3 and placed in a monitoredbed. Please see this patient and complete the necessaryassessment and management. You are required to fill inall the documentation you think is necessary to com-plete the consultation.

The simulated patient was a trained volunteer whowas instructed to provide an assessment consistentwith mild acute asthma according to current NationalAsthma Council guidelines.3 The same vignette andtraining was provided to all volunteers who receivedno inducements or incentives. The patient was acurrent cigarette smoker who ran out of his usualasthma puffers and presented to the ED. He hadnormal vital signs and mental state. He responded wellafter initial bronchodilator treatment, and was wellenough for discharge without further monitoring.Mild asthma was chosen as this was deemed toresult in a simple controlled consultation resulting indischarge.

Outcomes

The primary outcome was quality of asthma documen-tation – this was measured using 10 documentationvariables which were scored yes or no depending onpresence or absence in documented medical notes.These were classified into clinical parameters and dis-charge documentation (see Table 1), and were derivedfrom National Asthma Council guidelines (2002) andprevious studies evaluating asthma management in theED.3,4 Documentation of asthma management plan wasdefined as the provision of an individualized writtenshort-term asthma management plan. Smoking cessa-tion advice documentation was defined as the presenceof at least an entry in the medical notes or managementplan. For example, acceptable terms included ‘quitsmoking’ or ‘smoking cessation’.

The secondary outcome was consultation time. Thiswas defined as the time from commencement of thescenario until the time of completion of consultation.This included any time required to brief the participantand introduce the electronic platform to participantsrandomized to EI group. Thirty minutes was added tothe consultation time if investigations (laboratory testsor imaging studies) were requested as part of patientassessment. This was based on consensus regardingtime requirements for ordering and following up inves-tigations. No further time was added if multiple inves-tigations were ordered. Any observation time requestedfor observing treatment effect was also not included.

Data collection and analysis

Study investigators (BT, RK) assessed consultationsduring the scenario. This was limited to measuring

Table 1. Primary outcomes

Category Indicator

History Asthma precipitating factorsPrevious intensive care admissions

Examination Oxygen saturationsAbility to verbalize (words, phrases,

sentences)Chest auscultation findingsPeak expiratory flow (before and after

bronchodilator)Asthma severity (mild, moderate, severe)

Discharge Written asthma plan providedSmoking cessation adviceOral corticosteroid prescription

B Taylor et al.

40 © 2007 The AuthorsJournal compilation © 2007 Australasian College for Emergency Medicine and Australasian Society for Emergency Medicine

http://acafe.msdrg.org

http://acafe.msdrg.org

consultation times and a brief introduction to the system.Following the scenario, documentation generated fromall consultations was stored in a secure file. Consultationnotes generated using ACAFE were printed immediatelyafter and stored in the same fashion. Two independentdata abstractors not related to the present study andblinded to study outcomes then assessed documentationrelating to asthma consultation outcomes. A standarddata abstraction sheet was used indicating the 10documentation variables. Interrelater agreement wasdetermined and differences were assessed by paneladjudication with both data abstractors.

Data were stored in Excel format and analysed usingSPSS Version 12.1 (SPSS, Chicago, IL, USA). Values forproportions and continuous variables were expressedusing 95% confidence intervals (CI) and medians withinterquartile range (Q25-75), respectively. Categorical datawere compared for significance using c2-tests, and con-tinuous variables compared using Mann–WhitneyU-tests. Significance was set at P-values < 0.05. Becauserandomization was not stratified by seniority, multi-variate analysis was performed using a general linearmodel with consultation time or documentation out-comes as dependent variables, and EI or PD as fixedindependent variables, adjusting for seniority as therandom factor. Kappa values were used to assess inter-rater agreement. Based on previous studies of asthmaguideline implementation, investigators predicted thatkey asthma outcomes would be documented in 90% EIgroup compared with 50% in the control group. Assum-ing a two-tailed alpha value of 0.05 and a power of 0.80,we predicted that an evenly balanced sample of 50participants would be required.

Ethics

Approval from the Sydney South-west Area Health Ser-vices (Central Zone) Human Research Ethics Committeewas gained prior to commencement of the presentstudy.

Results

Study population

There were 50 ED doctors enrolled. Seniority rangedfrom emergency physicians to resident medical officers.Median age was 30 years (26–33 years). Forty-eightper cent of participants were male. Twenty-seven

participants were randomized to PD group and 23 in EIgroup. Both groups were similar with respect to age, sexand overall seniority.

Primary outcome

Electronic interface group was associated with signi-ficantly higher rate of documentation in 7 out of 10outcome variables. This included documentation ofpeak flow, ability to verbalize, asthma severity,smoking cessation advice, asthma management plansand documentation of oral corticosteroid prescription(See Table 2). There was no significant difference indocumentation of precipitating factors, previous ICU oroxygen saturations. Significance did not change withadjustment for seniority (adjusted R2 = 0.53, F = 56.0,P < 0.001). Overall interrelater agreement was 95%(kappa value = 0.86, 95% CI 0.80–0.92).

Consultation times

Median consultation time for the present study groupswere 43 min in PD group and 31 min in EI group(P = 0.04), and overall distributions are shown inFigure 1. There was no significant difference in consul-tation time after adjustment for seniority (adjustedR2 = 0.86, F = 4.0, P = 0.10). Investigations were orderedin 9/27 (33%) in PD group compared with 2/23 (9%) inEI group (P = 0.04). This might have also accounted forconsultation time differences.

Discussion

Asthma clinical assessment form and electronic deci-sion support was developed as a prototype onlinesystem combining EDSS, clinical pathways, order entryand document management. The purpose of the presentstudy was to assess its potential effectiveness. Thepresent study showed that use of this system was asso-ciated with improvements in asthma documentation insimulated scenarios. The EI group had significantlyhigher rates of documentation in 7 out of 10 documen-tation variables. This included documentation and pro-vision of short-term asthma discharge managementplans and smoking cessation advice. There was no sig-nificant difference in consultation time between studygroups after adjustment for seniority.

The management of acute asthma patients, ED dis-charge practice in particular, has been identified by theAustralian Department of Health and Ageing as apriority area for quality improvement.4 The Asthma



Table 2. Study groups and outcomes, medians expressed with IQR and proportions expressed with 95% CI

Category Study group

PD % (95% CI)or (IQR)

EI % (95% CI)or (IQR)

P-value

N 27 23Age Median age in years 29 (26–32) 30 (28–35) 0.11Sex Male (%) 14 52% (33–69) 10 43% (26–63) 0.55Seniority† Resident (%) 12 44% (27–63) 5 22% (10–42)

Senior resident (%) 5 19% (8–37) 6 26% (13–47)Registrar (%) 7 26% (13–45) 10 43% (25–63)Emergency physician (%) 3 11% (4–28) 2 9% (2–27) 0.57

OutcomesDocumentation Precipitating factors 26 96% (82–99) 23 100% (86–100) 0.35

Previous intensive care admissions 16 59% (41–75) 23 100% (86–100) 0.01*Oxygen saturations 22 81% (63–92) 21 91% (73–98) 0.32Chest auscultation 26 96% (82–99) 23 100% (86–100) 0.35Peak expiratory flow 14 52% (34–69) 19 82% (63–93) 0.02*Ability to verbalize 16 59% (41–75) 22 95% (79–99) 0.03*Asthma severity 17 63% (44–78) 23 100% (86–100) <0.01*Smoking cessation advice 8 29% (16–48) 22 95% (79–99) <0.01*Asthma management plan 15 55% (37–72) 23 100% (86–100) <0.01*Oral corticosteroid prescription 16 59% (41–75) 20 87% (68–95) 0.03*

Consultation times Median time in minutes 43 (30–70) 31 (29–40) 0.04*

*Significant value. †Resident refers to resident medical officer postgraduate years 1–2; senior resident refers to postgraduate years 3and over; and registrar refers to emergency medicine trainees. CI, confidence intervals; EI, electronic interface; IQR, interquartile range;PD, paper documentation.

120100806040200

Time

10

8

6

4

2

0

Fre

qu

ency

10

8

6

4

2

0

01

Gro

up

Figure 1. Distribution of consultation times (in minutes) between electronic interface (EI) (group 1) and paper documentation (PD)(group 0) (unadjusted for seniority).

B Taylor et al.


Discharge Management in Relation to EmergencyDepartments project identified key barriers to best prac-tice in asthma management.5 Such barriers includedpoor ED documentation and ED staff turnover.

These findings were confirmed by studies showingthat ED documentation of asthma management contin-ues to be suboptimal.6 Standardized written clinicalpathways based on national guidelines have been imple-mented in many ED, and have been shown to be aneffective way of improving asthma management.7

Written pathways, however, have their limitations inthe context of busy clinical environments and constantstaff rotations. Unless copies are readily available andnew staff reminded of their use, they are often underuti-lized. A number of studies have identified the potentialrole of electronic health records in clinical medicine andin ED.8–15 However, implementation trials involvingEDSS have, to date, yielded conflicting results.16 Suchinvestigators have identified lack of EDSS and clinicalpathway integration into workflow as major barriers touptake of such technologies.

We propose the use of dynamic and integrated EDSSsystems, such as ACAFE, as sustainable alternatives tocurrent clinical guideline implementation methods anddocumentation quality control. ACAFE was designedto integrate clinical pathways into current workflow,improve and streamline clinical documentation (e.g.automated discharge summaries and asthma manage-ment plans) and introduce real-time quality controlmechanisms into increasingly complex ED environ-ments. Therefore, ACAFE addresses some of the keybarriers to guideline implementation and is an impor-tant step in translational research in emergency medi-cine. Several other studies have investigated theimplementation of clinical pathways into electronichealth records; however, research in the ED context iscurrently limited.17,18

Improving workflow efficiency was an importantobjective of ACAFE. Increased data entry complexity atthe expense of actual patient care time is not desirable.We used consultation time as a marker of potentialworkflow efficiency. The results of the present studysuggested that consultation times were not differentwhen using an integrated EDSS compared with paperdocumentation. Confounding factors included partici-pant seniority and rate of investigations ordered. Thepresent study was not designed to investigate theeffect of differences in seniority on consultation time,and the potential effect of clinical seniority in relationto computerized systems warrants further exploration.Furthermore, workflow efficiency in the ED is complex

and consultation times might not be the most appropri-ate means of measuring this. Process optimizationthrough clinical pathways and quality control throughreductions in performance variation might be morerelevant.19

Limitations to the present study include small samplesize and simulated clinical scenarios. For the presentstudy, a clinical scenario was set up as realistically aspossible to how it would be in the ED; however, it wasnot tested on actual patients in real time. It is possiblethat differences noted through a simulation trial mightnot translate into clinically relevant differences in actualpractice. The system therefore needs to be validated inimplementation trials in actual clinical practice. Theideal system would be interoperable within the hospitalinformation system framework as well as facilitateelectronic access of discharge summaries to generalpractitioners.

Finally, it has yet to be proven in controlled trialswhether adherence to current asthma guidelines resultsin better patient outcomes. For example, it might beargued that improved documentation and provision ofasthma management plans are not clinically relevantoutcomes compared with asthma mortality and repre-sentation rate. Improved quality in health-care deliveryis multidimensional and not necessarily restricted toimprovements in mortality. Despite this, best practiceand translational research will remain an important pri-ority for health systems around the world.20

In conclusion, we found that the use of a dynamic andintegrated EDSS for asthma management was associ-ated with improved clinical documentation and patientdischarge information in a simulation trial. Such systemsmight provide sustainable solutions to future guidelineimplementation projects. Further studies are required toassess system effectiveness in actual practice.

Acknowledgements

We gratefully acknowledge Mr An Tran from Informa-tion Technology Services, The Canterbury Hospital forhis assistance with system implementation. We thankDr Kong Liew and Dr Wynne Sum for data abstractionand manuscript editing, and Dr Kai Zhang from theNSW Clinical Excellence Commission for data analysis.

Competing interest

None declared.

Accepted 20 September 2007



References

1. Hillestad R, Bigelow J, Bower A et al. Can electronic medicalrecord systems transform health care? Potential health benefits,savings and costs. Health Aff. 2005; 5: 1103–17.

2. Dinh M, Chu M, Kwok R, Taylor B, Dinh D. The asthma clinicalassessment form and electronic decision support (ACAFE)project: collaborative innovation, development. In: Morgan T, ed.Medinfo 2007. Proceedings of the 12th International HealthInformatics Congress. August 20–27th 2007, Brisbane Australia.Amsterdam: IOS Press, 2007; 435.

3. National Asthma Council Australia. Asthma Management Hand-book 2002. Melbourne: National Asthma Council, 2002.

4. Grimmer K, Peoples C, Steele E, Kumar S. Review of Best PracticeHospital Discharge Practices for Asthma Patients. Adelaide: Uni-versity of South Australia, 2003. [Cited April 2007] Availablefrom URL: http://www.wh.org.au/jecemr/asthma.pdf

5. Kelly AM, Clooney M. Improving asthma discharge managementin relation to emergency departments. The ADMIRE project.Emerg. Med. Australas. 2007; 19: 59–62.

6. Krym VF, Crawford B, MacDonald RD. Compliance with guide-lines for emergency management of asthma in adults: experienceat a tertiary care teaching hospital. CJEM 2004; 6: 321–6.

7. Doherty SR, Jones PD, Davis L et al. Evidence based implemen-tation of adult asthma based guidelines in the emergency depart-ment: a controlled trial. Emerg. Med. Australas. 2007; 19: 31–8.

8. Lee FCY, Chong WF, Ooi SBS. The emergency medicine depart-ment system: a study of the effects of computerization on thequality of medical records. Eur. J. Emerg. Med. 2001; 8: 107–9.

9. Schootman M, Zwerling C, Miller ER et al. Method to electroni-cally collect emergency department data. Ann. Emerg. Med.1996; 28: 213–19.

10. Hu S, Yen DH, Kao WF. The feasibility of full computerization inthe ED. Am. J. Emerg. Med. 2002; 20: 118–21.

11. Day F, Hoang LP, Ouk S, Nagada S, Schriger DL. The impact ofa guideline-driven computer charting system on the emergencydepartment care of patients with acute low back pain. Proc.Annu. Symp. Comp. App. Med. Care 1995, 567–80.

12. Cordell WH, Overhage JM, Waeckerle JF, for the InformationManagement Work Group. Strategies for improving informationmanagement in emergency medicine to meet clinical researchand administrative needs. Ann. Emerg. Med. 1998; 31: 172–8.

13. Stair TO. Reduction of redundant laboratory orders by access tocomputerized patient records. J. Emerg. Med. 1998; 16: 895–7.

14. Schenkel S. Promoting patient safety and preventing medicalerror in emergency departments. Acad. Emerg. Med. 2000; 7:1204–22.

15. Bizovi KE, Beckley BE, McDade MC et al. The effect of computer-assisted prescription writing on emergency department prescrip-tion errors. Acad. Emerg. Med. 2002; 9: 1168–75.

16. Garg AX, Adhikari NK, McDonald H et al. Effects of computer-ized clinical decision support systems on practitioner perfor-mance and patient outcomes. A systematic review. JAMA 2005;293: 1223–38.

17. Mikulich VJ, Liu YA, Steinfeldt J, Schriger D. Implementation ofclinical guidelines through an electronic medical record: physi-cian usage, assessment and satisfaction. Int. J. Med. Inform. 2001;63: 169–78.

18. Bossen C. Evaluation of a computerized problem orientatedmedical record in a hospital department: does it support dailyclinical practice? Int. J. Med. Inform. 2007; 76: 592–600.

19. Dinh MM, Chu M. Evolution of health information managementand information technology in emergency medicine. Emerg. Med.Australas. 2006; 18: 289–94.

20. Lang ES, Wyer PC, Haynes RB. Knowledge translation: closingthe evidence practice gap. Ann. Emerg. Med. 2007; 49: 355–63.

B Taylor et al.


http://www.wh.org.au/jecemr/asthma.pdf

electronic interface for emergency department management of asthma: a randomized control trial of...

Documents