eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld eHealth: Innovations and

IssuesLecture 9

Computerized Clinical Decision Support

Norm Archer

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Agenda

• Definitions• Medical procedure for patients• History of medical DSS• Consumer DSS• Clinical DSS• Treatments for diseases & conditions• Example use of DSS• Excerpts from Andre Kushniruk

presentation on usability testing of DSS

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Definitions• Decision support system

A computer system designed to assist users in decision-making activities.

• Decision support systems in healthcare Management DSS - Computer technologies which

allow providers to collect and analyze data in more sophisticated and complex ways. Activities supported include case mix, budgeting, cost accounting, clinical protocols and pathways, outcomes, and actuarial analysis.

Clinical (CDSS) - form a significant part of the field of clinical knowledge management technologies through their capacity to support the clinical process and use of knowledge, from diagnosis and investigation through treatment and long-term care.

Consumer DSS - help consumers become more aware and responsible for their own healthcare through a patient-centred approach, while cooperating with medical professionals who make informed treatment decisions.

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Definitions…

• Disease A disease is an abnormal condition of the body or

mind that causes discomfort, dysfunction, or distress to the person afflicted or those in contact with the person. Sometimes the term is used broadly to include injuries, disabilities, syndromes, symptoms, deviant behaviours, and atypical variations of structure and function, while in other contexts these may be considered distinguishable categories.

• Differential diagnosis Determination of which one of two or more

diseases or conditions with similar symptoms is the one from which the patient is suffering.

Correct diagnosis leads to the choice of the most effective treatments that will alleviate the disease or condition

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Medical Procedure for Patients

• 1. Ill patient has appointment with physician, or visit to emergency room or clinic

• 2. Physician takes patient history, symptoms, examines patient

• 3. Physician Makes preliminary determination of disease

• Writes prescription for medication to combat the illness, or

• Refers patient to specialist, or• Orders medical tests

If patient sent to specialist - orders additional tests and either reports back to physician or takes patient under care for treatment or surgery

If further tests ordered or specialist reports back, physician makes determination of illness and writes prescription for treatment

• Best practice information very helpful for physician or specialist in treating disease after it has been identified

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

History of Medical DSS

• See http://www.openclinical.org/dss.html

• Early history• 1970s and 80s, development of rule-

based expert systems – INTERNIST, MYCIN, PIP, ONCOCIN

• Some were commercialized as DSS E.g. DXPlain and QMR Demo of DXPlain at

http://www.openclinical.org/dm_dxplain.html

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Consumer DSS (Eysenbach

2000)

• Using self-analysis decision support tools, patients may be able to attain a healthy balance between self reliance and seeking professional help, by balancing of the need for face to face interaction with provision of virtual interaction.

• Focus group evaluations of HouseCall (for example) have shown that the program is easy to use and that consumers like using technology at home to investigate health issues and like participating in solving their medical problems.

• Obviously, such systems “do not and cannot replace visits with physicians; they can, however, make such encounters more productive, for both doctor and patient.” They may also help to triage patients. For consumers, the aim of such support systems would not be to make definitive diagnoses or to propose treatment but to answer simple questions such as “do I need to see a doctor?” or to alert patients to potential drug interactions or other health risks.

• The main challenge in developing comprehensive systems for consumers is that little is known about how patients interact with computer based informatics tools and how they digest and act on information.

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Online Consumer DSS• General

YourDiagnosis (U.S. $12.50 to $15.00) http://www.yourdiagnosis.com/start.htm

The Analyst (U.S. $25.00 to $77.00) http://www.diagnose-me.com/?page=main

MyElectronicMD (free) http://myelectronicmd.com/step1.php

EasyDiagnosis (Subscription cost) http://easydiagnosis.com/

• Cancer NexProfiler (free, operated by NexCura)

http://www.cancerfacts.com/DefaultSecure.asp• Cardiology

NexProfiler (free, operated by NexCura) http://health.discovery.com/jump/nexcura/heart_profiler.html

• Other diseases HealthCommunities (free, operated by NexCura)

http://www.healthcommunities.com/

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Clinical DSS for Physicians• Medexpert/WWW is a medical

knowledge base server – links to many online DSS http://medexpert.imc.akh-wien.ac.at/start.html

• Example: MedTechUSA (annual $1050 U.S. subscription) medical reference tools for a variety of diseases and conditions (includes handheld device versions) http://www.medicalamazon.com/

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Treatments for Diseases and Conditions

• List of diseases and treatments http://www.surgerydoor.co.uk/m

edical_conditions/

• Information on diseases and treatments http://www.mayoclinic.com/

• And many others

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Example 1• Symptoms

Tingling in upper right arm • At irregular intervals, when walking, computer work,

driving, etc. • Especially when looking upwards (e.g. changing a light

bulb in the ceiling)• Tingling stops when right hand is placed at back of neck in

stretching motion • Problem began August 2005 while swimming• No or little loss of strength and use of right arm

• Patient chart and history Gender Age Pre-existing health problems – episode of carpal tunnel

syndrome about 1990 (cleared up over a one year period after ergonomic adjustment of computer work environment)

Current medications Life style (exercise, diet) Psychological and physical (work, home) Medical test results from EMG (electromyography)

compared right & left arms.

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Example 2• Symptoms

Episodes or sudden onsets for 10 to 30 minutes of: • Dizziness• Sweating• Blurred vision• Excessive saliva• Déjà vu • Episodes have been occurring irregularly for several

years• Patient chart and history

Gender Age Parental health histories Pre-existing health problems Mental condition Current medications Life style (exercise, diet) Psychological and physical (work, home) Recent visits to foreign countries Medical test results

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Excerpts from “From Laboratory Usability

Testing to ‘Televaluation’ of

Web-based Information Systems”

2003 Presentation byAndré W. Kushniruk, Ph.D.

Director, Information Technology Program, Faculty of Arts

York University, Toronto, Ontario, Canada

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Needs-satisfaction curve of information technology

High Technology Human-ComputerInteraction

Unfilled Technology Need

Technology is “good enough”User Experience Dominates

Excess Functionality

Transition Point: Technology delivers basic need

Performance Required by Typical User

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Motivation – HCI Issues in Health

Informatics• Problems with information systems in health care Lack of acceptance of systems Poor usability Failure to support work practices Introduction of errors Inadvertent changes in workflow

• Issues related to human-computer interaction (individual and group/social level) may be single-most important barrier to successful implementation of systems in health care

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Evaluation in Health Informatics

• Summative Evaluation - need for assessment of whether systems meet the needs of users, are safe and effective

• Formative Evaluation – need for assessment of systems throughout their development

Traditional development approach – classic waterfall development cycle

Newer approach – rapid prototyping involving continued user input and testing

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

From Laboratory to Real-worldAnalysis and Evaluation

(Kushniruk,2001)

LABORATORY

- Fixed usability lab- Experimental tasks - “think aloud” - cognitive task analysis

- Simulations- E.g. “simulated” doctor-patient interviews

NATURALISTIC

-”Virtual” usability lab- Analysis of Web- based systems- Data mining

A Continuum of Studies

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

A Continuum of Approaches to Evaluation Along the SDLC

1. Planning (needs analysis)

-workflow analysis-job analysis-analysis of decision making-interviews

2. Analysis(requirements)

-interviews -questionnaires -focus groups -video analysis -cognitive task analysis

3. Design 4. Implementation(programming)

5. Support (maintenance)

Figure 1. The systems development life cycle (SDLC) in relation to evaluation methodologies.

-usability testing-usability inspection-design walk- throughs

-usability testing-code inspections-software unit testing

-outcome-based evaluations-randomized clinicaltrials-summative evaluations

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

•Study 2: Study of use in diabetes clinic over six month period – naturalistic

approach

•Interviews (pre and post) – 16 clinic staff

•Usability testing with subset of subjects

•Training recorded as well

•Logging of all system use

•Study of contents of paper and computer records

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

More irrelevant information in paper records Overall less information recorded in computer

based records• For corresponding records, CPR version contained

25% less information. Fewer diagnoses recorded in CPR for matched

records - typically only single primary diagnosis Change in reasoning -- from “hypothesis

driven” to “screen driven”

Study 2 Results…

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Changes in Reasoning

Data-directed (paper records)

Problem-directed (CPR)

Problem-directed (paper records)

Lasting change in reasoning patterns, even when CPR removed (“effects of” and “effects with”)

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Patient Data Multiple Hypotheses

Diagnostic Reasoning Using CPRPatient Data Hypotheses

Diagnostic Reasoning Using

Paper Record

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

10 fold decrease in usability problems

• Use of same method as above for improving prototype CPR system

• 9 subjects video recorded using the system• Number of problems and type of problems

identified from coding scheme• System refined based on the testing• 9 new subjects run• Found:

• Dramatic decrease in number of errors:– From 19 per session on average in first testing to

1.9 in second testing!

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Usability and the WWW(Kushniruk et al., 2001)

• Objective to adapt usability testing to the WWW

• how are people using health care sites? • Do they get information they want from

particular sites?• what problems do they have?• How are Web-based guidelines used?

• Remote tracking of Web users• Remote video-based usability testing

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Evaluation of Usability of Web-Based Health

Care Information Systems

• Varied users who interact from various locations

Less able to conduct controlled evaluative studies

• Current state-of-the-art track user actions (e.g. clicks) - tells what they do,

but not why on-line questionnaires/feedback forms - often not

filled in, limited questions interviews - problem that users often do not know

what they do

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Questions in the Evaluation of e-

Health Information Systems • What type of information do e-Health

consumers want?• Is the information provided useful,

helpful?• How to collect useful data from large

number of subjects remotely?• How to integrate data from multiple

sources?• How to analyze such data from varied

data sources to discover usage patterns?

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

1. Video BasedUsability Testing - from laboratory to remote

2. Interviews - from phone to electronic

UserInteract via WWW

3. E-mail(to evaluators)

Consumer Information System

4. Tracking User Actions - System Usage Database (log files)

5. On-line Questionnaire Data (triggered forms)

Remote evaluation of System Usage

Kushniruk, Patel, Patel, & Cimino, 2001

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Example: Evaluation of a Patient Clinical Information System

(PatCIS)• Over the WWW patients can

Review their own medical data (e.g. laboratory results

Enter their data (e.g. blood glucose levels) Receive advice Receive educational information

• Subjects recruited from private practices in New York state

• Followed over one year• Thousands of accesses

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Screen of a patient clinical information system (PatCIS) showing data review function

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Evaluation Questions

• What features of such systems are most used by patients, Why?

• What features are least used and why?• Are there usability issues that need to be

resolved?• How does use of such systems affect the

doctor-patient interaction?• Can patients comprehend information

presented?• Does use of these types of systems affect

decision making and disease management?

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Results

• Function Usage Most frequently accessed function

was “Review of Laboratory Data”oAccessed by patients at least once

in the majority of the sessions “Review of Reports” was second

most frequently accessed function Other functions (advice, education

and data entry) were used sparingly

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Analysis of User-System Interactions

Function Usage (number & percent of accesses)

ADVICE DATA ENTRY DATA REVIEW EDUCATION TOTAL

.3% 4% 93% 3% 100%

Majority of accesses by patients for Data Review• Laboratory details• Reports – admit/discharge, cardiology, radiology

• Discovery of patterns of usage related to both • demographic and medical data

• Most used and useful for patients with specific illnesses – chronic illness (e.g. diabetes)

• Patients liked the system since they felt greater ownership• Physicians liked the system as it streamlined their limited

face-to-face visits with patients (patients had often reviewed their data prior to the interview)

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Excerpts from interviews with Patients

“ Communication is less in the way of getting information now, and more in the way of discussing treatment options and agreeing on a course of action, so to me its more efficient than the old way”

“ I look for trends in my medical data and if I see something I can contact the doctor to see what’s going on, what we can do, change meds or whatever”

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Excerpts from Interviews with Physicians

“ Right now most of the communication takes place during the ten or fifteen minute visit and if I throw a lot of information at the patient about their condition or what I want them to do, its very hard for them to absorb all that.

It (PatCIS) gives them a chance to go back and look at things about their health record that they can then ask better questions about in the limited time that we have during the visit. Its another channel of communication”

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Figure 5a. Resource page showing links to clinical guidelinesavailable from within a computer-based patient record system.

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Figure 5b. Form to assess clinician’s reason for accessing a guideline(which appears when the user selects a guideline from the resource page).

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Implications

• Analysis of results led to guideline designers to modify

• the format and amount of information contained in guidelines

• Now includes text OR easy to read graphical representations (maps)

• Studying impact of changes

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Summary

• Need for range of approaches for assessing HCI / usability

• Usability is critical to success of health care information systems

• Usability will be recognized as a major success and marketing factor

• Consumer expectations for usability will increase

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

Future Directions

• Extension of methods for qualitative coding of data

• Development of newer portable and “virtual” approaches

• Usability engineering methods applied throughout system development• User needs analysis (before system

development)• During requirements gathering,

design, implementation• In general, the earlier the better

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

References• Eysenbach, G. (2000). Recent advances:

Consumer health informatics. British Medical Journal, 320, 1713-1716.

• Kushniruk, A. W. (2003). Human-computer interaction in health informatics: From laboratory usability testing to "televaluation" of Web-based information systems (pp. 57 - Powerpoint Presentation). Toronto, ON: York University.

• Schwitzer, G. (2002). A review of features in Internet consumer health decision-support tools. Journal of Medical Internet Research, 4(2), e11.

eBu

sin

ess

Kn

owle

dge

for

a C

han

gin

g W

orld

eHealth: Innovations and Issues

End Lecture 9

Computerized Clinical Decision Support

Norm Archer, Ph.D.archer@mcmaster.c

Ext. 23944