applying usability and ucd methodologies to achieve meaningful use
DESCRIPTION
Overview of the role of usability and user-centered design methodologies in the context of ONC meaningful use certification criteria.TRANSCRIPT
Applying Usability and User-Centered Design Methodologies to Achieve Meaningful Use
June 9, 2010, HIMSS Virtual Conference
Lisa Battle, Jasmin Phua & Duane Degler Design for Context
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 2
Conflict of Interest Disclosure
Lisa Battle User-Centered Design Lead
Jasmin Phua User Experience Researcher & Designer HIMSS EHR Usability Taskforce member
Duane Degler User-Centered Design Strategist & Semantic Web Expert
Consult for and have no real or apparent conflicts of interest to report.
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 3
Agenda
• Meaningful Use Overview
• Role of Usability in EHR Meaningful Use
• Build-in Usability through User-Centered Processes
• What is user-centered design
• Evaluating product usability
• Designing for usability
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 4
Objectives
This education session aims to help you:
• Understand the role of usability in helping achieve meaningful use
• Identify methods of evaluating usability
• Apply user-centered design methodologies to incorporate end-user feedback
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 5
“It’s not about technology,
it’s about transforming healthcare.”
Joshua Seidman, Acting Director, Meaningful Use Office of Provider Adoption Support, ONC
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 6
What is all this Meaningful Use buzz?
Through the provisions of the American Recovery and Reinvestment Act (ARRA; Recovery Act) of 2009, the Centers for Medicare & Medicaid Services (CMS) will provide incentive payments for the meaningful use of certified electronic health record (EHR) technology.
Source: Medicare and Medicaid Programs; Electronic Health Record Incentive Program; Proposed Rule. January 13, 2010
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 7
Who qualifies?
Eligible professionals (EPs), eligible hospitals, critical access hospitals (CAHs) for:
• Efforts to adopt, implement, or upgrade certified EHR technology,
or
• Meaningful use in first year of participation and for demonstrating meaningful use during each of 5 subsequent years.
Source: Medicare and Medicaid Programs; Electronic Health Record Incentive Program; Proposed Rule. January 13, 2010
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 8
How is Meaningful Use measured?
• Stage 1 has been defined; meaningful use objectives and certification criteria provided by CMS and ONC.
• Stages 2 & 3 have not been define yet but policy priorities have been painted in broad strokes.
• Different measures for eligible professionals (EPs) vs. hospitals.
Sources: Medicare and Medicaid Programs; Electronic Health Record Incentive Program; Proposed Rule. January 13, 2010. Health Information Technology: Initial Set of Standards, Implementation Specifications, and Certification Criteria for Electronic Health Record Technology; Interim final Rule. January 12, 2010.
Stage 1 Stage 2 Stage 3
2011 2016 incremental process
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 9
Meaningful Use: Stage One
Stage One (2011-2012) focuses on: • Electronically capturing health information in a coded format, • Using that information to track key clinical conditions, • Communicating that information for care coordination purposes, • Implementing clinical decision support tools to facilitate disease and
medication management, • Reporting clinical quality measures and public health information.
Defined measures for: • Eligible professionals (EPs): 25 measures • Eligible hospitals: 23 measures
Source: Medicare and Medicaid Programs; Electronic Health Record Incentive Program; Proposed Rule. January 13, 2010
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 10
Why Meaningful Use?
• Improvements in quality, safety, efficiency, and reductions in health disparities,
• Engagement of patients and families in their health care,
• Improvements in care coordination, • Improvements in population and public health, • Adequate privacy and security protections for
personal health information.
Anticipated health policy outcomes for meaningful use of EHR technology are:
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 11
Example: Meaningful Use Criteria
Health outcomes policy priority
Care goals
Objectives
Measures
Improving quality, safety, efficiency, and reducing health disparities.
Use evidence-based order sets and CPOE. Apply clinical decision support at the point of care.
Implement drug-drug, drug-allergy, drug-formulary checks. (same for eligible professionals and hospitals)
Eligible professional/hospital has enabled this functionality.
EHR meaningful use
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 12
Role of Usability in Meaningful Use
Achieving meaningful use requires successful implementation of certified EHR technology.
People get things done quickly and productively
They get the info they need, complete work accurately and achieve their goals
They feel confident and pleased
The technology does not get in the way
They don’t make mistakes
They are not frustrated
Efficiency
Effectiveness
Satisfaction
Quality attribute defined in ISO 9241, Part 11 Usability
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 13
Why does usability matter?
Good usability can help...
reduce increase Patient and staff safety Productivity and accuracy Staff morale Customer loyalty Competitive advantage
Training time Error rates Staff turnover Product liability Customer support
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 14
14
Why does usability matter?
Risks of poor usability • Endangers patients • Increases adverse events • Information overload leading to erroneous
decisions • High costs of training, errors, rework • Increased product & practice liability and last but not least...
Barrier to EHR implementation and adoption
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 15
Poor Usability = Adverse Events
Over a 2 year period, voluntary reporting of adverse events resulting from health IT malfunctions to FDA found:
260 reports with potential for patient harm 44 reports of injuries 6 deaths
Usability-related adverse events:
How many are unreported or caught before they become serious problems?
Source: Jeffrey Shuren, Director of CDRH, FDA. Testimony to ONC Health IT Policy Committee, February 25, 2010.
The user documented activities in the task list for one patient and used the “previous” or “next” arrows to select another patient chart, the patient’s task list displayed for second patient.
A sleep lab’s workstation software had confusing user interface, which led to the overwriting and replacement of one patient’s data with another patient’s study.
< prev next >
Task List
1. _______
2. _______
3. _______
BEST EMR SYSTEM
John Saint
John Smith
Janet Smith
L. Smite
Paul Smote
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 16
Poor Usability=EHR Adoption Barrier
Administrators, clinicians, CIOs, CMOs, and policymakers listed Top 10 barriers to EHR implementation:
Source: John Halamka, CIO, Beth Israel Deaconess Medical Center during “Leadership Strategies for Information Technology in Health Care” class at Harvard. February 1, 2010. http://geekdoctor.blogspot.com/2010/02/top-10-barriers-to-ehr-implementation.html
# 10 Usability Hard to use and not well engineered for clinician workflow.
# 9 Politics/naysayers Every organization has a powerful clinician or administrator who is convinced that EHRs will cause harm, disruption, and budget disasters.
# 8 Fear of lost productivity Concerned they will lose 25% productivity for 3 months after implementation.
# 7 Computer illiteracy/training Many clinicians are not comfortable with technology; often reluctant to attend training sessions.
# 6 Interoperability Applications do not seamlessly exchange data for coordination of care, performance reporting, and public health.
Solvable with good usability
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 17
Poor Usability=EHR Adoption Barrier
Administrators, clinicians, CIOs, CMOs, and policymakers listed Top 10 barriers to EHR implementation:
Source: John Halamka, CIO, Beth Israel Deaconess Medical Center during “Leadership Strategies for Information Technology in Health Care” class at Harvard. February 1, 2010. http://geekdoctor.blogspot.com/2010/02/top-10-barriers-to-ehr-implementation.html
# 5 Privacy Significant local variation in privacy policy and consent management strategies.
# 4 Infrastructure/IT reliability Many IT departments cannot provide reliable computing and storage support, leading to EHR downtime.
# 3 Vendor product selection/ suitability Hard to know what product to choose, particularly for specialists who have unique workflow needs.
# 2 Cost the stimulus money does not flow until meaningful use is achieved. Who will pay in the meantime?
#1 People Hard to get sponsorship from senior leaders, find clinician champions, and hire the trained workers to get the EHR rollout done.
Barriers mitigated with user-centered processes
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 18
Biggest EHR Usability Problem
HIMSS EHR Usability Pain Points Workgroup conducted a survey on the specific factors that resulted in poor usability (catch their presentation on Thursday, June 10).
Workflow is the overwhelming problem in almost all facets
“Must view many areas to capture the entire patient’s story”
“Too much info in too many different places, getting lost and overwhelming”
“Doesn't match clinician thought process”
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 19
“We failed to focus on the most important part of the decision—the human/computer interface....we didn’t listen to our guts on the design of the computer screen that we would have to look at for hours on end. It is about functionality and workflow.”
-Joseph G. Cramer, MD “Bought Wrong EMR” in Medical Economics Magazine
February 5, 2010, pp 28-30
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 20
Good Usability = a Good “Fit”
• “Fit” between the object and … • Its purpose • Human mental and physical capabilities • Target audience • Environment in which it will be used • Tasks it will be used for
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 21
Usability by Design
• Usability does not happen by chance • It can’t be “painted on” at the end • It can’t be achieved through testing alone...
but usability testing gives great insights as to improvements needed!
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 22
How to build-in usability?
Usability Test
Release Date
Timeline
Fantastic! And we don’t even
need any training
Usability Test
Release Date
Observe the work
Interview users
Timeline
Brainstorm design ideas
Get user feedback
Create designs
Usability test
Revise designs
Timeline
This isn’t what I
expected…
Common problem
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 23
What is User-Centered Design?
Industry best practice for creating usable products from the beginning
Risk mitigation strategy
Perspective
Toolkit of methods & guidelines
Discipline based on human factors
ISO 13407
task analysis
walkthroughs
usability testing
evidence-based design
ethnographic studies
usage tracking
affordances
predictability
working memory
feedback
mental models
Fitts’ law
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 24
How does UCD help with Meaningful Use?
When done right... • Meaningful use requirements are met in a way that fits
with how all staff members truly work. • Your EHR has features that your staff actually need,
rather than unnecessary “cool ideas”. • Finding clinician champions and gaining buy-in from
key stakeholders is easier because it is an inclusive process. (helps solve the “people” barrier!)
Participatory process that involves true end-users and other key stakeholders.
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 25
How does UCD help with Meaningful Use?
When done right... • Realistic, productive workflows for meaningful use
requirements are identified. • Scenarios where meaningful use criteria are applicable
and can be met are comprehensively identified. • The impact of “meaningful use” implementation on
staff duties and responsibilities is anticipated.
Practice predicated on getting into users’ heads to understand specific:
• Characteristics • Tasks and goals • Context in which they do work
EHR
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 26
User-Centered Design Process
Understanding the needs
Designing a solution that works
Who are the users?
What are their tasks and goals?
What situations bring them here?
What are their expectations?
Best practices for usable design
Progressive refinement
Multidisciplinary collaboration
ITERATION
Brainstorm
Test with users
Design
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 27
Always start with goals
• Business goals
• Stakeholder goals
• User goals
• Usability goals
Business goals include care goals and meaningful use objectives as defined in the proposed rules.
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 28
UCD: Observe & Analyze
Observe &
Analyze
Envision &
Design
Evaluate &
Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 29
Conduct user studies
A variety of activities that gather information about • Users • Tasks • Context of use
Users are the actual people who use the product. Users are not:
• The CEO • Their organizations and managers • You and the development team • Your public affairs or marketing department
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 30
User research methods
Observe & Analyze Envision & Design Evaluate & Refine
• Interviewing
• Site visits/contextual inquiry
• Usability testing
• Surveys
• Focus groups
• Analysis of emails, requests, or issues
• Conferences, training, user group meetings
• Usage logs
• Search logs
Not all at once—choose the techniques that fit best with your project
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 31
Different user groups have different needs
Emergency room unit
General practice physician
Patient Insurance and billing
Observe & Analyze Envision & Design Evaluate & Refine
When researching user needs, gather requirements from: • Direct users • Indirect users, e.g. billing • Others who have contact with users, e.g. caregivers
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 32
UCD in Meaningful Use: User Needs
Meaningful use objective: Implement drug-drug, drug-allergy, drug-formulary checks.
Certification criteria example: Automatically and electronically generate and indicate in real-time, alerts at the point of care for drug-drug and drug-allergy contraindications based on medication list, medication allergy list, age, and CPOE.
Who will use these drug-drug and drug-allergy checks? Do these user groups have the same needs? Which user group can override alerts? Should they?
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 33
Document user needs in Personas
Personas are: • Personal, composite sketches
of real users • Discovered through user
research • Representative of typical
users, not edge cases
Used to: • Debunk false assumptions • Help envision users and
design what is best for them • Keep a focus on the user
throughout the project
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 34
Analyze user tasks
Observe & Analyze Envision & Design Evaluate & Refine
Tasks are specific activities that people need to accomplish. • Examples:
• Complete all prescription refill requests • Discuss the MRI results with the neurologist • Call Ms. Nelson to discuss test results
• Many tasks include both information and action
Read over patient record, concentrating on problem list. Consider diagnosis possibilities.
Schedule discussion time with neurologist
Information Action
Discuss MRI results with the neurologist
Task
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 35
How to analyze tasks
• List them
• Match them to user groups
• Prioritize them based on
• Frequency
• Criticality of failure
• Break them down into
• their component parts, and/or
• the sequence of steps involved
• Write stories that illustrate them (scenarios)
Freq
uenc
y Criticality
Low Medium High
High
Medium
Low
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 36
Documenting tasks in scenario form
Scenarios are realistic narrative descriptions of activities that users engage in, detailed enough that design implications can be inferred
—Based on J.M. Carroll, 1995
How it works: • Write the story of the work from
the user’s perspective • Share the story with team to help
them visualize how tasks occur in the actual work environment
• Conduct walkthroughs of the proposed designs using these scenarios
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 37
Example: UCD in Physician Environment
Source: What’s Keeping Us So Busy in Primary Care? A Snapshot from One Practice by Richard J. Baron, M.D., New England Journal of Medicine, 362; 17, April 29, 2010S
Type of Service
Total No.
No. per Physician (day)
Telephone call 21, 796 23.7
Laboratory report 17,794 19.5
Visit 16,640 18.1
E-mail message 15,499 16.8
Consultation report 12,822 13.9
Prescription refill 11,145 12.1
Imaging report 10,229 11.1
4 FTE physicians, each working 50-60 hrs/wk, 230 days/yr.
Frequent (e.g. daily, weekly)
Infrequent (e.g. quarterly, annually)
One thing at a time (single)
Several at a time (multiple)
Observe & Analyze Envision & Design Evaluate & Refine
Would want to understand: • Typical amount of time spent
during each service interaction • Most beneficial patient
interaction • Other categories of services e.g.
health plan correspondence, FMLA forms
• Types of interruptions and frequency
• Tasks that are queued up to be addressed all at once vs. piecemeal
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 38
Example: UCD in ER or ICU
Adapted from Representing Reality: The Human Factors of Health Care Information, C P. Nemeth,M. O'Connor, M. Nunnally, and R I. Cook Chapter 28, Handbook of Human Factors and Ergonomics in Health Care and Patient Safety
Clinician is developing individual and shared mental model of patient
How would you automate an ER
whiteboard?
Patient care at individual clinical and unit levels
Pt 1 needs CT scan, had blood
work-up this morning
Pt 1 scheduled for CTC at 1500. Needs
line changed beforehand
Observe & Analyze Envision & Design Evaluate & Refine
Pt 2 doesn’t look good. May need to be intubated.
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 39
UCD in Meaningful Use: Task & Context
Meaningful use objective: Implement drug-drug, drug-allergy, drug-formulary checks.
Certification criteria example: Automatically and electronically generate and indicate in real-time, alerts at the point of care for drug-drug and drug-allergy contraindications based on medication list, medication allergy list, age, and CPOE.
In what situations will these alerts come up? • Acute episodes / hospitalizations • Chronic conditions • Primary care encounters • Preventative care
In what context will these alerts come up? Emergency room, general practice, pharmacy. ICU System
Drug alert! Do not use vancomycin
X
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 40
UCD: Envision & Design
Observe &
Analyze
Envision &
Design
Evaluate &
Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 41
Bridging the gap from analysis to design
Humans are good at some things…
…machines are good at other things.
To optimize the system, let each focus on what they are good at.
calculate BMI
lookup drug allergies
remind me about
drug interactions
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 42
“It is all about design, which we see every day, but mostly ignore....Design of the computer screen and the underlying program is how our brains see the whole picture of the patient.”
-Joseph G. Cramer, MD “Bought Wrong EMR” in Medical Economics Magazine
February 5, 2010, pp 28-30
EHR
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 43
Design is Hard
• You don’t get it right the first time • There are always trade-offs • Keys to success:
• Start with a deep knowledge of your users and their tasks
• Generate multiple ideas up front (e.g. through parallel design and brainstorming)
• Iteration – walkthroughs and user feedback • Progressive refinement • Following guidelines and patterns for usable design
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 44
“Computers are strong medicine. Done well, they are wonderful: done poorly they can kill people”
- Justin Starren MD Marshfield Clinic, Wisconsin
Source: As Doctors Shift to Electronic Health Systems, Signs of Harm Emerge, by Fred Schulte and Emma Schwartz, April 20, 2010, Huffington Post
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 45
UCD in Meaningful Use: Design
Meaningful use objective: Implement drug-drug, drug-allergy, drug-formulary checks.
Certification criteria example: Automatically and electronically generate and indicate in real-time, alerts at the point of care for drug-drug and drug-allergy contraindications based on medication list, medication allergy list, age, and CPOE.
I talked to my users and stakeholders, now magic happens!
It can still can go very wrong. For example, CPOE systems often flood doctors with warning alerts, leading physicians to ignore them, which is a human factor principle known as “alert fatigue” or “pop-up fatigue”.
Source: Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors, Ross Koppel; Joshua P. Metlay; Abigail Cohen; et al. JAMA. 2005;293(10):1197-1203
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 46
Magic happens or not...
Dangerous Drug Interaction: Warfarin x Sulfa
MyCPOE System Patient: Jane Smith
Patient Summary
Clinical Notes
Problem List
Medication
-----------------------
Consults
Lab Results
Warfarin x Sulfa interaction: Potential for bleeding
Pt currently on: Coumadin (warfarin)
Dangerous Drug Interaction
Don’t fill
Patient currently on: Coumadin, since 03/09/2001
Consider alternatives:
Warfarin x Sulfa Potential for bleeding details
Cephalexin Nitrofurantoin
Change order
Fill order
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 47
Visioning
What are we trying to build?
What would be best for our users?
What will help them accomplish their tasks?
How can we meet their expectations?
What is the best way to meet our goals?
Example: Designing a new house
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 48
Visioning
• In user-centered design, this involves:
• “Blue sky” brainstorming sessions
• Fast, informal generation of ideas
• Flip charts
• Whiteboard drawings
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 49
Parallel Design
Generates a wide range of design possibilities quickly The full group discusses pros and cons of each
Sketch #B “What I really like about this
idea is…?”
Serious alerts are easy to spot Like being able to see the rest of the patient’s record
Alert is helpful and lets me change the order if I need to. I don’t need to click through 5 screens again to do that.
It lets me get to more details so I don’t need to go back out to look it up
Observe & Analyze Envision & Design Evaluate & Refine
Sketch #C Sketch #A
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 50
How should it be organized?
What are its main sections?
What will people do in each area?
What will people expect each area to be called?
How can we streamline the path from one section to another?
Structural Level: Organizing
Example: Designing a new house
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 51
Structural Level: Organizing
Abstract prototype
• In user-centered design, this involves:
• Abstract prototypes
• Sitemaps
• Flow charts
• Card sorts
Sitemap
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 52
Abstract Prototype
Used in discussion with stakeholders to clarify content and organization
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 53
Which things should be near each other because they go together?
Is this the right style of interaction?
Does this layout support the flow of the task?
Representing and Refining
Example: Designing a new house
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 54
Representing and Refining
• In user-centered design, this involves:
• Sketches
• Low-fidelity prototypes or mockups
• Wireframes
Paper prototypes
Wireframes
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 55
Types of Prototypes
Good for testing:
- Concepts
- Organization
- Screen flow
- Main ideas
Good for testing:
- Terminology
- Headings
- Navigation
- User tasks
Good for testing:
- Visual appeal
- Interactions
- Accessibility
Observe & Analyze Envision & Design Evaluate & Refine
Low Fidelity High Fidelity
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 56
Iteration
• Design in repeated cycles • Results of each cycle feed into the next cycle
• Get input and feedback early and often • Prototypes don’t need to be working yet • Less “finished” looking, more options, elicit more feedback
ITERATION
Brainstorm Design
Test with users
(or walkthrough with specialists)
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 57
UCD: Evaluate & Refine
Observe &
Analyze
Envision &
Design
Evaluate &
Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 58
Many Ways to Get User Feedback
• In addition to usability testing, you can use: • Informal, scenario-based walkthroughs • Surveys • Web metrics and usage tracking • Management information on transactions • Help desk log • Emailed feedback
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 59
User Feedback Sessions
• Set expectations • Ask the user to “think aloud” and interpret what they see • Ask the user about realistic tasks • Ask the user to compare alternatives
Observe & Analyze Envision & Design Evaluate & Refine
Refer to handout for details
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 60
Testing for Usability
Get real users
Ask them to perform realistic tasks using your system, prototype, or web site
Observe, take notes, and see what works and what doesn’t work
If they have problems, fix them before the release!
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 61
Types of Usability Tests
• Formal vs informal • Formal: In a lab with two-way mirrors, logging
software, video cameras, observers • Informal: In a cafeteria, senior center, at home, or
wherever, with nothing but your prototype
• Formative vs summative • Formative: Identify problems and opportunities for
improvement • Summative: Determine whether performance
measures were met, or to set a baseline • In person vs remote
Observe & Analyze Envision & Design Evaluate & Refine
Refer to handout for details
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 62
Typical Measures for Usability
• Efficiency • Time to complete task • Number of clicks • Number of days/hours training reduced
• Effectiveness • Success rate (or completion rate) • Number or % of errors • Number of attempts before successful completion • Cost savings from reduced errors
• Satisfaction • Number of positive and negative statements or feedback
messages received from users • Satisfaction scores on a survey (e.g. SUS, QUIS, ACSI) • Number of users who rate the system as good or excellent • Actual usage (number of people, % increase)
Observe & Analyze Envision & Design Evaluate & Refine
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 63
Design Walkthroughs
Walkthrough Abstract prototype
Low-Fi Prototype
High-Fi Prototype
1) Scenario-Based Walkthrough 2) Persona-Based Walkthrough 3) Requirements Walkthrough 4) Subject-Matter Expert Walkthrough 5) Database Walkthrough 6) Information Needs Walkthrough 7) Usable Design Principles Walkthrough 8) Accessibility Principles Walkthrough
Observe & Analyze Envision & Design Evaluate & Refine
Refer to handout for details
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 64
Is there a magic number?
We have often been asked... “Is there a magic number to figure out if my product passes/fails usability?”
Why? Usability is measured by:
and is about balancing user needs. It is not a threshold measure.
Is there a magic number for human clinical trials? Why?
Efficiency Effectiveness Satisfaction
Observe & Analyze Envision & Design Evaluate & Refine
No, there isn’t.
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 65
When user requirements go awry
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 66
Takeaway: Do it early
Planning Requirements Design Development Roll-out Validation
Cost of
modifications
Identify usability and accessibility needs as early as possible to reduce costs
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 67
Takeaway: Build it in from the beginning
Involve real users early and often
Observe their actual work
Work collaboratively with a multi-disciplinary team
Follow human factors & usable design guidelines
Design the user interface deliberately
Iterate the design with user feedback
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 68
Takeaway: Start now!
Start now! You have the means...
• Look at your paper forms and their contents: • What you are collecting and why? • How do you intend to use it when it is digital?
• Analyze patterns of work e.g. patient requests, repetitive fixed events, tasks everyone can do
• Look at your current workflow. What’s optimal? • Scrutinize problem logs. Make sure you don’t
automate the problem source!
Copyright © 2010 Lisa Battle, Jasmin Phua & Duane Degler Slide 69
Better design, better healthcare
errors fatigue repetitive work stress & frustration loss of time training needs
ease of use productivity success rate human comfort user acceptance satisfaction
Transform healthcare through better design
reduce increase
Questions?
Lisa Battle: [email protected] Jasmin Phua: [email protected]
Duane Degler: [email protected]