usability & evaluation in visualizing biological data chris north, virginia tech vizbi

Usability & Evaluationin Visualizing Biological Data

Chris North, Virginia Tech

VizBi

Usomics & Evaluationin Visualizing Biological Data

Chris North, Virginia Tech

VizBi

Myths about Usability

Usability = Voodoo

Science of Usability

Measurement

Modeling

Engineering

Science

Phenomenon

…analogy to biology

Usability Engineering

User-centric

Iterative

Engineering = process to ensure usability goals are met

1. Analyze Requirements

2. Design

3. Develop

4. Evaluate

Myths about Usability

Usability = Voodoo

Usability = Learnability

Myths about Usability

Usability = Voodoo

Usability = Learnability

Usability = Simple task performance

Impact on Cognition

Spotfire

66

40

0

10

20

30

40

50

60

70

80

90

GeneSpring

Insight gained:

Myths about Usability

Usability = Voodoo

Usability = Learnability

Usability = Simple task performance

Usability = Expensivehttp://www.upassoc.org/usability_resources/usability_in_the_real_world/roi_of_usability.html

Usability Engineering

1. Analyze Requirements

2. Design

3. Develop

4. Evaluate

Requirements Analysis

Goal = understand the user & tasksMethods: Ethnographic observation, interviews, cognitive task analysis

Challenge: Find the hidden problem behind the apparent problem

Analysts’ Process

Pirolli & Card, PARC

Systems Biology Analysis

Beyond read-offs -> Model-based reasoning

Mirel, U. Michigan

Usability Engineering

1. Analyze Requirements

2. Design

3. Develop

4. Evaluate

Why Emphasize Evaluation?

Many useful guidelines, but…

Quantity of evidence

Exploit domain knowledgeHunter, Tipney, UC-Denver

Science of Usability

Measurement

Modeling

Phenomenon

Measuring Usability in Visualization

system,algorithm

Measurements

• frame-rate• capacity• …

• realism• data/ink• …

• market• ?

• ?

2 kinds of holes

visualperception,interaction

inference,insight

goal,problemsolving

Phenomena

• task time• accuracy• …

Time & Accuracy

Controlled Experiments Benchmark tasks

Results

Performance Time

0

0.5

1

1.5

2

2.5

3

T1* T2 T3 T4* T5* T6 T7*Tasks

Tim

e (

in m

in)

1 Tpt M Tpts. M. Graphs

Accuracy

0

2

4

6

8

10

T1 T2 T3 T4* T5 T6* T7Tasks

Co

un

t

1 Tpt M Tpts. M. Graphs

+ Consistent overall

+ Fast for single node analysis- Slow and inaccurate for expression across graph

+ Accurate for comparing timepoints

p<0.05

Cerebral Munzner, UBC

Insight-based Evaluation

Problem: Current measurements focus on low-level task performance and accuracy

What about Insight?

Idea: Treat tasks as dependent variableWhat do users learn from this Visualization?Realistic scenario, open-ended, think aloudInsight codingInformation-rich results

Insight?

Spotfire

GeneSpring

Cluster/Treeview

TimeSearcher

HCE

Gene expression visualizations

Cluster- Time- Gene- View Searcher HCE Spotfire Spring

4.6

7

14

8

16

0

2

4

6

8

10

12

14

16

18

Av

g T

im

e to

F

irs

t In

sig

ht

48 51

34

66

40

0

10

20

30

40

50

60

70

80

90V

alu

e

18

21

14

25

20

0

5

10

15

20

25

30

Co

un

t

Count of insights

Total value of insights

Average timeto first insight(minutes)

Results

Insight Summary

Time series Viralconditions

Lupusscreening

Clusterview

TimeSearcher

HCE

Spotfire

GeneSpring

Users’ Estimation

41

4842

67

52

0

10

20

30

40

50

60

70

80

Av

g F

ina

l A

mo

un

t48 51

34

66

40

0

10

20

30

40

50

60

70

80

90

Va

lue

Total value of insights

Users’ estimated insight percentage

Cluster- Time- Gene- View Searcher HCE Spotfire Spring

Insight Methodology

Difficulties:Labor intensive

Requires domain expert

Requires motivated subjects

Short training and trial time

Opportunities:Self reporting data capture

Insight trails over long-term usage – Insight Provenance

Trend towards Longitudinal Evaluation

Multidimensional in-depth long-term case studies (MILCS)Qualitative, ethnographic

GRID: Study graphics, find features, ranking guides insight, statistics confirm

But: Not replicable, Not comparative

Shneiderman, U. Maryland

Onward…

VAST ChallengeAnalytic dataset with ground truth

E.g. Goerg, Stasko – JigSaw study

BELIV Workshop – BEyond time and errors: novel evaLuation methods for Information Visualization

Visual Analytics

Visualization Visual Analytics

Perception, Interaction

Cognition, Sensemaking

Visualization tasks Whole analytic process

Visual representations, interaction techniques

Connection to data mining, statistics, …

Datatype scenarios Real usage scenarios, Analysts

Embodied Interaction

GigaPixel Display Lab, Virginia Tech Carpendale, U. Calgary

1) Cognition is situated. 2) Cognition is time-pressured. 3) We off-load cognitive work onto the environment. 4) The environment is part of the cognitive system. 5) Cognition is for action. 6) Off-line cognition is body-based.

-- Margaret Wilson, UCSC