hci methods for pathway visualization tools purvi saraiya, chris north, karen duca* virginia tech...
Post on 19-Dec-2015
217 views
TRANSCRIPT
![Page 1: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/1.jpg)
HCI Methods for Pathway Visualization Tools
Purvi Saraiya, Chris North, Karen Duca*
Virginia Tech
Dept. of Computer Science,Center for Human-Computer Interaction
*Virginia Bioinformatics Institute
![Page 2: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/2.jpg)
What is HCI?
• All aspects of human interaction with a computer system
• A discipline concerned with design, implementation, and evaluation of computer systems for human use
![Page 3: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/3.jpg)
Two Perspectives For Pathway Tools
• Developers:– What tool will be most helpful to
users?• Users:
– Many options, how to select the most appropriate tool?
![Page 4: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/4.jpg)
Software Development Cycle
Reqs Analysis
Evaluate Design
Develop
![Page 5: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/5.jpg)
Requirement Analysis Using Informal Interviews
Research Questions: • What kinds of tasks do
the users perform?• How do the tools fit with
overall research goals of the users?
Outcomes:• Usage Scenarios• Requirements
Method: 1-1 Interviews
![Page 6: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/6.jpg)
Usage Scenarios
• Scenarios: stories about users and their work activities
• User-oriented: focus on needs and concerns of users
• User-perception: how do users perceive their problems?
![Page 7: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/7.jpg)
User Requirements for Pathway Visualization Tools
Categories Requirements
Tasks
Pathway Construction
1. Create & Update
Collect and link pathways from multiple resources
2. Context Provide information about pathways
3. Uncertainty Show alternate hypotheses and information reliability
4. Collaboration
Enable group work
![Page 8: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/8.jpg)
Questionnaires and SurveysResearch Question: Which requirements are more important?
Pathway Questions Strongly Agree
Agree
Neutral
Disagree
Category: Pathway AssemblyR1: Create & Update
1
In my work, the entire pathway(s) is generally not available from a single source. 6 4
R1: Collaborate
2
I collaborate with others and need a tool to let them enter changes from remote sites 1 1 4 4
![Page 9: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/9.jpg)
Focus Groups• Research Questions:
– What is user feedback for the selected systems?
• Method:– Group discussion with
users • No of users:
– 5 - 10
![Page 10: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/10.jpg)
Systems Evaluated
GenMapp, PathwayAssist, Cytoscape, Patika, GScope
![Page 11: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/11.jpg)
Results• Example User Responses:
– Users were excited about the NLP features provided by PathwayAssist but were skeptical about its reliability
– Need more biological context for the pathways
End-Users:– Which system should I use for my work?
![Page 12: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/12.jpg)
Longitudinal Studies • Research Question:
– How are the tools used in actual real world scenarios?
– What interactions and features were actually used by the users for their tasks?
• Method:– Users: Log Keeping– Evaluators: Discuss user logs
![Page 13: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/13.jpg)
Example Log
Heatmap + Pathway Visualization in PathwayAssist
Date Visualization
Insight Value
9/01 Heatmap A list of genes that are suppressed by smoking but up-regulated by flu.
4
9/12 Pathway Visualization
The up-regulation of Mx by flu is suppressed by smoking even though smoking itself did not have an effect on basal Mx activity.
3
Example log for data analysis with PathwayAssist
![Page 14: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/14.jpg)
Empirical Studies
• Research Question: – Are the tools preferred by users actually better?
• Method:– Task-Based Method– Insight-Based Method
![Page 15: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/15.jpg)
Task-Based Studies
• Typical question:– Which visualization is better for defined
tasks?
Vs.
Vis. 1 Vis. 2
![Page 16: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/16.jpg)
Variables• Independent Variables:
• Visualization tool» GenMapp, PathwayAssist, etc.
• Task type» Find, count, pattern, compare
• Data size (# of items)» 100, 1000, 1000000
• Dependent Variables:• Task completion time• Errors• Subjective satisfaction (survey)
![Page 17: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/17.jpg)
Experiment Protocol
Participants may be videotaped
Evaluator takes notes
![Page 18: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/18.jpg)
Data Analysis
• t-test:• Compares 1 dep var on 2 treatments of 1 ind var
• ANOVA:• Compares 1 dep var on n treatments of m ind vars
Task1: Find the node with max value?
Task2 Task3
Vis 1 123245
… …
Vis 2 … … …
Ind Var 1: Vis. Tool
Ind Var 2: Task Type
Dep var: user performance times (3 users per cells)
![Page 19: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/19.jpg)
Insight-Based Method
Motivation:• Task-based methods uses pre-selected
tasks• Often not representative of the real world
visualization tool usageIssues:• How to eliminate benchmark tasks?• What is Insight? How to codify and
quantifiably measure insights across participants?
![Page 20: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/20.jpg)
Insight Characteristics• Insight = an individual data observation
•Observation•Time to discover•Domain value (importance)•Hypotheses generated?•Directed vs. unexpected•Correctness •Category (overview, details)
• Can be recognized by thinking-aloud
![Page 21: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/21.jpg)
Experiment Design
• Independent Variables:– Visualization Tools– Datasets– Participant Background
• Dependent Variables:– Insights– Time at which insights were reported– User feedback
![Page 22: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/22.jpg)
Tool 1 Tool 2Tool 3 Tool 4 Tool 5
Count of insights
Domain value of insights
Average timeto first insight(in mins)
18
21
14
25
20
0
5
10
15
20
25
48 51
34
66
40
0
15
30
45
60
75
4.5
7
14
8
16
0
5
10
15
20
![Page 23: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/23.jpg)
Data Bias
The tool works better for categorical data
36
20
9
26
42 2
14
0
10
20
30
Count of Insights Domain value of Insight Avg Time to 1st Insight(mins)
Time-series Categorical Multi-Categorical
![Page 24: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/24.jpg)
ConclusionsDevelopers:• Informal 1-1 interviews & questionnaires • Focus group meetings• Longitudinal StudiesUsers:• Controlled studies to evaluate tools• User studies may not always be feasible• Insight characteristics can be used as
checklists for selecting a tool
![Page 25: HCI Methods for Pathway Visualization Tools Purvi Saraiya, Chris North, Karen Duca* Virginia Tech Dept. of Computer Science, Center for Human-Computer](https://reader030.vdocument.in/reader030/viewer/2022033105/56649d3b5503460f94a15e22/html5/thumbnails/25.jpg)
Thank you