uniform: automatically generating consistent remote control user interfaces jeffrey nichols, brad a....
TRANSCRIPT
UNIFORM: Automatically Generating Consistent Remote Control User Interfaces
Jeffrey Nichols, Brad A. Myers, Brandon RothrockHuman-Computer Interaction InstituteSchool of Computer ScienceCarnegie Mellon University
Conference on Human Factors in Computing Systems (CHI 2006)April 25, 2006
2
Motivation
3
Motivation, cont.
Appliance interfaces could be more usable if their interfaces were consistent
Almost all appliances have some unique features
Question: How might additional functionality be presented to the user while still providing a consistent interface?
4
UNIFORM
Generates personally consistent user interfaces
Personal Mobile Devices
Personal Universal Controller (PUC) system
Appliances
Specifications
Control
State Feedback
5
UNIFORM
Copier A Copier B
Consistency disabled – Copier A used first
6
Consistency disabled – Copier A used first
UNIFORM
Copier A Copier B
Consistency enabled – Copier A used first
Consistent Copier B
7
Overview
• Motivation
• Previous Work on Consistency
• Specification Authoring Studies
• Requirements for Consistency
• Architecture
• Knowledge Base and Finding Similarity
• Generating Consistent Interfaces
• Discussion and Future Work
8
Related Work
Theory of Consistency• 1988 workshop at CHI was unable to define
consistency [Nielsen 88, Grudin 89]
• Loosely defined as “doing similar things in similar ways” [Reisner 90]
• Different dimensions of consistency [Kellogg 87]
• Work on consistency as knowledge transfer (e.g. [Keiras & Polson 85, Polson 88])
9
Related Work, cont.
Systems addressing Consistency
• Some tools can automatically test for consistencyGLEAN uses a GOMS-based approach [Kieras 95]
Sherlock generates metrics for interpretation by designer [Mahajan 97]
• Previous automatic generation systems have ensured consistency within an application or family of applications
ITS [Wiecha 90]
PUC [Nichols 02]
SUPPLE [Gajos 05]
10
Specification Authoring Study
Study of Authoring Appliance Specifications
• How can specifications vary for different appliances with similar functions?
• How can specifications vary for different authors?
Two User Studies• Study #1: An expert user (me!) wrote
specs forthree different VCRs
• Study #2: Three subjects wrote specifications
for the same VCR
All subjects instructed to: • Include all functions• Be faithful to the existing appliance design
Low End: Panasonic VCR
Mid-Range: Samsung DVD-VCR
High-End: Mitsubishi DVCR
11
Results of Both Studies
Functions• Some were identical• Others were similar but had different labels• Others had very different specifications
Structure• Structure for the first study was very similar across
appliances• Top-level structure was the same for the second study, but
many differences were found at lower levels
Results shows that specifications can differ substantially
Analysis of results showed important distinction between functional and structural consistency
12
Consistency in UNIFORM
We define a consistent interface to be one that incorporates elements and organization that are familiar to the user from previous interfaces
We have refined this definition into six requirements, based on previous research and our authoring study
13
Requirements for Consistency
1. Interfaces should manipulate functions in the same way
2. Interfaces should locate similar functions in the same place
3. Interfaces should use similar labels for similar functions
4. Interfaces should maintain a similar visual appearance
14
Requirements for Consistency
1. Interfaces should manipulate functions in the same way
2. Interfaces should locate similar functions in the same place
3. Interfaces should use similar labels for similar functions
4. Interfaces should maintain a similar visual appearance
5. Usability of unique functions is more important than consistency
15
Requirements for Consistency
1. Interfaces should manipulate functions in the same way
2. Interfaces should locate similar functions in the same place
3. Interfaces should use similar labels for similar functions
4. Interfaces should maintain a similar visual appearance
5. Usability of unique functions is more important than consistency
6. Users must be able to choose to which appliance consistency is ensured
16
Overview
• Motivation
• Previous Work on Consistency
• Specification Authoring Studies
• Requirements for Consistency
• Architecture
• Knowledge Base and Finding Similarity
• Generating Consistent Interfaces
• Discussion and Future Work
17
UNIFORM Architecture
Knowledge Base
UNIFORM Interface Generation
PUC ApplianceSpecification
Personally Consistent User Interface
18
UNIFORM Architecture
Knowledge Base
Previous Specifications
Functional Mappings Between Specifications
Previously Generated UIs
Mapping Phase
1
Abstract Consistency
Phase
3
Concrete UIBuilding Phase
4
Output:Consistent User Interface
Concrete Consistency Phase
5
2 Abstract UI Building Phase
Input:Appliance Specification
<?xml?>
<spec>
<stuff/>
</spec>
UNIFORM Interface Generation
19
UNIFORM Architecture
Mapping Phase
1
Abstract Consistency
Phase
3
Concrete UIBuilding Phase
4
Output:Consistent User Interface
Concrete Consistency Phase
5
2 Abstract UI Building Phase
Input:Appliance Specification
<?xml?>
<spec>
<stuff/>
</spec>
UNIFORM Interface Generation
Knowledge Base
Previous Specifications
Functional Mappings Between Specifications
Previously Generated UIs
20
UNIFORM Architecture
Knowledge Base
Previous Specifications
Functional Mappings Between Specifications
Previously Generated UIs
UNIFORM Interface GenerationMapping
Phase1
Abstract Consistency
Phase
3
Concrete UIBuilding Phase
4
Output:Consistent User Interface
Concrete Consistency Phase
5
2 Abstract UI Building Phase
Input:Appliance Specification
<?xml?>
<spec>
<stuff/>
</spec>
21
UNIFORM Architecture
Knowledge Base
Previous Specifications
Functional Mappings Between Specifications
Previously Generated UIs
Mapping Phase
1
Abstract Consistency
Phase
3
Output:Consistent User Interface
Concrete Consistency Phase
5
Input:Appliance Specification
<?xml?>
<spec>
<stuff/>
</spec>
UNIFORM Interface Generation
Concrete UIBuilding Phase
4
2 Abstract UI Building Phase
22
UNIFORM Architecture
Knowledge Base
Previous Specifications
Functional Mappings Between Specifications
Previously Generated UIs
Concrete UIBuilding Phase
4
Output:Consistent User Interface
2 Abstract UI Building Phase
Input:Appliance Specification
<?xml?>
<spec>
<stuff/>
</spec>
UNIFORM Interface GenerationMapping
Phase1
Abstract Consistency
Phase
3
Concrete Consistency Phase
5
23
Knowledge Base
Functional Mappings• We support six types of mappings,
based on the results of the specification authoring studies
general, state, node, group, list, and template
• Specifies that a function in one specification is similar to a function in another
Mappings are grouped within the knowledge base in mapping graphs
Knowledge Base
Previous Specifications
Functional Mappings Between Specifications
Previously Generated UIs
24
Mapping GraphsGroup semantically similar functions across appliances
One graph for each function
• Power• Volume• Media Controls• Etc.
Node counts • Store generation history• Used to find basis for
consistency
Edges• Track whether consistency
can be ensured
Choice in consistency supported by manipulating the node counts
Knowledge Base
Previous Specifications
Functional Mappings Between Specifications
Previously Generated UIs
3
1
DVD Player
State: Mode { Play, Stop, Pause }
Command: Next TrackCommand: Previous Track
Answering Machine
State: Mode { Play, Stop, Pause }
Command: Play New Messages
Cheap VCR
Command: PlayCommand: StopCommand: PauseCommand: RewindCommand: F-FwdCommand: Record
Mitsubishi VCR
State: Mode { Play, Stop, Pause, Rewind, F-Fwd, Record }
0
0
25
Where do mappings come from?Three ways mappings may be found
• Automatically extracted during mapping phase
• Challenging because of limited semantic information in specifications
• We have implemented two mapping algorithms, one based on work in automatic schema mapping from the database community
• Current best algorithm finds about 60% of mappings
• We are also exploring other techniques
• Specified by the user• Manually entered as text• Interaction techniques?
• Downloaded from the Internet• Manufacturers?• Internet communities? (e.g.
www.remotecentral.com)
Input:Appliance Specification
Mapping Phase
1
Abstract Consistency Phase
3
Concrete UIBuilding Phase
4
<?xml?>
<spec>
<stuff/>
</spec>
Output:Consistent User Interface
Concrete Consistency Phase
5
2 Abstract UI Building Phase
26
Abstract UI Building Phase
Rules from previous PUC system
Transforms specification into abstract user interface
Abstract user interface includes:• Tree organization• Control choice
Input:Appliance Specification
Mapping Phase
1
Abstract Consistency Phase
3
Concrete UIBuilding Phase
4
<?xml?>
<spec>
<stuff/>
</spec>
Output:Consistent User Interface
Concrete Consistency Phase
5
2 Abstract UI Building Phase
27
Abstract Consistency Phase
Most of the work to ensure consistency happens in this phase
• Operates on the abstract user interface
• We have found that most consistency can be ensured at the abstract level
• Also enables many of our rules to cross platforms with little or no additional work
This phase is made up of two parts:• Functional sub-phase
Similar functions are assigned similar labelsSimilar functions are converted, if necessary, to a consistent representation
• Abstract Structural sub-phaseSimilar functions are placed in similar groups
Input:Appliance Specification
Mapping Phase
1
Abstract Consistency Phase
3
Concrete UIBuilding Phase
4
<?xml?>
<spec>
<stuff/>
</spec>
Output:Consistent User Interface
Concrete Consistency Phase
5
2 Abstract UI Building Phase
28
Abstract Structural Sub-Phase
This sub-phase has two parts:• Moving
Similar functions are moved to previous groups
• Re-orderingWithin each group, functions are re-ordered to match previous order
29
Moving
Base
Status Setup
Clock
Channel
Base
Samsung DVD-VCR
Base
Mitsubishi DVCR
30
Moving
Base
Status Setup
Clock
Channel
Base
Setup
Samsung DVD-VCR
Base
Setup
Mitsubishi DVCR
31
Moving
Base
Status Setup
Clock
Channel
Base
Setup
Clock
Samsung DVD-VCR
Base
Status
Clock
Mitsubishi DVCR
32
Moving
Base
Status Setup
Base
Setup
Clock
Samsung DVD-VCR
Base
Status
Clock
Mitsubishi DVCR
Clock
Channel
Status
33
Moving
Base
Status Setup
Clock
Channel
Base
Status
Clock
Channel
Samsung DVD-VCR
Base
Setup
Channel
Mitsubishi DVCR
Channel
Setup
Unique functions are moved with their group but never by themselves
34
Re-Ordering
35
Re-Ordering
When
TimeDate
Channel ConflictSpeedSourceType
Start Stop
Length
VCR+ DayConflict Channel Speed
Timed Recordings for the Samsung DVD-VCR
Timed Recordings for the Mitsubishi DVCR
36
Re-Ordering
Conflict VCR+ Channel When Speed
When Type Channel Speed Conflict
Timed Recordings for the Samsung DVD-VCR
Timed Recordings for the Mitsubishi DVCR
When Type Channel Speed Conflict
37
Re-Ordering
Start StopVCR+ DayConflict Channel Speed
Timed Recordings for the Samsung DVD-VCR
Timed Recordings for the Mitsubishi DVCR
TimeDate
Conflict ChannelSource When Length Type Speed
38
Concrete UI Building Phase
Rules from previous PUC system
Transforms abstract user interface into concrete user interface
Concrete user interface includes:• Organization necessary for screen
size• Panel Structure• Instantiations of UI objects• Positions and sizes of controls
Input:Appliance Specification
Mapping Phase
1
Abstract Consistency Phase
3
Concrete UIBuilding Phase
4
<?xml?>
<spec>
<stuff/>
</spec>
Output:Consistent User Interface
Concrete Consistency Phase
5
2 Abstract UI Building Phase
39
Concrete Consistency Phase
Differences in the available labels or number of controls may have created inconsistencies
• For example, a rule may have not added organization in this interface that was in the previous interface
• The side-panel orientation in a previous interface might have been chosen to allow a long label to fit
Rules in this phase address these problems provided that the usability of unique functions is not affected
• For example, side-panel orientation might not change if needed for a large label or control
Input:Appliance Specification
Mapping Phase
1
Abstract Consistency Phase
3
Concrete UIBuilding Phase
4
<?xml?>
<spec>
<stuff/>
</spec>
Output:Consistent User Interface
Concrete Consistency Phase
5
2 Abstract UI Building Phase
40
Overview
• Motivation
• Previous Work on Consistency
• Specification Authoring Studies
• Requirements for Consistency
• Architecture
• Knowledge Base and Finding Similarity
• Generating Consistent Interfaces
• Discussion and Future Work
41
Discussion and Future Work
Evaluation of Generated Interfaces (in progress)
• Users will perform ~10 tasks with two all-in-one copier interfaces (HP or Canon)
• Three conditionsActual Appliance vs. PUC vs. Uniform
42
Discussion and Future Work
How else might we deal with unique functions?• Better semantic information• More detailed modeling?
Better knowledge of consistency• Elaborate and improve on requirements for consistency• Dimensions of consistency and relative importance• More operational knowledge
More consistency rules• Adding new organization• Support for “consistency ghosts” and other forms of
“transformational consistency” [Richter 2006]
43
AcknowledgementsCo-Authors
• Brad A. Myers• Brandon Rothrock
Thesis Committee• Scott Hudson• John Zimmerman• Dan Olsen Jr.
Funding• National Science
Foundation• Microsoft• General Motors• Intel• Pittsburgh Digital
Greenhouse
Equipment Grants• Mitsubishi (MERL)
PUC Project Members• Duen Horng Chau• Kevin Litwack• Thomas K. Harris• Michael Higgins• Joseph Hughes • Thomas Psik• Roni Rosenfeld• Rajesh Seenichamy• Pegeen Shen• Htet Htet Aung• Mathilde Pignol• Suporn Pongnumkul• Jeffrey Stylos• Stefanie Tomko• Peter Lucas
Thanks for listening!
For more information…http://www.cs.cmu.edu/~jeffreyn/uniform/http://www.pebbles.hcii.cmu.edu/puc/
UNIFORM: Automatically Generating Consistent Remote Control User Interfaces
Jeffrey Nichols, Brad A. Myers, Brandon RothrockHuman-Computer Interaction InstituteSchool of Computer ScienceCarnegie Mellon University
Conference on Human Factors in Computing Systems (CHI 2006)April 25, 2006
46
Questions
• Have you thought about using an approach where, given that you have 4-5 appliances, you generate all of the possible consistent interfaces and then find the right one (either by the user or through some kind of optimization or metric) – Allen Cypher
• What happens when I start with a simple interface and then get more complex over time? Does the consistent structure cause everything to become hidden? – Margaret Burnett
• Can you handle the case where on one appliance the function is a button and on another the function is multiple steps? – Henry Lieberman
47
Generated Phone Examples
Copier A Copier B
Consistency
disabled
48
Generated Phone Examples
Copier A Copier B
Consistency
enabled-
Copier A used first
49
Generated Phone Examples
Copier B Copier A
Consistency
enabled-
Copier B used first
50
Functional Sub-Phase
Inspects each function in the new specification
• Determines whether there is a previous function with which to ensure consistency (using mapping graph)
• Rules make changes to the abstract interface to ensure consistency
• Eight rules covering a number of potential functional differences
• Always ensure that previous labels are used
• May change the controls
Two sorting functions from different copier interfaces
51
Copier A Copier B
UNIFORMConsistency disabled – Copier B used firstConsistency disabled – Copier A used first
52
Copier ACopier B
UNIFORMConsistency disabled – Copier B used first
Consistent Copier A
Consistency enabled – Copier B used first
53
Motivation, cont.
54
Abstract Consistency Phase
Mitsubishi DVCR without consistency
Samsung DVD-VCR without consistency
Samsung DVD-VCR consistent with Mitsubishi DVCR