Chapter 0

IES 506 Human Computer InteractionLecture 11: Direct Manipulation InterfacesLecturer: Gazihan Alanku1Please look at the end of the presentation for assignments (marked with TODO)

Designing the User Interface:Strategies for Effective Human-Computer Interaction

Fifth Edition

Ben Shneiderman & Catherine Plaisantin collaboration with Maxine S. Cohen and Steven M. JacobsModified by Gazihan AlankuCHAPTER 5:Direct Manipulation and Virtual Environments 2010 Pearson Addison-Wesley. All rights reserved. Addison Wesley is an imprint of1-# 2010 Pearson Addison-Wesley. All rights reserved. 2OverviewMidterm exam reviewProject remindersCourse contentDirect ManipulationVirtual Environments6-3Project RemindersLets remember slides from week 1 in the next two slides here6-4Projects (reminder with a change)Come up with an ideaDiscuss it with me Present it in class Create it Do not start programming it!Find ideal users for your systemInterview them and understand their frame of mindCreate paper prototypes (optionally, together with users)Test the paper prototypes with the users. Learn more and improve the prototypes.At least three two iterations with the paper prototypes!Create a final prototype in Balsamiq mockupsAt least one more iterationCreate the software (now you can start programming!)At least one more iterationThis process should result in a working user interface with high usabilityVerify this with at least three user tests with new usersAssign them canonical tasks and watch them perform

5Projects (reminder)You need to videotape all user tests and interviewsYou need to document the whole progress. The documentation should include the prototypes

6OverviewDirect ManipulationVirtual Environments6-71-# 2010 Pearson Addison-Wesley. All rights reserved. Some emotions related to user interfaces6-8

1-# 2010 Pearson Addison-Wesley. All rights reserved. Some emotions related to user interfaces6-9

1-# 2010 Pearson Addison-Wesley. All rights reserved. IntroductionPositive feelings associated with good user interfaces: Mastery of the interface Competence in performing tasks Ease in learning the system originally and in assimilating advanced features Confidence in the capacity to retain mastery over time Enjoyment in using the system Eagerness to show the system off to novices Desire to explore more powerful aspects of the system

6-101-# 2010 Pearson Addison-Wesley. All rights reserved. We take for grantedAs developers we all want these10Direct-Manipulation Interfaces6-11

1-# 2010 Pearson Addison-Wesley. All rights reserved. Indirect Example6-12

1-# 2010 Pearson Addison-Wesley. All rights reserved. Direct-Manipulation Interfaces

2-131-# 2010 Pearson Addison-Wesley. All rights reserved. Direct-Manipulation InterfacesCentral ideas:Visibility of the objects and actions of interestRapid, reversible, incremental actions Replacement of typed commands by a pointing action on the object of interest

6-141-# 2010 Pearson Addison-Wesley. All rights reserved. On one end we have command line, on the other hand we have the real world14A History of Text Editors (with demonstrations)Ed (command line)Emacs, vi, etc (display editor)WYSIWYG (word processor)6-151-# 2010 Pearson Addison-Wesley. All rights reserved. Examples of Direct-Manipulation SystemsCommand line vs. display editors and word processors Training times with display editors are much less than line editors Line editors were initially more flexible and powerful The advances of WYSIWYG word processors: Display a full page of text Display of the document in the form that it will appear when the final printing is done Show cursor action Control cursor motion through physically obvious and intuitively natural means Use of labeled icon for actions Display of the results of an action immediately Provide rapid response and display Offer easily reversible actions

6-161-# 2010 Pearson Addison-Wesley. All rights reserved. Examples of Direct-Manipulation Systems6-17

Text editors / word processing software1-# 2010 Pearson Addison-Wesley. All rights reserved. Remember the example of curb cuts1-18

1-# 2010 Pearson Addison-Wesley. All rights reserved. Technologies that derive from the word processor: Integration of mediaDesktop publication software Slide-presentation software Hypermedia environments Improved macro facilities Spell checker and thesaurus Grammar checkers

6-191-# 2010 Pearson Addison-Wesley. All rights reserved. Since it is an important problem, it was well-studied. These are useful things other software also use. Remember the curb cut example!19Examples of Direct-Manipulation Systems (cont.)The VisiCalc spreadsheet and its descendants(Excel, etc)VisiCalc users delighted in watching the program propagate changes across the screen. 6-201-# 2010 Pearson Addison-Wesley. All rights reserved. Examples of Direct-Manipulation Systems (cont.): spreadsheet6-21

1-# 2010 Pearson Addison-Wesley. All rights reserved. Examples of Direct-Manipulation Systems (cont.)Spatial Data ManagementIn some cases, spatial representations provide a better model of reality (e.g., Google maps)Successful spatial data-management systems depend on choosing appropriate: Icons Graphical representations Natural and comprehensible data layouts

6-221-# 2010 Pearson Addison-Wesley. All rights reserved. Examples of Direct-Manipulation Systems (cont.) spatial data management

6-23

1-# 2010 Pearson Addison-Wesley. All rights reserved. Examples of Direct-Manipulation Systems (cont.)Video games Great example for direct manipulationGames require pleasant and easy experiences. Direct-manipulation interfaces can provide such experiences

6-241-# 2010 Pearson Addison-Wesley. All rights reserved. 24Examples of Direct-Manipulation Systems (cont.)Video games Nintendo Wii, Sony PlayStation, Microsoft Xbox, etc.Field of action is visual and compelling Commands are physical actions whose results are immediately shown on the screen No syntax to remember Most games continuously display a scoreDirect manipulation in SimCitySecond Life virtual worldSpore, WoW, Sims, etc.6-251-# 2010 Pearson Addison-Wesley. All rights reserved. 25Examples of Direct-Manipulation Systems (cont.)Video games

6-26

1-# 2010 Pearson Addison-Wesley. All rights reserved. Battleship, mud26Examples of Direct-Manipulation Systems (cont.)Guitar Hero video game 6-27

1-# 2010 Pearson Addison-Wesley. All rights reserved. Examples of Direct-Manipulation Systems (cont.)Computer-aided design 6-28

1-# 2010 Pearson Addison-Wesley. All rights reserved. Examples of Direct-Manipulation Systems (cont.)Computer-aided design Computer-aided design (CAD) use direct manipulationManipulate the object of interestGenerate alternatives easilyExplain the impactProblem solving by analogy to the real-world

6-291-# 2010 Pearson Addison-Wesley. All rights reserved. Examples of Direct-Manipulation Systems (cont.)Office automationXerox Star was a pioneer with sophisticated formattingApple Lisa SystemRapid and continuous graphical interactionMicrosoft Windows is a descendant

6-301-# 2010 Pearson Addison-Wesley. All rights reserved. Continuing evolution of Direct-Manipulation SystemsDirect-Manipulation interfaces are being used in a wide range of applications, e.g. management dashboard for a retail store6-31

1-# 2010 Pearson Addison-Wesley. All rights reserved. Continuing evolution of Direct-Manipulation Systems (cont.)6-32

1-# 2010 Pearson Addison-Wesley. All rights reserved. Discussion of Direct ManipulationSuccessful direct manipulation Present appropriate representation or model of realityDo not have to mimic the real worldCan use accepted metaphors insteadShould enrich the experience and make it intuitiveShould require minimal effort6-331-# 2010 Pearson Addison-Wesley. All rights reserved. Discussion of Direct ManipulationProblems with direct manipulation Spatial or visual representations can be too spread out High-level flowcharts and database-schema can become confusing Designs may force valuable information off of the screen Users must learn the graphical representations The visual representation may be misleading Typing commands with the keyboard may be faster

6-341-# 2010 Pearson Addison-Wesley. All rights reserved. Calendar exampleType in, select from combos, choose on calendarType in could be superior with many random dates

34Principles of Direct ManipulationContinuous representations of the objects and actions of interest with meaningful visual metaphors.Actions that resemble physical actions, instead of complex syntax.Rapid, incremental, reversible actions whose effects on the objects of interest are visible immediately.6-351-# 2010 Pearson Addison-Wesley. All rights reserved. Successful Use of Direct ManipulationNovices can learn quickly, usually through watching demonstrationsExperts can work rapidly without limitationKnowledgeable users can retain usageError messages are rarely neededUsers can immediately see if they are making progress towards their goals and can quickly change direction if notUsers have less anxiety because they can undoConfidence, mastery, feelings of control, ability to predict what will happen6-361-# 2010 Pearson Addison-Wesley. All rights reserved. Hyundai Equus Car Seat Controller6-37

1-# 2010 Pearson Addison-Wesley. All rights reserved. 6-38

1-# 2010 Pearson Addison-Wesley. All rights reserved. Multitouch mobile devices6-39

1-# 2010 Pearson Addison-Wesley. All rights reserved. OverviewDirect ManipulationVirtual Environments6-401-# 2010 Pearson Addison-Wesley. All rights reserved. 3D Interfaces

6-411-# 2010 Pearson Addison-Wesley. All rights reserved. 3D InterfacesPure 3D interfaces have strong utility in some contexts, e.g., medical, product design. In other situations, more constrained interaction may actually be preferable to simplify interactions.

Enhanced interfaces, better than reality, can help reduce the limitations of the real-world, e.g., providing simultaneous views.

Avatars in multiplayer 3D worlds and 3D games are a common usage of 3D interfaces6-421-# 2010 Pearson Addison-Wesley. All rights reserved. 3D Interfaces6-43

1-# 2010 Pearson Addison-Wesley. All rights reserved. 3D Interfaces (cont.)Features for effective 3D Use occlusion, shadows, perspective, and other 3D techniques carefully.Minimize the number of navigation steps for users to accomplish their tasks.Keep text readable.Avoid unnecessary visual clutter, distraction, contrast shifts, and reflections.Simplify user/camera movement.Simplify object movementOrganize groups of items in aligned structures to allow rapid visual search.Enable users to construct visual groups to support spatial recall. 6-441-# 2010 Pearson Addison-Wesley. All rights reserved. 3D Interfaces (cont.)Guidelines for inclusion of enhanced 3D features:Provide overviews so users can see the big pictureOffer X-ray vision so users can see into or beyond objects.Provide history keepingPermit rich user actions on objectsAllow teleoperationEnable remote collaborationGive users control over explanatory text and let users select for details on demand.Offer tools to select, mark, and measure.

6-451-# 2010 Pearson Addison-Wesley. All rights reserved. 3D Interfaces (cont.)Guidelines for inclusion of enhanced 3D features (cont.):Implement dynamic queries to rapidly filter out unneeded items.Support semantic zooming and movementEnable landmarks to show themselves even at a distanceAllow multiple coordinated viewsDevelop novel 3D icons to represent concepts that are more recognizable and memorable.

6-461-# 2010 Pearson Addison-Wesley. All rights reserved. TeleoperationControlling devices remotely and intuitively

Two parents: direct manipulation in personal computers and process control in complex environmentsPhysical operation is remote, observation is local6-47

1-# 2010 Pearson Addison-Wesley. All rights reserved. TeleoperationComplicating factors in the architecture of remote environments: Time delays transmission delays operation delays Incomplete feedback Feedback from multiple sources Unanticipated interferences

6-481-# 2010 Pearson Addison-Wesley. All rights reserved. Virtual and Augmented Reality6-49

1-# 2010 Pearson Addison-Wesley. All rights reserved. Lets watch the Google glass video49Virtual and Augmented RealityVirtual reality breaks the physical limitations of space and allow users to act as though they were somewhere else Augmented reality shows the real world with an overlay of additional informationSituational awareness shows information about the real world that surrounds you by tracking your movements in a computer model 6-501-# 2010 Pearson Addison-Wesley. All rights reserved. Virtual and Augmented Reality (cont.)Successful virtual environments depend on the smooth integration of: Visual Display Head position sensing Hand-position sensing Force feedback Sound input and output Other sensations Cooperative and competitive virtual reality

6-51

1-# 2010 Pearson Addison-Wesley. All rights reserved. Impact of this technology in our everyday lives6-52

1-# 2010 Pearson Addison-Wesley. All rights reserved. TODO Homework 5Improve your prototypes from HW4 by yourself. Use things that we learned in Heuristic Evaluation, as well as your own judgment. Conduct usability tests with two users. Each test should include at least these:Conceptual model extraction (before and after use)Talk-aloud usability testVideo recording and a retrospective testing interview (that is also recorded)Report findings and improvements to be done in the next iteration

Submission:E-mail to gazihan.ieu@gmail.com with subject [IES 506] HW5

Deadline: 14 May

6-53