multimodal, crossmedia, multi platform

49
INPUT MODALITIES UXD minor theme ‘Multimodal, Crossmedia and Multi-Platform

Upload: hans-kemp

Post on 23-Dec-2014

568 views

Category:

Documents


1 download

DESCRIPTION

 

TRANSCRIPT

Page 1: Multimodal, Crossmedia, Multi Platform

INPUT MODALITIESUXD minor theme ‘Multimodal, Crossmedia and Multi-Platform

Page 2: Multimodal, Crossmedia, Multi Platform

Theme program

March 23: ‘Input modalities’ (Hans), workshop and assignment kick-off

March 30: ‘Output modalities’ (Rolf) and assignment progress

April 6: Workshop with Pieter Jongerius (Fabrique)

April 13: No class, EasterApril 20: Final presentations assignment

Page 3: Multimodal, Crossmedia, Multi Platform

Theme in the scheme of things

Media, modalities and platforms provide us the nuts and bolts of the user experience.

The quality of the user experience is determined by our ability to utilize the media, modalities and platforms at our disposal.

Page 4: Multimodal, Crossmedia, Multi Platform

Crossmedia

‘Crossmedia (also known as Cross-Media, Cross-Media Entertainment, Cross-Media Communication) is a media property owned, service, story or experience distributed across media platforms using a variety of media forms.’

http://en.wikipedia.org/wiki/Crossmedia

Page 5: Multimodal, Crossmedia, Multi Platform

Multi-platform

‘In computing, cross-platform (also known as multi-platform) is a term used to refer to computer software or computing methods and concepts that are implemented and inter-operate on multiple computer platforms.’

http://en.wikipedia.org/wiki/Multiplatform

Page 6: Multimodal, Crossmedia, Multi Platform

Multimodal

‘Multimodal interaction provides the user with multiple modes of interfacing with a system beyond the traditional keyboard and mouse input/output.’

http://en.wikipedia.org/wiki/Multimodal_interaction

Page 7: Multimodal, Crossmedia, Multi Platform

Modality

‘A modality is a path of communication between the human and the computer.’

http://en.wikipedia.org/wiki/Modality_(human-computer_interaction)

Page 8: Multimodal, Crossmedia, Multi Platform

Input modalities and output modalities

‘In human-computer interaction, a modality is the general class of: a sense through which the human can receive the

output of the computer (for example, vision modality)

a sensor or device through which the computer can receive the input from the human’

http://en.wikipedia.org/wiki/Modality_(human-computer_interaction)

Page 9: Multimodal, Crossmedia, Multi Platform

Output modalities (computer-to-human)

‘Any human sense can be translated to a modality:

Major modalities Seeing or vision modality Hearing or audition modality

Haptic modalities Touch, tactile or tactition modality — the sense of pressure Proprioception modality — the perception of body awareness

Other modalities Taste or gustation modality Smell or olfaction modality Thermoception modality — the sense of heat and the cold Nociception modality — the perception of pain Equilibrioception modality — the perception of balance’

http://en.wikipedia.org/wiki/Modality_(human-computer_interaction)

Page 10: Multimodal, Crossmedia, Multi Platform

Input modalities (human-to-computer)

An input device is any peripheral (piece of computer hardware equipment) used to provide data and control signals to an information processing system (such as a computer).

http://en.wikipedia.org/wiki/Input_devices

Page 11: Multimodal, Crossmedia, Multi Platform

Pointing devices

Ivan Sutherland (MIT) demoing Sketchpad (1962)(introduced by Alan Kay in 1987)

Page 12: Multimodal, Crossmedia, Multi Platform

Pointing devices

‘Pointing devices are input devices used to specify a position in space. Direct/indirect Absolute/relative’

http://en.wikipedia.org/wiki/Input_devices

Page 13: Multimodal, Crossmedia, Multi Platform

Fitts’ law

‘The time it takes to move from a starting position to a final target is determined by the distance to the target and the size of the object.’ (Saffer, 2007)

Page 14: Multimodal, Crossmedia, Multi Platform

Pointing devices

And you can point at more than merely pixels on a screen…

Page 15: Multimodal, Crossmedia, Multi Platform

Alphanumeric input: keyboards

Page 16: Multimodal, Crossmedia, Multi Platform

Alphanumeric input: keyboards

Page 17: Multimodal, Crossmedia, Multi Platform

Alphanumeric input: keyboards

Page 18: Multimodal, Crossmedia, Multi Platform

Alphanumeric input: speech recognition

Speaker dependent/independentDiscrete-word/connected-word inputLimited/large vocabulary

Page 19: Multimodal, Crossmedia, Multi Platform

Alphanumeric input: handwriting recognition

‘Recognition’ patents as early as 1914

‘Electronic ink’ and recognition in Vista

http://www.freepatentsonline.com/1117184.pdf

Page 20: Multimodal, Crossmedia, Multi Platform

Pen Computing

‘The return of the pen’Switching modes:

‘pointing’ vs. ‘ink’

Page 21: Multimodal, Crossmedia, Multi Platform

Tap is the New Click

"One of the things our grandchildren will find quaintest about us is that we distinguish the digital from the real.“

William Gibson - from: Saffer (2009)

Page 22: Multimodal, Crossmedia, Multi Platform

Ubiquitous computing

‘Ubiquitous computing (ubicomp) is a post-desktop model of human-computer interaction in which information processing has been thoroughly integrated into everyday objects and activities.’

http://en.wikipedia.org/wiki/Ubiquitous_computing

Page 23: Multimodal, Crossmedia, Multi Platform

Wearable computing

‘Wearable computers are computers that are worn on the body.’

http://en.wikipedia.org/wiki/Wearable_computer

Page 24: Multimodal, Crossmedia, Multi Platform

Tangible user interfaces

Hiroshi Ishii (MIT)

Page 25: Multimodal, Crossmedia, Multi Platform

Sketching Mobile Experiences

Workshop in ‘Design This!’

Page 26: Multimodal, Crossmedia, Multi Platform

Gestural Interfaces

Touchscreen vs. Free-form

Page 27: Multimodal, Crossmedia, Multi Platform

Ergonomics of Interactive Gestures

"Hands are underrated. Eyes are in charge, mind gets all the study, and heads do all the talking. Hands type letters, push mice around, and grip steering wheels, so they are not idle, just underemployed."

—Malcolm McCullough, Abstracting Craft(from: Saffer, 2009)

Page 28: Multimodal, Crossmedia, Multi Platform

Ergonomics of Interactive Gestures

Limitations due to anatomy, physiology and mechanics of the human body (kinesiology)

Left-handedness (7-10%) Fingernails Screen Coverage

Page 29: Multimodal, Crossmedia, Multi Platform

Designing Touch Targets

No smaller than 1x1cm in an ideal world

In a not so ideal world:Iceberg TipsAdaptive Targets

Page 30: Multimodal, Crossmedia, Multi Platform

Designing Touch Targets

But even spaciously sized targets can be tricky…

Page 31: Multimodal, Crossmedia, Multi Platform

Patterns for Touchscreens and Interactive Surfaces

Tap to open/activate

Page 32: Multimodal, Crossmedia, Multi Platform

Patterns for Touchscreens and Interactive Surfaces

Tap to select

Page 33: Multimodal, Crossmedia, Multi Platform

Patterns for Touchscreens and Interactive Surfaces

Drag to move object

Page 34: Multimodal, Crossmedia, Multi Platform

Patterns for Touchscreens and Interactive Surfaces

Slide to scroll

Page 35: Multimodal, Crossmedia, Multi Platform

Patterns for Touchscreens and Interactive Surfaces

Spin to scroll

Page 36: Multimodal, Crossmedia, Multi Platform

Patterns for Touchscreens and Interactive Surfaces

Pinch to shrink and spread to enlarge

Page 37: Multimodal, Crossmedia, Multi Platform

Patterns for Free-Form Interactive Gestures

Proximity activates/deactivates

Page 38: Multimodal, Crossmedia, Multi Platform

Patterns for Free-Form Interactive Gestures

Point to select/activate

Page 39: Multimodal, Crossmedia, Multi Platform

Patterns for Free-Form Interactive Gestures

Rotate to change state

Page 40: Multimodal, Crossmedia, Multi Platform

Patterns for Free-Form Interactive Gestures

Shake to change

Page 41: Multimodal, Crossmedia, Multi Platform

Patterns for Free-Form Interactive Gestures

Tilt to move

Page 43: Multimodal, Crossmedia, Multi Platform

Reader

Wearable computers:Steve Mann. Eyetap.org. http://about.eyetap.org/

Ubiquitous computing: Mark Weiser (1991). The Computer for the 21st Century.

http://www.ubiq.com/hypertext/weiser/SciAmDraft3.htmlAdam Greenfield (2006). Everyware: The Dawning Age of

Ubiquitous Computing. New Riders, Berkeley, CA.Donald Norman (1998). The Invisible Computer: Why Good

Products Can Fail, The Personal Computer Is so Complex, and Information Appliances Are the Solution. The MIT Press, Cambridge, MA

Page 44: Multimodal, Crossmedia, Multi Platform

Reader

Input devicesDoug Engelbart (1968). The mother of all demos.

Google video streamWikipedia.

http://en.wikipedia.org/wiki/The_Mother_of_All_Demos

Page 45: Multimodal, Crossmedia, Multi Platform

Reader

Fitts’ LawDan Saffer (2007). Designing for Interaction: Creating

Smart Applications and Clever Devices. New Riders, Berkeley, CA. (page 53)

Speech recognitionMicrosoft. Microsoft Speech Technologies.

http://www.microsoft.com/speech/speech2007/default.mspx

Page 46: Multimodal, Crossmedia, Multi Platform

Reader

Handwriting recognitionWacom. Unleash Windows Vista With A Pen.

http://www.wacom.com/vista/index.php

Gestural InterfacesDan Saffer (2009). Designing Gestural Interfaces. O’Reilly Media,

Sebastopol, CA

ErgonomicsHenry Dreyfuss (1955). Designing for People. Allworth Press,

New York, NY.

Page 47: Multimodal, Crossmedia, Multi Platform

Theme assignment

Page 48: Multimodal, Crossmedia, Multi Platform

Today’s workshop assignment

Work together in teams of 2-3 students on one input device Each team will be investigating the following:

What’s the typical application of this device? What are typical patterns applied with this device? How can this device connect to a computer? What driver or other software is available for this device? How can I adjust the parameters of this device? How can I create application prototypes with this device?

Build a simple demonstrator for the device, using your laptop computer

Analyze the user experience with your demonstrator Present your demonstrator at the end of the afternoon Document your findings in a pdf document Link the document to a post on your blog

Page 49: Multimodal, Crossmedia, Multi Platform

Today’s workshop assignment

Available devices Touch screen (2) Wii mote (4) Xbox USB controller (2) Wacom (3) Web cam (5) SpaceNavigator (1) Presenter (3) Smartboard (1) iPhone (?)