1

Recommendations for Multi-Touch

Screen Design

Written by

Holly Peters, Rachel Gastrich, and Lyford Rome

Department of Educational Technology

San Diego State University

March 25, 2011

Prepared for the Museum of Photographic Arts

Balboa Park, San Diego, CA

2

Table of Contents Executive Summary ...................................................................................................................... 4 

1. Introduction ............................................................................................................................... 5 

1.1 Context ............................................................................................................................................... 5 

1.2 Description of Task ............................................................................................................................ 5 

1.3 Organizational Overview .................................................................................................................... 6 

2. Analysis Process ........................................................................................................................ 7 

2.1 Contextual Factors, Issues, and Concerns .......................................................................................... 7 

2.2 Background Research ......................................................................................................................... 8 

2.3 Survey ................................................................................................................................................. 8 

2.4 Site Visits ............................................................................................................................................ 9 

2.5 Literature Review ............................................................................................................................... 9 

3. What is Multi-Touch Technology? ........................................................................................ 10 

3.1 Definition .......................................................................................................................................... 10 

3.2 History .............................................................................................................................................. 10 

3.3 Variations.......................................................................................................................................... 11 

3.4 Methods of Interaction ...................................................................................................................... 13 

3.5 Advantages ....................................................................................................................................... 15 

4. Findings .................................................................................................................................... 16 

4.1 Survey Results .................................................................................................................................. 16 

4.2 Site Visits .......................................................................................................................................... 16 

4.3 Literature Review ............................................................................................................................. 19 

5. Implications and Recommendations for Design .................................................................. 21 

5.1 Physical Installation – Environmental Factors ................................................................................. 21 

5.2 Interaction Mechanics ....................................................................................................................... 22 

5.3 Designing for Groups and Multiple Users ........................................................................................ 25 

5.4 Designing for Various Age Groups .................................................................................................. 27 

5.5 Spatial Organization ......................................................................................................................... 27 

5.6 Programming .................................................................................................................................... 28 

6. Conclusions .............................................................................................................................. 29 

3

Appendixes................................................................................................................................... 30 

Appendix A – References ....................................................................................................................... 30 

Appendix B – Survey Questions and Responses .................................................................................... 33 

Appendix C – Annotated Bibliography .................................................................................................. 39 

Appendix D – List of Figures ................................................................................................................. 46 

4

Executive Summary This report describes best practices in multi-touch screen interactions. The Museum of

Photographic Arts (MoPA) in San Diego is installing a multi-touch screen in their gallery to help

visitors think critically about photography, improve the experience of patrons visiting the

museum in groups, create a crowd-sourced exhibit, and increase website traffic by linking multi-

touch screen interaction to MoPA’s website. A consulting team of three graduate students in

Educational Technology at San Diego State University were tasked with researching best

practices in multi-touch screen design that could be applied to a museum setting. The consultants

researched the capabilities of multi-touch screens, conducted site visits at institutions with multi-

touch screens, surveyed several museums with multi-touch screens, and reviewed previously

conducted research and case studies. This analysis effort revealed several best practices for

designing interactions using multi-touch screens, especially in the areas of physical installation,

interaction mechanics, target audience, spatial organization, and programming.

5

1. Introduction

1.1 Context

This report was prepared by Holly Peters, Lyford Rome, and Rachel Gastrich, graduate

students in the Educational Technology department at San Diego State University in San

Diego, California. It is the final deliverable for the ED 795A/B capstone course series

that students take to complete the Educational Technology Master’s degree. The team

partnered with Amber Lucero-Criswell, Director of Education and Public Programs, at

the Museum of Photographic Arts (MoPA) in San Diego, California to complete the

project, which culminated with this report. MoPA was founded in 1983 and is devoted to

collecting, conserving, and exhibiting the entire spectrum of the photographic medium.

The museum’s endeavors consistently address cultural, historical, and social issues

through its exhibitions and public programs. Approximately 100,000 people visit

MoPA’s museum and website annually.

1.2 Description of Task

As part of an Institute of Museum and Library Services (IMLS) grant, MoPA is installing

a multi-touch screen in their gallery entrance. The multi-touch screen will be vertically

mounted in the gallery entry, and will be visible when visitors enter the museum. By

installing the multi-touch screen, MoPA hopes to:

Help visitors understand and think critically about photography

Improve the experience of patrons visiting the museum in groups

Create a crowd-sourced exhibit through visitor selection of photos

Increase website traffic by linking the multi-touch screen interaction to

MoPA’s website

To this effort, the consultants were tasked with analyzing best practices in visitor

interactions with multi-touch screens at museums. Rather than identifying potential

content areas, the consultants researched general guidelines for multi-touch screen use in

museums that are applicable to all content areas.

6

1.3 Organizational Overview

This report is organized into five sections:

Introduction – An overview of the context, description of the task, and

an organizational overview

Analysis Process – An overview of the analysis process used by the

consultants

What is Multi-Touch Technology? – Definition, advantages, variations,

software, and methods of interactions specific to multi-touch technology

Findings – Results of research by the consultants

Implications and Recommendations for Design – Recommendations

based on best practices for MoPA to implement in their multi-touch

screen activities

7

2. Analysis Process

2.1 Contextual Factors, Issues, and Concerns

The recommendations in this report are based on the best research available to the

consultants during the time allotted to complete the appropriate analysis. The reader

should consider the following contextual factors, issues, and concerns that may have

influenced the research results.

New Technology

One factor that influenced our research is the sheer novelty of the topic. Not many

institutions have successfully incorporated multi-touch technology into their operations.

There are several possible factors contributing to the slow adoption of this technology

including cost, available space, unfamiliarity, lack of resources, and not recognizing a

need within their particular audience. The limited amount of multi-touch implementations

resulted in a narrow pool of reference material and led the consultant team to draw from a

wider variety of resources. In addition to museums, we also considered information from

other sources including retail stores and public spaces.

Different Orientation

Among the institutions that have implemented multi-touch technology, we have found

that the majority are in a horizontal orientation (table-style format). This format is

significantly different from the format that MoPA will use in several ways. With a

vertical format, users will physically interact with the screen in a standing position, view

the content at eye-level, and be able to see what other users are doing simultaneously. We

are confident, however, that examining the use of horizontal multi-touch screens yields

valuable information about best practices that can be applied to vertical orientations as

well.

Low Survey Response Rate

Of the thirteen institutions with multi-touch technology that we contacted and distributed

surveys to, only six returned a survey. Of these six, only three were complete. However,

the responses provided by the three respondents supported other research findings we

incorporated into our recommendations.

8

Single Observer for Site Visits

One person of the consulting team conducted on site observations at locations with multi-

touch technology. It is possible that a single observer may unconsciously distort data in

such a way as to make certain outcomes more likely, observing from their individual

perspective (gender, age, ethnicity, or other characteristics). The effect of this threat to

validity was reduced because the research team did not have a pre-determined hypothesis

that they were trying to prove. Also, the same team member observed each of the three

sites so that the findings would be as consistent as possible.

Financial and Temporal Limitations

While we would have liked to travel to other states and countries where multi-touch

technology is successfully used, we did not have the time or budget to do so. The team

also acknowledges that it would have been more desirable to visit each of the museums

we observed on more than one occasion, but our schedule could not accommodate this

luxury. In spite of any limitations, we believe that the time and effort invested into this

project was sufficient to produce a strong basis for our recommendations.

2.2 Background Research

Our analysis process began with background research to familiarize ourselves with multi-

touch technology, including its technical aspects and uses. We researched vendor sites,

including MultiTouch (www.multitouch.fi) and Ideum (www.ideum.com), to understand

the basic concepts behind multi-touch technology. Case studies linked on MultiTouch

and Ideum’s websites provided insight into the use of multi-touch screens in both

institutional and commercial settings. Although none of the case studies featured

museums, it was helpful to have an overview of topic.

2.3 Survey

At the beginning of the project, MoPA provided the consultants with a list of museums

using multi-touch technology. The team contacted each museum on the list to identify the

correct contact for further communication. Some museums were eliminated from that list

as they did not have a multi-touch screen, only had regular touch screens, or were using

their screens for research rather than with the general public. Thirteen organizations

confirmed that they had multi-touch screens and offered specific contact information. The

consultants designed a brief survey hosted on SurveyMonkey to poll the institutions

about their experience with their multi-touch screen(s). Six of the thirteen museums that

9

received the survey responded, including museums located in Sweden; Nashville, TN;

Anchorage, AL; Greenbelt, MD; and Vancouver, BC. Only three of the six, however,

returned a completed survey, with all the questions answered. Although the lack of

response was disappointing, some of the insights and experiences shared on the survey

proved to be helpful in our research. See Appendix II for a complete list of survey

questions and responses.

2.4 Site Visits

A member of the consulting team visited the following three locations that currently use

multi-touch technology:

The Microsoft Store (San Diego, CA) – This is a retail store owned and

operated by Microsoft, dealing in computers, computer software, and

consumer electronics.

The J. Paul Getty Museum (Los Angeles, CA) – The Getty Museum

houses European paintings, drawings, sculpture, manuscripts, decorative

arts, and European and American photographs. Their collection attracts a

wide variety of audiences spanning all age groups, education levels, and

nationalities.

The Annenberg Space for Photography (Los Angeles, CA) – The

home office need to provide support and direction during the rollout.

2.5 Literature Review

Finally, we conducted a literature review of best practices in multi-touch technology. We

reviewed articles, theses, and other research projects about multi-touch screens found

using Google, Google Scholar, and the Educational Resources Information Center

(ERIC). Though not all of the literature we reviewed focused on multi-touch screens in a

museum context, many of the findings were easily applicable to a broad range of multi-

touch screen uses. Our literature review provided us with a significant amount of useful

information.

10

3. What is Multi-Touch Technology?

3.1 Definition

The popularity of devices such as Apple’s iPad and iPhone, as well as the many new

tablets and touch screen computers, reflect the growing presence of touch technology in

day to day life. The phrase “multi-touch technology”, however, refers to any one of

several different interaction techniques and technologies that allow users to control a

computer using several fingers, or multiple inputs. These distinct approaches all utilize

different hardware and software, each with distinct advantages and disadvantages.

For the purposes of this report, we will define multi-touch as a system’s ability to register

more than three touch points directly on a display. Each input is detected and resolved

individually, even with multiple users. This greatly increases the speed, efficiency, and

intuitiveness of the technology. Our report will focus on best practices at user interaction

level, independent of a particular platform or vendor.

Figure 3.1: A vertical multi-user, multi-touch screen being used as an interactive directory at the Pori Jazz Festival. Image courtesy of Multitouch, Ltd.

3.2 History

Touch screen technology was first developed in the late 1960’s. Previous systems

allowed interaction with the computer screen using a light pen, but by 1972 technology

11

had advanced enough to enable a human finger to register input directly. This system was

developed by the Computer-based Education Research Laboratory at the University of

Illinois, Urbana-Champaign, and was called the PLATO IV Touch Screen Terminal

(Buxton, 2007). PLATO IV and other technology of this period recognized one single

touch at a time in separate zones on the screen.

By the mid 1980’s, systems had been developed to recognize multiple touches

simultaneously. Bob Boie at Bell Labs created the first multi-touch screen in 1984 that

used a transparent array of touch sensors overlaid on a cathode ray tube (CRT) screen

(Buxton, 2007).

In 1992, SUN Microsystems produced the Starfire demo, which imagined a day in the life

of a knowledge worker in the “far-off distant year of 2004.” This demonstration

introduced several multi-touch gestures and techniques (Tognazzini, 1994).

These ideas did not gain widespread adoption and remained in research labs until 2006

when NYU’s Jeffery Han presented a now-famous demonstration at the Technology,

Entertainment and Design (TED) Talks (Han, 2006). His system was relatively simple to

create as well as inexpensive, and it opened the door for much of the innovation we see

today.

3.3 Variations

With the advent of lower cost and more capable systems in the early 21st century, multi-

touch screens are now installed in many different settings and for as many different

purposes.

Modern multi-touch screens are used effectively in diverse environments such as the

military for command and control (Szymanski, 2008), science for data visualization and

collaboration (Wigdor, 2009), educational outreach for museums (Johnson, 2010), and

for business and data management applications (Morris, 2008).

Multi-touch systems are available in many form factors and sizes. Most of the literature

surveyed in our review described standalone table top systems, such as the Microsoft

Surface. A horizontal orientation permits more ergonomic virtual keyboard input and also

allows for the interaction with physical objects placed upon the table surface (Experience,

2011). A variation of the horizontal system is an angled, “drafting table” style surface.

This was the type first demonstrated by Jeffery Han in his 2006 TED Talk and was

described in an office setting by Morris (2009).

12

Figure 3.2: A multi-touch screen being used in a table format and with a stand for an angled horizontal orientation. Images courtesy of Multitouch, Ltd.

Multi-touch screens in a vertical orientation can offer their own unique advantages.

Large, wall-mounted screens are visible from far away, while horizontal screens require

the user to stand close by. Vertical screens are easier to read when displaying information

(Wimmer, 2009) and can be built to large sizes. Using modular units, a 4-meter-long

(13’) by 1.2-meter-high (4’) fully multi-touch video wall was constructed for the Chilean

Pavilion Wall of Chile display at Expo Shanghai 2010 (Multitouch.fi, 2010), for example.

Figure 3.3: A modular wall mounted cell-based design allows scalability. Image courtesy of Multitouch, Ltd.

MoPA has chosen to install a vertically oriented multi-touch screen in their front gallery.

The hardware technologies used to recognize multi-touch input vary by vendor and form

factor (Schöning, 2008). It is beyond the scope of this report to cover all the technical

details of the many different methods currently on the market. Rather, we will focus on

13

the software layer and the forms of interaction enabled by most modern multi-touch

displays.

3.4 Methods of Interaction

Despite multi-touch being a relatively newer method of interacting with a computer

display, there is already an emerging vocabulary of gestures that are standard on many

devices. A true multi-touch system does not just replace the mouse and keyboard with on

screen equivalents; on many such systems there is no cursor to move with a mouse at all.

Instead, a gestural “sign language” is used to interact with the display. Some of these

gestures are very intuitive as they mimic how one would naturally move, while some are

completely new to the experience of multi-touch.

Single Finger Tap or Touch

The simplest interaction with the screen is a single finger tap at a single point. This

gesture would be appropriate to activate button elements on screen, or any other status

controls such as “on/off” or “mute/unmute.” This is one of the most intuitive of all

gestures (Jacucci, 2010).

Single Finger Hold and Drag

The user holds down a single finger touch and then drags the finger across the screen.

This action can be interpreted different ways depending upon the screen element being

controlled (Malik, 2007). The movement can either be linear or rotational. This gesture is

appropriate for slider type controls such as volume or brightness, scrolling through lists

or mimicking rotating knobs. It is also used for moving onscreen objects, such as photos.

Single Finger Flick, Fling, or Swipe

This gesture resembles the single finger hold and drag but with a much faster motion.

This gesture can be interpreted by the system to mean the same as hold and drag but with

greater velocity, such as scrolling all the way to end of a list very quickly or moving a

photo off the screen.

Depending upon the physics of the software, this gesture could also be used to overcome

the “inertia” of some interface elements. Slower movements mean that the user is

intending to interact with what is displayed on screen, a fast swipe could tell the software

to switch to another view or part of the program.

14

Multiple Finger Touch, Drag, or Swipe

Variations of these gestures include using multiple fingers instead of a single touch.

Though, technically, these would be multi-touch gestures, all the fingers are performing

the same motion relative to each other. With a two finger swipe, both fingers would be

“swiping” in the same direction.

Figure 3.4: Examples of simple gestures: Single Touch, Multiple Touch & Drag, and Single Touch & Flick.

Multi-Touch Rotate/Spin

Rotating a two finger hold and drag around a common center will rotate or spin the

selected screen element. This could be a one or two handed gesture depending upon the

size of the element.

Multi-Touch Pinch/Shrink/Zoom Out

Holding two fingers and dragging them towards one another is interpreted by many

systems as a zoom out, shrinking whatever on screen element has been selected. This

may be performed with one or two hands, or with one stationary finger; what is important

is the motion relative to each point. If the motion is rotational as well, then the element

would be spun as well as shrunk (Malik, 2007).

Multi-Touch Unpinch/Stretch/Zoom In

Two fingers holding and dragging away from each other is the opposite of a pinch. This

gesture will zoom in or enlarge whatever element is selected. Again, the relative motion

between the fingers is what defines this gesture.

15

Figure 3.5: Examples of multi-touch gestures: Pinch/Zoom Out, Rotate/Spin, and Unpinch/Zoom In/Rotate combination.

3.5 Advantages

Multi-touch technology offers several advantages to museums over traditional kiosk style

installations (Multitouch.fi, 2011), such as:

The popularity of devices such as the iPad and touch screen phones has

raised the expectations of museum visitors who are drawn to the new

technology.

Modular multi-touch screens can be configured to make wall displays of

very large size, which become an integral part of exhibit experiences.

The size of wall mounted screens changes the relationship of the user to

the content. The visitor’s field of view is surrounded by the presentation.

Multi-touch displays by their nature allow for multiple users to interact at

the same time.

Multi-touch screens allow the visitor to interact more naturally with

onscreen content by allowing users to experience the presentation more

directly.

Visitors have a more memorable experience because they use more

senses than just sight.

16

4. Findings

4.1 Survey Results

Despite the low number of survey respondents, the information from our survey provided

several useful multi-touch screen practices. To summarize the findings:

Five out of six respondents used Ideum multi-touch screens.

Several of the respondents had multiple multi-touch screens; most were

oriented in the table-top format.

Most multi-touch screens were configured separately, rather than as one

large screen.

Visitors enjoyed interacting with screens that caught their eye, and liked

the relationship between exhibits and the screen.

Visitors preferred simple rather than complex interactions.

Interactions should be tested thoroughly before they are implemented.

Children tend to dive right into screen interactions, young adults

approach screen interactions with purpose, and older adults are more

cautious when interacting with multi-touch screens.

A complete list of survey questions and responses can be found in Appendix II.

4.2 Site Visits

The Microsoft Store

Our team first conducted a site visit at the Microsoft Store, where there are two horizontal

multi-touch screens built into sturdy table units. The units use Microsoft Surface

technology, a software and hardware combination product developed by Microsoft that

responds to natural hand gestures and real world objects placed on the screen by multiple

users. The platform features a 360-degree user interface on a 30 inch (76 cm.) reflective

surface. A projector underneath the surface projects an image onto its underside, while

five cameras in the machine's housing record reflections of infrared light from objects

and human fingertips on the surface.

17

A store employee demonstrated the basic functions of the multi-touch unit. He showed

how the technology allows for multiple points of contact to be recognized simultaneously

by asking two other people to place their fingertips on the screen with him all at the same

time. Each individual point of contact lit up on the screen. In addition, the screen was

able to detect and respond to physical objects such as a cup or identification card held

against the screen. The unit held games as well as applications for drawing and viewing

images. One of the two units in the store was not used as much as the other because,

according to the store employee, it was located by a window and the glare from the sun

caused problems.

The Getty Museum

Upon arriving at the Getty Museum, our team member realized that the Getty only had

single touch screens, not multi-touch screens. This means that each screen would

accommodate only one user and one input at a time instead of multiple users and inputs.

We did, however, learn valuable information as a result of the visit. Some of the screens

were in a horizontal table-top format, with one or two headphones attached, and located

in several locations throughout the museum, such as hallways and a computer room.

Additionally, units in the computer room were isolated from the rest of the museum

exhibits, and some visitors appeared to be reluctant to even enter the dimly lit room. In

fact, our team member observed that upon entering the computer room, several visitors

seemed as if they were attracted more to the comfortable seating as a place to rest rather

than the computers, as evidenced by their audible sighs and lack of computer usage.

A few computers were set up with a choice of three children’s games, and quite a few

families (adults with children) used these computers. Without exception, the children

began to interact with the touch screens immediately, while some of the adults just

watched their children or looked at the screen for several minutes before touching it. An

interview with a docent confirmed that, in his experience, children run right up to the

screens and adults are more hesitant.

Other computers throughout the room, either at counter-top height or at tables, were set

up to view pieces from the museum’s collection. The interactions available on most of

the touch screen computers consisted of viewing works of art in the museum collection,

detailed information about the piece, or a map of where the item is located in the

museum. The initial screen for these computers contained the text “Touch screen to

begin,” to prompt users. There were also a few computers that showed a library of videos

18

the user could choose from. The only observable gesture the screens recognized was

tapping, just as one would use a mouse to click in various areas on the screen.

The Annenberg Space for Photography

The last site visit conducted by the team was at the Annenberg Space for Photography.

The space housed two identical multi-touch screen units located in a corner of the

museum halfway between the front entrance and the beginning of the main exhibition.

The two units were arranged side by side in front of comfortable sofa-like seats with

shelves between them for users to place their belongings on. One of the units was not

working at the time of the site visit. The units were built into low horizontal tables in

front of each seating unit, which required the user to sit at the edge of the seat and lean

forward at a 45% angle in order to have a clear view of the screen, looking straight down

on it.

Once activated by touch, the screen displayed the following message along with an image

of a hand touching objects on the screen: “Have fun with photos and videos as you move,

resize, and rotate them. Enjoying photos and videos is a hands-on activity!” Our team

member observed that initially, users were unsure how to manipulate the onscreen images

until a museum employee came by to show them. Our team later learned that museum

employees had very limited, mostly on-the-job training on the screens due to the intuitive

design of the interactions. Employees did need, however, to know that they should not

tell visitors that the screens are “the same as” an iPhone because the technology is

different. Instead, they are instructed that they can say that the screens are “a little like”

an iPhone.

Users could drag images across the screen, and enlarge or reduce them by dragging the

corners. This could be done by multiple users simultaneously. One aspect of the design,

however, proved to be challenging for multiple users. Unless images were sized and

spaced far enough away from each other, only one image was visible at a time. The

interaction was designed so an image only appeared on the top of the “pile” of other

images while the user maintained physical contact. Once the user removes their hand

from an image, that image recedes behind other images, specifically behind images that

another user may be touching. As a result, we observed that when a group used the

screen, just one person from the group would touch the screen while the others looked on.

A museum employee also explained some of the features of the units that were removed

and why. For example, visitors were originally able to download and view photos from

19

their personal cameras, and also send photos from the current exhibit to a printer at the

front desk of the museum so they could take them home. These features were removed

due to usage time concerns and copyright issues.

4.3 Literature Review

Our literature review resulted in a broad spectrum of uses for multi-touch technology.

These various examples can be categorized on a continuum made up of the following

three categories: activation, manipulation, and creation. (See Figure 4.1)

Figure 4.1: Continuum of three types of multi-touch technology uses: activation, manipulation, and creation.

Activation

The activation level of interaction occurs when the user touches an area of the screen in

order to initiate a simple function. The outcome is the same for every user. Activation

includes accessing information from the organization or from outside the organization,

i.e. the Internet.

Activating information from the organization includes tapping a screen to see a list of

choices, filtering a selection, viewing more information, showing a map, or playing a

video. An example of this can be found in a study done at the Vancouver Aquarium

(Hinrichs and Carpendale, 2010), where visitors activated two types of multi-touch

learning experiences. The first experience was an interaction that launched photos,

videos, and graphic animations related to the museum’s Arctic exhibit. The second was a

timeline activity where users could overlay and compare different data sets on a timeline

by activating different layers. We also found examples of users accessing exclusive

content by bringing objects such as membership cards or bar codes into contact with the

multi-touch screen.

Information from outside the organization included items such as images from Flickr, a

video from YouTube, a Twitter feed, or a Facebook discussion. Flickr integration was

20

also used in the CityWall project (Peltonen, 2008). Photos tagged with "Helsinki" were

displayed in the top portion of the screen and mapped to a scrollable timeline with

thumbnails. Jacucci (2010) organized similar content into rotating 3D widget "worlds"

that unfolded into 2D planes when users interacted with them.

Manipulation

Manipulation occurs when the user’s action results in a corresponding change of the

information on the screen. The change is in direct relation to the degree of the user’s

interaction. The outcome is one of several possible outcomes available. Examples of the

manipulation level of interaction are the user generating special effects (like blowing

virtual bubbles, blowing virtual leaves in the air, or making ripples in a virtual pond of

water), resizing and rotating images, or playing games.

Again, the Vancouver Aquarium study shows that both types of their multi-touch

learning experiences feature fairly low levels of manipulation; users played videos,

resized and rotated images, and activated layers on a timeline, as opposed to higher levels

of manipulation found in games and similar interactions. The CityWall project also

included similar levels of manipulation, with users scrolling the timeline, selecting and

enlarging photographs. Jacucci's 3D worlds provided a greater degree of interaction,

with different widgets opening up to different activities. Kristensson's InfoTouch system

(2008) allowed direct interaction and manipulation of photo tag cloud. Users could drag

specific tags and keywords from the cloud to an onscreen "query area." The results of that

search would be displayed from the photo collection. The interactive screen at the Pori

Jazz Festival (MultiTouch Case Studies, 2011) allowed patrons to access schedule

information and track their favorites musicians by dragging spheres representing venues,

genres and artists.

Creation

Creation is the highest level of interaction and is when the user develops new content.

The result is something unique that has never existed before. Instances of creation include

editing images, adding comments to a discussion, or drawing “graffiti” on a virtual wall.

21

5. Implications and Recommendations for Design Based on our research, we have compiled the following best practices for multi-touch

screen interaction design. These practices are not only instructionally sound, but can also

be implemented across a wide range of content areas for multi-touch screen interactions.

5.1 Physical Installation – Environmental Factors

The physical location of multi-touch screens can greatly affect users’ experiences.

Careful attention should be paid to the physical location of screens to avoid any

detrimental effects to visitors’ enjoyment of the interaction resulting from environmental

factors.

Avoid Sunlight Glare

Feedback and observations from the California Science Center and the Microsoft Store

indicated that some of their screens are not used due the glare from sunlight. Multi-touch

screens should be installed in areas that are protected from sunlight glare or conversely,

from shadows that may make the screen difficult to view. Before a multi-touch screen is

installed, its intended location should be reviewed thoroughly to ensure the screen will

not be in an area with a lot of sunlight or other adverse environmental factors.

Install Screen in Well-Traveled Area

A site visit to the Getty Museum indicated that multi-touch screens should also be

installed in areas well-travelled by visitors. Several of the screens at the Getty Museum

are located in a side room off the main gallery, and many visitors would peek into the

room, but not actually enter the room to use the multi-touch screens. Hinrichs and

Carpendale (2010) reported a “honey pot effect” when they observed visitor interaction

with multi-touch screens at the Vancouver Aquarium; many visitors were drawn to the

multi-touch screens through other visitors’ interest in the screen. Thus, multi-touch

screens should be installed in an area where the honey-pot effect can be maximized.

Provide Storage Space

Hinrichs and Carpendale (2010) also observed that many users placed bags or backpacks

on the table-top multi-touch screen surface because there was no other place for them to

store their belongings when they used the screen. This caused an obstruction between the

user and the interaction, and caused several users to leave the interaction before it was

complete. Whether multi-touch screens are table-top or wall-mounted, users should be

22

provided with space to store belongings, such as a small shelf below the screen, in order

to free their hands for the on-screen interaction.

5.2 Interaction Mechanics

The haptic interaction users have with multi-touch screens, regardless of the content of

the interaction, can significantly affect the user’s experience. Poorly designed interaction

mechanics can frustrate users and render the intent of the interaction less effective. There

are several best practices in interaction mechanics to keep in mind when designing multi-

touch screen content.

Figure 5.1: Even with multi-touch screens, user interaction usually begins with a single touch (Jacucci 2010). Image courtesy of Multitouch, Ltd.

Mimic Natural Movements

Gestures and controls should mimic natural hand movements as much as possible

(Moscovich, 2007). For example, if users are required to grasp an object, the

programmed hand motion should mimic a natural grasping motion, with the thumb as one

input and the rest of the fingers acting as another input. Similarly, since many people are

already familiar with smartphone technology; gestures and controls such as expanding,

collapsing, or zooming in on objects should use similar motions on multi-touch screens as

are used with smartphones (Hinrichs and Carpendale, 2010). For example, on many

23

smartphones, expanding, collapsing, or zooming in on objects is controlled with a user’s

thumb and pointer finger. Any interactions on a multi-touch screen using this control

should mirror this action as much as possible to leverage users’ existing knowledge of the

motion.

Use Gradual Engagement

Brignull (2008) recommends providing an initial set of display-based interactions that are

intuitive and can be easily and comfortably followed. Jacucci (2010) found that users

were most likely to attempt to initially manipulate onscreen objects with one finger.

Initial interactions should require a single touch only, with more complex multi-touch

gestures allowed as the interaction unfolds. (See Figure 5.1)

Provide Large Controls

All onscreen controls should be large enough for users to easily manipulate. For example,

Hinrichs and Carpendale (2010) found that many guests at the Vancouver Aquarium

complained of a “fat finger” effect with controls that were too small or too close resulting

in unwanted actions. One guest said that when controls were too close they “would do

some unpredictable things” and launch unintended screens or interactions. This lack of

response or unintended action quickly caused frustration among users. Therefore, careful

consideration should be made when designing controls to ensure they are large enough

for users to easily interact with and control.

Avoid Easy Deletion

If users have the ability to delete or hide objects in an interaction, it should not be too

easy for users to delete or hide those objects. The accidental deletion of objects can

frustrate users (Hinrich and Carpendale, 2010) and cause them to leave the interaction.

The specific mechanics for deleting objects should be designed carefully; for example,

frequently used controls should not be placed near the delete control. If a gesture or

action is used to delete objects, those actions should not mirror other commonly used

gestures or actions.

Avoid Sustained Contact

Gestures should not require sustained contact to be effective. A vertically-oriented touch

screen can cause discomfort if users are required to hold their hands out at arms' length

for an extended period of time (Szymanski et al., 2008). This condition, coined "gorilla

arm," was described in the 1980's after light pen users reported fatigue while working

24

with monitors (Avons, 1983). Any interactions should require short periods of contact if

possible and activities should be designed to allow users to rest their arms periodically.

Limit Activities Requiring Text Entry

Similarly, Wimmer (2009) found text entry to be best handled with a keyboard in a

horizontal position. Requiring users to do text entry using a virtual QWERTY style

keyboard in a vertical orientation quickly leads to fatigue.

Ensure Onscreen Changes Result from User Interaction

All changes on the screen should be a direct result of user interaction (Moscovich, 2007).

Interactions should not change in ways that are not intuitive from direct user interaction.

An animation that automatically starts or an object that moves without input from the

user can confuse users, especially if they are focused on another object or interaction.

Provide Feedback for User Input

Animations should be used to acknowledge user input when a control is selected or

activated (Szymanski et al., 2008). For example, the Annenberg Space for Photography

featured a water ripple effect on one of their multi-touch screens that began when users

touched it, rather than on its own. This feedback confirmed when users had selected an

area of the screen, assuring that the user’s touch had been received by the screen.

Use Animation to Indicate Interface Controls

Since controls are not physical objects, animations are helpful to draw attention to screen

elements which are controls. Pulsing, color cycling and fading are animations that do not

change a control element’s location on the screen, but instead invite interaction from the

user (Szymanski et al., 2008).

Demonstrate Possibilities of the Interface

Vanacken (2008) proposed an avatar-based help system to assist users with onscreen

interactions. These “TouchGhosts” would be displayed when the system senses repeated

failed attempts at a gesture. Alternately, the multi-touch system would use the

TouchGhosts when the system has been inactive for some time, showing the possibilities

of the interface to bystanders. This guidance helps distinguish multi-touch screens from

other static video displays in the museum, and invites bystanders to interact. Another way

to demonstrate the possibilities of the interface is to have a museum representative

stationed near the interaction and show visitors how to use the screen, as observed during

the site visit to the Annenberg Space for Photography.

25

Move Objects Along the Shortest Path Possible

Users should also be able to move objects using the shortest path possible (Moscovich,

2007). Users will naturally move objects using the short path from the starting point to

ending point. Interaction designs should reflect this concept and allow users to move

objects efficiently, especially if the interaction requires users to move objects to a fixed

position or static target. The only exception to this practice is pursuit-type tasks often

found in games where users move objects in a more varied path will help you determine

whether the training was successful in the long term.

5.3 Designing for Groups and Multiple Users

The nature of multi-touch screens makes them ideal for group interactions. However,

special considerations must be made when designing interactions for multiple users.

Provide Workspace for Multiple Users

It is important to consider the space each user will control on the multi-touch screen if

designing onscreen workspaces for multiple users. A common problem with multi-

screens with simultaneous interactions for multiple users is interference with another

user’s workspace (Hinrich and Carpendale, 2010). With undefined workspaces, users can

“steal” other user’s objects unintentionally if they accidentally touch or move that object.

Users can also block another user’s work if they move objects onto another workspace.

Good design can mitigate this problem by defining clear workspaces for users,

prohibiting objects from being placed on top of each other, or preventing objects from

reacting to other touches when they are in use.

Use Audio Feedback Carefully

Audio feedback can help make users aware of the presence of other users, and indicate

“collisions” when actions run into a workspace border. Positive audio feedback (reward

sound) that is shared and synchronized between users can also help foster social

interaction across workspaces. However, if negative audio feedback (error sound) meant

for and individual is heard by all users it can cause confusion as to which actions caused

the error (Hancock et al, 2005).

Use Localized or Coded Audio

If sound must be used in an interaction, it should be limited and directed at the individual

user, rather than the entire screen. If the screen is divided into multiple workspaces, then

audio should be channeled separately to those zones with separate speakers. Coded audio

26

(where the waveforms are varied so that different sounds are played for each user) can be

useful if activities are individualized but a single audio speaker must be shared with the

display (Hancock et al, 2005).

Provide Challenges to Increase Cooperation

Several articles reported that although strangers completed on-screen interactions

separately, they interacted with each other when faced with a conflict while using the

multi-touch screen (Peltonen et al, 2008; Hinrichs and Carpendale, 2010). Visitors often

helped each other troubleshoot interactions or shared media items, which promoted brief,

conversational experiences. While interactions should not be designed with any controls

meant to intentionally confuse users, if the goal of an interaction is to foster cooperation

between users, then elements of that interaction should feature challenges that users must

work together to overcome. For example, Peltonen et. al. (2008) suggests designing

performative interactions where each user must take a different role to increase

interactions between visitors. However, Hinrichs and Carpendale (2010) reported that

some users preferred not to have a social experience while using the multi-touch screen.

Therefore, if possible, visitors should have the option of participating in an individual

interaction or a group interaction on the multi-touch screen. By designing individual and

group interactions, solitary visitors can complete interactions by themselves, and those

users visiting in groups or who wish to engage other patrons can participate in more

social interactions.

Plan for Varying Interaction Times

Several studies indicate that visitors use multi-touch assets for varying amounts of time

depending upon the complexity of the content and the age of the user. For a simple photo

collection application, the average interaction time was less than three minutes, with

children staying slightly longer than adults (Hinrichs, 2010 and Jacucci, 2010). For

material of more complexity and depth, adults spent longer with the display than

children. Hinrichs (2010) noted some adult visitor groups interacted with the Vancouver

Aquarium Arctic Choices table for as long as 20 minutes. Depending upon location, this

could affect foot traffic patterns if crowds gather and stay at the screen for long periods of

time. Alternatively, experiences could be designed to time out instead of being open

ended and unlimited.

27

5.4 Designing for Various Age Groups

As with all types of instruction, the age and maturity of the audience is a key

consideration in design. Interactions designed for children will differ from interactions

designed for adults.

Level of Prompting Differs for Age Groups

Based on interviews at sites and the survey results, interactions should feature guidance

appropriate to the age of the intended audience. Children tend to begin interactions with

little guidance, young adults approach interactions with purpose, and adults are more

cautious. Therefore, interactions designed for children do not need to feature lengthy

instructions or introduction screens as children tend to begin interactions with little

external prompting and tend to disregard onscreen text. Interactions designed for young

adults should feature at minimum a brief description of the interaction; although many

young adults are familiar with various types of technology, both site visits and survey

responses stated that young adults approach multi-touch screens with the intent of

figuring out the interaction. Conversely, designs for adults should provide more guidance

or step by step instructions, such as prompts, arrows, examples, or brief introduction text.

Data from site visits and survey responses indicates that adults are more cautious than

children or young adults when interacting with multi-touch screens, perhaps due to a lack

of familiarity with that type of technology. Many adults began interactions only after

watching other visitors interact with the screen for an extended period of time. Regardless

of the reason for their uncertainty, multi-touch screens for adults should feature the most

built-in guidance and prompting to encourage adults to use the screen.

5.5 Spatial Organization

The spatial organization of a multi-touch screen is also a key consideration in design.

Provide Central Work Area

Many examples of multi-touch screens feature objects that are manipulated by users.

Since users can only manipulate one or two objects at a time, this means that some

objects will be “active” and other objects will be “inactive.” Users typically organize

digital objects by storing inactive items around the outside of the screen, and using the

middle of the screen as a working area to manipulate active items (Steimle et.al. 2010).

Therefore, interactions where users manipulate multiple digital objects should feature a

working area in the center of the screen for users to manipulate active objects, and a

28

storage area around the outside of the screen for users to store inactive objects. For

example, if users are manipulating set of photographs in an interaction, users should be

able to manipulate chosen photographs in the center of the screen while storing inactive

photos around the outside of the screen, and easily move photographs between the

working and storage areas.

5.6 Programming

As with all types of technology, reliable programming is essential to a positive user

experience.

Test Interaction Thoroughly

Feedback from the survey indicated that interactions should be planned and tested

thoroughly before they are implemented. One respondent recommended building a paper

prototype, to scale, before the multi-touch screen is installed to help visualize the space

the screen will occupy. Another respondent recommended rigorously testing interactions

before they are shared with the public, and ensuring all firmware and software is up to

date. Outdated software or hardware upgrades could potentially cause interactions to slow

or malfunction, and frustrate users to the point where they leave the multi-touch screen

interaction.

29

6. Conclusions Multi-touch technology is a rapidly developing field, with new products and possibilities

constantly emerging. MoPA’s decision to embrace such a current, innovative tool places

them as a leader in visitor engagement at museums. While being ahead of the curve has

its advantages, implementing such a new technology does present challenges, especially

for those tasked with designing meaningful learning experiences. In order to create a

successful visitor experience, MoPA should consider.

The physical installation and environmental factors that may affect

multi-touch screens

The mechanics of an interaction

The intended audience, including patrons visiting in groups and their age

range

The spatial organization of onscreen interactions

The type of programming

While this report does not address specific content of the experience, these fundamental

design guidelines can be used in a wide variety of content areas. Sound instructional

design is critical in creating a lasting, memorable museum experience.

30

Appendixes

Appendix A – References

Avons, S., Beveridge, M., Hickman, A., & Hitch, G. (1983). Considerations on using a

lightpen-interactive system with young children. Behavior Research Methods &

Instrumentation, 15(1), 75-78.

Benko, H., Morris, M., Brush, A., & Wilson, A. (2009). Insights on interactive tabletops:

a survey of researchers and developers (Microsoft Research Technical Report

MSR-TR-2009-22). Redmond, Washington: Microsoft Research

Brignull, H., Izadi, S., Fitzpatrick, G., Rogers, Y., & Rodden, T. (2004). The introduction

of a shared interactive surface into a communal space. Proceedings of the 2004

acm conference on computer supported cooperative work (pp. 49-58). New York,

New York: ACM Press.

Buxton, B. (2007, January 12). Multi-touch systems that I have known and loved.

Retrieved from http://www.billbuxton.com/multitouchOverview.html Experience

things in a whole new way. (n.d.). Retrieved March 6, 2011 from

http://www.microsoft.com/surface/whatissurface.aspx

Han, J. (Writer). (2006). Jeff Han demos his breakthrough touchscreen [Web]. Available

from

http://www.ted.com/talks/jeff_han_demos_his_breakthrough_touchscreen.html

Hancock, M., Shen, C., Forlines, C., & Ryall, K. (2005). Exploring non-speech auditory

feedback at an interactive multi-user tabletop. In Proceedings of Graphics

Interface 2005 (GI '05). Canadian Human-Computer Communications Society,

School of Computer Science, University of Waterloo, Waterloo, Ontario,

Canada, 41-50.

Hinrichs, U., & Carpendale, S. (2010). Interactive tables in the wild: visitor experiences

with multi-touch tables in the arctic exhibit at the Vancouver aquarium.

Informally published manuscript, Computer Science, University of Calgary,

Calgary, Canada. Retrieved from http://hdl.handle.net/1880/48199

Jacucci, G., Morrison, A., Richard, G., Kleimola, J., Peltonen, P., Parisi, L., & Laitinen,

T. (2010). Worlds of information: designing for engagement at a public multi-

31

touch display. Proceedings of the 28th international conference on Human factors

in computing systems, 10, 2267-2276.

Johnson, L., Witchey, H., Smith, R., Levine, A., and Haywood, K., (2010). The 2010

Horizon Report: Museum Edition. Austin, Texas: The New Media Consortium.

Kristensson, P., Arnell, O., Björk, A., Dahlbäck, N., Pennerup, D., Prytz, E., Åström, N.

(2008). Infotouch: an explorative multi-touch visualization interface for tagged

photo collections. Proceedings of the 5th Nordic conference on Human-computer

interaction: building bridges, 2008, 491-494.

Malik, S., Brush, A., & Meyers, B. (2007). An exploration of multi-finger interaction on

multi-touch surfaces. Toronto, Ont., Canada: University of Toronto.

Morris, M. (2008). A field study of knowledge workers’ use of interactive horizontal

displays. Proceedings of the Ieee tabletops and interactive surfaces 2008 (pp.

105-112). Amsterdam: IEEE.

Moscovich, T. (2007). Principles and applications of multi-touch interaction. Providence,

Rhode Island: Brown University.

Multi-touch case studies. (n.d.). Retrieved March 6, 2011 from http://multitouch.fi/case-

studies/

Peltonen, P, Kurvinen, E, Salovaara, A, Jacucci, G, et al. (2008). It's mine, don't touch!:

interactions at a large multi-touch display in a city centre. Proceeding of the

twenty-sixth annual SIGCHI conference on Human factors in computing

systems, 08, 1285-1294.

Schöning, J., Brandl, P., Daiber, F., Echtler, F., Hilliges, O., Hook, J., Löchtefeld, M.,.

(2008). Multi-Touch Surfaces : A Technical Guide Technical Report TUM-

I0833, Technical Reports of the Technical University of Munich, October 2008,

1-19.

Steimle, J, Khalilbeigi, M, Mühlhäuser, M, & Hollan, J. (2010). Physical and digital

media usage patterns on interactive tabletop surfaces. Proceedings of the ACM

International Conference on Interactive Tabletops and Surfaces 2010.

Szymanski, R, Goldin, M, Palmer, N, Beckinger, R, Gilday, J. & Chase, T. (2008).

Command and control in a multitouch environment. Proceedings of the Army

science conference Orlando, Florida: http://handle.dtic.mil/100.2/ADA503423.

32

Tognazzini, B. (1994). The "starfire" video prototype project: a case history. Retrieved

March 6, 2011 from http://www.asktog.com/papers/videoPrototypePaper.html

Vanacken, D, Demeure, A, Luyten, K, & Coninx, K. (2008). Ghosts in the interface:

meta-user interface visualizations as guides for multi-touch interaction.

Horizontal Interactive Human Computer Systems, 2008, 81 - 84.

Wigdor, D., Jian, H., Forlines, C., Borkin, M., & Shen, C. (2009). The wespace: the

design, development, and deployment of a walk-up and share multi-surface

visual collaboration system. Proceedings of the ACM Conference on Human

Factors in Computing Systems (CHI) Boston, Massachusetts, 1237-1246

Wimmer, R., Schulz, F., Hennecke, F., Boring, S., & Hussman, H. (2009). Curve:

blending horizontal and vertical interactive surfaces. Proceedings of the 4th ieee

workshop on tabletops and interactive surfaces (pp. 49-58). Banff, Canada: IEEE.

.

33

Appendix B – Survey Questions and Responses

What type of multi-touch screen does your organization have? Answer Options Response Percent Response Count Ideum 83.3% 5 MultiTouch US 16.7% 1 HP 0.0% 0 Microsoft 0.0% 0 Perceptive Pixel 0.0% 0 Evoluce 0.0% 0 KeyTec 0.0% 0 24-7mediatechnology 0.0% 0 Other (please specify)

AV&C New York

AV&C, Design PT

33.3% 2

answered question 6 skipped question 0

What is the positioning of your multi-touch screen(s)? Answer Options Response Percent Response Count Wall-mounted 33.3% 2 Table-top 100.0% 6 Kiosk 33.3% 2 Other (please specify) 0.0% 0 answered question 6 skipped question 0 How many multi-touch screens do you use in your organization? Answer Options Response Percent Response Count 1 33.3% 2 2 16.7% 1 3 0.0% 0 4 0.0% 0 5 0.0% 0 More than 5 50.0% 3 answered question 6 skipped question 0

34

If you have multiple screens, are they used separately or configured together as one screen? Answer Options Response Percent Response Count Used separately 75.0% 3 Configured together as one unit

0.0% 0

Both 25.0% 1 answered question 4 skipped question 2 Are multiple people able to touch the screen(s) at once? Answer Options Response Percent Response Count No 0.0% 0 Yes (please explain) 100.0% 6 There is room for about 8 children or 5 adults around each table. I believe it has up to 50 points of contact. There are 12 points of interactive tracking. Multiple people can touch and interact with the screens simultaneously. It's a multitouch 100 inch table that about 8 people can interact with at a time. Multiple users can interact with the content. answered question 6 skipped question 0 Please describe specifically how your visitors interact with the multi-touch screen(s). "One is a top down view of the Arctic with two control stations on either end that are made up of iPhone style switches, sliders and roller wheels. Visitors can enable and disable various overlays such as sea ice extent, or migration routes. The other is a picture viewer with attached metadata. Pictures spawn and visitors select the ones that interest them. They can also select video. They can scale both and there is attached metadata so they can flip the picture and read about it. Sometimes a connection will spawn and link two pictures that are different parts of the same story." Touch. Move images of objects through different wavelengths of energy. answered question 3 skipped question 3

35

What types of multi-touch interactions have you found that work best? Visitors enjoy the non-linear interactions with our media database. They like to choose whatever catches their eye. They like the fact that the touchtable content is connected in them and information to print content that is part of the exhibits. It's an additive experience from table to wall or wall to table. Pairing of screens and other exhibits. Simple and straightforward; can't utilize for layering of content. answered question 3 skipped question 3 What types of multi-touch interactions have you found that do not work well? If people need to look at controls, then look at changes on the screen, those changes need to be clearly seen as the effect of the interaction. There can be accidental stealing of focus so you have to make sure your interactions can survive that. None. See above. answered question 3 skipped question 3 What challenges regarding visitor usage have you encountered using multi-touch technology? Shake-out. It took us a long time to get the applications stable. But we put a thousand people a day past the exhibits, and they are always busy and now they are rock solid. Testing can only be truly done in a live gallery setting. We tested extensively and creatively in the lab, but people still surprised us once it went live. Not enough Money for more, larger and faster screens. Ours is a projection screen unit that has to be aligned. answered question 3 skipped question 3

36

Do you collect statistics for how many visitors use the multi-touch screen? Answer Options Response Percent Response Count Yes 33.3% 1 No 66.7% 2 answered question 3 skipped question 3 If so, how do you collect that data? Answer Options Response Percent Response Count Survey 0.0% 0 Exit interviews 0.0% 0 Program internal to the multi-touch screen

0.0% 0

Other (please specify) 100.0% 3 One time study. No. We do have an evaluation form people can fill out. answered question 3 skipped question 3 On an average day, approximately what percentage of museum visitors use the multi-touch screen(s)? Answer Options Response Percent Response Count Less than 10% 0.0% 0 10% 0.0% 0 20% 0.0% 0 30% 66.7% 2 40% 0.0% 0 50% 0.0% 0 60% 0.0% 0 70% 33.3% 1 80% 0.0% 0 90% 0.0% 0 100% 0.0% 0 answered question 3 skipped question 3

37

Have you noticed any particular usage patterns? (For example, males use screens more than females, children more than adults, etc.) Children run to the screens and dive in. Young adults approach and act with a sense of purpose. Older adults 50+ watch from a distance as others interact and then move in cautiously. Nope. Adults, teenagers, and middle schoolers. answered question 3 skipped question 3 Have you received any feedback from your staff about the multi-touch technology? Answer Options Response Percent Response Count Yes 100.0% 3 No 0.0% 0 answered question 3 skipped question 3 If so, please summarize the feedback most commonly received. They love it, though IS hated the shake out process. Like it. Easy to care for; enjoyed by visitors. answered question 3 skipped question 3 Have you received any feedback from visitors about the multi-touch technology? Answer Options Response Percent Response Count Yes 100.0% 3 No 0.0% 0 answered question 3 skipped question 3 If so, please summarize the feedback most commonly received. They love that we have something different and new. They also like the self-selecting, non-linear nature of the storytelling. Like it. They like it; find it engaging. answered question 3 skipped question 3

38

What criteria do you use to determine this technology’s success? Answer Options Response Percent Response Count Multi-touch screen usage statistics

33.3% 1

Visitor feedback 100.0% 3 Increased museum web site traffic

0.0% 0

Increased attendance at museum

33.3% 1

Increased interest in other museum exhibits

33.3% 1

Increase in membership

33.3% 1

Other (please specify) 0.0% 0 answered question 3 skipped question 3 What suggestions do you have for a museum that is looking to implement similar technology? Paper prototype, to scale, before building. Make sure you are clear on your desired outcome. Rigorously test the equipment to work exactly the way you want it to work, before you share it with the public. Always make sure that all firmware and software is up to date. answered question 2 skipped question 4 Please provide us with your contact information in case we have any follow-up questions. Name: Nick Lynch Company: Anchorage Museum at Rasmuson Center Email Address: nlynch@anchoragemuseum.org Phone Number: 907-929-9252 Name: Herschell Parker Company: Adventure Science Center Email Address: hparker@adventuresci.com Phone Number: 615-401-5070 answered question 2 skipped question 4

39

Appendix C – Annotated Bibliography

Avons, S., Beveridge, M., Hickman, A., & Hitch, G. (1983). Considerations on using a

lightpen-interactive system with young children. Behavior Research Methods &

Instrumentation, 15(1), 75-78.

Summary: Describes a study in the 1980's with children, computers and lightpens,

an early form of touch screen interaction. Notable for describing the arm fatigue

that sets in when using a screen in a vertical orientation.

Benko, H., Morris, M., Brush, A., & Wilson, A. (2009). Insights on interactive tabletops:

a survey of researchers and developers (Microsoft Research Technical Report MSR-TR-

2009-22). Redmond, Washington: Microsoft Research.

Summary: A survey of 58 tabletop researchers and developers with longer-term

tabletop use experience, to find out how they use their devices, what they use

them for, and what features they consider important for novice, single-user, and

collaborative scenarios. Identifies obstacles to mainstream adoption and input and

ergonomic challenges.

Brignull, H., Izadi, S., Fitzpatrick, G., Rogers, Y., & Rodden, T. (2004). The introduction

of a shared interactive surface into a communal space. Proceedings of the 2004 acm

conference on computer supported cooperative work (pp. 49-58). New York, New York:

ACM Press.

Summary: Describes a user study of a large multi-user interactive surface

deployed for an initial period within a real world setting. The surface was

designed to enable the sharing and exchange of a wide variety of digital media.

The setting for the study was the common room of a high school where students

come together to mix, socialize, and collaborate throughout the day. Reports on

how the students use the new technology within their own established communal

space. Findings show that the system was used extensively by the students in a

variety of ways, including sharing of photos, video clips, and websites, and for

facilitating social interaction. Discusses how the interactive shared surface was

40

appropriated by the students and introduced into their everyday lives in ways that

both mirrored and extended their existing practices within the communal space.

Buxton, B. (2007, January 12). Multi-touch systems that I have known and loved .

Retrieved from http://www.billbuxton.com/multitouchOverview.html

Summary: Overview of the history and application of multi-touch systems.

Provides a good reference regarding both technical details and design from one of

the pioneers in the field.

Experience things in a whole new way. (n.d.). Retrieved March 6, 2011 from

http://www.microsoft.com/surface/whatissurface.aspx

Summary: Microsoft's website describes the features and functionality of their

Surface product line. Includes videos, case studies and software demonstrations.

Han, J. (Writer). (2006). Jeff Han demos his breakthrough touchscreen [Web]. Available

from http://www.ted.com/talks/jeff_han_demos_his_breakthrough_touchscreen.html

Summary: Video of Jeff Han's revolutionary demonstration at the 2006

Technology Entertainment and Design conference.

Hancock, M., Shen, C., Forlines, C., & Ryall, K. (2005). Exploring non-speech auditory

feedback at an interactive multi-user tabletop. In Proceedings of Graphics Interface 2005

(GI '05). Canadian Human-Computer Communications Society, School of Computer

Science, University of Waterloo, Waterloo, Ontario, Canada, 41-50.

Summary: Discusses experiments on the use of non-speech audio at an interactive

multi-touch, multi-user tabletop display. Investigates both affirmative sounds that

confirm user actions and negative sounds that indicate errors. The results show

that affirmative auditory feedback may improve one's awareness of group activity

at the expense of one's awareness of his or her own activity. Also compares

localized sound, where each user has his or her own speaker, to coded sound,

where users share one speaker, but the waveform of the sounds are varied so that

a different sound is played for each user. Users can identify who caused a sound

when either localized or coded sound is used.

Hinrichs, U., & Carpendale, S. (2010). Interactive tables in the wild: visitor experiences

with multi-touch tables in the arctic exhibit at the Vancouver aquarium. Informally

41

published manuscript, Computer Science, University of Calgary, Calgary, Canada.

Retrieved from http://hdl.handle.net/1880/48199

Summary: Investigates visitors’ use of two different tabletop applications—the

Collection Viewer and the Arctic Choices table—that are part of the Canada’s

Arctic exhibition at the Vancouver Aquarium. Describes what factors attracted

visitors’ attention and triggered interaction with both tabletop exhibits, the

character and duration of information exploration, general exploration strategies,

and factors that triggered social and collaborative information exploration.

Usability issues are also presented with possible solutions.

Jacucci, G., Morrison, A., Richard, G., Kleimola, J., Peltonen, P., Parisi, L., & Laitinen,

T. (2010). Worlds of information: designing for engagement at a public multi-touch

display. Proceedings of the 28th international conference on Human factors in computing

systems, 10, 2267-2276.

Summary: Study of different 3D interface elements and how they affect social

interaction on a public multi-touch screen in Helsinki. Describe methods of

effectively partitioning the large surface to allow simultaneous usage, the need

and location of help assets and how best to initiate a multi-touch session.

Johnson, L., Witchey, H., Smith, R., Levine, A., and Haywood, K., (2010).

The 2010 Horizon Report: Museum Edition. Austin, Texas: The New Media Consortium.

Summary: The New Media Consortium’s Horizon Reports focus on emerging

technologies that they have identified as being instrumental for the future of

educational organizations. The Museum Edition focuses on 6 such technologies

and provides context on how they will affect the experience of museums in the

near future. The six identified are: mobile devices, social media, augmented

reality, location based services, gesture based computing and the semantic web.

Kristensson, P., Arnell, O., Björk, A., Dahlbäck, N., Pennerup, D., Prytz, E., . . . Åström,

N. (2008). Infotouch: an explorative multi-touch visualization interface for tagged photo

collections. Proceedings of the 5th Nordic conference on Human-computer interaction:

building bridges, 2008, 491-494.

Summary: Describes multi-touch interfaces for manipulation of tag clouds of a

tagged photo collection. Allows for creation of queries by physically moving

42

different attributes of photos into a query location. Video of the technique at:

http://www.youtube.com/watch?v=DHMJJwouq5I

Malik, S., Brush, A., & Meyers, B. (2007). An exploration of multi-finger interaction on

multi-touch surfaces. Toronto, Ont., Canada: University of Toronto.

Summary: Describes the development of the Visual Touchpad, a low-cost vision-

based input device that allows for detecting multiple hands and fingertips over a

constrained planar surface. Investigates unanswered questions as to how multiple

fingers from a single hand can best be utilized on these touch-sensitive surfaces.

Proposes a set of more advanced bi-digit widgets and gestures that facilitate

smooth transitioning from novice to expert usage.

Morris, M. (2008). A field study of knowledge workers’ use of interactive horizontal

displays. Proceedings of the Ieee tabletops and interactive surfaces 2008 (pp. 105-112).

Amsterdam: IEEE.

Summary: Field study to collect and analyze over a month of use data from eight

participants who used horizontal displays in conjunction with their existing office

computer setups. Analyzes the system logs, observations, and interview data to

reveal clear differences in preference and use patterns for horizontal and vertical

display configurations. Formulates hardware and software design guidelines that

would increase the utility of interactive horizontal displays for office scenarios

based upon the findings.

Moscovich, T. (2007). Principles and applications of multi-touch interaction.

Providence, Rhode Island: Brown University.

Summary: Studies the human factors governing multi-touch interaction, with

special emphasis on finger coordination. Presents a number of novel interaction

techniques that illustrate the benefits of multi-touch interaction. These techniques

let users work faster and more fluently than traditional single-point interaction

methods. Describes a family of techniques that use multiple finger contacts to

control digital instruments for manipulating graphical objects, allowing for

parallel control of multiple parameters (such as position and orientation) and

reduce the need for separate interaction modes by unifying operations (such as

grouping and moving objects).

43

Multi-touch case studies. (n.d.). Retrieved March 6, 2011 from http://multitouch.fi/case-

studies/

Summary: Describes real world uses of multi-touch screens. Includes case studies,

photographs and videos of such multi-touch installations as the Pori Jazz Festival,

a Virtual Autopsy Table, an Interactive Bar and the world’s largest public

multitouch display is installed for Siemens in Vienna.

Peltonen, P, Kurvinen, E, Salovaara, A, Jacucci, G, et al. (2008). It's mine, don't touch!:

interactions at a large multi-touch display in a city centre. Proceeding of the twenty-sixth

annual SIGCHI conference on Human factors in computing systems, 08, 1285-1294.

Summary: Studies social interaction around a public multi-touch display in

Helsinki. Describes different interface methods, thresholds of interaction with the

device and fellow users, behaviors of users, methods of inviting usage and

obstacles to usage.

Schöning, J., Brandl, P., Daiber, F., Echtler, F., Hilliges, O., Hook, J., Löchtefeld, M.,.

(2008). Multi-Touch Surfaces : A Technical Guide Technical Report TUM-I0833,

Technical Reports of the Technical University of Munich, October 2008, 1-19.

Summary: Describes the different methods that devices use to detect multiple

touch inputs. Investigates the different hardware designs and provides a good

technical brief of the current state of the art.

Steimle, J, Khalilbeigi, M, Mühlhäuser, M, & Hollan, J. (2010). Physical and digital

media usage patterns on interactive tabletop surfaces. Proceedings of the ACM

International Conference on Interactive Tabletops and Surfaces 2010.

Summary: Investigates the challenges of hybrid usage: users with physical and

digital objects interacting with a multi-touch table. Reports the results of a study

of spatial usage patterns when physical and digital items are grouped and sorted

on a tabletop work surface. Discusses issues with occlusion, involving both

inconvenient and desirable aspects. Describes design implications for hybrid

tabletop systems.

Szymanski, R, Goldin, M, Palmer, N, Beckinger, R, Gilday, J. & Chase, T. (2008).

Command and control in a multitouch environment. Proceedings of the Army science

conference Orlando, Florida: http://handle.dtic.mil/100.2/ADA503423.

44

Summary: Discusses the use of advanced user interface technology to create a

computer enhanced multi-touch command and control collaborative environment.

The COMET (Command and Control Multitouch Enabled Table) team out of

CERDEC C2D Communications Electronics Research Development and

Engineering Center Command and Control Directorate) is building and

researching the system discussed in this paper.

Tognazzini, B. (1994). The "starfire" video prototype project: a case history. Retrieved

March 6, 2011 from http://www.asktog.com/papers/videoPrototypePaper.html

Summary: Discusses the development of the Starfire video project by Sun

Microsystems in the 1990's. Described many revolutionary technologies and user

interface conventions such as touch screen, voice activation and the World Wide

Web.

Vanacken, D, Demeure, A, Luyten, K, & Coninx, K. (2008). Ghosts in the interface:

meta-user interface visualizations as guides for multi-touch interaction. Horizontal

Interactive Human Computer Systems, 2008, 81 - 84.

Summary: Discusses how most users are not familiar with multi-touch interfaces

beyond the traditional move-rotate-scale interactions and higher order interactions

can become exceedingly complex. The paper introduces the concept of

TouchGhosts: visual guides that are embedded in the multi-touch user interface

and that demonstrate the available interactions to the user. These provide

guidance in real time and within the context-of-use. Strategies are presented so

that the online help should not interrupt the flow of use.

Wigdor, D., Jian, H., Forlines, C., Borkin, M., & Shen, C. (2009). The wespace: the

design, development, and deployment of a walk-up and share multi-surface visual

collaboration system. Proceedings of the ACM Conference on Human Factors in

Computing Systems (CHI) Boston, Massachusetts, 1237-1246

Summary: Describes the "WeSpace" – a collaborative work space that integrates a

large data wall with a multi-user multi-touch table, developed for scientists who

frequently meet in small groups for data exploration and visualization. Presents a

yearlong effort including ethnographic studies, design, development and user

45

testing results of scientists carrying out their collaborative research using these

tools.

Wimmer, R., Schulz, F., Hennecke, F., Boring, S., & Hussman, H. (2009). Curve:

blending horizontal and vertical interactive surfaces. Proceedings of the 4th ieee

workshop on tabletops and interactive surfaces (pp. 49-58). Banff, Canada: IEEE.

Summary: Argues that horizontal and vertical surfaces serve different purposes

and offer different advantages. Proposes a combination of both surfaces by a soft

curve forming one large, L-shaped surface. This setup preserves the unique

properties of horizontal and vertical surfaces while allowing interaction across

boundaries without disruption. Describes findings from ergonomic studies

supporting this idea.

46

Appendix D – List of Figures

Figure 3.1: A vertical multi-user, multi-touch screen being used as an interactive directory at the Pori Jazz Festival. Image courtesy of Multitouch, Ltd. Figure 3.2: A multi-touch screen being used in a table format and with a stand for an angled horizontal orientation. Images courtesy of Multitouch, Ltd. Figure 3.3: A modular wall mounted cell-based design allows scalability. Image courtesy of Multitouch, Ltd. Figure 3.4: Examples of simple gestures: Single Touch, Multiple Touch & Drag, and Single Touch & Flick. Figure 3.5: Examples of multi-touch gestures: Pinch/Zoom Out, Rotate/Spin, and Unpinch/Zoom In/Rotate combination. Figure 4.1: Continuum of three types of multi-touch technology uses: activation, manipulation, and creation. Figure 5.1: Even with multi-touch screens, user interaction usually begins with a single touch (Jacucci 2010). Image courtesy of Multitouch, Ltd.

.