tangible bits - dspace@mit homedspace.mit.edu/bitstream/handle/1721.1/82648/mas-963... ·...

© 2004 MIT Media Laboratory, Hiroshi Ishii

Tangible Bits Beyond Ubiquitous GUIs

2004 Tokyo

Hiroshi Ishii Tangible Media Group MIT Media Laboratory

1


Tangible Bits Designing the Seamless Interface between People, Bits, and Atoms

February 2004

Hiroshi Ishii Tangible Media Group Things That Think MIT Media Laboratory

2

Ubiquitous Computing Mark Weiser, Xerox PARC, 1991

• Computers should be " ." •

, Pads, and Boards,

each other.


transparentComputational services are delivered through a variety of computational devices such as Tabswith the infrastructure to allow these devices to talk with

Mark Weiser at Xerox PARC presented his vision of “Ubiquitous Computing” in 1991. He claimed computers should be transparent and invisible.

He claimed that the most profound technologies are invisible. They weave themselves into the fabric of everydaylife until they are indistinguishable from it.His team designed a variety of computational devices including Tabs, Pads, and Boards along with theinfrastructure to allow these devices to talk with each other.

Recently, he is introducing a concept of “calm technology” citing the LIVE WIRE by Natalie Jeremijenko as an example.

3

Misunderstood Ubiquitous • • • Multiple Computational Devices / User

–


Anytime & Anyplace Mobile, wireless, broadband, RFID, ….

c.f. Mainframe (TSS) computing: one computer / many users Personal computing: one computer / user




4

1997… i

i i l

..........

i

..........

incl

ject. i

project! ..........

i


Wieser wrote to me in

Date: Sun, 26 Jan 1997 23:34:10 PST Subject: "Tangible Bits"

Dear Hirosh and Brygg,

I recently had a chance to read your CHI 97 paper "Tangible Bits"! Great work! In my opinion this s the k nd of work that will characterize the technologica

landscape in the twenty-first century.

My request is that you help me stop the spread of misunderstanding of ubiquitous computing based s mply on its name. Ubicomp was never just about making "computers" ubiquitous. It was always, like your work, about awakening computation mediation into the environment.

I tried to stop using ubiquitous computing because of its misleading implication, but it keeps cropping up again, so I keep returning to it as my umbrella name for lots of work,

uding Things That Think. Augmented reality was in use for awhile, but again got balkanized in meaning. I have started to talk about Calm Technology as a theme, but it better names a goal than a research pro "Tangible Bits" is very n ce, and maybe could serve as an overall umbrella, but then you might lose it as the name of your research

-mark (Dr.) Mark We ser Chief Technologist, Xerox PARC

5

LIVE WIRE

• 8 foot piece of plastic rope that hangsfrom a small electric motor mounted in the ceiling. The motor is linked to a

• Bits tangible

through motion, sound, and touch.

•


Natalie Jeremijenko, 1995

nearby LAN, so that passing bits cause twitching of the motor.

flowing through the wires of a computer network become

Mark Wieser called “Calm Technology”

Natalie Jeremijenko designed a beautiful instrument called Live Wire while an artist in residence at Xerox PARC.It is a piece of plastic cord that hangs from a small electric motor mounted on the ceiling. The motor is linked to the Ethernet, and each passing digital packet causes a tiny twitch of the motor.

Bits flowing through the wires of a computer network become tangible through motion, sound without being obtrusive, taking advantage of peripheral cues. This work encouraged us to think about ambient media as a general mechanism for displaying activities in cyberspace.

6

“The Computer for the 21st Century”

“The most profound technologies are those thatdisappear.

Mark Weiser


They weave themselves into the fabric of everyday life until they are indistinguishable from it.”

July 23, 1952 - April 27, 1999

Mark Weiser wrote as follows in the 1st paragraph of his landmark paper on Ubiquitous Computing published in 1991 Scientific America . “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” If you learn the task quite well using the right tools, and become the master, the tools cease to be a center of users’ attention. �Tools disappear into the background and you can concentrate on the tasks themselves. This is the definition of transparent or invisible computers by Mark. Invisible does not mean you can not see with your eyes. It is the matter of user’s focus of attention and consciousness. It is the state in which tools do not get in your way and you can concentrate on the task. Mark showed a series of ubiquitous computers including Live Board, Pads, and Badges. All of them come with screens and GUIs. But I do not feel Ubiquitous GUI is the path to achieve invisible interface. Today, I would like to show my approach: Tangible Interfaces.

7

At the Border

Where the land meets the sea, there is a border.


Where the land meets the sea, there is a border. This borderline is constantly moving and changing the shape of the shore. However, it is not only a simple line of a landscape, but also a critical biological locale.

8

Living at the Border

Harsh, but also fertile environment.


Whenever the tides go in and out, the inhabitants in the intertidal zone experience harsh environmental changes. When the tidal animals are exposed, they must withstand the heat and drying. When they are submerged, they must face constant pounding of waves. Living at this border is hard, but it is also a fertile environment for many kinds of life and activity. Where the land meets the sea, countless creatures have evolved diverse and creative approaches for flourishing among the turbulence and constant changes of these competing worlds. Our ancestors came from the sea and landed crossing this border a million years ago.

9

At the Border

We live on the border where bits meet atoms. In the flood of pixels from the ubiquitous GUIscreens, we are losing our sense of body andplaces.


between Physical and Digital

Pixels impoverish human senses.

We are now crossing another border. Where bits meet atoms, there is another shoreline. Today, we live on the boundary between physical space and cyberspace. The constant pounding by waves of bits from screens distorts our experience of space. People are losing theirsense of body and places in the flood of pixels from the ubiquitous rectangular screens. Living on this border is hard because our bodies remain in the physical world, even while we are immersed within digital information. It is also hard because current interfaces are not designed for amphibians. We are halfdrowning in the sea of bits, squeezing plastic mice instead of breathing and swimming. People are standing at the water's edge at a loss.

10


At the Boundary Where the sea meets the land, life has blossomed into a myriad of unique forms in the turbulence of water, sand, and wind.

At another seashore between the land of atoms and the sea of bits, we are now facing the challenge of reconciling our dual citizenships in the physical and digital worlds. Windows to the digital world are confined to flat square screens and pixels, or "painted bits." Unfortunately, one can not feel and confirm the virtual existence of this digital information through one's body.

Tangible Bits seeks to realize seamless interfaces between humans, digital information, and the physical environment by giving physical form to digital information and computation, making bits directly manipulable and perceptible.

Hiroshi Ishii | Tangible Media Group | MIT Media Lab

11

• for the Boundaries

• Seamless Collaboration Media at NTT Human Interface Laboratories

• • )

• Tangible User Interfaces (96-00)

• • ( )


Seamless Interfaces

(88-94)

Beyond “Being There” CSCW (Computer-Supported Cooperative Workcommunities: ACM SIGOIS, SIGCHI, IFIP

at Tangible Media Group, MIT Media Laboratory Beyond “Painted Bits” HCI Human-Computer Interactionscommunities: ACM SIGCHI and SIGGRAPH

12

My Art Work in 1959


Let me show another art work. This is my drawing on a fridge door in 1959. I loved drawing. Since then, drawing has been a very important medium in which I think, express myself and communicate with my mother.

13

•

•

Shared Workspace Interpersonal Space © 2004 MIT Media Laboratory, Hiroshi Ishii

Goal of ClearBoard Design Seamless Integration of Shared Workspace and Interpersonal Space Natural and smooth transition between them using everyday cues

ClearBord, the shared drawing media I designed, is deeply rooted in my drawing practices and my liking for the whiteboard.

Our goal was to create a collaboration medium that seamlessly integrated the IPS and the SWS.

We designed ClearBoard to allow users to shift easily between these spaces by using familiar everyday cues such as the partner's gestures, head movements, eye contact, and gaze direction.

14

Beyond “Being There” Beyond “Talking Heads” • Tacit Goal of Telecommunication:

distance – But this goal is a mirage. –

• New Goal: Creation of added value

cannot provide – e.g. Virtual Shared Workspace


Imitation of "Being There" over

It can never be achieved 100%.

that an ordinary face to face meeting

Most current video telephony systems have been designed to see "talking heads," and to create an illusion as if all the participants sit in an imaginative conference room. But this goal to simulate as close as to the real face to face meeting, however, it is something like a mirage. It can never be achieved 100%. The gap between the recognizable reality and virtuality cannot be filled by technology.As long as we try to find the raison d'etre of communication media only in its being "tele", we shall never be able to solve this contradiction. Conventional logic that it saves travel expenses and time should be reviewed.Our goal is to go beyond this model and demonstrate new usage of video communication technologies. To do so, we have integratedgroupware technology and real-time video communication technology to go beyond the imitation of physical proximity.Instead of pursuing the illusion of "Being there", we should focus on the creation of added value which cannot be obtained from face to face conversation.

15

Beyond Multimedia

• Multimedia

•

– –

•


= more than 3 cables behind a computer? "Multi-ness" is not main issue; rather seamless media integration which hides

low level representations and complexity of underlying technologies.

Multimedia = a premature label that represents a stage of media evolution from the mono-media to the seamless media.

"Multimedia" is now becoming a big buzz word in the computer and communication industries. As a result, the number of cables behind a computer, the number of features users need to understand, and the number of incompatible data formats are increasing beyond the limits of human cognitive capability.

The communication channels of human beings are inherently multi-modal and seamless. It does not make much sense to decompose the representation of information into primitive data types such as text, video, audio, and graphics, and stress the "multi-ness" of the media. For example, we are speaking, gesturing, and drawing simultaneously in a design meeting. We have great skills to express ideas and understand each other in everyday contexts using all these media as a whole. We believe the multi-ness of media is not the main issue, but how to integrate them into a seamless media hiding the various low level representations, discontinuities among primitive media, and complexity of underlying technologies is the core issue in designing new applications.

"Multi-media" sounds like a premature label that represents a stage of media evolution from the mono-media to the seamless media.

16

Vision Video “Seamless Media Design”

• TED4 Kobe (94), Door of Perception 2 (94)


17


PDA in 1958 - Abacus

When I was 2 years old, I started playing with a PDA called the "abacus".It was more than my mother's financial calculator. The abacus became my musical instrument, imaginary toy train, even back-scratcher. I enjoyed the sound and tactile interaction with this simple physical artifact. The abacus also became a medium of awareness. When my mother used the abacus to manage household accounts, I learned she could not play with me while her abacus made its music. My childhood abacus is suggesting us a future direction of HCI. That is the reason why I started the Tangible Bits project at MIT.

18

Eyes are in charge, but hands are underemployed. By pointing, by pushing and

pulling, by picking up tools,hands act as conduits through which we extendour will to the world.

They serve also as conduitsin the other direction: hands bring us knowledgeof the world. Hands feel. They probe. They practice.


Malcolm McCullough “Abstracting Craft: The Practiced Digital Hand ” 1996

Eyes are fully employed, but hands are underusedHands are underemployed. GUIs (Graphical User Interfaces) are designed for human eyes, not for hands. Telepresence, or interpersonal communication, is not the exception. Communication media such as telephones, video conferencing, and e-mail are for human ears and/or eyes, but not for hands. McCullough articulated the importance of hands as follows, “By pointing, by pushing and pulling, by picking up tools, hands act as conduits through which we extend our will to the world. They serve also as conduits in the other direction: hands bring us knowledge of the world. Hands feel. They probe. They practice.” [McCullough, 1996 #1].

19

Drawing Instruments What we have lost in the flood of PCs


Aesthetics which value haptic interaction with specialized physical objects ... but much richness has been lost.

David Brewster’s Edinburgh Encyclopedia (1808-1830)

This slide shows the various architectural drawing instruments used in 19th century. Long before the invention of personal computers, our ancestors developed a variety of specialized physical artifacts to draw geometric shapes. Grasping and manipulating those instruments, users of the past must have developed unique and rich languages, cultures, and aesthetics which value haptic interaction with real physical objects. We began our investigation of "looking to the future of HCI" at a museum by looking for what we have lost in the flood of personal computers. Our intention is to find a way to rejoin the richness of the physical world in HCI.

20

Tangible User Interfaces

• digital information & computation, making bits –

) – perceptible through

our peripheral senses)

•


Giving physical forms to

directly manipulable with our hands (in manipulatory

(in ambulatorySeamless coupling between digital and physical worlds.

Tangible interface is an interface design approach to give physical forms to digital information to make it tangible. It makes bits directly manipulable with our hands, and perceptible through our peripheral senses. The first path to the Tangible Interface is the directmanipulation with hands. Second path is the use of ambient media to make people aware of digital information at the periphery of perception. Pinwheels is such an example.The Tangible Interface explores the way to weave interfaces with the digital world into the fabric of a physical and social context.

21

Center and Periphery

• to grasp & manipulate bits center

• to be aware of bits at the peripheryusing


Architectural Space as Interface

in the of user's focus by coupling bits with physical objects and surfaces, and

ambient display media such as light, sound, airflow, and water movement.

This slide illustrates the framework of “center and periphery” of human perceptions. The ambientROOM was designed to achieve two goals: 1) to grasp & manipulate bits in the center of user's focus by coupling bits with physical objects and surfaces, and 2) to be aware of bits at the periphery using ambient display media such as light, sound, airflow, and water movement. We get information in two main ways. First, we get information from what we are focusing on, where our center of attention is directed. Butat the same time, we also get information from ambient sources.We may have a sense of the weather outside from ambient cues such as light, temperature, sound, and air flow from the open windows. We may also have an idea of the activities of our colleagues from the ambient sound and the shadows of passers-by.

Our brain is not only capable of processing this ambient information while maintaining a focus elsewhere, but will naturally shift such information to our foreground when our subconscious deems it important. By using ambient media, we can take advantage of our brain's abilities as both a parallel processor and an attention manager. Current HCI research is primary focusing on foreground activity and neglecting the background. We would like to support seamless transitions of the user's focus of attention between the periphery (background) and the center (foreground) of human awareness.

22

Pinwheels: wind of bits

• Ambient information

• Architectural space

humans and online digital information.


Ren, Frei, Dahley, Wisneski, and Ishii, 1997-2000

display spinning in a "wind of bits.”

will be an ambient interface between

23


Pinwheels: wind of bits Ren, Frei, Dahley, Wisneski, and Ishii, 1997-2000

QuickTime™ and a decompressor

are needed to see this picture.



Ambient information display spinning in a"wind of bits.”

Architectural space will be an ambient interface.



24

created by a "rain of bits."




Water Lamp: rain of bits Dahley and Ishii, 1997

Water ripple shadow

25

Foreground --> Background

ambient

PCs

phone


Peripheral Awareness using Ambient Media

Always on, real-time Peripheral awareness Seamless with environment

Time-consuming Requires navigation Complex

Interruptive Intrusive

26

Orb

•

portfolio, traffic on

mountains, pollenindex, etc.

• remapped to


by Ambient Devices (Media Lab Spinoff) www.ambientdevices.com

This light glows different colors to help you monitor your

your commute, new snow in the

The behavior can be

summarize whatever information you’d like in your periphery.

New York Times Magazine, Dec. 2002

27

27


Information Gauges 情報計器courtesy of Ambient Devices

Market 市場 Pollen 花粉 Weather 天気

• Ever-present information gauges log data and show trends in pollen index, weather forecast, snow survey, today’s market, blood pressure, blood sugar, exercise, weight, etc.常時表示の情報計器がデータを収集記録し表示する。例えば花粉、天気、降雪量、市場、血圧、血糖値、体重など。

Ambient DisplaysDesign Principles

• –

• Calm –

• Persistent connection –

• Decision-driven data –

• Private –


Browser-less interface Glance-able, requires no navigation and no analysis, simple.

Non-intrusive, seamless with environment

Information is continuously updated.

Personalized and summarized data feeds to make a decision.

Encoded data

29

29


Wireless products will diversifyワイアレスプロダクトは進化の初期段階

• Like wireless, electricity started out with one application in mind: the light bulb （電気の最初のアプリケーションは電球）

• It took 20 years for the first new products using electricity to be introduced （最初の製品導入まで 20年を要した）

• Today electricity is an invisible carrier to virtually every consumer product（今日電気は遍在キャリア）

Electricity電気

Wirelessワイアレス

Light Bulb 電球 Mobile Phone 携帯電話

?

ambientROOM

on ceiling

Light projection

Ambient sound of rain drops

Bottles as containers of bits

Clock to navigate time


Architectural Space as Interface

Ripple shadows

on side wall

Open a bottle to release bits into air

Let me summarize what you saw in this video.We demonstrated three types of ambient media.First, ripple shadow display. We made a thin water tank outfitted with a small robot which taps water being "pulled by bits." With each pull, the robot creates a ripple on the surface of water. Light projected onto the water tank from above casts a subtle but poetic image of rippleson the ceiling of the room.Second, subtle light projection on a side wall provide awareness of activities in a remote working place.Third, ambient sound of rain drops or water stream is used to let people be aware of activities in cyberspace such as network traffic.

31

inTouch: Tangible Telepresence

create an illusion of touching the same object

inTouch-0: inTouch-1: inTouch-2:


Brave, Dahley, Frei, Su, and Ishii, 1998

“Synchronized Distributed Physical Objects”

using force-feedback technology.

mechanical mockup

early electronic prototype

distributed prototype

Touch is underdeveloped. Touch is a fundamental aspect especially of interpersonal communication. Physical contact is a basic means through which people achieve a sense of connection and express emotion. Communication through touch, however, has been left largely unexplored in telepresence research. Visual and auditory extension of space was dominant approach of telepresence as represented by ClearBoard. In this summer, we have started the implementation of a distributed prototype via a computer network. The goal of this design is to allow the virtual connection of inTouch-1 to extend over any arbitrary distance.

In experiments using inTouch-1, in conjunction with audio communication, for longer periods of two-person interactions, subjects often noted that the haptic interaction with inTouch-1 was distracting at first, but later fell more into the background of perception. When this happened subjects felt that the interaction became more natural and smooth. Subjects also general felt that the haptic channel presented a “subtle but deep and revealing” addition to audio communication. More specifically users commented that the haptic channel allowed them to get a better sense of the other person’s emotional state and sincerity. Finally, some users expressed feelings of awkwardness when interactions with an acquaintance became more rhythmical, since they associated such movements with more intimate relationships.

32


inTouch: Haptic Interpersonal Communication Medium





illusion of touching the same object using force-feedback technology.

QuickTime™ and a YUV420 codec decompressor


Touch is also a fundamental aspect of interpersonal communication. We use touch to achieve a sense of connection, indicate intention, and express emotion. Though often subtle, touch can be a very powerful mechanism through which we express our thoughts and feelings to others. Current telecommunications technology, however, lacks the mechanism for expression through touch. The goal of our work is to enhance real-time remote collaboration and communication by bringing a greater sense of touch and physicality to distributed multi-user interactions.

33

“Ghostly Presence”

Remote users remain isolated behind computer screen.

Movement of local objects suggests the physical presence of remote users.


shared physical work space user B’s physical space

traditional remote collaboration systems

user A’s physical space

tangible telepresence

We have primarily described Synchronized Distributed Physiacl Objects as a means for extending the Tangible Interface approach into the realm of distributed CSCW. In our observations of and discussions with users of both PSyBench and inTouch, however, we have found that this approach also suggests a method for providing a new type of awareness of remote users. In traditional telecommunication systems we get the distinct impression that the distant user is in a space that is separate from our own. They are, for example, isolated in the digital world on the other side of our computer screen, where we are unable to touch them or share our physical world with them. In providing distant users with a shared physical space, we begin to see a

new form of awareness of the physical presence of the distant user within our own space. As the objects on the surface of PSyBench or the rollers on inTouch move around "on their own", we are compelled to imagine the position and actions of the person causing that movement... much like a player piano compels us to imagine the a real body sitting at the piano bench with arms extending to the keys. And we tend to imagine that presence within our own space; a sort of ghostly presence in our environment which can see and manipulate the same objects that we have access to.

34

ComTouch:

• Goal: Use touch to enhance real-time remote communication

• tactile communication language can be developed


A Vibrotactile Communication Device A. Chang, Z. Kanji, A. O'Modhrain, E. Gunther R. Jacob, and H Ishii, 2002

Motivation: Explore how

The ComTouch project is about designing a touch communication device for mobile use. We aim to enhance interpersonal communication by using touch. Touch conveys a wealth of personal information. Although we know that tactile communication devices do exist, we were interested in further development of touch communication, particularly assessing in what ways touch can enhance audio interaction.

35

Touch to vibration mapping


We chose to implement the touch-to-vibration mapping for the hand as shown here. For each finger, there are three different areas under the finger. Fingertip pressure is converted into vibration frequency and intensity. Because this is such a non-intuitive mapping , we implement a feedback channel so that the sender can feel their squeeze. This vibration is sent to a corresponding partner, who can squeeze back. By pressing harder… stronger and faster vibrations will result.

36

PSyBenchBrave, Bulthaup, Ishii, 1998




Physically Synchronized Workbench for Remote Collaborative Design (very early crude prototype)

37

CurlybotFrei, Su, ishii, 2000

motion. QuickTime™ and a

YUV420 codec decompressorare needed to see this picture.


A toy that can record and playback physical

Children establish an affective and body syntonic connection with curlybot, and develop intuitions for concepts such as differential geometry.

curlybot is an toy that can record and playback physical motion. As one plays with it, it remembers how it has been moved and can replay that movement with all the intricacies of the original gesture; every pause, acceleration, and even the shaking in the user's hand, is recorded. curlybot then repeats that gesture indefinitely creating beautiful and expressive patterns. === We introduce an educational toy, called curlybot, as the basis for a new class of toys aimed at children in their early stages of development - ages four and up. curlybot is an autonomous two-wheeled vehicle with embedded electronics that can record how it has been moved on any flat surface and then play back that motion accurately and repeatedly. Children can use curlybot to develop intuitions for advanced mathematical and computational concepts, like differential geometry, through play away from a traditional computer.

In our preliminary studies, we found that children learn to use curlybot quickly. They readily establish an affective and body syntonic connection with curlybot, because of its ability to remember all of the intricacies of their original gesture; every pause, acceleration, and even the shaking in their hand is recorded. Programming by example in this context makes the educational ideas implicit in the design of curlybot accessible to young children.

38

CurlybotFrei, Su, ishii, 2000

•

• intuitions for concepts such as differential geometry,

from a traditional computer.


Children readily establish an affective and body syntonic connection with curlybot. They can develop

through play away

curlybot is an toy that can record and playback physical motion. As one plays with it, it remembers how it has been moved and can replay that movement with all the intricacies of the original gesture; every pause, acceleration, and even the shaking in the user's hand, is recorded. curlybot then repeats that gesture indefinitely creating beautiful and expressive patterns. === We introduce an educational toy, called curlybot, as the basis for a new class of toys aimed at children in their early stages of development - ages four and up. curlybot is an autonomous two-wheeled vehicle with embedded electronics that can record how it has been moved on any flat surface and then play back that motion accurately and repeatedly. Children can use curlybot to develop intuitions for advanced mathematical and computational concepts, like differential geometry, through play away from a traditional computer.

In our preliminary studies, we found that children learn to use curlybot quickly. They readily establish an affective and body syntonic connection with curlybot, because of its ability to remember all of the intricacies of their original gesture; every pause, acceleration, and even the shaking in their hand is recorded. Programming by example in this context makes the educational ideas implicit in the design of curlybot accessible to young children.

39

topobo

• (• • • • •


Building Blocks with kinetic memory Hayes Raffle, Amanda Parkes, and Hiroshi Ishii

made of active motorized) & passive (static) components passives geometry based on cubic & tetrahedral crystals coincident input & output space actives “programmed” by moving, pushing, twisting units recorded sequence automatically plays back repeatedly distributed computation and networking

40

topobo

• )• • • • •

QuickTimeý Ç²YUV420 ÉRÅ[ÉfÉbÉN êLí£ÉvÉçÉOÉâÉÄ

Ç™Ç±ÇÃÉsÉNÉ`ÉÉÇ¾å©ÇÈÇžÇ½Ç…ÇÕïKóvÇ-Ç�ÅB


Building Blocks with kinetic memory Hayes Raffle, Amanda Parkes, and Hiroshi Ishii

made of active (motorized & passive (static) components passives geometry based on cubic & tetrahedral crystals coincident input & output space actives “programmed” by moving, pushing, twisting units recorded sequence automatically plays back repeatedly distributed computation and networking

41

Coincidence of input and output spaces


Principle of Tangible Interface Design

inTouch 98 interpersonal communication

curlybot 00 mathematics and expression / narrative

topobo 04 building block with kinetic memory

Since 1996, we have been exploring several key design principles of Tangible Interface. One of these principle is the “coincidence of input and output space.” It is critical to blur the boundary of input/output, and representation/control to make information tangible. An initial example was inTouch designed for haptic interpersonal communication. Today, the curlybot will demonstrate a realization of this principle with special focus on mathematical education and narrative expression.

42

Constructive Assembly:

Triangles 99 /

Blocks

MERL © 2004 MIT Media Laboratory, Hiroshi Ishii

Physical/Digital Building Blocks

Blocks, Cubes, Triangles, …

Physical Digital Construction Kit

[Aish 79] [Anagnostou 89, Frazer 94]

[Anderson 00]


43


Triangles:Physical / Digital Construction Kit Gorbet, Orth and Ishii, 1998





• Physical embodiment of digital information topology

• Tactile feedback of digital connection from magnetic edge connectors

• Physically persistent representation

The Triangles allow users to feel as if they are actually holding and rearranging data itself, with the magnetic connectors providing tactile confirmation when connections are made or broken. Triangles is Not 'digitally enhanced' version of existing object. Triangles is the new language, digital and physical language.

Now let me introduce the Triangles, the final example of Tangible Bits project. The Triangles system is a physical/digital construction kit, which allows users to grasp and manipulate complex digital information with two hands. The system consists of a set of identical flat, plastic triangles, each with a microprocessor inside and magnetic edge connectors, which enable the Triangles to be physically connected to each other. The connectors pass electricity, allowing the Triangles to communicate digital information to each other and to a desktop computer. When the pieces contact one another, specific connection information is sent back to a computer that keeps track of the configuration of the system. The Triangles can be used to make two and three-dimensional objects whose exact configuration is known to the computer.

44

Triangles:

• digital information topology

• connection from magnetic edge connectors

• representation


Physical / Digital Construction Kit Gorbet, Orth and Ishii, 1998

Physical embodiment of

Tactile feedback of digital

Physically persistent

The Triangles allow users to feel as if they are actually holding and rearranging data itself, with the magnetic connectors providing tactile confirmation when connections are made or broken. Triangles is Not 'digitally enhanced' version of existing object. Triangles is the new language, digital and physical language.

Now let me introduce the Triangles, the final example of Tangible Bits project. The Triangles system is a physical/digital construction kit, which allows users to grasp and manipulate complex digital information with two hands. The system consists of a set of identical flat, plastic triangles, each with a microprocessor inside and magnetic edge connectors, which enable the Triangles to be physically connected to each other. The connectors pass electricity, allowing the Triangles to communicate digital information to each other and to a desktop computer. When the pieces contact one another, specific connection information is sent back to a computer that keeps track of the configuration of the system. The Triangles can be used to make two and three-dimensional objects whose exact configuration is known to the computer.

45

Painted Bits (GUI)

intangible representation (pixels on a screen)

information

output

digital

physical

pixels soundinput

remote control

intangible representation


General input devices as remote-controllers of

46

Tangible Bits (TUI)

Tangible representation as interactive control mechanism to manipulate the information represented in both tangible and intangible forms

control tangible

representation

digital

physical

physical objects e.g. building model e.g. video projection

of digital shadow

information

intangible representation


47

I/O Bulb and Luminous Room

• I/O Bulb –

• Luminous Room –

• Give life to architectural

• Enable direct manipulation of digital


Underkoffler and Ishii, 1997 - 1999

High resolution output, two-way information

Multiple I/O bulbs illuminating architectural space

surfaces and physical objects.

world by grasping and manipulating objects with digital shadows.

48

I/O Bulb Video

• –

• distributed

• seep –

• Urp –


Underkoffler and Ishii, 1997-99

illuminating light Holography

illuminating light

Fluid dynamics

Urban simulation

49

Sensetable Patten, Ishii, Pangaro, 2000

• TUI platform to track multiple objects and their

video projection

• simulation for Supply


states on a table with

System Dynamics

Chain Analysis

50

Sensetable: TUI platform

• TUI platform to trackmultiple objects and their

projection

• Applications –

– Music “Audiopad”

–


James Patten, Dan Chak & Hiroshi Ishii

states on a table with video

System Dynamics Simulation for Supply Chain Analysis (& Intel and Sloan School)

(James Patten & Ben Recht) Event Driven Simulation for IP Network Design (& NTT Comware)

51


System Dynamics Simulation for Supply Chain Analysis Patten, Hines, Malone, Murphy-Hoye & Ishii 00-03





Collaboration with Intel and MIT Sloan School

52

System Dynamics Simulation for

and MIT Sloan School

QuickTime™ and a Sorenson Video 3 decompressor are needed to see this picture.


Supply Chain Analysis Patten, Hines, Malone, Murphy-Hoye & Ishii 00-03

Collaboration with Intel

53

IP Network Design Workbench

• Event-Driven Simulation +

design consulting expertise

• TUI supports cooperative direct manipulation of IP


NTT Comware + TMG

NTT Comware’s network

Network simulator.

54

IP Network Design Workbench:

•

consulting expertise •

effects of changing

• TUI supports cooperative

QuickTime™ and a YUV420 codec decompressor



NTT Comware + TMG (sensetable)

Based on Event-Driven Simulation Engine and NTT Comware’s NW

This workbench helps designers to evaluate the

topology, bandwidth, server location in real time, to optimize the network performance.

direct manipulation of IP Network simulator.

55

QuickTime™ and a YUV420 codec decompressor are needed to see this picture.

IP Network Design Workbench

• •

Thanks to Mr. Kase, Mr. Hirano, Mr. Narita, Ms.


NTT Comware + TMG Event-Driven Simulation TUI supports cooperative direct manipulation of simulator to evaluate the effects of changing topology, bandwidth, server location in real time, to optimize the network performance.

Kobayashi, Mr. Tanaka, and many other NTT Comware people.

56

From Physical World Model to Computational Abstract Model



Urp 99 System Dynamics Simulation 03

57

AudiopadJames Patten and Ben Recht (

• •

• j


Physics & Media)

A new way to perform electronic music. Designed to combine the expressive power of traditional musical instruments with the modularity of a computer Based on the Sensetable pro ect.

58

AudiopadJames Patten and Ben Recht (

• •

• j

QuickTime™ and a H.263 decompressor are needed to see this picture.


Physics & Media)


59

AudiopadJames Patten, Ben Recht (Physics & Media)

• •

modularity of a computer •

QuickTime™ and a H.263 decompressor



A new way to perform electronic music. Designed to combine the expressive power of traditional musical instruments with the

Based on the Sensetable project.

60

AudiopadJames Patten and Ben Recht*

• •

• jQuickTimeý Ç²

YUV420 ÉRÅ[ÉfÉbÉN êLí£ÉvÉçÉOÉâÉÄÇ™Ç±ÇÃÉsÉNÉ`ÉÉÇ¾å©ÇÈÇžÇ½Ç…ÇÕïKóvÇ-Ç�ÅB


(*Physics & Media)


61

CircuiTUI:

•

• Immediate feedback –

• qualitative data

• Classroom testing in MIT


Real-time circuit simulation and analysis Dan Chak and Hiroshi Ishii

A middle ground between a laboratory and a text book

“Oscilloscope” voltage and current shown by each component

Displays all quantitative and

physics course this fall semester

62

• each component

• • Positive voltages are blue, negative are orange • • Circuit behavior can be understood immediately


CircuiTUI Visualizations

“Oscilloscope” voltage and current shown by

Wire thickness varies with the amount of current

Color intensity analogous to voltage intensity

63

Sensetable James Patten & Hiroshi Ishii

• TUI platform to trackmultiple objects and their

video projection

• Applications – Music “Audiopad”

–

–

QuickTime™ and a Sorenson Video decompressorare needed to see this picture.


states on a table with

in collaboration with Ben Recht System Dynamics simulation for Supply Chain Analysis Chemistry

64

Concrete and Abstract

specific / concrete

Puck of



generic / abstract

Phicon of metaDesk 97 Sensetable 00


65

Actuated Workbench

•

•

QuickTime™ and a Photo - JPEG decompressor


–

–


Dan Maynes-Aminzade and Gian Pangaro & Hiroshi Ishii

Augments Sensetable platform providing an additional physical dynamic display capability using a grid of electromagnets to move objects on surface of table in two dimensions. Pucks on table contain LC tag, permanent magnet, and toggle switch.

Applications Synchronization of distributed “Sensetables” in realtime remote collaboration Clearing up inconsistencies that arise from the computer's inability to move the objects on the table

66

Actuated Workbench





Function Application


Dan Maynes-Aminzade and Gian Pangaro & Hiroshi Ishii 02

Magnetic forces to move objects on a table in two dimensions.

Augment existing “Sensetable” providing an additional physical dynamic display capability.

67

Actuated Workbench

•

using a grid of electromagnets to move objects on surface of table.

• Pucks on table contain LC tag, permanent


Dan Maynes-Aminzade, Gian Pangaro & Hiroshi Ishii 02-03

Augments Sensetable platform providing an additional physical dynamic display capability

magnet, and toggle switch.

68

Actuated Workbench

Application 1 •


Dan Maynes-Aminzade and Gian Pangaro & Hiroshi Ishii 02-03

Clearing up inconsistencies that arise from the computer's inability to move the objects on the table

with actuation without actuation

"In the video on the left, we see several people positioning cell phone towers on an overhead map. They can scroll the map by locking down one of the towers and moving it. However, notice how this creates an inconsistency between the physical and the digital states of the system. The user must manually adjust the positions of the other towers on the table to fix it. Users can also rotate and zoom the map, but this creates an inconsistency as well."

"By adding actuation to the system, we can automatically correct these inconsistencies. In the video on the right, we see the same scroll, rotate, and zoom operations, but the positions of the other pucks on the table are automatically updated using magnetic actuation."

69

Actuated Workbench

Application 2 •

remote collaboration


Dan Maynes-Aminzade and Gian Pangaro & Hiroshi Ishii 02-03

Synchronization of distributed “Sensetables” in realtime

Actuation can also be used to synchronize two workbenches in different locations, facilitating remote collaboration. Notice how when a puck is moved on one table, it moves in the same way on the other table. This helps people to work together remotely using the Sensetable.

By placing a camera above the workbench, we can capture silhouettes of the users' hands. We then broadcast these hand silhouettes to the other workbench to show the activity of the remote user.

70

Senseboard Jacob, Ishii, Pangaro, Patten 2001

•

•


TUI platform to allow users to arrange small magnetic pucks bound to digital data and function on a grid of tag readers with video projection

Application: organizing and grouping information

71

Senseboard Jacob, Ishii, Pangaro, Patten




2001 (video)

72

Illuminating ClayPiper, Ratti, and Ishii 2001

•

understanding of spatial relationships • Landscape Simulation

–

• 3D Laser Scanner + Video Projector


Physical Clay as 3-D Physical Input & Visual Display for intuitive manipulation and

land erosion, visibility, shadow casting, etc.

73

Illuminating Clay



•

and understanding of

• 3D Laser Scanner + Video Projector


Ben Piper, Carlo Ratti & Hiroshi Ishii

Physical Clay as 3-D Physical Input & Visual Display for intuitive manipulation

spatial relationships

74

Systemvideo projector

glass beads

stone table

mode selection block

IR camera

A ceiling mounted IRcamera captures theradiance of the lightpassing through thesand model to determine

surface. The resulting landscape

back on to the surface.


3D view

IR light sources

the geometry of the

analysis is projected

75

Physical Design Media � � Cardboard � Wooden Blocks �


Clay

Found Objects

Frank O. Gehry, Architect

Physical Outcomes Stata Center 2002

76

and Digital Representation in Design

Clearer communication

Digital

Greater precision


Lack of Continuity Between Physical

Physical

Ease of manipulation

Aids spatial understanding Easy distribution Quantitative analysis

How can we merge these media?

77

Form Giving + Computational Reflection: Refresh Rate of Iterative Design Cycle

Digital

Upper Stream

simultaneous form giving +


Physical

Lower Stream

Rough and rapid form giving Precise and quantitative with hand for ideation computational reflection

computational reflection

78

Representation of Idea Matters …

• • • •


… because the mental operations are made possible by the representation. … GUI/CAD is not for ideation.

e.g. Mathematical representation Drawings Physical models Computational models

79

Media for Design Thinking

• Visual Thinking – sketch

• Tangible Thinking – tactile manipulation of

–


physical representations coupled with digital computation design + analysis

80

PingPongPlus

• Digital augmentation of

"reactive table." • Ball tracking using

microphone array underneath table.

• From competition to collaboration


Ishii, Lee, Wisneski, Orbanes 1999

ping pong play with

81

PingPongPlus

•

"reactive table." •

microphone array underneath table.

From competition to collaboration



Ishii, Lee, Wisneski, Orbanes 1999

Digital augmentation of ping pong play with

Ball tracking using

82

Augmentation of existing objects



Another principle is the augmentation of existing objects to develop on top of existing understanding and skills. The bottles demonstrated in SIGGRAPH 99 emerging technologies is such an example. HandSCAPE to be presented by Jay Lee is another example to augment existing tape measure to bridge to the world of digital modeling.

Now I would like to introduce Phil Fre who is presenting and demonstrating curlybot.

83

musicBottles

Jazz Techno Classical Weather

• Glass bottles as "containers" and "controls" for digital information

• Seamless extension of metaphors and


Ishii, Fletcher, Mazalek, Lee, Choo, Berzowska, Paradiso, 98-00

physical affordances into the digital domain

musicBottles proposes a new tangible interface using glass bottles as containers for digital information. Imagine an array of perfume bottles. Instead of scent, the bottles have been filled with music classical, jazz, and techno music. Opening each bottle releases the sound of a specific instrument.

84

Origin: Weather Bottle

• • Model: soy sauce bottle

in her kitchen © 2004 MIT Media Laboratory, Hiroshi Ishii

Present for my mother

The idea of a bottle interface originated from the concept of a “weather forecast bottle,” which I envisioned as a present for my mother.

Upon opening the weather bottle, she would be greeted by the sound of singing birds if the following day’s weather was forecasted to be clear. On the other hand, hearing the sound of rainfall from the bottle would indicate impending rain.

Such an interface would be consistent with the everyday interactions with her familiar physical environment, such as opening a bottle of soy sauce. She never clicked a mouse or typed a URL in her life, but opened soy sauce bottles thousands of times.

85

Origin: Weather Bottle



86

Tangible Bits •

digital information and computation, making bits –

• and digital representation indesign

• Supporting multi-usercollaboration and “tangible thinking”


Giving physical forms to

directly manipulable with two hands

Continuity between physical


87

Painted Bits (GUI) and Tangible Bits (TUI)

•

• controllers”

Tangible User Interface •

•


Graphical User Interface Intangible representation (pixels on a screen) + Generic input devices as “remote

Tangible representation as interactive control mechanism to manipulate the information and computation Continuity between physical and digital representation in design

88

Tangible Bits

• Reconciliation of our dual citizenship in the worlds of bits and atoms.

• Interaction Design – – – shaped by industrial design, media arts and


informed by sciences (HCI), materialized by technologies (CS, EE, ME), and

practical real-world applications.

89

Arts & Design Technologies

Tangible Interface Design

Tangible Bits Design Space


Sciences &

concept, philosophy, scientific theory, implementation, aesthetics evaluation, analysis

Interaction Design Computer Sci. Cog. Sci. Industrial / Product Design Mechanical Eng. Soc. Sci. Architectural / Env. Design Electronic Eng. Media Arts / Interactive Arts

90

Design?

Conceptual “eyeglasses” –


Inspire people’s imagination and creativity

91

“The Computer for the 21st Century”

“The most profound technologies are thosethat disappear.

Mark Weiser


They weave themselves into the fabric of everyday life until they are indistinguishable from it.”

July 23, 1952 - April 27, 1999

Mark Weiser wrote as follows in the 1st paragraph of his landmark paper on Ubiquitous Computing published in 1991 Scientific America . “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” If you learn the task quite well using the right tools, and become the master, the tools cease to be a center of users’ attention. �Tools disappear into the background and you can concentrate on the tasks themselves. This is the definition of transparent or invisible computers by Mark. Invisible does not mean you can not see with your eyes. It is the matter of user’s focus of attention and consciousness. It is the state in which tools do not get in your way and you can concentrate on the task. Mark showed a series of ubiquitous computers including Live Board, Pads, and Badges. All of them come with screens and GUIs. But I do not feel Ubiquitous GUI is the path to achieve invisible interface. Today, I would like to show my approach: Tangible Interfaces.

92

Exhibitions

Linz, Austria

“Get in Touch”

Centre Pompidou Paris, France,

PingPongPlus,


Ars Electronica Center

musicBottles SandScape Audiopad

11/19/03-1/6/04

Le Design Interactif

93

Acknowledgments • Tangible Media Group

Graduate students and UROPs • • •


Things That Think, Digital Life @ MIT Media Lab Friends in ACM SIGCHI, SIGGRAPH, IDSA, ICSID Ars Electronica Center, NTT-InterCommunication Center

We thank Bill Buxton and George Fitzmaurice at the University of Toronto for countless discussions about graspable UI, skill-based design, and foreground & background issues, through which many of the ideas in this paper were developed and shaped. Thanks are also due to Bill Verplank at Interval Research for his insightful comments and discussions on haptic interfaces and tangible media. We appreciate Mark Weiser for his inspiring Ubiquitous Computing work and the introduction of Live Wire to me. We thank TTT (Things That Think), a new consortium at the MIT Media Lab, for its ongoing support of the Tangible Bits project. We also would like to acknowledge the contribution of many hardworking graduate and undergraduate students at MIT for work on the implementation of the Tangible Bits platforms. In particular, we thank graduate students Scott Brave, Andrew Dahley, Matt Gorbet, and many other students for their work on the metaDESK and ambientROOM prototypes. Thanks are finally due to Steelcase, Inc. for their donation of Personal Harbor which became a plat form of the ambientROOM.

94

Thanks!

Hiroshi Ishii Tangible Media Group

/


MIT Media Laboratory http://tangible.media.mit.edu

95

Thanks!


/


MIT Media Laboratory http://tangible.media.mit.edu

Thanks for your kind attention!

96


At the Boundary Where the sea meets the land, life has blossomed into a myriad of unique forms in the turbulence of water, sand, and wind.

At another seashore between the land of atoms and the sea of bits, we are now facing the challenge of reconciling our dual citizenships in the physical and digital worlds. Windows to the digital world are confined to flat square screens and pixels, or "painted bits." Unfortunately, one can not feel and confirm the virtual existence of this digital information through one's body.

Tangible Bits seeks to realize seamless interfaces between humans, digital information, and the physical environment by giving physical form to digital information and computation, making bits directly manipulable and perceptible.

Hiroshi Ishii | Tangible Media Group | MIT Media Lab

97

Ubiquitous Computing Mark Weiser, Xerox PARC, 1991 • Computers should be "transparent." • Computational services are delivered through a

Tabs, Pads, and Boards


variety of computational devices such as , with the infrastructure to allow

these devices to talk with each other.




98

Research?

• – Local optimization –

problems – Reflection-driven

• Secure home – Orthodox – Established discipline –


Increment

Solve known

Homogeneity

99

Research?

Yes • Innovation

– Mutation –

problems – Invention-driven

• No home – – – Diversity

No •

– Local optimization –

problems – Reflection-driven

• Secure home – Orthodox – Established discipline –


Identify new

Heterodox - heretic Inter-disciplinary

Increment

Solve known

Homogeneity

100

Design

• Highly interdisciplinary – –


Enjoy identity crisis. Appreciate diversity.

101

Design


• New vision, no roadmap – Invent new paradigms, rather than do



incremental work, because life is short.

102

Design


• New vision, no roadmap – Invent new paradigms, rather than

doing incremental work. • Success is a danger

– © 2004 MIT Media Laboratory, Hiroshi Ishii


Do not stop. Keep redefining yourself.

103

To the Researchers •

–

• – –

• Ground to the earth –

• Success is a danger –


New vision, no roadmap Invent new paradigms, rather than doing incremental work, because life is short.

Highly interdisciplinary Enjoy identity crisis. Appreciate diversity.

Learn through real world problem solving

Do not stop. Keep redefining yourself.

104

Beyond Painted Bits, Toward Tangible Bits

•

• representation and interfaces for

GUI TUI


Beyond GUI (Graphical User Interface) Toward TUI (Tangible User Interface) Physical objects, surfaces, and spaces as

computationally mediated information.

Taking advantage of the multi-modality of human interactions with the physical world, we are trying to go beyond “painted bits”, and establish a new paradigm of “Tangible Bits.” Coupling bits with physical objects and phenomena is the key approach, we are trying to make the interfaces ubiquitous and transparent. Ultimately, we believe the architectural space itself should be an interface between people and digital information space

105

Thanks!


http://tangible.media.mit.edu/


MIT Media Laboratory

106

Thanks!



MIT Media Laboratory http://tangible.media.mit.edu/

107

Tangible Interface Design (1)

Coincidence of input and output spaces

interpersonal communication

computation / mathematics


Principles of

inTouch 98 curlybot 00

and expression / narrative


108

Tangible Interface Design (2)

Augmentation of existing objects

Bottles 99

containers © 2004 MIT Media Laboratory, Hiroshi Ishii

Principles of

Another principle is the augmentation of existing objects to develop on top of existing understanding and skills. The bottles demonstrated in SIGGRAPH 99 emerging technologies is such an example. HandSCAPE to be presented by Jay Lee is another example to augment existing tape measure to bridge to the world of digital modeling.

Now I would like to introduce Phil Fre who is presenting and demonstrating curlybot.

109

Oullim • Aufheben (• People, Bits, and Atoms

– • Arts, Design, Sciences,

and Technologies –


German)

Body, Cyberspace, and Physical Space

Analysis and Synthesis

110

Real + Virtual, but mainly Visual •

• –

synthetic environment. • Augmented Reality

– ld,

| digital


Personal Computing

Virtual Reality completely immerses a user inside a

allows the user to see the real worwith virtual objects superimposed upon the real world. supplements reality, rather than completely replacing it.

physical

111

Augmented Reality Def. by Ron Azuma Augmented Reality

l

1. 2. 3.

•

•


allows the user to see the real wor d, with virtual objects superimposed upon or composited with the real world. supplements reality, rather than completely replacing it.

Blends real and virtual, in real environment Real-time interactive Registered in 3-D

Azuma, A Survey of Augmented Reality. Presence, 6 (4), p.355-385, Aug. 1997 SIGGRAPH 2001 Course Notes 27, Augmented Reality: The Interface is Everywhere

112

Augmented RealityComputer-Augmented Environments

• media?

• by visual overlay

•

– – –


How to integrate “real world” and computational

Integrating “real world” and computational media of digital information onto real

world image with see-through display, hand-held display, or direct video projection Redefining “Human Interactions” with computational (virtual) artifacts within the social and physical settings of their use

From Desktop Computing to Ubiquitous Computing From Desktop UI to Ubiquitous UI From Virtual Reality to Augmented & Mixed Reality

Augmented Reality is a new research stream which tries to integrate the real world and computational media.

There are many important work including Myron Krueger’s pioneering work of VIDEOPLACE in 1985.DigitalDesk by Wellner is one of early work which demonstrated a way to merge physical and digital document by using video projection ofcomputer display onto a real desk with physical documents.

The most common AR approach is the visual overlay of digital information onto real-world imagery with see through head-mounted display, hand-held display, or direct video projection.

113

Eyes are in charge, but hands are underemployed.


Eyes are fully employed, but hands are underusedHands are underemployed. GUIs (Graphical User Interfaces) are designed for human eyes, not for hands. Telepresence, or interpersonal communication, is not the exception. Communication media such as telephones, video conferencing, and e-mail are for human ears and/or eyes, but not for hands. McCullough articulated the importance of hands as follows, “By pointing, by pushing and pulling, by picking up tools, hands act as conduits through which we extend our will to the world. They serve also as conduits in the other direction: hands bring us knowledge of the world. Hands feel. They probe. They practice.” [McCullough, 1996 #1].

114

Bottles as Interface


The core concept utilizes glass bottles as "containers" and "controls" for digital information. Physical manipulation of the bottles - opening and closing is the primary mode of interaction with digital information.

musicBottles proposes a new tangible interface using glass bottles as containers for digital information. Imagine an array of perfume bottles. Instead of scent, the bottles have been filled with music - classical, jazz, and techno music. Opening each bottle releases the sound of a specific instrument.

115

Sensetable & Senseboard as Generic TUI Platforms

TUI Applications

Urban Planning others

TUI Application Program Interface


Supply Chain Vis.

Sensetable / Senseboard Hardware

116

------

Tangible User Interfaces

• information, making bits

– our hands

– perceptible through our peripheral senses

•


Giving physical forms to digital

directly manipulable with

Seamless coupling between digital and physical worlds.


Tangible interface is our challenge to design seamless interface between the land of atoms and the sea of bits for the people.

117

People have developed sophisticated skills for sensing and manipulating their physical environments. However, most of these skills are not employed by the traditional Graphical User Interface. GUIs (Graphical User Interfaces) are designed for human eyes, not for hands. GUIs are in the boxes detached from our physical environment and social context. Tangible interface is an interface design approach to give physical forms to digital information to make it tangible. It makes bits directly manipulable with our hands, and perceptible through our peripheral senses. The Tangible Interface explores the way to weave interfaces with the digital world into the fabric of a physical and social context. The Tangible Interface seeks to build upon the skills of manipulating the physical world, by giving physical form to digital information and situating it in the architectural spaces. The Tangible Interface explores the way to weave interfaces with the digital world into the fabric of a physical and social context.

TouchCounters: Containers


Interactive Electronic Labels for Physical (Yarin and Ishii 99)

“Distributed visualization of usage history”: Physical objects and surfaces that display their history of use

118

mediaBlocks: Ullmer, Ishii, and Glas, 1998

• Small, electronically tagged

• containment, transport, and manipulation of online media.

•

• collections of media elements.


Physical Containers, Transports, and Controls for Online Media

wooden blocks Physical icons (phicons) for the

Rapid “copy” and “paste” from a media source into a media display. Racks allow navigation and sequencing of

119

Design Evolution

Paradigm Principles

Idea Hunch Hack

Product market

GUI, Ubiquitous Comp. WYSIWIS (GUI) /

NLS, Alto, Collab

Lisa, Mac, Navigator

PC, Web


Bill Verplank 1998

Prototype

WYSIWIS (CSCW)

120


Tangible Bits Designing the Seamless Interface between People, Bits, and Atoms

Conference Month Day, 2002

Hiroshi Ishii Tangible Media Group MIT Media Laboratory

121

Augmented RealityComputer-Augmented Environments

• media by of digitalinformation onto real world image with see-

video projection • Examples:

– – KARMA (Steven Feiner) – Chameleon (George Fitzmaurice) – Wearable Computing (Thad Starner)


Integrating “real world” and computational visual overlay

through display, hand-held display, or direct

DigitalDesk (Pierre Wellner)

Augmented Reality is a new research stream which tries to integrate the real world and computational media.

There are many important work including Myron Krueger’s pioneering work of VIDEOPLACE in 1985.DigitalDesk by Wellner is one of early work which demonstrated a way to merge physical and digital document byusing video projection of computer display onto a real desk with physical documents.

The most common AR approach is the visual overlay of digital information onto real-world imagery with see through head-mounted display, hand-held display, or direct video projection.

122

Graspable User InterfaceG. Fitzmaurice, H. Ishii, & W. Buxton, 1995

• Direct control of virtual objects through

• Blend physical and virtual affordances •


physical handles called “bricks”

Distribute cognitive load more evenly from visual to tactile system

This work of Graspable User Interface was done in collaboration with G. Fitzmaurice and Bill Buxton during my one year stay at the University of Toronto in 1994.The Graspable User Interface provide direct control of virtual objects throughphysical handles called “Bricks.”With Bricks, users can take advantage of our everyday skills in manipulating physical objects.

123

tangible bits - dspace@mit homedspace.mit.edu/bitstream/handle/1721.1/82648/mas-963... ·...

Documents