towards shaped and deformable interactive surfaces · beyond flat displays towards shaped and...
TRANSCRIPT
![Page 1: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/1.jpg)
Beyond Flat Displays Towards Shaped and Deformable Interactive SurfacesTutorial at ACM ITS 2012
Jürgen Steimle, Daniel Leithinger, Pol Pla Hrvoje BenkoMIT Media Lab Microsoft Research
![Page 2: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/2.jpg)
SHAPED DISPLAYSPol Pla and Hrvoje Benko
![Page 3: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/3.jpg)
![Page 4: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/4.jpg)
Novel Form Factorsor the world is not flat
![Page 5: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/5.jpg)
BendDeskMalte Weiss, Simon Voelker, and Jan Borchers
![Page 6: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/6.jpg)
Samsung SDI Bracelet Facet: A Multi-Segment Wrist Worn SystemKent Lyons, David. H. Nguyen, Daniel Ashbrook, Sean White
![Page 7: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/7.jpg)
Microsoft SphereHrvoje Benko, Andy Wilson, Ravin Balakrishnan, Billy Chen
![Page 8: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/8.jpg)
GCubikR Lopez-Gulliver, S Yoshida, S Yano, N Inoue
pCubeeI Stavness, B Lam, S Fels
![Page 9: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/9.jpg)
Audience Behavior around Large Interactive Cylindrical ScreensGilbert Beyer, Florian Alt, Jörg Müller, Albrecht Schmidt, Karsten Isakovic, Stefan Klose, Manuel Schiewe, Ivo Haulsen
![Page 10: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/10.jpg)
D20Ivan Poupyrev, Henry Newton-Dunn, Olivier Bau
![Page 11: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/11.jpg)
Display BlocksPol Pla and Pattie Maes
![Page 12: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/12.jpg)
“Beyond Flat Displays” @ Microsoft Research
Hrvoje Benko
Microsoft Research – Nov. 11, 2012
![Page 13: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/13.jpg)
Two research directions
New display forms Interactivity everywhere
In common: projector + (depth) camera
![Page 14: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/14.jpg)
New Display Forms
![Page 15: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/15.jpg)
Sphere Multi-Touch Interactions
UIST '08 4
Sphere
Benko, Wilson, Balakrishnan, UIST 2008.
![Page 16: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/16.jpg)
Pinch-the-Sky Dome
Benko & Wilson, ITS 2010.
![Page 17: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/17.jpg)
Mouse 2.0
Villar, Izadi, Rosenfeld, Benko, Helmes, Westhues, Hodges, Ofek, Butler, Cao & Chen, UIST 2009
MultiTouch Pen
Song, Benko, Guimretiere, Izadi, Cao & Hinckley, CHI 2011.
![Page 18: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/18.jpg)
Deformable Curved Displays
Bacim, Sinclair & Benko, ITS 2012
![Page 19: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/19.jpg)
What are curved surfaces good for?
![Page 20: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/20.jpg)
It depends…
If data matches shape
– Great
If viewing 2D data
– Probably worse
If viewing a 3D scene
– No different than flat displays
If exploiting some unique capability
– Potentially much better!
![Page 21: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/21.jpg)
Sphere Unique Characteristics
• Non-visible hemisphere
• Visibility changes with position• “Pseudo-private” and “public” areas
• No master user position / orientation
• Borderless, but finite display
• Few natural landmarks: poles
• Smooth transitions in depth and orientations• Near – far
• Horizontal – vertical
10UIST '08
![Page 22: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/22.jpg)
MirageTable
Benko, Jota, Wilson, CHI 2012
![Page 23: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/23.jpg)
MirageTable
![Page 24: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/24.jpg)
3D in your hand
Benko, Jota, Wilson, CHI 2012
![Page 25: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/25.jpg)
Interactivity everywhere
![Page 26: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/26.jpg)
What if you could use any available surface (including your body) as an interactive surface?
Rather than reach for a device…
simply touch where you want to see information…
and interact with it.
![Page 27: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/27.jpg)
![Page 28: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/28.jpg)
Analytic Approach• Problems:
• Slight variation in surface flatness
• Slight uncorrected lens distortion effect in depth image
• Noise in depth image
How to get the surface?
![Page 29: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/29.jpg)
Empirical Approach• Take per-pixel statistics of the empty surface
How to get the surface?
![Page 30: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/30.jpg)
KinectTouch
![Page 31: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/31.jpg)
But this works for static surfaces only!
![Page 32: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/32.jpg)
What about dynamic surfaces?
![Page 33: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/33.jpg)
How to get the surface?
Can we track the finger?
What is a surface?
• Hard in general• Simple from a body-centric perspective
![Page 34: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/34.jpg)
![Page 35: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/35.jpg)
Beamatron
![Page 36: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/36.jpg)
FLEXIBLE DISPLAYSJürgen Steimle
![Page 37: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/37.jpg)
Technology Trends
Flexible e‐paper and OLED displays are the most promising technologies for general‐purpose displays.Projection is likely to play an important role for providing in‐situ information by visually augmenting physical objects.There are some other technologies that have potential to support more specific use cases.
![Page 38: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/38.jpg)
Flexible ePaper
6Sony, 2011
![Page 39: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/39.jpg)
Flexible ePaper
LG, 2010 ITRI, 2011
![Page 40: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/40.jpg)
Flexible OLED
Samsung, 2011
![Page 41: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/41.jpg)
Flexible OLED
Samsung, 2009 Sony, 2010
![Page 42: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/42.jpg)
LEDs in Textiles
Philips Lumalive, 2006
![Page 43: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/43.jpg)
Thermochromic Ink
Saakes et al.: Shader Printer, 2012
![Page 44: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/44.jpg)
Projection
Cassinelli et al.: SkinGames, 2012
Takahashi et al.: Fluid Surface, 2012
Mistry et al.: Sixth Sense, 2009
![Page 45: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/45.jpg)
Flexible Sensing
Surface InputDeformation
External Internal
E.g.Omnitouch (UIST’11)Jamming User Interfaces (UIST’12)
E.g. printed: Ink‐jet Printed Ubiquitous Sensing (UbiComp’11)PyzoFlex (UIST’12)
optical:Twend (CHI’08)FlexAura (UIST’12)Stretchable sensor (UIST’12)
![Page 46: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/46.jpg)
INTERACTION
![Page 47: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/47.jpg)
Flexible displays allow for
• Incorporating displays into objects previously not compatible with displays (e.g. textiles)
• Highly portable devices• Malleable devices that adapt their shape to the current functionality
• Paper‐like/paper‐inspired interfaces that draw upon the affordances of paper
• More expressive, more varied ways of interaction and presentation of information
• + … ?
![Page 48: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/48.jpg)
Portability and Ergonomic Use
Lee et al.: Foldable Interactive Displays, UIST’08
Nokia Morph Concept
![Page 49: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/49.jpg)
Reconfigurable Devices: Folding
OLPC xo‐2, 2008
![Page 50: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/50.jpg)
Reconfigurable Devices: Rolling out
Khalilbeigi et al.: Xpaaand, CHI’11
![Page 51: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/51.jpg)
Reconfigurable Devices: Rolling out
Steimle and Olberding: Handheld Tabletop, CHI’12
![Page 52: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/52.jpg)
Paper‐like Interactions
Holman et al.: PaperWindows, CHI’05
![Page 53: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/53.jpg)
Paper‐like Interactions: Stacking
Girouard et al.: DisplayStacks, CHI’12
![Page 54: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/54.jpg)
Paper‐like Interactions: Arranging
Lissermann et al: PaperVideo, ACM Multimedia, 2012
![Page 55: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/55.jpg)
Paper‐like Interactions: Folding
Khalilbeigi et al.: FoldMe, TEI’12
![Page 56: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/56.jpg)
Navigation by Bending
Schwesig et al.: Gummi, CHI’04
![Page 57: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/57.jpg)
Navigation by Bending
Lahey et al.: PaperPhone, CHI’11
![Page 58: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/58.jpg)
Navigation by Bending
Nokia Kinetic, 2011
![Page 59: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/59.jpg)
Navigation by Bending
Cassinelli and Ishikawa: Khronos Projector, 2005
![Page 60: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/60.jpg)
ReferencesDaniel Saakes, Masahiko Inami, Takeo Igarashi, Naoya Koizumi, and Ramesh Raskar. 2012. Shader printer. In ACM SIGGRAPH 2012 Emerging Technologies (SIGGRAPH '12).A. Cassinelli, J. Angesleva, G. Frasca, M. Ishikawa. 2012. SkinGames. ITS 2012 Demo (ITS’12)Y. Takahashi, Y. Matoba, and H. Koike. 2012. Fluid Surface: Interactive water surface display for viewing information in a bathroom. ITS 2012 Demo (ITS’12)Pranav Mistry and Pattie Maes. 2009. SixthSense: a wearable gestural interface. In ACM SIGGRAPH ASIA 2009 Sketches (SIGGRAPH ASIA '09)Chris Harrison, Hrvoje Benko, and Andrew D. Wilson. 2011. OmniTouch: wearable multitouch interaction everywhere. In Proceedings of the 24th annual ACM symposium on User interface software and technology (UIST '11).Sean Follmer, Daniel Leithinger, Alex Olwal, Nadia Cheng, and Hiroshi Ishii. 2012. Jamming User Interfaces: Programmable Particle Stiffness and Sensing for Malleable and Shape‐Changing Devices. In Proceedings of the 25th annual ACM symposium on User interface software and technology (UIST ‘12).Nan‐Wei Gong, Steve Hodges, and Joseph A. Paradiso. 2011. Leveraging conductive inkjet technology to build a scalable and versatile surface for ubiquitous sensing. In Proceedings of the 13th international conference on Ubiquitous computing (UbiComp '11).\Christian Rendl, Patrick Greindl, Michael Haller, Martin Zirkl, Barbara Stadlober, and Paul Hartmann. 2012. PyzoFlex: printed piezoelectric pressure sensing foil. In Proceedings of the 25th annual ACM symposium on User interface software and technology (UIST '12).Gero Herkenrath, Thorsten Karrer, and Jan Borchers. 2008. Twend: twisting and bending as new interaction gesture in mobile devices. In CHI '08 Extended Abstracts on Human Factors in Computing Systems (CHI EA '08).Shenwei Liu and François Guimbretière. 2012. FlexAura: a flexible near‐surface range sensor. InProceedings of the 25th annual ACM symposium on User interface software and technology(UIST '12).Yuta Sugiura, Masahiko Inami, and Takeo Igarashi. 2012. A thin stretchable interface for tangential force measurement. In Proceedings of the 25th annual ACM symposium on User interface software and technology (UIST '12).Munehiko Sato, Ivan Poupyrev, and Chris Harrison. 2012. Touché: enhancing touch interaction on humans, screens, liquids, and everyday objects. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12).Johnny C. Lee, Scott E. Hudson, and Edward Tse. 2008. Foldable interactive displays. InProceedings of the 21st annual ACM symposium on User interface software and technology(UIST '08).Mohammadreza Khalilbeigi, Roman Lissermann, Max Mühlhäuser, and Jürgen Steimle. 2011. Xpaaand: interaction techniques for rollable displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11).Jürgen Steimle and Simon Olberding. 2012. When mobile phones expand into handheld tabletops. In CHI '12 Extended Abstracts on Human Factors in Computing Systems (alt.chi).David Holman, Roel Vertegaal, Mark Altosaar, Nikolaus Troje, and Derek Johns. 2005. Paper windows: interaction techniques for digital paper. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '05).Audrey Girouard, Aneesh Tarun, and Roel Vertegaal. 2012. DisplayStacks: interaction techniques for stacks of flexible thin‐film displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12).Roman Lissermann, Simon Olberding, Benjamin Petry, Max Mühlhäuser, Jürgen Steimle. 2012. PaperVideo: Interacting with Videos On Multiple Paper‐like Displays. Proceedings of ACM Multimedia (MM’12)Mohammadreza Khalilbeigi, Roman Lissermann, Wolfgang Kleine, and Jürgen Steimle. 2012. FoldMe: interacting with double‐sided foldable displays. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction (TEI '12)Carsten Schwesig, Ivan Poupyrev, and Eijiro Mori. 2004. Gummi: a bendable computer. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04).Byron Lahey, Audrey Girouard, Winslow Burleson, and Roel Vertegaal. 2011. PaperPhone: understanding the use of bend gestures in mobile devices with flexible electronic paper displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI '11).Johan Kildal, Susanna Paasovaara, and Viljakaisa Aaltonen. 2012. Kinetic device: designing interactions with a deformable mobile interface. In CHI '12 Extended Abstracts on Human Factors in Computing Systems (CHI EA '12).Alvaro Cassinelli and Masatoshi Ishikawa. 2005. Khronos projector. In ACM SIGGRAPH 2005 Emerging technologies (SIGGRAPH '05)
![Page 61: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/61.jpg)
ACTUATED DISPLAYSDaniel Leithinger
![Page 62: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/62.jpg)
Daniel LeithingerTangible Media Group (MIT Media Lab)
Shape Output Material Properties
1
![Page 63: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/63.jpg)
Bend, stretch, fold, shape, ...
Nokia Morph
Illuminating Clay [ Piper et al. 2002 ]SandScape [ Wang et al. 2003 ]Surface 2
2
![Page 64: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/64.jpg)
Piper et al. Illuminating clay: a 3-D tangible interface for landscape analysis. CHI ’02
Illuminating Clay, SandScapeBen Piper, Carlo Ratti, Yao Wang, Assaf Biderman, Bo Zhu, Saro Getzoyan, Hiroshi Ishii
3
![Page 65: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/65.jpg)
Tangible Media Group Vision:
4
![Page 66: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/66.jpg)
The Ultimate Display
The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal. With appropriate programming such a display could literally be the Wonderland into which Alice walked.
Ivan Sutherland (1965)
5
![Page 67: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/67.jpg)
Iwata et al. 2001. Project FEELEX: adding haptic surface to graphics. SIGGRAPH '01.
Shape Output: Related Work
Zhu et. al. 2003. Digital ClayXenotran XenoVision Mark III
Poupyrev et al. Lumen
6
![Page 68: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/68.jpg)
ReliefDaniel Leithinger, Hiroshi Ishii
7
![Page 69: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/69.jpg)
ReliefDaniel Leithinger, Hiroshi Ishii
8
![Page 70: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/70.jpg)
RecomposeMatt Blackshaw, David Lakatos, Tony DeVincenzi, Daniel Leithinger, Hiroshi Ishii
9
![Page 71: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/71.jpg)
Optical Properties■ Color, Opacity, Reflectivity, Luminosity
◦ Surface Properties■ Texture
◦ Thermal Properties:■ Temperature, specific heat, Thermal Conductivity
◦ Electro/Magnetic Properties:■ Conductivity, Dielectric Properties, Diamagnetism
◦ Mechanical Properties:■ Plasticity, Elasticity, Ductility, Hardness, Density, Coefficient of Friction,
Stiffness
Material Properties
10
![Page 72: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/72.jpg)
Jamming User InterfacesSean Follmer, Daniel Leithinger, Alex Olwal, Nadia Cheng, Hiroshi Ishii
11
![Page 73: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/73.jpg)
Jamming User InterfacesSean Follmer, Daniel Leithinger, Alex Olwal, Nadia Cheng, Hiroshi Ishii
12
![Page 74: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/74.jpg)
Programmable AffordancesSean Follmer, Daniel Leithinger, Alex Olwal, Nadia Cheng, Hiroshi Ishii
13
![Page 75: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/75.jpg)
Haptic FeedbackSean Follmer, Daniel Leithinger, Alex Olwal, Nadia Cheng, Hiroshi Ishii
14
![Page 76: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/76.jpg)
Shape ChangeSean Follmer, Daniel Leithinger, Alex Olwal, Nadia Cheng, Hiroshi Ishii
15
![Page 77: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/77.jpg)
Radical Atoms
16
![Page 78: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/78.jpg)
Wagner: Der Ring des NibelungenWagner's Dream, The Metropolitan Opera
17
![Page 79: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/79.jpg)
Wagner: Der Ring des NibelungenWagner's Dream, The Metropolitan Opera
18
![Page 80: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/80.jpg)
AmphormDavid Lakatos, Hiroshi Ishii
19
![Page 81: Towards Shaped and Deformable Interactive Surfaces · Beyond Flat Displays Towards Shaped and Deformable Interactive Surfaces Tutorial at ACM ITS 2012 Jürgen Steimle, Daniel Leithinger,](https://reader033.vdocument.in/reader033/viewer/2022060222/5f07a05b7e708231d41decc1/html5/thumbnails/81.jpg)
Thank you!
20