fifty shades of augmented reality: creating connection using ar
Post on 05-Apr-2017
1.633 Views
Preview:
TRANSCRIPT
FIFTY SHADES OF AUGMENTED REALITY: CREATING CONNECTION USING AR
Mark Billinghurstmark.billinghurst@unisa.edu.au
March 24th 2017
Laval Virtual
Modern Technology Trends
1. Improved Content Capture• Move from sharing faces to sharing places
2. Increased Network Bandwidth• Sharing natural communication cues
3. Implicit Understanding• Recognizing behaviour and emotion
These three trends combined with AR/MR technology can create new types of collaborative experiences
Content CaptureRealism
Time
Photo
Film
Live Video
Panorama
360 Video
3D Space
1850 1900 1940 1990 2000 2010
Capturing Reality• Panoramas (1790s)
• Immersive paintings• Photography (1820-30s)
• Oldest surviving photo (Niépce, 1826)• Stereo imagery (1830s)
• Wheatstone (1832)• Brewster (1851)
• Movies (1870s)• Muybridge (1878)• Roundhay Garden Scene (1888)
Capturing 360 images
Kodak 360 Fly 360 Gear 360 Theta S Nikon
LG 360 Pointgrey Ladybug Panono 360 Bublcam
360 Stereo Capture
Samsung Project Beyond
Jaunt VR camera
https://www.youtube.com/watch?v=FnBIvq8jzds
3D Space Capture
Google Project Tango
Occipital Structure Sensor
https://www.youtube.com/watch?v=XA7FMoNAK9M
Content CaptureRealism
Time
Photo
Film
Live Video
Panorama
360 Video
3D Space
1850 1900 1940 1990 2000 2010
2D Static
Immersive
Live
Experience
Networking Technology
• Telephone (1876)• Bell Picture Phone (1955)
• B+W live video calls• Mobile Phones (1985)• Desktop Video Conferencing (1990’s)
• Netmeeting, Skype
https://www.youtube.com/watch?v=OBkcYyenNBM
Mobile Conferencing (2010-)
• Video conferencing• Skype, facebook
• Live broadcasting• Meerkat, Periscope• Support for 360 video
Networking SpeedsLog (b/s)
Time
100 b/s
10 Kb/s
1 Mb/s
1980 1985 1990 1995 2000 2010
100 Mb/s
2005
Text
Audio
Natural
Video
Interaction TechnologyNatural
Time
PunchCard
Keyboard
Mouse
Speech
Gesture
Emotion
1950 1960 1980 1990 2000 2010
Thought
Natural Gesture Input
• Freehand input• Depth sensors, rich two handed gestures
Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J., Leichter, D. K. C. R. I., ... & Izadi, S. (2015, April). Accurate, Robust, and Flexible Real-time Hand Tracking. In Proc. CHI (Vol. 8).
https://www.youtube.com/watch?v=A-xXrMpOHyc
Interaction TechnologyNatural
Time
PunchCard
Keyboard
Mouse
Speech
Gesture
Emotion
1950 1960 1980 1990 2000 2010
Thought
Implicit
Explicit
Milgram’s Reality-Virtuality Continuum
Mixed Reality
Reality - Virtuality (RV) Continuum
RealEnvironment
AugmentedReality (AR)
AugmentedVirtuality (AV)
VirtualEnvironment
"...anywhere between the extrema of the virtuality continuum."
P. Milgram and A. F. Kishino, Taxonomy of Mixed Reality Visual Displays IEICE Transactions on Information and Systems, E77-D(12), pp. 1321-1329, 1994.
AR Video Conferencing (2001)
• Bringing conferencing into real world• Using AR video textures of remote people• Attaching AR video to real objects
Billinghurst, M., & Kato, H. (2002). Collaborative augmented reality. Communications of the ACM, 45(7), 64-70.
MR Point Cloud Sharing (2016)• Use video-see through HMD
• Occulus Rift• Depth camera on HMD
• Softkinetic• Create point cloud
• Remote hand gestures• User’s space
Holoportation (2016)
• Augmented Reality + 3D capture + high bandwidth • http://research.microsoft.com/en-us/projects/holoportation/
Social VR
• Facebook Social Virtual Reality, AltspaceVR• Bringing Avatars into VR space• Natural social interaction
AR/VR Collaboration (2017)
• Make 3D copy of real space• AR user in real world, VR user in 3D copy of real space
• Hololens for AR user, HTC Vive for VR user• Share virtual body cues (head, hands, gaze information)
Real World Virtual World
Empathy
“Seeing with the Eyes of another,
Listening with the Ears of another,
and Feeling with the Heart of another..”
Alfred Adler
Empathic Computing
Systems that allow us to share what we are seeing, hearing and feeling with others..
Example: Google Glass
• Camera + Processing + Display + Connectivity
• Ego-Vision Collaboration (But with Fixed View)
Current Collaboration on Wearables
• First person remote conferencing/hangouts• Limitations
• Single POV, no spatial cues, no annotations, etc
Social Panoramas (ISMAR 2014)
• Capture and share social spaces in real time• Supports independent views into Panorama
Reichherzer, C., Nassani, A., & Billinghurst, M. (2014, September). [Poster] Social panoramas using wearable computers. In Mixed and Augmented Reality (ISMAR), 2014 IEEE International Symposium on (pp. 303-304). IEEE.
Implementation
• Google Glass • Capture live image panorama (compass + camera)
• Remote device (tablet)• Immersive viewing, live annotation
JospehTame – Tokyo Marathon
• Live streaming from Tokyo marathon• http://josephta.me/en/tokyo-marathon/
Empathy Glasses (CHI 2016)
• Combine together eye-tracking, display, face expression• Impicit cues – eye gaze, face expression
++
Pupil Labs Epson BT-200 AffectiveWear
Masai, K., Sugimoto, M., Kunze, K., & Billinghurst, M. (2016, May). Empathy Glasses. In Proceedings of the 34th Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM.
AffectiveWear – Emotion Glasses
• Photo sensors to recognize expression• User calibration• Machine learning• Recognizing 8 face expressions
Empathy Glasses in Use
• Eye gaze pointer and remote pointing• Face expression display• In future integrated eye-tracking/display
Empathic VR Environments (2017)
• Player and Viewer• Viewer slaved to player
• Share emotional signals • Heart rate, GSR
• Remote affect measuring
VR Environments
• Butterfly World: calm scene, collect butterflies• Zombie Attack: scary scene, fighting zombies
AR and VR for Empathic Computing• VR systems are ideal for trying experiences:
• Strong story telling medium • Provide total immersion/3D experience• Easy to change virtual body scale and representation
• AR systems are idea for live sharing:• Allow overlay on real world view/can share viewpoints• Support remote annotation/communication• Enhance real world task
Conclusions•Trend towards Empathic Computing
• Understanding, Experiencing, Sharing
•AR/VR Enables Empathic Experiences• Changes perspective • Sharing space/experience• Supports annotation/communication
•Many directions for future research
top related