gvrdc beyond vr: mixed reality with tango and hololens
TRANSCRIPT
![Page 1: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/1.jpg)
Beyond VR: Mixed Reality with Tango and HoloLens
Cyrus Lum, CTO Ant Hive GamesAaron Pulkka, CEO Rabbx Inc
![Page 2: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/2.jpg)
Beyond VR: Mixed Reality with Tango and HoloLens
• Background• Mixed Reality• Tango vs HoloLens• ReFlow• Future
![Page 3: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/3.jpg)
Cyrus Lum
![Page 4: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/4.jpg)
AR Experience
![Page 5: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/5.jpg)
Aaron Pulkka
![Page 6: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/6.jpg)
VR Experience
![Page 7: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/7.jpg)
Mixed Reality
![Page 8: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/8.jpg)
Augmented Reality
![Page 9: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/9.jpg)
The VOID
![Page 10: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/10.jpg)
The VOID – Augmented Virtuality
![Page 11: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/11.jpg)
Mixed Reality
![Page 12: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/12.jpg)
Spatially Aware Devices
![Page 13: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/13.jpg)
• Powerful Android Device• Motion Tracking• Area Learning• Depth Sensing• Walk-around VR Ready
![Page 14: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/14.jpg)
Best Entertainment
WINNERProject Tango App
Contest 2015
![Page 15: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/15.jpg)
Story-Driven Physical Hidden-Object Game
![Page 16: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/16.jpg)
Motion Tracking driven design
“In this mystery sleuthing title, you play an imprisoned spirit, trying to find peace by solving the mystery of your own death.”
“It takes place in a spooky 3D mansion, and the entire game is controlled by walking around holding Tango.”
![Page 17: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/17.jpg)
Development Kit and Lenovo Phone
![Page 18: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/18.jpg)
Tango Development Environment
SDK Options• C, Java, Unity
UX Framework Active Community
![Page 19: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/19.jpg)
Tango Motion Tracking
• Visual-Inertial Odometry (VIO)• Tracking image differences• Inertial motion sensors• Combined for greater accuracy• Limits:• Drift• No memory• Lighting
![Page 20: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/20.jpg)
Tango Area Learning
• Simultaneous Location and Mapping (SLAM)
• Location memory • Drift correction• Limits:• Mapping time• Areas change over time
![Page 21: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/21.jpg)
Tango Depth Sensing
• Point Cloud • Infrared light• Markerless AR• Limits:• Environmental• 0.5 to 4 meters
![Page 22: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/22.jpg)
Environmental Issues
• “I can’t see.” – too dark• “I am lost.” – featureless • “I am dizzy.” – too fast
![Page 23: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/23.jpg)
Tetherless Walk-Around VR
![Page 24: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/24.jpg)
Mixed Reality with Tango
Google I/O 2016
![Page 25: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/25.jpg)
• Wearable Windows PC• Spatial Mapping• Motion Tracking• Gesture Tracking• Speech Recognition
![Page 26: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/26.jpg)
Wearable Windows 10 PC
![Page 27: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/27.jpg)
HoloLens Development Environment
![Page 28: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/28.jpg)
Rest API
![Page 29: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/29.jpg)
Infrared Camera Spatial Mapping
![Page 30: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/30.jpg)
Motion and Gesture Tracking
![Page 31: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/31.jpg)
Speech Recognition
• System level commands
• User configurable commands
![Page 32: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/32.jpg)
Best Spatial Mapping
WINNERSFVR HoloLens
Hackathon 2016
![Page 33: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/33.jpg)
Behemoth for HoloLens• Enabling epic cinematic
experiences everywhere!• Discover mysterious dangers
lurking beyond your walls• Find critical resources under
and around furniture• Conquer beast to win an epic
battle
![Page 34: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/34.jpg)
Mixed Reality UX
• Forget 2D UI design!• Integrate 3D UI elements• Physical movement and gaze• Use 3D audio and “offscreen” hints• ReFlow elements to environment
![Page 35: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/35.jpg)
Mixed Reality ReFlow
Each player’s gameplay environment is unique• Artificial Intelligence• MxR layout engine• Surface reconstruction• Planar identification
![Page 36: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/36.jpg)
vs
• Phablet• Touchscreen • Android• Consumer• Infrared mapping• Visual tracking
• Wearable• Gestures• Windows• Enterprise• Infrared mapping• Visual tracking
![Page 37: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/37.jpg)
FUTURE
![Page 38: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/38.jpg)
Smart Glasses Adoption Forecast
![Page 39: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/39.jpg)
ODG R-7 Smartglasses
![Page 40: GVRDC Beyond VR: Mixed Reality with Tango and HoloLens](https://reader036.vdocument.in/reader036/viewer/2022070516/587080931a28ab57368b6447/html5/thumbnails/40.jpg)
2017?