mobile day - webvr
TRANSCRIPT
![Page 1: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/1.jpg)
VR/AR/Mixed RealityRabimba Karanjai@rabimba
![Page 2: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/2.jpg)
VR/AR/MR in a NutshellRabimba Karanjai, Mozilla Research Mixed Reality Mozilla Tech Speaker@rabimba
![Page 3: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/3.jpg)
What the Heck Doodle?
ARKit ARCore
MRVR
XRAR
OculusSonyVive
Meta
HoloLens
![Page 4: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/4.jpg)
What You’ll Learn Today● What is VR / AR / MR?● Why is this all happening now?● What do I really need to know?
![Page 5: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/5.jpg)
What is VR/AR/MR?
VR: Virtual Reality AR: Augmented Reality
MR: Mixed Reality XR: We don’t know Reality
![Page 6: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/6.jpg)
Virtual Reality● Completely immersive● Wide field of view● Fast refresh rate (90+fps)● Fast sensor refresh
![Page 7: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/7.jpg)
Augmented Reality● Overlaid on real world● Anchored to real world● Non-immersive
![Page 8: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/8.jpg)
![Page 9: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/9.jpg)
Partial AR
![Page 10: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/10.jpg)
![Page 11: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/11.jpg)
![Page 12: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/12.jpg)
![Page 13: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/13.jpg)
Why Now?
![Page 14: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/14.jpg)
The Long History of VR/AR/MR
Ivan Sutherland, MIT
Sword of Damocles (1968)
3D wireframe rooms
![Page 16: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/16.jpg)
1980s: VPL
● Data Glove● Eye Phone● Data Suit
![Page 17: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/17.jpg)
Mattel’s Nintendo Power Glove
![Page 18: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/18.jpg)
1990s: Virtuality
![Page 19: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/19.jpg)
![Page 20: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/20.jpg)
Virtual Reality Markup Language
● Scene based● In the browser● Slow to download● Slow to render
![Page 21: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/21.jpg)
![Page 22: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/22.jpg)
Smartphone Era
![Page 23: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/23.jpg)
Smartphone+ Era
● Dual cameras● Depth sensing cameras● Machine Learning in
hardware● Even more powerful
GPU & CPU
![Page 24: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/24.jpg)
What do I need to know?
![Page 25: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/25.jpg)
Tech Stack for VR & AR
Input: Sensors & Camera Processing: Image & Sensor Fusion
Drawing: 3D Graphics API Application: Framework
![Page 26: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/26.jpg)
Input: Sensors & Camera● Accelerometer: relative to gravity● Magnetometer: relative to earth’s magnetic field● Gyroscope: relative angle movement
○ 9DOF sensor breakout board for 15$ @ Adafruit!
● Optical Camera: standard phone camera● Stereo Camera: dual camera setup● Depth Sensing Camera: Kinect, Face ID
![Page 27: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/27.jpg)
Processing the 3D Environment
● Markers: Symbols or Beacons● SLAM: Simultaneous Localization and Mapping● MonoSLAM: SLAM in real-time with just one camera
![Page 28: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/28.jpg)
HTC Vive Sensor Beacontwo beacons + headset sensors = highly accurate
![Page 29: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/29.jpg)
Processing: VR/AR APIs● Vuforia● ARToolkit● WebVR● ARKit● ARCore● WebXR
![Page 30: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/30.jpg)
Vuforia
![Page 31: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/31.jpg)
ARToolkit
![Page 32: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/32.jpg)
WebVR
● Mobile: used by web apps for Google Cardboard on any phone.
● Desktop: works with major HMDs
● Shipping: Firefox for Windows
● Beta: Chrome, FF for other platforms
![Page 35: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/35.jpg)
WebXR: Work In Progress!● Proposed Future Web Standard● Extend WebVR with new features● Work with ARKit & ARCore● Polyfill for old browsers using WebRTC + sensors
https://github.com/mozilla/webxr-apihttps://github.com/mozilla/webxr-polyfill
![Page 36: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/36.jpg)
Drawing: Graphics APIs
● Medium Level○ OpenGL (ES)○ DirectX○ WebGL
● Low Level○ Metal○ Vulkan○ Direct X 12
● High Level○ Unity○ Unreal○ Three.js○ Babylon.js
![Page 37: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/37.jpg)
Application: Frameworks● Scene Kit● Cardboard & Daydream● Hololens Academy● React VR● ArgonJS● A-Frame
![Page 38: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/38.jpg)
What do you *really* need to know?
● Start with web stack● Follow Mozilla and Google and Microsoft● Play with A-Frame● Get a Cardboard viewer.
![Page 39: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/39.jpg)
What can I do?● Experiment! We are still in the Model T days!● No one knows what the killer app will be.● First new medium in decades● Go play!
![Page 40: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/40.jpg)
State of WebVR and AframeThe web framework for building VR experiences
@rabimba | Mozilla VR | aframe.io
![Page 41: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/41.jpg)
Friction of VR Ecosystems
Gatekeepers Installs Closed
![Page 42: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/42.jpg)
WebVRAn open virtual reality platform with the advantages of
the Web
Open Connected Instant
![Page 43: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/43.jpg)
Browser APIs that enable WebGL rendering to headsetsand access to VR sensors
![Page 45: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/45.jpg)
FirefoxNightly
Chromium(Experimental)
SamsungInternet
MobilePolyfill
![Page 46: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/46.jpg)
A-FrameA declarative framework for building virtual reality
experiences on the Web
![Page 47: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/47.jpg)
Hello World<a-scene>
<a-box color="#4CC3D9" position="-1 0.5 -3" rotation="0 45 0"></a-box>
<a-cylinder color="#FFC65D" position="1 0.75 -3" radius="0.5" height="1.5
<a-sphere color="#EF2D5E" position="0 1.25 -5" radius="1.25"></a-sphere>
<a-plane color="#7BC8A4" position="0 0 -4" rotation="-90 0 0" width="4" h
<a-sky color="#ECECEC"></a-sky>
</a-scene>
![Page 48: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/48.jpg)
Hello World
![Page 49: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/49.jpg)
<a-animation> <a-box> <a-camera> <a-circle> <a-collada-model>
<a-cone> <a-cursor> <a-curvedimage> <a-cylinder>
<a-dodecahedron> <a-isocahedron> <a-image> <a-light>
<a-obj-model> <a-plane> <a-ring> <a-tetrahedron> <a-torus>
<a-torus-knot> <a-sky> <a-sound> <a-sphere> <a-videosphere>
![Page 52: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/52.jpg)
RegistryCurated collection of A-Frame components
![Page 53: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/53.jpg)
Motion CaptureRecord then develop room scale VR on the go
Remix Glitch
![Page 55: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/55.jpg)
Powerful and Extensible
![Page 56: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/56.jpg)
![Page 57: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/57.jpg)
Art - A-Painter@mozillavr
![Page 58: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/58.jpg)
Journalism - Fear of the SkyAmnesty International UK
![Page 59: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/59.jpg)
Journalism - Journey toMars
The Washington Post
![Page 60: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/60.jpg)
Guri VRBuild your own world in VR
![Page 61: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/61.jpg)
Sandbox - Aincraft@andgokevin
![Page 62: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/62.jpg)
AincraftLets play
![Page 63: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/63.jpg)
Data Visualization - Adit@datatitian
![Page 64: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/64.jpg)
Gaming - A-Blast@mozillavr
![Page 65: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/65.jpg)
AR - AR.js + A-Frame@jerome_etienne
![Page 66: Mobile Day - WebVR](https://reader031.vdocument.in/reader031/viewer/2022020411/5a64812f7f8b9a57568b49a3/html5/thumbnails/66.jpg)
Come build WebVR with us!
aframe.io/community