vr / ar fundamentals - amazon s3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · nyu...
TRANSCRIPT
![Page 1: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/1.jpg)
Michael Naimark Visiting Associate Arts Professor
Interactive Media Arts NYU Shanghai
VR / AR FundamentalsINTM-SHU 280C - 1 Topics in New Media & Entertainment Fall 2017
a practical guide to the big ideas
Jiayi Wang
Week 5: Input & Interactivity
Using our effectors and intentions as inputs and how they
shape interactive experiences.
![Page 2: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/2.jpg)
Week 1 - Introduction to VR and AR
Week 2 - FUNDAMENTALS 1: Audiovisual Resolution and Fidelity Fooling a single static eye and single static ear experiencing a framed representation.
Week 3 - FUNDAMENTALS 2: Audiovisual Spatiality and Immersion Fooling two static eyes and ears, then two moving eyes and ears, then removing the frame.
Week 4 - FUNDAMENTALS 3: Other Senses (Haptic, Smell, Taste, Mind) Fooling the other senses.
Week 5 - FUNDAMENTALS 4: Input and Interactivity Using our effectors and intentions as inputs and how they shape interactive experiences.
Week 6 - FUNDAMENTALS 5: Social and Liveness Being immersed with other people and live events.
Week 7 - Current Dilemmas This is a lead-in, “pre-production,” to the second half-semester.
First half semesterClass Structure
![Page 3: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/3.jpg)
Michael Naimark Visiting Associate Arts Professor
Interactive Media Arts NYU Shanghai
VR / AR FundamentalsINTM-SHU 280C - 1 Topics in New Media & Entertainment Fall 2017
a practical guide to the big ideas
Jiayi Wang
Week 5: Input & Interactivity
Using our effectors and intentions as inputs and how they
shape interactive experiences.
![Page 4: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/4.jpg)
VR / AR Fundamentals
Input & Interactivity Using our effectors and intentions as inputs and how they shape interactive experiences.
Jiayi Wang
Input
I/O Computation Sensors (Non-AV)
• Mechanical • Position & Movement (IMUs) • Biometric & Environment
Vision/Cameras • Body • Hands • Eyes
Sound/Microphones • Voice Control • Smart Mics
Mind • Electrodes & Big Machines • Optical Imaging & Holography • Psi & Consciousness Research
Interactivity
UI Navigation v Manipulation Symmetry Responsiveness Inference Control v Illusion of Control
![Page 5: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/5.jpg)
Mixing Board Analogy
![Page 7: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/7.jpg)
Human sensors sense signals from a system output. Human effectors effect to a system input.
![Page 8: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/8.jpg)
Human sensors sense signals from a system output. Human effectors effect to a system input.
(Not all intentional and explicit. System may infer intent.)
![Page 9: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/9.jpg)
VR / AR Fundamentals
Input & Interactivity Using our effectors and intentions as inputs and how they shape interactive experiences.
Jiayi Wang
Input
I/O Computation Sensors (Non-AV)
• Mechanical • Position & Movement (IMUs) • Biometric & Environment
Vision/Cameras • Body • Hands • Eyes
Sound/Microphones • Voice Control • Smart Mics
Mind • Electrodes & Big Machines • Optical Imaging & Holography • Psi & Consciousness Research
Interactivity
UI Navigation v Manipulation Symmetry Responsiveness Inference Control v Illusion of Control
![Page 11: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/11.jpg)
Smart Sensors Smart Cameras
Smart Mics
and much of the computation is now tiny and local.
![Page 12: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/12.jpg)
VR / AR Fundamentals
Input & Interactivity Using our effectors and intentions as inputs and how they shape interactive experiences.
Jiayi Wang
Input
I/O Computation Sensors (Non-AV)
• Mechanical • Position & Movement (IMUs) • Biometric & Environment
Vision/Cameras • Body • Hands • Eyes
Sound/Microphones • Voice Control • Smart Mics
Mind • Electrodes & Big Machines • Optical Imaging & Holography • Psi & Consciousness Research
Interactivity
UI Navigation v Manipulation Symmetry Responsiveness Inference Control v Illusion of Control
![Page 13: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/13.jpg)
Sparkfun Sensors
Buttons & Switches Encoders
Position & Movement (IMUs) (Intertial Measurement Unit)
Environment Biometric
![Page 14: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/14.jpg)
Heartmath HRV sensor
Heart Rate Variability
Empatica E4 wristband
![Page 16: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/16.jpg)
How would you measure breath?
some attempts
![Page 18: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/18.jpg)
VR / AR Fundamentals
Input & Interactivity Using our effectors and intentions as inputs and how they shape interactive experiences.
Jiayi Wang
Input
I/O Computation Sensors (Non-AV)
• Mechanical • Position & Movement (IMUs) • Biometric & Environment
Vision/Cameras • Body • Hands • Eyes
Sound/Microphones • Voice Control • Smart Mics
Mind • Electrodes & Big Machines • Optical Imaging & Holography • Psi & Consciousness Research
Interactivity
UI Navigation v Manipulation Symmetry Responsiveness Inference Control v Illusion of Control
![Page 19: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/19.jpg)
Motion Capture
Vicon IR cameras with retroreflective markers
Scatter / Depthkit Kinect (IR emitter) + DSLR
8i “Holograms” (NOT) (lots of cameras)
![Page 20: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/20.jpg)
VR / AR Fundamentals
Input & Interactivity Using our effectors and intentions as inputs and how they shape interactive experiences.
Jiayi Wang
Input
I/O Computation Sensors (Non-AV)
• Mechanical • Position & Movement (IMUs) • Biometric & Environment
Vision/Cameras • Body • Hands • Eyes
Sound/Microphones • Voice Control • Smart Mics
Mind • Electrodes & Big Machines • Optical Imaging & Holography • Psi & Consciousness Research
Interactivity
UI Navigation v Manipulation Symmetry Responsiveness Inference Control v Illusion of Control
![Page 21: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/21.jpg)
Leap Motion
technology
![Page 22: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/22.jpg)
VR / AR Fundamentals
Input & Interactivity Using our effectors and intentions as inputs and how they shape interactive experiences.
Jiayi Wang
Input
I/O Computation Sensors (Non-AV)
• Mechanical • Position & Movement (IMUs) • Biometric & Environment
Vision/Cameras • Body • Hands • Eyes
Sound/Microphones • Voice Control • Smart Mics
Mind • Electrodes & Big Machines • Optical Imaging & Holography • Psi & Consciousness Research
Interactivity
UI Navigation v Manipulation Symmetry Responsiveness Inference Control v Illusion of Control
![Page 23: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/23.jpg)
Gaze Tracking Head tracking + head position
Eye Tracking For control, for foveate imaging, and for gaze tracking
Fove
Eye Tribe video
![Page 25: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/25.jpg)
VR / AR Fundamentals
Input & Interactivity Using our effectors and intentions as inputs and how they shape interactive experiences.
Jiayi Wang
Input
I/O Computation Sensors (Non-AV)
• Mechanical • Position & Movement (IMUs) • Biometric & Environment
Vision/Cameras • Body • Hands • Eyes
Sound/Microphones • Voice Control • Smart Mics
Mind • Electrodes & Big Machines • Optical Imaging & Holography • Psi & Consciousness Research
Interactivity
UI Navigation v Manipulation Symmetry Responsiveness Inference Control v Illusion of Control
![Page 27: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/27.jpg)
Smart Mics
![Page 28: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/28.jpg)
VR / AR Fundamentals
Input & Interactivity Using our effectors and intentions as inputs and how they shape interactive experiences.
Jiayi Wang
Input
I/O Computation Sensors (Non-AV)
• Mechanical • Position & Movement (IMUs) • Biometric & Environment
Vision/Cameras • Body • Hands • Eyes
Sound/Microphones • Voice Control • Smart Mics
Mind • Electrodes & Big Machines • Optical Imaging & Holography • Psi & Consciousness Research
Interactivity
UI Navigation v Manipulation Symmetry Responsiveness Inference Control v Illusion of Control
![Page 30: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/30.jpg)
“Implanted electrodes simply won’t scale,”
“optical imaging”
Facebook “Typing with Thoughts” Goal is 100 words per minute
![Page 31: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/31.jpg)
Openwater
“optical imaging”
simple wearable with fMRI resolution
![Page 32: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/32.jpg)
Selected Psi Research Publications Dean Radin
Double Slit Experiment
Global Consciousness Project
![Page 33: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/33.jpg)
Typing with your mind at 100 words per minute?
Moving things with your mind?
![Page 34: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/34.jpg)
VR / AR Fundamentals
Input & Interactivity Using our effectors and intentions as inputs and how they shape interactive experiences.
Jiayi Wang
Input
I/O Computation Sensors (Non-AV)
• Mechanical • Position & Movement (IMUs) • Biometric & Environment
Vision/Cameras • Body • Hands • Eyes
Sound/Microphones • Voice Control • Smart Mics
Mind • Electrodes & Big Machines • Optical Imaging & Holography • Psi & Consciousness Research
Interactivity
UI Navigation v Manipulation Symmetry Responsiveness Inference Control v Illusion of Control
![Page 35: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/35.jpg)
UI = User Interface
CHI = Computer User Interface SIGCHI conference since 1982
UX = User eXperience
“X” is the new “I”
![Page 36: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/36.jpg)
VR / AR Fundamentals
Input & Interactivity Using our effectors and intentions as inputs and how they shape interactive experiences.
Jiayi Wang
Input
I/O Computation Sensors (Non-AV)
• Mechanical • Position & Movement (IMUs) • Biometric & Environment
Vision/Cameras • Body • Hands • Eyes
Sound/Microphones • Voice Control • Smart Mics
Mind • Electrodes & Big Machines • Optical Imaging & Holography • Psi & Consciousness Research
Interactivity
UI Navigation v Manipulation Symmetry Responsiveness Inference Control v Illusion of Control
![Page 37: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/37.jpg)
Navigation = interactively changing user’s orientation and position “You are a ghost.”
(Different for orientation and position in cinematic VR.)
Manipulation = interactively changing the state of the database “Moving things.” “Blowing things up.”
(Requires either 3D computer model or branching.)
![Page 38: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/38.jpg)
VR Interactivity
Some Useful Distinctions
https://medium.com/@michaelnaimark/vr-interactivity-59cd87ef9b6c
![Page 39: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/39.jpg)
VR / AR Fundamentals
Input & Interactivity Using our effectors and intentions as inputs and how they shape interactive experiences.
Jiayi Wang
Input
I/O Computation Sensors (Non-AV)
• Mechanical • Position & Movement (IMUs) • Biometric & Environment
Vision/Cameras • Body • Hands • Eyes
Sound/Microphones • Voice Control • Smart Mics
Mind • Electrodes & Big Machines • Optical Imaging & Holography • Psi & Consciousness Research
Interactivity
UI Navigation v Manipulation Symmetry Responsiveness Inference Control v Illusion of Control
![Page 42: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/42.jpg)
Conviviality: when means of production equals means of distribution
Ivan Illich, Tools for Conviviality (1972) (a driving force for the personal computer)
![Page 43: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/43.jpg)
VR / AR Fundamentals
Input & Interactivity Using our effectors and intentions as inputs and how they shape interactive experiences.
Jiayi Wang
Input
I/O Computation Sensors (Non-AV)
• Mechanical • Position & Movement (IMUs) • Biometric & Environment
Vision/Cameras • Body • Hands • Eyes
Sound/Microphones • Voice Control • Smart Mics
Mind • Electrodes & Big Machines • Optical Imaging & Holography • Psi & Consciousness Research
Interactivity
UI Navigation v Manipulation Symmetry Responsiveness Inference Control v Illusion of Control
![Page 44: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/44.jpg)
When do you think an interactive anything is not working?
![Page 45: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/45.jpg)
When do you think an interactive anything is not working?
Latency
“Dishonest” interface (bad design)
![Page 46: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/46.jpg)
VR / AR Fundamentals
Input & Interactivity Using our effectors and intentions as inputs and how they shape interactive experiences.
Jiayi Wang
Input
I/O Computation Sensors (Non-AV)
• Mechanical • Position & Movement (IMUs) • Biometric & Environment
Vision/Cameras • Body • Hands • Eyes
Sound/Microphones • Voice Control • Smart Mics
Mind • Electrodes & Big Machines • Optical Imaging & Holography • Psi & Consciousness Research
Interactivity
UI Navigation v Manipulation Symmetry Responsiveness Inference Control v Illusion of Control
![Page 48: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/48.jpg)
VR / AR Fundamentals
Input & Interactivity Using our effectors and intentions as inputs and how they shape interactive experiences.
Jiayi Wang
Input
I/O Computation Sensors (Non-AV)
• Mechanical • Position & Movement (IMUs) • Biometric & Environment
Vision/Cameras • Body • Hands • Eyes
Sound/Microphones • Voice Control • Smart Mics
Mind • Electrodes & Big Machines • Optical Imaging & Holography • Psi & Consciousness Research
Interactivity
UI Navigation v Manipulation Symmetry Responsiveness Inference Control v Illusion of Control
![Page 49: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/49.jpg)
What if you (the user/player/experiencer)
did everything as I (the author/maker) expected?
Would you know? Would you care?
![Page 51: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/51.jpg)
“Kino-Automat”, Raduz Cincera, Czech pavilion, Expo '67 Montreal
![Page 52: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/52.jpg)
“Cinelabyrinth” Raduz Cincera, Gas pavilion, Expo '90 Osaka
![Page 53: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/53.jpg)
“Cinelabyrinth” Raduz Cincera, Gas pavilion, Expo '90 Osaka
![Page 55: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/55.jpg)
VR / AR Fundamentals
For the next 2 Wednesdays, you are to experience, critique, post, and present your selected VR titles. This means:
1) Make sure your titles are here and running. (Work with platform point person.)
2) Write a brief review for each on the Documentation site.
3) Tag and rank your titles on the Catalog page. (Work with Ada and Dave.)
4) Present to class.
Week 3: Assignment for Wednesday
![Page 56: VR / AR Fundamentals - Amazon S3s3-ap-southeast-1.amazonaws.com/ima-wp/wp-content/... · NYU Shanghai VR / AR Fundamentals INTM-SHU 280C - 1 Topics in New Media & Entertainment Fall](https://reader030.vdocument.in/reader030/viewer/2022041110/5f0fd8037e708231d44629a7/html5/thumbnails/56.jpg)
VR / AR News of the Week Mon 9 Oct
VR / AR Fundamentals
Bad Content Oculus will start refunding users for bad VR content
Good Content Discovery, Google Launch Multi-Part Virtual Reality Trip Around the World ‘Everest VR’ Hits PlayStation VR Today TRIPP Raises $4 Million to Develop ‘Mood Altering’ VR Experiences
Hardware Wars Sony Isn’t ‘Entirely Comfortable’ With PSVR Market Lead By Current Margin Google, Facebook and Microsoft Line Up For VR Standalone Push How to get the new Microsoft and Samsung ‘mixed reality’ headset Snap CEO's Focus on Spectacles Underscores Hardware Ambitions
Hardware Innovations A Finnish VR startup is developing a ‘human eye-resolution' headset that is '10 years ahead of its time' Jaunt VR’s 6DOF Interactive Experience Free The Night Debuting Exclusively For Windows Mixed Reality The Basslet: a wearable subwoofer for your body
Surprises (or maybe not) Artist virtually vandalises Jeff Koons’ Snapchat artwork to protest “AR corporate invasion” Go ahead, try to guess what Magic Leap does based on this video