output devices - isweet.ua.pt/bss/aulas/rva-2017/rva-output-2017-i.pdf · 2017-10-20 · output...
TRANSCRIPT
Output Devices - I
Universidade de Aveiro Departamento de Electrónica, Telecomunicações e Informática
Realidade Virtual e Aumentada 2017/2018 Beatriz Sousa Santos
What is Virtual Reality?
“A high-end user interface that involves real-time simulation and interaction through multiple sensorial channels.” (vision, sound, touch, smell, taste) (Burdea and Coiffet., 2003)
2
Output Devices: Graphics, 3-D Sound, and Haptic Displays
3
(Burdea and Coiffet., 2003)
The human senses need specialized interfaces
• Graphics displays for visual feedback
• 3-D audio hardware for localized sound
• Haptic interfaces for force and touch feedback
Only a few examples of experimental devices providing smell olfactory and taste feedback
4
The ultimate display?
"The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal.” (Ivan Sutherland, 1965)
5
Visual Displays
“A graphics display is a computer interface that presents synthetic world images to one or several users interacting with the virtual world.”
(Burdea and Coiffet., 2003)
• Personal displays: Main technologies:
- HMDs (VR/AR)
- Binoculars LEDs/OLEDs
- Monitor-based displays/active glasses LCDs
- Autostereoscopic displays lenticular/barrier
• Large volume displays:
–Workbenches
– Caves projectors
– Walls, domes
– Surface … …
6
Human Visual System
And depth perception
• Vision is the dominant sensorial channel
• Depth perception in mono images is based:
- on occlusion (one object blocks another from view)
- shadows
- textures
- motion parallax
(closer images appear to move more than distant ones)
(Burdea and Coiffet., 2003)
7
• Depth perception in stereo is based on stereopsis
(when the brain registers and fuses two images)
• Image parallax means that the two eyes register different images
(horizontal shift)
• The amount of shift depends on the “inter-pupillary distance”
(IPD) (varies for each person in the range of 53-73 mm)
• Works in the near field (to a few meters from the eye)
• 3-5% of people are stereoblind
8
(Jerald., 2016)
• Many of the perceptual cues we use to visualize 3D structures are available in 2D projections
• Such cues include: – occlusion (one object partially covering another) – perspective (point of view) – familiar size (we know the real-world sizes of many objects) – atmospheric haze (objects further away look more washed out)
• Four cues are missing from 2D media:
– stereo parallax—seeing a different image with each eye – movement parallax—seeing different images when we move the head – accommodation—the eyes’ lenses focus on the object of interest – convergence—both eyes converge on the object of interest
9
• All 3D display technologies (stereoscopic displays) provide at least stereo parallax
• Autostereoscopic displays provide the 3D image without needing any eyewear
• Volume displays provide a “real” 3D image
• VR systems use the first type (stereoscopic displays)
10 https://voxon.co/stl-rendering-voxon-vx1-3d-volumetric-display
Stereopsis
Stereo ="solid" or "three-dimensional“
opsis = appearance or sight
'binocular vision‘ , 'binocular depth perception' , 'stereoscopic depth perception'
• Stereopsis is the impression of depth that is perceived when a scene is viewed with both eyes by someone with normal binocular vision
11
• Binocular disparity is due to the different position of our two eyes
Projection plane
eyes
object
Right eye image Left eye image
12
Projection plane
eyes
object (Hearn and Baker, 2002)
• Need to present two images of the same scene
• The two images can be presented:
• at the same time on two displays (HMD)
• time-sequenced on one display (active glasses)
• spatially-sequenced on one display (auto-stereoscopic displays)
13
Implications for Stereo Viewing devices
Left eye, right eye images
(Burdea and Coiffet., 2003)
Common ways to produce a 3D sensation
• Anaglyphs: two colored images and color coded glasses (red/cyan(green))
• Two images with different light polarization and polarizing glasses
– Linear and circular
• Double frame-rate displays combined with LCD shutter glasses
• Autostereoscopic displays
– Parallax barrier and lenticular lens
• Head Mounted Displays (HMDs)
and “exotic displays”
14
• All show the right image to the right eye and the left image to the left eye!
• All these technologies provide:
• stereo parallax
• When combined with head tracking,
they can provide
• movement parallax
for a single viewer
• In Virtual Reality the technologies used are:
- Different light polarization and polarizing glasses
- Double frame-rate displays combined with LCD shutter glasses
15
Polarized light stereoscopy
• 3D movies have used polarized technology since the 1930s
• Two images are projected on the same screen through different polarizing filters
• Gray linear-polarizing filters are easily manufactured, thus correct color rendition is possible
• Two types of polarization can be used: • Linear • Circular
Vertical polarizer
Circular polarized light
http://en.wikipedia.org/ wiki/Polarized_3D_glasses
Verical polarization of light
16
• Advantages of polarized glasses:
– are generally inexpensive
– don't require any power
– don't require synchronization with the display
– do not suffer from flicker
• Disadvantages:
– The images for polarized glasses may have to share the screen simultaneously, and therefore cannot have full resolution
– There are incompatible polarized systems (circular or linear polarized)
– The head should not be tilted to maintain the 3D effect with linear polarization
17
https://www.inition.co.uk/extraordinary-technology/stereoscopic-3d-displays/
Shutter glasses for stereoscopic displays
• Active-shutter glasses are small LCD screens that alternately dim the left and right "lenses" in succession
• They are synchronized with the display usually through IR or radio signals
• Each eye can see the image intended for it
18
• Can be bought for < 100€
• Need a battery
Passive (polarized) versus active (shutter)?
• Active
– Better image quality
19
• Passive
– cheaper
– lighter
– batteryless
– syncless glasses
Visual Displays in Virtual Reality
“A graphics display is a computer interface that presents synthetic world images to one or several users interacting with the virtual world.”
(Burdea and Coiffet., 2003)
• Personal displays: Main technologies:
- HMDs (VR/AR)
- Binoculars/hand held LEDs/OLEDs
- Monitor-based displays/active glasses LCDs
- Autostereoscopic displays lenticular/barrier
• Large volume displays:
–Workbenches
– Caves projectors
– Walls, domes
– Surface … …
20
Main Properties of Visual Displays
21
Visual presentation properties:
• Color
• Spatial resolution
• Contrast
• Number of display channels
• Focal distance
• Opacity
• Masking
• Field of view (FOV)
• Field of regard (FOR)
• Head position information
• Graphics latency tolerance
• Temporal resolution
Logistic Properties:
• User mobility
• Interface with tracking
• Environment requirements
• Associability with other sense displays
• Portability
• Throughput
• Encumbrance
• Safety
• Cost
22
Focal distance
Focal distance is the measurement between the participant's eyes and the virtual image. (A) projection display - distance to the display screen. (B) HMDs create a virtual image at some distance beyond the physical display
Field of view (FOV) and Field of regard (FOR)
23
(Sherman and Craig, 2003)
FOR is a measure of the amount of coverage a given display provides when head motion and other factors are considered. (A) Head-based displays can easily provide a loo% FOR, (B) stationary displays are limited to the area of the screens
FOV is the amount of the viewer's visual field covered by a display. Head-based displays (A) tend to have smaller, fixed FOV angles compared with those possible in projection- based displays
Visual Displays: a possible taxonomy (Burdea and Coiffet, 2003)
• Personal displays:
- HMDs (VR/AR)
- Binoculars
- Monitor-based displays/active glasses
- Autostereoscopic displays
• Large volume displays:
– Workbenches
– Caves
– Walls, domes
– Surface … …
25
Visual Displays: another taxonomy (Sherman and Craig, 2003)
26
• Head-based (occlusive)
• Non-occlusive head-based
• Handheld
• Monitor- based (Fishtank)
• Projection Displays Stationary displays
Personal Displays
A Visual display that outputs a virtual scene destined to be viewed by a single user. Such image may be monoscopic or stereoscopic, monocular (for a single eye) or binocular (displayed on both eyes).
• Head Mounted Displays (HMDs)
• 3-D Binoculars (hand supported)
• Auto-stereoscopic displays
(desk supported)
27
HMD integration in a VR system
simple HMD
More professional HMD
(Burdea and Coiffet., 2003) 28
29
- Bright, crisp, high-contrast OLED display
- Stereo
- High resolution full-color SXGA 1280×1024 pixels
- 60º field of view
- Integrated yaw/pitch/roll tracker
- Self contained, lightweight (450 g)
- Integrated high-quality stereo audio and microphone
- Price ~$13000
Example of a “professional” HMD zSight™ OLED HMD (Sensics)
http://sensics.com/portfolio-posts/zsight/
The low cost Head-Mounted Displays: Oculus Rift
http://www.oculusvr.com/
2014, DK2: - low persistence OLED display to eliminate
motion blur and judder (two of the biggest contributors to simulator sickness)
- It also makes the scene appear more visually stable, increasing the potential for presence
- 960×1080 pixels per-eye display improves clarity, color, and contrast.
Price: ~~$400
31
Oculus Rift and human factors in VR
http://www.youtube.com/watch?v=iACAS_RAneE
32
Oculus Rift vs PlayStation VR: What is the best VR gaming headset?
33
https://www.wareable.com/vr/oculus-rift-vs-playstation-vr
HTC Vive
https://www.vive.com/eu/
Price: ~~$700
34
Headsets using Smartphones: Google Cardboard
35
https://en.wikipedia.org/wiki/Google_Cardboard
36
https://techcrunch.com/2016/11/10/googles-daydream-view-made-me-a-believer-again-in-consumer-vr/
Popular Augmented Reality Headsets: Google Glass (2013)
http://www.youtube.com/watch?v=JSnB06um5r4
https://www.google.com/glass/start/
https://www.wired.com/story/google-glass-2-is-here/
Display - reflects light into the wearer's eye Camera – photos and video Touchpad – on the side Micro – voice control 1500 $
Google glass (2015)
38
http://www.pcadvisor.co.uk/feature/gadget/google-glass-release-date-uk-price-specs-3436249/
39
Google glass (2017?)
https://www.wired.com/story/google-glass-2-is-here/
Meta Glasses
• 3D See Through Display
- Resolution: 960 x 540 pixels per eye
- FOV : 35 degree field of view
• Cameras
- 3D Time-of-flight depth camera
- Color (RGB) Camera
• Head Tracking
- 360 degree tracking
- Accelerometer, gyroscope and compass
• Audio
- Dolby 3D audio
- Two built-in microphones
• Now there is a new version: Meta2
40 https://www.metavision.com/
Hololens Microsoft AR glasses
https://www.microsoft.com/en-us/hololens/
Developement edition: 3000 $ Thyssenkrupp – a new vision to elevator maintenance: Preparing the visit: On site:
https://www.youtube.com/watch?v=8OWhGiyR4Ns
42
Hand Held devices
- Binoculars (older and more expensive) - Tablets/ smartphones (more and more used for AR)
(Jerald, 2016)
Virtual Binoculars
(Burdea and Coiffet., 2003) 43
• Handheld, interactive, immersive displays
• combine high-resolution microdisplays with adjustable, wide field-of-view optics
• Have a familiar form (pair of binoculars)
• May have a tracker as accessory
Examples:
Virtual Binocular SX
Virtual Binocular SV
45
Virtual Binocular SX
Price: $7,900
Virtual Binocular SV
Price: $20,900
• Two technologies:
lenticular
barrier
• Do not require use of special glasses
• Allow several vantage points
Passive - do not track user’s head (restrict user’s position)
Active - track the head motion (give more freedom)
46
Auto-stereoscopic displays
Lenticular:
– an array of cylindrical lenslets is placed in front of the pixel raster
– lenslets direct the light from adjacent pixel columns to different viewing slots at the ideal viewing distance
48
Each of the viewer’s eyes sees light from only every second pixel column
Commercial examples: - 45”; 1920 x 1080 pixels; 8 view points - 84”; 3840 x 2160 pixels; 16 view points http://www.alioscopy.com/en/3Ddisplays.php
Parallax barrier:
– a barrier mask is placed in front of the pixel raster
49
Each of the viewer’s eyes sees light from only every second pixel column
Holographic displays
• The image source is based on standard flat panel technology of which the image is seen upon glass panel with several optical layers
• Objects appear to float in space
• For the maximum 3D effect, the background seen through the display should be several feet behind the display and dark in color
50
http://www.eonreality.com/eon-holographic-i
Large Volume Displays • Allow several co-located users to view a monoscopic or stereoscopic view of
the virtual world
- monitor-based large volume displays
- projector-based large volume displays
• Allow more freedom of motion vs. personal displays.
51
Projector based Large-volume displays
53
• CAVE type displays
• Wall-type displays
• Domes
• Workbanch-type displays
CAVE 3-D large volume display (Fakespace Co.)
(Plato’s allegory of the cave, where a philosopher contemplates reality and illusion ...)
CRT Projector
Mirror
Screen
55
CAVEs
58
Examples of several CAVE configurations and prices
http://www.visbox.com/products/cave/
http://www.mechdyne.com/cave2.aspx
CAVE 2TM
“near-seamless, 320-degree,
panoramic 2D/3D virtual
environment that matches
human visual acuity . . .
The CAVE2™ is a revolutionary
system that supports
information-rich analysis with
stunning immersive visuals
and intuitive interaction tools”
• Accommodate several users
• A single projector on a large wall means small image resolution
• Tiled displays place smaller images side-by-side so they need multiple projectors
• Images need to have overlap, to assure continuity
• However overlap from two projectors means intensity discontinuity (brighter
images in the overlap areas) • Projectors need to modulate intensities to dim their light for overlap pixels
Wall-type displays
60
http://www.mechdyne.com/portable-3d-display.aspx
62
• Widely installed immersive reconfigurable visual environment
• Adequate for applications that will benefit from more than one display configuration (there is a mobile version)
• Walls can be moved independently to create new formats (7.5' x 10' )
• Has three walls and one floor screen • In < 5 min can become: - A flat wall display - An angled theater,
- An L-shape - A CAVE-like immersive room
FLEX TM - Reconfigurable immersive display
http://www.mechdyne.com/flex.aspx http://www.mechdyne.com/plex.aspx
63
Wall-type displays
Typical uses:
• Multi-disciplinary research
• Product design reviews
• Large audience presentations
• Computational fluid dynamics
• Geophysical exploration and training
• Terrain mapping and situational awareness
• Any application where multiple display modes would be beneficial …
Issue: Tiled composite image from four projectors
65
Tiled composite image from four projectors after adjustment
66
67
Dome-type display Immersadome
http://inition.co.uk/3D-Technologies/immersive-display-immersadome-range
- 3D immersive visual environment
- Up to 20 people (depending on dome size)
- Resolution: 1400 x 1050
- 180°horizontal x 135°vertical (distortion
free projection)
- Multiple control options (mouse, keyboard, joystick, tracker-ball, haptic)
- Applications:
- museums, - group training - flight or driver simulation…
Advantages: • Accommodate many users (tens to hundreds)
• Give users more freedom of motion
Disadvantages: • Large cost (up to millions of dollars)
• Even with several displays resolution is low (larger area)
Dome-type displays
68
http://pacificdomes.com/immersive-virtual-reality-dome-environments/
69
Dome type displays
VisionDome 5(m); 2 to 20 users; price > $85,000.00
http://www.vrealities.com/products/vr-domes/visiondome-5
Visual Displays: another taxonomy (Sherman and Craig, 2003)
71
• Head-based (occlusive)
• Non-occlusive head-based
• Handheld
• Monitor- based (Fishtank)
• Projection Displays Stationary displays
72
Benefits of Stationary Displays (monitor and projection based)
• Higher resolution (than most HMDs)
• Wider field of view
• Longer user endurance (i.e., can stay immersed for longer periods)
• Higher tolerance for display latency
• Greater user mobility (fewer cables)
• Less encumbering
• Lower safety risk
• Better for group viewing
• Better throughput
73
Benefits of Head-based Displays (Occlusive and Non-occlusive)
• Lower cost (for lower resolution models)
• Complete field of regard
• Greater portability
• Can be used for augmenting reality
• Can occlude the real world
• Less physical space required
• Less concern for room lighting and other environmental factors
Benefits of Hand-based Displays
• Greater user mobility
• Greater portability
• Can be combined with stationary VR displays
Main bibliography
- G. Burdea and P. Coiffet, Virtual Reality Technology, 2nd ed. Jonh Wiley and Sons, 2003
- Craig, A., Sherman, W., Will, J., Developing Virtual Reality Applications: Foundations of Effective Design, Morgan Kaufmann, 2009
- Jerald, J., The VR Book: Human-Centered Design for Virtual Reality, ACM and Morgan & Claypool, 2016
74