Visual, auditory, and haptic displays

Dr. Xiangyu Wang

Acknowledgment of Dr. Doug Bowman’s lecture notes

Introduction to displays

• Display: device which presents perceptual information

• Often ‘display’ used to mean ‘visual display’

• Goal: display devices which accurately represent perceptions in simulated world (i.e., higher levels of immersion)

Vision

• Stimulus: light of wavelengths ~350-750 nm

Visual displays for VEs

• Standard monitor (mono/stereo)

• Head-mounted/head-referenced

• Projected (usually stereo)– single-screen– multiple, surrounding screens

• Retinal display

• Volumetric displays

Characteristics of visual displays

• Field of regard (FOR), field of view (FOV)

• Brightness, contrast ratio

• Resolution

• Screen geometry

• Light transfer

• Refresh rate

• Ergonomics

VR with a monitor

Head-mounted displays (HMDs)

• full surround (FOR=360)

• simple stereo

• cumbersome

• tethering to computer

• often small FOV

• single user

Surround-screen displays

• less obtrusive headgear

• multi-user

• better stereo

• occlusion problem

• missing sides

Surround-screen displays

• Video:

Six-sided CAVE (Duke U. DiVE)

Tabletop displays

• direct manipulation

• “god’s-eye” view

• change orientation

• small FOR

Virtual retinal display (VRD)

• HIT lab / Microvision

• image scanned directly onto retina

• great potential

Volumetric display

• Pixels displayed in actual 3D space

• Multi-user correct viewing

• Size issues

• Can’t move/reach into display

Volumetric display

• Video

Which visual display to use?

• Consider lists of pros and cons

• Consider depth cues supported

• Consider level of visual immersion

• But this is a very hard question to answer empirically

• Instead of comparing actual displays, compare levels of immersion

Auditory displays

• Second most studied sense

• Second most common VE display

• Stimulus: disturbance of molecules in a medium (air)

• Perceptions: pitch, loudness, location

Simple VE audio

• Intensity fall-off (1/d2)

• Headphones also block out real-world noises

• Ambient sound (e.g. stream)

• Present speech instead of text

3D auditory displays

• Technologies:– Speaker-based– Headphone-based

• Uses:– Virtual objects emitting sound– Sensory substitution

Auditory displays

• Video clip (1) shows the actual physical environment in which the virtual reality system was installed, and shows a participant during the course of the exploratory phase of the experiment.

• Video clip (2) shows the virtual environment during several phases of the experimental procedure. The virtual scene is shown from both the participants subjective view and also the overhead view. The virtual audio scene is also included and is rendered for the participant's position in the scene.

Haptic Displays

• Touch– Actually a range of skin senses– touch / pressure– hot / cold– pain

• Useful for object identification & understanding

Haptic displays

• Exoskeleton• Robot arms• Phantom• Tactile devices• Video: haptic

Prototyping– http://

www.youtube.com/watch?v=-i9Wm2rTsao

SPIDAR haptic display

Passive haptics

• Use of props - “poor man’s” haptic

• Display– ARtoolkit– pen & tablet

Haptic displays

• Displaying to other skin senses

• Simple, special-purpose “displays” for temperature, air movement, etc.– Fan– Heat lamp

Olfactory displays

• Sense of smell not studied extensively -use in VEs?

• A few academic projects

• Smell synthesis still in the future

Olfactory displays

• Video: "Fragra": Entertainment System Utilizing Olfactory Display by Arito Mochizuki

Vestibular/kinesthetic displays

• Sense of body, self-motion

• Virtual body representation

• Can “display” to these senses by:– Using motion platforms– Stimulating the proper parts of the brain

Conclusions

• Making VEs multi-sensory is becoming easier becoming cheaper

• But most VEs still use only visual display

• We need to study the effects of level of auditory and haptic immersion