Augmented and Virtual Reality6.S063 Engineering Interaction TechnologiesProf. Stefanie Mueller | MIT CSAIL | HCI Engineering Group

AugmentedReality(AR)

mixed reality

real environment augmented reality (AR) augmented virtuality (AV) virtual reality (VR)

AR supplements the real world VR replaces the real world

let’s look at thevery beginning…

not quite the first…

when was the first AR/VR headset developed?

<30 second brainstorming>

2012 - Occulus Rift

1965: Ivan Sutherland - ”Sword of Damocles” (first video headset)

binocular display (both eyes) displayed output

1965: Ivan Sutherland - ”Sword of Damocles” (first video headset)

• too heave to carry, thus suspended from ceiling

• designed for helicopter pilots to have light synchronized with head movement when flying in the dark

• included head & eye tracking

1965: Ivan Sutherland - ”Sword of Damocles” (first video headset)

a room where the computer can control the existence of matter

‘the ultimate display’::

(Ivan Sutherland, 1965)

Ivan created the first head mounted display…

but the idea of augmenting human senses has already around a bit longer…

1965: Marc Heiling - Sensorama (even before Sutherland)

• stereoscopic display • vibrating seat • body tilting • fragrance generating system • wind generation

1985: NASA virtual environment work station

firstdata glove

1985: NASA virtual environment work station

1985: NASA virtual environment work station

1985: NASA virtual environment work station

any ideas which tech advances made AR/VR suddenly mass-available in the last 5-10 years?

<30 second brainstorming>

• small and light-weight displays became low-cost• mobile computing: small & powerful computers for graphics

1996: Columbia - Mobile AR ”Touring Machine:”

firstmobilesystem

2000+: consumer AR / VR

commodity devices are finally sufficiently powerful, small, inexpensive

what’s next in terms of technology? how does the next AR/VR hardware look like?

<30 second brainstorming>

AR contact lenses (many challenges, max. 8 pixel right now)

display technologies

Head%attached))))))))))Body%attached)))))))))))))))))))))Spatial

hand-helddisplay

spatial see-through

display

head-worn

display

retinal display

projector

projector

[Ramesh Raskar]

retinal displayHead%attached))))))))))Body%attached)))))))))))))))))))))Spatial

hand-helddisplay

spatial see-through

display

head-worn

display

retinal display

projector

projector

[NaturalVue, ColorMax]

future• AR with digital overlay • future: VR

today• multifocal lenses • color blindness lenses • sensor lenses for body information

head-worn displaysHead%attached))))))))))Body%attached)))))))))))))))))))))Spatial

hand-helddisplay

spatial see-through

display

head-worn

display

retinal display

projector

projector

Vuzix Star 1200XL

optical see-through:• you see the real-world with your eyes in

the same way as using regular glasses • on top digital content is displayed

using a cellphone and materials from the workshop, how would you build an optical see-through display?

draw it on paper.

<2 minutes>

smart phone display

real world

semi-transparent mirror

optical see-through:

optical see-through:

display

mirror

what are some of the benefits and drawbacks of optical see through?

<30 second brainstorming>

• real world is full resolution and FOV• if you run out of battery, you still see something

benefits::

• no control over real world • you cannot display sth behind a real world object• virtual/real worlds focus at different distances

Vuzix Wrap 1200AR

video see-through:• real world is captured with cameras • camera image is digitally enhanced • you look at the camera image

full control over image you can display something behind a real world object

real world is low resolution and low field of view (FOV)

benefits::

field of view (FOV): here HoloLens, FOV < 100 degree)

hand-held displaysHead%attached))))))))))Body%attached)))))))))))))))))))))Spatial

hand-helddisplay

spatial see-through

display

head-worn

display

retinal display

projector

projector

kind of video see-throughdevice camera captures real world, then displays it on screen

markers help to align multiple devices

spatial augmented reality (SAR)Head%attached))))))))))Body%attached)))))))))))))))))))))Spatial

hand-helddisplay

spatial see-through

display

head-worn

display

retinal display

projector

projector

spatial AR:• project onto real-world objects • no need to wear or hold something • digital content directly in the real-world

spatial AR:• camera: find out where the object is • projector: enhance with digital content

[SoftAR, Punpongsanon]

Head%attached))))))))))Body%attached)))))))))))))))))))))Spatial

hand-helddisplay

spatial see-through

display

head-worn

display

retinal display

projector

projector

?[Ramesh Raskar]

if spatial AR brings digital content closer to the real-world object, what would be the next logical step in terms of technology?

<30 second brainstorming>

everything becomes a ‘display’the future?

“The ultimate display would, of course, be a room within which the computer can control the existence of matter.

A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal.”

— I. Sutherland, The Ultimate Display, Proc. IFIP 65, 506–508, 1965

and maybe even a ‘physical display’…

any many moreinteresting questions..

Liwei Chan: Frontface

Yoichi Ochiai: Augmented Mom

that’s it for today…

until wednesday• laser cut business card (HW1) • setup your micro-controller on your computer (HW2) • connect your micro-controller to wifi (HW3)

Jared’s tuesday office hour: 4.15 - 6pm

end.