David J. ZielinskiResearch and Development Engineer for the DiVE

DiVE Virtual Reality LabDuke ITAC Meeting

2/22/2018

1

Dr. Regis KopperAssistant Research Professor of MEMS, and Computer ScienceDirector of the DiVE

2

Media Hype (1992,1999)

History of VR

Ivan Sutherlandfirst HMD (1968)

https://en.wikipedia.org/wiki/Hype_cycle

Now?

Trough of Disillusionment2000s

Oculus acquired by Facebook for $2B (2014)

DiVE Lab Facilities

Duke immersive Virtual EnvironmentEstablished 2005, Upgraded 2015

OculusCV-1

HTCVIVE

MicrosoftHololens

Zspace

3

Consumer VR Devices

“A Visit to the DiVE” (Summer 2016)

4

VR Display Device Trade-offs?

5

CAVE-type systems (Duke DiVE)See own body (presence, motor tasks)See others (collaboration) Reduced sickness

Consumer VR Devices (HTC Vive, Oculus)Consumer ProductTranslational (expand outside the lab)Affordable

A few recent DiVE projects

6

7EmmanuelShiferaw

Dig@IT: Virtual Reality in ArchaeologyBass Connections. Oculus HMD. (2016)

DiVE version (2017)

Virtual Archeology: Çatalhöyük

Prof. MaurizioForte

Prof. RegisKopper

Nicola Lercari

Art History: Santa Chiara VR

LucasGiles

8Add in lost choir screen (Spring 2017)Prof. CarolineBruzelius

Elisa Castagna

Andrea Basso

David Zielinski

Blood Flow Through Arteries (Spring 2017)

BME 307: Transport Phenomena in Biological Systems

9

Prof. Amanda Randles

Molecules in the DiVE (Fall 2017)

10

NEUROBIO 719: Concepts in Neuroscience I

Prof Jorg GrandlDavid Zielinski

● Hard to do tangible interfaces in world-fixed systems (visible to user).

● Clear acrylic box● User can touch the box ● Can't reach contents● User study showed

○ users faster○ less rotations ○ less presence

Interaction Technique: Specimen Box (2017)

11

Inspiration: Historical “Specimen Jars”Late 1700s.

Published as: "Specimen box: A tangible interaction technique for world-fixed virtual reality displays". David J. Zielinski, Derek Nankivil, Regis Kopper. In 2017 IEEE Symposium on 3D User Interfaces (3DUI).

Patented as: PCT/US2017/053803.

David Zielinski

Derk Nankivil

Regis Kopper

12

Now with EEG! (P3+P4 electrodes show decrease over training)

Marksmanship in VR

Time spent training shows changes in time spent in ballistic vs refinement phases

Rao, H.M., Khanna, R., Zielinski, D.J., Lu, Y., Clements, J.M., Potter, N.D.D., Sommer, M.A., Kopper, R. and Appelbaum, L.G., 2018. Sensorimotor learning during a marksmanship task in immersive virtual reality. Frontiers in Psychology, 9, p.58.

Clements, J. M., Kopper, R., Zielinski, D. J., Rao, H., Sommer, M. A., Kirsch, E., Mainsah, B. O., Collins, L. M., and Appelbaum, L. G. Neurophysiology of visual-motor learning during a simulated marksmanship task in immersive virtual reality. In Proceedings of the 25th IEEE Conference on Virtual Reality and 3D User Interfaces, March 18-22, 2018, Reutlingen, Germany.

A few recent HMD projects

13

Motion Sickness Reduction Via Rest Frames

14

● Motion Sickness is a side effect of VR● Rest Frames are stable scene references,

which may reduce symptoms

Jason JeraldZekun Cao Regis Kopper

Z. Cao, J. Jerald, and R. Kopper. Visually-induced motion sickness reduction via static and dynamic rest frames. In Proceedings of the 2018 Conference on Virtual Reality and 3D User Interfaces, 2018.

User study found that rest frame helped reduce MS, and caused less sickness on follow-up experiences without rest frame

PanoVR: 360 Video. Multiple Users.

15

Visualize View Frustums of Other Users Annotation Call to Attention

Regis Kopper

Alex Blumenstock

MitchellBerger

Isaac Andersen

RayanTofique

Initially funded by DDI; won IgniteNC reverse pitch competition, allowing continued funding through 2018

Working on establishing 10G Link from Duke to Sao Paulo: Streaming 360 video.

Eye Tracking + SQUAD

Ray-casting. Common VR selection technique. Bad performance in cluttered/dense environment.

SQUAD. Sphere-casting refined by QUAD menu selection technique (Regis Kopper et al. 2011). 2D Select from among 4 screen areas. Each refinement step has less objects in it.

EyeSQUAD. Eye-controlled sphere-casting refined by QUAD-menu (EyeSQUAD) selection technique. FOVE headset.

Yunhan Wang

Prof. Regis Kopper

Augmented/Mixed Reality: EVD Placement Training

17

Microsoft Hololens

Patented as: PCT/US18/17381

DavidZielinski

RegisKopper

AndrewCutler

NandanLad

PatrickCodd

ShervinRahimpour

ClassesME 555.05: Virtual Reality Systems Research and Design (Spring 2017, Spring 2018)ISS 320S: Introduction to Programming and User Interface Design in Unity 3D

ISS 320S Final Projects Fall 2016https://youtu.be/S6MFgpLoT4Y 18

ISS320 Final Project Fall 2017https://youtu.be/IyXZ0jfFvE4

Discussion points

19

Sickness is the biggest challenge for VR- Limits immersion time and enjoyment of experience- Potential software solutions?- Potential hardware solutions?

Display trade-offs: HMDs vs. CAVEs (like the DiVE)- HMDs are getting widely adopted, but still issues remain: encumbrance, hygiene, embodiment, safety- CAVEs are expensive and not highly available, but there are advantages: multiple users, less sickness, embodiment

What about accessibility?- VR is visually focused, are we leaving out the blind?- Requirement for physical interaction is an issue- Increase realism of other senses in VR (audio, haptics)- Work with experts to explore potential accommodations- Determine legal rules and precedents, to reduce litigation potential

Thank you!

20

DiVE Open HousesThursday 5pm-6pm

David J. Zielinskidjzielin@duke.eduhttp://people.duke.edu/~djzielin/

Dr. Regis Kopperregis.kopper@duke.edu

http://virtualreality.duke.edu/