dive virtual reality lab - duke universitypeople.duke.edu/~djzielin/slides/itac_2_22_2018.pdf ·...

20
David J. Zielinski Research and Development Engineer for the DiVE DiVE Virtual Reality Lab Duke ITAC Meeting 2/22/2018 1 Dr. Regis Kopper Assistant Research Professor of MEMS, and Computer Science Director of the DiVE

Upload: others

Post on 18-Jan-2021

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

David J. ZielinskiResearch and Development Engineer for the DiVE

DiVE Virtual Reality LabDuke ITAC Meeting

2/22/2018

1

Dr. Regis KopperAssistant Research Professor of MEMS, and Computer ScienceDirector of the DiVE

Page 2: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

2

Media Hype (1992,1999)

History of VR

Ivan Sutherlandfirst HMD (1968)

https://en.wikipedia.org/wiki/Hype_cycle

Now?

Trough of Disillusionment2000s

Oculus acquired by Facebook for $2B (2014)

Page 3: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

DiVE Lab Facilities

Duke immersive Virtual EnvironmentEstablished 2005, Upgraded 2015

OculusCV-1

HTCVIVE

MicrosoftHololens

Zspace

3

Consumer VR Devices

Page 4: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

“A Visit to the DiVE” (Summer 2016)

4

Page 5: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

VR Display Device Trade-offs?

5

CAVE-type systems (Duke DiVE)See own body (presence, motor tasks)See others (collaboration) Reduced sickness

Consumer VR Devices (HTC Vive, Oculus)Consumer ProductTranslational (expand outside the lab)Affordable

Page 6: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

A few recent DiVE projects

6

Page 7: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

7EmmanuelShiferaw

Dig@IT: Virtual Reality in ArchaeologyBass Connections. Oculus HMD. (2016)

DiVE version (2017)

Virtual Archeology: Çatalhöyük

Prof. MaurizioForte

Prof. RegisKopper

Nicola Lercari

Page 8: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

Art History: Santa Chiara VR

LucasGiles

8Add in lost choir screen (Spring 2017)Prof. CarolineBruzelius

Elisa Castagna

Andrea Basso

David Zielinski

Page 9: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

Blood Flow Through Arteries (Spring 2017)

BME 307: Transport Phenomena in Biological Systems

9

Prof. Amanda Randles

Page 10: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

Molecules in the DiVE (Fall 2017)

10

NEUROBIO 719: Concepts in Neuroscience I

Prof Jorg GrandlDavid Zielinski

Page 11: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

● Hard to do tangible interfaces in world-fixed systems (visible to user).

● Clear acrylic box● User can touch the box ● Can't reach contents● User study showed

○ users faster○ less rotations ○ less presence

Interaction Technique: Specimen Box (2017)

11

Inspiration: Historical “Specimen Jars”Late 1700s.

Published as: "Specimen box: A tangible interaction technique for world-fixed virtual reality displays". David J. Zielinski, Derek Nankivil, Regis Kopper. In 2017 IEEE Symposium on 3D User Interfaces (3DUI).

Patented as: PCT/US2017/053803.

David Zielinski

Derk Nankivil

Regis Kopper

Page 12: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

12

Now with EEG! (P3+P4 electrodes show decrease over training)

Marksmanship in VR

Time spent training shows changes in time spent in ballistic vs refinement phases

Rao, H.M., Khanna, R., Zielinski, D.J., Lu, Y., Clements, J.M., Potter, N.D.D., Sommer, M.A., Kopper, R. and Appelbaum, L.G., 2018. Sensorimotor learning during a marksmanship task in immersive virtual reality. Frontiers in Psychology, 9, p.58.

Clements, J. M., Kopper, R., Zielinski, D. J., Rao, H., Sommer, M. A., Kirsch, E., Mainsah, B. O., Collins, L. M., and Appelbaum, L. G. Neurophysiology of visual-motor learning during a simulated marksmanship task in immersive virtual reality. In Proceedings of the 25th IEEE Conference on Virtual Reality and 3D User Interfaces, March 18-22, 2018, Reutlingen, Germany.

Page 13: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

A few recent HMD projects

13

Page 14: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

Motion Sickness Reduction Via Rest Frames

14

● Motion Sickness is a side effect of VR● Rest Frames are stable scene references,

which may reduce symptoms

Jason JeraldZekun Cao Regis Kopper

Z. Cao, J. Jerald, and R. Kopper. Visually-induced motion sickness reduction via static and dynamic rest frames. In Proceedings of the 2018 Conference on Virtual Reality and 3D User Interfaces, 2018.

User study found that rest frame helped reduce MS, and caused less sickness on follow-up experiences without rest frame

Page 15: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

PanoVR: 360 Video. Multiple Users.

15

Visualize View Frustums of Other Users Annotation Call to Attention

Regis Kopper

Alex Blumenstock

MitchellBerger

Isaac Andersen

RayanTofique

Initially funded by DDI; won IgniteNC reverse pitch competition, allowing continued funding through 2018

Working on establishing 10G Link from Duke to Sao Paulo: Streaming 360 video.

Page 16: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

Eye Tracking + SQUAD

Ray-casting. Common VR selection technique. Bad performance in cluttered/dense environment.

SQUAD. Sphere-casting refined by QUAD menu selection technique (Regis Kopper et al. 2011). 2D Select from among 4 screen areas. Each refinement step has less objects in it.

EyeSQUAD. Eye-controlled sphere-casting refined by QUAD-menu (EyeSQUAD) selection technique. FOVE headset.

Yunhan Wang

Prof. Regis Kopper

Page 17: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

Augmented/Mixed Reality: EVD Placement Training

17

Microsoft Hololens

Patented as: PCT/US18/17381

DavidZielinski

RegisKopper

AndrewCutler

NandanLad

PatrickCodd

ShervinRahimpour

Page 18: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

ClassesME 555.05: Virtual Reality Systems Research and Design (Spring 2017, Spring 2018)ISS 320S: Introduction to Programming and User Interface Design in Unity 3D

ISS 320S Final Projects Fall 2016https://youtu.be/S6MFgpLoT4Y 18

ISS320 Final Project Fall 2017https://youtu.be/IyXZ0jfFvE4

Page 19: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

Discussion points

19

Sickness is the biggest challenge for VR- Limits immersion time and enjoyment of experience- Potential software solutions?- Potential hardware solutions?

Display trade-offs: HMDs vs. CAVEs (like the DiVE)- HMDs are getting widely adopted, but still issues remain: encumbrance, hygiene, embodiment, safety- CAVEs are expensive and not highly available, but there are advantages: multiple users, less sickness, embodiment

What about accessibility?- VR is visually focused, are we leaving out the blind?- Requirement for physical interaction is an issue- Increase realism of other senses in VR (audio, haptics)- Work with experts to explore potential accommodations- Determine legal rules and precedents, to reduce litigation potential

Page 20: DiVE Virtual Reality Lab - Duke Universitypeople.duke.edu/~djzielin/slides/ITAC_2_22_2018.pdf · Prof. Caroline Add in lost choir screen (Spring 2017) 8 Bruzelius Elisa Castagna Andrea

Thank you!

20

DiVE Open HousesThursday 5pm-6pm

David J. [email protected]://people.duke.edu/~djzielin/

Dr. Regis [email protected]

http://virtualreality.duke.edu/