effects of handling real objects and avatar fidelity on cognitive task performance in virtual...
TRANSCRIPT
![Page 1: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/1.jpg)
Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task
Performance in Virtual Environments
Benjamin Lok
University of North Carolina at Charlotte
Samir Naik
Disney VR Studios
Mary Whitton, Frederick P. Brooks Jr.
University of North Carolina at Chapel Hill
March 25th, 2003
![Page 2: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/2.jpg)
Current ImmersiveVE Approaches
• System has limited shape and motion information of real objects.
• Most objects in the VE are purely virtual– User– Tools– Parts
• Most virtual objects are not registered with a corresponding real object.
![Page 3: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/3.jpg)
Ideally
• Would like:– Real objects in the VE, including accurate
virtual representations, or avatars– Virtual objects responding to real objects– Haptic feedback– Correct affordances– Constrained motion
• Example: Unscrewing a virtual oil filter from a car engine model
![Page 4: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/4.jpg)
Dynamic Real Objects
• By tracking and modeling dynamic objects (change shape and appearance) would– Improve interactivity– Enable visually faithful
virtual representations
![Page 5: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/5.jpg)
Previous Work: Incorporating Real Objects into VEs
• Non-Real Time– Virtualized Reality (Kanade, et al.)
• Real Time– Image Based Visual Hulls [Matusik00, 01]– 3D Tele-Immersion [Daniilidis00]
• Augment specific objects for interaction– Doll’s head [Hinkley94]– Plate [Hoffman98]
• How important is to get real objects into a virtual environment?
![Page 6: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/6.jpg)
Previous Work: Avatars• Self - Avatars in VEs
– What makes avatars believable? [Thalmann98]
– What avatars components are necessary? [Slater93, 94, Garau01]
• VEs currently have:– Choices from a library– Generic avatars– No avatars
• Generic avatars > no avatars [Slater93]
• Are visually faithful avatars better than generic avatars?
![Page 7: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/7.jpg)
Object Reconstruction System
• System takes n live camera images.
• At each frame, generates a view of the visual hull of the real objects from the user.
• It is combined with the virtual environment
• 15-18 fps• Allows incorporation of
dynamic real objects into a virtual environment
![Page 8: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/8.jpg)
Spatial Cognitive Task StudySpatial Cognitive Task Study
![Page 9: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/9.jpg)
Study Motivation
• Effects of– Interacting with real objects– Visual fidelity of self-avatars
• On– Task Performance
• For spatial cognitive manual tasks
![Page 10: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/10.jpg)
Spatial Cognitive Manual Tasks
• Spatial Ability– Visualizing a manipulation in 3-space
• Cognition– Psychological processes involved in the
acquisition, organization, and use of knowledge
![Page 11: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/11.jpg)
Task Performance Hypotheses
Real Objects: Participants will complete a spatial cognitive manual task faster when manipulating real objects, as opposed to virtual objects only.
Avatar Fidelity: Participants will complete a spatial cognitive manual task faster with visually faithful self-avatars, as opposed to generic avatars.
![Page 12: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/12.jpg)
Task
• Manipulated identical painted blocks to match target patterns
• Each block had six distinct patterns.
• Target patterns:– 2x2 blocks (small)– 3x3 blocks (large)
![Page 13: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/13.jpg)
Measures• Task performance
– Time to complete the patterns correctly – If errors were made:
• Participants were notified• Participants continued until all errors were corrected
• Other factors– (After experience) Steed-Usoh-Slater Sense of
Presence Questionnaire (SUS) (larger study)– (Before experience) spatial ability– (Before and after experience) simulator sickness
![Page 14: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/14.jpg)
Conditions
• All participants did the task in a real space environment.
• Each participant did the task in one of three VEs.
Real Space
Purely Virtual
Hybrid
Vis. Faithful Hybrid
![Page 15: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/15.jpg)
Conditions
Avatar Fidelity
GenericVisually
faithful
Interact with
Real objects
HE VFHE
Virtual objects
PVETask performance
Task performance
![Page 16: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/16.jpg)
Real Space Environment
• Task was conducted within a draped enclosure
• Participant watched monitor while performing task
• RSE performance was a baseline to compare against VE performance
![Page 17: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/17.jpg)
Purely Virtual Environment
• Participant manipulated virtual objects• Participant was presented with a generic avatar
![Page 18: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/18.jpg)
Hybrid Environment
• Participant manipulated real objects• Participant was presented with a generic avatar
![Page 19: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/19.jpg)
Visually-Faithful Hybrid Env.
• Participant manipulated real objects• Participant was presented with a visually faithful avatar
![Page 20: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/20.jpg)
Task Performance Results
Small Pattern Time (seconds) Large Pattern Time (seconds)
Mean S.D. Mean S.D.
Real Space (n=41) 16.8 6.3 37.2 9.0
Purely Virtual (n=13) 47.2 10.4 117.0 32.3
Hybrid (n=13) 31.7 5.7 86.8 26.8
Visually Faithful Hybrid (n=14) 28.9 7.6 72.3 16.4
![Page 21: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/21.jpg)
Small Pattern Time Large Pattern Time
T-test p T-test p
Purely Virtual vs. Vis. Faithful 3.32 0.0026** 4.39 0.00016***
Purely Virtual vs. Hybrid 2.81 0.0094** 2.45 0.021*
Hybrid vs. Vis. Faithful Hybrid 1.02 0.32 2.01 0.055
* - significant at the =0.05 level ** - =0.01 level *** - =0.001 level
Task Performance Results
![Page 22: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/22.jpg)
Debriefing Responses• They felt almost completely immersed while performing the task.
• They felt the virtual objects in the virtual room (such as the painting, plant, and lamp) improved their sense of presence, even though they had no direct interaction with these objects.
• VFHE and HE participants felt tactile feedback of working with real objects improved their performance and sense of presence.
• They felt that seeing an avatar added to their sense of presence.
• PVE and HE participants commented on the fidelity of motion, whereas VFHE participants commented on the fidelity of appearance.
• VFHE participants reported getting used to manipulating and interacting in the VE significantly faster than PVE participants.
![Page 23: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/23.jpg)
Conclusions
![Page 24: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/24.jpg)
Study Conclusions• Interacting with real objects provided a quite
substantial performance improvement over interacting with virtual objects for cognitive manual tasks.
• Avatar fidelity does not appear to affect cognitive task performance.
Handling real objects makes task performance and interaction in the VE
more like the actual task.
![Page 25: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/25.jpg)
Future Work
• Improved Object Reconstruction System– model fidelity– lag and latency (could improve results)
• Further studies to illuminate:– Effect of real objects and avatar fidelity on:
• Sense of Presence
– Expand on the relationship between avatar kinematic fidelity and visual fidelity
![Page 26: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments Benjamin Lok University of North Carolina at](https://reader035.vdocument.in/reader035/viewer/2022062719/56649edf5503460f94bef907/html5/thumbnails/26.jpg)
ThanksCollaborators
Dr. Larry F. Hodges
UNC-CH Effective Virtual Environments
Object Reconstruction SystemI3D2001 and I3D2003
For more information:http://www.cs.uncc.edu/~bclok/research/vr2003
Funding Agencies The LINK Foundation
NIH (Grant P41 RR02170)
National Science Foundation
Office of Naval Research