collaborative immersive analytics
TRANSCRIPT
COLLABORATIVE IMMERSIVE ANALYTICS
Mark [email protected]
November 7th 2017Workshop on Immersive Analytics
International Symposium on Big Data AnalyticsAdelaide, Australia
Visualization Today
Collaborative VisualizationThe shared use of computer-supported, (interactive,) visual representations of data by more than one person with the common goal of contribution to joint information processing activities.
Isenberg, P., Elmqvist, N., Scholtz, J., Cernea, D., Ma, K. L., & Hagen, H. (2011). Collaborative visualization: definition, challenges, and research agenda. Information Visualization, 10(4), 310-326.
Petra Isenberg, 2011
Example of Collaborative Visualization
Multi-disciplinary Research
• Collaborative Visualisation related to:• Scientific Visualisation• Information Visualisation• Visual Analytics
• All well established fields over last 20 years• But little existing research in Collaborative Visualisation
• E.g. from 1990 – 2010, over 1600 papers in main Viz conferences• Only 34 papers published in Collaborative Visualisation, ~2%
Collaborative Immersive Analytics (CIA) The shared use of new immersive interaction and display technologies by more than one person for supporting collaborative analytical reasoning and decision making.
• Key properties• Use of immersive technologies• Computer supported collaboration• Analytical reasoning and decision making
Immersive Technologies
Milgram’s Reality-Virtuality continuum
Mixed Reality
Reality - Virtuality (RV) Continuum
RealEnvironment
AugmentedReality (AR)
AugmentedVirtuality (AV)
VirtualEnvironment
"...anywhere between the extrema of the virtuality continuum."
P. Milgram and A. F. Kishino, Taxonomy of Mixed Reality Visual Displays IEICE Transactions on Information and Systems, E77-D(12), pp. 1321-1329, 1994.
Collaborative Immersive Analytics
• Relationship to Mixed Reality, Visual Analytics, CSCW
CIA Example: EVL CAVE2
Marai, G. E., Forbes, A. G., & Johnson, A. (2016, March). Interdisciplinary immersive analytics at the electronic visualization laboratory: Lessons learned and upcoming challenges. In Immersive Analytics (IA), 2016 Workshop on (pp. 54-59). IEEE.
CAVE2 Hardware and Software
• Hardware• 36 computers drive 72 LCD panels – 320 degree display• 74 megapixel 2D / 37 megapixel passive 3D hybrid reality environment• 14 Vicon tracking cameras – tracking up to 6 people/objects, 20.2 audio
• Software• OmegaLib – open source software driving CAVE2, rendering, input, etc.• SAGE2 – browser based collaboration/interaction platform• Support for hybrid devices + multi-user input
SAGE 2 Software Platform
• https://www.youtube.com/watch?v=V9zGmQpaRUU
ENDURANCE Case Study (2 Days)
• Working with NASA team • explore Lake Bonney in the McMurdo Dry Valleys of Antarctica• ice covered, used Auto. Underwater Vehicle to collect sonar data
• CAVE2 used as hybrid CIA system• CAVE walls – shared data representation• Laptops on tables – private workshop, individual data analysis• Shared VR data exploration – virtually swimming through data• Subgroups form at large screen to analyze data
“.. the team got more done in 2 days than in 6 months of email, Skype, and Google Hangout.”
Types of CIA systems
• Classify system using CSCW Space-Time Taxonomy
1. Co-Located Synchronous Collaboration
• Same time/Same place collaboration• E.g. CAVE2, shared tables, interactive walls
• Advantages• Shared awareness, use external tools (laptop, notes)• Easy moving between individual and group work
Reality VirtualityAugmented Reality (AR)
Augmented Virtuality (AV)
AR/VR Example: The MagicBook
The MagicBook
• Using AR to transition along Milgram’s continuum• Moving seamlessly from Reality to AR to VR
• Support for Collaboration• Face to Face, Shared AR/VR, Multi-scale
• Natural interaction• Handheld AR and VR viewer
Billinghurst, M., Kato, H., & Poupyrev, I. (2001). The MagicBook: a transitional AR interface. Computers & Graphics, 25(5), 745-753.
Demo: MagicBook
• https://www.youtube.com/watch?v=tNMljw0F-aw
2. Distributed Synchronous Collaboration
• Remote people working at same time• E.g. Collaborative VR, remote tabletops, shared browsers
• Advantages• Remote users in same collaborative space, spatial cues
• But: can’t convey same face to face cues
Example: Social VR
• Facebook Spaces, AltspaceVR• Bringing Avatars into VR space• Natural social interaction
Demo: Facebook Spaces (2016)
https://www.youtube.com/watch?v=PVf3m7e7OKU
3. Distributed Asynchronous Collaboration
• Collaboration at different time and different place• E.g. messages in VR, web annotation tools, doc. markup
• Advantages• Time for more considered response, work whenever• Combine information from many sources, better discussions
4. Co-Located Asynchronous Collaboration
• Collaborating at the same location but different times• E.g. Public displays, shared physical message walls, AR annotations• Not well studied for information visualisation
• Advantages• Collaborators viewing same physical space• Can use external objects to support collaboration (pens, notes)
Example: Hydrological Data Visualization
Hydrosys uses AR to display locations of stations in a global
sensor network as well as interpolated
temperature plotted as geodesic contours
Support for asynchronous
annotationImage: Eduardo Veas and Ernst Kruijff
Veas, E., Kruijff, E., & Mendez, E. (2009). HYDROSYS-first approaches towards on-site monitoring and management with handhelds. J. Hrebıcek, J. Hradec, E. Pelikán, O. Mırovský, W. Pillmann, I. Holoubek, TB, editor, Towards e-environment, EENVI2009, Prague, Czech Republic.
Mixed Presence Collaboration
• Combining collaborative spaces• Connect both co-located and distributed collaborators
Examples
• Many examples• Mixed Presence tabletop with multiple people at each end• CAVE VR connecting between multiple people
• Advantages• Support for distributed collaboration, benefits of face to face groups
• Challenges• Support for mutual awareness, representation of remote users
Lessons Learned
• In co-located systems the following is important:• supporting different independent viewpoints• enabling the use of different tools for different data• supporting face-to-face group work• support for different data representations
Marai, G. E., Forbes, A. G., & Johnson, A. (2016, March). Interdisciplinary immersive analytics at the electronic visualization laboratory: Lessons learned and upcoming challenges. In Immersive Analytics (IA), 2016 Workshop on (pp. 54-59). IEEE.
General Guidelines
• In general, collaborative systems should support:• Shared context – knowledge/context around data• Awareness of others – aware of others actions• Negotiation and communication – easy conversation• Flexible and multiple viewpoints - depending on roles
Churchill, E. F., Snowdon, D. N., & Munro, A. J. (Eds.). (2012). Collaborative virtual environments: digital places and spaces for interaction. Springer Science & Business Media.
Importance of Roles
• Asymmetic/Symmetric problem solving• Teacher/student vs. equal collaborators
• Three different levels of engagement [Isenberg 2011]:• Viewing: where people are consuming a data presentation
without interacting with the data, such as in a lecture.• Interacting/exploring: where people have the means to
choose alternate views or explore the data.• Sharing/creating: people are able to create and distribute
new datasets and visualizations to be explored.
• Need to design the interface differently for each role
Methods for Interacting in CIAs
• Goal: Natural interaction that supports collaboration• Techniques used
• Pointing and gestures – hand or full body• Dedicated devices – e.g. handheld tablet• Multimodal – touch + speech• Tangible interfaces – physical objects• Collaborative actions – working together
Opportunities for Research• Many opportunities for research
• Using VR for CIA• HMD vs. CAVE performance
• Next generation collaboration• Using AR/VR for FtF/remote collaboration
• Evaluation of CIA systems• Subjective/objective measures, cognitive evaluation
• Methods for Asynchronous collaboration• Especially remote asynchronous systems
• Novel interaction methods• Multimodal input, gaze based system, etc
• Exploring the CIA design space• Interaction metaphors, design patterns
Example: Holoportation (2016)
• Augmented Reality + 3D capture + high bandwidth • http://research.microsoft.com/en-us/projects/holoportation/
Holoportation Video
https://www.youtube.com/watch?v=7d59O6cfaM0
Example: Empathy Glasses (CHI 2016)
• Combine together eye-tracking, display, face expression• Impicit cues – eye gaze, face expression
++
Pupil Labs Epson BT-200 AffectiveWear
Masai, K., Sugimoto, M., Kunze, K., & Billinghurst, M. (2016, May). Empathy Glasses. In Proceedings of the 34th Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM.
AffectiveWear – Emotion Glasses
• Photo sensors to recognize expression• User calibration• Machine learning• Recognizing 8 face expressions
Empathy Glasses in Use
• Eye gaze pointer and remote pointing• Face expression display• In future integrated eye-tracking/display
Empathy Glasses Demo
https://www.youtube.com/watch?v=CdgWVDbMwp4
CONCLUSION
Conclusion• Need for research on Collaborative Visualisation
• Less than 5% of Visualisation papers
• New area: Collaborative Immersive Analytics• Visual Analytics + Mixed Reality + CSCW• Using Immersive Technologies• Early promising results – e.g. CAVE2 case studies
• Different classes of CIA systems• Classify according to space/time taxonomy
• Many directions for future research• Interaction, evaluation, asynchronous collaboration, etc.
