Full Terms & Conditions of access and use can be found athttp://www.tandfonline.com/action/journalInformation?journalCode=tjde20

Download by: [Pennsylvania State University], [Alexander Klippel] Date: 02 October 2017, At: 10:16

International Journal of Digital Earth

ISSN: 1753-8947 (Print) 1753-8955 (Online) Journal homepage: http://www.tandfonline.com/loi/tjde20

Collaborating remotely: an evaluation ofimmersive capabilities on spatial experiences andteam membership

Danielle Oprean, Mark Simpson & Alexander Klippel

To cite this article: Danielle Oprean, Mark Simpson & Alexander Klippel (2017): Collaboratingremotely: an evaluation of immersive capabilities on spatial experiences and team membership,International Journal of Digital Earth, DOI: 10.1080/17538947.2017.1381191

To link to this article: http://dx.doi.org/10.1080/17538947.2017.1381191

Published online: 27 Sep 2017.

Submit your article to this journal

View related articles

View Crossmark data

Collaborating remotely: an evaluation of immersive capabilities onspatial experiences and team membershipDanielle Opreana, Mark Simpsonb and Alexander Klippelb

aDepartment of Architecture, The Pennsylvania State University, University Park, PA, USA; bDepartment ofGeography, The Pennsylvania State University, University Park, PA, USA

ABSTRACTToday’s workforce environments are steadily becoming more distributedacross the globe, calling for improved ways of facilitating collaborationsat a distance, including geo-collaborations or collaborations at criticallocations. Newer technology is allowing distributed teams to move awayfrom traditional conference rooms, taking collaborations into the fieldand giving remote teams more information about the environment. Thisidea of situating a remote collaborator’s experiences in the field,virtually, promises to enhance the understanding of geographicallyremote spaces. Newer technologies in virtual reality (VR) hold promisefor providing mobile spatial experiences in real-time, without being tiedto fixed hardware, such as systems in conference rooms. An exploratorystudy using VR technology on remote user experiences in acollaboration was conducted to identify the added value for remotecollaborators. The findings suggest immersive capabilities improvefeelings of presence in the remote locations and perceptions of being inthe remote location increase feelings of team membership.

ARTICLE HISTORYReceived 12 May 2017Accepted 14 September 2017

KEYWORDSPresence; geo-collaboration;virtual reality

1. Introduction

Technology improves our ability to communicate and share information across distance, giving usaccess to remote sites, and creating a basis for distributed geo-collaborations (Wu et al. 2009). Col-laborations about location-specific activities, such as disaster management, may take remote collab-orators to locations that can be difficult or dangerous to reach. For instance, personnel on-scene for adisaster may need to plan responses with input from personnel who cannot travel there. In manysituations, decision-makers may not have time or resources to visit remote locations and oftentimesrely on geographic representations, such as maps created using geographic information systems.Geographic data help to build situational awareness of a location for informed decision-making(Resch, Schmidt, and Blaschke 2007; Chen et al. 2013). According to Sonnenwald, Maglaughlin,and Whitton (2004), situational awareness can significantly improve communication.

Situational awareness (also situation awareness) is defined as knowledge accessible at a given timeto help create a coherent picture for assessing a situation (Sarter and Woods 1991). An earlier defi-nition by Endsley (1988) identifies situational awareness in three parts: (1) perception of elements inthe environment within a certain time and space, (2) comprehending the meaning of those elements,and (3) projecting their near future status. In essence, situational awareness deals with how an indi-vidual comprehends what is happening in their immediate environment. To maximize situational

© 2017 Informa UK Limited, trading as Taylor & Francis Group

CONTACT Danielle Oprean dxo12@psu.edu Department of Architecture, The Pennsylvania State University, 149Stuckeman Family Building, University Park, PA 16801, USA

INTERNATIONAL JOURNAL OF DIGITAL EARTH, 2017https://doi.org/10.1080/17538947.2017.1381191

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

awareness, technology-mediated experiences need to provide immediate and accurate environmentinformation.

Site visits to remote locations are a natural way to increase situational awareness and help solveproblems where location-specific factors are important, such as infrastructure planning or naturalhazard management. Geospatial data often go hand-in-hand with field investigations, for example,site selection for landfills (Şener, Süzen, and Doyuran 2006), or landslide risk analysis (Lin and Tung2004). Within environmental management, there is need for local stakeholder participation indecision-making, and the role of the field visits which use the landscape as a reference in lieu of‘post-it notes and flip-charts’ (Reed 2008). However, visiting sites in person is not always practicalor cost-effective, such as for remote infrastructure like mountain top communications towers.This leaves a question of how to provide in-situ, site-specific collaboration experiences to remoteusers.

Since the 1990s, virtual reality (VR) has shown great promise providing a multi-modal platformfor collaboration (Ramesh and Andrews 1999; Chen et al. 2012). Currently, remote collaborators caneasily join meetings virtually through various equipment; however, limitations exist in what a remotecollaborator may experience and interact with through common audio and video communication(Gauglitz et al. 2014). Such limitations stem beyond technological and connectivity challenges,and relate more to experiences and organization of remote parties (Riemer, Steinfield, and Vogel2009). These experiences allow first-person engagement with remote locations potentially creatingawareness of environmental situations. So, the more engaged by the collaborative medium, themore a remote collaborator should feel part of a team, potentially making more contributions.

VR technology has long been a communication tool, but recent renewal in interest towards suchtechnology has created a boom in development and application (Bellini et al. 2016). The Oculus Rifthead-mounted display (HMD) released first in 2012 is widely credited with renewing interest in VR(Kumparak 2014). This market boom is largely due to breakthroughs in the quality and price ofnecessary hardware. Necessary technologies include miniaturized sensors and displays created forsmartphones (Wingfield 2013). These sensors in turn make possible, higher quality and moreresponsive experiences that significantly reduce the chance of simulator sickness, a known issuewith HMDs (Kumparak 2014).

Existing research on VR-based remote collaborations has generated only limited understanding ofthe role of aspects of technology, and is hampered by conceptual ambiguity. Studies often examinetechnology by comparing a single system against a real world experience (e.g. a co-located team) in a‘black-box’ approach, rather than comparing different types of technology to each other. Anotherkey issue in researching VR systems for collaboration is the ambiguity of concepts and terminology.Related but fundamentally different concepts such as presence and immersion are often confused(Slater 1999). By carefully considering conceptual clarity, studies can be made more replicableand comparable while advancing an understanding of the role of such technology.

The impact of technology on perceptions of team membership and feeling spatially present in aremote collaboration has not been fully explored. Collaborative systems provide technologically lim-ited access to remote collaborators, and those limitations can affect spatial experiences, feelings ofteam membership, and interactions. ‘Collaboration is situated in a medium for perception andaction; what we can see and do in this medium has profound effects on how we can communicateand interact’ (Gaver 1992, 21). This posits two questions: how different immersive technologiesenable remote collaborators to feel present in a meeting space and what the implications are forteam membership.

With a focus on geographically distributed collaboration, the potential of newer VR technology isexplored. We first present background on geo-collaborations and spatial experiences through VRtechnology and collaborative environments. We then describe our methods for conducting anexploratory study of remote collaborator perceptions using VR technology to complete a simple col-laborative task. We conclude with a discussion of our findings, implications for the geosciencedisciplines.

2 D. OPREAN ET AL.

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

2. Geo-collaboration and virtual geosciences

MacEachren and Brewer (2004) defined the term geo-collaboration to describe ‘visually-enabled col-laboration with geospatial information through geospatial technologies,’ and created a conceptualframework for research into how to best enable it. A wide range of domains including scientificresearch, science education, emergency management, and planning were identified as importantapplication areas. MacEachren and Brewer asserted that creating usable geo-collaboration systemsrequires depicting a participant’s location, viewpoint, and actions within the environment, both tothemselves and to other users. Despite this early emphasis, research in geo-collaboration hasoften focused on how to integrate simultaneous interaction with geospatial data and representations,such as Heard et al. (2014) and Convertino et al. (2005), rather than directly using an environment.However, Heiser, Tversky, and Silverman (2004) found that being in the same space as a collaboratorplays a crucial role in map-based collaborative problem-solving.

3. Spatial experience in virtual collaborative environments

Spatial experiences in collaborative environments can mean anything from 3D interactive environ-ments with avatars to live-streaming video. 3D interactive environments, virtual environments (VE),offer synthetic spatial experiences understood through the constructs of immersion and presence.With the renewed interest in development of virtual technology, the fidelity and quality of VEsare vastly improving. These improvements have shown direct impacts on spatial experiences (Balak-rishnan et al. 2012). In turn, spatial experiences have positive influences on knowledge developmentand transfer to real world applications (Choi and Hannafin 1995). Schroeder (2006) found that cam-era-based collaborative environments induce feelings of presence similar to VE. This suggests thatcamera-based collaborative environments can be examined from a similar perspective as VE.

3.1. Immersion

Immersion, an ambiguous concept in VR, is a result of multiple interpretations of the original defi-nition of submersion into a medium (Murray 1997). Immersion represents ‘submerging’ a user’ssenses (visual, audio, etc.) into a digital (synthetic) environment. This idea of submersion hasbeen translated to represent engagement with a user’s senses, which is facilitated through variousmechanisms including factors of cognitive absorption (Agarwal and Karahanna 2000). Slater(1999) distinguished immersion from similar concepts, defining it as an objective technology-focused phenomenon. The most common form of immersion is visual immersion, as most VR tech-nology focuses on what a user is able to see through a display device such as a monitor or HMD.Visual immersion refers to the engagement of a user’s visual senses through the capabilities of a dis-play device.

VR devices greatly differ, but can be compared in terms of specific features, or affordances.Greeno (1994) adapts Gibson’s idea of affordances, denoting them as properties of an environmentthat allow for action or perception of action by the appropriate entity. For instance, a coffee mughandle (property of an environment) allows a hand (an appropriate entity) to grasp. In our under-standing of affordances, we denote features of VR display devices. Adapting Bowman and McMa-han’s (2007) description of display device characteristics, visual immersion can be operationalizedas different features across different display devices. This leads us to identify a specific feature sharedacross most devices.

VR technology supports wider fields of view (FOV) than standard displays. FOV refers to thephysical field in which a user can see a simulated space. It relates to how much a user’s directand peripheral vision is filled by a specific display device. Humans generally have a specified FOVof 200° horizontally and 135° vertically (Arthur 2000). FOV is known to have an impact on senseof presence and memory (Balakrishnan and Sundar 2011; Balakrishnan et al. 2012). From this

INTERNATIONAL JOURNAL OF DIGITAL EARTH 3

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

information, we considered visual immersion by the affordance of FOV and vary it based on the dis-play technology involved.

3.2. Dimensions of presence

Presence is a commonly measured construct in spatial experience research. The construct of pres-ence consists of a number of dimensions specific to different contexts such as spatial, co-, or tele-presence. Each dimension has the same foundational meaning: that a user feels present within agiven medium, perceiving virtual content (Steuer 1992). The different dimensions help further dis-tinguish the context in which presence occurs. The dimensions of presence cover a range of contex-tually different concepts that are often used interchangeably (Zhao 2003).

In VR collaborations, two dimensions of presence are commonly considered: spatial presenceand co-presence. Spatial presence relates to the believability of being within a mediated space.Users identify a virtual space as the space they can act and perform in (Wirth et al. 2007).Spatial presence in high-fidelity environments forms through focused attention, attentionalallocation, allowing users to become absorbed in a VE (Balakrishnan and Sundar 2011;Oprean 2014). Self-report directly following an experience is a common form of measurement.Measures for spatial presence can include attentional allocation, spatial situation model, self-location, and possibilities for action (Vorderer et al. 2004); where self-location is the centralmeasure.

The other dimension of presence for VR collaborations is co-presence (Schroeder 2006). Co-pres-ence, also known as social presence, focuses on connection with others in virtual spaces (Nowak andBiocca 2003). This human–human relation is central to how co-presence forms (Zhao 2003). Co-presence is measured through self-report, with measures focusing on the believability of beingwith others in a computer-mediated space (Slater et al. 2000).

3.3. Presence and immersion

Immersion is based on the technology, making it more objective than presence. We follow Slater’s(1999) distinction: immersion as a product of technology providing sensory engagement and presenceas the subjective feeling of successfully submerging, or perception of, the senses (Steuer 1992). Ourfocus tests different levels of immersive technology to measure presence in various forms (spatial andco-). Therefore, providing higher levels of immersion should increase the sense of presenceexperienced.

The relationship between immersion and presence forms at the point when user attention isfocused (or absorbed) to allow for involvement in the content through sensory engagement (Oprean2014; Ryan 2015). This understanding gives us a framework for measuring the influence of immer-sion on presence and consideration for potential covariates related to attention and absorption. Fromthis framework, we can take our operationalization of immersion, FOV, and vary it to measureresulting sense of presence responses in terms of spatial presence and co-presence. We thereforehypothesize that:

H1: A wider field of view will improve spatial and co-presence.

Spatial presence is influenced by changes in FOV, where a wider FOV increased measures ofspatial presence in a VE (Balakrishnan et al. 2012). As presence is theorized to form based onattention, absorption, and involvement, there is a need to control for other influencing variables,such as enjoyment, attention, and absorption. We believe the wider FOV will produce similarresults in relation to self-location for live-streamed video. Co-presence is influenced by the inter-face of a collaborative system and how the interface identifies other individuals (Zhao 2003). Webelieve a wider field of view (FOV) will change the experience, increasing their sense of co-presence.

4 D. OPREAN ET AL.

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

3.4. Interactivity for knowledge building

The degree of control a remote collaborator has navigating a meeting space is a direct result of theinteraction afforded by a collaborative system. Balakrishnan and Sundar (2011) suggest the degree ofnavigable control can also improve spatial experiences. However, in collaborations using live-streaming video, cameras are often fixed, leaving little navigable control. Collaborative systemsthat afford remote user camera control are often confined to high-end meeting spaces. Newer con-sumer-grade 360° cameras have the potential to provide navigation control to remote collaborators.Therefore, we consider the overall influence of a live-streaming 360° camera as a testable format forfacilitating computer-mediated face-to-face collaborative meetings.

3.5. ‘Team satisfaction’ as team membership

Collaborative teams facilitated through technology are considered by degrees of virtuality, which refersto the degree a team is dependent on technology mediation (Schweitzer and Duxbury 2010). The defi-nition of virtuality distinguishes ‘virtual’ teams as different from proximate teams. Chudoba et al.(2005) formalize three criteria for identifying virtuality: team distribution, workplace mobility, and var-iety of practices. However, these criteria are influenced by factors of social interaction, knowledge shar-ing, and predictability. The culmination of factors that make up virtuality can be used to identifymembership in a team. Virtuality relates to team members feeling a part of a team, and can bemeasured through self-report on perceptions of team satisfaction and performance (Schweitzer andDuxbury 2010). Satisfaction with technology-mediated collaborations is a common measure for estab-lishing perceptions of team membership and activity (Schouten, van den Hooff, and Feldberg 2010).

In a virtual collaboration, it is also important to consider the relationship between satisfaction andthe sense of presence. So and Brush (2008) suggest no relationship exists between social presence andsatisfaction. This suggests the measure of co-presence will not relate to satisfaction. However, the roleof spatial presence has not been explored. Once spatial presence has been considered, satisfactionbased on the context of the collaboration can be explored. We therefore hypothesize that:

H2: A relationship exists between spatial presence and team satisfaction.

H3: The collaboration location will improve team satisfaction.

4. Method

We conducted an exploratory 3 (immersiveness) × 2 (location) between-subjects experiment wherewe manipulated two factors concerning remote collaborations with VR. We manipulated three levelsof immersiveness as FOV across two different remote locations. Our three levels of FOV were oper-ationalized as a single monitor for the narrow condition, three connected monitors for the mediumcondition, and a HMD for the wide condition. Our two locations were a conference room with stan-dard projector and screen and an outdoor plaza. This design randomly assigned participants into oneof six possible conditions, see Table 1.

The study enabled participants to take advantage of the wider FOV provided by the 360° cameraby controlling for interactivity. As each display type worked with a different type of input device for

Table 1. Conditions of experimental design.

Location

Immersiveness as field of view (FOV)

Narrow FOV(One-screen)

A

Medium FOV(Three-screen)

B

Wide FOV(Head-mounted display)

C

Conference room (1) A1 B1 C1Plaza (2) A2 B2 C2

INTERNATIONAL JOURNAL OF DIGITAL EARTH 5

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

interaction, it was important to make the interaction generalizable. Interactivity was operationalizedas navigability, as described in Balakrishnan and Sundar (2011). The level of navigability availablethrough the combination of the 360° camera and display device was rotational movement (rotatingthe viewpoint around a fixed point). This allowed for control over the level of interactivity. Move-ment was limited to rotating the camera to change the viewpoint. Our resulting design was threelevels of immersiveness as display devices with varying degrees of FOV and limited rotational inter-action through either keyboard or head-tracking interaction, see Figure 1.

4.1. Remote collaborator experiences as measures

All measures were collected as self-report measures using a 1–9 Likert scale format anchored with‘strongly agree’ to ‘strongly disagree.’ The measures were adapted from well-known studies toaddress our concepts of presence and teammembership. We selected specific dimensions of our con-cepts as dependent variables of spatial presence, co-presence, and team membership. We operatio-nalized these dependent variables into measurable dimensions where spatial presence was measuredas self-location, co-presence as co-presence, and team membership as team satisfaction.

Spatial presence measures were adapted from Measurement, Effects, Conditions Spatial PresenceQuestionnaire (MEC-SPQ) (Vorderer et al. 2004). The MEC-SPQ comprises several measures, seeVorderer et al. (2004), aligning with different aspects of spatial presence with the central measurebeing self-location. Self-location, consisting of four items, measures the degree of the ‘experienceof being located in the mediated environment’ (Wirth et al. 2007, 496). Co-presence measureswere adapted from Slater et al. (2000). Team membership was operationalized as team satisfaction,from Schweitzer and Duxbury’s (2010) team virtuality measures. We collected several additionalmeasures to explore as covariates, see Table 2.

The first covariate was attentional allocation adapted from the MEC-SPQ (Vorderer et al. 2004),and was designated as a covariate for all spatial presence measures. Attentional allocation consists of

Figure 1. Experimental design with levels of FOV as immersiveness.

Table 2. Measures adapted for use in this study.

Source Factor Measure Example of adapted measure

Schweitzer andDuxbury (2010)

Teammembership

Teamsatisfaction

All in all, I am satisfied with my experience with this virtualteam.

Slater et al. (2000) Co-presence Co-presence In the last meeting, to what extent did you have the sense ofthe other two people being together with you?

Vorderer et al. (2004) Spatial presence Attentionalallocation

I devoted my whole attention to the virtual space I just viewed.

Self-location I felt as though I was physically present in the environment ofthe virtual space I just viewed.

Agarwal and Karahanna(2000)

Cognitiveabsorption

Temporaldissociation

Time appeared to go by very quickly while viewing the virtualspace.

Davis, Bagozzi, andWarshaw (1992)

Perceivedenjoyment

Enjoyment I found viewing the virtual space fun.

6 D. OPREAN ET AL.

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

four items measuring ‘devotion of mental capacities to the media product’ (Wirth et al. 2007, 497).We also considered cognitive absorption and adapted temporal dissociation from Agarwal and Kar-ahanna (2000). Temporal dissociation, consisting of five items, measures ‘the inability to register thepassage of time while engaged in interaction’ (Agarwal and Karahanna 2000, 673). Next was enjoy-ment, made up of three items adapted from Davis, Bagozzi, and Warshaw (1992). Perceptions ofenjoyment could influence attention to the collaboration and subsequently feelings of presence.Enjoyment is a known covariate for presence measures (Oprean 2014) as it relates to involvement.Lastly, we included several demographic questions about the participant and previous experiencewith VR and collaborative technology.

4.2. Experimental setup

Participants, using one of our six conditions, would virtually meet with two researchers acting as col-laborators in a semi-structured meeting. Prior to the meeting, participants would fill out the demo-graphics survey and following the meeting, fill out a questionnaire responding to the experience. Ourexperimental setup consisted of several parts: technology, location, task, and script testing forconditions.

4.3. Collaboration technology setup

A combination of hardware and software technology was used to connect participants to the collab-orators (experimenters) visually and aurally. Each participant used the same desktop workstationwith an Intel Core i7-5930 K CPU (3.5 GHz, 6 cores), 32GB of RAM, an NVIDIA GeForce GTX980 Ti GPU, and a 240GB SSD running on Windows 10. The workstation in all conditions was con-nected to three 23′′ 1980 × 1280 monitors, an Oculus Rift Development Kit 2 (DK2) HMD, a USBcombination microphone and speaker, the Ricoh Theta S 360° camera (through an HDMI videoadapter), and a standard keyboard and mouse. The Theta S camera was used for our experimentas it could stream 360° video using two fisheye lenses. The collaborators were equipped withan Android smartphone and a wireless (Bluetooth) combination speaker and microphone. SeeFigure 2 for a schematic illustration of this setup.

The Oculus Rift development kit 2 (DK2) was selected as the HMD device as it represented themost advanced consumer-level hardware that was widely available at the time of the study. In terms

Figure 2. The overall hardware setup connecting the two locations (participant and remote team members).

INTERNATIONAL JOURNAL OF DIGITAL EARTH 7

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

of specifications, the DK2 has a full resolution of 1920 × 1080, divided in half per-eye. It uses a smallinfrared device placed in front of the user to track head movement. At the time of the study, no 360°video streaming applications were publically available. We used a custom-made platform using theUnity3D game engine to stitch and display the 360° video stream from the Theta S camera to eachdisplay device.

As our design focused on differences in FOV, we took steps to ensure we maintained the imagescale between each display device’s FOV and the virtual camera in Unity3D’s FOV (Draper et al.2001). In maintaining image scale, participants would experience less distortion of the streamed360° video. We also adjusted the resolution of the monitors to match that of the DK2, the displaywith the lowest resolution, to prevent undue influence of resolution. Lastly, the audio connectionwas through Microsoft Skype over a WiFi connection.

4.4. Collaboration location

Location was manipulated to represent a traditional conference setting where representations of anenvironment (i.e. a maps, photographs, and text descriptions) were used to convey details of the pro-blem versus meeting in the actual environment. The same environmental information, in the form ofbulleted lists, photographs, and a map, was provided to participants. Two sets of locations were usedas our manipulation, an office and a meeting room for the inside condition or traditional setting, anda different office and a plaza for the outdoor condition or non-conventional setting. See Figure 3 forthe indoor condition and Figure 4 for the outdoor condition. The change in offices was necessitatedby the maximum distance the camera could be from the desktop. Care was taken to keep the overallhardware setup the same in both conditions. The camera was mounted on a tripod, the location keptconstant between experiment runs.

4.5. Task for collaboration

To emulate an actual meeting, participants were asked to virtually meet with two collaborators todecide on placement of recycling bins in a plaza on a college campus. The simple spatial task allowedfor focus on user perceptions of the role of our technology manipulations during a collaborationwithout the complications of a more complex task. Participants were provided with the task require-ments and photos of the site for reference before joining the meeting, see Figure 5.

The task was to act as part of a committee deciding on where to place recycling bins in a plaza on auniversity campus. The task was limited to a 10-minute virtual meeting with two other collaboratorswho managed the meeting. In the meeting, the collaborators would share a simplified map of three

Figure 3. Indoor location with participant office space on the left and collaborator conference room on the right.

8 D. OPREAN ET AL.

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

potential locations where the recycling bins could be placed, see Figure 6. Participants were asked toparticipate by providing a choice with rationale while discussing the other collaborator’s choices withthe goal of coming to consensus on a solution.

4.6. Script testing for collaboration and conditions

In order to refine the experimental setup (particularly the semi-structured meeting), pre-testing wasconducted using a script for the two collaborators. This helped verify the meeting would keep to 10minutes, as well as finalizing other aspects such as camera placement and material provided toparticipants.

Two researchers were assigned a conversation script to conduct the remote meeting with the par-ticipant as a remote collaborator. The script gave each collaborator a role for conducting the meetingand options on how to respond, depending on specific participant responses. The collaboratorsmemorized, practiced, and helped refine the script prior to pilot testing. To control for the use ofthe 360° camera, the collaborators provided cues for the participant to move the camera at certain

Figure 4. Outdoor location with participant office space on the left and collaborator plaza on the right.

Figure 5. Reference photos shown to participants when introducing the task.

INTERNATIONAL JOURNAL OF DIGITAL EARTH 9

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

points during the discussion. This scripted use of the 360° camera would help to make sure all par-ticipants were able to take advantage of the capability of using a 360° camera regardless of thecondition.

5. Analysis and results

A total of N = 90 participants from a northeastern university in the United States participated in thestudy. Overall, there was an equal distribution of males (n = 45) to females (n = 45) with an averageage of 25.8 (SD = 0.50). For academic standing, graduate students accounted for 77% (n = 67) of themajority while seniors accounted for 13% (n = 12) of the sample. Juniors represented 8% (n = 7)while sophomores represented 2% (n = 2) and there was only one freshman accounting for 1% ofthe sample population. There were two missing data points for academic standing. Participants ran-ged across 42 different majors. The majority of participants were native English speakers (48.9%, n =44) closely followed by non-native speakers (42.2%, n = 38). Multilingual participants where Englishwas included accounted for 6.7% (n = 6) with two participants who did not indicate a nativelanguage.

The data were checked for the assumptions for all multivariate and univariate tests used. Twelveoutliers were removed from the dataset due to missing data points or being extreme values checkedvisually and through skewness and kurtosis methods (Tabachnick and Fidell 2013). After removingthe outliers, the data were found to be within acceptable bounds to continue the analysis.

The main analysis consisted of three parts. The first part used a MANCOVA measuringimmersiveness and location, controlling for temporal dissociation and enjoyment, on co-presenceand self-location. Follow-up analysis was conducted on co-presence and self-location to under-stand the findings of the MANCOVA. The second part used a Multiple Regression to identifythe amount of variance in Team Satisfaction scores predicted by co-presence, self-location, andattentional allocation. Lastly, an ANCOVA for immersiveness and location, covarying self-location and attentional allocation was used to see any impact of our manipulations on TeamSatisfaction.

Figure 6. The map of potential locations (A, B, C) in an outdoor plaza for recycling bin placement shown to participants.

10 D. OPREAN ET AL.

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

5.1. Immersiveness on self-location and co-presence

Our first hypothesis stated that immersiveness would improve our two measures of presence, self-location, and co-presence. Using a MANCOVA, we tested this hypothesis, covarying Enjoymentand Temporal Dissociation (absorption). The results from the MANCOVA found the level ofimmersiveness to be significant for Co-Presence and Self-Location, Wilks’ λ = .86, F(4, 138) =2.65, p < .05, partial η2 = .07. This suggests that scores for these presence measures combinedsignificantly differed between immersiveness conditions, where the higher the level ofimmersiveness had higher scores of presence, with the exception of the three-screen condition,see Table 3.

As the main effect of immersiveness was found significant, we then examined individualcomparisons for self-location (F(2, 70) = 1.94, p = .15) and co-presence (F(2, 70) = .95, p = .39).A bivariate correlation between co-presence and self-location revealed a strong relationship,r (78) = .70, p < .001. Individual tests revealed significant influences from the covariates:Temporal Dissociation was significant for covarying self-location, F(1, 70) = 4.71, p < .05, partialη2 = .06. Enjoyment was found to significantly influence both co-presence, F(1, 70) = 13.67,p < .05, partial η2 = .16, and self-location, F(1, 70) = 44.73, p < .05, partial η2 = .39. This suggeststhat our covariates were influential in revealing the influence of immersiveness on our twodependent variables, though our measure of temporal dissociation did not reveal an influenceon co-presence.

We conducted a follow-up univariate analysis, ANCOVA, of immersiveness and location on self-location, controlling for co-presence and enjoyment which showed a significant main effect forimmersiveness, F(2, 70) = 4.23, p < .05, partial η2 = .11. The significance in immersiveness showedthe Oculus Rift (M = 6.99, SE = .24) scored better than three screens (M = 6.14, SE = .21). Both co-presence (β = .90, t (70) = 6.74, p < .001) and enjoyment (β = .56, t (70) = 5.25, p < .001) significantlycovaried, meaning the role of co-presence and enjoyment allowed for the influence of immersivenessto become clear for self-location.

Similarly, we conducted a follow-up ANCOVA of immersiveness and location on co-presence,controlling for self-location and enjoyment. A significant main effect was found for immersivness,F(2, 70) = 3.63, p < .05, partial η2 = .09. The significance in immersiveness showed the three-screencondition (M = 7.11, SE = .15) scoring better than the single-screen condition (M = 6.58, SE = .16).Self-location (β = .44, t (70) = 6.74, p < .001) significantly covaried, while enjoyment did not(β = –.03, t (70) = –.34, p = 0.74). Self-location successfully helped reveal the influence of immersiv-ness on co-presence; however, enjoyment did not play a significant role.

The two follow-up univariate analysis revealed that both co-presence and self-location wereinfluenced by the level of immersiveness but only after controlling for the other measure ofpresence. This distinction revealed that a difference did exist between co-presence and self-location though they are still highly related as both measures of presence. A key differencelies in the extra covariate of enjoyment that significantly covaried for self-location but not forco-presence. When we covaried self-location in the analysis on co-presence, enjoyment wasno longer significant although in the original MANCOVA it did. This supports the notionthat co-presence, while correlated to self-location, is actually measuring a different aspect ofpresence.

Table 3. Adjusted means for co-presence and spatial presence.

Immersiveness Adj. M SE

Co-presence Single-screen 6.59 .20Three-screen 6.96 .19Oculus rift 6.85 .21

Self-location Single-screen 6.66 .28Three-screen 6.27 .26Oculus rift 7.05 .30

INTERNATIONAL JOURNAL OF DIGITAL EARTH 11

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

5.2. Self-location and team satisfaction

For our next hypothesis, we aimed to address how feeling present impacts team satisfaction throughour measures of co-presence and self-location. As we found in the MANCOVA and follow-upANCOVAs, co-presence and self-location are highly correlated but represent different dimensionsof presence. Based on these findings, we aimed to understand which measure of presence to considerinfluencing team satisfaction. We conducted a Multiple Stepwise Regression to determine how muchvariance our measures of presence could predict scores of Team Satisfaction.

Multiple Stepwise Regression analysis revealed two models significantly predicted Team Satisfac-tion scores. Both models excluded Co-Presence, similar to findings by So and Brush (2008). Thesecond model proved to be the best model and was selected for the analysis.

The second model was significant, including Attentional Allocation and Self-Location, F(2, 75) =25.49, p < .001. R2 for the model was .41, and adjusted R2 was .39. Table 4 displays the unstandar-dized regression coefficients (B), standard error, and standardized regression coefficients (β) forAttentional Allocation and Self-Location.

Looking closer at model 2 for the variance in Team Satisfaction, we examined the influence ofeach predictor. In terms of individual relationships between the independent variables and TeamSatisfaction, Attentional Allocation (t = 3.79, p < .001), and Self-Location (t = 3.67, p < .001), each sig-nificantly explained Team Satisfaction scores. Co-Presence was excluded from the model due to non-significance (t = .13, p = .89). Together, the two variables contributed 38.9% in shared variability.

5.3. Location on team satisfaction

From our two outcomes of immersion influencing both presence measures jointly and self-locationsignificantly predicting team satisfaction, we revisited the role our manipulations with self-locationand attentional allocation as covariates on Team Satisfaction. We aimed to address our hypothesisthat our manipulation of context would influence Team Satisfaction. Using an ANCOVA of immer-siveness and location, we covaried Self-Location and Attentional Allocation, as they were identified asprediction variables, on Team Satisfaction. We found a significant main effect for location in terms ofoverall Team Satisfaction (F(1,70) = 4.06, p < .05, partial η2 = .06). Immersiveness did not signifi-cantly impact team satisfaction, F(2, 70) = .45, p = .64. Participants with the plaza condition reportedsignificantly higher Team Satisfaction scores (M = 8.60, SE = .08) than those who viewed the confer-ence room (M = 8.36, SE = .09), see Figure 7. As predicted from the regression analysis, Self-Location(β = .13, t (70) = 3.57, p < .001) and Attentional Allocation (β = .29, t (70) = 4.11, p < .001) signifi-cantly covaried. Attentional Allocation alone explained 19.4% of the variance in satisfaction whereasSelf-Location only accounted for 15.4% of the variance.

5.4. Summary of results

For our analysis, we addressed our first hypothesis that immersiveness would influence measuresof presence, after controlling for individual differences. A MANCOVA showed significance forimmersiveness when controlling for enjoyment and temporal dissociation on the grouped vari-ables of co-presence and self-location. Looking at co-presence and self-location individually did

Table 4. Summary of multiple regression coefficients for model 2.

Variable B SE β

Intercept 5.51 .49Self-location 0.13 .04 .37**Attentional allocation 0.26 .07 .38**

Note: B = unstandardized regression coefficient; SE = standard error of the coefficient; β = stan-dardized coefficient.

**p < .001.

12 D. OPREAN ET AL.

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

not show any influence from immersion. A correlation showed the two variables were highlyrelated, calling for further follow-up. Individual ANCOVAs revealed that both were impactedby immersiveness where co-presence showed a difference between the single- and three-screenconditions and spatial presence had a difference between the single screen and HMD conditions.Next, we addressed our second hypothesis that feeling present would influence team satisfaction.Using multiple regression, we identified that self-location and attentional allocation explained alarge portion of the variance in team satisfaction scores. Co-presence did not influence team sat-isfaction. Our last analysis addressed the hypothesis that our manipulations will influence teamsatisfaction after controlling for spatial presence, self-location. Using an ANCOVA, controllingfor self-location and attentional allocation, the outdoor plaza reported higher scores for teamsatisfaction.

6. Discussion

Newer VR technology is promising for geographically distributed collaborations as it is increasinglyeasier to use and more affordable. Newer technology is also more mobile, deployable to remote sitesthat may be too dangerous or costly to visit in person, as found in disaster management. We con-sidered the use of VR technology for remote geo-collaborations and proposed that VR technologycan place individuals virtually into the location of interest with collaborators, providing increasingtheir situational awareness and fostering a sense of team membership. From this hypothesis, wederived an exploratory study that builds on existing theoretical frameworks for measuring the influ-ence of VR technology.

We posed the question of how different immersive technologies enable remote collaborators tofeel spatially present in a meeting space, and what the implications are for team membership. Inour exploratory study, we found the controllable first-person view provided by the 360° camera com-bined with a more immersive display was able to increase perceptions of both spatial experience(presence) and team membership (satisfaction).

Figure 7. Influence of Location after controlling for Spatial Presence on Team Satisfaction.

INTERNATIONAL JOURNAL OF DIGITAL EARTH 13

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

Looking at the role of presence, we found that spatial presence specifically was the most influen-tial. With a location-specific collaborative task, spatial presence logically makes sense. The role of thecovariates in particular supports the theoretical idea that cognitive absorption plays a role in the for-mation of spatial presence (Agarwal and Karahanna 2000). In dealing with online collaborations,however, the minimal role of co-presence to the overall results indicates our collaboration wherefocus was on a location, that inter-personal perceptions were less important. This could be due toa number of factors pertaining to the measures used, collaboration focus, or the short exposure time.

6.1. Outlook and future study

As this study was exploratory in nature, we identified several opportunities for future research onother affordances such as fidelity. Other differences may occur if we expanded both the collaborationsettings and measures for capturing the experiences in a more objective manner. A more intensivespatial problem, such as planning for large infrastructure or a complicated multi-stop route, mayhave different effects. In terms of measures, measuring presence for instance is debated it generallyrelies on subjective post-questionnaires (Riley, Kaber, and Draper 2004; Schuemie et al. 2001). Wedid not consider situational awareness as a measure, which could have had implications (Riley,Kaber, and Draper 2004). Additionally, we did not focus on the team dynamics (also importantfor collaborations; Schweitzer and Duxbury 2010), as our setup used two group members with ascripted discussion. Next steps will be to take these questions into studies focused on improvingmeasurement and affordances, focused on use of geographic representations.

Geo-collaborations, in combination with advances in virtual geographic environments (H. Lin et al.2015), are becoming more commonplace as newer technology offers better remote experiences. Onerelevant area for future study is the integration of live camera feeds with interactive geographic rep-resentations, such as a shared 2Dmap, or even AR-style data overlay onto the environment to increasesituational awareness. For example, future interfaces with AR type overlays over 360° video feeds tovisualize a new building or other planned project, or having a virtual map, while ‘in’ the photosphere.In considering the role of immersive technology in helping form team membership as an importantaspect in collaboratively engaging, for example, in virtual geographic environments (Lin, Chen, andLu 2012), we can gain insight into not only potential productivity but also into the actual adoptionand use of such technologies in the field. When we can allow spatial scientists to take remote teammembers to different locations around the world visually and aurally in high fidelity, the implicationsfor more advanced understanding of geographic phenomena are much improved.

Acknowledgements

The authors would like to thank Yooinn Hong and Xi Liu for participating in this study as moderators.

Disclosure statement

No potential conflict of interest was reported by the authors.

Funding

The authors would like to acknowledge the generous support of the Logistics Management Institute (LMI) ResearchInstitute through a grant [grant number EM160045] to study VR for Distributed Workforces.

References

Agarwal, Ritu, and Elena Karahanna. 2000. “Time Flies When You’re Having Fun: Cognitive Absorption and BeliefsAbout Information Technology Usage.” Management Information Systems Quarterly 24 (4). http://aisel.aisnet.org/misq/vol24/iss4/5

14 D. OPREAN ET AL.

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

Arthur, Kevin W. 2000. “Effects of Field of View on Performance with Head-Mounted Displays.” DoctoralDissertation, The University of North Carolina at Chapel Hill.

Balakrishnan, Bimal, Danielle Oprean, Brad Martin, and Melina Smith. 2012. “Virtual Reality: Factors DeterminingSpatial Presence, Comprehension, and Memory.” In Proceedings of the 12th International Conference onConstruction Applications of Virtual Reality, November 1–2, Taipei, Taiwan CONVR 2012, edited by Yu-ChengLin and Shih-Chung J. Kang, 451–459. Taipei: National Taiwan University Press.

Balakrishnan, Bimal, and S. S. Sundar. 2011. “Where Am I? How Can I Get There? Impact of Navigability andNarrative Transportation on Spatial Presence.” Human–Computer Interaction 26 (3): 161–204. doi:10.1080/07370024.2011.601689.

Bellini, Heather, Wei Chen, Masaru Sugiyama, Marcus Shin, Shateel Alam, and Daiki Takayama. 2016. “Virtual &Augmented Reality: Understanding the Race for the Next Computing Platform.” Profiles in Innovation 1.Accessed 6 August, 2017. http://www.goldmansachs.com/our-thinking/pages/technology-driving-innovation-folder/virtual-and-augmented-reality/report.pdf

Bowman, Doug A., and Ryan P. McMahan. 2007. “Virtual Reality: How Much Immersion Is Enough?” Computer 40(7): 36–43. doi:10.1109/MC.2007.257.

Chen, Min, Hui Lin, Mingyuan Hu, Li He, and Chunxiao Zhang. 2013. “Real-Geographic-Scenario-Based VirtualSocial Environments: Integrating Geography with Social Research.” Environment and Planning B: UrbanAnalytics and City Science 40 (6): 1103–1121. doi:10.1068/b38160.

Chen, Min, Hui Lin, Yongning Wen, Li He, and Mingyuan Hu. 2012. “Sino-VirtualMoon: A 3D web Platform UsingChang’E-1 Data for Collaborative Research.” Planetary and Space Science 65 (1): 130–136. doi:10.1016/j.pss.2012.01.005.

Choi, Jeong-Im, and Michael Hannafin. 1995. “Situated Cognition and Learning Environments: Roles, Structures, andImplications for Design.” Educational Technology Research and Development 43 (2): 53–69. doi:10.1007/BF02300472.

Chudoba, Katherine M., Eleanor Wynn, Mei Lu, and Mary B. Watson-Manheim. 2005. “How Virtual Are We?Measuring Virtuality and Understanding Its Impact in a Global Organization.” Information Systems Journal 15(4): 279–306. doi:10.1111/j.1365-2575.2005.00200.x.

Convertino, G., C. H. Ganoe, W. A. Schafer, B. Yost, and J. M. Carroll. 2005. “A Multiple View Approach to SupportCommon Ground in Distributed and Synchronous Geo-collaboration.” In Third International Conference onCoordinated & Multiple Views in Exploratory Visualization: CMV 2005 July 5, 2005, London, EnglandProceedings, edited by Jonathan Roberts, 121–132. Los Alamitos, CA: IEEE Computer Society.

Davis, Fred D., Richard P. Bagozzi, and Paul R. Warshaw. 1992. “Extrinsic and Intrinsic Motivation to Use Computersin the Workplace1.” Journal of Applied Social Psychology 22 (14): 1111–1132. doi:10.1111/j.1559-1816.1992.tb00945.x.

Draper, Mark H., Erik S. Viirre, Thomas A. Furness, and Valerie J. Gawron. 2001. “Effects of Image Scale and SystemTime Delay on Simulator Sickness Within Head-coupled Virtual Environments.” Human Factors 43 (1): 129–146.doi:10.1518/001872001775992552.

Endsley, M. R. 1988. “Situation Awareness Global Assessment Technique (SAGAT).” In Proceedings of the IEEE 1988National Aerospace and Electronics Conference, 23–27 May 1988, Dayton, OH, USA, 789–795. IEEE.

Gauglitz, Steffen, Benjamin Nuernberger, Matthew Turk, Tobias Höllerer, Hrvoje Benko, Mira Dontcheva, and DanielWigdor. 2014. “World-stabilized Annotations and Virtual Scene Navigation for Remote Collaboration.” InUIST’14:Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, 449–459. New York:ACM Press.

Gaver, William W. 1992. “The Affordances of media Spaces for Collaboration.” In CSCW ‘92: Sharing Perspectives:Conference on Computer-supported Cooperative Work: Papers, edited by Jon A. Turner and Robert Kraut, 17–24.New York: ACM.

Greeno, James G. 1994. “Gibson’s Affordances.” Psychological Review 101 (2): 336–342. doi:10.1037/0033-295X.101.2.336.Heard, Jefferson, Sidharth Thakur, Jessica Losego, and Ken Galluppi. 2014. “Big Board: Teleconferencing Over Maps

for Shared Situational Awareness.” Computer Supported Cooperative Work (CSCW) 23 (1): 51–74. doi:10.1007/s10606-013-9191-9.

Heiser, Julie, Barbara Tversky, and Mia Silverman. 2004. “Sketches for and from Collaboration.” In Visual and SpatialReasoning in Design III, edited by J. S. Gero, B. Tversky, and T. Knight, 69–78. Sydney: Key Centre for DesignResearch.

Kumparak, Greg. 2014. A Brief History of Oculus.” TechCrunch, March 26. Accessed 6 August 2017. https://techcrunch.com/2014/03/26/a-brief-history-of-oculus/

Lin, Hui, Michael Batty, Sven E. Jørgensen, Bojie Fu, Milan Konecny, Alexey Voinov, Paul Torrens, et al. 2015. “VirtualEnvironments Begin to Embrace Process-based Geographic Analysis.” Transactions in GIS 19 (4): 493–498. doi:10.1111/tgis.12167.

Lin, Hui, Min Chen, and Guonian Lu. 2012. “Virtual Geographic Environment: A Workspace for Computer-aidedGeographic Experiments.” Annals of the Association of American Geographers 103 (3): 465–482. doi:10.1080/00045608.2012.689234.

INTERNATIONAL JOURNAL OF DIGITAL EARTH 15

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

Lin, Meei-Ling, and Chi-Che Tung. 2004. “A GIS-based Potential Analysis of the Landslides Induced by the Chi-ChiEarthquake.” Engineering Geology 71 (1–2): 63–77. doi:10.1016/S0013-7952(03)00126-1.

MacEachren, Alan M., and Isaac Brewer. 2004. “Developing a Conceptual Framework for Visually-enabledGeocollaboration.” International Journal of Geographical Information Science 18 (1): 1–34. doi:10.1080/13658810310001596094.

Murray, Janet H. 1997. Hamlet on the Holodeck: The Future of Narrative in Cyberspace/Janet H. Murray. Cambridge:MIT Press.

Nowak, Kristine L., and Frank Biocca. 2003. “The Effect of the Agency and Anthropomorphism on Users’ Sense ofTelepresence, Copresence, and Social Presence in Virtual Environments.” Presence: Teleoperators and VirtualEnvironments 12 (5): 481–494. doi:10.1162/105474603322761289.

Oprean, Danielle. 2014. “Understanding the Immersive Experience: Examining the Influence of Visual Immersivenessand Interactivity on Spatial Experiences and Understanding.” Doctoral Dissertation, University of Missouri.

Ramesh, R., and Dee H. Andrews. 1999. “Distributed Mission Training: Teams, Virtual Reality, and Real-timeNetworking.” Communications of the ACM 42 (9): 64–67. doi:10.1145/315762.315775.

Reed, Mark S. 2008. “Stakeholder Participation for Environmental Management: A Literature Review.” BiologicalConservation 141 (10): 2417–2431. doi:10.1016/j.biocon.2008.07.014.

Resch, Bernd, Dirk Schmidt, and Thomas Blaschke. 2007. “Enabling Geographic Situational Awareness in EmergencyManagement.” In Proceedings of the 2nd Geospatial Integration for Public Safety Conference, 15–17 April 2007, 1–8.Park Ridge, IL: Urban and Regional Information Systems Association (URISA).

Riemer, Kai, Charles Steinfield, and Doug Vogel. 2009. “eCollaboration: On the Nature and Emergence ofCommunication and Collaboration Technologies.” Electronic Markets 19 (4): 181–188. doi:10.1007/s12525-009-0023-1.

Riley, Jennifer, David Kaber, and John V. Draper. 2004. “Situation Awareness and Attention Allocation Measures forQuantifying Telepresence Experiences in Teleoperation.” Human Factors and Ergonomics in Manufacturing 14 (1):51–67. doi:10.1002/hfm.10050.

Ryan, Marie-Laure. 2015. Narrative as Virtual Reality 2: Revisiting Immersion and Interactivity in Literature andElectronic media. 2nd ed. Baltimore, MD: Johns Hopkins University Press.

Sarter, Nadine B., and David D. Woods. 1991. “Situation Awareness: A Critical But Ill-defined Phenomenon.” TheInternational Journal of Aviation Psychology 1 (1): 45–57. doi:10.1207/s15327108ijap0101_4.

Schouten, Alexander, Bart van den Hooff, and Frans Feldberg. 2010. “Real Decisions in Virtual Worlds: TeamCollaboration and Decision Making in 3D Virtual Worlds.” ICIS 2010 Proceedings. http://aisel.aisnet.org/icis2010_submissions/18

Schroeder, Ralph. 2006. “Being There Together and the Future of Connected Presence.” Presence: Teleoperators andVirtual Environments 15 (4): 438–454. doi:10.1162/pres.15.4.438.

Schuemie, M. J., P. van der Straaten, M. Krijn, and C. A. van der Mast. 2001. “Research on Presence in Virtual Reality: aSurvey.” Cyberpsychology & Behavior: The Impact of the Internet, Multimedia and Virtual Reality on Behavior andSociety 4 (2): 183–201. doi:10.1089/109493101300117884.

Schweitzer, Linda, and Linda Duxbury. 2010. “Conceptualizing and Measuring the Virtuality of Teams.” InformationSystems Journal 20 (3): 267–295. doi:10.1111/j.1365-2575.2009.00326.x.

Şener, Başak, M. L. Süzen, and Vedat Doyuran. 2006. “Landfill Site Selection by Using Geographic InformationSystems.” Environmental Geology 49 (3): 376–388. doi:10.1007/s00254-005-0075-2.

Slater, Mel. 1999. “Measuring Presence: A Response to the Witmer and Singer Presence Questionnaire.” Presence:Teleoperators and Virtual Environments 8 (5): 560–565. doi:10.1162/105474699566477.

Slater, Mel, Amela Sadagic, Martin Usoh, and Ralph Schroeder. 2000. “Small-group Behavior in a Virtual and RealEnvironment: A Comparative Study.” Presence: Teleoperators and Virtual Environments 9 (1): 37–51. doi:10.1162/105474600566600.

So, Hyo-Jeong, and Thomas A. Brush. 2008. “Student Perceptions of Collaborative Learning, Social Presence andSatisfaction in a Blended Learning Environment: Relationships and Critical Factors.” Computers & Education 51(1): 318–336. doi:10.1016/j.compedu.2007.05.009.

Sonnenwald, Diane H., Kelly L. Maglaughlin, and Mary C. Whitton. 2004. “Designing to Support Situation AwarenessAcross Distances: An Example From a Scientific Collaboratory.” Information Processing & Management 40 (6):989–1011. doi:10.1016/j.ipm.2003.10.002.

Steuer, Jonathan. 1992. “Defining Virtual Reality: Dimensions Determining Telepresence.” Journal of Communication42 (4): 73–93. doi:10.1111/j.1460-2466.1992.tb00812.x.

Tabachnick, Barbara G., and Linda S. Fidell. 2013. Using Multivariate Statistics. 6th ed. Boston, MA: PearsonEducation.

Vorderer, P., W. Wirth, F. R. Gouveia, F. Biocca, T. Saari, F. Jäncke, S. Böcking, et al. 2004. “MEC Spatial PresenceQuestionnaire (MEC-SPQ): Short Documentation and Instructions for Application.” Project Presence: MEC(IST-2001-37661). http://www.ijk.hmt-hannover.de/presence

Wingfield, Nick. 2013. A Matter of Perception.” New York Times, February 17. Accessed 6 August 2017. http://www.nytimes.com/2013/02/18/technology/oculus-rift-headset-aims-for-affordable-virtual-reality.html

16 D. OPREAN ET AL.

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017

Wirth, Werner, Tilo Hartmann, Saskia Böcking, Peter Vorderer, Christoph Klimmt, Holger Schramm, Timo Saari,et al. 2007. “A Process Model of the Formation of Spatial Presence Experiences.” Media Psychology 9 (3): 493–525. doi:10.1080/15213260701283079.

Wu, Anna, Xiaolong Zhang, Gregorio Convertino, and John M. Carroll. 2009. “CIVIL.” In GROUP ‘09: Proceedings ofthe 2009 International ACM SIGGROUP Conference on Supporting Group Work : May 10-13, 2009, Sanibel Island,Florida, USA, edited by Kjeld Schmidt, Mark Pendergast, Mark Ackerman, and Gloria Mark, 273. New York:Association for Computing Machinery.

Zhao, Shanyang. 2003. “Toward a Taxonomy of Copresence.” Presence: Teleoperators and Virtual Environments 12 (5):445–455. doi:10.1162/105474603322761261.

INTERNATIONAL JOURNAL OF DIGITAL EARTH 17

Dow

nloa

ded

by [

Penn

sylv

ania

Sta

te U

nive

rsity

], [

Ale

xand

er K

lippe

l] a

t 10:

16 0

2 O

ctob

er 2

017