dynamic decision making lab social and decision sciences department carnegie mellon university 1...
Post on 14-Dec-2015
216 Views
Preview:
TRANSCRIPT
Dynamic Decision Making Labwww.cmu.edu/ddmlab
Social and Decision Sciences DepartmentCarnegie Mellon University
1
MODELING AND MEASURING SITUATION AWARENESS IN INDIVIDUALS AND TEAMS
Cleotilde Gonzalez
In Collaboration with: Lelyn Saner, Octavio Juarez, Mica Endsley, Cheryl Bolstad, Haydee
Cuevas, and Laura Strater
2
• Computational Models of SA– Individual aspects of SA– Design aspects of SA– Organizational aspects of SA
• Measures of SA– Individual SA– Shared SA
• Conclusions
Agenda
Situation Awareness
• the Perception of the Elements in the Environment within a Volume of Time and Space,
• the Comprehension of their Meaning, and• the Projection of their Status in the Near Future. • Formation of SA influenced by:
Individual abilities Interactions with others Environment
• Integrated theory of mind: ACT-R (Anderson & Lebiere, 1998)
– Shared attention (Juarez & Gonzalez, 2003, 2004)– Learning theory (Gonzalez, Lerch & Lebiere, 2003; Gonzalez &
Lebiere, 2005)– Representation of Recognition (Gonzalez & Quesada, 2003)– Learning and decision making in dynamic systems (Gonzalez et
al., 2003; Martin, Gonzalez & Lebiere, 2004)
• Micro and Macro Cognition: Convergence and Constraints Revealed in a Qualitative Model Comparison (Lebiere, Gonzalez & Warwick, 2009)
4
Computational Cognitive Models
6
A SA meta-architecture provided a full set of cognitive models interacting with OTB, and resulting in the “commander’s SA” (Gonzalez et al., 2004; Juarez & Gonzalez, 2003)
8
• Computational Models of SA– Individual aspects of SA– Design aspects of SA– Organizational aspects of SA
• Measures of SA– Individual SA– Shared SA
• Conclusions
Agenda
Individual Measures of SA: SAGAT
• Situation Awareness Global Assessment Technique (SAGAT)
• Human-in-the-loop simulation exercises• Use of SAGAT queries (from GDTA)• Stop at random times and query the user• Compare response with reality of the situation
– Examples: What is the aircraft altitude?– What is the aircraft activity in this sector (en route, inbound to
airport, outbound to airport)– Which aircraft will need a new clearance to achieve landing
requirements?
• SAGAT score: accuracy of the responses
Individual SA measures, learning and working memory
• Can we learn to be aware? Effects of task practice and working memory influence situation awareness (SA) - Gonzalez & Wimisberg, 2007
• How do we measure individual SA– Queries may be answered while the simulation
display is not visible or covered (Endsley, 1995) or while the display is visible, uncovered (Durso et al., 1995).
10
Methods
• The design was a 2 x 18 mixed design. Participants were randomly assigned to one of two conditions (covered or uncovered display) and they were asked to run the simulation 18 times (trials).
• Individuals were asked to answer SA queries while the simulation was paused
• Participants took the Visual Span Test (VSPAN) (Shah & Miyake, 1996).
11
• The correlation between SA scores and VSPAN decreased over time
• SA scores were higher in the uncovered condition than in the covered condition– This is due mostly to perception
• The effect of practice was significant only in the covered condition, but not in the uncovered condition
18
Summary of results relevant for individual measures of SA
Measures of Shared SA (Saner, Bolstad, Gonzalez & Cuevas, in press; Saner, Bolstad, Gonzalez & Cuevas, in preparation)
Ground TRUTH
X1X2X3X4X5X6X7
Person 1
X1 X3X4 X6X7X8X9
Person 2
X1X2 X4 X6 X8
Shared SA-the degree to which team members possess the same SA on shared SA requirements (i.e. on the information that they both need to know)
(Endsley,1995, 1995b; Endsley & Jones, 2001)
A good measure of shared SA needs to account for the ACCURACY
Shared SA
Person 1Q1
Q2
Q3
Q4
Q5
Q6
Q7
Person 2Q1
Q2Q3Q4Q5Q6Q7
SimQ1
SimQ2SimQ3SimQ4SimQ5SimQ6SimQ7
Situation Awareness Global Assessment Technique (SAGAT) - Seven queries while task is stopped - Objective knowledge of situation
Score Similarity = 1-absolute value of [(p1-p2)/(p1+p2)]
Range from 0 to 1
A good measure of shared SA needs to account for the SIMILARITY
Method
• Training at Joint Personnel Recovery Agency (JPRA) - JFCOM• 16 servicemen, 3 DoD contractors; Age M=33.85• Randomly assigned to one of four Teams:
– Navy, Army, Special Operations, or Joint Service
• Utilized Cross-Training– Five scenarios over 3 days – Each scenario had 3 to 12 incidents– Scenarios randomly stopped 3 times for SAGAT,
Communication, and Workload measures– Received training prior to the exercise
Methods and Procedure
Joint Service Cell
(p1, p2, p3, p4)
Special Operations Cell
(p13, p14, p15, p16, p17)
Army Cell
(p5, p6, p7, p8)
Navy Cell
(p9, p10, p11, p12)
• Joint Personnel Recovery Agency (JPRA) training exercise
• Four team groups (i.e. cells)• Five Predictors of Shared SA
– Experience Similarity- years in real service
– Shared JPRA Knowledge- prior experience with recovery operations
– Shared Cognitive Workload- subjective ratings, five NASA-TLX scales
– Communication Distance- inverse frequency of communication
– Organizational Hub Distance- degree of dissociation from Joint Service Cell
Results
Regression Models of True Shared SA
True
Shared SA
F Adj.
R2
Constant Experience
Similarity
Shared
Knowledge
Workload
Similarity
Organizational
Hub Distance
Communication
Distance
OVERALL 5.11** .21 -.03 .09 .26* .08 .50** -.18
Scenario 1 2.56* .09 .02 -.07 -.02 .18 -.26* -.08
Scenario 2 1.55 .03 .39 .04 -.05 -.19 -.26* .00
Scenario 3 1.66 .05 .10 .19 .09 .04 -.26* -.02
Scenario 4 2.62* .11 .26 .08 .31* -.03 -.17 -.06
Scenario 5 5.79* .24 .42 .11 -.19 -.16 .45* -.24*
*p < .0 5
**p < .0 1
Conclusions – Measures of Shared SA
• Development of a Shared SA measure must account for both, accuracy and similarity of SA between members of an organization
• As shared knowledge increased, so did shared SA. • Organizational Hub Distance (OHD) is key
predictor– Physical Distance and Joint Cell Membership
• Unexpected Role of OHD– Participants processed new information directly
Possible Models
ExpectedObserved
We observed that being in branch cells was associated with higher SSA rather than being in the joint cell
27
• The success of Computational Models of SA, depends on appropriate and robust measures of individual and shared SA– Although individual measures and procedures exist, there is
a huge need for defining the methods and procedures for measuring SA at the team level
• We investigated measures of SA at both, the individual and team levels– We created a shared SA measure that builds on individual
SA
• Computational models of both, SA and SSA can incorporate these measures.
Conclusions
top related