Situation Awareness in the Bulk Power System

Mica Endsley, PhD SA Technologies

2001 Woman Owned

Business Of the Year

-NASA-

Situation Awareness is Critical to Power Transmission & Distribution

• August 14, 2003 - Northeast US/Canada – “Inadequate situation awareness”

• August 10, 1996 - Western US – “train operators to make them aware of system

conditions and changes” – “develop displays that give operators immediate information on changes in status”

• July 2, 1996 - Western US – “review need … to monitor operating conditions on a regional scale”

• July 13, 1965 - Northeast US/Canada – “System control centers should be equipped with

display and recording equipment which provide the operator with as clear a picture of system conditions as possible”

What is Situation Awareness?

Situation Awareness is the Perception of elements in the environment within a volume of time and space, the Comprehension of their meaning, and the Projection of their status in the near future.*

*Endsley, 1988

• Frequency • System voltage • Direction of flow • Breaker status

• Impact of interchange

• Violation of thermal limit

• Frequency violation

• Projected impact on system of losing element

• Projected limit violations

Projection

What do I think will happen?

Situation Awareness

Comprehension

What does this mean to

me?

Perception

Which information do I need?

Data

What? So What?

Now What?

Got SA?

Consequences of Poor SA

As much as 88% of human error is due to problems with

situation awareness

Human Performance

Situation Awareness

Decision Making Performance

Situation awareness is key to good decision making and

good performance

What Kinds of SA Problems Do People Have?

Don’t Get Information That Is Needed

Don’t Correctly Understand Information They Do Get

Don’t Project What Will Happen in Future

SA Errors

0

10

20

30

40

50

60

Freq

uenc

y (%

)

Misperception Memory Loss

Pilot Errors

Hard to Discriminate/

Detect

Failure to Monitor/Observe

Data

Controller Errors

Majority of Level 1 Errors Due to failure to Monitor or Observe Data That is Present (50%)

Highest single cause of all SA error (30%)

Not Available

0

10

20

30

40

50

60

Freq

uenc

y (%

)

Vigilance Stress Workload Other

Controller Errors Pilot Errors

Other Distraction

Reliance on Automation

Task Distraction

Biggest Single Cause is “Task Distraction”

Team SA

The Degree to Which Every Team Member Possesses the SA Required for his/her Job

A - subgoal

C- subgoal B - subgoal

TEAM GOAL

Shared SA

The Degree to Which Team Members Possess the Same SA on Shared SA Requirements

Getting on the Same Page

• What is the current status of the system? • What has been done so far? • What are they doing now? • How will that affect my tasks? • How does what I’m doing affect them? • What will they do next?

Individual SA vs. Team SA

A S A S A S

Mental Models Goals

Mental Models Goals Goals

Mental Models

Displays Environment

Displays Environment

Displays Environment

Sometimes we just talk past each other…..

• Off the coast of Newfoundland in October, 1995.

• Americans: Please divert your course 15 degrees to the North to avoid a collision.

• Canadians: Recommend you divert YOUR course 15 degrees to the South to avoid a collision.

• Americans: This is the Captain of a US Navy ship. I say again, divert YOUR course.

• Canadians: No. I say again, you divert YOUR course.

• Americans: THIS IS THE AIRCRAFT CARRIER USS LINCOLN, THE SECOND LARGEST SHIP IN THE UNITED STATES' ATLANTIC FLEET. WE ARE ACCOMPANIED BY THREE DESTROYERS, THREE CRUISERS AND NUMEROUS SUPPORT VESSELS. I DEMAND THAT YOU CHANGE YOUR COURSE 15 DEGREES NORTH, THAT'S ONE FIVE DEGREES NORTH, OR COUNTER-MEASURES WILL BE UNDERTAKEN TO ENSURE THE SAFETY OF THIS SHIP.

• Canadians: This is a lighthouse. Your call.

Failures in Team SA

Cues

Perception

Is needed information clearly passed?

Comprehension Is information interpreted in the same way?

Projection Is same projection of actions formed to guide expectations?

SA Challenges in Power T&D

• Lack of real time information on global state of system – Data is piece-meal

• Plethora of alarms, many with high false alarm rates – Leads to reduced usage of

tools

• Support for diagnostics and projection of future events/contingencies is limited - Automation that is not integrated into tasks and for which SA

over state of automation is low • Lack of shared SA across various control centers and

reliability coordinators - Even though systems are inter-related

SA Demons Present in Power Systems

Data Overload

• Tremendous Volume of Information • Swift Rate of Change of Information • Limited Bandwidth for Input • Humans can only process so much

SA Demons Present in Power Systems

Misplaced Salience • Attention drawn by pre-

attentive features • Color, lights, movement • Loud noises, size, etc.

• Inappropriate use draws attention to less important information

• Overuse fights for attention

Complexity Creep

Systems with too many features make it difficult to develop an accurate mental model of how the system works

SA Demons Present in Power Systems

Out of the Loop Syndrome Low SA on how the automation is performing Low SA on the state of the system

• Slow to detect problems with system or automation • Slow to regain understanding of what it is doing and

taking over manually

Data Overload

Technology has taken us from here to here.

But we still can’t find what we really want to know……

Information Gap

Information Needed

Find Sort

Integrate

Process

Data Produced

Why the Information Gap?

Technology Centered Design Design Technologies

Let Human Adapt

Human can only adapt so far “Human Error” Resultant System is Sub-Optimized

Fatal Flaw

• Data is gathered and presented from different systems & sources

• Each new system is just added on

• Data not integrated or transformed into real needs of user

• Decision maker left to figure it out

User-Centered Design Philosophy

Design technology to fit capability of humans

Better Decision Making Improved Safety/Reduced Injury Improved User Acceptance & Satisfaction Improved Productivity

Result

• Integrate data around real needs of decision makers

• Present information in ways that are quickly understood and assimilated

SA-Oriented Design

50 Design Principles

SA Measurement SA Requirements

Analysis SA-Oriented

Design

Goals

Decisions

•Projection Requirements •Comprehension Requirements

•Data Requirements

Goal Directed Task Analysis (GDTA) • Goals

• Subgoals • Decisions

• Projection Requirements • Comprehension Requirements • Perception Requirements

Drives Design Drives Training

Drives Evaluation

SA Measurement SA Requirements Analysis

SA-Oriented Design

System Reliability Coordinator

Determine Violations & Potential Violations

SA-Oriented Design Principles • General Principles • Confidence and Uncertainty • Dealing with Complexity • Alarms, Diagnosis and SA • Automation and SA • Supporting SA in Multi-Person

Operations • SA for Unmanned and

Remotely Operated Vehicles • SA Oriented Training

50 Design Principles

SA Measurement SA Requirements

Analysis SA-Oriented

Design

Common Pitfalls

• Requires more than putting data on the same display – Must be the “right” data – Must be transformed into

true meaning – Like beauty “information”

is in the eye of the beholder

• Cool is not necessarily functional – Useful information display

must be based on good human factors

– Must optimize decision making processes Support Situation

Awareness

SA Oriented Designs 50 Design Principles

• Principle 1 - Organize Information Around Goals – Central organizing feature – Flexible to meet changing goals of operator

• Principle 2 - Support Comprehension - Present Level 2 SA Directly – Focus on integrated information

• Principle 3 - Support Level 3 Projections – Cognitively taxing & difficult for novices

• Principle 4 - Support Global SA – Awareness of status across goals

• Principle 5 - Support Tradeoffs Between Goal Driven and Data Driven Processing

– Avoid attentional narrowing • Principle 6 - Make critical cues for Schema Activation Salient

– Determine key breakpoints and classes of situations • Principle 7 - Take Advantage of Parallel Processing • Principle 8 - Use information filtering carefully

SA-Oriented Designs for Power Systems

Reservation of color to improve data salience Minimal use of animation/motion

SA-Oriented Designs for Power Systems

Data Visualizations to Support Comprehension (Level 2 SA)

SA-Oriented Designs for Power Systems

Organizing information to support goals and provide mechanisms for quick assimilation

SA-Oriented Designs for Power Systems

Integrated Information Dashboards to Support SA “at a glance”

Test & Evaluation in the Design Process

Human Performance Decision Making Workload Situation Awareness

Modeling Rapid Prototyping

User-in the-loop

Simulation Testing

Field Testing

Final Design

SA Measurement SA Measurement SA Requirements Analysis

SA-Oriented Design

Real-time man-in-the-loop simulation of system (rapid prototyping) At random times, freeze the simulation, blanking all displays Administer a rapid battery of queries to ascertain the subject's SA at that

point in time

Score the subject's SA on the basis of objective data derived from the simulation

Situation Awareness Global Assessment Technique (SAGAT)

Objective measure of SA

TIME

Only Validated Objective Direct Measure of SA

Power Systems SAGAT Results

SAGAT Query Correct 1 What value is closest to [the utility’s] current load ramp? 83% 2 What is the current load for [the utility]? 15% 3 What percent of [region] load is [the utility’s] load? 56%

4 What is the direction of flow for [area] interchange? 71%

5 Which interface currently has the highest transfer? 33% 6 Roughly how many alarms were there in the last 10 min period? 22%

7 Of the most recent contingency solution set, what type of equipment was affected most? 37%

8 In the last hour, how has the system frequency been running on average compared to 60 Hz? 63%

9 Where is the actual [region] load in relationship to the forecasted load? 94%

10 Is [the region] currently importing more than it is exporting? 55%

11 Where is [the utility’s] load in relationship to the forecasted load? 96%

12 In the past 10 minutes, which substation has had the majority of alarms? 80% 13 Within the past 15 minutes, were there any unsolved contingencies? 94% 14 Of the past 7 alarms, were more than one related to the same unit/element? 75%

15 In the most recent contingency set, of what type were the majority? 33%

16 Which direction to you expect [the utility’s] load to trend in the next 15 minutes? 81%

Power Systems SAGAT Results

0%

25%

50%

75%

100%

SAGAT Query Number

Perc

ent C

orre

ct

>75% = Problem Areas

Conclusion

• Situation Awareness is critical for effective decision making

• Many challenges for SA exist in Power T&D operations

• Situation Awareness can be directly enhanced through improved systems design to enhance information sharing and integration

• Tools for objectively measuring SA can be used to validate OCC system designs and training programs

mica@satechnologies.com

The Development of Expertise at SA

10 Fold Difference in SA among Experienced Personnel

Abilities Spatial Attention Memory Perceptual Cognitive

Skills Information management Communication System operations Scan patterns Planning

Knowledge Mental models Schema Critical cues Goals Preconceptions and objectives

What Allows People to Achieve High Levels of SA?

Mental Model

Schema Prototypical & Expected • Objects • Scenes • Order of Events

What information is attended to How information is interpreted

and integrated What projections are made

Critical Cues

Comprehension Projection

Situation Model (SA)

External Cues Perception

Situation Awareness Virtual Instructor (SAVI)

Interactive Situation Awareness Trainer (ISAT)

Virtual Environment Situation Awareness Rating System (VESARS)

SA Trainer

Basic Skills Meta-

Cognitive Skills

SA Feedback

Mental Models & Schema Training

Training Situation Awareness Knowledge, Skills, & Behaviors

Use of SAGAT

• Heavily Validated – Content Validity

• Inclusive of SA elements – (Endsley, 1990a)

– Construct Validity • Does not impact on performance

– (Endsley, 1989, 1990) – Criterion Validity

• Predictive of performance • (Endsley, 1990b; Jones & Endsley, 2004)

– High Reliability • (Endsley & Bolstad, 1994)

• Sensitive to Differences in – Display Designs

• Hardware • Software - GUI

– Avionics Systems – Operational paradigms – Automation LOAs – Individual Differences – Aging – Training Approaches

• Developed and used in – Aviation – UAV/Robotics – ATC – Military Command & Control – Infantry Operations – Medical – Driving – Power Systems – Cyber – Homeland Defense

• Best in Simulation Studies – has been used in Field Studies in some

domains • Variants

– SACRI (d’ sensitivity scoring) – QUASA (adds confidence ratings) – SAVANT (adds time to respond) – SALSA (weights information)

Command and Control

Unclassified

Unclassified

• Fast, easy operations on the move • One-step access to any screen or

task • Situation understanding at a glance • Tailored information organized and

integrated around key role goals and decisions

• Easy monitoring across multiple task demands

• Integrated collaboration tools for shared situation awareness across the distributed force

• Warfighter controlled flexibility for changing needs and priorities

• Intelligent assistance to manage workload without being intrusive

Approved for Public Release, Distribution Unlimited, TACOM 22 NOV 2006, case 06-274.

Bringing Systems to Support SA to the CDC BioPHusion Center