cognitive task analysis and its application to restoring system security by robin podmore, incsys...
TRANSCRIPT
Cognitive Task Analysisand its Application
to Restoring System Security
byRobin Podmore, IncSysFrank Greitzer, PNNL
Cognitive Task Analysis Background• Historically Task Analysis provides a list of tasks and subtasks as well as
input, actions, outputs, conditions for performance and proficiency requirements.
• Because we live in an age of Technological Complexity many jobs involve complex cognitive processing, rather than simply a collection of overt or observable behaviors.
• Analysts try to develop a model or theory of how experts in a field perform their tasks.
• This model can then be used to develop system interfaces, decision aids and training programs
• Cognitive Task Analysis is the extension of traditional task analysis techniques to yield information about the knowledge, thought processes, and goal structures that underlie observable task performance.
Respond to Forced Transmission OutageTask List
• Evaluate system conditions • Interpret relay targets, recorder readings, breaker operations, and
field observations to identify the location and cause of the event • Obtain details of equipment problem including estimated time of
return• Call out system personnel when appropriate • Direct steps needed to sectionalize a transmission line in order to
locate fault • Implement a plan for restoring the system to a safe operating
condition• Coordinate response with other affected systems as applicable • Implement a plan to restore system security including preparing for
the next contingency • Complete outage log, and update applicable displays in
SCADA/EMS • Make all required notifications.
CTA Objectives• Analysts speak with experts, journeymen, novices or any combination and
analyze their behavior to determine:
– Concepts and principles, their relationship to each other and their relationship to the tasks
– Goals and goal structures, including methods for achieving goals and their triggers
– Cognitive skills, rules, strategies and plans
– Perceptual learning, pattern recognition, implicit and explicit knowledge
– Mental models – how experts represent and run models of the system
– Problem models – how experts represent a problem and work within the problem space
– How novices move through all of the above in various stages in order to become experts
– Difficulties in acquiring domain knowledge and skills
– Instructional procedures useful for moving a person from novice to expert
Cognitive Task Analysis Framework
ODM and Cognitive
Task Analysis
Mental Model and Story Analysis
State and Situation Awareness Level Analysis
Object and Action Analysis
Conduct and Debrief Scenario – Script from Operator Interview
Scenario – Objectives, Base Case, Events, Roles
Simulator Environment – Power System Simulator, Control Center Model, Operator Consoles, Projection Map Board
Operator Decision Model
Real World Short termmemory
Long termmemory
Situation
Cues
That affect the
That let youcreate
That activate
Generates
Which youAssess by
Using your
Which youValidate by
Using your
TimeAvailable?N
MentalSimulation
Mental Simulations
ActionScript
Story(Patterns)
MentalModels
MentalModels
Y
Graphical Contingency Analysis (GCA)
• Contingency analysis, as a key function in the Energy Management System (EMS),
• Assesses the ability of the power grid to sustain various combinations of power grid component failures based on state estimates.
• Presently, most of the industrial software tools installed in control centers use tabular data to represent system status, which is not easy to interpret when the system is stressed
NORMAL
RESTORATIVE EMERGENCY
INSECURE
Preventative Action
Remedial Action
Island Blackout
Outage, Load Increase
Restoration Action
Power System States
Scenario• Prebrief
– System operating at 2:00 PM, Summer peak conditions
– Crawford – Oakdale 1 & 2 out for maintenance
– Lightning Storm activity in Northeast and Southwest regions
• Simulation – 20 minutes– Restore system to a secure state based on Contingency
Analysis
– Operator plays all the roles
• Debrief – 20 minutes– Walk through recording of screen activity
– Interview on operator thought process
Participants
• Experimental Group– Three participants– Using Graphical Contingency Analysis
• Control Group– Three participants– Using Tabular Contingency Analysis
• Common Requirements– NERC Certified System Operators– Some prior experience with PowerSimulator and
PALCO– Participants blind on Control and Experiment
Groups
Example of Actions Implemented
• Decrease Farlie output from 900 MW to 600 MW• Increase Doyle output from 357 MW to 550 MW• Increase PALCO import from External from 0
MW to 300 MW.
IncorrectCritical
Assessment
IncorrectCritical
Assessment
IncorrectCritical
Assessment
Situation Awareness
Analysis
Security Restoration Requirements
• Identification of every critical contingency • Identification of all critical system constraints• Identification of impact of re-dispatch on other
critical contingencies and constraints
Operator Decision Model - Enhanced
Real World Short termmemory
Long termmemory
Situation
Cues
That affect the
That let youcreate
That activate
Generates
Which youAssess by
Using your
Which youValidate by
Using your
TimeAvailable?N
Mental andComputer
Simulations
Mental andComputer
Simulations
ActionScript
Story(Patterns)
Mental andComputer
Models
Mental andComputer
Models
Y
Conclusions
• Generic PowerSimulator with PALCO system is an effective tool for Cognitive Task Analysis
• The environment is realistic and creates tension and stress for certified system operators
• Replay of dynamic screen capture provides basis to capture operator’s thought process.
• Level of wide area and local situation awareness can be identified at each step of the scenario.
• Mental models and mental simulations can be identified at each step of the scenario.
Cognitive Task Analysis Benefits
• Improved Curriculum– Need to add Contingency Analysis to PowerSimulator
• Improved Visualization Tools– Graphical vs. Tabular Contingency Analysis
• Improved – more specific, precise and constructive feedback to system operators
• Improved capture of expert knowledge to accelerate the spiral of knowledge
• Improved ability to learn from mistakes of others• Improved power system applications• Improved insight into the Operator Decision Model