euro control drm final

8/8/2019 Euro Control DRM Final

1/156

EUROPEAN ORGANISATIONFOR THE SAFETY OF AIR NAVIGATION

EUROCONTROL

DYNAMIC SAFETY MODELING

FOR FUTURE ATM CONCEPTS


2/156

DOCUMENT CHARACTERISTICS

TITLE

DYNAMIC SAFETY MODELING FOR FUTURE ATMCONCEPTS

EATMP Infocentre Reference:

Document Identifier Edition Number:

Edition Date: 08/09/06

Abstract

The DRM research project was aimed at developing a simulation approach able to provide aquantitative analysis of some critical operators activities considering the organizational context inwhich they take place and the main cognitive processes underneath. The process was able toprovide a trial application of it in a specific case study in the ATM context. This approach within thefield of HRA is able to interact with standard risk assessment methodologies in order to foresee

the possible criticalities arising from human performance in the ATC working contexts. Indeed, thesimulator that has been used (named PROCOS; Trucco & Leva, 2004), tries to integrate thequantification capabilities of the so called first generation human reliability assessment methodswith a cognitive evaluation of the operator.

KeywordsHRA cognitive simulation error recovery Future ATM conceptSESAR

Contact Person(s) Tel Unit

Daniela Grippa ~ 9 3330 DAP-SSHOliver Straeter ~ 9 5054 DAP-SSH

Author(s)

Maria Chiara Leva, Massimiliano De Ambroggi, Daniela Grippa, Randall De Garis, Paolo Trucco,Oliver Straeter

STATUS, AUDIENCE AND ACCESSIBILITY

Status Intended for Accessible via

Working Draft General Public Intranet Draft EATMP Stakeholders Extranet


3/156

EATMP InfocentreEUROCONTROL Headquarters96 Rue de la FuseB-1130 BRUSSELS

Tel: +32 (0)2 729 51 51Fax: +32 (0)2 729 99 84

E-mail: [email protected]

Open on 08:00 - 15:00 UTC from Monday to Thursday, incl.

DOCUMENT APPROVAL

The following table identifies all management authorities who have successively approvedthe present issue of this document.

AUTHORITY NAME AND SIGNATURE DATE

Please make sure that the EATMP Infocentre Reference is present on page ii.

DAP-SSHSafety Expert

Daniela Grippa

DAP-SSHSafety Expert Oliver Straeter

DAP-SSHSafety Domain Jacques Beaufays

DAP-SSH Alexander Skoniezki


4/156

DOCUMENT CHANGE RECORD

The following table records the complete history of the successive editions of the presentdocument.

EDITIONNUMBER

EDITIONDATE

INFOCENTREREFERENCE

REASON FOR CHANGEPAGES

AFFECTED

0.1 01.05.06 Initial draft all

0.5 01.09.06 Final draft all


5/156

CONTENTS

1. USE OF COGNITIVE SIMULATION FOR APPROCHING HUMANRELIABILITY ANALYSIS: Overview of dynamic risk modelingapproaches..........................................................................................................3

1.1 Introduction ...............................................................................................................................3

1.2 State of the art...........................................................................................................................5

1.2.1 The simulation CES (Cognitive Environmental Simulation)..............................................5

1.2.2 The simulator COSIMO (COgnitive Simulation MOdel) ....................................................7

1.2.3 The simulation SYBORG.................................................................................................10

1.2.4 The model TBNM (Team Behaviour Network Model) .....................................................14

1.2.5 The simulation AITRAM ..................................................................................................17

1.2.6 The simulation PROCRU (Procedures Oriented Crew Model) .......................................22

1.2.7 The simulation MIDAS (Man Machine Integration Design and Analysis System) ..........24

1.2.8 TOPAZ (Traffic Organization and Perturbation AnalyZer ) .............................................30

1.2.9 IDAC................................................................................................................................37

1.2.10 PROCOS.........................................................................................................................42

1.3 Summary of the Chapter.........................................................................................................48

2. USE OF COGNITIVE SIMULATION FOR APPROCHING HUMANRELIABILITY ANALYSIS: LINK TO EUROCONTROL ACTIVITIES.................53

2.1 Link with the ConOps Framework...........................................................................................53

2.2 The use of the cognitive Simulator PROCOS and the HERA predictive approach................58

2.3 Summary of the Chapter.........................................................................................................64

3. A QUANTITATIVE ANALYSIS OF SAFETY ISSUES: BY AN EXAMPLE


6/156

4.1.1 Scenario setting...............................................................................................................99

4.1.2 Number of repetition of simulation runs ....................................................................... 102

4.1.3 Summary of simulation campaign................................................................................ 103

4.2 Structure of the PROCOS reporting system........................................................................ 103

4.3 Collection and processing of results .................................................................................... 108

4.4 Normality test on the results of the simulation campaign .................................................... 112

5. ANALYSIS OF THE RESULTS FROM THE CASE STUDY: AN

EVALUATION OF THE EXPERIENCE GAINED..............................................1145.1 Discussion of the results of the case study.......................................................................... 114

5.2 Error type analysis ............................................................................................................... 118

5.3 Conclusions and potential developments of the approach .................................................. 119

5.3.1 Systematic integration of the PROCOS approach as applied to CONOPS................. 119

5.3.2 Strong findings from the pilot application ..................................................................... 120

5.3.3 Weaknesses of the current simulation approach......................................................... 121

5.3.4 Potential developments of the approach...................................................................... 121

6. REFERENCES..................................................................................................123

ANNEX I : CONOPS USE CASE Handle aircraft landing................................127

ANNEX II A: Task Analysis for Use Case Handling Aircraft Landing inflow chart format.............................................................................................130

ANNEX II B: Task Analysis for Use Case Handling Aircraft Landing inTable Format....................................................................................................132

ANNEX III: Cognitive Flowcharts Used Within The Simulator PROCOS AsValidated For ATC Applications.....................................................................145


7/156


8/156


9/156

EXECUTIVE SUMMARY

The DRM research project was aimed at developing a simulation approach able to provide a

quantitative analysis of some critical operators activities considering the organizational

context in which they take place and the main cognitive processes underneath. The process

was able to provide a trial application of it in a specific case study in the ATM context.

This approach within the field of HRA is able to interact with standard risk assessment

methodologies in order to foresee the possible criticalities arising from human performance

in the ATC working contexts. Indeed, the simulator that has been used (named PROCOS;

Trucco & Leva, 2004), tries to integrate the quantification capabilities of the so called first

generation human reliability assessment methods with a cognitive evaluation of the

operator. The simulator shall allow the analysis of both error prevention and error recovery. It

should integrate cognitive human error analysis with standard hazard analysis methods

(Event Tree and Fault Tree) by means of a semi static approach.

The dynamism of the simulator proposed in the present work is focused on the cognitive

simulation and, therefore, on the cognitive flow chart. However the operator actions are able

to modify only the state of some equipment of the plant according to:

- a limited set of the states in which the equipment can be turned;

- the error modes identified through the Task analysis and extracted as a result of the

cognitive simulation of the operator;

- an explicit relation between the actions outcomes (correct execution or Error modes)

and equipment status modifications (the relation has been derived from the Task

analysis).

Its focus is mainly in conveying a quantitative result, comparable to those of a traditional

HRA method, taking into account a cognitive analysis of the operator as well. As a further

step the simulator considers the evaluation of error management as part of the overall


10/156

The pilot study had two main objectives:

- Provide an overview of possible opportunities related to the use of a cognitive simulator

within CONOPS by investigating the future operational concept using the safety

fundamentals approach and using preliminary results of the Integrated Risk Picture

currently explored within EUROCONTROL.

- Evaluate the potential use of HERA-Predictive in combination with PROCOS for

concept evaluation (e.g., by analyzing the contributing factors to human error observed

in incidents, or by making use of experiences of approaches developed in other

industries like the CAHR method).


11/156

1. USE OF COGNITIVE SIMULATION FOR APPROCHING HUMANRELIABILITY ANALYSIS: OVERVIEW OF DYNAMIC RISK

MODELING APPROACHES

1.1 Introduction

The aim for this chapter is to provide an overview of the well know and commonly applied

cognitive simulation tools and compare them underlying their advantages and limits.

A definition of cognitive simulation, also referred as simulation of cognition, has been given

by Cacciabue and Hollnagel (1995):

the simulation of cognition can be defined as the replication, by means of computer

programs, of the performance of a person (or a group of persons) in a selected set ofsituations. The simulation must stipulate, in a pre-defined mode of representation, the way

in which the person (or persons) will respond to given events. The minimum requirement to

the simulation is that it produces the response the person would give. In addition the

simulation way may also produce a trace of the changing internal mental states of the

person.

In practice, a simulation is composed of three fundamental elements (Figure 1-1) that can be

considered necessary and sufficient for the development of a simulation of cognition:

- the theoretical cognitive model which defines conservation principles, criteria,

parameters and variable, that allow to describe cognitive and physical behaviour of

humans in a conceptual form;

- the numerical algorithms and the computational architecture, by which a theory isimplemented in a working computerised form;

- the task analysis technique, which is applied to evaluate tasks and associated

working context, and to describe procedures and actual human performances in a

f l


12/156

Figure 1-1 Simulation Model (Cacciabue 1998)

Cognitive simulation can be divided into two main types: qualitativeand quantitative.

Qualitative simulationdescribes the structure, the links and the logical and dynamic

evolution of a cognitive process, from the reception of an external stimulus to the

subsequent action. This type of simulation can be used for predicting expected

behaviours, in some well defined specific cases, where machine performance is also

simulated to the same level of precision.

Quantitative simulationis based on the structure of a qualitative one with the addition

of a computational section and can be used to make numerical estimates of human

behaviour. The qualitative study in this case is often coupled with a simulation of the

performance of the system the operator has to interact with. The final outcome of a

quantitative simulation can be the list of the types of action or errors performed by theoperator while executing a specific task, or a probability value for each type of action,

calculated through the simulation runs.

In a wider context of cognitive simulation two different types of analysis can be distinguished:

retrospective and prospective


13/156

Figure 1-2Types of simulation and types of analysis (Cacciabue 1998)

1.2 State of the art

In this section, some of the main approaches are discussed showing the architecture of the

cognitive simulations they propose and underlying their main properties.

1.2.1 The simulation CES (Cognitive Environmental Simulation)

The simulation CES (Woods, Roth, and People, 1987) has been developed for simulating

how people form intentions to act in nuclear power plant during emergency conditions.

CES is made of three basic kinds of cognitive activities (processing mechanism):

- monitoringand analysing the plant in order to decide if the plant is in an expected or


14/156

the operator in an optimal way. At any given time, the way in which knowledge is activateddepends on three different types of interaction:

- the interaction between knowledge driven and data driven processing;

- the interaction between resources and workload;

- The processing of the most evident and relevant information (the importance of a

process may be defined with respect on the ongoing one).

The performance of CES in different workload and environmental conditions is governed by

Performance Adjustment Factors (PAFs) by which the analyst can explore variability in

human behaviour.

The computational structure of CES contains two mayor elements:

- a knowledge base that represents the know-how of operator(s) in regard to the plant

and its behaviour;- an inference engine which is formulated in the form of processing mechanism.

CES considers two types of competencies that are generated from a number of studies and

analysis of the working environment in which CES operate and fed into the basis pool of

knowledge:

- the theoretical knowledgeof structures and functions of the plant under control;

- the empirical informationdeduced by investigation of operators and runs of simulation

in order to inspect the qualification of operator for emergency conditions.

The simulator CES is not able to analyse the interaction between two or more operator,

therefore it can not analyse the erroneous action produced by communication.

It presents a quite high complexity of application since it requires a simulation model for the

plant the operator has to interact with as well.


15/156

Figure 1-3 CES mechanism and cognitive process (Woods, Roth and Pople 1987)

1.2.2 The simulator COSIMO (COgnitive Simulation MOdel)

The Cognitive Simulation Model COSIMO (Cacciabue and Colleagues, 1992a) was

developed with the purpose to describe and predict quantitatively human behaviour during

dynamic human machine-interactions, mainly in highly automated working contexts like the

control rooms of nuclear power plants and air-traffic control rooms.

The simulator is composed of two main models: system model and operator model. The first


16/156

- RBFs are a snapshot of the configuration of the process controlled by theoperator and contain a set of appropriate actions for the management and the

performance of the selected tasks to deal with the current situation.

- KBFs are units of knowledge containing only heuristic rules as well as general

engineering and physical principles on the operation of the plant, usually

developed during training, experience and theoretical background. KBFs are

called into play in the working memory when a new planning process has to be

developed, as no RBF is available to handle the current situation.

The Working memory (WM) can be subdivided into two areas:

- Peripheral Working Memory (PWM), the area of vast capacity which receives

information directly from the KB and the outside world and makes selection;

- Focal Working Memory (FWM), the area of limited capacity which continuouslyreceives filtered information through the PWM.

The Cognitive Mechanisms, which are also referred as Primitives of Cognition,

governs the model and they are: Similarity Matching, Frequency Gambling and, less

frequently, Direct Interference.

- Similarity Matching (SM) primitive compares external cues (data perceived from

external world) and internal cues (elements that are included in KB) in order to

identify one or more procedures helpful to perform the current task.

- Frequency Gambling (FG) primitive resolves the conflict, which may occur if the

SM has selected more then one procedures, in favour of the most frequently

encountered and well know accidental situation.

- Direct Inference (DI) outlines a new action sequence not contemplated intonormal procedures on the basis of external stimuli and the KBF.

The Cognitive Functions are modelled and implemented through four interrelated

cognitive activities which produce the operator action on the basis of external stimuli

d ki Filt i Di i H th i E l ti d E ti


17/156

- Hypothesis valuation aims to decide whether a hypothesis can be trusted or hasto be rejected. If hypothesis selected after the diagnosis function is not supported

with sufficient evidence, the hypothesis is rejected and a new diagnosis is

initiated.

- Once a hypothesis has been selected, the WM is cleaned out and receives an

instantiation of the RBF associated with the selected explanation. This RBF is

called the Currently Instantiated Frame (CIF). The control and recovery actions

contained in the CIF are executed.

Like CES, the simulation COSIMO is not able to analyse the interaction between two or more

operator, therefore it can not analyse the erroneous action produced by communication. It

presents a quite high complexity of application as well, since it requires a simulation model

for the plant the operator has to interact with.


18/156

1.2.3 The simulation SYBORG

The simulation of the Behaviour of a Group of operators (SYBORG) has been developed

within the context of nuclear energy production studies by CRIEPI (Central Research

Institute of Energy Power Industry) and it aims at studying hypothetical severe accidents

involving human factors, as well as at supporting the design of intelligent interface and

control procedures.

The simulation has two major subsystems: a power plant modeland a human operator team

behaviour model. SYBORG exhibits the peculiarity of simulating two interfaces, one for the

interaction of human with the machine (Human-Machine Interaction, HMI) and one for the

group interactions (Human-Human Interaction, HHI).

Figure 1-5 SYBORG architecture (from Takano, Sasou and Yoshimura 1995)

The plant simulation models the power generation system, the controls, and the

alarms in the plant.

The operator modelaccounts for three operators: one is the leader of the team and

h h f ll i h diff l I i d h h l d d


19/156

- The short term memory accumulates temporarily information from the attentionmodel, conveying it smoothly to the thinking model, with a predefined time delay.

- The thinking module is the core of the single operator model; it introduces the

mental model mechanism that describes and illustrates how operators predict

plant behaviour and make decisions to prevent the deterioration of its conditions;

it calculates and defines the execution of procedures and actions to be carried

out.

- The medium term memory obtains information filtered by attention micro model

and information contained in long term memory and designs the mental

mechanism of the operator. In practice, the medium term memory serves as

buffer and sustains the transfer of information between the thinking model and the

long term memory.- The long term memorycontains the knowledge necessary for the thinking model,

including plant configuration, parameters, variables, dynamic behaviour, meaning

of alarms, and predefined procedures. Furthermore, the store knowledge contains

the relation between events and parameters, events and causes, change of

parameters and interlock, change of parameter and carrying out

countermeasures.

- The action micro model implements the control actions decided by the thinking

model. It is possible to calculate the value of the operation standard time of the

action and assesses the workload produced by action.

- The utterance micro model develops the communication between the team

members. The communications are distinguished in twelve categories, forexample: Report (reading the instruments), Application (application of the

procedures)


20/156

Figure 1-6 Individual Operator Behaviour Model (from Takano, Sasou and Yoshimura 1995)

The Human-Human interface (HHI) modelperforms three fundamental functions: the

task assignment, disagreement management and utterance management.

- The utterance management micro model, when communication takes place,

records the communication and sends itself at the receiver. The answer has to

feedback via HHI in order to confirm the success of the communication.

- The task assignment micro model incorporates the characteristics of team

behaviour related to the cooperation with each other to deal with a work that is

divided among operators.

- The disagreement management micro modelsimulates the characteristic of team

behaviour related to the fact that real operators communicate to exchange plant

information and their thoughts on the plant conditions, and they decide on

countermeasures that are thought to be the best ones for the plant. The


21/156

Figure 1-7 parameters using in the disagreement management ()

In order to obtain a quicker implementation, the model above explained is reduced by

applying appropriate aggregations. The properties of the Thinking, Short Term Memory,

Medium Term Memory and Long Term Memory micro-model have been assigned in two new

modules: the Skill Base Reaction (SBR) and the Knowledge Based Processing (KBP).

The SBR module regards the performance of the immediate reaction (when the

warning alarms go off the operator will carefully monitor the control panel).

The KBP module performs the following tasks:

- It receives the information from the external world and from the Long Term

Memory and produces the mental model;

- It selects a strategic objective on the basis the mental model produced;

- It researches the opportune countermeasures and checks the procedures carried

out until the operator notices some effects;

- It defines the priority;

It understands the situation of the system


22/156

output. The Leader module is the same of the follower, except for the Action micro-modelthat there is only in follower module. In fact, the leader does not have action tasks but has

only management tasks.

Figure 1-8 Flows of information

The simulator is well able to describe also interaction among members of the team, is


23/156

Behaviour Network Model (TBNM). This model is made up of four micro-models: TaskModel, Event Model, Team Model, and Human- Machine Interface Model.

Figure 1-9 Team Behaviour Network Model (Shu et al. 2002)

Task Model is used to depict team tasks and to identify the associated context in

which the interaction between the operators team is developed. Complex task is

subdivided and assigned to an operator in accordance with your individual

peculiarities.

Event Model specifies the developments of a situation after that an initial event

occurs.

Team Modeldefines a factors team (organizational structure, individual peculiarities

of the operator that are the root of the communication). In normal operation the team

structure is predetermined and each member of the team knows what you have to do

and how you have to communicate. The collaboration pattern is dynamic because the

environmental conditions change and the operators can execute abnormal action.

Human- Machine Interface Model shows the layout of the control room and all


24/156

Figure 1-10 Cognitive process team (Shu et al 2002)

The current state of the system is identified depending on know-how of the operator or by

information arose from the other member of the team.During Decision Makingprocess the decisor-making chooses, in the bound of his authority,

an option from emergency list.

During the Planning process the planner, selected depending on his knowledge and

responsibility, chooses a procedure from list of the plans.

During the Execution process the executor, selected depending on his responsibility and

capacity, performs an operation from action list in according to operative procedure.

The performance of the cognitive process is outlined by timing fault tree like reliability

assessment of the system. The representation includes the communication between

members of the team and the interaction with dynamic context. For a quantitative


25/156

1.2.5 The simulation AITRAMThe simulation AITRAM aim is to contribute to the improvement of the learning process by

developing an advanced training system for aeronautical maintenance technicians. This

simulator addresses both technical and Human Factors issues and is based on innovative

concepts, new cognitive approaches and simulation technologies such as Virtual Reality.

This model integrates Human Factors and Technical competency requirements in order to

satisfy those Human Factors and Technical training objectives, which are most frequently

applied as separate elements in aviation maintenance domain.

Figure 1-11 Process model for Human Factors and Technical training integration (Mauri, et al 2001)

The making of the simulator consists in three steps: creation of the model, conceptual design


26/156

SHELL model has been developed with the idea to describe the relationship betweenhumans and other elements of the working environmental through the following

elements: Software, Hardware, Environment, and Liveware.

In the context of aeronautic maintenance, the relationship between various elements

can be:

- Liveware-Environment: this kind of relationship covers social and technical aspect

of working context in which humans are working and that can be affect the

operator behaviour.

- Liveware-Hardware describes the relationship between the technician, plant and

working tools.

- Liveware-Software is the relationship based on interaction between technician

and procedures (AMM: Adres Maintenance Manual) that he musts follow.- Liveware-Liveware covers communications and the transfer of information

between two technicians. Furthermore, this kind of relationship includes possible

contacts with supervisor.

o PIPE model

PIPE model is based on the four main cognitive functions that describe the human

behaviour: Perception, Interpretation, Planning, and Execution. These functions are

controlled and supported by the cognitive processes of Memory and Allocation of

resources. These two cognitive processes affect the maintenance man by error

modelling and through the interrelation with other operator and environment.


27/156

The process starts with a stimulus and finishes with a response. Stimuli are producedfrom the control machine, the work environment or the contextual conditions, while

responses are the manual actions executed in according with stimuli and the related

cognitive process.

o Integration of SHELL and PIPE model

The operator model used in the simulator is the result of the integration of SHELL and

PIPE model. The four main cognitive functions of PIPE are managed through elements

of the SHELL model. Namely, during the task performance the operator interacts to

Hardware, Software, Environment and other operator through the perception and

interpretation functions that detect and process the stimuli coming out of the plant, the

procedures or the other operator. Having the information gathered, the operator can

plan the action to be executed. The execution of the action at the time tpermits thestart of a new cycle related at the time t+1.


28/156

b. Conceptual design

The conceptual design consists in two steps: Data Modellinge Function Specification.o Data Modelling

This step characterizes the elements of the model (Software, Hardware, Liveware,

and Environment) and determines the entity-relation diagram. This diagram is used to

create a database that is a fundamental for correct execution of the simulation run.

- Software: Task

The task simulation is performed by processing the Tabular Task Analysis (TTA)

(Schraagen et al. 2000). The task has to describe in great detail in order to

identify each action that the maintenance man performs and outline the effects of

the action itself. Then the TTA allows subdividing each task in units that represent

the individual action.

- Hardware: Objects and ToolsEvery objects and tools used during the execution of the task have to list and

label with an unambiguous codec.

- Liveware: Technician Performance Influencing Factors (Technician PIFs)

The Technician PIFs are those factors which influenced the operators

performance; examples are motivation, stress, experience The value of PIFs

can be fixed at the beginning of the simulation and can change during the

simulation run.

- Environment: Environment Performance Influencing Factors (Environment PIFs)

Environment PIFs are the external factors to the maintenance man which


29/156

The simulation process consists in three instalments: Initial Set Up, Simulation Runand Generation of Output Data.

Figure 1-14: Simulator stucture (Mauri, et al 2001)


30/156

Figure 1-15 Action Execution Flowchart (Mauri, et al 2001)

- Generation of Output Data: at the and of process, the simulator indicates the

pathway followed, action codec, a brief description, commission and omission

errors (if they occurred), time action and time task, trend of the

Environment/Technician PIFs during the run.

c. Implementation

In order to implement the model above described is used Microsoft Visual Basic 6.

D t d th h Mi ft A


31/156

The basic structure of the model comprises the Simulation of the Aircraftunder control and

the Simulation of the Single Operator.

Figure 1-16 The model PROCRU for individual crew member (Cacciabue 1998)

o The Simulation of the Aircraft includes Machine Dynamics, containing display and

control variables, and ATC/CREW model, which comprises communication with other

crew members and the external world, such as the air traffic control.


32/156

- The decision of the action or other cognitive activities to carry out, which is based

on the procedure oriented modelling (Procedure Selector) and is affected by the

previous cognitive activities, the aims of the operator and the assessment of

possible consequences.

- The action implementation (Execution), which implies a process of communication

with other crew members, or the external world, and the performance of actual

control activity, either by observing (Monitor Requirements) or by operating the

control system (Control Requirements).

The simulation PROCRU also comprises the model of Knowledge Base of the Operator,

which is made up of Procedures, Description of the aircraft, Interaction Module that describe

the interaction between crew members and ATC body.

The model includes, amongst the events that are considered for situation assessment, facts

that are not explicitly dependent on the vehicle state variables. This means that one of the

basic requirements for modelling cyclic cognitive processes is respected, i.e., a cognitive

activity may be generated by another cognitive process and is not only the result of a

machine or context stimuli. This qualifies PROCRU as a cyclic simulation.

The simulation of communications is performed by referring to standard procedural verbal

requests or responses as is required by procedures.

PROCRU presumes the use of cognitive task analysis for preliminary definition of procedures

and actual performances carried out in the cockpit.

It can be concluded that PROCRU, although developed in the early 80ies, remains, even

today, a remarkable simulation approach worth reviewing and considering as possible means

of representing pilots (operators) behaviour, even when dealing with highly automated

cockpits or control rooms, and multiple interaction processes.

1.2.7 The simulation MIDAS (Man Machine Integration Design and Analysis System)


33/156

MIDAS combines graphical equipment prototyping, a dynamic simulation, and human

performance modelling with the aim to reduce design cycle time, support quantitative

predictions of human-system effectiveness, and improve the design of crew stations and

their associated operating procedures. Furthermore, MIDAS has been conceived as a

modular structure and can, in principle, be apply to study different domain environments, at

different level of complexity.

The basic architecture of MIDAS contains a model of the system under control, the World

Representation, and the Operator Model.


34/156

expressed in three different formats: a time script, a stimulus response, or a finite

state machine representation. In addition to the physical and functional models for a

cockpit, the entire crew station can be place inside of a vehicle model, linked to

guidance and control models, and place inside a terrain database or gaming area.

The World Representation also contains the probabilistic module, by which failure and

malfunctions may be introduced on a probabilistic basis.

o Human Operator Model represented by MIDAS contains the following models and

structures.

- Physical Representation: a model of human figure anthropometry and dynamics.

The model, Jack, represent human figure data (e.g., size and joint limits) in the form

of a 3-D mannequin which dynamically moves through various postures and visual

fixations to represent the physical activities of a simulated human operator.

- Perception and Attention: MIDAS has focused on modelling perception agent

computes or cockpit objects imaged on the operators retina, tagging them as in/out

of peripheral and foveal fields of view, and in/out of focus, relative to the fixation

plane. Objects in the peripheral visual field are partially perceived. In order for

detailed information to be fully perceived, the data of interest must be in focus,

attended, and within the foveal vision for 200 ms. The perception agent also controls

simulation of commanded eye movements via defined scan, search, fixate, and

track modes. Differing stimuli salience are also accommodated through a model of

pre-attention in which specific attribute, e.g. colour or flashing, are monitored to

signal an attention shift.

- Updatable World Representation (UWR): this model contains the basic knowledge

of the operators, the information concerning procedures and equipment, the activity

of working memory on the information perceived from perception module, and the

know relationships between objects and system components. UWR contents are

d fi d b i l ti l di f i d i i d l d i t


35/156

describing preconditions, temporal or logical execution constraints, satisfaction

conditions, estimate duration, priority, and resource requirements. Resources

include both physical effectors and psychomotor task loading.

- Scheduler: Activities which have their preconditions met, temporal/logical execution

constraints satisfy, and required information retrieved from memory are queued and

passed to a model of operator scheduling behaviour. Based on the users selected

scheduling strategy (e.g., workload balancing or time minimization), activities are

executed in priority order, subject to the availability of required resources. MIDAS

contains support for parallel activity execution, the interruption of on-going activities

by those of higher priority, and the resumption of interrupted activities.

New MIDAS design

A major effort to redesign the MIDAS system is underway so as to enable a smaller

development time for new scenarios (from several months to one or two weeks), and in order

to increase the efficiency of the running system (from around 50 times real-time to near real-

time), to facilitate the process of replacing cognitive and perceptual models (from weeks to

days), and to expand the functionality of the system. There was also a desire to update a

human operator model, in particular to account for more widely accepted views on human

information processing and its likely underlying architecture.

The approach taken in MIDAS redesign is object-oriented rapid prototyping. Initial design

efforts produced a high-level system architecture with the following elements:

- a domain model supporting components necessary for running a simulation;

- a graphics system to enable simulation visualization;

- an interface for end user specification of the target domain models;

- a simulation system for controlling the simulation and collecting data;

- a results analysis system for examining simulation data after it has been collected.

The domain model is centred on a crew station, with the following models:

th i t i th t ti


36/156

hands and head was used), capturing physical aspects of human behaviour, permitting

visualization of reach, fit, and fixation activities. The processing architecture of the human

operator model considers as main components the following elements: input, memory and

central cognition, output, and attention.

Figure 1-18 New MIDAS Operator Architecture

o Operator Inputis received from the environment through the senses.


37/156

o Memorynow consists of both Long-Term and Working Memory components.

The former, similar to the existing UWR, contains both declarative and procedural

knowledge. Procedural knowledge is represented as Reactive Action Packages (RAPs)

which describe how to accomplish a given goal and consist of the methods possible for

achieving that goal, when each is most appropriate (according to the current context), and

how it is known that the goal is satisfied.

Working Memory has three main contents:

- Even Management in which new inputs are assessed to determine whether they were

expected or not (if so they are simply used to update the current context, if not, they

generally trigger the creation of new goals to handle an expected event);

- Agenda Management in which the goals on the Task Agenda are examined, based

upon priority and the current situation, to determine which one focus on next;

- Plan Execution which, after once goal is selected, is used to retrieve the appropriate

RAP from Long-Term Memory.

o Motor Control Process regulates bodily movement, manipulation of equipment, and

speech output. If required resources are available, a motor activity is created and

processed.

o Attention, within the new architecture, is planned as a limited central resource.

Therefore, for any of the behaviour described previously to occur, the responsible

process must first secure the necessary resources of attention. If these are not available,

then delay of that process, or an interruption of an ongoing activity, is necessary.


38/156

1.2.8 TOPAZ (Traffic Organization and Perturbation AnalyZer )

TOPAZ is a simulator that can be used for analysing errors of Air Traffic Controllers. It is

based on a stochastic analysis framework which implies the following five activities:

a. Develop a stochastic dynamical model for the situation considered,

b. Where necessary develop appropriate cognitive models for human operators involved,c. Perform the stochastic analysis necessary to decompose the risk assessment,

d. Execute the various assessment activities (e.g. through Monte Carlo simulation, numerical

evaluation, mathematical analysis, or a combination of these),

e. Validation of the risk assessment exercise.

The aim of the Topaz developers was to represent for the selected encounter scenarios the

results from the qualitative safety assessment in the form of a Stochastic Differential

Equation (SDE) on a hybrid state space. Unfortunately, the direct identification of the SDE

model would be very complicated for most ATM situations. In addition to a very large state

space of the corresponding SDE, there are many interactions between the many state

components. Therefore the developers shifted their attention towards a systematic approach

to develop an SDE instantiation through the development of a specific type of Petri Net: the

Dynamically Coloured Petri Net (DCPN), (a more detailed description is in the references:

M.H.C. Everdij, H.A.P. Blom and M.B. Klompstra 1997).

Operator Model

The Operator Model used consists of a contextual human task-network model, which is

formulated in terms of a DCPN, and which effectively combines the cognitive modes of

Hollnagel (1993) with the Multiple Resources Theory of Wickens (1992), the classical

slips/lapses model (Reason, 1990) and the human capability to recover from errors


39/156

independent from the scenario and operational concept. Secondly, the task is decomposed

according to a scenario/concept specific dimension, where the controller task is described at

the level of operational functions in the scenario. The task decomposition along the generic

dimension has been identified from (Buck et al., 1996). The following subtasks resulted:

1. Sensing (to gather all information which is needed to get an overview over the air traffic

situation).

2. Integration (to connect the gathered information thus forming a more global air traffic

picture).

3. Prediction (to use the more global picture to anticipate future situations and events).

4. Complementary communication (pass the information to aircraft in order to improve the

pilots understanding of the situation).

5. ATC problem solving planning (to use the understanding gained from the more global

perspective to plan and

prioritise aircraft actions).

6. Executive action (to communicate information and priorities as instructions to the aircraft in

the system).

7. Rule monitoring (to ensure that the active components of the system behave in

accordance with the rules; monitoring and taking corrective actions for exceptions).

8. Co-ordination (to coordinate laterally with other parts of the ATC organisation).

9. Over-all performance (to ensure that the objectives of the operation are achieved, and that

the infrastructure

functions correctly).

10. Maintenance and monitoring of non-human part (to ensure that all systems supporting

the controller work correctly).

In modelling the influence of the context on performance it has been adopted a mathematical

model that incorporates two control Modes from Hollnagels approach : tactical control and

t i ti t l Th h t i ti i fl f th t l d th f


40/156

Table 1-1: subtask related to Anticipation an Alerts (Blom, Daams, Nijhuis, 2000)

Sensing:Tactical: Whenever possible the controller scans his display to detect possible deviations from ATCintentions. The controller divides the display into regions of interest and assesses these regions in aparticular order. If scanning is interrupted at some time instant, the controller will resume scanning startingat the region that he was scanning when the interruption took place. Further information may also beobtained through R/T communication.

A1

Opportunistic: Whenever possible the controller scans his display to detect possible deviations. Thecontroller scans in a random fashion.Integration

Tactical: The ATCO systematically integrates the information derived from scanning to improve his mental

picture of the traffic situation. When some relevant information is not available, the ATCO may return tosensing to actively seek information to improve his assessment of the situation.

A2

Opportunistic: The ATCO integrates the randomly obtained information. An incomplete or even distortedmental picture may develop.Prediction

Tactical: The ATCO extrapolates his mental picture to the future traffic situation. On the basis of theassessment of the situation, the ATCO decides whether a problem may occur in the mid-term future

A3

Opportunistic: The assessment of the future situation is restricted to a short time horizon and is based onincomplete information. It is assessed whether a problem may be expected in the short-term future.

Problem solving/planning

Tactical: On the basis of the assessment of the (future) situation, the ATCO decides a resolution to theexpected problem. In principle, the resolution involves re-planning the aircraft trajectories in an optimalfashion with respect to safety, efficiency.

A5

Opportunistic:The resolution is aimed at solving the imminent problem only.Executive action

Tactical: The controller gives a series of R/T instructions to the aircraft involved. He verifies whether thepilot(s) readback these instructions correctly.

A6

Opportunistic:The verification of correct readback may be omitted.Rule monitoring

Tactical: After the R/T communication the controller verifies whether the aircraft comply to his clearances

A7

Opportunistic:This may be omitted or be performed less thoroughly.

Scheduling of subtasks

The subtasks have been scheduled according to a defined strategy. The scheduling strategy

is expressed in the following (input) task parameters:

Pre-emption For each subtask an assumption is made whether it may pre-empt another

subtask.

Concurrency For each subtask it is known whether it may be performed concurrently with

another subtask.


41/156

Rule 1: An initiated subtask will be placed in the stack before the subtasks that it may pre-

empt.

Rule 2: If the first two subtasks of the stack can be processed concurrently, this will be done

(subtask duration will be slightly longer, however). (Blom, Daams, Nijhuis 2000)

Mathematical model of tactical ATCO

The authors provided a description of the Topaz model from an input-output point of view

(Blom, Daams, Nijhuis 2000):

Initiation

Three stimuli for ATCO cognitive activity are identified: ATCOs Anticipation, Automation

alerts and other actions. Activity triggering situations that first have to be detected by the

operator (like an aircraft severely deviating from its route) are not considered as an initiation

stimulus, since general sensing is modelled as a part of the operators task and therefore the

sensing activity has to be initiated first. For the occurrence of certain stimuli various other

ATM modules may need to function properly, such as e.g. the ATCO HMI and surveillance

for an Automation alert. Using a Petri Net each stimulus is modelled as a place, connected

with one transition that fires if initiation of the corresponding cognitive activity occurs. These

transitions produce two tokens: one token returning to the stimulus place for future

generation of cognitive activity and one token in a stackplace. The stackplaces represent

the situation that the respective initiated cognitive activity has to wait until the operator has

completed other (more important) tasks. The places Anticipation, Alert and Other action

represent initiation of cognitive activity by own initiative, Automation alerts and other action

(e.g. a pilot request) respectively. Preconditions on occurrence of these stimuli are modelled

within the respective transitions: if the preconditions are not met the transition does not fire.

For example: the proper functioning of the ATCO HMI as a precondition for the occurrence of

an Automation alert triggering ATCO cognitive activity is modelled as a precondition for the

fi i f th t iti t d t th Al l


42/156

actions). Each subtask is represented by a place in the Petri Net, which is named after the

cognitive activity it represents.

The tokens then model cognitive activity on the subtask that corresponds to the place that

they reside in. Some cognitive activities may be performed for several purposes, leading to

several places with the same name. Below we describe the places with respect to the

cognitive activities that they represent. The places named sensing represent the situation

that the ATCO is gathering information to improve his picture of the traffic situation. The

places named integration represent the situation that the ATCO incorporates the newly

obtained information into this mental picture. The place named communicationrepresents

the situation where the ATCO makes his knowledge of the situation available to the pilots.

The place named over-all performancedescribes the evaluation of sector performance as a

whole. In the predictionplace, the ATCO extrapolates his picture of the traffic to the future,

while in the problem solving/planning place he synthesizes solutions to possible (future)

problems. In the executive actionplace the operator gives clearances to aircraft, followed by

a monitoringplace where it is verified whether the aircraft complies with these clearances. In

the outplace the tokens are collected after performance.

Whenever one subtask is logically performed after performing another (e.g. prediction is

performed afterintegration) and they have the same scenario specific purpose a transition is

drawn between those two subtasks.

The subtask scheduling then follow the rules previous mentioned. Scheduling depends on

the relative priority of a subtask and the possible simultaneous performance of two subtasks.

Priority is coded as a number 1,2low numbers have higher priority, and each priority level

corresponds to a colour for the Petri Net. The priority colours are up-dated whenever a new

token is initiated and when a token is collected in the outplace, according to a suitable set of

assumptions.

For each subtask the time needed to complete it has a certain probability density, given the

t t l d f th ATCO d ibl t f f th bt k


43/156

assumed that the type of clearance given is determined during the executive action subtask

only and that it depends on the control mode only. So the firing of the transitions after the

executive action places also affects the Petri nets of other ATM modules: completion of

executive action means that a decision to give a clearance to an aircraft has been carried out

and therefore the firing of these transitions describes the ATCO control actions.

In the Topaz model the ATCO performance depends on the control mode, scheduling rules

and it results in a clearance. In the DCPN model of the ATCO the two control modes

identified, which are each represented by a place in the Petri Net: the place named Tactical

models the situation that the controller has a relatively high degree of control and the place

named Opportunistic models a relatively low degree of control. The switching between

control modes is modelled by transitions between the Tactical and Opportunistic places. The

resulting subnet contains one token, the place of which defines the current degree of control.

The firing of the transitions between the control modes depends on the number of tokens in

the stack places, which should provide an indication for the subjectively available time, and

the number of times that monitoring was followed by another executive action during the last

few minutes, which should be a proxy of the outcome of previous actions measured as the

number of clearances that the controller considers to be insufficiently effective.

The Petri Net of the ATC model is represented in Figure 1-19.

In the model ATCO may give erroneous clearances (e.g. switching heading and speed, or

clearance given to a different aircraft than he intended to: call-signs mixed up). These errors

are incorporated as random variations in the ATCO actions, and the error types are

represented as a colour value of the tokens in the place Clearances.

It is not clear however what type of data calibration is used for these random variations since

it is a HEP type of data.


44/156

Figure 1-19 : Petri Net of tactical ATCO model (Blom, Daams, Nijhuis 2000)

The developers performed a comparison against statistical data for the ATCO routine


45/156

1.2.9 IDAC

IDAC is a cognitive simulator based on many other HRA first and second generation HRA

methodologies. IDAC has been mainly designed to be applied in the probabilistic accident

simulation environment (ADS Accident Dynamics Simulator) (Chang and Mosleh 1998)

developed to perform dynamic probabilistic risk assessment of Nuclear Power plants.

IDAC represents the behaviour of a single operator or of a group of operators, taking intoaccount three generic types: decision maker, action taker and consultant ( Chang and

Mosleh 1999, Mosleh and Chang 2004).

The acronym stands for the various modules that composed the simulator which is to say a

model for information processing (I), problem solving and Decision Making (D), action

execution (A) of a crew (C).

The ADS code simulates accident scenarios and generates information about the external

world, this is used as an input for the IDAC code that in turn generates the possible response

of the crew.

The architecture of the simulator can be represented in Figure 1-20.


46/156

The main elements of the IDAC response model have been described by the authors

(Mosleh and Chang 2004) as reproduced in Figure 20 where the model is placed within a

similarly high level model of how an individual operator interacts with the external world. The

blocks shown in Figure 1-20 are:

(1) The external world to a specific operator includes the system, the physical environment,

other operators, and the external resources. These are the entities that the operator has to

interact with and that are provided by the DAS.

(2) The external filter is any factor external to the operator that can block or distort the

information from the external world before being detected by the operators sensory organs

(e.g., visual and auditory organs). Examples of external filters are noise and view obstruction.

(3) The information that has passed the external filter enters the inner world of the operator.

The main components of the internal world are Mental State (MS), Memory, and Rules of

Behaviour.

a. The MS is represented by a set of inter-related variables (i.e., internal PIFs). It

defines the operators state of mind in various dimensions such as individual

differences, situation perception and appraisal, feelings about the situation, and

certain cognitive behavioural modes. MS could act as an internal filter by which the

incoming information is masked.

b. IDAC model includes three types of memory: working memory (WM), intermediate

memory (IM), and knowledge base (KB). WM stores limited information related to

the current cognitive process. IM, theoretically unlimited in capacity, stores

information related to recent cognitive processes which could be easily retrieved at

any time given appropriate stimuli. KB, also theoretically unlimited in capacity,

stores all PS/DM related knowledge obtained from training and experience.

c. Rules of Behaviour govern the cognitive, emotional, and physical responses of an

individual for a given state of PIFs and the content of memory. More specifically,

th i ti f d MS f t t t th d i th f


47/156

strategy. There is hierarchy of goals and sub-goals, such that complex problems

are broken down into simpler ones, and solved one at a time or concurrently, using

corresponding strategies. The problem solving process involves a series of

decisions to be made or solutions to be selected based on available alternatives.

The decisions making stage has its own strategy which is cost-benefit

optimization.

(4) Within the time window of interest (i.e., duration of an accident,) some PIFs are static

(e.g., human-system interface quality) and some are dynamic (e.g., number of alarms control

room generated in an accident).

(5) Actions are external manifestation of decisions (to act) formed by the cognitive process of

Problem Solving/Decision Making. The action performing process (A) executes the decision

made through the D process. The actions are skill-based, requiring little mental effort .

Through action the operators interact with the external world, which in turn generates new

information starting another Problem Solving/Decision Making cycle. The operators actions

could be blocked or distorted by the external filter. This interaction loop continues until the

desired system state is reached (e.g., problem solved or an undesired state of the system

reached). (Mosleh, Chang 2004).

Any cognitive response of the operator or of the crew to an external situation perceived, is

translated into a problem statement or a goal, requiring solution. The model tries to cover

also why and how a response process is initiated and why and how a goal or a solution is

selected or abandoned. In order to go through the I-D-A process dynamically and in

response to external dynamics IDACs model has an internal engine comprised of the Mental

States with its set of states variables and rule of behaviour, plus the information processing

engine of the Working Memory. The stimuli are an individual perception of the external world.

The tendency to act on stimuli include the individuals internal feelings pertaining to the

ti li ( ti t i t k l d t ) Th lt i i h l i l d


48/156

discrete cognitive events, such as the step of the information processing, goal selection and

execution of problem solving strategies to achieve the goal. The cognitive basic events and

the resulting observable actions are stochastically selected among the possible alternative

paths and the related outcomes that have been identified as potential, (each of them with an

assigned conditional probability). These probabilities are conditioned on the past history of

the sequence preceding the events, and their values are calculated as a function of the

states of the various parameters identified as influencing factors. The uncertainties identified

by the authors in connection with the probabilities evaluated by IDAC are:

- uncertain effects and variability of factors which are not included in the current model

- Uncertainty of the degree of influence of factors included in the model on each other

and on the model output

- Stochastic variability of the spectrum of situations that are collectively approximated

by individual basic events and parameters of the model

- Intrinsic residual unpredictability of human behaviour

In current application IDAC uses qualitative and quantitative scales in order to asses the

state of input variables and parameters (PSFs). Those elements are then used to calculate

the score for each alternative response. The completeness of the set of possible alternativesis assumed therefore the probability of each alternative is calculated as the normalized score

of that alternative:

=

=N

j

j

i

i

score

scoreP

1

Each PSF values ranges from 0 to 10. Static PSF are input to the model and quantified by

HRA analysts using conventional methods such s expert judgment and surveys. Dynamics

PSFs are function of the scenario and of the static PSFs.

In IDAC observable human actions are classified as errors in respect of the external


49/156

The premise of internal reference points is that the error has occurred in the module for

which there was a correct input but an incorrect output (I.E. an error will be defined as under

action execution, A-element if the action was incorrect given a correct problem solving

process, D-element).

Currently IDAC is implemented as the HRA module of a Dynamic PSA computer code

Accident Dynamic Simulator (ADS) with its embedded models for a nuclear power plant

(which include the Relap5 thermal hydraulic simulation code). ADS uses the DiscreteDynamic Event Tree (D-DET) approach (Amendola 1988, Acosta, Siu 1991) to generate

possible time dependent scenarios based on dynamically changing states of various systems

and operator response. Probability of a scenario overall is calculated as the product of

conditional probabilities of branches that constitute the scenario and the operator responses

are among these branches.


50/156

1.2.10 PROCOS

PROCOS is a probabilistic cognitive simulator for HRA studies. It has been developed within

the Politecnico of Milan in Italy for approaching human errors for highly procedural tasks

such as operator involved in the commissioning phase within the control room of an

ammonia urate plant.

This simulator is based on a semi-static approach: it provides a quantitative result,

comparable to those traditional and static first generation methods, but it takes also into

account a cognitive analysis of the operator.

PROCOS differs from the way traditional human reliability methods represent human actions

because it considers the recovery phase as well. In fact there are two different flow charts: a

flow chart to simulate the operator behaviour in normal operations and a new flow chart for

the recovery phase.

The simulator does not imply the development of a detailed model for the interaction

operator-context; the context is taken into account mainly through the use of Performance

Shaping Factors, as proposed in traditional HRA methods.

The Cognitive model of the operator

The cognitive model of the operator is based on a combination of SHELL (Edwards, 1988)

and PIPE (Cacciabue, 1998). The two models have been combined as already proposed in

the AITRAM project (see above paragraph on AITRAM).

PIPE represents the process of human cognition according to the definition of Minimal

Modelling Manifesto given by Hollnagel:

A Minimal model is a representation of the main principles of control and regulation that are

established for a domain-as well as for the capabilities and limitations of the controlling

system (Hollnagel, 1993).

SHELL has been used for organizing the information regarding the context and the


51/156

- Error in Perception: errors regarding issues related to the detection and to the

understanding of information;

- Error in Memory: errors related to both short-term storage and more permanent

information based on the persons training and experience;

- Error in Decision: errors related to the judgement and decision making process

required to the operators;

- Error in Response: it is sometimes possible to carry out actions that have not beenintended, an example of this is often referred as a slip of the tongue.

The error types have been linked with the error modes through a correlation matrix that

is specific for every task the error type and the connected error modes have to be

referred to. An example is shown in Table 1-2.

ERROR TYPE

Perception Memory Decision Response

Not Done Weak weak

Other Then Medium Strong medium

... Weak medium

Strong mediumERROR

MODE

Part of strong

Table 1-2: Correlation matrix between Error Mode and cognitive Error Type

The taxonomy used for the recovery phase has been proposed by Kontogiannis

(Kontogiannis, 1997), breaking down the error handling process in three phases: Detection,

Localization or explanation and correction.

- Error in Detection:the error happen in the phase in which the error is detected. The

detection can take place at different stages of the task execution:

o Detection in outcome stage


52/156


53/156

- Steps of the task (Task Analysis);

- Possible error modes to be considered.

The main Output is to provide a probability value in respect of the operator actions identified

as critical and a probability value for the corrective action in the recovery phase as well.

The architecture of the simulator is centred on the cognitive flowchart. A cognitive flowchart

is a decision blocks diagram through which it is possible to represent the succession of

cognitive functions used by the operator in order to execute an action.

Decision blocks criteria

The mathematical model for decision blocks criteria of the flowchart is the main critical

feature of the operator module of the simulator.

Each decision block has to possible exits: Yes and No. The exit process is stochastic and

it depends on the PSFs values and the influence they have on each decision block.

If we indicate with X the possible outcome of a decision Block, X is a Bernoullis variable.

If the following values are associated with X:

YesX= 1

NoX= 0

Then the probability density function fx(x) is equal to:

==

==

otherwise

xorxperpp

pxfxf

xx

xx

0

10)1(

),()(

1

(1.1)

where : 0 p 1

1 p = q.

The probability of having Yes as a possible exit of the block can be expressed as [P(X= 1)]


54/156

changes in human responses induced by changes in external conditions can be described by

a logarithmic relationship(Fujita & Hollnagel, 2004):

bSLIaHEP +=)(log10 (1.2)

where:

HEP

Human Error ProbabilitySLI = f(PSF)Success Likelihood Index

a, b parameters

The SLI index is defined as follow:

= =jN

iii rwSLI

1)( (1.3)

where:

winormalised weight of the i-th PSF for the cognitive process of the j-th block

ri i-th PSF value

Nj number of PSFs for the j-th block

and =

=jN

i

iw1

1


55/156

representative of the cognitive aspect described in each decision block. The value of the

median has been used in order to calculate the two parameters a and b, from the formula

(1.1), in correspondence to a SLI mean (SLImean) value for the nominal working condition

(central value of the interval for each PSF involved). The second condition was to consider

SP=0 for SLI=0 as a bound condition.

In this way has been possible to define a and b for each block.

0 = 1-10a 0 + b

b = 0 (1.4)

SPTHERP = 1-10aSLI

mean a (1.5)

In this way it is therefore possible to determine the probability of each exits from the block

using the SLI index:

blockSPpq == 11 (1.6)

THERPblock HEPSP =1 (1.7)At the beginning of a simulation process, the value and the weight wi for each PFS are

extracted as a random variable from a uniform distribution in an interval e-f and winf-wsup

respectively.

The strong point of this simulator is the medium-low application complexity, especially it is

very easier then each other quantitative method present in literature. Furthermore PROCOS

can be applied to many different fields with a few efforts to perform the necessary changes.


56/156

1.3 Summary of the Chapter

The main elements analysed in this overview of dynamic risk modelling in Human Reliability

analysis that focused on Cognitive simulators is presented in Table 1-3. Each one of the

methods presented is classified according to some criteria:

- The Model for humanenvironment interaction

- Application complexity

- If it is Quantitative or Qualitative

- Cognitive model for the operator

- If it allow interaction between operators

- Field of Application

Furthermore table 1-4 provides a comparison among the various methods: for each of them

are emphasized the strong points, the weaknesses and the opportunities in relation to

possible application within The ConOps Concept. As far as ConOps is concern in fact it is

considered the capability of the cognitive simulator to be used as a supporting tool to carry

out Human Reliability Analysis for the use cases proposed within the ConOps framework.

Of the 9 cognitive simulators reviewed only 5 are capable of providing quantitative results

suitable for a risk assessment application.

However we can conclude that only the cognitive simulators able to provide quantitative

results are of interest for the possible applications of HRA related to CONOPS and among

the ones analyzed above apart from PROCOS, only four cognitive simulators are able to

provide quantitative results:

- CES, COSIMO, MIDAS, TOPAZ, IDAC

Of these five:

o CES and COSIMO do not have a model for the interaction between the operator and

the external environment


57/156

Table 1-3: Summary Table for the Cognitive Simulator analysed

Model forhuman-

environmentinteraction

Applicationcomplexity

Quantitative/Qualitative

Cognitive modelfor the operator

Interactionbetween

operators

Field ofApplication

PROCRU(1980)

Yes Medium-High Qualitative Sequential Yes Aviation

CES(1987)

No HighQualitative/Quantitative

Cyclic No Nuclear

COSIMO

(1992) No High

Qualitative/

Quantitative Cyclic No NuclearMIDAS(1993)

Yes Quantitative Sequential No Aviation

SYBORG(1995)

Yes Medium-High Qualitative Cyclic Yes Nuclear

TBNM(2002)

No Qualitative Sequential Yes Nuclear

AITRAM(2002)

Yes Medium Qualitative Sequential Yes Aviation

TOPAZ(2000)

Yes High Quantitative Cyclic No Aviation

IDAC(2004)

Yes High Quantitative Cyclic/Sequential Yes Nuclear

PROCOS(2004)

Yes* Medium/Low Quantitative Sequential** Yes*** Industrial

Notes:

- Yes* Procos do have a model for the interaction between operator and environment

however is quasi-static, which means that the behaviour of the external plant is not

simulated but it is taken into account using:

o a limited set of the states in which the equipment can be turned;

o an explicit relation between the actions outcomes (correct execution or Error

modes) and equipment status modifications (the relation has been derived

from the Hazop analysis).

- Sequential**The cognitive model is based on an information processing approach,


58/156

Table 1-4: SWOT Analysis of the simulation models in relation to applications within the ConOps concept

Strong points WeaknessesOpportunities within

ConOps

CES(1987)

- It can provide an objectivemeans of distinguishingwhich event scenarios arelikely to be straightforwardto diagnose and whichscenarios requiring longer

diagnosing and which canlead to human error.- It can implement bothquantitative andqualitative analysis;- It can be used to predicthuman errors byestimating the mismatchbetween cognitive

resources and demandsof the particular problem-solving task.

It presents a quite highcomplexity of applicationsince it requires a

simulation model for theplant the operator has tointeract with as well;It is not able to analyse theinteraction between two ormore operator;It cant analyse theerroneous action producedby communication.

CES has been developedfor simulating how peopleform intention to act innuclear power plant

during emergencycondition. It is impossibleto adjust the method tothe ATM domain becauseCES can not simulate theinteraction betweenoperators (it does notinclude thecommunication module).

PROCRU(1980)

- It comprises thecommunication model;- It permits the investigation

of questions concerningthe impact of proceduraland system designchanges on theperformance and safety ofcommercial aircraft;- It presents a model forhuman environmentinteraction.

It presents a highcomplexity of applicationsince it is a closed-loopsystem modelincorporating sub modelsfor the aircraft, the

approach and landing aidsprovided by ATC;It is focused on a cockpitcrew (flying pilot and pilotnot flying); it does notconsider the ATCO point ofview.It implements a qualitativeanalysis but there is not a

computational section: itcant be used to makenumerical estimate ofhuman error probabilities.

PROCRU is a simulationfor aircraft crew but itcould be difficult to adjust

it in order to simulate thecrew of air trafficcontrollers Its main focusshould be shifted from theaircraft crew to ATCO.Furthermore It doesntpermit a quantitativeanalysis.

- It can implement bothIt presents a highcomplexity of application

As the structure ofCOSIMO is actually built it


59/156


ConOpsstatistical result can beobtain;- It presents a model forhuman environmentinteraction.- It is modular, with theuser able to specify whichmodules are active.

Lack ofvalidation/verification ofmodels;It presents an extremelyslow speed of simulation.

to ConOps because itdoes not simulatecommunication processesbetween two or morepeople.

SYBORG(1995)

- It exhibits two interfaces,one for the interaction ofhuman with the machine(HMI) and one for thegroup interaction (Human,Human Interaction, HHI).Therefore it is well able todescribe also interaction

among members of theteam.

It implements onlyqualitative analysis, thereis no computationalsection, therefore it cantbe used to make numericalestimate of humanbehaviour;It presents a highcomplexity of application

since it requires asimulation model for theplant the operator has tointeract with;

SYBORG is tailored on aspecific application(nuclear power plant) andin order to be used itneeds the input comingfrom the plant simulator forwhich it has been build. Soit is very difficult to adjustthe method to other field ofapplication.

TBNM(2002)

- It is able to analyse theinteraction between two ormore operator; itcomprises a team modelwell structured.

- It is the first to try todetermine how theemotions personnel, willexperience when dealingwith difficult nuclear powerplant events, affectattention, thought, actionand utterances.

It implements onlyqualitative analysis, thereis no a computational

section, therefore it cantbe used to make numericalestimate of humanbehaviour;

It was developed to

nuclear application and itdoesnt know how it canbe applied on other fieldsof study.

AITRAM(2002)

- It exhibits a model for

human-environmentinteraction;- It integrates HumanFactors and Technicalcompetency. It is suitableto evaluate the

It implements onlyqualitative analysis, thereis no computationalsection therefore it cant

Despite the application

field of the method ishuman factorsmaintenance industry it isnot simple to adjustAITRAM to the specificneed of ConOps


60/156


ConOps- It can be used to identifyhazards, to combinehazard into riskframework, to evaluaterisk and to identifypotential mitigatingmeasure to reduce risk.

calibration of the simulator. The simulator can be usedfor analyzing scenarios;however a very high levelof details is required in thescenario description.Therefore it is not clearwhether this is compatiblewith the tactical level of

detail at which the taskanalysis is currentlydeveloped within ConOps.

IDAC(2004)

- It exhibits a model forhuman environmentinteraction;- It can implement both

quantitative andqualitative analysis;- It includes a crew model

of three types of operatorsand characterizes theinteraction in terms ofcommunication andcoordination.

It presents a very highcomplexity of applicationand in the analysis of theresults;The causal model used byIDAC are at a preliminary

stage and they are still notadequately supported ontheoretical or experimentalground;There is not an explicitrepresentation of theimpact of memory of thepast on future actions ofthe operator;

Although IDAC wasdeveloped to use on topredict the likely responsequantitatively of nuclearpower plant control roomoperating crew in accident

conditions, maybe it couldbe adjusted to the specificneed in ConOps. Howeverat the moment thesimulator works only ifcoupled with a code thatsimulates accidentscenarios and generatesinformation about the

external world in a nuclearpower plant.

PROCOS(2004)

- It presents a quasi-staticmodel for humanenvironment interaction;- It takes into account the

interaction betweenoperator through the useof part of the cognitiveflowchart especiallydedicated tocommunication

The behaviour of theexternal plant is notsimulated but is taken intoaccount using a limited setof the states in which theequipment can be turnedand an explicit relation

between the actionoutcomes and equipmentstatus modifications;Even for Procos theCognitive simulator hasbeen validated only

PROCOS is adaptable tomany field of study andthen it is not difficult toarrange to the specificneed of ConOps.

Its model is relativelysimple and easy to becommunicated to Expert ofthe field of analysis(namely ATC) even if theyh th ti l


61/156


62/156

Process model cover the full scope of ATM

- It will relate directly to the ConOps, Scenarios and Use Cases;

- It is possible to map and check the Logical Architecture against the process model;

- Operational Improvements and Enablers and the Performance view can also relateeasily to it;

- Economical aspects related with ATM Value Chain within the Aviation Industry canalso be related;

- Can be a candidate to be used as reference either for Validation or Safety

The overview of the ATM Process Model is presented in Figure 2-1.

Strategic

Phase

Air Traffic Control

Aircraft Operator

ATFCM

Airspace Management

Airport Operator

Pre-Tactical

Phase

Tactical

Phase

1

4

7

10

13 14 15

1211

89

65

2 3

Strategic

Phase

Air Traffic Control

Aircraft Operator

ATFCM

Airspace Management

Airport Operator

Pre-Tactical

Phase

Tactical

Phase

1

4

7

10

13 14 15

1211

89

65

2 3


63/156


64/156


65/156

- This Use Case for instance, describes how a Tower Runway Controller uses the

System to control the landing of an aircraft. It starts when the intermediary

approach phase is completed and the aircraft is ready for final approach and

ends when the Tower Runway Controller is ensured that the aircraft has vacated

the runway

o Actors

- Description of the main actors involved

o Preconditions

- Scenario inputs to the analysis

o Post conditions

- Possible success end states

- Possible failure end states

o Definitions

- List of the main term and abbreviations used

o Triggered

- Elements that triggered the use case events (i.e. : The Use Case starts when the

System detects that the aircraft is on final approach)

o Main Flow

- Main path, or nominal path that should be followed by the chain of events that

lead to a success end state

o Alternative Flow

- Possible deviations from the nominal path

Within the ConOps framework a model able to provide a quantitative reliability analysis of

use cases can provide useful inputs to the ATM Process model. Cognitive Simulators in

general and PROCOS in particular can constitute a useful tool for carrying out a Human

Reliability Analysis (HRA) as far as the use cases tasks are concerned, this in turn can

tit t i t d l f D i i ki d d f Wid D i Ri k


66/156

2.2 The use of the cognitive Simulator PROCOS and the HERA-Predictive

approach

Within EURCONTROL Human Reliability Analysis has been already carried out with some

in house and ad hoc methods. A more systematic approach is under development to make

better use of incident analysis data collected with the HERA retrospective tool. This

approach, called HERA-Predictive keeps the taxonomy and qualitative structure of HERA

retrospective and complements the data collected with a statistical approach, which allows

using the data in predictive safety assessments (Isaac, Van Damme & Strter 2004). The

approach is an adaptation of the CAHR approach developed in the nuclear domain to the

ATM environment (Strter 2000). Currently this approach is further developed under the

heading Virtual Advisor as the approach should support safety assessments as some kind

of virtual expert. The following outlines how the HERA-Predictive approach in principal works

based on the retrospective analysis of events.

Regarding the structure of the prospective and retrospective HERA approach, a research

project has been set up at EUROCONTROL that reviewed the theoretical and practical

literature to determine the best conceptual framework upon which to base an ATM incident

analysis tool. The conceptual framework chosen is that of human performance from an

information processing perspective (Shorrock, Kirwan 2002; Isaac et al., 2003). The

technique and the related taxonomy are model-based. A model in fact allows causes and

their inter-relations to be better understood. An error model provides an organizing principle

to guide learning from errors. Trends and Patterns tend to make more sense when seen

against the background of a model and more strategic approaches to error reduction may

arise, rather than short term error reduction initiatives following each single error event.

(Shorrock et al 2003).

The main purpose of the HERA (retrospective and prospective) classification of human error


67/156

of human error data to detect trends over time and differences in recorded error types

between different systems and

euro control drm final

Documents