Critique of Current Human Reliability Analysis Methods

PSA GROUP I I1 111 IV V

A.J. Spurgin, Senior Member, IEEE, and B.O.Y. Lydell

HRA METHODOLOGY ASEP (NUREGICR4772) SLIM andNUREG/CR-1278 FLIM and NUREGICR-1278 Decision tree methods and NUREG/CR-1278 Time-reliability correlations and NUREGICR-

Abstract - This paper is a critical review of human reliability analysis methods & techniques as applied in current probabilistic safety assessments of nuclear power plants. Practitioners continue to rely on concepts, data and methodologies that have seen relatively little change since the early 1970s, however. The generic human reliability data developed for the seminal 1975 Reactor Safety Study is still being used. Using the review insights, the paper develops recommendations for using plant- specific information from operator training and procedure validation in the human reliability analysis applications.

Index Terms - Basic Characteristics, Contemporary Human Reliability Analysis (HRA), Nuclear Power Plants, Performance Shaping Factors, Plant Procedures, Probabilistic Safety Assessment (PSA), Training Insights.

I. CURRENT HRA STUDIES

PLANT-specific probabilistic safety assessment (PSA) studies exist for all operating U.S. nuclear power plants. Human reliability analysis (HRA) forms an important element of these PSA studies. Following completion of the regulatory mandated Individual Plant Examination (IPE) program during the 1990's, the PSAs that evolved from this program have been subjected to updates and integration with living PSA programs and PSA quality management programs. Listed below are some typical HRA features of current U.S. plant-specific PSA models:

Between 50 to 150 unique operator actions are included in a PSA model. These actions include pre-accident (test, maintenance and calibration) and post- accident actions, and recovery actions.

Each PSA is updated every two to three years or as warranted by plant modifications. These updates recognize changes made to plant procedures (e.g., emergency operating procedures, EOPs) and control room design.

A. J. Spurgin, Private Consultant, San Diego, CA 92103 USA

B. 0. Y. Lydell is with ERIN@' Engineering and Research, Inc. (e-mail: a-jspurgin@cox.net).

Carlsbad, CA 92009-141 9 USA (e-mail: bolydell@erineng.com).

Through the reactor owners groups (i.e., BWROG, BWOG, CEOG and WOG), the PSAs (with their HRAs) benefit from peer review. The peer review guidelines address the quality of HRA.

Human error contributions to the assessed core damage frequency tend to be significant with risk-important operator actions having risk achievement worth (RAW) values in the range of 2 to 50 (or more).

Many U.S. utility organizations have implemented PC-based HRA tools to support living PSA applications. As an example, all HRA calculations are performed and documented using spreadsheet programs to facilitate updates and sensitivity evaluations. Also, these new tools enable effective peer reviews since the calculations are more easily traceable.

Increased emphasis on operator interviews to validate assumptions about task complexity, effectiveness of procedural guidance, influence by time-constraints, relevance of training, etc.

While progress has been made to better integrate HRA with other PSA tasks, there remain questions about the capability of these HRA studies to provide plant-specific estimates of human error probability WEP) that reflect underlying causes of potential human performance problems. These questions arise as a result of the known limitations in human error modeling by the HRA methods and techniques in current use. Table I is a summary of methods and techniques in current U.S. plant-specific PSA studies.

Table I. Basic Characteristics of HFU Methods & Techniques in

I 1278

IEEE I" Human Factors Meeting Scottsdale Arizona 2002 0-7803-7450-9/$17.00Q2002 TEEE

3-12

As implied by Table I, when characterized on the basis of HRA methodology, the PSA studies fall into five basic groups. Noteworthy is the fact these studies continue to rely on generic data and model concepts developed during 1970 to 1990. Expert judgment is a key ingredient of the HRA methodologies that are used to analyze pre- and post-accident operator actions. Often, the expert judgment is limited to a single analyst’s understanding of human performance in the context of different plant conditions, however. A contention of this paper is that there remains a significant gap between the academic research and the needs of the practical HRA.

II. THE STATE-OF-HRA-PRACTICE

HRA in the context of PSA of nuclear power plants has been the subject of numerous critical reviews, e.g., [1]-[4]. Each plant-specific HRA represents a highly iterative analysis process. Together, system analysts and human reliability analysts define the type operator actions to be incorporated in a PSA model and how to best represent them (e.g., at the event tree or fault tree level). As a model matures, the analytical focus shifts to the risk-important actions and further decomposition of certain actions may be warranted to ensure realism. The level of effort afforded a HRA task tends to be between 5% and 10% of the overall project resources that are allocated PSA model development. Therefore, selected HRA methods and techniques must be chosen in such a way that all operator actions be evaluated in a consistent and logical manner. Furthermore, there needs to be sufficient analytical resolution so that the plant- specific aspects of procedure design and content and training practice are reflected in representation and quantification of human error probabilities (HEPs).

A typical HRA convention is to represent post- accident actions by cognitioddiagnosis and execution. The former captures the potential challenges in correctly interpreting control room indications and alarms. The latter addresses the task complexity. The pioneering work by Swain and Guttman [5] continues to have a major influence on how operator actions are evaluated, both qualitatively and quantitatively. It needs to be recognized that this pioneering work evolved from human factors research performed during the 1960’s and 1970’s. As a result, it is not a trivial task to adapt these early human reliability concepts and data to today’s HRA requirements. The pre-publication peer review efforts of the ‘Swain-Guttman’ work resulted in consideration of a time-reliability correlation (TRC) as an enhancement to the decompositional or task-analysis approach to HRA.

Much of the R&D in the 1980’s focused on different TRC concepts. The basis for the TRCs was the extensive work undertaken by General Physics and Oak

TEEE 7” Human Factors Meeting Scottsdale Arizona 2002

3-13

Ridge National Laboratory on early plant simulators [6],[7]. The simulator was used to collect crew response data for both PWRs and BWRs. At this time, the emergency operating procedures were event based and the range of scenarios addressed were quite limited. The objective of the studies was to support assessments of the appropriate balance between manual and automatic actions in safety situations.

In a parallel, expert judgment methods were evaluated [SI. A method called the Success Likelihood Method (SLIM) was developed under sponsorship of Brookhaven National Laboratory and the U.S. Nuclear Regulatory Commission [9]. Following a critical review of the SLIM method [IO], a modified version called the Failure Likelihood Method (FLIM) was developed. The SLIM/FLIM methods are based on the assumption that the HEP in a given situation depends on the combined effects of a relatively small set of performance-shaping factors (PSF) that influence the operators’ ability to perform an action. Using expert judgment, the PSFs are ranked and converted into success rate (SLIM) or HEP (FLIM) .

One of the TRCs developed under contract to the Electric Power Research Institute (EPRI) was the Human Cognitive Reliability (HCR) correlation [ 1 I]. EPRI then engaged in an extensive research program to try to validate HCR by the use of simulators. The Operator Reliability Experiments (ORE) project was the result of this research project in which multiple scenarios were run at six utility simulators [12]. The development of both the Holistic Decision Tree (HDT) [ 131 and indirectly to the Caused Based Decision Tree (CBDT) [14] methods was the result of a change in attitude stemming from the analysis of the data from ORE simulator sessions.

Subsequently, there has been a philosophical change in thinking about human errors and their causality. The current idea is that the totality of the job and the context as it relates to the operators force the errors. This concept is developed in the work of Hollnagel [ 151. The NRC sponsored work ATHEANA [4] reflects this concept in the idea of error forcing hnctions. The HDT method also reflects this change in attitude to error causality. There is also a growing belief that the best way to replicate the reliability of the station personnel in dealing with accidents is to use the power plant training simulator so that HRA reflects an understanding of the context under which the personnel perform the tasks.

III. REQUIREMENTS FOR AN HRA METHOD

HRA provides predictive estimates of averaged human performance and this implies some limitation in the representation of human activities in the PSA context. Equally, the objective of the PSA ought to be

reflected in the choice of the HRA approach. So if the PSA were to be used by regulators to assess the general risk of the industry, one would use a generic method representative of a nominal power plant. However, if one wants to represent a specific plant, then one would try to represent that plant and its personnel as well as one can, commensurate with project budget and schedule. The HRA is an essential part of PSA and, therefore, the accuracy or validity of the HRA may go a long way to deciding the utility of the PSA for various applications. The US utilities have engaged in a round of independent reviews of their PSA models to establish the quality and usefulness for risk-informed applications. The quality of the HRA is one of the requirements to setting the usage of the PSA. Quality is reflected in both what method is chosen and how it is applied. For achievement of a ‘good’ quality rating, one is expected to tie the method to the use of a non-generic method supported by the use of plant personnel (experts in their fields) and feedback from plant experience and the use of the simulator.

[I11 SLIM [9], FLIM [ 101 HEART [ 171

Cause Based Decision Tree (CBDT) [14]

Holistic Decision Tree ~ 3 1

I K HRA METHODS

PSF Contributions are evaluated by plant personnel and used to HEPs via anchor points Data driven approach related to generic tasks and a set of analyst selected weighting hnctions related to context or error producing conditions (EPCs). .Decision tree format addressing causal factors of human error (e.g., errors associated information-operator interface or operator- procedure interface). Decision tree format based on contextual importance ranged within anchor values. Method uses simulator data [I31 and expert

HRA methods used in current U.S. PSA studies are listed in Table 11. There is considerable overlap between the different methods. That is, most methods that have been proposed since the publication of NUREG/CR-1278 [5] share some common, basic concept(s). For example, the basic concept may be a

ATHEANA [41

Table 2. HRA Methods & Techniques Supporting Current U.S. PSA

judgment Structured technique for examination of EPCs. Qualification is based on HEART

Studies THEW with

ASEP [ 161 Human Cognitive Reliability (HCR) model

TRC [51, , Modified task analysis with PSF contribution.

HFiPs are based on expert judgment

TRC developed from considerations of simulator data and the concepts of skill-, rule- and knowledge-based behavior

TRC, but the estimates for the parameters of the TRC are derived differently (e.g., expert judgment versus operator response time measurements).

V. CRITIQUE OF HRA METHODS

The objective of this paper is to examine the basic assumptions made in the development of the methods, critique those assumptions and then draw conclusions about the method and its potential use. Most HRA methods have been developed for specific reasons and requirements. These may not satisfy the user’s needs or requirements, however. There is some commonality in all proposed methods. We will discuss some of the common aspects and then critique the specific method. For example, there is a whole range of TRCs based on the same general concept and use, but with differences.

A. THEW Currently, there is no universally approved or

accepted HRA method. Historically the development of HRA has gone from generic models to plant specific models as HRA experts have tried to capture the essence of how human affect given plants. The steps in the development process have gone from the assembly of individual human reliability values, such the reliability of a person reading a gage, to the reliability of a crew responding to an accident judged as an entity. The estimate of HEP in the first case was by adding up the contributions from a set of sub-tasks. THEW is such a model and is based upon performing a task analysis of the set of actions to be taken. Swain then uses task analysis to develop a human reliability tree in which the elements of the task analysis are incorporated, including potential recoveries. One problem is selecting the key sub-tasks that affect the overall reliability, and another is that the most significant element was missing, i.e. the cognitive element. Situational factors affecting the overall task were used to adjust the basic HEP to make it more plant specific.

In the later HRA methods estimation of the HEP is based on the context of the accident as it affects the crew. In other words, the context determines the HEP value, not the task or series of tasks. The context forms the intention in the minds of the crew and leads to the possibility of errors. Errors in intent are difficult to correct as stated in the literature and have been shown in simulated accidents [ 181, whereas manual execution tasks are much easier to recover from. Even feedback incorporated into the EOPs is not too successful in increasing recoveries from errors of intent and this includes interpretational difficulties with EOPs. The PSFs, the task and the accident situation have to be integrated to produce the correct HEP. For complex, dynamic actions the simple decompositional models of

IEEE 7Ih Human Factors Meeting Scottsdale Arizona 2002

3-14

human reliability fail to capture the underlying causes of human error.

B. Time Reliability Curves In an attempt to defuse the above opinion, Swain

and Guttman introduced their version of the time reliability curve. The concept of the TRC was originally a General Atomics concept [19] based on expected manual control responses to disturbances. The original model was a success model derived from hybrid simulator studies (man in the loop), then converted to a failure model. Subsequent to the IEEE meeting HRA TRC models were derived from simulator data [6],[7]. It was suggested that the data could be explained based on Rasmussen’s skill-, rule-, and knowledge-based behavior [20]. The TRC proved a popular concept and EPRI developed the Human Cognitive Reliability (HCR) model, also based on Rasmussen, but differently organized. HCR was developed to try to explain how tasks were essentially the same, but stretched in time and the important feature was the cognitive characteristics of the crew relative to the task.

EPRI funded a project [12] to attempt validation of the HCR concept. The answer was that it did not validate the basic hypothesis. However, insights from this and other simulator studies were useful in understanding how crews performed. The data also showed that crews could fail and the number of crews failing could be high [21] depending on the type of procedures, training, etc. In other words, the context under which the crews operated affected their failure rate. The Swain type human reliability model did not seem to be appropriate, in other words the dominant effect was the context not the task. Also, a closer consideration of the TRC indicated that it was generated by the random variation of the crews, which led to the characteristic shape caught by the HCR together with either actual failures or delayed responses. Examination of TRC data reveals that the order of crews in responding can be quite different from one human action to another. Further supporting the fact that the basic shape of the TRC is really due to the random response of crews to the initiator.

TRCs have been used to predict HEP based on extrapolating the curve to lower probability levels. The HEP used in a PSA would be given by the intersection of the extrapolated curve and the time at which core or plant damage would occur. In some cases these calculated values test the creditability of the approach, so much so that the users prefer to assume a much lower value, such as 1.OE-4.

So the simulator results show that TRCs can have different ‘tail’ contributors to the curves for various different initiators. Also, for different plants of similar design the crews can have different responses. This does not confirm the concept of task dominant

responses. The PSFs are not sufficiently different to account for the different ‘tails’. Now Swain’s TRC has a median curve together with a long ‘tail’. This particular TRC fails to account for high reliability in the case of Anticipated Transient Without Scram (ATWS) responses and also the variability between plants of similar design. Time is just one of the variables of interest not the defining variable.

C. Performance Shaping Factors Many HRA methods assume that Performance

Shaping Factors (PSFs) are independent. PSFs are not models but they are a vital part of a number of models. It has been shown by Montgomery [22] that crew skills, related to crew performance and context are dependent.

Montgomery classifies the crew skills into two semi-independent classes of dependent crew skills. This being the case, it is highly unlikely that the normal linear, independent set of PSFs really represent human actions. Doubly difficult is for a group of operators to correctly estimate the separate effects of PSFs on HEPs. This casts doubt on the use of PSFs in any formulation, especially one relying on so-called domain experts.

D. SLIWFLIM Methodology SLIM [9] and FLIM [lo] are based on the

combination of using anchor values and estimation of the HEP for a particular human action on the concept of the PSF, the value of the PSF and the weighting of the PSFs as determined by a group of relatively untrained observers. These methods therefore are very problematical. It is important that biases, introduced into the process by the manner of elucidation, be avoided. The results of elucidation should be examined most carefully before using the results to estimate the final HEPs. It has been found in some cases this elucidation process has not been carried out carefully.

E. Expert Judgment Commonly used in HRA studies, structured or

unstructured expert judgment forms a vital part of all methods and techniques. Questions relative the applicability of expert judgment include:

1. Who is the expert? 2. How is the elucidation process carried out? 3. How do you get a reproducible result? 4. How do you validate the result?

All are very difficult questions. The answers could be:

1. Use the best observers of the operators. This seems to preclude the operators tllemselves; maybe the instructors are the best.

2. The development of the scenario is difficult since one does not wish to give the game away or lead the experts. Perhaps the thing

IEEE 7’ Human Factors Meeting Scottsdale Arizona 2002

3-15

to do is to treat the experts more like the simulator and define the scenario by the failures that occur.

3. As Comer 11111 said use more than six experts and interview them independently.

It still does not really satisfy either question 3. or 4. This is the difficulty of expert judgment ad HRA, since one is often forced to use expert opinion.

F. HEART Approach The HEART method is a concept similar to the task

analysis approach in NUREG/CR-1278 [5]. Its uniqueness lies in the assessment of the effect of error- producing conditions (EPCs). HEART uses different EPC elements to modify a basic HEP. A series of generic tasks define the nominal (or basic) HEP together with a range. For example, task ‘D’ (defined as a fairly simple task performed rapidly or given scant attention) gives a basic HEP of 0.09 (0.06 to 0.13). The approach requires interpretation.

Operator response to ATWS is frequently trained and assuming that there is no failure of trip signal, the indications would be clear, hence partly fitting the description. In practice, the ATWS action is more likely to be 0.003 (.01 to .OOl). The EPCs are equally difficult to substantiate, for example ‘a need for absolute judgments which are beyond the capabilities or experience of an operator, (x 1.6)’. So if the crew does not have the experience with the ATWS would be 1 . 6 ~ BHEP.

However, it is difficult to predict human actions based on a nominal set of discrete task descriptors. It is the context of a situation that determines the error rate. Another issue that comes up from time to time is the limitation of the HRA to deal with certain specific situations related to individuals. HEART deals with individual situations, so some of the elements in the EPCs are inappropriate for an HRA. It looks as though many of the considerations in HEART come from non- nuclear accidendincident reports and not all of these can be reflected directly into the HRA. This point can be made with respect to other methods as well.

G. Cause-Based Decision Tree (CBDT) method The CBDT method [8] is a recast of Swain’s tables

in a decision tree formulation. The author of the method was experienced with Swain’s THERP and had been involved with both the Oak Ridge [6],[7] and ORE [12]simulator experiments, but it was trying to use the SPOA experiments to confirm the data in some of the tables in reference [5] that lead to the formulation. The use of the-decision tree was suggested during the attempts to more directly relate the ORE data to HEP estimation. In the EPRI report there are a number of inaccuracies in modeling including the mixing of

IEEE 7” Human Factors Meeting Scottsdale Arizona 2002

3-16

reliability numbers with shaping factors. Also the idea of summing the HEPs from different influences, which follows THERP, has not been substantiated. Again the independence of these effects is not indicated by experience in simulation studies or by Montgomery [22]. The CBDT is an interesting modification of THERP, but does appear to be in need of a change in the formulation of effects.

H. Holistic Decision Tree (HDVMethod The HDT method is a direct result of trying to use

simulator data to the HRA. It is embedded in the idea of context defining the error probabilities. Expert judgment is used to define the selection and ranking of the Influence Factors (IFS), equivalent to Hollnagel’s CPCs (as described below). The range of the HEPs is fixed at 1 .O to 1 .OE-3 as anchor values.

The trees are constructed like an event tree with multiple paths with end-states representing the combination of IF levels representing the assessment of the contribution of each IF. Each IF is treated as a heading of the tree and the range of values is represented by a limited number of branches.

The importance of one IF versus another IF is taken care of by the rule that the importance of a contributor is higher on left hand side and lowest on the right hand side of the tree. It is usual to use either two or three levels representing the possible variation in an IF.

Due to the extensive simulator studies at Paks NPP [21], it was possible to refine the HDT method so that it was possible to use both simulator data, at the high failure rate end and expert judgment derived fiom a better understanding of the ways the crews responded. The Paks experience showed how it was possible to closely connect the simulator data, observer data and the HDT method. The observer records were very much like those used in ORE [ 121 and by Montgomery [22], but were better organized. In fact, later Paks used barcode readers to collect the observer data.

The HDT method is in need of further development. The emphasis is on overview and not details. An improvement would be to replace ternary and binary branches with a smoother transition of a distribution. The IF-contributions are assessed by expert opinion or simulator observations. This is a complex task, however. The contribution of the IFS can be non-linear, but this just increases the difficulty of determining the relative contributions.

A check of the modeling of the trees has been performed by using two groups of experts, one to define end-state HEPs from the pathways and the other independently define the weight and importance of the headings and branches. The basic concept is based on the importance of the cognitive actions and the fact that errors of intention are difficult to recover, whereas manual actions, like switch operations, are mostly

easily recovered. But still some manual operations need to be considereh in the context of the overall modeling of human activities. I. ATHEANA

ATHEANA [4] is not an HRA method in the opinion of the authors. It does, however, help the HRA analyst to more clearly understand the kinds of situations that can lead to errors, i.e. the concept of error forcing contexts (EFCs). The concepts related to EFCs could be very useful for persons developing observation forms for simulator data collection sessions. The taxonomy should be part of the background to the design of scenarios. Of course, the responses of the operators may not show significant errors due to excellence in training and the other factors influencing the performance of the operators. The requirement of an HRA method is to take these ideas and use them to predict the systematic failures of the operating personnel of a nuclear power plant. The method seems to have grown out of accident analysis rather than HRA. The method of itself does not generate HEPs, but rests on HEART quantification values.

J. Other Methods ‘Connectionism Assessment of Human Reliability’

(CAHR) is not a HRA method, but it is worth mentioning since its basis is the use of actual data to generate HEP values [23]. There are a number of similar approaches under consideration at the moment. A database on industrial accidents is associated with the CAHR computer program. Before deriving a HEP for a specific task, each element in the database must be examined for applicability.

The Cognitive Reliability and Error Analysis Method (CREAM) is a method that is based on the use of context to formulate HEPs [15]. There are two versions of CREAM, one is the basic method, for use as a screening approach, and the other for a more developed HRA or Extended Method. The basic method has been used for screening application on the International Space Station PRA to provide ranges of HEPs.

Hollnagel use the term ‘Common Performance Conditions’, which is analogous to Performance Shaping Factors except that they are the basis of shaping the HEPs not just a modifier. Nine CPCs are defined, including Adequacy of Organization, Working Conditions, etc. Further, he defines ranges in the quality of these CPCs, there may be three or four ranges. For example, in the case of ‘Adequacy of Organization’ there are four levels; Very efficient, Efficient, Inefficient, and Deficient. Hollnagel recognizes the possibility of dependence between the CPCs. He then uses a computer code to evaluate the impact of all of the CPC values to predict a HEP range. These have

been classified as ‘Strategic’, ‘Tactical’, Opportunistic’, and ‘Scrambled’. The ranges are overlapping. The approach is useful to gain some appreciation of the interplay between the CPCs and the resulting ranges. The ranges have to be interpreted for screening purposes.

The extended method builds upon the concept of the cognitive task analysis and calls for a level of understanding and effort usually beyond that of HRA in PSA. It takes one into a much deeper attempt to understand how the cognitive processing of the crews progresses during the execution of a response to an accident. More work needs to be done with this part of CREAM to enable it to become a viable HRA method. Even in the case of access to training simulator it is not a straightforward task to characterize elements of operator cognitive processing. Post scenario debriefing may help, but it is difficult to probe the processing of normal control room crews after the fact.

VI. CONCL USIONS

There appears to be a shift from the traditional HRA model concepts to new, contextual concepts. The change is away from the decompositional to a more holistic approach. In part this development is performed using power plant simulators to more clearly understand the effect and importance of the various aspects of the context as it affects the reliability of operators. The same concepts can be applied to other plant personnel (e.g., ex-control room operators). The TRC idea is quite useful in the training process to see how crews respond, but time is not the only determinant of operator reliability, context is and time is only one component of that. One can appreciate this when one examines control room responses. Crew responses can be affected by the man-machine interface, the organization, training, the characteristics of the accident, and so on.

The event analysis approach as delineated in the ATHEANA provides one source of relevant information about how crews and organizations both cause and respond to accidents. Control room crews are exposed to a variety of situations during each simulator training session. Data from the simulators offer a vast pool of information that we are not tapping. In part this is so because of a lack of user-friendly analytical tools. The correct use of data can enhance our ability to improve training and at the same time enhance our understanding of crew responses to accidents and incorporate this data and understanding into the assessment of risk and improvement of plant availability. To take this step requires a different attitude to the use of the simulator, better training on the part of the instructors to improve their observer skills and a more deliberate purpose in the design of

IEEE 7’ Human Factors Meeting Scottsdale Arizona 2002

3-17

scenarios to help the crews more closely identify the simulator with the actual plant.

WI. REFERENCES

[I] D.I. Gerhnan and H.S. Blackman, Human Reliability & Safety Analysis Data Handbook, New York: Wiley-Interscience,

[2] V. Joksimovich and D.D. Orvis, “Experiences gained in the application of human reliability analysis in various U.S. PSA studies applicable to the evaluation of the PSAs of Swiss nuclear power plants,” APG Report No. 32, Accident Prevention Group, San Diego (CA), 1995.

[3]. S. Hirschberg and V.N. Dang (Eds), “Critical operator actions - human reliability modeling and data issues,” NEA/CSNVR(98)1, OECD Nuclear Energy Agency, Issy-les- Moulineaux (France), 1998.

[4] M. Bamere et al, “Technical basis and implementation guidelines for a technique for human event analysis (ATHEANA), NUREG-1624, Rev. 1, U.S. Nuclear Regulatory Commission, Washington (DC), May 2000.

[5] A.D. Swain and H.E. Guttman. “Handbook of human reliability analysis with emphasis on nuclear power plant applications,” NUREG-CW1278, US Nuclear Regulatory Commission, Washington (DC), 1983.

[6] T.F. Bott, E.J. Kozinsky, C. Crowe and P.M. Haas, “Criteria for safety-related operator actions: Initial PWR simulator exercises,” NUREG/CR-I908, US. Nuclear Regulatory Commission, Washington (DC), 1981.

[7] A.N. Beare et al, “Criteria for safety-related NPP operator actions: Initial BWR simulator exercises,” NUREGICR-2535, U.S. Nuclear Regulatory Commission, Washington @C), 1982.

[8] M.K. Comer et al., “General human reliability estimates using expert judgment,” NUREGICR-I 984, U.S. Nuclear Regulatory Commission, Washington (DC), 1984.

[9] D.E. Embrey et al., “SLIM-MAW: An approach to assessing human error probabilities using structured expert judgment,” NUREGICR-3518, U.S. Nuclear Regulatory Commission, Washington (DC), 1984.

[IO] S.H. Chien et al, “Quantification of human error rates using a SLIM-based approach,” in Proc. 1988 IEEE 41h Conf. Human Factors and Power Plants, pp 297-302.

[I 11 G.W. Hannaman, A.J. Spurgin and Y. Lukic, “Human cognitive reliability model for PRA analysis,” NUS-4531, Draft EPRI Report, Electric Power Research Institute, 1984.

[I21 A.J. Spurgin et al, “Operator reliability experiments using nuclear power plant simulators: Methods and results,’’ NP- 6937, Electric Power Research Institute, 1990.

[13] A.J. Spurgin, “Experience with the decision tree methof for several applications,” presented at the 5’ Int. Conf. Probabilistic Safety Assessment & Management, Osaka, Japan, 2000.

[I41 G.W. Parry et al, “An approach to the analysis of operator actions in probabilistic risk assessment,” TR-100259, Electric Power Research Institute, Palo Alto (CA), 1992.

[ 151 E. Hollnagel, “CREAM: Cognitive Reliability and Error Analysis Method,” Elsevier, Barking, Essex, United Kingdom, 1998.

[ 161 A.D. Swain, “Accident Sequence evaluation program human reliability analysis procedure,” NUREGICR-4772, U.S. Nuclear Regulatory Commission (DC), 1987.

[I71 J.C. Williams, “A data-based method for assessing and reducing human error to improve operational performance,” in Proc.

1994, pp 44-76.

1988 IEEE 4th Conf. Human Factors and Power Plants, pp 436-453.

[I81 A.J. Spurgin et al, “The BWR emergency procedures tracking system (EOPTS): Evaluation by control room operating crews,” “-6846, Electric Power Research Institute, Palo Alto (CA), 1990.

[I93 K.N. Fleming et al, “HTGR accident initiation and progression analysis (AIPA) status report,” GA/A13617, Gnereal Atomic Company, San Diego (CA), 1975.

[20] 3. Rasmussen, “Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models,” IEEE Trans. Systems, Man, and Cybernetics, vol

[21] A. Bareith, E. Hollo and Z. Karsa, “HRA and human factors evaluations in support of safety assessment and improvement at the Paks NPP,” presented at 4” Int. Tnf. Exchange Forum on Safety Analysis for NPPs of VVER and RBMK Types, Obninsk (Russia), 1999.

[22] J.C. Montgomery et al., “Team skills evaluation criteria for nuclear power plant control room crews,” Draft report available from the Public Document Room, U.S. Nuclear Regulatory Commission, Washington (DC), 1992.

[23] 0. Strater, “Evaluation of human reliability on the basis of operational experience,” GRS-170, Gesellschaft f i r Anlagen- und Reaktorsicherheit, Cologne (Germany), 2000.

SMV-13, pp 257-266.

VIII. BIOGRAPHIES

A.J. Spurgin, Senior Member, IEEE, is a Private Consultant specializing in human reliability analysis. He was the principal investigator of the 1987-1 990 Operator Reliability Experiments Project by the Electric Power Research Institute.

B.O.Y. Lydell has 27 years of specialized risk & reliability analysis experience. Since 1983 he has performed numerous domestic and international HRA tasks. He was the HRA task leader for Perry and Surry Individual Plant Examinations (1989 - 1992). More recently he performed the Browns Ferry, Byron & Braidwood, and Watts Bar HRA Updates.

TEEE 71h Human Factors Meeting Scottsdale Arizona 2002

3-18