Probabilistic Safety Assessment International Topical Meeting Clearwater Beuch, Florida January 26-29, 7 993

PROCEEDINGS Vol. 1

DISCLAIMER

This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employm, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or use- fulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any spe- cific commercial product, process, or service by trade name, trademark, manufac- turer, or otherwise does not necessarily constitute or imply its endorsement, recom- mendhtion, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.

ABSTRACT

USE OF BEHAVIORALLY ANCHORED RATING SCALES (BARS) FOR DEEP TECHNICAL KNOWLEDGE

David Okrent', Edward C. Abbott", John 0. Leonard, JrP, Yongjie Xiong'

The UnbenQ of Cafifomia a! Lo$ Angeles (UCLA) participated in a U.S. Nuclear Regulatory Commission research program to investigate methods to measure the effect of management and organization on nuclear plant safety. The UCCA research team focused Its efforts on understanding 'deep technical knowledge,' and its relation to probabilistic risk assessment. As a result, the research team combined deep technical knowledge with I commonly used rating system for understanding the effectiveness of management and organirations.

INTRODUCTION

The U.S. Nuclear Regulatory Commission has been supporting a mufti-institution research program on the influence of management and Organization of reactor safety for several years. The various groups involved have usually approached the issues involved from different points of view.'b Some have looked more deeply at the sociological and psychological aspects, while others have introduced engineering emphasis early on. With time, the efforts have begun to interact more and more strongly, and this paper reflects one example of such interaction.

'Unkersity of California, Lot Angeles; 5532 Bolter Hall; lor Angeles, CA 90024-1597

'A82, incorporated; 14595 Avion Parkway, Suite 900; Chantilly, VA 22021

tong Wand Lighting Company; 131 Hoffman Lane; Central Islip, NY 11722

.

LICENSEE EVENT REPORT REVIEW

The importance of deep technicat knowledge a Illustrated in Table 1. A mvkw was performed or summaries ot licensee went reports (ERs) lor February and March 1992. Each went was reviewed with emphasis on .Vent causes which were rotated to a tack of technical knowledge. that is, if the proper technical knowledge of the subject were known, would the event have occuned. This review is preliminary and the hdividual events cited had little impact in degrading plant safety. However, the number of events that can be construed to involve lack of deep technical knowledge b higher than expected. Additional research in this area is wananted.

KNOWLEDGE Dt STRlBUTlON

in addition to a review of the LERt , the UCU research team also developed mechanisms lor judging the adequacy of the knowledge of plan1 staffs in areas related to safe operation. The approach was to define areas and fevels of expeflao that might be used to quantify the effect of deep technical knowledge on the probabilistic risk 8stessment (PRA). A list of staff positions for 8 typical nuclear power plant and the corresponding knowledge areas was developed. For each posilion,

\ i

I I

234

23 I

i

the level of knowledge was determined based on a threepoint scale. The threopoint scale is:

1. Passing Knowledge. Familiar with the knowledge area in a very general way. Would have difficulty in providing a detailed expranation of a problem in the area without considerable additional trairilng or study. Some understanding of the knowledge area's safety signfficance.

2. Working Knowledge. Capable of functioning adequately in the subject area and understanding the knowledge area's rafety signilicance. Capable of participating in solving safety problems in the area, but not making final decisions on coirective actions.

3. Detailed Knowledge. Capable of performing in-depth assessments or oval uations of safety problems in the knowledge area. Has an In-depth undentandhg of the safety significance of problems in the area. Capable of solving problems in the area with little or no assistance and developink corrective actions.

the results of this effort are shown in Figure 1. The figure illustrates the depth of knowledge associated with each plant position based' on a limited consensus of the research team and b presented as an example of the approach. Such information could be used as input to a model of the plant's 'software processes.'

These software processes are those, used to manage and implement work, such as work requests, procedure changes, safety reviows, etc. The combination of the knowledge matrbt and the process model could be used to quantify the effect of management on the PR4. Additional development of this approach cciuld prove to be helpful in assessing the managenent effectiveness of plant staffs.

EIEHAW0RAU.Y ANCHORED RATING SCALES

A s i m h r approach was usrsd in developing Behaviorally Anchored Rating Scales [BARS). BARS have been identified as a polentially valuable instrument In the measurement of various attributes.' Using the information developed above, BARS were developed for several pknt positions. The method

of BARS represents a technique to reduce the problems of halo and leniency, among othom, in judgments about performance. In one commonly used form, the procedure k divided into a sequence of live steps, as follows: (8) identification and definition d performance dimension; (b) generation of behavior examples; (c) rllocation of behavior examples; (d) scaling d behavior ewmples; and (e) sufe anchor selection.

Among other things, the UCLA research team has called out deep technical knowledge 8s 8 particularly bnportant facet of management, with respect to safety. The UCU research team has developed a preliminary categofiation of deep technical knowledge brto six major topics; namely, risk rrsetsment, details of the pknt, tfanrient behavior, severe accident management, physical sciences and safety basis. These principle utegories have been subdivided, in turn, usually into three subcategories.'" These ategoriet are similar to the knowledge areas in Figure 1.

tt & not completely stralghtlorward to cast the dimensions of deep technical knowledge into behavior examples as b done in the typical BARS technique. Nevertheless, the UCLA research team has tried to carry the interactive mffort into 8 next logical step. The subcategories of deep technical knowledge have become the performance dimensions. For oach of several dimensions, a generic set of performance measures has been prepared, each representing 8 dinering combination of aspedr of deep technical knowtedge applicable to the particular performance dimension. For the dimension 'reactor physics: which is a subcategory of transients, ten performance measures, which take the place of the behavior examples usually formulated in a BARS application, were developed to make 8 list from which selections have been made to provide preliminary live-point BARS for ten different positions at the plmnt. The BAR scale expanded the threepoint wale used in Figure 1.

For reactor physics, the drall ten pertomante measures are the following:

1. Good working knowledge (quantitative) of steady state and transient neuttonict, including all relevant react'kity contributors: deep familiarity with all plant-specific reactivity control features, reactivity accident potential and recritiulii considerations under severe

2.

8ccident conditions; quantitative grasp of the interaction between thermal-hydraulic and neutronic phenomena.

Phenomenological understanding (semi-quantitative) of all important reacthriky reiated effects in steady state, startup, shutdown and accident conditions, including severe 8ccident recriticality; details of plant specific reactivity contrd features, including indirect reactivi control effects; capable of understanding the interaction between neutronic and thennabhydraulic phenomena.

3. Capable of recognizing abnormal reactivity conditions, performing an estimated critical position, estimating the magnitude of changes in power associated with anticipated transients, (e.g., dropped rods, loss of feedwater, etc.).

4. Continuing familiarity with major relevant reactor physics concepts (e.g., multiplication, bumup, fission product poisons, readivity feedbacks): familiarity with reactor physics rote in safety for specific plan!.

5. Capable of visualizing the plant response to changes in reactivity due to plant activiies (e.& startup, shutdown); anticipate abnormal reactor states (e.g., high flux tilt, inoperable control rods, etc.); understand thermal-hydraulic effects on power (e.g., cool-water accident, loss of feedwater heating).

6. Some familiarity with concepts of criticality, shutdown, reactiviity feedback, reactivity transients, influence of system failure on ability to shutdown; understand safety function of critical components in systems important to reactivity control.

7. Familiar with the concept of reactiVQ control (rods, boron, etc.); understands the important systems and components in controlling reactivity during normal plant operation and accident conditions.

8. Some familiarity with the concept of fission, reactor control, .and systems for controlling the fission process; understands ' the importance of maintenance on reactivity

236

control system, especially maintenance on redundant trains.

9. Understands the basic lksion process and concept of criticality. Can name the major systems related lo shutdown of the reactor,

10. Know the plant Uses nuclear energy as a heat source, u n find his way through the plant, understands the concepts of safety (similar to general employee).

The draft performance measures for the positions of shin technical advisor and maintenance foreman follow. (Note that an excellent fating k not appropriate for each position for each dimension.)

ShR Technical Advisor Excellent 1 Generic Measure 2

2 Generic Measure 3 Good 3 Generic Measure 4

4 Generic Measure 5 Poor S Generic Measure 6

Maintenance Foreman Excellent 1 Generic Measure 5

2 Generic Measure 6 Good 3 GenericMeasure7

4 Generic Measure 8 Poor 5 Generic Measure 9

The method has been rpplied thus ta r in draft form for four dimensions (or subcategories) of deep technical knowledge. Ten or twelve generic me8sures appeared to suffice for ten positions; however, lt k anticipated several more generic measures would be useful to cover twenty plant positions.

CONCLUSION

The approach appears to offer promise as it provides framework to make decisions on the distribution of deep technical knowledge in a nuclear organization by consensus without being judgmental. In addition, !he quantitative nature of the scales could be used in a process Or management model to measure the effect of deep technical knowledge on risk. This last step is, Of course, the most difficult, but the research thus far has shown that such an approach is feasible.

REFERENCES

I. 0. APOSTOLAKIS, 0. GRUSKY, 0, OKRENf, et a\, 'Inclusion d Organizationa? and Management Factors into Probabilistic Safely Assessments of Nuclear Power Plants: NUREG/CI3-5751.

2. S. HABER, et at, 'Influence of Organkational Factors o n Performance Reliability,. NUREG/CR-SW (1990).

3. A. MARCUS, et ai, 'Organization m d Safety in Nuclear Power Plants,' NUREGICR-5437 (1990).

4. J. W R E A T W et al, The Development md Evaluation of Programmatic Performance tndicatora Associated with hfaintenance at Nuclear Power Plants: NUREGICR-5436.

5. R. JACOBS, et all 'Organizational Processes and Nuclear Power Plant Safety: 191'3 Water Reador Safety Meeting (October 1991).

6. V. JOKSIMOVICH, et at, 'Organizational and Management Influences on sltfety ot Nuclear Power Plants,' NUREG/CR-S752 (DRAfT).

f . F. IANDY and J. L. FARR, The Measurement of Work Performance,' NY Acadernfc Press (1982).

8. G. APOSTOIAKIS, 0. OKRENT, 0. GRUSKY, J. S. WU, R. ADAMS, K. DAVO\!DlAN, Y. XIONG, 'Inclusion of Organizational Factors into Probabilistic Safety Assessm ents of Nuclear Power Piants,' lEEE Fifth Conference on Human factors and Power Plants, Julie 7.1 1, Monterey (1992).

.

t

237

TABLE 1

LICENSEE EVENT REPORT (LER) REVIEW c-

P U N T TlTLE CAUSE Callaway 1 utem to maimin rMctOr and turbine power at

Cook 2 IncOmplCtS miXln0 kr the W S T , less than precise L€R 92901 blonde? tantrOl and hadquato administrative

controt of RWST boron concentration.

Dresden 2 Inoperable intormediato range monitor. - P.nonn.l error h that tho cabk was not i E R 92005 rrconnsctbd IS rquired.

Fermi 2 Unplanned breach of HPCl oil system. foccnnicin remvod the temperature sensor LER 92-001 ' themrowcll.

lndlan Point 3 Procedural inadequacy that could have resulted in Insutriient correlation between plant design LER 91-012 documents and tho plant emergency procedures.

N I n e lnstrumentrtlon rootvalveforthereactor protecton Point 1 personnel error. system found closedd. LER 92001

Oconee 1 LER 92003 was not raognized.

South Texas 1 Containment integrty technical specifration Misinterpmution of the containment integrty LER 92402 violation. requirementr.

T h r e e Lack of understanding tfat the test involved fuel Island 1 containment isolation. movement. LER 97904

Trojan Failure to recognize that these deluQe system LER 92-001 spray pettern. n0zzl.r were part of a fire protection system.

Arnold Potential bss of vessel b e l instrumentation Failure to recognize and account for the effects of LER 0 2 a 1 tunctions. high drywtll temperatures on the high level trip

instrumentation when the instrument line reference k g s were ro-muted during the 1988 retuel outage.

Operator dd not insert control rods during shfl

Hi$h steam generator level due to bedwater

Refueling water storage tank boron concentration Qreater than specification limit dum to incomplete mixmng.

LZR 92403 control problem. kw k d s .

overloading vital buses during a LOCA

lack of configuration management caused by M i i e

Equipment failure in emetgegy power system. No action taken when required because the ned

M I 1 e Movement of irradtated fuel assembly without

Erection of scaffold which obstructed the noule

Big Rock Point LER 91- wolution. turnovor @?ter shutdown.

Return to critkalty during a reactor shutdown

Catawba 1 Two inoperable trains of tho control mom Control room operators did not recognize the need LER 02402 ventilation system to Open ventilation damper in order k r System to

bo operable.

Comanche 1 LER 91929

Technical specfition violation due to steam .upply valves to the turbine driven auxiliary feedwater pump b e i i hohtd in Mode 3.

Personnd error while taking manual control Of the generator primary water system leads to a raactor trip.

Cognitive error by operators assuming that there was a T/S 3.0.4 exception of the limning condnron for operation,

Failure to understand the potentlsl conrequcntar of controlling primry water (iow in manual and failure of the shift $upclrviror to adcquatcly mOnItOr the evolution.

Comanche 1 LER 92901

b

c c s" A

I(

3

B

w

3 x

W U 2)

L

.. c

P

- U

c

Y

- Y

Y

c

c - CI

h

c

c - e

e

7 *' I

L

- L

L

c - c

w

c1

c

L

m

239