high-confidence, low- probability-of-failure screening january 11, 2014 rlgm? p[p[failure|eq]
TRANSCRIPT
High-Confidence, Low-Probability-of-Failure Screening
January 11, 2014
0 2 4 6 8 10 12 14 16 18 200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
RLGM?
P[P[Failure|Eq]<0.05]95%
Magnitude
HCLPF depends on whom you believe
• Which components need seismic safety improvements? – If HCLPF < RLGM (Review Level Ground Motion), improve them– “HCLPF is defined as the earthquake [ground] motion level at which there is a high
(95 percent) confidence of a low (at most 5 percent) probability of failure” Interim Staff Guidance, DC/COL-ISG-020
• HCLPF: SoV vs. CDFM – AmExp[-2.32bC] “CDFM” (Conservative Deterministic Failure Margin, “because
engineers understand it”) [Kennedy et al.] – AmExp[-1.65(bR+bU)] [“SoV” (Separation of Variables), (EPRI TR-103959, Ravindra,
and others]• Am should be the a “High-Confidence” limit on the natural logarithm of median strength• -1.65 and -2.32 are the number of 5% and 1% standard deviations below the normal distribution
[log] mean Am
• The b-parameters represent randomness or aleatory variability (R), epistemic uncertainty (U), or combined uncertainty (C)
• Notice any sample uncertainty?
Objective of Seismic Screening: Component HCLPF < RLGM?
• Guidance unspecific, encourages industry initiative, abhors statistics?– ASCE/SEI 41-06 and -13, FEMA E-74 Chapter 4.1, and IEAE NS-G-1.6– RG 1.208, “A Performance-based Approach to Define the Site-
specific Earthquake Ground Motion” (NRC, 2007b) “The desired performance being expressed as the target value of 10-5 for the mean annual probability of exceedance (frequency) of the onset of significant inelastic deformation “
• Plants with higher seismic risk are already shut down– Vallecitos, Humboldt Bay, Zion, Fukushima Daiichi, San Onofre– Older plants have little time left, except for 20-or-30-year
extensions!
Alternative Uncertainty Model
Q = P[f < f’ | a]; i.e., the subjective probability (confidence) that the conditional probability of failure, f, is less than f’ for a peak ground acceleration a (e.g., RLGM)-1(.)= the inverse of the standard Gaussian cumulative distribution function.
R
Um
QAa
In
f
1-
How about some Rational Economic Resource Allocation?
• Cost-benefit analysis and budget constrained resource allocation is not under current consideration– Cost of alternatives? Fix some? Shut down plant?
• How to allocate resources to aleatory (random) or epistemic (unknown) uncertainty?– Is reducing epistemic uncertainty [unknown unknowns] a scam?
Quants use scenarios. Elizabeth Paté-Cornell says insure or hedge (Allin’s wife)• Price-Anderson act provides insurance to nuclear utility industry• NRC insures industry for costs incurred by delays caused by NRC
– How to allocate to high confidence vs. low probability of failure• “The public appears to heavily value confidence and places a much
smaller, although still positive, emphasis on accuracy.” [Smith and Wooten]
Legal, Precautionary Principle, and Social Equity
• “…if an action or policy has a suspected risk of causing harm to the public or to the environment, in the absence of scientific consensus that the action or policy is harmful, the burden of proof that it is not harmful falls on those taking an action.” http://en.wikipedia.org/wiki/Precautionary_principle
• Search for “Kip Viscusi and Risk Equity” for guidance • ISO 26000 Social Responsibility, ISO 16269-6
“Determination of Statistical Tolerance Intervals”
HCLPF Borrows Credibility• To statisticians, HCLPF is a tolerance limit or a confidence limit on a sample
percentile (ISO 16269-6 and http://en.wikipedia.org/wiki/Tolerance_limit– The “High-Confidence” part of HCLPF corresponds to the statistical confidence
limit, typically 95%– The “Low-Probability-of-Failure” part of HCLPF corresponds to the percentile,
typically 5-th percentile of the fragility function– Tolerance limits are supposed to be estimated from data, perhaps censored,
perhaps extrapolated• To reliability engineers, HCLPF is a R95C95 reliability demonstration test:
95% confidence that reliability > 95%– RnnCmm demonstration uses sample test or field (observed) data
• HCLPF uses subjective opinions or response analysis to relate component fragility to RLGM instead of local component response– Awkwardly related to material strength distributions [ASME et al.]
Methods: Economics
• Minimize seismic risk subject to budget constraint– Allocate resources to biggest bang-per-buck = Seismic Risk/$
$$(component fragility parameter)• Seismic risk = E[Discounted future costs due to earthquakes]
– = exp(-dt)*P[Eq of magnitude m at age t]*E[Cost|Eq of magnitude m at age t]dF(m,t)
– integrate from now to license expiration– Conditional on components to be improved
• $$$(component fragility parameter) = cost per unit change in component fragility parameter
– Use chain rule (next slide) – Use law of diminishing marginal returns: the more you spend
on one component, the less you get
Chain Rule
• Seismic Risk / $$$(component fragility parameter) = – Seismic Risk / Subsystem Risk* Subsystem Risk / $$$
(component fragility parameter)– Subsystem Risk / $$$(component fragility parameter) =
Subsystem Risk / Component type risk * Component Type Risk / $$$(component fragility parameter)
– Component Type Risk / $$$(component fragility parameter) = P[Component Failure|Eq] / $$$(component fragility parameter)
– Where component parameters include fragility correlations!
Why is HCLPF = AmExp[-1.65b]?
• Assume strength at failure RV is AmeReU
– Where Am is lower 5-th confidence limit on median component strength at failure and
– ln(eR) and ln(eU) are normally distributed, independent random variables with means 0 and (logarithmic) standard deviations bR and bU
• Then P[Stress > Strength] = [ln(Stress/Am)/(bR2+bU
2)] and– Screen is P[RLGM > HCLPF?] = 5% – HCLPF? = -1[0.05, ln(Am), (bR
2 + bU2)] or
– AmExp[1.64485(bR2+bU
2)] where standard normal z(0.05) = 1.64485
– i.e., P[Stress/(AmeReU) > 1] = [ln(Stress/Am) > 0] = 0.05
Methods: Data and Subjective Opinions to Obtain HCLPF
• Data: ASME and other material strength distributions, component test data, and post-earthquake failure observations– Maximum likelihood, Bayes, and method of moments– “The seismic capacity of such equipment in regard of their
functionality during and after an earthquake is impossible, difficult or unreliable to evaluate by other methods” [Other than seismic test, Tengler]
• Subjective opinions on median, percentiles, (logarithmic) standard deviations, and Y1|Y2 [NUREG/CR-3558 and others]– Least squares and weighted least squares– Output Am and b from “High-Confidence” fragility function, and r
Excel Workbook Implementations
• HCLPF….xlsx– Computes HCLPFs and P[RLGM > HCLPF] for various components
from various input parameters• SubjFrag.xlsx
– Computes High-Confidence fragility functions from test and seismic observation data, subjective opinions, and multiple-failure-mode fragility functions
– Estimates fragility correlations from subjective opinions of P[X1|X2] • NoFail.xlsm
– Estimates lognormal fragility function parameters, including correlation, from earthquake responses and component no-failure observations
HCLPF…xlsx SpreadsheetsSpreadsheet Name Contents
IP2ESEL Jim Moody’s list of components
BoM FLEX PWR list of components
Inherent ESEL Summary table list of components
HCLPF Compute HCLPF and P[RLGM > HCLPF] from component lists and fragility parameters
HELParameters Bandyopadhyay fragility parameters
Structures Structure CDFM fragility parameters
Equipment Equipment CDFM fragility parameters
ToleranceInterval Estimate correct tolerance limit from sample data 95% confidence on 5-th percentile
HCLPF Inputs
• List of component candidates for screening, QPA, structure function (series, parallel, RBD, fault tree)
• Design basis and RLGM and their units • Method(s) (CDFM or SoV?) • Component “fragility” parameters Am and b
– Strength-at-failure distributions for materials and some components or “High-Confidence” Am (median) and bR (logarithmic) standard deviation assuming lognormal strength
– Uncertainties due to components’ strength and responses to seismic ground motions, bU
– Safety and other fudge factors for CDFM computation of structure and equipment fragility parameters
HCLPF…xlsx:HCLPF Spreadsheet
• HCLPF spreadsheet does the HCLPF and P[RLGM>HCLPF]? and other computations– From component lists, parameter lists, and parameter
computations in other spreadsheets– Table 1 contains notes– Table 2 originated from AREVA proposal form
• Added QPA, Units, Method, Factors, Am, b, and computation columns• Computations include: z(p)Amb, RLGM>HCLPF?, (ln(RLGM/Am)/b), ({ln(RLGM/Am)+ bU-1(Q(RLGM)))/bR), and P[All QPA components survive]
• Could use VLOOKUP() function or links to enter parameters from other spreadsheets
HCLPF Spreadsheet Table 2
ESEL Item # ID Description
QPA assumed in series
Reference Ground Motion (Design)
Review Level Ground Motion RLGM Units
Method SoV? CDFM?
TRS(C)Test Response Spectrum
RRS(C) Required Response Spectrum
F(D) Device capacity factor
F(RS) Response factor for structure
S(A) Avg. spectral acceleration
Scale Factor SF for reference ground Am Units R U C
Am*exp(-1.65*R)
RLGM > HCLPF?
Am*exp(-1.65*(R+U))
RLGM > HCLPF?
Am*exp(-2.326*C)
RLGM > HCLPF?
(ln(RLGM/Am)/R)
(ln(RLGM/Am)/Sqrt(R^2+bU^2))
((ln(RLGM/Am)+U*^(-1)(Q(a)))/R)
P[min > RLGM]
MechanicalTurbine driven AFW pump 1 0.2 pga, g SoV? CDFM? 1 pga, g 0.2 0.2 0.282843 0.718924 FALSE 0.516851 FALSE 0.517942 FALSE 4.23585E-16 6.3437E-09 4.23585E-16 0.00E+00Turbine driven AFW valves 2 0.2 pga, g SoV? CDFM? 1 pga, g 0.2 0.2 0.282843 0.718924 FALSE 0.516851 FALSE 0.517942 FALSE 4.23585E-16 6.3437E-09 4.23585E-16 0.00E+00SG PORVs 2 0.2 pga, g SoV? CDFM? 1 pga, g 0.2 0.2 0.282843 0.718924 FALSE 0.516851 FALSE 0.517942 FALSE 4.23585E-16 6.3437E-09 4.23585E-16 0.00E+00Condensate storage tank 1 0.1 0.2 pga, g SoV? CDFM? 1 pga, g 0.2 0.2 0.282843 0.718924 FALSE 0.516851 FALSE 0.517942 FALSE 4.23585E-16 6.3437E-09 4.23585E-16 0.00E+00Tanks, Ch. 7 TR-103959 2 0.1 0.2 pga, g SoV 1.14 0.537736 0.053774 pga, g 0.22 0.17 0.278029 0.037404 TRUE 0.028255 TRUE 0.028165 TRUE 0.999999999 0.999998846 0.999999999 1.00E+00Tanks, Table 7-11 and NP-6041 2 0.1 0.2 pga, g CDFM 0.52 pga, g 0.229 0.244 0.33463 0.356373 FALSE 0.238264 FALSE 0.238765 FALSE 1.50612E-05 0.002148932 1.50612E-05 3.01E-05SG injection valves 2 0.2 pga, g SoV? CDFM? 1 pga, g 0.2 0.2 0.282843 0.718924 FALSE 0.516851 FALSE 0.517942 FALSE 4.23585E-16 6.3437E-09 4.23585E-16 0.00E+00RCS injection valves 2 0.2 pga, g SoV? CDFM? 1 pga, g 0.2 0.2 0.282843 0.718924 FALSE 0.516851 FALSE 0.517942 FALSE 4.23585E-16 6.3437E-09 4.23585E-16 0.00E+00Other mechanical equipment 10 0.2 pga, g SoV? CDFM? 1 pga, g 0.2 0.2 0.282843 0.718924 FALSE 0.516851 FALSE 0.517942 FALSE 4.23585E-16 6.3437E-09 4.23585E-16 4.44E-15Electrical 0.2 pga, g SoV? CDFM?Batteries 2 0.2 pga, g SoV? CDFM? 0.5 pga, g 0.2 0.2 0.282843 0.359462 FALSE 0.258426 FALSE 0.258971 FALSE 2.30877E-06 0.000598536 2.30877E-06 4.62E-06DC distribution panels 2 0.1 0.2 pga, g SoV 3.3 0.33 pga, g 0.11 0.27 0.291548 0.275226 FALSE 0.176283 TRUE 0.167495 TRUE 2.65058E-06 0.042930687 2.65058E-06 5.30E-06DC MCCs 2 0.1 0.2 pga, g SoV 2.8 0.28 pga, g 0.1 0.23 0.250799 0.23741 FALSE 0.162437 TRUE 0.156246 TRUE 0.000383104 0.089862431 0.000383104 7.66E-04DC switchgear 2 0.1 0.2 pga, g SoV 4.4 0.44 pga, g 0.13 0.37 0.392173 0.355056 FALSE 0.192823 TRUE 0.176722 TRUE 6.59536E-10 0.022190135 6.59536E-10 1.32E-09Vital AC distribution panels 2 0.1 0.2 pga, g SoV 3.3 0.33 pga, g 0.11 0.27 0.291548 0.275226 FALSE 0.176283 TRUE 0.167495 TRUE 2.65058E-06 0.042930687 2.65058E-06 5.30E-06Battery chargers 1 0.2 pga, g SoV? CDFM? 0.5 pga, g 0.2 0.2 0.282843 0.359462 FALSE 0.258426 FALSE 0.258971 FALSE 2.30877E-06 0.000598536 2.30877E-06 2.31E-06Inverters 1 0.2 pga, g SoV? CDFM? 0.5 pga, g 0.2 0.2 0.282843 0.359462 FALSE 0.258426 FALSE 0.258971 FALSE 2.30877E-06 0.000598536 2.30877E-06 2.31E-06Instrument racks 4 0.1 0.2 pga, g SoV 1 1 1 1 1 0.1 pga, g 0.2 0.2 0.282843 0.071892 TRUE 0.051685 TRUE 0.051794 TRUE 0.999735609 0.992869981 0.999735609 1.00E+00
If C is not SQRT(R^2+U 2), please enter your value
HCLPF Computations Include…
• AmExp[z(p) ]b : HCLPF = AmExp[z(p)b] etc. for alternative z(p) and b (p = “Low-Probability-of-Failure” 5% or 1%)
• RLGM>HCLPF?: TRUE or FALSE • (ln(RLGM/Am)/b) = P[RLGM > HCLPF] for alternative b
• ({ln(RLGM/Am)+bU-1(Q(RLGM))}/bR) [Ravindra, Kennedy, et al.]
• P[min>RLGM] = P[All QPA components survive] – Assumes QPA co-located, series components with iid fragilities– Change the formula for parallel components or other
configuration (FTA) or correlated fragilities
SubjFrag.xlsx SpreadsheetsSpreadsheet Name Contains
ToleranceInterval Same as HCLPF…xlsx:ToleranceInterval
19experts Compute least-squares lognormal fit to subjective lower 5th percentiles
19expertsWeighted Ditto with weighted least squares
10expertsBootstrap Bootstrap 95% “High-Confidence” lognormal fit with fewer than 19 experts
SubjCorr Estimate r from subjective distribution of Y1|Y2 by least squares
Tolerance Limit on Lognormal RV Fragility Function
• Estimate correct tolerance limit on a lognormal distribution from a sample of means m (= ln(Am)) and (logarithmic) standard deviations b
• Input desired “High-Confidence” and “Low-Probability-of-Failure) and a sample of means and standard deviations from tests or subjective opinions– "Exposure Assessment: Tolerance Limits, Confidence Intervals, and Power
Calculation…" K. Krisnamoorthy et al.• Confidence limit for m + z(p)*b is constructed of the form…• m +Q(p,alpha)* b from estimates m and b are the MLEs and Q(p,alpha) is the
tolerance factor determined so that …• P[ m +Q(p,alpha)* b < something] > 1 alpha…• where (somethingm)/b is approximately ~ (z(p)m)/b
• Output is tolerance limit or HCLPF (table 2 is simulated)
“High-Confidence” Subjective Fragility Function Estimation
• Suppose 19 experts give 19 opinions on fragility median Am and (logarithmic) standard deviation
• For each (discrete) value of strength y, find maximum of 19 cdfs and connect with a curve Fmax(y)– P[Strength < y|Expert Am and b], (or Am and some percentile) and – Assume each represents the upper 95% confidence limit (“High-
Confidence”)– Fit a lognormal distribution to minimum curve to represent a 95%
“High-Confidence” fragility function using (weighted) least squares
• Note that P[F(y) ≤ Fmax(y) for all y] 0.95
Lognormal Fit to Lower 5-th Percentiles
0 1 2 3 4 5 6 7 8 9 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
19 opinions, maxima, and 5th percentile fragility function
lower 5th %ileExpert 1Expert 2Expert 3Expert 4Expert 5Expert 6Expert 7Expert 8Expert 9Expert 10Expert 11Expert 12Expert 13Expert 14Expert 15Expert 16Expert 17Expert 18Expert 19Max
ln[strength at failure]
By Weighted Least Squares
0 1 2 3 4 5 6 7 8 9 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
19 opinions, maxima, and 5th percentile fragility function
lower 5th %ileExpert 1Expert 2Expert 3Expert 4Expert 5Expert 6Expert 7Expert 8Expert 9Expert 10Expert 11Expert 12Expert 13Expert 14Expert 15Expert 16Expert 17Expert 18Expert 19Max
ln[strength at failure]
What if there aren’t 19 experts?
• Bootstrap• Correlation estimate requires at least one
subjective opinion of distribution of X1|X2• Use least squares to combine experts’
subjective distribution information– Sum of squared errors indicted magnitude of
experts’ deviation from lognormal distribution
Bootstrap 10 experts
0 2 4 6 8 10 12 14 16 18 200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
lower 5th %ileExpert 1Expert 2Expert 3Expert 4Expert 5Expert 6Expert 7Expert 8Expert 9Expert 10Max
NoFail.xlsm Spreadsheets
Spreadsheet Name Contents
Freq Mle of lognormal fragility parameters from 19 non-failure responses, assuming 20-th would be failure
Bayes MoM estimates of lognormal fragility parameters of a-posteriori distribution of P[Failure|Eq] from 19 non-failure responses assuming non-informative prior
BayesCorr Same as Bayes, including estimate of fragility correlations
• Imagine inspections after earthquakes indicate component responses and failure or non-failure
Freq Spreadsheet
• Input iid responses for which no failures occurred– 19 responses were simulated for example and
convenient interpretation of mean and standard deviation estimates as “High-Confidence”
– Assume ln[Stress]-ln[Strength} ~ N[muX-muY, Sqrt(sigmaX^2+sigmaY^2)]
• Use Solver to maximize log likelihood of PP[Non-failure|Response] subject to constraint– Either constrain CV or P[Failure]
• Output is ln(Am) and b
Bayes Spreadsheets
• Bayes estimate of reliability r = P{ln[Stress]-ln{strength] > 0]– Non-informative prior distribution of r– Same inputs as Freq: responses and non-failures– Use MoM to find ln(Am) and b to match posterior E[r]
= n/n+1 and Var[r] = n/((n+1)^2*(n+2))– Ditto to find correlation r from third moment of a-
posteriori distribution of r• Bayes posterior P[ESEL component life > 72
hours|Eq and plant test data]
Parameter Estimates from 19 Non-Failure Responses
• Given 19 earthquake responses with ln(Median) = 0.5 and b = 0.1 and reliability P[ln(Stress) < ln(strength)] ~ 95% • Bayes non-informative prior on reliability P[Response < strength] => posterior
distribution • Use Method of Moments to estimate parameters for a-posteriori distribution of
reliability
Parameter Assume 20-th is failure
Bayes Bayes Correlation
m(ln(strength)) 0.781404 0.781389 0.950047891 b ln(strength) 0.148132 0.148123 0.226336118 R ln(strength) N/A N/A 0.752266708
What is the correlation of fragilities?
• See SubjFrag.xlsx:SubjCorr and NoFail.xlsm:BayesCorr spreadsheets to estimate correlations from subjective opinions on Y1|Y2 or from no-failure response observation
• HCLPF ignores fragility correlation• Risk doesn’t ignore it
What if multiple, co-located components?
• Responses are same (Refer to work for Howard last year)
• In series? Parallel? RBD? Fault tree?• Using event trees, Jim Moody argues that
HCLPF for one component is representative of all like, co-located components
What if like-components are dependent?
• Fragilities could be dependent too!• But not necessarily all fail if one fails – True, P[Response > strength] may be same for all
like, co-located components– But what is P[g(Stress, strength) = failure] for
system structure function g(.,.)?
More References• NAP, “Review of Recommendations for Probabilistic Seismic Hazard Analysis:
Guidance on Uncertainty and the Use of Experts (1997) / Treatment of Uncertainty,” National Academies Press, http://en.wikipedia.org/wiki/Quantification_of_margins_and_uncertainties
• Viscusi, W. Kip, “Risk Equity,” ISSN 1045-6333, (2000) http://www.law.harvard.edu/programs/olin_center/papers/pdf/294.pdf
• Der Khiureghian, Armen and Ove Ditlevsen, “Aleatory or Epistemic? Does It Matter?” (2007), Risk Acceptance and Communication Workshop, Stanford
• Mannes, Albert and Don Moore, “I Know I’m Right, A Behaviourial View of Overconfidence,” Significance, vol. 10, issue 4, August 2013
• Smith, Ben and Jadrian Wooten, “Pundits: The Confidence Trick,” Significance, vol. 10, issue 4, August 2013
• Tengler, Marek, “Seismic Qualification of NPP Structures, Systems and Other Components,” Seminar, Nov. 2011
• Smith, Ben and Jadrian Wooten, “Pundits: The Confidence Trick,” Significance, vol. 10, issue 4, August 2013