bayesian component reliability estimation: an f-35 case study · 2019. 4. 18. · bayesian methods...

Bayesian Component Reliability Estimation: an F-35 Case Study

Bram Lillard Rebecca Medlin

April 2019 0

F-35 is a complex aircraft…

1

Estimating Component Reliability is essential for Operations and Sustainment

2

Air Vehicle Systems # components within category

PWR & THERMAL MGMTSYS (PTMS) 88270VDC GENERATION AND DIST 28CONTROL PANELS 26SENSORS, WPNS BAY, ENG BAY 46CONTROL SURFACES 49FUEL SYSTEM 141ICE DETECTION 5LANDING GEAR 261LIGHTING 31IMU & IEU 16OXYGEN GEN 7HELMET AND DATA PROCESSORS 52PHM AIR VEHICLE 7VEHICLE SYS PROCESSING (VSP) 16CNI SYSTEM 70STANDARD PRACTICES, STRUCTURES 38DOORS & COVERS 330FRAME, BULKHEADS 113STABILIZERS, RUDDER 40CANOPY 27STRUCTURE, FARINGS, FLAPS 92PROPULSION AIRCRAFT INTERFACE 9THROTTLE 6DOOR ACTUATORS (STOVL ONLY) 49RADAR SYSTEM 149EJECTION SEAT, SYSTEM 34ELECTRONIC WARFARE 81

Over 2,000 parts

Reliability estimates drive :• Spares purchases• Program budgeting• Cost estimation• Readiness

What comprises F-35 Costs per Flying Hour?

3

Air Vehiclecomponent removals

for repair/replacement

Engine modules removalsfor repair/replacement

Manpower(operations & maintenance) Maintenance

Fuel, Expendables

Depot level repair and other maintenance costs

Support,Training

SystemImprovements


4

𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 = �𝑖𝑖=1

𝑛𝑛[𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 𝑜𝑜𝑜𝑜 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑜𝑜 𝐶𝐶𝑜𝑜𝐶𝐶𝑅𝑅]𝑖𝑖× (𝑅𝑅𝑛𝑛𝑅𝑅 𝑓𝑓𝑅𝑅𝑅𝑅𝑅𝑅𝑛𝑛𝑜𝑜𝑅𝑅𝐶𝐶)𝑖𝑖

𝑇𝑇𝑜𝑜𝑅𝑅𝑅𝑅𝑅𝑅 𝐶𝐶𝑅𝑅𝑅𝑅𝐹𝐹𝐹𝑅𝑅 𝐶𝐶𝑜𝑜𝑛𝑛𝑜𝑜𝐶𝐶

CPFH = Cost per Flight Hour MFHBR = Mean Flight Hours Between Removals


Fuel, Expendables


Support,Training

SystemImprovements


5

𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 = �𝑖𝑖=1

𝑛𝑛[𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 𝑜𝑜𝑜𝑜 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑜𝑜 𝐶𝐶𝑜𝑜𝐶𝐶𝑅𝑅]𝑖𝑖

MFHBR𝑖𝑖

Accurate component Reliability estimates are essential for cost estimation

CPFH = Cost per Flight Hour MFHBR = Mean Flight Hours Between Removals


Fuel, Expendables


Support,Training

SystemImprovements

Data is often scarce for reliability estimation

6

0

10,000

20,000

30,000

40,000

50,000

2014 2015 2016 2017

Cumulative total Flight

Hours

Early in a Program we only have Engineering Estimates

for com pone nt re liab ility(a lso whe n a ne w

variant/ configurationbe g ins flying )

Late r, suffic ie nt failure s have occurre d , flying hours accum ulate d , to be g in

e stim ating re liab ility for e ach com pone nt

Early Lots Upgrade d Lots

Component Reliability Estimates – Many methods

7

Three CasesMany failures (N>20)

𝑀𝑀𝐶𝐶𝐶𝐶𝑀𝑀𝑅𝑅𝑖𝑖 =𝑇𝑇𝑜𝑜𝑅𝑅𝑅𝑅𝑅𝑅 𝐶𝐶𝑅𝑅𝑅𝑅𝐹𝐹𝐹𝑅𝑅 𝐶𝐶𝑜𝑜𝑛𝑛𝑜𝑜𝐶𝐶

𝑁𝑁𝑖𝑖

No Failure s to date (N=0)

𝑀𝑀𝐶𝐶𝐶𝐶𝑀𝑀𝑅𝑅𝑖𝑖 = 𝑀𝑀𝐶𝐶𝐶𝐶𝑀𝑀𝑅𝑅𝑖𝑖𝐸𝐸𝑛𝑛𝐸𝐸𝑖𝑖𝑛𝑛𝐸𝐸𝐸𝐸𝐸𝐸 𝐸𝐸𝐸𝐸𝑇𝑇.

What if FH >> 𝑀𝑀𝐶𝐶𝐶𝐶𝑀𝑀𝑅𝑅𝑖𝑖𝐸𝐸𝑛𝑛𝐸𝐸𝑖𝑖𝑛𝑛𝐸𝐸𝐸𝐸𝐸𝐸 𝐸𝐸𝐸𝐸𝑇𝑇 ?

Do we use the : • lowe r CI bound?• se t e qual to FH?• e ng ine e ring e st.?

Fe w failure s (1 < N < 20)

Do we use : • FH/ N (ignore unce rtainty)?

• Re port a we ighte d ave rage ?• E.g ., 0 .3*(FH/ N) + 0 .7*Eng . Est.

N = num be r of fa ilure s; FH = Flight Hours

Alternatively we can use a Bayesian approach (s lid ing scale we ighte d ave rage )

(assum e fa ilure tim e s follow an Expone ntia l Distribution)

Bayesian statistics combine “prior” knowledge with observed data to produce an estimate

8

Example for Component X:• Engineering Estimate MFHBR = 990 hours• Flight Hours flown to date: 40,000 hours• Observed 2 Failures…. traditional methods estimate:• MFHBR = 40,000 / 2 = 20,000 hours

What’s the best number to use for MFHBR?990 or 20,000?

Average the two? (~10,500?)Weigh one more than the other? Which one?

One math slide for the presentation…

9

• Likelihood Distribution: Exponential (𝜆𝜆)• 𝑀𝑀𝐶𝐶𝐶𝐶𝑀𝑀𝑅𝑅 = 1

𝜆𝜆= 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝐹𝐹𝑇𝑇𝑖𝑖𝐸𝐸𝐹𝑇𝑇 𝐻𝐻𝑇𝑇𝐻𝐻𝐸𝐸𝐸𝐸

𝑛𝑛

• Prior Distribution: Gam m a (𝛼𝛼,𝛽𝛽)• We can use the e ng ine e ring e stim ate s to solve for 𝛼𝛼 and 𝛽𝛽.• Inv. Gam m a m e an = 𝑀𝑀𝐶𝐶𝐶𝐶𝑀𝑀𝑅𝑅𝐸𝐸𝑛𝑛𝐸𝐸𝑖𝑖𝑛𝑛𝐸𝐸𝐸𝐸𝐸𝐸 𝐸𝐸𝐸𝐸𝑇𝑇• Inv. Gam m a std . = 𝑀𝑀𝐶𝐶𝐶𝐶𝑀𝑀𝑅𝑅𝐸𝐸𝑛𝑛𝐸𝐸𝑖𝑖𝑛𝑛𝐸𝐸𝐸𝐸𝐸𝐸 𝐸𝐸𝐸𝐸𝑇𝑇 𝑥𝑥 𝑅𝑅

• Posterior Distribution: Gam m a (𝛼𝛼`,𝛽𝛽`)• 𝛼𝛼𝛼 = 𝑁𝑁 + 𝛼𝛼• 𝛽𝛽𝛼 = 𝑇𝑇𝑜𝑜𝑅𝑅𝑅𝑅𝑅𝑅 𝐶𝐶𝑅𝑅𝑅𝑅𝐹𝐹𝐹𝑅𝑅 𝐹𝑜𝑜𝑛𝑛𝑜𝑜𝐶𝐶 + 𝛽𝛽

Baye sian approach to e stim ating Re liab ility

𝜋𝜋 𝜆𝜆 𝒙𝒙 ∝ 𝐿𝐿 𝒙𝒙 𝜆𝜆 𝜋𝜋 𝜆𝜆

Posterior DistributionLikelihood Distribution

PriorDistribution

p = the “confide nce ” we p lace in the p rior inform ation. We use d a p = 1.5.


10

Example for Component X:• Engineering Estimate MFHBR = 990 hours (yellow “prior” below)• Flight Hours flown to date: 40,000 hours• Observed 2 Failures…. traditional methods estimate:• MFHBR = 40,000 / 2 = 20,000 hours (blue “likelihood” below)

Posterior Median/Mean

(best estimate of “true” reliability)

MFHBR est.from data alone

MFHBR from engineeringestimates


11

Example for Component X:• Engineering Estimate MFHBR = 990 hours (yellow “prior” below)• Flight Hours flown to date: 40,000 hours• Observed 2 Failures…. traditional methods estimate:• MFHBR = 40,000 / 2 = 20,000 hours (blue “likelihood” below)

We’re showing our trust in the Engineering Estimates with the

narrow width of the prior

The final estimate is influencedsignificantly by the Engineering Estimate because 1. few data (failures) exist, 2. we chose a narrow distribution

for the prior

With more failure data available, the final estimate is less influenced by the MAC value

12

Example for Component Y:• MAC/SPUD estimates the MFHBR = 990 hours (yellow prior below)• Flight Hours flown to date: 40,000• Observed 10 Failures, so, traditional methods estimate:• MFHBR = 40,000 / 10 = 4,000 hours (blue “likelihood” below)

Posterior Median/Mean(best estimate of “true” reliability)

MFHBR est.from data alone

MFHBR from engineering estimates

A robust methodology for all cases

13

• Bayesian method appropriately moves MFHBR estimate towards the traditional result as the available data increases

• The approach also automatically handles cases where N=0 (something not satisfactorily handled with traditional approaches)

A frequent debate:How do we estimate MFHBR for a new configuration?

14

Example for Component Z:

• Bayesian method provides an ideal (and defensible) calculation method for this case

• MFHBR for Older Lots serves as the new prior estimate for the Ne w Lots calculation • Appropriate ly using the availab le data as a s tarting point, but a llowing the

availab le Ne w data to d ic tate how m uch the final e stim ate is m ove d

• Baye sian re sults for Ne w Lots : MFHBR = 7,576

Older Lots New Lots (anticipated improvement)

Engine e ring Estim ate 900 900

Flight hours 20 ,000 10,000

Failure s obse rve d 5 0

MFHBR 4,0003,338? (95% Lowe r bound)4,000? (LRIP 2-5 e stim ate )

900? (Eng . Est.)

Distributed Aperture System Sensors’ reliability show the benefit of the Bayesian approach

15

Confidence interval cut off for readability

(21)(4)

(19)

(1)

(11)

(0) (5)

(6)

(4)(2)

(14)

(4)

Engineering Estimate


16


17


18

Only one failure observed; point

estimate is highly uncertain

Bayesian estimate uses previous Lot

estimate as starting point – much better reliability estimate

Cases with lots of data (failures),

estimates are the same


19

Sensor #3 saw 11 failures in older Lots, and zero failures in

New Lots

Only a lower bound can be estimated –

and is more pessimistic than the

Bayesian point estimate


20

Traditional method is highly uncertain (2

failures), but suggest a degradation in performance

Bayesian method is a properly weighted average between

Engineering Estimate and Traditional

Point estimates reflect the fact that there is no

real difference in performance


21

Air Vehiclecomponent removals

for repair/replacement

Engine modules removalsfor repair/replacement


Fuel, Expendables


Support,Training

SystemImprovements

Bayesian Reliability results in a more informed estimate of maintenance costs

22

Maintenance Costs ($)

Last Estimate (2015)(uses Engineering Estimates

for Reliability)

New Estimate (uses IDA-Bayesian method

for Reliability)

Overhauls Consumables Air VehicleRepairables

EngineRepairables


Fuel, Expendables

Support,Training

SystemImprovements

Conclusion

23

Bayesian methods provide a means to combine available knowledge of reliability with operational data to estimate component reliability, resulting in a more informed estimate of F -35 maintenance costs.

• Updated from early engineering estimates• Updated from previous system/variant data• Handles cases with few data (even no failures!)

bayesian component reliability estimation: an f-35 case study · 2019. 4. 18. · bayesian methods...

Documents