confidence limits

RELIABILITY ENGINEERING UNITASST4403

Lecture 18-19 CONFIDENCE LIMITS

Learning outcomes

Select and apply the concepts of statistical fidconfidence

Construct and use confidence intervals and limits for exponential distribution

Construct and use confidence intervals and limitsConstruct and use confidence intervals and limits for normal distribution

Construct confidence limits for Weibull plot Construct confidence limits for Weibull plot

Random sampling and the and the

Central Limit Theorem

What is a random sample?p Practically: A random sample is a collection of observations taken from the same population.observations taken from the same population.

Mathematically: A random sample of size n (eg. X1, X2, , Xn) is a collection of n iid random variables.

X s are independent and identically distributedXi s are independent and identically distributed (i.i.d.) iff:

X i i d d t d i bl d every Xi is an independent random variable; and

every Xi has the same probability distribution.

Random Sample

These conditions are exactly satisfied if: sampling with replacement; or sampling from an infinite conceptual population.p g p p p

These conditions are approximately satisfied if:These conditions are approximately satisfied if: sampling without replacement; and sampling from a finite population with size N>>n sampling from a finite population with size N>>n.

The observed values of X X X are denoted The observed values of X1, X2, , Xn are denoted by x1, x2, , xn

Sample Mean

Sample mean of {X1, X2, , Xn} is given by:

1 nX X 1

ii

X Xn

Note: is also a random variable!X

The Central Limit Theorem (CLT)

If X1, X2, , Xn are a random sample from a 1, 2, , n pdistribution with mean and standard deviation

th h i t l l, then has approximately a normal distribution with

X

mean : and standard deviation:

X / n /X n

provided that n is sufficiently large (n>30).

Statistical Confidence

Statistical confidence

Statistical confidence is the exact fraction of times the confidence interval will include the true value if theconfidence interval will include the true value, if the experiment is repeated many times

Confidence interval is the interval between the upper and lower confidence limits.

Statistical intervals are used to make assertion about a population given data from a sample.

It is not the same as engineering confidence which takes into account other factors which may make the data non-yrepresentative.

To what do we need confidence interval/limit/

Generally one of the parameters of a population, hi h d id di t ib tiwhich decides a distribution

Can often be e.g. the mean, standard deviation etc that have some practical meaning in reliability such as MTTF, MTTR

Confidence intervals...

Consider a population of bearing failures in heavy crude bottoms pumps...

Each circle representsEach circle represents an age at failure.

This bearing population will have a true mean and standard deviation.

Confidence intervals...Co de ce te a s...

In our current reliability analysis, we can collect data from 8 bearings.

We calculate theWe calculate the distribution variables (eg. Mean, std dev etc)( g , )

Confidence intervals...Confidence intervals...

In other refineries, engineers looks atengineers looks at their bearing failures.

13


What does the true population look like?

Failure age14


Each sample set gives a different result!

What value(s) can we use?use?

And how confident are we that we are using

Failure agethe right value?


It depends on: the range of data in


the range of data in our sample set;

the number of lsamples in our set;

how confident we want to be that we want to be that we have the right estimate.

Confidence intervals are bounds that help us do

Failure agethis. 16

Confidence intervals... EG. Define 80% bounds Confidence intervals...

Define bounds on our sample estimate that

on the sample means

sample estimate that allow us to be x% confident that the trueconfident that the true population value lies within these bounds,

The true mean will lie within the confidence

intervals in 4 out of 5 estimatesx% of the time. 5 estimates

The distance between the bounds is known as Population mean

Failure agethe bounds is known as a confidence interval.17

Population mean


Confidence interval: an interval of plausible values for the parameter being estimated

The degree of plausibility is specified by a The degree of plausibility is specified by a confidence level, e.g. 95%, 99%

If the confidence level is high and the resulting confidence interval is narrow, we have a reasonably precise estimate of the parameters value.

18

Confidence intervals and risk

We are prepared to accept a level of risk, expressed in a %, that the value of our sample falls outside a given %, p gconfidence interval.

A double-sided confidence interval applies a certainA double sided confidence interval applies a certain risk on either side of the parameter studied and gives upper and lower limit

E.g. 1000 hrs MTBF 1100hrs

A single-sided confidence interval means a certainA single-sided confidence interval means a certain risk at one side of the studied parameter and gives an upper or lower limit

E.g. 800 hrs MTBF or MTBF 2000hrs

Mathematically: is a Two sided confidence interval:

parameter for the population

1P L U L U i k h 100(1 )% id d LU is known as the 100(1-)% two sided confidence interval.

/ 2 / 21-/ 2 / 2

L U

Mathematically:

One sided confidence interval:

1P L L i k h 100(1 )% i l id d L is known as the 100(1-)% single sided confidence interval. Note: No Note: No

upperboundupper bound

1-L

How do we work out our CI?

Depends on: Type of underlying distribution;

Type of statistical parameter Type of statistical parameter (e.g. Mean or variance);

Number of samples in our sample set.

When we can construct confidence interval?

Data has been collected;

Distribution has been identified and its parameters estimatedparameters estimated.

Commonly used distributions

Normal distribution

Exponential distribution

Weibull distribution

Confidence interval Confidence interval for normal distribution

Confidence on Mean of Normal Distn:

If population is s-normally distributed OR if n is

Population Variance Known or n large

If population is s-normally distributed, OR if n is large (>30), standard error of the distribution is determined by:determined by:

StdError SE From central limit is either given l l d

L li i

StdError SEn central limit

theoremor calculated from sample

Lower limit: LCL X Zn X

Z

Upper limit: UCL X Z /Z n

nNote: X bar = ^Z is found using std normal

table for a given %

Confidence on Mean of Normal Distribution

Use CDF of s-normal distribution to determine fid li it

Distribution

confidence limits.

1XP Z Z /2 /2 1/P Z ZnE.g. : for a 90% 2 sided confidence interval, or a 95% 1 sided confidence interval, we want 5% in the t il( ) S l k f th Z l hi h ill i thtail(s). So look for the Z-value which will give you the closest number to 0.95 (=1.65).

Note: is our risk, typically small such as 0.1 or 0.051 our confidence typically large such as 0 9 or 0 951- = our confidence, typically large such as 0.9 or 0.95

/2 in the formula is because the risk is half on each side

Example 1Example 1

A sample of 100 values has a mean of 27.56 and standard d i ti 1 1 D i 95% fid li it f thdeviation 1.1. Derive 95% confidence limits for the population mean (assume the sample means are normally distributed)distributed)

The standard error of the sample is: 1.1 0.11100

SSE

nUsing Z tables:

the closest value of z such that (z)=0.95 is 1.65. 100n

95% single sided CL of the popn mean:

27.561.65*0.11=27.56 0.18.56 65 0 56 0 890% double sided CL of the popn mean:

LCL= 27.56-0.18=27.38, UCL= 27.56+0.18 = 27.74

Example 2

In a maintainability demonstration test 50 times

51 71 75 67 86demonstration test, 50 times taken to repair are collected and the data is normally distributed.

58 52 64 41 74

48 55 43 72 30y

Sum of all repair times = 3096 mins

39 64 45 63 37

70 37 48 71 69mins.

min92.6150

3096 n

tMTTR i

70 37 48 71 69

83 57 83 46 72

Sample standard deviation 50n

33 59 97 66 93

76 68 50 65 63

min74.151

)(50

1

2

n

MTTRtS i

i 75 63 51 69 75

1n 64 54 53 59 92

Adapted from the Defence Reliability Management Course, 2/2005

Example 2p

Q: What is the value we can be sure with 90% confidence that the MTTR will not be exceeded?confidence that the MTTR will not be exceeded?

A: We need to find the 90% upper confidence limit on the MTTRthe MTTRSince n=50 > 30, we can use:

z S= Sample

90% single -sided confidence (z)=0.9. n

zMTTRMTTR Note: No

S= Sample Std Dev

g ( )

From standard normal table, z=1.28.

Note: No upper bound

Therefore:

min77.6475.1528.192.61 MTTR1-

50L


Confidence on Mean of Normal Distn: Population Variance Unknown and n

Student-T distribution

pdf

where is the number of degrees of freedomwhere is the number of degrees of freedom

T-table ith right with right

tail probabilities

t =1 345t0.1,14=1.345

Example 2a

Re-consider example 2 by assuming there are only 15 repair times but with the same mean and standardrepair times, but with the same mean and standard deviation (Sample MTTR = 61.92 min, sample std dev = 15.74(Sample MTTR 61.92 min, sample std dev 15.74 min)

Q: What is the value we can be sure with 90%Q: What is the value we can be sure with 90% confidence that the MTTR will not exceed?


Example 2a

A: We need 90% upper limit on the MTTR. Since n = 15 < 30 we cant assume sample std dev = population15 < 30, we can t assume sample std dev = population std dev. Instead we use

t n

tMTTRMTTR

,

where t, is the Student t distribution with =n-1 and the risk level

With =0.1, n=15, we find in the t-distribution table t0.1,14=1.345, so

min39.6715

74.15345.192.61 MTTR


Example 2: summary

The level of confidence on the mean is dependent on the l isample size

Sample size n=50, 90% UCL MTTR 64.77 min

Sample size n=15, 90% UCL MTTR 67.39 min

The larger sample size is, the closer the confidence limit is to the true mean value.

The smaller sample size is, the further away of the confidence limit is from the true mean value.


Confidence Intervals for of a N l P l i

Population known? Sample Statistic 100(1-)%

Normal Population

Population distribution

known? Sample size

Statistic 100(1 )%C.I.

N l Y n X Normal Y n

Large App o imatel App o imatel/

XZn

/ 2 /x z n

Any Y Large(n>30)

Approximately Approximately

L/

XZn

/ 2 /x z n

Any N Large(n>30)

Approximately Approximately

/XZS n

/ 2 /x z s n

Normal N Small(n

Confidence interval for Confidence interval for exponential distribution

Exponential Distribution

The exponential distribution is characterised by (f il t ) 1/ (MTTF)(failure rate) or =1/ (MTTF)

The failure rate can be estimated asTotal number of failuresTotal observation time

Equivalently MTTF can be estimated as

Total observation time

q yTotal observation time

Total number of failures

Confidence on exponential pdistribution - Time terminated (Failures + suspensions)(Failures + suspensions)

Failure data is highly skewed, so normal approximation i t lidis not valid.

Use 2 distribution to determine confidence limits. UCL on mean:

LCL on mean:2

)22,(

2

r

TMTTF LCL on mean:2

2 TMTTF

)22,( r

T = total observation time (failures + suspensions) r=number of failures during T

)22,1( r

r=number of failures during T=risk (1-level of confidence)

Example 3Example 3

A reliability test of an item is conducted for 1000 hrs and 10 failures occurred during this time. When the item failed, it was repaired and returned to operation.

The 10th failure occurred before the test was terminated at 1000 hrs.

Assume an exponential distribution (e.g. a complex system).

Q: With 90% confidence, what value will the MTTF will exceed?

A: We need to find the 90% lower confidence limit on the MTTF for a time terminated test.

Example 3 solutionExample 3 solution

Estimate of MTTF = total time/no. of failures = 1000/10 = 100 hrs= 1000/10 = 100 hrs

2 TMTTFUsing the formula

ith 1 0 9 10 T 1000 h

2)22,1(

r

MTTF

with 1-=0.9, r=10, T=1000 hrsFrom 2 distribution table, we find 2(0.9,22)=30.81( , )So

hrs916410002MTTF hrs91.6481.30

MTTF


2 distribution The pdf is (a special case of the

Gamma distribution , when =/2 and =0.5)

2/1)2/(2/ )2/(2

1)( tettf

where is the only parameter, called degree of freedom

2/ )2/(2)(f

degree of freedom

It is a distribution of sums of squares f i d d t t d d lof n independent standard normal

variables.

Used for statistical testing, goodness-of-fit and confidence intervals

Cumulative distribution values can be found in tables.

Note: we try to find the 2 value that gives the given right tail that gives the given right tail area, using

the right tail =3 area = 0.1

23, 0.1=6.25

Exponential distribution Failure terminated (No Failure terminated (No

suspensions)Single-sided confidence interval can be determined

using:

LCL on mean: 2

(1 ,2 )

2

r

TLCL

For

UCL

(1 ,2 )r

2T

double sided CI,

0 UCL on mean:

2

( ,2 )

2

r

TUCL use 0.5,

not

Where T=total period of observation = risk (1-level of confidence) risk (1 level of confidence)r = number of failures during the period of observation

Example 4Example 4

A reliability test of an item is conducted for 1000 hrs and 10 failures occurred during this time. Each time when it failed the item was repaired and returned to operation (except the last one)(except the last one)

The test was terminated after the 10th and last failure that occurred at 1000 hrs.

Assume an exponential distribution.p

Q: What is the value we can be sure with 90%Q: What is the value we can be sure with 90% confidence that the MTTF will exceed?

A W d t fi d th 90% l li it th MTTF fA: We need to find the 90% lower limit on the MTTF for a failure terminated test

Example 4 solution

Estimate of MTTF = period of observation/no. of failures

Example 4 solution

= 1000/10 = 100 hrs

2TUsing the formula: 2 )2,1(

2

r

TMTTF

with 1-=0.9, r=10, T=1000 hrsFrom 2 distribution table we find 2(0 9 20)=28 41From distribution table, we find (0.9,20) 28.41

So: hrs407010002 MTTFSo: hrs40.7041.28MTTF


C.I. for Exponential Distribution: Example 1

A sewage pump were monitored for 10000 hours, and 10failures were recorded. Every time it failed, it was repaired and restored to service (as good as new). What is the value we can be sure with 95% confidence that the MTBFvalue we can be sure with 95% confidence that the MTBF of the pump will exceed?

Solution

C.I. for Exponential Distribution: Example 2

A sewage pump were monitored for failures. Every time it failed, it was repaired and restored to service (as good as new) The monitoring finished after the 10th failure whichnew). The monitoring finished after the 10th failure, which happened at 9829 hours. What is the value we can be sure with 95% confidence that the MTBF of the pump willwith 95% confidence that the MTBF of the pump will exceed?

Solution

C.I. for Exponential Distribution The 2 distribution can be used to deriveThe distribution can be used to derive

confidence interval around an estimated MTTFMTTF

C.I. Time truncated Failure truncated Two-sided

(LCL, UCL) 2 2/ 2, 2 2 1 / 2, 2 2

2 2,n n

T T

2 2/ 2 2 1 / 2 2

2 2,n n

T T

One-sided(LCL)

/ 2, 2 1 / 2, 2n n 2

2T2

2T(LCL) 2, 2 2n , 2n

n : total number of failures Note we have been n : total number of failuresT : total observation time100(1-)% : confidence level

Note we have been using r for this

About confidence interval and the (dreaded) point estimate of MTTFp

MTTF often calculated simply as = T/r

Assumes (and is only valid for) an exponential distribution.

Why not test for the shortest possible time?

And better et e perience ero fail res And better yet experience zero failures Infinite MTTF The perfect systemThe perfect system

We all know the answers, or do we?


confidence limits

Documents

engineering confidence

confidence intervallimit

lower confidence limits

times confidence interval

x n x n

statistical intervals

bearing population

random sample of size