reliability models

7/28/2019 Reliability Models

http://slidepdf.com/reader/full/reliability-models 1/41

Software Reliability

Models



2

Reliability Models

• usage

– assessing product reliability

– estimating the number of latent defects at thetime when the product is released

• helps planning resources for maintenance

• the variable being studied

– the defect number/rate in a time interval – the time between failures



3

Expression

• the variable being studied has a form as below

y=f(x1,x2,...,xk )+e – y - dependent (studied) variable

– xi - independent variables

• attributes of the product/development process

• e.g., the complexity of the project, the team'straining level, etc.

– e - error



4

Static and Dynamic Models (1)

• static models

– the coefficients of the independent variables are

estimated based on data from previous processes

• dynamic models

– the coefficients of the independent variables are

estimated based on data collected from the

current process

• thus, the resulting model is product-specific



5

Static and Dynamic Models (2)

• static models are better suited for testinghypotheses rather than estimating reliability

•

dynamic models - 2 categories – applicable for the whole development process

• e.g., the Rayleigh model

– applicable for the testing phases

• e.g., reliability growth models



The Rayleigh Model



7

The Weibull Distribution (1)



8

The Weibull Distribution (2)

• m - shape parameter

• c - scale parameter

•

t - time – in theory, may be any independent variable

• standard distribution - the area below the probability density curve is 1

• the probability density curve approaches 0asymptotically



9

Usage

• many reliability processes evolve accordingto the Weibull distribution

– corresponding to various values of m

• examples: electron tube failures, river overflows, etc.

• software engineering

– CDF - cumulative defect arrival pattern

– PDF - defect rate (density) over time



10

The Rayleigh Model

• particular case of the Weibull distribution

– m = 2

2

ctm

• probability density grows until a moment tm,

then decreases asimptotically to 0



11

Usage in Software Engineering (1)

• in most cases, the life cycle of a software

project observes the Rayleigh distribution

–

including the defect removal pattern• in practice, the probability functions are

multiplied by a factor K

–

where K is the total number of defects – so the area below the probability density curve

is K



12

Example

• GA - General Availability phase

–

when the product is available for customers – we are interested in estimating the defect rate

during this phase



13

Assumptions

• applying the Rayleigh model in software

quality relies on two basic assumptions

1. the defect rate observed during thedevelopment process is positively

correlated with the defect rate in the field

2. if more defects are discovered during theearly phases, fewer defects remain in the

later phases



14

Assumptions (continued)

• assumption 1

– a higher rate of defects discovered means thatmore defects were introduced• if defect removal effectiveness remains the same

– thus, it is normal that more defects are revealedin the field as well

• assumption 2

– if the PDF curve reaches the maximum value atan earlier moment, then the area below thecurve is smaller in the final part (the GA phase)



15

Predictive Validity

• refers to the accuracy of the estimates made

based on the model we use

•how to achieve it – ensure the accuracy of input data

• in many projects, defect monitoring is not well

implemented in the early phases

– permanently compare the estimates to actual

results

• useful for the subsequent phases



16

Predictive Validity (continued)

• the Rayleigh model often underestimates

the defect rate in the field

•the value of parameter m may be too big if input data are incomplete

• a Weibull distribution with m less than 2

(but still close to 2) may be better suited for modeling the software process

– e.g., 1.8



Reliability Growth Models



18

Exponential Model

• particular case of the Weibull distribution

– m = 1

λt-t/c

e-1e-1tF:CDF

λt-t/c-λee

c

1tf :PDF

– =1/c - instantaneous failure rate

• the parameters to be estimated for describing the model: K (total number of

defects) and



19

Cumulative Distribution Function



20

Probability Density Function



21

Usage

• models the defect rate during the testing

phases for many software projects

•

assumption – the testing effort is homogeneous throughout all

testing phases

•

if the testing effort is homogeneous, anormalization procedure needs to be applied



22

Normalization

• analyze the number of person-hours per week allocated for testing

– compute its average value (n) for the entire

testing process• divide the testing effort into n person-hours

units

– the defect rate for the first n person-hours unitis assigned to the first week

– proceed similarly for the second unit and thesecond week and so on



23

Reliability Growth Models

• used for testing phases

• they describe the evolution of the number of

defects discovered throughout the testing phases

• they are all based on the exponential model

• the dependent (studied) variable may be – the time between failures

– the number of defect arrivals



24

Time between Failures Models

• the time between two consecutive failures

grows as defects are fixed

•

the time between failures i-1 and i follow acertain distribution

– the parameters of the distribution depend on the

number of latent defects remaining after failure

i-1



25

Jelinski-Moranda Model (1)

• assumptions

– the number of failures at the start of testing: N

–

failures occur at random – all defects have equal contributions to failures

– the time necessary for fixing a defect is

negligible

– each defect fix is perfect



26

Jelinski-Moranda Model (2)

• let us consider the interval ti

– between failures i-1 and i

•

the instantaneous failure rate functionduring interval ti is of this form

1-i- NtZ i

• - proportionality constant

• the instantaneous failure rate

– is constant between failures

– decreases by after each defect fix



27

Littlewood Model

• similar to the Jelinski-Moranda model

• but the defects have different contributions

to failures – we say the defects have different sizes

• during the testing process we expect a

decrease in – the number of defects

– the average size of the defects



28

Goel-Okumoto Imperfect Debugging

Model• also derived from the Jelinski-Moranda

model

•

but considers that defect fixes may beimperfect

– i.e., they could introduce new defects

λ 1-i p- NtZ i

• p - the probability of imperfect debugging

• - the failure rate per defect



29

Fault Count Models

• the number of defects in a time interval

decreases as defects are fixed

•

the time interval used may be – CPU time

– calendar: hour, week, month, etc.

•

the parameter to estimate - the number of remaining defects



30

Goel-Okumoto Nonhomogeneous

Poisson Process Model (1)• the number N(t) of defects observed by time

t is modelled as a nonhomogeneous Poisson

process• the failure rate depends on time, following

an exponential distribution

0,1,2..y,ey!

tm

yt NPtm-

y

-bte-1atm

-btabetm'tλ



31


Poisson Process Model (2)• parameters

– m(t) - the expected number of failures observed

by time t

– (t) - failure density

– a - the expected total number of failures to be

observed

– b - defect detection rate



32


Poisson Process Model (3)• the model is an application of the

exponential model

–

m(t) corresponds to the cumulative distributionF(t)

– (t) corresponds to the density probability f(t)

– parameters a and b correspond to parameters K

and , respectively

• parameter a is treated as a random variable



33

Musa-Okumoto Model

• similar to the previous model

• the later a defect is fixed, the smaller effect

it has on the reliability of the software – it is thus a logarithmic Poisson process

1θλ lnθ

1tm t0

– 0 - initial failure intensity

– - rate of reduction of failure intensity



34

Delayed S Model

• considers the time necessary for failureanalysis

– the curve of the cumulated number of defects is

S-shaped• the maximum value is not reached at start, as in the

exponential model

– still a nonhomogeneous Poisson process

t-eλt-1-1K tm

– - error detection rate

– K - total number of defects



35

Inflection S Model

• errors are no longer considered as

independent

•

the more failures are detected, the moreundetected failures become detectable

– again, a nonhomogeneous Poisson process

t-

t-

ie1

e-1K tm

– i - inflection factor



36

Model Evaluation Criteria

• predictive validity

– the ability to predict the failure behavior or thenumber of defects based on currently available

data

• simplicity

– collecting data - simple and inexpensive

–

the concept does not require extensivemathematical background

– easy to implement by computer programs



37

Model Evaluation Criteria (cont'd)

• quality of assumptions

– the assumptions are plausible and likely to bemet in practice

• capability

– the ability to estimate quantities that are usefulfor managers, software engineers, users, etc.

•applicability – the model can be equally applied on projects

that differ in size, structure, functions, etc.



38

Reliability Modeling Process (1)

• data examination

– the kind of available data: number of defects vs.

time between failures

– the time unit that is used (CPU time, week, etc.)

– data reliability

• select one or several models that fit the

available data

• estimate the parameters of the model



39

Reliability Modeling Process (2)

• test to which extent the model (with theestimated parameters) fits the data

– if the model does not fit the data, another model

must be chosen

• make reliability predictions based on themodel

–

predictions are assessed based on other information

– e.g., through the analysis of similar products



40

• defect arrival patterns differ between the

testing phases and the maintenance phase

–

testing phases are specifically focused onfinding defects

– in the field, defects arise more rarely

• thus, defect density is different

• such diference is called compression factor

Test Compression Factor



41

Test Compression Factor (continued)

• the value of the compression factor may

vary depending on

–

the testing environment – the customer usage profile

• if the software is more complex, defects

tend to arise later

– so the curve of the probability density with

respect to time exhibits a slower evolution

reliability models

Documents