chapter 2 literature review - inflibnetshodhganga.inflibnet.ac.in/bitstream/10603/45303/6/06_chapter...

A Study of Discrete and Continuous Software Reliability Growth Models

Department of Mathematics, DNPG College, Gulaothi, Bulandshahar, CCS University, Meerut 28

Chapter 2

Literature Review

The Software Reliability Growth Model (SRGM) is the tool, which can be used to evaluate

the software quantitatively, develop test status, schedule status and monitor the changes in

reliability performance. There have been many Software Reliability models developed in

the last two decades. Most of these are based upon historical failure data collected during

the testing phase. These models have been utilized to evaluate the quality of the software

and for future reliability predictions. They have further been used in many management

decision-making problems that occur during the testing phase. But none of these models



can claim to be the best and hence there is a need for further research. In this section

different modelling approaches are briefly discussed.

2.1 Classification Scheme

The software reliability models are broadly classified into black box (single system) and

white box (multi component software) models. The black box models are classified

further into models based on

Inter Failure Times,

Failure Count and

Static Models.

Markov assumptions are also made for some models and they are overlapping with the

Failure Count and Inter Failure Time Modelling. All known models can be extended to be

a Bayesian model. If the model is Bayesian then we are estimating the model parameters

using Bayesian techniques. White box models are those that model modular software

systems considering the architecture of the system.

Popstajanova and Trivedi (Popstajanova and Trivedi 2001) presented a classification of

software reliability models that classified multi component software systems as stated in

Figure 2.1.

The white box models are broadly classified into State Based, Path Based and Additive

Models. The State Based Models are further classified into Continuous Time Markov

Chains (CTMC), Discrete Time Markov Chain (DTMC) and semi-Markov models.

All these Markov models fall into either absorbing or irreducible class of Markov chains.

Absorbing type represents terminating software and continuously operating system is

represented by irreducible chain.



Figure 2.1 : Classification Scheme

Software Reliability Models

Black Box Models

(Single system)

White Box Models(Multi Components)

Type I State Based Models

Failure rate models

Type IIBayesianModels

Static Models

Fault Seeding

Input Domain Models

Markov Reliability Models

DTMC (Discrete)

CTMC (Continuous)

Semi Markov

Absorbino

Irreducible

Composite

Hierarchical

Path BasedModels

Additive Models


Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 32

Bayesian models are extensions of Software Reliability Growth Models. In Bayesian models the

stochastic model reflects the system accurately at that instant based on the latest failure data. In

Bayesian models the distributions representing the failure behaviour is represented by a prior and

posterior distribution. The prior distribution reflects the expert knowledge available on the

specific failure process, which is incorporated in the stochastic model. Bayesian models have

wide acceptance due to these reasons in the software industry. In the following sections we

describe each class of software reliability models with a representative reliability model.

Single System Software

This model considers the whole software as a single monolithic system and the structure of the

model is not considered in the process of reliability estimation of the software system. The

reliability is solely estimated based on the failure history. Popular reliability estimation models

include the Goel-Okumoto (Goel and Okumoto 1979) model, and the Jelinski-Moranda model

(Jelinski and Moranda 1972). Thus, for example, these models do not account for the structure of

the software. Single system software failure behaviour is usually modelled using Software

Reliability Growth Model and classified as times between failure and failure count models.

Failure Rate Models

Inter failure times are the main modelling parameter for these models. Generally it is expected

that the time for next failure increase as more bugs are detected and corrected. This may not be

always correct as inter failure times are random variables and subjected to statistical fluctuations.

The failure rate is a modelling tool for type I models. The reliability function is a non-decreasing

function of mission time. This demonstrates an increase in the software credibility.

Markovian Models

When a markov process represents the failure process, the resultant model is called Markovian

model. The software can attain many states at any particular time with respect to number of

faults remaining or number of faults already removed. The transition between states depends on

the current state of the software and the transition probability. The memory less property of the

Markovian process implies that the time between failures follows an exponential distribution.

Numerous attempts have been made to develop Markovian models; especially in earlier days due



to similar theory in hardware reliability was already well developed. One of the popular

reliability models developed by Jelinski and Moranda (Jelinski and Moranda 1972) is a Markov

process model. Littlewood (Littlewood 1973, Littlewood 1979, Littlewood 1987) proposed a

model based on semi-Markovian process to describe the failure phenomenon of software with

module structure. Cheung (Cheung 1980) has also proposed a Markovian model to describe

module-structured software. Kremer (Kremer 1983) proposed birth-death model, which

incorporates the probabilities of fault removal and introduction. Goel (Goel 1985) modified

Jelinski-Moranda model by introducing the concept of imperfect debugging. Kapur et al. (Kapur,

Garg and Kumar 1999) proposed a model incorporating imperfect debugging and fault

introduction with upper bound on the number of faults.

Fault Counting Models

This category includes the models, which describe the failure phenomenon by stochastic

processes like Homogeneous Poisson Process (HPP), Non-Homogeneous Poisson Process

(NHPP), and Compound Poisson Process (CPP) etc. The majority of these failure count models

are based upon the NHPP. Schneidewind (Schneidewind 1975) proposed a fault detection model

based on NHPP. Different NHPP models are distinguished by their unique mean value functions.

Yamada et al. (Yamada, Ohba and Osaki 1983) proposed the delayed S-Shaped model. Ohba

(Ohba 1984) proposed the inflection S-Shaped model. Musa et al. (Musa, Iannino and Okumoto

1987) have proposed the basic execution time model and Log-Poisson model. Goel (Goel 1985)

modified Musa’s original model by introducing the test quality parameter. Yamada et al.

(Yamada, Osaki and Narithisa 1985) also proposed a discrete time model. The effect of testing

effort on failure process was taken into consideration by Yamada et al. (Yamada, Ohtera and

Narithisa 1986). Kapur et. al (Kapur, Garg and Kumar 1999) modified Goel-Okumoto (Goel and

Okumoto 1979) model by introducing the concept of imperfect debugging. Kareer et al. (Kareer

1990) proposed a model with two types of faults where each fault type is modelled as by an S-

Shaped curve model. Xie and Zhao (Xie and Zhao 1992) have illustrated how the Schneidewind

model can be modified to result in many of the above SRGMs. Zeephongsekul et al.

(Zeephongsekul P, Xia G and Kumar S 1993) proposed a model describing the case when a

primary fault introduces secondary faults. Attempts as listed above for new models were made

with the primary intention of getting flexible models that could describe a range of failure count



curves or reliability growth curves like exponential curves and highly S-Shaped curves. Models

with such property are termed as flexible SRGMs (Kapur, Garg and Kumar 1999). Recently

efforts have been directed towards development of general SRGMs (Kapur, Garg and Kumar

1999). General SRGMs are flexible models and many of the above models can be derived from

them. In this thesis new NHPP based SRGMs are proposed and the theory and methodologies

developed in the rest of the thesis are based upon NHPP models. Therefore later in this chapter

we briefly discuss the Non-Homogeneous Poisson Process and some of the above SRGMs.

Recently reliability modelling for distributed development environment has caught the attention

of many researchers. Large software systems have modular design. A system is said to be

modular when each activity of the system is performed by exactly one component, and when the

inputs and outputs of each component are well defined (Lyu 1996). Often different development

teams develop such components of software separately. With availability of communication

networks at cheaper rates some software components are developed at separate geographic

locations also. Software developed under this distributed development environment has proved

to be economical. Many a time components from other software projects are also reused.

Therefore SRGMs for software developed under distributed environment needs to have different

approach. But very few attempts have been made in this regard (Yamada, Tamura and Kimura

2000). Most of the models proposed in the thesis have been developed under distributed

development environment, which assumes that the software system consists of a finite number of

reused and newly developed components.

Models based on Bayesian Analysis

In the previous two categories the unknown parameters of the models are estimated either by the

least squares method or by the maximum likelihood method (later in this chapter both these

methods are briefly discussed). But in this category of models, the Bayesian analysis technique is

used to estimate the unknown parameters of the models. This technique facilitates the use of

information obtained by developing similar software projects. Based on this information the

parameter of given model are assumed to follow some distribution (known as priori distribution).

Given the software test data and based on a priori distribution, a posterior distribution can be

obtained which in turn describes the failure phenomenon. Littlewood and Verral proposed the



first software reliability model based on Bayesian Analysis (Xie 1993). Singpurwalla

(Singpurwalla 1995, Singpurwalla 1999) have proposed a number of Bayesian software

reliability models for different testing environments.

The following four categories of Software Reliability models do not make any dynamic

assumptions of the failure process. These are defined very briefly here.

Static Models

These models use statistical techniques to evaluate software reliability and can be evaluated only

if complete failure data is available. These models were developed in the initial stage of

reliability model evolution and are now seldom used, as it cannot incorporate the structure of the

software.

The Input Domain and Fault Seeding Models

In fault seeding models, a known number of bugs are seeded (planted) in the program. The

software is then tested for faults. The bugs detected would have a combination of inherent faults

and seeded faults. The number of inherent faults in the software is calculated based on the

inherent and seeded faults detected using maximum likelihood estimation and combinatory. The

drawback of this approach is that the seeded faults and the inherent faults must have the same

detection probability. This is difficult to achieve. The basic approach in input domain based

model is to generate a set of test cases from an input distribution. The reliability measure is

calculated from the number of failures observed during symbolic or physical execution of

sampled test cases. The test cases selected from the representative input space are executed

recording the results. The probability of the success can be evaluated using statistical techniques.

It is generally difficult to estimate the input distribution (operational profile); generally the input

distribution is obtained based on the different paths that exist in the software.

Software Metrics Models

The models in this class relate the fault content in the software to some features of the software

program such as program length, complexity, volume etc. These models are empirically built and



the result obtained by a model is dependent on the software development process environment,

which may not be the same in the other projects.

Models other than those falling under above categories exist in the literature and more are being

proposed. A number of review papers have been written, to name among others Littlewood

(Littlewood 1973), Goel (Goel 1985), Xie (Xie 1991), Kapur et al. (Kapur, Garg and Kumar

1999). In the following section we briefly describe some Software Reliability Growth models

based upon NHPP assumptions. Estimation procedures and Criteria for comparison among

Software Reliability Growth Models are also discussed.

Multi Component Models

White Box model or Multi component system considers the Software Architecture of the system,

its interactions with the different modules. White box models are used to model component

based software. Such models seek to explicitly incorporate the testing method used during the

testing phase, as well as the structure of the software being tested. Modular systems have a

general framework in which the system reliability is calculated. All modular systems are grouped

as State Based, Path Based and Additive models.

State Based Models

Architecture based models assume that components fail independently and that a component

failure will ultimately lead to a system failure. Unlike hardware reliability every component is

always in use, software components need a utilization factor in State Based Models. State Based

Models are generally modelled using Markov models like CTMC, DTMC or semi Markov

models. System reliability estimates are obtained using both architecture and failure model. This

is achieved using two methods, Composite and Hierarchical solution approach.

Composite-Hierarchical Approach

The State Based Models are further classified into composite and hierarchical based on the

solution approach to obtain the reliability of the system. Composite method combines the

architecture model with the failure model and then solved for reliability prediction. If the

architecture model is first solved first and then superimposed on the failure behaviour on the

architecture model solution to predict reliability.



Path Based Models

Similar to State Based Models, the Path Based Models consider software architecture with

components and interfaces. Initially the different paths in system are obtained

either experimentally or algorithmically. Path reliability is the product of all component

reliabilities along the path. The system reliability is average of all the path reliabilities. State

based models analytically account for the infinite loops in a path but path-based models

terminate the loop to one or to an average execution time of the path. Shooman Model considers

reliability of modular software introducing the path based approach by using the frequencies

with which different paths are run. Krishnamurthy and Mathur developed a method to combine

architecture and failure process by estimating the path reliabilities based on the sequence of

components executed for a single test run and the average over all test runs to obtain the system

reliability. State Based Models are an important category of models for modular systems.

Additive Models

Additive models consider software testing phase and each component reliability is modeled by

NHPP. This implies system failure rate is also NHPP with cumulative number of failures and

failure intensity functions that are the sums of corresponding function of each component.

Additive model do not consider architecture of the software. Xie & Wohlin (Xie & Wohlin 1995)

developed an additive based architecture model. Semi Markov and Markov regenerative models

attempt to relax the Markovian assumption of exponential failure and repair times description, in

a restrictive manner. They are also exposed to the state space explosion problem. Discrete-event

simulation on the other hand offers an attractive alternative to analytical models as it can capture

a detailed system structure and facilitate the study of influence of various factors such as

reliability growth, various repair policies. All software development teams have to answer an

obvious question of when to stop testing and release the software, whether the software is single

component or multi-component.



2.2 NHPP based Software Reliability Growth Modeling

Stochastic modeling has been to develop models to represent the real system and analyze their

operation since years. There are two main types of stochastic process: continuous and discrete.

Among the class of discrete process, counting process is used in reliability engineering widely to

describe the occurrence of an event of time (e.g. failure, repair, etc.). Poisson process is used

most widely to describe a counting process as in reliability engineering. NHPP has been used

successfully in hardware reliability analysis to describe the reliability growth and deteriorating

trends. Following the trends in hardware reliability analysis many researchers proposed and

validated several NHPP based SRGM. SRGM describe the failure occurrence and\or failure

removal phenomenon with respect to time (CPU time, calendar time or execution time or test

cases as unit of time) and\or resources spent on testing and debugging during testing and

operational phases of the software development.

NHPP based SRGM are broadly classified into two categories first – continuous time models,

which uses time (CPU time, calendar time or execution time) as a unit of fault detection period

and second – Discrete time models, which adopt the number of test occasions/cases as a unit of

fault detection period. Models can also be categorized as concave and S-shaped depending upon

the shape of the failure curve described by them. Concave models describe an exponential failure

curve while second category of models describes an S-shaped failure curve [Kapur et al, (1999);

Pham, (2006)]. The two types of failure growth curves are shown in figure 2.2 and 2.3. The

most important property of these models is that they have the same asymptotic behavior in the

sense that the fault detection rate decreases as the number of detected defects increases and

approaches a finite value asymptotically. The S-shaped curve describes the early testing process

to be less efficient as compared to the later testing i.e. it depicts the learning phenomenon

observed during testing and debugging process.



Figure 2.2 Exponential Failure Curve Figure 2.3 S-Shaped Failure Curve

During the last three decades several researchers devoted their research interest to NHPP based

software reliability modeling and contributed significantly in understanding the testing and

debugging process and developing quality software. Schneidewind, (1975) made the preliminary

attempt in NHPP based software reliability modeling. He assumed exponentially decaying failure

intensity and rate of fault correction proportional to the number of faults to be corrected. Goel

and Okumoto, (1979) presented a reliability model (GO model); assuming hazard rate is

proportional to the remaining number of faults in the software. This research paper was a

pioneering attempt in the field of software reliability growth modeling and paved the way for

research on NHPP based software reliability modeling. The model describes the failure

occurrence phenomenon by an exponential curve. The research following GO model was mainly

modifying the existing research in the way of incorporating the various aspects of the real testing

environment and strategy.

In the upcoming sections we briefly discuss some of the most widely known/preliminary

attempts in some of the above modeling categories.

2.3 General Description of a Non-Homogeneous Poisson Process

2.3.1 NHPP in Continuous Time Space

A counting process ( ( ), 0)N t t is said to be an NHPP with mean value function m(t), if it

satisfies the following conditions:



1. There are no failures experienced at t=0, that is, 0)0(N .

2. The counting process has independent increments, i.e. for any finite collection of times

t1<t2<�<tk, the k random variables N(t1), N(t2)-N(t1),…, N(tk)-N(tk-1) are independent.

3. Pr(exactly one failure in ( , )) ( ) ( )t t t t o t

4. Pr(two or more failures in ( , )) ( )t t t o t

Where ( )t is intensity function of ( )N t . If we let 0

( ) ( )t

m t x dx them m(t) is a non-

decreasing, bounded function representing the mean of number of faults removed in the time

interval (0, ]t [Kapur et al, (1999)]. It can be shown that

- ( )( )Pr[ ( ) ]

!

k m tm t eN t k

k, 0,1, 2,....n

i.e. ( )N t has a Poisson distribution with expected value [ ( )] ( )E N t m t for 0t .

and the reliability of the software in the time interval of length x is given as

- ( ) ( )( | ) m t x m tR x t e

In case we have n, the number of test cases as the unit of time then t is replaced by n and t is

replaced by 1, and describes NHPP based SRGM in discrete time space.

2.3.2 NHPP in Discrete Time Space

A discrete counting process[ ( ), 0]N n n , ( 0,1, 2,...)n is said to be an NHPP with mean value

function ( )m n , if it satisfies the following two conditions

1. No failures are experienced at n=0, that is, (0) 0N .

2. The counting process has independent increments, implies the number of failures

experienced during thth , 1n n test cases is independent of the history. The state ( 1)m n



of the process depends only on the present state ( )m n and is independent of its past state ( )m x ,

for x n .

For any two test cases in and jn where (0 )i jn n , we have

( ) ( )Pr{ ( ) ( ) } exp ( ) ( )

!

xj i

j i j im n m n

N n N n x m n m nx

, x = 0,1,2,

The mean value function )(nm which is a non-decreasing in n represents the expected

cumulative number of faults detected by n test cases. Then the NHPP model is formulated as

( )( )Pr{ ( ) } exp

!

xm nm n

N n xx

Let ( )N n denote the number of faults remaining in the system after execution of the thn test run.

Then we have

( ) ( ) ( )N n N N n

where, ( )N represents the total initial fault content of the software.

The expected value of ( )N n is given by

( ) ( ) ( )E n m m n

where, ( )m represents the expected number of faults to be eventually detected.

Suppose that dn faults have been detected by the execution of n test cases. The conditional

distribution of ( )N n , given that ( ) dN n n , is given by

( )( )Pr{ ( ) | ( ) } exp

!

yE n

dE n

N n y N n ny

and the probability of no faults detected between the thn and th( )n h test cases, given that dn

faults have been detected by n test cases i.e. software reliability, is given by



( / ) exp ( ) ( )R h n m n h m n ,

2.4 NHPP based SRGM: Literature Survey

A large number of SRGMs have been developed in the literature to describe the failure and fault

removal phenomenon of software in continuous and discrete time space. In this section we

introduce briefly some of the existing SRGM in the literature

Notations

m(t) : expected number of failure/removal by time t, m(0) = 0.

mf(t) : expected number of failure by time t, mf(0) = 0.

mr(t) : expected number of removals by time t, mr(0)=0.

mi(t) : expected number of failure due to independent faults in time t, mi(0) = 0.

md(t) : expected number of failure due to dependent faults by time t, md(0)=0.

a : number of fault in the software at the time of start of software testing.

a(t) : expected initial fault content at time t, a>0.

p : probability of perfect debugging of a fault, 0<p<1.

b(t) : time dependent rate of fault removal per remaining faults.

, c : constants

2.4.1 Modeling Under Perfect Debugging Environment

The earlier attempts in NHPP based reliability growth modeling assumed a perfect debugging

environment. Which means whenever an attempt is made to remove a detected fault it is removed

perfectly and no new faults are generated. Various models have been proposed in this category.

The earliest model was due to Goel and Okumoto, (1979).



The Exponential SRGM [Goel and Okumoto, (1979)]

The model is based on the following assumptions

1. Failure observation phenomenon is modeled by NHPP.

2. Software is subject to failures during execution caused by faults remaining in the software.

3. Each time a failure is observed, an immediate effort takes place to isolate and remove the

cause of the failure.

4. All faults in the software are mutually independent.

Under the assumptions 1- 4 the differential equation describing the GO model is

( ) [ ( )]dm t dt b a m t (2.1)

solving (2.1) with the initial condition m(0)=0 we get

( ) (1 )btm t a e (2.2)

The model is known as exponential NHPP model as it describes an exponential failure curve.

GO model has been applied to a variety of testing environment in practice. Following, the

general assumptions of GO model other early exponential SRGM are proposed by Obha, (1984)

and Yamada and Osaki, (1985). Obha assumed that the software consists of a number of

independent modules whereas Yamada and Osaki assumed there are two types of faults in the

software. Both these models describe the failure phenomenon for each module\fault type by GO

model with different parameters and the mean value function is the sum of mean value function

for each module\fault type. Musa, (1975) and Musa and Okumoto (1984) assumed failure

intensity to decay exponentially in the basic and Logarithmic Poisson execution time models

respectively. Yamada et al, 1983 refined the GO model describing testing as a two-stage process

namely fault detection and removal. The model describes an S-shaped failure curve. SRGM

proposed by Obha, (1984a); Bittanti et al, (1988) and Kapur and Garg, (1992) are similar to

Yamada et al, (1983) but are developed under different set of assumptions. These models can

describe both exponential and S-shaped growth curves depending on the parameter values and



therefore are termed as flexible models or S-shaped models. Some of these models are discussed

here.

Yamada Delayed S-shaped Model [Yamada et al, 1983]

This model defines the testing as a two-stage process - failure observation and the corresponding

fault removal phenomenon.

( ) ( ( ))f fdm t dt b a m t (2.3)

( ) ( ( ) ( ))r f rdm t dt b m t m t (2.4)

solution of above differential equations under initial condition mf(0)= mr(0)= 0, is

( ) (1 )btfm t a e (2.5)

( ) (1 (1 ) )btrm t a bt e (2.6)

Model can be derived alternatively in single stage if we assume 2( ) (1 )b t bt as the fault detection

rate [Kapur et al, (1999, 2011)].

SRGM for Error Removal Phenomenon [Kapur and Garg, (1992)]

This model is based on the assumption that the debugging team can also remove some additional

faults while removing some faults that have caused a failure without these faults causing any

failure. The faults identified on a failure are termed as independent faults while the faults

removed additionally are termed as dependent faults.

( )( ) ( ( )) ( )r

r r rm td m t p a m t q a m t

dt a (2.7)

The mean value function obtained under the initial condition mr (0) = 0 is



( )

( )1

( )1

p q tr p q t

em t aq p e

(2.8)

If c=0 the model reduces to GO model and failure phenomenon is same as removal else the

failure phenomenon is given as

( )0( ) ( ) ln

tf r p q t

ap p qm t p a m x dxq p qe

(2.9)

The model can be derived alternatively if we assume logistic learning fault detection rate given

by ( ) 1 btb t b e with b=(p+q) and =(q/p).

2.4.2 Reliability Growth Modeling with Testing Effort

Most of these earlier SRGM use calendar (execution) time as the unit of fault detection/removal

period and either assume that the testing effort consumption (TEC) rate is constant, or do not

explicitly consider the testing efforts (TE) and its effectiveness. The achieved reliability during

testing phase is highly related to the amount of development resources (test efforts) spent on

detecting and correcting latent software faults. The TE are measured by the manpower spent

during the testing phase, the number of CPU hours, the number of executed test cases, and so on.

A testing effort function describes the distribution of TE (CPU time, manpower, etc) during the

testing period. Putnam et al (1978), Yamada et al (1983,1985,1991,1993), Kuo et al (2001),

Kapur et al (2004b, 2004c), Bokhari and Ahmad (2006) and Huang et al (2002,2007) proposed

SRGM describing the relationship among the testing time, TEC and the number of software

faults detected. Most existing SRGM belong to exponential-type models. Haung et al (2007)

proposed S-shaped TE dependent SRGM based on Yamada delayed S-shaped model. In the

literature the time-dependent behavior of test-effort function is described by Weibull, Rayleigh,

exponential, Gamma, logistic, log logistic curves etc.

The testing effort (resources) that govern the pace of testing for almost all the software projects

are (Musa, Iannino and Okumoto 1987):

1. Manpower that includes



i) Failure identification personnel

ii) Failure correction personnel.

2. Computer time

These resources very closely interact during the system test phase. The function of the failure

identification personnel is to run test cases and compare the test results against program

requirement to establish the failures that have occurred. The failure correction personnel are the

debuggers or developers available to repair the software. The Computer facility represents the

computer(s) necessary for the failure identification personnel and failure correction personnel to

do their tasks. Computer time is the measure that is used for allocating computer resources.

SRGMs have been proposed in the Software Reliability Engineering literature under sets of

assumptions and for different testing environments. But most of them do not consider the

consumption pattern of resources such as computer time and manpower during testing. In this

thesis some new SRGMs incorporating testing effort have been proposed.

As the time increases, testing effort also increases. If testing time becomes quite large, testing

effort also becomes quite large. The testing effort rate is proportional to the testing resources

available i.e.,

)()()( tWtv

dttdW

(2.10)

where is the total testing resource consumption and )(tv is the time dependent rate at which

testing resources are consumed, applied to remaining available resources. Solving equation

(2.10) under the initial condition 0)0(W we get:

0

( )

( ) 1

t

v x d x

W t e (2.11)

Though various types of testing effort function like Exponentials, Rayleigh, Weibull and Logistic

have been studied in the literature. In this thesis, we have used weibull function to model the

testing resources modeling. The detailed description of these effort functions is given in chapter

6.



Model using Testing Effort Function (Yamada et al 1983)

Under the assumptions described above and using equation (2.11), the fault removal process can

be described by the following differential equation:

tmatbtw

dttmd

)(

)(

(2.12)

Using the different forms of b(t) several SRGMs can be formulated

Let the fault detection rate function is defined as:

b t b (2.13)

Here w(t) is the current testing effort consumption at time t.

.1 b W tm W t a e (2.14)

The above model is known as GO Model with respect to Testing Effort Function Yamada et al

(1983)

2.5 Multi Up-gradations of Software

The present software development environment is very competitive and advanced. Many

independent and well established developing organizations are competing in the market with

similar products and capabilities to attain the maximum market share and brand value. As such

software delivered with full functionalities and very high reliability built over a period of time

may turn out to be unsuccessful due to technological obsolescence. Therefore now a day’s the

software are rather developed in multiple releases where the latest releases might be developed

by improving the existing functionality and revisions, increasing the functionality, a combination

of both or improving the quality of the software in terms of reliability etc. For example we can

see the various software in the market named as Windows 98, Windows 2000, Windows ME,

Windows XP, Windows Vista, Windows 7 etc. For another illustration consider a development

firm developing antivirus software. Such a firm can begin with releasing the product that detects

and remove viruses and spywares from the computer system. In their second release they can

provide the feature of protecting the system from virus infected emails. Next, they can add the



trait of blocking spyware automatically for the next release. Finally, the fourth release can

provide the root kit protection along with removing hidden threats in the operating system.

This step by step release (base software with features enhancement) is advantageous for the

developing firms in various contexts. Firstly, if a firm implements the complete characteristic

capabilities in first release, than that would delay the product release in the market in the desired

time window. Secondly, launching of new software product may bring the developing firm in

limelight, but the stream of subsequent product releases is the source of their bread and butter.

Moreover releasing different versions of the product lengthen the market life of product, protect

competitive advantages and sustain crucial maintenance revenue streams.

Software products aren’t static and each release has a limited existence. As soon as a software

product reaches the market, a variety of factors begin creating demand for changes (Figure 2.4 ).

Defects require repairs. Competitors offer software with added features and functions. Evolving

technology requires upgrades to support new hardware and updated versions of operating

software. Sales demands new features to win over prospects. Customers want new functionality

to justify maintenance expenditures. These demands accumulate over time, eventually reaching

the point when the software product must be upgraded with a new version to remain viable in the

market. As soon as the new version is released, the cycle begins again.

For software developing organizations it is not an easy task to design software in isolation.

Developing reliable software is one of the most difficult problems faced by them. Timely

development, resource limitations, and unrealistic requirements can all negatively impact

software reliability. Moreover, there is some interdependence between their developments. The

interdependence between their developments exists in many ways, which also affects their

reliability. A new release (an upgraded version of the base software) may come into existence

even during its development, at the time of release or during its operation. The code and other

documents related to a release may be some modification of the existing code and documents

and/or addition of new modules and related modification in the documents. The dependence of

the development process of successive releases necessitates considering this dependence in the

reliability growth analysis.



Figure 2.4: Necessity for Up-gradations

Not much work has been done in the modeling framework for multi releases failure process. In

the earlier work Kapur et al (2007) proposed initial development in this field by proposing

multiple releases of a software in operational phase for product and project type software. Later

Kapur et al (2010) also described the phenomenon specially for testing phase by reformulating

modeling methodology that shows the dependence of operational phase of the previous release

with testing phase of the current release. In this thesis we have formulated SRGMs applicable to

describe the failure\removal phenomenon of multiple releases of software in testing phase and

also considered its dependency with operational phase wherein the expected number of failure

and consequently the faults removed in testing phase is modeled by the joint effect of

failure\removal phenomenon of successive releases.

2.6 Parameter Estimation

The success of mathematical modeling approach to reliability evaluation depends heavily upon

quality of failure data collected. The parameters of the SRGMs are estimated based upon these

data. Hence efforts should be made to make the data collection more explicit and scientific.

Usually data is collected in one of the following two ways. In the first case the times between



successive failures are recorded. Though this type of data collection is more desirable, it may not

be simple. Complication can arise in measuring the testing effort for each fault and it may not be

very convenient to note the time at each failure report. The other easier and commonly collected

data type is known as the grouped data. Here testing intervals are specified and number of

failures experienced during each such interval is noted. The proposed models presented in the

thesis are non-linear and presents extra problems in estimating the parameters.

Technically, it is more difficult to find the solution for non-linear models using Least Square

method and requires numerical algorithms to solve it. Statistical software packages such as SPSS

helps to overcome this problem. SPSS is a Statistical Package for Social Sciences. It is a

comprehensive and flexible statistical analysis and data management system. SPSS can take data

from almost any type of file and use them to generate tabulated reports, charts, and plots of

distributions and trends, descriptive statistics, and conduct complex statistical analysis.

SPSS Regression Models enables the user to apply more sophisticated models to the data using

its wide range of nonlinear regression models. For the estimation of the parameters of the

proposed models, Method of Least Square (Non Linear Regression method) has been used. Non

Linear Regression is a method of finding a nonlinear model of the relationship between the

dependent variable and a set of independent variables. Unlike traditional linear regression, which

is restricted to estimating linear models, nonlinear regression can estimate models with arbitrary

relationships between independent and dependent variables.

2.6.1 METHOD OF LEAST SQUARES

In this method the square of the difference between observed response and value predicted by the

model is minimized. If the expected value of the response variable is given by )(ˆ tm (can be a

mean value function of an SRGM), then the least square estimators of the parameters of the

model may be obtained from n pairs of sample values (t1, y1), (t2, y2), …, (tn, yn) by minimizing J

given by

n

ii tmyJ

1

2)(ˆ (2.15)



ti and yi observed values of explanatory and dependent variables respectively. For small and

medium size samples least square estimation is preferred (Musa, Iannino and Okumoto 1987).

The method has also been applied for estimating parameter of testing effort curves.

2.6.2 MAXIMUM LIKELIHOOD ESTIMATION

Maximum Likelihood Estimation (MLE) method has been extensively adopted for estimation of

parameters of SRGMs based upon NHPP (Musa, Iannino and Okumoto 1987). We briefly

discuss below the MLE procedure for two types of software failure data discussed above.

For the first type of data, suppose that estimation is to be performed at a specified time tk, not

necessarily corresponding to a failure, and with total of mk failures being experienced at time

kmttt ,...,, 21 . Then the likelihood function for the NHPP (Musa, Iannino and Okumoto 1987)

discussed above is:

ktdxxk

ii etL 0

)(

1

)( (2.16)

The MLE of the Parameters can be obtained by maximizing Likelihood function or its Log

likelihood function (log L).

If the software failure data is grouped into k points ),( ii yt ; i = 1,2,…,k, where yi is the

cumulative number of failure reports at time ti. Then the Likelihood function L is given as

follows:

k

i

tmtm

ii

yyii ii

ii

eyytmtmL

1

)()(

1

1 1

1

)!(

)()( (2.17)

Taking natural logarithm of (2.17) we get the loglikelihood function

k

iii

k

ikiiii yytmtmtmyyLogL

11

111 ])!ln[()()]()(ln[)( (2.18)



The MLE of the parameters of SRGM can be obtained by maximizing (2.18) with respect to the

model parameters.

Likelihood functions for NHPP models are defined above as most of the stochastic models

discussed in this thesis are based upon NHPP assumptions. Both the estimation procedures can

also be applied to other stochastic processes. Maximum likelihood estimators possess many

desirable properties such as consistency, efficiency, asymptotic normality and the invariance

property (Musa, Iannino and Okumoto 1987). Hence it is the most preferred estimation

procedure for relatively large sample size.

2.7 Comparison Criteria for SRGMs

The performance of SRGMs are judged by their ability to fit the past software fault data

(goodness of fit) and to predict satisfactorily the future behaviour of the software fault removal

process (predictive validity). Kapur, Garg and Kumar (1999), Musa (1987, 1999) have suggested

the following attributes for choosing an SRGM.

Capability: The model should possess the ability to estimate with satisfactory accuracy metrics

needed by the software managers.

Quality of assumptions: The assumptions should be plausible and must depict the testing

environment.

Applicability: A model can be adjudged as the better one if it can be applied across software

products of different sizes, structures, platforms and functionalities.

Simplicity: The data required for an ideal SRGM should be simple and inexpensive to collect.

The parameter estimation should not be too complex and is easy to understand and apply even

for persons without extensive mathematical background.

Other than the above qualitative aspects the following indices help in comparing SRGMs.



2.7.1 GOODNESS OF FIT CRITERIA

The term goodness of fit is used in two different contexts. In one context, it denotes the question

if a sample of data came from a population with a specific distribution. In another context, it

denotes the question of “How good does a mathematical model (for example a linear regression

model) fit to the data”?

1. The Mean Square Fitting Error (MSE): The model under comparison is used to

simulate the fault data, the difference between the expected values, )(ˆ itm and the observed data

yi is measured by MSE as follows.

k

i

iik

ytmMSE1

2))(ˆ( (2.19)

where k is the number of observations. The lower MSE indicates less fitting error, thus better

goodness of fit (Kapur, Garg and Kumar 1999).

2. Coefficient of Multiple Determination (R2) This Goodness-of-fit measure can be used

to investigate whether a significant trend exists in the observed failure intensity. We define this

coefficient as the ratio of the Sum of Squares (SS) resulting from the trend model to that from a

constant model subtracted from 1, that is

SScorrectedSSresidualR 12 (2.20)

R2 measures the percentage of the total variation about the mean accounted for by the fitted

curve. It ranges in value from 0 to 1. Small values indicate that the model does not fit the data

well. The larger, the better the model explains the variation in the data (Kapur, Garg and Kumar

1999).

3. Prediction Error (PE): The difference between the observation and prediction of

number of failures at any instant of time i is known as PEi. Lower the value of Prediction Error

better is the goodness of fit (Pillai and Nair 1997).

4. Bias: The average of PEs is known as bias. Lower the value of Bias better is the

goodness of fit (Pillai and Nair 1997).



5. Variation: The standard deviation of PE is known as variation.

2

11 BiasPENVariation i (2.21)

Lower the value of Variation better is the goodness of fit (Pillai and Nair 1997).

6. Root Mean Square Prediction Error: It is a measure of closeness with which a model

predicts the observation.

22 VariationBiasRMSPE (2.22)

Lower the value of Root Mean Square Prediction Error better is the goodness of fit (Pillai and

Nair 1997)



References

ANSI/IEEE, Standard Glossary of Software Engineering Terminology, STD-729-1991.

Bittanti S., Bolzern P., Pedrotti E., Pozzi N. and Scattolini R., “A flexible modelling approach for software reliability growth”, Goos, G., and Harmanis, J., (Ed.), Software Reliability Modelling and Identification, Springer Verlag, Berlin, pp. 101-140, 1988.

Bittanti S. (Editor) “Software Reliability Modelling and Identification” Lecture notes in Computer Science Berlin: Springer-Verlag, 1987.

Brooks W.D. and Motley R.W., “Analysis of Discrete Software Reliability Models-Technical Report (RADC-TR-80-84)”, Rome Air Development Center, New York, 1980.

Bardhan AK. “Modeling in Software Reliability and its Interdisciplinary Nature”. Ph.D. Thesis, University of Delhi, Delhi, 2002.

Chen Y., Singpurwalla N. D., Unification of software reliability models by self-exciting point processes, Adv. Appl. Probab., 29, 337–352, 1997.

Daisuke Satoh, Yamada Shigeru, “discrete equation and software reliability growth models” IEEE, pp. 176-184, 2004.

Gupta Anu “Some Contributions to Flexible Software Reliability Growth Modelling –Ph.D. Thesis” University of Delhi, Delhi, 2007.

Garg RB. “Reliability Modelling In Software and Computing Systems-Ph.D. Thesis” University of Delhi, Delhi, 1990.

Goel A.L. and K. Okumoto, “Time dependent error detection rate model for software reliability and other performance measures”, IEEE Transactions on Reliability, Vol. R-28 No. 3, pp. 206-211, 1979.

Gokhale S.S., Lyu M.R. and Trivedi K.S., “Analysis of software fault removal policies using a non-homogenous continuous time Markov chain”, Software Quality Journal, vol. 12, pp.211-230, 2004.

Goswami, DN. “Software Reliability Growth Modelling And Its Impact On Testing Resources- Ph.D. Thesis”, Jiwaji University, Gwalior, 2004.

Huang C. Y., Lyu M. R. and Kuo S. Y., “A unified scheme of some non-homogeneous Poisson process models for software reliability estimation”, IEEE Trans. on Software Eng., 29, pp. 261–269, 2003.



Huang Yu Chin “Performance Analysis of Software Reliability Growth Models with Testing Effort and Change-Point “, Journal of Systems and Software Vol. 76, Issue 2, pp 181-194, 2005.

Innoue S. and Yamada S. “Generalised Discrete Software Reliabilty Modelling with effect of Program Size”, IEEE transactions Syst. Man Cybern. A, Syst. Humans, vol. 37,No.2, pp. 170-179,2007.

Inoue S. and Yamada S., “An infinite server queuing model considering time distribution of fault isolation process for software reliability assessment”, (in Jananese) IEICE Technical Report [Reliability], Vol. 102 No. 454, pp. 119-124, 2002.

Inoue, S. and S. Yamada, Discrete Software Reliability Assessment with Discretized NHPP Models, Computer & Mathematics with Applications, Vol. 51, No. 2, pp.161-170, 2006.

Kapur P. K., Garg R.B. and S. Kumar, Contributions to Hardware and Software Reliability, World Scientific, Singapore, 1999.

K. Pillai and V.S.S. Nair, “A Model for Software Development effort and Cost Estimation”, IEEE Transactions on Software Engineering; vol. 23(8), pp. 485-497, 1997.

Kapur P. K, Singh V. B, Anand Sameer and Yadavalli V.S.S.,“Software Reliability Growth Model with Change-Point and Effort Control Using a Power Function of Testing Time” International Journal of Production Research, Vol. 46 Issue no. 3 pp.771-787,2008.

Kapur P. K, Singh V. B, Anand Sameer “Effect of Change-Point on Software Reliability Growth Models Using Stochastic Differential Equations” published in the proceedings of 3rd International Conference on Reliability and Safety Engineering, (Eds. R.B.Misra, V.N.A. Naikan, S.K.Chaturvedi, and N.K.Goyal), (INCRESE-2007), Udaipur, pp. 320-333, 2007.

Kapur P. K., Pham, H., Gupta, A. & Jha, P. C. Software Reliability Assessment with OR Applications, UK, Springer, 2011.

Kapur P.K. Kumar Archana ,Yadav Kalpana and Khatri Sunil “Software Reliability Growth Modelling for Errors of Different Severity using Change Point” International Journal of Quality ,Reliability and Safety Engineering ,Vol.14,No.4, pp 311-326,2007.

Kapur P.K., Garg R.B. A software reliability growth model for an error-removal phenomenon, Software Engineering Journal, July. Pp. 291-294, 1992.



Kapur P.K., Goswami D.N., Bardhan a.K., Singh Ompal, “Flexible Software reliability Growth Model with testing effort Dependent Learning Process” applied Mathematical Modelling, Vol. 32,No. 7 pp. 1298-1307, 2008.

Kapur P.K., Gupta A., Kumar A., Yamada S.; “Flexible Software Reliability Growth Models for Distributed Systems” OPSEARCH, 42 (4): 378-398, 2005.

Kapur P.K., Younes S., Agarwala S.; “Generalised Erlang Model with n types of faults”, ASOR Bulletin 14 pp. 5-11, 1995.

Kapur PK, Bardhan AK and Shatnawi O, “Why Software Reliability Growth Modelling Should Define Errors of Different Severity”, Journal of the Indian Statistical Association (Special Issue on Reliability Theory and Its Applications) 40(2), pp. 119-143, 2002.

Kapur PK, Bardhan AK, Kumar S. “On Categorization Of Errors In A Software” International Journal of Management and System 16(1): pp. 37-48, 2006.

Kapur PK, Bhusan S, Younes S. “An Exponential Software Reliability Growth Model with Upper Bound on Number of Failures” Microelectronics and Reliability, 33(9): pp. 1245-1249, 1993.

Kapur PK, Singh Ompal, Gupta Amit, “Some Modelling Peculiarities in Software Reliability” in Quality, Reliability and IT (Trends & future Directions), Eds.: P.K.Kapur and A.K Verma, Narosa Publications Pvt. Ltd., New Delhi 2005.

Kapur PK, Yadavalli VSS, and Gupta Anu “Software Reliability Growth Modeling Using Power Function Of Testing Time” International Journal of Operations and Quantitative Management, Vol.12, No.2, June 2006.

Kapur PK, Younes S, Sehgal VK and Yadav HS. “Discrete Imperfect Debugging Software Reliability Growth Model” Bulletin of Pure and Applied Sciences 15 E (2), pp 121-126, 1996.

Kapur, P.K. and Younes, S. “Software reliability growth model with error dependency”, Microelectronics and Reliability, Vol. 35 No. 2, pp. 273-278, 1995.

Kapur, P.K., Gupta Amit, Gupta Anu, Kumar A., Discrete Software Reliability Growth Modelling, Published in “Quality, Reliability and IT (Trends & Future Directions)”, Eds.: P.K. Kapur and A. K. Verma, Narora Publications Pvt.Ltd., New Delhi, 2005.

Kapur, P.K., Xie, M., Garg, R.B. and Jha, A.K. A discrete software reliability growth model with testing effort, Published in the Proceedings of 1st International Conference on Software Testing, Reliability and Quality Assurance (STRQA), held during Dec 21-22 at Hyatt Regency, New Delhi, 16-20, 1994.



Khoshgoftaar TM “Non-Homogenous Poisson Process for Software Reliability Growth”, COMPSTAT’88, Copenhagen: Denmark, pp. 13-14, 1988.

Kimura M, Yamada S and Osaki S, “Software Reliability Assessment for an Exponential S-shaped Reliability Growth Phenomenon”, Computers and Mathematics with Applications 24, pp. 71-78, 1992.

Kumar Archana, Ph.D. Thesis “A Study in Software Reliability Growth Modelling Under Distributed Development Environment”, Department of Operational Research, University of Delhi, Delhi, 2007.

Littlewood B. “Stochastic Reliability Growth: A Model for Fault Removal in Computer Program and Hardware Design”, IEEE Transactions on Reliability; R-30, No. 4, pp.313-320, 1981.

Littlewood, B. “Modeling Growth in Software Reliability. In Software Reliability Handbook” Ed Paul Rook. Elsevier Applied Science, pp. 111-136, 1991.

Musa JD, Software Reliability Engineering, McGraw-Hill, 1999.

Musa J.D. “Validity of execution time theory of software Reliability” IEEE Transactions on Reliability, R-28: pp 181-191, 1979.

Musa J.D., A. Iannino and K. Okumoto, Software Reliability: Measurement, Prediction, Applications, McGraw Hill, 1987.

Musa JD “A Theory of Software Reliability and Its Applications”, IEEE Transactions on Software Engineering; SE-1, pp. 312-327, 1975.

Musa JD and Okumoto “A Logarithmic Poisson Execution Time Model For Software Reliability Measurements”, Proceedings of the 7th International Conference on Software Engineering, Orlando, F1, pp. 230-237, 1984.

N. Langberg and N.D. Singpurwalla, “A Unification of Some Software Reliability Models,” SIAM J. Scientific and Statistical Computing, vol. 6, no. 3, pp. 781-790, Mar. 1985.

Ohba M and Yamada S “S-shaped Software Reliability Growth Model”, Proceedings of the 4th International Conference on Reliability and Maintainability, pp.430-436, 1984.

Ohba M., “ Software Reliability Analysis Models”, IBM Journal of Research and Development, Vol. 28, pp. 428-443, 1984.

Ohba M., “Inflection S-Shaped Software Reliability Growth Models. Osaki, S. and Hatoyama, Y. (Eds.), Stochastic Models in Reliability Theory, Berlin, Germany: Springer-Varlag, pp. 44-162, 1984.



Pham H. “Software Reliability” Springer-Verlag Singapore, 2000.

Pham H. and X. Zhang, “An NHPP Software Reliability Models and its Comparison “International Journal of Reliability, Quality and Safety Engineering, 4, pp. 269-282, 1997.

Pham H., System Software Reliability, Reliability Engineering Series Springer, 2006.

Pham, “Hanbok of Engineering statistics” Springer, 2006(a).

Popstajanova, K. and K. Trivedi: “Architecture Based approach to Reliability Assessment of Software Systems”, Performance Evaluation, Vol. 45, No.2, pp.179-204, 2001.

Pressman RS, Software Engineering: A Practitioner’s Approach, McGraw-Hill: Fifth Edition, 2001.

Schneidewind N.F., Modelling the fault correction process, Proceedings of the 12th International Symposium on Software Reliability Engineering, IEEE Computer Society Press: Los Alamitos, CA, pp. 185–190, 2001.

Shanthikumar, J. G. “A general software reliability model for performance prediction”, Microelectronics Reliability, Vol. 21, pp. 671–682, 1981.

Shanthikumar, J. G. “Software reliability models: a review”, Microelectronics Reliability, Vol. 23, pp. 903–943, 1983.

Singh V.B. “A Study on Software Reliability Growth Model using Change-Point Fault Dependency” Ph.D. Thesis, University of Delhi, Delhi, 2008.

Singh, V.B., Yadav, K., Kapur, R. and Yadavalli, V.S.S. “Considering fault dependency concept with debugging time lag in software reliability growth modeling using a power function of testing time”, International Journal of Automation and Computing, Vol. 04 No. 4, pp. 359-368, 2007.Singpurwalla ND, Wilson SP. “Statistical Methods in Software Engineering – Reliability and Risk” New York: Springer-Verlag, 1999.

Singpurwalla ND. “The Failure Rate Of Software: Does It Exist?” IEEE Transactions on Reliability 44(3): pp.463-469, 1995.

Wood A. “Predicting Software Reliability” IEEE Computers, 11: pp. 69-77, 1996.

Xie M. “Software Reliability Modelling” World Scientific, 1991.

Xie M., Zhao M., The Schneidewind software reliability model revisited, In Proceedings of the 3rd International Symposium on Software Reliability Engineering, pp. 184–192, 1992.



Yamada S, Ohtera H, and Narithisa H. “Software Reliability Growth Models With Testing Effort” IEEE Trans. On Reliability, R-35 (1): pp. 19-23, 1986.

Yamada S., Kimura M., Tanaka H., Osaki S., Software reliability measurement and assessment with stochastic differential equations, IEICE Transactions on Fundamentals of Electronics and Computer Sciences, Vol. E77-A, No. 1, pp. 109-116, 1994.

Yamada S., Nishigaki A., Kimura M., A Stochastic Differential Equation Model for Software Reliability Assessment and its Goodness of Fit, International Journal of Reliability and Applications, Vol. 4, No. 1, pp. 1-11, 2003.

Yamada S., Obha M. and Osaki S., “S-shaped software reliability growth modelling for software error detection”, IEEE Trans. On Reliability, Vol. R-32 No. 5, pp. 475-484, 1983.

Yamada S., Ohba M., Osaki S.; “S-Shaped Software Reliability Growth Models and their Applications”, IEEE Transactions on Reliability; R-33: 169-175, 1984.

Yamada S., Osaki S. Software reliability growth models”, Applied- Stochastic Models and Data analysis,1,65-77, 1985.

Yamada S., Tamura Y and Kimura M.; A Software Reliability Growth Model For a Distributed Development Environment-Electronics & Communication in Japan Part 3, Vol. 83, pp 46-53, 2000.

Yamada S., Tamura Y., A Flexible Stochastic Differential Equation Model in Distributed Development Environment , European Journal of Operational Research, Vol. 168, pp. 143-152,2006.

Yamada, S., M. Obha and S. Osaki, S-Shaped Software Reliability Growth Modeling For Software Error Detection, IEEE Trans. On Reliability, Vol. R-32, No. 5, pp. 475-484, 1983.

Younes S. “Studies in Software Reliability Modelling–Ph.D. Thesis” University of Delhi, Delhi, 1990.

Zhang X and Pham H.” An Analysis of Factors Affecting Software Reliability “ Journal of Systems and Software, Vol. 50 No.1, pp. 43-56(14), 2000.

Zhao, M “Change-Point Problems In Software And Hardware Reliability”, Communications in Statistics-Theory and Methods, 22(3), pp. 757-768, 1993.

chapter 2 literature review - inflibnetshodhganga.inflibnet.ac.in/bitstream/10603/45303/6/06_chapter...

Documents