logo evaluating quality-in-use using bayesian networks m.a. moraga 1, m.f. bertoa 2, m.c. morcillo...

LOGO

Evaluating Quality-in-Use Using Bayesian Networks

M.A. Moraga1, M.F. Bertoa2, M.C. Morcillo2, C. Calero1, A. Vallecillo2

1Universidad de Castilla-La Mancha 2Universidad de Málaga

(Spain)

Paphos, Cyprus - July 8th, 2008 LOGO

Contents

Introduction1

ISO/IEC 91262

Bayesian Belief Networks3

BNs for evaluating QiU4

Conclusions5


Introduction. The ISO approach

Assessing the quality of a software product is, in general, a complex and difficult task.

ISO/IEC 9126 proposes three levels of quality: Internal quality uses a “white box” view of software products External quality uses a “black box” view Quality in use (QiU) is the end-user viewpoint when the

software product is used in its working environment


ISO/IEC 9126

Quality

Efficiency

Maintainability

Portability

Functionality

Reliability

Usability

Efficiency

Maintainability

Portability

Functionality

Reliability

UsabilityQiU

Intern

al

Extern

al

Productivity

Safety

Satisfaction

Effectiveness


ISO current approach is questionable

Most efforts so far have been put on evaluating the internal and external quality

… based on the assumption that a good external quality would guarantee a good quality-in-use (QiU)

But we all know that quality does not work that way!

The QiU should be the driving force!


Reasons to challenge the traditional approach

1 Not all the external quality characteristics of a software product have the same influence on its QiU

2 The direct dependency between the EQ and the QiU, forces to over-specify some of the product aspects (which are non critical for the end user), for the sake of ensuring a certain level of QiU. Unnecessarily increasing costs and development efforts, without a

direct effect on the advantages that the end-user perceives

3 Users normally tend to perceive as equally important all the external quality characteristics of a software product when asked about them in the abstract However, when the context-of-use of a soft-ware product is fixed, users are

able to tell apart the product characteristics that really matter (from their point view), from those that are desirable, but not critical


Our position in this paper

We change the focus of the quality assessment concentrating on the QiU as the driving factor for designing a

software product, or for selecting the product that better fits a user’s needs

We propose a “backwards” analysis of the relationship between the external quality and the QiU

The analysis tries to determine the external quality sub-characteristics that are really relevant to ensure the required level of quality in a given context of use in order to avoid superfluous costs or irrelevant features – which

may unnecessarily in-crease the final price of the product

We propose Bayesian Belief Networks to model such relationships, and propose a method to build them for different contexts of use.


Contents

Introduction1

ISO/IEC 91262



Conclusions5


Bayesian Belief Networks

A BN is a directed acyclic graph, whose nodes are the uncertain variables and the edges are the casual or influential links between variables.

The different relationships between the characteristics and sub-characteristics of the external quality and the QiU

Using BNs We can model

The degree of dependence or influence between them


An example of a Bayesian Network

[Taken from Fenton and Neil “Software Metrics Roadmap”]


Bayesian Belief Networks

Steps

Make inferences about the values of the variables

Define the conditional probability tables

Define the structure

Train the BN through a set of controlled experiments


Contents

Introduction1

ISO/IEC 91262



Conclusions5


BNs for evaluating QiU

HYPOTHESIS

EQ has influence on the QiU and this influence can be modelled and studied through a BN

We can conduct backwards analysis on the required level of EQ to ensure a given level of QiU



1)

3) Link2)


BNs for evaluating QiU1st Approach Study the relationships between the different characteristics, identifying

the EQ characteristics that have a significant influence on the QiU characteristics.

Quality in Use

Safety Satisfaction Productivity Effectiveness

ExternalQuality

Functionality X X X X

Reliability X X X

Usability X X

Efficiency X X

Portability X

Maintainability X


BNs for evaluating QiU1st Approach BN


BNs for evaluating QiU1st Approach Probability tables:

3 categories (IEEE Std. 1061):Acceptable, Marginal and Unacceptable The tables of the upper level nodes have a single entry with the evaluation of

the external quality sub-characteristics (opinion of experts). The probability tables of the nodes in the next level have been defined taking

into account the degree of influence that each one has on the corresponding QiU characteristic.


BNs for evaluating QiU1st Approach


BNs for evaluating QiU1st Approach Problem: all sub-characteristics of an external quality characteristic

have the same influence on all related QiU characteristics

Is Suitability as “important” for Safety as for Satisfaction? The answer is, of course, No.

2nd Approach


BNs for evaluating QiU2nd Approach Define relationships between EQ sub-

characteristics and QiU characteristics


BNs for evaluating QiU2nd Approach BN


BNs for evaluating QiU2nd Approach Problem: it produces a very high number of entries on the nodes that

represent the QiU characteristics. And therefore, the definition of the probability tables is very laborious and

cumbersome. Solution: the introduction of synthetic nodes

3rd Approach


BNs for evaluating QiU3rd Approach A synthetic node among sub-characteristics of each

EQ characteristic and each QiU characteristic.


BNs for evaluating QiU3rd Approach Advantages: It reduces the number of entries in the probability tables. It avoids the undesirable influence of EQ sub-characteristics on the QiU characteristics when they do not have

a direct relationship

This network achieved a finer granularity while avoiding inconsistent relations Disadvantages: the concept of external quality characteristic is lost.

However, we could easily create four individual BNs, one for each QiU characteristic, and then combine them to form the complete Bayesian Network.



Each QiU characteristic can be studied independently


Contents

Introduction1

ISO/IEC 91262



Conclusions5


Conclusions We have proposed to change the focus of the Evaluation of

the Overall Quality of a Software Product Evaluation should be focused on the QiU – and then use a

backwards analysis for predicting the minimum acceptance levels for EQ (sub)characteristics that ensure the required level of QiU

We have shown how to build a BNs for conducting such analysis

Future work Empirically validate the proposal by:

exercising the BNs in several contexts of use. adapting the BNs to the specific peculiarities of each context checking that the BNs learn as they should for these contexts

Refine this approach by combining it with some Principal Component Analysis during the initial definition of the BN

Provide a useful input to the ISO Working Group defining the new SQUARE family of standards based on our researches

logo evaluating quality-in-use using bayesian networks m.a. moraga 1, m.f. bertoa 2, m.c. morcillo...

Documents