university of sunderland cifm03lecture 4 1 software measurement and reliability cifm03 lecture 4

1University of Sunderland CIFM03 Lecture 4

Software Measurement and Reliability

CIFM03

Lecture 4


Where does Software Measurement fit in?

Look at software measurement from the perspective of software quality assurance…..

SQA

REVIEWS TESTING

MEASURE-MENT

STANDARDS&

PROCEDURES


Software Measurement is . . .

“the objective assignment of numbers or symbols to attributes of a software process or product in such a way as to describe them according to clearly defined rules...”

see ‘Software Metrics, A Rigorous & Practical Approach’, Fenton & Pfleeger


Why should we measure?

what are our goals?

what information is required?

what can we collect?

How do we act upon the data?


DON’T!

Don’t define what you won’t collect..

Don’t collect what you won’t analyse..

Don’t analyse what you won’t use..


Goal / Question / Metric

• Goal - achieve a 10-fold improvement in

post-release defects.

• Question - how many existing faults? How

effective are inspection techniques? How

effective are tests?

• Metric - no. & type of faults, no. of defects

found, % paths tested, complexity of code.


Hard & Soft Data

HARD - objective figures - no.of people on a project, duration of tasks, size of documentation, etc.

SOFT - subjective - level of experience, balance of teams, etc.


Obstacles to Effective Measurement

no agreement over standard measures

patchy tool support

initial set-up expense


Basic Measurement Theory

Nominal - names / categories

Ordinal - ‘cold, warm, hot’ ‘small, med, large’

Interval - temperature ‘20oC, 40oC’ - scale

Ratio - absolute temperature ‘20oK, 40oK’, time

Absolute - counting no. of occurrences


Analysis..

Nominal - equivalence

Ordinal - as above + greater than

Interval - as above + ratio of any intervals

Ratio - as above + ratio of any 2 scale values

Absolute - as above + geometric mean,

coeff. of variation


Function Points & Cyclomatic Complexity

Function Points:

Measure of functionality for ‘size’

- no. of external inputs- no. of external outputs- no. of enquiries- no. of files- no. external interfaces

Cyclomatic Complexity:

Module complexity based on a ‘graph’ of the module

- nodes

- predicate nodes- paths


Measurement and the CMM

Maturity Level Characteristics Type of Metrics to Use

5. Optimizing Improvement fed- back to the process

Process plus feedback for changing the process

4. Managed Measured process Process plus feedback for control

3. Defined Process defined andinstitutionalised

Product

2. Repeatable Process dependent on individuals

Project management

1. Initial Ad hoc Baseline


Software Reliability is…

The probability that the software will;

operate as required (i.e., without fail),

for a specified time,

in a specified environment.


Software Reliability - features• failures in software are design faults,

• reliability during test changes continually(new problems are found as old ones are fixed / new code is never perfect)

• phenomenon of software reliability growth

• environment is important (platform/inputs) - new envt. may require s/w retest


Reliability - General (i)

• Of all software quality characteristics, reliability is consistently identified as most important.

• Reliability, for both hardware and software, is a probability - SYSTEM reliability can be derived from combining the hardware and software reliabilities.

• Reliability is directly related to the number of failures. [ p(Reliable) =1 - p(Fail) ]


Reliability - General (ii)

• Reliability measures are much more useful than fault measures - from the user perspective.

• Measures of faults/defects are developer oriented.

• Change of environment (operating platform, operational data profile) changes reliability.


Fault, Error, Failure

(defined by BS5760 part8: 1998)

• Faulta defect in a software component which, when

triggered, causes one or many errors

• Errora discrepancy between its actual state and the

‘correct state’ which, if it propagates, means that the component ceases to perform as required

• Failure the component ceases to perform as required


Fault, Error, Failure

FAULTS

ERRORS

FAILURE

ENVIRONMENT

OPERATOR

INPUT

OR

REVEALINGMECHANISMAND

LEADS TO ZERO OR MANY

LEADS TO ZERO OR MANY

POTENTIALLYLEADS TO ZERO OR MANY

MISTAKES

(PERSON MAKES)

CAN BE ATTRIBUTED TO ONE OR MANY



(This slide from BS5760 Part 8: 1998)


Reliability Measures

When we talk of reliability measures the irony is that we invariably talk about failure measures. There are four general ways of measuring failures against time;

time of failure, interval between failures, cumulative failures experienced up to a given time, failures experienced in a time interval.


Availability & Maintainability

• Availability – is the probability that the software is operating at

a given time. It is equal to the mean time to failure divided by the mean time between failures.

• Maintainability – is the mean time to repair the software and

restore to working order.


Uses of S/w Reliability Measurement

• System Reliability evaluation

• Monitoring operational performance

– e.g., decisions on when to go for enhancement

• Evaluation of test status / test schedule

• Management of new features and design changes

university of sunderland cifm03lecture 4 1 software measurement and reliability cifm03 lecture 4

Documents