product quality: metrics, verification, validation, testing

Arab Open University2nd Semester, 2006-2007

M301

Unit 5.3Product Quality: Metrics,

Verification, Validation, Testing [email protected]

mailto:[email protected]

Reem AlAttas © 2

Topic Road Map

Introduction to software qualityVerification, validation and testingMeasuring software quality factorsTesting techniquesFormal technical reviews

Reem AlAttas © 3

What is software quality?

Software that is fit for its purpose and is of sufficiently high quality is said to be of appropriate quality, i.e. in conformance to the requirements and expectations of the customer.

Reem AlAttas © 4

Requirements

There are three general types of requirements that have an effect on software product quality: Product operation requirements how the product will

be used by the end user? Product revision requirements how the product will be

changed? Product transition requirements how the product will

be modified for different operating environments?

Reem AlAttas © 5

Product operation requirements

1. Correctness: that attribute which determines how well a system fulfils the customer’s overall objectives;

2. Reliability: that attribute which determines system can be expected to perform its intended function;

3. Efficiency: that attribute which determines the level of computing resources required perform its function;

4. Integrity: that attribute which determines how well the data is secured;

5. Usability: that attribute which determines how easy the system is to use.

Reem AlAttas © 6

Product revision requirements

1. Maintainability: that attribute which determines how easily bugs can be found and fixed.

2. Flexibility: that attribute which determines the effort required to modify an operational system.

3. Testability: that attribute which determines how easily the system can be tested to show that the customer’s requirements have been met.

Reem AlAttas © 7

Product transition requirements

1. Portability: that attribute which determines how easily the system can be used on another machine

2. Reusability: that attribute which determines how easy it is to reuse some of the software to make future developments more cost-effective

3. Interoperability: that attribute which determines the effort required to couple one system to another.

Reem AlAttas © 8

Primary Software Quality Factors (SQFs)

1. Correctness.

2. Integrity.

3. Maintainability.

4. Usability.

Reem AlAttas © 9

Correctness Measure

The popular measure for assessing correctness is defects per thousand lines of code (defects per KLOC)

Defect (bug): verified lack of conformance to requirements.

Reem AlAttas © 10

Integrity Measure

Integrity is measured by considering the proportion of ‘attacks’ on a product as opposed to bona fide uses.

Reem AlAttas © 11

Example

Reem AlAttas © 12

Topic Road Map

Introduction to software quality Verification, validation and testingMeasuring software quality factorsTesting techniquesFormal technical reviews

Reem AlAttas © 13

Verification

The process of checking that each system description is selfconsistent, and that different system descriptions are consistent and complete with respect to each other.

Reem AlAttas © 14

Validation

The process of checking that each system description is consistent with the customer’s requirements.

Reem AlAttas © 15

Consistency

Two system descriptions are consistent simply means that, where they describe the same part of the system, they do not contradict each other.

Consistency with a customer requirement means that the requirement is met.

Reem AlAttas © 16

Completeness

Every aspect of the customer’s requirements must be met by the system description.

More difficult task than showing consistency.

Reem AlAttas © 17

Testing

The process of executing a software system, or part of a software system, in order to check that it meets its requirements.

It is any form of validation or verification that operates on program code.

Reem AlAttas © 18

Test Procedure

Select some test data.Predict the results that should be

expected.Check whether the software conforms to

our expectations.

Reem AlAttas © 19

Role of Testing

Help finding and fixing bugs.Improve the customer’s confidence in a

software product.Improve the testing process itself.

A cost–benefit analysis of ‘bugs found’ versus ‘time taken’ can be performed to determine when testing should stop.

Reem AlAttas © 20

Types of Testing

1. Usability testing.2. Developmental testing: checks that developmental

activities have been carried out correctly.3. Requirements-based testing: checks that a system

meets the customer’s requirements.4. Regression testing: occurs during developmental

testing and system maintenance, and checks that fixing one bug has not introduced others.

Reem AlAttas © 21

Developmental testing stages

Unit testing, in which units of functionality (e.g. The classes in an object-oriented system) are tested in isolation.

Integration testing, in which previously unit-tested units are tested together.

System testing, in which the completed system is tested against the customer’s requirements.

Reem AlAttas © 22

Requirements testing stage

Acceptance testing, which is performed by the customer, and after which (all being well) the system is accepted as complete.

Reem AlAttas © 23

Topic Road Map

Introduction to software qualityVerification, validation and testing Measuring software quality factorsTesting techniquesFormal technical reviews

Reem AlAttas © 24

Measuring Quality The quality of a software product

can be measured by obtaining values for the eleven software quality factors (SQFs)

Obtaining such values is not a trivial task, as there is no obvious way of evaluating a product against these factors.

What we must do is to find properties of software products which are measurable, and use these to infer the values for the SQFs.

1. Correctness2. Reliability3. Efficiency4. Integrity5. Usability

6. Maintainability7. Flexibility8. Testability

9. Portability10. Reusability11. Interoperability

Reem AlAttas © 25

Software Metrics

1. Accuracy2. Auditability3. Communication commonality4. Completeness5. Complexity6. Conciseness7. Consistency8. Data commonality9. Execution efficiency10. Expandability

11. Generality12. Hardware independence13. Instrumentation14. Modularity15. Operability16. Robustness17. Security18. Self-documentation19. System independence20. Traceability21. Training

Reem AlAttas © 26

The Dependence of the SQFs on the Metrics

Reem AlAttas © 27

Ex. The Relationship Between Reliability and the Metrics

F value for the SQF, reliability. m measured values for the metrics which relate to the SQF,

reliability. c weightings indicating the relative importance of the metrics with

respect to the SQF, reliability.

Reem AlAttas © 28

Measuring System Complexity

1. Lines-of-code Metric: measures the complexity of a method by counting the number of lines-of-code.

More LOC more errors. Should comment lines be included in the line

count?

Reem AlAttas © 29

… Measuring System Complexity

2. McCabe’s Cyclomatic-complexity Metric: measures the complexity of a method by counting the number of independent paths through a method body. A method body starts and completes its execution defines one

independent path. The number of independent paths through a piece of code can be

calculated by counting the number of decision points. Each if statement adds a decision point. Each while, do-while and for loop adds a decision point. Each switch statement adds one decision point for each of the cases it

tests for, excluding any default case. Each try statement adds one decision point per catch block, but any

finally block is not a decision point, as it will always be executed. The operators && and || each add a decision point.

Reem AlAttas © 30

… Measuring System Complexity

A cyclomatic-complexity of ten or more should be a hint that a method body should be restructured into two or more simpler methods.

Reem AlAttas © 31

Modern Object-oriented Metrics Depth of inheritance tree (DIT): its number of ancestors.

Coupling between objects (CBO): the number of has-a relationships the class has with other classes.

Number of children (NOC): the number of children for that class.

Response for a class (RFAC): the size of the response set for the class, which consists of all the methods of that class together with all the methods of other classes called by those methods.

Lack of cohesion in methods (LCOM): its cohesiveness.

Weighted methods per class (WMPC): its complexity of behavior the sum of the cyclomatic complexities of each method of the class.

Reem AlAttas © 32

Topic Road Map

Introduction to software qualityVerification, validation and testingMeasuring software quality factors Testing techniquesFormal technical reviews

Reem AlAttas © 33

Testing Techniques

Black-box testing we choose test cases by looking at the specification (i.e. requirements and high-level design) of the system to be tested.

White-box testing we choose test cases by looking at the detail of the implementation of the system to be tested.

Reem AlAttas © 34

A strategy for black-box testing (using partitioning)

1. For each method in the class, determine the input data space.

2. Partition the input data space into sub-domains.

3. Test all sub-domains given by the case analysis.

Reem AlAttas © 39

Basis-path testing

Ensures that all reachable statements in a method are tested at least once. Select data which will exercise the straight-through

path, i.e., loop and if conditions evaluate to false, and only default cases selected in switch statements.

Then, find data that deviates from the straight-through path at the first decision point.

The process is then repeated for each subsequent decision point in the program, varying the flow in each case.

Reem AlAttas © 40

Loop testing

Used to test loop constructs. In Java, there are three ways in which

loops can occur: Simple loops. Nested loops. Concatenated loops.

Reem AlAttas © 41

Simple loops forms

Test data should be found for the following cases: the loop is skipped entirely the loop is passed through exactly once the loop is passed through more than once, maximum n times

m times, where 3 < m < n – 2 pass through the loop n – 1, n and n + 1 times.

Reem AlAttas © 42

Nested loops form

Approach: Start with the innermost loop. Work outwards. Repeat step 2 until all loops are tested.

Reem AlAttas © 44

Topic Road Map

Introduction to software qualityVerification, validation and testingMeasuring software quality factorsTesting techniques Formal technical reviews

Reem AlAttas © 45

Inspections (Formal Technical Reviews)

A formal, efficient, and economical method of finding errors in design and code.

TMA5 – Q3

Thank You!

product quality: metrics, verification, validation, testing

Education