tcs2411 software engineering1 software testing techniques “how to find bugs in the software?”

TCS2411 Software Engineering 1

Software Testing Techniques

“How to find bugs in the software?”


Lecture ObjectivesTo understand the role of testing in

ensuring software qualityTo discuss issues relevant to software

testingTo describe the different techniques

used for testing software


Validation & Verification

Validation “Are we building the product right?” Ensure software meets customer’s

needs

Verification “Are we building the right product?” Ensure software meet specification

(error-free)


Testing Terminology

Failure Incorrect or unexpected output Symptom of at fault

Fault Invalid execution state Symptom of an error May or may not produce failure

Error Defect or anomaly in source code Commonly referred as bug May or may not produce fault


What is Software Testing

Software Testing is a process of executing a program with the intent of finding an error.

A good test case is one that has a high probability of finding an as yet undiscovered error

A successful test is one that uncovers an as yet undiscovered error


Testing Objective

To design tests that systematically uncover different classes of errors and to do so with a minimum amount of time and effort


Why We Need Software Testing?

Reveal faults/failures/errorsLocate faults/failures/errorsShow system correctness Improved confidence that system performs

as specified (verification) Improved confidence that system performs

as desired (validation) Indicator of system reliability and system

quality


Importance of TestingCritical Element of Software Quality

AssuranceWell-planned and thorough testing are

important in reducing the high cost of software failure

Can take up to 40% of development effort

In systems that require high reliability, testing can be 3 to 5 times of all the other steps combined


Testing Principles

All test should be traceable to customer requirements.

Tests should be planned before testing begins.

Testing should begin with individual components and move towards to integrated cluster of components.

The most effective testing should be conducted by an independent third party.


Pre-requisites For Good Testing

Adequate staffAdequate support facilities (tools)Adequate time


Who Should Test The Software?

Developer (individual units)Independent test group (ITG)

removes conflict of interest reports to SQA team


Successful Testing

Uncover errors in the softwareDemonstrates that the software

functions appear to be working according to specification

Data collected as testing is conducted provide a good indication of software reliability and software quality


Testing Guidelines

Testing a system’s capabilities is more important than testing its components

Testing old capabilities is more important than testing new capabilities

Testing typical situations is more important than testing boundary value cases


Test Cases

Test case : unit of testing activityTest cases have 3 parts :-

Goal Aspect of the system being tested

Input and system stateData provided to the system under stated condition

Expected behaviorThe output or action the system should take

according to its requirements


Test Cases (Cont)

Test cases are derived during all phase of development cycle

Determine the expected result before running the test case

Record test cases and its resultsAllocate

money people resourcestime


Type Of Test Cases

Test cases are derived for Valid and expected input Invalid and unexpected input Test if the system does less than

specified requirement Test if the system does more than

specified requirement


White Box Testing

Derived from knowledge of program’s structure & implementation

Structural testing - analyse code & use knowledge of the structure of a component to derive test data

Logical paths are tested by providing test cases that exercise specific sets of conditions and/or loops


White Box Testing (Continued)

Thorough white box testing would lead to “100 percent correct programs”

Exhaustive testing are impractical - too many tests!

A limited number of logical paths can be selected and exercised

Important data structures can be probed for validity


White Box Test Cases

Guarantee that all independent paths have been exercised at least once

Exercise all logical decisions on their true and false sides

Execute all loops at their boundaries and within their operational bounds

Exercise internal data structures to ensure their validity


White Box Testing Techniques

Basis path testing Flow graph notation Cyclomatic complexity Derived test cases Graph metrics

Control structure testing Condition testing Data Flow testing Loop testing


Flow Graph Notation

1

2

3

6

7 8

4

5

9

1011

1

2,3

6

7 8

9

10

11

4,5

NodeEdge

R3

R2

R1

R4

Region


Cyclomatic Complexity

Provide quantitative measure for program logical complexity.

Defined number of independent path Any path that introduce one ser of processing

statements or new conditionEg :-

Path 1 : 1-11 Path 2 : 1-2-3-4-5-10-1-11 Path 3 : 1-2-3-6-8-9-10-1-11 Path 4 : 1-2-3-6-7-9-10-1-11


How Is Cyclomatic Complexity Computed?

1. Number of regions The flow graph has 4 regions

2. V(G) = E – N + 2 E : Number of flow graph edges N : Number of flow graph nodes

V(G) = 11 edges – 9 nodes + 2 = 4

3. V(G) = P + 1 P : Number of predicate nodes

V(G) = 3 predicate nodes + 1 = 4


Deriving Test Cases1. Draw flow graph from design/code as foundation.

…i=1;total.input = total.valid=0;sum=0;do while value[i] <> -999 and total.input<100 increment total.input by 1; if value[i] >= minimum AND value[i] <= maximum then increment total.valid by 1; sum = sum + value[i] else skip end if increment i by 1End doIf total.valid > 0 then average = sum / total valid; else average = -999;End if…

1

2

3

4

56

7

8

910

11

12

13

1

2

3

4

5

6

78

9

10

1112

13


Deriving Test Cases (cont)

2. Determine cyclomatic complexity V(G) = 6 regions V(G) = 17 edges – 13 nodes + 2 = 6 V(G) = 5 predicates nodes + 1 = 6

3. Determine a basis set of linearly independent graph

Path 1 = 1-2-10-11-13 Path 2 = 1-2-10-12-13 ….

4. Prepare test cases Path 1 test case :

value(k) = valid input where k < i defined below value(i) = -999 where 2 <= i <= 100 Expected result : Correct average based on k value and

proper totals


Discussion on White Box Testing

Advantages Find errors on code level Typically based on a very systematic approach,

covering the complete internal module structure

Constraints Does not find missing or additional functionality Does not really check the interface Difficult for large and complex module


Black Box Testing

Derived from program specificationFunctional testing of a component of

a systemExamine behaviour through inputs &

the corresponding outputsInput is properly accepted, output is

correctly producedDisregard internal logical structure


Black Box Testing (Continued)

Attempts to find the following errors:Incorrect or missing functionsInterface errorsErrors in data structures or external

database accessPerformance errorsInitialisation and termination errors


Black Box Testing Techniques

Graph Based Testing MethodsEquivalence PartitioningBoundary Value AnalysisComparison TestingOrthogonal Array Testing


Equivalence Partitioning

Divide input domain into classes of dataBased on an evaluation of equivalence classes

for an input conditionGuidelines to define equivalence classes

Range input : One valid and two invalid equivalence Specific value : One valid and two invalid

equivalence A member of a set : One valid and one invalid

equivalence Boolean : One valid and one invalid equivalence


Example – Data for Automated Banking Application

The use can access the bank using his personal computer, provide a six digit password, and follow with a series of typed commands that trigger various banking function. During the log on sequence the software supplied for the banking application accepts data in the form:

area code – blank or 3 digit numbersprefix – 3 digit numbers, nor beginning with 0 or 1suffix – 4 digit numberspassword – 6 digits alphanumeric stringscommands – “check”, “deposit”, “bill pay” etc

Input condition

area code : Input condition : Boolean – area code may or may not present Input condition : Range – 200 – 999 with specific exception

prefix : Input condition : Range – specified value > 200 with no 0 digitssuffix : Input condition : Value – 4 digit lengthpassword : Input condition : Boolean – password may or may not present

Input condition : Value – six character stringcommand : Input condition : Set – containing command


Boundary Value Analysis

Complement equivalence partitioningTest both sides of each boundary Look at output boundaries for test

cases Test min, min-1, max, max+1, typical

valuesExample : 1 <= x <=100

Valid : 1, 2, 99, 100 Invalid : 0 and 101


Discussion on Black Box Testing

Advantages Find missing functionality Independent from code size and

functionality Find some coding errors

Constraints No systematic search for low level errors Specification errors not found


References

“Software Engineering: A Practitioner’s Approach” 5th Ed. by Roger S. Pressman, Mc-Graw-Hill, 2001

“Software Engineering” by Ian Sommerville, Addison-Wesley, 2001

tcs2411 software engineering1 software testing techniques “how to find bugs in the software?”

Documents

testing software slide

software testing zto

software quality slide

yensure software

software zdemonstrates

software functions

tcs2411 software engineering5

tcs2411 software engineering11