lect9 ch9 unit4 part1

7/30/2019 Lect9 Ch9 Unit4 Part1

1/29

1

ECE 453 CS 447 SE 465Software Testing &Quality Assurance

InstructorKostas Kontogiannis


2/29

2

OverviewStructural Testing

Introduction General ConceptsFlow Graph Testing

DD-PathsTest Coverage MetricsBasis Path TestingGuidelines and Observations

Data Flow TestingHybrid MethodsRetrospective on Structural Testing


3/29

3

Structural TestingAlso known as glass box , structural , clear box and

open box testing . A software testing techniquewhereby explicit knowledge of the internal

workings of the item being tested are used toselect the test data. Unlike black box testing that is

using the program specification to examine

outputs, white box testing is based on specificknowledge of the source code to define the testcases and to examine outputs.

http://www.webopedia.com/TERM/W/White_Box_Testing.html


4/29

4

Structural Testing

Structural testing methods are veryamenable to:

Rigorous definitions Data flow, control flow, objectives, coverage

criteria, relation to programming language semantics

Mathematical analysis

Graphs, path analysis Precise measurement

Metrics, coverage analysis


5/29

5

Program Graph - Definition

Given a program written in an imperativeprogramming language, its Program Graph, is adirected labeled graph in which nodes are eithergroups of entire statements or fragments of astatement, and edges represent flow of control

P. Jorgensen, Software Testing a Craftsmans Approach


6/29

6

Program Graph

If i, j, are nodes in the program graph, there is an edgefrom node i, to node j in the program graph if an only if,the statement corresponding to node j, can be executed

immediately after the last statement of the group of statement(s) that correspond to node i.

The groups of statements that make up a node in theProgram Graph is called a basic block.

There is a straightforward algorithm to segment a codefragment into basic blocks and create the correspondingProgram Graph.


7/29

7

White-box Testing: Determining the BasicBlocks

FindMean (FILE ScoreFile)

{ float SumOfScores = 0.0;int NumberOfScores = 0;float Mean=0.0; float Score;Read(ScoreFile, Score);while (! EOF(ScoreFile) {

if (Score > 0.0 ) {

SumOfScores = SumOfScores + Score;NumberOfScores++;}

Read(ScoreFile, Score);}

/* Compute the mean and print the result */if (NumberOfScores > 0) {

Mean = SumOfScores / NumberOfScores;printf( The mean score is %f \ n, Mean);

} elseprintf (No scores found in file \ n);

}

1

23

4

5

7

6

8

9


8/29

8

Constructing the Logic Flow DiagramStart

2

3

4 5

6

7

8 9

Exit

1

F

T F

T F

T


9/29

9

Program Graph - Example


10/29

10

x = z+5 z = 4*3 - y if(x > z) goto A; for( u=0; u < x; u++) {

z = z+1; };

A: y = z + k

x = z+5 z = 4*3 - y

x > z

z = z+1 u++

y = z+k

u = 0

f

f t

t

u < x

1

2

3

4

5

6

7

Example of a simple control flowgraph.

R1 R2

R3

V( G) = 3

Basis set: 1, 2, 3, 4, 6, 7 1, 2, 3, 4, 5, 4, 6, 7 1, 2, 6, 7

Flow Graphs Use in determiningPaths for Testing


11/29

11

Program Graph - Paths

4

5

6

7

8

10

9

11 12 13

21

14 15

22

16 17

18 19

20

Lets consider the following program graph:


12/29

12

DD-Paths (1) The best known form of structural testing is based on a construct

known as a decision-to-decision path.

A DD-Path is a chains obtained from a program graph , where a chainis a path in which the initial and terminal nodes are distinct, and every

interior node has indegree = 1, and outdegree = 1. We show in the nextslide on selecting the nodes from a program graph to form a DD-Pathchain. DD-Paths are used to create DD-Path Graphs .

Note that the initial node is 2-connected to every other node in thechain, and there are no instances of 1- or 3- connected nodes.

An example of a chain is shown below:

Initial node Internal nodes Final node


13/29

13

DD-Paths (2)

More formally a DD-Path is a chain obtained froma program graph such that:

Case1: it consists of a single node with indeg =0. Case2: it consists of a single node with outdeg=0, Case3: it consists of a single node with indeg 2 or

outdeg 2

Case4: it consists of a single node with indeg =1, andoutdeg = 1

Case5: it is a maximal chain of length 1


14/29

14

DD-Path Formation - Example4

5

6

7

8

10

9

11 12 13

21

14 15

22

16 17

18 19

20

Program Graph Nodes DD-Path Name Case #

4 first 1

5-8 A 5

9 B 4

10 C 411 D 3

12-14 E 5

15 F 4

16 G 3

17 H 4

18 I 319 J 4

20 K 3

21 L 4

22 last 2


15/29

15

DD-Path Graph

Given a program written in an imperativelanguage, its DD-Path graph is a labeled directed

graph, in which nodes are DD-Paths pf itsprogram graph, and edges represent control flowbetween successor DD-Paths.

In this respect, a DD-Path is a condensation graph.For example 2-connected program graph nodes arecollapsed to a single DD-Path graph node.


16/29

16

4

5

6

7

8

10

9

11 12 13

21

14 15

22

16 17

18 19

20

Program Graph Nodes DD-Path Name Case #

4 first 1

5-8 A 5

9 B 4

10 C 4

11 D 3

12-14 E 5

15 F 4

16 G 3

17 H 4

18 I 3

19 J 4

20 K 3

21 L 4

22 last 2

first

A

B C

D

L

E

F G

H

A

I

J K

last


17/29

17

Test Coverage Metrics

The motivation of using DD-paths is that they enable veryprecise descriptions of test coverage.

In our quest to identify gaps and redundancy in our testcases as these are used to exercise (test) different aspectsof a program we use formal models of the programstructure to reason about testing effectiveness.

Test coverage metrics are a device to measure the extendto which a set of test cases covers a program.


18/29

18

Test Coverage MetricsMetric Description of Coverage

C0 Every Statement

C1 Every DD-Path

C1P Every predicate to each outcomeC2 C1 Coverage + loop coverage

Cd C1 Coverage + every dependent pair of DD-Paths

CMCC Multiple condition coverage

C ik Every program path that contains up tok repetitions of a loop (usually k=2)

Cstat Statistically significant fraction of paths

C

All possible execution paths


19/29

19

Statement and Predicate CoverageTesting

Statement coverage based testing aims to devise test cases thatcollectively exercise all statements in a program.

Predicate coverage (or branch coverage, or decision coverage) basedtesting aims to devise test cases that evaluate each simple predicate of the program to True and False. Here the term simple predicate refers toeither a single predicate or a compound Boolean expression that isconsidered as a single unit that evaluates to True or False. Thisamounts to traversing every edge in the DD-Path graph.

For example in predicate coverage for the conditionif(A or B) then C we could consider the test cases A=True, B= False(true case), and A=False, B=False (false case). Note if the programwas encoded as if(A) then C we would not detect any problem.


20/29

20

DD-Path Graph Edge Coverage C1

1 2

T F

T F

Here a T,T andF,F combination willsuffice to have DD-PathGraph edge coverage or

Predicate coverage C1

P1

P2


21/29

21

DD-Path Coverage Testing C 1P

This is the same as the C 1 butnow we must consider testcases that exercise all allpossible outcomes of thechoices T,T, T,F, F,T, F,F forthe predicates P1, and P2respectively, in the DD-Pathgraph.

T F

T F

P1

P2


22/29

22

Multiple Condition Coverage

Testing Now if we consider that the predicates P1 is a compound

predicate (i.e. (A or B)) then Multiple Condition CoverageTesting requires that each possible combination of inputsbe tested for each decision.

Example: if (A or B) requires 4 test cases: A = True, B = TrueA = True, B = False

A = False, B = TrueA = False, B = False

The problem: For n conditions, 2 n test cases are needed,and this grows exponentially with n


23/29

23

Dependent DD-Path Pairs Coverage

Testing C d In simple C 1 coverage criterion we are interested simply to traverse all

edges in the DD-Path graph.

If we enhance this coverage criterion by ensuring that we also traversedependent pairs of DD-Paths also we may have the chance of revealingmore errors that are based on data flow dependencies.

More specifically, two DD-Paths are said to be dependent iff there is adefine/reference relationship between these DD-Paths, in which avariable is defined (receives a value) in one DD-Path and is referenced

in the other.

In C d testing we are interested on covering all edges of the DD-Pathgraph and all dependent DD-Path pairs.


24/29

24

Loop Coverage The simple view of loop testing coverage is that we must devise test

cases that exercise the two possible outcomes of the decision of a loopcondition that is one to traverse the loop and the other to exit (or notenter) the loop.

An extension would be to consider a modified boundary value analysisapproach where the loop index is given a minimum, minimum +, anominal, a maximum -, and a maximum value or even robustnesstesting.

Once a loop is tested, then the tester can collapse it into a single nodeto simplify the graph for the next loop tests. In the case of nested loopswe start with the inner most loop and we proceed outwards.

If loops are knotted then we must apply data flow analysis testingtechniques, that we will examine later in the course.


25/29

25

Statistically Significant Path

Coverage Testing Exhaustive testing of software is not practical because variable input

values and variable sequencing of inputs result in too many possiblecombinations to test.

NIST developed techniques for applying statistical methods to derivesample test cases would address how to select the best sample of testcases and would provide a statistical level of confidence or probabilitythat a program implements its functional specification correctly.

The goal of statistically significant coverage is to develop methods forsoftware testing based on statistical methods, such as MultivariableTesting, Design of Experiments, and Markov Chain usage models, andto develop methods for software testing based on statistical measuresand confidence levels.

Source: http://www.itl.nist.gov/div897/ctg/stat/mar98ir.pdf


26/29

26

Basis Path Testing

We can apply McCabes CyclomaticComplexity metric

gives an upper bound on number of test cases toensure edge coverage is satisfied.

in practice, it is usually the lower bound of the number of test cases due to the presence of loops.


27/29

27

Basis Path Testing - Motivation

If we consider the paths in a program graph (or DD-Graph)to form a vector space V , we are interested to devise asubset of V say B that captures the essence of V; that is

every element of V can be represented as a linearcombination of elements of B. Addition of paths meansthat one path is followed by another and multiplication of anumber by a path denotes the repetition of a path.

If such a vector space B contains linearly independent paths and forms a basis for V then it certainly capturesthe essence of V .


28/29

28

McCabe Algorithm to Determine

Basis Paths The algorithm is straightforward:

The method begins with the selection of a baseline path, which should correspond to a normal execution

of a program (from start node to end node, and has asmany decisions as possible). The algorithm proceeds by retracing the paths visited

and flipping the conditions one at a time. The process repeats up to the point all flips have been

considered.

The objective is to generate test cases that exercisethese basis paths.


29/29

29

x = z+5 z = 4*3 - y if(x > z) goto A; for( u=0; u < x; u++) {

z = z+1; };

A: y = z + k

x = z+5 z = 4*3 - y

x > z

z = z+1 u++

y = z+k u = 0

f

f t

t

u < x

1

2

3

4

5

6

7

Example of a simple control flowgraph

R1 R2

R3

V( G) = 3

Basis set: 1, 2, 3, 4, 6, 7 1, 2, 3, 4, 5, 4, 6, 7 1, 2, 6, 7

Flow Graphs Use in determiningPaths for Testing - Revisited

lect9 ch9 unit4 part1

Documents