Chapter 8: Path TestingCSCI 565

Objectives

Program Graphs

Decision-to-decision path (DD-Paths)

Test Coverage Metrics

Basis Path Testing

Observation on McCabe Basis Path Method

Essential Complexity

Structural Coverage Analysis

The specification based testing can be used to verify the intended functionality

The techniques cannot be used with respect to unintended functionality

Structural testing (white box) is complementary solution

Shows reachability

Program Graph

Program Graph (PG) A directed graph G= (V, E) Where

V = a set of nodes corresponding to program statements in an imperative programming languages (e.g., C)

E = a set of edges corresponding to flow of control

Resilient properties of PG can be nested, which is the equivalent of a subroutine call

can be sequenced, which is the equivalent of sequential execution of two instructions

Program Graphs for elements of Programming langauges

ignore

t f

t

ff

f

f

Test Selection Criteria?

Test Selection coverage Criteria

Help to measure the adequacy of a test suite

E.g., if we use statement coverage to test 40% of the code, it means 60% of code was never executed

Help to decide when to STOP testing

E.g., use other criteria to cover 85%

Coverage?

refers to the extent by which a given verification activity has satisfied its objectives

a measure, not a method or a test

expressed as the percentage of an activity that is

accomplished.

Types of Structural Coverages

Typically structural coverage criteria are divided into two types: Data flow Control flow

Data flow criteria? measure the flow of data between variable

assignments and references to the variables Control flow criteria?

measure the flow of control between statements and sequences of statements

E.F. Millers’s Coverage Metrics

Overview of the families of test selection criteria Families of test selection criteria include

Structural model coverage criteria All statements (C0)

All DD-path (C1) (chain)

All conditions (predicates) to each outcome (T orF) (C1p)

All paths (C)

C C1p C0

Multiple condition coverage (CMCC)

A test set achieves CMCC if it exercises all possible combinations of condition outcomes in each decision For example, for N conditions, it requires 2N tests

(truth table) Dependent pairs Cd (Define/Usage paths)

Example of MCC: a compound condition of three variables of a, b, c.

Statement testing (C0)

Let T be a test suite for a Program P, then T satisfies the statement criterion for P, iff, for each statement S of P, there exists at least one test case in T that causes the execution of S.

In terms of Flow graph model of program P, it is the same as visiting each single node on some execution path exercised by a test case in T

C0 = number of executed statements/ total number of statements

Statement testing: Small Java Program

Statement testing: Flow Graph

C0: path aceTest case: A=2, B=0, X=3

C1p: Every predicate

Let T be a test suite for a program P. T satisfies the decision adequacy criterion for P, iff, for each branch B (or predicate P), there exists at

least one test case in T that causes execution of B.

This is the same as stating that every edge in the flow graph model of program P belongs to some execution path exercised by a test case in T

C1p = number of executed branches/ total number of branches

T satisfies the branch adequacy criterion if C1p =1

predicate Testing: Flow Graph

C1: p1: acd and p2: abeT1: A=3, B=0, X=1T2: A=2, B=1, X=1

P1:a,c,d

predicateTesting: Flow Graph

C1: p1: acd T1: A=3, B=0, X=1

predicate Testing: Flow Graph

C1: p2: abeT2: A=2, B=1, X=1

Predicate Testing: Flow Graph

Four conditions:A>1, B=0, and A=2, X>1Need test cases to force when 1) A>1, and A<=12) B=0, and B<>13) A=2, and A<> 24) X>1, and X<=1

C2 metric : Simple Loop Coverage + loop coverage

C2 metric demands the coverage of C1 and loop testing

Simple loop testingRequires that every loop decision

to be tested for two possible outcomes:One to force traversing the loop

bodyThe other one to exist

Loop boundary adequacy criterion (LC)

A test suite T for a program P satisfies the LC criterion iff for each loop l in P:

There is at least one execution in which control reaches the loop, and then the loop control condition evaluates to False the first time it is evaluated (zero time)

There is at least on e execution in which control reaches the loop and then the body of the loop is executed exactly once before control leaves the loop (one time)

There is at least one execution in which the body of the loop is repeated more than once (many times)

Dd Metric: C1 coverage + every define/usage

Refers to the most common dependency in DD-paths known as define/usages (covered in chapter 9: dataflow testing)

The dependency issues closely related to infeasible path problem associated with the control flow testing

E.g., C and H, and D and H pair in figure 8.5 The H branch can be traversed s when C = true The H branch can not be traversed when D=

false The path is syntactically feasible but

semantically is impossible

CMCC: Multiple Condition Coverage

A more complete extension that includes both the basic condition and branch adequacy criteria is called the multiple condition coverage CMCC

CMCC requires a test case for each possible evaluation of compound conditions. For N basic conditions, we need 2N

combinations of N basic conditionsShort-circuit evaluation is effective in

reducing the above number to a more manageable number.

CMCC: Multiple Condition Coverage

Four conditions:A>1, B=0, and A=2, X>1Need test cases to force when 1) A>1, and A<=12) B=0, and B<>13) A=2, and A<> 24) X>1, and X<=1

Test cases:T1: A=2, B=0, X=4T2: A=1, B=1, X=1

Truth table for (A>1 AND B =0)

conditions

T1 T2 T3 T4

A true true false false

B true false true falseA>1 AND B=0

true false false false

ACTIONS

X=X/A x - - -

Truth table for (A=2 or X>1)

conditions

T5 T6 T7 T8

A true true false false

X true false true falseA=2 OR x>1

true true true false

ACTIONS

X=X+1 x - - -

Test Data and Test cases for CMCC

General Form for Compound Condition

Example 2: Compound Condition

Modified Condition/Decision Coverage (MC/DC)

MC/DC Requires that each basic condition be shown to

independently affect the outcome of each decision a condition has independent effect when that condition

alone determines the outcome of the decision (AKA unique-cause approach)

For each basic condition C, select two test cases such that the truth values of all

evaluated conditions except C are the same, Also,the compound condition as a whole evaluates to

True for one of those test cases and False for the otherMC/DC is mandated by RTCA/Do-178B

Terminologies: Condition, Decision

Condition? A Boolean expression containing no Boolean

operators. E.g., A>B

Decision (compound Boolean expressions)? A Boolean expression composed of conditions and

zero or more Boolean operators. (AND, OR) (A>B OR (C=B AND D=H))

A decision without a Boolean operator is a condition. Also, If a condition appears more than once in a

decision, each occurrence is a distinct condition.

Understanding Logical operator

Minimum testing to meet MC/DC for AND

Minimum testing to meet MC/DC for OR

Minimum testing to meet MC/DC for OR

MC/DC for Z= (A or B) AND (C OR D)

Test case 2 is coupled by test case 10

for A

Test cases from 1-16

Path testing C (or P)

A test suite T for a program P satisfies the path adequacy criterion iff

for each path pi in P, there exists at least one test case in T that causes the execution of pi

C = # of executed paths/total number of paths

Linearly independent paths

ex1= p2+p3-p1

ex1= (1,0,1,2,0,0,0,0,1,0) +(1,0,0,0,1,0,0,1,0,1)-(1,0,0,1,0,0,0,0,1,0)

ex1 = (1,0,1,1,1,0,0,1,0,1)=A,B,C,B,E,F,G

ex2=2p2-p1

ex2=(2,0,2,4,0,0,0,0,2,0) – (1,0,0,1,0,0,0,0,1,0)= (1,0,2,3,0,0,0,0,1,0)=A,B,C,B,C,B,C,G

McCabe’s Algorithmic Procedure (Baseline method)

McCable’s Algorithmic Procedure can be used to identify a set of basis paths Select a path with highest number of decision nodes Retrace each decision in baseline path (i.e., normal path

that can be arbitrary path) Flip each decision to create a new path

E.g. p1: A,B,C,B,E,F,G (baseline, flip A)

P2: A,D,E,F,G (flip p1 at D)

P3:A,D,F,G (take D)

P4:A,B,E,F,G (flip p1 at B)

P5:A,B,C,G (flip p1 at C)

-The basis paths is different from the one in table 2

-A basis path is not required to be unique

t f

t

ff

f

f

Example: Applying McCabe Procedure

p1

p2

p3

p4

p5

Observations on McCabe’s Basis Path

Infeasible paths

Essential Complexity

V(G)=1wellStr(P)

V(G) >=10 requires substantial testing

Use the characteristics of code to select appropriate criteria

More on White-box testing

The purpose of structural coverage analysis with the associated structural coverage analysis resolution is to complement requirements-based testing as follows:

Provide evidence that the code structure was verified to the degree required for the applicable software level;

Provide a means to support demonstration of absence of unintended functions;

no information about whether the code is doing what it is supposed to be doing as specified in the requirements

Establish the thoroughness of requirements-based testing.

References

Material in this presentation is taken from the following resources: Paul C. Jorgensen. Software Testing: A

Craftsman’s Approach, 3rd Edition. Auerbach Publication, 2008

Mauro Pezze and Michal Young. Software Testing And Analysis. Wiley 2008

Kelly j. Hayhurst et al. A Practical Tutoring on Modified Condition/Decision Coverage NASA/TM-2001-210876, May 2001.