chapter 8 path testing. path testing path testing is a “design structural testing” in that it is...
TRANSCRIPT
Chapter 8
Path Testing
Path testing
• Path testing is a “design structural testing” in that it is based on detailed design & the source code of the program to be tested.
• The methodology uses the graphical representation of the source code– Thus it is very much “control flow” or “path”
oriented
• This methodology has been available since the mid-1970’s and is an important White Box testing technique
Path Testing
• Paths derived from some graph construct.
• When a test case executes, it traverses a path.
• Huge number of paths implies some simplification is needed.
• Big Problem: infeasible paths.
• By itself, path testing can lead to a false sense of security.
Program Graphs
Definition: Given a program written in an imperative programming language, its program graph is a directed graph in which nodes are statement fragments, and edges represent flow of control. (A complete statement is a “default” statement fragment.)
Program Graphs of Structured Programming Constructs
Case/Switch
1 Case n Of 32 n=1: 3 <case 1 statements>4 n=2:5 <case 2 statements>6 n=3:7 <case 3 statements>8 End Case
1
2
3
64
5 7
8
Post-test Loop
1 Do2 <repeated body>3 Until <condition>4 <next statement>
1
2
3
4
If-Then-Else
1 If <condition>2 Then 3 <then statements>4 Else5 <else statements>6 End If7 <next statement>
1
2
3
6
4
5
7
Pre-test Loop
1 While <condition>2 <repeated body>3 End While4 <next statement>
1
2
3
4
Sample Program Graph
4
23
5 6 7 8
9
10 11
12
16
17 18
19
13
21
14
15
22
20
1 Program triangle2 2 Dim a,b,c As Integer3 Dim IsATriangle As Boolean4 Output("Enter 3 integers which are sides of a triangle")5 Input(a,b,c)6 Output("Side A is ",a)7 Output("Side B is ",b)8 Output("Side C is ",c)9 If (a < b + c) AND (b < a + c) AND (c < a + b)10 Then IsATriangle = True11 Else IsATriangle = False12 EndIf 13 If IsATriangle 14 Then If (a = b) AND (b = c)15 Then Output ("Equilateral")16 Else If (a≠b) AND (a≠c) AND (b≠c)17 Then Output ("Scalene")18 Else Output ("Isosceles")19 EndIf20 EndIf21 Else Output("Not a Triangle")22 EndIf23 End triangle2
A
B
C D E
F
G
first
last
Common Objection to Path-Based Testing
(Trillions of Paths)
If the loop executes up to 18 times, there are 4.77 Trillion paths. Impossible, or at least, infeasible, to test them all. [Schach]
50 + 51 + 52 + … + 518 = 4,768,371,582,030
Stephen R. Schach, Software Engineering, , (2nd edition)Richard D. Irwin, Inc. and Aksen Associates, Inc. 1993
Schach’s flowchart
Char = ?
Condition 1 Condition 2
Condition 3
Block A Block B
Block C
Block D
‘A’ ‘B’
‘C’
‘True’ ‘True’
‘False’‘False’
Loop <= 18 times
Schach’s 4.77 Trillion loop
Test Cases for Schach’s “Program”
1. First-A-B-C-F-G-Last2. First-A-B-C-D-F-G-Last3. First-A-B-D-F-G-A-B-D-F-G-Last 4. First-A-B-E-F-G-Last 5. First-A-B-E-D-F-G-Last
These test cases cover• Every node• Every edge• Normal repeat of the loop• Exiting the loop
A
B
C D E
F
G
first
last
Transforming Code to Diagram
• In early chapter (chapter 4) on graph theory, we mentioned condensation graph as a graph that was derived by “condensing” a component (or set of code) into a node as a way to simplify the graphical diagram of source code.
• Here we will :– assume that we have a condensed graph and
discuss path analysis
Path Analysis
• Why path analysis for test case design?– Provides a systematic methodology of White Box
testing.• Reproducible• Traceable• Countable
• What is path analysis?– Analyzes the number of paths that exist in the
system– Facilitates the decision process of how many paths
to include in the test
Reasons similar to why documenttest cases
Path, as previously defined in Graph Theory, is a sequence of connected nodes
Linearly Independent Path
• A path through the system is Linearly Independent** from other paths only if it includes some segment or edge that is not covered in the other paths.
S1
S3
S2
C1
1
4
2
3
- The statements are representedby the rectangular and diamond blocks (nodes).- The segments between the blocks arethe edges, labeled with numbered circles.
Path1 : S1 – C1 – S3Path2 : S1 – C1 – S2 – S3 ORPath1: edges (1,4)Path2: edges (1,2,3)
Path1 and Path2 are linearly independent because each includes some edge that is not included in the others. (note: not necessarily nodes)
** This definition will require more explanation later.
Another Example of Linearly Independent Paths
S1
S2
S3
S4
S5
C1
C2
C3
1
5
6
7
2
4
3
10
9
8
Path1: edges (1,2,8)Path2: edges (1,5,3,9)Path3: edges (1,5,6,4,10)Path4: edges (1,5,6,7)
Note that these are all linearly independent
Statement Coverage Method• Count all the linearly independent paths• Pick the minimum number of linearly independent paths that will
include all the statements (nodes) (S’s and C’s in the diagram)
S1
S3
S2
C1
1
4
2
3
Path1 : S1 – C1 – S3 : edges (1, 4)Path2 : S1 – C1 – S2 – S3 : edges (1, 2. 3 )
Are both Path1 and Path2 needed to cover all the statements: (S1,C1,S2,S3) ? What would you say?
Another Example of Statement Coverage
S1
S2
S3
S4
S5
C1
C2
C3
1
5
6
7
2
4
3
10
9
8
The 4 Linearly Independent Paths Covers:
Path1: includes S1-C1-S2-S5 : edges (1,2,8)Path2: includes S1-C1-C2-S3-S5 : edges ( 1,5,3,9)Path3: includes S1-C1-C2-C3-S4-S5 : edges (1,5,6,4,10)Path4: includes S1-C1-C2-C3-S5 : edges (1,5,6,7)
For 100% Statement Coverage, all we need are 3 paths : Path1, Path2, and Path3 to cover all the statements (S1,C1,S2,C2,S3,C3,S4,S5) - - - no need for Path4 - - - - !!
Statement Coverage
Now - - -
What do you think of a Software Companywhich boasts that they run every statementin the software before release?
Try this on some students or other professorsand see their reaction.
Branch Coverage Method(Also DD-Paths of our text book)
1. Identify all the decisions
2. Count all the branches from each of the decisions ( “out-degree” of the decision node)
3. Pick the minimum number of paths that will cover all the branches from the decisions.
Branch Coverage Method
S1
S3
S2
C1
1
4
2
3
One decision C1 :
B1 : Path1 : C1 – S3 B2 : Path2 : C1 – S2
Are both Path1 and Path2 needed to cover Branch1 and Branch2 from C1?
Branch 1 Branch 2
2 linearly independent paths cover :
B1 : Path1 : S1 - C1 – S3 B2 : Path2 : S1 - C1 – S2 – S3
Another Example of Branch Coverage
S1
S2
S3
S4
S5
C1
C2
C3
1
5
6
7
2
4
3
10
9
8
The 3 Decisions and the Branches:
C1: - B1 : C1- S2 - B2 : C1- C2 C2: - B3 : C2 – S3 - B4 : C2 – C3 C3: - B5 : C3 – S4 - B6 ; C3 – S5
We need: Path1 to cover B1, Path2 to cover B2 and B3, Path3 to cover B4 and B5, Path4 to cover B6
The 4 Linearly Independent Paths Cover:
Path1: includes S1-C1-S2-S5Path2: includes S1-C1-C2-S3-S5Path3: includes S1-C1-C2-C3-S4-S5Path4: includes S1-C1-C2-C3-S5
Branch Coverage
How do you feel about a Software Companywho states that they test every branch in theSoftware before release?
Does All Branch Coverage subsume All Statements Coverage?
DD-Paths
A DD-Path (decision-to-decision) is a chain in a program graph such that
Case 1: it consists of a single node with indeg = 0, Case 2: it consists of a single node with outdeg = 0, Case 3: it consists of a single node with indeg ≥ 2 or
outdeg ≥ 2, Case 4: it consists of a single node with indeg = 1 and
outdeg = 1, Case 5: it is a maximal chain of length ≥ 1.
DD-Path Graph
Given a program written in an imperative language, its DD-Path graph is the directed graph in which nodes are DD-Paths of its program graph, and edges represent control flow between successor DD-Paths. • a form of condensation graph • 2-connected components are collapsed into an individual node • single node DD-Paths (corresponding to Cases 1 - 4 ) preserve the convention that a statement fragment is in exactly one DD-Path
Condensation of Code to Table then to Graph
1. Program Triangle2. Dim a, b,c As Integer3. Dim IsTriangle As Boolean
4. Output ( “enter a,b, and c integers”)5. Input (a,b,c)6. Output (“side 1 is”, a)7. Output (“side 2 is”, b)8. Output (”side 3 is”, c)
9. If (a<b+c) AND (b<a+c) And (c<b+a)10. then IsTriangle = True11. else IsTriangle = False 12. endif
13. If IsTriangle14. then if (a=b) AND (b=c)15. then Output (“equilateral”)16. else if (a NE b) AND (a NE b) AND (b NE c)17. then Output ( “Scalene”)18. else Output (“Isosceles”)19. endif20. endif21. else Output (“not a triangle”)22. endif23. end Triangle2
Psuedo-code
Def of DD-paths on
Page 140
code statementPath/nodename
DD-pathDef.
Skip 1- 3(or w/4)4 first 15 – 8 A 5
9 B 310 C 411 D 412 E 313 F 314 H 315 I 416 J 317 K 418 L 419 M 320 N 321 G 422 O 323 last 2
Condensation Graph from Table
first
A
B
C D
E
F
G H
I J
K L
MN
O
Last
- Statements coverage - 4 paths- Branch (DD-path) coverage - 4 paths- Cyclomatic # = 4+1 = 5 - 5 lin. Ind paths- All combinations - 8 paths
Closer Look into Path Testing
• Look at 2 paths from A to E. The paths of either it is a triangle or not a triangle.
– But there are many combinations to get “not triangle” ; so we still need to consider utilizing boundary values and equivalence class. Just one test case to cover the path may not be enough
• Look at the path that leads from D to L (D= not triangle and L= Isosceles triangle). Is that path possible - - - can you generate a test case for that path?
• Look at the path from C to G (C = Is a triangle and G = output “not triangle”) - - - is this a possible path?
– There may be logical dependencies that prevent us from generating test case to traverse certain paths. Thus we also need to consider employing decision tables from Black Box testing technique.
Condensation of Code to Table then to Graph
1. Program Triangle2. Dim a, b,c As Integer3. Dim IsTriangle As Boolean
4. Output ( “enter a,b, and c integers”)5. Input (a,b,c)6. Output (“side 1 is”, a)7. Output (“side 2 is”, b)8. Output (”side 3 is”, c)
9. If (a<b+c) AND (b<a+c) And (c<b+a)10. then IsTriangle = True11. else IsTriangle = False 12. endif
13. If IsTriangle14. then if (a=b) AND (b=c)15. then Output (“equilateral”)16. else if (a NE b) AND (a NE b) AND (b NE c)17. then Output ( “Scalene”)18. else Output (“Isosceles”)19. endif20. endif21. else Output (“not a triangle”)22. endif23. end Triangle2
Psuedo-code
Def of DD-paths on
Page 140
code statementPath/nodename
DD-pathDef.
Skip 1- 3(or w/4)4 first 15 – 8 A 5
9 B 310 C 411 D 412 E 313 F 314 H 315 I 416 J 317 K 418 L 419 M 320 N 321 G 422 O 323 last 2
McCabe’s Cyclomatic Number
• Is there a way to know how many linearly independent paths exist?– McCabe’s Cyclomatic number which we used to
study program complexity may be applied. There are 3 ways to get the Cyclomatic Complexity number from a flow diagram.
• # of binary decisions + 1 • # of edges - # of nodes + 2• # of closed regions + 1
McCabe’s Cyclomatic Complexity NumberEarlier Example
S1
S3
S2
C1
1
4
2
3
We know there are 2 linearly independentpaths from before: Path1 : C1 – S3 Path2 : C1 – S2 – S3
McCabe’s Cyclomatic Number:
a) # of binary decisions +1 = 1 +1 = 2
b) # of edges - # of nodes +2 = 4-4+2 = 2
c) # of closed regions + 1 = 1 + 1 = 2
Closed region
McCabe’s Cyclomatic Complexity NumberAnother Example
S1
S3
S2
C1
1
4
2
3
McCabe’s Cyclomatic Number:
a) # of binary decisions +1 = 2 +1 = 3
b) # of edges - # of nodes +2 = 7-6+2 = 3
c) # of closed regions + 1 = 2 + 1 = 3
Closed Region
C2
S47
5
6
Closed Region
There are 3 Linearly Independent Paths
An example of 2n total path
S1
C1
C2
C3
S2
S3
S4S5
2
1
3
4
5
6
8
9
7
Since for each binary decision, there are 2 paths andthere are 3 in sequence, there are 23 = 8 total “logical” paths path1 : S1-C1-S2-C2-C3-S4 path2 : S1-C1-S2-C2-C3-S5 path3 : S1-C1-S2-C2-S3-C3-S4 path4 : S1-C1-S2-C2-S3-C3-S5
path5 : S1-C1-C2-C3-S4 path6 : S1-C1-C2-C3-S5 path7 : S1-C1-C2-S3-C3-S4 path8 : S1-C1-C2-S3-C3-S5
How many Linearly Independent paths are there?Using Cyclomatic number = 3 decisions +1 = 4
One set would be: path1 : includes edges (1,2,4,6,9) path2 : includes edges (1,2,4,6,8) path3 : includes edges (1,2,4,5,7,9) path5 : includes edges (1,3,6,9)
Note 1: with just 2 paths ( Path1 and Path8) all the statements are covered.
Note2: with just 2 paths ( Path1 and Path8) all the branches are covered.
Example with a Loop
S1
S2
S3C1
3
2
1
4
Total number of ‘logical’ paths may be “infinite” (very large) because of the loop ---- so we will only look at this “statically”
Linearly Independent Paths = 1 decision +1 = 2
path1 : S1-C1-S3 (segments 1,4 ) path2 : S1-C1-S2-C1-S3 (segments 1,2,3,4 )
“One path” will cover all statements: path2 : S1-C1-S2-C1-S3 (S1,C1,S2,S3)
“One path” will cover all branches: path2 : S1-C1-S2-C1-S3 branch1 (C1-S2) and branch 2 (C1-S3)
*** Note***: To get one path --- S2 must somehow change C1
Example with a Loop (cont.)
S1
S2
S3C1
3
2
1
4
Total number of ‘logical’ paths may be “infinite” (very large) because of the loop
Linearly Independent Paths = 1 decision +1 = 2
path1 : S1-C1-S3 (segments 1,4 ) path2 : S1-C1-S2-C1-S3 (segments 1,2,3,4 )
“One path” will cover all branches: path2 : S1-C1-S2-C1-S3 branch1 (C1-S2) and branch 2 (C1-S3)
Should we considerthe loop boundaries in C1? & also bring in our boundary value testing thoughts?
McCabe Basis Paths in the Triangle Program
V(G) = 23 - 20 + 2(1) = 5
Basis Path Set B1p1: 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 16, 18, 19, 20, 22, 23 (mainline)p2: 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 16, 18, 19, 20, 22, 23 (flipped at 9)p3: 4, 5, 6, 7, 8, 9, 11, 12, 13, 21, 22, 23 (flipped at 13)p4: 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 20, 22, 23 (flipped at 14)p5: 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 16, 17, 19, 20, 22, 23 (flipped at 16)
There are 8 topologically possible paths.4 are feasible, and 4 are infeasible.
Exercise: Is every basis path feasible?13
14
15 16
17 18
19
20
21
22
4 5 6 7 8
10
9
11
12
23
Condensation with Structured Programming Constructsfirst
A
B
C D
E
J
K L
M
F
G H
I
O
N
last
a
first
A
a
b
F
G H
I
O
N
last
c
b
first
A
a
J
K L
M
F
G H
I
O
N
last
first
A
a
F
G c
O
last
d
first
A
a
d
last
e
first
e
Violations of Structured Programming Precepts
Branching out of a decision
1
2
3
6
4
5
7
8
Branching into a decision
1
2
3
6
4
5
7
8
Branching out of a loop
5
1
2
3
4
Branching into a loop
1
2
3
4
5
More on Linearly Independent Paths
• In discussing dimensionality, we talks about orthogonal vectors (or “basis” of vector space) .
– (e.g.) Two dimensional space has two orthogonal vector from which all the other vectors in two dimension can be obtained via “linear combination” of these 2 vectors:
• [1,0]• [0,1]
[1,0]
[0,1]
[2,4]
e.g. [2,4] = 2[1,0] + 4[0,1]
More on Linearly Independent Paths
• A set of paths is considered to be a Linearly Independent Set if every path may be constructed as a “linear combination” of paths from the linearly independent set. For example:
C1
C2
S1
S2
6
5
4
3
21
We already know: a) there are a total of 22=4 logical paths. b) 1 path (path4) will cover all statements c) 2 paths will cover all branches. d) 2 branches +1 = 3 linearly independent paths.
path1
path2
path3
path4
1 2 3 4 5 6
1 1 1
1 1
1 1 1
1 1 1 1
path 4 = path3 + path1 – path2 = (0,1,1,1,0,0)+(1,0,0,0,1,1)- (1,0,0,1,0,0) = (1,1,1,1,1,1) - (1,0,0,1,0,0) = (0,1,1,0,1,1)
We pick: path1,path2 and path3 as the Linearly Independent Set
Other Sets of Linearly Independent Paths
-Consider the set of linearly independent paths: 1, 2 & 4 instead.
- Can we get Path3 = (0,1,1,1,0,0)?-Consider path3 = path2 + path4 – path1 = (1,0,0,1,0,0,)+(0,1,1,0,1,1) – (1,0,0,0,1,1) = (0,1,1,1,0,0)
path1
path2
path3
path4
1 2 3 4 5 6
1 1 1
1 1
1 1 1
1 1 1 1
-Consider another set of linearly independent paths: 2, 3 & 4 instead.
- Can we get Path 1 = (1,0,0,0,1,1))?-Consider path1 = path2 + path4 – path3 = (1,0,0,1,0,0,)+(0,1,1,0,1,1) – (0,1,1,1,0,0) = (1,0,0,0,1,1)
More on Linearly Independent Paths
C1
C2
S1
S2
6
5
4
3
21
path1
path2
path3
path4
1 2 3 4 5 6
1 1 1
1 1
1 1 1
1 1 1 1
Note : Although path1 and path3 are linearly independent, they do NOT form a Linearly Independent Set because no linear combination of path1 and path3 can get , say, path4.
We already know: a) there are a total of 22=4 logical paths. b) 1 path (path4) will cover all statements c) 2 paths will cover all branches. d) 2 branches +1 = 3 linearly independent paths.
More on Linearly Independent Paths
• Because the Linearly Independent Set of paths display the same characteristics as the mathematical concept of basis in n-dimensional vector space, the testing using the Linearly Independent Set of paths is sometimes called the “basis” testing.
• The main notion is that since the “linear independent set of paths” as a set can span all the paths for the design/code construct, then basis testing covers the “essence” of the whole structure.
Then ------ is there a way to find a Linearly Independent Set ?
An Algorithm to Find the Basis Set1. Select a baseline path, an arbitrary, normal execution path
that contains as many decisions as possible.2. Retrace the baseline path and “flip” each of the decision
encountered; each flip creates a new path. Continue until all the decisions are flipped
3. The basis set is composed of all the paths generated from steps 1 and 2 above
C1
C2
S1
S2
6
4
3
21
1. pick baseline path P1: C1 –S1- C2 – S2: <0,1,1,0,1,1> 2. flip C1 P2 : C1 – C2 – S2 : <1,0,0,0.1,1>3. flip C2 P3: C1 - C2 : <1,0,0,1,0,0>
Cyclomatic # = 2+ 1 = 3;So there are 3 linearly independent paths
5
Can we get the 4th path : C1 – S1 – C2 : <0,1,1,1,0,0> from the above basis set?
How about : (P1 + P3) – P2 ?(P1 + P3) – P2 = (<0,1,1,0,1,1> + <1,0,0,1,0,0>) - <1,0,0,0,1,1> = <1,1,1,1,1,1> - <1,0,0,0,1,1> = <0,1,1,1,0,0> = P4
Total “Possible” Logical Paths can be Big!
s1
c1
s2 c3
s3
c2
c4
s4
4
1
2
35
There are 5 choices each time we process throughthis loop.
For passing through the loop n times we have 5n
possibilities of logical paths.
If we go through the loop just 3 times, we have (5)3 = 125possible paths!
You also note that for nloops there may be 3n decisions. e.g.:We have 3*3 = 9 decisions for looping 3 times.
Paths Analysis
• Interested in total number of all possible combinations of “logical” paths
• Interested in Linearly Independent paths• Interest in Branch coverage or DD-path• Interested in Statement coverage