cmpt 473 software quality assurance graph coveragewsumner/teaching/473/06-graphcoverage.pdf ·...
TRANSCRIPT
![Page 1: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/1.jpg)
Graph Coverage
CMPT 473Software Quality Assurance
Nick Sumner
![Page 2: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/2.jpg)
2
Recall: Coverage/Adequacy
● Can't look at all possible inputs.● Need to determine if a test suite covers / is
adequate for our quality objectives.
![Page 3: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/3.jpg)
3
Recall: Coverage/Adequacy
● Can't look at all possible inputs.● Need to determine if a test suite covers / is
adequate for our quality objectives.● So far: Input & Requirements based
![Page 4: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/4.jpg)
4
Recall: Coverage/Adequacy
● Can't look at all possible inputs.● Need to determine if a test suite covers / is
adequate for our quality objectives.● So far: Input & Requirements based
Efficiently sort a provided list in under X seconds
void sortEfficiently(List list) { if (list.size() < THRESHOLD) { sort1(list); } else { sort2(list); }}
![Page 5: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/5.jpg)
5
Recall: Coverage/Adequacy
● Can't look at all possible inputs.● Need to determine if a test suite covers / is
adequate for our quality objectives.● So far: Input & Requirements based
void sortEfficiently(List list) { if (list.size() < THRESHOLD) { sort1(list); } else { sort2(list); }}
How well can input basedtechniques test this?
Efficiently sort a provided list in under X seconds
![Page 6: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/6.jpg)
6
Recall: Coverage/Adequacy
● Can't look at all possible inputs.● Need to determine if a test suite covers / is
adequate for our quality objectives.● So far: Input & Requirements based
void sortEfficiently(List list) { if (list.size() < THRESHOLD) { sort1(list); } else { sort2(list); }}
How might we do better?
Efficiently sort a provided list in under X seconds
How well can input basedtechniques test this?
![Page 7: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/7.jpg)
7
White Box / Black Blox
● Considering only the requirements or input is a black box approach– Treats the program like an opaque box– No deep knowledge of the program's structure
![Page 8: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/8.jpg)
8
White Box / Black Blox
● Considering only the requirements or input is a black box approach– Treats the program like an opaque box– No deep knowledge of the program's structure
● Techniques that use artifacts of the program structure are white box approaches– They can 'see into' the program's implementation
![Page 9: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/9.jpg)
9
White Box Testing
● What is a simple approach that solves our problem here?
void sortEfficiently(List list) { if (list.size() < THRESHOLD) { sort1(list); } else { sort2(list); }}
![Page 10: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/10.jpg)
10
White Box Testing
● What is a simple approach that solves our problem here?
● Statement Coverage– How many of the statements did the suite test?
void sortEfficiently(List list) { if (list.size() < THRESHOLD) { sort1(list); } else { sort2(list); }}
![Page 11: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/11.jpg)
11
White Box Testing
● What is a simple approach that solves our problem here?
● Statement Coverage– How many of the statements did the suite test?
● Branch Coverage– How many of the condition outcomes were tested?
void sortEfficiently(List list) { if (list.size() < THRESHOLD) { sort1(list); } else { sort2(list); }}
![Page 12: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/12.jpg)
12
White Box Testing
● In this course, we'll mostly look at graph coverage based techniques
![Page 13: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/13.jpg)
13
White Box Testing
● In this course, we'll mostly look at graph coverage based techniques– Most commonly used metrics in the real world
![Page 14: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/14.jpg)
14
White Box Testing
● In this course, we'll mostly look at graph coverage based techniques– Most commonly used metrics in the real world– Most concepts can be modeled through graphs
e.g. programs, protocols, use patterns, designs, ...
So a bit of review...
![Page 15: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/15.jpg)
15
Graphs
● What is a graph G?
![Page 16: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/16.jpg)
16
Graphs
● What is a graph G?– A set N of nodes
B A
DEC
![Page 17: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/17.jpg)
17
Graphs
● What is a graph G?– A set N of nodes– A set E of edges
B A
DEC
![Page 18: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/18.jpg)
18
Graphs
● What is a graph G?– A set N of nodes– A set E of edges
● When edges are directed from one node to another, the graph is a directed graph
B A
DEC
![Page 19: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/19.jpg)
19
Graphs
● What is a graph G?– A set N of nodes– A set E of edges
● When edges are directed from one node to another, the graph is a directed graph
● A path is a list of pairwise connected nodes
B A
DEC
![Page 20: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/20.jpg)
20
Graphs
● What is a graph G?– A set N of nodes– A set E of edges
● When edges are directed from one node to another, the graph is a directed graph
● A path is a list of pairwise connected nodes
B A
DEC
ABCDE
![Page 21: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/21.jpg)
21
Graphs
● What is a graph G?– A set N of nodes– A set E of edges
● When edges are directed from one node to another, the graph is a directed graph
● A path is a list of pairwise connected nodes
B A
DEC
ABCDEABDEAEBCDEBD
![Page 22: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/22.jpg)
22
Control Flow Graphs (CFGs)
● Programs can be modeled as graphs– Used extensively in compilers
![Page 23: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/23.jpg)
23
Control Flow Graphs (CFGs)
● Programs can be modeled as graphs– Used extensively in compilers– Also used in testing!
![Page 24: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/24.jpg)
24
Control Flow Graphs (CFGs)
● Programs can be modeled as graphs– Used extensively in compilers– Also used in testing!
● Control Flow Graphs– Nodes comprise the code of a program
![Page 25: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/25.jpg)
25
Control Flow Graphs (CFGs)
● Programs can be modeled as graphs– Used extensively in compilers– Also used in testing!
● Control Flow Graphs– Nodes comprise the code of a program– Edges show the paths that an execution may take
through a program
![Page 26: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/26.jpg)
26
Control Flow Graphs
Example:void sortEfficiently(List list) { if (list.size() < THRESHOLD) { sort1(list); } else { sort2(list); }}
![Page 27: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/27.jpg)
27
Control Flow Graphs
Example:void sortEfficiently(List list) { if (list.size() < THRESHOLD) { sort1(list); } else { sort2(list); }}
if (list.size() < THRESHOLD)
sort1(list); sort2(list);
return;
![Page 28: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/28.jpg)
28
Control Flow Graphs
Example:void sortEfficiently(List list) { if (list.size() < THRESHOLD) { sort1(list); } else { sort2(list); }}
if (list.size() < THRESHOLD)
sort1(list); sort2(list);
return;
![Page 29: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/29.jpg)
29
Control Flow Graphs
Example:void sortEfficiently(List list) { if (list.size() < THRESHOLD) { sort1(list); } else { sort2(list); }}
if (list.size() < THRESHOLD)
sort1(list); sort2(list);
return;
size < THRESHOLD size >= THRESHOLD
![Page 30: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/30.jpg)
30
Control Flow Graphs
Example:void sortEfficiently(List list) { if (list.size() < THRESHOLD) { sort1(list); } else { sort2(list); }}
if (list.size() < THRESHOLD)
sort1(list); sort2(list);
return;
size < THRESHOLD size >= THRESHOLD
![Page 31: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/31.jpg)
31
Control Flow Graphs
● Many types of nodes:
if (list.size() < THRESHOLD)
sort1(list); sort2(list);
print(list);
list = ...
return;
size < THRESHOLD size >= THRESHOLD
![Page 32: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/32.jpg)
32
Control Flow Graphs
● Many types of nodes:
if (list.size() < THRESHOLD)
sort1(list); sort2(list);
print(list);
Entry Node
list = ...
return;
0 incoming edges
size < THRESHOLD size >= THRESHOLD
![Page 33: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/33.jpg)
33
Control Flow Graphs
● Many types of nodes:
if (list.size() < THRESHOLD)
sort1(list); sort2(list);
print(list);
Entry Node
list = ...
return;
0 incoming edges
Exit Node0 outgoing
edges
size < THRESHOLD size >= THRESHOLD
![Page 34: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/34.jpg)
34
Control Flow Graphs
● Many types of nodes:
if (list.size() < THRESHOLD)
sort1(list); sort2(list);
print(list);
Entry Node
>1 outgoing edges
list = ...Decision/Branch Node
return;
0 incoming edges
Exit Node0 outgoing
edges
size < THRESHOLD size >= THRESHOLD
![Page 35: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/35.jpg)
35
Control Flow Graphs
● Many types of nodes:
if (list.size() < THRESHOLD)
sort1(list); sort2(list);
print(list);
Entry Node
list = ...
return;
0 incoming edges
Exit Node0 outgoing
edges
>1 incoming edges
Join Node
size < THRESHOLD size >= THRESHOLD
Decision/Branch Node>1 outgoing
edges
![Page 36: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/36.jpg)
36
Control Flow Graphs
● Straight-line sequences of code are grouped into basic blocks:
if (list.size() < THRESHOLD)
sort1(list); sort2(list);
print(list);
list = ...
return;
size < THRESHOLD size >= THRESHOLD
![Page 37: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/37.jpg)
37
Control Flow Graphs
● Straight-line sequences of code are grouped into basic blocks:
list = ...if (list.size() < THRESHOLD)
sort1(list); sort2(list);
print(list);return;
size < THRESHOLD size >= THRESHOLD
![Page 38: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/38.jpg)
38
Control Flow Graphs
● Many patterns arise from common constructs
![Page 39: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/39.jpg)
39
Control Flow Graphs
q = 0;r = x;while r >= y { r = r - y; q = q + 1;}
● Many patterns arise from common constructs● What is the CFG for this program?
Tell me how to draw it.
![Page 40: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/40.jpg)
40
Control Flow Graphs
for (i = 0; i < n; i++) { foo(i);}
● What is the CFG?
![Page 41: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/41.jpg)
41
Control Flow Graphs
switch (x) { case a: foo(x); break; case b: bar(x); case c: baz(x); break;}
● What is the CFG?– Don't forget implicit behavior like default!
![Page 42: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/42.jpg)
42
Control Flow Graphs
if (x == 0 || y/x > 1) { foo(x, y);}
● What is the CFG?– Short circuit operators can lead to subtle behavior!
![Page 43: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/43.jpg)
43
Control Flow Graphs
● What is the CFG?– Control flow can become complex!
From Ammann & Offutt:
x = 0;while (x < y) { y = f (x, y); if (y == 0) { break; } else if (y < 0) { y = y*2; continue; } x = x + 1;}print (y);
![Page 44: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/44.jpg)
44
Control Flow Graphs
● What is the CFG?– Control flow can become complex!
From Ammann & Offutt:
123456789101112
x = 0;while (x < y) { y = f (x, y); if (y == 0) { break; } else if (y < 0) { y = y*2; continue; } x = x + 1;}print (y);
![Page 45: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/45.jpg)
45
Control Flow Graphs
● What is the CFG?– Control flow can become complex!
From Ammann & Offutt:
1234
6
7
10
12
x = 0;while (x < y) { y = f (x, y); if (y == 0) { break; } else if (y < 0) { y = y*2; continue; } x = x + 1;}print (y);
![Page 46: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/46.jpg)
46
Control Flow Graphs
x = 0;while (x < y) { y = f (x, y); if (y == 0) { break; } else if (y < 0) { y = y*2; continue; } x = x + 1;}print (y);
● What is the CFG?– Control flow can become complex!
From Ammann & Offutt:
1234
6
7
10
12
![Page 47: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/47.jpg)
47
Control Flow Graphs
x = 0;while (x < y) { y = f (x, y); if (y == 0) { break; } else if (y < 0) { y = y*2; continue; } x = x + 1;}print (y);
● What is the CFG?– Control flow can become complex!
From Ammann & Offutt:
1234
6
7
10
12
![Page 48: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/48.jpg)
48
Control Flow Graphs
x = 0;while (x < y) { y = f (x, y); if (y == 0) { break; } else if (y < 0) { y = y*2; continue; } x = x + 1;}print (y);
● What is the CFG?– Control flow can become complex!
From Ammann & Offutt:
1234
6
7
10
12
![Page 49: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/49.jpg)
49
Control Flow Graphs
x = 0;while (x < y) { y = f (x, y); if (y == 0) { break; } else if (y < 0) { y = y*2; continue; } x = x + 1;}print (y);
● What is the CFG?– Control flow can become complex!
From Ammann & Offutt:
1234
6
7
10
12
![Page 50: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/50.jpg)
50
Control Flow Graphs
x = 0;while (x < y) { y = f (x, y); if (y == 0) { break; } else if (y < 0) { y = y*2; continue; } x = x + 1;}print (y);
● What is the CFG?– Control flow can become complex!
From Ammann & Offutt:
1234
6
7
10
12
![Page 51: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/51.jpg)
51
Control Flow Graphs
x = 0;while (x < y) { y = f (x, y); if (y == 0) { break; } else if (y < 0) { y = y*2; continue; } x = x + 1;}print (y);
● What is the CFG?– Control flow can become complex!
From Ammann & Offutt:
1234
6
7
10
12
![Page 52: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/52.jpg)
52
Control Flow Graphs
x = 0;while (x < y) { y = f (x, y); if (y == 0) { break; } else if (y < 0) { y = y*2; continue; } x = x + 1;}print (y);
● What is the CFG?– Control flow can become complex!
From Ammann & Offutt:
1234
6
7
10
12
![Page 53: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/53.jpg)
53
CFG Coverage
● Statement Coverage → Node Coverage
Try to cover all reachable basic blocks.
![Page 54: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/54.jpg)
54
CFG Coverage
● Statement Coverage → Node Coverage
Thinking in terms of node coveragecan be more efficient. Why?
![Page 55: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/55.jpg)
55
CFG Coverage
● Statement Coverage → Node Coverage● Branch Coverage → Edge Coverage
Try to cover all reachable paths of length ≤ 1.
![Page 56: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/56.jpg)
56
CFG Coverage
● Statement Coverage → Node Coverage● Branch Coverage → Edge Coverage
How do node & edge coveragecompare? Why?
![Page 57: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/57.jpg)
57
CFG Coverage
● Statement Coverage → Node Coverage● Branch Coverage → Edge Coverage
Are these notions of coverageenough? Why?
How do node & edge coveragecompare? Why?
![Page 58: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/58.jpg)
58
Pragmatic Concerns
● The goal is full coverage (of whatever criteria)
![Page 59: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/59.jpg)
59
Pragmatic Concerns
● The goal is full coverage (of whatever criteria)
Is that reasonable in practice?Why?
![Page 60: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/60.jpg)
60
Pragmatic Concerns
● The goal is full coverage (of whatever criteria)● We must consider reachability
![Page 61: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/61.jpg)
61
Pragmatic Concerns
● The goal is full coverage (of whatever criteria)● We must consider reachability
Try to cover all reachable basic blocks.
Try to cover all reachable paths of length ≤ 1.
![Page 62: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/62.jpg)
62
Pragmatic Concerns
● The goal is full coverage (of whatever criteria)● We must consider reachability
– Syntactic Reachability● Based on the structure of the code
ENTER
...
return;
print(“Got Here”)
return;
![Page 63: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/63.jpg)
63
Pragmatic Concerns
● The goal is full coverage (of whatever criteria)● We must consider reachability
– Syntactic Reachability● Based on the structure of the code
ENTER
...
return;
print(“Got Here”)
return;
How could this happen?
![Page 64: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/64.jpg)
64
Pragmatic Concerns
● The goal is full coverage (of whatever criteria)● We must consider reachability
– Syntactic Reachability● Based on the structure of the code
– Semantic Reachability● Based on the meaning of the code
condition = false;if (condition)
... ...
return;
![Page 65: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/65.jpg)
65
Pragmatic Concerns
● The goal is full coverage (of whatever criteria)● We must consider reachability
– Syntactic Reachability● Based on the structure of the code
– Semantic Reachability● Based on the meaning of the code
condition = false;if (condition)
... ...
return;
This can beundecidable!
![Page 66: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/66.jpg)
66
Pragmatic Concerns
● The goal is full coverage (of whatever criteria)● We must consider reachability
– Syntactic Reachability● Based on the structure of the code
– Semantic Reachability● Based on the meaning of the code
● So what do you do in practice?– No, really. What have you done in practice?
![Page 67: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/67.jpg)
67
Pragmatic Concerns
● The goal is full coverage (of whatever criteria)● We must consider reachability
– Syntactic Reachability● Based on the structure of the code
– Semantic Reachability● Based on the meaning of the code
● So what do you do in practice?– No, really. What have you done in practice?– Relative degrees of coverage matter (40%? 80%?)
![Page 68: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/68.jpg)
68
CFG Coverage
● The path taken by each test can matter
if (condition 1)
... x = 0;
if (condition2)
... z = y/x;
return;
![Page 69: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/69.jpg)
69
CFG Coverage
● The path taken by each test can matter
if (condition 1)
... x = 0;
if (condition2)
... z = y/x;
return;
![Page 70: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/70.jpg)
70
CFG Coverage
● The path taken by each test can matter
if (condition 1)
... x = 0;
if (condition2)
... z = y/x;
return;
![Page 71: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/71.jpg)
71
CFG Coverage
● The path taken by each test can matter
if (condition 1)
... x = 0;
if (condition2)
... z = y/x;
return;
Full edge coverage& no bugs found
![Page 72: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/72.jpg)
72
CFG Coverage
● The path taken by each test can matter
if (condition 1)
... x = 0;
if (condition2)
... z = y/x;
return;
How can we make sure to find this bug?
![Page 73: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/73.jpg)
73
CFG Coverage
● The path taken by each test can matter
if (condition 1)
... x = 0;
if (condition2)
... z = y/x;
return;
![Page 74: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/74.jpg)
74
CFG Coverage
● The path taken by each test can matter
if (condition 1)
... x = 0;
if (condition2)
... z = y/x;
return;
![Page 75: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/75.jpg)
75
CFG Coverage
● The path taken by each test can matter
if (condition 1)
... x = 0;
if (condition2)
... z = y/x;
return;
Testing all pathsexposes this bug.
![Page 76: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/76.jpg)
76
Path Coverage
● Complete Path Coverage– Test all paths through the graph
![Page 77: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/77.jpg)
77
Path Coverage
● Complete Path Coverage– Test all paths through the graph
Is this reasonable?Why?
![Page 78: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/78.jpg)
78
Path Coverage
● Complete Path Coverage– Test all paths through the graph
B
CD
A
Is this reasonable?Why?
![Page 79: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/79.jpg)
79
Path Coverage
● Complete Path Coverage– Test all paths through the graph
B
CD
A
Infeasibility
Is this reasonable?Why?
![Page 80: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/80.jpg)
80
Path Coverage
● Complete Path Coverage– Test all paths through the graph
B
CD
A
Infeasibility
How many paths?How does this relate to input
based approaches?
A
CB
D
FE
G
H
JI
K
... ...
Is this reasonable?Why?
![Page 81: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/81.jpg)
81
Path Coverage
● Complete Path Coverage– Test all paths through the graph
B
CD
A
Infeasibility
A
CB
D
FE
G
H
JI
K
... ... Intractability
Is this reasonable?Why?
How many paths?How does this relate to input
based approaches?
![Page 82: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/82.jpg)
82
Compromises?
● What could we do instead?(How did we handle the input based approaches?)
![Page 83: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/83.jpg)
83
Compromises?
● Edge Pair Coverage– Each path of length <= 2 is tested.
● Specified Path Coverage– Given a number k, test k paths
![Page 84: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/84.jpg)
84
Compromises?
● Edge Pair Coverage– Each path of length <= 2 is tested.
● Specified Path Coverage– Given a number k, test k paths
A
CB
D
FE
G
H
JI
K
... ...
B
CD
A
What do these look like?
![Page 85: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/85.jpg)
85
Compromises?
● Edge Pair Coverage– Each path of length <= 2 is tested.
● Specified Path Coverage– Given a number k, test k paths
A
CB
D
FE
G
H
JI
K
... ...
B
CD
A
What do these look like?
Are they good?
![Page 86: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/86.jpg)
86
Coping With Loops
● What criteria do you use when testing loops?
![Page 87: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/87.jpg)
87
Coping With Loops
● What criteria do you use when testing loops?● Simple Paths
– A path between nodes is simple if no node appears more than once. (Except maybe the first and last)
– Captures the acyclic behaviors of a program
![Page 88: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/88.jpg)
88
Coping With Loops
● What criteria do you use when testing loops?● Simple Paths
– A path between nodes is simple if no node appears more than once. (Except maybe the first and last)
– Captures the acyclic behaviors of a program
How many may there be?
A
B C
D
![Page 89: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/89.jpg)
89
Coping With Loops
● What criteria do you use when testing loops?● Simple Paths
– A path between nodes is simple if no node appears more than once. (Except maybe the first and last)
– Captures the acyclic behaviors of a program
How many may there be?
A
B C
D
![Page 90: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/90.jpg)
90
Coping With Loops
● What criteria do you use when testing loops?● Simple Paths
– A path between nodes is simple if no node appears more than once. (Except maybe the first and last)
– Captures the acyclic behaviors of a program
How many may there be?
A
B C
D
![Page 91: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/91.jpg)
91
Coping With Loops
● What criteria do you use when testing loops?● Simple Paths
– A path between nodes is simple if no node appears more than once. (Except maybe the first and last)
– Captures the acyclic behaviors of a program● Prime Paths
– A simple path that is not a subpath of any other simple path
![Page 92: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/92.jpg)
92
Coping With Loops
● What criteria do you use when testing loops?● Simple Paths
– A path between nodes is simple if no node appears more than once. (Except maybe the first and last)
– Captures the acyclic behaviors of a program● Prime Paths
– A simple path that is not a subpath of any other simple path
What does this provide?What do they look like?
A
B C
D
![Page 93: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/93.jpg)
93
Coping With Loops
● Prime Path Coverage– Cover all prime paths
![Page 94: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/94.jpg)
94
Coping With Loops
● Prime Path Coverage– Cover all prime paths
A
G
B
C
ED
F
Example from Ammann & Offutt
![Page 95: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/95.jpg)
95
Coping With Loops
● Prime Path Coverage– Cover all prime paths
A
G
B
C
ED
F
Example from Ammann & Offutt
What are the prime paths?
![Page 96: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/96.jpg)
96
Coping With Loops
● Prime Path Coverage– Cover all prime paths
A
G
B
C
ED
F
Example from Ammann & Offutt
What are the prime paths?
How many simple paths?
![Page 97: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/97.jpg)
97
Coping With Loops
● Prime Path Coverage– Cover all prime paths
A
G
B
C
ED
F
Example from Ammann & Offutt
What are the prime paths?
How many simple paths?
Can you intuitively explain what prime paths capture?
![Page 98: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/98.jpg)
98
Coping With Loops
● Prime Path Coverage– Cover all prime paths
Are these tests good or bad?
![Page 99: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/99.jpg)
99
Coping With Loops
● Prime Path Coverage– Cover all prime paths
Are these tests good or bad?
Do they address all of the problems with path coverage?
![Page 100: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/100.jpg)
100
Coping With Loops
● Prime Path Coverage– Cover all prime paths
Are these tests good or bad?
Do they address all of the problems with path coverage?
Can you think of things they miss?
![Page 101: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/101.jpg)
101
How Do They Relate?All / Complete Path Coverage
How do they compare w/ edge coverage?
Prime Path Coverage
![Page 102: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/102.jpg)
102
How Do They Relate?All / Complete Path Coverage
Prime Path Coverage
Edge Coverage
Node Coverage
![Page 103: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/103.jpg)
103
How Do They Relate?All / Complete Path Coverage
Prime Path Coverage
Edge Coverage
Node Coverage
How do the compare w/ edge pair coverage?
![Page 104: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/104.jpg)
104
How Do They Relate?All / Complete Path Coverage
Prime Path CoverageEdge Pair Coverage
Edge Coverage
Node Coverage
![Page 105: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/105.jpg)
105
How Do They Relate?All / Complete Path Coverage
Prime Path CoverageEdge Pair Coverage
Edge Coverage
Node Coverage
Consider:
A
B
![Page 106: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/106.jpg)
106
How Do They Relate?All / Complete Path Coverage
Prime Path CoverageEdge Pair Coverage
Edge Coverage
Node Coverage
Why did we look at prime paths at all?
![Page 107: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/107.jpg)
107
How Do They Relate?All / Complete Path Coverage
Prime Path CoverageEdge Pair Coverage
Edge Coverage
Node Coverage
Why did we look at prime paths at all?
So let's only consider loops....
![Page 108: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/108.jpg)
108
How Do They Relate?All / Complete Path Coverage
Prime Path CoverageEdge Pair Coverage
Edge Coverage
Node Coverage
Complete RoundTrip Coverage
Simple RoundTrip Coverage
Round Trip – A prime path starting and ending with the same node
![Page 109: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/109.jpg)
109
How Do They Relate?All / Complete Path Coverage
Prime Path CoverageEdge Pair Coverage
Edge Coverage
Node Coverage
Complete RoundTrip Coverage
All round trips for each node
![Page 110: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/110.jpg)
110
How Do They Relate?All / Complete Path Coverage
Prime Path CoverageEdge Pair Coverage
Edge Coverage
Node Coverage
Complete RoundTrip Coverage
Simple RoundTrip Coverage
Just one round trip for each node
![Page 111: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/111.jpg)
111
Turning Them Into Tests
● Reconsider:
A
G
B
C
ED
F
![Page 112: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/112.jpg)
112
Turning Them Into Tests
● Reconsider:
A
G
B
C
ED
F
Is this path prime?
![Page 113: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/113.jpg)
113
Turning Them Into Tests
● Reconsider:
A
G
B
C
ED
F
Is this path prime?
Is it still useful?
![Page 114: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/114.jpg)
114
Turning Them Into Tests
● Reconsider:
A
G
B
C
ED
F
Is this path prime?
Is it still useful?
One test may cover multiple prime paths!Requirements ≠ Tests
![Page 115: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/115.jpg)
115
Turning Them Into Tests
● Relaxing our path requirements can help, too
![Page 116: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/116.jpg)
116
Turning Them Into Tests
● Relaxing our path requirements can help, too● Tour
– A path p tours path q if q is a subpath of p– A test covers any prime path it tours
![Page 117: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/117.jpg)
117
Turning Them Into Tests
● Relaxing our path requirements can help, too● Tour
– A path p tours path q if q is a subpath of p– A test covers any prime path it tours
This is strict!Can we relax it?
![Page 118: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/118.jpg)
118
Turning Them Into Tests
● Relaxing our path requirements can help, too● Tour
– A path p tours path q if q is a subpath of p– A test covers any prime path it tours
● Tour with sidetrips– Iff every edge of q appears in the same order in p– “Return to where you left”
![Page 119: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/119.jpg)
119
Turning Them Into Tests
● Relaxing our path requirements can help, too● Tour
– A path p tours path q if q is a subpath of p– A test covers any prime path it tours
● Tour with sidetrips– Iff every edge of q appears in the same order in p– “Return to where you left”
● Tour with detours– Iff every node of q appears in the same order in p
![Page 120: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/120.jpg)
120
Turning Them Into Tests
A B C D E
FStrict Tour: ABCDE
![Page 121: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/121.jpg)
121
Turning Them Into Tests
A B C D E
FStrict Tour: ABCDE
![Page 122: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/122.jpg)
122
Turning Them Into Tests
A B C D E
FStrict Tour: ABCDE
A B C D E
FWith Sidetrips?
![Page 123: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/123.jpg)
123
Turning Them Into Tests
A B C D E
FStrict Tour: ABCDE
A B C D E
FWith Sidetrips?
![Page 124: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/124.jpg)
124
Turning Them Into Tests
A B C D E
FStrict Tour: ABCDE
A B C D E
FWith Sidetrips?
A B C D E
FWith Detours?
![Page 125: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/125.jpg)
125
Turning Them Into Tests
A B C D E
FStrict Tour: ABCDE
A B C D E
FWith Sidetrips?
A B C D E
FWith Detours?
![Page 126: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/126.jpg)
126
Turning Them Into Tests
● Do these relaxations help us with problems we have seen?
![Page 127: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/127.jpg)
127
Turning Them Into Tests
● Do these relaxations help us with problems we have seen?
● Can you see any problems they may introduce?
![Page 128: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/128.jpg)
128
Turning Them Into Tests
● Do these relaxations help us with problems we have seen?
● Can you see any problems they may introduce?● How might this affect how you use them?
![Page 129: CMPT 473 Software Quality Assurance Graph Coveragewsumner/teaching/473/06-graphcoverage.pdf · Recall: Coverage/Adequacy ... – Nodes comprise the code of a program. 25 Control Flow](https://reader034.vdocument.in/reader034/viewer/2022042321/5f0ac8527e708231d42d5128/html5/thumbnails/129.jpg)
129
Onward
● But sometimes the structure of the CFG is not enough...