chapter6-part1
TRANSCRIPT
![Page 1: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/1.jpg)
Ch. 6 1
Verification
![Page 2: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/2.jpg)
Ch. 6 2
Outline• What are the goals of verification?• What are the main approaches to
verification?– What kind of assurance do we get
through testing?– How can testing be done systematically?– How can we remove defects
(debugging)?• What are the main approaches to
software analysis?– informal vs. formal
![Page 3: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/3.jpg)
Ch. 6 3
Need for verification
• Designers make mistakes even if they are skilled and follow sound principles
• Everything must be verified, every required quality, process and products– even verification itself…
![Page 4: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/4.jpg)
Ch. 6 4
Properties of verification
• May not be binary (OK, not OK)– severity of defect is important– some defects may be tolerated
• May be subjective or objective– e.g., usability
• Even implicit qualities should be verified– because requirements are often incomplete– e.g., robustness
![Page 5: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/5.jpg)
Ch. 6 5
Approaches to verification
• Experiment with behavior of product– sample behaviors via testing– goal is to find "counterexamples"– dynamic technique
• Analyze product to deduce its adequacy– analytic study of properties– static technique
![Page 6: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/6.jpg)
Ch. 6 6
Testing and lack of "continuity"
• Testing samples behaviors by examining "test cases"
• Impossible to extrapolate behavior of software from a finite set of test cases
• No continuity of behavior– it can exhibit correct behavior in
infinitely many cases, but may still be incorrect in some cases
![Page 7: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/7.jpg)
Ch. 6 7
Verification in engineering
• Example of bridge design• One test assures infinite correct
situations
![Page 8: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/8.jpg)
Ch. 6 8
procedure binary-search (key: in element; table: in elementTable; found: out Boolean)
beginbottom := table'first; top := table'last; while bottom < top loop
if (bottom + top) rem 2 ≠ 0 then middle := (bottom + top - 1) / 2;
end if;if key ≤ table (middle) then
top := middle;else
bottom := middle + 1;end if;
end loop;found := key = table (top);
end binary-search
![Page 9: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/9.jpg)
Ch. 6 9
procedure binary-search (key: in element; table: in elementTable; found: out Boolean)
beginbottom := table'first; top := table'last; while bottom < top loop
if (bottom + top) rem 2 ≠ 0 then middle := (bottom + top - 1) / 2;
else middle := (bottom + top) / 2;
end if;if key ≤ table (middle) then
top := middle;else
bottom := middle + 1;end if;
end loop;found := key = table (top);
end binary-search
if we omit thisthe routineworks if the elseis never hit!(i.e. if size of table is a power of 2)
![Page 10: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/10.jpg)
Ch. 6 10
Goals of testing
• To show the presence of bugs (Dijkstra, 1987)
• If tests do detect failures, we cannot conclude that software is defect-free
• Still, we need to do testing– driven by sound and systematic
principles
![Page 11: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/11.jpg)
Ch. 6 11
Goals of testing (cont.)
• Should help isolate errors– to facilitate debugging
• Should be repeatable– repeating the same experiment, we
should get the same results• this may not be true because of the effect
of execution environment on testing• because of nondeterminism
• Should be accurate
![Page 12: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/12.jpg)
Ch. 6 12
Theoretical foundations of testing
![Page 13: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/13.jpg)
Ch. 6 13
Definitions (1)• P (program), D (input domain), R
(output domain)– P: D R (may be partial)– Let OR be the requirement on output
values of P as stated in P’s specification
• Correctness defined by OR D R– P(d) correct if <d, P(d)> OR– P correct if all P(d) are correct
![Page 14: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/14.jpg)
Ch. 6 14
Definitions (2)• FAILURE
– A manifest symptom of the presence of an error.– P(d) is not correct
• may be undefined (error state) or may be the wrong result
• ERROR (DEFECT)– anything that may cause a failure
• typing mistake• programmer forgot to test “x = 0”
• FAULT– incorrect intermediate state entered by program
![Page 15: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/15.jpg)
Ch. 6 15
Definitions (2.1)• FAILURE
– Occurs only if a fault happens during execution
• ERROR (DEFECT)– P(d) does not satisfy the output
requirements
• FAULT– Occurs only if the programs contains an
error
![Page 16: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/16.jpg)
Ch. 6 16
Definitions (3)
• Test case t – an element of D
• Test set T– a finite subset of D
• Test is successful if P(t) is correct• Test set successful if P correct for all
t in T
![Page 17: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/17.jpg)
Ch. 6 17
Definitions (4)
• Ideal test set T– if P is incorrect, there is an element of
T such that P(d) is incorrect
• if an ideal test set exists for any program, we could prove program correctness by testing
![Page 18: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/18.jpg)
Ch. 6 18
Test criterion• A criterion C defines finite subsets of D
(test sets)– C 2D (the set of all finite subsets of D)
• A test set T satisfies C if it is an element of CExampleC = {<x1, x2,..., xn> | n 3 i, j, k, ( xi<0 xj=0
xk>0)}<-5, 0, 22> is a test set that satisfies C<-10, 2, 8, 33, 0, -19> also does<1, 3, 99> does not
![Page 19: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/19.jpg)
Ch. 6 19
Properties of criteria (1)• C is consistent
– for any pairs T1, T2 satisfying C, T1 is successful iff T2 is successful• so either of them provides the “same”
information
• C is complete– if P is incorrect, there is a test set T of
C that is not successful• C is complete and consistent
– identifies an ideal test set– allows correctness to be proved!
![Page 20: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/20.jpg)
Ch. 6 20
Properties of criteria (2)
• C1 is finer than C2– for any program P
• for any T1 satisfying C1 there is a subset T2 of T1 which satisfies C2
![Page 21: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/21.jpg)
Ch. 6 21
Properties of definitions
• None is effective, i.e., no algorithms exist to state if a program, test set, or criterion has that property
• In particular, there is no algorithm to derive a test set that would prove program correctness– there is no constructive criterion that
is consistent and complete
![Page 22: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/22.jpg)
Ch. 6 22
Empirical testing principles
• Attempted compromise between the impossible and the inadequate
• Find strategy to select significant test cases– significant=has high potential of
uncovering presence of error
![Page 23: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/23.jpg)
Ch. 6 23
Complete-Coverage Principle
• Try to group elements of D into subdomains D1, D2, …, Dn where any element of each Di is likely to have similar behavior– D = D1 D2 … Dn
• Select one test as a representative of the subdomain
• If Dj Dk for all j, k (partition), any element can be chosen from each subdomain
• Otherwise choose representatives to minimize number of tests, yet fulfilling the principle
![Page 24: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/24.jpg)
Ch. 6 24
Complete-Coverage Principle
example of a partition
![Page 25: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/25.jpg)
Ch. 6 25
Testing in the smallWe test individual modules• BLACK BOX (functional) testing
– partitioning criteria based on the module’s specification
– tests what the program is supposed to do
• WHITE BOX (structural) testing– partitioning criteria based on module’s
internal code– tests what the program does
![Page 26: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/26.jpg)
Ch. 6 26
White box testing
derives test cases from program code
![Page 27: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/27.jpg)
Ch. 6 27
Structural Coverage Testing
• (In)adequacy criteria – If significant parts of program
structure are not tested, testing is inadequate
• Control flow coverage criteria– Statement coverage– Edge coverage– Condition coverage– Path coverage
![Page 28: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/28.jpg)
Ch. 6 28
Statement-coverage criterion
• Select a test set T such that every elementary statement in P is executed at least once by some d in T– an input datum executes many
statements try to minimize the number of test cases still preserving the desired coverage
![Page 29: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/29.jpg)
Ch. 6 29
Exampleread (x); read (y);if x > 0 then
write ("1");else
write ("2");end if;if y > 0 then
write ("3");else
write ("4");end if;
![Page 30: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/30.jpg)
Ch. 6 30
Exampleread (x); read (y);if x > 0 then
write ("1");else
write ("2");end if;if y > 0 then
write ("3");else
write ("4");end if;
{<x = 2, y = 3>, <x = - 13, y = 51>, <x = 97, y = 17>, <x = - 1, y = - 1>}covers all statements
{<x = - 13, y = 51>, <x = 2, y = - 3>} is minimal
![Page 31: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/31.jpg)
Ch. 6 31
Weakness of the criterion
if x < 0 then x := -x;
end if;z := x;
{<x=-3} covers allstatements
![Page 32: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/32.jpg)
Ch. 6 32
Weakness of the criterion
if x < 0 then x := -x;
end if;z := x;
{<x=-3} covers allstatements
it does not exercise the case when x is positiveand the then branch isnot entered
![Page 33: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/33.jpg)
Ch. 6 33
Edge-coverage criterion
• Select a test set T such that every edge (branch) of the control flow is exercised at least once by some d in Tthis requires formalizing the concept of the control graph, and how to construct it– edges represent statements– nodes at the ends of an edge represent entry into
the statement and exit
![Page 34: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/34.jpg)
Ch. 6 34
G G1 2 G1
G1
G1
G 2
I/O, assignment, or procedure call
if-then-else if-then
while loop
two sequential statements
Control graph construction rules
![Page 35: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/35.jpg)
Ch. 6 35
Simplification
a sequence of edges can be collapsed into just one edge
. . .n n nnn k-1 k1 2 3
n1n
k
![Page 36: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/36.jpg)
Ch. 6 36
beginread (x); read (y);while x ≠ y loop
if x > y then x := x - y;
else y := y - x;
end if;end loop;gcd : = x;
end;
Exemple: Euclid's algorithm
![Page 37: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/37.jpg)
Ch. 6 37
Weaknessfound := false; counter := 1;while ((not found) and counter < number_of_items) loop
if (table[counter] = desired_element) then found := true;
end if;counter := counter + 1;
end loop;if found then
write ("the desired element is in the table");else
write ("the desired element is not in the table");end if;
![Page 38: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/38.jpg)
Ch. 6 38
Condition-coverage criterion
• Select a test set T such that every edge of P’s control flow is traversed and all possible values of the constituents of compound conditions are exercised at least once– it is finer than edge coverage
![Page 39: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/39.jpg)
Ch. 6 39
Weakness if x ≠ 0 then
y := 5; else
z := z - x; end if;if z > 1 then
z := z / x; else
z := 0; end if;
![Page 40: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/40.jpg)
Ch. 6 40
Weakness
{<x = 0, z = 1>, <x = 1, z = 3>}
if x ≠ 0 then
y := 5; else
z := z - x; end if;if z > 1 then
z := z / x; else
z := 0; end if;
![Page 41: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/41.jpg)
Ch. 6 41
Weakness
{<x = 0, z = 1>, <x = 1, z = 3>} causes the execution of all edges, but fails to expose the risk of a division by zero
if x ≠ 0 then
y := 5; else
z := z - x; end if;if z > 1 then
z := z / x; else
z := 0; end if;
![Page 42: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/42.jpg)
Ch. 6 42
Path-coverage criterion
• Select a test set T which traverses all paths from the initial to the final node of P’s control flow– it is finer than previous kinds of
coverage– however, number of paths may be too
large, or even infinite (see while loops)• additional constraints must be provided
![Page 43: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/43.jpg)
Path-coverage Criterion
For loop <= 20 times what would be the number of uniquePaths through the program?
Control-flow graph
![Page 44: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/44.jpg)
Ch. 6 44
Problemmain (){ int i, j, k, match;
cin >> i >> j >> k; cout << i << j << k;
if (i <= 0 || j <= 0 || k <= 0 || i+j <= k || j+k <= i || k+i <= j) match = 4; else if !(i == j || j == k || k == i) match = 3;else if (i != j || j != k || k != i) match = 2;else
match = 1;cout << match << endl;
}
![Page 45: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/45.jpg)
Ch. 6 45
Problem1) If, for an assignment of values to the input variables i, j, and
k, the output variable match will assume a correct value upon execution of the program, we can assert that the program is correct for this particular test case.
2) And if we can test the program for all possible assignments to i, j , and k, we will be able to determine its correctness.
3) Assume an ordinary integer variable inC++ to be a value in the range −32,768 to +32,767 (i.e., 216).
![Page 46: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/46.jpg)
Ch. 6 46
The infeasibility problem• Read( x ); z := 2 * x; if( z = 4 ) then statements
• If( x > 0 ) then if( x < 0 ) then statements
![Page 47: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/47.jpg)
Ch. 6 47
The infeasibility problem• Syntactically indicated behaviors
(statements, edges, etc.) are often impossible– unreachable code, infeasible edges, paths,
etc.
• Adequacy criteria may be impossible to satisfy – manual justification for omitting each
impossible test case– adequacy “scores” based on coverage
• example: 95% statement coverage
![Page 48: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/48.jpg)
Ch. 6 48
Further problem
• What if the code omits the implementation of some part of the specification?
• White box test cases derived from the code will ignore that part of the specification!
![Page 49: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/49.jpg)
Ch. 6 49
Further problem1) Write a program that takes three positive integers
as input and determine if they represent three sides of a triangle, and if they do, indicate what type of triangle it is.
![Page 50: chapter6-part1](https://reader036.vdocument.in/reader036/viewer/2022081511/55cf8ef8550346703b97943a/html5/thumbnails/50.jpg)
Ch. 6 50
Further problem
To be more specific, it should read three integers and setflag as follows:1) If they represent a scalene triangle, set the flag to 1.2) If they represent an isosceles triangle, set the flag to 2. 3) If they represent an equilateral triangle, set the flag to 3.4) If they do not represent a triangle, set the flag to 4.