software testing sudipto ghosh cs 406 fall 99 november 16, 1999
Post on 20-Dec-2015
222 views
TRANSCRIPT
11/16/99 CS 406 Testing 3
Problems with path testing
• With each predicate, there is a combinatorial explosion in the number of paths.
• Potentially infinite number of paths because of loops.
• Not all paths a feasible, i.e., there may not be inputs for which the path is executed.
• Suppose that somehow, we satisfied this criterion. Would we find all the faults?
11/16/99 CS 406 Testing 4
More problems
• A path is tested only if it is present!!• Consider the example:
double logBase19 (double x) {
return ln(x)/ln(19);
}
• Missing condition is:if(x > 0)
11/16/99 CS 406 Testing 5
More problems
• It is possible to test every path without detecting the fault in the product.
• Function to test the equality of 3 integers:• (Fallacious) assumption:
• If the average of the 3 numbers is equal to the first, then they are equal.
boolean areEqual(int x,int y,int z){if ((x+y+z)/3 == x) return TRUE;else return FALSE;
}
• Exercise: Think of test cases.
11/16/99 CS 406 Testing 6
Reducing the number of paths to be covered• Find something between branch and path
coverage.• Select a set of paths that ensure branch
coverage and try some other paths that help reveal errors.• Consider two paths same if they differ only in
their sub-paths that are caused due to loops.• Restrict test cases to linear code sequences
• Identify set of points L, from which control flow may jump; includes entry, exit, branch statements
• Paths begin and end at elements in L.
11/16/99 CS 406 Testing 7
Data Flow-Based Testing
• Select paths to be executed based on data flow analysis
• Information about where • variables are defined• variables are used
• Idea is to make sure that the definitions of variables and their subsequent use is tested
11/16/99 CS 406 Testing 8
Variables in statements
• Variables occur in statements as• Definition – def
• Variables on the left side of a statement ( c )• c := 10;
• Variable is used in a computation – c-use• Variables on the right-hand side of a statement ( c )• temp := c + 5;• Variables used in a write statement
• Variable is used in a predicate – p-use• if( c!= MAXLENGTH ) {
11/16/99 CS 406 Testing 9
An example (xy)
1. scanf(x, y); if(y < 0)
2. pow = 0 – y;
3. else pow = y;
4. z = 1.0;
5. while(pow != 0)
6. { z = z * x; pow = pow – 1; }
7. if ( y < 0 )
8. z = 1.0/z;
9. printf(z);
11/16/99 CS 406 Testing 10
Def/use graph of example
1
2 3
6
4
5
7
8 9
def = {x, y}: c-use = {}
def = {pow}:c-use = {y}
def = {pow}:c-use = {y}
def = {z}: c-use = {}
def = {}: c-use = {}
def = {}: c-use = {}def = {z, pow}:c-use = {x, z, pow}
def = {z}:c-use = {z}
def = {}:c-use = {z}
y
y
y
pow
y
pow
11/16/99 CS 406 Testing 11
More on def/use graphs
• With each node, we associate the c-uses of a variable
• With each edge, we associate the p-uses of a variable
11/16/99 CS 406 Testing 12
Def-clear paths
• Def-clear path with respect to a variable x:• Path from node i to node j, such that there is
no def of x in the nodes in the path from i to j. Nodes i and j may have a def.
• Find out examples from the previous graph.• Path from node i to edge (j, k), such that there
is no def of x in the nodes in the path.• Find out examples from the previous graph.
11/16/99 CS 406 Testing 13
Global c-use
• A c-use of a variable is considered global if:• there is not def of x within the block preceding
the c-use• Example?
11/16/99 CS 406 Testing 14
Global def
• A def is global if it can be used outside the block in which it is defined.
• A def of a variable x, in a node i, is aglobal def, if:• it is the last def of x in the block (node) i• a def-clear path exists from i to another node
which has a global c-use of x• Example?
11/16/99 CS 406 Testing 15
Set def(i)
• Set of variables for which there is a global def in the node i.
• Examples:
11/16/99 CS 406 Testing 16
Set c-use(i)
• Set of variables for which there is a global def in the node i.
• Examples:
11/16/99 CS 406 Testing 17
Set p-use(i, j)
• For an edge (i, j), the set p-use(i, j), is the set of variables for which there is a p-use for the edge (i, j).
• Examples:
11/16/99 CS 406 Testing 18
Set dcu(x, i)
• dcu(x, i) represents all those nodes in which the global c-use of x uses the value assigned by the def of x in i.
• Suppose a variable x is in def(i) of node i.• Set of nodes, such that :
• Each node has x in its c-use• There is a def-clear path from i to j
• Examples:
11/16/99 CS 406 Testing 19
Set dpu(x, i)
• dpu(x, i) represents all those edges in which the p-use of x uses the value assigned by the def of x in i.
• Suppose a variable x is in def(i) of node i.• Set of edges, such that :
• each edge has x in its p-use• There is a def-clear path from i to (j, k)
11/16/99 CS 406 Testing 20
Test case selection criteria
• Let G be the def-use graph for a program• Let P be a set of all the complete paths of
G that were executed during the test.• Examples:
11/16/99 CS 406 Testing 21
All-defs criterion
• Make sure that the definitions of all variables are tested.
• For every def of every variable, one of its uses (either p-use or c-use) must be included in a path.
• For every node i in G and every x in def(i), P includes a def-clear path w.r.t. x to some member of dcu(x, i)
11/16/99 CS 406 Testing 22
All-p-uses criterion
• All p-uses of all definitions should be tested.
• For every x def (i), P includes a def-clear path w.r.t. x from i to all members of dpu(x, i).
• Note that a c-use may not be tested• Can require
• all p-uses, some c-uses criterion
11/16/99 CS 406 Testing 23
All c-uses criterion
• All c-uses of all definitions should be tested.
• For every x def (i), P includes a def-clear path w.r.t. x from i to all members of dcu(x, i).
• Note that a p-use may not be tested• Can require
• all c-uses, some p-uses criterion
11/16/99 CS 406 Testing 24
All-uses criterion
• All p-uses and all c-uses of a definition must be exercised.
• The set P must include,• for every node i and every x def(i)• a def-clear path w.r.t. x from i to all elements
of dcu(x, i) and to all elements of dpu(x, i).
11/16/99 CS 406 Testing 25
Number of test cases
• Can be shown theoretically that the limit of the size of test set is up to quadratic in the number of two-way decision statements in the program.
• Empirical studies have shown that the actual number of test cases that satisfy a criterion is quite small and increases linearly with the number of two-way decisions.
11/16/99 CS 406 Testing 26
Inclusion relationship between criteria
all-paths (path coverage)
all-uses
all-defs
all-c-uses/some-p-uses
all-p-uses/some-c-uses
all-p-uses
all-edges (branch coverage)
all-nodes (statement coverage)
11/16/99 CS 406 Testing 27
Implication of inclusion
• Suppose C1 includes C2 (C1 C2), I.e test cases satisfying C1 also satisfy C2
• Does it mean that C1 is always better than C2?
• Statistically, the set of test cases satisfying C1 will be better than those satisfying C2.
• Testing done using these criteria performs better than randomly selected test cases.
11/16/99 CS 406 Testing 28
Test assessment
• Develop• a test set T• a collection of test inputs
• Question:• How good is T?• Test assessment is the measurement of the
goodness of T.• Test assessment is carried out based on one
or more criteria.
11/16/99 CS 406 Testing 29
Test adequacy assessment
• These criteria are known as test adequacy criteria.
• Test assessment is also known as test adequacy assessment.
11/16/99 CS 406 Testing 30
Test adequacy assessment (contd)• Test adequacy assessment provides the
following information:• A metric, also known as adequacy score, or
coverage, usually between 0 and 1.• A list of all the weaknesses found in T, which
when removed, will raise the score to 1.• The weaknesses depend on the criteria used for
assessment.
11/16/99 CS 406 Testing 31
Test adequacy assessment (contd)• Once the coverage has been computed,
and the weaknesses identified, one can improve T.
• Improvement of T is done by examining one or more weaknesses and constructing new test requirements designed to overcome the weaknesses.
11/16/99 CS 406 Testing 32
Test adequacy assessment (contd)• This is continued until all weaknesses are
overcome, i.e., the adequacy criterion is satisfied (coverage = 1)
• In some instances, it may not be possible to satisfy the adequacy criteria for one or more of the following reasons:• Lack of sufficient resources (people, time)• Weaknesses that cannot be removed because
they are infeasible• The cost of removing the weakness is not
justified
11/16/99 CS 406 Testing 33
Test adequacy assessment (contd)• While improving T by removing its
weaknesses, one usually tests the program more thoroughly than it has been tested so far.
• This additional testing is likely to result in the discovery of remaining errors.
• Hence, we say that test assessment and improvement helps in the improvement of software reliability.