white box testing arun lakhotia university of southwestern louisiana p.o. box 44330 lafayette, la...
TRANSCRIPT
White Box Testing
Arun Lakhotia
University of Southwestern Louisiana
P.O. Box 44330
Lafayette, LA 70504, USA
References:1) Jalote, section 7.3
2) Roger Pressman, Software
Engineering: A Practitioner’s
Approach.
Testing Scenario 1
If we know the function that the product has been designed to perform, tests can show that each function is fully operational.
This approach called Black-Box Testing.
Testing Scenario 2
If we know the internal workings of a product, tests can show that internal operations perform according to specification, and all components have been exercised.
This approach is called White-Box (Structural) Testing.
Question Why do we need both?
White box testing wants to force some desired coverage of different structures -- coverage based testing.
Coverage Types
1) Statement Coverage
2) Branch Coverage
3) Path Testing
Statement Coverage The goal is to execute
every statement at least once.
Is this a good or bad idea?int abs( int x )
{
if ( x >= 0 )
x = 0 - x;
return x;
}
If we execute this with {x=0} the criteria will be satisfied - every statement will be executed. But the error will not be detected. Why?
Branch Coverage
Each branch is executed once. Each decision is evaluated to T and F at least once.
This will usually imply statement coverage.
Does this criteria “find” the fault in the abs() function?
Branch Coverage (cont’d) Branch coverage has
problems with multiple (compound) conditions within a decision (&&, ||)
Consider the following:/* x is valid if 0<=x<=100 */
int check( int x )
{
if( x>=0 && x<=200 )
return TRUE;
else return FALSE;
}
If we test this module with
x {-5,5}, then the error will not be revealed.
Path Testing
A path is a sequence of branches executed when a program is executed from start to finish.
Also called a logical path.
The problem with path testing is that even small programs can have very large numbers of paths, especially if they contain loops.
Path Testing (cont’d)
Consider the following flowchart:
Path Testing (cont’d) There are 100 trillion
possible paths to traverse! This would take over 3,000 years.
Obviously, we need to limit the number of paths to execute...
Basis Path Testing
Involves the construction of a flow graph and using the graph to determine a basis set of execution paths for the program.
McCabe’s Cyclomatic Complexity
C.C. = # paths in the basis set.
Constructing a Flow Graph
“primitives”
sequence
if-else
F T
while-do
Flow Graphs (cont’d)
more “primitives”
do-until
case (switch)
Flow Graphs (advanced)
if( a || b )
x( );
else
y( );
x
a
y x
b
if( a && b )
x( );
else
y( );
x
a
yx
b
TT
F
F
Example: Procedure Sort1) while records remain do read record2) if record.field1 = 03) process record; store record; increment counter;4) else if record.field2 = 0;5) reset counter;6) else process record;7) endif endif8) enddo9) end
Procedure Sort Flow Graph
2
1
3
1
2
5
34
8
9
7
6
Cyclomatic Complexity
Once we’ve found the flow graph (G), we can compute the cyclomatic complexity, V(G).
It is simply the number of enclosed regions in the graph plus 1:
• V(G) = # Regions + 1
• V(G) = 3 + 1 = 4 This means that we need
4 independent paths through the graph.
Independent Paths By independent paths, we
mean that no path is a combination of the others in the set.
When constructing them, each “new” path should include some node not previously used.
A good guidelne is to start with a simple path, and change each condition.
Example Path 1: 1-9 Path 2: 1-2-3-8-1-9 Path 3: 1-2-4-6-7-8-1-9 Path 4: 1-2-4-5-7-8-1-9 Note: 1-2-3-8-1-2-4-5-7-8-1-9 is
not independent because it simply combines paths 2 and 4.
If we choose data to force execution of these 4 basis paths, then we will be executed and every condition will be evaluated on TRUE and FALSE sides.
Loop Testing
Path testing may only force the execution of loops with one TRUE and one FALSE value.
Since loops are the cornerstone of many algorithms, we should derive specific tests to test loop constructs.
Loop Classes
Simple
Nested
Concatenated
Unstructured (FORTRAN, COBOL)
Testing Simple Loops Assume N is the
maximum number of allowable passes through the loop:
1. Skip the loop entirely
2. One pass through the loop
3. Two passes through the loop
4. m passes through the loop
where m < N
5. N - 1, N + 1, N passes.
Testing Nested Loops
1. Start at innermost loop.
2. Perform simple loop test for
innermost loop.
3. Work outward, conducting tests
for the next loop, keeping outer
loops at minimum values and
other nested loops to “typical”
values.
4. Continue until all loops have been
tested.
Testing Concatenated Loops Independend vs.
dependent
See text pg. 399
Testing Unstructured Loops
rewrite them
Test Case Generation
Manual generation is tedious.
Some automated methods: Random generation until
desired coverage is reached -- inefficient.
Tools to tell the tester what paths have been covered. Then tester can derive additional test cases to increase coverage.
Instrumentation Programs are
instrumented with probes.
Probes keep track of which branches have been executed.
A preprocessor inserts the probes at strategic locations.
The program is executed on the test data.
A postprocessor analyzes the results.
btool, bmake, bsummary, breport