Download - SWTest 04 - White Box Testing
-
7/28/2019 SWTest 04 - White Box Testing
1/56
Software Testing Slide 1
Software Testing(09CS 491)
Unit IIWhite Box Testing(Chapter 3)
Testing can prove presence of defects and not their absence!
-
7/28/2019 SWTest 04 - White Box Testing
2/56
Software Testing Slide 2
Objectives
l To review the V Model and set the context for different types of
tests
l To focus on white box testing
l To distinguish between different types of white box testing
l To place special emphasis on
Code reviews / inspections
Methods of code coverage and relationship to quality
Code complexity
-
7/28/2019 SWTest 04 - White Box Testing
3/56
Software Testing Slide 3
Bending the Waterfall
High-LevelDesign
Low-LevelDesign
Development
Testing
Unit Testing
ComponentTesting
IntegrationTesting
SystemTesting
AcceptanceTesting
SoftwareRequirements
OverallBusinessRequirements
-
7/28/2019 SWTest 04 - White Box Testing
4/56
Software Testing Slide 4
OverallBusinessRequirements
SoftwareRequirements
High-LevelDesign
Low-Level
Design
Development Unit TestExecution
ComponentTestExecution
IntegrationTest Execution
SystemTest Execution
AcceptanceTest Execution
AcceptanceTest Design
SystemTest Design
IntegrationTest Design
Component
Test Design
Unit TestDesign
V Model
-
7/28/2019 SWTest 04 - White Box Testing
5/56
Software Testing Slide 5
Principles of the V Model
l Do an early test design.
l Involve the right people in the design of the right type of tests.
l Make the tests reflect closely the steps on the left-hand side ofthe V.
l Unit testing
White box testing
Black box testing
-
7/28/2019 SWTest 04 - White Box Testing
6/56
Software Testing Slide 6
White Box Testing
l Requires access to code
l Looks at the program code and performs testing by
mapping program code to functionality
-
7/28/2019 SWTest 04 - White Box Testing
7/56 Software Testing Slide 7
White-box testing is a method of testing software that tests internal
structures or workings of an application.
White box testing is also known as clear, open, structural, and glass
box testing.
White Box Testing method is applicable to the following levels of software
testing:
Unit Testing : For testing paths within a unit
Integration Testing : For testing paths between units
System Testing : For testing paths between subsystems
White Box Testing
-
7/28/2019 SWTest 04 - White Box Testing
8/56 Software Testing Slide 8
Types of White Box Testing
-
7/28/2019 SWTest 04 - White Box Testing
9/56 Software Testing Slide 9
Why White Box Testing?
l The program code truly represents what the program actually
does and not just what it is intended to do!
l It minimizes delay between defect injection and defect detection
(i.e., does not postpone detection to be done by someone else).
l Can catch obvious programming errors that do not necessarily
map to common user scenarios (e.g., divide by zero).
-
7/28/2019 SWTest 04 - White Box Testing
10/56 Software Testing Slide 10
Static Testing
l Involves only the source code and not the executable or binaries
l Does not involve executing the program on a machine but rather
humans going through it or the usage of specialized tools
l Some of the things tested by static testing:
Whether the code works according to the functional requirement
Whether the code has been written in accordance with the design
developed earlier in the project life cycle
Whether the code follows all applicable standards
Whether the code for any functionality has been missed out
Whether the code handles errors properly
-
7/28/2019 SWTest 04 - White Box Testing
11/56 Software Testing Slide 11
Static Testing by Humans
l Humans can find errors that computers cant.
l Multiple humans can provide multiple perspectives.
l A human evaluation can compare the code against thespecifications more thoroughly.
l It can detect multiple defects at one go.
l It reduces downstream, inline pressure.
-
7/28/2019 SWTest 04 - White Box Testing
12/56 Software Testing Slide 12
Static Testing Types
l Different types
Desk checking of the code
Code walkthrough
Code review
Code inspection
l QA vs QC argument
l Increasing the involvement of people
l More variety of perspectives
l Increasing formalism
l Increasing the likelihood of identifying more complex defects
-
7/28/2019 SWTest 04 - White Box Testing
13/56 Software Testing Slide 13
Desk Checking
l Author informally checks the code against the specifications and
corrects defects found.
l No structured method or formalism is required to ensure
completeness.
l No log or checklist is maintained.
l It relies only on the thoroughness of the author.
l It may suffice for obvious programming errors but may not be
effective for incomplete / misunderstood requirements.
-
7/28/2019 SWTest 04 - White Box Testing
14/56 Software Testing Slide 14
Advantages and Disadvantages of
Desk Checking
l Advantages
The programmer knows the code and programming language well
and hence is best suited to read the program
Less scheduling and logistics overheads
Reduces delay in defect detection and correction
l Disadvantages
Person dependent, not scalable or reproducible
Tunnel visioning of developers
Developers prefer writing new code and dont like testing
-
7/28/2019 SWTest 04 - White Box Testing
15/56 Software Testing Slide 15
Code Walkthrough
l Group oriented (as against desk checking)
l Brings in multiple perspectives
l Multiple specific roles (author, moderator, inspector, etc.),
as discussed in Fagan Inspection
-
7/28/2019 SWTest 04 - White Box Testing
16/56 Software Testing Slide 16
Formal Inspection / Fagan Inspection
l Group-oriented activity
l Highly formal and structured
l Has specific roles
l Requires thorough preparation
-
7/28/2019 SWTest 04 - White Box Testing
17/56
Software Testing Slide 17
Roles in a formal inspection
l Author Author of the work product
Makes available the required material to the reviewers
Fixes defects that are reported
l Moderator
Controls the meeting(s)
l Inspectors (reviewers)
Prepare by reading the required documents
Take part in the meeting(s) and report defects
l Scribe
Takes down notes during the meeting
Assigned in advance by turns
Can participate to review to the extent possible
Documents the minutes and circulates them to participants
-
7/28/2019 SWTest 04 - White Box Testing
18/56
Software Testing Slide 18
Process in a Fagan Inspection
Typical documents circulated:
Program code
Design / program specifications
SRS (if needed)
Any applicable standards (e.g., coding standards)
Any necessary checklists (e.g., code review checklist)
-
7/28/2019 SWTest 04 - White Box Testing
19/56
Software Testing Slide 19
Meetings in a Fagan Inspection
l Preliminary meeting (optional)
Author explains his / her perspective
Makes available the necessary documents
Highlights concern areas, if any, for which review comments are sought
l Defect Logging Meeting
All come prepared!
Moderator goes through the code sequentially
Each reviewer comes up with comments
Comments / defects categorized as defect / observation, major /
minor, systemic / mis-execution
Scribe documents all the findings and circulates them
-
7/28/2019 SWTest 04 - White Box Testing
20/56
Software Testing Slide 20
Meetings in a Fagan Inspection
l Follow-up meeting (optional)
Author fixes defects
If required, a follow-up meeting is called to verify completeness of fixes
-
7/28/2019 SWTest 04 - White Box Testing
21/56
Software Testing Slide 21
Advantages and Disadvantages of
Fagan Inspection
l Advantages
Thorough, when prepared well
Brings in multiple perspectives
Has been found to be very effective
l Disadvantages
Logistically difficult
Time consuming
May not be possible to exhaustively go through the entire code
White Box TestinSlide : 21
-
7/28/2019 SWTest 04 - White Box Testing
22/56
Software Testing Slide 22
Combinations of various methods
l Prioritizing various programs
High priority / important codes subject to formal
inspection, medium ones for walkthrough and low ones
for desk checking
l Deciding between 100% coverage and partial coverage
-
7/28/2019 SWTest 04 - White Box Testing
23/56
Software Testing Slide 23
Static Analysis Tools
Are used to identify errors such as:l Whether there are unreachable codes (usage of goto statements sometimes
creates this situation; there could be other reasons too)
l Variables declared but not used
l Mismatch in definition and assignment of values to variables
l Illegal or error-prone type-casting of variables
l Use of non-portable or architecture-dependent programming constructs
l Memory allocated but not having corresponding statements for freeing up
memory
l For calculation of cyclomatic complexity
l As an extension of compilers (lint, compiler flag driven checking)
-
7/28/2019 SWTest 04 - White Box Testing
24/56
Software Testing Slide 24
Use of a Code Review Checklist
l The code review checklist aids in organizational learning
by indicating what to look for.
l It should be kept current as we learn.
l A discussion of a sample checklist is given on Page 54.
-
7/28/2019 SWTest 04 - White Box Testing
25/56
Software Testing Slide 25
Structural Testing
l Done by running the executable on the machine
l Entails running the product against some predefined test cases and
comparing the results against the expected results
l Designing test cases and test data to exercise certain portions of
code
l Types of structural testing
Unit / code functional testing
Code coverage testing
Code complexity testing
-
7/28/2019 SWTest 04 - White Box Testing
26/56
Software Testing Slide 26
Unit / Code Functional Testing
l Initial quick checks by developer
l Removes obvious errors
l Done before more expensive checks
l Building debug versions
l Running through an IDE
l Debugging or testing? (Who cares?!)
-
7/28/2019 SWTest 04 - White Box Testing
27/56
Software Testing Slide 27
Code Coverage Testing
l Exercises different parts of code
l Maps parts of code to required functionality
l Finds out percentage of code covered by testing by adopting a
technique called instrumentation of code.
l Instrumentation rebuilds the product, linking it with libraries
provided by the tool vendors.
l Instrumented code can monitor what parts of code is covered
l Can help identify critical portions of code, executed often
-
7/28/2019 SWTest 04 - White Box Testing
28/56
Software Testing Slide 28
Types of Code Coverage
l Statement coverage
l Path coverage
l Condition coverage
l Function coverage
-
7/28/2019 SWTest 04 - White Box Testing
29/56
Software Testing Slide 29
Programming Constructs
l Sequential instructions
l Two-way decision statements (if then else)
l Multi-way decision statements (switch statements)
l Loops, like while/do, for, repeat/until, etc
-
7/28/2019 SWTest 04 - White Box Testing
30/56
Software Testing Slide 30
Testing sequential instructions
l Generate test data to make the program enter the sequential
block, to make it go through the entire block (Is this valid?)
Asynchronous exceptions
Multiple entry points, in non-structured programming
l Statement coverage as a metric:
# of statements exercised / Total # of statements
-
7/28/2019 SWTest 04 - White Box Testing
31/56
Software Testing Slide 31
Testing IF THEN ELSE
l Have data to test the THEN part
l Have data to test the ELSE part
Total = 0; /* set total to zero */
if (code == M) {
stmt1;
stmt2;
}
else
percent = value/Total*100; /* divide by zero */
-
7/28/2019 SWTest 04 - White Box Testing
32/56
Software Testing Slide 32
Path Coverage
l Statement Coverage may not indicate true coverage
l Path Coverage provides better representation
Previous example: Having data go through the THEN part and
ELSE part gives only 50% coverage, irrespective of number of
statements in each path
l More detailed example of path coverage in next slide
l Path coverage = # of paths exercised / total # of paths in the
program
-
7/28/2019 SWTest 04 - White Box Testing
33/56
Software Testing Slide 33
Path Coverage.. ExampleFig: flowchart for a date validation routine
-
7/28/2019 SWTest 04 - White Box Testing
34/56
Software Testing Slide 34
Condition Coverage
l Further refinement of path coverage
l Makes sure each constituent condition in a Boolean expression
is covered
l Protects against compiler optimizations
l Condition coverage = # of conditions covered / # of conditions in the program
-
7/28/2019 SWTest 04 - White Box Testing
35/56
Software Testing Slide 35
Function Coverage
l Finds how many functions are covered by test cases
l Higher level of abstraction; hence possible to achieve 100%
coverage
l More logical than the other types of coverage
l Can be derived from RTM
l Easy to prioritize as they are based on requirements
l Leads more naturally to black box tests
l Can also be a natural predecessor to performance testing
-
7/28/2019 SWTest 04 - White Box Testing
36/56
Software Testing Slide 36
Other Uses of Code Coverage Methods
l Performance analysis and optimization
l Resource usage analysis
l Checking of critical sections
l Identifying memory leaks
l Dynamically generated code (e.g., dynamic generation of
URLs and security checking)
-
7/28/2019 SWTest 04 - White Box Testing
37/56
Software Testing Slide 37
Code Complexity Testing
l Which of the paths are independent? (If paths are not
independent, we may be able to minimize the number of tests.)
l Is there an upper bound on the number of tests to be executed to
ensure that all the statements have been executed at least once?
l Cyclomatic complexity provides answers to some of these
questions.
-
7/28/2019 SWTest 04 - White Box Testing
38/56
Software Testing Slide 38
Intuitive Meaning of Cyclomatic Complexity
l Provides an indication of how many independent paths
are there in a program
l What gives birth to new paths?
l How can one count the new paths that are born?
Steps in Determining
-
7/28/2019 SWTest 04 - White Box Testing
39/56
Software Testing Slide 39
Steps in Determining
Cyclomatic Complexity
l Start with a conventional flow chart.
l Identify the decision points.
l Convert complex predicates to simple predicates (see
next slide).
-
7/28/2019 SWTest 04 - White Box Testing
40/56
Software Testing Slide 40
Steps in Determining
-
7/28/2019 SWTest 04 - White Box Testing
41/56
Software Testing Slide 41
Steps in Determining
Cyclomatic Complexity.. contd
l If there are loops, convert the loop conditions to simple predicates.
l Combine all the sequential statements into a single node in the flow
graph.
l When a set of sequential statements are followed by a simple predicate,
combine all the sequential statements and the predicate check into one
node and have two edges emanating out of this node. Nodes with two
emanating edges are called predicate nodes.
l Make sure that all edges terminate in some node, adding a node to
represent all the statements at the end of the program.
-
7/28/2019 SWTest 04 - White Box Testing
42/56
Software Testing Slide 42
Conventional Flow ChartFlow Diagram for
Calculating Complexity
Formal and Intuitive Interpretation of Cyclomatic
-
7/28/2019 SWTest 04 - White Box Testing
43/56
Software Testing Slide 43
Formal and Intuitive Interpretation of Cyclomatic
Complexity
l Formal definitions
CC = # of predicate nodes + 1
CC = E N + 2 (E = edges, N = nodes)
l Intuitive definitions
When a condition is introduced, adds a new path
See the next two slides
-
7/28/2019 SWTest 04 - White Box Testing
44/56
Software Testing Slide 44
End
Case (i): A hypothetical program with no decision node
#of independent paths = 1 #of nodes, N = 2; #of edges, E = 1; Cyclomatic complexity = E N + 2 = 1 #of predicate nodes, P = 0; Cyclomatic complexity = P + 1 = 1
-
7/28/2019 SWTest 04 - White Box Testing
45/56
Software Testing Slide 45
End
Case (ii): Adding one decision node
#of independent paths = 2 #of nodes, N = 4;
#of edges, E = 4; Cyclomatic complexity = E N + 2 = 2 #of predicate nodes, P = 1; Cyclomatic complexity = P + 1 = 2
-
7/28/2019 SWTest 04 - White Box Testing
46/56
Software Testing Slide 46
Independent Paths and Basis Sets
l An independent path is a path in the flow graph that has
not been traversed before in other paths.
l A set of independent paths that cover all the edges iscalled a basis set.
l Find the basis set and write the test cases to execute all
the paths in the basis set.
Meaning and Interpretation of
-
7/28/2019 SWTest 04 - White Box Testing
47/56
Software Testing Slide 47
Meaning and Interpretation of
Cyclomatic Complexity
Complexity What it means
1-10Well-written code, testability is high, cost / effort to
maintain is low
10-20Moderately complex, testability is medium, cost / effort
to maintain is medium
20-40 Very complex, testability is low, cost / effort to maintainis high
>40Not testable, any amount of money / effort to maintain
may not be enough
-
7/28/2019 SWTest 04 - White Box Testing
48/56
Software Testing Slide 48
Challenges in White Box Testing
l Developers not finding time for testing
l They develop blind spots for their own defects
l Fully covered code may not correspond to realistic
scenarios
-
7/28/2019 SWTest 04 - White Box Testing
49/56
Software Testing Slide 49
Objectives (Recap)
l To review the V Model and set the context for different types of
tests
l To focus on white box testing
l To distinguish between different types of white box testing
l To place special emphasis on
Code reviews / inspections
Methods of code coverage and relationship to quality
Code complexity
-
7/28/2019 SWTest 04 - White Box Testing
50/56
Software Testing Slide 50
Appendix
EXAMPLES
-
7/28/2019 SWTest 04 - White Box Testing
51/56
-
7/28/2019 SWTest 04 - White Box Testing
52/56
-
7/28/2019 SWTest 04 - White Box Testing
53/56
Software Testing Slide 53
Constructing the Logic Flow Diagram
Start
2
3
4 5
6
7
8 9
Exit
1
F
T F
T F
T
-
7/28/2019 SWTest 04 - White Box Testing
54/56
Software Testing Slide 54
Finding the Test Cases
Start
2
3
4 5
6
7
8 9
Exit
1
b
d e
gf
i j
h
c
k l
a (Covered by any data)
(Data set must
(Data set must contain at least one value)
be empty)
(Total score > 0.0)(Total score < 0.0)
(Positive score) (Negative score)
(Reached if either f or
e is reached)
Self reading
-
7/28/2019 SWTest 04 - White Box Testing
55/56
Software Testing Slide 55
Unit-testing Heuristics1. Create unit tests as soon as object
design is completed: Black-box test: Test the use
cases & functional model
White-box test: Test thedynamic model
Data-structure test: Test theobject model
2. Develop the test cases
Goal: Find the minimal numberof test cases to cover as manypaths as possible
3. Cross-check the test cases toeliminate duplicates
Don't waste your time!
4. Desk check your source code
Reduces testing time5. Create a test harness
Test drivers and test stubsare needed for integrationtesting
6. Describe the test oracle
Often the result of the firstsuccessfully executed test
7. Execute the test cases
Dont forget regressiontesting
Re-execute test cases everytime a change is made.
8. Compare the results of the testwith the test oracle
Automate as much as
possible
Big cost -> what should be done?
-
7/28/2019 SWTest 04 - White Box Testing
56/56
THANKS!