softtest : a framework for software testing of java programs
DESCRIPTION
SoftTest : A Framework for Software Testing of Java Programs. Presented by : Priyanka Puri 1000676053. B. Childers, M. L. Soffa , J. Beaver, L. Ber , K. Cammarata , J. Litman , J. Misurda. Structural Software Testing. Assure quality, robust software - PowerPoint PPT PresentationTRANSCRIPT
B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda
Presented by:
Priyanka Puri
1000676053
SOFTTEST: A FRAMEWORK FOR SOFTWARE TESTING OF JAVA PROGRAMS
STRUCTURAL SOFTWARE TESTING
• Assure quality, robust software
• Collect coverage information about the program
• E.g., branch coverage records which edges are hit
• Uses program instrumentation
..Continued..
• Automatically apply multiple test strategies
• Multiple languages and platforms
• Handle large programs
Static instrumentation approaches
• Not scalable: Instrumentation left in place• Time, memory
• Inflexible: Only certain tests, languages, platforms
• Generally not detailed
Approach
• Specification driven: User written test
• Test plans: Recipe of how & where to test
• Path specific: Instrument only what is needed
• Dynamic: Insert & remove instrumentation
TEST FRAMEWORK INSTANCE FOR JAVAWizard for creating the test specification
From test specification determine how to testTest plan uses instrumentation primitives
Source level coverage results
• Apply different tests to various code regions.• may be applied with different coverage criteria• A GUI for• What tests to apply, where to apply, under what
conditions• Select code regions• Conditions can be specified
TEST SPECIFICATION
public class Example
{
public int foo(int x)
{
if(x==200)
x = x+100;
else x = x-100;
return x;
}
}
DEFINITIONS
{
NAME: X, REGION_D,
LOCATION: FILE Example.java {
CLASS example, METHOD foo
}
}
BODY
{
DO BRANCH_TEST
ON REGION X UNTIL:90%
}
Example test specification
TEST PLANNER• Test plan: Where and how to test program
• Test table
• Where to instrument in the target machine code
• Test information (results)
• Branch coverage: when to insert & remove instrumentation, records which edges are hit/not hit
• Test payload
– What to do at an instrumentation point
– Target machine code
– Branch coverage: Update edge coverage
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 N,N
2 4 N
3 4 N
4 1 N
Instrumentation
Hit
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 N,N
2 4 N
3 4 N
4 1 N
Instrumentation
Hit
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 N,N
2 4 N
3 4 N
4 1 N
Instrumentation
Hit
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 N,N
2 4 N
3 4 N
4 1 N
Instrumentation
Hit
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,N
2 4 N
3 4 N
4 1 N
Instrumentation
Hit
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,N
2 4 N
3 4 N
4 1 N
Instrumentation
Hit
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,N
2 4 Y
3 4 N
4 1 N
Instrumentation
Hit
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,N
2 4 Y
3 4 N
4 1 N
Instrumentation
Hit
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,N
2 4 Y
3 4 N
4 1 Y
Instrumentation
Hit
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,Y
2 4 Y
3 4 N
4 1 Y
Instrumentation
Hit
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,Y
2 4 Y
3 4 N
4 1 Y
Instrumentation
Hit
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,Y
2 4 Y
3 4 Y
4 1 Y
Instrumentation
Hit
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,Y
2 4 Y
3 4 Y
4 1 Y
Instrumentation
Hit
Example CFG with block as its own successor
TEST VIRTUAL MACHINE• Provides functionality to insert and remove
instrumentation at run-time.
• Implements an interface for inserting and removing instrumentation with fast breakpoints*.
• API provides primitives, such as• the placement of successor breakpoints, • storing test- specific data• removal of breakpoints, for constructing fast
breakpoints with varying payloads
TEST ANALYZER
• Displays the results of tests
• For branch coverage, the analyzer displays
the CFG for a method
• CFG for target machine code
Observations• SoftTest significantly reduces the number of tests resulting in savings in
scripting effort.
• Significant impact on the acceptability of code coverage.
• Provides cost effective approach to testing.
• Low performance and memory overhead.
• For the benchmarks, the memory overhead was 178 to 822 bytes for the test table.
• The performance overhead for coverage varied from 1% to 5.5% of the total execution time.
Preliminary Results: Branch Coverage Testing
Overhead includes:
1. Test Planner
2. Instrumentation
Run-time performance overhead
[3]
REFERENCES
[1] http://veryoldwww.cs.pitt.edu/copa/eclipse.html
[2] SoftTest: A Framework for Software Testing of Java Programs
[3] http://www.cs.pitt.edu/coco/papers/SoftTest-Etx-2003
[4] http://www.softwaretestingmentor.com/types-of-testing/static-testing.php
QUESTIONS?