fault based testing
DESCRIPTION
TRANSCRIPT
FAULT BASED TESTING
INTRODUCTION
o It is necessary to test an object-oriented system at a variety of different levels
o The goal is to uncover errors that may occur as classes collaborate with one another and subsystems communicate across architectural layers
• Testing begins "in the small" on methods within a class and on collaboration between classes
• As class integration occurs, use-based testing and fault-based testing are applied
• Finally, use cases are used to uncover errors during the software validation phase
o Conventional test case design is driven by an input-process-output view of software
oObject-oriented testing focuses on designing appropriate sequences of methods to exercise the states of a class
TEST OBJECTIVES
oUntil now, we have not been very precise in our use of the notion of an 'error'.
o In order to appreciate the following discussion, it is important to make a careful distinction between the notions error, fault, and failure.
An error is a human action that produces an incorrect result. The consequence of an error is software containing a fault.
Error
A fault thus is the manifestation of an error. If encountered, a fault may result in a failure
Fault
OBJECTIVE
Design tests that have a high likelihood
of uncovering plausible faults
Begins with the analysis model
- The tester looks for plausible faults (i.e., aspects
of the implementation of the system that may result
in defects)- To determine whether
these faults exist, test cases are designed to exercise the
design or code
If the analysis and design models can provide insight
into what is likely to go wrong, then fault-based
testing can find a significant number of errors
FAULT BASED TESTING
• In fault-based testing strategies, we do not directly consider the artifact being tested when assessing the test adequacy. We only take into account the test set. Fault-based techniques are aimed at finding a test set with a high ability to detect faults.
• We will discuss two fault-based testing techniques: error seeding and mutation testing
ERROR SEEDING AND MUTATION TESTING
Error Seeding Technique:
No mutants are present here.
Here source code is tested within itself.
Errors are introduced directly
Test cases which detect errors are used for
testing.
It is less efficient error testing technique.
It requires less time.
Mutation Technique:
Mutants are developed for testing.
Here mutants are combined, compared for testing to find error introduced.
Special techniques are used to introduce errors.
Here, test cases which killmutants are used for testing.
It is more efficient than error seeding.
It is more time consuming
Error seeding and mutation testing are both error-oriented techniques and are generally, applicable to all levels of testing.
MUTATION TECHNIQUE
In Mutation Testing, we mutate (change) certain statements in the source code and check if the test cases are able to find the errors. It is a type of white box testing which is mainly used for unit testing. The changes in mutant program are kept extremely small, so it does not affect the overall objective of the program.
GOAL OF MUTATION TESTING
The goal of Mutation Testing is to assess the quality of the test cases which should be robust enough to fail mutant code. This method is also called as Fault based testing strategy as it involves creating fault in the program
HOW TO EXECUTE MUTATION TESTING?
Following are the steps to execute mutation testing:
Step 1: Faults are introduced into the source code of the program by creating many versions called mutants. Each mutant should contain a single fault, and the goal is to cause the mutant version to fail which demonstrates the effectiveness of the test cases.
Step 2: Test cases are applied to the original program and also to the mutant program. A test case should be adequate, and it is tweaked to detect faults in a program.
Step 3: Compare the results of original and mutant program.
Step 4: If the original program and mutant programs generate the same output, then that the mutant is killed by the test case. Hence the test case is good enough to detect the change between the original and the mutant program.
Step 5: If the original program and mutant program generate different output, Mutant is kept alive. In such cases , more effective test cases need to be created that kill all mutants.
HOW TO CREATE MUTANT PROGRAMS?
Original Program
If (x>y)
Print "Hello"
Else
Print "Hi"
Mutant Program
If(x)
Print "Hello"
Else
Print "Hi"
A mutation is nothing but a single syntactic change that is made to the program statement. Each mutant program should differ from the original program by one mutation.
WHAT TO CHANGE IN A MUTANT PROGRAM?
MUTATION SCORE
• The mutation score is defined as the percentage of killed mutants with the total number of mutants.
• Mutation Score = (Killed Mutants / Total number of Mutants) * 100
Test cases are mutation adequate if the score is 100%. Experimental results have shown that mutation testing is an effective approach for the measuring the adequacy of the test cases. But, the main drawback is that the high cost of generating the mutants and executing each test case against that mutant program.
FAULT INJECTION
FAULT INJECTION IS A SOFTWARE TESTING TECHNIQUE BY INTRODUCING FAULTS INTO THE CODE FOR IMPROVING THE COVERAGE AND USUALLY USED WITH STRESS TESTING FOR ROBUSTNESS OF THE DEVELOPED SOFTWARE.
WHAT IS FAULT?
A fault is the cause of an error. An incorrect step, process, or data definition in a computer program which causes the program to perform in an unintended or unanticipated manner. A software fault lies in software, a hardware fault lies in hardware.
SecurityRecoverability
Performance
Look & Feel
Functional & Logical
Business Logic
FAULT INJECTION METHODS:
• It is a fault injection technique where source code is modified to inject simulated faults into a system.
Compile Time Injections
• It makes use of software trigger to inject a fault into a software system during run time.
• The Trigger can be of two types, Time Based triggers and Interrupt Based Triggers.
Run-Time Injections