mutation tesingmutation
DESCRIPTION
testingTRANSCRIPT
EFFECTIVENESS OF MUTATION TESTING TOOL IN OBJECT ORIENTED
SYSTEMSA DISSERTATION
Submitted in partial fulfillment of the requirement for the award of Degree of Master of Technology in Information Technology Discipline
Submitted To
RAJIV GANDHI PRODYOGIKI VISHWAVIDYALAYA, BHOPAL (M.P.)
Submitted By:
Shikha SinghEnrollment No –
Under The Supervision Of:
Guide Name Co-Guide Name
DEPARTMENT OF INFORMATION TECHNOLOGY TECHNOCRATS INSTITUTE OF TECHNOLOGY, BHOPAL
ABSTRACT
Software testing allows programmers to determine the quality of the software.
Mutation testing is a branch of software testing which does more than this. It helps
determine whether the test cases that have been created, effectively detect all the
possible faults in the software. This allows the development of better test sets,
thus, ensuring that maximum software quality is achieved. This may seem very
inviting to software testers, but there is a major issue being faced which has
hindered the widespread use of mutation testing.
Mutation testing works by seeding faults in the software program. Various
mutation operators are used to create these faulty programs. These programs are
called mutants. The mutants depict software faults that may be caused by
programmers while writing the software. Test cases are then executed on these
mutants to determine if they have been killed or not. Test sets that kill all the
mutants are considered to be good as they successfully detect all the possible
program faults.
Sometimes, it is difficult to kill all the mutants. The reason is that some of the
mutants, although syntactically different than the original program, are still
semantically the same. Any test case will not be able to differentiate between the
original program and the mutant. Such mutants are called equivalent mutants.
This thesis targets this issue by determining which mutation operators create
mutants, which are more probable to create equivalent mutants. A tool called
MuJava, Jody is used for this purpose. An empirical analysis has been carried out
for this purpose, which helps determine which mutation operators develop more
equivalent mutants.
INTRODUCTION
Computer software can have two kinds of problems: faults or failures. A failure
does not permit a program to perform its intended function. A fault is an error in
the correctness of a program’s semantics. Testing is a process of determining
faults. Software testing is performed to ensure that the algorithms used in a
program are correct, the program can execute correctly in all possible scenarios
and it conforms to all the requirements specified. Correctness, completeness and
quality are some of the characteristics that all software programs must meet.
Software testing can be performed in a number of ways. There is no fixed process;
it is a simple method of trial and error and investigating various problems
encountered whilst execution.
Software testing is performed using test cases. These make use of some variables
and determine the actual output of the program against the expected output. Good
test cases help determine faults that occur rarely.
1.1 Mutation Testing
Mutation testing is one of the many types of methodologies used for testing a
program. While other software testing techniques focus on the correct
functionality of the program, mutation testing focuses on the test cases used to test
the programs. The main idea is to create good sets of test cases rather than trying
to find all the faults in a particular program. Good test cases are those which are
able to discover all the easy and hard-to-find faults in the program. An ideal test
case will detect all the possible faults in a software program.
Mutation testing is a white-box testing technique i.e. it examines the internal
structure of a program to detect faults. It is used to determine the effectiveness of
test cases i.e. how good they are in detecting faults. Mutation testing has been
developed using two basic ideas:
1. Competent Programmer Hypothesis: Software programs usually differ from the
correct version of the program in minute ways. Most of the programs written are
nearly correct.
2. Coupling Effect Hypothesis: Larger programming faults are coupled with
smaller faults of the same nature.
These hypotheses form the platform for mutation testing. Since incorrect
programs differ from the correct versions in minute ways, mutation testing works
by making the correct programs incorrect by using the concept of fault seeding.
This is done by the use of mutation operators.
These operators are simple rules which create different versions of the same
program with minor changes. These versions are called mutants. Typically,
mutation operators replace operands with other similar operands or delete entire
statements, etc. The process used in mutation testing can be explained simply in
the following steps:
1. Suppose we have a program P
2. Apply mutation operators to P to produce mutant P’
3. Apply test case t to P and P’
4. Analyze outputs of P and P’
5. If the outputs are different, then test case is effective i.e. it can detect the fault
and has killed the mutant.
6. If the outputs are same, there could be two reasons:
i. The mutant is difficult to kill. Write another test case to detect this fault.
ii. The mutant has the same semantic meaning as the original program i.e. it is
equivalent.
We need to achieve a high mutation score for any given set of test cases. This
mutation score indicates how well a particular test set has detected faults in a
program. Mutation score can be defined as:
Mutation score = # of mutants killed / total # of non-equivalent mutants
Where 0 <= M.S. <= 1 or 0% <= M.S. <= 100%
A low score means that the majority of faults cannot be detected accurately by the
test set. A higher score indicates that most of the faults have been identified with
this particular test set. A good test set will have a mutation score close to 100% or
ideally 100%.
A simple example of mutation testing has been illustrated below:
Program P
Apply mutation operation
Mutant P’
Apply test case t to P and P’
Result R Result R’
If R != R’, then t has killed mutant P’
………
………
c = a + b;
C=a*b;
D=c/a;
c = a + b;
d = c / a;
C=a**b;
D=c/a;
If R == R’, then mutant P’ is still alive
Figure 1: Process of Mutation testing
Mutation testing faces a huge challenge. Since every program may have a fault in
many possible ways, one problem is that a large number of mutants are created for
very small programs.
Another problem is that sometimes most of these programs are equivalent to the
original program.
The latter problem is the topic of discussion for this thesis.
This project focuses on the use of mutation operators to eliminate the problem of
equivalent mutants. The technique adopted suggests that some mutation operators
may contribute more towards the creation of equivalent mutants. Hence, a detailed
analysis is carried out for all the mutation operators used in MuJava (mutation
testing tool). Scenarios are sketched which indicate situations always creating
equivalent mutants for certain operators. For the extension of this thesis, it is
suggested that the scenarios sketched out be transformed into algorithms and
incorporated into the respective mutation operators.
LITERATURE REVIEW
The aim of mutation testing is to generate a good set of test cases that successfully
help find all possible faults in a software program. We need to achieve a high
mutation score i.e. a score close to 100%. To get the maximum mutation score, we
need a test set which successfully kills all the mutants. But some of the mutants
created are equivalent. Therefore, to get the highest possible score, all the non-
equivalent mutants need to be detected.
An equivalent mutant is one, which is syntactically different from the original
program, but semantically the same. Such mutants only contribute in increasing
the computational cost. They do not help in establishing whether a particular test
case is effective in discovering faults in a program or not. The problem of
determining whether a mutant is equivalent to the original mutant is theoretically
decidable.
Sample Program P
……
……
if(x == 2 && y == 2)
z = x + y;
……
Mutant P’
……
……
if(x == 2 && y == 2)
z = x * y;
……
The value of Z will be equal to 4; any test set will be unable to determine any
faults with this program because the value will always be equal to 4.
Example of Equivalent Mutants
Mutation testing is not popular in the industry. Reason being, that a large number
of mutants are generated for very small programs. This incurs a high
computational cost, as each test case needs to be executed on each mutant. The
number of mutants generated for a software unit is proportional to the product of
the number of data references and the number of data objects. To reduce the
computational cost, some approaches have been proposed which are explained
below:
2.1 ‘Do Smarter’ approaches
These approaches work by dividing computational cost amongst several machines
or by preserving some information, such as compiled code, so that the same code
need not be generated repetitively. They may also avoid complete execution of the
entire program. E.g. suppose there is a program that has 10,000 lines of code.
Assuming the 5th line is changed to create 3 different mutants. It will take a long
time to compile the mutated program again and again.
To avoid this, the compiled form of the original program is saved. This compiled
form of the original program is changed for the single line modified in the
mutants. Thus, computation cost is saved.
2.1.1 Weak Mutation
Weak mutation is an approximation technique. It compares the internal states of
the mutated program and the original program immediately after executing the
mutated portion of the program. If the state of the mutated program is different
from that of the original program, the test case has killed the mutant. Otherwise
the mutant is still alive. The first tool developed for mutation testing called
Mothra used this strategy.
Original program Mutant
result = obj.add(a,b)
………
………
Display(result);
result = obj.multiply(a,b)
/* inspect value of result */
………
Display(result);
2.1.2 Distributed Architectures
This approach works by dividing computational cost over multiple machines.
Some work has been carried out on vector processors, SIMD machines,
Hypercube (MIMD) machines, and Network (MIMD) computers to adapt them
for mutation analysis. This is easy to perform as each mutant is independent of
the other mutants and can be executed individually.
2.2 ‘Do Faster’ approaches
These approaches work by generating and running each mutant program as
quickly as possible.
2.2.1 Schema-based Mutation Analysis
Many mutation systems compile their programs using interpretive methods. This
makes the task of compilation low and more tedious to build. Untch has built a
Mutant Schema Generation system for this purpose. This system encodes all the
mutants into one source-level program to create a ‘metamutant’ . This metamutant
is compiled once and executed in the same programming environment. Since only
a fraction of code is different for each schema, repetitive runs of large programs
can be avoided on the compiler, thus saving computational cost.
Original program Metamutant
result = obj.add(a,b) Switch(n)
Case 1:
result = obj.add(a,b)
Case 2:
result = obj.subtract(a,b)
Case 3:
result = obj.multiply(a,b)
Case 4:
result = obj.divide(a,b)
Case 5:
result = obj.modulus(a,b)
Case 6:
…………
Figure 4: Example of Schema-based mutation analysis
2.2.2 Separate Compilation Approach
This method avoids the interpretative style of execution. It creates, compiles,
executes and runs each mutant individually. When mutant run times greatly
exceed individual compilation/link times, a system based on such a strategy will
execute 15 to 20 times faster than an interpretative system. One example of such a
system is Proteum.
2.3 ‘Do Fewer’ approaches
These approaches run lesser number of mutants. A subset of mutants is selected
from all the mutants created in such a way that it is sufficient to determine a good
set of test cases.
2.3.1 Selective Mutation
The aim of selective mutation is to achieve maximum coverage by using the least
number of mutation operators. Operators are selected in such a way that most of
the mutants are generated from them. E.g. multiple mutation operators, which
produce the same mutant, are discarded when generating mutants. This idea was
developed by Offutt and was called ‘selective mutation’. Since lesser operators
are used, the number of mutants produced also decreases significantly.
2.3.2 Mutation Sampling
This technique randomly selects a subset of mutants produced. This subset is then
tested with the specified test set to determine its sufficiency. If not satisfied,
another subset of mutants is selected randomly. Such subsets are continuously
selected till we find one which helps determine the effectiveness of the particular
test set.
Another method of sampling does not use a priori fixed size of mutants; it uses a
Bayesian sequential probability ratio test to determine whether a statistically
appropriate sample size of mutants has been reached.
2.4 Techniques
Some techniques have been proposed which deal with the problem of equivalent
mutants.
Some of these have been implemented using some algorithms. Tools have also
been created for mutation testing. It was imperative to study these tools as this
thesis involves the use of one of these tools.
2.4.1 Overcoming the Equivalent Mutant Problem and Achieve Tailored
Selective Mutation Using Co-evolution
This technique makes use of genetic algorithms. It is suggested that the use of
genetic algorithms may help reduce the problem of generating a large number of
mutants as well as the problem of equivalent mutants. It is argued that this
strategy helps attain selective mutation without the need of decreasing the
mutation operators applied to the programs. This technique performs three main
operations:
1. Evolution of subsets of mutants against a fixed set of test cases:
In this step, mutants are created and validated against a fixed set of test cases. The
score generated indicates the performance level for the mutant against a particular
set of test cases. A low score indicates that the mutant is difficult to kill. Subsets
of these mutants are then created. Each subset corresponds to an individual of the
genetic algorithm. A subset of non-equivalent mutants is then selected with a
higher adequacy score. Genetic algorithms are then used to evolve this set. The
fitness function used ensures that the mutants selected are such that they are
definitely killed by some test case. Uniform crossover is used to enable the
selection of highly fit individuals in the population.
2. Evolution of subsets of test cases against a fixed set of test mutants:
The same strategy is used to create a subset of test cases. Test sets are generated
randomly. Mutation scores are then assigned to each set of test cases by executing
them against a fixed set of mutants. This mutation score corresponds to the
performance of the test set against a given set of mutants. Again, each test set is
evolved using a fitness function; it results in another set of test cases which when
executed against a set of mutants give a higher adequacy score.
3. Combine the above two sets using co-evolution:
The two populations generated above are then combined using a fitness function.
The score of each mutant is re-evaluated with respect to the present population of
test cases.
Similarly, the score of each test case is re-evaluated with respect to the present
population of the mutants.
This is due to the design of the fitness functions. The fitness function created
assigns an adequacy score of 0 to all the mutants which are difficult to kill since
they are more likely to create equivalent mutants. And as explained, only mutants
and test cases with higher adequacy scores are selected.
Summarizing the study presented in this paper, selective mutation has been
achieved by the use of genetic algorithms without reducing the mutation operators
used thus reducing the computational cost. Also, equivalent mutants are not
considered to determine the effectiveness of a particular test set due to the
ingenious design of the fitness function.
2.4.2 Using Program Slicing to Assist in the Detection of Equivalent Mutants
It is often argued that it is difficult to apply mutation testing to large programs.
This is because the detection of equivalent mutants becomes more tedious and
almost impossible since it is done manually. It is debated in this paper that the
creation of program slices makes detection of equivalence easier.
A formal notation is used. Let the program be p and mutant p’. An input σ is
applied to both p and p’. p and p’ can be compared by either considering their
internal states or final output. To determine whether a mutant has been killed or
not, three different choices are considered:
1. Strong mutation:
Under strong mutation, either the final output of the two programs is considered
(strong output mutation) or the final state of p and p’ is considered (strong state
mutation). For strong output mutation, an equivalent mutant will produce the same
value of variable x as the original program. Similarly, for strong state mutation, an
equivalent mutant will produce the same state of variable x as the original
program at the end of execution.
2. Weak mutation:
In weak mutation, if the value of some variable x is different immediately after the
execution of some point in the program, say n, for both programs p and p’ when
some input σ is applied, then the mutant is killed. Hence, a mutant will be
equivalent only when the value of x is identical in p and p’ at any point n.
3. Firm mutation:
Firm mutation creates p’ by mutating some part of p, for example a loop, a control
structure, etc. It then examines p and p’ after applying some input σ to determine
if the mutant has been killed or not. If the value of variable x is the same after the
final node in the structure, the mutant is equivalent.
The strategy discussed in this paper uses weak mutation. A new Boolean variable,
say z, is introduced in the program p. A slice is then created on this variable and
analyzed to determine if the mutant has been killed or not. The value of z is
initially set to true. Every time the node n is met, the value of z is changed. If the
mutant is killed it becomes false, otherwise it remains true.
Equivalence is indicated if the value of z remains true at the end of execution.
Program p
1 substring (int from, int to, char target[], char source[]){
2 from++;
3 start = from;
4 if(from < to){
5 while(from != to){
6 target[from-start] = source[from];
7 from++;
8 }
9 target[to-start] = ‘/0’;
10 }
11 else
12 target[0] = ‘/0’;
13 }
Sample program to show strategy using program slicing to detect equivalent
mutants
Mutant p’
1 substring (int from, int to, char target[], char source[]){
2 from++;
3 start = from;
4 if(from < to){
5 while(from != to){
6 target[from-start] = source[from];
7 from++;
8 }
9 target[from-start] = ‘/0’;
10 }
11 else
12 target[0] = ‘/0’;
13 }
Example of mutant created for use with program slicing
The Boolean variable z is introduced at this point on line 2. The statement on line
10 shows that the variable to has been modified to the variable from in p’. A slice
is then created with respect to z and the variable from. This slice is then tested to
determine if the mutant has been killed or is still alive at the end of test execution.
Mutant p’ to be sliced
1 substring (int from, int to, char target[], char source[]){
2 z = TRUE;
3 from++;
4 start = from;
5 if(from < to){
6 while(from != to){
7 target[from-start] = source[from];
8 from++;
9 }
10 z = z && from == to;
11 target[from-start] = ‘/0’;
12 }
13 else
14 target[0] = ‘/0’;
15 }
Sliced mutant
1 substring (int from, int to, char target[], char source[]){
2 z = TRUE;
3 from++;
5 if(from < to)
6 while(from != to)
8 from++;
10 z = z && from == to;
15 }
Introduction of Boolean variable z in p’ & the creation of a program slice
using z
The authors also discuss that creating amorphous slices may reduce the problem
of detecting equivalent mutants even more by applying further transformations to
the traditional slices.
Amorphous Slice of mutant p’
substring (int from, int to, char target[], char source[]){
z = TRUE;
from++;
if(from < to)
z = TRUE;
}
Creation of an amorphous slice from a traditional program slice for the
detection of Equivalent mutants
An interesting observation stated in the paper states that “an ideal amorphous
slicing algorithm would yield this slice for all equivalent mutants, and the
equivalent mutant problem would disappear”.
In summary, the use of program slices can ease the process of detection of
equivalent mutants. it reduces the work carried out manually in identifying
equivalent mutants. Also, it has been suggested that equivalent mutants will
always create slices which are identical for all mutants.
2.4.3 Using Compiler Optimization Techniques to Detect Equivalent
Mutants
The strategy presented in this paper makes use of compiler optimization
techniques. The authors of this paper suggest that “many equivalent mutants are
either optimizations or de-optimizations of the original program”. Equivalent
mutants can always be detected by their optimized code by the use of algorithms.
Following are the 6 techniques explained in this paper:
1. Dead code detection
Mutants that change dead code, i.e. code that will never be executed or is
irrelevant, will not affect the output of the program and will thus be equivalent.
2. Constant propagation
It creates a constant table, which keeps entries of variables with constant
definitions.
Mutants that are still alive at the end of the execution of a test case can be checked
for equivalence using this table. A mutant that has an entry in this table will be
equivalent.
3. Invariant propagation
This technique works by saving the relationship between 2 variables, also called
an invariant, in an invariant table. An equivalent mutant will have the same
definitions in the invariant table, thus not changing the value in the mutated
program form that in the original program.
4. Common sub-expressions
This technique does not identify equivalent mutants directly. It is used in
conjunction with other compiler optimization techniques. It keeps track of all the
temporary variables created during compilation and determines the equality of
relationships between variables if any.
Normal Code Optimized Code
T1 = B + C
T2 = T1 - D
A = T2
T3 = B + C
T4 = T3 – D
X = T4
T1 = B + C
T2 = T1 – D
A = T2
X = T2
Figure 5: Common Sub-expression Detection Example
5. Loop invariant detection
Here, an equivalent mutant is one, which changes the boundary of a loop by
moving it inside or outside when the code is compiled.
6. Hoisting and sinking
This also uses the same technique as loop invariant detection. Equivalent mutants
will generate the same result regardless of hoisting or sinking the code.
2.4.4 Automatically Detecting Equivalent Mutants and Infeasible Paths
The authors of the paper presume that the problem of detecting equivalent mutants
is in-fact a type of a feasible path problem. The feasible path problem states that
some test requirements are infeasible because of the nature of the program
semantics. These requirements cannot be satisfied by the program.
The strategy presented makes use of the constraint-based testing technique. This
technique specifies a number of mathematical properties, which need to be met by
any test case. Assuming that P is a program, M is a mutant of P on statement S,
and t is a test case for P. three mathematical conditions need to be met to kill a
mutant:
1. Reachability:
The test case must execute the mutated statement. I.e. if t cannot reach S, then t
will never kill M.
2. Necessity:
To kill a mutant, the test case must cause the mutant to have an incorrect state if it
reaches the mutated statement. I.e. for t to kill M, it is necessary that if S is
reached, the state of M immediately following some execution of S must be
different from the state of P at the same point.
3. Sufficiency:
The test case must cause the final state of the mutant to be different from the
original program. I.e. the final state of M differs from that of P.
These mathematical conditions are then used to apply constraints on the
programs. The constraints are used to determine which mutants are equivalent and
which are not.
2.5 Tools
As part of the literature survey, all the tools that have been created for mutation
testing were also studied. None of these tools detect equivalent mutants
automatically. The tester has to detect them by hand. Tools have been developed
for a number of languages:
2.5.1 Mothra
It is the oldest tool developed for mutation testing. It was developed as a project
at Georgia Institute of Technology’s Software Engineering Research Centre. This
project was initiated in 1986 and developed using C language. It was developed
for programs written in FORTRAN and made for the Linux platform.
Mothra is a set of tools that can be called separately or called using one of
Mothra’s interfaces. Since it is based on a set of data structures, many researchers
have tried to expand Mothra by adding new tools. Mothra works by creating
intermediate code of the program. This requires more compilations and an
increase in performance costs.
2.5.2 Jester/Nester/Pester
The tool that was developed first was Jester. It applied mutation testing to Java
programs.
Jester works for Java code with JUnit tests. Jester does simple modifications to the
programs such as changing If statements to true or false, etc. After making these
modifications, it runs tests on the modified programs. It then generated web pages
displaying the results of the tests using a built-in script.
Pester is the same as Jester only that it was written for programs created in
Python. Test cases are written in PyUnit. Nester is also a variation of Jester
written for C# programs. Unfortunately, no literature was available to study it in
more detail.
2.5.3 MuJava
MuJava is a tool written for java programs. It uses two sets of mutation operators;
method-level and class-level. MuJava creates mutants using the various method-
level and class-level operators.
It then runs test cases on them and evaluates the mutation coverage for them. Test
cases are written as separate classes that call methods in the classes that need to be
tested.
Figure 6: Structure of MuJava
Behavior mutant
Behavior mutant generator
Compile time reflection+ms
gStandard java
API[47]
Behavioiural mutant executr
Mutant generator Mutant executor
Different engines create the structural and behavioral mutants. For the behavioral
mutants, compile time reflection is used to analyze the original program. The
MSG engine then uses compile time reflection as well to create a meta-mutant
program. For the structural mutants, the original source code is compiled using the
Java compiler. BCEL API is then used to add or delete class members in the
resulting byte code translation.
3.2 Motivation
Mutation testing has a high computational cost. A large number of mutants are
created for very small programs. This is because each program can have various
faulty versions. Each version could be a simple replacement of an addition
operator, for example, with all other arithmetic operators in a particular language.
All these mutants need to be compiled and executed against a specific set of test
cases.
In addition to this high cost, sometimes most of the mutants created are
equivalent. Equivalent mutants produce the same output for the original program
as well as the modified program or the mutant.
These issues have prevented mutation testing to gain wide-spread acceptance in
the software industry. The aim of this thesis is to help determine a technique by
the use of which the number of equivalent mutants produced is reduced.
Quite a few techniques have been proposed to date to deal with this issue as
discussed in chapter 2. Some algorithms have been proposed for them, but none of
them have been implemented so far. Most authors have resorted to combining
other software engineering strategies such as genetic algorithms and compiler
optimization techniques with mutation testing.
In view of this thesis, this has made the process of detecting equivalence more
arduous. The implementation of any of these strategies would require the creation
of an entirely new tool for mutation testing rather than reusing any one of those
already developed.
The strategy discussed here makes use of MuJava, a mutation testing tool
developed in 2003.
None of the other mutation testing tools have been used due to lack of support for
the Windows platform. Also, some of the tools did not contain enough literature to
support their operation. The main idea is that some mutation operators may
develop a greater number of equivalent mutants as compared to other operators.
To support this argument, the mutation operators of MuJava are discussed in great
detail. An analysis is carried out to see which operators create a large number of
equivalent mutants.
3.3 The Strategy
For this project, the main aim is to help identify those mutation operators that
contribute more towards creating equivalent mutants. The procedure that was
followed is explained below:
1. Test programs were created for method-level and class-level operators. These
test programs serve as original programs for which mutants will be generated
later.
2. Two different approaches were adopted for method-level and class-level
operators
a. For method-level operators, different programs were developed each for
arithmetic, shift, logical operators etc. Then all the method-level mutation
operators were applied to each type of program to create the mutants.
b. For the class-level operators, programs were written for the common data
structures of Linked list. A Stack class was then derived from it. Programs were
then written which used these classes. These programs differed for each of the
class-level mutation operators. All the class-level mutation operators were applied
to all the programs to create mutants.
3. Empirical data was gathered indicating which mutation operators created a
greater number of mutants as compared to other mutation operators.
4. Also, a manual inspection was conducted on all the mutants to discover the
equivalent versions.
5. Java programs were then created containing the test cases for each of the
method-level and class-level operators original programs created in step 2. These
test cases perform branch coverage on the original program.
6. The test cases were executed against the original programs using MuJava.
7. The mutation scores were obtained.
8. Also, it was determined that how many mutants were left alive by each of the
operators at the end of test execution.
9. Of the mutants left alive, the equivalent mutants were determined.
10. An empirical analysis was conducted to determine which mutation operators
contribute more towards creating equivalent mutants.
RESEARCH METHODOLOGY
The tool used in this project is MuJava. Before explaining the strategy adopted for
this project, it is imperative to explain the working of MuJava in more detail.
3.1 MuJava
Mutation operators make syntactic changes to the program under test. These
syntactic changes depict usual syntactical mistakes made by programmers while
writing code. MuJava implements a ‘do faster’ approach to mutation testing to
save compilation time. This approach has been adopted primarily for object-
oriented programs.
The architecture of MuJava makes use of the Mutant Schemata Generation (MSG)
approach.
This approach works by creating one meta-mutant of all the mutant programs and
requires only two compilations: compilation of the original program and
compilation of the meta-mutant program.
MuJava uses two types of mutant operators:
Operators that change the structure of the program
Operators that change the behavior of the program
The method-level operators implemented in MuJava. The operators considered for
MuJava modify the expressions by inserting, replacing or deleting the primitive
operators. MuJava contains six types of primitive operators.
1. Arithmetic operators
2. Relational operators
3. Conditional operators
4. Shift operators
5. Logical operators
6. Assignment operators
Some of these operators have short-cut versions as well. Because of this, some
operators are subdivided into binary, unary and short-cut versions. In all, there are
12 method-level operators in MuJava.
Since MuJava has been created for Java programs, only operators used in Java are
Considered for the creation of the tool
Table 1: Method-level operators in MuJava
3.2 Arithmetic Operators
Arithmetic operators perform mathematical computations on all integers and
floating-point numbers. The arithmetic operators supported in Java are:
For binary: op + op (addition), op – op (subtraction), op * op
(multiplication), op / op (division) and op % op (modulus).
For unary: + (plus indicating positive value), - (minus indicating negative
value)
For short-cut: op++ (post-increment), ++op (pre-increment), op-- (post-
decrement), --op (pre-decrement)
The arithmetic operators used in MuJava are explained below [9]:
AORB / AORB U / AORS: Arithmetic Operator Replacement
These operators replace binary, unary or short-cut arithmetic operators with other
binary, unary or short-cut operators, respectively.
AOIU / AOIS: Arithmetic Operator Insertion
These operators insert unary or short-cut arithmetic operators, respectively.
AODU / AODS: Arithmetic Operator Deletion
These operators delete unary or short-cut arithmetic operators, respectively.
3.3 Relational Operators
Relational operators compare the values of two operands. The relational operators
supported in Java are:
op > op (greater than), op >= op (greater than or equal to), op < op (less
than), op <= op (less than or equal to), op == op (equal to) and op != op (not
equal to).
Since these operators require two operands, only replacement is allowed for these
operators. The relational operator used in MuJava is explained below:
ROR: Relational Operator Replacement
This operator replaces relational operators with other relational operators.
3.4 Conditional Operators
Conditional operators or bitwise operators perform computations on the binary
values of its operands. These operators exhibit short-circuit behavior. The
conditional operators supported in Java are:
For binary: op && op (conditional AND), op || op (conditional OR), op &
op (bitwise AND), op | op (bitwise OR) and ^ (bitwise XOR).
For unary: !op (bitwise logical complement) The arithmetic operators in
MuJava are explained below :
COR: Conditional Operator Replacement This operator replaces binary
conditional operators with other binary conditional operators.
COI: Conditional Operator Insertion
This operator inserts unary conditional operators.
COD: Conditional Operator Deletion
This operator deletes unary conditional operators.
3.5 Shift Operators
Shift operators require two operands. They manipulate the bits of the first operand
in the expression by either shifting them to the right or the left. The shift operators
supported in Java are:
op >> op (signed right shift), op << op (signed left shift) and op >>> op
(unsigned right shift).
The arithmetic operators in MuJava are explained below:
SOR: Shift Operator Replacement
This operator replaces binary shift operators with other binary shift operator.
3.6 Logical Operators
Logical operators perform logical comparisons to produce a Boolean result for
comparison statements. The logical operators supported in Java are:
For binary: op & op (AND), op | op (OR) and op ôp (XOR).
For unary: ~op (bitwise complement)
The arithmetic operators in MuJava are explained below:
LOR: Logical Operator Replacement
This operator replaces binary logical operators with other binary logical operators.
LOI: Logical Operator Insertion
This operator inserts unary logical operators.
LOD: Logical Operator Deletion
This operator deletes unary logical operators.
3.7 Assignment Operators
Assignment operators set the values of an operand. The short-cut assignment
operators perform a computation on the right hand operand and then assign its
value to the left hand operand. The assignment operators supported in Java are:
For short-cut: op += op (addition assignment), op -= op (subtraction
assignment), op *= op (multiplication assignment), op /= op (division
assignment), op %= op (modulus assignment), op &= op (bitwise AND
assignment), op |= op (bitwise OR assignment), op ^= op (bitwise XOR
assignment), op <<= op (right shift assignment), op >>= op (left shift
assignment), op >>>= op (unsigned right shift assignment).
The arithmetic operators in MuJava are explained below:
ASRS: Assignment Operator Replacement Short-cut
This operator replaces short-cut assignment operators with other short-cut
assignment operators.
3.8 Class Level Mutation Operator
The operators considered for MuJava have been divided in four categories
according to their usage in Object Oriented programming. The first 3 groups
target features common to all OO languages. The last group includes features that
are specific to Java. These groups are:
1. Encapsulation
2. Inheritance
3. Polymorphism
4. Java-specific Features
Like method-level operators, the class-level operators make changes to the
program syntax by inserting, deleting or modifying the expressions under test.
Operators have been defined for each category. In all, there are 29 class-level
operators in MuJava. These operators are explained in detail below. All code
examples have been taken from.
3.8.1 Encapsulation
Encapsulation in OOP deals with data hiding. It is the ability of an object to create
a boundary around its data and methods. Encapsulation allows a programmer to
define the access levels for various objects. For this purpose many access
modifiers are used. Specifying the wrong access modifier can lead to incorrect
results.
There is only one mutation operator in MuJava, which deals with encapsulation.
AMC: Access Modifier Change
This operator replaces the access modifiers for various instance variables and
methods in a program. This allows a tester to ensure that the correct level of
accessibility is used in a program.
Original Code Mutants
public Stack s; private Stack s;
protected Stack s;
3.8.2 Inheritance
Inheritance allows the data and methods of one class (parent class) to be used by
another class (child class). Inheritance promotes code reusability.
Eight mutation operators have been developed for MuJava that deal with
inheritance.
Variable shadowing allows a child class to hide or shadow the variables in the
parent class.
This may cause incorrect variable to be accessed. Mutation operators IHD and IHI
test this issue in OOP.
IHD: Hiding variable deletion
This operator deletes a hiding variable in a child class. In this way, the variable in
the parent class can be accessed.
Original Code Mutants
class List { class List {
int size; int size;
… …..
} }
class Stack extends List { class Stack extends List {
// int size; int size;
… …..
} }
IHI: Hiding variable insertion
This operator inserts a hiding variable in a child class. It is reverse of IHD. Newly
Defined and overriding methods in a subclass reference the hiding variable
whereas inherited methods reference the hidden variable as before.
Original Code Mutants
class List { class List {
int size; int size;
… ….
} }
class Stack extends List { class Stack extends List {
… int size;
… } }
A child class can modify the behavior of its parent class by creating a method with
the same name and arguments as the parent class. This is called method
overriding. Mutation operators IOD, IOP and IOR are used to test issues related to
method overriding in MuJava.
IOD: Overriding method deletion
This operator deletes the entire declaration of the overriding method in the child
class.
All references to this function then access the method in the parent class.
Original Code Mutants
class Stack extends List { class Stack extends List {
… ….
void push(int a){ // void push(int a){
… ….
} }
} }
IOP: Overridden method calling position change
Sometimes the overriding method in the child class may need to call the method it
overrides in the parent class. It is necessary to test that the method in the parent
class is called at the right point in the program otherwise it would cause the
program to be in an incorrect state.
IOR: Overridden method rename
This operator is used to check if an overriding method adversely affects other
methods. It renames the method being overridden in the parent class so that the
overriding method in the child class doesn’t affect the method in the parent class.
The super keyword allows a programmer to access the member variables and
functions of the parent class when using variable shadowing or method overriding.
Mutation operators ISI and ISD are used to test issues regarding use of the super
keyword in MuJava.
ISI: super keyword insertion
This operator inserts the super keyword so that any references to the variable or
Method go to the overridden method or variable.
ISD: super keyword deletion
This operator deletes the super keyword so that any references to the variable or
Method go to the overriding method or variable.
The constructors of the parent class are not inherited like other methods.
Whenever an object of a child class is created, it automatically invokes the default
constructor of the parent’s class before invoking its own constructor. The child
class can also use the super keyword to invoke a specific parent class constructor.
Mutation operator IPC is used to test this in MuJava.
IPC: explicit call of a parent’s constructor deletion
This operator deletes calls to the parent class’s constructor. This causes the default
Constructor of the parent class to be called.
3.8.3 Polymorphism
Polymorphism allows objects to react differently to the same method. It is
implemented by having many methods with the same name.
Ten mutation operators have been developed for MuJava that deal with
polymorphism.
PNC: new method call with child class type
This operator changes the constructor used to instantiate an object i.e. it changes
the type with which the object is instantiated. It makes the object reference refer to
an object of a different type than that with which it is declared
Original Code Mutants
Parent a; Parent a;
a = new Parent(); //a = new Child();
PMD: member variable declaration with parent class type
This operator changes the declared type of an object reference to the parent of the
Original declared type.
Original Code Mutants
Child b; // Parents b;
b = new Child(); b = new Child();
3.9 Method-level Mutation Operators
There are 12 method-level operators in MuJava. To develop mutants for these
operators the following strategy was used:
1. One type of mutation operator was chosen, e.g. Arithmetic mutation operators
i.e. AOR, AOI, AOD.
2. A simple Java program (say ‘ArithOper.java’) was developed, which used this
type of mutation operator, in this case, arithmetic operators.
3. MuJava was started and all the method-level operators were selected.
4. Using MuJava, mutants were created for ‘ArithOper.java’.
5. Another java program was created (say ‘ArithOperTest.java’) which contained
the test cases for ‘ArithOper.java’. These test cases must exercise branch coverage
for ‘ArithOper.java’.
6. ‘ArithOperTest.java’ was run against ‘ArithOper.java’ to determine how many
mutants were killed and the mutation score was calculated.
7. Of the mutants still alive at the end of the execution, the equivalent mutants
were determined manually.
8. The entire process from steps 1 to 7 was repeated to develop java programs for
each type of method-level mutation operator i.e. relational, conditional, shift,
logical and assignment.
Using the above process, the data collected in the first instance indicates the
number of mutants created for each method-level operator.
OBJECTIVE OF THE STUDY
This thesis targets this issue by determining which mutation operators create
mutants, which are more probable to create equivalent mutants. A tool called
MuJava, Jody is used for this purpose. An empirical analysis has been carried out
for this purpose, which helps determine which mutation operators develop more
equivalent mutants.
EXPERIMANTAL SETUP & RESULTS
The muJava system requires that the Java CLASSPATH be modified to include
the muJava jar and the Java tools.jar files. Also, the general PATH variable must
include the java bin directory; this is usually set automatically when java was
installed, but not always. Then one GUI (Java applet) is used to generate mutants,
the tester must create tests, and another GUI is used to run mutants.
1. Environment Settings for the muJava System
There are three steps to setting up the environment for muJava, (1) CLASSPATH,
(2) setting the config file, and (3) creating subdirectories.
i. The Java CLASSPATH must include two µJava jar files and one standard
Java jar file. tools.jar is standard with Java compilers and is probably
located in the "lib/" directory. The two µJava files are mujava.jar and
openjava2005.jar, which are downloaded from this site. One slightly
awkward requirement is that the CLASSPATH must include the location of
the classes under test when generating mutants, but NOT when running
mutants. (If it does, no mutants can be killed.) This location is the "classes/"
directory under where the mujava.config is stored.
In a DOS window, use the following command (assuming that classes is
under C:\mujava):
set CLASSPATH=%CLASSPATH%;C:\mujava\mujava.jar;C:\mujava\openjava2005.jar;C:\j2sdk1.4.0_01\lib\tools.jar;C:\mujava\classes
In a Cygwin window, use the following command:
CLASSPATH="$CLASSPATH;C:\mujava\mujava.jar;C:\mujava\openjava2005.jar;C:\j2sdk1.4.0_01\lib\tools.jar;C:\mujava\classes" ; export CLASSPATH
To change your CLASSPATH permanently in Win2000 and WinXP, go to
Start-settings-Control Panel. Double-click System, go to the Advanced tab,
and choose Environment Variables. Edit the CLASSPATH variable or
create a new variable if there is none. Add the full path to mujava.jar and
openjava2005.jar to the CLASSPATH.
In Unix, set the CLASSPATH environment variable. Assuming the jar files
are stored in the home directory of user gpd:
CLASSPATH=$CLASSPATH:/home/gpd/mujava.jar:/home/gpd/openjava2
005.jar:/java1.4/j2sdk1.4.0_01/lib/tools.jar ; export CLASSPATH
Note that the syntax will vary under different shells, and it will be more
convenient to put the command in your setup file such as .login, or .bashrc.
ii. Next, modify the mujava.config file to point to a directory where you wish
to store source Java files and muJava temporary files. The directory must
be the complete path (either Windows or Unix). For example, the config
file may contain the line: MuJava_HOME=C:\home\gpd\exp.
IMPORTANT: It is necessary to copy the config file to the directory you
run the muJava system in.
iii. Finally, create a directory structure for the muJava system in the
$MuJava_HOME directory. Assuming your MuJava_HOME directory is
called MuJava, the subdirectories should look like:
There should be four subdirectories, used in the following ways.
<TDMuJava_HOME\
srcdirectory for Java files to be tested
MuJava_HOME\classesdirectory for compiled classes of Java files from
MuJava_HOME\src
MuJava_HOME\testset directory for test sets
MuJava_HOME\result directory for generated mutants
You can create these subdirectories by hand or by using the muJava class
"mujava.makeMuJavaStructure".
java mujava.makeMuJavaStructure
We have identified several potential problems with installing µJava.
µJava does not work with Java 1.5.
It is important that the MuJava_HOME variable NOT have a trailing slash.
This will confuse µJava.
If you have a different version of the java compiler and the JVM, µJava
may get confused. This happens sometimes when a new application on your
computer updates the JVM. If you have problems compiling or killing
mutants, we suggest deleting all Java components and reinstalling the latest
version.
If your tools.jar file is out of date (pre Java 1.4, we think), parts of µJava
may not work.
2. Generating Mutants with muJava
Important note: You should run all commands in a directory that contains
"mujava.config"
i. Put the source files to test to MuJava_HOME\src directory. muJava does
not check for compilation errors, so all Java files should compile correctly.
If the Java file to test needs other Java files or class files, they should also
be placed in MuJava_HOME\src. For example, suppose you want to test B,
which is a child class of A. Then, you should put both A.java and B.java
into MuJava_HOME\src. If the file has a package structure, you should
store the entire package underneath MuJava_HOME\src.
ii. Compile all the Java files in MuJava_HOME\src and copy the .class files
into the MuJava_HOME\classes\ directory.
iii. Start the GUI from the command line. Use it to generate mutants:
java mujava.gui.GenMutantsMain
iv. This command should bring a up a screen similar to the following:
i. Select the files you want to mutate by clicking in the boxes on the left.
Select the mutation operators you want to use by slecting their boxes. Then
push RUN.
ii. Note: The class mutation operators produce far fewer mutants. Also
note that a number of status messages go to the command window, but
not the GUI.
iii. After mutants are generated, you can view the mutants in the "Class
Mutants Viewer" and "Traditional Mutants Viewer" tabs, as shown in the
following two figures.
iv.v.
vi.
vii. You may be interested in knowing where the mutant versions of the classes
are stored. They are underneath the MuJava_HOME\result\ directory. The
following example shows the directory Stack underneath result, with
object-oriented mutants in class_mutants and traditional mutants in a
separate directory.
3. Making a test set
A testset in muJava is a Java file that contains executable test scripts. Each test is
a method that contains a sequence of calls to methods in the class under test. Each
test method returns a string result that is used to compare outputs of mutants with
outputs of the original class. Each test method should start with the string "test".
The test methods and the test class should have public access.
Below is an example of a testset class for the class Stack. Its name is StackTest.
StackTest contains two test case: test1() and test2(). The testset .class file should
be in the directory MuJava_HOME\testset\. (By the way, although these test
scripts are reminiscient of JUnit tests, µJava does not accept JUnit tests. The first
version of µJava predated JUnit so we implemented a general test driver facility.)
public class StackTest{
public String test1() { String result = ""; Stack obj = new Stack(); obj.push(2); obj.push(4); result = result + obj.isFull(); result = result + obj.pop(); return result; }
public String test2()
{ String result = ""; Stack obj = new Stack(); obj.push(5); obj.push(3); result = result + obj.pop(); result = result + obj.pop(); return result; }
}
4. Running mutants.
Run the mutants from another GUI. Start it with the following command:
java mujava.gui.RunTestMain
Note: Your CLASSPATH must not include MuJava_HOME\classes\. If it
does, no mutants can be killed.
You should see the following GUI. You can select which collection of mutants to
run, and which testset to use. The "Class Mutants Viewer" and "Traditional
Mutants Viewer" tabs will show the source view of the mutants. You can design
tests to kill mutants by finding a live mutant, then analyzing the program to decide
what input will kill it. Remember that between 5% to 20% of the mutants are
typically equivalent.
CONCLUSION AND FUTURE WORK
Detailed work has been carried out to determine which mutation operators
contribute more towards creating equivalent mutants. The following results were
obtained:
Method-level operators:
The majority of mutants were created for AOIS, ROR and LOI.
Almost half of the mutants created for AOIS were found to be equivalent.
However, majority of the mutants created for ROR and LOI were killed
successfully.
All the mutants created for COI failed the software.
Class-level operators:
Majority of mutants were created for PRV, JSI, IHI and IOD.
Almost 50% of the mutants created for PRV, failed the software. Of the remaining
mutants, almost 80% were killed and the remaining was found to be equivalent.
Nearly 95% of the mutants created for PMD caused MuJava to fail.
All the mutants created for JDC were found to be equivalent.
Many class-mutation operators such as IOR, ISI, etc, which created a very low
number of mutants, were found to be equivalent.
Various scenarios have been sketched out for method-level operator AOIS and
class-level operators IHD, IHI, IOR, PRV, JID and JDC. These scenarios indicate
situations which always create equivalent mutants.
Looking at these results, different conclusions can be made for method-level
mutation operators and class-level mutation operators. For method-level operators,
AOIS is more probable to create equivalent mutants. For class-level operators,
PRV usually creates the greatest number of mutants. But most of these mutants
are simply program failures. Also, operators that create a very low number of
mutants as compared to other class-level operators are also most likely to create
equivalent mutants.
For the future work of this project, the operators that create a larger number of
equivalent mutants can be tweaked such that they do not create equivalent
mutants. This can be done by using the scenarios sketched out in chapter 7. These
scenarios can be incorporated in the algorithms implemented for the concerned
operators. Thus, the creation of equivalent mutants can be avoided.
APPENDIX A – USER GUIDE for MuJava
MuJava requires 2 jar files and one config file. These files can be found at:
http://ise.gmu.edu/~ofut/mujava/
These 2 files, (mujava.jar & adaptedOJ.jar) should be placed in the same
directory. Let’s assume the directory is C:\MuJava\.
Before executing the software, the following steps should be taken:
1. Your computer must have a JRE installed and the Java path set. To learn how to
do this refer to the java.sun.com website for details.
2. After this, firsts et the classpath using the following command:
set CLASSPATH=%CLASSPATH%;C:\mujava\mujava.jar;C:\mujava\
adaptedOJ.jar;C:\j2sdk1.4.0_01\lib\tools.jar;C:\mujava\classes
This command allows you to run MuJava to create mutants. To run test cases a
different path must be set which is explained later.
3. After this open the mujava.config file. You should see ‘MuJava_HOME=C:\
ofut\mujava’.
Change this to point to ‘MuJava_HOME=C:\MuJava_Home’. This command tells
MuJava where to find the source files and the test programs when executing
MuJava.
4. Now we need to make a directory structure for MuJava. All the source
programs and test programs will be contained in this directory structure. This
structure can be made either manually or by using the command:
java mujava.makeMuJavaStructure
The directory structure will now look like:
C:\MuJava_Home\classes
C:\MuJava_Home\testset
C:\MuJava_Home\src
C:\MuJava_Home\result
Now, you will have 2 different directories; one will be ‘C:\MuJava’ and the other
will have ‘C:\MuJava_Home\’. The config file should be in the directory of
MuJava.
Creating Mutants
1. Create your source file (say ‘AccessorModifier.java’) that you want to test in
C:\MuJava_Home\src.
2. Compile this file using the DOS prompt and the command ‘javac
AccessorModifier.java’. This
command will create a class file in the folder C:\MuJava_Home\src\.
3. Remove this file from the
C:\MuJava_Home\src\ and copy it to the folder
C:\MuJava_Home\classes\.
4. Now change the path in the dos prompt to point to C:\MuJava\ instead of
C:\MuJava_Home\src\.
5. Now set the classpath using:
set classpath=%classpath%;c:\MuJava\mujava.jar;c:\MuJava\adaptedOj.jar;c:\
J2ee\jdk\lib\tools.jar;c:\MuJava_Home\classes;
6. Then use the following command to start MuJava:
java mujava.gui.GenMutantsMain
7. Now select the programs from the list shown for which you want to create the
mutants. Also select the mutation operators for which you want to create mutants.
Then press ‘Generate’.
Whilst mutants are being generated, many messages will be sent to the command
prompt in the background. The ‘Generate’ button will be disabled. When all the
mutants have been generated, this button will turn yellow again and the different
mutants that have been generated can then be viewed.
APPENDIX B – PROGRAME WRITTEN FOR Mujava (source code)
Method-level Mutation Operators The programs written in this section were
developed to create mutants for method-level operators in MuJava. They were
written arbitrarly. Test cases written for all the programs are also included.
ArithOper.java
/* @author: Maryam Umar */
public class ArithOper {
public String oper(int i){
int val = 1 + 2 + 3*i + (4 + 8)/3;
String result = "";
result = Integer.toString(val);
return result;
}
}
ArithOperTest.java
/* @author: Maryam Umar */
public class ArithOperTest{
public String test1(){
String result = "";
ArithOper obj = new ArithOper();
result = result + obj.oper(0);
return result;
}
public String test2(){
String result = "";
ArithOper obj = new ArithOper();
result = result + obj.oper(1);
return result;
}
public String test3(){
String result = "";
ArithOper obj = new ArithOper();
result = result + obj.oper(2);
return result;
}
public String test4(){
String result = "";
ArithOper obj = new ArithOper();
result = result + obj.oper(3);
return result;
}
}
AssignOper.java
/* @author: Maryam Umar */
public class AssignOper {
public String operation(int i){
String result = "";
i += 10;
result = Integer.toString(i);
return result;
}
}
AssignOperTest.java
/* @author: Maryam Umar */
public class AssignOperTest{
public String test1(){
String result = "";
AssignOper obj = new AssignOper();
result = result + obj.operation(0);
return result;
}
public String test2(){
String result = "";
AssignOper obj = new AssignOper();
result = result + obj.operation(1);
//All Mutants Generator Code
package mujava;
import openjava.mop.*;
import openjava.ptree.*;
import java.io.*;
import mujava.op.*;
import mujava.op.basic.*;
import mujava.op.util.*;
import mujava.util.Debug;
/**
* <p>Generate all mutants</p>
* <p>Copyright: Copyright (c) 2005 by Yu-Seung Ma, ALL RIGHTS
RESERVED </p>
* @author Yu-Seung Ma
* @version 1.0
*/
public class AllMutantsGenerator extends MutantsGenerator
{
boolean existIHD = false;
String[] classOp;
String[] traditionalOp;
public AllMutantsGenerator(File f)
{
super(f);
classOp = MutationSystem.cm_operators;
traditionalOp = MutationSystem.tm_operators;
}
public AllMutantsGenerator(File f, boolean debug)
{
super(f, debug);
classOp = MutationSystem.cm_operators;
traditionalOp = MutationSystem.tm_operators;
}
public AllMutantsGenerator(File f, String[] cOP, String[] tOP)
{
super(f);
classOp = cOP;
traditionalOp = tOP;
}
void genMutants()
{
if (comp_unit == null)
{
System.err.println(original_file + " is skipped.");
}
ClassDeclarationList cdecls = comp_unit.getClassDeclarations();
if (cdecls == null || cdecls.size() == 0)
return;
if (traditionalOp != null && traditionalOp.length > 0)
{
Debug.println("* Generating traditional mutants");
MutationSystem.clearPreviousTraditionalMutants();
MutationSystem.MUTANT_PATH =
MutationSystem.TRADITIONAL_MUTANT_PATH;
CodeChangeLog.openLogFile();
genTraditionalMutants(cdecls);
CodeChangeLog.closeLogFile();
}
if (classOp != null && classOp.length > 0)
{
Debug.println("* Generating class mutants");
MutationSystem.clearPreviousClassMutants();
MutationSystem.MUTANT_PATH =
MutationSystem.CLASS_MUTANT_PATH;
CodeChangeLog.openLogFile();
genClassMutants(cdecls);
CodeChangeLog.closeLogFile();
}
}
void genClassMutants(ClassDeclarationList cdecls)
{
genClassMutants1(cdecls);
genClassMutants2(cdecls);
}
void genClassMutants2(ClassDeclarationList cdecls)
{
for (int j=0; j<cdecls.size(); ++j)
{
ClassDeclaration cdecl = cdecls.get(j);
if (cdecl.getName().equals(MutationSystem.CLASS_NAME))
{
DeclAnalyzer mutant_op;
if (hasOperator(classOp, "IHD"))
{
Debug.println(" Applying IHD ... ... ");
mutant_op = new IHD(file_env, null, cdecl);
generateMutant(mutant_op);
if (((IHD)mutant_op).getTotal() > 0)
existIHD = true;
}
if (hasOperator(classOp, "IHI"))
{
Debug.println(" Applying IHI ... ... ");
mutant_op = new IHI(file_env, null, cdecl);
generateMutant(mutant_op);
}
if (hasOperator(classOp, "IOD"))
{
Debug.println(" Applying IOD ... ... ");
mutant_op = new IOD(file_env, null, cdecl);
generateMutant(mutant_op);
}
if (hasOperator(classOp, "OMR"))
{
Debug.println(" Applying OMR ... ... ");
mutant_op = new OMR(file_env, null, cdecl);
generateMutant(mutant_op);
}
if (hasOperator(classOp, "OMD"))
{
Debug.println(" Applying OMD ... ... ");
mutant_op = new OMD(file_env, null, cdecl);
generateMutant(mutant_op);
}
if (hasOperator(classOp, "JDC"))
{
Debug.println(" Applying JDC ... ... ");
mutant_op = new JDC(file_env, null, cdecl);
generateMutant(mutant_op);
}
}
}
}
void genClassMutants1(ClassDeclarationList cdecls)
{
for (int j=0; j<cdecls.size(); ++j)
{
ClassDeclaration cdecl = cdecls.get(j);
if (cdecl.getName().equals(MutationSystem.CLASS_NAME))
{
String qname = file_env.toQualifiedName(cdecl.getName());
try
{
mujava.op.util.Mutator mutant_op;
if (hasOperator(classOp,"AMC"))
{
Debug.println(" Applying AMC ... ... ");
mutant_op = new AMC(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "IOR"))
{
Debug.println(" Applying IOR ... ... ");
try
{
Class parent_class = Class.forName(qname).getSuperclass();
if (!(parent_class.getName().equals("java.lang.Object")))
{
String temp_str = parent_class.getName();
String result_str = "";
for (int k=0; k<temp_str.length(); k++)
{
char c = temp_str.charAt(k);
if (c == '.')
{
result_str = result_str + "/";
}
else
{
result_str = result_str + c;
}
}
File f = new File(MutationSystem.SRC_PATH, result_str +
".java");
if (f.exists())
{
CompilationUnit[] parent_comp_unit = new CompilationUnit[1];
FileEnvironment[] parent_file_env = new FileEnvironment[1];
this.generateParseTree(f, parent_comp_unit, parent_file_env);
this.initParseTree(parent_comp_unit, parent_file_env);
mutant_op = new IOR(file_env, cdecl, comp_unit);
((IOR)mutant_op).setParentEnv(parent_file_env[0],
parent_comp_unit[0]);
comp_unit.accept(mutant_op);
}
}
} catch (ClassNotFoundException e)
{
System.out.println(" Exception at generating IOR mutant. File :
AllMutantsGenerator.java ");
} catch (NullPointerException e1)
{
System.out.print(" IOP ^^; ");
}
}
if (hasOperator(classOp, "ISD"))
{
Debug.println(" Applying ISD ... ... ");
mutant_op = new ISD( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "IOP"))
{
Debug.println(" Applying IOP ... ... ");
mutant_op = new IOP(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "IPC"))
{
Debug.println(" Applying IPC ... ... ");
mutant_op = new IPC( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "PNC"))
{
Debug.println(" Applying PNC ... ... ");
mutant_op = new PNC( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "PMD"))
{
Debug.println(" Applying PMD ... ... ");
// if(existIHD){
mutant_op = new PMD( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
//}
}
if (hasOperator(classOp, "PPD"))
{
Debug.println(" Applying PPD ... ... ");
// if(existIHD){
mutant_op = new PPD( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
// }
}
if (hasOperator (classOp, "PRV"))
{
Debug.println(" Applying PRV ... ... ");
mutant_op = new PRV( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "PCI"))
{
Debug.println(" Applying PCI ... ... ");
mutant_op = new PCI( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "PCC"))
{
Debug.println(" Applying PCC ... ... ");
mutant_op = new PCC( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "PCD"))
{
Debug.println(" Applying PCD ... ... ");
mutant_op = new PCD( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "JSD"))
{
Debug.println(" Applying JSC ... ... ");
mutant_op = new JSD( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "JSI"))
{
Debug.println(" Applying JSI ... ... ");
mutant_op = new JSI( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "JTD"))
{
Debug.println(" Applying JTD ... ... ");
mutant_op = new JTD( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "JTI"))
{
Debug.println(" Applying JTI ... ... ");
mutant_op = new JTI( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "JID"))
{
Debug.println(" Applying JID ... ... ");
mutant_op = new JID( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "OAN"))
{
Debug.println(" Applying OAN ... ... ");
mutant_op = new OAN( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "EOA"))
{
Debug.println(" Applying EOA ... ... ");
mutant_op = new EOA( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "EOC"))
{
Debug.println(" Applying EOC ... ... ");
mutant_op = new EOC( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "EAM"))
{
Debug.println(" Applying EAM ... ... ");
mutant_op = new EAM( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "EMM"))
{
Debug.println(" Applying EMM ... ... ");
mutant_op = new EMM( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
} catch (ParseTreeException e )
{
System.err.println( "Encountered errors during generating mutants." );
e.printStackTrace();
}
}
}
}
/**
* Compile mutants into bytecode
*/
public void compileMutants()
{
if (traditionalOp != null && traditionalOp.length > 0)
{
Debug.println("* Compiling traditional mutants into bytecode");
MutationSystem.MUTANT_PATH =
MutationSystem.TRADITIONAL_MUTANT_PATH;
super.compileMutants();
}
if (classOp != null && classOp.length > 0)
{
Debug.println("* Compiling class mutants into bytecode");
MutationSystem.MUTANT_PATH =
MutationSystem.CLASS_MUTANT_PATH;
super.compileMutants();
}
}
void genTraditionalMutants(ClassDeclarationList cdecls)
{
for(int j=0; j<cdecls.size(); ++j)
{
ClassDeclaration cdecl = cdecls.get(j);
if (cdecl.getName().equals(MutationSystem.CLASS_NAME))
{
try
{
mujava.op.util.Mutator mutant_op;
boolean AOR_FLAG = false;
if (hasOperator(traditionalOp, "AORB"))
{
Debug.println(" Applying AOR-Binary ... ... ");
AOR_FLAG = true;
mutant_op = new AORB(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(traditionalOp, "AORS"))
{
Debug.println(" Applying AOR-Short-Cut ... ... ");
AOR_FLAG = true;
mutant_op = new AORS(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(traditionalOp, "AODU"))
{
Debug.println(" Applying AOD-Normal-Unary ... ... ");
mutant_op = new AODU(file_env, cdecl, comp_unit);
((AODU)mutant_op).setAORflag(AOR_FLAG);
comp_unit.accept(mutant_op);
}
if (hasOperator(traditionalOp, "AODS"))
{
Debug.println(" Applying AOD-Short-Cut ... ... ");
mutant_op = new AODS(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(traditionalOp, "AOIU"))
{
Debug.println(" Applying AOI-Normal-Unary ... ... ");
mutant_op = new AOIU(file_env, cdecl, comp_unit);
((AOIU)mutant_op).setAORflag(AOR_FLAG);
comp_unit.accept(mutant_op);
}
if (hasOperator(traditionalOp, "AOIS"))
{
Debug.println(" Applying AOI-Short-Cut ... ... ");
mutant_op = new AOIS(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(traditionalOp, "ROR"))
{
Debug.println(" Applying ROR ... ... ");
mutant_op = new ROR(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(traditionalOp, "COR"))
{
Debug.println(" Applying COR ... ... ");
mutant_op = new COR(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(traditionalOp, "COD"))
{
Debug.println(" Applying COD ... ... ");
mutant_op = new COD(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(traditionalOp, "COI"))
{
Debug.println(" Applying COI ... ... ");
mutant_op = new COI(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(traditionalOp, "SOR"))
{
Debug.println(" Applying SOR ... ... ");
mutant_op = new SOR(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(traditionalOp, "LOR"))
{
Debug.println(" Applying LOR ... ... ");
mutant_op = new LOR(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(traditionalOp, "LOI"))
{
Debug.println(" Applying LOI ... ... ");
mutant_op = new LOI(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(traditionalOp, "LOD"))
{
Debug.println(" Applying LOD ... ... ");
mutant_op = new LOD(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(traditionalOp, "ASRS"))
{
Debug.println(" Applying ASR-Short-Cut ... ... ");
mutant_op = new ASRS(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
} catch (ParseTreeException e)
{
System.err.println( "Exception, during generating traditional mutants for
the class "
+ MutationSystem.CLASS_NAME);
e.printStackTrace();
}
}
}
}
}
Class Mutant Generator
package mujava;
import openjava.mop.*;
import openjava.ptree.*;
import java.io.*;
import mujava.op.*;
import mujava.op.util.*;
import mujava.util.Debug;
/**
* <p>Generate class mutants according to selected
* class mutation operator(s) from gui.GenMutantsMain.
* The original version is loaded, mutated, and compiled.
* Outputs (mutated source and class files) are in the
* class-mutants folder. </p>
*
* <p> Currently available class mutation operators:
* (1) AMC: Access modifier change,
* (2) IHD: Hiding variable deletion,
* (3) IHI: Hiding variable insertion,
* (4) IOD: Overriding method deletion,
* (5) IOP: Overriding method calling position change,
* (6) IOR: Overriding method rename,
* (7) ISI: Super keyword insertion,
* (8) ISD: Super keyword deletion,
* (9) IPC: Explicit call to parent's constructor deletion,
* (10) PNC: New method call with child class type,
* (11) PMD: Member variable declaration with parent class type,
* (12) PPD: Parameter variable declaration with child class type,
* (13) PCI: Type cast operator insertion,
* (14) PCC: Cast type change,
* (15) PCD: Type cast operator deletion,
* (16) PRV: Reference assignment with other compatible variable,
* (17) OMR: Overloading method contents replace,
* (18) OMD: Overloading method deletion,
* (19) OAN: Arguments of overloading method call change,
* (20) JTI: Java-specific this keyword insertion,
* (21) JTD: Java-specific this keyword deletion,
* (22) JSI: Java-specific static modifier insertion,
* (23) JSD: Java-specific static modifier deletion,
* (24) JID: Java-specific member variable initialization deletion,
* (25) JDC: Java-supported default constructor creation,
* (26) EOA: Java-specific reference assignment and content assignment
replacement,
* (27) EOC: Java-specific reference comparison and content assignment
replacement,
* (28) EAM: Java-specific accessor method change,
* (29) EMM: Java-specific modifier method change
* </p>
* <p>Copyright: Copyright (c) 2005 by Yu-Seung Ma, ALL RIGHTS
RESERVED </p>
* @author Yu-Seung Ma
* @version 1.0
*/
public class ClassMutantsGenerator extends MutantsGenerator
{
boolean existIHD = false;
String[] classOp;
public ClassMutantsGenerator (File f)
{
super(f);
classOp = MutationSystem.cm_operators;
}
public ClassMutantsGenerator (File f, boolean debug)
{
super(f, debug);
classOp = MutationSystem.cm_operators;
}
public ClassMutantsGenerator (File f, String[] cOP)
{
super(f);
classOp = cOP;
}
/**
* Verify if the target Java source and class files exist,
* generate class mutants
*/
void genMutants()
{
if (comp_unit == null)
{
System.err.println(original_file + " is skipped.");
}
ClassDeclarationList cdecls = comp_unit.getClassDeclarations();
if (cdecls == null || cdecls.size() == 0)
return;
if (classOp != null && classOp.length > 0)
{
Debug.println("* Generating class mutants");
MutationSystem.clearPreviousClassMutants();
MutationSystem.MUTANT_PATH =
MutationSystem.CLASS_MUTANT_PATH;
CodeChangeLog.openLogFile();
genClassMutants(cdecls);
CodeChangeLog.closeLogFile();
}
}
/**
* Apply selected class mutation operators
* @param cdecls
*/
void genClassMutants (ClassDeclarationList cdecls)
{
genClassMutants1(cdecls);
genClassMutants2(cdecls);
}
/**
* Apply selected class mutation operators: IHD, IHI, IOD, OMR, OMD, JDC
* @param cdecls
*/
void genClassMutants2 (ClassDeclarationList cdecls)
{
for (int j=0; j<cdecls.size(); ++j)
{
ClassDeclaration cdecl = cdecls.get(j);
if (cdecl.getName().equals(MutationSystem.CLASS_NAME))
{
DeclAnalyzer mutant_op;
if (hasOperator(classOp, "IHD"))
{
Debug.println(" Applying IHD ... ... ");
mutant_op = new IHD(file_env, null, cdecl);
generateMutant(mutant_op);
if ( ( (IHD)mutant_op).getTotal() > 0 )
existIHD = true;
}
if (hasOperator(classOp, "IHI"))
{
Debug.println(" Applying IHI ... ... ");
mutant_op = new IHI(file_env, null, cdecl);
generateMutant(mutant_op);
}
if (hasOperator(classOp, "IOD"))
{
Debug.println(" Applying IOD ... ... ");
mutant_op = new IOD(file_env, null, cdecl);
generateMutant(mutant_op);
}
if (hasOperator(classOp, "OMR"))
{
Debug.println(" Applying OMR ... ... ");
mutant_op = new OMR(file_env, null, cdecl);
generateMutant(mutant_op);
}
if (hasOperator(classOp, "OMD"))
{
Debug.println(" Applying OMD ... ... ");
mutant_op = new OMD(file_env, null, cdecl);
generateMutant(mutant_op);
}
if (hasOperator(classOp, "JDC"))
{
Debug.println(" Applying JDC ... ... ");
mutant_op = new JDC(file_env, null, cdecl);
generateMutant(mutant_op);
}
}
}
}
/**
* Apply selected class mutation operators:
* AMC, IOR, ISD, IOP, IPC, PNC, PMD, PPD,
* PRV, PCI, PCC, PCD, JSD, JSI, JTD, JTI,
* JID, OAN, EOA, EOC, EAM, EMM
* @param cdecls
*/
void genClassMutants1(ClassDeclarationList cdecls)
{
for (int j=0; j<cdecls.size(); ++j)
{
ClassDeclaration cdecl = cdecls.get(j);
if (cdecl.getName().equals(MutationSystem.CLASS_NAME))
{
String qname = file_env.toQualifiedName(cdecl.getName());
try
{
mujava.op.util.Mutator mutant_op;
if (hasOperator(classOp, "AMC"))
{
Debug.println(" Applying AMC ... ... ");
mutant_op = new AMC(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "IOR"))
{
Debug.println(" Applying IOR ... ... ");
try
{
Class parent_class = Class.forName(qname).getSuperclass();
if ( !(parent_class.getName().equals("java.lang.Object")) )
{
String temp_str = parent_class.getName();
String result_str = "";
for (int k=0; k<temp_str.length(); k++)
{
char c = temp_str.charAt(k);
if (c == '.')
{
result_str = result_str + "/";
}
else
{
result_str = result_str + c;
}
}
File f = new File(MutationSystem.SRC_PATH, result_str +
".java");
if (f.exists())
{
CompilationUnit[] parent_comp_unit = new CompilationUnit[1];
FileEnvironment[] parent_file_env = new FileEnvironment[1];
this.generateParseTree(f, parent_comp_unit, parent_file_env);
this.initParseTree(parent_comp_unit, parent_file_env);
mutant_op = new IOR(file_env, cdecl, comp_unit);
((IOR)mutant_op).setParentEnv(parent_file_env[0],
parent_comp_unit[0]);
comp_unit.accept(mutant_op);
}
}
}
catch (ClassNotFoundException e)
{
System.out.println(" Exception at generating IOR mutant. file :
AllMutantsGenerator.java ");
}
catch (NullPointerException e1)
{
System.out.print(" IOP ^^; ");
}
}
if (hasOperator(classOp, "ISD"))
{
Debug.println(" Applying ISD ... ... ");
mutant_op = new ISD( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "IOP"))
{
Debug.println(" Applying IOP ... ... ");
mutant_op = new IOP(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "IPC"))
{
Debug.println(" Applying IPC ... ... ");
mutant_op = new IPC( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "PNC"))
{
Debug.println(" Applying PNC ... ... ");
mutant_op = new PNC( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "PMD"))
{
Debug.println(" Applying PMD ... ... ");
//if(existIHD){
mutant_op = new PMD( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
//}
}
if (hasOperator(classOp, "PPD"))
{
Debug.println(" Applying PPD ... ... ");
// if(existIHD){
mutant_op = new PPD( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
//}
}
if (hasOperator(classOp, "PRV"))
{
Debug.println(" Applying PRV ... ... ");
mutant_op = new PRV( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "PCI"))
{
Debug.println(" Applying PCI ... ... ");
mutant_op = new PCI( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "PCC"))
{
Debug.println(" Applying PCC ... ... ");
mutant_op = new PCC( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "PCD"))
{
Debug.println(" Applying PCD ... ... ");
mutant_op = new PCD( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "JSD"))
{
Debug.println(" Applying JSC ... ... ");
mutant_op = new JSD( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "JSI"))
{
Debug.println(" Applying JSI ... ... ");
mutant_op = new JSI( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "JTD"))
{
Debug.println(" Applying JTD ... ... ");
mutant_op = new JTD( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "JTI"))
{
Debug.println(" Applying JTI ... ... ");
mutant_op = new JTI( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "JID"))
{
Debug.println(" Applying JID ... ... ");
mutant_op = new JID( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "OAN"))
{
Debug.println(" Applying OAN ... ... ");
mutant_op = new OAN( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "EOA"))
{
Debug.println(" Applying EOA ... ... ");
mutant_op = new EOA( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "EOC"))
{
Debug.println(" Applying EOC ... ... ");
mutant_op = new EOC( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "EAM"))
{
Debug.println(" Applying EAM ... ... ");
mutant_op = new EAM( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
if (hasOperator(classOp, "EMM"))
{
Debug.println(" Applying EMM ... ... ");
mutant_op = new EMM( file_env, cdecl, comp_unit );
comp_unit.accept(mutant_op);
}
}
catch (ParseTreeException e )
{
System.err.println( "Encountered errors during generating mutants." );
e.printStackTrace();
}
}
}
}
/**
* Compile class mutants into bytecode
*/
public void compileMutants()
{
if (classOp != null && classOp.length > 0)
{
Debug.println("* Compiling class mutants into bytecode");
MutationSystem.MUTANT_PATH =
MutationSystem.CLASS_MUTANT_PATH;
super.compileMutants();
}
}
}
Compile Test Case
package mujava;
import mujava.MutationSystem;
import mujava.util.Debug;
import mujava.util.ExtensionFilter;
import com.sun.tools.javac.Main;
import java.io.*;
/**
* <p>Description: </p>
* <p>Copyright: Copyright (c) 2005 by Yu-Seung Ma, ALL RIGHTS
RESERVED </p>
* @author Yu-Seung Ma
* @version 1.0
*/
public class compileTestcase
{
public static void main(String[] args)
{
Debug.setDebugLevel(3);
File f = new File(MutationSystem.TESTSET_PATH);
String[] s = f.list(new ExtensionFilter("java"));
String[] pars = new String[2+s.length];
pars[0] = "-classpath";
pars[1] = MutationSystem.CLASS_PATH;
for (int i=0; i<s.length; i++)
{
pars[i+2] = MutationSystem.TESTSET_PATH + "/" + s[i];
}
try
{
// result = 0 : SUCCESS, result = 1 : FALSE
int result = Main.compile(pars,new PrintWriter(System.out));
if (result == 0)
{
Debug.println("Compile Finished");
}
} catch (Exception e)
{
e.printStackTrace();
}
}
}
Exception Mutant Generator
package mujava;
import openjava.ptree.*;
import java.io.*;
import mujava.op.exception.*;
import mujava.op.util.*;
import mujava.util.Debug;
/**
* <p>Generate exception-related mutants
* </p>
* <p>Copyright: Copyright (c) 2005 by Yu-Seung Ma, ALL RIGHTS
RESERVED </p>
* @author Yu-Seung Ma
* @version 1.0
*/
public class ExceptionMutantsGenerator extends MutantsGenerator
{
String[] exceptionOp;
public ExceptionMutantsGenerator(File f)
{
super(f);
exceptionOp = MutationSystem.em_operators;
}
public ExceptionMutantsGenerator(File f, boolean debug)
{
super(f, debug);
exceptionOp = MutationSystem.em_operators;
}
public ExceptionMutantsGenerator(File f, String[] eOP)
{
super(f);
exceptionOp = eOP;
}
void genMutants()
{
if (comp_unit == null)
{
System.err.println(original_file + " is skipped.");
}
ClassDeclarationList cdecls = comp_unit.getClassDeclarations();
if (cdecls == null || cdecls.size() == 0)
return;
if (exceptionOp != null && exceptionOp.length > 0)
{
MutationSystem.clearPreviousMutants();
MutationSystem.MUTANT_PATH =
MutationSystem.EXCEPTION_MUTANT_PATH;
CodeChangeLog.openLogFile();
genExceptionMutants(cdecls);
CodeChangeLog.closeLogFile();
}
}
/**
* Compile exception-related mutants into bytecode
*/
public void compileMutants()
{
if (exceptionOp != null && exceptionOp.length > 0)
{
Debug.println("* Compiling exception-related mutants into bytecode");
MutationSystem.MUTANT_PATH =
MutationSystem.EXCEPTION_MUTANT_PATH;
super.compileMutants();
}
}
void genExceptionMutants(ClassDeclarationList cdecls)
{
for (int j=0; j<cdecls.size(); ++j)
{
ClassDeclaration cdecl = cdecls.get(j);
if (cdecl.getName().equals(MutationSystem.CLASS_NAME))
{
try
{
mujava.op.util.Mutator mutant_op;
if (hasOperator(exceptionOp, "EFD"))
{
Debug.println(" Applying EFD ... ... ");
mutant_op = new EFD(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(exceptionOp, "EHC"))
{
Debug.println(" Applying EHC ... ... ");
mutant_op = new EHC(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(exceptionOp, "EHD"))
{
Debug.println(" Applying EHD ... ... ");
mutant_op = new EHD(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(exceptionOp, "EHI"))
{
Debug.println(" Applying EHI ... ... ");
mutant_op = new EHI(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(exceptionOp, "ETC"))
{
Debug.println(" Applying ETC ... ... ");
mutant_op = new ETC(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator(exceptionOp, "ETD"))
{
Debug.println(" Applying ETD ... ... ");
mutant_op = new ETD(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
} catch (ParseTreeException e)
{
System.err.println( "Exception, during generating traditional mutants for
the class "
+ MutationSystem.CLASS_NAME);
e.printStackTrace();
}
}
}
}
}
Make MUJAVA Structure
package mujava;
import java.io.*;
/**
* <p>Description: </p>
* <p>Copyright: Copyright (c) 2005 by Yu-Seung Ma, ALL RIGHTS
RESERVED </p>
* @author Yu-Seung Ma
* @version 1.0
*/
public class makeMuJavaStructure {
public static void main(String[] args) {
MutationSystem.setJMutationStructure();
makeDir(new File(MutationSystem.SYSTEM_HOME));
makeDir(new File(MutationSystem.SRC_PATH));
makeDir(new File(MutationSystem.CLASS_PATH));
makeDir(new File(MutationSystem.MUTANT_HOME));
makeDir(new File(MutationSystem.TESTSET_PATH));
}
static void makeDir(File dir){
System.out.println("\nMake " + dir.getAbsolutePath() + " directory...");
boolean newly_made = dir.mkdir();
if(!newly_made){
System.out.println(dir.getAbsolutePath() + " directory exists already.");
}else{
System.out.println("Making " + dir.getAbsolutePath() + " directory " +
" ...done.");
}
}
}
Mutants Generator
package mujava;
import openjava.mop.*;
import openjava.ptree.*;
import openjava.tools.parser.*;
import openjava.ptree.util.*;
import java.io.*;
import java.util.*;
import java.lang.reflect.Constructor;
import mujava.op.util.*;
import mujava.util.*;
import com.sun.tools.javac.Main;
/**
* <p>Generate mutants according to selected mutation
* operator(s) from gui.GenMutantsMain.
* The original version is loaded, mutated, and compiled.
* Outputs (mutated source and class files) are in the
* -mutants folder. </p>
* <p>Copyright: Copyright (c) 2005 by Yu-Seung Ma, ALL RIGHTS
RESERVED </p>
* @author Yu-Seung Ma
* @version 1.0
*/
public abstract class MutantsGenerator
{
//private boolean debug = false;
/** Java source file where mutation operators are applied to */
File original_file; // mutationÀ» Àû¿ëÇÒ file
/** mutation operators to apply */
String[] operators = null;
FileEnvironment file_env = null;
CompilationUnit comp_unit = null;
public MutantsGenerator(File f)
{
this.original_file = f;
initPrimitiveTypes();
}
public MutantsGenerator(File f, boolean debug_flag)
{
this(f);
//debug = debug_flag;
}
public MutantsGenerator(File f, String[] operator_list)
{
this(f);
operators = operator_list;
}
public MutantsGenerator(File f, String[] operator_list, boolean debug_flag)
{
this(f, operator_list);
//debug = debug_flag;
}
/**
* Generate and initialize parse tree from the original Java source file.
* Generate mutants. Arrange and compile the original Java source file.
* @return
* @throws OpenJavaException
*/
public boolean makeMutants() throws OpenJavaException
{
Debug.print("-------------------------------------------------------\n");
Debug.print("* Generating parse tree. \n" );
generateParseTree();
Debug.print("..done. \n" );
//System.out.println("0");
Debug.print("* Initializing parse tree. \n" );
initParseTree();
Debug.print("..done. \n" );
//System.out.println("1");
Debug.print("* Generating Mutants \n" );
genMutants();
Debug.print("..done.\n" );
//System.out.println("2");
Debug.print("* Arranging original soure code. \n" );
arrangeOriginal();
//System.out.println("3");
compileOriginal();
Debug.print("..done. \n" );
Debug.flush();
return true;
}
abstract void genMutants();
/*void generateMutant(OJClass mutant_op){
try {
mutant_op.translateDefinition(comp_unit);
}catch (Exception ex){
System.err.println("fail to translate " +mutant_op.getName()+" : " + ex);
ex.printStackTrace();
}
}*/
/**
* Generate mutants from Java bytecode
*/
void generateMutant(DeclAnalyzer mutant_op)
{
try
{
mutant_op.translateDefinition(comp_unit);
}
catch (Exception ex)
{
System.err.println("fail to translate " + mutant_op.getName() + " : " + ex);
ex.printStackTrace();
}
}
/**
* Arrange the original source file into an appropriate directory
*/
private void arrangeOriginal()
{
if (comp_unit == null)
{
System.err.println(original_file + " is skipped.");
}
ClassDeclarationList cdecls = comp_unit.getClassDeclarations();
for (int j=0; j<cdecls.size(); ++j)
{
ClassDeclaration cdecl = cdecls.get(j);
File outfile = null;
try
{
outfile = new File(MutationSystem.ORIGINAL_PATH,
MutationSystem.CLASS_NAME + ".java");
FileWriter fout = new FileWriter( outfile );
PrintWriter out = new PrintWriter( fout );
MutantCodeWriter writer = new MutantCodeWriter( out );
writer.setClassName(cdecl.getName());
comp_unit.accept( writer );
out.flush();
out.close();
}
catch ( IOException e )
{
System.err.println( "fails to create " + outfile );
}
catch ( ParseTreeException e )
{
System.err.println( "errors during printing " + outfile );
e.printStackTrace();
}
}
}
/**
* Initialize parse tree
* @throws OpenJavaException
*/
private void initParseTree() throws OpenJavaException
{
try
{
//System.out.println("OJSystem.env0 :" + OJSystem.env );
comp_unit.accept(new TypeNameQualifier (file_env));
//System.out.println("OJSystem.env1 :" + OJSystem.env );
MemberAccessCorrector corrector = new
MemberAccessCorrector(file_env);
// System.out.println("OJSystem.env2 :" + OJSystem.env );
comp_unit.accept(corrector);
//System.out.println("OJSystem.env3 :" + OJSystem.env );
}
catch (ParseTreeException e)
{
throw new OpenJavaException("can't initialize parse tree");
}
}
/**
* Initialize parse tree
* @param parent_comp_unit
* @param parent_file_env
*/
void initParseTree(CompilationUnit[] parent_comp_unit,FileEnvironment[]
parent_file_env)
{
try
{
parent_comp_unit[0].accept(new TypeNameQualifier(parent_file_env[0]));
MemberAccessCorrector corrector = new
MemberAccessCorrector(parent_file_env[0]);
parent_comp_unit[0].accept(corrector);
}
catch (ParseTreeException e)
{
System.err.println("Encountered errors during analysis.");
e.printStackTrace();
}
}
/**
* Generate parse tree
* @throws OpenJavaException
*/
private void generateParseTree() throws OpenJavaException
{
try
{
comp_unit = parse(original_file);
String pubcls_name = getMainClassName(file_env, comp_unit);
if (pubcls_name == null)
{
int len = original_file.getName().length();
pubcls_name = original_file.getName().substring(0, len-6);
}
file_env = new FileEnvironment(OJSystem.env, comp_unit, pubcls_name);
ClassDeclarationList typedecls = comp_unit.getClassDeclarations();
for (int j = 0; j < typedecls.size(); ++j)
{
ClassDeclaration class_decl = typedecls.get(j);
OJClass c = makeOJClass(file_env, class_decl);
OJSystem.env.record(c.getName(), c);
recordInnerClasses(c);
}
}
catch (OpenJavaException e1)
{
throw e1;
}
catch (Exception e)
{
System.err.println("errors during parsing. " + e);
System.out.println(e);
e.printStackTrace();
}
}
/**
* Evaluate whether a parse tree is successfully generated
* @param f
* @param comp_unit
* @param file_env
* @return
*/
boolean generateParseTree(File f, CompilationUnit[] comp_unit,
FileEnvironment[] file_env)
{
try
{
comp_unit[0] = parse(f);
String pubcls_name = getMainClassName(file_env[0], comp_unit[0]);
if (pubcls_name == null)
{
int len = f.getName().length();
pubcls_name = f.getName().substring(0, len-6);
}
file_env[0] = new FileEnvironment(OJSystem.env, comp_unit[0],
pubcls_name);
ClassDeclarationList typedecls = comp_unit[0].getClassDeclarations();
for (int j=0; j<typedecls.size(); ++j)
{
ClassDeclaration class_decl = typedecls.get(j);
if ( class_decl.getName().equals(MutationSystem.CLASS_NAME) )
{
if ( class_decl.isInterface() ||
class_decl.getModifiers().contains(ModifierList.ABSTRACT) )
{
return false;
}
}
OJClass c = makeOJClass(file_env[0], class_decl);
OJSystem.env.record(c.getName(), c);
recordInnerClasses(c);
}
}
catch (Exception e)
{
System.err.println("errors during parsing. " + e);
e.printStackTrace();
return false;
}
return true;
}
/**
* Record inner-classes
* @param c
*/
private static void recordInnerClasses( OJClass c )
{
OJClass[] inners = c.getDeclaredClasses();
for (int i = 0; i < inners.length; ++i)
{
OJSystem.env.record( inners[i].getName(), inners[i] );
recordInnerClasses( inners[i] );
}
}
/** -> to move to OJClass.forParseTree() **/
private OJClass makeOJClass( Environment env, ClassDeclaration cdecl )
{
OJClass result;
String qname = env.toQualifiedName( cdecl.getName() );
Class meta = OJSystem.getMetabind( qname );
try
{
Constructor constr = meta.getConstructor( new Class[]{
Environment . class,OJClass . class, ClassDeclaration . class } );
Object[] args = new Object[]{env, null, cdecl};
result = (OJClass) constr.newInstance(args);
}
catch (Exception ex)
{
System.err.println("errors during gererating a metaobject for " + qname);
ex.printStackTrace();
result = new OJClass( env, null, cdecl );
}
return result;
}
/**
* Prepare a compilation unit
* @param file
* @return
* @throws OpenJavaException
*/
private static CompilationUnit parse( File file ) throws OpenJavaException
{
Parser parser;
try
{
parser = new Parser(new java.io.FileInputStream( file ) );
}
catch ( java.io.FileNotFoundException e )
{
System.err.println( "File " + file + " not found." );
return null;
}
CompilationUnit result;
try
{
System.out.println( "File " + file );
result = parser.CompilationUnit( OJSystem.env );
}
catch (ParseException e)
{
throw new OpenJavaException(" can't generate parse tree");
}
catch (Exception e)
{
e.printStackTrace();
result = null;
}
return result;
}
/**
*
* @param env
* @param comp_unit
* @return
* @throws ParseTreeException
*/
private static String getMainClassName(FileEnvironment env,
CompilationUnit comp_unit) throws ParseTreeException
{
ClassDeclaration cd = comp_unit.getPublicClass();
if (cd != null)
{
return cd.getName();
}
else
return null;
}
/**
* Compile mutants
*/
public void compileMutants()
{
File f = new File(MutationSystem.MUTANT_PATH);
String[] s = f.list(new MutantDirFilter());
for (int i=0; i<s.length; i++)
{
File target_dir = new File(MutationSystem.MUTANT_PATH + "/" + s[i]);
String[] target_file = target_dir.list(new ExtensionFilter("java"));
Vector v = new Vector();
for (int j=0; j<target_file.length; j++)
{
v.add(MutationSystem.MUTANT_PATH + "/" + s[i] + "/" +
target_file[j]);
}
String[] pars = new String[v.size()+2];
pars[0] = "-classpath";
pars[1] = MutationSystem.CLASS_PATH;
for (int j=0; j<v.size(); j++)
{
pars[2+j] = v.get(j).toString();
}
try
{
// result = 0 : SUCCESS, result = 1 : FALSE
//int result = Main.compile(pars,new PrintWriter(new
FileOutputStream("temp")));
int result = Main.compile(pars);
if (result == 0)
{
Debug.print("+" + s[i] + " ");
}
else
{
Debug.print("-" + s[i] + " ");
// delete directory
File dir_name = new File(MutationSystem.MUTANT_PATH + "/" +
s[i]);
File[] mutants = dir_name.listFiles();
boolean tr = false;
for (int j=0; j<mutants.length; j++)
{
// [tricky solution] It can produce loop -_-;;
while (!tr)
{
tr = mutants[j].delete();
}
tr = false;
}
while (!tr)
{
tr = dir_name.delete();
}
}
}
catch (Exception e)
{
System.err.println(e);
}
}
Debug.println();
}
/**
* Compile original java source file
*/
private void compileOriginal()
{
String[] pars= { "-classpath",
MutationSystem.CLASS_PATH,
MutationSystem.ORIGINAL_PATH + "/" +
MutationSystem.CLASS_NAME + ".java"};
try
{
// result = 0 : SUCCESS, result = 1 : FALSE
//int result = Main.compile(pars,new PrintWriter(new
FileOutputStream("temp")));
Main.compile(pars);
}
catch (Exception e)
{
System.err.println(e);
}
}
private static void initPrimitiveTypes()
{
OJSystem.initConstants();
}
/**
* Determine whether a string contain a certain operator
* @param list
* @param item
* @return true if a string contain the operator, false otherwise
*/
protected boolean hasOperator (String[] list, String item)
{
for (int i=0; i<list.length; i++)
{
if (list[i].equals(item))
return true;
}
return false;
}
}
Test Executer
package mujava;
import java.io.*;
import java.lang.annotation.Annotation;
import java.lang.reflect.*;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import java.util.Vector;
import mujava.test.*;
import mujava.util.*;
import org.junit.*;
import org.junit.internal.RealSystem;
import org.junit.runner.JUnitCore;
import org.junit.runner.Result;
import org.junit.runner.notification.Failure;
import org.junit.runners.*;
/**
* <p>Description: </p>
* <p>Copyright: Copyright (c) 2005 by Yu-Seung Ma, ALL RIGHTS
RESERVED </p>
* @author Yu-Seung Ma
* @update by Nan Li May 2012 integration with JUnit
* @version 1.0
*/
public class TestExecuter {
Object lockObject = new Object();
//int TIMEOUT = 3000;
int TIMEOUT;
final int MAX_TRY = 100;
Class original_executer;
Object original_obj; // instancitation of the test set class
volatile Object mutant_result;
Class mutant_executer; // test set class for a mutant
volatile Object mutant_obj; // test set object for a mutant
Method[] testCases;
volatile Method testcase;
String whole_class_name;
String testSet;
boolean mutantRunning = true;
//original test results
Map<String, String> originalResults = new HashMap<String, String>();
//results for mutants
Map<String, String> mutantResults = null;
//JUnit test cases
List<String> junitTests = new ArrayList<String>();
//result of a test case
Result result = null;
//results as to how many mutants are killed by each test
Map<String, String> finalTestResults = new HashMap<String, String>();
//results as to how many tests can kill each single mutant
Map<String, String> finalMutantResults = new HashMap<String, String>();
public TestExecuter(String targetClassName) {
int index = targetClassName.lastIndexOf(".");
if(index<0){
MutationSystem.CLASS_NAME = targetClassName;
}else{
MutationSystem.CLASS_NAME =
targetClassName.substring(index+1,targetClassName.length());
}
MutationSystem.DIR_NAME = targetClassName;
MutationSystem.CLASS_MUTANT_PATH =
MutationSystem.MUTANT_HOME+"/"+targetClassName
+"/"+MutationSystem.CM_DIR_NAME;
MutationSystem.TRADITIONAL_MUTANT_PATH =
MutationSystem.MUTANT_HOME+"/"+targetClassName
+"/"+MutationSystem.TM_DIR_NAME;
MutationSystem.EXCEPTION_MUTANT_PATH =
MutationSystem.MUTANT_HOME+"/"+targetClassName
+"/"+MutationSystem.EM_DIR_NAME;
whole_class_name = targetClassName;
}
public void setTimeOut(int msecs){
TIMEOUT = msecs;
}
public boolean readTestSet(String testSetName){
try{
testSet = testSetName;
// Class loader for the original class
OriginalLoader myLoader = new OriginalLoader();
System.out.println(testSet);
original_executer = myLoader.loadTestClass(testSet);
original_obj = original_executer.newInstance(); // initialization of the test
set class
if(original_obj == null){
System.out.println("Can't instantiace original object");
return false;
}
// read testcases from the test set class
testCases = original_executer.getDeclaredMethods();
if(testCases==null){
System.out.println(" No test case exist ");
return false;
}
}catch(Exception e){
System.err.println(e);
return false;
}
return true;
}
boolean sameResult(Object result1,Object result2){
if( !(result1.toString().equals(result2.toString())) ) return false;
return true;
}
public TestResult runClassMutants() throws
NoMutantException,NoMutantDirException{
MutationSystem.MUTANT_PATH =
MutationSystem.CLASS_MUTANT_PATH;
TestResult test_result = new TestResult();
runMutants(test_result, "");
return test_result;
}
public TestResult runExceptionMutants() throws
NoMutantException,NoMutantDirException{
MutationSystem.MUTANT_PATH =
MutationSystem.EXCEPTION_MUTANT_PATH;
TestResult test_result = new TestResult();
runMutants(test_result, "");
return test_result;
}
public TestResult runTraditionalMutants(String methodSignature) throws
NoMutantException,NoMutantDirException{
MutationSystem.MUTANT_PATH =
MutationSystem.TRADITIONAL_MUTANT_PATH;
String original_mutant_path = MutationSystem.MUTANT_PATH;
TestResult test_result = new TestResult();
if(methodSignature.equals("All method")){
try{
//setMutantPath();
//computeOriginalTestResults();
File f = new File(MutationSystem.TRADITIONAL_MUTANT_PATH,
"method_list");
FileReader r = new FileReader(f);
BufferedReader reader = new BufferedReader(r);
String readSignature = reader.readLine();
while(readSignature != null){
MutationSystem.MUTANT_PATH = original_mutant_path + "/" +
readSignature;
try{
runMutants(test_result, readSignature);
}catch(NoMutantException e){
}
readSignature = reader.readLine();
}
reader.close();
}catch(Exception e){
System.err.println("Error in update() in
TraditioanlMutantsViewerPanel.java");
}
}else{
MutationSystem.MUTANT_PATH = original_mutant_path + "/" +
methodSignature;
runMutants(test_result, methodSignature);
}
return test_result;
}
/**
* get the result of the test under the mutanted program
* @deprecated
* @param mutant
* @param testcase
* @throws InterruptedException
*/
void runMutants(Object mutant,Method testcase) throws InterruptedException{
mutantRunning = true;
try{
// testcase execution
mutant_result = testcase.invoke(mutant_obj,null);
}catch(Exception e){
// execption occurred -> abnormal execution
mutant_result = e.getCause().getClass().getName()+" : "
+e.getCause().getMessage();
}
mutantRunning = false;
synchronized(lockObject){
lockObject.notify();
}
//throw new InterruptedException();
}
synchronized void waitUntilAtLeast(long timeOut) throws
InterruptedException{
wait(timeOut);
}
/**
* get the mutants for one method based on the method signature
* @param methodSignature
* @return
* @throws NoMutantDirException
* @throws NoMutantException
*/
private String[] getMutants (String methodSignature) throws
NoMutantDirException, NoMutantException{
// Read mutants
//System.out.println("mutant_path: " +
MutationSystem.MUTANT_PATH);
File f = new File(MutationSystem.MUTANT_PATH);
if(!f.exists()){
System.err.println(" There is no directory for the mutants of " +
MutationSystem.CLASS_NAME);
System.err.println(" Please generate mutants for " +
MutationSystem.CLASS_NAME);
throw new NoMutantDirException();
}
// mutantDirectories match the names of mutants
String[] mutantDirectories = f.list(new MutantDirFilter());
if(mutantDirectories == null || mutantDirectories.length == 0){
if(!methodSignature.equals(""))
System.err.println(" No mutants have been generated for the
method " + methodSignature + " of the class" +
MutationSystem.CLASS_NAME);
else
System.err.println(" No mutants have been generated for the
class " + MutationSystem.CLASS_NAME);
// System.err.println(" Please check if zero mutant is correct.");
// throw new NoMutantException();
}
return mutantDirectories;
}
/**
* compute the result of a test under the original program
*/
public void computeOriginalTestResults(){
Debug.println("\n\
n======================================== Generating Original
Test Results ========================================");
try{
//initialize the original results to "pass"
//later the results of the failed test cases will be updated
for(int k = 0;k < testCases.length;k++){
Annotation[] annotations = testCases[k].getDeclaredAnnotations();
for(Annotation annotation : annotations)
{
//System.out.println("name: " + testCases[k].getName() +
annotation.toString() + annotation.toString().indexOf("@org.junit.Test"));
if(annotation.toString().indexOf("@org.junit.Test") != -1){
//killed_mutants[k]= ""; // At first, no mutants are killed by
each test case
originalResults.put(testCases[k].getName(), "pass");
junitTests.add(testCases[k].getName());
finalTestResults.put(testCases[k].getName(), "");
continue;
}
}
}
JUnitCore jCore = new JUnitCore();
//result = jCore.runMain(new RealSystem(), "VMTEST1");
result = jCore.run(original_executer);
//get the failure report and update the original result of the test with the
failures
List<Failure> listOfFailure = result.getFailures();
for(Failure failure: listOfFailure){
String nameOfTest = failure.getTestHeader().substring(0,
failure.getTestHeader().indexOf("("));
String testSourceName = testSet + "." + nameOfTest;
//System.out.println("failure message: " + failure.getMessage()
+ failure.getMessage().equals(""));
String[] sb = failure.getTrace().split("\\n");
String lineNumber = "";
for(int i = 0; i < sb.length;i++){
if(sb[i].indexOf(testSourceName) != -1){
lineNumber = sb[i].substring(sb[i].indexOf(":") +
1, sb[i].indexOf(")"));
}
}
//put the failure messages into the test results
if(failure.getMessage() == null)
originalResults.put(nameOfTest, nameOfTest + ": " +
lineNumber + "; " + "fail");
else{
if(failure.getMessage().equals(""))
originalResults.put(nameOfTest, nameOfTest + ":
" + lineNumber + "; " + "fail");
else
originalResults.put(nameOfTest, nameOfTest + ":
" + lineNumber + "; " + failure.getMessage());
}
}
System.out.println(originalResults.toString());
// System.out.println(System.getProperty("user.dir"));
// System.out.println(System.getProperty("java.class.path"));
// System.out.println(System.getProperty("java.library.path"));
}
catch(Exception e){
System.out.println(e.getMessage());
e.printStackTrace();
// original_results[k] = e.getCause().getClass().getName()+" : "
+e.getCause().getMessage();
// Debug.println("Result for " + testName + " : " +original_results[k] );
// Debug.println(" [warining] " + testName + " generate exception as a result "
);
// ----------------------------------
}finally{
//originalResultFileRead();
}
}
private TestResult runMutants(TestResult tr, String methodSignature) throws
NoMutantException,NoMutantDirException{
try{
String[] mutantDirectories = getMutants(methodSignature);
int mutant_num = mutantDirectories.length;
tr.setMutants();
for(int i = 0;i < mutant_num;i++){
// set live mutnats
tr.mutants.add(mutantDirectories[i]);
}
// result againg original class for each test case
// Object[] original_results = new Object[testCases.length];
// list of the names of killed mutants with each test case
//String[] killed_mutants = new String[testCases.length];
Debug.println("\n\n========================================
Executing Mutants ========================================");
for(int i = 0; i < tr.mutants.size(); i++){
// read the information for the "i"th live mutant
String mutant_name = tr.mutants.get(i).toString();
finalMutantResults.put(mutant_name, "");
JMutationLoader mutantLoader = new JMutationLoader(mutant_name);
//mutantLoader.loadMutant();
mutant_executer = mutantLoader.loadTestClass(testSet);
mutant_obj = mutant_executer.newInstance();
Debug.print(" " + mutant_name);
try{
// Mutants are runned using Thread to detect infinite loop caused by mutation
Runnable r = new Runnable(){
public void run(){
try{
mutantRunning = true;
//original test results
mutantResults = new HashMap<String, String>();
for(int k = 0;k < testCases.length;k++){
Annotation[] annotations = testCases[k].getDeclaredAnnotations();
for(Annotation annotation : annotations)
{
//System.out.println("name: " + testCases[k].getName()
+ annotation.toString() + annotation.toString().indexOf("@org.junit.Test"));
if(annotation.toString().indexOf("@org.junit.Test") != -
1){
//killed_mutants[k]= ""; // At first, no mutants are
killed by each test case
mutantResults.put(testCases[k].getName(),
"pass");
continue;
}
}
}
JUnitCore jCore = new JUnitCore();
result = jCore.run(mutant_executer);
List<Failure> listOfFailure = result.getFailures();
for(Failure failure: listOfFailure){
String nameOfTest =
failure.getTestHeader().substring(0, failure.getTestHeader().indexOf("("));
String testSourceName = testSet + "." + nameOfTest;
//System.out.println(testSourceName);
String[] sb = failure.getTrace().split("\\n");
String lineNumber = "";
for(int i = 0; i < sb.length;i++){
//System.out.println("sb-trace: " + sb[i]);
if(sb[i].indexOf(testSourceName) != -1){
lineNumber =
sb[i].substring(sb[i].indexOf(":") + 1, sb[i].indexOf(")"));
}
}
//get the line where the error happens
/*
String tempLineNumber = "";
if(failure.getTrace().indexOf(testSourceName) != -1){
tempLineNumber =
failure.getTrace().substring(failure.getTrace().indexOf(testSourceName) +
testSourceName.length() + 1, failure.getTrace().indexOf(testSourceName) +
testSourceName.length() + 5);
System.out.println("tempLineNumber: " +
tempLineNumber);
lineNumber = tempLineNumber.substring(0,
tempLineNumber.indexOf(")"));
//System.out.print("LineNumber: " +
lineNumber);
} */
//get the test name that has the error and save the failure
info to the results for mutants
if(failure.getMessage() == null)
mutantResults.put(nameOfTest, nameOfTest + ":
" + lineNumber + "; " + "fail");
else if(failure.getMessage().equals(""))
mutantResults.put(nameOfTest, nameOfTest + ":
" + lineNumber + "; " + "fail");
else
mutantResults.put(nameOfTest, nameOfTest + ":
" + lineNumber + "; " + failure.getMessage());
}
System.out.println(mutantResults.toString());
mutantRunning = false;
synchronized(lockObject){
lockObject.notify();
}
}catch(Exception e){
e.printStackTrace();
//System.out.println("e.getMessage()");
//System.out.println(e.getMessage());
}
}
};
Thread t = new Thread(r);
t.start();
synchronized(lockObject){
lockObject.wait(TIMEOUT); // Check out if a mutant is in infinite loop
}
if(mutantRunning){
//System.out.println("check point4");
t.interrupt();
mutant_result = "time_out: more than " + TIMEOUT + " seconds";
//mutantResults.put(nameOfTest, nameOfTest + ": " + lineNumber + "; " +
failure.getMessage());
}
}catch(Exception e){
mutant_result = e.getCause().getClass().getName()+" : "
+e.getCause().getMessage();
}
//determine whether a mutant is killed or not
//update the test report
boolean sign = false;
for(int k = 0;k < junitTests.size();k++){
String name = junitTests.get(k);
if(!mutantResults.get(name).equals(originalResults.get(name))){
sign = true;
//update the final results by tests
if(finalTestResults.get(name).equals(""))
finalTestResults.put(name, mutant_name);
else
finalTestResults.put(name, finalTestResults.get(name) + ", "
+ mutant_name);
//update the final results by mutants
if(finalMutantResults.get(mutant_name).equals(""))
finalMutantResults.put(mutant_name, name);
else
finalMutantResults.put(mutant_name,
finalMutantResults.get(mutant_name) + ", " + name);
}
}
if(sign == true)
tr.killed_mutants.add(mutant_name);
else
tr.live_mutants.add(mutant_name);
mutantLoader = null;
mutant_executer = null;
System.gc();
}
for(int i = 0;i < tr.killed_mutants.size();i++){
tr.live_mutants.remove(tr.killed_mutants.get(i));
}
/*
System.out.println(" Analysis of testcases ");
for(int i = 0;i < killed_mutants.length;i++){
System.out.println(" test " + (i+1) + " kill ==> " + killed_mutants[i]);
}
*/
}catch(NoMutantException e1){
throw e1;
}catch(NoMutantDirException e2){
throw e2;
}
/*catch(ClassNotFoundException e3){
System.err.println("[Execution 1] " + e3);
return null;
}
*/catch(Exception e){
System.err.println("[Exception 2]" + e);
return null;
}
System.out.println("test report: " + finalTestResults);
System.out.println("mutant report: " + finalMutantResults);
return tr;
}
void erase_killed_mutants(Vector v){
System.out.println("Deleting directories of killed mutants");
for(int i=0;i<v.size();i++){
System.out.print(v.get(i).toString()+" ");
erase_directory(v.get(i).toString());
}
}
void erase_directory(String mutant_name){
File mutant_dir = new
File(MutationSystem.MUTANT_PATH+"/"+mutant_name);
File[] f = mutant_dir.listFiles();
boolean flag = false;
for(int i=0;i<f.length;i++){
while(!flag){
flag = f[i].delete();
}
flag = false;
}
while(!flag){
flag = mutant_dir.delete();
}
}
}
Traditional Mutants Generator
package mujava;
import openjava.ptree.*;
import java.io.*;
import mujava.op.basic.*;
import mujava.op.util.*;
import mujava.util.Debug;
/**
* <p>Generate traditional mutants according to selected
* operator(s) from gui.GenMutantsMain.
* The original version is loaded, mutated, and compiled.
* Outputs (mutated source and class files) are in
* the traditional-mutants folder. </p>
*
* <p>Currently available traditional mutation operators:
* (1) AORB: Arithmetic Operator Replacement (Binary),
* (2) AORU: Arithmetic Operator Replacement (Unary),
* (3) AORS: Arithmetic Operator Replacement (Short-cut),
* (4) AODU: Arithmetic Operator Deletion (Unary),
* (5) AODS: Arithmetic Operator Deletion (Short-cut),
* (6) AOIU: Arithmetic Operator Insertion (Unary),
* (7) AOIS: Arithmetic Operator Insertion (Short-cut),
* (8) ROR: Rational Operator Replacement,
* (9) COR: Conditional Operator Replacement,
* (10) COD: Conditional Operator Deletion,
* (11) COI: Conditional Operator Insertion,
* (12) SOR: Shift Operator Replacement,
* (13) LOR: Logical Operator Replacement,
* (14) LOI: Logical Operator Insertion,
* (15) LOD: Logical Operator Deletion,
* (16) ASRS: Assignment Operator Replacement (short-cut)
* </p>
* <p>Copyright: Copyright (c) 2005 by Yu-Seung Ma, ALL RIGHTS
RESERVED </p>
* @author Yu-Seung Ma
* @version 1.0
*/
/* upsorn: 11/08/2009 - need to work on nesting
* (17) SID: If-Statement Deletion,
* (18) SFD: For-Statement Deletion,
* (19) SSD: Switch-Statement Deletion,
* (20) SWD: While-Statement Deletion
*/
public class TraditionalMutantsGenerator extends MutantsGenerator
{
String[] traditionalOp;
public TraditionalMutantsGenerator(File f)
{
super(f);
traditionalOp = MutationSystem.tm_operators;
}
public TraditionalMutantsGenerator(File f, boolean debug)
{
super (f, debug);
traditionalOp = MutationSystem.tm_operators;
}
public TraditionalMutantsGenerator(File f, String[] tOP)
{
super(f);
traditionalOp = tOP;
}
/**
* Verify if the target Java source and class files exist,
* generate traditional mutants
*/
void genMutants()
{
if (comp_unit == null)
{
System.err.println (original_file + " is skipped.");
}
ClassDeclarationList cdecls = comp_unit.getClassDeclarations();
if (cdecls == null || cdecls.size() == 0)
return;
if (traditionalOp != null && traditionalOp.length > 0)
{
Debug.println("* Generating traditional mutants");
MutationSystem.clearPreviousTraditionalMutants();
MutationSystem.MUTANT_PATH =
MutationSystem.TRADITIONAL_MUTANT_PATH;
CodeChangeLog.openLogFile();
genTraditionalMutants(cdecls);
CodeChangeLog.closeLogFile();
}
}
/**
* Compile traditional mutants into bytecode
*/
public void compileMutants()
{
if (traditionalOp != null && traditionalOp.length > 0)
{
try
{
Debug.println("* Compiling traditional mutants into bytecode");
String original_tm_path =
MutationSystem.TRADITIONAL_MUTANT_PATH;
File f = new File(original_tm_path, "method_list");
FileReader r = new FileReader(f);
BufferedReader reader = new BufferedReader(r);
String str = reader.readLine();
while (str != null)
{
MutationSystem.MUTANT_PATH = original_tm_path + "/" + str;
super.compileMutants();
str = reader.readLine();
}
reader.close();
MutationSystem.MUTANT_PATH = original_tm_path;
} catch (Exception e)
{
e.printStackTrace();
System.err.println("Error at compileMutants() in
TraditionalMutantsGenerator.java");
}
}
}
/**
* Apply selected traditional mutation operators:
* AORB, AORS, AODU, AODS, AOIU, AOIS, ROR, COR, COD, COI,
* SOR, LOR, LOI, LOD, ASRS, SID, SWD, SFD, SSD
* @param cdecls
*/
void genTraditionalMutants(ClassDeclarationList cdecls)
{
for (int j=0; j<cdecls.size(); ++j)
{
ClassDeclaration cdecl = cdecls.get(j);
//take care of the case for generics
String tempName = cdecl.getName();
if(tempName.indexOf("<") != -1 && tempName.indexOf(">")!= -1)
tempName = tempName.substring(0, tempName.indexOf("<")) +
tempName.substring(tempName.lastIndexOf(">") + 1, tempName.length());
if (tempName.equals(MutationSystem.CLASS_NAME))
{
try
{
mujava.op.util.Mutator mutant_op;
boolean AOR_FLAG = false;
try
{
//generate a list of methods from the original java class
//System.out.println("MutationSystem.MUTANT_PATH: " +
MutationSystem.MUTANT_PATH);
File f = new File(MutationSystem.MUTANT_PATH, "method_list");
FileOutputStream fout = new FileOutputStream(f);
PrintWriter out = new PrintWriter(fout);
mutant_op = new CreateDirForEachMethod(file_env, cdecl,
comp_unit, out);
comp_unit.accept(mutant_op);
out.flush();
out.close();
} catch (Exception e)
{
System.err.println("Error in writing method list");
return;
}
if (hasOperator (traditionalOp, "AORB") )
{
Debug.println(" Applying AOR-Binary ... ... ");
AOR_FLAG = true;
mutant_op = new AORB(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator (traditionalOp, "AORS") )
{
Debug.println(" Applying AOR-Short-Cut ... ... ");
AOR_FLAG = true;
mutant_op = new AORS(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator (traditionalOp, "AODU") )
{
Debug.println(" Applying AOD-Normal-Unary ... ... ");
mutant_op = new AODU(file_env, cdecl, comp_unit);
((AODU)mutant_op).setAORflag(AOR_FLAG);
comp_unit.accept(mutant_op);
}
if (hasOperator (traditionalOp, "AODS") )
{
Debug.println(" Applying AOD-Short-Cut ... ... ");
mutant_op = new AODS(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator (traditionalOp, "AOIU") )
{
Debug.println(" Applying AOI-Normal-Unary ... ... ");
mutant_op = new AOIU(file_env,cdecl,comp_unit);
((AOIU)mutant_op).setAORflag(AOR_FLAG);
comp_unit.accept(mutant_op);
}
if (hasOperator (traditionalOp, "AOIS") )
{
Debug.println(" Applying AOI-Short-Cut ... ... ");
mutant_op = new AOIS(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator (traditionalOp, "ROR") )
{
Debug.println(" Applying ROR ... ... ");
mutant_op = new ROR(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator (traditionalOp, "COR") )
{
Debug.println(" Applying COR ... ... ");
mutant_op = new COR(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator (traditionalOp, "COD") )
{
Debug.println(" Applying COD ... ... ");
mutant_op = new COD(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator (traditionalOp, "COI") )
{
Debug.println(" Applying COI ... ... ");
mutant_op = new COI(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator (traditionalOp, "SOR") )
{
Debug.println(" Applying SOR ... ... ");
mutant_op = new SOR(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator (traditionalOp, "LOR") )
{
Debug.println(" Applying LOR ... ... ");
mutant_op = new LOR(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator (traditionalOp, "LOI") )
{
Debug.println(" Applying LOI ... ... ");
mutant_op = new LOI(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator (traditionalOp, "LOD") )
{
Debug.println(" Applying LOD ... ... ");
mutant_op = new LOD(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator (traditionalOp, "ASRS") )
{
Debug.println(" Applying ASR-Short-Cut ... ... ");
mutant_op = new ASRS(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
// upsorn: First attempt: statement deletion operator
// if (hasOperator (traditionalOp, "SDL") )
// {
// Debug.println(" Applying SDL ... ... ");
// mutant_op = new SDL(file_env, cdecl, comp_unit);
// comp_unit.accept(mutant_op);
// }
/*
if (hasOperator (traditionalOp, "SID") )
{
Debug.println(" Applying SID ... ... ");
mutant_op = new SID(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
if (hasOperator (traditionalOp, "SWD") )
{
Debug.println(" Applying SWD ... ... ");
mutant_op = new SWD(file_env, cdecl, comp_unit);
comp_unit.accept(mutant_op);
}
*/
} catch (ParseTreeException e)
{
System.err.println( "Exception, during generating traditional mutants for
the class "
+ MutationSystem.CLASS_NAME);
e.printStackTrace();
}
}
}
}
}
REFERENCES
Offutt, J. A. & Untch, R. H. (October 2000). Mutation 2000: Uniting the Orthogonal.
Retrieved from http://cs.gmu.edu/~offutt/rsrch/papers/mut00.pdf
Bakewell, G. (2010).
Mutation-Based Testing Technologies Close the “Quality Gap” in Functional
Verification for Complex Chip Designs. Retrieved January 18, 2011 from
http://electronicdesign.com/article/eda/Mutation-Based-Testing-Technologies-Close-the-
Quality-Gap-in-Functional-Verification-for-Complex-Chip-Designs/4.aspx
Offut, J. A. (June 1995). A Practical System for Mutation Testing: Help for the Common
Programmer. Retrieved from http://cs.gmu.edu/~offutt/rsrch/papers/practical.pdf
Usaola, M. & Mateo, P. (2010). Mutation Testing Cost Reduction Techniques: A
Survey. IEEE Software, 27(3), 80-86. Retrieved from ABI/INFORM Global.
(Document ID: 2012103051).
Alexander, R. T.; Bieman, J. M.; Ghosh, S.; & Ji, B. (2002). Mutation of Java Objects.
Retrieved from
http://www.cs.colostate.edu/~bieman/Pubs/AlexanderBiemanGhoshJiISSRE02.pdf
Nester - Free Software that Helps to do Effective Unit Testing in C#. Retrieved January
23, 2011 from http://nester.sourceforge.net/
Kolawa, Adam (1999). Mutation Testing: A New Approach to Automatic Error-
Detection. Retrieved January 18, 2011 from
http://www.parasoft.com/jsp/products/article.jsp?articleId=291
Eclipse Profiler plugin. http://sourceforge.net/projects/eclipsecolorer.
M. Hafiz. User Manual of Muatation Engine. https://netfiles.uiuc.edu/
mhafiz/www/ResearchandPublications/MutationTestingTool.htm
jclasslib byte-code viewer.EJ-Technologies, http://www.ej-technologies.com/
products/jclasslib/overview.html
J. Offutt. Investigation of the software testing coupling effect. ACM Transactions on
Software Engineering Methodology, vol. 1, pp. 3-18.
J. Offutt, Ammei Lee, G. Rothermel, R. Untch, and C. Zapf. An experimental
determination of sufficient mutation operators. ACM Transaction on Software
Engineering Methodology, 5(2):99-118, April 1996.
http://javabdd.sourceforge.net/
Sourceforge. Metrics, http://metrics.sourceforge.net/. accessed May 2010.
Sourceforge. Jumble. http://jumble.sourceforge.net/, 2007. accessed May 2010.
P. Trinidad, D. Benavides, A. Ruiz-Cortés, S. Segura, and A.Jimenez. Fama framework.
In 12th Software Product Lines Conference (SPLC), 2008.
J. Whaley. Javabdd. http://javabdd.sourceforge.net/. accessed May 2010.