white box testing in practice - qai global...

XPANXION INTERNATIONAL PVT. LTD.

Server Space, A. G. Technology Park, Off ITI Road, Aundh, Pune – 411007

White Box Testing In Practice

STC 2013 Conference

By -

Smita More (email : [email protected] )

Chaitanya Dharmadhikari (email : [email protected] )

White Box Testing In Practice Page | 2

ABSTRACT

White box testing encompasses several testing techniques used to evaluate the logical

functionality of an application, block of code or specific software package. It helps to

improve the quality of the software application by identifying a majority of bugs early in the

software development lifecycle. This in turn reduces the cost of fixing them at later stages.

This paper introduces white box testing along with the details of how it is implemented in

real world projects. This paper will elaborate the challenges in white box integration testing

and BPM unit testing from the practitioner‟s point of view and the approach used to

overcome them.

The paper attempts to describe:

1. Different methods that can be used to add unit and integration test cases (using the

Spring Framework (open source application framework), Maven and JUnit).

2. The guidelines that should be followed while writing unit test cases.

3. Different approaches that aid in writing unit test cases. (We have used the Mockito

framework with JUnit and its best practices as an example).

4. White box testing of BPM processes: With the increase in use of BPM software in

nearly every organization, this paper tries to explain how white-box testing can be

applied to test BPM processes as well.

Our approach has helped us to reduce manual testing efforts and improve the overall

productivity of automating unit and integration test cases.

White Box Testing

Let us first understand what white box testing is before focusing on its numerous benefits.

White box testing is testing based on an analysis of the internal structure of the component

or system. It means testing the minute internal workings of the component or system under

test. For example, opening the engine block of a car to check whether the pistons are

working properly. An example in terms of the software industry is writing tests in order to

execute every statement inside a particular method of a class.

We will explain the following approaches to white box testing along with the benefits of each

in this paper -

1. Unit testing – Testing of individual methods of a given class.

2. Integration testing – Testing end to end flows through multiple classes.

3. Testing BPM processes – Testing of high level business flows.

http://en.wikipedia.org/wiki/Open_source

http://en.wikipedia.org/wiki/Application_framework


1. Unit Testing

Unit testing in its simplest form is testing each part of the system individually. E.g. when we

test the indicators of a vehicle, we aren‟t bothered whether the vehicle has two wheels or

four. We are focused on whether the indicators blink their intensity etc. Similarly in

software, we test the functionality of an individual class by means of a unit test.

A unit test is a test related to a single functionality of a method of a class. The purpose of

unit tests is to verify that the specific unit of code works as expected. The idea behind unit

testing is that we want to test our bit of code regardless of its dependencies. A dependency

is when one class in an application depends upon another in order to perform its intended

function.

A unit test is written using fake replacements of the required dependencies. Mocks are a

type of such fake replacements. Mocking by definition is creating a copy of a real object. In

case of Figure 1.1 below, class A has dependency on classes B & C. So in order to test class

A we provide fake functionalities by mocking classes B & C.

Figure 1.1

Some easily available tools or frameworks like Mockito or PowerMock enable us to create

mocks in a simple and intuitive way, while at the same time providing great control of the

whole process. Mockito artifacts are available in the Maven Central Repository (MCR).

Maven is a commonly used build tool (For more information, please see

http://maven.apache.org/). The easiest way to make MCR available in your project is to put

the following configuration in your dependency manager:

Maven:

<dependency>

<groupId>org.mockito</groupId>

<artifactId>mockito-core</artifactId>

<version>1.9.0</version>

<scope>test</scope>

</dependency>

Using Mockito, we can focus more on the class under test than the dependencies that are

required to carry out the test. This reduces testing efforts as we need not spend time in

Class A

Class B Mock

Class C Mock

http://maven.apache.org/


looking for suitable dependent objects that match our requirements. We can just create

temporary objects of our own as and when required.

Let us now understand how to use mocking in our projects for different types of scenarios.

The project code base is in java and we have used the JUnit test framework for our

examples.

a. Mocking methods that return a value

Consider the following line of code -

VendorOrder vendorOrder = VendorOrderService.findById(vendorOrderId);

Suppose we have a java class named “AvmServiceImpl‟ under test that uses an

object of class “VendorOrder”. Now if this VendorOrder object is retrieved from a

method of a service say “VendorOrderService”, we cannot spend efforts on getting a

real value from this new service. Our focus is on the class under test and not its

dependencies.

We use a mock in this case. One of the basic functions of mocking frameworks is an

ability to return a given value when a specific method is called. It can be done using

Mockito.when() in conjunction with thenReturn(). This process of defining how a

given mock method should behave is called stubbing.

Here, VendorOrderService is an external service used in the class under test. Hence

we need to stub the returned value of the method findById() of VendorOrderService

and return a fake replacement of the real value. This can be stubbed as below:

when(VendorOrderService.findById(vendorOrderId)).thenReturn(vendorOrder);

where this “vendorOrder” object can be created in our JUnit test class.

If you want a method to throw an exception, above method can be stubbed to throw

an exception.

when(VendorOrderService.findById(vendorOrderId)).thenThrow(Exception.class);

Mockito provides a family of functions for requesting such specific behaviors. Few of

them which are used often are

Method Description

thenReturn(T valueToBeReturned) returns given value

thenThrow(Throwable toBeThrown) throws given exception


b. Mocking void methods

As we have seen above, the stubbed method is passed as a parameter to a method

that returns some value. This obviously means that you cannot use this construct for

void methods. Instead, you should use a doSomehing ()..when () construct. See a

few methods below:

Method Description

doThrow(Throwable toBeThrown) Throws given exception

doNothing() Does nothing

c. Argument matching

Mockito, by default, compares arguments using equals methods. Sometimes it‟s

convenient to know exactly what parameter the method will be called with. We can

match exact object values, the types of objects, the types of object lists and so on.

An example of Argument matching is as below:

when(postalAddressService.createPostalAddress(eq(address1), eq(address2),

anyString(), anyString(), anyString(), anyBoolean(), Matchers.anyList(),

Matchers.anyListOf(MasterZip.class))).thenReturn(new PostalAddress());

d. Verifying Behavior

Once created, a mock remembers all operations performed on it. These operations

can easily be verified using Mockito.verify(T mock) on the mocked method. By

default, Mockito checks if a given method (with given arguments) was called exactly

the required number of times. Mockito provides a number of verification modes. It is

also possible to create a custom verification mode.

Name Verifying method was...

times(int wantedNumberOfInvocations) called exactly n times (one by default)

never() never called

atLeastOnce() called at least once


A Simple Mockito test case example is as below:

Method under test –

JUnit test –

Since Unit tests cover every single statement or condition within a given class method, the

test coverage will always be high. Coverage tools like Clover can be used to calculate the

code coverage of tests that are executed. Using Mockito increases the productivity of white

box testers. We were able to test more than 90% of the source code though the client

expectation was only 70% and that too within the initially estimated time frame. Hence, unit

testing overall gives the client more confidence in the system that is being built.

We will now look at the second approach in white box testing i.e. integration testing.


2. Integration Testing

Let us use the car example to understand integration tests. Now that we are sure that each

individual part of the car is working as expected, we need to make sure that these parts

function properly when they are combined. We take the car for a test drive and check

whether the engine is running smoothly; the brakes slow the car down; the indicators work

as designed and so on.

Similarly in software, integration tests are used to test the entire end-to-end functionality.

In most projects we need to obtain data, process it and save the updated data to some

database. Using integration tests we need to verify whether the right data is retrieved, the

right processing is done and that correct results are saved in the database. Hence, our

tests too require a database to reflect changes at the different intermediate levels.

A logical representation of a real world project can be as shown in Figure 2.1 below -

Figure 2.1

Consider an example of a Controller under test. A controller interacts with a Service “B”,

which then gives a call to the data layer (i.e. repository) that interacts with the database.

Though the basic structure of an integration test is similar to a unit test, integration tests

require a call to real services and repositories unlike unit tests using Mocks.

These integration tests can be time consuming as they cover multiple class methods and

require the database to be loaded before execution. One significant improvement in this

regard is to load only the tables necessary for execution of the specific tests instead of

loading the entire database each time. We provided only the main tables necessary for

execution in the tests. The DBUnit Framework picks up the dependency tables

automatically. This reduced the manual efforts needed for identifying the dependency tables

from the entire database. Also, loading only selective tables reduced the overall build time

from „40 minutes‟ to „30 minutes‟. One of the approaches used for white box testing

required insertion of data in the xml files. Approach of inserting this data through White Box

Integration Testing was reduced and the client could gain a profit of “90 days”. So just

imagine in terms of cost gained by the client. I am sure every client would like to adopt

such approach for their projects.


Integration tests check the complete flow through different modules and ensure that the

system as a whole works fine. However, we cannot say that all conditions within a given

class method will be covered. Hence, integration test coverage will rarely be as good as unit

test coverage.

The next challenging area in White Box Testing is testing BPM Processes.

3. Testing BPM Processes

Many Organizations implement BPM (Business Process Management) solutions. BPM

processes can range from simple (having a single flow with a few tasks in it) to very

complex business flows. Those who need to understand the BPM processes in detail can

refer to the References section of this White Paper.

With the complexity of the business flows, there lies a challenge in understanding the flows

and testing them. Like Unit testing, there are different tools available in the market that can

create a unit test for the BPM process. For example, there is a tool called „Activiti‟ by

Alfresco. This tool can be added as a plugin to the Eclipse editor. It provides a designer view

where one can just right click on the process name in their project and add a unit test. This

can be done when you just need to add a unit test for a simple BPM process (no sub-

processes in it).

We faced a challenge in testing BPM processes that had different sub-processes in it. Here

the challenge is to test one process at a time. Before we go into any details about the

approach, we would like to first explain the context in our project. Unit testing of more than

90% of the Java classes was completed and we needed an optimum approach to test the

flow of each process where we verified invocation of the tasks associated with those

processes. There is no defined approach or any standard guideline that can help you in

adding test cases for BPM. We came up with this custom approach to test each of the BPM

processes. We would rather call this a hybrid approach because the approach involves use

of both unit and integration testing. How it is both unit and integration is explained in detail

below.

Let us take a live example used in the project. At this point, do not bother about what each

of the tasks and processes below do. Our intent now is to just understand the high level

flow and the approach to test it.


Figure 3.1

Above diagram represents one of the sub processes named „PfcSubProcess‟ in a main

process called „Vendor Order Main Process‟. The Activiti tool provides an interface to view

the properties associated with each construct (this can be viewed in any BPM tool). Let us

now go through each of the construct properties and understand the flow.

Figure 3.2

As seen in the properties, this above task does not have any associated Java class. Any

script can be run through this task if required. In this case it just prints a message.

Figure 3.3

After the completion of the Script task, the PFC Notification Service Task would be invoked.


Figure 3.4

Figure 3.5

After the PFC Notification Service Task, above two user tasks are „Corrections Required‟ and

„Follow up with Vendor for Fulfillment Corrections‟. Both tasks will be run in parallel.


Listeners would be called after completion of each of the tasks. In a BPM test, user tasks

need to be completed through code, as these tasks are carried out by a user manually

through user interface. We will look into more detail below.

As mentioned earlier, the two processes are sub processes of the main process. Our

objective is to test these sub processes. Each sub process may require input from another

process and give some output at the end which again can be an input to another process.

Let us now understand the technical aspect required for testing the BPM process.

It is a best practice to have a base class with all common methods required for testing

processes. This Base class should include:

a. Context Configuration required for your project. These xml files should include

the mock beans for the java classes used in the Listeners, actual beans for the

service/user tasks in your BPM process. For example above, we will need to add a

real object for PFC Notification Service Task and mock objects for the service calls

within the PFC Notification Service Task.

b. Methods to start the job executor and stop job executor. The JobExecutor

is an interface provided by Activiti that is used to start and stop the BPM process.

c. Method to get the sub process instance id. This can be done by querying

database using the main process instance ID.

The JUnit class for the BPM process should include:

a. Autowired beans. (Injection of dependent objects or beans)

b. Application configuration. This requires the name and version of the process


c. A variable map based on the inputs required for the process. This can be

analyzed based on the flow of the BPM process.

d. Stubbed calls used in the listeners to return expected object. You can refer to

Unit testing section.

e. Execute the process using RuntimeService and Repository Service. (You can find

details of the Core interfaces in the links provided in References Section)

ProcessInstance instance = runtimeService.startProcessInstanceById(processDefinitionId,

businessKey, paramMap);

ProcessDefinition definition =

repositoryService.createProcessDefinitionQuery().processDefinitionId(processDefinitionId)

.singleResult();

As mentioned above, if the BPM process includes a user task, this task needs to be

completed using code. Please see the example below.

Next important point to conclude a test case is assertion. Once we execute the process

using runtimeService, we should check if the expected tasks are created. In this example,

we should assert if the two user tasks are created. Please see code snippet above.


The advantage of testing the BPM process is that you can uncover the gaps in the flow even

before the application goes into the Black box testing phase.

In conclusion we would like to summarize the overall benefits of white box testing…

Benefits of White Box Testing –

Unit tests clean up the source code. They ensure that class methods are performing exactly

as required and that no extra functions are carried out; unnecessary conditions are

removed; looping structures are simplified and that overall proper coding standards are

followed. This improves overall code quality and performance of the system. Statistics have

shown that finding and correcting defects at earlier stages of development is more cost

effective. Figures from “Applied Software Measurement”, (Capers Jones, McGraw-Hill 1991),

for the time taken to prepare tests, execute tests and fix defects show that unit testing is

about twice as cost effective as integration testing and more than three times as cost

effective as system testing.

Effectiveness of Early Testing

Integration tests ensure that all modules are interacting perfectly and that there are no

breakages. The database functionality is also tested thoroughly.

BPM tests ensure that process flows have the best possible design and that they are

efficient.

Since overall quality of the entire system improves by white box testing, clients are satisfied

with the quality that they receive at the end and customer satisfaction leads to customer

retention. Customer retention increases the overall brand value.


References

Mockito FrameWork:http://docs.mockito.googlecode.com/hg/org/mockito/Mockito.html

BPM Process tool Activiti with details of interfaces required for adding unit test cases

forBPM:http://www.mif.vu.lt/~donatas/PSArchitekturaProjektavimas/Library/BP/2010%20B

PMN%202.0%20whats%20in%20it%20for%20developers%20%28Activiti%29.pdf

Foundations of Software Testing

Authors’ biography

Smita More, having 6+ years of experience in IT industry. During this tenure I have mostly

worked in Media Data Advertising and Broadcasting Domain, Financial and Mortgage

Services. I am working with Xpanxion International Private Ltd since last 4 years. I am

working as a QA Lead and handling the team size of 5. In our current project we are

working on White Box Testing using technologies Spring Framework, Spring Integration,

JUnit and Maven. I also have experience in Automation using Selenium Web Driver, Test

Complete, Sikuli, Cucumber, ETL Testing and Web Services Testing. Technical Languages

that I am familiar with are Core Java, SQL, Python, Shell scripting. Databases used are SQL

server, My SQL, Oracle, PostGreSql.

Chaitanya Dharmadhikari, having 2 years of experience in the IT industry. I have worked

in Hotel, Real Estate and Weather Forecasting domains. I am working with Xpanxion, India

right from my internship days. I have done both black box testing and white box testing. In

our current project I am working on automation testing for an iOS application. I have

worked on various tools like HP-QC, Jira, XCode and Idea. The technical languages I am

familiar with are Java, UI Automation JavaScript and SQL. The databases used are SQL

server and MySQL.

http://docs.mockito.googlecode.com/hg/org/mockito/Mockito.html

http://www.mif.vu.lt/~donatas/PSArchitekturaProjektavimas/Library/BP/2010%20BPMN%202.0%20whats%20in%20it%20for%20developers%20%28Activiti%29.pdf



white box testing in practice - qai global...

Documents