TestManager Java Scalability

by Kevin Mooney

Editor's Note: Each month, we will feature one or two articles from the Rational Developer Network, just to give you a sense of the content you can find there. If you have a current Rational Support contract, you should join the Rational Developer Network now!

Introduction

This paper describes a technique for playing back Java test scripts with TestManager that reduces by half the memory required for each virtual tester. The technique is also useful for extending TestManager with detached Java code by allowing the code access to TestManager logs. Using this technique, a Java test script that would normally require 11 megabytes per virtual tester now requires only 4 megabytes per virtual tester. This memory savings enables you to play back hundreds of Java virtual testers with only a fraction of the memory required now.

TestManager Extension Architecture

TestManager extensions enable you to add new recording technologies, new script generation technologies, and new playback technologies. The technique described in this paper focuses on playback extensions made possible using a Test Script Execution Adapter (TSEA). The Test Script Execution Engine (TSEE) within TestManager uses different TSEAs to execute scripts of different types. For example, one TSEA executes command line scripts, and another TSEA executes Java scripts. Figure 1 shows the architecture of a typical Java test script playback using the Java TSEA.

Click to enlarge

Figure 1: TSEE to TSEA Relationship for Java Playback.

In this scenario, typical memory usage per virtual tester is 11 megabytes for a relatively simple Java test script, because each process that represents a virtual tester requires a separate JVM.

Challenges to Scalable Java Playback

Given the architecture described in the previous section, it becomes clear that as you increase the number of virtual testers, memory requirements become prohibitive. With each virtual tester requiring a minimum of 10-15 megabytes of memory, only a few virtual testers can be supported by what could be considered a typical machine. Once RAM is exhausted, and the machine begins to page and swap, the fidelity of transaction timing is compromised.

One approach to allowing more Java virtual testers to execute on one machine is to consider ways to reduce the number of JVMs required. If virtual testers can run in Java threads, then one JVM could support more than one virtual tester. This idea is good in principle, but in the TestManager TSEE scenario with the Java TSEA, multiple threads cannot work.

When we look closer at how each virtual tester logs distinct information, it becomes clear that using threads with the Java TSEA is not the right solution for running multiple virtual testers in one JVM. The TSS library provides the ability for a virtual tester to perform delays, report if the System Under Test (SUT) is functioning correctly, log data, and log timing information for transactions, plus much more. In the typical TSEE/Java TSEA/TSS scenario as described in the previous section, any TSS calls are logged in the context of the one virtual tester represented by the process executing. If the Java application creates threads that make TSS calls, those calls will go into the log, but when viewed later will appear as though they were generated by the single virtual tester for that process.

What is needed to solve this problem is the ability for Java threads to specify they are logging information on behalf of a specified virtual tester, not just the single virtual tester that has a TSS direct connection, which is the case when running with the Java TSEA.

Test Script Services Remote

The winter 2001 TestStudio release, 2002.05.00, enables each virtual tester to establish a remote connection to the Test Script Services (TSS) remote listener. Some TSEAs provided by Rational, such as the Command Line TSEA and the Java TSEA, publish the host name, port number, and ID for each virtual tester. With this TSS listener information, a remote or detached process that is not part of the TestManager play back can access the TSS, including test logs, available to a virtual tester.

Figure 2 shows a remote detached virtual tester interacting with TestManager during playback.

Click to enlarge

Figure 2: Java application running outside TestManager logs data using TSS Remote.

With TSS remote connection information now available to a Java test script, individual threads within the Java application can connect to a virtual tester TSS remote listener. Once connected, subsequent TSS calls for logging or timing within that thread are logged in that virtual tester's log.

Greater Scalability for Java

A more scalable Java play back can be achieved by starting a number of "drone" virtual testers equal to the number of Java threads that you intend to run. The purpose of the drones is to publish their TSS remote connection information, and then wait for the test to complete. For example, each drone virtual tester can create a file in the %TEMP% directory whose name is identical to the virtual tester ID, and place in that file the port number for the TSS remote listener. In this paper, we assume the hostname for all remote connections will be to the local computer. After the drones are started, you can then start a threaded Java test script to perform the load testing work.

Drone virtual testers should consume as few resources as possible. One way to use minimal resources is to add a new Test Script Execution Adapter that satisfies the API requirements of a TSEA, but does no work other than publishing the TSS remote listener port information. This new TSEA is called the Drone TSEA.

The number of drone testers should equal the number of Java threads you plan to create. The Java application can be initiated as a command line script within the TestManager suite. The Java application, using the

remote TSS, can make TSS connect() calls to establish a remote TSS connection to the appropriate drone virtual tester's remote TSS listener port. Once connected in this manner, a Java thread can make subsequent TSS calls to log information in the drone virtual tester's log.

Figure 3 shows the TSEE, TSEA, and Java application interactions that make this type of logging possible.

Click to enlarge

Figure 3: Threads within a Java application log like distinct virtual testers.

During play back, a threaded Java program that uses TSS remote can reduce resource usage by connecting threads to the TestManager drones and logging data on behalf of the virtual tester represented by each drone. With this architecture, only one JVM is created, thus saving many megabytes of memory during playback.

Note that the Java application is invoked using the Command Line TSEA, and not using the Java TSEA. The Java TSEA, by design, uses the TSS direct library, and does not provide for the possibility of using the TSS remote library. Therefore, the Java application must run under the Command Line TSEA, and use the rt.tssjava.remote property to indicate the application wishes to use the TSS remote library. It is also critical that the application being played back is thread safe.

Example

This section provides instructions for demonstrating scalable Java play back. To run this example, you must have the following:

● Windows NT or 2000

● Java JDK 1.3

● TestStudio 2002.05.00

The following sections show, step by step, how to apply the concepts

described in this paper to achieve scalable Java play back.

1. Click here to obtain a copy of the sample code zip file, tm_java_scalability.zip.

2. Unzip and place the files into a directory. For example, unzip the files into a directory called c:\scale.

3. Copy the Drone TSEA, rttseadrone.dll, from the c:\scale directory to the install TSEA directory. The follwing figure shows where the dll should reside in a typical installation.

Click to enlarge

Figure 4

4. Modify the CLASSPATH environment variable to allow for compiling and executing the Java application. You need to add the following paths:

%RATL_RTHOME%\tsea\rttseajava.jar%RATL_RTHOME%\rttssjava.jar%RATL_RTHOME%\rtutil.jarc:\scale

For example:

Figure 5

Start TestManager, and then create a new test script type for the Drone TSEA. From the TestManager window, select Tools -> Manage -> Test Script Types.

Figure 6

After the window above appears, click New.

Under Name, enter rttseadrone. Your screen should look like the following:

Figure 7

Select the Execution Adapter Type tab, click Use a custom execution

adapter, and then specify the name of the Execution adapter library as rttseadrone as follows:

Figure 8

Select the Sources tab, and click Insert. Click Yes when asked if the type should be created. Then specify the test script source name as rttseadrone as follows:

Figure 9

Select the Connection Data tab. Enter a data path for the test script source location. You can enter any path because this execution adapter does not execute scripts. The following dialog box specifies the path as c:\scale. Click OK or Close as needed to complete the process.

Figure 10

Compile the sample test scripts. Start a Command Prompt window, change directory to c:\scale, and enter the following command:

c:\scale> build

This command compiles two Java programs: EmulMany.java and VirtualTester.java.

EmulMany.java is the application that each virtual tester executes. Unlike Java test scripts that execute under TestManager, this application, which runs outside TestManager, should not extend the TestScript class. Instead, the script should implement the TSSConstants interface. For example,

public class EmulMany implementscom.rational.test.tss.TSSConstants {

VirtualTester.java is the application that creates the Java threads which execute EmulMany.java. It can be reused for other applications. Note the commented area which indicates where and how you specify what Java class to execute for each virtual tester:

// Insert thread work here

EmulMany em = new EmulMany();String[] args = null;em.testMain(args);

// End thread work

Note that the build command simply compiles the Java test scripts. Access to libraries such as the TSS remote Java library is controlled through a Java property. The property, and its correct value, will be specified later when we create the TestManager suite.

From TestManager, select File -> New Suite and create a Blank performance testing suite. Select Suite -> Insert -> User Group to insert a new user group with ten virtual testers as shown below.

Figure 11

Select Suite -> Insert -> Test Script. Select rttseadrone as the test script source as follows:

Figure 12

Click Browse, and then select any file. The Drone TSEA does not execute this file or test script; instead, the TSEA publishes TSS remote listener port information into the TEMP directory. For example, you could even select readme.txt as the test script file.

Select Suite -> Insert -> Synchronization Point and insert two synchronization points, one called 'init', and the other called 'done'. Your TestManager window should look like the following:

Click to enlarge

Figure 13

Create another user group with one virtual tester. Insert the synchronization point 'init'. Then insert a test script to execute a command line script as follows:

Figure 14

Click Browse, and enter the following information to execute the Java script VirtualTester.

Figure 15

The VirtualTester Java test script accepts arguments for the number of threads to execute and the starting virtual tester ID. These arguments enable you to start more than one VirtualTester Java test script at playback. You may want to start more than one Java test script to spread the threads over more than one JVM. For example, in a 500-user play back, the first test script can start 100 threads beginning with virtual tester ID 1, the second can start 100 threads beginning with virtual tester ID 101, and so on.

Suite -> Insert -> Synchronization Point and insert a synchronization point called 'done'. Your suite should look like the following:

Click to enlarge

Figure 16

Run the test suite with 11 users.

Figure 17

The EmulMany test script performs several delays, each of different lengths, over an elapsed period of several minutes, with some emulated commands passing, and some failing. This script is designed to ensure some emulation commands fail to provide results that are varied and interesting. The failure rate is approximately 10%, and your results should be similar. Note that although 10 distinct virtual testers appear to be working, the work is actually performed by 10 Java threads.

Click to enlarge

Figure 18

Conclusion

In summary, TestManager Java scalability enables a Java test script to use many threads to simulate virtual testers. Using this technique, once a

multi-user run completes, the log appears as if many distinct users have logged information regarding the timing or success of methods or transactions. In previous releases, any TSS calls made by Java threads would appear as though they were logged by only one virtual tester. Now, each thread can appear as a unique virtual tester. In our tests, simple Java scripts that had required 11 megabytes per virtual tester now required only 4 megabytes per virtual tester. Of course, the exact amount of memory required per virtual tester depends on the work of each individual thread.

For more information on the products or services discussed in this article, please click here and follow the instructions provided. Thank you!

Copyright Rational Software 2002 | Privacy/Legal Information