run and analyze software tests with rational clearquest test

Run and Analyze Software Tests with RationalClearQuest Test ManagementTest Management with IBM Rational ClearQuest

Skill Level: Intermediate

Brian Bryson ([email protected])Technology EvangelistIBM

17 Oct 2006

Learn how to handle test management with the new functionality and features inIBM® Rational® ClearQuest® V 7.0 TestManager, which now handles the testmanagement functions previously provided by IBM Rational TestManager. Thistutorial gives you a basic understanding of installation, configuration, test planning,and running different types of tests with this new tool. It also covers how to useClearQuest query and reporting capabilities for test analysis.

Section 1. Introduction

Test Management with IBM Rational ClearQuest

The Version 7.0 release of IBM Rational ClearQuest heralds a significant strategyshift in the IBM Rational approach to test management. Before this release, testmanagement functionality was provided by IBM Rational TestManager, whichincluded test planning, test execution, and test results analysis for quality assurance(QA) organizations.

Starting with the V 7.0 release, ClearQuest handles test management, instead.ClearQuest provides the same core functionality available in Rational TestManagerbut extends it with features that address the challenges of the modern QAorganization. Among the benefits are these new features:

• Web interface

Run and Analyze Software Tests with Rational ClearQuest Test Management© Copyright IBM Corporation 1994, 2006. All rights reserved. of 32

mailto:[email protected]

http://www-306.ibm.com/software/awdtools/test/manager/

http://www.ibm.com/legal/copytrade.shtml

• Eclipse integration

• Tighter defect integration

• Enterprise-level database support

• More powerful reporting

• Query capabilities for regulatory compliance through eSignatures andAuditTrails

• Support for geographically distributed test teams, using ClearQuestMultiSite

Tutorial overview

The purpose of this tutorial is to provide a basic understanding of the new testmanagement functionality introduced in the 7.0 version of IBM Rational ClearQuest.This tutorial takes you through the following steps:

• Installation and configuration of ClearQuest and associated packages

• Test planning, including creating a test asset registry, test plan, and testcases

• Execution of different types of tests

• Test analysis, focusing on the ClearQuest query and reporting capabilities

Intended audience

This tutorial is intended for first-time users of the ClearQuest test managementcapabilities. This includes users who are evaluating ClearQuest for the first time, aswell as those who already use ClearQuest for change management and would liketo take advantage of the new test management capabilities.

Time to complete tutorial

After you have the software installed, this tutorial should take approximately 30-60minutes to complete.

Section 2. Tutorial Setup

There are no pre-requisites for this tutorial, other than having the IBM RationalClearQuest software and time to complete the tutorial. To help you understand the

developerWorks® ibm.com/developerWorks

Run and Analyze Software Tests with Rational ClearQuest Test Management of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.

http://www.eclipse.org/

http://www-306.ibm.com/software/awdtools/clearquest/


full capabilities of ClearQuest, this tutotrial includes a .zip file of sample tests, whichyou will use to better understand various concepts. (See Figure 1.) Follow thesesteps to install the files:

1. Using a tool such as WinZip, expand the included .zip file to your C drive(C:\). The files will be expanded to a directory called CQTM Test Scripts.

2. ClearQuest requires that this directory be accessible through a Microsoft®Windows® network shared location. To enable this directory as a sharedresource, open Windows Explorer

3. Within Windows Explorer, right-click the C:\CQTM Test Script FileLocations directory, and then select Sharing and Security.

4. If you have never enabled file sharing on your machine, proceed to thenext step. If you have previously enabled file sharing, skip ahead to Step7.

5. In the resulting CQTM Test Scripts Properties dialog Sharing tab, clickthis statement: If you understand the security risks but want to sharefiles without running the wizard, click here.

6. In the resulting Enable File Sharing dialog, select Just enable filesharing, and then click OK.

7. In the CQTM Test Scripts Properties dialog, select Share this folder onthe network. Accept the default name of CQTM Test Scripts

8. Although not necessary for this tutorial, check Allow network users tochange my files, because ClearQuest writes test results to this sharedfolder.

9. Click OK to close the CQTM Test Scripts Properties dialog.

10. Finally, click Yes to acknowledge the warning about shared folder namesthat are longer than 12 characters.

Figure 1. CQTM test scripts dialog

ibm.com/developerWorks developerWorks®


http://www.winzip.com/


Section 3. Install ClearQuest and Create Your Database

This tutorial assumes no previous knowledge of IBM Rational ClearQuest. It walksyou through every step, from installation through reporting. In this first section, youwill complete your first two steps. First, you'll install ClearQuest, and then you'llcreate the databases necessary to store your ClearQuest data.

Install IBM Rational ClearQuest

Get ClearQuest software




At the time of publication, IBM Rational ClearQuest was notavailable for trial download. To get the software, contact your localIBM Rational representative, and request an evaluation copy andlicense key.

Follow these steps to install IBM Rational ClearQuest:

1. Run setup.exe, and then proceed through the wizard to perform adesktop installation.

2. Accept all defaults. Feel free to install to any location or to addcomponents, such as the ClearQuest Web interface. This tutorial doesn'tcover them, but you can install them now for future use.

3. In the configuration wizard panel, click Done. You do not need toconfigure anything at this point.

4. If you have received trial software with an evaluation key, uponcompletion of the installation, you will be taken to the IBM RationalLicense Key Administrator to enter your evaluation license key. Currentcustomers can use the wizard to point to their current license server.

Create a schema repository and sample database

Follow these steps to create a sample database:

1. Start the ClearQuest Maintenance Tool.

2. In the ClearQuest Maintenance Tool, select Schema Repository >Create.

3. Select MS_ACCESS as the vendor, and then choose a location on yourhard drive to store the schema repository. This tutorial uses C:\CQTMTutorial\CQTM_SchemaRepo.mdb.

4. In the Existing Connections field, change the connection name to CQTM

5. Click Next, and then acknowledge the warning about placing therepository in a shared location.

6. Click Next to accept the ClearQuest Data Code Page. This will differ,depending on the language of your OS. In Canada and the U.S., thedefault will be: 1252 (MS Windows Latin 1).

7. On the next screen, check the Create sample database box, and thenselect the Enterprise Schema.

8. Enter the database name: CQTST. You have the option of providing a



http://www-306.ibm.com/software/rational/howtobuy/contact/?S_TACT=105AGX23&amp;amp;amp;amp;amp;S_CMP=RMAIN

http://www-306.ibm.com/software/rational/howtobuy/contact/?S_TACT=105AGX23&amp;amp;amp;amp;amp;S_CMP=RMAIN


description.

9. Click Next.

10. Again, select MS_ACCESS as the database type for the sample data,,and then provide a Physical Database Name of C:\CQTMTutorial\CQTM_Data.mdb.

11. Click Finish to complete the process, again acknowledging the warningabout storing data in a shared location.

12. After you have reviewed the status messages, click Done, and then clickExit the ClearQuest Maintenance Tool.

Launch ClearQuest and log in

IBM Rational ClearQuest clientsThere are many interfaces or clients for IBM Rational ClearQuest.Looking into the program group that was created when you installedClearQuest reveals two clients for ClearQuest. The first is theClearQuest Eclipse client, and the second is the ClearQuest forWindows client.

Note:

There is also a third ClearQuest Eclipse plug-in and, of course, theClearQuest Web client. There are no icons in the program group foreither of those.

The difference between the Eclipse client and the Eclipse plug-in isthat the Eclipse client provides its own Eclipse shell. The plug-in isfor users who have already installed Eclipse and merely want toinclude ClearQuest functionality in their pre-existing shells.

Users who do not wish to install a full ClearQuest client or thosewho are using Apple hardware can access ClearQuest data throughthe ClearQuest Web interface. For Linux and Unix users, there arealso native Unix and Linux clients for ClearQuest.

For this tutorial, you'll use the ClearQuest Eclipse client, which is an Eclipse-basedstand-alone tool. Everything you do in this tutorial can be done from the Eclipseplug-in, but some of the menu navigation differs slightly between the two clients.Much of what you'll do in this tutorial will be based on the ClearQuest Test Managerview, which is available only in the Eclipse clients. Therefore, do not attempt tocomplete this tutorial using the ClearQuest Windows or other native OS clients.Follow these steps (also see Figure 2):

1. Click Start > All Program > IBM Rational > IBM Rational ClearQuest >ClearQuest to start ClearQuest

2. Optional: Click the Overview icon on the Welcome screen. From there,



http://www.ibm.com/developerworks/rational/library/04/r-3089/

http://www.eclipse.org/


you can watch a 15-minute Rational ClearQuest TestManager video tour.Then close the Welcome screen. You can return here later if you wish bygoing to Help > Welcome.

3. Select File > Database > Manage connections.

4. Select the CQTM database set, and then click Add Connection....

5. Accept the default Schema Repository of CQTM, and then click Next.

6. Enter the user ID: admin. There is no password on this account.

7. In the Connect dialog, select Remember the password, and then clickOK to log in.

8. The Test Manager view should be visible, although it will not containdata.

Figure 2. IBM Rational ClearQuest

You are now connected to the CQTST database that you created previously, andyou are ready to start this tutorial.

Section 4. Test Project Configuration

Before you begin to create your test plans and test cases, you need to complete aone-time setup and configuration. You must provide core information to ClearQuest,such as the location where you'd like to store your test assets.

The asset registry




The asset registry will hold all of your assets. It will contain all of your test plans,which contain all of your test cases. It will contain all of your test suites, as well asfile locations for all of your IBM Rational Manual Tester, IBM Rational FunctionalTester, and IBM Rational Performance Tester tests. It will also contain all of your testresults.

See Figure 3 and Figure 4. Complete these steps to create your asset registry:

1. Right-click the Asset Registries branch in the Test Manager view, andthen select New Asset Registry.

2. Type the name CQTM Tutorial, and then your sample testassets description.

3. Click OK to create your asset registry

Figure 3. Asset registry creation dialog

Figure 4. Newly created asset registry with test plans, test suites, iterations,and file locations




Configurations and configuration attributes

What if I don't use configurations?Although configurations are a powerful feature of IBM RationalClearQuest, not every team tests multiple configurations.ClearQuest requires at least one base configuration and one baseconfiguration attribute to run tests. If you need only oneconfiguration, create only a single attribute and a singleconfiguration.

ClearQuest TestManager recognizes that most test teams run tests across a varietyof configurations. As an example, QA teams often run the same tests on variousoperating systems and versions, such as Microsoft® Windows XP®, Microsoft®Windows Server®, or Linux® operating systems. Alternatively, teams might need totest different hardware configurations, such as a machine with 512 MB of RAM andanother machine with 2 GB of RAM. Whatever the configuration, ClearQuestTestManager can record various configuration attributes (such as operating system,memory, disk space, and so forth) so that you can group attributes into commonconfigurations.

Configurations and configuration attributes are not part of your asset registry.Therefore, they can be shared across asset registries. The benefit of this is that youneed to enter this information only one time, rather than having to recreateconfigurations for every asset registry.

Create configuration attributes

Follow these steps to create basic configuration attributes (also see Figure 5):

1. Expand the Configurations folder to reveal the ConfigurationAttributes folder.

2. Right-click the Configuration Attributes folder, and then select New




Configuration Attribute.

3. Type Operating System in the Name field, and then click OK to createthe attribute.

4. Right-click the newly created attribute, and then select NewConfiguration Value.

5. Enter Linux in the Name field, and then click OK to create the attributevalue.

6. Repeat the last two steps, entering Windows XP in the Name field.

7. Right-click the Configuration Attributes folder, and then select NewConfiguration Attribute.

8. Type Memory in the Name field, and then click OK.

9. Right-click the newly created attribute, and then select NewConfiguration Value.

10. Type 2 GB in the Name field, and then click OK to create the attributevalue.

11. Repeat the last two steps, this time typing 512 MB in the Name field.

Figure 5. Configuration attributes

Create configurations

1. Right-click the Configurations folder, and then select NewConfiguration.

2. Provide the Name: Minimum Memory Windows XP.

3. Next to the Configuration Attributes List, click Add.

4. In the Browse Record Type dialog, click Search to reveal all of theconfiguration attributes that you just created.

5. Hold down the CTRL key while you click to select both Windows XP




operating system and 2 GB Memory.

6. Click OK to close the Browse dialog.

7. Click OK to close the Create (TMConfiguration)dialog.

8. Repeat the process to create a configuration called RecommendedWindows XP Configuration. Again, select the Windows XP operatingsystem and 512 MB of memory simultaneously. (See Figure 6).

Figure 6. Completed configurations and configuration attributes

Section 5. Test Planning

Asset registry and configuration are simple, one-time steps that you typically doearly in a test project, and then forget. However, test planning is process thatcontinues throughout a test project.

In the next section, you will create a multi-tiered test plan. A test plan is anorganized structure of test cases. Test cases are the specific tests or validationsthat you want to perform on your system. Thus, after you create your test plans, youwill insert your test cases into those test plans.

Finally, you will associate the configurations that you created in the previous sectionwith your test cases. A simple test case is an abstract concept, in that it merelydefines something that you want to test. By associating a configuration with a testcase, you create a configured test case, which identifies not only what you want totest, but where you want to run your tests (on which platform or configuration).

Create test plans

Sample application: Classics Online




From this point forward in this tutorial, you will base all of your testplans, test cases, and suites on the >Classics Online sampleapplication. Classics Online is a simple, online music CD-orderingsystem. It's a lightweight Java™ application that serves as yourapplication under test.

You can download the sample application from the downloadssection of this tutorial. To run the sample application, expand the fileto any location on your hard drive, and then double-click theClassicsJavaA.jar application.

Note:

You must have a Java Runtime Engine installed to run thisapplication.

The first step in your test planning process is to create a test plan to hold all of yourtest cases. There are many ways to organize your test plans and test cases. Ratherthan reading through a long discussion of the pros and cons of test plan organizationhere, you will simply use the very common functional decomposition structure tocreate child test plans for various functional areas of the application. By using thisstructure, your test plan architecture matches your application architecture.

Follow these steps to create your basic test plan structure:

1. Right-click the Test Plans branch of the CQTM Tutorial asset registrythat you created earlier, and then select New Test Plan.

2. Type Master Functional Test Plan as the headline.

3. Click OK to create your test plan.

Now that you have created your main test plan, you can create test plans for someof the functional areas of the Classics Online application (see sidebar for details onthe sample application).

1. Right-click the Master Functional Test Plan branch that you just created,and then select New Test Plan.

2. Enter Order Placement Tests as the headline.

3. Click OK to create your test plan.

4. Repeat the process to create a test plan for Administrative FunctionTests.

5. Repeat the process to create a test plan for Order Review Tests.

Figure 7. ClearQuest supports hierarchical test plans for better test plan andtest case organization



http://www.java.com/en/download/windows_ie.jsp


Create test cases

Feel free to wander aboutThe goal of this tutorial is to walk though the process of enterprisetest management. To get through the entire process within thealloted time, we're not going delve deeply into some subjects.Nonetheless, there are interesting side attractions for you. As anexample, in the next exercise we'll create some very basic testcases, but if you flip through the various tabs of a test case, you'llsee that there's lots of information that you could be entering, suchas:

Test Motivator: Here you can attach any supporting file ordocumentation that might provide more information about the testcase. You can attach any type of file, such as a spreadsheet orscreen captures, to add more context to your test case.

Legacy data: These fields map to ClearQuest TestManager fieldsfor backward compatibility.

Notes: These are freeform text fields where you can make anygeneral notes about the test case.

Requirements: Here is where ClearQuest can integrate with IBMRational RequisitePro®, the Rational requirements managementtool. If you are running RequisitePro, you can associate yourrequirements with your tests, which later enables you to associateyour test verdicts with your requirements. This gives you a fulllifecycle view of the progress of your application, from requirementsto tests and defects.

After you have established your test plan structure, the next step is to populate thetest plan with test cases. The test cases define each of the validations that you planto perform on the system. (See Figure 8.)

Follow these steps to create a test case for each of your test plans:

1. Right-click Order Placement Tests, and then select New Test Case.




2. Type Place Valid Order as the headline.

3. Under Priority, select 1 - Critical.

4. Now right-click Administrative Function Tests, and then select NewTest Case.

5. Enter View Customer List as the headline.

6. For Priority, select 3 - Medium.

7. Right-click Administrative Function Tests, and then select New TestCase again.

8. Type View Existing Order Status as the headline.

9. For Priority, select 2 - High.

Figure 8. Test plans with associated test cases

Associate test scripts with test cases

The test plan and test cases represent your logical test plan. You can think of thetest plan as a list of test cases to run. But how will you run these tests? And how willyou know if a test passed or failed? Answer: The Execution tab.

You use the Execution tab to link your test case to an actual test script. The testscript then issues the pass or fail verdict when it runs.

The ClearQuest TestManager supports execution of test scripts that were created inIBM Rational Functional Tester, IBM Rational Manual Tester, and IBM RationalPerformance Tester test automation tools. It also supports the Test and




Performance Tool Platform, or TPTP-compatible tests, such as TPTP JUnit tests.

• Rational Functional Tester is a script-based regression testing tool thatcaptures user actions against an application in script form for subsequentplayback and system validation.

• Rational Manual Tester is a manual test authoring and execution tool.You can use it to capture test script steps and actions that you want to runon an application, and then run the tests. It then automatically enters andvalidates data as the you run the test.

• Rational Performance Tester is a tool to help eveluate system responsetimes for Web-based and enterprise resource planning (ERP)applications. It simulates simultaneous user activity on your system andmeasures response times to help you poinpoint performance issues andbottlenecks before you release your system to production.

Establish file locations

Where did these file locations come from?In this section, you use the tutorial working assets that youestablished during the Tutorial Setup section. If you skipped thatstep, go back now to create these sample assets.

Before you can associate your test scripts with your test cases, you need to tellClearQuest the file locations of your tests. This is a one-time setup for each assetregistry. After that setup, all test cases will be able to access the scripts in the filelocations that you specified. (See Figure 9.)

Follow these steps to establish file locations for your Rational Manual Tester andRational Functional Tester scripts:

1. Right-click the File Locations folder in the asset registry, and then selectNew File Location.

2. Click Next to accept CQTM Tutorial as the default asset registry.

3. Type Functional Tester Tests in the Name field.

4. Provide this file location: C:\CQTM Test Scripts\FunctionalTester\Functional Tests.

5. Provide the test log location: C:\CQTM Test Scripts\FunctionalTester\Functional Tests_logs.

6. Click Finish to create the file locations.

7. Repeat the process to create a file location with the following properties:



http://www-306.ibm.com/software/awdtools/tester/functional/

http://www-306.ibm.com/software/awdtools/tester/manual/

http://www-306.ibm.com/software/awdtools/tester/performance/


• File location name: Manual Tester Tests

• File location: C:\CQTM Test Scripts\Manual Tester\ManualTester Scripts

• Test log location: C:\CQTM Test Scripts\ManualTester\Manual Tester Logs

Figure 9. File locations tell ClearQuest where to find test automation scripts

Establish associations between test scripts and test cases

Now that ClearQuest knows where your scripts are, follow these steps to associateRational Functional Tester and Rational Manual Tester scripts with your test cases(also see Figure 10):

1. Right-click the Place Valid Order test case, and then select Modify.

2. Navigate to the Execution tab.

3. Click the Browse button.

4. Select Rational Functional Tester as the test type.

5. Expand the Functional Tester Tests file location branch.

6. Select the Place_Valid_Order script.

7. Click Finish to complete the association.

8. Click OK to close the test case.

9. Note the icon change on your test case. There is now a small script iconon the top-right corner of your test case icon.

10. Repeat the process to associate the Functional Tester Test calledView_Existing_Order with the View Existing Order Status test case.




11. Switch the test type to Rational Manual Tester.

12. Repeat the process to associate the Manual Tester Test calledView_Customer_List with the View Customer List test case.

Figure 10. Test script associated with test case

The View Existing Order test script is associated with the View Existing Order Statustest case. (Your record ID, in this case CQTST00000047, may be different.)

Section 6. Test Execution

More to come...Customers who are migrating from Rational TestManager will noticethat ClearQuest's execution model is less mature than that ofClearQuest TestManager -- for now. Currently, ClearQuest supportsbasic sequential test execution. Many customers have requestedadditional functionality, and the IBM Rational development teamsare increasing capabilities in this area, which will be released in thenear future.

To make this tutorial as generally applicable as possible, we did not want to assumethat you had installed any test script execution tools, such as Rational Manual Testeror Rational Functional Tester. That is why what you do in the next section goes onlyup to the point of actually executing a test. First, you run an individual test case, andthen you create a suite of test cases. Your first step, regardless of how you executetests, is to associate configurations with test cases. (See Figure 11.)

Associate configurations with test cases

Test cases are an abstract artifact. A test case represents something that needs tobe tested, but a test case itself cannot be executed. A test case becomes executablewhen you associate it with a specific configuration. You have already created twoconfigurations during this tutorial, so your next step is to tie your test cases to your




configurations.

Follow these steps to associate your configurations with your test cases:

1. While holding down the control key to select all three test casessimultaneously, click all three of the test cases you created in theprevious step: Place Valid Order, View Customer List, and ViewExisting Order Status.

2. Right-click the Place Valid Order test case (or any other test case), andthen select Add Configured Test Cases.

3. In the Add Configured Test Cases dialog, select RecommendedWindows XP Configuration.

4. Click OK to add the configuration to the three test cases.

5. Repeat the process, adding the Minimum Memory XP configuration tothe Place Valid Order test case only.

Figure 11. Your test planning process is complete, and your configured testcases are ready to run

Schedule test iterations

You can think of configurations as defining where to run a test case. In ClearQuest,you also define when to run a test case. The when of a test case is called aniteration. (See Figure 12.)

If you are familiar with the IBM Rational Unified Process®, or RUP®, then theconcept of iterations will not be new. Rational TestManager users will also befamiliar with the concept. The idea of an iteration is to specify at what point in time,




or at what stage of development, a test case can be run.

Iterations are specific to your organization and your development process. To keepthings simple, use the following steps to create three simple stages of development,or iterations:

1. Right-click the Iterations folder of your asset registry, and then selectNew Iteration.

2. Provide the Name: Inception.

3. Provide the Start Date: January 1, 2007.

4. Provide the End Date: January 31, 2007.

5. Click OK to create your iteration.

6. Repeat the process to create an iteration called Elaboration that runsfrom February 1, 2007 to February 28, 2007.

7. Repeat the process again to create an iteration called Construction thatruns from March 1, 2007 to March 31, 2007.

Figure 12. Iterations

Run a single configured test case

For such an important step, the actual process of running or executing a test issurprisingly simple. You just right-click a configured test case, and then selectExecute. To execute a configured test case, the automation tool must be installed. Ifit is not installed, the Execute option will be disabled.

Note:

Test execution is not possible from all ClearQuest clients. The Windows and Webclients cannot execute any scripts. The Linux client can execute only Rational




Functional Tester Java scripts. The ClearQuest client that you are using in thistutorial can execute only Rational Manual Tester scripts. Only the Eclipse plug-incan execute all types of scripts. Therefore, for any customers who will be launchingscripts directly from ClearQuest, the recommendation is to use the Eclipse plug-in asyour primary ClearQuest client.

Create a suite of configured test cases

Suites are configuration-specificWhen you create a test suite, you need to specify a configuration forthe suite. This selection is important, because the suite will be ableto execute only configured test cases that are associated with thisconfiguration.

Although possible, you will not often execute individual configured test cases. It is farmore likely that you will assemble a series of configured test cases for sequentialexecution. In ClearQuest terminology, this is called a test suite. (See Figure 13.)

Follow these steps to create a test suite of configured test cases:

1. Right-click the Test Suites folder of your asset registry, and then selectNew Test Suite.

2. Within the Main tab, provide this headline: Smoke Test Suite.

3. Within the Execution tab, select Recommended Windows XPConfiguration.

4. Click OK to create the test suite.

Take the following steps to add configured test cases to you test suite:

1. Right-click your newly created Smoke Test Suite, and then selectConfigure....

2. Click Add....

3. A list of all configured test cases that configurations that match the suitesconfiguration will display.

4. Select all of the test cases by clicking the first test case, and thenholding down the Shift key until you have selected the last test case.

5. Click OK to add the test cases.

6. Click the last configured test case, View Existing Order Status.




7. Click Move Up to move this test case into the second position. Testscripts are executed in the that order they are presented in this dialog.

8. Click Finish to complete the test suite configuration,

Figure 13. The configured test cases will be executed in the order that they arepresented in this dialog

Run a test suite

Much like running an individual configured test case, all you need to do to execute asuite is to right-click the test suite, and then select Execute. Again, for your presentpurposes, you will not be executing the tests. In the upcoming step, you will simulatean execution by importing the results from these tests that the IBM Rational test labsalready ran for you.




Import test results

Import test log formatThe format for a raw test log file comes from the Test andPerformance Tools Platform (TPTP) specification. The file itself is azip-compressed XML file. You can use a file-compression utility,such as Winzip, to open the file and look at the XML that it contains.

Why? Because this is a very powerful feature for getting results intoClearQuest. If you are using tool that are not supported byClearQuest for test execution, this is how you can export resultsfrom those tools into ClearQuest for consolidated test reporting.

At this point, you would normally run your configured test cases or your test suite,and your results would be populated automatically in ClearQuest as test log records.However, because we're working on the assumption that you are not using thesetools, you will use an alternate method to populate ClearQuest with the test logrecords.

ClearQuest has an import function that enables you to associate a test log recordwith a configured test case. A test log record contains two key pieces of information:a time and date stamp and a verdict. The verdict is Pass, Fail, or Inconclusive. Thetest log record is the key asset. When assimilated with all other test log records, itprovides you with the most information about your testing project.

Take the following steps to import test log records for your configured test cases:

1. Right-click the Place Valid Order - Recommended Windows XPConfiguration configured test case, and then select import Test Log....

2. For Test Type, select Rational Functional Tester.

3. Select the test log: Functional Tests_logs > Place_Valid_Order > null.

4. Enter the name: Build 1.

5. Click Finish to import the record.

6. In your Test Results view, right-click the newly imported record, and thenselect Commit.

Where did that build come from?You've seen configurations and iterations, but when you import yourtest log record, you're also asked for build information. A build is afurther level of granularity that you can use to subdivide and classifyyour test results. Although not really necessary in a small examplesuch as this one, it is an invaluable feature for large projects, wherenew builds of the application under test are a daily occurrence.




Section 7. Test Analysis

Finally, you reach the payoff point. Here is where you see the benefit of your hardwork of capturing and executing all of your test cases. At this stage, you will be ableto run queries and reports to give you a definitive diagnosis of the health of your testproject. You can run these reports as often as you like to get up to get anup-to-the-minute status report.

You can also share the reports and query results with other members of your team.Users who have configured ClearQuest Web can provide Web access to all reports.Users who do not set up this free feature of ClearQuest can simply export and e-mailresults to other team members.

Regardless of how you convey the information to your colleagues, ClearQuest givesyou all of the information you need to assess your application under test.

Switch to Sample database

Rather than having to import hundreds of test logs, for this tutorial, you're going to doyour analysis against a pre-populated database of results to save time. This sampledatabase also includes several custom-created queries from IBM Rational labs thatyou can use for your own projects. (See Figure 14.)

Follow these steps to connect to the sample database.

1. Select File > Database > Manage connections.

2. Click Add Database Set....

3. In the Name field, type Tutorial, and then select MS_ACCESS as theVendor.

4. Click Next, and then browse to C:\CQTM Test Scripts\CQTM SampleData\SampleMaster.mdb.

5. Click Add Connection....

6. Select the Schema Repository called Tutorial, and then click Next.

7. Provide the user ID: jan (which is case-sensitive), and then click Finish.

8. In the Connect dialog, provide the password: jan. Select Remember thepassword, and then click OK.




9. Click Close to close the ClearQuest Connection Management dialog.

Figure 14. Newly established connection to the tutorial Sample Data database

ClearQuest provides charts, queries, and reports

ClearQuest provides three formats to analyze your data: charts, queries, andr\Reports. (See Figure 15.) You will navigate through examples of all three of these.

Take the following steps to navigate to the analysis tools:

1. Switch to the ClearQuest Navigator tab.

2. Collapse the admin, CQTM@CQTST connection that you have beenworking with.

3. Expand the jan, Tutorial@AUCT connection.

4. Open the public Queries folder, and then find the TM Charts, TMQueries, and TM Reports folders. All of your next steps will be workingwithin these folders.

Figure 15. TM Charts, TM Queries, and TM Reports folders contain analyticaltools that come with ClearQuest




Run ClearQuest TestManager Charts

Take the following steps to examine some of the out of the box charts thatClearQuest includes (see Figure 17 and Figure 1 for examples):

1. Expand the TM Charts folder.

2. Double-click the Execution - Verdict Distribution chart.

3. Highlight the Auction asset registry.

4. Click OK to run the report

5. Double-click the Clear Quest Chart: Execution - VerdictDistributiontab to expand the report to full size.

Figure 16. This chart gives a graphical representation of all pass and failverdicts in the asset registry selected. This is a high-level, global summary ofall tests.




1. Double-click the Clear Quest Chart: Execution - Verdict Distributiontab to return the tab to normal size.

2. Close the Clear Quest Chart: Execution - Verdict Distribution tab.

3. Run the CTC - Iteration Distribution chart by double-clicking on thechart name. Expand the chart as necessary.

Figure 17. This chart gives a graphical representation of all of the configuredtest cases that are tagged with an iteration number. You can use this report tosee at which stage you will perform most of your testing.




Run ClearQuest TestManager Reports

Report customizationClearQuest's reports are created with the Crystal Reports reportingtool from Business Objects. To learn more about creating andmodifying ClearQuest reports, see the IBM Technote on How tocreate reports in ClearQuest using Crystal Reports.

Whereas charts contain purely graphical data, reports can contain both graphics andtext.

To examine the reports that ship with ClearQuest, follow these steps:

1. Collapse the TM Charts folder

2. Expand the TM Reports folder

3. Double-click Verdict Summary by Build report to run that report.

4. Select Asset Registry Auction.

5. Click OK to run the report

6. Double-click the Verdict Summary by Build tab to expand the report to



http://www-1.ibm.com/support/docview.wss?uid=swg21217221

http://www-1.ibm.com/support/docview.wss?uid=swg21217221


full size.

Figure 18. Verdict Summary by Build report

ClearQuest Reports can contain both graphic and text data. In Figure 19, the VerdictSummary by Build report breaks down the pass-or-fail verdict summary by builds.You also have the option of editing ClearQuest reports with the Crystal Reportsreport editing software, which is available from Business Objects.

Run ClearQuest TestManager Queries

Queries do not offer the visual representation that Charts and Reports can offer, butthey are probably the most powerful tool in the data analysis toolkit. You can extractany data you want with a query. They're easy to build, quick to run, and can provideinformation at any level.

Take the following steps to run some of the queries included with ClearQuestTestManager:

1. Collapse the TM Reports folder




2. Expand the TM Queries folder

3. Double-click Configured Test Cases - Verdict History to run that query.

4. Select Asset Registry Auction.

5. Click OK to run the report.

Figure 19. Configured Test Cases - Verdict History query

The Configured Test Cases - Verdict History query provides us with a listing ofevery execution verdict of every configured test case. You can use this query toinvestigate whether a given configured test case has a history of failing.

Create your own query

It is quite likely that you are going to want and need to create your own queries toanalyze your data. Imagine a scenario where your project lead comes to you andasks for a list of all of the test cases that are currently failing. How would yourespond? ClearQuest has all the information, but how would you get it out?

Take the following steps to create a custom query to show you which configured testcases are failing.

1. Right-click the Personal Queries folder of the jan, Tutorial@AUCTconnection, and then select New Query.

2. Provide the query name: Failing Tests.

3. Select the record type TMConfiguredTestCase to run a query containinginformation on your configured test cases.




4. You must first provide your query filter to select only configured test casesthat failed the most recent execution. To do this, you'll need to filter theTest Log Verdict and the Test Log Latest fields. First, expandTestLogs branch of the TMConfiguredTestCase folder, and thendouble-click to select both the Latest and Verdict fields.

5. Click Next to specify the values to filter for these fields.

6. Expand the And filter on the right, and then select the TestLogs Verdictfield. In the Define Filters box, make sure that the operator is Equal.

7. Click Values, and then select Fail.

8. Click OK to close the dialog.

9. Select the TestLogs.Latest filter, and again ensure that the operator isset to Equal.

10. In the Values field, type True. Setting these two filters means that youwill see only records for configured test cases that failed the most recenttest.

11. Click Next to select the fields to display as part of the query.

12. Double-click the ID field to add it to the display list. In the sorting column,select Ascending.

13. Double-click the Headline field to add it to the display list.

14. Drill down, and then double-click the TestLogs > Verdict field to add it tothe display list.

15. Click Finish.

16. Your query is now complete and will display in your personal queriesfolder.

17. Double-click the query to see the results (also Figure 20).

Figure 20. Query for Failing Configured Test Cases




Your custom query shows a list of all of the configured test cases with fail as themost recent verdictl. You can double-click any of these records to read the details ofthe record.

Examine custom queries included with ClearQuest

You will notice that there are several queries in the Personal Queries folder. Theseare all custom-designed queries that testers in the IBM Rational lab built for you toexamine and try. Many of these queries contain information from different types ofrecords. In the Integrated Requirement-Defect-TestCase Queries folder, as anexample, you will find records that relate test cases to requirements and then todefects. Use, examine, and learn from these queries to see how ClearQuest canprovide you with all of the information that you need to test the quality of yoursoftware and track your testing projects.

Section 8. Recap and Next Steps

You covered a lot of ground in this tutorial. Starting from nothing, you created a testplan and test cases. You created configurations and iterations, learned how to run aClearQuest test, and imported test case results. The goal was to get you to a pointwhere you can turn all of the data about your testing project into useful information.You did this in the analysis phase by examining the queries, charts, and reports thatClearQuest provides. Then you created queries to extract information specific toyour needs.

You have just completed ClearQuest TestManager basic training, yet you've onlyscratched the surface. You have not touched on eSignatures, audit trails, norsharing data with remote sites by using ClearQuest MultiSite and ClearQuest Web.You haven't customized any fields, forms, or actions. Whenever you're ready orneed more information, there is more to learn.

That's the beauty of IBM Rational ClearQuest. For the novice, you can use theconfiguration provided to get up and running in an hour. For those who want tocustomize the many options of ClearQuest, the flexibility is unlimited.




Downloads

Description Name Size Download method

Classics Online Sample Application ClassicsOnlineSampleApp.zip274 KB FTP

Sample tests and results CQTMTestScripts.zip5.9 MB FTP

Information about download methods

About the author

Brian BrysonBrian Bryson joined Rational Software in 1995 with a strong background in QualityAssurance. Currently, he is a member of the product team that plans and delivers theIBM Rational automated toolset. A former instructor at Carleton University in Ottawa,Canada, Mr. Bryson has taught the Rational approach to quality automation tocollege students as well as employees of organizations such as Microsoft, SunMicrosystems, and Oracle.



ftp://www6.software.ibm.com/software/developer/library/rational/zip/ClassicsOnlineSampleApp.zip

ftp://www6.software.ibm.com/software/developer/library/rational/zip/CQTMTestScripts.zip

http://www.ibm.com/developerworks/library/whichmethod.html


run and analyze software tests with rational clearquest test

Documents