ibm rational software quality solutions · pdf fileibm rational software quality solutions ......

®

IBM Software Group

© 2006 IBM Corporation

IBM Rational Software Quality Solutions- IBM Rational Performance Tester

Denice WongTechnical Consultant

Rational Software, IBM Hong Kong

IBM Software Group | Rational software

2

Agenda

� IBM Rational Software Quality Solution Overview

� IBM Rational Performance Testing Solution

�IBM Rational Performance Tester


3

Business Driven DevelopmentSoftware quality best practices

Leverage open standards

Support testing of modular systems

Govern and control testing to ensure compliance

Integrate quality into the life cycle

Focus on quality early

Best Practices


4

Focus on quality earlyLate defect discovery exponentially increases repair costs

� An IBM & Gartner “rule of thumb” for the relative costs to fix defects

1 AC Market Analysis: Self-healing market June 20042 Source: Tivoli Marketplace Assessment, February 2004

Model the Business Define

Requirements

Analyze& Design

Implement

TestDeploy

Manage

Optimize

1.0

1.5

60

100

10

Lost revenue for each hour of downtime

ranges from $89K to $4.5M1

Application maintenance cost is

18.5% of total I/T spending2


5

Integrated quality capabilities

FUNCTIONAL & SYSTEM TEST

PERFORMANCE TEST

DEPLOY & MANAGE

ANALYZE DESIGN VALIDATION

OPTIMIZE

UNIT TESTING

BUILD TESTING


6

Govern and control testing

GOVERN TEST LIFE-

CYCLE

FUNCTIONAL & SYSTEM TEST

PERFORMANCE TEST

DEPLOY & MANAGE

ANALYZE DESIGN VALIDATION

OPTIMIZE

UNIT TESTING

BUILD TESTING


7

Rational Software Development Platform

� IBM Rational Performance Tester

� IBM Rational Functional Tester� IBM Rational Manual Tester� IBM Rational PurifyPlus� IBM Rational Robot� IBM Test RealTime

Software quality

� IBM Rational ClearCase®

� IBM Rational ClearQuest®

� IBM Rational Team Unifying Platform™

� IBM Tivoli Provisioning Manager

� IBM Tivoli Configuration Manager

� IBM Tivoli Intelligent Orchestrator

� IBM Rational Software Architect

� IBM Rational Software Modeler

� IBM Rational Data Architect

� IBM Rational Application Developer

� IBM Rational Web Developer

� IBM WebSphere®

Business Modeler

� IBM Rational RequisitePro®

� IBM Rational Software Architect

� IBM Rational Software Modeler

� IBM Rational Rose®

Data Modeler

Design & construction

Change & configuration management

Process & portfolio management

� Best practices content (IBM Rational Unified Process®

IBM Tivoli Unified Process®, Portfolio Management)� IBM Rational® Portfolio Manager� IBM Rational Method Composer

Partner ecosystem & open computing: Eclipse, Linux®, Microsoft® Windows®, UNIX®, IBM z/OS®

Solutions for geographically distributed development, compliance, SOA

Requirements & analysis

GOVERNANCE DASHBOARD


8

Rational Software Quality Solutions

Developer Test Functional Test

Automated Manual

Requirements Test Change Defects

� Govern testing� Trace to business needs� Audit progress/change� Monitor quality

� Run-time analysis� Profile memory� Code coverage� Component & services

testing

� Automated test creation

� Find regression defects

� Early business process tests

� Modular components

� Capture requirements� Document use cases� Monitor change

Project Dashboards Detailed Test Results Quality Reports

Performance Test

� Model users� Simulate workload� Find bottlenecks� Pinpoint production

problems

� Track defects to resolution

� Trace to business needs

� Manage code change� Associate with builds� Deploy to servers

SOFTWARE QUALITY SOLUTIONS

Test and Change Management

Test Automation

Quality Metrics

DE

VE

LO

PM

EN

T

OP

ER

AT

OIN

S

BUSINESS

Security Test

AppScan

WebXM


9

Rational Software Quality Solutions

Developer Test Functional Test

Automated Manual

Rational RequisitePro Rational ClearQuest Rational ClearQuest

Defects

Project Dashboards Detailed Test Results Quality Reports

Performance Test

SOFTWARE QUALITY SOLUTIONS

Test and Change Management

Test Automation

Quality Metrics

DE

VE

LO

PM

EN

T

OP

ER

AT

OIN

S

BUSINESS

Rational ClearQuest

Requirements Test Change

Rational PurifyPlus

Rational Test RealTime

Rational Functional Tester Plus

Rational Functional Tester

Rational Robot

Rational Manual Tester

Rational Performance Tester

Security and Compliance Test

AppScan

WebXM


10

Agenda

� IBM Rational Software Quality Solution Overview

� IBM Rational Performance Testing Solution

�IBM Rational Performance Tester


11

What Is Automated Performance Testing?

� The process of exercising an application by emulating actual users with a load generation tool for the purpose of finding system bottlenecks

Controller

System Under Test


12

Why do Performance Testing?

� Because a break at any point in your system means your customers are not getting the service you think they are

System Under Test


13

Challenges of Performance TestingChallenges of Performance Testing

� Complex Tools & Complex Task

�Tool complexity and lack of experience intimidates many first time users



“We don’t have a lot of experience building

good load tests.”

“Most load test tools are much more

complicated than we require.”


14




� Testing Tools Lack Insight

�Tool can find the problem, but not diagnose the root cause





“We don’t have a good understanding of what is causing the problem”

“I need to know more than simply where a

problem is occurring.”


15






� Hidden Costs

�Hardware intensive tools and inability to share information with team mates increases cost of testing





� Hidden Costs

�Hardware intensive tools and inability to share information with team mates increases cost of testing

“Our developers have no way of performing

load tests.”

“We don’t have enough hardware to run our

tests.”


16

� IBM Rational Performance Tester

�Performance Testing for Web Based Applications

� Key Benefits

�Built for Day 1 Productivity

� Mask complexity to get the job done

�Advanced Data Access & Manipulation

� Automated data variation and synchronization

�Root Cause Analysis

� Identifies location and root cause of performance problem in hardware and software

Performance Testing with IBM Rational Performance TesterTest automation for the novice and the professional


17

Creating a Performance TestCreating a performance test is a three step process

� Script Creation Considerations

�Visual test editor, varying input data & correlating server responses

Build Scripts


18

No-Code Tests

� Tests are represented in a tree view as a list of web pages and their elements, such as images.

� No programming necessary to create a test

� Underlying detail is accessible but hidden to simplify test creation and maintenance


19

Visual Test Editor

Test Navigator

Access to Tests and Schedules

Test

Organized as tree view of access

pages. No code

Additional Info

View detailed info about requests and

responses

Organization

Top level branches are pages. First branch under the page is the html container. Under that are all the page elements, such as images, cascading style sheets or java

scripts


20

Visual Test Editor

� Enhance tests without programming

�Loops

�Conditions

�Verification Points

� Validate server response code

� Validate server response size

� Power User Features

�Manually add HTTP requests

�Add custom Java code

Tests not Scripts

With Atlantic, we are no longer using the word “script”, instead,

the tool now creates “tests”.


21

Creating a Performance TestBuilding Scripts

� Automated creation of parameterized tests

�Literal values used during recording are replaced with variables

�At playback time, variables are populated with data from a datapool

�Data for playback can be imported from external data sources

�No user coding involved

User Name: JSmith

User Name: CBryson

User Name: TJones

Session No. 100

Session No. 101

Session No. 102


22

Automatic Identification of Variable Data

� Automatic server response correlation

� Items in purple indicate that RPT has automatically correlated a piece of data unique to this run

� In the example on the left three values have been correlated: custID, orderIDand SessionID. Lets focus on orderID:

� orderID is a unique order number that is generated for every purchase This value cannot be hardcoded in the test – and is not. RPT will automatically use the order ID that is generated by the web site for every playback.

Data correlation

The automatic process of flagging and parameterizing data that is

unique for every test run.


23

Built-in Datapool Editor for Varying Test Data

� RPT Automatically identifies candidates for datapooling

� All data entered by the user is a candidate for datapooling

� The next 6 slides walk through the process of datapooling values


24



� Scheduling Considerations

�Accurately representing a true user workload

Schedule WorkloadBuild Scripts



25

LoadSmart schedulingEasy to use

� Model complexreal world scenarios

� Point and clickscheduling


26

Creating a Performance TestSchedule Workload

� Powerful and flexible scheduling

�Visual schedule editor for no code scheduling

�Coordinate timing and dependencies

�Accurately model real users workloads

�Dynamically increase the load during the run


27

Execute & AnalyzeSchedule WorkloadBuild Scripts



� Scheduling Considerations

�Accurately representing a true user workload

� Execute and Analyze Considerations

�Validating responses & finding the bottleneck



28

Root Cause Analysis

� Root Cause Analysis is a collection of features which provide the user with information to diagnose the cause of their performance problems

� Root Cause Analysis collectively refers to the following functionality

� Problem Analysis Tools

� Technology formerly bundled as the IBM Performance Optimization Toolkit

� Response Time Breakdown

� Agentless Resource Monitoring

� Report Data Overlay

� Report Filtering by Time


29

Performance Problem Identification During Test

� Rational Performance Tester easily identifies bottlenecks

�Next logical question is Why?

�Root Cause Analysis provides to tools to answer this question

Page Performance ReportShows average response time

per page.

Highest bar = Performance Problem


30

Performance Problem Identification During Test

� Rational Performance Tester easily identifies bottlenecks

�Next logical question is Why?

�Root Cause Analysis provides to tools to answer this question


31

Live Response Data CollectionProblem Diagnostic data capture with the Data Collection Infrastructure

� Primary Use Case:

� Performance test is run with problem analysis features enabled capturing live diagnostic data via DCI

� Data collection infrastructure

� Pre-instruments WebSphere and Weblogic servers to capture class and method level information

� Is run on all tiers of the application

� Uses Application Response Measurement (ARM) technology and Java JVMPI Profiler Interface to capture data

� Configuration Process

� Install on all tiers of application (Windows, Linux Redhat/SuSE only)

� Run Application Server Instrumenter to instrument WebSphere and WebLogicServers

� Start DCI via Start Menu

� Communicates over port 10002

Application Server Instrumenter

GUI Interface to ARM instrument WebSphere and Weblogic servers


32

Problem Analysis Tools

� On schedule, specify tests for which to capture data

� Specify detail level and sample size

� Best Practice: Minimize superfluous data

� Capture problem diagnosis data only for relevant pages

On schedule, specify tests for which to

capture additional data


33

Problem Analysis Tools – Page Level Diagnostics

� Feature:

� Response Time Breakdown can be done at schedule level or at page level

� Benefit:

� Page level granularity enables you to filter out irrelevant information to your investigation

On test, specify pages for which to capture additional data


34

Response Time Breakdown

� Feature:

� Breakdown page response times into composite element response time

� Benefit:

� Breakdown data drills down to highlight slowest components of a given page. Breakdown into tiers and components is a unique competitive advantage for RPT.

Response data broken down by tier and by transaction component (JDBC, JSP,

Servlet, etc…)


35

Deep Diagnostic DataProviding the details you need to solve the performance problem mystery

UML Sequence diagram showing method calls

between classes with timing information. User can drill

down to source code.

Execution Statistics Shows response time for

every method


36

Resouce Monitoring Configuration on Schedule

� Agentless monitoring of local or remote Windows and Unix machines

� Unix requires rstatd daemon to be running

� Windows requires File Sharing enabled and a connection established with an account with Administrator access

� Minimum requirement is remote registry monitoring


37

Agentless Resource Monitoring� Feature

� During a performance test run, RPT can capture resource monitoring information from

� IBM Tivoli Monitoring

� UNIX rstatd monitor

� Windows Performance Monitor

� Does not require any installation on any tier of the application

� Subset of default counters selected to monitor key resources

� Benefit

� Provides additional data during results analysis to identify root cause of performance problems


38

Resource Monitoring DataIncorporate resource data into response data

� Tech Tip

�Ensure time clocks are synchronized across machines


39

Performance & Resource Statistic Report Overlay

� Data from different graphs can easily be consolidated on to one graph.

� Includes resource monitoring data


40

Report Filtering by Time Range

� Feature:

� Filter results on any report to focuson a given time range

� Benefit:

� For long runs, enables focused analysis on subset of data most interesting to customer


41

Ensure Application Scalability and Performance IBM Rational Performance Tester Extension for Siebel

� Improve capacity planning which can lead to lower hardware costs

� Simulate multiple concurrent user types and business transactions

� Generate work-loads that stress all points in the architecture

� Isolate the source of performance bottlenecks and optimize resource utilization before deployment

� Siebel Validation ensures technical readiness

Web ServerWeb Server

OLTPOLTP

OLTPOLTP OLTPOLTP


42

Ensure Application Scalability and Performance IBM Rational Performance Tester Extension forCitrix Presentation Server

� Reduce the cost and time associated with performance testing with an easy to use, "no-code" test recording and execution solution

� Test system scalability and performance by emulating users with varying business transactions, usage patterns and work-loads

� Perform pre-deployment capacity planning tests to ensure efficient allocation of server resources

Citrix Presentation Citrix Presentation

ServerServer


43

Performance Testing of SAP Applications� The process of testing SAP applications

� Follows the same steps as testing web based applications

� Record

� Edit

� Execute & measure response times

� But requires new technology to enable the process

� SAP Recorder

� SAP Protocol Browser for test editing

� SAP Performance Reports

� SAP HTTP Interface now supported

� SAP 4.6, 4.7 and 5.0 offer an HTTP interface to SAP.

� Visual Design is a feature of SAPGUI which allow the customer to modify the colors & fonts of SAPGUI


44

Service Quality Management Functional and Performance Testing of Web Services from a common interface

Rational Tester for SOA Quality

Automated regression and functional testing for GUI-less Web services

Rational Performance Tester

Extension for SOA Quality

Performance Testing for Web Service based applications


45

ibm rational software quality solutions · pdf fileibm rational software quality solutions ......

Documents