pega bpm system z benchmark test results bpm system z... · how well does the pega bpm application...

IBM Global Services

© Copyright IBM Corporation 2011

Pega BPM System z Benchmark Test Results

March 1, 2011

IBM Global Services

© Copyright IBM Corporation 2011Pega BPM Benchmark Test Results | 1-March-20112

Document ContentsTest Objectives – Key Questions to be Answered

Success Criteria – What the Test Proves re Pega BPM on System z

Test Description – What Workload Scenarios were Executed

User Loading Histograms – Ramp Up, Steady State, Ramp Down

Overall Test Results – 100% Successful vs. Test Objectives

Performance Scalability Curves – Linear to Near Linear– Call Center (CC)– Customer Process Manager (CPM)– Hybrid (CPM-BPM) – Customer Process Manager and Native Pega BPM

Test Results Summary – Performance Metrics and Response Times

Appendices– Appendix A – Pega BPM Test Environment Deployment Topologies– Appendix B – Pega BPM Benchmark Software Stack

IBM Global Services


Test ObjectivesThe singular purpose of this benchmark test was to answer a straight-forward, but critical set of questions:

Using the IBM System z processor as the host, can Pega BPM and CPM be effectively deployed across an entire corporate enterprise – meaning that all user-facing applications (with a servicing population of approximately 10,000 users) could be replaced with a common Pega BPM desktop while maintaining adequate performance levels?

What are the System z memory and CPU resource requirements, as measured in Gartner MIPS, to adequately support the ensuing workload – consistent with specified workflows, transaction rates, process durations and end-user response times?

How well does the Pega BPM application scale in the IBM System z environment – will Pega BPM take advantage of z architecture and exhibit both horizontal and vertical scalability – and will the scalability be linear? What is the optimal configuration of Linux guests, WAS servers and JVMs to take advantage of the z architecture and scale?

IBM Global Services


Test Results SummaryPega BPM application has met or exceeded all performance goals and success criteria as set forth in the Benchmark test. It scales linearly both vertically within a single LPAR and horizontally across the four z10 LPARs. – Call Center scenario with 35,000 users required 0.82 MIPS per user and the end-to-

end browser response time was 0.832 seconds.– CPM scenario with 9,000 users required 3.24 MIPS per user and the end-to-end

browser response time was 1.12 seconds. – Hybrid CPM-BPM scenario with 12,000 users required 3.03 MIPS per user and the

end-to-end browser response time was 1.22 seconds against targets of 3 MIPS per user and a 2.0 second browser response time.

In order to better manage resources, the Benchmark test used dedicated CPs in each LPAR. The shared CP test resulted in 10-25% improvement in response time and 2% improvement in MIPS because this allowed the z10 hardware to optimize resources, both CPU and associated caches, across the processor complex.

The endurance tests for each workload (8-16 hours in length) showed the same performance numbers as the normal (1 hour) duration tests.

IBM Global Services


Success CriteriaTo produce linear or near-linear Performance Scalability Curves with respect to the MIPS required and work performed per unit of time within the load range supported by the z10 for the following scenarios– Call Center (CC) workload– Customer Process Manager (CPM) workload – Hybrid CPM-BPM. CPM-Industry Specific Framework and Pega BPM Native Application

To show that no more than 3 z/VM MIPS per user are required to support a steady state load of 12,000 simulated, concurrent users performing the prescribed Hybrid CPM-BPM transaction workload for the duration of the test.

To achieve an end-to-end browser response time of < 2 seconds. Given an average of 4 HTTP requests per interaction, the component JMeter HTTP response times must therefore be < 500 ms.

Technical Notes – The success criteria are stated in terms of MIPS/user instead of Transactions Per Second (TPS)

because the conditions which motivated this test were expressed in MIPS/user. – Given steady state conditions and linear scale, with all users continuously interacting with Pega

BPM, the MIPS required are directly proportional to TPS (the relationship between MIPS and TPS is a constant given a specific user workload)

– The MIP ratings reported in this test as well as those from the Poughkeepsie test are based on Gartner’s document titled, “MIP Rating for IBM’s System z10 EC Processors” (Document ID G00156987).

IBM Global Services


Test DescriptionThe benchmark test involved three separate business process workflows:– Pega BPM (PRPC Version 5.5 SP1) Call Center (CC) – a simple “Out of the Box”

Customer Service test application used for demos and benchmarks. Reps receive phone calls, access/update client/member data.

– Customer Process Manager 6.1 (CPM) – a complex “Out of the Box” generic Customer Service Framework which underlies the Industry Specific Framework used in this benchmark.

– Hybrid (CPM-BPM) – a combination of an industry-specific CPM framework and a native Pega BPM application.

For each workflow, between 5 and 7 test cases were run with increasing numbers of users to demonstrate vertical and horizontal scalability and to create a sufficient number of points needed to construct the performance scalability curves. For each flow, the system was loaded up to the point of “failure” in the last test case.

The base test cases were followed by additional tests in which an altered hardware (shared v dedicated CPs) configuration was used. In addition, there was one endurance test (8-16 hours long) performed for each workflow.

With the exception of the endurance tests, each test lasted between 40 and 100 minutes during which time the simulated users continuously executed assigned transactions. Calls were 4-8 mins long.

IBM Global Services


User Loading Histogram for Call Center

IBM Global Services


User Loading Histogram for CPM

IBM Global Services


User Loading Histogram for Hybrid CPM-BPM

IBM Global Services


Call Center Performance Scalability Curve

IBM Global Services


CPM Performance Scalability Curve

IBM Global Services


Hybrid CPM-BPM Performance Scalability Curve

IBM Global Services


Test Metrics Results (CC, CPM, CPM-BPM)

IBM Global Services


Appendix A1 – Pega BPM Benchmark Test Configuration

IBM Global Services


Appendix A2 – Pega BPM CC Workload Test Environment

IBM Global Services


Appendix A3 – Pega BPM CPM Workload Test Environment

IBM Global Services


Appendix A4 – CPM-Pega BPM Hybrid Test Environment

IBM Global Services


Appendix A5 – DASD Configuration Map

IBM Global Services


Appendix B – Pega BPM Benchmark Software Stack

pega bpm system z benchmark test results bpm system z... · how well does the pega bpm application...

Documents