Cost Effective WebApplication Load Testing

Presented by:Scott Tiedemann

Project Manager / Lead DeveloperLBi Software

July 20, 2011 1

Agenda

• Introduction

• What is Load Testing

• Load Testing Goals

• Load Testing Tools

• Tool Demonstration

• Summary

• LBi Overview

• Q&A

July 20, 20112

What is Load Testing?

• Load testing is the process of putting demandon an application and measuring its response

• Models the expected usage of a softwareapplication by simulating multiple userslogging on concurrently

• Primary goal is to define the maximumamount of work a system can handle withoutsignificant performance degradation

July 20, 20113

Is it the same as Stress Testing?

• Stress testing is used to evaluate the extent towhich a system keeps working when subjectedto extreme work loads or when some of itshardware or software has been compromised

July 20, 20114

Why Load Test?

• Critical for multi-user systems such as webapplications

– Must determine if system can handle theexpected user base

• Tune hardware sizing and configuration

• Identify deficiencies in software architecture

July 20, 20115

Issues you may uncover• Hardware Sizing

– CPU/Memory Requirements– Internet Connection Speed– Database Size

• Hardware Configuration– Web Server Request Limits– Thread Counts– Database Connections– Record Locking

• Software Issues– Deadlock– Synchronizations and Concurrency

July 20, 20116

What are you measuring?

• Page Load Times

• Number of Concurrent Users

• Transactions Completed within Timeframe

July 20, 20117

Analyzing Results

• What do the results really mean?

– Are findings acceptable, but not optimal?

– Were test parameters appropriate? Should theybe tweaked?

– Are application or hardware changes necessary?

July 20, 20118

Automated Vs. Manual Testing

• Manual testing rarely simulates actual load

– Hard to get enough users on the systemperforming actions simultaneously

• Automated testing enables simulations with awide range of parameters

– Number of users

– Ramp up time

– Time in between requests

July 20, 20119

Examples of Load Testing Tools

• Enterprise Level– WebKing (Parasoft)– LoadRunner (HP)– Neoload (Neotys)

• Free / Open Source– JMeter (Apache)– Eclipse TPTP (Test & Performance Tools Platform)– TestMaker (PushToTest)– The Grinder

• Other– BrowserMob

July 20, 201110

Enterprise Tools• Pros

– Robust test management and execution platforms

– Leverage functional/regression tests in loadtesting

– License usually includes support

• Cons– Can be expensive

– Hardware requirements can be high

– Load testing not always primary application focus

July 20, 201111

Open Source Tools

• Pros

– Low Cost or Free

– Generally Lightweight

– Targeted to Load Testing

• Cons

– Almost always require technical know-how to use

– Support, if available, is not free

July 20, 201112

Implementing the Right Tool

• Up front time evaluating the options will payoff in the long run, as you avoid choosing atool that isn’t a good fit

• Factor in:– Cost

– Resources available

– Hardware available

– Number of applications to be tested

– Complexity of applications to be tested

July 20, 201113

Scaling Up• Initial tests should simulate minimal load;

main purpose to verify testing tool isperforming as expected

• Once verified, ramp up testing untilapplication fails

• If application does not fail at the maximumload testing capacity, scale up the load testing– Run load testing on more powerful hardware

– Create multiple “nodes” of the load tester

July 20, 201114

Use of the Cloud in Load Testing• Often times, the hardware required by the load

testing software itself can be expensive

• If in-house servers cannot handle the load, use ofcloud based services can provide the extracomputing power necessary at a fraction of thecost of actually purchasing the hardware

• Cloud servers generally scale; can start out withlower powered machines and scale up as testsare successful

July 20, 201115

What is Cost Effective?

• Largely dependent on scale of project

– Enterprise solutions may be worth the price onlarge applications

– More often, however, a low cost tool will providethe necessary results

• Will tool be used on other initiatives?

– Is this a “one off”, or can the test suite be used totest multiple applications

July 20, 201116

Load Test Before Release!• Often times, a system is not load tested until

after it is in production and problems havebegun cropping up

• Changes at this stage are more costly thanchanges during the development cycle

• Performance problems shortly after go-livecripple user confidence in the system, whichcan make it hard to gain user acceptance

July 20, 201117

Potential Issues

• Creating and executing tests can requiretechnical resources, particularly on the opensource solutions

• “AJAXified” applications can be challenging toscript for

• Maintenance burden

• Often requires testing in production, or oncomparable hardware, to avoid misleadingresults

July 20, 201118

JMeter Demo

July 20, 201119

Summary• What is “Cost Effective” varies from

organization to organization

• Choose a tool which is right for yourorganization and your applications

• Reusing the tool across multiple applicationslowers the effective cost

• If necessary, base tool choice on resourcesavailable to implement and perform testing

July 20, 201120

LBi Software• Established 1982

• Primary Focus on Human Capital Management(HCM) Solutions

• Legacy in custom HCM development

• Custom & Packaged solutions

• Web 2.0 Development expertise

• IVR/CTI/Speech Recognition expertise

• Mission/Business Critical solution delivery

July 20, 201121

LBi Software Products

July 20, 201122

LBi SoftwareDevelopment Support Services

• Source Control

• Issue Tracking

• Automated Testing

• Load Testing

• Development Environment

• Application Servers

July 20, 201123

Q & A

July 20, 201124

Thank You!

July 20, 201125

www.LBisoftware.com