Stéphane Landelle

CTOeBusiness Information

@slandelle

Performance & Load Testing

Concepts

Define performance?● fast?● robust?● resource effective?

=> Define your requirements!

Web speed is a user experience

● <0,5 s : loading looks instantaneous

● 0,5-2 s : not instantaneous, but acceptable

● > 3 s : users start leaving your site

Put things into perspective

Product Owner requirement : Every page should show up under 200ms

WRONG !

User expectations = per use case!

Speed matters

● More traffic

● Better conversion rate

● Less angry users

Some figures

Google :Results/page : 10 => 30 -> +500 ms -> -20% pages seen

Amazon :+100 ms -> -1 % salesestimated annual cost : $160M

Load tests : why ?

Load tests =

know your app & infrastructure

No load tests=

potential problems in production=

angry users

Methodology

Clicking everywhere/Selenium = not a load test !

● You expect more than 1 user

● "Seems fast enough." => Not a metric !

Methodology

Comes after proper local perf testing

● Client/Network optimization: minification, javascript, sprites, caching...○ Tools : Developer Tools, YSlow, Google PageSpeed

Insights...

● Application debugging: 50 SQL queries/page = BUG!○ Tools : VisualVM, Yourkit, JProfiler, MAT...

Methodology

● Analyze = Data = Monitoring

● Fix hotspot

● Iterate!

Tooling

● Load injector: Gatling, JMeter, Locust.io...

● JVM monitoring: JMXtrans, Yammer Metrics...

● System monitoring: Nagios

● Network monitoring

● Database monitoring

● Dashboard: Graphite, Ganglia

● Webapp mock: H. Gomez's basic perf webapp

Types of load tests

● Capacity Test

● Stress Test

● Endurance Test

Capacity Test

Goal : Determine how much load your system can hold

How :Repeat the same scenario over and over, but add virtual users every time until performance starts degrading.

A response time under 1 s is expected

-> user cap : 1500 users

Stress Test

Goal :Study system behavior in case of heavy load, during AND after

How : Find the max load your system can handle, and then run the scenario with a heavier load.

10k users for 1 min, then 1k : webapp struggles, but stabilizes

Endurance Test

Goal :Validate system behavior after a long period of activity.

How :Run the scenario with a manageable load, but for a long period of time (several hours at least).

Fast memory leak :Runs fine for 2 minutes until heap's full...

Ramps

Start virtual users progressively, because that's what real users do !Ramps also help to warm up your system.

Reports

The purpose of load injectors : stress your app and produces reports

● Meaningful reports help developers analyze stress tests results and what to make of them

● Something shiny to give to your boss

The good, the bad and the ugly metrics

Every metric can be useful, but some less than others...

● Response time min/max = worst case/best case

● Mean can be biased in case of extreme values

Response time is a physical phenomenon=> Statistically distributed

Percentiles to the rescue

nth percentiles =

n % of users' response time

Part of the development process

Like any other functional test, load testing should be :

● integrated early in the development cycle

● automated

● versioned

Gatling

Yet Another Stress Tool

JMeter, Grinder, Tsung, LoadUI,

LoadRunner, Neoload…

High PerformanceIssue #1

http://www.shopfbparts.com/catalog/nal-19201331_w.jpg

1 user = 1 thread

With 50 threads on a JVM

With 2000 threads on a JVM

Blocking I/O

Threads? Waiting…

… and sleeping

Is that a real problem?

Can you trust your results?

JMeter 2.8 perf test, expecting 300 tr/sec

UsabilityIssue #2

Listen, it's not that complicated...

Graphical User Interface

MaintainabilityIssue #3

What was this change about?

Gatling can change all that!

http://static.lexpress.fr/medias/15/mai-68_124.jpg, copyright by AFP

Version 1.4.1Released January 2013

Say hello to my little friend…

Be asynchronous, embrace the actor model

Use non-blocking I/O

● Async HTTP Client● Netty

Scenario = Code (Scala) = DSL

Easy

Use the rich DSL…Checks

● regex / css / xpath / jsonPath

● find / findAll / count

● is / in / not / whatever

Structures

● doIf / repeat / during / asLongAs

● randomSwitch / roundRobinSwitch

Error handling

● tryMax / exitBlockOnFail

Feeders

● csv / tsv / jdbc

… or write your own Scala code…

…or use the Recorder

Integrations● Maven Plugin● Maven archetype (run in IDE)

● Jenkins plugin

● Graphite live reporting

Coming soon…

● Websockets, JDBC…● Clustering

Demo

Gatling at Ezakus

Ezakus Architecture

Metrics

110 M Http hits by day

Peak : 3 000 req/s

Gatling usage - Simulations

Gatling usage - Example

Gatling usage - Results

Gatling usage - Results

Conclusion

Really efficient?

Jmeter perf test run with Gatling, expecting 300

tr/sec

http://gatling-tool.orghttps://github.com/excilys/gatling@GatlingTool

https://github.com/slandelle@slandelle