load tests 101! why and where to start?

(RE)LOAD TESTS 101

WHY AND WHERE TO START?

AURÉLIEN DELEUSIÈRETHE ONLINE COLDFUSION MEETUP

(c)2016 A. Deleusière

WHO AM I?

• Software engineer specializing in web technologies, consulting, development, architecture and troubleshooting

• Founder of Webellian based in France and Poland

• ColdFusion since 1996, Java, PHP (mainly for CMS), AngularJs, ExtJS…


WHY LOAD TESTING?

Load Testing

Performance

Resources

Scalability

Robustness


WHERE TO START?

• Depends on the architecture!

• Two approaches:

• Define the limit of a system in the absolute

• Validate the capability of a system to resist to an estimated traffic, anticipate spikes link to special events


SCENARIO “CALIBRATION”

• have a picture of the “background noise” of the scope tested

• have the best response time to calibrate results analysis


SCENARIO “HEALTHINESS”

• have a picture of the profile of resources consumption (JVM Memory and Garbage Collection, System CPU, I/O)

• validate the healthiness of the component before Stress Scenario


SCENARIO “STRESS”

• determine the maximum load capacity (MAX) where the target conditions are reach in term of response time and stability

• determine acceptable response time variation (in percentile)MAX


SCENARIO “ENDURANCE”

• validate the maximum capacity determined during Stress Scenario

• validate the healthiness of the component under target load


SCENARIO “BURST”

• validate the capacity of the component to support a sudden high pressure

• this case of sudden high pressure can happen when you put a server back in the pool, or when a special campaign is starting


ABOUT GATLING

• Very high performance testing Engine

• 1 thread ≠ 1 request

• Distributed multi-nodes injectors

• Scala descripted scenarios (≈ DSL)


ABOUT GATLING

Question: How much performant?

How many requests per second with a single injector (16 vcpu on AWS)?


A SIMPLE EXAMPLE

• Produce a chain “ABCDEFGHIJKLMNOPQRSTUVWXYZ”

• By concatenating ASCII characters from 65 to 90

• Tests done locally:

• MacBook Pro running Virtual Box guest Ubuntu

• ColdFusion 2016u3 (2016.0.03.300466) Developper Edition

• Lucee 5.0.0.254


REAL EXAMPLE: A QUOTATION SERVICE

• Service Layer is composed by several Component (S1, S2

…) behind a Load Balancer (LB) in a stateless fashion.

• The Service Layer is behind a Gateway in charge of Authentication, Transformation, Routing and Logging that is also composed by several instances (G1, G2) behind a Load Balancer.

• No database, configuration in memory from configuration (XML, DSL) files.

• What to be tested?...

Serv

ices

LB

Gate

way G3 G2

LB

TRAFFIC

Datacenter 1 Datacenter 2

S1 S3 S5 S7 S2 S4 S6 S8

G1 G4


Agents


• Isolate a representative architecture to be tested…

• …in an environment identical to production

• Install the load tests Agent as close as possible to the component to be tested

• Install tools to monitor the resources (JVM memory details, GC, System CPU) like Fusion Reactor or Mission Control

• But you can also do test as a “black box”

Services S1 S2

LB

Gateway G1 G2

LB

TRAFFIC

A1 A2


FUSION REACTOR


MISSION CONTROL



The Load Tests are run against one single component to measure the intrinsic capability of one single instance.

Main objectives:• Measure the maximum capacity of one single component

• Measure the response time at component level

Services S1 S2

LB

Gateway G1 G2

LB

TRAFFIC

A



The Load Tests are run against a couple of component behind the Load Balancer.

Main objectives:• Determine the capacity factor between 1 instance and 2

instance that is generally is not linear

• Measure the response time to evaluate the latency introduced by the network devices

Services S1 S2

LB

Gateway G1 G2

LB

TRAFFIC

B



The Load Tests are run against the full Service layer through one single Gateway instance.

Main objectives:

• Measure the capacity of one single Gateway instance

• Measure the response time to evaluate the latency introduced by one Gateway instance

Services S1 S2

LB

Gateway G1 G2

LB

TRAFFIC

C



The Load Tests are run against the full stack.

Main objectives:• measure the response time with the full stack to validate the

order of magnitude for SLA

If possible validate this result with the full stack.

Services S1 S2

LB

Gateway G1 G2

LB

TRAFFIC

D


INITIAL TEST CAMPAIGN


CHRISTMAS 2016 SPIKE CAMPAIGN


1. GATHER EXISTING DATA

Experience from 2015:

• brutal ramp up at 6:00am from almost nothing to 2000 rq/min;

• peak around 7:00am around 1850 rq/min;

• plateau between 9:00am and 10:30am around 1500 rq/min;

• then 30% to 40% of sales more than usual during the next weeks.

Let’s focus on the 2000 rq/min

ILLUSTRATIVE


2. CHALLENGE BUSINESS EXPECTATIONS

Based on the current trends, the business expects an increase of 30%.

Let’s challenge our system with:• 2600 rq/min• A brutal ramp up from 0


3. FORECAST YOUR TRAFFIC

3. 2600 rq / min => 44 rq/sWrong! You have to correct the linear projection

“Feel” the volatility of what is your case.Here for a starting spike it should quite stable => 2x

Let’s challenge our system with:• 88 rq/s• A brutal ramp up from 0


4. SIZE YOUR SYSTEM

Original system design

Serv

ices

LB

Gate

way G3 G2

LB

TRAFFIC

Datacenter 1 Datacenter 2

S1 S3 S5 S7 S2 S4 S6 S8

G1 G4


SERVICE CAPABILITY


SERVICE CAPABILITY

Service 80 rq/s per server


GATEWAY CAPABILITY


GATEWAY CAPABILITY

Gateway 25 rq/s per server



4. Size your system

Target 88 rq/s

Service: 80 rq/s x 8 = 640 rq/s OKGateway: 25 rq/s x 4 = 100 rq/s OK

But….you have to expect a disaster!Service: 80 rq/s x 4 = 320 rq/s OKGateway: 25 rq/s x 2 = 50 rq/s NOT OK!

Required action: add 4 new nodes in Gateway layer.

Serv

ices

LB

Gate

way G3

LB

TRAFFIC

Datacenter 1

S1 S3 S5 S7

G1 G2

Datacenter 2

S2 S4 S6 S8

G4


1. Gather existing data2. Challenge business expectations3. Forecast your traffic4. Size your system5. Seat back in comfort on D-Day and bring the popcorns



• Measured results



Estimated: 2600



Estimated: 88


WHEN LOAD TESTING?

• Usually done on Major version releases

• Often by a specialist consultant

• Should be integrated into your automated tests


SPECIAL THANKS

• Mohamadou Mballo, a genius developer that just felt in love with CFML

• Stéphane Landelle father of Gatling for his support and the goodies to give away

• Yamina, my supportive girlfriend, that would be upset if she’s not listed on this page…

…and all of you making CFML so special…


You liked it? Tweet it!@adeleusiere