with apache jmeter? frank pientka · why performance testing? testing right! ... jmeter distributed...

Performance testing

With Apache JMeter? Frank Pientka

© Materna GmbH 2016 www.materna.com

Where and who is Materna GmbH?

2© Materna GmbH 2016

Founded: 1980Employees: 1,700Turnover 2015: € 210 million

ITK-Full-service provider in the premium segment

Family-owned company in the ITC sector

Target groups: IT organisations and user departments in both private and public sectors

Dortmund

Düsseldorf

Bad Vilbel

Stuttgart

Erlangen

München

Dresden

Bremen Hamburg

BerlinWolfsburg

Lüneburg

Köln

Nürnberg

Heidelberg

FI

SE

DK

PL

SK

UKNL

DECZ

NOR

Orlando, Florida, USA


Who is Frank Pientka?

Dipl.-Informatiker (TH Karlsruhe)

Principal Software Architect at Dortmund

iSAQB founding member

heise.de/developer/Federlesen columne

over 20 years IT experiencelots of articles and talks


Agenda

Why performance testing? Testing right! What is Apache JMeter? What’s new in Jmeter 3.0? JMeter main elements? Have a (test-)plan! Distributed performance testing Further steps, extensions, challenges Resume

4


Why performance tests?

5


What went wrong while testing?

How secure&safe?

The Heisenbug!

Volkswagen St. (VW) share

6


Java performance testing tools (selection)

JUnitPerf ab Gattling

7


May I introduce Apache JMeter? 18 years of experience

Since 1998 for Apache Tomcat loadtest (Stefano Mazzocchi) Recent version 3.0 (2016-05-17) Supported protocols

Web - HTTP, HTTPS, FTP SOAP/XML-RPC Database via JDBC, MongoDB, HADOOP LDAP JMS Mail - SMTP(S), POP3(S) and IMAP(S) Native commands or shell scripts

+ Highly extensible and configurable, tutorial, example templates, demos- Limited reporting capacities, Short outdated documentation, i18n

8


https://jmeter-plugins.org/stats/

9

© Materna GmbH 2016 www.materna.com 10

Apache JMeter 3.0 Changes http://jmeter.apache.org/changes.html

Test plan creation and debugging improvements Reporting improvements (dynamic HTML-Dashboard) Protocols and Load Testing improvements UX Improvements Core improvements Components updates Incompatible changes, deprecations


A JMeter test plan example

11


How does Apache JMeter simulate a single user?

http://jmeter-expert.blogspot.de

12


Typical steps

Create Testplan Test Testplan Run Testplan and evaluate results Document Testplan and results

Extras/schematic.cmd Testplan.jmx Testplan.html

TestPlanCheck --jmx <filename> [ --stats --tree-dump ]

jmeter -n -t <test JMX file> -l <test log file> -e -o <Path to output folder>


How does Apache JMeter simulate multiple users?


14


The main elements of a JMeter testplan


15

© Materna GmbH 2016 www.materna.com16

JMeter testplan elements

Workbench• Test Recorder

ConfigTestElement• Request Defaults• Autorisation Manager• Cookie Manager

Sampler• HTTP Request

Timer

Assertion

Listener• SummaryReport• Results Tree• Results Table

Start


Login testing scenario

17

Thread GroupThread: 1000Ramp-up: 500

ConstantTimer300ms

HTTP sampler GETHostname/urlStart

End

AssertiomCheck combination

ist OK

RegExHTTP Status code

HTTP sampler POSTHostname/url

J_username=$userJ_password=$pwd

ConstantTimer300ms

HTTP sampler GETlogout

AssertiomCheck login

string


Example: Tomcat admin uses manager app

18


Example: Tomcat manager app JMeter testplan (skeleton & meat)

19


JMeter test plan recorder (HTTP Proxy Server) elements

21

Alternative: BlazeMeter's Google Chrome extension, .har file to a JMeter test plan (.jmx file) converter


Tips&Tricks

Use user defined variables with same name as in request Place Login CSV Data Set Config element on login page request Use Counter element and reference {counter} variable in request i.e. {user} Use non-GUI mode: jmeter -n -t test.jmx -l test.jtl Use as few Listeners as possible e.g. disable the “View Result Tree” Use CSV output rather than XML. Only save the data that you need. Use as few Assertions as possible. Disable all JMeter graphs as they consume a lot of memory Check, document testplanTestPlanCheck': --jmx <filename> [ --stats --tree-dump ]


Four Ways To Launch a JMeter Test without JMeter GUI

1. Command line: jmeter -n -t your_script.jmx2. JMeter Ant Task ant extras/build.xml3. JMeter Maven Plugin. pom.xml mvn verify jmeter:jmeter4. from Java code


JMeter distributed testing with Master-Slave

SystemunderTest

Slave1

Slave1

Slave3

Master-GUIjmeter.propertiesremote_hosts=

Slave1,Slave2,Slave3

Remote start Slave1

Remote start Slave3

Remote start All

In bin/jmeter.properties "remote_hosts=localhost:1099,localhost:2010" add IP addresses of Slave-Clients, server.rmi.port=1099, server.rmi.localport=1099, client.rmi.localport=2010 On Master: jmeter -n -t script.jmx -R slave1,slave2,slave3.„Remote Start All" distribute and execute testplan and receive results, SET SERVER_PORT=1099, set REMOTE_HOSTS=master:1099"Remote Stop“, control jmeter-server.log, and have check and assert elements, to handle errors

Remote stop

26


Distributed testing

27

EC2, VPNSSH-Tunneling

Running Apache JMeter on Amazon EC2ec2 Script


How to document test plans in HTML?

Bildschirm als Bild speicherncd apache-jmeter-3.0\extrasschematic.cmd test-plan.jmxant -Dtest=test-plan run reportTestPlanCheck.bat --jmx testplan.jmx --stats --tree-dump

Cold/warm start!Think Time!

28


Apache JMeter Dashboard 3.0

29

http://jmeter.apache.org/usermanual/generating-dashboard.html

Save JPG.image


Report Dashboard with APDEX (Application Performance Index) table

30

https://www.blazemeter.com/blog/5-key-things-you-need-know-about-jmeter-30jmeter.bat -n -t WorkBench_JMETER.jmx -l WorkBench_JMETER.log -e -o reporthttp://www.apdex.org


https://sense.blazemeter.com


Realtime results: Async BackendListener with Graphite/Grafana

32


https://flood.io/har2jmx

33


Continuous integration

34https://wiki.jenkins-ci.org/display/JENKINS/Performance+Plugin


Useful extensions http://jmeter-plugins.org

35


Further challenges with JMeter

How to integrate JMeter in the CI-Build? How to document test plans and results? What further reporting tools are needed? Extending the Web Test Plan (ForEach Controller with User Defined

Variables (UDV), While Controller with StringFromFile Function, RegEx) How to use JMeter with Selenium/JS/mobile Apps/Cloud?

36


Further info

Gatling http://gatling.io Grinder3: http://grinder.sourceforge.net JMeter Info's: http://jmeter.apache.org, http://sourceforge.net/projects/jmeterforwindows JMeter extensions: http://jmeter-plugins.org, http://www.atlantbh.com/jmeter-components JMeter Tips: http://blazemeter.com/apache-jmeter, http://jmeter-expert.blogspot.de JMeter Master-Slave-Test: http://jmeter.apache.org/usermanual/remote-test.html JMeter MAVEN Plugin: https://github.com/jmeter-maven-plugin, Online-Reports: https://sense.blazemeter.com/ https://blazemeter.com/blog/open-source-load-testing-tools-which-one-should-you-use JMeter-ec2 script: https://github.com/oliverlloyd/jmeter-ec2

37


Eights Anniversary of the Twitter Fail Whale

38


Fragen?


Vernetzt. Kontakt.

Materna GmbHFrank PientkaVoßkuhle 3744141 Dortmund+49 231 [email protected]

with apache jmeter? frank pientka · why performance testing? testing right! ... jmeter distributed...

Documents