enhacement of jmeter - it.iitb.ac.in choudhary shekhar saurav surabhi mour ... mr. sukhdeo gupta ......

Enhacement of JMeter

B. Sushmitha Dhruv Joshi Manisha Choudhury

KIET NIT Rourkela NIT Rourkela

Naman Choudhary Shekhar Saurav Surabhi Mour

NIT Jamshedpur NIT Jamshedpur SVNIT Surat

03 July 2013

JMeter Team () Enhacement of JMeter 03 July 2013 1 / 78

Guides

Our Guides

Principle InvestigatorProf. D. B. Phatak

Project In-ChargeMr. Nagesh Karmali

MentorsMiss. Silpa T.Miss Firuza AibaraMr. Sukhdeo Gupta

Introduction

INTRODUCTION

Introduction

Why do we use JMeter?

Considering a typical web-application, we could have 1000s of userstrying to access it concurrently. We may not choose to employee1000 testers for every such user for performance evaluation of theapplication.

Introduction

More on Testing

Testing Parameters:

Performance testing Comprises of:

1 Load Testing

2 Stress Testing

3 Scalability Testing

and calculating Response Time, Latency, Throughput and other suchMetrics.

Introduction

A sample JMeter ’TESTPLAN’

Some JMeter Plugins

Thread group Plugins

Stepping Thread GroupUltimate Thread Group

Timeline Graph Plugins

Active Threads Over TimeResponse Times Over TimeResponse Latency Over TimeTransactions per SecondServer Hits per SecondsBytes Throughput Over Time

Enhancements

JMeter Enhancements Implemented

Dynamic Bandwidth Throttlingfor the requests being sent, based on response error percentage

IP Spoofingdistinct IP addresses for each virtual user

Auto CSV Generationcreating A .csv file directly from the database table mentioned

Automating TPC-C TestingTest script for Oracle and MySQL that enables a tester to carryout preliminary TPCC testing

Enhancements

Filtered Results TableFilters the sampler results, based on user specified parameters

Constant Increasing TimerStepping Up time interval between Samples requested

Enhanced Assertion resultsDetails of the Sampler passing or failing the test

SMTP DefaultsA configuration element for setting data for SMTP Samplersunder it

Enhancements

AutoCSV Generation

AutoCSV Generation Application to be tested

Application to be tested

AutoCSV Generation The Auto CSV Generation GUI

The Auto CSV Generation GUI

AutoCSV Generation The .csv file generated in /bin folder

The .csv file generated in /bin folder

AutoCSV Generation The ams user.csv generated

The ams user.csv generated

AutoCSV Generation

The CSV data config element is added as child of HTTP Sampler

AutoCSV Generation

The HTTP sampler where the parameters of .csv file are set

AutoCSV Generation

Output - CSV data verification

Bandwidth Throttling

Bandwidth throttling is the intentional slowing of Internetservice.

In real world scenario, people use different web services atdifferent bandwidths.

Using Bandwidth Throttling, JMeter can be used to create testplans to simulate slower connections.

Bandwidth Throttling User Interface

User Interface

To use bandwidth throttling in JMeter, a gui component hasbeen added to HTTP Request Defaults Config Element.

Bandwidth Throttling Working of Bandwidth Throttling in JMeter

Test Plan

Thread groups : 2

Thread count : 5

Loop count : 5

HTTP Samplers : 1

Bandwidth Throttling Working of Bandwidth Throttling in JMeter

HTTP Default Settings for Samplers

Bandwidth Throttling Test Results

Result Table for Thread group 1

Result Table for Thread group 2

Results Comparison

Bandwidth: 1MBps Bandwidth: 1KBps

Aggregate Reports

Bandwidth: 1MBps

Bandwidth: 1KBps

Dynamic Bandwidth Throttling

1 DBT deals with the variation of bandwidth at runtime.

2 DBT can be used to test performance of web services undervarying bandwidth (load).

3 DBT can be used to measure and manage errors during the testat runtime.

4 Based on error rate.

5 A distributed testing can be simulated using DBT and IPspoofing.

Dynamic Bandwidth Throttling User Interface

User Interface

To use Dynamic Bandwidth Throttling in JMeter, an extended GUIcomponent has been added to Bandwidth Throttling in HTTPRequest Defaults.

Dynamic Bandwidth Throttling Working of Dynamic Bandwidth Throttling

Test Plan

Thread Group: 1

Thread Count: 1000

Ramp Up period: 0 sec

Timeout period: 22 sec

No. of Samplers: 11

Permisibble error: 7%

Applicable bandwidth:1MBps

Minimum applicablebandwidth: 1KBps

Dynamic Bandwidth Throttling Test Results

Aggregate Report

Dynamic Bandwidth Throttling Test Results

Aggregate Report

Dynamic Bandwidth Throttling JMeter Log Report

JMeter Log Report

Dynamic Bandwidth Throttling JMeter Log Report

JMeter Log Report

IP Spoofing

IP Spoofing What is IP address Spoofing?

What is IP address Spoofing?

It is the creation of IP Packets with forged source IP address, withthe purpose of concealing identity of the sender or for impersonatinganother computer system.

IP Spoofing IP Spoofing in JMeter

IP Spoofing in JMeter

1 JMeter is capable of generating thousands of threads that act asvirtual users

2 On the server side, these requests appear from the same IPaddress on which JMeter resides

3 On servers which have IP dependent response, the testplan witha thousand virtual fails

4 To eliminate this drawback, we use IP Spoofing in JMeter

IP Spoofing

Without IP spoofing

Load balancing inactive

Figure: Server treating multiple requests without IP spoofing

IP Spoofing

With IP spoofing

Load balancing active

Figure: Server treating multiple requests with IP spoofing

IP Spoofing JMeter Implementation

JMeter Implementation

IP and Subnet Details

We need to provide JMeter with IP address of the machine and thesubnet it belongs to, and also specify the number of IP addressesrequired

IP allocation

JMeter internally allocates virtual IPs to the same machine, and eachvirtual user can send request from a distinct IP from newly allocatedvirtual IPs

IP Spoofing Interface of IP Spoofing Config Element in JMeter

Interface of IP Spoofing Config Element in JMeter

Figure: GUI of IP Spoofing

IP Spoofing Virtual IPs allocated to machine

Virtual IPs allocated to machine

Figure: Server records same IP for each virtual user

IP Spoofing Server Response Without IP Spoofing

Server Response Without IP Spoofing

Figure: Server records same IP for each virtual user

IP Spoofing Server Response With IP Spoofing

Server Response With IP Spoofing

Figure: Server records distinct IP for each virtual userJMeter Team () Enhacement of JMeter 03 July 2013 44 / 78

Automating TPC-C Benchmarking

TPC- Transaction Processing Council It defines transactionprocessing and database benchmarks and delivers trusted results.

Some benchmarks undertaken under TPC:

TPC-APPTPC-HTPC-C

TPC- Transaction Processing Council It defines transactionprocessing and database benchmarks and delivers trusted results.

Some benchmarks undertaken under TPC:

TPC-APPTPC-HTPC-C

Automating TPC-C Benchmarking WHY TPC-C?

WHY TPC-C?

A number of these benchmarks have been deprecated. TPC-C iscurrently in use and a rather complex process.

A number of business houses use this benchmark to showcasetheir performance for OLTP transactions. It gives the measureof Server speed for online transaction processing.

The actual benchmarking process is a time taking and a costlyaffair.

A preliminary test would be a highly useful tool to test a serverfor performance and hence improve it where it lacks.

Automating TPC-C Benchmarking The Benchmarking Model

The Benchmarking Model

Automating TPC-C Benchmarking TPC-C Model

The Benchmarking Model

The model emulated for TPC-C

Automating TPC-C Benchmarking Tables in the TPC-C schema

Tables in the TPC-C schema

Warehouse

History

District

Customer

New Order

Orders

Order Line

Automating TPC-C Benchmarking Transactions

Transactions

New-order: enter a new order from a customer

Payment: update customer balance to reflect a payment

Delivery: deliver orders

Order-status: retrieve status of customers most recent order

Stock-level: monitor warehouse inventory

Automating TPC-C Benchmarking TPC-C Workflow

TPC-C Workflow

Automating TPC-C Benchmarking Why JMeter to automate??

Why JMeter to automate??

JMeter is already capable of spawning a large number of virtualusers to simulate the interaction of the real users with thesystem under test.

The firing of a request and measurement of response time aswell as throughput is embedded in JMeter.

Automating TPC-C Benchmarking Why JMeter to automate??

Why JMeter to automate??

JMeter is already capable of spawning a large number of virtualusers to simulate the interaction of the real users with thesystem under test.

The firing of a request and measurement of response time aswell as throughput is embedded in JMeter.

Automating TPC-C Benchmarking TPC-C Testing in JMeter

TPC-C Testing in JMeter

Automating TPC-C Benchmarking Controllers

Controllers

Include Controller is a component of JMeter used to run saved scriptsin JMeter.

Automating TPC-C Benchmarking Test with 1 warehouse

Test with 1 warehouse

Automating TPC-C Benchmarking Procedures

Procedures

Automating TPC-C Benchmarking Test with 33 Warehouses

Test with 33 Warehouses

Automating TPC-C Benchmarking JDBC Configuration

JDBC Configuration

Automating TPC-C Benchmarking Controllers

Controllers

Automating TPC-C Benchmarking Timers

Timers

Automating TPC-C Benchmarking Transaction Call

Transaction Call

Automating TPC-C Benchmarking Function Helper

Function Helper

Automating TPC-C Benchmarking Procedure Request

Procedure Request

Automating TPC-C Benchmarking Response

Response

Automating TPC-C Benchmarking Aggregate Report

Aggregate Report

Automating TPC-C Benchmarking Transactions per second

Transactions per second

Automating TPC-C Benchmarking Aggregate Graph

Aggregate Graph

Automating TPC-C Benchmarking Response Time over time

Response time over time

Challenges

1 Auto CSV Generation:Generating CSV file for a particular table from a specifiedDatabase

2 Dynamic Bandwidth Throttling:Changing bandwidth in runtime and determining a reliableparameter (Percentage Error) to implement throttle bandwidth

3 IP Spoofing:Finding an array of un-used IPs and automating allocation ofvirtual IPs for a system

4 Automating TPC-C testing:Making tpcc testing capable of being run with differentdatabases incroporating all the standards into JMeter

Challenges

Future Work

1 Incorporating other Benchmarking support such as TPC-H,TPC-E etc. into JMeter.

2 Automation of the test scripts, as in user may not have to createthe test script, and Jmeter can itself do it for the tester bytechniques such as web crawling, etc.

3 The instability of JMeter on large loads could be worked outwith some solution.

4 Bringing large download efficiency into JMeter.

5 Better analysis of the results produced by JMeter via somecomplex graphs and better comparison between different graphresults.

Future Work

Conclusion

The basic aim of the project was to enhance JMeter with theintroduction of some new features and overcome somedrawbacks.

We have made JMeter capable of performing more practical testswith the introduction of bandwitdh throttling and IP Spoofing.

The user friendliness of JMeter has been improved with theintroduction of auto csv generation and results filtering.

Finally Jmeter has been extended from just a load testing tool toa Preliminary TPC-C benchmarking tool.

Conclusion

References

References[1] E. H. Hallili, Apache JMeter: A practical beginner’s guide to automated testing and performance measurement for your

websites.Packt Publishing Ltd., 2008.

[2] “Apache Jmeter. Available athttp://jmeter.apache.org/,” February 2013.

[3] “ Apache Jmeter usermanual. Available at/http://jmeter.apache.org/usermanual/component reference.htmlintroduction.”

[4] “Installation of Jmeter Googlecode pluginshttp://www.code.google.com/p/jmeter-plugins/,” February 2012.

[5] “Jmeter Googlecode plugins testplanhttp://shantonusarker.blogspot.in/2013/05/introdution-to-jmeter-google-plugin.html/,” May 2013.

[6] “ Standard Benchmarks for database systems. Available atwww.tpc.org/information/sessions/sigmod/sigmod97.ppt,” 1997.

[7] “TPC-C Benchmark standard specification. Available atwww.tpc.org/tpcc/spec/tpcc current.pdf,” February 2010.

[8] “Benchmark overview of TPC-C. Available athttp://globalsp.ts.fujitsu.com/dmsp/Publications/public/Benchmark Overview TPC-C.pdf,” October 2003.

[9] “Tpc-c benchmark kit. available athttp://docs.openlinksw.com/virtuoso/tpcc.html.”

[10] “comparisons between LoadRunner vs. Grinder vs. JMeter. Available athttp://blackanvil.blogspot.in/2006/06/shootout-load-runner-vs-grinder-vs.html,” June 2006.

[11] “Jmeter Plug-in Resultshttp://www.methodsandtools.com/tools/jmeterplugins.php/,” May 2012.

[12] GauravMutreja, “ Exporting your database to csv file. Available athttp://gauravmutreja.wordpress.com/2011/10/13/exporting-your-database-to-csv-file-in-java/,” October 2011.

Educational Application

Learning Shapes

A simple application to teachyoung minds the sense of shapesand colors. This app tests therecognition skill shapes and colorsamong children

Traingles for High School

An application to aid learningabout Triangles, allowsconstruction of Triangles, alsodrawing Triangles from user Input

Mathematics Playground

Best friend of any student duringexamination for quick revision ofthe formulas.The detailed proofsand animation helps the student inrecalling the logic behind theformula

Base Conversion

This application teaches BaseConversion, it describes theconversion procedure from anypositive integral base to any otherpositive integral base

Area of Polygons

This application teaches methodsto calculate area of any Polygon.Polygon is drawn by user inputand area is calculated anddisplayed with explanation

Currency Converter

This application provides themethods of currency conversion aswell as the details of the countryto which the currency belongs.

Team Members

Enhancement of JMeter - Team Members

B. Sushmitha Dhruv Joshi Manisha Choudhury

Naman Choudhary Shekhar Saurav Surabhi Mour

enhacement of jmeter - it.iitb.ac.in choudhary shekhar saurav surabhi mour ... mr. sukhdeo gupta ......

Documents