performance testing of a road tolling system · testing of web services • webservice mocking...
TRANSCRIPT
![Page 1: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/1.jpg)
Performance Testing of a Road Tolling System
Oliver BaendlSiegfried Goeschl
1Donnerstag, 27. September 12
![Page 2: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/2.jpg)
Gauteng Open Road TollingSouth Africa
2Donnerstag, 27. September 12
![Page 3: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/3.jpg)
42 Tolling Points3Donnerstag, 27. September 12
![Page 4: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/4.jpg)
1.500.000 registered road-user4Donnerstag, 27. September 12
![Page 5: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/5.jpg)
11 Satellite Centers5Donnerstag, 27. September 12
![Page 6: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/6.jpg)
41 E-Toll Shops
6Donnerstag, 27. September 12
![Page 7: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/7.jpg)
20 Mobile Payment Units7Donnerstag, 27. September 12
![Page 8: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/8.jpg)
Central Operation Center8Donnerstag, 27. September 12
![Page 9: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/9.jpg)
750 Concurrent CRM Client Users
9Donnerstag, 27. September 12
![Page 10: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/10.jpg)
162 vehicle passges per second10Donnerstag, 27. September 12
![Page 11: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/11.jpg)
400 concurrent web users11Donnerstag, 27. September 12
![Page 12: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/12.jpg)
Two Data Centers
12Donnerstag, 27. September 12
![Page 13: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/13.jpg)
Master Control Program
13Donnerstag, 27. September 12
![Page 14: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/14.jpg)
Project Overview
• Vehicles are passing through a tolling point
• The vehicle is identified either using the license plate number (via OCR) or a tag
• Vehicle passage are transmitted to Open Road Tolling Back Office (ORTBO)
• 90% video passages with three images
• 10% tag passages
14Donnerstag, 27. September 12
![Page 15: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/15.jpg)
Project Overview
• Vehicle passages are collected and assigned to road-user accounts in the Transaction Clearing House (TCH)
• TCH responsible for billing the customer
• Passages of unregistered or black-listed cars are processed by the Violation Processing Center (VPC)
15Donnerstag, 27. September 12
![Page 16: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/16.jpg)
Project Overview
• Overall system contains multiple Oracle RACs and MSSQL clusters
• Interfaces between the subsystems mostly web services
• Backup data center replicates the “Central Operation Center” over dark fiber
16Donnerstag, 27. September 12
![Page 17: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/17.jpg)
Schedule Phase One
• Performance testing of CRM Server simulating 771 concurrent & active CRM client sessions
• Performance testing of public-facing web portal simulating 400 concurrent & active web user sessions
• On success the CRM subsystem goes live and has real user accounts
17Donnerstag, 27. September 12
![Page 18: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/18.jpg)
Schedule Phase Two
• Simulating up to 162 vehicle passage per second
• Validating the end-to-end processing under load (passage processing, customer billing and violation processing)
• On success the overall system goes live
18Donnerstag, 27. September 12
![Page 19: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/19.jpg)
Performance Test Scope
• End to end performance acceptance test
• Mandatory and client-witnessed test
• Strict acceptance criteria regarding
‣ throughput
‣ number and severity of errors
‣ validation of test runs
19Donnerstag, 27. September 12
![Page 20: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/20.jpg)
Performance Test Types
Baseline Test One hour average load
Stress Test One hour peak load
Endurance Test 24 hours maximum load
20Donnerstag, 27. September 12
![Page 21: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/21.jpg)
Performance Test Hardware
• Six dedicated load injectors
‣ Windows 2008 Server
‣ 3 physical and 3 virtual boxes
‣ Dual quad-cores with 8 GB RAM
• No direct access outside the data center
‣ Citrix over RDP over RDP
21Donnerstag, 27. September 12
![Page 22: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/22.jpg)
Performance Test Tools• All performance test tools have issues
‣ Make sure that they work for you
• We prefer JVM-based test tools
‣ Easy to migrate between different OS
‣ We are Java developers
• We prefer command-line invocation
‣ RDP over slow network is really slow
22Donnerstag, 27. September 12
![Page 23: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/23.jpg)
Performance Test Tools
• We use a non-distributed installation
‣ Complex network and routing
‣ Firewall not under our control
• Check the license and price tags
‣ Virtual users can be expensive
‣ Connectivity to license server?
23Donnerstag, 27. September 12
![Page 24: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/24.jpg)
The Test Tools We Used
Subsystem Interface Planned Delivered
CRM Server WebService
Public Web Portal HTML
Vehicle PassageProcessing WebService
24Donnerstag, 27. September 12
![Page 25: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/25.jpg)
Apache JMeter
• Open Source and free
• Implemented a new reporting backend to scope with huge result files
• Rock solid - only minor issues
• Extensible using scripting and Java libraries
• Rough GUI compared to commercial tools
• Unsuitable for complex ASP.NET web sites
25Donnerstag, 27. September 12
![Page 26: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/26.jpg)
WAPT Pro
• Excellent value for money
• Good support
• ASP.NET module saved my day
• Extensible through Javascript
• One major issue with generation of test values under high load (requires restart)
26Donnerstag, 27. September 12
![Page 27: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/27.jpg)
soapUI Pro
• Excellent for prototyping and functional testing of web services
• WebService mocking saved my day
• Extensible using Groovy and Java libraries
• Various issues during load testing
• soapUI was replaced with JMeter for performance tests
27Donnerstag, 27. September 12
![Page 28: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/28.jpg)
• No software installation
• Take two (or more)
• Automation is your friend
• Performance test for everyone
• Performance tests are a valuable asset
Some More Thoughts
28Donnerstag, 27. September 12
![Page 29: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/29.jpg)
No Software Installation
• Portable Apps on USB Stick
‣ Run tests directly from USB Stick
‣ Clone USB Stick to hard disk
• Have all your tools on the USB stick
‣ Java, JMeter, Editor, Git, ...
29Donnerstag, 27. September 12
![Page 30: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/30.jpg)
Take Two (or more)
• A minimum of two share-nothing load injectors are required to prove that the production servers is causing the performance bottleneck (and not your test tool)
• We used a maximum of five share-nothing load injectors simultaneously
30Donnerstag, 27. September 12
![Page 31: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/31.jpg)
• Performance test were run mostly at night and over the weekend
• Test execution managed by Hudson
• Test failures triggers email notification
• Test protocols are copied to FTP server
Automation Is Your Friend
31Donnerstag, 27. September 12
![Page 32: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/32.jpg)
Performance Tests for Everyone
• All test scenarios are configured as Hudson jobs
‣ Baseline, stress & endurance test
‣ Some other internal tests
• Everyone can start and monitor a performance test scenario over the web browser
• Even developers can run performance tests
32Donnerstag, 27. September 12
![Page 33: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/33.jpg)
• Performance tests can be used in creative and un-planned ways
‣ Smoke test for new deployment
‣ Testing database failover
‣ Testing data center replication & failover
‣ Testing different database setups
Performance Tests as Asset
33Donnerstag, 27. September 12
![Page 34: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/34.jpg)
The few things you should take home
34Donnerstag, 27. September 12
![Page 35: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/35.jpg)
Lessons Learned
• Tests tools are buggy
• Client-witnessed test are hard
• Performance tests are a valuable asset
• Creation and maintenance of complex performance test suite is a project on its own
35Donnerstag, 27. September 12
![Page 36: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/36.jpg)
Questions & Answers
36Donnerstag, 27. September 12
![Page 37: Performance Testing of a Road Tolling System · testing of web services • WebService mocking saved my day • Extensible using Groovy and Java libraries • Various issues during](https://reader033.vdocument.in/reader033/viewer/2022042304/5ecf1d096fb970708516740f/html5/thumbnails/37.jpg)
Resources
JMeter SLA Report https://github.com/sgoeschl/jmeter-sla-report
JAMon API http://jamonapi.sourceforge.net/
JMeter ANT Tasks http://www.programmerplanet.org/pages/projects/jmeter-ant-task.php
WAPT Pro http://www.loadtestingtool.com
soapUI http://www.soapui.org/
37Donnerstag, 27. September 12