ciprian balea automated performance-testing
DESCRIPTION
TRANSCRIPT
D I S C I P L I N E D I N N O VAT I O N
AUTOMATED PERFORMANCE TESTING
FOR DESKTOP APPLICATIONS
CIPRIAN BALEA
THE SCARY QUESTION OF APPLICATION
DEVELOPMENT
How does it perform in real life conditions?
Never as first expected!
BENCHMARKING
Ideally: • consumming as little resources as possible without loosing
functionality• the app should not break / slow the system down
What impact does the app have on the system running it?(is the system still reliable)
How does the app perform when running on the host system? (does it keep all functionality)
HOW TO PREDICT THE FUTURE?
Run it multiple times, and average out.
A slingle run doesn’t acurately predict real life usage.
HOW TO GET A CREDIBLE RESULT
Avoid inconsistencies induced by the outside world=> no Internet connection, own webserver & assets
WHAT’S OUR APPLICATION ABOUT
A custom web browser
HARDWARE SETUP
© Copyright 2013 3PILLAR GLOBAL All rights reserved 7
BIG PROBLEM #1: HOW TO KNOW HOW MUCH
THE PAGE TOOK TO LOAD
Small solution:
Based on the HTML content, loading time cannot be computed.(element loading order is not consistent across multiple runs)
BIG PROBLEM #2: HOW TO READ SYSTEM
RESOURCE USAGE WITHOUT 3RD PARY TOOLS
Small solution (a):
BIG PROBLEM #2: HOW TO READ SYSTEM
RESOURCE USAGE WITHOUT 3RD PARY TOOLS
Small solution (b)
BIG PROBLEM #3: HOW TO MANIPULATE THE
APPLICATION’S GUI
Small solution:
WHAT DATA WE COLLECT
Test 1• Webpage load time• CPU usage• Memory usage• Disk I/O
Test 2• Timing a „complex” operation
TEST 1:
READING SYSTEM STATISTICS
HOW IS THE DATA COLLECTED
Not at the start of the game
Not at the end of the game
Throughout the entire game
HOW IS THE DATA COLLECTED
RUNNING THE TEST
One test sequence:
• clear cache
• open the browser
• start reading system data
• load webpage
• close the browser
• save result in an Excel spreadsheet
Full test:
• restore the “clean” snapshot of the test environment
• automatically run the test sequence for 1000 times on the
native browser
• automatically run the the test sequence for 1000 times on
the custom browser
RUNNING THE TEST
TEST 2:
TIMING A „COMPLEX” OPERATION
„COMPLEX” OPERATION
Open the browser
Load a webpage
Bookmark page
Go to homepage
Open bookmark
Wait for page to load
RESULTS
RESULTS
CONCLUSIONS
It can be automated with an initial effor, but the returned value is huge.
Offers very concludent results on what needs to be improved in the application to boost performance.
Performance testing is very important as it can reveal performance flaws.
Is very time & effor consuming to do manually.
Q & A
THANK YOU !