automated performance testing for desktop applications by ciprian balea
TRANSCRIPT
![Page 1: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/1.jpg)
Automated Performance Testing for Desktop Applications by Ciprian Balea
![Page 2: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/2.jpg)
2
THE SCARY QUESTION OF APPLICATION DEVELOPMENT
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
How does it perform in real conditions?Never as first expected!
![Page 3: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/3.jpg)
3
BENCHMARKING• What impact does the app have on the system running it? (is the
system still reliable)• How does the app perform when running on the host system?
(does it keep all functionality)
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Ideally,• consuming as little resources as possible without loosing
functionality• the app should not break /slow the system down
![Page 4: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/4.jpg)
4
HOW TO PREDICT THE FUTURE?
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Run it multiple times and average out.
A single run doesn’t accurately predict real life usage.
![Page 5: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/5.jpg)
5
HOW TO GET A CREDIBLE RESULT?
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Avoid inconsistencies induced by the outside world => no Internet connection, own
webserver & assets
![Page 6: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/6.jpg)
6
WHAT’S OUR APPLICATION ABOUT?
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
A Custom Web Browser
![Page 7: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/7.jpg)
7
HARDWARE SETUP
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
![Page 8: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/8.jpg)
8
BIG PROBLEM #1: HOW TO KNOW HOW MUCH TIME THE PAGE TOOK TO LOAD
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Based on the HTML content, loading time cannot be computed (element loading order is not consistent across
multiple runs).
Small solution:
![Page 9: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/9.jpg)
9
BIG PROBLEM #2: HOW TO READ SYSTEM RESOURCE USAGE WITHOUT 3RD PARTY TOOLS
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Small solution (a):
![Page 10: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/10.jpg)
10
BIG PROBLEM #2: HOW TO READ SYSTEM RESOURCE USAGE WITHOUT 3RD PARTY TOOLS
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Small solution (b):
![Page 11: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/11.jpg)
11
BIG PROBLEM #3: HOW TO MANIPULATE THE APPLICATION’S GUI
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Small solution:
![Page 12: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/12.jpg)
12
WHAT DATA WE COLLECT?• Test 1
• Webpage load time• CPU usage• Memory usage• Disk I/O
• Test 2• Timing a “complex” operation
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
![Page 13: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/13.jpg)
13
TEST 1: READING SYSTEM
STATISTICS
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
![Page 14: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/14.jpg)
14
HOW IS THE DATA COLLECTED?
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Not at the start of the game
Not at the end of the game
Throughout the entire
game
![Page 15: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/15.jpg)
15
HOW IS THE DATA COLLECTED?
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
![Page 16: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/16.jpg)
16
RUNNING THE TEST• One test sequence:
• clear cache• open the browser• start reading system data• load webpage• close the browser• save result in an Excel spreadsheet
• Full test:• Restore the “clean” snapshot of the test
environment• Automatically runs the test sequence for
1000 times on the native browser• Automatically runs the test sequence for
1000 times on the custom browser
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
![Page 17: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/17.jpg)
17
RUNNING THE TEST
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
![Page 18: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/18.jpg)
18
TEST 2: TIMING A
“COMPLEX” OPERATION
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
![Page 19: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/19.jpg)
19
“COMPLEX” OPERATION
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Open the browser
Load a webpage
Bookmark page
Go to homepage
Open bookmark
Wait for page to load
![Page 20: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/20.jpg)
20
RESULTS
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
![Page 21: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/21.jpg)
21
RESULTS
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
![Page 22: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/22.jpg)
22
CONCLUSIONSPerformance testing is very important as it can reveal performance flaws.
It is very time and effort consuming to do it manually.
It can be automated with some initial effort, but the returned value is huge.
Offers very conclusive results on what needs to be improved in the application to boost performance.
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
![Page 23: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/23.jpg)
23
Q&A
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
![Page 24: Automated Performance Testing for Desktop Applications by Ciprian Balea](https://reader037.vdocument.in/reader037/viewer/2022102802/554bef46b4c9056b348b54b9/html5/thumbnails/24.jpg)
THANKS!Now let’s get to
work!
24© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential