concepts on software performance testing
DESCRIPTION
Concepts on Software Performance Testing. Presented by Chris SAUNDERS - 2014 @chrisrsaunders. Presentation. Presentation. Presentation's Purpose [1%] What’s Software Performance ? [ 4 %] General Measures [4%] The Business Case [1%] Performance Goals - 5 Examples Types [45%] - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/1.jpg)
testrun.co.nz@testrunnzConcepts on
Software Performance
Testing
Chris SAUNDERS 2014
testrun.co.nz@testrunnz
![Page 2: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/2.jpg)
testrun.co.nz@testrunnz
Presentation's Purpose
![Page 3: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/3.jpg)
testrun.co.nz@testrunnz
Presentation's Purpose
• To talk about software performance testing
• To explore concepts, ideas and things to consider
• Not talking about tools
![Page 4: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/4.jpg)
testrun.co.nz@testrunnz
What is Software Performance?
![Page 5: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/5.jpg)
testrun.co.nz@testrunnz
What is Software Performance?Software performance is the manner in which
software functions, operates, or behaves; based on a given set of factors.
Software performance testing is to determine how a system performs in terms of responsiveness and stability with a defined set of factors.
What factors?
Each person takes two sticky notes and writes their answer down, and places it on the wall
Note this question may be asked again later
![Page 6: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/6.jpg)
testrun.co.nz@testrunnz
General Measures
![Page 7: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/7.jpg)
testrun.co.nz@testrunnz
General Measures
Measures Description
1. Known measures Measure where the system meets a given predefined performance criteria
For example: The CPU & Memory resources will not exceed 80% for more than 5 second duration.
![Page 8: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/8.jpg)
testrun.co.nz@testrunnz
General Measures
Measures Description
1. Known measures Measure where the system meets a given predefined performance criteria
2. Comparable measures Measure between the performance of two systems to determine which has better performance
For example: The new calculation engine will have a 20% increase in processing time and data volume
throughput.
![Page 9: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/9.jpg)
testrun.co.nz@testrunnz
General Measures
Measures Description
1. Known measures Measure where the system meets a given predefined performance criteria
2. Comparable measures Measure between the performance of two systems to determine which has better performance
3. Unknown measures Measure where the system has a possible performance issue in a given area
(One does not exactly no where or how it will fail)
For example: Determine the system performance on the system due to teamed network card failure.
![Page 10: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/10.jpg)
testrun.co.nz@testrunnz
The Business Case
![Page 11: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/11.jpg)
testrun.co.nz@testrunnz
The Business Case
“What is the business case for performance testing?”
“How does it add business value?”
Need to define “realistic performance related goals”
![Page 12: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/12.jpg)
testrun.co.nz@testrunnz
Performance Goals5 Examples Types
![Page 13: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/13.jpg)
testrun.co.nz@testrunnz
Performance Goals:Water pipe analogy
Consider software performance testing like Christchurch’s water pipe system
![Page 14: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/14.jpg)
testrun.co.nz@testrunnz
Performance Goals:Response Time
Measuring the duration of time between two points
For example: The duration for a user waiting on a file to upload
A BDuration
0:00:00.29 or 290ms
![Page 15: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/15.jpg)
testrun.co.nz@testrunnz
Performance Goals:Concurrency
Measuring the effect on the system with concurrent processes
For example: The effect of the different number of users trying to upload files at almost the same time
![Page 16: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/16.jpg)
testrun.co.nz@testrunnz
Performance Goals:Throughput
Measuring the effect on the system with throughput of information (volume)
For example: The effect of different sized files being uploaded
20 l/s98 l/s
![Page 17: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/17.jpg)
testrun.co.nz@testrunnz
Performance Goals:Interdependence
Measuring how the system responds to different factors
For example: Consider the relationship between the following1. Response time 2. Concurrency 3. Throughput
A BDuration
![Page 18: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/18.jpg)
testrun.co.nz@testrunnz
Performance Goals:Interdependence
Compare these two scenarios:1. 100 users are each uploading 20MB files, start
uploading files within a 5 minutes duration.2. 10 users are each uploading 200MB files, start
uploading files within a 10 seconds duration.
Which scenario has better performance?
![Page 19: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/19.jpg)
testrun.co.nz@testrunnz
Performance Goals:MTBF & MTTR
Mean Time Between Failures & Mean Time To Repair
Determine the performance of when the system fails and how long does it take to repair?
ConsIder the relationship of the system to other systems or internal components
![Page 20: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/20.jpg)
testrun.co.nz@testrunnz
Performance Goals:MTBF & MTTR
Source : http://www.greenm3.com/gdcblog/2012/7/31/netflix-sets-chaos-monkey-free-for-all-to-use-next-comes-mor.html
The Chaos Monkey is a software tool that was
developed by Netflix engineers to test
the resiliency and recoverability of their Amazon Web Services
Open Source https://github.com/Netflix/SimianArmy
Netflix engineers develop automatic recovery mechanisms
![Page 21: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/21.jpg)
testrun.co.nz@testrunnz
Performance Goals:Summary
1. Response Time2. Concurrency3. Throughput4. Interdependence5. MTBF & MTTR6. …
![Page 22: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/22.jpg)
testrun.co.nz@testrunnz
Some Performance Testing Approaches
![Page 23: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/23.jpg)
testrun.co.nz@testrunnz
Performance Testing Approach:Load Testing
Measure how the system behaves under a specific expected load.
Load characteristics: Define number of concurrent users, with a determined throughput for a given duration
Could be looking for bottlenecks
![Page 24: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/24.jpg)
testrun.co.nz@testrunnz
Performance Testing Approach:Stress Testing
Measure how the system behaves with loads beyond normal operational capacity.
Measure modes of failure.
Determine the robustness of the system: How, when & where is it going to fail
![Page 25: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/25.jpg)
testrun.co.nz@testrunnz
Performance Testing Approach:Spike Testing
Measure how the system behaves with the sudden increase in load
Determine system response; performance profile or system failure
![Page 26: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/26.jpg)
testrun.co.nz@testrunnz
Performance Testing Approach:Endurance (Soak) Testing
Measure how the system operates under a continuous expected load
Determine performance degradation
![Page 27: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/27.jpg)
testrun.co.nz@testrunnz
Environmental changes to determine system performance
System controlled failure
System configuration changes
![Page 28: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/28.jpg)
testrun.co.nz@testrunnz
Performance Testing Approach:Summary
1. Load Testing2. Stress Testing3. Spike Testing4. Endurance Testing5. Environmental changes to
determine system performance6. …
![Page 29: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/29.jpg)
testrun.co.nz@testrunnz
What’s Software Performance?
![Page 30: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/30.jpg)
testrun.co.nz@testrunnz
What is Software Performance Testing?
Software performance testing is to determine how a system performs in terms of responsiveness and stability with a defined set of factors.
What factors?(Each person can change their two sticky notes on the wall)
Discuss software performance testing…
![Page 31: Concepts on Software Performance Testing](https://reader036.vdocument.in/reader036/viewer/2022062321/56813c8e550346895da63879/html5/thumbnails/31.jpg)
testrun.co.nz@testrunnzConcepts on
Software Performance
Testing
Chris SAUNDERS 2014