the right way to manage your load testing project
DESCRIPTION
Load testing essentials, Lior Katz - TACT TestingTRANSCRIPT
The right way to manage The right way to manage your Load testing projectsyour Load testing projects
Performance test main goalsPerformance test main goals Validates system under load with KPI/SLA system Validates system under load with KPI/SLA system
requirementsrequirements Verify system functionality under loadVerify system functionality under load Help configure the system from performance aspects Help configure the system from performance aspects Identify Performance system issuesIdentify Performance system issues
Test Performance on What LevelTest Performance on What Level??
• Unit testUnit test
• Sub system testSub system test
• System testSystem test
• ProductionProduction
The Cost of Fixing Software Defects:The later in the development cycle—the higher the cost
$0
$100
$1,000
$10,000
Design Code Unit Integration System Release Maintenance
Testing Tools Management Methodology
Performance projects Step by Step
• Analyze the system• Performance Test Plan & Detail design• Define Load testing tools• Monitor tools• Estimation module• Work plan• Prepare the test environment and data
preparation
Performance projects Step by Step (Cont.)
• Script Development• Scenario configuration• Execution 20% 50% 100%• Analyze test result and recommendation• Rerun solving issues• Summary report and conclusions• Lesson learn
Load TestingLoad Testing
• # of CSRs # of CSRs
• # of transactions per peak hour# of transactions per peak hour
• Runs this test for a long period of time without Runs this test for a long period of time without shutting downshutting down
• Checks the transaction’s time in the application under Checks the transaction’s time in the application under
load conditionsload conditions
Verifies that the system comply with the load requirements (i.e. operates according to a defined response time when maximum inputs are loaded into the system)
Verifies that the system comply with the load requirements (i.e. operates according to a defined response time when maximum inputs are loaded into the system)
Volume TestingVolume Testing
Verifies that the system complies with the volume requirements (i.e. operates in a defined response time when the system data bases are populated to maximum)
Verifies that the system complies with the volume requirements (i.e. operates in a defined response time when the system data bases are populated to maximum)
# #of data poolof data poolLoad DB with Max dataLoad DB with Max dataTest the system with Max dataTest the system with Max data
Stress TestingStress Testing
• Examples:Examples:
Maximum data inflation Maximum data inflation
# of maximum records # of maximum records
# of maximum users# of maximum users
Evaluates the system behavior at or beyond the limits of its specified load and volume requirements
Evaluates the system behavior at or beyond the limits ofits specified load and volume requirements
First step Definition – analyze the system
First step Definition – Performance Test Plan
Test Plan document include
• Environment architecture• Testing tools• Performance Test scope• Running scenario• Entry/Exit Criteria• Time table• Defect Tracking• Roles & responsibilities• Measure of success• Risk assessment
First step Definition – Define Load Testing tool
Load Runner
First step Definition – Define Load Testing toolJMeter server types (web-http,https SOAP DB via jdbc LDAP JMS mail POP 3)
First step DefinitionWebLoad web load support
First step DefinitionTeam System
First step Definition – Monitor Tools
Windows Performance monitor
First step Definition – Monitor Tools
Unix Performance monitor - Top
Preparation - Estimation module
A tool helps to calculate the exact dates will need in order to perform the performance activities
Microsoft Excel 97-2003 Worksheet
Preparation – Work Plan
After we got the number of working days required for the project will gather it on a calendar
Microsoft Excel 97-2003 Worksheet
Development & configuration
• Test environment preparation and data inflation
• Script Development• Scenario configuration
Execution
• Execute 30% 50% 100%
Analyze results & Recommendation
• There is a problem OR the problem is here???• Help the development team to find the problem• Configuration issues and recommendation
Load Testing reports –Templates General Summary and main problems
Saml Load executions held at Mar/ 22 with following: Details: Executed by: Load Runner Period: 3 hours Scenario: 300 reps, 300 visitors, 200 real chats
Environment: App server: stg-app3 Web server: stg-web1 Version: 9.0.2.0, build 22 Results:
1) Load tests executed without errors. 2) No performance issues found. 3) Attached execution results.
Defect Status
Application Defect # Description Status Due date
Activities Status
Execution Name Due date
% Completion Duration of Run Status Comments Actual Planned
SAML Load Test Mar/22/10 100% 100% 3 hours Pass
Load Testing reports – Templates cont.
Comparison XLS report table
Microsoft Excel 97-2003 Worksheet
Run & Analyze the scenario Run & Analyze the scenario Bottlenecks Statistics ExampleBottlenecks Statistics Example
bottlenecks statistics
Application23%
WebServer20%
Network25%
Other5%
DB27%
Summary Report
• Activities summary• Performance tests problem and defects• Baseline scenario• After tuning and connection• Result comparison• Monitoring summary• Conclusion and recommendation
Thank you All!!!
Any Questions?