Introduction to Performance

Testing

By Sajid Manzoor

• What is Performance Testing• Goals of Performance Testing• Importance of Performance Testing• Types of Performance Testing• Pre-requisites of Performance Testing• Performance Management• Performance Testing Cycle• Performance Testing Activities• Performance Symptoms and Issus• Typical Order of Fixes• Challenges with Performance Testing• Performance Testing Best Practices• Difference Between Performance Testing and Performance

Engineering

AGENDA OF THE DISCUSSION

• Non-Functional testing to determine the system responsiveness, stability, reliability and scalability

WHAT IS PERFORMANCE

TESTING??

• Access production readiness• Compare two platform to compare which performs

better• Compare performance characteristics of system

configurations• Evaluate system against performance criteria• Discover what parts of the application perform

poorly and under what conditions• Finding the source of performance problems• Support system tuning

PERFORMANCE TESTING GOALS

• Most users click away after 8 seconds of delay• $4.4 billion business revenue loss due to poor web

applications performance• Failure of US Based Airlines computerized system to

maintain flight plans in Nov 2009• Aberdeen found that inadequate performance could

impact revenue by up to 9% • Business performance begins to suffer at 5.1

seconds of delay in response times of web applications and 3.9 for critical applications

IMPORTANCE OF PERFORMANCE TESTING

• An additional second of waiting on a website significantly impact customer satisfaction and visitor conversions

• Page viewers, conversions rate and customer satisfaction drops 11%, 7% and 16% respectively

IMPORTANCE OF PERFORMANCE TESTING

IMPORTANCE OF PERFORMANCE TESTING

• Load Testing• Stress Testing• Soak/Endurance Testing• Volume/Spike Testing• Scalability Testing• Configuration Testing

TYPES OF PERFORMANCE TESTING

TYPES OF PERFORMANCE TESTING (COND..)

• Stable build free from all major functional defects• Performance testing environment similar to

production environment• No other testing should be performed while

Performance testing• Conduct Performance testing before going live• Complete understanding and knowledge of the

application

PRE-REQUISITES OF PERFORMANCE TESTING

CALL CENTER APPLICATION

INFRASTRUCTURE EXAMPLE

PERFORMANCE

MANAGEMENT

Proactive Approach

Reactive Approach

TYPES OF PERFORMANCE MANAGEMENT

• Performance Testing after the System Testing• Advantages

• Cost effective• Disadvantages

• Difficult to resolve the performance bottlenecks after the complete development

• Defect removal cost will exponentially increase • Whole system can be useless

REACTIVE APPROACH

PERFORMANCE TESTING CYCLE

CORE PERFORMANCE TESTING ACTIVITIES

Analyze, Report and Retest

Execute Tests

Implement Test Design

Configure Test Environment

Plan and Design Tests

Identify Performance Acceptance Criteria

Identify Test Environment

• Identify Test Environment: Performance test environment should be exact replica of production environment

• Identify Performance Acceptance Criteria: Identification of performance testing matrices and acceptable value of each matrix

• Plan and Design Test: Identification of user test scenarios based on their importance, frequency and performance impact

• Configure Test Environment: Test environment setup as identified at first step

• Implement Test Design: Scripting of user scenarios with the help of a performance testing tool

• Execute Tests: Execution of scripted user scenarios against different user sets with different configurations

• Analyze, Report and Retest: Analyze the executed tests results with identified matrices, report results to stake holders and Retest if tests were not passed in previous iteration

PERFORMANCE TESTING ACTIVITIES

PERFORMANCE SYMPTOMS AND

ISSUES Symptoms of Web Application

Performance Problems

Database Problems

Web Server Problems

Application Server problems

Network Problems

• Long user response time• Memory leaks• High CPU usage• Too many open connections • Length queues for requests• Too many table scans of

database• Database deadlocks• Erroneous data returned• HTTP errors• Pages not available

WEB APPLICATIONS PERFORMANCE

PROBLEMS

• Insufficient Indexing• Fragmented Database• Out-of-date Statistics• Faulty Application Design

DATABASE PERFORMANCE PROBLEMS

WEB SERVER

PERFORMANCE

PROBLEMS • High CPU Usage• Poor Server Design• Memory Problems

• Poor Database Tuning

• Poor Cache Management

• Poor Session Management

• Poor Security Design

APPLICATION SERVER PERFORMANCE PROBLEMS

NETWORK PERFORMANCE PROBLEMS

TYPICAL ORDER OF FIXES

Improve current application design (Algorithms, caching, DB

calls, memory use)

Upgrade hardware (RAM, CPU, network bandwidth)

Upgrade software infrastructure (OS, web server, database)

Upgrade system architecture (Client-server to basic n-tier, software and hardware changes,

Use hardware load balancing /SSL

• Infrastructure Setup• Collection and Analysis of Huge Data• Identifying the Problem Root cause• Team effort required (Product vendors, Architects, Developers,

Testers, Database administrators, Network administrators)• Accuracy of results• Client involvement• Testing inside the firewall• Multidimensional applications (web, SAAS, mobile etc.)• Testing on Live Server• Expensive

CHALLENGES WITH PERFORMANCE

TESTING

• Use Ramp up approach• Repeat the tests 2-3 times to confirm the results• Run tests at different times• Run individual tests before combining them in a

single test

PERFORMANCE TESTING BEST PRACTICES

info@swaam.com

+92-42-35782346

Contact Us

www.Twitter.com/SwaamTech

www.FaceBook.com/SwaamTech