performance testing for web applications – techniques, metrics and profiling

Performance Testing for Web Applications Techniques, Metrics and Profiling

Ștefan PopaSoftware Test Automation Specialist

Performance Testing for Web Applications: Techniques, Metrics and ProfilingSlide 2 of 18

Agenda

Techniques Stage the environment Build test assets Test procedureMetrics analysis

System analysis Database analysisProfiling

Application profiling Topology profiling


Performance testing for web applications1. Techniques

Stage the environment:

Deploy testbed

Eliminate deployment issues

Build tests assets:

Analyze client data to develop an approach strategy

Build scenarios

Test procedure:

Manual & automated tests

Automate data collecting and processing


Stage the environment: Deploy testbed

Administrator / Management

Database

Payment Aggregator

Frontend

Backend

Users

Create a testbed similar to client’s production


Stage the environment: Eliminate deployment issues

Client’s working infrastructure

Application server tuning:

- Network issues- Data compression- Connection pools- Memory allocation

Test DB server tuning:

- Best practices or DBA expertise- Resource allocation for the shared pool- Database parameters


Build test’s assets: Investigate logs and database

Analyze client data Investigate logs Investigate database

Build scenarios


Build test’s assets: Analyze client data to develop an approach strategy

Analyze client’s previous product logs or establish new possible scenarios based on existing clients


Build test’s assets: Build the scenarios

Create the actions

Set the goals(expected latencies)


Build test’s assets: Manual performance tests: Fiddler / httpwatch

Manual testing:Finding issues/bugs with minimal effort

Have a reference for automated tests


Build test’s assets: Manual performance tests: results

1 Login 1.42 0.68 0.592 Press "Existing customer" button. 0.33 0.23 0.313 Enter the account's Billing Name. Click the "Find" button 0.29 0.17 0.034 Click the "Continue" button. 1.88 1.53 1.125 Click the button "Update Resources". 0.16 0.21 0.036 Select a value from the "Service Number" drop down list. 0.68 0.78 0.437 Insert an available "IMEI“. Click the "Retrieve" button. 0.45 0.5 0.218 Press add to cart 2.58 2.56 1.859 Click the "Service payments" button. 0.49 0.17 0.17

10 Fill in a value for "Amount to pay” Press "Add to cart" 1.66 1.37 1.2111 Press "Check Out" button. 1.75 1.06 0.7812 Press "Cash" button. 0.33 0.2 0.1713 Press "Submit" 3.50 2.81 2.1714 Press "Finish Sale" 0.23 0.12 0.06

Test RUN ID 1 2 3Test Tool Manual - Fiddler Manual - Fiddler Manual - Fiddler# of users 1 1 1# of cycles 10 10 10


Build test’s assets: Develop an automated solution for testingTransposing the manual scenario into an automated script


Build test’s assets: Calibrate testsValidate script by running with 1 user as in previous manual run.

Data must be consistent with manual tests

Validate actual results

Differences between manual and automated tests

Test RUN ID 3 4

Test Tool Manual - Fiddler

Automatic - jMeter

# of users 1 1# of cycles 3 10

Login 0.57 0.53

Press "Existing customer" button. 0.03 0.06

Enter the first name/last name Click the "Find" button 0.07 0.08

Click the "Continue" button. 0.7 0.76

Click "Update Service" 0.17 0.17

Select Service Number 1.4 1.36

Change "Rate Plan" 0.51 0.51

Set "Effective Date" (Immediate) 0.1 0.1

Add to cart. 2.17 2.03

Click Check Out 0.75 0.91

Press "Check" 0.29 0.31

Submit Check number and press Submit 2.51 2.55

Press Close & Finish sale button. 0.35 0.15


Test runs, analyze resultsFirst stage – Detect low load performance issues

Raise limit to client’s expected load

Run ID 5 6 7 8 9 10 11 Test Tool Automatic

(jmeter)Automatic

(jmeter)Automatic

(jmeter)Automatic

(jmeter)Automatic

(jmeter)Automatic

(jmeter)Automatic (jmeter)

# of users 1 10 20 50 50 50 1001 Open OnlineStore main page. 0.62 0.16 0.94 0.14 0.08 0.07 0.08

2 Sort by provider: LG 0.07 0.04 0.46 0.13 0.06 0.05 0.05

3 Select a preconfigured phone model with promo 0.42 0.35 1.38 6.8 3.1 0.08 0.09

4 Add to cart the phone. 0.03 0.02 0.25 0.08 0.05 0.01 0.02

5 Enter a valid ZIP code and click the Submit button. 1.41 1.52 2.97 13.96 0.05 0.87 0.94

6 Click the Continue button. 0.82 0.88 2.58 6.21 1.24 0.6 0.63

7 Click on the "Postpaid plan type" hyperlink. 0.06 0.03 0.49 0.04 1.16 0.03 0.03

8 Select a rate plan. 0.02 0.01 0.37 0.02 0.01 0.02 0.03

9 Acknowledge age restrictions 0.05 0.03 0.25 0.05 0.02 0.03 0.04

10 Click the "Add to cart" button. 0.3 0.37 1.14 7.14 2.67 0.06 0.07

11 Skip add to cart an accessory. 0.06 0.06 0.4 0.1 0.02 0.04 0.04

12 Click the Check Out button. 0.05 0.03 0.53 0.05 0.03 0.07 0.08

13 Select New Account 0.42 0.81 1.69 4.06 0.71 0.32 0.37

14 Enter new user information. Press Continue Checkout 0.49 0.98 2.15 3.76 0.59 0.35 0.37

15 Enter Account Owner Info 0.38 0.44 1.32 4.7 0.24 0.24 0.25

16 Select Personal address or Postal Service address 1.32 4.6 9.44 47.06 28.18 7.46 11.04

17 Verify Cart 0.05 0.03 0.32 0.12 0.02 0.03 0.03

18 Select Subscriber info and select “new number" 0.17 0.24 0.8 5.54 0.19 1.47 1.62

19 Select Personal address or Postal Service address 1.99 3.17 5.75 10.34 1.91 0.07 0.08

20 Select Shipping Details & select Shipping method 0.5 1.09 2.24 5.31 0.99 0.66 0.87

21 Review cart and confirm Checkout 1.43 2.63 5.79 6.42 3.89 0.04 0.05

22 Agree License and Contract 3.23 3.54 4.87 4.76 0.41 0.33 0.33

23 Enter & submit credit card details. 2.21 5.64 9.53 27.89 6.67 3.01 2.88

24 Order Complete. Press Continue Shopping 0.11 0.09 0.8 0.14 0.06 0.05 0.06

25 Main page loads. Press logout 0.09 0.1 0.52 0.12 0.05 0.06 0.06


Performance testing for web applicationsMetrics analysis

Automatic data gathering

Automatic data processing

Metrics analysis


Metrics analysis: Automate collecting and processing test data


Metrics analysis: Analyze database issues


Metrics analysis: Analyze application server issues


Metrics analysis: Analyze network issues

Analyze system’s working parameters to help build system requirements.


Performance testing for web applicationsProfiling

System-wide load distribution analysis

Database profiling

Application profiling


Profiling: System-wide load distribution analysis

Performance profiling of staging environment’s overview

Use of APM tools


Profiling: Database profiling

Workload summary & CPU load


Profiling: Database profilingSQL query analysis


Profiling: Database profilingExecution plan analysis


Profiling: Application profilingBreaking down the problems

Finding atomic method’s elapsed time to reduce application server load


Performance tests: Publish the results after multiple iterations Run ID 30 31 66 68 72 73 # of users 1 10 10 50 50 501 Open OnlineStore main page. 0.98 0.3 0.09 0.07 0.07 0.062 Sort by provider: LG 0.52 0.28 0.05 0.05 0.04 0.053 Select a preconfigured phone model with promo 1.38 1.18 0.09 0.08 0.08 0.084 Add to cart the phone. 0.15 0.11 0.01 0.01 0.01 0.015 Enter a valid ZIP code (57722) and click the Submit button. 1.19 1.37 1.28 0.87 0.85 0.916 Click the Continue button. 0.88 1.04 0.82 0.6 0.58 0.617 Click on the "Postpaid plan type" hyperlink. 0.66 0.27 0.03 0.03 0.02 0.048 Select a rate plan. 0.07 0.07 0.02 0.02 0.02 0.029 Acknowledge age restrictions 0.66 0.28 0.03 0.03 0.03 0.02

10 Click the "Add to cart" button. 1.27 0.97 0.06 0.06 0.06 0.0711 Skip add to cart an accessory. 0.19 0.24 0.04 0.04 0.04 0.0412 Click the Check Out button. 0.2 0.14 0.07 0.07 0.07 0.0713 Select New Account 0.49 0.56 0.28 0.32 0.29 0.3

14 Enter new user information. Press Continue Checkout 0.84 0.86 0.34 0.35 0.34 0.35

15 Enter Account Owner Info 0.47 0.9 0.2 0.24 0.19 0.3516 Select Personal address or Postal Service address 9.16 9.33 6.65 7.46 5.41 3.3817 Verify Cart 0.24 0.18 0.03 0.03 0.03 0.0418 Select Subscriber info and select "Give me an Viaero number" 0.49 0.5 1.34 1.47 1.37 0.4919 Select Personal address or Postal Service address 2.79 2.24 0.07 0.07 0.07 0.0720 Select Shipping Details & select Shipping method 0.95 1.05 0.62 0.66 0.61 0.7421 Review cart and confirm Checkout 1.77 1.92 0.04 0.04 0.04 0.0422 Agree License and Contract 0.72 0.71 0.29 0.33 0.3 0.4223 Enter & submit credit card details. 4.73 5.26 3.01 3.01 2.73 2.1524 Order Complete. Press Continue Shopping 0.37 0.24 0.06 0.05 0.05 0.0525 Site Loads. Press logout 0.13 0.26 0.07 0.06 0.06 0.04


Ștefan Popa - Software Test Automation Specialist

performance testing for web applications – techniques, metrics and profiling

Technology