performance testing driving capacity management at easyjet
DESCRIPTION
TRANSCRIPT
![Page 1: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/1.jpg)
![Page 2: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/2.jpg)
Positionner le logo de la société du présentateur
à la place de ce bloc texte
Président de filière
Orateur
Sponsor
Administrateur itSMF
2Titre de la présentation
Performance Testing Driving Capacity Management at easyJet
Andy Bolton
Nathalie Grandvuillemin
Andy Caddy
Benoit de Savignac
![Page 3: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/3.jpg)
Performance Testing Driving Capacity Management at easyJet
easyJet.com
3
![Page 4: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/4.jpg)
Performance Testing Driving Capacity Management at easyJet
easyJet continues to groweasyJet 2009
£2.6bn revenue
181 aircraft
45m passengers
422 routes
“Low cost with care and convenience”
4
easyJet 2011
£3bn+ revenue
204 aircraft
50m+ passengers
560routes
“Passionate about making travel easy & affordable”
easyJet 2014
£4bn+ revenue
230+ aircraft
60m+ passengers
650+ routes
““Europe’s preferred short haul airline”
![Page 5: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/5.jpg)
Performance Testing Driving Capacity Management at easyJet
easyJet 19963 Airports
2 Countries
2 Aircraft
2 Routes
5
Base Airport
Other Airport
![Page 6: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/6.jpg)
Performance Testing Driving Capacity Management at easyJet 6
easyJet 2011150 Airports
34 Countries
210 Aircraft
560 Routes
1500+ Flights
60m+ Passengers
Base Airport
Other Airport
![Page 7: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/7.jpg)
Performance Testing Driving Capacity Management at easyJet
easyJet a growth storyIn 15 years easyJet has become the UK’s #1 airline
7
![Page 8: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/8.jpg)
Performance Testing Driving Capacity Management at easyJet
IT Budgets
8
![Page 9: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/9.jpg)
Performance Testing Driving Capacity Management at easyJet
easyJet Revenue
9
Flight revenue (easyJet.com + B2B)
Flight revenue (easyJet.com + B2B)
Non Flight Revenue(Ancilary Partners)Non Flight Revenue(Ancilary Partners)
100% ITReliance100% ITReliance
![Page 10: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/10.jpg)
Performance Testing Driving Capacity Management at easyJet
How important is IT?
10
![Page 11: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/11.jpg)
Performance Testing Driving Capacity Management at easyJet 11
How important is IT?
![Page 12: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/12.jpg)
Performance Testing Driving Capacity Management at easyJet
Agile Development Process
12
Product Backlog
Sprint Planning Meeting
Sprint Backlog
Release
Sprint(15-30 days)
INOUT
SprintRetrospective
Meeting
Scrum(Daily
Meeting)
LESSONS LEARNED
Pre-release Activities(e.g. Training,
Documentation)
Sprint Review
Product Backlog
Sprint Planning Meeting
Sprint Backlog
Release
Sprint(15-30 days)
INOUT
SprintRetrospective
Meeting
Scrum(Daily
Meeting)
LESSONS LEARNED
Pre-release Activities(e.g. Training,
Documentation)
Sprint Review
![Page 13: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/13.jpg)
Performance Testing Driving Capacity Management at easyJet
Business Context
Capacity planning service www.easyJet.com
Business critical service
98% of revenue earned through the e-commerce service
Highly competitive market
Frequent innovation by adopting agile development
Monthly software releases
Legislative changes
Consistent growth; typically 20% p.a.
Low-cost operating model
Requirement to manage risk
13
![Page 14: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/14.jpg)
Performance Testing Driving Capacity Management at easyJet
easyJet’s Goals
14
![Page 15: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/15.jpg)
Performance Testing Driving Capacity Management at easyJet
easyJet’s Goals
15
![Page 16: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/16.jpg)
Performance Testing Driving Capacity Management at easyJet
Challenges Associated with Agile
Frequent change introduces risk
Window available for performance testing is typically 1-2 weeks
Final production candidate code often not available at the start of testing
Multiple code releases within the testing window
Increased probability of functional and environmental defects
Frequent rework of code necessitates modifications to test scripts
Cumulative effect on capacity from 12 releases per annum
Narrow time window to resolve performance defects
16
![Page 17: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/17.jpg)
Performance Testing Driving Capacity Management at easyJet
Challenges Associated with Agile
17
![Page 18: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/18.jpg)
Performance Testing Driving Capacity Management at easyJet
Typical Causes of Degraded Service Performance
18
![Page 19: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/19.jpg)
Performance Testing Driving Capacity Management at easyJet
How Can We Reduce Risk?
19
Cause of degraded performance Mitigated by performance testing?
Mitigated by modelling?
Changes in user patterns No Yes
Poor performance of 3rd parties May be limited by environment
Yes
Physical capacity bottlenecks
May be limited by size of the environment
Yes
Logical bottlenecks Yes If known
Software configuration changes Yes Yes, if tested
Increases in demand May be limited by size of the environment
Yes
![Page 20: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/20.jpg)
Performance Testing Driving Capacity Management at easyJet
Why Test?
Performance testing is used to understand the software’s performance characteristics
Do unknown time and growth related defects exist?
Memory leaks
Poor database design/indexing
Do unknown concurrency related defects exist?
Database locking issues
Memory locking issues
Race conditions
Obtain model inputs
20
![Page 21: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/21.jpg)
Performance Testing Driving Capacity Management at easyJet
Why Model?
Modelling uses our understanding of the software to predict end to end service performance under different scenarios
21
![Page 22: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/22.jpg)
Performance Testing Driving Capacity Management at easyJet
Scenario Modelling
22
![Page 23: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/23.jpg)
Performance Testing Driving Capacity Management at easyJet
Coverage of Testing and Modelling
23
ModellingTesting
Application InfrastructureBusiness ScenariosTechnical Scenarios
![Page 24: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/24.jpg)
Performance Testing Driving Capacity Management at easyJet
High Level Process
24
![Page 25: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/25.jpg)
Performance Testing Driving Capacity Management at easyJet
Risk Assessment
25
![Page 26: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/26.jpg)
Performance Testing Driving Capacity Management at easyJet
Risk Assessment
Assess the potential performance impact on different elements of infrastructure
Plan tests to help mitigate that impact
Focus on areas of highest risk due to limited timeframes
26
Web DB OtherProject X --- Medium Medium Low Test 1Project Y --- Low High Medium Tests 2 & 3Project Z --- Low Low Low None
Description IDRisks
Risk Mitigation
![Page 27: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/27.jpg)
Performance Testing Driving Capacity Management at easyJet
Performance Tests Employed
27
![Page 28: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/28.jpg)
Performance Testing Driving Capacity Management at easyJet
Baseline Tests
Used to determine the impact of the planned release
Comparative performance between releases
May be Load, Stress or Soak Tests
Service times and response times are compared to each other
28
Database Server Service Time
Web Server Service Time
Test 1.1 Release X 752 1240
Test 1.1 Release Y 813 1192
% Difference 8% -4%
0
200
400
600
800
1000
1200
1400
Serv
ice T
ime (
ms)
Example service time comparison between two fictional baseline tests
![Page 29: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/29.jpg)
Performance Testing Driving Capacity Management at easyJet
Single Transaction Type Tests
29
To determine the impact on the infrastructure of individual user transactions
Every business process which is to be modelled will have a calculated service time for each hardware element
132
233
156
122
221
328
Milliseconds Web CPU per page (fictional data)
Home Search Select
Options Login Purchase
![Page 30: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/30.jpg)
Performance Testing Driving Capacity Management at easyJet
Modelling
30
Performance Test Results
![Page 31: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/31.jpg)
Performance Testing Driving Capacity Management at easyJet
Performance Modelling
31
Business
Service
Component
Virtual Infrastructure
![Page 32: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/32.jpg)
Performance Testing Driving Capacity Management at easyJet
Performance Modelling
32
Business
Service
Component
Virtual Infrastructure
Aircraft
Home Page Search Booking
HTTP1 APP1 APP2 DB1
![Page 33: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/33.jpg)
Performance Testing Driving Capacity Management at easyJet
Performance Modelling
33
Business
Service
Component
Virtual Infrastructure
600
100 80 5
160 70 10 80
3.5 transactions/s
![Page 34: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/34.jpg)
Performance Testing Driving Capacity Management at easyJet
Demand Model Validation
The model should be validated on an ongoing basis against actual values
The first stage is to validate the demand part of the model to ensure that arrival rates are being calculated correctly
Following this the true demand can be used to validate the utilisation calculations
34
600
100 80 5
610
80 80 6
![Page 35: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/35.jpg)
Performance Testing Driving Capacity Management at easyJet
Capacity and Performance Model Validation
35
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Comparison of Actual Database Load to Predicted Database Load
Actual Database CPU Predicted Database CPU
![Page 36: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/36.jpg)
Performance Testing Driving Capacity Management at easyJet
What you need – People
Capacity and performance management skills:
Demand forecasting
Modelling
Load testing
A strong understanding of risk management
A good understanding of the business model
Flexibility
Project management skills
36
![Page 37: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/37.jpg)
Performance Testing Driving Capacity Management at easyJet
What you need – Process
37
Interface with the release team
Participation in the agile development `scrums`
Access to developers to understand application functionality
Interface with the test environment management team for test planning
Interface to the business to access business forecasts
Interface to operations to access production measurements
![Page 38: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/38.jpg)
Performance Testing Driving Capacity Management at easyJet
What you need – Tool & Environment
An appropriately sized test environment
Production-like environment is not required
Consider the ratio of capacity between presentation, business and data tiers
Toolset
Performance monitoring tools (production and test)
Load generation tool
Modelling tool
38
![Page 39: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/39.jpg)
Performance Testing Driving Capacity Management at easyJet
What you need – Data
Software/hardware configuration (production and test)
Performance measurements from production and test environments
Benchmark data (Spec, TPC, etc.)
Service times derived from performance test results
SLAs
Internal
SLAs with third-parties
Historical and forecast business demand
Historical and forecast service demand
Measured error of previous forecasts
39
![Page 40: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/40.jpg)
Performance Testing Driving Capacity Management at easyJet
Historical Business Demand
40
y = 0.0461x - 1673.1R² = 0.9668
0
20
40
60
80
100
120
140
160
180
Mar-04
Jun-
04
Sep-
04
Dec-
04
Mar-05
Jun-
05
Sep-
05
Dec-
05
Mar-06
Jun-
06
Sep-
06
Dec-
06
Mar-07
Jun-
07
Sep-
07
Dec-
07
Mar-08
Jun-
08
Sep-
08
Dec-
08
Mar-09
Num
ber
of
Ow
ned A
ircr
aft
Delivery Date
Fleet Plan to April 2009
GB Airways acquisition
![Page 41: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/41.jpg)
Performance Testing Driving Capacity Management at easyJet
Historical Service Demand
41
![Page 42: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/42.jpg)
Performance Testing Driving Capacity Management at easyJet
Costs
42
![Page 43: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/43.jpg)
Performance Testing Driving Capacity Management at easyJet
Benefits
Testing and modelling complement one another
The combined approach provides effective risk-mitigation in the context of:
Frequent change
Narrow performance test window
Business growth
The approach enables us to rapidly answer business and technical ‘what if’ scenarios
Costs are limited through the use of a scaled-down performance test environment and low-cost modelling tools
Provides performance assurance in a highly competitive market place
“Capacitas give us the confidence in making the correct release decision”
James Mackay, Test Manager, easyJet
43
![Page 44: Performance Testing Driving Capacity Management at easyJet](https://reader034.vdocument.in/reader034/viewer/2022051816/546437d5af79594f398b47dd/html5/thumbnails/44.jpg)
Performance Testing Driving Capacity Management at easyJet
Questions
Presentation will be available for download at www.capacitas.fr
Please visit us at our stand for any further questions
44