realize the full power of the cloud for performance ... mortimer - release the full power.pdf ·...
TRANSCRIPT
Gen-i Test Services
Realize The Full Power Of The Cloud For Performance Testing
Ian Mortimer and Gareth Shackel
Agenda
2
Agenda
1. Why Performance Test?
2. What is Performance Testing?
3. Why use the Cloud for Performance Testing?
4. Types of Cloud Providers
5. Which Cloud Provider(s)
6. Key features of this approach6. Key features of this approach
7. Demo� Capability -- What we can do
� Setup
� Example
� Learnings
8. Key caveats
9. Wrap-up
3
Save time & money with the Cloud
To
4
Why Performance Test?
To prevent poor system performance
5
What is Performance Testing?Performance Testing is usually divided into three categories:
� Performance Testing:
To determine or validate speed, scalability, and/or stability
� Load Testing:To verify behaviour under normal and peak conditions to verify that your application can meet your desired performance objectives
� Stress Testing:To determine an application’s behaviour when it is pushed beyond normal or peak load conditions
6
Source: Performance Testing Guidance for Web Applications: patterns & practices; J.D. Mier, Carlos Farre, Prashant
Bansode, Scott Barber, Dennis Rea
Traditional Performance Testing
Challenges� Agreeing the need for Performance Testing
� Fitting it into tight project schedules
� Affordable Tools� Affordable Tools
� Affordable Bandwidth
� Network Latency
� Affordable Test Environment
� Real-time monitoring of Application Infrastructure
7
Why use the Cloud for
Performance Testing?� Access to cheap scalable hardware
� Access to cheap bandwidth
� Only pay for what you use� Only pay for what you use
� Quick view of the user experience
� You can get a quick view of the user experience on your website
� At a fraction of the normal cost and effort of performance testing
� Quicker to identify bottlenecks
8
Types of Cloud ProvidersIaaS (Infrastructure)
Get a new server (or many servers) in minutes
Pay per hour, so you’re free to try things out
Scale your infrastructure up and down at will
9
PaaS (Platform)
Patching and scaling are done for you
Developers concentrate on adding value, not the ‘plumbing’
SaaS (Software)
The business decides on the solution – no infrastructure to worry about
Latest version always available, wherever you are
Backups are part of the service
Which Cloud Provider?
� Cloud Providers that fit the IaaS Model� i.e. need an actual server
� Need a pay only for what you use model
� Choose your cloud provider locations based on:� Choose your cloud provider locations based on:� Where are your websites hosted
� Where your customers / users of the website are based
� Network Latency is key� If choose irrelevant locations, then get misleading network
latency results� Due to more or less hops
10
Goal
“Utilize Cloud Computing capabilities to create a low cost
performance testing benchmark service to enable
customers to understand how their web application will
perform”perform”
This includes:
� Using the Cloud to test a Web Application
� Using the Cloud to test a Cloud based Application
11
Cloud Performance Testing
Architecture
12
Key features of this approach …� Minimal stand-up time
� Can provide a basic core package of tests and users
� Plus can ramp up for more complexity in the website
� Or to focus on different technical or business risks
� Easy to script� Easy to script
� Can add any number of users using a “Controller – Robot” test rig architecture
� Can schedule the tests for any time you want
… Key features of this approach
� Test results can be monitored in real time
� Monitor for potential performance bottlenecks both:� Website response time
� And the underlying infrastructure's performance
� Requires externally accessible web-sites� Requires externally accessible web-sites� As the "Cloud" has to be able to access what we're testing
� Requires a small program to be copied onto the web application’s web server
Demo
15
We’ve Tested:
� Amazon EC2
� US based
� Huge scale
� Melbourne IT� Melbourne IT
� Australian based
� Uses VMWare Virtualized
� In beta and is currently free
� iSERVE NZ
� NZ based
� Uses Citrx XenWeb
16
Key caveats …
� Spin up time - “your virtual machine will be available in 30
minutes… “
� …2 days and 5 support calls later you can finally access the machine
� Registration time
� “We will process and respond to your application in less than two
business days.”
� Location - watch out for this clause:
� “…may provide the services as well as house any data of the
customer or otherwise from any jurisdiction or location that in its
absolute discretion it chooses”
17
… Key caveats
� Results are dependent on the users specified what web
actions they expect to be used
� Key to performance testing being able to show what your users' experience of the website will really be like
� If the customer provides the wrong input, then the results won't represent how the site will be used
� Infrastructure Monitoring currently supports:� Windows with IIS Web Server
� Linux with Tomcat and Axis 2
18
Which Tools Support this
Approach?� Jmeter
� Visual Studio 2010 Ultimate & Predecessors
� HP Loadrunner in the cloud
19
Summary
20
Questions
Contact Details:
Ian Mortimer
Test Services Manager
Gen-i
Phone: +64 275 525 067 Phone: +64 275 525 067
Email: [email protected]
Gareth Shackel
Performance Test Architect
Phone: +64 27 600 1427
Email: [email protected]