performance testing and apm - ncc group · @devopsguys devopsguys devopsguys the benefits of using...

DevOpsGuys @DevOpsGuys www.devopsguys.com

DevOpsGuys The benefits of using an

APM solution while performance testing

Why Perf testing without APM is like Corona without the Lime…

1

DevOpsGuys @DevOpsGuys www.devopsguys.com 2 http://www.flickr.com/photos/lendog64/5781379311/


Perf Testing & APM in 30mins

• Why Performance Test?

• What is APM anyway?

• Benefits of APM when Perf Testing

• Q&A

3


Why Performance Test?

1. Ensure that we meet our customer’s

expectations of website response time

– So that they convert and we make £££

2. Ensure that we have sufficient

computing capacity to meet #1 under

load (load testing)

3. Understand how the system reacts when

things go wrong (stress testing)

4

DevOpsGuys @DevOpsGuys www.devopsguys.com 5


APM Defined

“In the fields of information technology and

systems management, application performance

management (APM), is the monitoring and

managing of performance and availability of

software applications. APM strives to detect

and diagnose application performance

problems to maintain an expected level of

service. APM is the translation of IT metrics into

business meaning (i.e. value).”

6

http://en.wikipedia.org/wiki/Application_Performance_Management


“5 Dimensions of APM*”

1. End user experience monitoring - (Active and passive)

2. Application runtime architecture discovery and modeling

3. User-defined transaction profiling (also called business transaction management)

4. Application component monitoring

5. Reporting & Application data analytics

7

*According to Gartner


CA

Quest HP

OpTier

Opnet

IBM

Compuware

Oracle

Microsoft

SL

Inetco

NetScout

eG Innovations

Progress

ASG

Idera

Visual networks

Info vista

Network Instruments

Knoa

Aternity

Coradient

Correlsense Nastel

Precise

Arctures Technologies

BM

C

Managed Engine

New Relic

AppDynamics

2011 Riverbed

OpTier

2012

AppNeta

Dell

2013

Gartner APM Magic Quadrant


BENEFITS OF LOAD TESTING WITH APM TOOLS

Never leave home without it…

9


#1

See the Big Picture

(aka Systems Thinking)

10


Systems Thinking 101

“Systems thinking is a framework for seeing

interrelationships rather than things, for

seeing patterns rather than static snapshots.

It is a set of general principles spanning

fields as diverse as physical and social

sciences, engineering and management.” -

Peter Senge

11


Command & Control vs Systems Thinking

12 http://www.systemsthinking.co.uk/2-1.asp


The Scenario

• E-commerce customer

• New website

• High data-start times on page load

• Poor load testing results

• Initial conversation with client

• 2 week delay while hosting provider scaled out the hardware…

• Problem still not fixed…

13


What’s the root cause of the ~1sec Data Start?

Waterfall Diagram of page load times


#2

Drill down to the Detail

(Code-level)

16


Find the slow code!

17


#3

Faster Iteration

= Better Value

18


Maximise Testing Value

Test

Analyse Fix

Deploy

19


CASE STUDY #1 56X FASTER IN A DAY…

9am to 9pm. 12hrs. 56x faster!

20


The Scenario

• New website pre-launch testing

• Customer Portal (SaaS solution)

• Start Load testing 9am

• 4 “Test & Fix” cycles in the day

• Final testing completed at 9pm

21


The End Results = 56x Average

Response

Throughput

(Req/sec)

Initial Testing 3024ms 16/min

Final Result 54ms 59/min

56x faster 3.5x throughput

22

Before

3024ms After

54ms


Nice flat line!

23

Before

After


#4

Team Work

“The Enemy is Performance,

not each other…”

24


The Blame Game…

25

Ops Dev QA


The Fix-It Game…

26

Ops

Dev QA


Summary

1. See the Big Picture (systems thinking)

2. Drill down to the details

3. Faster Iteration = Better Value

4. Stop the “Blame Game”

– aka “There’s your Problem!”

28


DevOpsGuys provide development &

operations support (DevOps) services

for your online applications.

You run your business. We run your website.

It’s as simple as that.

What We Do

30


Our service portfolio

• 24 x 7 Application-Centric web operations

• DevOps Transformation & Training

• Continuous Delivery implementation

• Infrastructure Automation & “Infrastructure as Code”

• Application Performance Management

• AppDynamics Implementation & Consultancy

• Web Performance consulting

• Web monitoring

• Load Testing

• Test Automation and scripting

• JetBrains Product Implementation & Consultancy

• Octopus Deploy implementation

31


James Smith (@TheDevMgr)

• 16 years of software

delivery experience.

• Proven track record

– Pizza Hut

– KFC

– Dell

– RAC

– Aviva

– Fujitsu Siemens

– Whitbread

Steve Thair (@TheOpsMgr)

• 25 years of IT Operations

experience.

• Proven track record

– Vodafone

– Credit Suisse Group

– IBM

– BNP Paribas

– Zurich Financial Services

– Totaljobs Group

– TSL Education

32

The Leadership Team


Contact Us Visit us at www.devopsguys.com

Call our team on +44 (0)20 8798 3079

Email our engineers at [email protected]

33

performance testing and apm - ncc group · @devopsguys devopsguys devopsguys the benefits of using...

Documents