tom levey (appdynamics) - the devops payrise

Realising the true value of DevOpsThe DevOps Payrise

Tom LeveyTech Evangelist

@tlevey

DevOps?

Developers working together with

Operations to get things done faster in an

automated and repeatable way

DevOps Success?

2005 Portal-services & handset testing

my����������� ������������������  desk

space

flashing����������� ������������������  lights

IREGInternational Roaming

Expert Group

International roaming

voice calls

video calls

signalingnetwork backbone

portal services

messaging

france network

spain network

italy network

germany network

norway network

Runbook1. Call L2 ops team 2. Call infrastructure ops 3. Call signaling partner 4. Call radio ops 5. Call backbone provider 6. Call international L2 ops team 7. Wait for reply... 8. Repeat 1-7 until resolution

“Everything is fine from our end.”

OPS:

Typical Day1. Open 30 new tickets 2. Make 200 phone calls 3. Attend executive P1 status update meeting 4. Argue about what a P1 and P2 really is 5. Reprioritise P2 tickets to P1 6. Reprioritise P3 tickets to P2 7. Close tickets as ‘Cannot reproduce’ or ‘Duplicate’

What did I care about?

P1’sSLA’s

What did I really care about?

What did the Business care about?

£

P1 = £Even though the business could never prove it.

How the Business really viewed me

How L2 & L3 Support viewed me

How Ops viewed me

False Alarms

Site is down

404 Errors

My search is slow

2am Friday - #FFS

We have had an alert that the load on one of your staging servers is critical.

How much time do false alarms waste?

Role Hours Per Week Cost Per Week Cost Per Year

Me 20 £400 £20,800

L2 10 £200 £10,400

L3 15 £300 £15,600

Hosting 6 £120 £6240

Network 6 £120 £6240

CMS 10 £200 £10,400

Total 55 £1,340 £69,680

Conservative estimates assuming £20/hour

How much revenue did the business lose?

No idea

Typical Day1. Open 30 new tickets 2. Make 300 phone calls 3. Attend executive P1 status update meeting 4. Argue about what a P1 and P2 really is 5. Reprioritize P2 tickets to P1 6. Reprioritize P3 tickets to P2 7. Close tickets as ‘Cannot reproduce’ or ‘Duplicate’

Things I wish I’d had

1. Automation

2. Collaboration

3. Better Tooling

4. Business Metrics

Things I wish I’d done

1. Baseline where I started

2. Measured my progress

3. Calculated Business Impact

4. Promote success not problems

5. Proved my value

Present Day

3 billion daily transactions

250 milliseconds

500+ updates/yr

1 million+ servers

100 million GB

1,000 man years

1,500 miles

Konstantin Karpov

Users Expectations

Web server 1

Internet FirewallLoad

Balancer

Web server 2

Database

Napkin architecture…

Key:

= bad

= not bad

Pre$Produc)on+APM+–+“Non+Produc)on+Data”+

Development Operations

Dev Test Staging Live

Monitor & Manage Profile QA Load Test

Pre-Production Production

Produc'on)APM)–)“Produc'on)Data”)

6

Development Operations

Dev Test Staging Live

Monitor & Manage

Pre-Production Production

Profile QA Load Test

tools can be helpful

right tools

right hands

right use

How much time and £ do these tools save?

INFRASTRUCTURE AUTOMATION

How much time and £ do these tools save?

DEPLOYMENT AUTOMATION

How much time and £ do these tools save?

LOG AUTOMATION

LogStash

Monitoring

How much time and $ do these tools save?

severe outage?

PLAN FOR FAILURE!be stronger than the weakest link

Traditional monitoring approach is limited

APPLICATION

BUSINESS TRANSACTION

Server

OS DB

MQ

Web

JVM

Silo’d domain visibility

EXISTING APPROACH

EXPANDED APPROACH

Business transaction

99.9% 99.9% 99.9%99.9%

END USER EXPERIENCE

How many of you use performance

management tools?

Identify early !Troubleshoot fast !Resolve quickly !Quantify impact

x

FOCUS

automatically map your application

Who is using my app?

device, browser, network, Carrier…

DETAILED CRASH ANALYSIS

device, carrier & browser performance

§

Show me the daily deals in my local area…

47 nodes

Search for a hotel in London…

215 nodes

Executed code diagnostics

Understand the impact of slow performance

* Screenshot from US e-Commerce AppDynamics Customer

Application Revenue

Application Errors

Application Response time

$64,499 per min

$11,987 per min

10.1 s

100 ms

Understand the benefit of an application release

Application Revenue

Application Response time

code release 1

code release 2

code release 3

$44,499 per min

$58,237 per min

1.9 s3.1 sec

Collaboration = ??

Productivity saves time Time is money.

DevOps Tasks Annual Cost £

Annual Benefit £

Infrastructure Automation ? ?

Deployment Automation ? ?

Log Automation ? ?

Monitoring ? ?

Collaboration Pride ?

Total X Y

DevOps Value £ Y - X

!

Tell the Business how much you save them.

DevOps

Automation & collaboration Time Money= =

Free trial: appdynamics.com