top lessons learned from the devops handbook

@RealGeneKim1

WELCOMES YOU TO THE

#DevOpsSummit17

@RealGeneKim

Gene Kim | KEYNOTE

9

@RealGeneKim

Session ID:Gene Kim

Top Lessons Learned SinceThe Phoenix Project Was Released

@RealGeneKim

The DownwardSpiral

@RealGeneKim

@RealGeneKim

IT Ops And Dev At War

7

@RealGeneKim

@RealGeneKim

IT Operations

CBS Photo Archive/Star Trek: The Original Series/Getty Images

@RealGeneKim

The Developers

@RealGeneKim

The Product Managers

Source: Flickr: birdsandanchors

@RealGeneKim“Hot Fuzz”: Rogue Pictures

@RealGeneKim

Session ID:

Surprise #1:

The Business Value Of DevOps Is Even Higher Than We Thought

@RealGeneKim

High Performers Are More Agile

46x 440xmore frequent

deployments

faster lead times

than their peers

Source: Puppet/DORA: 2017 State Of DevOps Report: https://puppet.com/resources/whitepaper/state-of-devops-report

@RealGeneKim

High Performers Are More Reliable

5x 96xlower change

failure rate

faster mean time

to recover (MTTR)


@RealGeneKim

High Performers Are More Secure And Controlled

2x 29%less time spent

remediating

security issues

more time spent

on new work

Source: Puppet Labs 2016 State Of DevOps Report: https://puppet.com/resources/white-paper/2016-state-of-devops-report

@RealGeneKim

High Performers Win In The Marketplace

2x 2xmore likely to

exceed profitability,

market share &

productivity goals

more likely to achieve

organizational and

mission goals, customer

satisfaction, quantity &

quality goalsSource: Puppet/DORA: 2017 State Of DevOps Report: https://puppet.com/resources/whitepaper/state-of-devops-report

@RealGeneKim

High Performers Win In The Marketplace

2.2xhigher employee

Net Promoter Score

Source: Puppet Labs 2016 State Of DevOps Report: https://puppet.com/resources/white-paper/2016-state-of-devops-report

50%higher market

capitalization growth

over 3 years*

@RealGeneKim

“deploys / day”

“deploys / day / dev”

@RealGeneKim

High (linear)

Low

Med

Source: Puppet Labs 2015 State Of DevOps: https://puppetlabs.com/2015-devops-report

de

plo

ys

/ d

ay

# of developers

@RealGeneKim

We can achieve all the goals, dreams and aspirations of our

business…

@RealGeneKim

Session ID:

Surprise #2:

DevOps Is As Good For Ops…

@RealGeneKim

Session ID:

Surprise #2:

DevOps Is As Good For Ops…...As It Is For Dev!

@RealGeneKim

Deploy Smaller Changes, More Frequently *

Source: http://www.facebook.com/note.php?note_id=14218138919

@RealGeneKim

“As a lifelong Ops practitioner, I know we need DevOps to make our work humane.

In the past, I’ve worked every holiday, on my birthday, my spouse’s birthday, and even on the day my son was born.”

Nathan ShimekEngineering Manager, New Context

@nathan_shimek

@RealGeneKim

CSG: COBOL App + 20 tech stacks

Source: Scott Prugh, Chief Architect, CSG, Inc.

And the customer got the feature in half the

time!

Apps supporting bill printing and customer care for 50MM customer, 6B transactions per month

20 technology platforms, including mainframe VSAM and DB2, Java, desktop client

Moved from 2 to 4 releases per year

Shared Operations Team performed daily deployments to UAT

@RealGeneKim

Developers Carry Pagers

“We found that when we woke up developers at

2am, defects got fixed faster than ever”

– Patrick Lightbody,

VP Prod Mgmt, New Relic

“You build it, you run it.”

– Werner Vogels

CTO, Amazon

@RealGeneKim

“As a developer, the most satisfying points in my career?

“It’s when I wrote the code, pushed the button to deploy it, watched the metrics to see if it actually worked in production, and fixed it if it broke.”

Tim TischlerDirector of Operations EngineeringNike, Inc.

@RealGeneKim

Session ID:

Surprise #3:

The Importance Of MeasuringCode Deployment Lead Time

@RealGeneKim

“deploys per day”

vs.

“lead time”

@RealGeneKim

“What is your lead time for changes?”

“How long does it take to go from code committed to code successfully

running in production?”

@RealGeneKimSource: The DevOps Handbook

Change Committed Into Version Control

@RealGeneKim

What Is The One Question That Predicts Performance With

Startling Accuracy?

@RealGeneKim

“To what degree do we fear doing deployments?”

Source: Puppet Labs 2015 State Of DevOps: https://puppetlabs.com/2015-devops-report

@RealGeneKim

Session ID:

Surprise #4:

The Surprising Implications Of Conway’s Law

@RealGeneKim

Conway’s Law

Eric S. Raymond: “If you have four groups

working on a compiler, you’ll get a four pass

compiler”

(summarizing results of Dr. Melvin Conway’s

experiment in 1968)

@RealGeneKim

The Birth And Death Of Etsy Sprouter

A story about teams of engineers implementing

changes

2008: Devs and DBAs

2009: Devs and DBAs and Sprouter team

2010: Devs

@RealGeneKim

Lesson:The Organization and

The Architecture Of Our SoftwareMust Be Congruent

@RealGeneKim

Lead Time = 9 months

Source: Damon Edwards (@damonedwards)

@RealGeneKim

Architecture Enables Teams To…

…make large scale changes to the design of its system without the permission of someone outside the team, or depending on other teams

...complete its work without fine-grained communication and coordination with people outside the team

...deploy and release its product or service on demand, independently of other services the product or service depends upon

...do most of its testing on demand, without requiring an integrated test environment

...perform deployments during normal business hours with negligible downtime


@RealGeneKim

Session ID:

Surprise #5:

DevOps EnablesDynamic, Learning Organizations…

@RealGeneKim

Dr. Steven Spear

“While designing perfectly safe systems is likely beyond our abilities, safe systems are close to achievable” when the four following conditions are met…”

Source: Dr. Steven Spear

@RealGeneKim

Dr. Steven Spear’s Four Capabilities

1. See problems as they occur

2. Swarm and solve problems to create new knowledge

3. Spread new knowledge throughout the organization

4. Leaders create new leaders


@RealGeneKim

DevOps Practices In Capability 3

Single, shared source code repositories

Blameless post-mortems

Chaos Monkeys

@RealGeneKim

DevOps Practices In Capability 3

Learning days and internal technology conferences

Embracing open source

Internal architecture to propagate best known patterns (“buoys, not boundaries”)

Functional organizations

@RealGeneKim

You Don’t Choose Chaos Monkey…Chaos Monkey Chooses You

@RealGeneKim

Capability 4

Leading by developing

The job of leaders is not to command and control, but to create other capable leaders who can perpetuate this system of work


“My goal is not to direct and control, but to guide and enable”

@RealGeneKim

Session ID:

Surprise #6:

DevOps Is For The Unicorns…...And The Horses, Too

@RealGeneKim

DevOps Enterprise: Lessons Learned

On Nov. 7-9, we held the third DevOps Enterprise Summit, a conference for horses, by horses

Speakers included fifty leaders from:

Barclays, ING Bank, UK HMRC, Hiscox, Zurich Insurance, LV, UK GDS, iTV, Unilever, SAP, Macy’s, Disney, Target, GE Capital, Western Union, Sherwin Williams, Blackboard, Nordstrom, Telstra, US Department of Homeland Security, CSG, Raytheon, IBM, Ticketmaster, MITRE, Marks and Spencer, Barclays Capital, Microsoft, Nationwide Insurance, Capital One, Gov.UK, Fidelity, Rally Software, Neustar, Walmart, PNC, ADP, …

@RealGeneKim

Observations

They were using the same technical practices and getting the same

sort of metrics as the unicorns

Target: 100+ deploys per week, < 10 incidents per month, enabled 53

business initiatives

Capital One: 100s of deploys per day, lead time of minutes

Macy’s: 1,500 manual tests every 10 days, now 100Ks automated tests run

daily

Disney: Has embedded nearly 100 Ops engineers into LOB teams across the

enterprise

Nationwide Insurance: Retirement Plans app (COBOL on mainframe)

Raytheon: testing and certification from months to a day

Nordstrom: 20% lead time reduction into executive bonuses

@RealGeneKim

Mieke DeenenProject Manager Deployment AutomationUWV, Netherlands

@RealGeneKim

@RealGeneKimSource: Puppet/DORA: 2017 State Of DevOps Report: https://puppet.com/resources/whitepaper/state-of-devops-report

@RealGeneKim

Leaders Cannot Do It Alone

Teams with the least transformational leaders (the bottom third) were one-half as likely to be high IT performers

Leaders cannot do it alone! Teams with the top 10% of transformational leaders performed no better than the median


@RealGeneKim

Leaders Affect Outcomes Through…


@RealGeneKim

Aspirational Suggestion

Congratulations for engineering yourselves into a unique situation where you can affect how much organizations fear performing deployments

Find ways to enable your “pocket of greatness” to be leveraged throughout your entire organization

May require partnering with developers, Dev and Ops leadership, and architecture

The rewards are worth it

@RealGeneKim

Why Do I Think This Is Important?

@RealGeneKim

@RealGeneKimSource: Pingdom

@RealGeneKim

The DevOps Handbook

5+ years in the making

23 chapters

48 case studies

98,124 words

48 images

503 endnotes

192 footnotes

@RealGeneKim

Want More Learn More?

To receive the following:

A copy of this presentation

The 140 page excerpt of The DevOps Handbook

The 140 page excerpt of The Phoenix Project

Videos and slides from DevOps Enterprise 2014-2017

Whitepaper from DevOps Research and Assessment

The DevOps Enterprise Forum Guidance Papers

Link to the DevOps Audit Defense Toolkit

One hour excerpt of The Phoenix Project audiobook

Just pick up your phone, and send an email:

To: [email protected]

Subject: devops

[email protected]

devops