devnet-2015devops in depth - damon edwards on devops kaizen: building an enterprise’s capability...

90
DevOps Kaizen: Building and Enterprise’s Capability to Change © DTO Solutions, Inc. v1.2

Upload: cisco-devnet

Post on 15-Aug-2015

191 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen: Building and Enterprise’s Capability to Change

© DTO Solutions, Inc. v1.2

Page 2: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

@damonedwards

Damon Edwards

Page 3: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

@damonedwards

Damon Edwards

I’ll post link to slides later today

Page 4: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Consulting

Automation Design

Tools

Page 5: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

What’s the real difference between high-performers and low-performers?

Page 6: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

It’s the single most important skill for any company to master…

Page 7: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

The ability to improve.

Page 8: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

a quick note about improvement…

Page 9: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

The only problems that matter are those that prevent the business from reaching its goals.

Page 10: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

OpsDev

We are in business to give the customer... • What they want • When they want it • At the lowest cost possible

Page 11: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

OpsDev

DevOps Problems!

Page 12: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Ah-ha!

OpsDev Remove

Shorten

Lead Time

Feedback

Page 13: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Why aren’t most organizations “good at getting better”?

Page 14: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Improvement already has a well known recipe: Plan - Do - Study - Act (PDSA)

Other variants: PDCA OODA

W. Edwards Deming - 1950

© The Deming Institute

Page 15: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Then why hasn’t everyone followed the recipe?

Page 16: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Main culprit: the work isn’t visible

Page 17: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Photo credit: Doc Searls on Flickr

The “hands on the keyboard” experience

Page 18: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Photo credit: Doc Searls on Flickr

OpsDev

The “hands on the keyboard” experience

Page 19: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Photo credit: Doc Searls on Flickr

The “hands on the keyboard” experience

I can fix it!

Page 20: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Photo credit: Doc Searls on Flickr

Local optimization often hurts the system

Page 21: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

The management experience: the mirage

Ah-ha!

Page 22: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

The management experience: the reality

?Ah-ha!

“Everything takes too long” “Everyone is busy but not much is getting done” “Things break, and then break again and again” “All hands on deck and heroics to get anything done” “Handoffs are painful. Lots of miscommunication and rewok”

Page 23: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

?Ah-ha!

The management experience: the reality

“Everything takes too long” “Everyone is busy but not much is getting done” “Things break, and then break again and again” “All hands on deck and heroics to get anything done” “Lots of miscommunication and rework”

Page 24: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

?Ah-ha!

Decisions are more of a (hopeful) educated guess

Page 25: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

So how do you “get good at getting better”?…

Page 26: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

1. Make the work visible

So how do you “get good at getting better”?…

Page 27: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

1. Make the work visible2. Use that visibility to align the organization

So how do you “get good at getting better”?…

Page 28: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen

Page 29: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen

• “Kaizen”: Japanese word for improvement

Page 30: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen

• “Kaizen”: Japanese word for improvement

• Modern business context:

Page 31: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen

• “Kaizen”: Japanese word for improvement

• Modern business context:• Continuous improvement

Page 32: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen

• “Kaizen”: Japanese word for improvement

• Modern business context:• Continuous improvement• Systematic, scientific-method approach

Page 33: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen

• “Kaizen”: Japanese word for improvement

• Modern business context:• Continuous improvement• Systematic, scientific-method approach • Total engagement of the workforce

Page 34: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen

• “Kaizen”: Japanese word for improvement

• Modern business context:• Continuous improvement• Systematic, scientific-method approach • Total engagement of the workforce• Valuing small changes as much as large changes

(outcome is what matters)

Page 35: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen

• “Kaizen”: Japanese word for improvement

• Modern business context:• Continuous improvement• Systematic, scientific-method approach • Total engagement of the workforce• Valuing small changes as much as large changes

(outcome is what matters)

• Kaizen in DevOps context:

Page 36: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen

• “Kaizen”: Japanese word for improvement

• Modern business context:• Continuous improvement• Systematic, scientific-method approach • Total engagement of the workforce• Valuing small changes as much as large changes

(outcome is what matters)

• Kaizen in DevOps context: • Continuously improve the flow of work through the full

value stream in order to improve customer outcomes

Page 37: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

ServiceDeliveryMetrics

KaizenProgramOversight

Planning&

Retrospectives Informs Informs

Countermeasures &Blockers

Elements of a DevOps Kaizen Program

Page 38: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

ServiceDeliveryMetrics

KaizenProgramOversight

Planning&

Retrospectives Informs Informs

Countermeasures &Blockers

Elements of a DevOps Kaizen Program

Page 39: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

ServiceDeliveryMetrics

KaizenProgramOversight

Planning&

Retrospectives Informs Informs

Countermeasures &Blockers

Elements of a DevOps Kaizen Program

Page 40: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

1. The will to make change happen 2. The resources to make change happen 3. Follow-through / clear obstacles

Leading the Improvement

Page 41: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

1. The will to make change happen 2. The resources to make change happen 3. Follow-through / clear obstacles

Leading the Improvement

This (and only this) is what the Kaizen Program Oversight Group

does!

Page 42: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

ServiceDeliveryMetrics

KaizenProgramOversight

Planning&

Retrospectives Informs Informs

Countermeasures &Blockers

Elements of a DevOps Kaizen Program

Page 43: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

ServiceDeliveryMetrics

KaizenProgramOversight

Planning&

Retrospectives Informs Informs

Countermeasures &Blockers

Elements of a DevOps Kaizen Program

This is where the work becomes visible!

Page 44: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Map end-to-end process1

Include key process metrics:

Lead Time Processing Time Scrap Rate Head Count

DevOps Kaizen: Retrospective Technique

Page 45: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Identify wastes, inefficiencies, bottlenecks

PD - Partially Done TS - Task Switching W - Waiting M - Motion / Manual

D - Defects EP - Extra Process EF - Extra Features HB - Heroics

Structured approach building on DevOps adaptation of “7 deadly wastes” from Lean / Agile:

2

DevOps Kaizen: Retrospective Technique

Page 46: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Creative

Business

Release Engineering

Development

QA

Technical Operations

Program Management

L/T Lead timeP/T Process timeH/C Head countS/R Scrap rate

D DefectsEP Extra processesM Motion

PD Partially doneTS Task switchingW Waiting

Core business service delivery process Current state value stream map

DevelopmentEnvironments

ReleaseBuild and deploy

QAEnvironments

PerformanceEnvironment

StagingEnvironment

ProductionEnvironment

Merge project branch

Deploy to Performance

Web Assets, flows and

wireframes

Delta features and

fixesSet of service

branches

Documented test plans and cases

Request QA build

Daily Scrum meeting

Daily Scrum meeting

Deployment Meeting Business Systems

Support

Stan Walters

Change Control

Project Manager

Erica Switzer

Change request

Business Analyst

Erica Switzer

Steering Committee

Erica Switzer

Full Projects / Expedites

Estimates and

requirements

Technical Specification

Deployment Document

Change Control Forms

RT tickets

Jira issues

Jira issues

Tested packages

Development

Joe Harbaugh

L/T 8 weeksP/TH/CS/R

4 weeks525%

Developweb assets

Bob McNulty

L/T 6 weeksP/TH/CS/R

4 weeks125%

DevelopmentBuild and deploy

Joe Harbaugh

L/T 2 daysP/TH/CS/R

4 hours110%

Development Integration testing

Joe Harbaugh

L/T 1 dayP/TH/CS/R

1 day595%

Performance testing

Deploy toProduction

Build test plan

Raj Lee

L/T 5 daysP/TH/CS/R

4 days110%

Build test cases

Raj Lee

L/T 2 weeksP/TH/CS/R

8 days220%

Fernando Gomez

L/T 8 hoursP/TH/CS/R

7 hours12%

Acceptance testing

Raj Lee

L/T 5 daysP/TH/CS/R

5 hours210%

Feature testing

Raj Lee

L/T 4 weeksP/TH/CS/R

2.5 weeks420%

Allen Cannata

L/T 1 weekP/TH/CS/R

3 days166%

RT tickets

Cut project branch

Chris Holmes

L/T 15 minutesP/TH/CS/R

5 minutes10%

Deploy to Staging

Frances Middleton

L/T 6 daysP/TH/CS/R

90 minutes366%

Staging testing

Frances Middleton

L/T 1 weekP/TH/CS/R

3.5 days466%

Allen Cannata

L/T 9 hoursP/TH/CS/R

8 hours666%

Frances Middleton

L/T 6 daysP/TH/CS/R

90 minutes366%

Fernando Gomez

L/T 30 minutesP/TH/CS/R

30 minutes133%

Process Time

D

EP

Dx2EP

Mx2W

DPD

Dx3Mx3

TSx5 Dx2 Wx2 PDx4

EP

DPDx3 Mx8

DWx2

PDx2

What should you be looking for?• Long lead times (vs processing time) • Request and approval queues • Large batch sizes • Human to human information transfer • Scrap / Rework / “Backwash” • Process or tooling inconsistency • Manual deployment / configuration • Reliance on manual testing / verification • Injecting requirements late in the process • Fixing in place (vs rebuilding) • “Snowflake” servers and environments • Manual dependency management • Ad-hoc or unstructured artifact management

Page 47: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen: Retrospective Technique

Identify countermeasures

Countermeasures must be actionable, backlog ready. Focus on short-term “baby steps”. Note broader, strategic recommendations.

3

Page 48: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen: Behaviors to Encourage

Page 49: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Learn Fast, Fail Early Verification tests catch problems early. Tests accumulate to build the “business immune system.

DevOps Kaizen: Behaviors to Encourage

Page 50: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Learn Fast, Fail Early Verification tests catch problems early. Tests accumulate to build the “business immune system.

Work in Small Batches Work is planned and executed in small batches (made continuously and operations-ready early).

DevOps Kaizen: Behaviors to Encourage

Page 51: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Learn Fast, Fail Early Verification tests catch problems early. Tests accumulate to build the “business immune system.

Standard Processes Making Standard Parts Specs and recipes produce infrastructure and app components. No more “snowflakes”.

Work in Small Batches Work is planned and executed in small batches (made continuously and operations-ready early).

DevOps Kaizen: Behaviors to Encourage

Page 52: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Learn Fast, Fail Early Verification tests catch problems early. Tests accumulate to build the “business immune system.

Shared Ops Visibility Live visibility into apps/environments. Single page, simple shared conventions part of DoD

Standard Processes Making Standard Parts Specs and recipes produce infrastructure and app components. No more “snowflakes”.

Work in Small Batches Work is planned and executed in small batches (made continuously and operations-ready early).

DevOps Kaizen: Behaviors to Encourage

Page 53: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Learn Fast, Fail Early Verification tests catch problems early. Tests accumulate to build the “business immune system.

Shared Ops Visibility Live visibility into apps/environments. Single page, simple shared conventions part of DoD

Standard Processes Making Standard Parts Specs and recipes produce infrastructure and app components. No more “snowflakes”.

Ops Requirements are Product Features Performance, Stability, Configurability are all essential features for a successful product.

Work in Small Batches Work is planned and executed in small batches (made continuously and operations-ready early).

DevOps Kaizen: Behaviors to Encourage

Page 54: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Learn Fast, Fail Early Verification tests catch problems early. Tests accumulate to build the “business immune system.

Shared Ops Visibility Live visibility into apps/environments. Single page, simple shared conventions part of DoD

Standard Processes Making Standard Parts Specs and recipes produce infrastructure and app components. No more “snowflakes”.

Ops Requirements are Product Features Performance, Stability, Configurability are all essential features for a successful product.

Self-Service Operations People consume operations services rather than opening tickets for others to do it for them.

Work in Small Batches Work is planned and executed in small batches (made continuously and operations-ready early).

DevOps Kaizen: Behaviors to Encourage

Page 55: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen: Retrospective Technique

Create Improvement Storyboards (Kata Style) 4

Page 56: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen: Planning Technique

Draw map in this direction

What are we going to do? Who do we need to do it? What do we need to do it? What’s going to get in our way?

Page 57: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Parts Fulfillment Networks (a division of Parts Unlimited)

Page 58: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Parts Fulfillment Networks (a division of Parts Unlimited)

● “How do we protect the previous $2B in revenue while innovating towards next $2B… without doubling the team?”

Page 59: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Parts Fulfillment Networks (a division of Parts Unlimited)

● “How do we protect the previous $2B in revenue while innovating towards next $2B… without doubling the team?”

● Schedule slippage and cost of delay impacting customer relationships

Page 60: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Parts Fulfillment Networks (a division of Parts Unlimited)

● “How do we protect the previous $2B in revenue while innovating towards next $2B… without doubling the team?”

● Schedule slippage and cost of delay impacting customer relationships

● Everybody looks busy (40+ concurrent projects) but nothing gets done

Page 61: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Parts Fulfillment Networks (a division of Parts Unlimited)

● “How do we protect the previous $2B in revenue while innovating towards next $2B… without doubling the team?”

● Schedule slippage and cost of delay impacting customer relationships

● Everybody looks busy (40+ concurrent projects) but nothing gets done

● “We love doing things so much we do them 3 or 4 times”

Page 62: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Parts Fulfillment Networks (a division of Parts Unlimited)

● “How do we protect the previous $2B in revenue while innovating towards next $2B… without doubling the team?”

● Schedule slippage and cost of delay impacting customer relationships

● Everybody looks busy (40+ concurrent projects) but nothing gets done

● “We love doing things so much we do them 3 or 4 times”● “Customers find the problems before we do”

Page 63: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

Parts Fulfillment Networks (a division of Parts Unlimited)

● “How do we protect the previous $2B in revenue while innovating towards next $2B… without doubling the team?”

● Schedule slippage and cost of delay impacting customer relationships

● Everybody looks busy (40+ concurrent projects) but nothing gets done

● “We love doing things so much we do them 3 or 4 times”● “Customers find the problems before we do”● Business unit leaders pressuring to “route around” Core IT

Page 64: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

PunchdownApprovals

Parts Unlimited - High Availability for Acme Co.

Evaluate Needs

Pro Serve Leads

Share Point

3 months

"Susan T."VP of Vertical

Act Mgmt

"Pat"Owner of Acme Co

Act.

Acme Co.

Reqs

Cust.Config

Pro Services PM

Begin Contract Negotiation

Evaluate Risk for Deliveries to

SLA

Pro Services

Legal

Ops

Reqs

"David"Service Level Manager

Finance Reporting

Project Delivery

ExpectationBU Owner

5 months total

Pro Services

BA's

Write Specs for Fulfillment

Share Point

.doc

Discuss Planning

Attendees

Project Kickoff Meeting

ProServices Ops

PM

AppSupport

Share Point

AppSupport

TicketsHigh Level

About Project

Estimate and Priorities

"Linda"Ops PM

Child ticket

"Project Estimate"

Create Project

Estimate Package

SystemsNetworksSecurityLeads

Core Network Provider

Share Point

.ppt

BOM.xls

Function Specific

Project Estimate estimate

"BOM.xls"

Set Timeline

Ops PM

Project Estimate

Judge Costs & Approval

AppSupport

Partner BU Business

Case

3 weeks

Create App Spec

Share Point

Tech Leads Review App

Spec

Dev LeadsPro Services

Pro Services PM

App Spec.doc

DevelopmentApp Spec

Reqs

Status Updates

Availability

Pro Services Lead

Acme Co.

Share Point

Create Build Request

Project Estimate

"Linda"Ops PM

"Linda""Denis"Ops PM

Add Design

.xls

Ops Engineer"Chuck"

Ops Specialists

1 month

Share Point

Build.xls

Create Change Tickets > 100

Build.xls

Service Now

NetFacilityCablingStorage

"Linda"Ops PM

RESET DELIVERY

DATE!

Acme Co.

Fix Tickets!

Senior Ops Engr

Create DB Schema

Ops DBA

Conflict Resolution

3 weeks

BA AppDev

NetEngrLead

AssignTickets

Service Now

"Chris"Ops

Engineer

Procure Hardware

Procurement SaaS

Install Hardware

SA Install OS Version (2)

Jumpstart Images

Service Now

Run CablingOps

Stock Clerk

Receive and Tag

Create DB VMs(6)

Install Hw Req

Cluster Setup

Creat App servers

Solaris Zones(10) Websphere

NFS

Write Data Migration Scripts

App DBA

UATUS

UATEurope

PRODUS

PRODEurope

F5 RulesPro

Services

External DNS

SSLCertificate

CreateVIP

CreateExternal IP

CreateSubnet

FirewallRules

Status Event

ProServices

Service Now

Create UAT Deploy Ticket

Middleware (code)DBA (SQL)Network

Approvals

"Marvin"Middleware

Ops

Deploy EAR

Configure to the data sources

Restart App

Request Testing to

Verify

Admin01

NFS

UAT

App SupportPro Services

BuildServer

EAR copied (FTP)

Promote AppDev

SmokeTest

App Support

Share Point

App SupportTest Script

Run Functional

Test

Test Plan.xls or .doc

Pro Services

BAWrite Test Plan

Customer Sign Off on Test

Plan

Acme Co.

"Linda"PM

CoordinateOperationalReadiness

Testing

UAT PROD

Acme Co.

Testing

Repeated in Prod

Middleware

DB Migration

Re-Sync DB

DBA

(happened earlier)

1 Month

Create Production Readiness

Meeting

DRHelpdeskTeam LeadsAppsOps PM

Share Point

TicketSystem

Go Live

Ops PM

Phone Bridge

1 Week 2 Weeks

Turn off old site

Run final DB sync

Customer uses new

VIP

Renegotiate

Create Build Out Doc

QA Test DITUS

4 hour change window2x per week

W

UAT Deploy and QA

Go Live .xls

Go Live .xls

UAT and PROD Network Config

UAT and PROD Server Provisioning

NetworkOps

Teams

Pro Services

Address Management

M

PROD Deploy and QACreate Ops

Tickets

App Dev and QA

App Specification

Financial Approvaland Project Commitment

Project Estimate

Sales and Customer Analysis

Sales and Delivery Risk

Analysis

Data Setup

Verification

PD

MW

PD

D

M

PD

PDM

TS

D M TS

HB

HB

M

TS

D

PD

W

M

M

M

M

M

PD

D

PDM

M

PD

D M

TSD

W

TSD

PD

W

TS W

"Incorrect or invalid rules"

"Missing Routes"

"Certificate Errors"

"Stale or conflicts"

Ops Requirements VerificationOps

Requirements

Verification

Small Batches

Ops Requirements

Verification

Verification

Ops Requirements

Standard Parts Small Batches

Verification

Small Batches

Ops Requirements

Verification

Verification

Standard Parts

Verification Standard Parts

Standard Parts

Ops Requirements

Verification

Ops Requirements

Ops HealthDashboard

Standard Parts

Ops HealthDashboard

Ops HealthDashboard

Ops HealthDashboardOps Health

Dashboard

Standard Parts

Verification

Verification

Verification

Ops Requirements

Ops HealthDashboard

Verification

Standard Parts

Standard Parts

Reqs Cust.Config

AppSpec Project EstimateProject

Estimate

Biz CaseApproval

Build Doc

Design & Spec

Project Plan

Tickets

~8 months before implementation starts

(and information / requirements still

incorrect or incomplete!)

Tickets Tickets TicketsAd-HocEmails-Missing details-New technical requirements-Missing context

Information Flow

PD

Page 65: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

PunchdownApprovals

Parts Unlimited - High Availability for Acme Co.

Evaluate Needs

Pro Serve Leads

Share Point

3 months

"Susan T."VP of Vertical

Act Mgmt

"Pat"Owner of Acme Co

Act.

Acme Co.

Reqs

Cust.Config

Pro Services PM

Begin Contract Negotiation

Evaluate Risk for Deliveries to

SLA

Pro Services

Legal

Ops

Reqs

"David"Service Level Manager

Finance Reporting

Project Delivery

ExpectationBU Owner

5 months total

Pro Services

BA's

Write Specs for Fulfillment

Share Point

.doc

Discuss Planning

Attendees

Project Kickoff Meeting

ProServices Ops

PM

AppSupport

Share Point

AppSupport

TicketsHigh Level

About Project

Estimate and Priorities

"Linda"Ops PM

Child ticket

"Project Estimate"

Create Project

Estimate Package

SystemsNetworksSecurityLeads

Core Network Provider

Share Point

.ppt

BOM.xls

Function Specific

Project Estimate estimate

"BOM.xls"

Set Timeline

Ops PM

Project Estimate

Judge Costs & Approval

AppSupport

Partner BU Business

Case

3 weeks

Create App Spec

Share Point

Tech Leads Review App

Spec

Dev LeadsPro Services

Pro Services PM

App Spec.doc

DevelopmentApp Spec

Reqs

Status Updates

Availability

Pro Services Lead

Acme Co.

Share Point

Create Build Request

Project Estimate

"Linda"Ops PM

"Linda""Denis"Ops PM

Add Design

.xls

Ops Engineer"Chuck"

Ops Specialists

1 month

Share Point

Build.xls

Create Change Tickets > 100

Build.xls

Service Now

NetFacilityCablingStorage

"Linda"Ops PM

RESET DELIVERY

DATE!

Acme Co.

Fix Tickets!

Senior Ops Engr

Create DB Schema

Ops DBA

Conflict Resolution

3 weeks

BA AppDev

NetEngrLead

AssignTickets

Service Now

"Chris"Ops

Engineer

Procure Hardware

Procurement SaaS

Install Hardware

SA Install OS Version (2)

Jumpstart Images

Service Now

Run CablingOps

Stock Clerk

Receive and Tag

Create DB VMs(6)

Install Hw Req

Cluster Setup

Creat App servers

Solaris Zones(10) Websphere

NFS

Write Data Migration Scripts

App DBA

UATUS

UATEurope

PRODUS

PRODEurope

F5 RulesPro

Services

External DNS

SSLCertificate

CreateVIP

CreateExternal IP

CreateSubnet

FirewallRules

Status Event

ProServices

Service Now

Create UAT Deploy Ticket

Middleware (code)DBA (SQL)Network

Approvals

"Marvin"Middleware

Ops

Deploy EAR

Configure to the data sources

Restart App

Request Testing to

Verify

Admin01

NFS

UAT

App SupportPro Services

BuildServer

EAR copied (FTP)

Promote AppDev

SmokeTest

App Support

Share Point

App SupportTest Script

Run Functional

Test

Test Plan.xls or .doc

Pro Services

BAWrite Test Plan

Customer Sign Off on Test

Plan

Acme Co.

"Linda"PM

CoordinateOperationalReadiness

Testing

UAT PROD

Acme Co.

Testing

Repeated in Prod

Middleware

DB Migration

Re-Sync DB

DBA

(happened earlier)

1 Month

Create Production Readiness

Meeting

DRHelpdeskTeam LeadsAppsOps PM

Share Point

TicketSystem

Go Live

Ops PM

Phone Bridge

1 Week 2 Weeks

Turn off old site

Run final DB sync

Customer uses new

VIP

Renegotiate

Create Build Out Doc

QA Test DITUS

4 hour change window2x per week

W

UAT Deploy and QA

Go Live .xls

Go Live .xls

UAT and PROD Network Config

UAT and PROD Server Provisioning

NetworkOps

Teams

Pro Services

Address Management

M

PROD Deploy and QACreate Ops

Tickets

App Dev and QA

App Specification

Financial Approvaland Project Commitment

Project Estimate

Sales and Customer Analysis

Sales and Delivery Risk

Analysis

Data Setup

Verification

PD

MW

PD

D

M

PD

PDM

TS

D M TS

HB

HB

M

TS

D

PD

W

M

M

M

M

M

PD

D

PDM

M

PD

D M

TSD

W

TSD

PD

W

TS W

"Incorrect or invalid rules"

"Missing Routes"

"Certificate Errors"

"Stale or conflicts"

Ops Requirements VerificationOps

Requirements

Verification

Small Batches

Ops Requirements

Verification

Verification

Ops Requirements

Standard Parts Small Batches

Verification

Small Batches

Ops Requirements

Verification

Verification

Standard Parts

Verification Standard Parts

Standard Parts

Ops Requirements

Verification

Ops Requirements

Ops HealthDashboard

Standard Parts

Ops HealthDashboard

Ops HealthDashboard

Ops HealthDashboardOps Health

Dashboard

Standard Parts

Verification

Verification

Verification

Ops Requirements

Ops HealthDashboard

Verification

Standard Parts

Standard Parts

Reqs Cust.Config

AppSpec Project EstimateProject

Estimate

Biz CaseApproval

Build Doc

Design & Spec

Project Plan

Tickets

~8 months before implementation starts

(and information / requirements still

incorrect or incomplete!)

Tickets Tickets TicketsAd-HocEmails-Missing details-New technical requirements-Missing context

Information Flow

PD

Page 66: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

PunchdownApprovals

Parts Unlimited - High Availability for Acme Co.

Evaluate Needs

Pro Serve Leads

Share Point

3 months

"Susan T."VP of Vertical

Act Mgmt

"Pat"Owner of Acme Co

Act.

Acme Co.

Reqs

Cust.Config

Pro Services PM

Begin Contract Negotiation

Evaluate Risk for Deliveries to

SLA

Pro Services

Legal

Ops

Reqs

"David"Service Level Manager

Finance Reporting

Project Delivery

ExpectationBU Owner

5 months total

Pro Services

BA's

Write Specs for Fulfillment

Share Point

.doc

Discuss Planning

Attendees

Project Kickoff Meeting

ProServices Ops

PM

AppSupport

Share Point

AppSupport

TicketsHigh Level

About Project

Estimate and Priorities

"Linda"Ops PM

Child ticket

"Project Estimate"

Create Project

Estimate Package

SystemsNetworksSecurityLeads

Core Network Provider

Share Point

.ppt

BOM.xls

Function Specific

Project Estimate estimate

"BOM.xls"

Set Timeline

Ops PM

Project Estimate

Judge Costs & Approval

AppSupport

Partner BU Business

Case

3 weeks

Create App Spec

Share Point

Tech Leads Review App

Spec

Dev LeadsPro Services

Pro Services PM

App Spec.doc

DevelopmentApp Spec

Reqs

Status Updates

Availability

Pro Services Lead

Acme Co.

Share Point

Create Build Request

Project Estimate

"Linda"Ops PM

"Linda""Denis"Ops PM

Add Design

.xls

Ops Engineer"Chuck"

Ops Specialists

1 month

Share Point

Build.xls

Create Change Tickets > 100

Build.xls

Service Now

NetFacilityCablingStorage

"Linda"Ops PM

RESET DELIVERY

DATE!

Acme Co.

Fix Tickets!

Senior Ops Engr

Create DB Schema

Ops DBA

Conflict Resolution

3 weeks

BA AppDev

NetEngrLead

AssignTickets

Service Now

"Chris"Ops

Engineer

Procure Hardware

Procurement SaaS

Install Hardware

SA Install OS Version (2)

Jumpstart Images

Service Now

Run CablingOps

Stock Clerk

Receive and Tag

Create DB VMs(6)

Install Hw Req

Cluster Setup

Creat App servers

Solaris Zones(10) Websphere

NFS

Write Data Migration Scripts

App DBA

UATUS

UATEurope

PRODUS

PRODEurope

F5 RulesPro

Services

External DNS

SSLCertificate

CreateVIP

CreateExternal IP

CreateSubnet

FirewallRules

Status Event

ProServices

Service Now

Create UAT Deploy Ticket

Middleware (code)DBA (SQL)Network

Approvals

"Marvin"Middleware

Ops

Deploy EAR

Configure to the data sources

Restart App

Request Testing to

Verify

Admin01

NFS

UAT

App SupportPro Services

BuildServer

EAR copied (FTP)

Promote AppDev

SmokeTest

App Support

Share Point

App SupportTest Script

Run Functional

Test

Test Plan.xls or .doc

Pro Services

BAWrite Test Plan

Customer Sign Off on Test

Plan

Acme Co.

"Linda"PM

CoordinateOperationalReadiness

Testing

UAT PROD

Acme Co.

Testing

Repeated in Prod

Middleware

DB Migration

Re-Sync DB

DBA

(happened earlier)

1 Month

Create Production Readiness

Meeting

DRHelpdeskTeam LeadsAppsOps PM

Share Point

TicketSystem

Go Live

Ops PM

Phone Bridge

1 Week 2 Weeks

Turn off old site

Run final DB sync

Customer uses new

VIP

Renegotiate

Create Build Out Doc

QA Test DITUS

4 hour change window2x per week

W

UAT Deploy and QA

Go Live .xls

Go Live .xls

UAT and PROD Network Config

UAT and PROD Server Provisioning

NetworkOps

Teams

Pro Services

Address Management

M

PROD Deploy and QACreate Ops

Tickets

App Dev and QA

App Specification

Financial Approvaland Project Commitment

Project Estimate

Sales and Customer Analysis

Sales and Delivery Risk

Analysis

Data Setup

Verification

PD

MW

PD

D

M

PD

PDM

TS

D M TS

HB

HB

M

TS

D

PD

W

M

M

M

M

M

PD

D

PDM

M

PD

D M

TSD

W

TSD

PD

W

TS W

"Incorrect or invalid rules"

"Missing Routes"

"Certificate Errors"

"Stale or conflicts"

Ops Requirements VerificationOps

Requirements

Verification

Small Batches

Ops Requirements

Verification

Verification

Ops Requirements

Standard Parts Small Batches

Verification

Small Batches

Ops Requirements

Verification

Verification

Standard Parts

Verification Standard Parts

Standard Parts

Ops Requirements

Verification

Ops Requirements

Ops HealthDashboard

Standard Parts

Ops HealthDashboard

Ops HealthDashboard

Ops HealthDashboardOps Health

Dashboard

Standard Parts

Verification

Verification

Verification

Ops Requirements

Ops HealthDashboard

Verification

Standard Parts

Standard Parts

Reqs Cust.Config

AppSpec Project EstimateProject

Estimate

Biz CaseApproval

Build Doc

Design & Spec

Project Plan

Tickets

~8 months before implementation starts

(and information / requirements still

incorrect or incomplete!)

Tickets Tickets TicketsAd-HocEmails-Missing details-New technical requirements-Missing context

Information Flow

PD

Page 67: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

PunchdownApprovals

Parts Unlimited - High Availability for Acme Co.

Evaluate Needs

Pro Serve Leads

Share Point

3 months

"Susan T."VP of Vertical

Act Mgmt

"Pat"Owner of Acme Co

Act.

Acme Co.

Reqs

Cust.Config

Pro Services PM

Begin Contract Negotiation

Evaluate Risk for Deliveries to

SLA

Pro Services

Legal

Ops

Reqs

"David"Service Level Manager

Finance Reporting

Project Delivery

ExpectationBU Owner

5 months total

Pro Services

BA's

Write Specs for Fulfillment

Share Point

.doc

Discuss Planning

Attendees

Project Kickoff Meeting

ProServices Ops

PM

AppSupport

Share Point

AppSupport

TicketsHigh Level

About Project

Estimate and Priorities

"Linda"Ops PM

Child ticket

"Project Estimate"

Create Project

Estimate Package

SystemsNetworksSecurityLeads

Core Network Provider

Share Point

.ppt

BOM.xls

Function Specific

Project Estimate estimate

"BOM.xls"

Set Timeline

Ops PM

Project Estimate

Judge Costs & Approval

AppSupport

Partner BU Business

Case

3 weeks

Create App Spec

Share Point

Tech Leads Review App

Spec

Dev LeadsPro Services

Pro Services PM

App Spec.doc

DevelopmentApp Spec

Reqs

Status Updates

Availability

Pro Services Lead

Acme Co.

Share Point

Create Build Request

Project Estimate

"Linda"Ops PM

"Linda""Denis"Ops PM

Add Design

.xls

Ops Engineer"Chuck"

Ops Specialists

1 month

Share Point

Build.xls

Create Change Tickets > 100

Build.xls

Service Now

NetFacilityCablingStorage

"Linda"Ops PM

RESET DELIVERY

DATE!

Acme Co.

Fix Tickets!

Senior Ops Engr

Create DB Schema

Ops DBA

Conflict Resolution

3 weeks

BA AppDev

NetEngrLead

AssignTickets

Service Now

"Chris"Ops

Engineer

Procure Hardware

Procurement SaaS

Install Hardware

SA Install OS Version (2)

Jumpstart Images

Service Now

Run CablingOps

Stock Clerk

Receive and Tag

Create DB VMs(6)

Install Hw Req

Cluster Setup

Creat App servers

Solaris Zones(10) Websphere

NFS

Write Data Migration Scripts

App DBA

UATUS

UATEurope

PRODUS

PRODEurope

F5 RulesPro

Services

External DNS

SSLCertificate

CreateVIP

CreateExternal IP

CreateSubnet

FirewallRules

Status Event

ProServices

Service Now

Create UAT Deploy Ticket

Middleware (code)DBA (SQL)Network

Approvals

"Marvin"Middleware

Ops

Deploy EAR

Configure to the data sources

Restart App

Request Testing to

Verify

Admin01

NFS

UAT

App SupportPro Services

BuildServer

EAR copied (FTP)

Promote AppDev

SmokeTest

App Support

Share Point

App SupportTest Script

Run Functional

Test

Test Plan.xls or .doc

Pro Services

BAWrite Test Plan

Customer Sign Off on Test

Plan

Acme Co.

"Linda"PM

CoordinateOperationalReadiness

Testing

UAT PROD

Acme Co.

Testing

Repeated in Prod

Middleware

DB Migration

Re-Sync DB

DBA

(happened earlier)

1 Month

Create Production Readiness

Meeting

DRHelpdeskTeam LeadsAppsOps PM

Share Point

TicketSystem

Go Live

Ops PM

Phone Bridge

1 Week 2 Weeks

Turn off old site

Run final DB sync

Customer uses new

VIP

Renegotiate

Create Build Out Doc

QA Test DITUS

4 hour change window2x per week

W

UAT Deploy and QA

Go Live .xls

Go Live .xls

UAT and PROD Network Config

UAT and PROD Server Provisioning

NetworkOps

Teams

Pro Services

Address Management

M

PROD Deploy and QACreate Ops

Tickets

App Dev and QA

App Specification

Financial Approvaland Project Commitment

Project Estimate

Sales and Customer Analysis

Sales and Delivery Risk

Analysis

Data Setup

Verification

PD

MW

PD

D

M

PD

PDM

TS

D M TS

HB

HB

M

TS

D

PD

W

M

M

M

M

M

PD

D

PDM

M

PD

D M

TSD

W

TSD

PD

W

TS W

"Incorrect or invalid rules"

"Missing Routes"

"Certificate Errors"

"Stale or conflicts"

Ops Requirements VerificationOps

Requirements

Verification

Small Batches

Ops Requirements

Verification

Verification

Ops Requirements

Standard Parts Small Batches

Verification

Small Batches

Ops Requirements

Verification

Verification

Standard Parts

Verification Standard Parts

Standard Parts

Ops Requirements

Verification

Ops Requirements

Ops HealthDashboard

Standard Parts

Ops HealthDashboard

Ops HealthDashboard

Ops HealthDashboardOps Health

Dashboard

Standard Parts

Verification

Verification

Verification

Ops Requirements

Ops HealthDashboard

Verification

Standard Parts

Standard Parts

Reqs Cust.Config

AppSpec Project EstimateProject

Estimate

Biz CaseApproval

Build Doc

Design & Spec

Project Plan

Tickets

~8 months before implementation starts

(and information / requirements still

incorrect or incomplete!)

Tickets Tickets TicketsAd-HocEmails-Missing details-New technical requirements-Missing context

Information Flow

PD

+ Work in small batches + Early Ops Involvement + Standardized Catalog + Plan and Design by Those Who Do

Page 68: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

PunchdownApprovals

Parts Unlimited - High Availability for Acme Co.

Evaluate Needs

Pro Serve Leads

Share Point

3 months

"Susan T."VP of Vertical

Act Mgmt

"Pat"Owner of Acme Co

Act.

Acme Co.

Reqs

Cust.Config

Pro Services PM

Begin Contract Negotiation

Evaluate Risk for Deliveries to

SLA

Pro Services

Legal

Ops

Reqs

"David"Service Level Manager

Finance Reporting

Project Delivery

ExpectationBU Owner

5 months total

Pro Services

BA's

Write Specs for Fulfillment

Share Point

.doc

Discuss Planning

Attendees

Project Kickoff Meeting

ProServices Ops

PM

AppSupport

Share Point

AppSupport

TicketsHigh Level

About Project

Estimate and Priorities

"Linda"Ops PM

Child ticket

"Project Estimate"

Create Project

Estimate Package

SystemsNetworksSecurityLeads

Core Network Provider

Share Point

.ppt

BOM.xls

Function Specific

Project Estimate estimate

"BOM.xls"

Set Timeline

Ops PM

Project Estimate

Judge Costs & Approval

AppSupport

Partner BU Business

Case

3 weeks

Create App Spec

Share Point

Tech Leads Review App

Spec

Dev LeadsPro Services

Pro Services PM

App Spec.doc

DevelopmentApp Spec

Reqs

Status Updates

Availability

Pro Services Lead

Acme Co.

Share Point

Create Build Request

Project Estimate

"Linda"Ops PM

"Linda""Denis"Ops PM

Add Design

.xls

Ops Engineer"Chuck"

Ops Specialists

1 month

Share Point

Build.xls

Create Change Tickets > 100

Build.xls

Service Now

NetFacilityCablingStorage

"Linda"Ops PM

RESET DELIVERY

DATE!

Acme Co.

Fix Tickets!

Senior Ops Engr

Create DB Schema

Ops DBA

Conflict Resolution

3 weeks

BA AppDev

NetEngrLead

AssignTickets

Service Now

"Chris"Ops

Engineer

Procure Hardware

Procurement SaaS

Install Hardware

SA Install OS Version (2)

Jumpstart Images

Service Now

Run CablingOps

Stock Clerk

Receive and Tag

Create DB VMs(6)

Install Hw Req

Cluster Setup

Creat App servers

Solaris Zones(10) Websphere

NFS

Write Data Migration Scripts

App DBA

UATUS

UATEurope

PRODUS

PRODEurope

F5 RulesPro

Services

External DNS

SSLCertificate

CreateVIP

CreateExternal IP

CreateSubnet

FirewallRules

Status Event

ProServices

Service Now

Create UAT Deploy Ticket

Middleware (code)DBA (SQL)Network

Approvals

"Marvin"Middleware

Ops

Deploy EAR

Configure to the data sources

Restart App

Request Testing to

Verify

Admin01

NFS

UAT

App SupportPro Services

BuildServer

EAR copied (FTP)

Promote AppDev

SmokeTest

App Support

Share Point

App SupportTest Script

Run Functional

Test

Test Plan.xls or .doc

Pro Services

BAWrite Test Plan

Customer Sign Off on Test

Plan

Acme Co.

"Linda"PM

CoordinateOperationalReadiness

Testing

UAT PROD

Acme Co.

Testing

Repeated in Prod

Middleware

DB Migration

Re-Sync DB

DBA

(happened earlier)

1 Month

Create Production Readiness

Meeting

DRHelpdeskTeam LeadsAppsOps PM

Share Point

TicketSystem

Go Live

Ops PM

Phone Bridge

1 Week 2 Weeks

Turn off old site

Run final DB sync

Customer uses new

VIP

Renegotiate

Create Build Out Doc

QA Test DITUS

4 hour change window2x per week

W

UAT Deploy and QA

Go Live .xls

Go Live .xls

UAT and PROD Network Config

UAT and PROD Server Provisioning

NetworkOps

Teams

Pro Services

Address Management

M

PROD Deploy and QACreate Ops

Tickets

App Dev and QA

App Specification

Financial Approvaland Project Commitment

Project Estimate

Sales and Customer Analysis

Sales and Delivery Risk

Analysis

Data Setup

Verification

PD

MW

PD

D

M

PD

PDM

TS

D M TS

HB

HB

M

TS

D

PD

W

M

M

M

M

M

PD

D

PDM

M

PD

D M

TSD

W

TSD

PD

W

TS W

"Incorrect or invalid rules"

"Missing Routes"

"Certificate Errors"

"Stale or conflicts"

Ops Requirements VerificationOps

Requirements

Verification

Small Batches

Ops Requirements

Verification

Verification

Ops Requirements

Standard Parts Small Batches

Verification

Small Batches

Ops Requirements

Verification

Verification

Standard Parts

Verification Standard Parts

Standard Parts

Ops Requirements

Verification

Ops Requirements

Ops HealthDashboard

Standard Parts

Ops HealthDashboard

Ops HealthDashboard

Ops HealthDashboardOps Health

Dashboard

Standard Parts

Verification

Verification

Verification

Ops Requirements

Ops HealthDashboard

Verification

Standard Parts

Standard Parts

Reqs Cust.Config

AppSpec Project EstimateProject

Estimate

Biz CaseApproval

Build Doc

Design & Spec

Project Plan

Tickets

~8 months before implementation starts

(and information / requirements still

incorrect or incomplete!)

Tickets Tickets TicketsAd-HocEmails-Missing details-New technical requirements-Missing context

Information Flow

PD

Page 69: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

PunchdownApprovals

Parts Unlimited - High Availability for Acme Co.

Evaluate Needs

Pro Serve Leads

Share Point

3 months

"Susan T."VP of Vertical

Act Mgmt

"Pat"Owner of Acme Co

Act.

Acme Co.

Reqs

Cust.Config

Pro Services PM

Begin Contract Negotiation

Evaluate Risk for Deliveries to

SLA

Pro Services

Legal

Ops

Reqs

"David"Service Level Manager

Finance Reporting

Project Delivery

ExpectationBU Owner

5 months total

Pro Services

BA's

Write Specs for Fulfillment

Share Point

.doc

Discuss Planning

Attendees

Project Kickoff Meeting

ProServices Ops

PM

AppSupport

Share Point

AppSupport

TicketsHigh Level

About Project

Estimate and Priorities

"Linda"Ops PM

Child ticket

"Project Estimate"

Create Project

Estimate Package

SystemsNetworksSecurityLeads

Core Network Provider

Share Point

.ppt

BOM.xls

Function Specific

Project Estimate estimate

"BOM.xls"

Set Timeline

Ops PM

Project Estimate

Judge Costs & Approval

AppSupport

Partner BU Business

Case

3 weeks

Create App Spec

Share Point

Tech Leads Review App

Spec

Dev LeadsPro Services

Pro Services PM

App Spec.doc

DevelopmentApp Spec

Reqs

Status Updates

Availability

Pro Services Lead

Acme Co.

Share Point

Create Build Request

Project Estimate

"Linda"Ops PM

"Linda""Denis"Ops PM

Add Design

.xls

Ops Engineer"Chuck"

Ops Specialists

1 month

Share Point

Build.xls

Create Change Tickets > 100

Build.xls

Service Now

NetFacilityCablingStorage

"Linda"Ops PM

RESET DELIVERY

DATE!

Acme Co.

Fix Tickets!

Senior Ops Engr

Create DB Schema

Ops DBA

Conflict Resolution

3 weeks

BA AppDev

NetEngrLead

AssignTickets

Service Now

"Chris"Ops

Engineer

Procure Hardware

Procurement SaaS

Install Hardware

SA Install OS Version (2)

Jumpstart Images

Service Now

Run CablingOps

Stock Clerk

Receive and Tag

Create DB VMs(6)

Install Hw Req

Cluster Setup

Creat App servers

Solaris Zones(10) Websphere

NFS

Write Data Migration Scripts

App DBA

UATUS

UATEurope

PRODUS

PRODEurope

F5 RulesPro

Services

External DNS

SSLCertificate

CreateVIP

CreateExternal IP

CreateSubnet

FirewallRules

Status Event

ProServices

Service Now

Create UAT Deploy Ticket

Middleware (code)DBA (SQL)Network

Approvals

"Marvin"Middleware

Ops

Deploy EAR

Configure to the data sources

Restart App

Request Testing to

Verify

Admin01

NFS

UAT

App SupportPro Services

BuildServer

EAR copied (FTP)

Promote AppDev

SmokeTest

App Support

Share Point

App SupportTest Script

Run Functional

Test

Test Plan.xls or .doc

Pro Services

BAWrite Test Plan

Customer Sign Off on Test

Plan

Acme Co.

"Linda"PM

CoordinateOperationalReadiness

Testing

UAT PROD

Acme Co.

Testing

Repeated in Prod

Middleware

DB Migration

Re-Sync DB

DBA

(happened earlier)

1 Month

Create Production Readiness

Meeting

DRHelpdeskTeam LeadsAppsOps PM

Share Point

TicketSystem

Go Live

Ops PM

Phone Bridge

1 Week 2 Weeks

Turn off old site

Run final DB sync

Customer uses new

VIP

Renegotiate

Create Build Out Doc

QA Test DITUS

4 hour change window2x per week

W

UAT Deploy and QA

Go Live .xls

Go Live .xls

UAT and PROD Network Config

UAT and PROD Server Provisioning

NetworkOps

Teams

Pro Services

Address Management

M

PROD Deploy and QACreate Ops

Tickets

App Dev and QA

App Specification

Financial Approvaland Project Commitment

Project Estimate

Sales and Customer Analysis

Sales and Delivery Risk

Analysis

Data Setup

Verification

PD

MW

PD

D

M

PD

PDM

TS

D M TS

HB

HB

M

TS

D

PD

W

M

M

M

M

M

PD

D

PDM

M

PD

D M

TSD

W

TSD

PD

W

TS W

"Incorrect or invalid rules"

"Missing Routes"

"Certificate Errors"

"Stale or conflicts"

Ops Requirements VerificationOps

Requirements

Verification

Small Batches

Ops Requirements

Verification

Verification

Ops Requirements

Standard Parts Small Batches

Verification

Small Batches

Ops Requirements

Verification

Verification

Standard Parts

Verification Standard Parts

Standard Parts

Ops Requirements

Verification

Ops Requirements

Ops HealthDashboard

Standard Parts

Ops HealthDashboard

Ops HealthDashboard

Ops HealthDashboardOps Health

Dashboard

Standard Parts

Verification

Verification

Verification

Ops Requirements

Ops HealthDashboard

Verification

Standard Parts

Standard Parts

Reqs Cust.Config

AppSpec Project EstimateProject

Estimate

Biz CaseApproval

Build Doc

Design & Spec

Project Plan

Tickets

~8 months before implementation starts

(and information / requirements still

incorrect or incomplete!)

Tickets Tickets TicketsAd-HocEmails-Missing details-New technical requirements-Missing context

Information Flow

PD

Page 70: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

PunchdownApprovals

Parts Unlimited - High Availability for Acme Co.

Evaluate Needs

Pro Serve Leads

Share Point

3 months

"Susan T."VP of Vertical

Act Mgmt

"Pat"Owner of Acme Co

Act.

Acme Co.

Reqs

Cust.Config

Pro Services PM

Begin Contract Negotiation

Evaluate Risk for Deliveries to

SLA

Pro Services

Legal

Ops

Reqs

"David"Service Level Manager

Finance Reporting

Project Delivery

ExpectationBU Owner

5 months total

Pro Services

BA's

Write Specs for Fulfillment

Share Point

.doc

Discuss Planning

Attendees

Project Kickoff Meeting

ProServices Ops

PM

AppSupport

Share Point

AppSupport

TicketsHigh Level

About Project

Estimate and Priorities

"Linda"Ops PM

Child ticket

"Project Estimate"

Create Project

Estimate Package

SystemsNetworksSecurityLeads

Core Network Provider

Share Point

.ppt

BOM.xls

Function Specific

Project Estimate estimate

"BOM.xls"

Set Timeline

Ops PM

Project Estimate

Judge Costs & Approval

AppSupport

Partner BU Business

Case

3 weeks

Create App Spec

Share Point

Tech Leads Review App

Spec

Dev LeadsPro Services

Pro Services PM

App Spec.doc

DevelopmentApp Spec

Reqs

Status Updates

Availability

Pro Services Lead

Acme Co.

Share Point

Create Build Request

Project Estimate

"Linda"Ops PM

"Linda""Denis"Ops PM

Add Design

.xls

Ops Engineer"Chuck"

Ops Specialists

1 month

Share Point

Build.xls

Create Change Tickets > 100

Build.xls

Service Now

NetFacilityCablingStorage

"Linda"Ops PM

RESET DELIVERY

DATE!

Acme Co.

Fix Tickets!

Senior Ops Engr

Create DB Schema

Ops DBA

Conflict Resolution

3 weeks

BA AppDev

NetEngrLead

AssignTickets

Service Now

"Chris"Ops

Engineer

Procure Hardware

Procurement SaaS

Install Hardware

SA Install OS Version (2)

Jumpstart Images

Service Now

Run CablingOps

Stock Clerk

Receive and Tag

Create DB VMs(6)

Install Hw Req

Cluster Setup

Creat App servers

Solaris Zones(10) Websphere

NFS

Write Data Migration Scripts

App DBA

UATUS

UATEurope

PRODUS

PRODEurope

F5 RulesPro

Services

External DNS

SSLCertificate

CreateVIP

CreateExternal IP

CreateSubnet

FirewallRules

Status Event

ProServices

Service Now

Create UAT Deploy Ticket

Middleware (code)DBA (SQL)Network

Approvals

"Marvin"Middleware

Ops

Deploy EAR

Configure to the data sources

Restart App

Request Testing to

Verify

Admin01

NFS

UAT

App SupportPro Services

BuildServer

EAR copied (FTP)

Promote AppDev

SmokeTest

App Support

Share Point

App SupportTest Script

Run Functional

Test

Test Plan.xls or .doc

Pro Services

BAWrite Test Plan

Customer Sign Off on Test

Plan

Acme Co.

"Linda"PM

CoordinateOperationalReadiness

Testing

UAT PROD

Acme Co.

Testing

Repeated in Prod

Middleware

DB Migration

Re-Sync DB

DBA

(happened earlier)

1 Month

Create Production Readiness

Meeting

DRHelpdeskTeam LeadsAppsOps PM

Share Point

TicketSystem

Go Live

Ops PM

Phone Bridge

1 Week 2 Weeks

Turn off old site

Run final DB sync

Customer uses new

VIP

Renegotiate

Create Build Out Doc

QA Test DITUS

4 hour change window2x per week

W

UAT Deploy and QA

Go Live .xls

Go Live .xls

UAT and PROD Network Config

UAT and PROD Server Provisioning

NetworkOps

Teams

Pro Services

Address Management

M

PROD Deploy and QACreate Ops

Tickets

App Dev and QA

App Specification

Financial Approvaland Project Commitment

Project Estimate

Sales and Customer Analysis

Sales and Delivery Risk

Analysis

Data Setup

Verification

PD

MW

PD

D

M

PD

PDM

TS

D M TS

HB

HB

M

TS

D

PD

W

M

M

M

M

M

PD

D

PDM

M

PD

D M

TSD

W

TSD

PD

W

TS W

"Incorrect or invalid rules"

"Missing Routes"

"Certificate Errors"

"Stale or conflicts"

Ops Requirements VerificationOps

Requirements

Verification

Small Batches

Ops Requirements

Verification

Verification

Ops Requirements

Standard Parts Small Batches

Verification

Small Batches

Ops Requirements

Verification

Verification

Standard Parts

Verification Standard Parts

Standard Parts

Ops Requirements

Verification

Ops Requirements

Ops HealthDashboard

Standard Parts

Ops HealthDashboard

Ops HealthDashboard

Ops HealthDashboardOps Health

Dashboard

Standard Parts

Verification

Verification

Verification

Ops Requirements

Ops HealthDashboard

Verification

Standard Parts

Standard Parts

Reqs Cust.Config

AppSpec Project EstimateProject

Estimate

Biz CaseApproval

Build Doc

Design & Spec

Project Plan

Tickets

~8 months before implementation starts

(and information / requirements still

incorrect or incomplete!)

Tickets Tickets TicketsAd-HocEmails-Missing details-New technical requirements-Missing context

Information Flow

PD

Page 71: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

PunchdownApprovals

Parts Unlimited - High Availability for Acme Co.

Evaluate Needs

Pro Serve Leads

Share Point

3 months

"Susan T."VP of Vertical

Act Mgmt

"Pat"Owner of Acme Co

Act.

Acme Co.

Reqs

Cust.Config

Pro Services PM

Begin Contract Negotiation

Evaluate Risk for Deliveries to

SLA

Pro Services

Legal

Ops

Reqs

"David"Service Level Manager

Finance Reporting

Project Delivery

ExpectationBU Owner

5 months total

Pro Services

BA's

Write Specs for Fulfillment

Share Point

.doc

Discuss Planning

Attendees

Project Kickoff Meeting

ProServices Ops

PM

AppSupport

Share Point

AppSupport

TicketsHigh Level

About Project

Estimate and Priorities

"Linda"Ops PM

Child ticket

"Project Estimate"

Create Project

Estimate Package

SystemsNetworksSecurityLeads

Core Network Provider

Share Point

.ppt

BOM.xls

Function Specific

Project Estimate estimate

"BOM.xls"

Set Timeline

Ops PM

Project Estimate

Judge Costs & Approval

AppSupport

Partner BU Business

Case

3 weeks

Create App Spec

Share Point

Tech Leads Review App

Spec

Dev LeadsPro Services

Pro Services PM

App Spec.doc

DevelopmentApp Spec

Reqs

Status Updates

Availability

Pro Services Lead

Acme Co.

Share Point

Create Build Request

Project Estimate

"Linda"Ops PM

"Linda""Denis"Ops PM

Add Design

.xls

Ops Engineer"Chuck"

Ops Specialists

1 month

Share Point

Build.xls

Create Change Tickets > 100

Build.xls

Service Now

NetFacilityCablingStorage

"Linda"Ops PM

RESET DELIVERY

DATE!

Acme Co.

Fix Tickets!

Senior Ops Engr

Create DB Schema

Ops DBA

Conflict Resolution

3 weeks

BA AppDev

NetEngrLead

AssignTickets

Service Now

"Chris"Ops

Engineer

Procure Hardware

Procurement SaaS

Install Hardware

SA Install OS Version (2)

Jumpstart Images

Service Now

Run CablingOps

Stock Clerk

Receive and Tag

Create DB VMs(6)

Install Hw Req

Cluster Setup

Creat App servers

Solaris Zones(10) Websphere

NFS

Write Data Migration Scripts

App DBA

UATUS

UATEurope

PRODUS

PRODEurope

F5 RulesPro

Services

External DNS

SSLCertificate

CreateVIP

CreateExternal IP

CreateSubnet

FirewallRules

Status Event

ProServices

Service Now

Create UAT Deploy Ticket

Middleware (code)DBA (SQL)Network

Approvals

"Marvin"Middleware

Ops

Deploy EAR

Configure to the data sources

Restart App

Request Testing to

Verify

Admin01

NFS

UAT

App SupportPro Services

BuildServer

EAR copied (FTP)

Promote AppDev

SmokeTest

App Support

Share Point

App SupportTest Script

Run Functional

Test

Test Plan.xls or .doc

Pro Services

BAWrite Test Plan

Customer Sign Off on Test

Plan

Acme Co.

"Linda"PM

CoordinateOperationalReadiness

Testing

UAT PROD

Acme Co.

Testing

Repeated in Prod

Middleware

DB Migration

Re-Sync DB

DBA

(happened earlier)

1 Month

Create Production Readiness

Meeting

DRHelpdeskTeam LeadsAppsOps PM

Share Point

TicketSystem

Go Live

Ops PM

Phone Bridge

1 Week 2 Weeks

Turn off old site

Run final DB sync

Customer uses new

VIP

Renegotiate

Create Build Out Doc

QA Test DITUS

4 hour change window2x per week

W

UAT Deploy and QA

Go Live .xls

Go Live .xls

UAT and PROD Network Config

UAT and PROD Server Provisioning

NetworkOps

Teams

Pro Services

Address Management

M

PROD Deploy and QACreate Ops

Tickets

App Dev and QA

App Specification

Financial Approvaland Project Commitment

Project Estimate

Sales and Customer Analysis

Sales and Delivery Risk

Analysis

Data Setup

Verification

PD

MW

PD

D

M

PD

PDM

TS

D M TS

HB

HB

M

TS

D

PD

W

M

M

M

M

M

PD

D

PDM

M

PD

D M

TSD

W

TSD

PD

W

TS W

"Incorrect or invalid rules"

"Missing Routes"

"Certificate Errors"

"Stale or conflicts"

Ops Requirements VerificationOps

Requirements

Verification

Small Batches

Ops Requirements

Verification

Verification

Ops Requirements

Standard Parts Small Batches

Verification

Small Batches

Ops Requirements

Verification

Verification

Standard Parts

Verification Standard Parts

Standard Parts

Ops Requirements

Verification

Ops Requirements

Ops HealthDashboard

Standard Parts

Ops HealthDashboard

Ops HealthDashboard

Ops HealthDashboardOps Health

Dashboard

Standard Parts

Verification

Verification

Verification

Ops Requirements

Ops HealthDashboard

Verification

Standard Parts

Standard Parts

Reqs Cust.Config

AppSpec Project EstimateProject

Estimate

Biz CaseApproval

Build Doc

Design & Spec

Project Plan

Tickets

~8 months before implementation starts

(and information / requirements still

incorrect or incomplete!)

Tickets Tickets TicketsAd-HocEmails-Missing details-New technical requirements-Missing context

Information Flow

PD

ReworkDevTeam

Page 72: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

PunchdownApprovals

Parts Unlimited - High Availability for Acme Co.

Evaluate Needs

Pro Serve Leads

Share Point

3 months

"Susan T."VP of Vertical

Act Mgmt

"Pat"Owner of Acme Co

Act.

Acme Co.

Reqs

Cust.Config

Pro Services PM

Begin Contract Negotiation

Evaluate Risk for Deliveries to

SLA

Pro Services

Legal

Ops

Reqs

"David"Service Level Manager

Finance Reporting

Project Delivery

ExpectationBU Owner

5 months total

Pro Services

BA's

Write Specs for Fulfillment

Share Point

.doc

Discuss Planning

Attendees

Project Kickoff Meeting

ProServices Ops

PM

AppSupport

Share Point

AppSupport

TicketsHigh Level

About Project

Estimate and Priorities

"Linda"Ops PM

Child ticket

"Project Estimate"

Create Project

Estimate Package

SystemsNetworksSecurityLeads

Core Network Provider

Share Point

.ppt

BOM.xls

Function Specific

Project Estimate estimate

"BOM.xls"

Set Timeline

Ops PM

Project Estimate

Judge Costs & Approval

AppSupport

Partner BU Business

Case

3 weeks

Create App Spec

Share Point

Tech Leads Review App

Spec

Dev LeadsPro Services

Pro Services PM

App Spec.doc

DevelopmentApp Spec

Reqs

Status Updates

Availability

Pro Services Lead

Acme Co.

Share Point

Create Build Request

Project Estimate

"Linda"Ops PM

"Linda""Denis"Ops PM

Add Design

.xls

Ops Engineer"Chuck"

Ops Specialists

1 month

Share Point

Build.xls

Create Change Tickets > 100

Build.xls

Service Now

NetFacilityCablingStorage

"Linda"Ops PM

RESET DELIVERY

DATE!

Acme Co.

Fix Tickets!

Senior Ops Engr

Create DB Schema

Ops DBA

Conflict Resolution

3 weeks

BA AppDev

NetEngrLead

AssignTickets

Service Now

"Chris"Ops

Engineer

Procure Hardware

Procurement SaaS

Install Hardware

SA Install OS Version (2)

Jumpstart Images

Service Now

Run CablingOps

Stock Clerk

Receive and Tag

Create DB VMs(6)

Install Hw Req

Cluster Setup

Creat App servers

Solaris Zones(10) Websphere

NFS

Write Data Migration Scripts

App DBA

UATUS

UATEurope

PRODUS

PRODEurope

F5 RulesPro

Services

External DNS

SSLCertificate

CreateVIP

CreateExternal IP

CreateSubnet

FirewallRules

Status Event

ProServices

Service Now

Create UAT Deploy Ticket

Middleware (code)DBA (SQL)Network

Approvals

"Marvin"Middleware

Ops

Deploy EAR

Configure to the data sources

Restart App

Request Testing to

Verify

Admin01

NFS

UAT

App SupportPro Services

BuildServer

EAR copied (FTP)

Promote AppDev

SmokeTest

App Support

Share Point

App SupportTest Script

Run Functional

Test

Test Plan.xls or .doc

Pro Services

BAWrite Test Plan

Customer Sign Off on Test

Plan

Acme Co.

"Linda"PM

CoordinateOperationalReadiness

Testing

UAT PROD

Acme Co.

Testing

Repeated in Prod

Middleware

DB Migration

Re-Sync DB

DBA

(happened earlier)

1 Month

Create Production Readiness

Meeting

DRHelpdeskTeam LeadsAppsOps PM

Share Point

TicketSystem

Go Live

Ops PM

Phone Bridge

1 Week 2 Weeks

Turn off old site

Run final DB sync

Customer uses new

VIP

Renegotiate

Create Build Out Doc

QA Test DITUS

4 hour change window2x per week

W

UAT Deploy and QA

Go Live .xls

Go Live .xls

UAT and PROD Network Config

UAT and PROD Server Provisioning

NetworkOps

Teams

Pro Services

Address Management

M

PROD Deploy and QACreate Ops

Tickets

App Dev and QA

App Specification

Financial Approvaland Project Commitment

Project Estimate

Sales and Customer Analysis

Sales and Delivery Risk

Analysis

Data Setup

Verification

PD

MW

PD

D

M

PD

PDM

TS

D M TS

HB

HB

M

TS

D

PD

W

M

M

M

M

M

PD

D

PDM

M

PD

D M

TSD

W

TSD

PD

W

TS W

"Incorrect or invalid rules"

"Missing Routes"

"Certificate Errors"

"Stale or conflicts"

Ops Requirements VerificationOps

Requirements

Verification

Small Batches

Ops Requirements

Verification

Verification

Ops Requirements

Standard Parts Small Batches

Verification

Small Batches

Ops Requirements

Verification

Verification

Standard Parts

Verification Standard Parts

Standard Parts

Ops Requirements

Verification

Ops Requirements

Ops HealthDashboard

Standard Parts

Ops HealthDashboard

Ops HealthDashboard

Ops HealthDashboardOps Health

Dashboard

Standard Parts

Verification

Verification

Verification

Ops Requirements

Ops HealthDashboard

Verification

Standard Parts

Standard Parts

Reqs Cust.Config

AppSpec Project EstimateProject

Estimate

Biz CaseApproval

Build Doc

Design & Spec

Project Plan

Tickets

~8 months before implementation starts

(and information / requirements still

incorrect or incomplete!)

Tickets Tickets TicketsAd-HocEmails-Missing details-New technical requirements-Missing context

Information Flow

PD

+ GTM/LTM in Early Environments

ReworkDevTeam

Page 73: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

PunchdownApprovals

Parts Unlimited - High Availability for Acme Co.

Evaluate Needs

Pro Serve Leads

Share Point

3 months

"Susan T."VP of Vertical

Act Mgmt

"Pat"Owner of Acme Co

Act.

Acme Co.

Reqs

Cust.Config

Pro Services PM

Begin Contract Negotiation

Evaluate Risk for Deliveries to

SLA

Pro Services

Legal

Ops

Reqs

"David"Service Level Manager

Finance Reporting

Project Delivery

ExpectationBU Owner

5 months total

Pro Services

BA's

Write Specs for Fulfillment

Share Point

.doc

Discuss Planning

Attendees

Project Kickoff Meeting

ProServices Ops

PM

AppSupport

Share Point

AppSupport

TicketsHigh Level

About Project

Estimate and Priorities

"Linda"Ops PM

Child ticket

"Project Estimate"

Create Project

Estimate Package

SystemsNetworksSecurityLeads

Core Network Provider

Share Point

.ppt

BOM.xls

Function Specific

Project Estimate estimate

"BOM.xls"

Set Timeline

Ops PM

Project Estimate

Judge Costs & Approval

AppSupport

Partner BU Business

Case

3 weeks

Create App Spec

Share Point

Tech Leads Review App

Spec

Dev LeadsPro Services

Pro Services PM

App Spec.doc

DevelopmentApp Spec

Reqs

Status Updates

Availability

Pro Services Lead

Acme Co.

Share Point

Create Build Request

Project Estimate

"Linda"Ops PM

"Linda""Denis"Ops PM

Add Design

.xls

Ops Engineer"Chuck"

Ops Specialists

1 month

Share Point

Build.xls

Create Change Tickets > 100

Build.xls

Service Now

NetFacilityCablingStorage

"Linda"Ops PM

RESET DELIVERY

DATE!

Acme Co.

Fix Tickets!

Senior Ops Engr

Create DB Schema

Ops DBA

Conflict Resolution

3 weeks

BA AppDev

NetEngrLead

AssignTickets

Service Now

"Chris"Ops

Engineer

Procure Hardware

Procurement SaaS

Install Hardware

SA Install OS Version (2)

Jumpstart Images

Service Now

Run CablingOps

Stock Clerk

Receive and Tag

Create DB VMs(6)

Install Hw Req

Cluster Setup

Creat App servers

Solaris Zones(10) Websphere

NFS

Write Data Migration Scripts

App DBA

UATUS

UATEurope

PRODUS

PRODEurope

F5 RulesPro

Services

External DNS

SSLCertificate

CreateVIP

CreateExternal IP

CreateSubnet

FirewallRules

Status Event

ProServices

Service Now

Create UAT Deploy Ticket

Middleware (code)DBA (SQL)Network

Approvals

"Marvin"Middleware

Ops

Deploy EAR

Configure to the data sources

Restart App

Request Testing to

Verify

Admin01

NFS

UAT

App SupportPro Services

BuildServer

EAR copied (FTP)

Promote AppDev

SmokeTest

App Support

Share Point

App SupportTest Script

Run Functional

Test

Test Plan.xls or .doc

Pro Services

BAWrite Test Plan

Customer Sign Off on Test

Plan

Acme Co.

"Linda"PM

CoordinateOperationalReadiness

Testing

UAT PROD

Acme Co.

Testing

Repeated in Prod

Middleware

DB Migration

Re-Sync DB

DBA

(happened earlier)

1 Month

Create Production Readiness

Meeting

DRHelpdeskTeam LeadsAppsOps PM

Share Point

TicketSystem

Go Live

Ops PM

Phone Bridge

1 Week 2 Weeks

Turn off old site

Run final DB sync

Customer uses new

VIP

Renegotiate

Create Build Out Doc

QA Test DITUS

4 hour change window2x per week

W

UAT Deploy and QA

Go Live .xls

Go Live .xls

UAT and PROD Network Config

UAT and PROD Server Provisioning

NetworkOps

Teams

Pro Services

Address Management

M

PROD Deploy and QACreate Ops

Tickets

App Dev and QA

App Specification

Financial Approvaland Project Commitment

Project Estimate

Sales and Customer Analysis

Sales and Delivery Risk

Analysis

Data Setup

Verification

PD

MW

PD

D

M

PD

PDM

TS

D M TS

HB

HB

M

TS

D

PD

W

M

M

M

M

M

PD

D

PDM

M

PD

D M

TSD

W

TSD

PD

W

TS W

"Incorrect or invalid rules"

"Missing Routes"

"Certificate Errors"

"Stale or conflicts"

Ops Requirements VerificationOps

Requirements

Verification

Small Batches

Ops Requirements

Verification

Verification

Ops Requirements

Standard Parts Small Batches

Verification

Small Batches

Ops Requirements

Verification

Verification

Standard Parts

Verification Standard Parts

Standard Parts

Ops Requirements

Verification

Ops Requirements

Ops HealthDashboard

Standard Parts

Ops HealthDashboard

Ops HealthDashboard

Ops HealthDashboardOps Health

Dashboard

Standard Parts

Verification

Verification

Verification

Ops Requirements

Ops HealthDashboard

Verification

Standard Parts

Standard Parts

Reqs Cust.Config

AppSpec Project EstimateProject

Estimate

Biz CaseApproval

Build Doc

Design & Spec

Project Plan

Tickets

~8 months before implementation starts

(and information / requirements still

incorrect or incomplete!)

Tickets Tickets TicketsAd-HocEmails-Missing details-New technical requirements-Missing context

Information Flow

PD

Page 74: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

PunchdownApprovals

Parts Unlimited - High Availability for Acme Co.

Evaluate Needs

Pro Serve Leads

Share Point

3 months

"Susan T."VP of Vertical

Act Mgmt

"Pat"Owner of Acme Co

Act.

Acme Co.

Reqs

Cust.Config

Pro Services PM

Begin Contract Negotiation

Evaluate Risk for Deliveries to

SLA

Pro Services

Legal

Ops

Reqs

"David"Service Level Manager

Finance Reporting

Project Delivery

ExpectationBU Owner

5 months total

Pro Services

BA's

Write Specs for Fulfillment

Share Point

.doc

Discuss Planning

Attendees

Project Kickoff Meeting

ProServices Ops

PM

AppSupport

Share Point

AppSupport

TicketsHigh Level

About Project

Estimate and Priorities

"Linda"Ops PM

Child ticket

"Project Estimate"

Create Project

Estimate Package

SystemsNetworksSecurityLeads

Core Network Provider

Share Point

.ppt

BOM.xls

Function Specific

Project Estimate estimate

"BOM.xls"

Set Timeline

Ops PM

Project Estimate

Judge Costs & Approval

AppSupport

Partner BU Business

Case

3 weeks

Create App Spec

Share Point

Tech Leads Review App

Spec

Dev LeadsPro Services

Pro Services PM

App Spec.doc

DevelopmentApp Spec

Reqs

Status Updates

Availability

Pro Services Lead

Acme Co.

Share Point

Create Build Request

Project Estimate

"Linda"Ops PM

"Linda""Denis"Ops PM

Add Design

.xls

Ops Engineer"Chuck"

Ops Specialists

1 month

Share Point

Build.xls

Create Change Tickets > 100

Build.xls

Service Now

NetFacilityCablingStorage

"Linda"Ops PM

RESET DELIVERY

DATE!

Acme Co.

Fix Tickets!

Senior Ops Engr

Create DB Schema

Ops DBA

Conflict Resolution

3 weeks

BA AppDev

NetEngrLead

AssignTickets

Service Now

"Chris"Ops

Engineer

Procure Hardware

Procurement SaaS

Install Hardware

SA Install OS Version (2)

Jumpstart Images

Service Now

Run CablingOps

Stock Clerk

Receive and Tag

Create DB VMs(6)

Install Hw Req

Cluster Setup

Creat App servers

Solaris Zones(10) Websphere

NFS

Write Data Migration Scripts

App DBA

UATUS

UATEurope

PRODUS

PRODEurope

F5 RulesPro

Services

External DNS

SSLCertificate

CreateVIP

CreateExternal IP

CreateSubnet

FirewallRules

Status Event

ProServices

Service Now

Create UAT Deploy Ticket

Middleware (code)DBA (SQL)Network

Approvals

"Marvin"Middleware

Ops

Deploy EAR

Configure to the data sources

Restart App

Request Testing to

Verify

Admin01

NFS

UAT

App SupportPro Services

BuildServer

EAR copied (FTP)

Promote AppDev

SmokeTest

App Support

Share Point

App SupportTest Script

Run Functional

Test

Test Plan.xls or .doc

Pro Services

BAWrite Test Plan

Customer Sign Off on Test

Plan

Acme Co.

"Linda"PM

CoordinateOperationalReadiness

Testing

UAT PROD

Acme Co.

Testing

Repeated in Prod

Middleware

DB Migration

Re-Sync DB

DBA

(happened earlier)

1 Month

Create Production Readiness

Meeting

DRHelpdeskTeam LeadsAppsOps PM

Share Point

TicketSystem

Go Live

Ops PM

Phone Bridge

1 Week 2 Weeks

Turn off old site

Run final DB sync

Customer uses new

VIP

Renegotiate

Create Build Out Doc

QA Test DITUS

4 hour change window2x per week

W

UAT Deploy and QA

Go Live .xls

Go Live .xls

UAT and PROD Network Config

UAT and PROD Server Provisioning

NetworkOps

Teams

Pro Services

Address Management

M

PROD Deploy and QACreate Ops

Tickets

App Dev and QA

App Specification

Financial Approvaland Project Commitment

Project Estimate

Sales and Customer Analysis

Sales and Delivery Risk

Analysis

Data Setup

Verification

PD

MW

PD

D

M

PD

PDM

TS

D M TS

HB

HB

M

TS

D

PD

W

M

M

M

M

M

PD

D

PDM

M

PD

D M

TSD

W

TSD

PD

W

TS W

"Incorrect or invalid rules"

"Missing Routes"

"Certificate Errors"

"Stale or conflicts"

Ops Requirements VerificationOps

Requirements

Verification

Small Batches

Ops Requirements

Verification

Verification

Ops Requirements

Standard Parts Small Batches

Verification

Small Batches

Ops Requirements

Verification

Verification

Standard Parts

Verification Standard Parts

Standard Parts

Ops Requirements

Verification

Ops Requirements

Ops HealthDashboard

Standard Parts

Ops HealthDashboard

Ops HealthDashboard

Ops HealthDashboardOps Health

Dashboard

Standard Parts

Verification

Verification

Verification

Ops Requirements

Ops HealthDashboard

Verification

Standard Parts

Standard Parts

Reqs Cust.Config

AppSpec Project EstimateProject

Estimate

Biz CaseApproval

Build Doc

Design & Spec

Project Plan

Tickets

~8 months before implementation starts

(and information / requirements still

incorrect or incomplete!)

Tickets Tickets TicketsAd-HocEmails-Missing details-New technical requirements-Missing context

Information Flow

PD

Page 75: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

PunchdownApprovals

Parts Unlimited - High Availability for Acme Co.

Evaluate Needs

Pro Serve Leads

Share Point

3 months

"Susan T."VP of Vertical

Act Mgmt

"Pat"Owner of Acme Co

Act.

Acme Co.

Reqs

Cust.Config

Pro Services PM

Begin Contract Negotiation

Evaluate Risk for Deliveries to

SLA

Pro Services

Legal

Ops

Reqs

"David"Service Level Manager

Finance Reporting

Project Delivery

ExpectationBU Owner

5 months total

Pro Services

BA's

Write Specs for Fulfillment

Share Point

.doc

Discuss Planning

Attendees

Project Kickoff Meeting

ProServices Ops

PM

AppSupport

Share Point

AppSupport

TicketsHigh Level

About Project

Estimate and Priorities

"Linda"Ops PM

Child ticket

"Project Estimate"

Create Project

Estimate Package

SystemsNetworksSecurityLeads

Core Network Provider

Share Point

.ppt

BOM.xls

Function Specific

Project Estimate estimate

"BOM.xls"

Set Timeline

Ops PM

Project Estimate

Judge Costs & Approval

AppSupport

Partner BU Business

Case

3 weeks

Create App Spec

Share Point

Tech Leads Review App

Spec

Dev LeadsPro Services

Pro Services PM

App Spec.doc

DevelopmentApp Spec

Reqs

Status Updates

Availability

Pro Services Lead

Acme Co.

Share Point

Create Build Request

Project Estimate

"Linda"Ops PM

"Linda""Denis"Ops PM

Add Design

.xls

Ops Engineer"Chuck"

Ops Specialists

1 month

Share Point

Build.xls

Create Change Tickets > 100

Build.xls

Service Now

NetFacilityCablingStorage

"Linda"Ops PM

RESET DELIVERY

DATE!

Acme Co.

Fix Tickets!

Senior Ops Engr

Create DB Schema

Ops DBA

Conflict Resolution

3 weeks

BA AppDev

NetEngrLead

AssignTickets

Service Now

"Chris"Ops

Engineer

Procure Hardware

Procurement SaaS

Install Hardware

SA Install OS Version (2)

Jumpstart Images

Service Now

Run CablingOps

Stock Clerk

Receive and Tag

Create DB VMs(6)

Install Hw Req

Cluster Setup

Creat App servers

Solaris Zones(10) Websphere

NFS

Write Data Migration Scripts

App DBA

UATUS

UATEurope

PRODUS

PRODEurope

F5 RulesPro

Services

External DNS

SSLCertificate

CreateVIP

CreateExternal IP

CreateSubnet

FirewallRules

Status Event

ProServices

Service Now

Create UAT Deploy Ticket

Middleware (code)DBA (SQL)Network

Approvals

"Marvin"Middleware

Ops

Deploy EAR

Configure to the data sources

Restart App

Request Testing to

Verify

Admin01

NFS

UAT

App SupportPro Services

BuildServer

EAR copied (FTP)

Promote AppDev

SmokeTest

App Support

Share Point

App SupportTest Script

Run Functional

Test

Test Plan.xls or .doc

Pro Services

BAWrite Test Plan

Customer Sign Off on Test

Plan

Acme Co.

"Linda"PM

CoordinateOperationalReadiness

Testing

UAT PROD

Acme Co.

Testing

Repeated in Prod

Middleware

DB Migration

Re-Sync DB

DBA

(happened earlier)

1 Month

Create Production Readiness

Meeting

DRHelpdeskTeam LeadsAppsOps PM

Share Point

TicketSystem

Go Live

Ops PM

Phone Bridge

1 Week 2 Weeks

Turn off old site

Run final DB sync

Customer uses new

VIP

Renegotiate

Create Build Out Doc

QA Test DITUS

4 hour change window2x per week

W

UAT Deploy and QA

Go Live .xls

Go Live .xls

UAT and PROD Network Config

UAT and PROD Server Provisioning

NetworkOps

Teams

Pro Services

Address Management

M

PROD Deploy and QACreate Ops

Tickets

App Dev and QA

App Specification

Financial Approvaland Project Commitment

Project Estimate

Sales and Customer Analysis

Sales and Delivery Risk

Analysis

Data Setup

Verification

PD

MW

PD

D

M

PD

PDM

TS

D M TS

HB

HB

M

TS

D

PD

W

M

M

M

M

M

PD

D

PDM

M

PD

D M

TSD

W

TSD

PD

W

TS W

"Incorrect or invalid rules"

"Missing Routes"

"Certificate Errors"

"Stale or conflicts"

Ops Requirements VerificationOps

Requirements

Verification

Small Batches

Ops Requirements

Verification

Verification

Ops Requirements

Standard Parts Small Batches

Verification

Small Batches

Ops Requirements

Verification

Verification

Standard Parts

Verification Standard Parts

Standard Parts

Ops Requirements

Verification

Ops Requirements

Ops HealthDashboard

Standard Parts

Ops HealthDashboard

Ops HealthDashboard

Ops HealthDashboardOps Health

Dashboard

Standard Parts

Verification

Verification

Verification

Ops Requirements

Ops HealthDashboard

Verification

Standard Parts

Standard Parts

Reqs Cust.Config

AppSpec Project EstimateProject

Estimate

Biz CaseApproval

Build Doc

Design & Spec

Project Plan

Tickets

~8 months before implementation starts

(and information / requirements still

incorrect or incomplete!)

Tickets Tickets TicketsAd-HocEmails-Missing details-New technical requirements-Missing context

Information Flow

PD

Page 76: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

PunchdownApprovals

Parts Unlimited - High Availability for Acme Co.

Evaluate Needs

Pro Serve Leads

Share Point

3 months

"Susan T."VP of Vertical

Act Mgmt

"Pat"Owner of Acme Co

Act.

Acme Co.

Reqs

Cust.Config

Pro Services PM

Begin Contract Negotiation

Evaluate Risk for Deliveries to

SLA

Pro Services

Legal

Ops

Reqs

"David"Service Level Manager

Finance Reporting

Project Delivery

ExpectationBU Owner

5 months total

Pro Services

BA's

Write Specs for Fulfillment

Share Point

.doc

Discuss Planning

Attendees

Project Kickoff Meeting

ProServices Ops

PM

AppSupport

Share Point

AppSupport

TicketsHigh Level

About Project

Estimate and Priorities

"Linda"Ops PM

Child ticket

"Project Estimate"

Create Project

Estimate Package

SystemsNetworksSecurityLeads

Core Network Provider

Share Point

.ppt

BOM.xls

Function Specific

Project Estimate estimate

"BOM.xls"

Set Timeline

Ops PM

Project Estimate

Judge Costs & Approval

AppSupport

Partner BU Business

Case

3 weeks

Create App Spec

Share Point

Tech Leads Review App

Spec

Dev LeadsPro Services

Pro Services PM

App Spec.doc

DevelopmentApp Spec

Reqs

Status Updates

Availability

Pro Services Lead

Acme Co.

Share Point

Create Build Request

Project Estimate

"Linda"Ops PM

"Linda""Denis"Ops PM

Add Design

.xls

Ops Engineer"Chuck"

Ops Specialists

1 month

Share Point

Build.xls

Create Change Tickets > 100

Build.xls

Service Now

NetFacilityCablingStorage

"Linda"Ops PM

RESET DELIVERY

DATE!

Acme Co.

Fix Tickets!

Senior Ops Engr

Create DB Schema

Ops DBA

Conflict Resolution

3 weeks

BA AppDev

NetEngrLead

AssignTickets

Service Now

"Chris"Ops

Engineer

Procure Hardware

Procurement SaaS

Install Hardware

SA Install OS Version (2)

Jumpstart Images

Service Now

Run CablingOps

Stock Clerk

Receive and Tag

Create DB VMs(6)

Install Hw Req

Cluster Setup

Creat App servers

Solaris Zones(10) Websphere

NFS

Write Data Migration Scripts

App DBA

UATUS

UATEurope

PRODUS

PRODEurope

F5 RulesPro

Services

External DNS

SSLCertificate

CreateVIP

CreateExternal IP

CreateSubnet

FirewallRules

Status Event

ProServices

Service Now

Create UAT Deploy Ticket

Middleware (code)DBA (SQL)Network

Approvals

"Marvin"Middleware

Ops

Deploy EAR

Configure to the data sources

Restart App

Request Testing to

Verify

Admin01

NFS

UAT

App SupportPro Services

BuildServer

EAR copied (FTP)

Promote AppDev

SmokeTest

App Support

Share Point

App SupportTest Script

Run Functional

Test

Test Plan.xls or .doc

Pro Services

BAWrite Test Plan

Customer Sign Off on Test

Plan

Acme Co.

"Linda"PM

CoordinateOperationalReadiness

Testing

UAT PROD

Acme Co.

Testing

Repeated in Prod

Middleware

DB Migration

Re-Sync DB

DBA

(happened earlier)

1 Month

Create Production Readiness

Meeting

DRHelpdeskTeam LeadsAppsOps PM

Share Point

TicketSystem

Go Live

Ops PM

Phone Bridge

1 Week 2 Weeks

Turn off old site

Run final DB sync

Customer uses new

VIP

Renegotiate

Create Build Out Doc

QA Test DITUS

4 hour change window2x per week

W

UAT Deploy and QA

Go Live .xls

Go Live .xls

UAT and PROD Network Config

UAT and PROD Server Provisioning

NetworkOps

Teams

Pro Services

Address Management

M

PROD Deploy and QACreate Ops

Tickets

App Dev and QA

App Specification

Financial Approvaland Project Commitment

Project Estimate

Sales and Customer Analysis

Sales and Delivery Risk

Analysis

Data Setup

Verification

PD

MW

PD

D

M

PD

PDM

TS

D M TS

HB

HB

M

TS

D

PD

W

M

M

M

M

M

PD

D

PDM

M

PD

D M

TSD

W

TSD

PD

W

TS W

"Incorrect or invalid rules"

"Missing Routes"

"Certificate Errors"

"Stale or conflicts"

Ops Requirements VerificationOps

Requirements

Verification

Small Batches

Ops Requirements

Verification

Verification

Ops Requirements

Standard Parts Small Batches

Verification

Small Batches

Ops Requirements

Verification

Verification

Standard Parts

Verification Standard Parts

Standard Parts

Ops Requirements

Verification

Ops Requirements

Ops HealthDashboard

Standard Parts

Ops HealthDashboard

Ops HealthDashboard

Ops HealthDashboardOps Health

Dashboard

Standard Parts

Verification

Verification

Verification

Ops Requirements

Ops HealthDashboard

Verification

Standard Parts

Standard Parts

Reqs Cust.Config

AppSpec Project EstimateProject

Estimate

Biz CaseApproval

Build Doc

Design & Spec

Project Plan

Tickets

~8 months before implementation starts

(and information / requirements still

incorrect or incomplete!)

Tickets Tickets TicketsAd-HocEmails-Missing details-New technical requirements-Missing context

Information Flow

PD

+ Dev provide verification script

Page 77: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

ServiceDeliveryMetrics

KaizenProgramOversight

Planning&

Retrospectives Informs Informs

Countermeasures &Blockers

Elements of a DevOps Kaizen Program

Page 78: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen: Designed for the Enterprise Needs

Page 79: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen: Designed for the Enterprise Needs

•Scale improvement quickly

Page 80: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen: Designed for the Enterprise Needs

•Scale improvement quickly

•Span multiple organizational boundaries

Page 81: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen: Designed for the Enterprise Needs

•Scale improvement quickly

•Span multiple organizational boundaries

•Work with substantial numbers of legacy technologies

Page 82: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen: Designed for the Enterprise Needs

•Scale improvement quickly

•Span multiple organizational boundaries

•Work with substantial numbers of legacy technologies

•Develop your existing staff in mass

Page 83: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen: Designed for the Enterprise Needs

•Scale improvement quickly

•Span multiple organizational boundaries

•Work with substantial numbers of legacy technologies

•Develop your existing staff in mass

•Be self-funding after initial seed investment

Page 84: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

“No need for this. We’ve got a big project lined up that will replace everything and solve our problems…”

Page 85: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

The “Big Bang” Transformation Dream

Start

Finish

Page 86: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

The “Big Bang” Transformation Reality

Start

Finish

Fear

Panic

Abort

Maybe

Page 87: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

The “Big Bang” Transformation Reality

Start

Finish

Fear

Panic

Abort

Maybe

People revert to legacy behaviors

Page 88: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

“Big J” vs “Little J’s”

Start

Finish

Start

Finish

“Big Bang” Continuous Improvement

Fear

PanicAbort

Maybe

Page 89: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

DevOps Kaizen: Let’s Recap!

ServiceDeliveryMetrics

KaizenProgramOversight

Planning&

Retrospectives Informs Informs

Countermeasures &Blockers

Ah-ha!

OpsDev Remove

Shorten

Lead Time

Feedback

It’s about the business

Make the work visible Continuous Improvement

Actions to support improvement

Page 90: DEVNET-2015DevOps In Depth - Damon Edwards on DevOps Kaizen: Building an Enterprise’s Capability to Change

@damonedwards

Damon Edwards

[email protected]