agile2013 learning to see: the practical application of lean in devops

Learning to See

The Practical Application of Lean in DevOps

Robert BenefieldEvolveBeyond, [email protected]

Robert Benefield - Evolve Beyond, Ltd 2013

Business Outcomes

Make, Save, Protect


Release to market fasterImprove speed and responsivenessEnsure a smooth, trouble-free service

Make – increase revenue


Lower costs for setting up & maintaining servicesReduce infrastructure & people needed to run services

Save – reduce costs


Eliminate OutagesHigh-performance, scalable serviceRespond rapidly to problems

Protect – revenue & market


If you are going too fast and ops can’t keep up, it won’t matter

Ops is essential, but…


High friction


Siloed thinking


Defensive and obstructive


Opaque


Align Ops Outcomes to Business Outcomes

Create transparency

Make it flow

How do we fix this?


PERHAPS LEAN CAN HELP


War Manpower CommissionTraining Within Industry (TWI)


MacArthur Brings Deming and Lean to

Post War Japan

Do

CheckAct

Plan

Toyota autonomation


Visual Management

Continuous Flow Just In Time

Continuous Improvement Collaboration Autonomation

Walking The Gemba

Commander’s Intent Intuitiveness

Lean Principles


Learning to See

Walking the Gemba


OHM Energy


Who are they?

What is important to them

Risk appetites

Internal vs External

Understand the communities


Speed & Accuracy

Reliability & Safety

Community drivers


Within Ops

End-to-End

Alignment & transparency


Visualize Workflow

Employ Information Sherpas/Fire Wardens

Use A3 Problem Solving

Create Value Stream MapsExploit Information Radiators

Work Board

Accepted Scheduled In Progress Blocked Done

How lean helps you see


GETTING ORGANIZED


Move from Fire Fighting to Fire PreventionUnderstand proactive/reactive ratiosMake causes visibleMove towards prevention

Queuing theory


• Ringfence Fire Prevention– Fires aren’t a desired business outcome!– Keep focus (and disruptions to a minimum)

• Make fires and their sources visible– Firemen can become unintentional arsonists– Expedite means it is a fire!

• Reduce flammable fuel– Reduce batch sizes– Big batches can become all-engulfing infernos!

Create the fire line


QUEUE MASTER AND WORKFLOW


Team

Team

Team

Team

Unscheduled Queue

Queue Master

Clarification Queue

Prioritized Queue

Task

Task

Task

Task

Work Board

Accepted Scheduled In Progress Blocked Done

Visualize the workflow


Incoming to Workflow

Makes sure that tasks are clear, correct, and sized appropriatelyMakes sure tasks have the right priority

Unscheduled Queue

Queue Master

Clarification Queue

Prioritized Queue

Task

Task

Task

Task

Queue master


Looks for bottlenecks in queues

Facilitates

Notes patterns for discussion

Can take on tasks if idle, but lower priority

Queue master


All work goes into flow via the Queue Master

Work is prioritized by the Queue Master

Rules


Week long iterations (at most!)

Queue Master position should rotate at regular iterations

Iterations


Review queues and workflowGo through blocked tasksReview flow problemsArchive Done

Team iteration reviewGreat place to go over A3s!

Retrospective


Handover of Queue Master hatChoose which improvement items to actionReview any key upcoming/scheduled activitiesSort out any change alignment challenges

Planning


INFORMATION SHERPAS AND FIRE WARDENS


Move from reactive to proactive and predictive

Align Organizational Focus

Alignment at speed


Schwerpunkt

John Boyd

Intuitiveness


Fire WardenSpot arsonists and fire hazardsCoordinate cross functional fire prevention

Embedded Ops Leads


Information Sherpa

Ensures information flow

Aligns tools, processes and outcomes

Embedded Ops Leads


IMPROVE THE FLOW


DevOps and Service Engineering

Continuous Delivery

Continuous Integration

Automated Deployment

Environment Configuration Management

Workflow and Service Transparency

Kanban/Scrumban

Dashboarding and

Instrumentation

Cross Functional Working

Elastic Services

Architecture

Loose Coupling Design

Building Resilience

In

DevOps Ecosystem


Total Cost of Ownership

People Intensity

Risk

Metrics that matter


A3 thinking


Find information gaps and bottlenecksPrioritize automation & architectural targetsImprove agility & alignment

Value stream mapping


Information radiators


Source Code

Information sources


Build trends

Test trends

Information sources


Ops• Workflow

• Incidents• Performance

Accepted Scheduled

In Progress

Blocked Done

T12T12

T12T12

T34 T19 T9T16

T12 T2

2 T44 T5

4T62 T72

Information sources


Information is powerful

Information brings focus & alignment

Information helps decision making

Information wants to be free


Automate to improve flow and transparencyAutomate to reduce people intensity & frictionAutomate to instrument & find defects early

Don’t hide problemsDon’t lose touch with reality

Autonomation


THE TALE OF THE BILLING DATABASE


Walk the Gemba and understand the ecosystemVisualize the flowInstrument and let information go free!Get IntuitiveContinuously improve

Learn to see!


QUESTIONS?

LeanDevOps.com

Picture Credits• Doc Searls Siloes http://www.flickr.com/photos/docsearls/5500714140/• RWE http://www.rwe.com• Michael Loke: http://www.flickr.com/photos/jymloke/4476177332/• CTPO Operations Centre: http://www.flickr.com/photos/ctbto/8443792513/in/photostream/• Malcomb Paynes• Becky Simmons: http://www.flickr.com/photos/southerncomposition/5345607040/• Slackware: http://www.flickr.com/photos/slackware/• David Singleton http://www.flickr.com/photos/davidsingleton/4112671768/• John Rork http://www.flickr.com/photos/johnrork/5395933785/• Docklandsboy http://www.flickr.com/photos/mogwai_83/3022261893/

http://www.flickr.com/photos/docsearls/5500714140/

http://www.rwe.com/

http://www.flickr.com/photos/jymloke/4476177332/

http://www.flickr.com/photos/ctbto/8443792513/in/photostream/

http://www.flickr.com/photos/ctbto/8443792513/in/photostream/

http://www.flickr.com/photos/southerncomposition/5345607040/

http://www.flickr.com/photos/southerncomposition/5345607040/

http://www.flickr.com/photos/davidsingleton/4112671768/

http://www.flickr.com/photos/johnrork/5395933785/

http://www.flickr.com/photos/mogwai_83/3022261893/

agile2013 learning to see: the practical application of lean in devops

Technology