2020 scanagile - continuous delivery in 4 months for 15 ... · establish a deployment pipeline...

33
@tdpauw thinkinglabs.io @tdpauw thinkinglabs.io Continuous Delivery in 4 months for 15 teams and their 1 monolith Thierry de Pauw | Continuous Delivery advocate

Upload: others

Post on 26-Jul-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io@tdpauw thinkinglabs.io

Continuous Delivery in 4 months for 15 teams and their 1 monolith

Thierry de Pauw | Continuous Delivery advocate

Page 2: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

A Belgian federal agency

15 teams (~ 150 people) and 1 monolith

Page 3: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

3-4 planned major releases per year

2017: 3 major releases2018: 2 major releases

deployment lead team of 28 days334 person-days to achieve those 28 days

Page 4: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

September 5th 2018:

Can you help us achieve Continuous Delivery with fortnightly release cyclesby end of December 2018?

Page 5: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

“In preparing for battle I have always found that plans are useless, but planning is indispensable.”-- Dwight Eisenhower, 1957

Page 6: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

What not to do? Maturity Models

They are fundamentally flawed.

Page 7: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

● Improvement Kata

● Value Stream Mapping

● Measuring Continuous Delivery

● Theory of Constraints

Page 8: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

Establish a core teamrepresenting all roles to lead the Continuous Delivery adoption.

Page 9: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

Investing in Continuous Delivery is very valuable.But it isn’t easy nor cheap !

Page 10: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

The Improvement Kata

Page 11: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

The Agency's Improvement Kata

Page 12: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

The Agency's Improvement Kata

Page 13: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

Major release ~ 6 months lead time

5 months: accumulation of features 3 weeks: code freeze 1 week:productiondeployment

Champagne 🍾🎉

🥂

334 person-days

Page 14: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

Patch releases - every fortnight

Page 15: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

Theory of Constraints

Every system has a bottleneck. Spending time optimising anything other than the bottleneck is an illusion.

Page 16: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

Where is the bottleneck?

activity lead time

activity failure rate

not here

Page 17: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

Where is the bottleneck?

activity lead time

activity failure rate

VCS Branches

but here !!!

Page 18: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

First experiment: the deployment pipeline and its metrics

Page 19: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

The Agency's Improvement Kata

Page 20: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

The Agency's Improvement Kata

Page 21: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

The Agency's Improvement Kata

Establish a Deployment Pipeline ✔ Daily evaluation of failing tests Dedicated environment for automated acceptance tests Recreate database before running acceptance tests Stub 3rd party services Auto collect lead time and failure rate metrics from the pipeline

Page 22: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

“... but plans are worthless when the fighting is once begun, and all depends on the inspiration of the moment.”-- a war correspondent for the British newspaper “The Daily News”, 1877

Page 23: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

Fear 1: Fear of Complexity

Page 24: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

Complex branching strategy

Page 25: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

Meanwhile …18/10: LT of 8h, 21/3529 failing tests29/10: LT of 4h, 545/5390 failing tests05/11: no pipeline run for 5 days due to a Jenkins upgrade23/11: first green pipeline, all tests are passing 🎉

After which the pipeline went red for another 2 weeks 😒

Page 26: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

Fear 2: Fear of Deadlines

Page 27: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

Introduce a manual overrule step

“Sunlight is said to be the best disinfectants.”-- Louis Brandeis

Page 28: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

Fear 3: Fear of Bugs

Page 29: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

Introduce quarantined tests

Page 30: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

“They did not know it was impossible, so they did it.”-- Mark Twain

Page 31: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

WTF am I doing here?And yet, it happens. Not as you expected.

Page 32: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io@tdpauw thinkinglabs.io

Hello, I am Thierry de Pauw

is shy, introvert and opinionatedis chief imposter

likes dark chocolatedark means > 50% cacao, prefers 70% and more

Acknowledgments:

Els, my beloved one, for supporting me in all my crazy endeavours, always believing in me.

Steve Smith for reviewing this presentation, his endless kindness and being my CD mentor.

Jeffrey Fredrick and Douglas Squirrel for including this case as a fear conversations example in their upcoming book “Agile Conversations” which will be available from May 12th - https://www.conversationaltransformation.com/

Engineer Lead at the fintech startup PaxFamilia

Founder of ThinkingLabs, advising on CI and CD

Page 33: 2020 ScanAgile - Continuous Delivery in 4 months for 15 ... · Establish a Deployment Pipeline Daily evaluation of failing tests Dedicated environment for automated acceptance tests

@tdpauw thinkinglabs.io

Bibliography

Lean Enterprise (ch 6 Deploy Continuous Improvement), Jez Humble, Joanne Molesky, Barry O'Reilly Accelerate - Extra chapter: How to transform, Jez HumbleThe Improvement Kata, Mike RotherMeasuring Continuous Delivery, Steve SmithThe Goal, Eliyahu GoldrattA Practical Approach to Large-Scale Agile Development: How HP Transformed LaserJet FutureSmart Firmware, Gary Gruver, Mike Young, Pat FulghumDual Value Streams, Steve SmithGot a wicked problem? First, tell me how you make toast, Tom Wujec, TEDGlobal 2013