fosdem'14: configuration management in an enterprise linux team

26
automated How I myself out job of my @remibergsma in an enterprise Linux Team FOSDEM’14 - Feb 1st, 2014 - Brussels Use case: Configuration Management

Upload: remi-bergsma

Post on 22-Apr-2015

2.890 views

Category:

Technology


0 download

DESCRIPTION

About a year ago I accepted a new job in an enterprise Linux environment, running ~450 Linux servers. These servers were running on an internal network and had never been updated. Most work was done ad-hoc and in response to issues or failure. I transformed the team to a pro-active way of working where automation was key. By solving the most frequent problems first, we found the time to automate more and more. Every server was updated and configuration management was introduced. One interesting year later I've automated myself out of my job. The team can easily handle the (now much lighter) workload without me. Users are happy, so mission completed! Configuration management is done using CFEngine 3 and we use other DevOps style tools like Git and Vagrant. In this talk I'll talk about the journey we took and answer: - What was the problem with the old way of working? - How did we implement configuration management in an already running environment? - Why did we choose CFEngine 3? - What is the new way of working? - What are the lessons learned? - How to automate yourself out of your job? The presentation will also focus on the process of getting there and the lessons learned, so others can benefit. The presentation was given in the FOSDEM Configuration Management devroom at February 1st 2014 @12:00

TRANSCRIPT

Page 1: FOSDEM'14: Configuration Management in an Enterprise Linux Team

automatedHow I

myself outjob of my

@remibergsma

in an enterprise Linux Team

FOSDEM’14 - Feb 1st, 2014 - Brussels

Use case: Configuration Management

Page 2: FOSDEM'14: Configuration Management in an Enterprise Linux Team

@remibergsma

Page 3: FOSDEM'14: Configuration Management in an Enterprise Linux Team

@remibergsma

1 2Use case with results.

How to do it yourself.

Today’s story:

Page 4: FOSDEM'14: Configuration Management in an Enterprise Linux Team

Remi BergsmaI am

Engineer with a passion for automation.

@remibergsma

Page 5: FOSDEM'14: Configuration Management in an Enterprise Linux Team

Enterprise?Semi-government organization.

@remibergsma

Page 6: FOSDEM'14: Configuration Management in an Enterprise Linux Team

@remibergsma

Page 7: FOSDEM'14: Configuration Management in an Enterprise Linux Team

Too many of the same incidents.

@remibergsma

users

Page 8: FOSDEM'14: Configuration Management in an Enterprise Linux Team

firefightingTo go from

prevention.to

@remibergsma

Page 9: FOSDEM'14: Configuration Management in an Enterprise Linux Team

Git, CFEngine3and Vagrant.

The building blocks are

@remibergsma

Page 10: FOSDEM'14: Configuration Management in an Enterprise Linux Team

Workflow

@remibergsma

Dev Beta PreProd Production

FeatureFeature

Page 11: FOSDEM'14: Configuration Management in an Enterprise Linux Team

Git repository

Page 12: FOSDEM'14: Configuration Management in an Enterprise Linux Team

Involve users,they look from a different perspective.

@remibergsma

Page 13: FOSDEM'14: Configuration Management in an Enterprise Linux Team

@remibergsma

Page 14: FOSDEM'14: Configuration Management in an Enterprise Linux Team

Significant drop in incidents.

@remibergsma

users

Page 15: FOSDEM'14: Configuration Management in an Enterprise Linux Team

WTF!Phase 1:

Find out what problems to fix.

@remibergsma

Page 16: FOSDEM'14: Configuration Management in an Enterprise Linux Team

quick wins.Save time by applying

@remibergsma

Phase 2:

Page 17: FOSDEM'14: Configuration Management in an Enterprise Linux Team

configmgmtand build your baseline.

@remibergsma

ImplementPhase 3:

Page 18: FOSDEM'14: Configuration Management in an Enterprise Linux Team

scaleoutPhase 4:

up.and

@remibergsma

Page 19: FOSDEM'14: Configuration Management in an Enterprise Linux Team

Phase 5:

you’re now in control.Relax,

@remibergsma

Page 20: FOSDEM'14: Configuration Management in an Enterprise Linux Team

automate--- Someone @Google, long time ago (Thanks Kris!)

@remibergsma

“Every 18 months,

yourself out of your job

Page 21: FOSDEM'14: Configuration Management in an Enterprise Linux Team

Recap1. WTF: Find out what to fix 2. Save time with quick wins 3. Setup configmgmt baseline 4. Scale out & scale up 5. Relax: full control!

Replace manual work with configmgmt

@remibergsma

Always version your code

Page 22: FOSDEM'14: Configuration Management in an Enterprise Linux Team

use configmgmt.Whatever happens,

And please spread the word!

@remibergsma

Page 23: FOSDEM'14: Configuration Management in an Enterprise Linux Team

Get in touch!

blog.remibergsma.com@remibergsma

Follow me on Twitter I will post the slides soon.

[email protected]

Page 24: FOSDEM'14: Configuration Management in an Enterprise Linux Team

Thank you!

@remibergsma

Page 25: FOSDEM'14: Configuration Management in an Enterprise Linux Team

Credits

@remibergsma

Changing permissions animation: http://devopsreactions.tumblr.com/post/73295491766/changing-permissions-that-are-enforced-by-cfengine Fireman Sam Team: http://kidzpadz.co.uk/images/products/63500102343_00.jpg Fireman Sam relax: http://www.flickr.com/photos/thebigsheep/5905236905/sizes/l/ Kris Buytaert mentions the quote “Every 18 months, automate yourself out of your job” in his presentation: http://www.slideshare.net/KrisBuytaert/the-future-of-sysadmin on slide 38.

Page 26: FOSDEM'14: Configuration Management in an Enterprise Linux Team

License

@remibergsma

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.