docker & chef and chef...8/4/14 1 docker & chef containers • linux containers (lxc) •...

8/4/14

1

Docker & Chef

Containers • Linux containers (LXC)

• Single operating system

• Multiple isolated, working configurations

Containers and Docker • Open-source engine for container management

• Docker registry for sharing containers

8/4/14

2

Virtual Machines vs. Docker

https://www.docker.com/whatisdocker/

Chef & Docker • Create & Deploy Docker Containers

• Configure and protect containers

• Manage host platform

Docker & Chef • knife container • docker build – build a Docker container • docker init – setup Dockerfile

8/4/14

3

Docker & Chef Demo

Do I still need configuration management? • Yes! • Image sprawl is a thing

Typical Infrastructure

Memcache

Postgres Slaves

Postgres Master

Nagios Graphite

Rails App

8/4/14

4

New Compliance Mandate!

Rails App

Memcache

Postgres Slaves

Postgres Master

Nagios Graphite

• Move SSH off port 22

•  Lets put it on 2022

6 Golden Image Updates

Rails App

Memcache

Postgres Slaves

Postgres Master

Nagios Graphite

•  edit /etc/ssh/sshd_config

1 2

3

4

5

6

12 Instance Replacements

Rails App

Memcache

Postgres Slaves

Postgres Master

Nagios Graphite

•  Delete, launch

1 2

3 4 5 6 7

8 9

10 11

12

•  Repeat

•  Typically manually

8/4/14

5

Done in Maintenance Windows

•  Don’t break anything!

•  Bob just got fired =(

5

Rails App

Memcache

Postgres Slaves

Postgres Master

Nagios Graphite 1 2

4 5 6 7

8 9

10 11

12

3

12

Different IP Addresses?

Rails App

Memcache

Postgres Slaves

Postgres Master

Nagios Graphite

•  Invalid configs!

Chef & Docker • Create & Deploy Docker Containers

• Configure and protect containers

• Manage host platform

8/4/14

6

Choose Your Own Adventure • What is DevOps? •  Infrastructure as

Code • Version Control • Quit your job! •  Incident Management • Stages of Learning • How to hire DevOps • Podcasts &

Podcasting • Ask me anything

• Docker & Chef • Test Driven Development

• Chef DK • Chef Metal • Chef for OSX • Chef Search • Knife • One repo or many? • Chef vs …

Test Driven Development For your infrastructure code!

Typical Development Workflow • Typical Software

Development Workflow:

• Write Source

• Compile Source

• Deploy Artifacts

• Write Tests

• Run Tests

8/4/14

7

Typical Development Workflow • Source Code

• The recipe for a computer program

• Edited directly

• Managed via Source Control Software

Typical Development Workflow

• Compiler

• Takes source code and converts it into executable programs


• Artifact

• Artifacts are executable programs created by compilers.

• Compiled artifacts cannot be edited directly. Source code must be changed and re-compiled to produce a new build artifact.

8/4/14

8

Typical Development Workflow • Testing

• Write tests to verify that code works as intended

• Run tests at different stages of the code lifecycle to ensure correctness


• When developing software, most time isn’t actually spent coding

• 10-second changes to source code can take minutes to vet

• Compiling code

• Deploying code

• Writing & Running tests

8/4/14

9

Typical Development Workflow Fun!

Boring!

•  Things that are fun:

• Designing programs!

• Writing source code!

•  Things that are boring:

• Compiling code

• Deploying artifacts

• Running tests

Cookbooks Are Source Code • Chef Cookbook source code is just like the source code for other programs. •  Changes are tracked and versioned with source control software such as

git •  Code is inert until it has been deployed •  Code can be used to re-create the desired system configuration from

scratch

Cookbooks Aren’t Pre-Compiled • Because Ruby is an interpreted language, Chef code is not compiled until it is executed by the Chef-Client

• There is no lengthy compilation step between committing Chef Cookbooks and deploying them

X

8/4/14

10

Cookbooks Are Artifacts • Chef Cookbook artifacts are just like the

artifacts for other programs.

• Cookbooks are deployed

• Cookbooks are released with versions • Cookbooks are immutable once

deployed

• Cookbooks are packaged for distribution

• Cookbooks can depend on one another X

Cookbooks Are Artifacts

• Depending entirely on context, Chef Cookbooks can be either source code OR artifacts.

Close Encounters of the Third Kind, copyright 1977, Columbia Pictures

“This means something! This is important!”

Cookbooks Are Artifacts • This sounds obvious, but it’s easy to overlook: Treating cookbooks only as source code or only as artifacts will lead to long-term problems when using Chef cookbooks to manage infrastructure.

• Because Chef cookbooks are source code and artifacts, depending on the context, they have all the properties of each.

8/4/14

11

Cookbooks Are Artifacts

• Cookbooks are source!

• Versioned • Stored in a

source repository

•  Inert while in development

A good Chef workflow treats cookbooks appropriately depending on context • Cookbooks are

artifacts! • Versioned • Deployed to a

Chef Server • Executable by

Chef-client once deployed

Keeping Repos in Sync A brief aside about deploying cookbook artifacts: • Common question: “How do I automatically

keep my Chef Code Repository and my Chef Server Artifact Repository in sync?”

• Answer: The same way that one would automatically keep a code repo and an artifact repo in sync with any other software released to production. Deploy with care!

• Cookbooks are Source! •  knife cookbook create •  Code: recipes/default.rb

•  Version & dependencies: metadata.rb

•  git commit •  git tag

Cookbooks as Source Code

X

8/4/14

12

Cookbooks as Artifacts • Cookbooks are Artifacts!

•  knife cookbook upload•  Immutable once deployed,

can only be replaced •  Version & dependencies:

metadata.rb •  chef-client

Testing Cookbooks • Three things necessary to test a Chef cookbook: 1)  A place to store the cookbook artifact. 2)  A Chef-client with access to the cookbook. 3)  A target server running the same OS as production.

X

Traditional Cookbook Development: First Chef Run

8/4/14

13

Traditional Cookbook Development: Subsequent Runs

Boooooooring •  Too much time doing “paperwork” •  vi recipes/

something.rb •  knife cookbook upload •  sudo pkill -USR1 chef-

client •  #%$#%$ something

broke, let me do that all again

•  Not enough time doing fun stuff! •  Writing recipes

Less Fun!

More Boring!

XTHIS

SUCKS!

Developing for Chef: Rapid Iteration • Less time waiting

around for cookbook deploys and Chef runs

• More frequent testing

• Better code

• Business needs met more quickly

More Fun!

Less Boring!

X

8/4/14

14

Test Kitchen

Serverspec describe "awesome appliance repair" do! it "has a webserver listening on port 80" do!

expect(port('80')).to be_listening!

end!

!

it "does not display the default apache home page" do! expect(command("curl http://localhost")).not_to return_stdout /Ubuntu/!

end!

!

it "displays the home page" do! expect(command("curl http://localhost")).to return_stdout /Awesome/!

end!

end!!

ChefSpec require 'chefspec'!!describe 'awesome_appliance_repair_chef::default' do! let(:chef_run) { ChefSpec::Runner.new.converge(described_recipe) }!

! it 'installs foo' do! expect(chef_run).to install_package('apache2')! end!

end!!

8/4/14

15

$!

FC002: Avoid string interpolation where not required: ./recipes/default.rb:104!

Foodcritic

foodcritic .!






Chef DK The Chef Development Kit

8/4/14

16

Chef DK • Chef • ohai • chef-client • chef-apply • chef-solo • knife

• Berkshelf • Test Kitchen • ChefSpec • Foodcritic

Chef – the command line utility • chef generate • cookbook • recipe • attribute • template • file • lwrp

• chef gem • chef verify • chef exec

Chef DK – The Workflow • Install Vagrant • Install Chef DK • Generate cookbook, recipe, etc. • Use default test-kitchen

• http://bit.ly/1kaZKpF

8/4/14

17

Demo Time






Chef Metal machine resource and provisioners

8/4/14

18






Knife

8/4/14

19

Demo Time






Chef vs … How does Chef compare to …?

8/4/14

20

Chef vs …

http://ronblacks.com/wp-content/uploads/2013/07/153359_60_redo_web.jpg






Infrastructure as Code • Programmatically provision and configure components

90

http://www.flickr.com/photos/louisb/4555295187/

8/4/14

21

Infrastructure as Code • Treat like any other code base

91


Infrastructure as Code • Reconstruct business from code repository, data backup, and compute resources

92


Infrastructure as Code • Programmatically provision and configure components

• Treat like any other code base

• Reconstruct business from code repository, data backup, and compute resources

93


8/4/14

22






Version Control

Which Version Control System Do You Use? • cp foo foo.bak • cp foo{,.`date +%Y%m%d%H%M`}

8/4/14

23

Which Version Control System Do You Use? • cp foo foo.bak • cp foo{,.`date +%Y%m%d%H%M`-`$USER`}

Use Tools that Reflect & Reinforce Your Culture • Open Communication • Incentive and Responsibility Alignment

• Respect • Trust

Choose Your Own Adventure • What is DevOps? • Infrastructure as Code

• Version Control • Quit your job! • Incident Management

• Stages of Learning

• How to hire DevOps

• Podcasts & Podcasting

• Ask me anything


• Chef DK • Chef Metal • Chef for OSX • Chef Search • Knife • One repo or many?

• Chef vs …

8/4/14

24

Incident Management …and Post Mortems

Shit Happens! • Have a plan • Identify an Incident Commander • First responder • Decides what to do next • Coordinates resources • Can hand off command • Communicates Status • Not about rank

Post Mortems • Learning Opportunity • Blameless – here to learn, not blame

• EVERYONE is invited • Establish the timeline • Identify contributing factors • Describe customer impact • Open tickets for remediation tasks • Prioritize those tickets

8/4/14

25

Post Mortems • There is no such thing as a “root cause”

• Focus on MTTD and MTTR • MTBF holds much less weight









• Chef vs …

8/4/14

26

Stages of Learning

Shuhari • First learn, then detach, and finally transcend • shu - "obey" - traditional wisdom • ha - "detach" - break with tradition • ri - "separate" - transcend









• Chef vs …

8/4/14

27

Hiring DevOps

Who here cares?









• Chef vs …

8/4/14

28

Podcasts & Podcasting

DevOps-related Podcasts

Other podcasts I listen to • The Ruby Rogues • Ruby5 • Software Defined Talk • This American Life • The Truth

8/4/14

29

Podcasting • How do we do it on the Food Fight Show? • Google Hangouts

• Hangouts on Air • The Ruby Rep • Libsyn.com

Food Fight Show Downloads

115 Total Episodes 155,700+ Total Downloads









• Chef vs …

8/4/14

30

One repo or many? WTF, community?!

How many repos do I need? • One repo! • N + 1 repos!









• Chef vs …

8/4/14

31

This slide intentionally left blank

docker & chef and chef...8/4/14 1 docker & chef containers • linux containers (lxc) •...

Documents