continuously testing infrastructure - beyond module testing - puppetconf 2014

111
Continuously Testing Infrastructure Puppet Conf, San Francisco, 2014 Gareth Rushgrove Beyond Module Testing

Upload: puppet-labs

Post on 01-Dec-2014

1.302 views

Category:

Technology


0 download

DESCRIPTION

Continuously Testing Infrastructure - Beyond Module Testing - Gareth Rushgrove, Puppet Labs

TRANSCRIPT

Page 1: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Continuously Testing Infrastructure

Puppet Conf, San Francisco, 2014

Gareth Rushgrove

Beyond Module Testing

Page 2: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

@garethr

Page 3: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 4: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 5: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 6: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Not talking about

Page 7: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Finished software

Gareth Rushgrove

Page 8: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Testing individual modules

Gareth Rushgrove

Page 9: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

puppet-lint, puppet-syntax, rspec-puppet, beaker

Gareth Rushgrove

Page 10: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 11: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Am talking about

Page 12: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Experiments

Gareth Rushgrove

Page 13: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Testing images and containers

Gareth Rushgrove

Page 14: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Test driving infrastructure as a service

Gareth Rushgrove

Page 15: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Testing with PuppetDB

Gareth Rushgrove

Page 16: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Testing images and containers

1

Page 17: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 18: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Packer builds images based on a JSON template

Gareth Rushgrove

Page 19: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 20: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

It has some Puppet integration too

Gareth Rushgrove

Page 21: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 22: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

But how do we know the image works?

Gareth Rushgrove

Page 23: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Lets add some tests!

Gareth Rushgrove

Page 24: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 25: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

shaunduncan/packer-provisioner-host-command

Gareth Rushgrove

Page 26: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

serverspec.org

Gareth Rushgrove

Page 27: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 28: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 29: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 30: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Serverspec also supports port, file, ppa, selinux, user, group, lxc, iptables, cron and more

Gareth Rushgrove

Page 31: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Only publish the image if the tests pass

Gareth Rushgrove

Page 32: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Run tests automatically with a continuous integration system

Gareth Rushgrove

Page 33: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 34: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 35: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

garethr/packer-serverspec-example

Gareth Rushgrove

Page 36: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 37: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Same approach works with containers too

Gareth Rushgrove

Page 38: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 39: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

garethr/docker-spec-example

Gareth Rushgrove

Page 40: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Test drive your IaaS

2

Page 41: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Test driven development

Gareth Rushgrove

Page 42: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

First the developer writes an automated test case that defines a desired improvement or new function

Gareth Rushgrove

Page 43: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Then produces the minimum amount of code to pass that test

Gareth Rushgrove

Page 44: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

And finally refactors the new code

Gareth Rushgrove

Page 45: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

First the developer writes an automated test case that defines a desired improvement or new function

Page 46: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Your infrastructure should!have an API

Gareth Rushgrove

Page 47: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

What if we write assertions against!that API?

Gareth Rushgrove

Page 48: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Aside: Clojure

2.1

Page 49: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 50: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Great for building DSLs

Gareth Rushgrove

Page 51: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Don’t worry, you could write the examples in any language

Gareth Rushgrove

Page 52: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Policy driven development

Gareth Rushgrove

Page 53: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

I don’t want to launch too many nodes, they’re expensive

Gareth Rushgrove

Policy

Page 54: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 55: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

I don’t want any stopped nodes, they are costing me money

Gareth Rushgrove

Policy

Page 56: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 57: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Large nodes are really expensive, so limit their usage

Gareth Rushgrove

Policy

Page 58: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 59: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

We should be backing up every node

Gareth Rushgrove

Policy

Page 60: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 61: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

I only want nodes in London and !San Francisco

Gareth Rushgrove

Policy

Page 62: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 63: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

All our nodes should be named environment-name

Gareth Rushgrove

Policy

Page 64: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 65: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

garethr/digitalocean-expect

Gareth Rushgrove

Page 66: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 67: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Now we have the tests, we can provision some infrastructure

Gareth Rushgrove

Page 68: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Aside: Provisioning with Puppet

2.2

Page 69: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 70: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 71: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

puppetlabs/gce_compute

Gareth Rushgrove

Page 72: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 73: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 74: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

garethr/digitalocean

Gareth Rushgrove

Page 75: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 76: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

bobtfish/aws_api

Gareth Rushgrove

Page 77: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Testing with PuppetDB

3

Page 78: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Aside: PuppetDB

3.1

Page 79: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

puppetlabs/puppetdb

Gareth Rushgrove

Page 80: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

PuppetDB can store a lot of data about your infrastructure

Gareth Rushgrove

Page 81: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

The most recent facts from every node

Gareth Rushgrove

Page 82: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

The most recent catalog for every node

Gareth Rushgrove

Page 83: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

A wide range of metrics

Gareth Rushgrove

Page 84: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 85: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

I want to run the same operating system on all hosts

Gareth Rushgrove

Policy

Page 86: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 87: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Security enforcing packages should be installed everywhere

Gareth Rushgrove

Policy

Page 88: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 89: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

I want to limit how many puppet resources I’m using

Gareth Rushgrove

Policy

Page 90: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 91: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

We should avoid heavy I/O load on the database by maintaining a high catalog duplication rate

Gareth Rushgrove

Policy

Page 92: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 93: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

garethr/puppetdb-expect

Gareth Rushgrove

Page 94: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Testing based on PuppetDB

3.2

Page 95: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

PuppetDB is a great source of context for tests

Gareth Rushgrove

Page 96: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Generate serverspec tests from PuppetDB data

Gareth Rushgrove

Page 97: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Automatically detect hosts, and generate commands

Gareth Rushgrove

Page 98: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 99: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Match puppet resources to serverspec resources

Gareth Rushgrove

Page 100: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 101: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

For instance on a Puppet Enterprise master

Gareth Rushgrove

Page 102: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 103: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Run serverspec tests on all puppet managed hosts

Gareth Rushgrove

Page 104: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Gareth Rushgrove

Page 105: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

garethr/serverspec-puppetdb

Gareth Rushgrove

Page 106: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Conclusions

Page 107: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Is this monitoring?

Gareth Rushgrove

Page 108: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

We’re still moving towards infrastructure as code

Gareth Rushgrove

Page 109: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Infrastructure as code rather than infrastructure from code

Gareth Rushgrove

Page 110: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Taking about policy as code might help communicate intent

Gareth Rushgrove

Page 111: Continuously Testing Infrastructure - Beyond Module Testing - PuppetConf 2014

Questions?And thanks for listening