1

Taking Devops to the Next Level!

Max Martin Developer & Manager

@maxmartin

2

So  you’ve  automated  your  infrastructure…  

•  Configura7on  as  code  •  Be9er  insight  into  the  structure  of  your  systems  

•  More  robust  deployments  •  But  how  do  you  reach  the  next  level?  

3

Advances  in  the  Puppet  Ecosystem  

•  Puppet  3.0+  •  Hiera  •  PuppetDB  •  Mcollec7ve  2.0+  •  Geppe9o  •  Puppet  Forge  

4

PuppetDB 0.9.0 •  Big Data for Puppet •  Fast, scalable Stored

Configs

Mcollective 2.0 •  Sub-collectives •  Advanced Filter Syntax

Puppet OSS 3.0 •  Performance •  Data Bindings

Puppet Enterprise 2.8 •  AIX

Puppet Enterprise 2.6/2.7 •  RBAC and external auth •  Certificate Management

Puppet Enterprise 2.5 •  Windows •  Puppet Forge Integration •  Puppet Data Library

Puppet Enterprise 2.0 •  Discovery •  Cloud Provisioning •  Orchestration

2011   2012   2013   You Are Here  

Puppet Enterprise 1.0 •  Configuration Management •  Reporting

Puppet Enterprise 3.0 •  Enhanced Scalability & Performance •  Dynamic Discovery •  Progressive Deployment •  Orchestration for Windows •  Module Reusability

Puppet OSS 3.1 and 3.2 •  API docs •  Future parser

Mcollective 2.2 •  Direct addressing •  Pluggable Discovery

PuppetDB 1.4 * Report Storage

5

Puppet  3.0  –  Speed  and  Scalability  

    +100%   2.7x Speed-up  

6

Puppet  3.0  –  Hiera  func7ons  +  data  bindings  

•  Hiera:  hierarchical  key-­‐value  store  •  Keeps  site-­‐specific  data  out  of  Puppet  code  •  Parameter  values  are  now  automa7cally  looked  up  in  Hiera  

•  Hiera  func7ons  in  core  give  7ghter  integra7on  

7

Puppet  3.0  –  Hiera  Example  node ”ntp.example.com" {

class { "ntp": servers => [ '0.us.pool.ntp.org iburst’,

'1.us.pool.ntp.org iburst',

'2.us.pool.ntp.org iburst',

'3.us.pool.ntp.org iburst'], autoupdate => false, restrict => false, enable => true, }

}

8

Puppet  3.0  –  Hiera  Example  { "ntp::restrict" : false, "ntp::autoupdate" : false, "ntp::enable" : true, "ntp::servers" : [ "0.us.pool.ntp.org iburst", "1.us.pool.ntp.org iburst", "2.us.pool.ntp.org iburst", "3.us.pool.ntp.org iburst" ]

}

9

Puppet  3.0  –  Hiera  Example  

node "ntp.example.com" { include ntp }

10

PuppetDB  

•  Fast,  durable  data  storage  for  data  generated  by  Puppet,  including:  – Facts  – Catalogs  – Reports  – Exported  resources  

•  Replaces  exis7ng  libraries  (including  storeconfigs  and  the  old  Inventory  Service  backend),  is  much  faster  &  more  reliable  

11

PuppetDB  -­‐  Technology  

•  Based  on  PostgreSQL:  reliable,  ba9le-­‐tested  •  Wri9en  in  Clojure:  fast,  mul7-­‐core  •  Based  on  the  JVM:  can  be  shipped  as  a  JAR  

12

PuppetDB  -­‐  Technology  

•  Thousands  of  PuppetDB  deployments  •  Hundreds  of  threads  per  install  •  Zero  deadlocks  •  Zero  bugs  involving  state  

13

PuppetDB  –  Query  Syntax  and  APIs  •  AST-­‐based  query  syntax  

["and", ["=", "type", "User”], ["=", "title", ”max"]] ["and", ["=”,["fact", "operatingsystem"], "Debian"], ["<”,["fact", "uptime_seconds"], 10000]]

•  API  for  defining  your  own  query  syntax  –  Erik  Dalén,  Spo7fy:  h9ps://github.com/dalen/puppet-­‐puppetdbquery  

•  Rich  APIs  make  it  easy  to  build  your  own  tools  to  consume  Puppet  data  

14

PuppetDB  –  Event  Inspector  

15

PuppetDB  -­‐  Puppetboard  h9ps://github.com/nedap/puppetboard    

16

PuppetDB  -­‐  Metrics  

17

MCollec7ve  

A  powerful  orchestra7on  engine  

18

Mcollec7ve  -­‐  Subcollec7ves  $ mco inventory --list-collectives * [ ==================================== ] 52 / 52 Collective Nodes ========== ===== za_collective 2 us_collective 7 uk_collective 19 de_collective 24 eu_collective 45 mcollective 52 Total nodes: 52

19

Mcollec7ve  –  Pluggable  Discovery  

•  Query  a  file:  $ mco rpc service restart service=httpd --nodes=hosts.txt

•  Discover  using  PuppetDB:  $ mco rpc service restart service=httpd -W country=uk --dm=puppetdb

•  Direct  addressing:  $ mco rpc rpcutil ping –I example.com

20

MCollec7ve  –  Ruby  Integra7on  c = rpcclient("service") c.discover :nodes => File.readline("hosts.txt").map {|i| i.chomp}

printrpc c.restart(:service => "httpd")

21

Geppe9o  

•  IDE  for  developing  Puppet  modules  &  code  •  Integrates  with  git  &  SVN  •  Linux,  OS  X  &  Windows  •  Debug  your  Puppet  modules  as  you  develop  them,  and  publish  directly  to  the  Puppet  Forge  

22

Puppet  Forge  

•  Puppet  Labs’  module  repository  •  1,500+  community-­‐contributed  modules  •  Improved  search  •  Team  of  module  engineers  •  Standards  &  best  prac7ces  

23

Taking  you  to  the  next  level  of…  

•  Speed  •  Scalability  •  Extensibility  •  Reliability  •  Flexibility  •  Agility  •  Community