Puppet Camp Sydney

The (im)perfect Puppet module

2 AGENDA

Who we are

The common pattern

Pitfalls

Module standards

Module walk-through �

WHO IS ODECEE? Innovation for the Enterprise

A digital world where technology is simple and business gets what it wants

170+ staff

Banking/financial services

Communications Logistics Insurance Government

Offices in Melbourne & Sydney

Digital

DevOps

Sprout is designed to collaborate on new ideas and develop innovative ‘real world’ solutions.

4 WHO WE ARE

PHILIP JAY PETER HALL

•  Ten years industry experience •  Odecee Lead Engineer

•  Swinburne University Centre for Astrophysics & Supercomputing

•  realestate.com.au •  catchoftheday.com.au

@peterkh

•  Eight years industry experience •  Odecee Senior Engineer

•  Ericsson Mobile Packet Core •  Softel Systems •  Puppet certified professional #454

@ps_jay

5

Data

SCOPE

Business logic goes here

(not sharable)

Node

Role

Profile

No business logic Sharable on the Puppet Forge

Class

Module

Hiera

Explicit Hiera lookups

Automatic lookups only

Scope of our talk

6

Style guide:

•  v1.1.2: Existed 2011 – 2015 •  v2.0.1: Current 2015 onwards

Guiding principles: • Readability matters • Scoping and simplicity are key • Your module is a piece of

software

Covers writing code but not how to structure modules

CODE STANDARDS

docs.puppetlabs.com/guides/style_guide.html

7 MODULE STANDARDS

Two documents •  Module fundamentals: covers file-system layout

docs.puppetlabs.com/puppet/latest/reference/modules_fundamentals.html

•  Beginners guide to modules: step-by-step guide docs.puppetlabs.com/guides/module_guides/bgtm.html

8

>  puppet  module  generate  pcsyd15-­‐foo    pcsyd15-­‐foo/Gemfile  pcsyd15-­‐foo/metadata.json  pcsyd15-­‐foo/Rakefile  pcsyd15-­‐foo/README.md  pcsyd15-­‐foo/manifests/init.pp  pcsyd15-­‐foo/spec/spec_helper.rb  pcsyd15-­‐foo/spec/classes/init_spec.rb  pcsyd15-­‐foo/tests/init.pp  

MODULE SKELETON

9 AUTOMATIC PARAMETER LOOKUP

Puppet 3 – automatic lookup

class  foo  (      param_1,    #  no  default      param_2  =  “a  default”,  )  

Since puppet 3, parameters get their value automatically

Explicitly set in a resource-like declaration

Hiera lookup (automatic)

Default provided by class

If no default, catalog compile fails

Equivalent Puppet 2 code

class  foo  (      param_1  =  hiera(“foo::param_1”),      param_2  =  hiera(“foo::param_2”,  “a  default”)  )  

10 THE COMMON MODULE PATTERN

params.pp

•  Defaults •  Handle operating system

differences

init.pp

•  Inherit from params.pp •  Contain (or anchor) other

subclasses (install, config, …)

Hiera

config.pp, install.pp, etc.

•  Inherit from init.pp •  Contained by init.pp

init.pp variables referenced by all other

classes/defines

puppet-stdlib used for param validation

Well known Puppet Forge contributors following this pattern: •  puppetlabs •  example42 •  garethr

11 PITFALL #1: Looking up Hiera directly

• Module less sharable: not all installations use Hiera

• Harder to debug: more areas of code to identify Hiera lookups

• Non-obvious: Module user needs to know all “magic” Hiera values to set for desired outcome

• Complex unit testing: Need to feed Hiera values as well as parameters

12 PITFALL #1: Hunting for Hiera lookups

class  nginx::params  {      $worker_processes  =  $::processorcount      $worker_connections  =  ‘1024’      $conf_template  =  ‘nginx/nginx.conf.erb’      $error_pages_default  =  false      $set_real_ip  =  hiera(“nginx::real_ip_range”)      $real_ip_header  =  ‘X-­‐Forwarded-­‐For’      $blocked_ips  =  []      $certs  =  {}      $server_template  =  'nginx/server-­‐template.conf.erb’  ...  

13 PITFALL #2: Module Scope

• A module should only have one area of responsibility

• That module should be the only module with that area of responsibility

• Anti-pattern: •  Apache module for installing httpd •  MyPHPApp module installs a vhost config in: /etc/httpd/conf.d/

14 PITFALL #3: Dependencies

• Don’t have “too many” dependencies •  puppet-stdlib is okay & expected •  Aim for zero dependencies

• Testing is more complex: need to include the dependency

• Module less sharable: now dependent on additional classes that others may not have

• From Module Fundamentals: “Be wary of having classes declare classes from other modules”

15

Example of a class including a class: #  Class  for  creating  the  PuppetDB  postgresql  database  class  puppetdb::database::postgresql(      …  )  inherits  puppetdb::params  {      …      #  create  the  puppetdb  database      postgresql::server::db  {  $database_name:          …      }  }  

Must also have puppetlabs-postgresql module

Alternatively: •  Document in the README

and then link in a “profile” class

•  Create module parameter for inclusion of the external class

PITFALL #3: puppetlabs-puppetdb

Module Walk-through

puppetlabs-ntp

github.com/puppetlabs/puppetlabs-ntp

CONSIDERATIONS

Anchor pattern or contain function

needed for classes inside classes

Hiera needed to keep profiles and

roles sensible

Puppet version > 2.6.2 needed for

param inherits

18

Separation between modules keeps testing simple

Common patterns reduces staff ramp up time

Mix puppet forge and local modules

with ease

IN SUMMARY

Questions?

@odecee www.facebook.com/odeceesocial www.odecee.com.au www.linkedin.com/company/odecee

Philip Jay Peter Hall Lead DevOps engineer

@peterkh @ps_jay Senior DevOps Engineer