doing it wrong with puppet -

1 / 33

Doing It Wrong with PuppetA small collection of anti-patterns, pitfalls

and bad practices

Felix Frank

PuppetCamp Berlin 2014

April 11, 2014

2 / 33

Bio

Felix FrankI 2004 – 2009 sysadmin and FOSS dev at DESY

I 2009 CS diploma at Tech Uni

I since 2009 Puppeteer at

– we are a company thatI manages complex IT systems and services for business

customersI hosts a sizable server fleetI relies on Puppet and git consequently

3 / 33

Agenda

Use boolean facts

Expect C-like values for parameters

Make excessive use of “if defined()”

Use large numbers of execs

Rely on dynamic scoping

4 / 33

Use boolean values for facts

On wanton ambiguity in your tool chain

5 / 33

False is relativeConsider this fact

I is virtual: true or false

Broken manifest I

I if ! $::is virtual {include hardware monitoring

}

Broken manifest II

I if $::is virtual != false {include hardware monitoring

}

Stupid manifest

I if $::is virtual != "false" {include hardware monitoring

}

6 / 33

Some background

True values in the puppet DSLI true or any non-empty string

Limitation in facter 1.x

master

agent fact value

fact code

ruby

as String

Correct way to implement such factsI return the empty string for false

7 / 33

It’s gonna be the future soon

I Facter 2 will allow boolean and other valuesI widespread adoption quite far off still

8 / 33

Up next

Use boolean facts

Expect C-like values for parameters

Make excessive use of “if defined()”

Use large numbers of execs

Rely on dynamic scoping

9 / 33

Expect C-like values for parameters

Or: treating Puppet like a scripting language pt. 1

10 / 33

The Perl trap

The puppet user baseI . . . comprises lots of admins with *NIX backgroundsI . . . also writes plenty of Shell and Perl scripts (also C)

. . . and these languages have no pure boolean values

True values e.g.in Puppet “foo”, any array, typically true

in Perl “foo”, non-empty array, typically 1False values e.g.

in Puppet empty string, undef, falsein Perl empty string/array/hash, typically 0

11 / 33

Building confusing modules

I define server module($enabled=0) {$dir = "/etc/..."file { "$dir/$title.conf": ...}if $enabled == 1 {

...# take some action}

}

inevitable WTF momentI server module { "foo": enabled => true }

I handled by documentationat best, and likely not untilafter the fact

12 / 33

Up next

Use boolean facts

Expect C-like values for parameters

Make excessive use of “if defined()”

Use large numbers of execs

Rely on dynamic scoping

13 / 33

Make excessive use of “if defined()”

A tale of borderline non-determinism

14 / 33

A common problemSeveral modules will sometimes have to manage a commonset or resources

I a subtree of /etc of mutual interestI a package for required functionality etc.

The naive implementation won’t work because Puppetdoesn’t allow multiple declaration of the same resource

I class php {...package { "imagemagick":

ensure => present }}class tomcat {

...package { "imagemagick":

ensure => present }}

15 / 33

A common workaround

Protect declarations with a function callclass php {

...if !defined(Package[imagemagick]) {

package { "imagemagick":ensure => present }

}}class tomcat {

...if !defined(Package[imagemagick]) {

package { "imagemagick":ensure => present }

}}

16 / 33

A possible issue with that

There is no protection against contradictionclass php {

...if !defined(Package[imagemagick]) {

package { "imagemagick":ensure => present }

}}class graphicsmagick {

...if !defined(Package[imagemagick]) {

package { "imagemagick":ensure => absent }

}}

17 / 33

A more likely scenarioIt’s easy to lose metaparametersclass php {

...if !defined(Package[imagemagick]) {

package { "imagemagick":ensure => present,require => File[...] }

}}class tomcat {

...if !defined(Package[imagemagick]) {

package { "imagemagick":ensure => present,notify => Exec[...] }

}}

18 / 33

By the way. . .

The latter issue can be worked aroundclass php {

...if !defined(Package[imagemagick]) {

package { "imagemagick":ensure => present,require => File[...] }

}else {

Package<| title == "imagemagick" |> {require +> File[...]

}}

}

19 / 33

A word about stdlib

I puppetlabs-stdlib, a collection of helpful parser functions

In theory, ensure resource() solves this more cleanlyclass php {

ensure resource(‘package’,‘imagemagick’,{ ensure => present } )

}I avoids conflicts for basic propertiesI more expressive power

It cannot solve the whole problem thoughI issue with metaparameters remainsI pertains to possible additional properties as wellI only slightly superior to if defined()

20 / 33

The ideal(ized) solutionWrapper classes for shared dependenciesclass php {

include imagemagick}class tomcat {

include imagemagick}

I still won’t allow the easy handling of metaparameters etc.I but you won’t even be tempted to tryI just require/subscribe/notify/. . . the class

I contradictions are not addressedI but there is no sensible way to do that

How is this better then?I the manifest has clear, nonambiguous semanticsI parse order dependencies avoided, see final slides

(virtual resources work too, but less flexibly)

21 / 33

Up next

Use boolean facts

Expect C-like values for parameters

Make excessive use of “if defined()”

Use large numbers of execs

Rely on dynamic scoping

22 / 33

Use large numbers of execs

Or: treating Puppet like a scripting language pt. 2

23 / 33

Implementing a HOWTO in a manifestSetting up software often comprises

I editing filesI running scripts and programs

. . . and often both of them in a set and mingled orderI it can be tempting to translate this verbatim

exec { "curl http://... >/tmp/...":creates => "..." }

->exec { "unzip /tmp/...":

creates => "/usr/local/..." }->file { "/usr/local/.../etc/...":

content => template(...) }->exec { "/usr/local/...": ... }->...

24 / 33

So what?

Problems with this approach (likely among others)I contradicts Puppet’s idea of resourcesI the catalog becomes complex with items and relationshipsI leads to plentiful error output in case of problems

A more maintainable pattern consists ofI a monolithic, robust script to perform all setup

I either templated or with a managed config fileI a single exec resource to invoke it

I with precise condition(s) for when to runI or better yet: create a deb or rpm package

Also – a quick word on refreshonly

I nice antipattern: use it to run script after retrieving itI prone for false positives and lost events

25 / 33

So remember

A small mnemonic

26 / 33

Up next

Use boolean facts

Expect C-like values for parameters

Make excessive use of “if defined()”

Use large numbers of execs

Rely on dynamic scoping

27 / 33

Rely on dynamic scoping

Or: how to jumble up your own manifest’s opinions

. . . which is another bout with nondeterminism

28 / 33

Brief reviewDynamic scoping

I in Puppet 2.x mainly for variable valuesclass foo {

$limited = trueinclude bar

}class bar {

if $limited {...

}}

I in Puppet 3.x only for resource defaultsclass foo {

File { ensure => present }include bar

}

29 / 33

The jumble

role::webserver

apache

tcpserver

sysctl

apache

tcpserver

sysctlsysctl

include

include

include

File { mode => 644 }

thread optimization

include

include

File { mode => 640 }

thread optimization

which default is in effect for sysctl?either, depending on parse order

30 / 33

Mitigation?

I Idea: just take care that the parse order is correctI only possible in very confined class structuresI scopes are generally too complex

scopes of classes late in the chain change through unexpectedfactors

31 / 33

Mixing things upscopes of classes late in the chain change through

I inclusion of more classesI removal of one or more classesI refactoring of manifests

32 / 33

ConclusionAvoid!

I parameters and Hiera will get you there much safer

You may want to move away from dynamic scopes anywayI they will likely get deprecated and removed

33 / 33

Thanks for your attentionImage sources

I https://www.pinterest.com/pin/418553359088246576/I http://www.kulfoto.com/funny-pictures/17395/its-called-

wireless-tech-and-its-the-futureI http://www.cacbasketball.com/b2-5v5-unification-finals-

uhhh-ditka/I http://www.someecards.com/usercards/

viewcard/MjAxMy00MzdlNjAzZjE2MWRkMjk0I http://www.marketingpilgrim.com/2013/08/google-glass-

update-like-having-an-admin-assistant-on-your-shoulder.html

I http://www.aboutbradsugars.com/tag/executive-coaching/I http://themetapicture.com/schrodingers-cat/I http://www.mrlovenstein.com/comic/50I http://funny-pics.co/photo/funny-cat-cheering-up-dog/

34 / 33

We are hiring

Always looking for techs whoI know their way around Puppet (or would like to)I further the development of our homegrown

infrastructure and toolsI will implement more technologies in our

management ecosystem

Visit us

http://mpexnetworks.de/ueber-uns/jobs.htmljobs@mpexnetworks.de

35 / 33

Bonus content!

36 / 33

Preferring new style class declaration

I the good thing about classes: they are singletonsI a class can be declared an arbitrary number of times

Class parameterizationI a class with parameters must be one of a kindI multiple declarations with different parameters just as

contradictory as with resources (or more so)Additional fun

I declaration using include implies all parameters use theirrespective default value

I does not mix with new style class { } declarationI mixing is allowed but only with all include statements

before the class { }I more parse order dependencies (yay!)