Download - How I hack on puppet modules
How I hack PuppetHow I hack Puppetand made my life soo much easier
by
Kris Buytaert
Kris BuytaertKris Buytaert● I used to be a Dev,I used to be a Dev,● Then Became an OpThen Became an Op● Chief Trolling Officer and Open Source Chief Trolling Officer and Open Source
Consultant @inuits.euConsultant @inuits.eu● Everything is an effing DNS ProblemEverything is an effing DNS Problem● Building Clouds since before the bookstoreBuilding Clouds since before the bookstore● Some books, some papers, some blogsSome books, some papers, some blogs● Evangelizing devopsEvangelizing devops● But mostly, trying to be good at my jobBut mostly, trying to be good at my job
TodayToday
• VagrantVagrant
• StyleStyle
• Testing PuppetTesting Puppet
• Modules vs ModulesModules vs Modules
• JenkinsJenkins
• Demo ?Demo ?
Typical EnvironmentsTypical EnvironmentsFor DevsFor Devs
● ScrumScrum
● Version ControlVersion Control
● Automated Build Automated Build
● Bugtracking Bugtracking
● Continous integrationContinous integration
● Integrated testingIntegrated testing
● Automated Automated deployment deployment
For OpsFor Ops
● KanbanKanban
● Version ControlVersion Control
● Automated Build Automated Build
● Bugtracking Bugtracking
● Continous integrationContinous integration
● Integrated testingIntegrated testing
● Automated Automated deployment deployment
Everybody is a developerEverybody is a developer
● Yes we write code alsoYes we write code also
● httpd.conf, squid.conf, my.cnfhttpd.conf, squid.conf, my.cnf
● Just crappy languages :)Just crappy languages :)
● shell, perl, ruby, python, puppetshell, perl, ruby, python, puppet
● So those rules apply for EveryoneSo those rules apply for Everyone
VagrantVagrant● Abstraction layer for Abstraction layer for
VirtualBoxVirtualBox
● Integrates well with Integrates well with Puppet/ChefPuppet/Chef
● Project = Project =
● VagrantfileVagrantfile
● Manifests / Manifests / CookbooksCookbooks
● Portable, Small , Portable, Small , VersionableVersionable
VeeweeVeewee● Vabgrantbox.esVabgrantbox.es
● Use veewee to build your boxenUse veewee to build your boxen
•gem install veeweegem install veewee
•veewee templatesveewee templates
•veewee init natty ubuntu-11.04-server-amd64veewee init natty ubuntu-11.04-server-amd64
•vagrant basebox build nattyvagrant basebox build natty
•vagrant box add 'natty' 'natty.box'vagrant box add 'natty' 'natty.box'
A Vagrant projectA Vagrant project
[sdog@mine vagrant-graphite]$ lsmanifests modules README TODO Vagrantfile [sdog@mine vagrant-graphite]$ tree -dL 2.├── manifests│ └── hosts└── modules ├── apache ├── collectd ├── graphite ├── jmxtrans ├── logster ├── statsd └── tattle
10 directories
A VagrantfileA VagrantfileVagrant::Config.run do |config| # All Vagrant configuration is done here. The most common configuration # options are documented and commented below. For a complete reference, # please see the online documentation at vagrantup.com.
config.vm.define :mongo1 do |mongo1_config| mongo1_config.ssh.max_tries = 100 mongo1_config.vm.box = "MyCentOS2" mongo1_config.vm.network("192.168.99.101") mongo1_config.vm.host_name = "mongo1" mongo1_config.vm.provision :puppet do |mongo1_puppet| mongo1_puppet.pp_path = "/tmp/vagrant-puppet" mongo1_puppet.manifests_path = "manifests" mongo1_puppet.module_path = "modules" mongo1_puppet.manifest_file = "site.pp" end end config.vm.define :mongo2 do |mongo2_config| mongo2_config.ssh.max_tries = 100 mongo2_config.vm.box = "MyCentOS2" mongo2_config.vm.network("192.168.99.102") mongo2_config.vm.host_name = "mongo2" mongo2_config.vm.provision :puppet do |mongo2_puppet| mongo2_puppet.pp_path = "/tmp/vagrant-puppet" mongo2_puppet.manifests_path = "manifests" mongo2_puppet.module_path = "modules" mongo2_puppet.manifest_file = "site.pp" end end
Vagrant RocksVagrant Rocks● Vagrant init Vagrant init
● Vagrant up Vagrant up
● Vagrant provision Vagrant provision
● Vagrant down Vagrant down
● Vagrant destroyVagrant destroy
Booting UpBooting Up[sdog@stillmine vagrant-graphite]$ vagrant up[default] VM already created. Booting if its not already running...[default] Preparing host only network...[default] Clearing any previously set forwarded ports...[default] Forwarding ports...[default] -- carbon: 2003 => 2021 (adapter 1)[default] -- http2: 8080 => 50051 (adapter 1)[default] -- http: 80 => 50050 (adapter 1)[default] -- ssh: 22 => 2222 (adapter 1)[default] Cleaning previously set shared folders...[default] Creating shared folders metadata...[default] Running any VM customizations...[default] Booting VM...[default] Waiting for VM to boot. This can take a few minutes.[default] VM booted and ready for use![default] Enabling host only network...[default] Setting host name...[default] Mounting shared folders...[default] -- v-root: /vagrant[default] -- manifests: /tmp/vagrant-puppet/manifests[default] -- v-pp-m0: /tmp/vagrant-puppet/modules-0[default] Running provisioner: Vagrant::Provisioners::Puppet...[default] Running Puppet with site.pp...[default] notice: /Stage[main]/Default-repo/Yumrepo[epel]/descr: descr changed 'Extra Packages for Enterprise Linux 6.x' to 'Extra Packages for Enterprise Linux 6.x '[default] [default] notice: /Stage[main]/Up-graph::Document-throughput/File[/usr/local/up-graph/document-throughput/graph.sh]/content: content changed '{md5}5085413bb8465edc1815024e1e06c0e6' to '{md5}c7d73b7f91a751632742f569030cb3d7'[default] [default] notice: /Stage[main]/Collectd/Service[collectd]/ensure: ensure changed 'stopped' to 'running'[default] [default] notice: Finished catalog run in 12.45 seconds[default]
Failed ProvisioningFailed Provisioning[sdog@stillmine vagrant-graphite]$ vagrant provision [default] Running provisioner: Vagrant::Provisioners::Puppet...[default] Running Puppet with site.pp...[default] Could not parse for environment production: Syntax error at 'import'; expected '}' at /tmp/vagrant-puppet/manifests/site.pp:5 on node graphite.lan[default] The following SSH command responded with a non-zero exit status.Vagrant assumes that this means the command failed!
cd /tmp/vagrant-puppet/manifestspuppet apply --modulepath '/tmp/vagrant-puppet/modules-0' site.pp
The output of the command prior to failing is outputted below:
[no output]
Vagrant TipsVagrant Tips
Set up a puppetmaster node Set up a puppetmaster node
Stored configsStored configs
Hiera Hiera
Mount Symlink /etc/puppet dirs Mount Symlink /etc/puppet dirs
/vagrant is your host folder/vagrant is your host folder
Handover to devsHandover to devs● Gitrepo Gitrepo
•VagrantfileVagrantfile
•A Box definitionA Box definition
•Puppet Manifests Puppet Manifests
•Puppet modules Puppet modules
•Their codeTheir code
=> A reproducable environment easy to => A reproducable environment easy to migrate to test/uat/prod.migrate to test/uat/prod.
StyleStyle● I never caredI never cared
● JanJan Coworkers complained Coworkers complained
● http://docs.puppetlabs.com/guides/style_guidehttp://docs.puppetlabs.com/guides/style_guide
BeforeBeforenode 'nova' {
yumrepo {
'epel': baseurl => $operatingsystemrelease ? { '6.0' => "http://mirror.eurid.eu/epel/6/$hardwaremodel/", '*' => "http://mirror.eurid.eu/epel/5/$hardwaremodel/", }, descr => $operatingsystemrelease ? { '6.0' => 'Extra Packages for Enterprise Linux 6.x ', '*' => 'Extra Packages for Enterprise Linux 5.x', }, gpgcheck => 0, enabled => 1;
'openstack': baseurl => "http://yum.griddynamics.net/yum/diablo-centos", descr => "openstack diablo at griddynamics", gpgcheck => 0, enabled => 1;
}
service { "iptables": ensure => "stopped", enable => "false"; }
package { "centos-release-cr": ensure => "present"; "openstack-nova": ensure => "present"; }
AfterAfternode 'nova' {
yumrepo { 'epel': baseurl => $operatingsystemrelease ? { '6.0' => "http://mirror.eurid.eu/epel/6/$hardwaremodel/", '*' => "http://mirror.eurid.eu/epel/5/$hardwaremodel/", }, descr => $operatingsystemrelease ? { '6.0' => 'Extra Packages for Enterprise Linux 6.x', '*' => 'Extra Packages for Enterprise Linux 5.x', }, gpgcheck => 0, enabled => 1;
'openstack': baseurl => "http://yum.griddynamics.net/yum/diablo-centos", descr => "openstack diablo at griddynamics", gpgcheck => 0, enabled => 1; }
service { "iptables": ensure => "stopped", enable => "false"; }
package { "centos-release-cr": ensure => "present"; "openstack-nova": ensure => "present"; }
Puppet LintPuppet Lintpuppet-lint nova.pp WARNING: double quoted string containing no variables on line 18WARNING: double quoted string containing no variables on line 19WARNING: double quoted string containing no variables on line 30WARNING: double quoted string containing no variables on line 31WARNING: double quoted string containing no variables on line 32WARNING: double quoted string containing no variables on line 36WARNING: double quoted string containing no variables on line 37WARNING: double quoted string containing no variables on line 38WARNING: double quoted string containing no variables on line 39WARNING: => on line 47 isn't aligned with the previous lineWARNING: => on line 48 isn't aligned with the previous lineWARNING: => on line 49 isn't aligned with the previous lineWARNING: => on line 52 isn't aligned with the previous lineWARNING: => on line 53 isn't aligned with the previous lineWARNING: => on line 54 isn't aligned with the previous lineWARNING: => on line 55 isn't aligned with the previous line
Note: Buy Tim Sharpe more beer ...
Testing PuppetTesting Puppet● puppet --parseonly test.pppuppet --parseonly test.pp
● puppet parser validate test.pppuppet parser validate test.pp
● erb -x -T '-' foreman-vhost.conf.erb | ruby -c erb -x -T '-' foreman-vhost.conf.erb | ruby -c
● Cucumber-puppetCucumber-puppet
● rspec-puppetrspec-puppet
Editor pluginsEditor plugins● Vim : https://github.com/rodjek/vim-puppetVim : https://github.com/rodjek/vim-puppet
● Emacs etc .. Emacs etc ..
Puppet-modulePuppet-module● Generates a clean structureGenerates a clean structure
● and unused code and unused code
● Does stuff with forge (Does stuff with forge (Does anyone actually use forge ? )Does anyone actually use forge ? )
● + shell script also create class stubs + shell script also create class stubs
Classes vs ModulesClasses vs Modules● Module : Module :
● Abstract definition on configuring a Abstract definition on configuring a serviceservice
● Class : Class :
● Specific implementation of your use case Specific implementation of your use case of such a module of such a module
•e.g usernames / passwords / hosts do not e.g usernames / passwords / hosts do not belong in modulesbelong in modules
Parametrized ClassesParametrized Classes
Services and ModulesServices and ModulesNot in my cluster Not in my cluster please ! please !
service {service {
'blah':'blah':
Ensure => running,Ensure => running,
}}
class autofs::disable inherits autofs {class autofs::disable inherits autofs {
Service['autofs'] {Service['autofs'] {
ensure => 'stopped' ,ensure => 'stopped' ,
enable => false,enable => false,
}}
}}
GitHubGitHub
The Nr 1 place for your modulesThe Nr 1 place for your modules(Till forge.puppetlabs.org is revamped)(Till forge.puppetlabs.org is revamped)
A github projectA github project
[sdog@mine vagrant-graphite]$ lsmanifests modules README TODO Vagrantfile [sdog@mine vagrant-graphite]$ tree -dL 2.├── manifests│ └── hosts└── modules ├── apache ├── collectd ├── graphite ├── jmxtrans ├── logster ├── statsd └── tattle
10 directories
SubmodulesSubmodules git submodule git submodule
-d132d2ba4b8c4848acee3ee8e6144b68e86bedc3 modules/apache-d132d2ba4b8c4848acee3ee8e6144b68e86bedc3 modules/apache
6a44fb1179e23ff42f476233d9c65b545be76470 modules/collectd 6a44fb1179e23ff42f476233d9c65b545be76470 modules/collectd (heads/master)(heads/master)
+24560116dd498033d81a444f5e326ba9b8b7aefc modules/graphite +24560116dd498033d81a444f5e326ba9b8b7aefc modules/graphite (heads/master)(heads/master)
9b5c07afcf4e0b18f48346093053a7d46222a19a modules/jmxtrans 9b5c07afcf4e0b18f48346093053a7d46222a19a modules/jmxtrans (heads/master)(heads/master)
1a200f9dfae4f346b7f10f7b8f33076d99670ef8 modules/logster (heads/master)1a200f9dfae4f346b7f10f7b8f33076d99670ef8 modules/logster (heads/master)
-7f801701898dec12b578aafa35f7df8fa64f2c05 modules/statsd-7f801701898dec12b578aafa35f7df8fa64f2c05 modules/statsd
2d506cac88a731d142b434a99646926c015af6a3 modules/tattle (heads/master)2d506cac88a731d142b434a99646926c015af6a3 modules/tattle (heads/master)
Git FlowGit Flow● git flow feature start blahgit flow feature start blah
● git flow feature finish blah git flow feature finish blah
● git flow release start blah git flow release start blah
● git flow release finish blah git flow release finish blah
JenkinsJenkins● Pull git Pull git
● ““Build” Build”
•puppet-syntax-puppet-syntax-check.shcheck.sh
•puppet-lint-check.shpuppet-lint-check.sh
•Generate-puppet-Generate-puppet-doc.shdoc.sh
● Package & deploy Package & deploy
DemoDemo$git clon$git clone e [email protected]@github.com:KrisBuytaert/vagrant-:KrisBuytaert/vagrant-graphite.gitgraphite.git
$cd vagrant-graphite$cd vagrant-graphite
$git submodule init$git submodule init
$git submodule update $git submodule update There's a submodule in a submodule !There's a submodule in a submodule !
$vagrant up$vagrant up
$vagrant provision$vagrant provision
$vagrant provision (or fix your ordering) $vagrant provision (or fix your ordering)
My code sucked, but I try to make it suck less My code sucked, but I try to make it suck less step by step, step by step,
patches welcomepatches welcome
Homework: Homework: ● puppet-rspecpuppet-rspec
● cucumber-puppetcucumber-puppet
● guardguard
● jenkins vagrant pluginjenkins vagrant plugin
ContactContactKris Buytaert Kris Buytaert [email protected]@inuits.eu
Further ReadingFurther Reading@krisbuytaert @krisbuytaert http://krisbuytaert.be/blog/http://krisbuytaert.be/blog/http://www.inuits.eu/http://www.inuits.eu/
InuitsInuitsDuboistraat 50Duboistraat 502060 Antwerpen2060 AntwerpenBelgiumBelgium891.514.231891.514.231
+32 475 961221+32 475 961221