puppet camp 2012

20
From zero to Puppet http://www.rankpop.com/you-need-to-start-structuring-your-blog-posts-asap/

Upload: server-density

Post on 21-Jan-2015

544 views

Category:

Documents


1 download

DESCRIPTION

Going from zero to Puppet by Pedro Pessoa, Operations Engineer at Server Density. Abstract: Using out-of-the-box Puppet for non-sysadmin work - steps from going from no config management to managing 100 nodes and allowing non-sysadmin tasks to be performed. Speaker Bio: Linux admin for 10+ years. Java/Python/C developer 12+ years. Ops engineer at http://www.serverdensity.com - a hosted server and website monitoring service. Currently processing 12TB+ per month into MongoDB running on dedicated and virtual instances. www.serverdensity.com/puppetcamp/

TRANSCRIPT

Page 1: Puppet Camp 2012

From zero to Puppet

http://www.rankpop.com/you-need-to-start-structuring-your-blog-posts-asap/

Page 2: Puppet Camp 2012
Page 3: Puppet Camp 2012

+15TB / mth

+1bn docs /mth

2-5k inserts/s @ 3ms

10K RPM @ 140ms

Page 4: Puppet Camp 2012

Servers

HTTP Load Balancer – 5x

Apache - 14x

Build - 2x

MongoDB - 19x data, 13x routing, 6x configuration, 6x arbiter,

Page 5: Puppet Camp 2012

HTTP Load Balancer

$globalIPs_array = split($globalIPs, ',')

<% globalIPs_array.each do |globalIP| ­%>ListenHTTP        # primary public IP address        Address <%= globalIP %>        Port    80

Page 6: Puppet Camp 2012

Apache

Page 7: Puppet Camp 2012

file{ 'vhost':  path    =>  "/etc/apache2/sites­enabled/${::siteDomain}",  ensure  => file,  content => template('apache­php/vhost.erb'),  notify  => Service['apache2'],}

Apache

Page 8: Puppet Camp 2012

<VirtualHost *:80>ServerName <%= siteDomain %>DocumentRoot <%= documentRoot %>...        ErrorLog /var/log/apache2/error­<%= siteDomain %>.log<% if @requestLogging and requestLogging == "yes" %> CustomLog /var/log/apache2/access­<%= siteDomain %>.log vhost_combined<% end %>

<% if enableSSL == "yes" %><VirtualHost *:443>   SSLEngine OnSSLCertificateFile /var/www/ssl/<%= siteDomain %>.crtSSLCertificateKeyFile /var/www/ssl/<%= siteDomain %>.key...

Apache

Page 9: Puppet Camp 2012

$excess_bagage = [                    "ppp",                    "bind9­host",                    "laptop­detect",                    "open­iscsi",                    "libnss3:i386"                  ]package{ $excess_bagage: ensure => purged,}

Page 10: Puppet Camp 2012

exec{ 'ufw allow is­1':   command     => '/usr/sbin/ufw allow from 184.173.178.67',   unless      => '/usr/sbin/ufw status verbose | grep "184.173.178.67"',}exec{ 'ufw enable':   command     => '/usr/sbin/ufw enable',   unless      => '/usr/sbin/ufw status | grep "Status: active"',   Require     => [Exec['ufw allow is­1']],   notify      => Exec['ufw reload'],}exec{ 'ufw reload':   command     => '/usr/sbin/ufw reload',   require     => Exec['ufw enable'],   refreshonly => true,}

Page 11: Puppet Camp 2012

SSH

file{ 'sshd_config':   path    => '/etc/ssh/sshd_config',   ensure  => file,   content  => template('sshd/sshd_config.erb'),}

Port 22Protocol 2AllowUsers david pessoa chris rob wesCompression yes...<% if @duoSecKey and @duoIntKey and @duoHost %> ForceCommand /usr/sbin/login_duo PermitTunnel no AllowTcpForwarding no<% end %>

Page 12: Puppet Camp 2012

System updatesCANARIES=" \

aws­prod­apac­singapore­exm­a1 \a2.wdc.sl \exm1.wdc.sl \mtx­web1.wdc.sl \sdapp­web1.wdc.sl \sdapi­web1.wdc.sl \"

function canaries {echo "REMOVE *­web1.wdc.sl FROM ROTATION!"echo "press ENTER"readecho "have you REALLY removed *­web1.wdc.sl from rotation?"echo "press ENTER"read

for i in $CANARIESdo

echo $imco rpc puppetral ­I $i create type=exec \

                       title="/bin/bash ­c 'apt­get dist­upgrade ­y'"done

}

Page 13: Puppet Camp 2012

System updates

function reboot_canaries {echo "REMOVE *­web1.wdc.sl FROM ROTATION!"echo "press ENTER"readecho "have you REALLY removed *­web1.wdc.sl from rotation?"echo "press ENTER"readfor i in $CANARIESdo

echo $imco rpc puppetral ­I $i create type=exec \

                       title="/bin/bash ­c 'reboot'"echo "­­­­­­­­­­­­­­"

done}

Page 14: Puppet Camp 2012

What to Reboot

mco rpc puppetral create type=exec \   title="/bin/bash ­c 'ls /var/run/reboot­required'" \   | grep ­B 1  "Resource was created" \   | grep ".sl"

exm­md1a.wdc.sl                               Status: Resource was created   Resource: {"tags"=>["exec"],              "title"=>"/bin/bash ­c 'ls /var/run/reboot­required'",              "type"=>"Exec",              "parameters"=>{:returns=>:notrun},              "exported"=>false}

exm1.wdc.sl                                 Status: change from notrun to 0 failed:    /bin/bash ­c 'ls /var/run/reboot­required'    returned 2 instead of one of [0]

Page 15: Puppet Camp 2012

Live Management- Emergency

# Metrics<%= metricsIP %> metrics­svc

# Infrastructure services184.173.178.66  puppet

...

Page 16: Puppet Camp 2012

Live Management- Emergency

Page 17: Puppet Camp 2012

Live Management- Emergency

Page 18: Puppet Camp 2012

Live Management- Emergency

Page 19: Puppet Camp 2012

Live Management- Emergency

Page 20: Puppet Camp 2012

Pedro Pessoa

[email protected]

www.serverdensity.com