puppet at the centre of everything
DESCRIPTION
Puppet can be used as a control panel to perform a wide variety of tasks within your infrastructure. It can be used to trigger failover between hot standby servers or entire data centres. It can be used to deploy package updates across large clusters. It can be used to deploy code to staging and then onto production, ensuring the right versions are present. It can be used to replicate production environments locally for all our engineers. And it can even be used for config management! This talk will take a hands on technical look at how we use Puppet to achieve all these things. It will include code samples, hints and tips and explain how the flexibility of Puppet can be used to really control your entire infrastructure. This talk was given by David Mytton at PuppetCamp 2013: Berlin, London and Munich.TRANSCRIPT
![Page 1: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/1.jpg)
Puppet at the centre of everything
![Page 2: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/2.jpg)
David Mytton
Woop Japan!
![Page 3: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/3.jpg)
![Page 4: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/4.jpg)
Server Density Architecture
•100 servers - Ubuntu 12.04
![Page 5: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/5.jpg)
Server Density Architecture
•100 servers - Ubuntu 12.04
•50% virtual
![Page 6: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/6.jpg)
Server Density Architecture
•100 servers - Ubuntu 12.04
•Nginx, Python, MongoDB
•50% virtual
![Page 7: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/7.jpg)
Server Density Architecture
•100 servers - Ubuntu 12.04
•Nginx, Python, MongoDB
•25TB data per month
•50% virtual
![Page 8: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/8.jpg)
Use cases
• Config
![Page 9: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/9.jpg)
Use cases
• Config
• Failover
![Page 10: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/10.jpg)
Use cases
• Config
• Code deploys
• Failover
![Page 11: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/11.jpg)
Use cases
• Config
• Code deploys
• System updates
• Failover
![Page 12: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/12.jpg)
• Base manifest
Config
![Page 13: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/13.jpg)
![Page 14: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/14.jpg)
![Page 15: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/15.jpg)
• Base manifest
Config
• Sub manifests
![Page 16: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/16.jpg)
![Page 17: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/17.jpg)
• base
• softlayer
• softlayer-dedicated
• rackspace
• ...
![Page 18: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/18.jpg)
Failover
• /etc/hosts
<%= metrics2IP %> alerts.honshuu.prod audit.honshuu.prod auth.honshuu.prod cloud.honshuu.prod datastore.honshuu.prod inventory.honshuu.prod ...
![Page 19: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/19.jpg)
![Page 20: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/20.jpg)
![Page 21: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/21.jpg)
![Page 22: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/22.jpg)
![Page 23: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/23.jpg)
Failover
• /etc/hosts
108.168.254.33 puppet
![Page 24: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/24.jpg)
![Page 25: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/25.jpg)
Expected
Dealing with outages
![Page 26: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/26.jpg)
Really test your vendors
Dealing with outages
![Page 27: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/27.jpg)
Simulations
Dealing with outages
![Page 28: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/28.jpg)
• Commit
Code deploys
![Page 29: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/29.jpg)
![Page 30: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/30.jpg)
• Commit
Code deploys
• Build
![Page 31: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/31.jpg)
![Page 32: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/32.jpg)
• Commit
Code deploys
• Build
• Deploy
![Page 33: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/33.jpg)
![Page 34: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/34.jpg)
![Page 35: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/35.jpg)
![Page 36: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/36.jpg)
![Page 37: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/37.jpg)
/opt/puppet/bin/mco puppetd runonce -I <host>
![Page 38: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/38.jpg)
• Commit
Code deploys
• Build
• Deploy
• Puppet run
![Page 39: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/39.jpg)
![Page 40: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/40.jpg)
Code deploys
Advantages
APIs
![Page 41: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/41.jpg)
Code deploys
Advantages
APIs
Already there
![Page 42: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/42.jpg)
Code deploys
Advantages
APIs
Already there
System facts
![Page 43: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/43.jpg)
Code deploys
Disadvantages
Slow
![Page 44: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/44.jpg)
Code deploys
Disadvantages
Slow
Puppet run already
![Page 45: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/45.jpg)
Code deploys
Disadvantages
Slow
Puppet run already
Eventually consistent
![Page 46: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/46.jpg)
System updates
• Bash script
![Page 47: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/47.jpg)
System updates
• Bash script
• Remote commands
![Page 48: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/48.jpg)
![Page 49: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/49.jpg)
System updates
• Bash script
• Remote commands
• Notifications
![Page 50: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/50.jpg)
System updates
• Bash script
• Remote commands
• Notifications
• Canaries
![Page 51: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/51.jpg)
![Page 52: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/52.jpg)
•Build master (buildbot): VM x2 CPU 2.0Ghz, 2GB RAM – $89/m
•Build slave (buildbot): VM x1 CPU 2.0Ghz, 1GB RAM – $40/m
•Staging load balancer: VM x1 CPU 2.0Ghz, 1GB RAM – $40/m
•Staging server 1: VM x2 CPU 2.0Ghz, 8GB RAM – $165/m
•Staging server 2: VM x1 CPU 2.0Ghz, 2GB RAM – $50/m
•Puppet master: VM x2 CPU 2.0Ghz, 2GB RAM – $89/m
Total: $473/m
Colo experiment
![Page 53: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/53.jpg)
![Page 54: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/54.jpg)
Colo experiment
•Dell 1U R415
•x2 8C AMD 2.8Ghz
•32GB RAM
![Page 55: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/55.jpg)
Colo experiment
•Dell 1U R415
•x2 8C AMD 2.8Ghz
•32GB RAM
•Dual PSU, NIC
![Page 56: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/56.jpg)
Colo experiment
•Dell 1U R415
•x2 8C AMD 2.8Ghz
•32GB RAM
•Dual PSU, NIC
•x4 1TB SATA hot swappable
![Page 57: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/57.jpg)
Dedicated?
![Page 58: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/58.jpg)
Dedicated?
•Hardware replacement
![Page 59: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/59.jpg)
Dedicated?
•Hardware replacement
•Managed/support
![Page 60: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/60.jpg)
Dedicated?
•Hardware replacement
•Managed/support
•Networking
![Page 61: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/61.jpg)
Colo?
![Page 62: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/62.jpg)
Colo?
•Hardware spec/value
![Page 63: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/63.jpg)
Colo?
•Hardware spec/value
•Total cost
![Page 64: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/64.jpg)
Colo?
•Hardware spec/value
•Total cost
•Internal skills?
![Page 65: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/65.jpg)
Colo?
•Hardware spec/value
•Total cost
•Internal skills?
•More fun?!
![Page 66: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/66.jpg)
![Page 67: Puppet at the centre of everything](https://reader034.vdocument.in/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/67.jpg)
Colo: Power
•£300-350/kWh/m
•4.5A = £520/m
•9A = £900/m