using puppet - netways · using puppet why it exists, how it works, and why it works this way. ......
TRANSCRIPT
Luke [email protected], Reductive Labs
Nashville, TennesseeUSA
Using Puppet
Why it exists, how it works, and why it works this way
Introductions
Image from http://www.flickr.com/photos/tafa/27605946/sizes/o/
* How many of you would consider yourselves sysadmins?* How many of you manage sysadmins?* Frightening: How many of you are managed by sysadmins?* How many of you have heard of Puppet?* How many of you use Puppet?
Puppet is a system for centrally specifying and
managing your infrastructure
Written in Ruby
Three major pieces
• A declarative, resource-oriented language
Three major pieces
• A declarative, resource-oriented language
• An execution engine
Three major pieces
• A declarative, resource-oriented language
• An execution engine
• A bunch of daemons and tools
Three major pieces
1) A resource-oriented language
Image from http://www.flickr.com/photos/heather/147388906/sizes/l/
Your infrastructure is code
Image from http://www.flickr.com/photos/echo_29/25829591/sizes/o/
And code is great because we have tooling like version control, and decades of experience with it
Resource Abstraction Layer (RAL)
Do you really care how rpm works?
rpm/apt/etc becomes packages/etc/hosts becomes hostsuseradd/netinfo, etc. becomes users/etc/init.d/smf/launchd/daemontools becomes services
Resource Providers
23 package types
Users in NetInfo, useradd, pw
Support for Debian, Ubuntu, Red Hat, Solaris, OS X, Gentoo, SuSE, FreeBSD, and more
Windows is being actively worked on
2) An engine for applying
that specification repeatably
Image from http://www.flickr.com/photos/jurvetson/480227362/sizes/l/
2) An engine for applying
that specification repeatably
Image from http://www.flickr.com/photos/jurvetson/480227362/sizes/l/
Specifications are idempotent
Idempotency is what allows us to manage a machine through its whole lifecycle
Specifications are idempotent
Idempotency is what allows us to manage a machine through its whole lifecycle
3) Daemons and executables that
determine Puppet’s architecture
client/server, cert mgmt, etc.
Centralized Management
puppetd in the cloud
Puppetmasterd
puppetdOS X puppetd
Linux
Code
Uses SSL, and provides a Certificate Authority
Image from http://www.flickr.com/photos/piet_musterd/2307596484/sizes/l/
* Every connection is encrypted, and the only connection that isn’t authenticated is the one that asks for a signed cert* Client certs* Autosign, manual sign, manual certificate generation* You don’t even have to use it
ralsh - a thin API wrapper
This uses the same model as the rest of puppet -- it chooses the appropriate provider for the local system. You can edit resources, and it even works over the network.
Who’s using Puppet?
“...at Google we're currently using Puppet to
manage close to 6,000 Macs, and it's likely our deployment will expand
dramatically beyond that....”
Testimonials
. . .at Google we're currently using it to manage close to 6,000 Macs, and it's likely our deployment will expand dramatically beyond that. . . Nigel KerstenMacOps
Thank You Puppet!! . . .We had a guy on staff who was doing only these things almost full time, so puppet's allowed him to do bigger and better things (like take Christmas break off :) Eugene Ventimiglia Director of Systems GridApp Systemsd
User Community Who Volunteer Info(http://reductivelabs.com/trac/puppet/wiki/WhosUsingPuppet)
The Ohio State University Department of Mathematics- Primarily used for: Mac OS X workstation and GNU/Linux server management.
Wordtracker- We introduced Puppet gradually by managing a handful of office workstations, and are now developing recipes for building servers with it
Imperial College London HEP group- We use puppet to manage our HPC clusters, our desktops and some of our servers
Bashton Ltd- Country: UK- We use Puppet to manage our own cluster of servers, as well as creating and supporting configurations for our customers
Fab:IT ApS- We use puppet to administrate our own servers placed in the local datacenter, and our customers placed both in the same datacenter, and scattered around the country. Puppet is ideal
because of it's ease of use, and especially because the puppet server compiles the configuration specifically for the individual host, so that no irrelevant information goes to the wrong nodes. Thanks, Luke, for a great tool!
- Country: Denmark
Geni Inc. (www.geni.com)- We use puppet to manage the configurations of all our Linux servers. With puppet my server build time is roughly 12-15 minutes to get a server ready for production.
Institute of Cognitive Science at the University of Osnabrueck, Germany- After having some headaches with cfengine, we started managing our linux workstations with puppet.- Shortly after Linux, we also started managing our Mac OS X workstations.
Why does Puppet exist?
Somebody has to do something, and it's just incredibly pathetic that it has to be us.
-- Jerry Garcia
Community
• 1000+ people on the mailing lists
• 200+ people in IRC all day
• More than 50 contributors
Programming SysAdmin
Low-level, non-portable
Assembly commands and files
Abstract, portable C* Resources
An Analogy
* For small values of abstract
* The assembly programmers fought the adoption of C* Fear for your career if you’re a bit too fond of assembly* It’s not about fewer people, it’s about higher quality and productivity* Are there more or fewer programmers today than in the days of assembly?
Image from http://flickr.com/photos/kenskritters/2128853769/
Speciation
These are house finches, reminding one of the finches Darwin observed in the Galapagos. I want to cause sysadmin speciation.
Sysadmins do too much
Image from http://flickr.com/photos/shirleytwofeathers/2068713495/
Firefighter? Architect? Developer? Tape-changer? All of the above?
Let’s talk about you
Do you have any computers?
Some questions
Some questions
• How many of you have written software to manage computers?
Some questions
• How many of you have written software to manage computers?
• How many have published this software?
Developer Sysadmin
Developer Sysadmin
Three people
Puppet allows you to bridge that gap
Image from http://www.flickr.com/photos/patrick-smith-photography/2969769911/sizes/o/
Either you can manage many machines with little effort
Either you can manage many machines with little effort
Or you can’t
How to use Puppet
The strategical version
Seek the pain
Image from http://www.flickr.com/photos/pagedooley/2147718252/sizes/l/
Solve the simple problems
Add the infrastructure features you always
wanted
How to use Puppet
The tactical version
Think like Puppet thinks
Think like Puppet thinks
• Resources, not text snippets or lines added to files
Think like Puppet thinks
• Resources, not text snippets or lines added to files
• What resources are you managing?
Think like Puppet thinks
• Resources, not text snippets or lines added to files
• What resources are you managing?
• How are they related to each other?
Replace Shell Scripts with ResourcesThis:
Replace Shell Scripts with Resources
Becomes:
This:
Replace Shell Scripts with Resources
Becomes:
This:
Replace Shell Scripts with Resources
Becomes:
This:
Replace Shell Scripts with Resources
Becomes:
This:
Relationships matter but are often implicit
Relationships matter but are often implicit
Package
Relationships matter but are often implicit
Package
Configuration
Configuration should get modifed after package
installation
Relationships matter but are often implicit
Package
Service
Service should restart whenconfiguration changesConfiguration
Configuration should get modifed after package
installation
Explicit Relationships
We’ll come back to abstraction
Relationships provide ordering and notification
"Exec[createrepo-PM-RHEL5-noarch]"
"Yumrepo[PM-RHEL5-x86_64]"
"Yumrepo[PM-RHEL5-noarch]"
"Package[postgresql-server]"
"Package[thttpd]"
"File[/var/www/thttpd/html/yum-PM-RHEL5-noarch]""File[/var/www/thttpd/html/yum-PM-RHEL5-x86_64]"
"Exec[rsync-rpmdir-PM-RHEL5-x86_64]"
"Exec[createrepo-PM-RHEL5-x86_64]"
"Postgres::Role[puppet]"
"Exec[rsync-rpmdir-PM-RHEL5-noarch]"
Classes document Intent
This is shareable, releasable code.Classes are analogous with tags
Organize files into modules
Provide platform abstraction
Debian
We’re doing the same thing with different commands on different platforms
Provide platform abstraction
Debian
Red Hat
We’re doing the same thing with different commands on different platforms
Provide platform abstraction
Debian
Red Hat
We’re doing the same thing with different commands on different platforms
Portability and Naming
Class Membership
Class Membership
Puppet in the cloud
Heavily used in EC2, VMWare, Xen, etc
Puppet allows you to avoid the foil ball
Image from http://www.flickr.com/photos/fungep/2516767121/sizes/l_
The Gory Details
Status and Roadmap
• In production usage for years
• About to release a new version with significant internal refactoring
• Adding simple graphical tools this year
Puppet and Capistrano
or ControlTier
Puppet vs. Cfengine
It scales like HTTPS
because it is https
And you don’t even need to centralize it.
Logs go to syslog (by default)
The people behind Puppet
This is all I do
Bad product,hungry Luke
Employs three people and still growing
Support, Paid features, Consulting, Training
Questions?