infrastructure as software - puppetconf 2014
DESCRIPTION
Infrastructure as Software - Dustin J. Mitchell, Mozilla, Inc.TRANSCRIPT
INFRASTRUCTURE AS SOFTWAREINFRASTRUCTURE AS SOFTWAREDustin J. Mitchell
[email protected] 24, 2014
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
1 of 37 09/24/2014 03:32 PM
CODECODEInfrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
2 of 37 09/24/2014 03:32 PM
CODE = SNIPPETCODE = SNIPPET
var width=150; // width of the eyes in pixelsvar colour="#06f"; // colour of the eye - bluey green in this casevar iris="#000"; // colour of the iris (normally black);/***************************\* Moving Eyeballs Effect **(c)2012-3 mf2fm web-design ** http://www.mf2fm.com/rv ** DON'T EDIT BELOW THIS BOX *\***************************/var swide=800;function addLoadEvent(funky) { var oldonload=window.onload; if (typeof(oldonload)!='function') window.onload=funky; else window.onload=function() {...
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
3 of 37 09/24/2014 03:32 PM
CODE = SNIPPETCODE = SNIPPET
# == Class: baseconfig## Performs initial configuration tasks for all Vagrant boxes.#class baseconfig { exec { 'apt-get update': command => '/usr/bin/apt-get update'; }
host { 'hostmachine': ip => '192.168.0.1'; }
file { '/home/vagrant/.bashrc': owner => 'vagrant', group => 'vagrant', mode => '0644', source => 'puppet:///modules/baseconfig/bashrc'; }}
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
4 of 37 09/24/2014 03:32 PM
CODE = SCRIPTCODE = SCRIPT
#! /usr/bin/env python
class RequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
def do_POST(self): content_length = int(self.headers['content-length']) data = self.rfile.read(content_length) self.send_response(200) self.end_headers() self.wfile.write("accepted.\n") self.wfile.close() now = int(time.time()) filename = os.path.join(REPORT_DIR, "report-%s.yaml" % (now,)) fd = os.open(filename, os.O_EXCL|os.O_CREAT|os.O_WRONLY) os.fdopen(fd, "w").write(data)
def main(): logging.basicConfig(format="%(asctime)s %(message)s", level=logging.DEBUG) SocketServer.TCPServer.allow_reuse_address = True httpd = SocketServer.TCPServer(("", PORT), RequestHandler) httpd.serve_forever()
main()
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
5 of 37 09/24/2014 03:32 PM
--
“My friend Clift Norris has identified a fundamental constantthat I call Norris’ number, the average amount of code an
untrained programmer can write before he or she hits a wall. Cliftestimates this as 1,500 lines. Beyond that the code becomes so
tangled that the author cannot debug or modify it withoutherculean effort.”
John D. Cook
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
6 of 37 09/24/2014 03:32 PM
SOFTWARE IN PUPPET?SOFTWARE IN PUPPET?Let me tell you a story..
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
7 of 37 09/24/2014 03:32 PM
tinyurl.com/puppetagainMEET PUPPETAGAINMEET PUPPETAGAINOpen SourceWritten in PuppetCross-PlatformHighly AvailableSecureDesigned to Manage Job-Runners
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
8 of 37 09/24/2014 03:32 PM
SOFTWARESOFTWAREArchitectural models
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
9 of 37 09/24/2014 03:32 PM
OPENSTACK ARCHITECTUREOPENSTACK ARCHITECTUREInfrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
10 of 37 09/24/2014 03:32 PM
APPROACHABILITYAPPROACHABILITYIf I change this, what will happen?
Where should I start reading code?
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
11 of 37 09/24/2014 03:32 PM
PUPPETAGAIN: TOPLEVELPUPPETAGAIN: TOPLEVELclass toplevel::base { include users::root}class toplevel::server inherits toplevel::base { include puppet::periodic include cron}class toplevel::server::mozpool inherits toplevel::server { include bmm include mozpool}
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
12 of 37 09/24/2014 03:32 PM
PUPPETAGAIN: TOPLEVELPUPPETAGAIN: TOPLEVELnode "mobile-imaging1.p1.releng.scl3.mozilla.com" { include toplevel::server::mozpool}
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
13 of 37 09/24/2014 03:32 PM
PUPPETAGAIN: CONFIGURATIONPUPPETAGAIN: CONFIGURATION# modules/config/manifests/base.ppclass config::base { $ntp_server = ''}# manifests/moco-config.ppclass config inherits config::base { $ntp_server = 'time.mozilla.org'}
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
14 of 37 09/24/2014 03:32 PM
PUPPETAGAIN: CONFIGURATIONPUPPETAGAIN: CONFIGURATIONclass ntp::config { include ::config if ($::config::ntp_server) { .. }}
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
15 of 37 09/24/2014 03:32 PM
SOFTWARESOFTWAREArchitectural modelsControlled Interdependencies
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
16 of 37 09/24/2014 03:32 PM
APACHE: MODULESAPACHE: MODULESInfrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
17 of 37 09/24/2014 03:32 PM
PUPPETAGAIN: DIRSPUPPETAGAIN: DIRSclass dirs::builds { file { "/builds": ensure => directory; }}class dirs::builds::slave { include dirs::builds file { "/builds/slave": ensure => directory; }}
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
18 of 37 09/24/2014 03:32 PM
PUPPETAGAIN: DIRSPUPPETAGAIN: DIRSclass talos { include dirs::builds::slave file { "/builds/slave/talos-slave": ensure => directory; }}
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
19 of 37 09/24/2014 03:32 PM
SOFTWARESOFTWAREArchitectural modelsControlled InterdependenciesOrganizing Principles
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
20 of 37 09/24/2014 03:32 PM
PA: PRINCIPLE OF LEAST SURPRISEPA: PRINCIPLE OF LEAST SURPRISE
.. installs mig-agent .. or dies trying
include mig_agent::install
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
21 of 37 09/24/2014 03:32 PM
PA: PRINCIPLE OF LEAST SURPRISEPA: PRINCIPLE OF LEAST SURPRISEclass mig_agent::install { case $operatingsystem { CentOS: { .. } default: { fail("Cannot install on $operatingsystem") } }}
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
22 of 37 09/24/2014 03:32 PM
SOFTWARESOFTWAREArchitectural modelsControlled InterdependenciesOrganizing PrinciplesAbstractions
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
23 of 37 09/24/2014 03:32 PM
PUPPET: PACKAGESPUPPET: PACKAGES
Write once, run everywhere, right?
package { 'httpd': ensure => '2.2.15';}
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
24 of 37 09/24/2014 03:32 PM
PUPPETAGAIN: PACKAGESPUPPETAGAIN: PACKAGESinclude packages::httpd
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
25 of 37 09/24/2014 03:32 PM
PUPPETAGAIN: PACKAGESPUPPETAGAIN: PACKAGESclass packages::httpd { case $::operatingsystem { CentOS: { package { "httpd": ensure => latest; } } Ubuntu: { package { "apache2": ensure => latest; } } Darwin: { # installed by default } default: { fail("cannot install on $::operatingsystem") } } }
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
26 of 37 09/24/2014 03:32 PM
WRITING SOFTWAREWRITING SOFTWAREIS HARDIS HARD
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
27 of 37 09/24/2014 03:32 PM
CHALLENGESCHALLENGESPackage repositories are part of the code
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
28 of 37 09/24/2014 03:32 PM
REPOSITORIES:REPOSITORIES:PINNING IS HARDPINNING IS HARD
Un-specified prerequisite packages aren't pinnedCan confuse package managersModifying the repo makes production changes
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
29 of 37 09/24/2014 03:32 PM
REPOSITORIES:REPOSITORIES:MIRRORING IS HARDMIRRORING IS HARD
“Can we update mirrors now?”
“No.”
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
30 of 37 09/24/2014 03:32 PM
REPOSITORIES:REPOSITORIES:SHARING IS HARDSHARING IS HARD
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
31 of 37 09/24/2014 03:32 PM
REPOSITORIES:REPOSITORIES:THEY'RE HUGETHEY'RE HUGE
[[email protected] dmitchell]# df -h /dataFilesystem Size Used Avail Use% Mounted on/dev/mapper/vg_relengpuppet2-lv_data 414G 315G 78G 81% /data
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
32 of 37 09/24/2014 03:32 PM
CHALLENGESCHALLENGESPackage repositories are part of the codeInteractions are hard to model
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
33 of 37 09/24/2014 03:32 PM
MODULE INTERACTIONSMODULE INTERACTIONS'httpd' module installs Apache'rsyslog' module installs rsyslogWhen both are installed, we want to send access logs to rsyslog
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
34 of 37 09/24/2014 03:32 PM
MODULE INTERACTIONSMODULE INTERACTIONS
Httpd has to know about rsyslog?
class httpd::logging { include rsyslog::config_dir file { "${rsyslog::config_dir::dir}/httpd.conf": content => template("${module_name}/rsyslogd_httpd.conf.erb"); }}
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
35 of 37 09/24/2014 03:32 PM
CHALLENGESCHALLENGESPackage repositories are part of the codeInteractions are hard to modelAcceptance-level testing is hard
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
36 of 37 09/24/2014 03:32 PM
GO FORTH ANDGO FORTH ANDWRITE SOFTWAREWRITE SOFTWARE
Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
37 of 37 09/24/2014 03:32 PM