What?
Wednesday, June 27, 12
What?
• Chef at Etsy
Wednesday, June 27, 12
What?
• Chef at Etsy
• Familiarity and Understanding
Wednesday, June 27, 12
What?
• Chef at Etsy
• Familiarity and Understanding
• Critical Approach and Experimentation
Wednesday, June 27, 12
What?
• Chef at Etsy
• Familiarity and Understanding
• Critical Approach and Experimentation
• Use The Source
Wednesday, June 27, 12
What?
• Chef at Etsy
• Familiarity and Understanding
• Critical Approach and Experimentation
• Use The Source
• A liberal sprinkling of screwups
Wednesday, June 27, 12
What?
• Chef at Etsy
• Familiarity and Understanding
• Critical Approach and Experimentation
• Use The Source
• A liberal sprinkling of screwups
• Open Sourced Goodness - We’re all here!
Wednesday, June 27, 12
What?
• Chef at Etsy
• Familiarity and Understanding
• Critical Approach and Experimentation
• Use The Source
• A liberal sprinkling of screwups
• Open Sourced Goodness - We’re all here!
• [x] = http://tiny.cc/velocity2012
Wednesday, June 27, 12
Opscode is Orange,Velocity is Blue.In Soviet Russia,
Cookbook writes you.
Wednesday, June 27, 12
Chef at Etsy
Wednesday, June 27, 12
Our Setup
Wednesday, June 27, 12
Our Setup• Open Source chef server 0.10.4
Wednesday, June 27, 12
Our Setup• Open Source chef server 0.10.4
• Backup to Opscode Platform
Wednesday, June 27, 12
Our Setup• Open Source chef server 0.10.4
• Backup to Opscode Platform
• ~800 self-hosted nodes (Mainly CentOS, some RHEL & mac)
Wednesday, June 27, 12
Our Setup• Open Source chef server 0.10.4
• Backup to Opscode Platform
• ~800 self-hosted nodes (Mainly CentOS, some RHEL & mac)
• KVM & lxc virts, self hosted too.
Wednesday, June 27, 12
Our Setup• Open Source chef server 0.10.4
• Backup to Opscode Platform
• ~800 self-hosted nodes (Mainly CentOS, some RHEL & mac)
• KVM & lxc virts, self hosted too.
• Never test in production!
Wednesday, June 27, 12
Our Setup• Open Source chef server 0.10.4
• Backup to Opscode Platform
• ~800 self-hosted nodes (Mainly CentOS, some RHEL & mac)
• KVM & lxc virts, self hosted too.
• Never test in production!
• Many chefs don’t spoil our soup
Wednesday, June 27, 12
Our Setup• Open Source chef server 0.10.4
• Backup to Opscode Platform
• ~800 self-hosted nodes (Mainly CentOS, some RHEL & mac)
• KVM & lxc virts, self hosted too.
• Never test in production!
• Many chefs don’t spoil our soup
Wednesday, June 27, 12
Familiarity and Understanding
Wednesday, June 27, 12
not
http://www.flickr.com/photos/photo_secessionist/5555167113/ (C) Alexander McQueen
Wednesday, June 27, 12
Wednesday, June 27, 12
• Insight
Wednesday, June 27, 12
• Insight
• Simplicity
Wednesday, June 27, 12
• Insight
• Simplicity
• Standards
Wednesday, June 27, 12
Insight
Wednesday, June 27, 12
Insight
• You should never have to say “I don’t know.”
Wednesday, June 27, 12
Insight
• You should never have to say “I don’t know.”
• What, where, when, why, how long?
Wednesday, June 27, 12
Insight
• You should never have to say “I don’t know.”
• What, where, when, why, how long?
• Easy, and I’ll show you how!
Wednesday, June 27, 12
Wednesday, June 27, 12
Chef Dashboard
Wednesday, June 27, 12
Chef Dashboard• Chef handler sends metrics to graphite [4]
Wednesday, June 27, 12
Chef Dashboard• Chef handler sends metrics to graphite [4]• git clone git://github.com/etsy/chef-handlers.git
Wednesday, June 27, 12
Chef Dashboard• Chef handler sends metrics to graphite [4]• git clone git://github.com/etsy/chef-handlers.git
• Set your graphite server’s URL in graphite.rb
Wednesday, June 27, 12
Chef Dashboard• Chef handler sends metrics to graphite [4]• git clone git://github.com/etsy/chef-handlers.git
• Set your graphite server’s URL in graphite.rb
• Add the following to client.rb
Wednesday, June 27, 12
Chef Dashboard• Chef handler sends metrics to graphite [4]• git clone git://github.com/etsy/chef-handlers.git
• Set your graphite server’s URL in graphite.rb
• Add the following to client.rb• require "<clonedir>/graphite.rb"
graphite_handler = GraphiteReporting.newreport_handlers << graphite_handlerexception_handlers << graphite_handler
Wednesday, June 27, 12
Chef Dashboard• Chef handler sends metrics to graphite [4]• git clone git://github.com/etsy/chef-handlers.git
• Set your graphite server’s URL in graphite.rb
• Add the following to client.rb• require "<clonedir>/graphite.rb"
graphite_handler = GraphiteReporting.newreport_handlers << graphite_handlerexception_handlers << graphite_handler
• Etsy dashboards framework [5]
Wednesday, June 27, 12
[12:54:01] <irccat> Chef run failed on gfernandez.vm.ny4dev.etsy.com[12:54:02] <irccat> https://github.etsycorp.com/gist/384228[12:54:02] <irccat>
[12:54:07] <irccat> Chef run failed on buildtest11.ny4dev.etsy.com[12:54:07] <irccat> https://github.etsycorp.com/gist/384227[12:54:07] <irccat>
Wednesday, June 27, 12
Chef irccat Alerts
Wednesday, June 27, 12
Chef irccat Alerts
• Chef handler send fails to irc[4] via irccat [6]
Wednesday, June 27, 12
Chef irccat Alerts
• Chef handler send fails to irc[4] via irccat [6]• git clone git://github.com/etsy/chef-handlers.git
Wednesday, June 27, 12
Chef irccat Alerts
• Chef handler send fails to irc[4] via irccat [6]• git clone git://github.com/etsy/chef-handlers.git
• Set your irccat[6] server’s URL in logtoirc.rb
Wednesday, June 27, 12
Chef irccat Alerts
• Chef handler send fails to irc[4] via irccat [6]• git clone git://github.com/etsy/chef-handlers.git
• Set your irccat[6] server’s URL in logtoirc.rb
• Add the following to client.rb
Wednesday, June 27, 12
Chef irccat Alerts
• Chef handler send fails to irc[4] via irccat [6]• git clone git://github.com/etsy/chef-handlers.git
• Set your irccat[6] server’s URL in logtoirc.rb
• Add the following to client.rb• require "<clonedir>/logtoirc.rb"
exception_handlers << Etsy::LogToIRC.new
Wednesday, June 27, 12
~ > knife node lastrun buildtest11.ny4dev.etsy.comStatus failed Elapsed Time 4.628171438 Start Time 2012-06-18 10:06:28 +0000End Time 2012-06-18 10:06:32 +0000
Recipe Action Resource Type Resource
Backtrace<snip>
ExceptionChef::Exceptions::Package: package[php] (php::buildtest line 20) had an error: Version 5.3.10-1.el5 of php not found. Did you specify both version and release? (version-release, e.g. 1.84-10.fc6)
Wednesday, June 27, 12
~ > knife search node 'lastrun_debug_formatted_exception:Chef\:\:Exceptions\:\:Package*' -a lastrun.debug.formatted_exception
5 items found
id: masterrestore.ny4.etsy.comlastrun.debug.formatted_exception: Chef::Exceptions::Package: package[postgresql-server] (postgresql::server-8.3 line 1) had an error: Installed package postgresql-server-8.3.16-1PGDG_id is newer than candidate package postgresql-server-8.3.11-1PGDG_id.rhel5
id: buildtest11.ny4dev.etsy.comlastrun.debug.formatted_exception: Chef::Exceptions::Package: package[php] (php::buildtest line 20) had an error: Version 5.3.10-1.el5 of php not found. Did you specify both version and release? (version-release, e.g. 1.84-10.fc6)
<snip>
Wednesday, June 27, 12
Chef lastrun Info
Wednesday, June 27, 12
Chef lastrun Info
• Chef handler and knife plugin [7]
Wednesday, June 27, 12
Chef lastrun Info
• Chef handler and knife plugin [7]
• gem install knife-lastrun
Wednesday, June 27, 12
Chef lastrun Info
• Chef handler and knife plugin [7]
• gem install knife-lastrun
• Add the following to client.rb
Wednesday, June 27, 12
Chef lastrun Info
• Chef handler and knife plugin [7]
• gem install knife-lastrun
• Add the following to client.rb• require "lastrun_update"
handler = LastRunUpdateHandler.newreport_handlers << handlerexception_handlers << handler
Wednesday, June 27, 12
Chef lastrun Info
• Chef handler and knife plugin [7]
• gem install knife-lastrun
• Add the following to client.rb• require "lastrun_update"
handler = LastRunUpdateHandler.newreport_handlers << handlerexception_handlers << handler
• knife node lastrun <nodename>
Wednesday, June 27, 12
Simplicity
Wednesday, June 27, 12
Simplicity
• Think of yourself at 3AM!
Wednesday, June 27, 12
Simplicity
• Think of yourself at 3AM!
• Please, won’t you think of the new guy?
Wednesday, June 27, 12
Simplicity
• Think of yourself at 3AM!
• Please, won’t you think of the new guy?
• Minimize the logics!
Wednesday, June 27, 12
Simplicity
• Think of yourself at 3AM!
• Please, won’t you think of the new guy?
• Minimize the logics!
• As few logical steps from start to finish as possible.
Wednesday, June 27, 12
Simplicity - Not!Date: Mon Dec 05 2011 23:07:18 GMT+0000 (GMT)
Subject: so close to death
# Don't install v2 on search or Cent 5.6 nodes-if node[:fqdn] !~ /\b(^(preprod-)?search[0-9]{2}|ny4dev\.etsy\.com|^(preprod-)?giftsweb[0-9]{2}|^db(shard|spare|data)[0-9]{2}|^qa-web01|^devsearch[0-9]{2}|^nagios01|^webnest[0-9]{2}|^prodking[0-9]{2}|^sandboxweb[0-9]{2}|^virt((0[5-9])|(1[0-9]))|^msysmgr[0-9]{2}|^msysmta[0-9]{2}|^dbconvo[0-9]{2}|^dbshowcase01|atlasweb[0-9]{2}|devnagios[0-9]{2}|cimaster02|worker[0-9]{2}|^ganglia[0-9]{2}|^imgcache[0-9]{2}|imgconvert[0-9]{2}|^imgwriter[0-9]{2}|dev-img02|^datacache04|^graphite01|^graphite03|^webutil03|^webutil04|^statsd01|^maintweb[0-9]{2}|^(dev-|preprod-)?convosearch[0-9]{2}|deployinator[0-9]{2}|^wpadmin01|^(preprod-)?dbtasks[0-9]{2})\b/ and node.role?("Web56") == false and node.role?("Preprodweb56") == false and node.role?("Princess53") == false+if node[:fqdn] !~ /\b(^(preprod-)?search[0-9]{2}|ny4dev\.etsy\.com|^(preprod-)?giftsweb[0-9]{2}|^db(shard|spare|data)[0-9]{2}|^qa-web01|^devsearch[0-9]{2}|^nagios01|^webnest[0-9]{2}|^prodking[0-9]{2}|^sandboxweb[0-9]{2}|^virt((0[5-9])|(1[0-9]))|^msysmgr[0-9]{2}|^msysmta[0-9]{2}|^dbconvo[0-9]{2}|^dbshowcase01|atlasweb[0-9]{2}|devnagios[0-9]{2}|cimaster02|worker[0-9]{2}|^ganglia[0-9]{2}|^imgcache[0-9]{2}|imgconvert[0-9]{2}|^imgwriter[0-9]{2}|dev-img02|^datacache04|^graphite01|^graphite03|^webutil03|^webutil04|^statsd01|^maintweb[0-9]{2}|^(dev-|preprod-)?convosearch[0-9]{2}|deployinator[0-9]{2}|^wpadmin01|^(preprod-)?dbtasks[0-9]{2})\b/ and node.role?("Web56") == false and node.role?("Preprodweb56") == false and node.role?("Princess53") == false and node.role?("Auth") == false
Wednesday, June 27, 12
Simplicity - Better!if node.chef_environment == "libmemcached_upgrade"
package "libmemcached" do
version "1.0.4-1"
action :install
end
<snip>
else
package "libmemcached" do
version "0.53-1.1"
action :install
end
<snip>
end
Wednesday, June 27, 12
Simplicity - Complexity
Wednesday, June 27, 12
Simplicity - Complexity
• Sometimes you need complex behaviour
Wednesday, June 27, 12
Simplicity - Complexity
• Sometimes you need complex behaviour
• Don’t fight it, try to abstract it.
Wednesday, June 27, 12
Simplicity - Complexity
• Sometimes you need complex behaviour
• Don’t fight it, try to abstract it.
• Case in point: Syslog-ng refactor
Wednesday, June 27, 12
Case Study: Syslog-ng
Wednesday, June 27, 12
Case Study: Syslog-ng
• 36 recipes
Wednesday, June 27, 12
Case Study: Syslog-ng
• 36 recipes
• 30 versions of syslog-ng.conf
Wednesday, June 27, 12
Case Study: Syslog-ng
• 36 recipes
• 30 versions of syslog-ng.conf
• 27 manually configured files in /etc/syslog-ng.d on central server
Wednesday, June 27, 12
Case Study: Syslog-ng
• 36 recipes
• 30 versions of syslog-ng.conf
• 27 manually configured files in /etc/syslog-ng.d on central server
• Edge cases and exceptions galore
Wednesday, June 27, 12
Case Study: Syslog-ng
Wednesday, June 27, 12
Case Study: Syslog-ng
• Down to:
Wednesday, June 27, 12
Case Study: Syslog-ng
• Down to:
• 2 recipes (one client, one server)
Wednesday, June 27, 12
Case Study: Syslog-ng
• Down to:
• 2 recipes (one client, one server)
• 2 templates (one for syslog-ng.conf, one for stuff in /etc/syslog-ng.d)
Wednesday, June 27, 12
Case Study: Syslog-ng
• Down to:
• 2 recipes (one client, one server)
• 2 templates (one for syslog-ng.conf, one for stuff in /etc/syslog-ng.d)
• Attributes in roles
Wednesday, June 27, 12
Case Study: Syslog-ng
• Down to:
• 2 recipes (one client, one server)
• 2 templates (one for syslog-ng.conf, one for stuff in /etc/syslog-ng.d)
• Attributes in roles
• Not open sourced yet, sorry :(
Wednesday, June 27, 12
Case Study: Syslog-ng"syslog": {
"group": "preprod_web",
"items": {
"web_apache_access_log": {
"source": "/var/log/httpd/access_log",
"source_program_override": "APACHEACCESS: ",
"destination": "<snip>/access.log",
"destination_filters": [
"host('^preprod-web')",
"match('APACHEACCESS')"
],
"destination_options": [
"template_escape(no)"
]
},
}
}
Wednesday, June 27, 12
Remember, No Panacea!
Wednesday, June 27, 12
Remember, No Panacea!
• A new package hits the repo.
Wednesday, June 27, 12
Remember, No Panacea!
• A new package hits the repo.
• Are you in control of when it goes out?
Wednesday, June 27, 12
Remember, No Panacea!
• A new package hits the repo.
• Are you in control of when it goes out?
• Memcached Outage
Wednesday, June 27, 12
Remember, No Panacea!
• A new package hits the repo.
• Are you in control of when it goes out?
• Memcached Outage
• Do you know what services are going to restart and when?
Wednesday, June 27, 12
Remember, No Panacea!
• A new package hits the repo.
• Are you in control of when it goes out?
• Memcached Outage
• Do you know what services are going to restart and when?
• Image Service Outage
Wednesday, June 27, 12
Standards
Wednesday, June 27, 12
Standards
• Not going to talk about testing [8]!
Wednesday, June 27, 12
Standards
• Not going to talk about testing [8]!
• But I don’t have time for standards!
Wednesday, June 27, 12
Standards - No Time!
Wednesday, June 27, 12
Standards - No Time!
• I won’t say “Make Time”, but you should...
Wednesday, June 27, 12
Standards - No Time!
• I won’t say “Make Time”, but you should...
• For a quick win, try Foodcritic
Wednesday, June 27, 12
Standards - No Time!
• I won’t say “Make Time”, but you should...
• For a quick win, try Foodcritic
• Good out of the box rules
Wednesday, June 27, 12
Standards - No Time!
• I won’t say “Make Time”, but you should...
• For a quick win, try Foodcritic
• Good out of the box rules
• Jenkins integration in seconds
Wednesday, June 27, 12
Standards - No Time!
• I won’t say “Make Time”, but you should...
• For a quick win, try Foodcritic
• Good out of the box rules
• Jenkins integration in seconds
• Supports custom rules
Wednesday, June 27, 12
Standards - No Time!
• I won’t say “Make Time”, but you should...
• For a quick win, try Foodcritic
• Good out of the box rules
• Jenkins integration in seconds
• Supports custom rules
• Plays well with others
Wednesday, June 27, 12
Foodcritic
Wednesday, June 27, 12
Foodcritic•gem install foodcritic
Wednesday, June 27, 12
Foodcritic•gem install foodcritic
•foodcritic <cookbook_repo>
Wednesday, June 27, 12
Foodcritic•gem install foodcritic
•foodcritic <cookbook_repo>
•Simple Jenkins job:
Wednesday, June 27, 12
Foodcritic•gem install foodcritic
•foodcritic <cookbook_repo>
•Simple Jenkins job:
#!/usr/bin/env rvm-shell 1.9.3foodcritic -f correctness .
Wednesday, June 27, 12
Standards at Etsy
Wednesday, June 27, 12
Standards at Etsy• “style” not “correctness”[9]
Wednesday, June 27, 12
Standards at Etsy• “style” not “correctness”[9]• ETSY001 - Package or yum_package resource used with :upgrade action
Wednesday, June 27, 12
Standards at Etsy• “style” not “correctness”[9]• ETSY001 - Package or yum_package resource used with :upgrade action
• ETSY002 - Execute resource used to run git commands
Wednesday, June 27, 12
Standards at Etsy• “style” not “correctness”[9]• ETSY001 - Package or yum_package resource used with :upgrade action
• ETSY002 - Execute resource used to run git commands
• ETSY003 - Execute resource used to run curl or wget commands
Wednesday, June 27, 12
Standards at Etsy• “style” not “correctness”[9]• ETSY001 - Package or yum_package resource used with :upgrade action
• ETSY002 - Execute resource used to run git commands
• ETSY003 - Execute resource used to run curl or wget commands
• ETSY004 - Execute resource defined without conditional or action :nothing
Wednesday, June 27, 12
Standards at Etsy• “style” not “correctness”[9]• ETSY001 - Package or yum_package resource used with :upgrade action
• ETSY002 - Execute resource used to run git commands
• ETSY003 - Execute resource used to run curl or wget commands
• ETSY004 - Execute resource defined without conditional or action :nothing
• ETSY005 - Action :restart sent to a core service
Wednesday, June 27, 12
Standards at Etsy• “style” not “correctness”[9]• ETSY001 - Package or yum_package resource used with :upgrade action
• ETSY002 - Execute resource used to run git commands
• ETSY003 - Execute resource used to run curl or wget commands
• ETSY004 - Execute resource defined without conditional or action :nothing
• ETSY005 - Action :restart sent to a core service
• ETSY006 - Execute resource used to run chef-provided command
Wednesday, June 27, 12
Standards at Etsy• “style” not “correctness”[9]• ETSY001 - Package or yum_package resource used with :upgrade action
• ETSY002 - Execute resource used to run git commands
• ETSY003 - Execute resource used to run curl or wget commands
• ETSY004 - Execute resource defined without conditional or action :nothing
• ETSY005 - Action :restart sent to a core service
• ETSY006 - Execute resource used to run chef-provided command
• ETSY007 - Package or yum_package resource used to install core package without specific version number
Wednesday, June 27, 12
Standards at Etsy• “style” not “correctness”[9]• ETSY001 - Package or yum_package resource used with :upgrade action
• ETSY002 - Execute resource used to run git commands
• ETSY003 - Execute resource used to run curl or wget commands
• ETSY004 - Execute resource defined without conditional or action :nothing
• ETSY005 - Action :restart sent to a core service
• ETSY006 - Execute resource used to run chef-provided command
• ETSY007 - Package or yum_package resource used to install core package without specific version number
Wednesday, June 27, 12
Standards at Etsy
Wednesday, June 27, 12
Standards at Etsy
• ETSY001 - Written after Memcached Outage
Wednesday, June 27, 12
Standards at Etsy
• ETSY001 - Written after Memcached Outage
• Package or yum_package resource used with :upgrade action
Wednesday, June 27, 12
Standards at Etsy
• ETSY001 - Written after Memcached Outage
• Package or yum_package resource used with :upgrade action
• package "memcached" do action :upgradeend
Wednesday, June 27, 12
Standards at Etsy
Wednesday, June 27, 12
Standards at Etsy
• ETSY005 - Written after a total Image service outage
Wednesday, June 27, 12
Standards at Etsy
• ETSY005 - Written after a total Image service outage
• Action :restart sent to a core service
Wednesday, June 27, 12
Standards at Etsy
• ETSY005 - Written after a total Image service outage
• Action :restart sent to a core service• cookbook_file "/etc/httpd/conf.d/myvhost.conf" do
source "myvhost.conf" notifies :restart, resources(:service => "httpd")end
Wednesday, June 27, 12
Critical Approach and Experimentation
Wednesday, June 27, 12
CA&E
Wednesday, June 27, 12
CA&E
• Chef is by necessity generic
Wednesday, June 27, 12
CA&E
• Chef is by necessity generic
• ...so don’t take Opscode’s word for it.
Wednesday, June 27, 12
CA&E
• Chef is by necessity generic
• ...so don’t take Opscode’s word for it.
• If it doesn’t work well for you, change it!
Wednesday, June 27, 12
CA&E
• Chef is by necessity generic
• ...so don’t take Opscode’s word for it.
• If it doesn’t work well for you, change it!
• Case Study - Etsy Environments rollout
Wednesday, June 27, 12
Environments Rollout
Wednesday, June 27, 12
Environments Rollout
• We knew we needed them
Wednesday, June 27, 12
Environments Rollout
• We knew we needed them
• Simple enough, right?
Wednesday, June 27, 12
Environments Rollout
• We knew we needed them
• Simple enough, right?
• Let’s look at the workflow...
Wednesday, June 27, 12
Env: Standard Workflow
Wednesday, June 27, 12
Env: Standard Workflow
• knife cookbook show php
Wednesday, June 27, 12
Env: Standard Workflow
• knife cookbook show php
• Change version number in metadata.rb
Wednesday, June 27, 12
Env: Standard Workflow
• knife cookbook show php
• Change version number in metadata.rb
• Change version constraint in foo.json
Wednesday, June 27, 12
Env: Standard Workflow
• knife cookbook show php
• Change version number in metadata.rb
• Change version constraint in foo.json
• Commit and push changes to git
Wednesday, June 27, 12
Env: Standard Workflow
• knife cookbook show php
• Change version number in metadata.rb
• Change version constraint in foo.json
• Commit and push changes to git
• knife cookbook upload php --freeze
Wednesday, June 27, 12
Env: Standard Workflow
• knife cookbook show php
• Change version number in metadata.rb
• Change version constraint in foo.json
• Commit and push changes to git
• knife cookbook upload php --freeze
• knife environment from file foo.json
Wednesday, June 27, 12
Env: Our Issues
Wednesday, June 27, 12
Env: Our Issues
• 41 people with Chef repo access
Wednesday, June 27, 12
Env: Our Issues
• 41 people with Chef repo access
• Most with knife keys
Wednesday, June 27, 12
Env: Our Issues
• 41 people with Chef repo access
• Most with knife keys
• All editing the same 2 files with every change...
Wednesday, June 27, 12
Env: Solutions?
Wednesday, June 27, 12
Env: Solutions?
• Go with it and hope for the best?
Wednesday, June 27, 12
Env: Solutions?
• Go with it and hope for the best?
• Don’t use environments?
Wednesday, June 27, 12
Env: Solutions?
• Go with it and hope for the best?
• Don’t use environments?
• Write a totally new workflow?
Wednesday, June 27, 12
Env: Solutions?
• Go with it and hope for the best?
• Don’t use environments?
• Write a totally new workflow?
• Tweak the existing one with some tooling?
Wednesday, June 27, 12
Env: Solution!
Wednesday, June 27, 12
Env: Solution!
• Tweak with some tooling!
Wednesday, June 27, 12
Env: Solution!
• Tweak with some tooling!
• Presenting knife-spork [9]...
Wednesday, June 27, 12
Spork: Workflow
Wednesday, June 27, 12
Spork: Workflow
• Wrapper around standard environments workflow
Wednesday, June 27, 12
Spork: Workflow
• Wrapper around standard environments workflow
• check - cookbook versioning
Wednesday, June 27, 12
Spork: Workflow
• Wrapper around standard environments workflow
• check - cookbook versioning
• bump - increment version component
Wednesday, June 27, 12
Spork: Workflow
• Wrapper around standard environments workflow
• check - cookbook versioning
• bump - increment version component
• upload - upload and freeze
Wednesday, June 27, 12
Spork: Workflow
• Wrapper around standard environments workflow
• check - cookbook versioning
• bump - increment version component
• upload - upload and freeze
• promote - set env constraints
Wednesday, June 27, 12
Spork: Check$> knife spork check apache2
Checking versions for cookbook apache2...
Current local version: 1.0.6
Remote versions (Max. 5 most recent only):*1.0.6, frozen1.0.5, frozen<snip>
DANGER: Your local cookbook has same version number as the starred version above!
Please bump your local version or you won't be able to upload.
Wednesday, June 27, 12
Spork: Bump
$> knife spork bump apache2 <major|minor|patch|manual>
Bumping patch level of the apache2 cookbook from 1.0.6 to 1.0.7
Wednesday, June 27, 12
Spork: Upload
$> knife spork upload apache2
Uploading and freezing apache2 [1.0.7]upload complete
Wednesday, June 27, 12
Spork: Promote$> knife spork promote foo php
Adding version constraint php = 1.0.6
Saving changes into foo.json
Promotion complete! Please remember to upload your changed Environment file to the Chef Server.
---
$> knife spork promote foo php --remoteAdding version constraint php = 0.1.0
Saving changes into foo.json
Uploading foo to server
Wednesday, June 27, 12
Spork
Wednesday, June 27, 12
Spork
• Worked well, avoided the issues it was designed for
Wednesday, June 27, 12
Spork
• Worked well, avoided the issues it was designed for
• Subsequently evolved
Wednesday, June 27, 12
Spork
• Worked well, avoided the issues it was designed for
• Subsequently evolved
• A lot of input & work came from Devs
Wednesday, June 27, 12
Spork
• Worked well, avoided the issues it was designed for
• Subsequently evolved
• A lot of input & work came from Devs
• Organic evolution
Wednesday, June 27, 12
Spork
• Worked well, avoided the issues it was designed for
• Subsequently evolved
• A lot of input & work came from Devs
• Organic evolution
• Open sourced, of course
Wednesday, June 27, 12
Spork: Evolution
Wednesday, June 27, 12
Spork: Evolution
• Safety Checks
Wednesday, June 27, 12
Spork: Evolution
• Safety Checks
• Extra Features
Wednesday, June 27, 12
Spork: Safety Checks
Wednesday, June 27, 12
Spork: Safety Checks
• Before promoting, check version is uploaded...
Wednesday, June 27, 12
Spork: Safety Checks
$> knife spork promote php --remote
<snip>
WARNING: It looks like you have multiple cookbook paths defined so I can't tell if you're running inside a git repo.
Checking that php version 0.1.93 exists on the server before promoting (any error means it hasn't been uploaded yet)...
ERROR: The object you are looking for could not be found
Response: Cannot find a cookbook named php with version 0.1.93
Wednesday, June 27, 12
Spork: Safety Checks
• Before promoting, check version is uploaded...
• Check if you’re promoting changes to more than you thought....
Wednesday, June 27, 12
Spork: Safety ChecksWARNING: You're about to promote changes to several cookbooks:
WARNING: ganglia: = 0.1.26 changed to = 0.1.25installerz: = 0.1.66 changed to = 0.1.65php: = 0.1.92 changed to = 0.1.93
Are you sure you want to continue? (Y/N) N
You said no, so I'm done here.
Would you like to reset your local development.json to match the server?? (Y/N) Y
<snip>
development.json reset.
Wednesday, June 27, 12
Spork: Features
Wednesday, June 27, 12
Spork: Features• Default Environments
Wednesday, June 27, 12
Spork: Features• Default Environments
• Git support
Wednesday, June 27, 12
Spork: Features• Default Environments
• Git support
• Chat notifications + Gist (upload and promote)
Wednesday, June 27, 12
Spork: Features• Default Environments
• Git support
• Chat notifications + Gist (upload and promote)
• IRCCat
Wednesday, June 27, 12
Spork: Features• Default Environments
• Git support
• Chat notifications + Gist (upload and promote)
• IRCCat
• Hipchat (courtesy of Secondmarket)
Wednesday, June 27, 12
Features: Chat Notifications
[19:43:46] <irccat> CHEF: pmcdonnell uploaded and froze cookbook immount version 0.0.24
[19:44:00] <irccat> CHEF: pmcdonnell uploaded environment production https://github.etsycorp.com/gist/385043
[19:44:01] <irccat> CHEF: pmcdonnell uploaded environment development https://github.etsycorp.com/gist/385044
Wednesday, June 27, 12
Spork: Features• Default Environments
• Git support
• Chat notifications + Gist (upload and promote)
• IRCCat
• Hipchat (courtesy of Secondmarket)
• Graphite (on promote)
Wednesday, June 27, 12
Features: Graphite
Wednesday, June 27, 12
Spork: Features• Default Environments
• Git support
• Chat notifications + Gist (upload and promote)
• IRCCat
• Hipchat (courtesy of Secondmarket)
• Graphite (on promote)
• Foodcritic (on upload)Wednesday, June 27, 12
Features: Foodcritic
$> knife spork upload system
<snip>
Lint checking system...
ERROR: Lint check failed. Halting upload.
ERROR: Lint check output:
ERROR: ETSY003: Execute resource used to run curl or wget commands: /Users/jcowie/dev/etsy/chef/cookbooks/system/recipes/dev-ssl.rb:41
Wednesday, June 27, 12
Use the Source!
tiny.cc/velocity2012
Wednesday, June 27, 12
We’re hiring!
BoF on Tuesdayor just come and say Hi :)
Wednesday, June 27, 12