reaching deployment nirvana using buildout
DESCRIPTION
In the past, deploying a Zope/Plone application was a cumbersome task wrought with peril. Buildout has ushered in a new era of repeatability and ease of use. Spending hours to get your co-workers set up with just the right set of products and python libraries has come to an end. Six Feet Up's Lead Developer Clayton Parker will show you how to use buildout from your local development instance to the full blown production server instance with one buildout.TRANSCRIPT
open sourcenowhere to go but
sixfeetup.com/dw08
ReachingDeployment Nirvana
Using BuildoutClayton Parker
Plone Conference 2008
sixfeetup.com/dw08
Here’s to Plone
sixfeetup.com/dw08
• Buildout
• Parts
• Recipes
• Command Line
• ZopeSkel
• Create Buildouts
• Custom Recipes
To-do List
sixfeetup.com/dw08
Why buildout?
sixfeetup.com/dw08
Eliminate Confusion
sixfeetup.com/dw08
Tracking Dependencies
sixfeetup.com/dw08
Roll Your Own
• Installer
• ZopeSkel
sixfeetup.com/dw08
ZopeSkel
$ easy_install ZopeSkel$ paster create --list-templatesplone3_buildout: A buildout for Plone 3 projects$ paster create -t plone3_buildout mybuildout
sixfeetup.com/dw08
Lingo
sixfeetup.com/dw08
Syntax[buildout]parts = zope2 plone instance
[plone]recipe = plone.recipe.plone
[zope2]recipe = plone.recipe.zope2install
[instance]recipe = plone.recipe.zope2instanceuser = admin:admineggs = ${plone:eggs}products = ${plone:products}
sixfeetup.com/dw08
Advanced Syntax
${part:option}
${plone:eggs}${buildout:directory}option = ${buildout:directory}/somefolder
Variable Substitution
options = foo bar
options += bazoptions -= foo
Option Addition and Removal
sixfeetup.com/dw08
Reserved Characters
: $ % { }
sixfeetup.com/dw08
Parts
sixfeetup.com/dw08
[buildout]parts = instance
[instance]recipe = plone.recipe.zope2instanceuser = admin:admineggs = ${plone:eggs} archetypes.schemaextenderzcml = archetypes.schemaextenderproducts = ${plone:products}environment-vars = TZ America/Chicagohttp-address = 51060zeo-address = 53060debug-mode = onverbose-security = onevent-log-level = debug
sixfeetup.com/dw08
Recipes
sixfeetup.com/dw08
[buildout]parts = zope2 plone instance
[plone]recipe = plone.recipe.plone
[zope2]recipe = plone.recipe.zope2install
[instance]recipe = plone.recipe.zope2instanceuser = admin:admineggs = ${plone:eggs}products = ${plone:products}
sixfeetup.com/dw08
Buildout Directory• bin/
• bootstrap.py
• buildout.cfg
• develop-eggs/
• downloads/
• eggs/
• parts/
• products/
• src/
sixfeetup.com/dw08
Buildout options• bin-directory
• develop-eggs-directory
• eggs-directory
• parts
• parts-directory
• log-level
sixfeetup.com/dw08
Defaults
[buildout]eggs-directory = /home/clayton/.buildout/eggsdownload-cache = /home/clayton/.buildout/downloadszope-directory = /home/clayton/.buildout/zopeindex = http://download.zope.org/ppix
[instance]event-log-level = debug
sixfeetup.com/dw08
Recipes
sixfeetup.com/dw08
Plone Recipes
• plone.recipe.plone
• plone.recipe.zope2install
• plone.recipe.zope2instance
• plone.recipe.squid
sixfeetup.com/dw08
plone.recipe.zope2install
[zope2]recipe = plone.recipe.zope2installurl = ${plone:zope2-url}fake-zope-eggs = trueadditional-fake-eggs = ZODB3skip-fake-eggs = zope.testing zope.component zope.i18n
sixfeetup.com/dw08
In The Wild
sixfeetup.com/dw08
Extending Configuration• buildout.cfg
• profiles/
• base.cfg
• development.cfg
• debug.cfg
• qa.cfg
• prod.cfg
• versions.cfg
sixfeetup.com/dw08
buildout.cfg
[buildout]# the profile we want to useextends = profiles/development.cfg
find-links = http://dist.plone.org http://download.zope.org/distribution/ http://effbot.org/downloads
sixfeetup.com/dw08
base.cfg[buildout]parts = PILwoTK plone zope2 instance zeoserverdevelop = src/my.package
[PILwoTK]recipe = zc.recipe.eggfind-links = http://download.zope.org/distribution/
[zope2]recipe = plone.recipe.zope2installurl = ${plone:zope2-url}fake-zope-eggs = trueadditional-fake-eggs = ZODB3skip-fake-eggs = zope.testing zope.component zope.i18n
sixfeetup.com/dw08
[plone]recipe = plone.recipe.plone
[instance]recipe = plone.recipe.zope2instancezeo-client = Truezope2-location = ${zope2:location}eggs = PILwoTK elementtree ${plone:eggs} my.packagezcml = my.packageproducts = ${plone:products}environment-vars = TZ America/New_York
[zeoserver]recipe = plone.recipe.zope2zeoserverzope2-location = ${instance:zope2-location}zeo-address = ${instance:zeo-address}
sixfeetup.com/dw08
development.cfg[buildout]extends = base.cfg debug.cfgparts += ${debugging:parts} omelette
[instance]user = admin:adminhttp-address = 8080zeo-address = 8100debug-mode = onverbose-security = onevent-log-level = debugproducts += ${buildout:directory}/products ${debugging:debug-products}eggs += ${debugging:eggs}zcml += ${debugging:zcml}
[omelette]recipe = collective.recipe.omeletteeggs = ${instance:eggs}ignore-develop = Trueignores = setuptools
sixfeetup.com/dw08
debug.cfg[debugging]parts = debug-products debug-products-svn ipzope zopepydebug-products = ${debug-products:location}eggs = plone.reload Products.PDBDebugMode Products.DocFinderTab Products.Clouseau Products.PrintingMailHostzcml = plone.reload
sixfeetup.com/dw08
[debug-products]recipe = plone.recipe.distrosurls = ...dcworkflowgraph-0_3.tgz ...PTProfiler-1.2.tgz
[ipzope]recipe = zc.recipe.eggeggs = ipython ${instance:eggs}initialization = import sys, os os.environ["SOFTWARE_HOME"] = "${instance:zope2-location}/lib/python" os.environ["INSTANCE_HOME"] = "${instance:location}" sys.argv[1:1] = "-p zope".split()extra-paths = ${instance:zope2-location}/lib/pythonscripts = ipython=ipzope
[zopepy]recipe = zc.recipe.eggeggs = ${instance:eggs}interpreter = zopepyextra-paths = ${instance:zope2-location}/lib/pythonscripts = zopepy
sixfeetup.com/dw08
versions.cfg[versions]# Use the following from the command line to get the latest versions:# bin/buildout -vvvvv |sed -ne 's/^Picked: //p' | sort | uniqPILwoTk = 1.1.6.4elementtree = 1.2.7-20070827-previewinfrae.subversion = 1.1plone.recipe.distros = 1.3plone.recipe.plone = 3.1.6plone.recipe.zope2install = 2.3plone.recipe.zope2instance = 2.5plone.recipe.zope2zeoserver = 0.13python-openid = 2.2.1my.package = 1.0
sixfeetup.com/dw08
prod.cfg[buildout]extends = base.cfg versions.cfgparts += instance2versions = versions
[instance]zope2-location = ${zope2:location}http-address = 9080zeo-address = 9100eggs += Products.CacheSetupz2-log-level = CRITICALzodb-cache-size = 15000
[instance2]recipe = collective.recipe.zope2clusterinstance-clone = instancehttp-address = 10080
sixfeetup.com/dw08
Command Line
sixfeetup.com/dw08
Baby Steps$ cd path/to/mybuildout
$ python2.4 bootstrap.pyCreating directory 'mybuildout/bin'.Creating directory 'mybuildout/parts'.Creating directory 'mybuildout/develop-eggs'.Generated script 'mybuildout/bin/buildout'.
$ bin/buildout
$ bin/instance start
sixfeetup.com/dw08
Options• -v and -q
• increase and decrease verbosity
• -n and -N
• Newest and non-newest modes
• -O and -o
• online and offline mode
• -t
• socket timeout
sixfeetup.com/dw08
Update your buildout
$ bin/buildout -v
$ bin/buildout -Nvvv
$ bin/buildout -No
$ bin/buildout -t 60
sixfeetup.com/dw08
Assignments
$ bin/buildout instance:debug-mode=on
$ bin/buildout buildout:log-level=70
$ bin/buildout -N instance:debug-mode=on -v
sixfeetup.com/dw08
Commands
$ bin/buildout install
$ bin/buildout -Nv install zope2 instance
$ bin/buildout -nv install instance
sixfeetup.com/dw08
Create Recipes
$ paster create -t recipe my.recipe.example
sixfeetup.com/dw08
Recipe
• Recipe class
• constructor
• install
• update
• uninstall (optional)
sixfeetup.com/dw08
class Recipe: """"A recipe """" def __init__(self, buildout, name, options): self.buildout = buildout self.name = name self.options = options # gather options from other parts here options['notmine'] = buildout['someotherpart']['foobar'] def install(self): """"Install method """ options = self.options location = options['location'] # must return a string, or an iterable of strings return location def update(self): """Update method """ pass
sixfeetup.com/dw08
Real Worldclass Recipe: """infrae.subversion recipe. """
def __init__(self, buildout, name, options): self.buildout = buildout self.name = name self.options = options options['location'] = self.location = os.path.join( buildout['buildout']['parts-directory'], self.name) self.urls = [l.split() for l in options['urls'].splitlines() if l.strip()] self.export = options.get('export') self.newest = ( buildout['buildout'].get('offline', 'false') == 'false' and buildout['buildout'].get('newest', 'true') == 'true' ) self.verbose = buildout['buildout'].get('verbosity', 0)
sixfeetup.com/dw08
Install Method
def install(self): """Checkout the checkouts. """ for (url, name) in self.urls: wc = py.path.svnwc(self.location).join(name) if self.export: raise Exception('Unimplemented feature') if self.verbose: print "Fetch %s" % url wc.checkout(url) return self.location
sixfeetup.com/dw08
def update(self): """Update the checkouts""" if not self.newest: return self.location if self.export: return self.location if self.options.get('ignore_updates', False): return self.location num_release = re.compile('.*@[0-9]+$') part = py.path.local(self.location) for link, sub_path in self.urls: if num_release.match(link): if self.verbose: print "Given num release for %s, skipping." % link continue wc = py.path.svnwc(self.location).join(sub_path) if self.verbose: print "Updating %s" % link wc.update() return self.location
sixfeetup.com/dw08
Wrapping Up• Buildout
• Parts
• Recipes
• Command Line
• ZopeSkel
• Create Buildouts
• Custom Recipes
Plone Deployment Workshop
Register by October 17and save $100!
sixfeetup.com/dw08
sixfeetup.com/dw08
Links
• http://buildout.zope.org
• http://pypi.python.org/pypi/zc.buildout
• https://svn.sixfeetup.com/svn/public/buildout/debug.cfg
• http://www.sixfeetup.com/swag/buildout-quick-reference-card
• http://plone.org/documentation/tutorial/buildout
• http://pypi.python.org
sixfeetup.com/dw08
Photo Credits• http://flickr.com/photos/monsieurlam/2645956083/
• http://flickr.com/photos/_boris/2796908072/
• http://flickr.com/photos/b-tal/163450213/
• http://flickr.com/photos/bullish1974/2648544508/
• http://flickr.com/photos/haydnseek/87432002/
• http://flickr.com/photos/disowned/1158260369/
• http://flickr.com/photos/7603557@N08/2662531345/
• http://flickr.com/photos/julishannon/2151986631/
• http://flickr.com/photos/julishannon/2152778524/
• http://flickr.com/photos/lollyknit/1155225799/
• http://flickr.com/photos/binary_koala/86227485/