buildout for the future
DESCRIPTION
Buildout is a great tool for creating repeatable environments. But what happens when things change and you are left with a broken buildout? This talk will focus on creating a buildout that can survive the test of time. The talk will cover how to keep your packages pinned so that releases don't cause you headaches. We will also discuss how to set up your own package server in order to make sure that you always have the packages that you need.TRANSCRIPT
Clayton Parker, Senior Web Developer
Buildout for the FuturePLONE CONFERENCE 2010
Thursday, October 28, 2010
PLONE CONFERENCE 2010Who Am I?
• claytron
• Python dev since 2003
• Plone Core Committer
• Foundation Member
Thursday, October 28, 2010
PLONE CONFERENCE 2010What Will We Learn?• Pinning
• Indexes
• Buildout Tricks
Thursday, October 28, 2010
PLONE CONFERENCE 2010Problems with Buildout• PyPi
• Lost/deleted packages
• Version conflicts
Thursday, October 28, 2010
PLONE CONFERENCE 2010Pinning Versions
• [versions]
• Local config
• Extended URLs
Thursday, October 28, 2010
PLONE CONFERENCE 2010Versions
[buildout]# tell buildout to use the [versions] partversions = versions
[versions]my.package = 1.0some.other.package = 2.0
Thursday, October 28, 2010
PLONE CONFERENCE 2010Plone Versions
[buildout]extends = http://dist.plone.org/release/4.0.1/versions.cfg
Thursday, October 28, 2010
PLONE CONFERENCE 2010Plone versions.cfg[buildout]extends = http://download.zope.org/Zope2/index/2.12.11/versions.cfg
[versions]# Buildout infrastructureplone.recipe.zope2instance = 4.0.4plone.recipe.zeoserver = 1.1.1...
# External dependenciesMarkdown = 2.0.3PIL = 1.1.6...
# Plone releasePlone= 4.0.1...
Thursday, October 28, 2010
PLONE CONFERENCE 2010Zope versions.cfg
[buildout]versions = versions
[versions]Zope2 = 2.12.11...
Thursday, October 28, 2010
PLONE CONFERENCE 2010Our versions.cfg
## buildout.cfg[buildout]extends = http://dist.plone.org/release/4.0.1/versions.cfg versions.cfg
## versions.cfg[versions]my.package = 1.0some.other.package = 2.0
Thursday, October 28, 2010
PLONE CONFERENCE 2010Getting Versions$ bin/buildout -v...Installing instanceWe have the distribution that satisfies 'Plone==4.0.1'.We have the distribution that satisfies 'plone.app.caching==1.0b1'.We have the distribution that satisfies 'plone.app.ldap==1.2.3'.We have the distribution that satisfies 'plone.reload==1.5'.We have the distribution that satisfies 'zope.testrecorder==0.4'.We have the distribution that satisfies 'Products.PDBDebugMode==1.1'.We have the distribution that satisfies 'Products.PrintingMailHost==0.7'.We have the distribution that satisfies 'Products.DocFinderTab==1.0.4'.We have the distribution that satisfies 'Products.signalstack==1.0rc2'.We have the distribution that satisfies 'Products.PTProfiler==1.4'.We have the distribution that satisfies 'Products.DCWorkflowGraph==0.4'.We have the distribution that satisfies 'ipdb==0.2'.We have the distribution that satisfies 'plone.recipe.zope2instance==4.0.4'.Getting required 'Zope2==2.12.11'We have the distribution that satisfies 'Zope2==2.12.11'.
Thursday, October 28, 2010
PLONE CONFERENCE 2010Dump Picked Versions
[buildout]extensions = buildout.dumppickedversions
Thursday, October 28, 2010
PLONE CONFERENCE 2010Dump Picked Versions
$ bin/buildout -v...*************** PICKED VERSIONS ****************[versions]PILwoTk = 1.1.6.4Products.DCWorkflowGraph = 0.4Products.DocFinderTab = 1.0.4Products.PDBDebugMode = 1.1Products.PTProfiler = 1.4Products.PrintingMailHost = 0.7Products.signalstack = 1.0rc2ipdb = 0.2plone.app.caching = 1.0b1plone.app.ldap = 1.2.3zope.testrecorder = 0.4
*************** /PICKED VERSIONS ***************
Thursday, October 28, 2010
PLONE CONFERENCE 2010Extended URLs
[buildout]extends =# Plone's set of version dependencies http://dist.plone.org/release/4.0.1/versions.cfg# specific versions for plone.app.caching package http://good-py.appspot.com/release/plone.app.caching/1.0b1# our version overrides and add-on packages versions.cfg
Thursday, October 28, 2010
PLONE CONFERENCE 2010Public Indexes
• PyPi
• Mirrors
• Plone dist
• http://dist.plone.org/
• Company dist
• http://dist.company.com/public/
Thursday, October 28, 2010
PLONE CONFERENCE 2010Find Links
[buildout]find-links = http://dist.plone.org/release/4.0.1/
[buildout]plone-version = 4.0.1extends = http://dist.plone.org/release/${:plone-version}/versions.cfgfind-links = http://dist.plone.org/release/${:plone-version}/
Thursday, October 28, 2010
PLONE CONFERENCE 2010Private Indexes
• Customer eggs
• Customer archives
• Not ready for pypi
• Branches / trunk
Thursday, October 28, 2010
PLONE CONFERENCE 2010Solutions
• Apache / nginx
• Plone Software Center
• basketweaver
• haufe.eggserver
• ClueReleaseManager
Thursday, October 28, 2010
PLONE CONFERENCE 2010Apache
• Use directory listings
• Basic auth for protection
• mod_ldap for integration
Thursday, October 28, 2010
PLONE CONFERENCE 2010Config
<Location /private/myproject> AuthType Basic AuthName "myproject" # hook up to ldap or a password file # ...</Location>
Thursday, October 28, 2010
PLONE CONFERENCE 2010lovely.buildouthttp
• Access private egg repos
• Also tied into download recipes
myproject,https://dist.example.com/private/myproject,username,password
Thursday, October 28, 2010
PLONE CONFERENCE 2010Releasing Eggs
• jarn.mkrelease
• zest.releaser
Thursday, October 28, 2010
PLONE CONFERENCE 2010~/.pypirc[distutils]index-servers = pypi plone.org
[pypi]username: usernamepassword: password
[plone.org]repository: http://plone.org/productsusername: usernamepassword: password
Thursday, October 28, 2010
PLONE CONFERENCE 2010~/.mkrelease[defaults]distbase = distdefault = public
[aliases]public = dist.company.com:/var/dist/publicmyproject = dist.company.com:/var/dist/private/myprojectworld = public pypiplone-world = world plone.org
Thursday, October 28, 2010
PLONE CONFERENCE 2010mkrelease
$ mkrelease -d myproject
$ mkrelease -d public
$ mkrelease -d plone-world
Thursday, October 28, 2010
PLONE CONFERENCE 2010Buildout Profiles
• Debugging / local dev
• Production
Thursday, October 28, 2010
PLONE CONFERENCE 2010Profiles
my-buildout "## buildout.cfg "## parts "## profiles $ "## base.cfg $ "## debug.cfg $ "## local.cfg $ "## prod.cfg $ &## versions.cfg &## var
Thursday, October 28, 2010
PLONE CONFERENCE 2010Annotate
• Check config
• Debug issues
• Verify settings
Thursday, October 28, 2010
PLONE CONFERENCE 2010Annotate
$ bin/buildout annotate
$ bin/buildout -c profiles/prod.cfg annotate
Thursday, October 28, 2010
Check out
sixfeetup.com/demos
Thursday, October 28, 2010