practical project automation (pygrunn conference)
TRANSCRIPT
Practical project automation
donderdag 19 mei 2011
Practical project automation
• Reinout van Rees
• Nelen & Schuurmans
• Utrecht
• Water consultancy
• Lots of python and django
donderdag 19 mei 2011
Project: more than code
• Projects are important
• Deployment
• Colleagues
• Rebuilding
• Solution: automation
donderdag 19 mei 2011
Generic idea: automation
• Cobbler’s children
• Automate your projects
• Less manual work
• Less menial work
• Fewer errors
• More of the fun work
donderdag 19 mei 2011
Build automation
• ./configure;make
• Ant, maven
• Buildout (python)
• Single command to get started
donderdag 19 mei 2011
Build tool: good spot for more automation
• Most are extensible
• Examples:
• Apache config file generation
• Setting up cronjobs
donderdag 19 mei 2011
Apache config file:[apacheconf]recipe = collective.recipe.templateinput = ${buildout:directory}/etc/mysite.apache.conf.inoutput = ${buildout:directory}/etc/mysite.apache.conf
<VirtualHost *:80> ServerName mysite.example.org CustomLog ${buildout:directory}/var/log/access.log combined ErrorLog ${buildout:directory}/var/log/error.log DocumentRoot ${buildout:directory}/var/www ... RewriteRule ^(.*) http://localhost:${django:gunicorn_port}$1 [P]
<VirtualHost *:80> ServerName mysite.example.org CustomLog /srv/mysite/var/log/access.log combined ErrorLog /srv/mysite/var/log/error.log DocumentRoot /srv/mysite/var/www ... RewriteRule ^(.*) http://localhost:10003$1 [P]
donderdag 19 mei 2011
Prevent missing apache log dir (automatically!)
[mkdir]recipe = z3c.recipe.mkdirpaths = ${buildout:directory}/generated ${buildout:directory}/var/geotiffs ${buildout:directory}/var/log ${buildout:directory}/var/media ${buildout:directory}/var/sqlite ${buildout:directory}/var/static
donderdag 19 mei 2011
Automatic crontab
[supervisor-cronjob]recipe = z3c.recipe.usercrontabtimes = @rebootcommand = ${buildout:bin-directory}/supervisord
[whoosh-cronjob]recipe = z3c.recipe.usercrontabtimes = 13 */8 * * *command = ${buildout:bin-directory}/django update_index
donderdag 19 mei 2011
Test automation
• Jenkins/buildbot
• Deployment ok...
• ... so tests possible
• Prevent mistakes
donderdag 19 mei 2011
donderdag 19 mei 2011
Extra jenkins automation: quality checks
• Code coverage (‘coverage’ on pypi)
• Pyflakes and pep8 (also on pypi)
• Jslint (or jshint)
donderdag 19 mei 2011
donderdag 19 mei 2011
Quality: make it easy to do the right thing
• “I really should do x, but it takes too much time so I’ll skip it”
• Releasing packages
• Skeleton
donderdag 19 mei 2011
donderdag 19 mei 2011
Releasing python packages
• zest.releaser (on pypi)
• prerelease
• release
• postrelease
donderdag 19 mei 2011
donderdag 19 mei 2011
Practical project automation
• Make it easier to do the right thing: let laziness work for you
• Automate your builds
• Automate tests
• Extend where practical
donderdag 19 mei 2011
Questions?
reinout.vanrees.org
@reinoutvanrees
www.nelen-schuurmans.nl
photos.reinout.vanrees.org
donderdag 19 mei 2011