from zero to hero @ pygrunn 2014
DESCRIPTION
Deploying and maintaining (Django) applications professionally using puppet, git and scripting.TRANSCRIPT
From Zero to Hero
Job GanzevoortDouwe van der Meij
Goldmund, Wyldebeast & Wunderliebe
{ ganzevoort, vandermeij } @gw20e.com
Outline
● Introduction● Initial Setup● Deployment● Maintenance● Deployment continued● Conclusion
Introduction
● People have great ideas● Django is perfect for RAD
○ But...● How to deploy to production?● How to keep the system maintainable?
○ Or even…● When is my application production ready?
Why this presentation?
Initial Setup
● Before you do anything related to code:○ git init
● Make sure you track everything you do in a VCS
Version Control System
● In the first stage of development, program only the parts that are the core of your application
● But how to do this in git?
Define Minimum Viable Product
● Have base/production code in master branch
● git branch for every code change● Merge all branches with acceptance
branch before going live (again)
Use branches (1/2)
Use branches (2/2)
master
acceptance
time
bi-weekl
y
release
change #123bran
chmerg
emerge
Deployment
● VPS● Provision with e.g., Puppet● Find or create scripts● Put the scripts in version control!
Server(s)
node 'kiezel.gw20e.com' {
class { 'ssh':
server_options => {
'PasswordAuthentication' => 'no',
'PermitRootLogin' => 'no',
},
}
appie::app { "mysite":
envs => ["tst", "acc", "prd"],
secret => "secret",
accountinfo => $gw20e::user_accounts,
accounts => ['ganzevoort', 'vandermeij'],
}
}
● Create a user per environment-layer combination○ app-mysite-tst○ app-mysite-acc○ app-mysite-prd
● With their own home dir, postgresql DB, nginx glue
● Check out (and fork) our puppet module:○ puppet-appie @ GitHub
Separate users
● Use fabric to script deploying to your servers:○ fab deploy:layer=tst
● Put the scripts in version control!
● Check out (and fork) our template:○ templateproject @ GitHub○ It includes gunicorn and supervisor configurations
Use scripts to deploy
● Don’t just change Django’s settings.py● Keep dev, tst, acc and prd specific
settings in separate files● Put the settings in version control!
Different DTAP layers (1/6)
● Deploy to their respective environment
Different DTAP layers (2/6)
DEV● Laboratory setup● Switch to the branch you’re working on● Work with dev settings● Deploy to TST, ACC or PRD environment
Different DTAP layers (3/6)
TST● Real deployment, but alpha● Work with tst settings
Different DTAP layers (4/6)
ACC● Real deployment, but beta● Work with acc settings
Different DTAP layers (5/6)
PRD● Real deployment!● Work with prd settings● Setup monitoring
○ Nagios○ Sentry○ ...
● Setup backup○ Database○ Uploaded media
Different DTAP layers (6/6)
● Move settings.py as is to: settings/core.py
● Create: settings/base.py○ Add: from settings.core import *○ Add your own generic settings
● Create: settings/{dev,tst,acc,prd}.py○ Add: from settings.base import *○ Add your own layer specific settings
Django settings (1/4)
Django settings (2/4)
settings.py core.py
base.py
LAYER.py
settings/
● Use Fabric to create: settings/__init__.py
● This command:○ fab pick_settings:layer=prd
● Results in:○ from settings.prd import *
● Stackable settings scale○ Imagine (white) labeling
Django settings (3/4)
Django settings (4/4)
__init__.pysettings/
dev.py tst.py acc.py prd.py
base.py
core.py
Django settings (5/4)
Database passwords in version control?
from .base import *read_pgpass('app-mysite-prd')
Maintenance
● Scrum, RUP, etc.○ Any iterative methodology will do
● Basicly:○ Have periodic deadlines/releases
■ Bi-weekly○ Deliver each iteration
■ To ACC, if OK, to PRD○ Release early, release often (MVP)
Iterative development
● Create a ticket for every change (RFC)● Estimate the ticket● Have discussion in ticket thread● Create a code branch per ticket
○ Let git help you● Deploy to a separate TST environment
○ (with separate database)
(How to) use a ticket system
Branching revisited
master
acceptance
time
bi-weekl
y
release
change #123bran
chmerg
emerge
iteration
● On your test machine, let each ticket/branch have its own environment (with database)
● Test implementations individually, care for code merging later
● Multiple TST environments means multiple deployments○ So not only three (= TST, ACC, PRD)
Separate environments for TST (1/5)
● change_123/$ fab deploy
● Is shorthand for:○ fab deploy:layer=tst,branch=change_123
Separate environments for TST (2/5)
● Isn’t setting up separate webserver(s), database(s), etc. causing a lot of overhead?
Separate environments for TST (3/5)
● Computer says NO○ If you do it the right way
Separate environments for TST (4/5)
● Keep your (webserver) configuration files in version control○ Use e.g., Django templates to construct environment
specific files● Keep your database scripts in version
control○ To copy a new DB from a production fixture○ To install a new DB from scratch with fixtures○ Or both!
Separate environments for TST (5/5)
Deployment continued
● Have separate deployment settings per layer:○ use_https○ path_to_certificate○ sitename○ deploy_user_at_host○ dir_to_deploy_to○ ...
Deployment settings (1/4)
Deployment settings (2/4)
LAYER.pydeployment/
● Use the Django template engine to construct configuration files on deployment:
{% if use_https %}
server {
listen 80;
server_name {{ sitename }};
rewrite ^(.*) https://$server_name$request_uri?;
}
...
Deployment settings (3/4)
● Symlink the generated configuration files from the user homedir into the place they’re needed
● E.g.○ ln -s ~/current/etc/nginx.conf
/etc/nginx/sites-enabled/SITENAME.conf○ service nginx reload
Deployment settings (4/4)
● Create a tag for each PRD release● Switch to that tag on deploy to PRD
○ For easy rollback■ Switch to previous tag to rollback
● Run migration scripts after switch○ Watch out for backwards incompatible changes!
Use Git tagging
● When using Pip○ pip freeze > frozen.txt
● Add to version control!
● pip install -r frozen.txt
Freeze dependencies
Conclusion
● Use a VCS● Have separate DTAP layers● Script and store● Work with RFCs
● Make everything repeatable○ By your colleague
Conclusion(s)
● github.com/○ Goldmund-Wyldebeast-Wunderliebe/
■ templateproject■ puppet-appie
References
Thank you!Job Ganzevoort,
Douwe van der Meij
Goldmund, Wyldebeast & Wunderliebe
{ ganzevoort, vandermeij } @gw20e.com