from zero to hero @ pygrunn 2014

From Zero to Hero

Job GanzevoortDouwe van der Meij

Goldmund, Wyldebeast & Wunderliebe

{ ganzevoort, vandermeij } @gw20e.com

Outline

● Introduction● Initial Setup● Deployment● Maintenance● Deployment continued● Conclusion

Introduction

● People have great ideas● Django is perfect for RAD

○ But...● How to deploy to production?● How to keep the system maintainable?

○ Or even…● When is my application production ready?

Why this presentation?

Initial Setup

● Before you do anything related to code:○ git init

● Make sure you track everything you do in a VCS

Version Control System

● In the first stage of development, program only the parts that are the core of your application

● But how to do this in git?

Define Minimum Viable Product

● Have base/production code in master branch

● git branch for every code change● Merge all branches with acceptance

branch before going live (again)

Use branches (1/2)

Use branches (2/2)

master

acceptance

time

bi-weekl

y

release

change #123bran

chmerg

emerge

Deployment

● VPS● Provision with e.g., Puppet● Find or create scripts● Put the scripts in version control!

Server(s)

node 'kiezel.gw20e.com' {

class { 'ssh':

server_options => {

'PasswordAuthentication' => 'no',

'PermitRootLogin' => 'no',

},

}

appie::app { "mysite":

envs => ["tst", "acc", "prd"],

secret => "secret",

accountinfo => $gw20e::user_accounts,

accounts => ['ganzevoort', 'vandermeij'],

}

}

● Create a user per environment-layer combination○ app-mysite-tst○ app-mysite-acc○ app-mysite-prd

● With their own home dir, postgresql DB, nginx glue

● Check out (and fork) our puppet module:○ puppet-appie @ GitHub

Separate users

● Use fabric to script deploying to your servers:○ fab deploy:layer=tst

● Put the scripts in version control!

● Check out (and fork) our template:○ templateproject @ GitHub○ It includes gunicorn and supervisor configurations

Use scripts to deploy

● Don’t just change Django’s settings.py● Keep dev, tst, acc and prd specific

settings in separate files● Put the settings in version control!

Different DTAP layers (1/6)

● Deploy to their respective environment

Different DTAP layers (2/6)

DEV● Laboratory setup● Switch to the branch you’re working on● Work with dev settings● Deploy to TST, ACC or PRD environment

Different DTAP layers (3/6)

TST● Real deployment, but alpha● Work with tst settings

Different DTAP layers (4/6)

ACC● Real deployment, but beta● Work with acc settings

Different DTAP layers (5/6)

PRD● Real deployment!● Work with prd settings● Setup monitoring

○ Nagios○ Sentry○ ...

● Setup backup○ Database○ Uploaded media

Different DTAP layers (6/6)

● Move settings.py as is to: settings/core.py

● Create: settings/base.py○ Add: from settings.core import *○ Add your own generic settings

● Create: settings/{dev,tst,acc,prd}.py○ Add: from settings.base import *○ Add your own layer specific settings

Django settings (1/4)

Django settings (2/4)

settings.py core.py

base.py

LAYER.py

settings/

● Use Fabric to create: settings/__init__.py

● This command:○ fab pick_settings:layer=prd

● Results in:○ from settings.prd import *

● Stackable settings scale○ Imagine (white) labeling

Django settings (3/4)

Django settings (4/4)

__init__.pysettings/

dev.py tst.py acc.py prd.py

base.py

core.py

Django settings (5/4)

Database passwords in version control?

from .base import *read_pgpass('app-mysite-prd')

Maintenance

● Scrum, RUP, etc.○ Any iterative methodology will do

● Basicly:○ Have periodic deadlines/releases

■ Bi-weekly○ Deliver each iteration

■ To ACC, if OK, to PRD○ Release early, release often (MVP)

Iterative development

● Create a ticket for every change (RFC)● Estimate the ticket● Have discussion in ticket thread● Create a code branch per ticket

○ Let git help you● Deploy to a separate TST environment

○ (with separate database)

(How to) use a ticket system

Branching revisited

master

acceptance

time

bi-weekl

y

release

change #123bran

chmerg

emerge

iteration

● On your test machine, let each ticket/branch have its own environment (with database)

● Test implementations individually, care for code merging later

● Multiple TST environments means multiple deployments○ So not only three (= TST, ACC, PRD)

Separate environments for TST (1/5)

● change_123/$ fab deploy

● Is shorthand for:○ fab deploy:layer=tst,branch=change_123

Separate environments for TST (2/5)

● Isn’t setting up separate webserver(s), database(s), etc. causing a lot of overhead?

Separate environments for TST (3/5)

● Computer says NO○ If you do it the right way

Separate environments for TST (4/5)

● Keep your (webserver) configuration files in version control○ Use e.g., Django templates to construct environment

specific files● Keep your database scripts in version

control○ To copy a new DB from a production fixture○ To install a new DB from scratch with fixtures○ Or both!

Separate environments for TST (5/5)

Deployment continued

● Have separate deployment settings per layer:○ use_https○ path_to_certificate○ sitename○ deploy_user_at_host○ dir_to_deploy_to○ ...

Deployment settings (1/4)

Deployment settings (2/4)

LAYER.pydeployment/

● Use the Django template engine to construct configuration files on deployment:

{% if use_https %}

server {

listen 80;

server_name {{ sitename }};

rewrite ^(.*) https://$server_name$request_uri?;

}

...

Deployment settings (3/4)

● Symlink the generated configuration files from the user homedir into the place they’re needed

● E.g.○ ln -s ~/current/etc/nginx.conf

/etc/nginx/sites-enabled/SITENAME.conf○ service nginx reload

Deployment settings (4/4)

● Create a tag for each PRD release● Switch to that tag on deploy to PRD

○ For easy rollback■ Switch to previous tag to rollback

● Run migration scripts after switch○ Watch out for backwards incompatible changes!

Use Git tagging

● When using Pip○ pip freeze > frozen.txt

● Add to version control!

● pip install -r frozen.txt

Freeze dependencies

Conclusion

● Use a VCS● Have separate DTAP layers● Script and store● Work with RFCs

● Make everything repeatable○ By your colleague

Conclusion(s)

● github.com/○ Goldmund-Wyldebeast-Wunderliebe/

■ templateproject■ puppet-appie

References

Thank you!Job Ganzevoort,

Douwe van der Meij

Goldmund, Wyldebeast & Wunderliebe

{ ganzevoort, vandermeij } @gw20e.com