from zero to hero @ pygrunn 2014

45
From Zero to Hero Job Ganzevoort Douwe van der Meij Goldmund, Wyldebeast & Wunderliebe { ganzevoort, vandermeij } @gw20e.com

Upload: meij200

Post on 10-May-2015

273 views

Category:

Software


0 download

DESCRIPTION

Deploying and maintaining (Django) applications professionally using puppet, git and scripting.

TRANSCRIPT

Page 1: From Zero to Hero @ PyGrunn 2014

From Zero to Hero

Job GanzevoortDouwe van der Meij

Goldmund, Wyldebeast & Wunderliebe

{ ganzevoort, vandermeij } @gw20e.com

Page 2: From Zero to Hero @ PyGrunn 2014

Outline

● Introduction● Initial Setup● Deployment● Maintenance● Deployment continued● Conclusion

Page 3: From Zero to Hero @ PyGrunn 2014

Introduction

Page 4: From Zero to Hero @ PyGrunn 2014

● People have great ideas● Django is perfect for RAD

○ But...● How to deploy to production?● How to keep the system maintainable?

○ Or even…● When is my application production ready?

Why this presentation?

Page 5: From Zero to Hero @ PyGrunn 2014

Initial Setup

Page 6: From Zero to Hero @ PyGrunn 2014

● Before you do anything related to code:○ git init

● Make sure you track everything you do in a VCS

Version Control System

Page 7: From Zero to Hero @ PyGrunn 2014

● In the first stage of development, program only the parts that are the core of your application

● But how to do this in git?

Define Minimum Viable Product

Page 8: From Zero to Hero @ PyGrunn 2014

● Have base/production code in master branch

● git branch for every code change● Merge all branches with acceptance

branch before going live (again)

Use branches (1/2)

Page 9: From Zero to Hero @ PyGrunn 2014

Use branches (2/2)

master

acceptance

time

bi-weekl

y

release

change #123bran

chmerg

emerge

Page 10: From Zero to Hero @ PyGrunn 2014

Deployment

Page 11: From Zero to Hero @ PyGrunn 2014

● VPS● Provision with e.g., Puppet● Find or create scripts● Put the scripts in version control!

Server(s)

Page 12: From Zero to Hero @ PyGrunn 2014

node 'kiezel.gw20e.com' {

class { 'ssh':

server_options => {

'PasswordAuthentication' => 'no',

'PermitRootLogin' => 'no',

},

}

appie::app { "mysite":

envs => ["tst", "acc", "prd"],

secret => "secret",

accountinfo => $gw20e::user_accounts,

accounts => ['ganzevoort', 'vandermeij'],

}

}

Page 13: From Zero to Hero @ PyGrunn 2014

● Create a user per environment-layer combination○ app-mysite-tst○ app-mysite-acc○ app-mysite-prd

● With their own home dir, postgresql DB, nginx glue

● Check out (and fork) our puppet module:○ puppet-appie @ GitHub

Separate users

Page 14: From Zero to Hero @ PyGrunn 2014

● Use fabric to script deploying to your servers:○ fab deploy:layer=tst

● Put the scripts in version control!

● Check out (and fork) our template:○ templateproject @ GitHub○ It includes gunicorn and supervisor configurations

Use scripts to deploy

Page 15: From Zero to Hero @ PyGrunn 2014

● Don’t just change Django’s settings.py● Keep dev, tst, acc and prd specific

settings in separate files● Put the settings in version control!

Different DTAP layers (1/6)

Page 16: From Zero to Hero @ PyGrunn 2014

● Deploy to their respective environment

Different DTAP layers (2/6)

Page 17: From Zero to Hero @ PyGrunn 2014

DEV● Laboratory setup● Switch to the branch you’re working on● Work with dev settings● Deploy to TST, ACC or PRD environment

Different DTAP layers (3/6)

Page 18: From Zero to Hero @ PyGrunn 2014

TST● Real deployment, but alpha● Work with tst settings

Different DTAP layers (4/6)

Page 19: From Zero to Hero @ PyGrunn 2014

ACC● Real deployment, but beta● Work with acc settings

Different DTAP layers (5/6)

Page 20: From Zero to Hero @ PyGrunn 2014

PRD● Real deployment!● Work with prd settings● Setup monitoring

○ Nagios○ Sentry○ ...

● Setup backup○ Database○ Uploaded media

Different DTAP layers (6/6)

Page 21: From Zero to Hero @ PyGrunn 2014

● Move settings.py as is to: settings/core.py

● Create: settings/base.py○ Add: from settings.core import *○ Add your own generic settings

● Create: settings/{dev,tst,acc,prd}.py○ Add: from settings.base import *○ Add your own layer specific settings

Django settings (1/4)

Page 22: From Zero to Hero @ PyGrunn 2014

Django settings (2/4)

settings.py core.py

base.py

LAYER.py

settings/

Page 23: From Zero to Hero @ PyGrunn 2014

● Use Fabric to create: settings/__init__.py

● This command:○ fab pick_settings:layer=prd

● Results in:○ from settings.prd import *

● Stackable settings scale○ Imagine (white) labeling

Django settings (3/4)

Page 24: From Zero to Hero @ PyGrunn 2014

Django settings (4/4)

__init__.pysettings/

dev.py tst.py acc.py prd.py

base.py

core.py

Page 25: From Zero to Hero @ PyGrunn 2014

Django settings (5/4)

Database passwords in version control?

from .base import *read_pgpass('app-mysite-prd')

Page 26: From Zero to Hero @ PyGrunn 2014

Maintenance

Page 27: From Zero to Hero @ PyGrunn 2014

● Scrum, RUP, etc.○ Any iterative methodology will do

● Basicly:○ Have periodic deadlines/releases

■ Bi-weekly○ Deliver each iteration

■ To ACC, if OK, to PRD○ Release early, release often (MVP)

Iterative development

Page 28: From Zero to Hero @ PyGrunn 2014

● Create a ticket for every change (RFC)● Estimate the ticket● Have discussion in ticket thread● Create a code branch per ticket

○ Let git help you● Deploy to a separate TST environment

○ (with separate database)

(How to) use a ticket system

Page 29: From Zero to Hero @ PyGrunn 2014

Branching revisited

master

acceptance

time

bi-weekl

y

release

change #123bran

chmerg

emerge

iteration

Page 30: From Zero to Hero @ PyGrunn 2014

● On your test machine, let each ticket/branch have its own environment (with database)

● Test implementations individually, care for code merging later

● Multiple TST environments means multiple deployments○ So not only three (= TST, ACC, PRD)

Separate environments for TST (1/5)

Page 31: From Zero to Hero @ PyGrunn 2014

● change_123/$ fab deploy

● Is shorthand for:○ fab deploy:layer=tst,branch=change_123

Separate environments for TST (2/5)

Page 32: From Zero to Hero @ PyGrunn 2014

● Isn’t setting up separate webserver(s), database(s), etc. causing a lot of overhead?

Separate environments for TST (3/5)

Page 33: From Zero to Hero @ PyGrunn 2014

● Computer says NO○ If you do it the right way

Separate environments for TST (4/5)

Page 34: From Zero to Hero @ PyGrunn 2014

● Keep your (webserver) configuration files in version control○ Use e.g., Django templates to construct environment

specific files● Keep your database scripts in version

control○ To copy a new DB from a production fixture○ To install a new DB from scratch with fixtures○ Or both!

Separate environments for TST (5/5)

Page 35: From Zero to Hero @ PyGrunn 2014

Deployment continued

Page 36: From Zero to Hero @ PyGrunn 2014

● Have separate deployment settings per layer:○ use_https○ path_to_certificate○ sitename○ deploy_user_at_host○ dir_to_deploy_to○ ...

Deployment settings (1/4)

Page 37: From Zero to Hero @ PyGrunn 2014

Deployment settings (2/4)

LAYER.pydeployment/

Page 38: From Zero to Hero @ PyGrunn 2014

● Use the Django template engine to construct configuration files on deployment:

{% if use_https %}

server {

listen 80;

server_name {{ sitename }};

rewrite ^(.*) https://$server_name$request_uri?;

}

...

Deployment settings (3/4)

Page 39: From Zero to Hero @ PyGrunn 2014

● Symlink the generated configuration files from the user homedir into the place they’re needed

● E.g.○ ln -s ~/current/etc/nginx.conf

/etc/nginx/sites-enabled/SITENAME.conf○ service nginx reload

Deployment settings (4/4)

Page 40: From Zero to Hero @ PyGrunn 2014

● Create a tag for each PRD release● Switch to that tag on deploy to PRD

○ For easy rollback■ Switch to previous tag to rollback

● Run migration scripts after switch○ Watch out for backwards incompatible changes!

Use Git tagging

Page 41: From Zero to Hero @ PyGrunn 2014

● When using Pip○ pip freeze > frozen.txt

● Add to version control!

● pip install -r frozen.txt

Freeze dependencies

Page 42: From Zero to Hero @ PyGrunn 2014

Conclusion

Page 43: From Zero to Hero @ PyGrunn 2014

● Use a VCS● Have separate DTAP layers● Script and store● Work with RFCs

● Make everything repeatable○ By your colleague

Conclusion(s)

Page 45: From Zero to Hero @ PyGrunn 2014

Thank you!Job Ganzevoort,

Douwe van der Meij

Goldmund, Wyldebeast & Wunderliebe

{ ganzevoort, vandermeij } @gw20e.com