PowerPoint-presentatie

Deploying your apps.. the right way!

Robbert van den Bogerd

June 8th, 2013

Introduction

whoami

Developer at Ibuildings

Web enthusiast

Traveller

The Average Joe

Disclaimer

This talk is heavily opinionated. All tools mentioned could be replaced by alternatives, but I'll show you some tools and techniques which I consider best for the job.

This is not a talk about continuous deployment, although most of the ideas and best practices shown are part of the concept.

DEPLOYMENT

So what are we here for?

TOC

Horror storiesCommon pitfallsWhat we wantDeployment from beginning to endIntegration tools

Horror stories

Horror stories

FTP: Forever Taking Process

FTP is a great protocol for transferring (large) files between computers, but what it's not so great for is deploying your 25000 files counting web project from your 2mb/s home network while your web visitors are waiting for it to finish.

Horror stories

Rsync: for oldschool sysadmins

Horror stories

Versioning checkout

Horror stories

Teamviewer: REALLY?

Problems & solutions

Common pitfalls

Manual operations

Sloow deployment process

Downtime

Cache

Migrations

No turning back

Obviously, we would all prefer to never take our website down, not even when updating. But in practice most developers find it acceptable and even customers don't find it odd that their website is down for maintenance (when announced on beforehand of course). But, ideally, our customers and, more importantly, their customers should have no idea of what's going on on the server while buying new shoes in your awesome webshop.As discussed in the previous slides, most traditional deployment strategies require the developer to perform some additional manual operations after or before uploading the updated code. This often makes the developer want to work fast, because the website is down or even broken. This results in stress and often leads to mistakes made by the deployer.Some applications are very heavy and are heavily dependant on caching. After updating the code, the cache also needs to be renewed, which willl cause the first couple of page views to be slow. This might sound as a trivial problem, but in practice, when deploying multiple times a day, it will also result in multiple slow downs each day.Some deployment processes take forever to complete, which might not be a big thing when you're releasing every 3 months, but when your team wants to get new features out of the door quickly, your productivity rate will decrease dramatically when your team mates are waiting 40 minutes a day for 3 deployments to finish.There is soo much that could potentially go wrong when deploying. Database migrations gone wrong, permission errors, connection problems and so on. But, no matter what goes wrong, you should ALWAYS be able to go back and revert, so that you can investigate the problem, at your own pace, in your own time. Not when your website is throwing fatals at your visitors.

Manual operations

No!

The Bus Factor

Anyplace, anywhere, anytime

Confidence

No turning back

Zero downtime

No more UC pages!

Quick deploys

Smaller releases

Automate

Wherever you can optimize, optimize!Higher release frequency means smaller deploys means less risk of exceptionsMain point of presentation: optimize & automate!

Cache

Migrations

Common pitfalls

Manual operations

Sloow deployment process

Downtime

Cache

Migrations

No turning back

Obviously, we would all prefer to never take our website down, not even when updating. But in practice most developers find it acceptable and even customers don't find it odd that their website is down for maintenance (when announced on beforehand of course). But, ideally, our customers and, more importantly, their customers should have no idea of what's going on on the server while buying new shoes in your awesome webshop.As discussed in the previous slides, most traditional deployment strategies require the developer to perform some additional manual operations after or before uploading the updated code. This often makes the developer want to work fast, because the website is down or even broken. This results in stress and often leads to mistakes made by the deployer.Some applications are very heavy and are heavily dependant on caching. After updating the code, the cache also needs to be renewed, which willl cause the first couple of page views to be slow. This might sound as a trivial problem, but in practice, when deploying multiple times a day, it will also result in multiple slow downs each day.Some deployment processes take forever to complete, which might not be a big thing when you're releasing every 3 months, but when your team wants to get new features out of the door quickly, your productivity rate will decrease dramatically when your team mates are waiting 40 minutes a day for 3 deployments to finish.There is soo much that could potentially go wrong when deploying. Database migrations gone wrong, permission errors, connection problems and so on. But, no matter what goes wrong, you should ALWAYS be able to go back and revert, so that you can investigate the problem, at your own pace, in your own time. Not when your website is throwing fatals at your visitors.

What we want

Manual operations Automation

Sloow deployments Quick rollouts

Zero Downtime

Cache warmed up

Migrations

No turning back Rollbacks

From the beginning..

Victorious warriors first win and then go to war

- Sun Tzu

Deployment is an often forgotten topic when sitting around the drawing table. As before mentioned requirements will take time and therefore cost more money, you should mention and discuss this topic in the early stages of your project. It is essential for a successful deployment strategy.

Think before you (sign a contr)act!

Deployment is an often forgotten topic when sitting around the drawing table. As before mentioned requirements will take time and therefore cost more money, you should mention and discuss this topic in the early stages of your project. It is essential for a successful deployment strategy.

Managing expectations

Customer

UsersProject managerDevs/testers

Also, you need to manage the expectations for everybody involved in the project. Since deployment is such a critical part of your development cycle, everyone should know how to act and what to expect when working on the project.Also, users should of course be informed about updates, maintenance windows and receive proper error messages when something goes wrong while deploying.

Deployment strategy

Release cycle

Server setup

Branching strategy

Tools to use

DTAP

So, let's cut to the point..

Use git

Seriously, use git!

Reliable

Super fast

Easy branching/merging

Distributed

Github:

Awesome ui

Extra's

PR's

Integration with existing deployment tools

Dependency management

Framework

Libraries

Build/QA tools

Company private repo's

No matter how cool your versioning system is, and how good your migrations script can rollback, always make sure there is a backup of your stuff before you actually start breaking things.

Drumroll please..

Composer

Migrations

Write your own script

Use one of the great existing ones:

- Doctrine migrations

- DbPatch

- Shipped with your framework/orm?

Backups

Always create one!

..

unless it's not possible =]

No matter how cool your versioning system is, and how good your migrations script can rollback, always make sure there is a backup of your stuff before you actually start breaking things.

Directory structure

/var/www/rule34/

/var/www/rule34/releases

/var/www/rule34/releases/20130606143000

/var/www/rule34/releases/20130606143400

Live www dir:

/var/www/rule34/current > releases/20130606143400

Deployment script

> Do a backup

> Create new directory

> Update your code

> Update dependencies

> Run migrations

> Warmup cache

> ...

So now we're getting more and more close to having us a deployment script

Let's start scripting!

Wait a second..

Existing tools

Fabric

Phing/Ant

Chef

Capistrano

Capistrano

Powerful

Easy setup

Event/task based, easy to hook into

Written in ruby

Multiple servers

Multistage extension

Capifony

The power of Capistrano

Pre-configured for Symfony

Deploy with 1 single config file

Setup

Local setup

gem install capifony

cd /var/www/rule34/

capifony install .

Capifony config

# app/config/deploy.rb

set :application, "My App"

set :deploy_to, "/var/www/my-app.com"

set :domain, "my-app.com"

set :scm, :git

set :repository, "ssh-gitrepo-domain.com:/path/to/repo.git"

Remote setup

cd /var/www/rule34/

cap deploy:setup

Deploy

Capifony deploy

cd /var/www/rule34/

cap deploy

# made a boo-boo?

cap deploy:rollback

Configuration

set :model_manager, "doctrine"

Configuration

set :update_vendors, false

set :use_composer, true

set :vendors_mode, "install"

Configuration

set :dump_assetic_assets, true

Configuration

set :shared_children, [app_path+"/logs", "data"]

set :writable_dirs, [app_path+"/logs", app_path+"/cache", "data"]

Directory structure

/var/www/rule34/releases

/shared

/shared/data

/shared/app/logs

Symlink:

/var/www/rule34/releases/20130606143400/data > ../../shared/data

DTAP

Multistaging

# app/config/deploy.rb

require 'capistrano/ext/multistage'

set :stages, %w(prod acc test dev)

set :default_stage, "test"

set :stage_dir, "app/config/deploy/stages"

# app/config/deploy/stages/test.rb

server 'rule34.mytestserver.com'

set :symfony_env_prod, "test"

User permissions

set :use_sudo, false

set :user, "deployment"

set :group, "www-data"

set :webserver_user, "www-data"

:use_set_permissions, true

:permission_method, chmod # or acl or chown

Maintenance page

cap deploy:web:disable \

REASON="hardware upgrade" \

UNTIL="12pm Central Time"

# override template

set :maintenance_template_path, app/Resources/uc.html

Tips 'n Tricks

# determine tag/branch/commit/sha1 on the fly

set :branch do

default_tag = `git describe --abbrev=0 --tags`.split("\n").last

tag = Capistrano::CLI.ui.ask "Tag/branch to deploy (make sure to push the tag first): [#{default_tag}] "

tag = default_tag if tag.empty?

tag

end

Tips 'n Tricks

# enable ssh client interaction

default_run_options[:pty] = true

Tips 'n Tricks

# enable agent forwarding

ssh_options[:forward_agent] = true

Tips 'n Tricks

# cleanup old release dirs automatically

set :keep_releases, 3

after "deploy", "deploy:cleanup"

Tips 'n Tricks

# run doctrine migrations after code update

after update_code, doctrine:migrations:migrate

Tips 'n Tricks

# run any symfony console commandbefore any_task, symfony:cache:clear

What's next?

System provisioning

Upgrade PHP 5.3 > 5.4

Upgrade MySql

Install Redis

Install Node.js

Etc..

Chef

Version control your infrastructure

Identical environments

Integration with Vagrant

Chef

# run chef after code update

after update_code, provision

task :provision do

end

Chef

# run chef after code update

after update_code, provision

task :provision do

run "#{sudo} chef-solo -c #{release_path}/chef/config.rb #{release_path}/chef/chef_#{stage}.json"

end

Vagrant

Build automation

Jenkins

The end result

The usual stuff

Twitter: @nickedname

Joind.in:https://joind.in/8461

Further reading

A basic git capistrano workflow

http://labs.headshift.com/2012/02/29/a-basic-gitcapistrano-workflow/

A successful git branching model

http://nvie.com/posts/a-successful-git-branching-model/

The dbPatch project

http://www.dbpatch-project.com/

Continuous deployment with symfony2, jenkins and capifony

http://jeremycook.ca/2012/11/04/continuous-deployment-with-symfony2-jenkins-and-capifony/

Software branching and parallel universes

http://www.codinghorror.com/blog/2007/10/software-branching-and-parallel-universes.html

Naam Presentator