continuous integration and deployment patterns for magento

for Magento

San Francisco, CA

Janine

Fiona

that’s me

Leo

for Magento

= 18.55 slides/min (1 slide every 3.23 sec)

for Magento

4

(and that’s ok…)

type “YESIKNOWWHATIMDOING” + click

automatically

Continuous Development

Continuous Delivery

Continuous Integration

Continuous Deployment

Continuous Monitoring

Continuous

Jenkins

Everything sounds better with “continuous”

“Deployment Pipeline”

every time you change code

passed all tests

deployed to production

One size doesn’t fit all!

What is it? A simple shell script. That’s it! No Ant, no Phing, no fancy PHP tool/framework

Where is it used?

In the build job of a Jenkins server

What does it need to know? Where to get all the source code from

What does it do? add metadata, composer, minify, packaging, upload to S3

What is it? A couple of chef cookbooks. Run by AWS OpsWorks or Vagrant,…

Where is it used?

On the target servers

What does it need to know? The role of the server (web server, devbox, Redis,…)

What does it do? Installing and configuring web server, PHP, MySQL, Redis,…

What is it? A simple shell script. Or the “deploy” chef cookbook Or EasyDeployWorkflows

Where is it used?

On the target servers

What does it need to know? How to get the build and where to deploy it to

What does it do? (Resetting the environment), getting the build, extracting it, managing release folders and symlinks, triggering installer, cleanup old releases

What is it? A simple shell script. Sometimes triggered inside “deploy” before_migrate.rb

Where is it used?

On the target servers, called from deployment step

What does it need to know? What environment this is. Injected settings

What does it do? Run modman, apply settings, trigger update scripts, customize cache prefix, indexing,…

doesn’t know anything about the app, except where to get it

from and where to put it

knows how to get the app up and

running

no creating CMS blocks

no creating product attributes

no updating of any configuration settings!

no clearing the cache

no purging CDN or Reverse Proxy

content!

media directory (sometimes minified)

Database dump

(Vanilla) Magento core

Modules

Scripts

Settings

Tools

Urls

Database Settings

Payment Provider Configuration Feature

Flags…

Everything that’s different between two

environments

Every file is a potential

security risk

Don’t deploy anything you don’t need to operate the

website

.git*

./htdocs/dev

./htdocs/includes

./htdocs/downloader

./htdocs/pkginfo

./htdocs/LICENSE*

./htdocs/RELEASE_NOTES.txt

./htdocs/phpunit.xml*

./htdocs/*.sample

./.modman/Aoe_TemplateHints

./.modman/Aoe_Profiler

./.modman/EcomDev_PHPUnit

composer.*

…

production, staging,…

devbox and integration environment

Use Jenkins to implement a full deployment pipeline for your

projects!

Test our Open Source Magento modules with

Travis CI!

Vagrant Boxes

Awesome stuff

Every single line potentially

can break production!

Bratwurst (Germans)

Donut (Americans)

Pizza (Italians)

master

Feature B

Feature A

Hotfix (if master is not modified)

Hotfix (if master is modified)

Feature C

build

install on integration

tests

build


tests

install on deploy

build


tests

deploy to stage

deploy to prod

build


tests

deploy to stage

deploy to prod

build


tests

deploy to stage

deploy to prod

single mainline

no commits, only --no-ff

merges

Version Control

GIT, SVN,…

PHP lint

Version Control Code Reviews

Crucible,…

Version Control Code Reviews Build

Files

DB

Installer

Settings

Triggered by commits, by time

or manually

Build job in Jenkins

minify js/css

.tar.gz

“virtual package” (revision)

Version Control

Code Reviews

Build

Files

DB

Installer

Settings

eat your own dogfood


Files

DB

Installer

Settings

Static Code Analysis

CodeSniffer

php-pmd

php-cpd

php-depend


Files

DB

Installer

Settings


Install on “Integration”

First system where code from all

developers/teams meets each other

unstable system, might be broken,

will be rebuilt without warning


Files

DB

Installer

Settings



Unit Tests

PHPUnit


Files

DB

Installer

Settings



Unit Tests Integration Tests

MPMD broken links,

HTTP headers, W3C validity,…

Test features that rely on other parts (e.g. framework, external services,…)

External components (SSL, Redis, Varnish, ERP, Data Import,…)

Basically this is also PHPUnit, but addressing

other things


Files

DB

Installer

Settings



Unit Tests Integration Tests Acceptance Tests

Selenium

US languages

UK DE FR IT

size

640

980

1280

1900

US languages

UK DE FR IT

language

size

browser

language

size

browser

touchstone job


Files

DB

Installer

Settings




Install on “Staging”

Behat, PDIFF,…

Same setup as production, but different machines

Client QA / approval


Files

DB

Installer

Settings





Stress

Tests

JMeter

Spin up EC2 instances to

produce traffic

Again: multiple servers or autoscaling setup


Files

DB

Installer

Settings





Stress

Tests

Install on “Production”

A/B Deployments on different

scales

Plan B: rollback

Zero Downtime


Files

DB

Installer

Settings





Stress

Tests


Deployment != Feature Release


Files

DB

Installer

Settings





Stress

Tests


Enable Features

Version Control

Code Reviews Build Static Code Analysis


Unit Tests Integration Tests

Acceptance Tests



Enable Features

Pipeline Visualization

#284

#283

#282

#281

#280

Build Static Code

Analysis Unit Tests


Integration Tests

Acceptance Tests

#285

for team dashboards

Tom E.

Follow me on twitter!

My blog

continuous integration and deployment patterns for magento

Technology