the continuous php pipeline

Continuous PHP Pipeline

in it2PROFESSIONAL PHP SERVICES

Michelangelo van DamPHP Consultant, Community Leader & Trainer

http

s://w

ww.

flick

r.com

/pho

tos/

akra

bat/8

7843

1881

3

Schedule

• Development processes

• Automation first

• Continuous Integration

• The PHP Pipeline

• Do what you do best: code

• Round up http

s://w

ww.

flick

r.com

/pho

tos/

wen

zday

01/3

0052

9735

5

Knowledge Check

• What’s PHP?

• What’s a SCM?

• What’s unit testing?

• What are distributed systems?

• What is provisioning?

• What are merge conflicts? http

s://w

ww.

flick

r.com

/pho

tos/

didm

ysel

f/653

0383

417

Development Management

http

s://w

ww.

flick

r.com

/pho

tos/

2743

3628

@N

05/2

5973

0832

8

WaterfallProject Start

Kick-off meetingDefining Approach

Meeting Execs

Project Initiate

Business CasesProject ControlsRisk Definition

Project Control

Progress Monitoring

Project Delivery

Project finalisation

Maintenance

Ongoing process

ScrumWeekly

backlog review

Dailystandups

Product Backlog

AcceptanceCriteria

Sprint Backlog Prototype Delivery

Product Backlog

Product BacklogProduct

Backlog

Sprint

Something (ad-hock)Project

Requirements

Ad-hockIssues

TaskAssignment Delivery

Need to deliver

Weeklybacklog review

Dailystandups

Product Backlog

AcceptanceCriteria

Sprint Backlog Prototype Delivery

Product Backlog

Product BacklogProduct

Backlog

Sprint

Project Start

Kick-off meetingDefining Approach

Meeting Execs

Project Initiate

Business CasesProject ControlsRisk Definition

Project Control

Progress Monitoring

Project Delivery

Project finalisation

Maintenance

Ongoing process

Project Requirements

Ad-hockIssues

TaskAssignment Delivery

Delivery

Delivery “event”• Off-hours release planning • Set up a maintenance page • Pre-release actions (backups, stopping crons, …) • Release checklist -> manual deployment • Post-release actions (crons, caches, …) • Removing maintenance page • Manually verify everything is working (old & new) • If lucky, going home for the night/morning

A nightmare

http

s://w

ww.

flick

r.com

/pho

tos/

bern

atcg

/223

1649

285

Automation First

http

s://w

ww.

flick

r.com

/pho

tos/

freef

oto/

5982

5499

38

Computers are great at doing repetitive task very well

Development

Source Code

Static Files

Database Deltas

Platform

Monitoring Database

Storage Workers Caches

Web Server

Infrastructure

Bare Metal

Virtual Machines Cloud

Infrastructure

Bare Metal

Virtual Machines Cloud

Infrastructure

https://www.gnu.org/graphics/heckert_gnu.small.png

Hosting Providers

On PremiseCloud Solution Providers

A “simple” architecture

Web Web

Proxy Proxy Proxy

DB Slave DB Slave DB Slave DB Slave

DB Master DB Master

NoSQL NoSQL NoSQL

0MQ

Mai

lShared IO

Web

Database

Are you ready?Things get very complex very fast

http

s://w

ww.

flick

r.com

/pho

tos/

mar

iano

-man

tel/9

1943

4426

8

Platform

Monitoring Database

Storage Workers Caches

Web Server

Automation tools

https://www.gnu.org/graphics/heckert_gnu.small.png

Why automate platform?• It should be optimised to run your application

• Continuous updates and tweaks possible

• Setting up your architecture in a consistent way

• For production

• For staging

• For testing

• For development

Choose wisely

http

s://w

ww.

flick

r.com

/pho

tos/

keep

itsur

real

/610

7919

083

Give Docker a try

• Requires no virtualisation software • Saves on resources!

• Allows to set up a distributed architecture • Even for development purposes

• Isolates processes in a container • Helps with securing infrastructure

Vagrant up

Monitoring Database

Storage Workers Caches

Web Server

Development

Source Code

Static Files

Database Deltas

SCM is a must!

FTP is not a SCM

Commit and push

http

s://w

ww.

flick

r.com

/pho

tos/

mab

i/383

0712

4

Why CI?

• Monitors continuously

• Executes directly after change

• Gives feedback

• Has dashboards

• Automate post-development processes

http

s://w

ww.

flick

r.com

/pho

tos/

ambe

rand

clin

t/326

6859

324

CI takes care of your codePre build steps!

- Stop crons, queues and workers - Remove target from load balancer

Build steps!- Check out new changes - Run unit tests - Provision target(s) - Provision databases

Post build steps!- Warm up caches - Reactivate crons, queues and workers - Add target back into loadbalancer - Sent status out

Common CI systems

Online CI systems

The PHP Pipeline

http

s://w

ww.

flick

r.com

/pho

tos/

mm

mav

ocad

o/45

9739

7875

Prioritise build steps

Unit tests Metrics

Provisioningnew target

Provisioningdatabases

Provisioningworkers

Integrationtests

Promotebranch

CI

Stop execution at first fail

Unit tests Metrics

Provisioningnew target

Provisioningdatabases

Provisioningworkers

Integrationtests

Promotebranch

CI

When good, promote!Unit tests Metrics

Provisioningnew target

Provisioningdatabases

Provisioningworkers

Integrationtests

Promotebranch

CI

Unit tests Metrics

Provisioningnew target

Provisioningdatabases

Provisioningworkers

Integrationtests

Promotebranch

CI

Delivery vs Deployment

Source: http://blog.crisp.se/2013/02/05/yassalsundman/continuous-delivery-vs-continuous-deployment

Continuous Delivery

Build Feedback

Build Feedback

SCMCommits SCM Change

Polling

Deployment ManagementDashboard

test

staging

production

Continuous Deployment

Build Feedback

Execute Delivery

SCMCommits SCM Change

Pollingtest

staging

production

All code is live!

    if ($application->getFeature('myawesomefeature')->isComplete()) { "!        // Here comes the new feature functionality "!    }

Other usages• A/B Testing

• Partial based roll-out (GeoLocation, Age, Gender)

• SaaS premium features

• Automated assisted training programs

• Netflix Chaos Monkey Testing

• See http://in2.se/nfchaosmonkey for details

• April’s fools day fun!

http

s://w

ww.

flick

r.com

/pho

tos/

ihta

tho/

6272

2631

5

Benefits

Failures do happen

Our tools

Our Stats

Total commits today 129Total releases 118Total failures 11Total DB deployments 4Average failure resolution time (minutes) 5Average release time (minutes) 3

http

s://w

ww.

flick

r.com

/pho

tos/

timyp

enbu

rg/5

1720

4200

7

Your turn now!• Automate everything that’s not coding • Create repeatable, reliable processes • Fail as quickly as possible

• If it hurts, do it more frequently • Communicate each step

• Everyone is responsible! • Continuously improve

• On your code • On your processes

Recommended Reading

in it2PROFESSIONAL PHP SERVICES

Michelangelo van Dam!Zend Certified Engineer

PHP Consulting - Training - Quality Assurance

www.in2it.be | contact@in2it.be

http

s://w

ww.

flick

r.com

/pho

tos/

5621

8409

@N

03/1

5371

2624

55