the continuous php pipeline
TRANSCRIPT
Continuous PHP Pipeline
in it2PROFESSIONAL PHP SERVICES
Michelangelo van DamPHP Consultant, Community Leader & Trainer
http
s://w
ww.
flick
r.com
/pho
tos/
akra
bat/8
7843
1881
3
Schedule
• Development processes
• Automation first
• Continuous Integration
• The PHP Pipeline
• Do what you do best: code
• Round up http
s://w
ww.
flick
r.com
/pho
tos/
wen
zday
01/3
0052
9735
5
Knowledge Check
• What’s PHP?
• What’s a SCM?
• What’s unit testing?
• What are distributed systems?
• What is provisioning?
• What are merge conflicts? http
s://w
ww.
flick
r.com
/pho
tos/
didm
ysel
f/653
0383
417
Development Management
http
s://w
ww.
flick
r.com
/pho
tos/
2743
3628
@N
05/2
5973
0832
8
WaterfallProject Start
Kick-off meetingDefining Approach
Meeting Execs
Project Initiate
Business CasesProject ControlsRisk Definition
Project Control
Progress Monitoring
Project Delivery
Project finalisation
Maintenance
Ongoing process
ScrumWeekly
backlog review
Dailystandups
Product Backlog
AcceptanceCriteria
Sprint Backlog Prototype Delivery
Product Backlog
Product BacklogProduct
Backlog
Sprint
Something (ad-hock)Project
Requirements
Ad-hockIssues
TaskAssignment Delivery
Need to deliver
Weeklybacklog review
Dailystandups
Product Backlog
AcceptanceCriteria
Sprint Backlog Prototype Delivery
Product Backlog
Product BacklogProduct
Backlog
Sprint
Project Start
Kick-off meetingDefining Approach
Meeting Execs
Project Initiate
Business CasesProject ControlsRisk Definition
Project Control
Progress Monitoring
Project Delivery
Project finalisation
Maintenance
Ongoing process
Project Requirements
Ad-hockIssues
TaskAssignment Delivery
Delivery
Delivery “event”• Off-hours release planning • Set up a maintenance page • Pre-release actions (backups, stopping crons, …) • Release checklist -> manual deployment • Post-release actions (crons, caches, …) • Removing maintenance page • Manually verify everything is working (old & new) • If lucky, going home for the night/morning
A nightmare
http
s://w
ww.
flick
r.com
/pho
tos/
bern
atcg
/223
1649
285
Automation First
http
s://w
ww.
flick
r.com
/pho
tos/
freef
oto/
5982
5499
38
Computers are great at doing repetitive task very well
Development
Source Code
Static Files
Database Deltas
Platform
Monitoring Database
Storage Workers Caches
Web Server
Infrastructure
Bare Metal
Virtual Machines Cloud
Infrastructure
Bare Metal
Virtual Machines Cloud
Infrastructure
https://www.gnu.org/graphics/heckert_gnu.small.png
Hosting Providers
On PremiseCloud Solution Providers
A “simple” architecture
Web Web
Proxy Proxy Proxy
DB Slave DB Slave DB Slave DB Slave
DB Master DB Master
NoSQL NoSQL NoSQL
0MQ
Mai
lShared IO
Web
Database
Are you ready?Things get very complex very fast
http
s://w
ww.
flick
r.com
/pho
tos/
mar
iano
-man
tel/9
1943
4426
8
Platform
Monitoring Database
Storage Workers Caches
Web Server
Automation tools
https://www.gnu.org/graphics/heckert_gnu.small.png
Why automate platform?• It should be optimised to run your application
• Continuous updates and tweaks possible
• Setting up your architecture in a consistent way
• For production
• For staging
• For testing
• For development
Choose wisely
http
s://w
ww.
flick
r.com
/pho
tos/
keep
itsur
real
/610
7919
083
Give Docker a try
• Requires no virtualisation software • Saves on resources!
• Allows to set up a distributed architecture • Even for development purposes
• Isolates processes in a container • Helps with securing infrastructure
Vagrant up
Monitoring Database
Storage Workers Caches
Web Server
Development
Source Code
Static Files
Database Deltas
SCM is a must!
FTP is not a SCM
Commit and push
Why CI?
• Monitors continuously
• Executes directly after change
• Gives feedback
• Has dashboards
• Automate post-development processes
http
s://w
ww.
flick
r.com
/pho
tos/
ambe
rand
clin
t/326
6859
324
CI takes care of your codePre build steps!
- Stop crons, queues and workers - Remove target from load balancer
Build steps!- Check out new changes - Run unit tests - Provision target(s) - Provision databases
Post build steps!- Warm up caches - Reactivate crons, queues and workers - Add target back into loadbalancer - Sent status out
Common CI systems
Online CI systems
The PHP Pipeline
http
s://w
ww.
flick
r.com
/pho
tos/
mm
mav
ocad
o/45
9739
7875
Prioritise build steps
Unit tests Metrics
Provisioningnew target
Provisioningdatabases
Provisioningworkers
Integrationtests
Promotebranch
CI
Stop execution at first fail
Unit tests Metrics
Provisioningnew target
Provisioningdatabases
Provisioningworkers
Integrationtests
Promotebranch
CI
When good, promote!Unit tests Metrics
Provisioningnew target
Provisioningdatabases
Provisioningworkers
Integrationtests
Promotebranch
CI
Unit tests Metrics
Provisioningnew target
Provisioningdatabases
Provisioningworkers
Integrationtests
Promotebranch
CI
Delivery vs Deployment
Source: http://blog.crisp.se/2013/02/05/yassalsundman/continuous-delivery-vs-continuous-deployment
Continuous Delivery
Build Feedback
Build Feedback
SCMCommits SCM Change
Polling
Deployment ManagementDashboard
test
staging
production
Continuous Deployment
Build Feedback
Execute Delivery
SCMCommits SCM Change
Pollingtest
staging
production
All code is live!
if ($application->getFeature('myawesomefeature')->isComplete()) { "! // Here comes the new feature functionality "! }
Other usages• A/B Testing
• Partial based roll-out (GeoLocation, Age, Gender)
• SaaS premium features
• Automated assisted training programs
• Netflix Chaos Monkey Testing
• See http://in2.se/nfchaosmonkey for details
• April’s fools day fun!
http
s://w
ww.
flick
r.com
/pho
tos/
ihta
tho/
6272
2631
5
Benefits
Failures do happen
Our tools
Our Stats
Total commits today 129Total releases 118Total failures 11Total DB deployments 4Average failure resolution time (minutes) 5Average release time (minutes) 3
http
s://w
ww.
flick
r.com
/pho
tos/
timyp
enbu
rg/5
1720
4200
7
Your turn now!• Automate everything that’s not coding • Create repeatable, reliable processes • Fail as quickly as possible
• If it hurts, do it more frequently • Communicate each step
• Everyone is responsible! • Continuously improve
• On your code • On your processes
in it2PROFESSIONAL PHP SERVICES
Michelangelo van Dam!Zend Certified Engineer
PHP Consulting - Training - Quality Assurance
www.in2it.be | [email protected]
http
s://w
ww.
flick
r.com
/pho
tos/
5621
8409
@N
03/1
5371
2624
55