symfony under control. continuous integration and automated deployments in symfony-based projects
DESCRIPTION
Minsk PHP User Group Meetup #3TRANSCRIPT
![Page 1: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/1.jpg)
Continuous Integration and Automated Deployments for Symfony-based projects
Symfony 2 under control
PHP Minsk User Group
![Page 2: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/2.jpg)
2
INTRODUCTION
![Page 3: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/3.jpg)
3
Max Romanovsky
maxromanovsky.com
Twitter, LinkedIn, SensioLabs Connect, MoiKrug: maxromanovsky
Who am I
![Page 4: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/4.jpg)
4
Continuous integration (CI) is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day.
http://en.wikipedia.org/wiki/Continuous_integration
http://www.martinfowler.com/articles/continuousIntegration.html
What is Continuous Integration?
![Page 5: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/5.jpg)
5
• Improve quality of application
• Understand current situation on project
• Automate your checklists & verifications
• Notify different people on different events
– Build broken? – Developers
– New build deployed to QA? – QA engineers
– New build deployed to UAT/Production? – Customers
How could it help you?
![Page 6: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/6.jpg)
6
Dependencies• Composer• Platform packages• Phar• Pear• Pecl
Build• Phing• phpqatools.org
Continuous Integration• Jenkins CI• Additional PHP-
related & general plugins
• Git
Deployment• Package with
vendors• Capifony• Doctrine Migrations
Our proposal for Symfony-based projects
![Page 7: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/7.jpg)
7
DEPENDENCIES
![Page 8: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/8.jpg)
8
Manages 3rd party dependencies
required at runtime*
• Bundles• PHP libraries• JS/CSS libraries• *Phpunit for hphpa
Composer
getcomposer.org
![Page 9: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/9.jpg)
9
Checklist
• stability• lib-*, ext-*• install vs update• require• version masks• Keep composer.phar
under VCS• Satis for private packages
Composer
github.com/composer/satis
lib-*, ext-*: goo.gl/Z8dFal
![Page 10: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/10.jpg)
10
Phar• Only for CLI libraries used for development and build• Composer, php-cs-fixer
Pear• Build tools• Phing, phpunit*, phpqatools
Pecl• For PHP extensions missing in platform packages
Platform packages (apt, yum, ports)• Use it for all purposes if you can (PHP, extensions etc.)
Other tools
![Page 11: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/11.jpg)
11
What to update?• Composer dependencies• PHARs• PEARs• PECLs• Platform packages
Keep in mind• Update at the beginning of the
iteration• Read changelogs• Subscribe to news feeds• Use * as Composer dependency
version to receive all updates• Not for Symfony itself
Dev
QA
Staging• Performance testing• Check logs
Production
Don’t forget about updates!
![Page 12: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/12.jpg)
12
BUILD
![Page 13: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/13.jpg)
13
Ant
• From Java world
• XML declarative config
• Can run tasks in parallel
• No PHP-specific tasks
• Could be extended (Java)
• IDE support
Phing
• Written in PHP• Ant config
syntax• All tasks by
default are running in the single PHP process
• Could be extended (PHP)
• IDE support
Pake
• Written in PHP• Not so popular• PHP-based
config• github.com/inde
yets/pake/wiki
Build tools
![Page 14: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/14.jpg)
14
build-symfony-ci.xml• Default workflow
build.xml• Overwrites for current
project
build.properties.dist• Default configuration
build.properties• Project configuration
Phing
phing.info
![Page 15: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/15.jpg)
15
Target
• A set of tasks• Could depend
on other targets
• phing <target_name>
Task
• Action executed as a part of target
• Custom tasks can be implemented
Types
• Reference• FileSet• Property• Other
• FileList• Filters
Phing Terminology
![Page 16: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/16.jpg)
16
build
• for dev environment
• outputs to the console
build-ci
• for CI environment
• outputs to XML files
package
• generates artifacts for deployment
• tar.gz archive
• SQL schema dump
deploy
• deploys tar.gz archive to the environment
Main Phing Targets
![Page 17: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/17.jpg)
17
Special clean
prepare
build-tools-self-update
Composer-specific
composer-validate
dependencies-install
dependencies-list-updates update –dry-run
symfony-security-checker security.sensiolabs.org
Phing Targets (1/6)
![Page 18: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/18.jpg)
18
General phplintphp -l
phploc
pdependcharts
phpmd+ custom config
phpcs/phpcs-js+ custom config (PSR-1&2, no var_dump(). console.log() etc)
phpcpd
Phing Targets (2/6)
![Page 19: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/19.jpg)
19
General apidocPHPDoc
projectdocrst2html
phpunitvia exec
phpcbPHP_CodeBrowser UI
php-cs-fixer
hphpaHipHop static analysis
No target for KPHP yet
Phing Targets (3/6)
![Page 20: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/20.jpg)
20
Symfony-specific
copy-configs-ci parameters.yml for CI
cache:warmup app/cache is removed during build
twig:lint for Bundles and app/Resources
assets:install
assetic:dump YUICompressor, Google Closure Compressor, lessphp
Phing Targets (4/6)
![Page 21: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/21.jpg)
21
Symfony-specific
doctrine:database:drop
doctrine:database:create
doctrine:migrations:migrate
doctrine:fixtures:load
doctrine:schema:validate
doctrine:schema:dump –dump-sql
for DDL schema artifact
Phing Targets (5/6)
![Page 22: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/22.jpg)
22
Package package-version version.txt
package-db-schema schema.sql
package-archive
Deployment deploy-maintenance- enable/disable
deploy-backup-db
deploy-clear-doctrine-cache Only if not in APC
deploy-cleanup-old-releases
deploy-artifact artifact name provided via CLI argument
Phing Targets (6/6)
![Page 23: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/23.jpg)
23
CONTINUOUS INTEGRATION
![Page 24: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/24.jpg)
24
CruiseControl• +PHP Under
Control
TeamCity• Commercial
Travis CI• SaaS
Scrutinizer CI• SaaS• PHP-oriented
Jenkins CI• ex-Hudson CI
CI servers
![Page 25: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/25.jpg)
25
• Open-source
• Enterprise
• Lot of plugins
• PHP Template from Sebastian Bergmann
• IDE Integration
• RESTful API
Jenkins CI overview
jenkins-ci.org
![Page 26: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/26.jpg)
26
• No comments git
• green & red signs for buildsgreenballs
• parameterized buildsparameterized-trigger
• git tag selector for builds• disable internal tagging!git-parameter
• copy artifact from one job to anothercopyartifact
• sends emails on successful builds• useful for deployment jobsemail-ext
Jenkins Plugins (common)
goo.gl/IZ2gKE
![Page 27: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/27.jpg)
27
phing
•phpcs•hphpacheckstyle
•phpcpddry
•php_dependjdepend
•phploc using CSV filesplot
•phpmdpmd
•aggregates info from phpcs, hphpa, phpmd, phpcpdviolations
•phpunitxunit
•HTML artifactshtmlpublisher
Jenkins Plugins (PHP-specific)
jenkins-php.org
![Page 28: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/28.jpg)
28
Master
• Bleeding edge• For completed features
Feature
branch
• Merge to master on completion• After code review• Developer executes phing build at least before code review
Tag
• Stable version from master
Bugfix & develop?
• Sprint branch• Master branch• Merge feature branches to sprint branch• Jenkins job for sprint branch
Git workflow
![Page 29: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/29.jpg)
29
• Builds specific branch (master)• Features are verified after merge
build-<branchname>
• Parameterized• Produces .tar.gz artifact from specific
Git tag
build-package-tag
Jenkins Jobs for build
![Page 30: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/30.jpg)
30
• Produces .tar.gz artifact from latest revision of branch• Deploys to QA• Not recommended to have more than one such job
(issues with migrations)• Workaround: multiple QA or complete DB purge on deploy
deploy-qa-<branchname>
• Deploys specific package from build-package-tag to specified environment
• Ability to enable/disable maintenance mode and error message
• Email after deployment
deploy-package-tag
Jenkins Jobs for deployment
![Page 31: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/31.jpg)
31
Deployed .tar.gz archive
• <tag>.<build#>• v0.0.3.26
• <branchname>-<build#>-<revision>-SNAPSHOT• master-12-7c041a90366ce9d958760094c763d809ffbc629e-
SNAPSHOT
Previous deployed version
DB dump before deployment
Doctrine Migrations info
DB schema
Jenkins Jobs for deployment. Artifacts
![Page 32: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/32.jpg)
32
DEPLOYMENT
![Page 33: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/33.jpg)
33
Capistrano• from Ruby world
Capifony• Based on
Capistrano
Shell script• VCS update• Rsync
Phar• Don’t use WebPhar• Silex does not use
it anymore
Zend Server package• No support for
vanilla PHP
PaaS• AWS Elastic
Beanstalk• PagodaBox
Deployment tools
![Page 34: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/34.jpg)
34
Based on Capistrano• Supports Capistrano plugins
Symfony 1&2 deployment
Multistage• qa, staging, uat, production
Maintenance mode
Clustered environments
Capifony
capifony.org
![Page 35: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/35.jpg)
35
Directories• releases/• current/ (symlink)• shared/
Shared• app/config/parameters.yml• app/cache• app/logs• web/uploads
Shared files & dirs are symlink’ed
Shared files & dirs are not shared across
nodes
Directory structure for Capistrano projects
![Page 36: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/36.jpg)
36
`-- /var/www/my-app.com
|-- current → /var/www/my-app.com/releases/20100512131539
|-- releases
| `-- 20100512131539
| `-- 20100509150741
| `-- 20100509145325
`-- shared
|-- web
| `-- uploads
|-- log
`-- config
`-- databases.yml
Directory structure example
![Page 37: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/37.jpg)
37
All you need is package• capistrano-deploy-strategy-archive
Github not available?• Goodbye, vendors!
Decreases deployment time• Lot of vendors• Slow internet connection
Reproducible• You know what you deploy• You can deploy exactly the same package to multiple environments
Deployment via SCM vs package
![Page 38: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/38.jpg)
38
.tar.gz Optimized autoloader
Contains only necessary dirs & files
assets:install assetic:dumpversion.txt &
db-schema.sql
Packaging with Phing
![Page 39: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/39.jpg)
39
DBDeploy Liquibase
Doctrine Migrations
Doctrine Fixtures
Database Migrations
![Page 40: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/40.jpg)
40
User
• Triggers job on Jenkins CI• Choose options (env, maintenance mode)
Jenkins
• Executes single phing target with additional command-line arguments
Phing
• Executes several Capifony commands according to command-line arguments
Capifony
• Performs actions on remote machines
Capifony & Jenkins CI
![Page 41: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/41.jpg)
41
Enables Maintenance mode• maintenance.html in webroot
Backups DB
Clears Doctrine cache• Not for APC cache
Downloads previous deploy metadata• package version• Doctrine Migrations info
Deploys tar.gz archive
Capifony & Phing (1/2)
![Page 42: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/42.jpg)
42
Restarts services• php-fpm or Apache• Doctrine APC cache is cleared
Symfony-specific tasks• copy app/config/parameters.yml• doctrine:migrations:migrate• cache:warmup
Disables Maintenance mode
Performs cleanup for old releases
Capifony & Phing (2/2)
![Page 43: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/43.jpg)
43
Semi-automatical
• from deployment server
• usually CI server
Enable Maintenance
• manually via phing
Manually backup for analysis
• DB• current version• Doctrine
Migrations status
• codebase
Rollback (1/2)
![Page 44: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/44.jpg)
44
Switch to previous release
• Manually via capifony CLI
• Recover DB• Rollback
Doctrine Migrations
• Restore DB dump
• Custom
Additional tasks
• Restore web server config
• Restart service• cache:warmup
Disable Maintenance
• Manually via phing
Rollback (2/2)
![Page 45: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/45.jpg)
45
Write rollback guide• Step-by-step• Verify it periodically
Enable monitoring on prod• Zabbix• Munin• New Relic• Pinba• Graylog2
Rollback tips
![Page 46: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/46.jpg)
46
SUMMARY
![Page 47: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/47.jpg)
47
• Currently not available online
• Planning to release in 1-2 months on GitHub
Where to download?
![Page 48: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/48.jpg)
48
Deployment•Cronjob / CLI script handling•Web server (Apache, nginx) config manipulations
•Ext- and lib- dependency verification via Composer
What is not implemented yet
![Page 49: Symfony under control. Continuous Integration and Automated Deployments in Symfony-based projects](https://reader034.vdocument.in/reader034/viewer/2022050808/554fb4b7b4c9057b298b5324/html5/thumbnails/49.jpg)
Questions?