devops: cooking drupal deployment

Drupal Developers Day 2013Drupal Developers Day 2013

Microsoft PHMicrosoft PH6750 Ayala Avenue, Makati City

Our Goal?Our Goal?

Zero Touch Zero Touch DeploymentDeployment

Zero Downtime Zero Downtime During During

DeploymentDeployment

Deployment Deployment SyndromeSyndrome

Fear of Change (before and after delivery)

Deployment Deployment SyndromeSyndrome

It works on my machine*

Deployment Deployment SyndromeSyndromeSilo-isolation (different software stack)

DevOpsDevOps {{ 'Drupal': 'Drupal': cooking => deployment, cooking => deployment,}}

Gerald VillorenteSenior DevOps Engineer | Open-Source Enthusiast

FTPFTPused to transfer files from one host to another host over a TCP-based used to transfer files from one host to another host over a TCP-based

networknetwork

PrehistoricPrehistoric

SlowSlow

Not SecureNot Secure

No version No version controlcontrol

Manual Manual ConfigurationConfiguration

RiskyRisky

RepetitiveRepetitive

Solution?Solution?

AutomateAutomatenot applicable in shared-hostingnot applicable in shared-hosting

ToolsTools

GitGit Git / t/ is a distributed ɡɪGit / t/ is a distributed ɡɪ revision controlrevision control and source code and source code

management (SCM) system with an emphasis on speed. management (SCM) system with an emphasis on speed. - Wikipedia- Wikipedia

Git Hooks:Git Hooks: PostPost Post-code-deploy (post-merge, post-checkout, etc) Post-db-copy Post-files-copy

Git Hooks:Git Hooks: Post-code-deployPost-code-deploy

drush -vdy @domain.prod updbdrush -vdy @domain.prod fradrush -vdy @domain.prod cc all

Git Hooks:Git Hooks: Post-code-deployPost-code-deploy

drush -vdy @domain.uat cc alldrush -vdy @domain.uat updbdrush -vdy @domain.uat fradrush -vdy @domain.uat cc all

Git Hooks:Git Hooks: Post-code-deployPost-code-deploy

drush -vdy @domain.stage cc alldrush -vdy @domain.stage updbdrush -vdy @domain.stage fradrush -vdy @domain.stage cc all

Git Hooks:Git Hooks: Post-code-deployPost-code-deploy

drush -vdy @domain.dev cc alldrush -vdy @domain.dev updbdrush -vdy @domain.dev fradrush -vdy @domain.dev cc all

Git Hooks:Git Hooks: Post-db-copyPost-db-copydrush @domain.uat cc alldrush @domain.uat updb -ydrush @domain.uat fra -ydrush @domain.uat cc all

Git Hooks:Git Hooks: Post-db-copyPost-db-copydrush @domain.stage cc alldrush @domain.stage updb -ydrush @domain.stage en views_ui -ydrush @domain.stage en context_ui -ydrush @domain.stage fra -ydrush @domain.stage dis chartbeat -ydrush @domain.stage vset cdn_status 0 -ydrush @domain.stage solr-vset conf[apachesolr_read_only] 1 -ydrush @domain.stage vset preprocess_js 0 -ydrush @domain.stage vset preprocess_css 0 -ydrush @domain.stage vset purge_proxy_urls 'http://stage.domain.com/?purge_method=ah' -ydrush @domain.stage cc all

Git Hooks:Git Hooks: Post-db-copyPost-db-copydrush @domain.dev cc alldrush @domain.dev updb -ydrush @domain.dev en views_ui -ydrush @domain.dev en context_ui -ydrush @domain.dev fra -ydrush @domain.dev dis chartbeat -ydrush @domain.dev vset cdn_status 0 -ydrush @domain.dev vset conf[apachesolr_read_only] 1 -ydrush @domain.dev vset preprocess_js 0 -ydrush @domain.dev vset preprocess_css 0 -ydrush @domain.dev vset purge_proxy_urls 'http://stage.domain.com/?purge_method=ah' -ydrush @domain.dev cc all

Git Hooks: Post-files-copy

Will run after finished copying files

- from prod to uat, stage, and or dev

- from uat to stage or dev

- from stage to uat or dev

- from dev to uat or stage

Managing FilesManaging Files

Don't put in version control Use rsync

Git Hooks: PreGit Hooks: Pre Pre-commit Prepare-commit-msg Pre-applypatch Pre-rebase

DrushDrushis a command-line shell and scripting interface for Drupal, a veritable is a command-line shell and scripting interface for Drupal, a veritable Swiss Army knife designed to make life easier for those who spend Swiss Army knife designed to make life easier for those who spend

their working hours hacking away at the command prompttheir working hours hacking away at the command prompt

Drush:Drush: Clear CacheClear Cache

Command: $ drush cc all

Drush:Drush: Run hook_updateRun hook_update

Command: $ drush updb -y $ drush @domain.prod updb -y

Drush MakeDrush Make

Drush:Drush: Make CapabilitiesMake Capabilities

Downloading Drupal core, as well as contrib modules from drupal.org. Checking code out from SVN, git, and bzr repositories. Getting plain `.tar.gz` and `.zip` files (particularly useful for libraries

that can not be distributed directly with drupal core or modules). Fetching and applying patches. Fetching modules, themes, and installation profiles, but also external

libraries.

Useful Drush Useful Drush CommandsCommands

not really important in deploymentnot really important in deployment

Drush:Drush: SQL LoginSQL Login

Command:$ drush sql-cli

Drush:Drush: SQL QuerySQL Query

Command:$ drush sqlq “UPDATE users SET pass=$S$DHaDeIifGS5qiS7ghhEolZ9lbaRTkZh/UcU.dhZY0Rpp1gUooEpQ WHERE uid=1”

Drush: Drush: SQL SyncSQL Sync

Command:$ drush sql-sync @prod @dev

Drush:Drush: SQL CreateSQL Create

Command:$ drush sql-create$ drush @domain.dev sql-create

Drush:Drush: SQL DropSQL Drop

Command:$ drush sql-drop$ drush @domain.dev sql-drop

Drush: Drush: SQL DumpSQL Dump Command:$ drush sql-dump --result-file=../bck.sql

Options:--gzip--result-file

Drush:Drush: Variable SetVariable Set

Command:$ drush vset preprocess_js 1 -y

Drush:Drush: Module EnableModule Enable

Command:$ drush en views -y$ drush en features views_ui -y

Drush:Drush: Module DisableModule Disable

Command:$ drush dis views -y$ drush dis features views_ui -y

Drush:Drush: Module UninstallModule Uninstall

Command:$ drush pm-uninstall views -y$ drush pm-uninstall ctools views -y

Drush:Drush: One-time LoginOne-time Login

Command:$ drush uli gerald$ drush uli –browser=firefox --mail=drush@example.org

Drush:Drush: Update PasswordUpdate Password

Command:$ drush upwd gerald --password=foo

Use Use Site AliasSite Alias

Allows you to execute Drush commands in target remote server via ssh

Command:$ drush @domain.prod updb -y

FeaturesFeaturesEnables the capture and management of features in Drupal. A feature Enables the capture and management of features in Drupal. A feature

is a collection of Drupal entities which taken together satisfy a is a collection of Drupal entities which taken together satisfy a certain use-case.certain use-case.

Features: Features: CreateCreate

Features: Features: ManageManage

Features: Features: ModuleModule

Features: Features: Drush IntegrationDrush Integrationfeatures-revert-all (fra) Revert all enabled feature module on your site.

Command:drush fra -y

Options:--force

BashBashA command processor, typically run in a text window, allowing the A command processor, typically run in a text window, allowing the

user to type commands which cause actions. Bash can also read user to type commands which cause actions. Bash can also read commands from a file, called a script. Like all Unix shells, it supports commands from a file, called a script. Like all Unix shells, it supports

filename wildcarding, piping, here documents, command filename wildcarding, piping, here documents, command substitution, variables and control structures for condition-testing substitution, variables and control structures for condition-testing

and iteration. and iteration. - Wikipedia- Wikipedia

Bash Scripting: Bash Scripting: SampleSample#!/usr/bin/env bashpath=”sites/all/default/files”if [ ! -d “$path”]; then mkdir $path chown www-data:www-data $path fi

Bash Scripting: Bash Scripting: DrushDrush#!/usr/bin/env bashdrush cc alldrush updb -ydrush fra -ydrush cc allecho “Delployment done!”

I don't know Bash!I don't know Bash!

You can also use Python, Shell, or Perl. Use the one that you're comfortable with. Just remember, DRY and KISS.

Drupal APIDrupal API

hook_update_Nhook_update_N

hook_update_N: hook_update_N: SampleSample/** * Enable Test module. */function test_update_7001() { $module = array('test'); module_enable($module);}

DemoDemo

What we What we achieved?achieved?

We used Git to managed We used Git to managed the entire deployment.the entire deployment.

We used Features to We used Features to capture database capture database

changes. changes.

We used Drush to run the We used Drush to run the hook_update_N, revert hook_update_N, revert the features, and clear the features, and clear

the cache.the cache.

We created a Bash script We created a Bash script that contains deployment that contains deployment

process run by Git.process run by Git.

Related Related DevOps ToolsDevOps Tools

PuppetPuppetOpen-souce automation software that helps system administrators Open-souce automation software that helps system administrators

manage infrastructure throughout its lifecycle, from provisioning and manage infrastructure throughout its lifecycle, from provisioning and configuration to orchestration and reporting.configuration to orchestration and reporting.

ChefChefA systems and cloud infrastructure automation framework that A systems and cloud infrastructure automation framework that makes it easy to deploy servers and applications to any physical, makes it easy to deploy servers and applications to any physical, virtual, or cloud location, no matter the size of the infrastructure.virtual, or cloud location, no matter the size of the infrastructure.

AnsibleAnsibleAnsible configures operating systems, deploys applications, runs Ansible configures operating systems, deploys applications, runs

parallel commands, and orchestrates IT processes like zero-downtime parallel commands, and orchestrates IT processes like zero-downtime rolling updates. It uses SSH by default, so no special software has to rolling updates. It uses SSH by default, so no special software has to

be installed to start managing remote machines. Modules can be be installed to start managing remote machines. Modules can be written in any language.written in any language.

JujuJujuJuju is a powerful service orchestration tool from Ubuntu that helps Juju is a powerful service orchestration tool from Ubuntu that helps you define, configure and deploy services to any cloud quickly and you define, configure and deploy services to any cloud quickly and

easily.easily.

CapistranoCapistranoCapistrano was originally designed to simplify and automate Capistrano was originally designed to simplify and automate deployment of web applications to distributed environments.deployment of web applications to distributed environments.

DockerDockerDocker is an open-source project to easily create lightweight, Docker is an open-source project to easily create lightweight,

portable, self-sufficient containers from any application. The same portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at container that a developer builds and tests on a laptop can run at

scale, in production, on VMs, bare metal, OpenStack clusters, public scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.clouds and more.

ForemanForemanfor managing physical and virtual serverfor managing physical and virtual server

Travis CITravis CIdistributed continuous integration service used to build and test distributed continuous integration service used to build and test

projectsprojects

JenkinsJenkinsAn extendable open source continuous integration server. Built with An extendable open source continuous integration server. Built with Java, it provides 829 plugins to support building and testing virtually Java, it provides 829 plugins to support building and testing virtually

any project.any project.

MavenMavenMaven allows a project to build using its project object model (POM) Maven allows a project to build using its project object model (POM)

and a set of plugins that are shared by all projects using Maven, and a set of plugins that are shared by all projects using Maven, providing a uniform build system.providing a uniform build system.

rsyncrsyncIs a file transfer program for Unix systems. rsync uses the "rsync Is a file transfer program for Unix systems. rsync uses the "rsync

algorithm" which provides a very fast method for bringing remote algorithm" which provides a very fast method for bringing remote files into sync. It does this by sending just the differences in the files files into sync. It does this by sending just the differences in the files across the link, without requiring that both sets of files are present across the link, without requiring that both sets of files are present

at one of the ends of the link beforehand.at one of the ends of the link beforehand.

Questions?Questions?

ResouresResoures http://devops.com http://devo.ps http://dev2ops.org http://devopsweekly.com http://devopsdays.org http://drush.ws http://try.github.io http://tldp.org/LDP/abs/html

Contact DetailsContact Details

http://about.me/gerald.villorente

10-26-1310-26-13