best practices for development deployment & distributions: capital camp + govdays

Best Practices for Development Deployment & Distributions

August 1, 2014

Friday, August 1, 14

Best Practices...• Defining environments (Dev, Stage, Prod, etc)

• Smooth and Automated deployments

• Distributions and How to use them

• Useful development tools

Mike PotterSoftware Architect @ Phase2

Email: [email protected]: mpotter


mailto:[email protected]


Principles & Goals• Create an easy-to-understand and simple codebase

• Know exactly what code is present in each environment

• Isolate developers to avoid stepping on toes

• Track contributed code and patches

• Automate putting desired code into an environment

• Automate putting desired configuration into an environment

• Test upcoming release code against latest production data/config


Environments


Environments - DEVelopment

• One for each Individual developer(whether Local, VMs, or via VHosts)

• Use git-flow branching strategy

• Put Everything into Code!

• Isolate Developers

Development

Integration

Staging / QA

Production


Environments - INTegration

• “Integrates” all DEV environments

• Runs on “develop” branch

• First round of testing

• Automated or Manual rebuilding

Development

Integration

Staging / QA

Production


Environments - QA (Staging)

• Contains Production Content

• Runs on “release” branches or tags

• Final testing ground

• Used to test hotfixes

• Essentially a copy of Production for the next release

Development

Integration

Staging / QA

Production


Environments - PRODuction

• “Database of Record” for Content

• Runs on tagged “master” branch

• Should NEVER manually touch Production

• Use Hotfix process on QA for emergencies

Development

Integration

Staging / QA

Production


Deployment


Drupal DB

Images, FilesPHP CodeDevelopment

Integration / Staging

Production

GIT Repo

Code

Code

Images, Files

Images, Files

DB Content

DB Content

Configuration Code

git commitgit push

git pull

Features

rsync @prod >> @stage

rsync @stage >> @dev

sql dump

sql restore

sql restorebranch: develop

branch: release

git pull

branch: master DB of Record

Code FilesDatabaseContent

• CODE is for Developers

• CONTENT is for Production


Git Flow

• Each major feature has a branch

• Current release is “master”

• Next release is “develop”

• http://github.com/nvie/gitflowprovides easy git commands

• git flow feature start <name>git flow feature publish <name>

http://nvie.com/posts/a-successful-git-branching-model/


https://github.com/nvie/gitflow

https://github.com/nvie/gitflow



Everything in Code• Use Features module to export configuration

http://drupal.org/project/features

• Keep it modular! (don’t put everything into one feature)

• Use Features Override module to export site-specific changes(when using a base distribution)

• Use update-hooks for anything else(custom hook_update functions in code)




Features Strategies

• In general, more, smaller Features is better

• Group by functionality, such as configuration related to a specific content type

• Keep stuff that is often changed (permissions) separate from configuration that doesn’t change (fields)


Good Featuregallery (content type)main_image (field)gallery_thumbnail (image style)gallery_view (view)

Bad Featuregallery_view (view)content_view (view)event_view (view)news_view (view)user_view (view)


Use Update Hooks

• Enable/disable modules

• Update database

• Revert features(if not using drush fra)


Automated Deployment

• Use drush command line tool

• Pull code (git fetch, git checkout branch/tag)

• Rsync files @prod >> @stage (*)

• Run updates (drush updb)

• Revert all features (drush fra -y)

• Clear cache (drush cc all)

• (*) See also the Stage File Proxy module


Tools for Continuous Integration

• Open Source : Jenkins, CruiseControl

• Commercial : Bamboo (Atlassian)

• Create jobs to execute automated scripts

• Run job automatically when code changes


Distributions


Distributions & Profiles• A Distribution is:

A full copy of Drupal core along with additional modules, themes, and installation profiles

• A Profile is:A set of installation instructionshttps://www.drupal.org/node/1022020

• Drupal searches in /sites before /profiles


https://www.drupal.org/node/1022020

https://www.drupal.org/node/1022020

Why use Profiles?

• Easily separate common code from site-specific code

• Documents exact module dependencies and patches(stops you from hacking core)

• Easier to control module updates

• Can build upon other profiles (*)

• Can be more of a “drush make” wrapper rather than something actually “installed”


Module search order

• Drupal searches for modules in this order:/sites/sitename/modules (for multisites)/sites/default/modules/sites/all/modules/profiles/ProfileName/modules/profiles/BaseProfileName/modules (if using a base profile)/modules

• Modules from your Profile are kept separate fromsite-specific module overrides


Building your own Install Profile• ProfileName.info

Defines the dependencies (other modules)

• ProfileName.profileCan add steps to the installer, or just have an emptyProfileName_install() hook function

• ProfileName.makeDefines the specific versions and patches for modules


ProfileName.info file

• Same as Module.info files

• Determines what modules areenabled when profile installed

• Can contain:base[] = BaseProfileName

to build on another profilewhen using this patch:http://drupal.org/files/1356276-make-D7-21.patch

name = Profile Namedescription = Description of what the profile does.core = 7.xdependencies[] = blockdependencies[] = colordependencies[] = commentdependencies[] = contextualdependencies[] = dashboarddependencies[] = helpdependencies[] = imagedependencies[] = listdependencies[] = menudependencies[] = numberdependencies[] = optionsdependencies[] = pathdependencies[] = taxonomydependencies[] = dblogdependencies[] = searchdependencies[] = shortcutdependencies[] = toolbardependencies[] = overlaydependencies[] = field_uidependencies[] = filedependencies[] = rdf


http://drupal.org/files/1356276-make-D7-21.patch

http://drupal.org/files/1356276-make-D7-21.patch

ProfileName.profile File

• Can add steps to installer

• Return empty array for default

<?php/** * Implements hook_install_tasks(). */function PROFILENAME_tasks($install_state) { $tasks = array(); return $tasks;}?>


Profile Make files

• Controls what modules are downloaded when built

• Three make files:drupal-org-core.make (contains Drupal Core)drupal-org.make (contains everything else)build-profile.make (references the other two)

• Allows you to build a Distribution from your Profileusing “drush make”


drupal-org-core.make

• Determines what version of Drupal Core

• Contains any needed patches to core


drupal-org.make• Reference exact versions.

• Apply specific patches

• Modules install into/profiles/profilename/modules

• Can specify exact git revision


build-profile.make

• Just referencesthe other two make files.


Building using Drush Make• Pull your latest profile into temp directory

• Run drush:drush make --prepare-install --contrib-destination=profiles/ProfileName ProfileLocation/build-profile.make DestinationDir

• Example:git clone [email protected] make --prepare-install --contrib-destination=profiles/myprofile myProfileRepository/build-myprofile.make /var/www/mysite

• Will download modules, patch, etc

(ProfileLocation)




Merge the distribution

• Use rsync to merge your profile distribution with what

“drush make” installed

rsync -r ProfileLocation/ DestinationDir/profiles/profileName/

• Then just go to http://mysite/install.php to install

Profile

Drush Make

Full Drupal Site


http://mysite/install.php

http://mysite/install.php

Automated Building• You only need to run install.php ONCE to set up your initial site

• For future changes (e.g. via Jenkins),

• This will delete DestDir (drupal root) and recreate your entire site!

cd ProfileLocationgit checkout developgit pull origin developmv DestDir/sites /tmp/sitesdrush make --contrib-destination=profiles/myprofile build-profile.make DestDirrsync -r ProfileLocation/ DestDir/profiles/myprofilemv /tmp/sites DestDir/sitescd DestDirdrush updbdrush fra -ydrush cc all


“Fake” profiles• Install Drupal normally

• Use “drush make” to create files in profiles/profileName

• Tell Drupal you installed with a profile:drush vset install_profile profileName

• Now Drupal will look in profiles/profileName for modules

• Don’t need profileName.profile or profileName.install files

• profileName.info can be a “stub” file

(ADVANCED)


Why use Profiles? (revisited)

• Easily separate common code from site-specific code

• Documents exact module dependencies and patches(via drush make files)

• Easier to control module updates

• Can build upon other profiles (*)


Development Tools


Making DEV like PROD

• Development environment often doesn’t matchconfiguration of Production environment:

• PROD has caching (Varnish, etc)

• OS Differences (PROD on Linux, DEV on Mac)

• Version consistency of Apache, PHP, MySQL, etc

• Configuration consistency


Virtual Machines (VM)

• VirtualBox (free), VMware, etc

• Allows you to run a completely separate machine within your own computer

• Can create a separate VM for each client project

• Works on PC, Mac, Linux

• Can take a while to configure

• Can use a lot of resources (memory, disk)


Vagrant

• Vagrant is like a MAKE file for VMs

• Vagrant configuration sets RAM, CPU, etc

• Allows you to spin up a VM locally that matches PROD(can also spin up on Amazon, Rackspace, etc)

• Acts as a command-line wrapper around VirtualBox/VMWare


Puppet (or Chef)

• Puppet is like a MAKE file for the Software configuration.

• Controls version and configuration of Apache, nginx, PHP, drush, MySQL, etc.

• Can be used to configure both DEV and PROD(doesn’t require a VM)

• Helps document exact server configurations

• See https://puphpet.com/ for interactive puppet script creator.


https://puphpet.com/

https://puphpet.com/

IDE (Integrated Development Environment)

• Code styling (Drupal coding standard)

• Code analysis (unused variables, missing returns, etc)

• Integrated Debugging (breakpoints, step, variable inspect)

• Integration with version control (git)

• Integration with other tools (Vagrant, ssh, gitflow)

• Drupal integration

• (I personally use PHPStorm)


Summary

• Define your environments

• Create a deployment process (and stick with it)

• Match DEV and PROD as much as possible

• Keep hands off PROD! (everything in code)

• Automate all the things

• Use all the tools


Questions?


PHASE2TECHNOLOGY.COM


best practices for development deployment & distributions: capital camp + govdays

Technology

view view content

code git fetch

production content

code use features module

release git

desired code

code changes

git push git