devops for drupal
DESCRIPTION
Deploying your Drupal site, Upgrading your Drupal Site, Scaling, Clustering and Monitoring it ... all topics Developers are often not involved with ... Devops For Drupal explains the Devops problem, to a Drupal audience .TRANSCRIPT
![Page 1: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/1.jpg)
Devops, Devops, DrupalDevops, Devops, DrupalDries Vanlerberghe and Kris Buytaert
![Page 2: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/2.jpg)
Kris BuytaertKris Buytaert@krisbuytaert@krisbuytaert● I used to be a Dev, Then Became an OpI used to be a Dev, Then Became an Op● Senior Linux and Open Source Consultant Senior Linux and Open Source Consultant
@[email protected]● „„Infrastructure Architect“Infrastructure Architect“● Building Clouds since before the Cloud Building Clouds since before the Cloud ● Surviving the 10Surviving the 10thth floor test floor test● Co-Author of some books Co-Author of some books ● Guest Editor at some sitesGuest Editor at some sites
![Page 3: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/3.jpg)
Dries VanlerbergheDries Vanlerberghe
● I used to be a Dev, and kept it that way!I used to be a Dev, and kept it that way!● Open Source Consultant @inuits.beOpen Source Consultant @inuits.be● „„Developer“Developer“● +10 years Web development+10 years Web development● Translating business needs to working Translating business needs to working
applicationsapplications
![Page 4: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/4.jpg)
What's this Devops thing about ?What's this Devops thing about ?
![Page 5: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/5.jpg)
Alternative TitlesAlternative Titles● Tales from Grumpy old Sysadmin Tales from Grumpy old Sysadmin
● Devops, Devops, DevopsDevops, Devops, Devops
● Surviving the 10Surviving the 10thth floor test floor test
● Why devs hated opsWhy devs hated ops
![Page 6: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/6.jpg)
● Devops is a growing movementDevops is a growing movement
● We don't have all the answers yetWe don't have all the answers yet
● We are reaching out to different communities We are reaching out to different communities
● We will point out problems we see..We will point out problems we see..
While we are still working out the solutionsWhile we are still working out the solutions
![Page 7: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/7.jpg)
CAMSCAMS● CultureCulture
● AutomationAutomation
● MeasurementMeasurement
● SharingSharing
Damon Edwards and John WillisDamon Edwards and John Willis
![Page 8: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/8.jpg)
The community of developers whose work you The community of developers whose work you see on the Web, who probably don’t know what see on the Web, who probably don’t know what ADO or UML or JPA even stand for, deploy ADO or UML or JPA even stand for, deploy betterbetter systems at systems at less costless cost in in less timeless time at at lower risklower risk than we see in the Enterprise. This is true even than we see in the Enterprise. This is true even when you factor in the greater flexibility and when you factor in the greater flexibility and velocity of startups.velocity of startups.
Tim Bray , on his blog January 2010Tim Bray , on his blog January 2010
What's the problem ?What's the problem ?
![Page 9: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/9.jpg)
How did we get here ?How did we get here ?
![Page 10: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/10.jpg)
The Old DaysThe Old Days● ““Put this Code Live, here's a tarball” NOW! Put this Code Live, here's a tarball” NOW!
● What dependencies ?What dependencies ?
● No machines available ?No machines available ?
● What database ?What database ?
● Security ?Security ?
● High Availability ? High Availability ?
● Scalability ?Scalability ?
● My computer can't install this ?My computer can't install this ?
![Page 11: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/11.jpg)
Devs vs OpsDevs vs Ops
![Page 12: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/12.jpg)
10 days into production10 days into production● What High Load ? What Memory usage ?What High Load ? What Memory usage ?
● Are these Logs ? Or this is actualy customer Are these Logs ? Or this is actualy customer data ?data ?
● How many users are there , should they launch How many users are there , should they launch 100 queries each ?? Oh we're having 10K 100 queries each ?? Oh we're having 10K users users
● Why is debugging enabled ?Why is debugging enabled ?
● Who the fsck wrote this crap ? Who the fsck wrote this crap ?
![Page 13: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/13.jpg)
11 days into production11 days into production
![Page 14: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/14.jpg)
We can solve this !We can solve this !
● Some people think Some people think the Ops work starts the Ops work starts on deploymenton deployment
• Ops = both system, Ops = both system, app as platform !app as platform !
● It starts much earlierIt starts much earlier
● Get Devs and Ops to Get Devs and Ops to talk asaptalk asap
![Page 15: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/15.jpg)
Talk about Non Talk about Non functional Reqs NOW!functional Reqs NOW!
● SecuritySecurity
● BackupsBackups
● High AvailabilityHigh Availability
● UpgradabilityUpgradability
● DeploymentDeployment
● Monitoring Monitoring
● ScaleScale
![Page 16: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/16.jpg)
Breaking the SilosBreaking the Silos
Getting AlongGetting AlongOpsOpsDevs Devs
![Page 17: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/17.jpg)
NirvanaNirvanaAn “ecosystem” that supports continuous delivery, from An “ecosystem” that supports continuous delivery, from infrastructure, data and configuration management to infrastructure, data and configuration management to business.business.
Through automation of the build, deployment, and testing Through automation of the build, deployment, and testing process, and improved collaboration between developers, process, and improved collaboration between developers, testers, and operations, delivery teams can get changes testers, and operations, delivery teams can get changes released in a matter of hours — sometimes even minutes–no released in a matter of hours — sometimes even minutes–no matter what the size of a project or the complexity of its code matter what the size of a project or the complexity of its code base.base.
Continuous Delivery , Jez HumbleContinuous Delivery , Jez Humble
![Page 18: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/18.jpg)
How many times a day ?How many times a day ?● 10 @ Flickr10 @ Flickr
● Deployments used to be painDeployments used to be pain
● Nobody dared to deploy a siteNobody dared to deploy a site
● Practice makes perfectPractice makes perfect
● Knowing you can vs constantly doing it Knowing you can vs constantly doing it
![Page 19: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/19.jpg)
How do we get there ?How do we get there ?
![Page 20: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/20.jpg)
Todays EnviromentsTodays EnviromentsFor DevsFor Devs
● Version ControlVersion Control
● Automated Build Automated Build
● Bugtracking Bugtracking
● Continous integrationContinous integration
● Integrated testingIntegrated testing
● Automated Automated deployment deployment
For OpsFor Ops
● Version ControlVersion Control
● Automated Build Automated Build
● Bugtracking Bugtracking
● Continous integrationContinous integration
● Integrated testingIntegrated testing
● Automated Automated deployment deployment
![Page 21: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/21.jpg)
![Page 22: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/22.jpg)
![Page 23: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/23.jpg)
![Page 24: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/24.jpg)
Infrastructure as CodeInfrastructure as Code● Automated DeploymentsAutomated Deployments
● If my computer can't install it , the installer is If my computer can't install it , the installer is borkenborken
● Reproducable Reproducable
● With configuration mgmtThink : With configuration mgmtThink :
• Cfengine,Puppet, ChefCfengine,Puppet, Chef
● Put configs under version controlPut configs under version control
![Page 25: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/25.jpg)
Deploying a SiteDeploying a Site● Code vs ConfigCode vs Config
● Think:Think:
• Capistrano, Controltier, ..Capistrano, Controltier, ..
● Pulling stuff from version controlPulling stuff from version control
• Is your repo accessible from production ?Is your repo accessible from production ?
● Ops people like stuf packagedOps people like stuf packaged
• .rpm, .deb, not .tgz , not .war.rpm, .deb, not .tgz , not .war
![Page 26: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/26.jpg)
Why ops like to package Why ops like to package ● Uniquely identify where files come fromUniquely identify where files come from
• Package or cfg-mgmt Package or cfg-mgmt
● Source repo not always availableSource repo not always available
• Firewall / Cloud etc .. Firewall / Cloud etc ..
● Weird deployment locations , no easy accessWeird deployment locations , no easy access
● Little overhead when you automateLittle overhead when you automate
● CONFIG does not belong in a packageCONFIG does not belong in a package
![Page 27: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/27.jpg)
Looking for ?Looking for ?““As a system administrator, I can tell when software As a system administrator, I can tell when software vendors hate me. It shows in their products.”vendors hate me. It shows in their products.”
““DON'T make the administrative interface a GUI. System DON'T make the administrative interface a GUI. System administrators need a command-line tool for constructing administrators need a command-line tool for constructing repeatable processes. Procedures are best documented by repeatable processes. Procedures are best documented by providing commands that we can copy and paste from the providing commands that we can copy and paste from the procedure document to the command line. We cannot procedure document to the command line. We cannot achieve the same repeatability when the instructions are: achieve the same repeatability when the instructions are: "Checkmark the 3rd and 5th options, but not the 2nd "Checkmark the 3rd and 5th options, but not the 2nd option, then click OK." Sysadmins do not want a GUI that option, then click OK." Sysadmins do not want a GUI that requires 25 clicks for each new user.”requires 25 clicks for each new user.”
Thomas A. Limoncelli in ACM Queue December 2010Thomas A. Limoncelli in ACM Queue December 2010
http://queue.acm.org/detail.cfm?id=1921361http://queue.acm.org/detail.cfm?id=1921361
![Page 28: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/28.jpg)
Can you please talk about Drupal now ?Can you please talk about Drupal now ?
![Page 29: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/29.jpg)
Deploying a Drupal SiteDeploying a Drupal Site● Drush_make Drush_make
● ProfilesProfiles
● Features ,strongarm , Custom codeFeatures ,strongarm , Custom code
• Configuration as CodeConfiguration as Code
● What with external apps and libraries?What with external apps and libraries?
• Lucene – Solr – Tika – BI softwareLucene – Solr – Tika – BI software
● Test before you drive! Test before you drive!
![Page 30: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/30.jpg)
Upgrading a siteUpgrading a site
● Drush Drush
• pm-updatecode pm-updatecode
• UpdatedbUpdatedb
• Features-updateFeatures-update
● Once again, test before you drive!Once again, test before you drive!
● But ... what about userdata ?But ... what about userdata ?
![Page 31: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/31.jpg)
ChallengesChallenges● What about the data ?What about the data ?
● Content vs Config Content vs Config
• Drupal provides no clear distinctionDrupal provides no clear distinction
● e.g a billing application e.g a billing application
• Table CreationTable Creation
• Provisioning (e.g. Rates, Call plan types)Provisioning (e.g. Rates, Call plan types)
• Actual Data Actual Data
● D8 ?D8 ?
![Page 32: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/32.jpg)
Deploying a Drupal SiteDeploying a Drupal Site● Drush_make Drush_make
● Profiles ? Profiles ?
• Not reusable for upgradesNot reusable for upgrades
• Drush site-install profileDrush site-install profile
● Features ,strongarm , Custom codeFeatures ,strongarm , Custom code
• Configuration as CodeConfiguration as Code
● D7 Looks BetterD7 Looks Better
![Page 33: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/33.jpg)
Upgrading a siteUpgrading a site
● Drush Drush
• upgrade upgrade
• UpdatedbUpdatedb
• Features-updateFeatures-update
● But ... what about userdata ?But ... what about userdata ?
![Page 34: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/34.jpg)
ChallengesChallenges● What about the data ?What about the data ?
● Content vs Config Content vs Config
• Drupal provides no clear distinctionDrupal provides no clear distinction
● e.g a billing application e.g a billing application
• Table CreationTable Creation
• Provisioning (e.g. Rates, Call plan types)Provisioning (e.g. Rates, Call plan types)
• Actual Data Actual Data
● D8 ?D8 ?
![Page 35: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/35.jpg)
Going Live Going Live
![Page 36: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/36.jpg)
High AvailabiltyHigh Availabilty
![Page 37: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/37.jpg)
ScalabilityScalability
![Page 38: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/38.jpg)
MonitorMonitor
![Page 39: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/39.jpg)
Devops a definitionDevops a definition● There is no definitionThere is no definition
● It certainly isn't a personIt certainly isn't a person
● No strict rulesNo strict rules
● No strict toolsNo strict tools
● It's not even new It's not even new
● If you aren't doing it already ...If you aren't doing it already ...
.. failure is upon you ... .. failure is upon you ...
![Page 40: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/40.jpg)
![Page 41: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/41.jpg)
Surviving the test !Surviving the test !● After 5+ years of After 5+ years of
preaching I`m not preaching I`m not alone anymorealone anymore
● Devops, a new Devops, a new Movement ! Movement !
● Join the movement ! Join the movement !
• Devopsdays.org Devopsdays.org
• Agile System Agile System Adminstration Adminstration GoogleGroupsGoogleGroups
![Page 42: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/42.jpg)
Contact :Contact :Kris Buytaert Kris Buytaert [email protected]@inuits.be
Further ReadingFurther Reading@krisbuytaert @krisbuytaert http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/http://www.inuits.be/http://www.inuits.be/http://www.virtualization.cohttp://www.virtualization.com/m/http://www.oreillygmt.com/http://www.oreillygmt.com/
EsquimauxEsquimauxKheops Business Kheops Business CenterCenterAvenue Georges Avenue Georges Lemaître 54Lemaître 546041 Gosselies6041 Gosselies889.780.406889.780.406+32 495 698 668 +32 495 698 668
InuitsInuits't Hemeltje't HemeltjeGemeentepark 2Gemeentepark 22930 Brasschaat2930 Brasschaat891.514.231891.514.231
+32 473 441 636 +32 473 441 636
![Page 43: Devops For Drupal](https://reader033.vdocument.in/reader033/viewer/2022051111/5549b40bb4c90564768b498a/html5/thumbnails/43.jpg)
Images: Images: http://www.flickr.com/photos/huffstutterrobertl/4135257384/http://www.flickr.com/photos/huffstutterrobertl/4135257384/
http://www.flickr.com/photos/brighton/2153602543/http://www.flickr.com/photos/brighton/2153602543/
http://www.flickr.com/photos/gchorus/2074271352/http://www.flickr.com/photos/gchorus/2074271352/
http://www.flickr.com/photos/49024304@N00/2951673691/sizes/l/http://www.flickr.com/photos/49024304@N00/2951673691/sizes/l/
http://www.flickr.com/photos/30302096@N06/2953698548/http://www.flickr.com/photos/30302096@N06/2953698548/
http://www.flickr.com/photos/jamescridland/613445810/http://www.flickr.com/photos/jamescridland/613445810/
http://www.flickr.com/photos/johnmcga/4468003947/http://www.flickr.com/photos/johnmcga/4468003947/
And some stuff from my previous devops presentation with And some stuff from my previous devops presentation with @patrickdebois@patrickdebois