building an automated database deployment pipeline
DESCRIPTION
The pace of business accelerates fairly continuously and application development moves right with it. But we’re still trying to deploy databases the same way we did 10 years ago. This session addresses the need for changes in organizational structure, process and technology necessary to arrive at a nimble, fast, automatable and continuous database deployment process. We’ll use actual customer case studies to illustrate both the common methods and the unique context that led to a continuous delivery process that is best described as a pipeline. You will learn how to customize common practices and tool sets to build a database deployment pipeline unique to your environment in order to speed your own database delivery while still protecting your organization’s most valuable asset, it’s data.TRANSCRIPT
BUILDING AN AUTOMATED DATABASE DEPLOYMENT PIPELINE
Grant Fritchey
Red Gate Software
Continuous delivery for databases
Goals
Understand the technology and process requirements to work towards automation step-by-step in your release pipeline.
Learn about the organizational changes necessary to support process modifications.
Appreciate why these changes are necessary in support of modern development and deployment methodologies.
[email protected] www.scarydba.com
@gfritchey www.linkedin.com/in/scarydba
Grant Fritchey
Product Evangelist, Red Gate Software
ALM – and where the database fits in
Three core processes in Application Lifecycle Management:
Governance Development Operations
Natural friction across pipeline
Development → Operations
Natural friction across pipeline
Development → Operations
Operations → Development
Why?
Development focus is on speed
Agile
Scrum
Lean
Feature-driven Development
Iterative
Operations focus is on production protectionprotection
Monitoring
Deployment
Integrity
Data Management
Databases as a bottleneck – historically
Odd languages
SQL
Cubes
X-Query
3 reasons why databases have traditionally slowed down
deployments:
Data persistence
Data outlives
applications
Data can’t be
replaced
DBA paranoia
Frankly…
1 2 3
A quick primer on continuous delivery
Common misconception: it’s about automating to Production
A quick primer on continuous delivery
Common misconception: it’s about automating to Production
In fact –
there’s a difference between three continuous approaches:
DevelopmentSource control CI server Test Production
Continuous integration
Continuous deployment
Continuous delivery
The goals of continuous delivery for
databases
Faster feedback on changes made
o Continuously integrate team changes
o Automated testing
o Releases rehearsed in testing environments before
deployed to Production
= repeatability, repeatability and
a strong team process for changes
Focus on the pipelineThink of the Toyota production system…
Start with Lean
o Focus on the customer, eliminate waste
o Continuously Improve
o Empower the team
o Optimize the whole
o Plan for change
o Automate processes
o Build quality in
SOURCE
CONTROL
CONTINUOUS
INTEGRATION:
FUNDAMENTALS
CONTINUOUS
INTEGRATION:
ADVANCED
AUTOMATED
DEPLOYMENT
Four key stages of the deployment pipeline
SOURCE
CONTROL
Greenfields
Existing systems
Test data
Four key stages of the deployment pipeline
Automate builds as first step
o Incremental
o Complete
Build library of manual tests
Automate testsCONTINUOUS
INTEGRATION:
FUNDAMENTALS
Four key stages of the deployment pipeline
Select CI tooling
Team discipline
Customization of build steps
Automate the testingCONTINUOUS
INTEGRATION:
ADVANCED
Four key stages of the deployment pipeline
Testing
Automation
Safety
o Backups
o Rollback strategyAUTOMATED
DEPLOYMENT
Four key stages of the deployment pipeline
Create development environment
for automationEmpower Development
Environment should let them work at their speed
Take part in Development Instead of stopping bad deployments, stop bad development
Automate your build process Supply clean production data or supply good sample data
Get started on writing tests Build your library
Different types of testing for different stages of the pipeline:
DevelopmentIntegration
TestingQA
Pre-Production/Staging
Production
Automate testing
Unit tests Integration tests
Automated tests
Deployment validation
Behaviour validation
Other validations
B C D
A
Always Be Continuously Delivering
Deliver
Every delivery is practice
Fail faster
But less frequently
Smooth the process
Testing, testing, testing
Select the right tests for each stage
Review and act on the results
Automate
DBAs must work with Devs…
Yes, you must protect the data for the business, but
that must be tempered with helping deliver functionality.
Changes to philosophy
Devs must work with DBAs…
Yes, you may know more about business needs,
so educate rather than isolate.
Changes to philosophy
Project Management must think of operations as part
of development…
Yes:
Deployment is part of development
Release 1.1 and on are part of development
o And planning for 1.1 is not premature optimization
Data retention is part of development
Changes to philosophy
Changes to workplace
SOURCE
CONTROL
CONTINUOUS
INTEGRATION:
FUNDAMENTALS
CONTINUOUS
INTEGRATION:
ADVANCED
AUTOMATED
DEPLOYMENT
Changes to workplace for continuous
database delivery
Empower developers to do more
Bring DBAs into development teams
Management must buy-in
Management must get out of the way
How are other companies getting started?
Case Study: Boxon
Global packaging and labelling company
Based in Sweden; sub-division in China
Three business areas: profitable packing
solutions; intelligent marking; customized
big-bags solutions for bulk handling
One developer responsible for ASP.NET
application with SQL Server backend,
enabling customers to control consistent
printing of labels wherever the print shop
is located in the world
Boxon – Initial Process
Deploying to production was scary…
Database changes not version-
controlled
2-hour window - at night - to
complete deployments to production
– difficult to find quiet period with a
global customer basePAIN
POINTS
Boxon – Improved Process Deployments are less scary now…
Tickets are logged and prioritized in Unfuddle(unfuddle.com)
Ticket numbers are logged in SVN and TeamCity to track items
Now use SQL Source Control and Subversion to version DB changeso Source of authority on database build
TeamCity (CI build server) is triggered on check-in of changeso Fast response – changes checked in frequently
Notification if build fails
Build packaged into a Nuget file for deployment
BUILDING A
PIPELINE
Case Study: Move with Us
Risk management solution and
sales and marketing channel
provider for residential property
businesses in the UK
Located near Cambridge, UK
Customers put a heavy load on
databases, so updates need to
be efficient
Move with Us – Initial Process
Database code often not checked in to VCS due to risk of conflicts –changes were communicated by email instead
Hot fixes sometimes done directly on integration server
1 day needed to release updates –a lot of work in building and testing
Problems with merging code led to a lot of firefighting
PAIN
POINTS
Move with Us – Improved Process
Each developer has a local copy of database
SQL Source Control and Subversion used to check in database changes to version control
TeamCity (CI build server) automatically compiles changes on check-in
Successful builds packaged into a Nuget file
automaticallyo Fast getting here – multiple database changes
checked in per day
Nuget package deployed to Test on demand
If problems detected, return to Dev to fix
BUILDING A
PIPELINE
Case Study: Practice Fusion
Web-based electronic health record (EHR) company
Founded in 2005
Based in San Francisco
Hosts over 50 million patient records
Used in all 50 states and by 150k+ physicians
3 DBA engineers; 1 data architect; 10+ developers
Practice Fusion – Initial Process
Hand-crafted SQL scripts - inconsistent
Database changes not always version controlled
Changes released by opening dozens of files in SSMS
Long-winded team comms if further changes needed
Smoke-testing deployments showed some objects had been deployed straight to Prod and weren’t in deployment script
PAIN
POINTS
Practice Fusion – Improved Process
SQL Source Control and Subversion to version DB changeso Source of authority on database build
Jenkins (CI build server) is triggered on check-in of changeso More time to develop; less time managing scripts
Build failed by Jenkins if problems detectedo Fewer issues deploying to production
Jenkins, SQL Compare and SQL Data Compare used to deploy changes to environments
Custom scripts for replication
BUILDING A
PIPELINE
Summary
Eliminate or mitigate frictiono Slow is smooth, smooth is
fast
Adopt lean methodologiesand focus
Four key stageso Source control
o Build automation
o Continuous Integration
o Continous Deployment
A-B-C-D
Change your philosophy
Change your workplace
Goals
Understand the technology and process requirements to work towards automation step-by-step in your release pipeline.
Learn about the organizational changes necessary to support process modifications.
Appreciate why these changes are necessary in support of modern development and deployment methodologies.
Documentation and resources
Continuous Delivery by Jez Humble and David Farley (Addison Wesley)
The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win by Gene Kim, Kevin Behr and George Spafford (IT Revolution Press)
The Goal: A Process of Ongoing Improvement by Eliyahu M. Goldratt and Jeff Cox (Gower Publishing Ltd.)
Agile Organization by the agile admin (theagileadmin.com)
Further resources:
Database Delivery Learning program: www.red-gate.com/delivery
o Patterns and practices on Simple-Talk
o Tutorials in Red Gate training academy
www.youtube.com/user/RedGateVideos - for recorded seminars
Image sourcesAuthor Source Information
Chiltepinster Wikimedia Commons Mocking Bird Argument.jpg – Wikimedia Commons. This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license. Source on Wikimedia Commons: “Own work”
Tableatny Wikimedia Commons Athlete at Starting block.jpg – Wikimedia Commons. This file is licensed under the Creative Commons Attribution 2.0 Genericlicense. Source on Wikimedia Commons: “BXP135671”
n.raveender Wikimedia Commons Stop No Entry.jpg – Wikimedia Commons. This file is in the public domain. Source on Wikimedia Commons: “Own work.”
Original uploader was Lifaceat en.wikipedia
Wikimedia Commons Cross Country US.jpg – Wikimedia Commons. This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license. Source on Wikimedia Commons: “Transferred from en.wikipedia; transfer was stated to be made by User:TFCforever.” Notes on license: Liface at the English language Wikipedia, the copyright holder of this work, hereby publishes it under the following license:GNU head Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.Subject to disclaimers.
Horia Varlan Flickr Graduated cylinders and beaker filled with chemical compounds – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.
Henry Mühlpfordt Flickr CERN Atlas Control Room 2010-07-01 – Flickr. This file is licensed under the Creative Commons Attribution-ShareAlike 2.0 Generic license.
Department for Business, Innovation and Skills
Flickr Toyota’s new Auris – Flickr. This file is licensed under the Creative Commons Attribution-NoDerivs 2.0 Generic license.
Qrodo Photos Flickr Sport Action – Fencing – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.
William Warby Flickr Gears – Flickr. This file is licensed under the Creative Commons Attribution 2.0 Generic license.