continuous delivery for databases
Post on 22-Nov-2014
483 Views
Preview:
DESCRIPTION
TRANSCRIPT
Continuous Delivery for DatabasesSQL Relay – Cardiff - 2014
Presented by: James SmithCo-Founder, DevOpsGuys
ABOUT ME
Co-founder @ DevOpsGuys
Two decades building & delivering enterprise web systems
Helped many, many companies implement Continuous Delivery practices
Found High Quality Belgian Beer in late nineties!
Who is this guy?
ABOUT DEVOPSGUYS
We provide
web application management
for customers, giving them access to the expertise necessary for
building, launching, maintaining & optimising applications
allowing them to
accelerate time-to-market
and to
focus on adding value to their business
Who are these guys?
WHY?Why is this important?
WHY?Why is this important?
BUSINESS HAS CHANGED
SPEED MATTERS
8
9
DEPLOYMENT RATES
300 Deployments / Year
50-60 Deployments / Day
10+ Deployments / Day
Every 11.6 seconds
10
HIGH PERFORMERS
HIGH PERFORMING IT ORGANISATIONS DEPLOY 30 TIMES
MORE FREQUENTLY, WITH 50% FEWER FAILURES WITH 8000X
FASTER LEAD TIMES THAN THEIR PEERS.
11
HIGH PERFORMERS
THEY ARE ALSO TWO TIMES MORE LIKELY TO EXCEED PROFITABILITY, MARKET SHARE &
PRODUCTIVITY GOALS
THEY EXPERIENCE 50% HIGHER MARKET CAPITALIZATION GROWTH OVER 3 YEARS
HOW?How have things changed?
12
2010
2012
2001 (1998)
2001 (and earlier)
DEV HAS CHANGED
WE KNOW AGILE WORKS
15
Plan
Code
Build
Test
Release
Deploy
Operate
CONTINUOUS X?
Continuous Integration
Continuous Delivery
Continuous Deployment
CONTINUOUS DELIVERYWhat is it?
REGULAR, RAPID, RELIABLE AND CONTROLLED DELIVERY OF WORKING SOFTWARE SYSTEMS INTO PRODUCTION
Matthew Skelton
REGULAR, RAPID, RELIABLE AND CONTROLLED DELIVERY OF WORKING SOFTWARE SYSTEMS INTO PRODUCTION
Matthew Skelton
Nhan Ngo, a QA engineer at Spotify
20
CONTINUOUS DELIVERY
1. Build Management & CI
2. Environments & Deployment
3. Release Management & Compliance
4. Testing
5. Data Management
22
BUT DATABASE CHANGE IS SCARY
Photo Credit: http://cache.lego.com/r/www/r/movie/-/media/franchises/the%20lego%20movie/explore/downloads/wallpapers/lego_wps_1600_emmet.jpg?l.r=-1410608028
NO, THIS IS SCARY
Photo Credit: ?
SO WE MINIMISE RISK
Photo Credit: https://ideas.lego.com/projects/62456
WE DEFEND THE THREAT
Photo Credit: "Rumrunner" via Compfight cc
26
WE STOP CHANGE
Photo Credit: s.kosoris via Compfight cc
27
CUSTOMERS GET SAD
Photo Credit: Kalexanderson via Compfight cc
28
THE BUSINESS GETS ANGRY
Photo Credit: powerpig builds via Compfight cc
29
AND IT GET THE BLAME!
Photo Credit: bobsfever via Compfight cc
30
YES BUT….
Dev are shipping application changes – frequently
Ops are creating servers - automatically
WHY IS THERE SO MUCH LEGO IN YOUR SLIDE DECK?
Perfectly normal question
LEGO = DEVOPS
Rapid proto-typing & experimentation
Building blocks – no right or wrong way
Promotes collaboration
Strong cultural appeal
Small batch sizes
Visibly measureable
We’ve even automated it! Manufacturing Zenon Mindstorms
Danish phrase leg godt, which means "play well".
WHY?Why is database automation so hard?
!= CTRL C, CTRL V
Database deployment is not copying and replacing.
It is the transformation from a previous version to the next version while preserving data integrity.
Deploying database change is hard
Deploying database change frequently is even harder
WHAT?What are the common challenges?
#1: VERSION CONTROL
Photo Credit: ntr23 via Compfight cc
VERSION CONTROL
Database changes not under VCS
Worse – Changes not “always” committed to VCS
Communication of change
Living in “the sea of ‘branch/merge’ filth”
38
#2: AGAINST THE FLOW
Photo Credit: slang589 via Compfight cc
Photo Credit: slang589 via Compfight cc
39
AGAINST THE FLOW
DEV PRODUCTION
DEV PRODUCTION
CODE
DATA
40
#3: COMPLEXITY
Photo Credit: L@go via Compfight cc
COMPLEXITY
Dependence
Centralisation
#4: DATA VOLUME
Photo Credit: http://www.calgaryherald.com/news/calgary/Gallery+LEGO+KidsFest/9848406/story.html
DATA VOLUME
Transit time
Historical data – no archiving
One of the biggest impacts on Cycle time
#5: CONWAY'S LAW
Photo Credit: kirk_arts via Compfight cc
45
CONWAY'S LAW
"Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design
whose structure is a copy of the organization's communication structure.“
- Conway, 1968
CONWAY'S LAW
Is (was) a database really needed? 3 Tiers anyone?
Centralised vs Decentralised?
Formalised [change] control
HOW?So what are some of the solutions?
#1: GET UNDER CONTROL
Photo Credit: RHiNO NEAL via Compfight cc
VERSION CONTROL
It’s code – it should be in VCS!
It’s code – it should be in VCS!
It’s code – it should be in VCS!
Schema & Static/Reference Data
Reverse engineer existing schema & reference data
50
#2: INTEGRATE CONTINUOUSLY
Photo Credit: ....Tim via Compfight cc
51
CONTINUOUS INTEGRATION
Automate your build steps (Full vs Incremental)
Build a library of manual tests
Automate your tests
52
#3: AUTOMATE TESTING
Photo Credit: s3aphotography via Compfight cc
53
TEST, TEST, TEST
Select the right tests for each stage;Unit testing Integration TestingDeployment ValidationBehaviour Validation
Determine the right data for testingDo you need it all?
54
#4: AUTOMATION
Photo Credit: pasukaru76 via Compfight cc
55
AUTOMATE ALL THE THINGS
Testing
Deployment
Back-up and more importantly restore
Archiving
Rollback
56
#5: MICROSERVICES
57
AS A SERVICE
Treat your database as a service
Contracts – that you do not break (lightly)
58
RE-ARCHITECT
Split data along sensible partitions
Polyglot persistence
59
POLYGLOT PERSISTENCE
60
#6: CHANGE YOUR WAYS
Photo Credit: Stéfan via Compfight cc
61
IT’S CULTURE
Technology is only half of the story
Dev’s must work with DBA’s (no silo’s)
Management must think of operations as part of development
Deployment is part of development
Data retention is part of development
Fail faster, but fail safely
TOOLINGCan tools help?
62
THE PIPELINESource
Control
Integration
TestDeploy
64
IN REALITY
Development
Plan Code Build Test Release Deploy Operate
Operations
SOURCE CONTROL
Source control is the foundation of Continuous Integration
Plus many, many more
66
CONTINUOUS INTEGRATION
Continuous Integration is the common orchestration point
67
DEPLOYMENT
Repeatable and consistent deployments every time.
MONITORING
Continuous delivery elevates the need for monitoring in production
THE TRUTH
Continuous Delivery relies on having 2 basic things;
1. Version Control
2. Automation
70
AND FINALLY…
Customers see results and new features more quickly.
Shorter feedback cycles increases our ability to learn.
Improve the whole system.
Reduce firefighting.
Everyone wins!
71
EVERYBODY WINS!
Photo Credit: Kalexanderson via Compfight cc
72
THAT’S ALL FOLKS
Photo Credit: Walter Benson via Compfight cc
top related