cdSummitContinuous Delivery Seminars
Automating the deployment process9th September 2014
A bit about me…
• Co-founder and Director
• Worked in Deployment Automation Solutions in
regulated environments for 10 years
• With MidVision since 2008
• A toolsmith at heart
Gartner think’s we’re cool
• Born in a Bank – regulation, risk and compliance
• Application Release Automation Platform, RapidDeploy deploy
environments AND applications at scale
• Cross industry customer success – proven in large complex
regulated enterprises
• Core Platform – with ability to integrate to Open Source and
Commercial tool chains
• Gartner DevOps Cool Vendor
Customers
Agenda
• Relationship between DevOps, CD and Automation
• Desired state, Packaging and versioning
• Deployment models
• Toolchains
• Automation and self service
DevOps, Continuous Delivery and Automation
DevOps
(Philosophy)
Continuous Delivery
(Practice)
Application Release
Automation (ARA)Application Release Automation
a.k.a. Deployment Automation
“The process of packaging and deploying an
application or update of an application from
development, across various environments,
and ultimately to production”
Application Deploy
Infrastructure Automation
Image Provisioning
Release Management
Application Configuration
APPLICATION DEPLOYMENT
Continuous Delivery / Release Pipelines
Automated Deployment Principles
• Version control and build configuration
• Release versions of configuration change in the
same way as code
• Allows linking of configuration releases and
application releases – together comprising a
business application
• “Build Once, Deploy Anywhere”
8
Model Driven Deployment
• Model defines complete configuration for a logical
unit of automation
• Define similarities and differences
• Always contain the complete configuration for a
target technology
• Deltas become part of the template – so
configuration changes are not made in isolation
• Idempotent
9
Properties / Template
MQSC Template
------<QMGR_NAME>--------
------<QMGR_HOST>--------
------<QMGR_PORT>--------
DevTst01
QMGR_NAME SALESDEVQM01
QMGR_HOST lnxdev01
QMGR_PORT 1414
10
Properties / Template
MQSC Template
------<QMGR_NAME>--------
------<QMGR_HOST>--------
------<QMGR_PORT>--------
DevTst01
QMGR_NAME SALESDEVQM01
QMGR_HOST lnxdev01
QMGR_PORT 1414
SysTst01
QMGR_NAME SALESTSTQM02
QMGR_HOST lnxtst01
QMGR_PORT 1415
11
Properties / Template
MQSC Template
------<QMGR_NAME>--------
------<QMGR_HOST>--------
------<QMGR_PORT>--------
------<DB_USER>--------
DevTst01
QMGR_NAME SALESDEVQM01
QMGR_HOST lnxdev01
QMGR_PORT 1414
DB_USER devtstuser
SysTst01
QMGR_NAME SALESTSTQM02
QMGR_HOST lnxtst01
QMGR_PORT 1415
DB_USER syststuser
Business Appliationdev_area \DeployDesc
\database(s)\source
Infrastructure ‘Desired State’release_area \jee
\scripts\static\database
Baseline File: SALESAPP_BUILD_01_02_003
DEPLOY PACKAGE
SALESAPP_BUILD_01_02_003.zip
Creating the Deploy Package
Implementation-Version=SALESAPP_BUILD_01_02_003
DevOps Toolchain
`
Release Manager
Directed Deployment IHS
WAS
MQ
cluster/ds/jms
httpd.conf
qmgr/queue
httpd.conf
IHS
WAS
MQ
MQ
WAS
IHSIHS
WAS
MQ
cluster/ds/jms
httpd.conf
qmgr/queue
cluster/ds/jms
httpd.conf
qmgr/queue
cluster/ds/jms
qmgr/queueImplementation Manager selects:
• Application
• Environment
• Version
SALESAPP_BUILD_01_02_003.tarSALESAPP_BUILD_01_02_004.tar
Composite Deployment
DevOps Toolchain
17
Version
Control
Configuration
Management
Build Process Release Mechanism Targets
Code
Configuration
Commandline /
WebUI / REST /
Eclipse
Generate Versioned Release
Components
Ideally web based, self
service
Web / Application /
Database / etc
18
Version
Control
Configuration
Management
Build Process Release Mechanism Targets
Code
Configuration
Commandline /
WebUI / REST /
Eclipse
Generate Versioned Release
Components
Ideally web based, self
service
Web / Application /
Database / etc
x.mqsc Alter channel
19
Version
Control
Configuration
Management
Build Process Release Mechanism Targets
Code
Configuration
Commandline /
WebUI / REST /
Eclipse
Generate Versioned Release
Components
Ideally web based, self
service
Web / Application /
Database / etc
x.mqsc Alter channel SALES_MQ_01_02_033.zip
20
Version
Control
Configuration
Management
Build Process Release Mechanism Targets
Code
Configuration
Commandline /
WebUI / REST /
Eclipse
Generate Versioned Release
Components
Ideally web based, self
service
Web / Application /
Database / etc
x.mqsc Alter channel SALES_MQ_01_02_033.zip SALES_MQ_01_02_033 MQ
21
Version
Control
Configuration
Management
Build Process Release Mechanism Targets
Code
Configuration
Commandline /
WebUI / REST /
Eclipse
Generate Versioned Release
Components
Ideally web based, self
service
Web / Application /
Database / etc
x.mqsc Alter channel SALES_MQ_01_02_033.zip SALES_MQ_01_02_033 MQ
x.py JMS Queue Dest
22
Version
Control
Configuration
Management
Build Process Release Mechanism Targets
Code
Configuration
Commandline /
WebUI / REST /
Eclipse
Generate Versioned Release
Components
Ideally web based, self
service
Web / Application /
Database / etc
x.mqsc Alter channel SALES_MQ_01_02_033.zip SALES_MQ_01_02_033 MQ
x.py JMS Queue Dest SALES_JBOSS_01_02_046.zip
23
Version
Control
Configuration
Management
Build Process Release Mechanism Targets
Code
Configuration
Commandline /
WebUI / REST /
Eclipse
Generate Versioned Release
Components
Ideally web based, self
service
Web / Application /
Database / etc
x.mqsc Alter channel SALES_MQ_01_02_033.zip SALES_MQ_01_02_033 MQ
x.py JMS Queue Dest SALES_JBOSS_01_02_046.zip SALES_JBOSS_01_02_046 JBoss
24
Version
Control
Configuration
Management
Build Process Release Mechanism Targets
Code
Configuration
Commandline /
WebUI / REST /
Eclipse
Generate Versioned Release
Components
Ideally web based, self
service
Web / Application /
Database / etc
x.mqsc Alter channel SALES_MQ_01_02_033.zip SALES_MQ_01_02_033 MQ
x.py JMS Queue Dest SALES_JBOSS_01_02_046.zip SALES_JBOSS_01_02_046 JBoss
x.ddl Column, View, etc
25
Version
Control
Configuration
Management
Build Process Release Mechanism Targets
Code
Configuration
Commandline /
WebUI / REST /
Eclipse
Generate Versioned Release
Components
Ideally web based, self
service
Web / Application /
Database / etc
x.mqsc Alter channel SALES_MQ_01_02_033.zip SALES_MQ_01_02_033 MQ
x.py JMS Queue Dest SALES_JBOSS_01_02_046.zip SALES_JBOSS_01_02_046 JBoss
x.ddl Column, View, etc SALES_DB_01_02_012.zip
26
Version
Control
Configuration
Management
Build Process Release Mechanism Targets
Code
Configuration
Commandline /
WebUI / REST /
Eclipse
Generate Versioned Release
Components
Ideally web based, self
service
Web / Application /
Database / etc
x.mqsc Alter channel SALES_MQ_01_02_033.zip SALES_MQ_01_02_033 MQ
x.py JMS Queue Dest SALES_JBOSS_01_02_046.zip SALES_JBOSS_01_02_046 JBoss
x.ddl Column, View, etc SALES_DB_01_02_012.zip SALES_DB_01_02_012 Oracle
27
AppServer - products
Apache HTTP
Jboss 6.2
RabbitMQ
RHEL 6 Update 5
Database - products
Oracle 11.1.0.7.0
RHEL 6 Update 5
Provisioning New EnvironmentsProducts
28
AppServer - products Configuration / App
Apache HTTP WEB_CONFIG_SALES_1_2_056
Jboss 6.2 JBOSS_CONFIG_SALES_1_2_087
RabbitMQ MQ_CONFIG_SALES_1_2_032
RHEL 6 Update 5 STANDARD ORG IMAGE
Database - products Configuration / App
Oracle 11.1.0.7.0 ORA_CONFIG_SALES_1_2_042
RHEL 6 Update 5 STANDARD ORG IMAGE
Provisioning New EnvironmentsProducts > Configuration
29
SalesRelease 1.2
AppServer - products Configuration / App
APP_CONTENT_SALES_1_2_124
Apache HTTP WEB_CONFIG_SALES_1_2_056
Jboss 6.2 JBOSS_CONFIG_SALES_1_2_087
RabbitMQ MQ_CONFIG_SALES_1_2_032
RHEL 6 Update 5 STANDARD ORG IMAGE
Database - products Configuration / App
Oracle 11.1.0.7.0 ORA_CONFIG_SALES_1_2_042
RHEL 6 Update 5 STANDARD ORG IMAGE
Provisioning New EnvironmentsProducts > Configuration > Code
Continuous Delivery Overview
Packaging Principles, Build Once, Deploy Anywhere… when does Continuous Beer start!
Packaging PrincipleBuild Once, Deploy Anywhere
• Build once, deploy to any (defined) environment in the pipeline.
• A package should be a single compressed, versioned file.– Package integrity across all environments
– Check-summed and labeled
• The package should contains all deployment instructions, environment configuration and resources (or address locations to remote resources).
• Package should be runnable as a standalone process.- not dependent on the deployment tool.
• Store and retrieve deployment artifacts in a DSL.
• Use Jenkins and to build your deployment package and store it in the DSL.
Packaging Model for Deployment Artifacts
• Topology of different environments can define the packaging model to use.
• Single App / Single Cluster – Deploy application and configuration together
• Multiple App / Multiple Cluster– Deploy applications and configurations separately
Deployment PackageApp Config
Deployment Package 1App 1
Deployment Package 2ConfigApp 2
Prerequisite Resource & Environment Testing
• Define which environment resources need testing:– file permissions– disk space– port availability– user accounts
• Do not continue with the deployment if the prerequisite tests fail.• Automating prerequisite testing of resources saves a lot of pain and time.• Keep adding tests to the deployment as you get a better understanding of
the environment.• Sometimes known as a “Deployment Dry Run”.
Automated Testing Strategy
• Use Jenkins post build plugins to initiate the tests after successful a deployment.
• Building pipelines can be quick, converting your manual tests to automated tests can be laborious and take months…..stick with it.
Rollback Strategy• Rollback strategy depends on the technology of the
deployment.• Database changes cannot be rolled back automatically
if business data has been added or amended since the deployment took place. Need a DBA.
• Make sure DB changes are backwards compatible.• Generally with databases, use a fix forward strategy.• Middleware can be rolled back if the code is deployed
with the configuration.
Metrics, Analytics & Feedback Loop
• Measure the release process.– Deployment times– No. of deployments (successful & failed).– Velocity
• Report on the release process measurements.• Show management the improvements CD is making to your organization.• Fully automating CD is not cheap. Use the metrics to get management on-
side with investing in CD.• Make use of the CI and CD metrics from Jenkins to help with the analytics.
What are we trying to achieve?
Faster(Acceleration)
We automate processes carried out by IT
Faster deployments through whole lifecycle
Bring products to market quicker
Safer (Lower Operational Risk)
Eliminate outages
Workflow-scheduling
Audit & compliance controls
Cheaper(Reduce TCO)
More resource time for value-creation
Why outsource when you can automate
Continuous Delivery ….e2e Pipeline
• Continuous Integration
• Automated Unit Testing
• Artifact creation
• Automated Acceptance testing /
Approval
• Automated Deployment
AutomationAutomate your tool chainempower your team
III