seacon continuous delivery pipeline tools track

SEACON:UK 2017 Breakout Track

Exploring different approaches to Continuous Delivery Pipelines

Diversion into Inclusion & Diversity1.It should be down to all of us to champion this at every

opportunity.2.We all suffer from Unconscious Bias and even if this has just a

0.1% on the opportunities that we give others, cumulatively this stacks up. (Take a test: https://implicit.harvard.edu/implicit/selectatest.html)

3.Mirco aggression is a term for behaviour that intimidates others and make it harder for them to perform a their best in the working environment. Be vigilant and call it out. (Real examples here: http://www.microaggressions.com/)

Other links:• https://www.fastcompany.com/3044738/hit-the-ground-running/7-simple-methods-to-fight-against-your-

unconscious-biases• https://codeascraft.com/2016/10/19/being-an-effective-ally-to-women-and-non-binary-people/• http://weprinciples.org/files/attachments/

WHITE_PAPER_Nudging_the_Unconscious_Mind_Nielsen_&_Kepinski_Oct_2014.pdf• https://managingbias.fb.com/• http://www.cmu.edu/news/stories/archives/2015/june/bias-blind-spot.html

Agenda• Introduction to Continuous Delivery In Practice

• This Exercise

• XebiaLabs - XL Deploy and XL Release

• ADOP and Jenkins

• Thoughtworks GoCD

• General Q&A

The Objective: Continuous Delivery of Ideas

FEEDBACK

BuildStatic Code Unit Tests

Source CodeRepo

CI pipeline

Deploy Code

Run Test Harness

Deploy CodePPE

Perf,Security

TestDeploy

CD pipeline(rest of the way to Prod)

Automating the Workflow from Commit to Release

The Workflow needs to hold up

How do we control flow?

What versions

were tested

together?

What should

we release?

Tests configurationBuild tools configuration

Deployment software configurationMiddleware configuration

Operating System configuration Virtual infrastructure configuration

Hardware configuration

Business

Application

Business

Application

Business

Application

Business

Application

Business

Application

Compile Code AnalysisCommitter: jdoe

Story:25

Commit ID: 113 Unit Test Deploy PPE Functional Test Security and Perf Test

Story:25

We should probably do this all the way down!

Commits of any Source Code

triggers a pipeline

One pipeline won’t work - the tools need handle complexity

BuildStatic Code Unit Tests

Deploy Code

Run Test

Harness

Deploy CodePPE

Perf,Securit

y Test

Deploy Prod

Luckily in 2017 they all do now support a lot things e.g.:

• Agents for secure remote execution on different platforms

• Application security

• Role Based Access Control

• Reporting and metrics

• Automated and manual triggers and flow control

• Commercial support

• Extensibility

But how do they differ?

THE EXCERCISE

Application DesignReference Application Stack• Frontend App• Backend App (tightly coupled to frontend – both must be released

together)• Configuration Management = Chef or Puppet• Provisioning = TerraformOrchestration & Automation Scripts = (you choose) Tools• Software configuration Management = (you choose but not of

concern)• Binary storage = (you choose but not of concern)• Testing = (you choose and but of concern)• Monitoring = (you choose but not of concern)• Automation of build = (you choose but not of concern)• IaaS platform = (you choose but not of concern)Automation of deployment = (you choose)Continuous Delivery Pipeline = (you choose)

Example PipelineFrontend

Code, Config

BackendCode, Config

Config MgmtScripts

ProvisioningScripts

Orchestration& Automation

Scripts

Build & Package

Unit TestCode Analysis

Build & Package

Unit Test &Code Analysis

Sandbox Testing(various)

Install &ConfigureNon-Prod

Runtime Testing(various)

Install &ConfigureProduction

ProvisionNon-Prod

ProvisionProduction

UpdateJobs /Flows

MonitoringBackupHealingetc.

System TestDeploy

Install &ConfigureSystem Test

Non-ProdDeploy

ProductionDeploy

ProvisionSystem Test

FrontendCode, Config

BackendCode, Config

Config MgmtScripts

ProvisioningScripts

Scripts

Build & Package

ProvisionNon-Prod

ProvisionProduction

UpdateJobs /Flows

System TestDeploy

Non-ProdDeploy

ProductionDeploy

Visualisation and release Tracking1. How can “what is deployed where” be visualised and tracked (over time) and how can tested combinations of components be tracked and locked-in?

2. How can it be enforced that a consistent use of automation is used across deployments to different environments?

BackendCode, Config

Config MgmtScripts

ProvisioningScripts

Scripts

Build & Package

ProvisionNon-Prod

ProvisionProduction

UpdateJobs /Flows

System TestDeploy

Non-ProdDeploy

ProductionDeploy

Developing and Testing Automation3. What does the activity of creating automation and orchestration entail (Coding, DSL, Drag and Drop etc.)?

4. How is automation and orchestration tested and new versions released?

5. How can people share and re-use automation packages?

BackendCode, Config

Config MgmtScripts

ProvisioningScripts

Scripts

Build & Package

ProvisionNon-Prod

ProvisionProduction

UpdateJobs /Flows

System TestDeploy

Non-ProdDeploy

ProductionDeploy

Build & Package

System TestDeploy

Operational Concerns7. How are the Continuous Delivery Pipeline tools upgraded / patched and typically how often does this happen and why?

6. How can the platform be extended e.g. Via Plugins?

The main event!

• Ben Hählen – Xebia Labs – XL Deploy and XL Release

• Hitesh Joshi – Accenture – ADOP and Jenkins

• Marius Ciotlos – Betfair – Thoughtworks GoCD

XL Release and XL Deploy

XL Release

XL Deploy

To facilitate comparison we would like to propose the following scenario. The objective of the exercise is to design (no need to implement) a solution for achieving continuous delivery of the following.

Reference Application Stack• App 1 = Front-End• App 2 = Back-End• Configuration Management = Chef or Puppet• Provisioning = Terraform Orchestration & Automation Scripts = XL Release Tools• Software configuration Management = Git, SVN• Binary storage = Nexus, Artifactory, etc• Testing = Sonar, Cucumber, etc• Monitoring = ELK, Splunk, etc• Automation of build = Jenkins, VSTS, TFS, etc• IaaS platform = On-premise, Azure, Kubernetes, Docker, etc Automation of deployment = XL Deploy Continuous Delivery Pipeline = XL Release

Application Design

1. How can “what is deployed where” be visualised and tracked over time and how can tested combinations of components be tracked and locked-in?

2. How can it be enforced that a consistent use of automation is used across deployments to different environments?

3. What does the activity of creating automation and orchestration entail (coding, DSL, drag and drop etc.)?

4. How is automation and orchestration tested and new versions released?

5. How can people share and re-use automation packages?

6. How can the platform be extended e.g. via plugins?

7. How are the Continuous Delivery Pipeline tools upgraded / patched and typically how often does this happen and why?

Short Term Support : Every 2 months

Long Term Support : Every 6 monthsMaintenance Release : Every 2 months

THANKS

About Me

RAPID MOBILISATION

GLOBAL SHARED SERVICES TEAM

SUPPORTING 24 / 7

Software Configuration Management

Release ManagementAssessments

Environment Management

DevOps TransformationBuild & Deploy Automation Continuous Delivery

Rational and Jira

Infrastructure as code CloudPaaS Tooling

Jenkins

Jenkins with ADOP• All open source• Commercial Jenkins option though CloudBees• Accenture created ADOP so we can help

1: December 2016: https://www.cloudbees.com/jenkins/about

Stats• 150,000 Jenkins active installations, 1M users, 12.5M jobs (1)

• Jenkins plugins: 1200+• Jenkins Pipeline installations: 77,000+• ADOP: ~550 forks + ~200 stars• ADOP plugins (cartridge) > 50

On Premise

Docker Machine

Docker Compose

ADOP Platform Extension

ADOP Cartridge

AWS Cloud

AWS regionAvailability

Internet

Selenium

Platform for Reusable Automation

Cartridges

Solution ComponentsReference Application Stack• App 1 = Hybris• App 2 = SOLR• Configuration Management = Chef or Puppet• Provisioning = Terraform Orchestration & Automation Scripts = ADOP, Jenkins, Ansible

Tools• Software configuration Management = Git• Binary storage = Nexus (with Lifecycle)• Testing = Cucumber, Selenium Grid• Monitoring = Sensu and ELK• Automation of build = Ant and Maven• IaaS platform = AWS Automation of deployment = Ansible Continuous Delivery Pipeline = ADOP, Jenkins

Visualisation and release tracking1. How can “what is deployed where” be visualised and tracked over time and how can tested combinations of components be tracked and locked-in?

Components

Environments

Visualisation and release tracking2. How can it be enforced that a consistent use of automation is used across deployments to different environments?

Developing and Testing Automation3. What does the activity of creating automation and orchestration entail (coding, DSL, drag and drop etc.)?

Developing and Testing Automation5. How can people share and re-use automation packages?

A Cartridge encapsulates a reference complete implementation for a particular technology or product, typically consisting of:• Reference application• Build/Deploy Pipeline• Environment Creator – depending on technology

Hybris Cartridge

CreateEnvironment(AWS EC2,

RDS) Base Hybris Build

(Platform + Code) Deploy Hybris

Docker image with config +

Developing and Testing Automation4. How is automation and orchestration tested and new versions released?

Same Continuous Delivery Pipeline for Jenkins and Ansible

Operational Concerns6. How can the platform be extended e.g. via plugins?

Operational Concerns7. How are the Continuous Delivery Pipeline tools upgraded / patched and typically how often does this happen and why?

THANKS

seacon continuous delivery pipeline tools track

Technology

seacon sustainable concrete using seawater, salt

a continuous dataflow pipeline for low latency...

seacon logistics

seafloor construction experiment, seacon ii an instrumented...

scrum, kanban und (r)evolutionäre veränderungen - seacon...

continuous delivery: automating the deployment pipeline ·...

an almost complete continuous delivery pipeline including...

anatomy of a continuous integration and delivery (cicd)...

building a continuous delivery pipeline with gradle and...

pipeline: continuous delivery as code in jenkins 2.0

bursting the pipeline: evolving continuous delivery at...

story in the slaughterhouse (seacon 2012)

altitude sf 2017: building a continuous deployment pipeline

architecture reconstruction via dynamic...

best practices for continuous pipeline development€¦ ·...

10 questions to ask about our continuous delivery pipeline

building a containeri{s|z}ed continuous deployment pipeline

testing in a continuous delivery pipeline - better, faster,...

establish a continuous delivery pipeline: ibm urbancode...

automated accessibility testing in the continuous...