continuous delivery at deli xl

Continuous Delivery @ Deli XL

Ernst de Haan, E-commerce Architect, Deli XL June 6, 2012

Executive Summary

> PROD = 18 machines (16 virtual)> Developer to PROD: ± 50 min> Monday after Sprint → PROD> Multi-site, multi-branch> Deployment = 1 click (takes ± 20 min)> Rollback = 1 click> Deployment issues = 0

✓ 1K suppliers✓ 30K customers

✓ 110K products✓ € 750M/year

✓ 2M consumers?✓ 100 catalogs

Process

> Scrum, 3 week Sprints> Acceptance test inside Sprint> Separate streets: D→A→P and T> Monday after Sprint → Prod (06:00 h)> Prod deployment by Ops team in India> Developers/testers deploy to D/T/A

Architecture

> ATG 10 (multi-site)> Commerce> Service

> XINS> integration layers> high testability

> GigaSpaces> service grid + data grid> high scalability

web browsers, mobile apps, external systems

Image API(XINS)

InternetInternet

ATG In API(XINS)

ATG Out API(XINS)

External systems: SAP, IDM, AS/400, etc.

CMS(First Spirit)

Apache

JSPs

E-commerceArchitecture

Domain

Passthru API(GigaSpaces)

Data Grid(GigaSpaces)

JavaScript API(XINS)


Why we did it

What we did

Demo

Discussion

1

2

3

4

Typical IT Headaches

> Creating builds = manual labour> Deploying to test environments = slow> Deployment reliability = error prone> Get Sprint ready for PROD = challenging> Root cause analysis = difficult> Elastic scaling = not possible

been there…

Typical Business Headaches

> TTM> long, even with agile (Developer → P in weeks?)> much variance (typically due to issues after Sprint)

> Cost> manual builds & deployments are recurring costs> the later an issue is found, the more expensive

Why Continuous Delivery

> Improve TTM (shorten, reduce variance)> Automate tedious work> Increase project efficiency (within Sprints)> Increase Ops efficiency> Make offshoring easier> Simplify new initiatives/brands> Reduce (impact of) Production defects> Elastic scaling

Why Continuous Delivery - Summary

> Reduce overhead cost> Increase reliability> Increase agility

and because it reduces headaches for IT :-)


Why we did it

What we did

Demo

Discussion

1

2

3

4

At Project Start

> Manual quality control> Manual builds & deployments> Single site/brand> Single project (1 branch in version control)> Slow environment provisioning

Steps Taken During Project

> Unified vision & NFRs> Standardized all processes> Continuous Integration including QC> Automated builds & deployments> Workflow control system (Jenkins)> Standardized environment provisioning> Environment cloning procedures

Design

> Environment-agnostic builds/packages> Commit Stages - one per branch> e.g. per project: previous release/hotfixes, next release> includes QC> includes automated deployments to CI environment> includes automated functional regression tests

> Deployment Pipelines> independent from Commit Stages> configuration easy to change

Technology

> Maven> Jenkins> Puppet

less relevant, it’s about how you set it up…

Dealing with Parallel Projects

> Example:> Sprint 22 – 3 weeks – regular Sprint> Sprint 21+ – 1 week – started at same time

> Approach:> separate branch (10 min)> separate commit stage (10 min)> allocated a deployment pipeline (= environments)> communication> daily merge, one way> final merge after P delivery

Dealing with Data

> Simple approach for typical situations> repeatable data model changes

> In exceptional cases intervention is needed

Hotfixes

> Again: single click> Regular deployments from release branch

Rollbacks

> Typical:> one click of a button

> Exceptional cases:> intervention needed because of data (structure)

Resourcing Vision (1/2)

> Business & management consulting:> local Solution Architect

> Analysis> Deli XL

> Development:> local ATG/Java expertise, leads (Mindcurv Europe)> offshore ATG/Java capacity (Mindcurv India)

Resourcing Vision (2/2)

> Testing:> local testers documenting test cases> offshore Testing-as-a-Service (Mindcurv)

> Support & maintenance:> mostly offshore> local Service Delivery Manager


Why we did it

What we did

Demo

Discussion

1

2

3

4