© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

DevOps Summit AGM Continuous Delivery Adam Spektor/ January 28, 2013

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Structure

Continuous Delivery

Jenkins OO Nexu

s

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3

8 Principles of Continuous Delivery

Principles

• The process for releasing/deploying software

MUST be repeatable and reliable

• Automate everything!

• If something's difficult or painful, do it more often

• Keep everything in source control

• Done means “released”

• Build quality in!

• Everybody has responsibility for the release

process

Martin

Fowler

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Continuous integration

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5

CI infrastructure structure

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 6

Continuous Integration

Dev L

AB

Yehud

Jenkin

s

Farm

SaaS

Data

Cente

r

Je

nkin

s

Farm

1

2

How ?

What ?

• UT

• Rest system tests

• UI selenium tests

• Upgrade tests

• Performance tests (single user experience)

• Static code analysis/Coverage – Sonar (FindBugs, PMD,

JaCoCo)

• Maven site

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 7

Continuous Integration

Performance test : WHY ?

•Enable performance testing as part of CI

•Utilize Integration & UI testing as Performance tests

•Track performance trending and KPI in addition to functional testing

•Mark build as unstable in case of performance degradation above

threshold

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 8

Continuous Integration

Performance test : test perspective

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9

Continuous Integration

Performance test : transaction perspective

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 10

Jenkins

Master

1 - Developer checkin

5 - Split tests to chunks based previous execution time

1

2

2 - Jenkins trigger

4 - Build & copy into the Vagrant VM’s shared directory

3 - Execute chef OS modifications & configurations

0 - Prerequisites

1. Pool with vms prepared (include initial Sahara

state) each vm use Jenkins warm plug-in (auto-

discover slave)

2. Each pool have assigned to predefined OS item

Virtual box

Jenkins slaves pool

Jenkins

Physical

Slave 3

4

5

6

6 - Clean vm with Sahara & swarm

installation

1. OS changes like code changes should be tested in CI

system

2. OS changes should be trackable

3. Independent development – no IT involved

4. Quick and updated development environment

Continuous Integration – near future

Developer Lab

Why

?

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Nexus artifact management

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 12

Nexus infrastructure

• LB – active/passive Linux heart beat

clustering model

• Guarantee high availability to read users

– Nexus proxy duplication

– Smart proxy – guarantee that

requested artifacts are available

• Isolation level – only Jenkins can upload

artifacts to Nexus (only after tests

verification)

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Continuous deployment

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 14

Continuous Delivery flow

Circular deployment Staging Productio

n

Automatic/By request

By request :

Approved by QA

By request :

Approved by QA &

SaaS

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 15

Continuous deployment

CI Master mydtbld0004

1

CI Farm

1 - Time Scheduled/Manual Build

Nexus

master

2

2 - Upload to QA

repository DevLab

IL

SaaS Lab

(Single)

3

Nexus Proxy

3 - Start deploy QA/DEV

QA

5

4

4 - Sync proxy & Upload to RAS

5 - Deploy

Automatic nightly deployment to QA/DEV

environment

OO Server Dev

3 3

3

OO Lab

Server

5

RAS

4

4

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 18

Continuous deployment 1 -Request deploy version

2 - Download to RAS

SaaS Lab

(Single)

3 - Deploy

Manually deployment to Security/PCoE

Nexus Proxy

1

PCoE

Security

engineer

OO Lab

Server

RAS

3

PCoE

engineer

Security

1

2

2 3

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 19

Continuous deployment 1 - Request promote to staging

2 - Build

DevLab

IL

SaaS Lab

(Single)

Nexus Proxy

OO Lab

Server

3 - Upload to Staging repository

4

4 - Sync proxy

5 - Request to Deploy

Promote & Deploy to Staging

SaaS Production

(Multi)

1

Staging

QA

operator

SaaS operator

RAS

CI Master

CI Farm

Nexus

master

3

2

RAS 4

6 - Download to RAS & Deploy

5

5

6 6

4

OO Production

Server

2

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 20

Continuous deployment 1 - Request promote to production

Nexus

master

2 - Nexus promote to production

DevLab

IL

SaaS Lab

(Single)

2

Nexus Proxy

OO Production

Server

2

Promote to Production

SaaS Production

(Multi)

1

SaaS

Operator

RAS

2

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 21

Continuous deployment 1 - Request to deploy

2 - Upload to appropriate

geography RAS

Sidney Austin

3 -Deploy

Deploy to Production

London

SaaS operator

OO Production

server

RAS

Nexus Proxy

RAS RAS

1

2

2

SaaS operator SaaS operator

1 1

2 2

3 3 3 3

Internal

External

External External

2

2

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 22

Deployment Demo

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Thank you