Continuous Integration for Fun and Profit

Arnold Bechtoldt

Darmstadt, 16.06.2016v2

Arnold BechtoldtSystems Engineer / Consultant @ inovex

〉 Software-Defined Datacenters

〉 Continuous Integration/Delivery

〉 DevOps

2

Scenario

The Product

4

Frontend Middleware

MySQL Backend

SMTP Backend

IMAP/POP Backend

Test Matrix

5

Rails-FE

Mail-MW

Calendar-MW

DBMS

K/V 1

K/V 2

MDA Mail-Store

MDA IMAP/POP Proxy

Mail-FTS

MTA Mail-In

MTA Mail-Out

Customer Feature Set 1

Customer Feature Set 2

Customer Feature Set 3

Customer Feature Set 4

...

v1, v2, v3, ...

v1, v2, v3, ...

v1, v2, v3, ...

v1, v2, v3, ...

...

X X

Sad Stories from Real World

We can‘t run integration tests in parallel.

Running all of our tests takes ages.

Our integration tests are fragile.

Producing a bugfix release is hard work.

Time For A Plan

The Plan: Building A Private Travis CI Clone

12

GitLab to the Rescue!

gitlab.com/bechtoldt/tabellarius 13

1. Git-Pushes go to feature/bugfix branches

2. Every Git-Push triggers a test

3. Tests run in prod-like environments

4. Tests run in isolated/dedicated environments

5. Master branch keeps “stable”

6. Make it easy to improve/add tests

Improving the Workflow

14

15

Pipeline (simplified)

run-tests.node5

run-tests.node4

release-artifact deploy-artifactGit Push

IntegrationDelivery

GitLab CI Config

16

stages:

- tests

- release

- deploy

run-tests.node4:

stage: tests

image: node4

script:

- npm test

run-tests.node5:

stage: tests

image: node5

script:

- npm test

release-artifact:

stage: release

script:

- npm release.prod

deploy-artifact:

stage: deploy

script:

- npm deploy.qa

17

GitLab Code Build Status

18

GitLab CI Pipeline

19

GitLab CI Pipeline (2)

Conclusions

〉 Spawning integration test environments at ludicrous speed

〉 Similarity to production leads to faster bugfixing

〉 Isolated testing improves test (=product) quality

〉 Parallel testing increases work efficiency

Conclusions

21

A voice in my head..

22

Arnold Bechtoldtinovex GmbH

abechtoldt@inovex.de

CC BY-NC-ND inovex.de +ArnoldBechtoldtGER

github.com/bechtoldt arbe.io youtube.com/inovexGmbH

24

Backup

25

GitHub/Travis Code Build Status

26

Travis CI Pipeline

27

Travis CI Pipeline (2)

28

Jenkins CI Pipeline

29

Jenkins CI Pipeline (3)

30

Jenkins CI Pipeline (4)

31

Gitlab Jenkins Master

Jenkins CI Architecture

Jenkins Slave

ArtifactoryJenkins Slave

Jenkins Slave

Toolset

32

Tool PurposeGitLab Git Repository + CI ServerDocker Container TechnologyDocker-Compose Container/Service ManagementBash + Confd Image/Container ConfigurationArtifactory Artifact Repository