game of codes: the battle for ci

Games of Codes, the CI battleA view behind the Atlassian curtains

ESTHER ASENJO • BAMBOO DEV TEAM LEAD • ATLASSIAN • @STHREO

Who am I?

Agenda

C I A N ATO M Y

T H E C I B AT T L E , AT AT L A S S I A N

C I C U LT U R E

C I I N T R O D U C T I O N

T H E C D B AT T L E , AT AT L A S S I A N

C O N C L U S I O N

Continuous Integration is a software development practice where members of a team integrate their work frequently.Each integration is verified by an automated build to detect integration errors as quickly as possible. This leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.”

“

- M A R T I N F O W L E R , T H O U G H T W O R K S

do it FASTER.make us STRONGER.

make it BETTER.work it HARDER.

FASTER delivery.STRONGER teams.

BETTER quality.HARDER processes.

Agenda

C I A N ATO M Y


C I C U LT U R E


C O N C L U S I O N


Atlassian, a software company

Products Services

Ecosystem

Questions for ConfluenceTeam Calendars for ConfluenceShare Point Connector for Confluence

www.atlassian.com

my.atlassian.com

Atlassian AccountJIRA Service DeskJIRA Core

JIRA Software

Portfolio for JIRACapture for JIRA

Continuous Integration is a battle.

The Continuous Integration battle, IT DOES NOT COUNT UNTIL IT’S SHIPPED

When you play the Game of Codes, you ship or you fail.

Agenda

C I A N ATO M Y


C I C U LT U R E


C O N C L U S I O N


Infrastructure

Processes

People

Infrastructure

Build InfrastructureEngineering Collaboration JIRA family Dev Tools Internal Ecosystem

Collaboration JIRA family Dev Tools Internal EcosystemEngineering

Build Infrastructure

Build EngineeringBuild infrastructure

Maven repositoriesRemote agents

Elastic agents

12

Bamboo instances

1200 elastic agents 20 AMIs

4

remote agents

3,5k

build plans

15 Nexus instances

14k branch build plans

Tools Processes

Processes Tools : The CI anatomy

Developer Test DeliverBuildCommit

Developer Test DeliverBuildCommit

Processes Tools : The CI anatomy

- C O N T I N U O U S D E L I V E RY ( A D D I S O N - W E S L E Y ) , J E Z H U M B L E A N D D AV I D FA R L E Y

Continuously is more often than you think.”“

Confluence pipeline

100 developers

320 commits/day

20,000 tests

435 build plans

Confluence pipeline

1,5 days

I’m not slacking off, my build plans are still running.

“”

- M E

I’m not slacking off, my build plans are queued.

“”

- D A N I E L K J E L L I N , B A M B O O D E V E L O P M E N T T E A M L E A D E R

I’m not slacking off, my build plans are

CI challenges our processes

Master branch production-ready Feature branch development

Long running branches

Big&slow test suite

Quick feedback

Feature flags

Build grid

Build strategy

Confluence pipeline strategy

30 min

Core Test Suite

90 min

Cloud Test Suite

Ecosystem testing

1,5 days

Full Test Suite

Performance testing

Cross-Platform testing

Acceptance testing

Confluence team dog fooding

1 per day

1 per week*

Atlassian dog fooding

Monthly bug fixing

Server release

Quarterly release

Cloud release

Every 2 weeks*

+ +++ =

The continuous integration pipeline stages

Commit Automated testing

technically

functionally non-functionally

usable provides value

deliverable fully-operational

Manual testing

Release

The continuous integration pipeline stages

Commit Automated testing

Release Manual testing

QA Dog fooding

817quarantined tests out of > 200k test

1,888occurrences of flakiness in our Confluence wiki

But hey: I am code too, you know? I need attention!“

”- M A U R I E D O ( @ M A U R I _ E D O ) , O P E N L E T T E R F R O M A N @ I G N O R E D T E S T, D E V E L O P E R S . AT L A S S I A N . C O M

http://developers.atlassian.com

People

CI is a practice, not a tool, and requires discipline from the development team.

“”

- C O N T I N U O U S D E L I V E RY ( A D D I S O N - W E S L E Y ) , J E Z H U M B L E A N D D AV I D FA R L E Y

The Culture of Green powered by

Agenda

C I A N ATO M Y


C I C U LT U R E


C O N C L U S I O N


Culture is what happens when the founders are not around.

The Build Master

The Build Breaker

Red is the new green.

Give feedback to people, so it’s fed-back.

0quarantined tests out of > 200k test

Agenda

C I A N ATO M Y


C I C U LT U R E



C O N C L U S I O N

CI is not enough. ”“- C O N T I N U O U S D E L I V E RY ( A D D I S O N - W E S L E Y ) , J E B H U M B L E A N D D AV I D FA R L E Y

The non-CD syndrome

I cannot fix that bug… I’m busy writing a new one.

“”

- M E , O R Y O U , O R A N Y D E V E L O P E R A F F E C T E D B Y T H E L A C K O F C D

A deployment pipeline

A deployment pipeline, at Atlassian

09:0009:00 dev

09:00

DEV STAGING PROD


dev

dev

09:00

10:00


dev

dev

dev 09:00

10:00DEV STAGING PROD


09:00 dev

dev

stg

09:00

10:00

16:00


09:00 dev

stg

dev

09:00

10:00

16:00

DEV STAGING PROD


09:00 dev

dev

09:00

10:00

16:00

16:15

stg


09:00 dev 10:00 dev

09:00

10:00

16:15

16:00 stg

16:30


09:00 dev 10:00 dev

09:00

10:00

16:15

16:00 stg

16:30

10:00 prd


09:00 dev 10:00 dev

09:00

10:00

16:15

16:00 stg

16:30

10:00

DEV STAGING PROD

prd


Free people to do the interesting stuff,

and let Bamboo do the rest.

Wax in, wax off.

Agenda

C I A N ATO M Y


C I C U LT U R E



C O N C L U S I O N

19

34

18

42

Do it, or do not. There is no try. ”

“

Do it, with . ”

“

Thank you!

ESTHER ASENJO • BAMBOO DEV TEAM LEAD • ATLASSIAN • @STHREO