continuously serving the oss community with continuous integration and delivery, by akshay karle and...

Continuously serving the OSS community with

Continuous Integration and Delivery

Akshay Karle !Fernando Júnior

“How long would it take your organization to deploy a change that involves just one single line of code?”

Mary and Tom Poppendieck

“(…) build software in such a way that the software can be released to

production at any time.” Martin Fowler

CONTINUOUS INTEGRATION

CONTINUOUS INTEGRATION

CONTINUOUS INTEGRATION

CONTINUOUS DELIVERY

CONTINUOUS DELIVERY

!

!

▫︎ Continuous Integration

!

▫︎ Automated tasks

!

▫︎ Repeatable/reliable process

THERE’S MORE…

CONTINUOUS DELIVERY

CONTINUOUS DELIVERY

DEPLOYING YOUR APPLICATION

!

!

▫︎ Database migrations

!

▫︎ Infrastructure update

!

▫︎ Restarting services

ZERO DOWNTIME DEPLOYMENTS

Deployment process of your application has got to be transparent for end users

WHO WE ARE?

Developer, Snap CI

/nandopaf

/fernando-alves

/nand0paf

Developer, Snap CI

/akshay_karle

/akshaykarle

/akshay_ka

WHAT WE DO?

!

!

!

!

▫︎Continuous Integration for repositories on GitHub

▫︎ SaaS

▫︎Helps you do Continuous Delivery

▫︎ Free for open source projects

SNAP INSIDE-OUT

LIFECYCLE OF A BUILD

LIFECYCLE OF A BUILD

LIFECYCLE OF A BUILD

LIFECYCLE OF A BUILD

head repository commit

LIFECYCLE OF A BUILD

head repository commit

LIFECYCLE OF A BUILD

head repository commit

DATABASE

LIFECYCLE OF A BUILD

head repository commit

DATABASE

LIFECYCLE OF A BUILD

head repository commit

DATABASE

Babysitters

LIFECYCLE OF A BUILD

head repository commit

DATABASE

Babysitters

LIFECYCLE OF A BUILD

head repository commit

DATABASE

Babysitters

LIFECYCLE OF A BUILD

head repository commit

DATABASE

LIFE CYCLE

!

▫︎ Prepare the container

▫︎ Starts database

▫︎ Sets PATH

▫︎ Download artifacts

▫︎ Git clone

▫︎ Runs each pipeline stage

▫︎ Upload artifacts

DATABASE

Babysitters

LIFECYCLE OF A BUILD

head repository commit

DATABASE

Babysitters

LIFECYCLE OF A BUILD

head repository commit

DATABASE

ARCHITECTURE OVERVIEW

L B

ARCHITECTURE OVERVIEW

L B

Database

web server

ARCHITECTURE OVERVIEW

!

!

▫︎ Rails app fronted by apache

!

▫︎ Receives the hooks from GitHub

Database

web server

ARCHITECTURE OVERVIEW

L B

web server

Database

ARCHITECTURE OVERVIEW

L B

Build Server

web server

Database

ARCHITECTURE OVERVIEW

!

▫︎ Background jobs

▫︎ Babysitters

▫︎ Build Queue

▫︎ Artifacts

!

▫︎ OpenVZ Containers

▫︎ Virtual machines where the pipeline runs

Build Server

ARCHITECTURE OVERVIEW

L B

Build Server

web server

Database

OUR DEPLOYMENT

!

▫︎ Do have automated scripts

!

▫︎ Deployment pipeline

!

▫︎ 1-click deploy

▫︎ Sort of…

Babysitters

DATABASE

L B

Build Server

Database

web server

OUR DEPLOYMENT

!

▫︎ Wait for all builds to finish

!

▫︎ Put app on maintenance mode

▫︎ No new requests picked up

!

▫︎ Deploy and wait for migrations

Babysitters

DATABASE

L B

Database

VZHOSTBuild Server

web server

ZERO DOWNTIME DEPLOYMENTS

BLUE-GREEN DEPLOYMENT

Web Server App Server Database

BLUE-GREEN DEPLOYMENT

Web Server App Server Database

App ServerApp ServerWeb Server Database

BLUE-GREEN DEPLOYMENT

Web Server App Server Database

App ServerApp ServerWeb Server Database

BLUE-GREEN DEPLOYMENT

App ServerApp ServerWeb Server Database

BLUE-GREEN DEPLOYMENTS FOR SNAP

VZHOSTBuild Server

web server

VZHOSTBuild Server

web server

L B DATABASE

BLUE-GREEN DEPLOYMENTS FOR SNAP

VZHOSTBuild Server

web server

VZHOSTBuild Server

web server

L B DATABASE

CHALLENGES IN SNAP

Long running builds

Database migrations

CHALLENGES IN SNAP

Long running builds

LONG RUNNING BUILDS

VZHOSTBuild Server

web server

VZHOSTBuild Server

web server

L B DATABASE

LONG RUNNING BUILDS

VZHOSTBuild Server

web server

VZHOSTBuild Server

web server

L B DATABASE

LONG RUNNING BUILDS

!

▫︎ Builds should continue to run

!

▫︎ Artifacts should continue to be served

COLOR AWARE BABYSITTERS

Build Server

Green Stack

Build Server

Blue Stack

COLOR AWARE BABYSITTERS

Build Server

Green Stack

Build Server

Blue Stack

web server

BUILD LIFECYCLE

head repository commit

DATABASEDATABASE

L B

BUILD LIFECYCLE

head repository commit

DATABASEDATABASE

L B

BUILD LIFECYCLE

head repository commit

DATABASEDATABASE

L B

THE SWITCH

L B

THE SWITCH

L B

THE SWITCH

L B

THE SWITCH

L B

THE SWITCH

L B

THE SWITCH

L B

THE SWITCH

L B

CHALLENGES IN SNAP

Long running builds

Database migrations

CHALLENGES IN SNAP

Database migrations

DATABASE MIGRATIONS

PipelinesCounterDuration

App v1.0

App v1.0

L B

DATABASE MIGRATIONS

PipelinesCounterDuration

App v1.0

App v1.0

L B

Upgrading to v2.0, migrating

DATABASE MIGRATIONS

PipelinesCounter

App v2.0

App v1.0

L B

Cannot have destructive migrations

MIGRATIONS IN PHASES

PipelinesCounterDuration

App v1.0

App v1.0

L B

MIGRATIONS IN PHASES

PipelinesCounterDuration

App v1.0

App v1.0

Upgrading to v2.0

L B

MIGRATIONS IN PHASES

PipelinesCounterDuration

App v2.0

App v1.0

L B

MIGRATIONS IN PHASES

PipelinesCounterDuration

App v2.0

App v1.0

L B

MIGRATIONS IN PHASES

PipelinesCounterDuration

App v2.0

App v1.0

L B

MIGRATIONS IN PHASES

PipelinesCounterDuration

App v2.0

App v1.0Upgrading to v3.0

L B

MIGRATIONS IN PHASES

App v2.0

App v3.0

PipelinesCounter

L B

MIGRATIONS IN PHASES

App v2.0

App v3.0

PipelinesCounter

L B

LESSONS LEARNT

Automation is beauty

Watch out your data

Things will go wrong

THANK YOUQuestions?

https://snap-ci.com