delivering ebay's ci solution with apache mesos & docker - dockercon 2014

12
Delivering eBay's CI Solution with Apache Mesos & Docker Mohit Soni Software Engineer Ashish Hunnargikar Software Engineer

Upload: ahunnargikar

Post on 18-Dec-2014

640 views

Category:

Software


3 download

DESCRIPTION

Delivering eBay's CI Solution with Apache Mesos & Docker - DockerCon 2014

TRANSCRIPT

Page 1: Delivering eBay's CI Solution with Apache Mesos & Docker - DockerCon 2014

Delivering eBay's CI Solution with Apache Mesos & Docker

Mohit SoniSoftware Engineer

Ashish HunnargikarSoftware Engineer

Page 2: Delivering eBay's CI Solution with Apache Mesos & Docker - DockerCon 2014

2

Why we did it....

• Thousands of VMs dedicated to run Jenkins!• Utilization is less than 5% • VMs idle most of time (no one runs CI builds frequently)

VM sprawl and poor resource utilization

• Polyglot application stacks (Java, C++, Node.js, Python, Scala)• Different OS flavors (Ubuntu & RHEL)• Software version conflicts• Special hardware requirements

Build slave management and maintenance is a nightmare

Page 3: Delivering eBay's CI Solution with Apache Mesos & Docker - DockerCon 2014

3

And here’s what we did....

• Mesos for cluster management• Marathon to launch Jenkins masters

• Jenkins Mesos-plugin to launch build slaves• Build slaves Dockerized as well as regular

Page 4: Delivering eBay's CI Solution with Apache Mesos & Docker - DockerCon 2014

4

Page 5: Delivering eBay's CI Solution with Apache Mesos & Docker - DockerCon 2014

5

Page 6: Delivering eBay's CI Solution with Apache Mesos & Docker - DockerCon 2014

6

Why use Docker containers?

• Managing varied slave flavors in a large cluster is hard!• Docker effectively isolates dependencies inside the container• Docker allows host OS to remain independent of workload dependencies

Why build Docker-in-Docker?

• Run Docker build/push/pull inside a Docker container• Eliminates redundant CI build dependency downloads• Helps to enforce a heterogeneous Mesos slave cluster

Page 7: Delivering eBay's CI Solution with Apache Mesos & Docker - DockerCon 2014

7

Basic Use Case

Build C++ & Java applications on both RHEL & Ubuntu

• Mesos slave host server is running Ubuntu

• Jenkins job is running builds in a RHEL or Ubuntu container

• Application dependencies can be preinstalled in base images OR can be downloaded during build time

Page 8: Delivering eBay's CI Solution with Apache Mesos & Docker - DockerCon 2014

8

Advanced Use Case

Build RHEL & Ubuntu Docker images

• Mesos slave host server is running Ubuntu

• Jenkins job is running inside a Docker container

• Jenkins job is building a RHEL/Ubuntu Docker image from an application Dockerfile

Page 9: Delivering eBay's CI Solution with Apache Mesos & Docker - DockerCon 2014

9

Enabling the Docker CI workflow using Jenkins & Mesos

Page 10: Delivering eBay's CI Solution with Apache Mesos & Docker - DockerCon 2014

10

Initial Build Strategy

Individual Docker-in-Docker Setup

Complete isolation

Docker daemon running inside the Docker container requires privileged mode

I/O overhead of downloading and duplicating all the AUFS layers

Page 11: Delivering eBay's CI Solution with Apache Mesos & Docker - DockerCon 2014

11

Optimized Build Strategy

Shared Docker-in-Docker Setup

Docker daemon not required inside the container so no privileged mode

Single reusable AUFS image layer cache

Redundant network and disk I/O eliminated for layer downloads

Lack of isolation for layers

Page 12: Delivering eBay's CI Solution with Apache Mesos & Docker - DockerCon 2014

12

For more information visit:

1. Apache Mesos: http://mesos.apache.org

2. Jenkins Mesos Plugin: https://github.com/jenkinsci/mesos-plugin

3. Docker: https://www.docker.io

4. Marathon Framework: https://github.com/mesosphere/marathon

Q&A

http://www.ebaytechblog.com/2014/04/04/delivering-ebays-ci-solution-with-apache-mesos-part-i/

http://www.ebaytechblog.com/2014/05/12/delivering-ebays-ci-solution-with-apache-mesos-part-ii/

www.ebaytechblog.com