building web scale apps with docker and mesos by alex rukletsov (mesosphere)

Building Web Scale Apps with Docker and Mesos

Alexander RukletsovSoftware Engineer @ Mesosphere

Why should we care?

Applications in the Cloud Era

Client-Server Era:Small apps, big servers

Cloud Era:Big apps, small servers

Serv Serv Serv Serv

VirtualizationAggregation

App

Server

App App App App

Major Components

● Hardware

● Orchestration, deployment and isolation

● Cluster and resource management

● Scale-aware applications, service discovery, etc.

Major Components

● Hardware

● Orchestration, deployment and isolation

● Cluster and resource management

● Scale-aware applications, service discovery

Applications in the Cloud Era

Client-Server Era:Small apps, big servers

Cloud Era:Big apps, small servers

Serv Serv Serv Serv

VirtualizationAggregation

App

Server

App App App App

“Divide and rule [and package]”

—Philip II of Macedon, paraphrased

“Divide et impera [et sarcina]”

Containers

● Lightweight Linux execution environment

● Static application composition

● Reliable deployment

● Unit of resource isolation

● Execution isolation

● Multi-tenancy without heavyweight VMs

Docker

● Open source

● Configurable layers

● Reproducible

● Version-controlled

● Plenty of other people’s containers

Docker

● Open source● Configurable layers● Reproducible● Version-controlled● Plenty of other people’s containers

● First-class citizen in Mesos and the Mesosphere stack

● Kubernetes employs and promotes Docker

© Gerard Julien/AFP

Run everything in containers!

What about container management?

© Unknown

© Unknown

Apache Mesos facts

● Created in 2009 at UC Berkeley, hardened in Twitter● Top-level Apache project● Mesosphere, Twitter, and Airbnb are major users /

contributors● Scales to 10 000s of nodes, production grade● Packages and support through Mesosphere● Google officially endorsed Mesos for Kubernetes● Built-in containerization, including Docker

Mesos as a Distributed OS kernel

● Two level resource scheduling● Launch tasks across the cluster● Communication between tasks (like IPC)● APIs for building “native” applications (aka frameworks):

program against the datacenter● APIs in C++, Python, JVM-languages, Go and counting● Pluggable CPU, memory, IO isolation● Multi-tenant workloads● Failure detection● Easy failover and HA

How Mesos works (HA mode)

Zookeeper

MesosMasterMesos

MasterMesosMaster

MesosMaster

MesosSlave

Executor Task

Task

Framework TaskScheduler

Application

Executor

Task

TaskExecutor

MesosSlave

© ect.nl

Manage containers with Mesos!

Mesos + Docker =

Native Docker support in Mesosmessage DockerInfo {

required string image = 1;

// Network options.

enum Network {

HOST = 1;

BRIDGE = 2;

NONE = 3;

}

message PortMapping {

required uint32 host_port = 1;

required uint32 container_port = 2;

optional string protocol = 3; // Protocol to expose (ie: tcp, udp).

}

optional Network network = 2 [default = HOST];

repeated PortMapping port_mappings = 3;

optional bool privileged = 4 [default = false];

// Allowing arbitrary parameters to be passed to docker CLI.

repeated Parameter parameters = 5;

}

Native Docker support in MesosCommandInfo command;

command.set_value("dd if=/dev/zero of=/dev/null");

ContainerInfo::DockerInfo dockerInfo;

dockerInfo.set_image("mesosphere/inky");

ContainerInfo containerInfo;

containerInfo.set_type(ContainerInfo::DOCKER);

containerInfo.mutable_docker()->CopyFrom(dockerInfo);

TaskInfo task;

task.set_name("");

task.mutable_task_id()->set_value("1");

task.mutable_slave_id()->CopyFrom(offer.slave_id());

task.mutable_resources()->CopyFrom(offer.resource());

task.mutable_command()->CopyFrom(command);

task.mutable_container()->CopyFrom(containerInfo);

vector<TaskInfo> tasks;

tasks.push_back(task);

driver.launchTasks(offer.id(), tasks);

Native Docker support in Marathon// nginx-task.json

{

"container": {

"type": "DOCKER",

"docker": {

"image": "nginx",

"network": "BRIDGE",

"portMappings": [

{ "containerPort": 80,

"hostPort": 0,

"servicePort": 80,

"protocol": "tcp" }

]

}

},

"id": "nginx",

"instances": "1",

"cpus": "0.25",

"mem": "256",

"uris": []

}

$ cat nginx-task.json | http http://dev1.mesosphere.com:8080/v2/apps

Future Docker Swarm API support

Mesos

Mesos Framework

Serv

DockerApp

Docker Swarm API

DockerApp

DockerApp

DockerApp

DockerApp

DockerApp

DockerApp

Serv Serv Serv Serv

What Mesos contributes

● Multi-framework: weighted fair sharing, roles, etc.● Run Docker containers alongside other popular frameworks

(e.g. Spark, Rails, Hadoop, …)● Run services and batch apps in the same cluster● Advanced scheduling: resources, constraints, global view● High resource availability, cluster self-healing● Proven at scale, battle-tested in production● GUI / CLI cluster management console

Ways of running Dockers

● Marathoneasy to setup, reliable orchestration

● Multiple Marathonsbenefit from two level scheduling, reservations, framework

roles

● Custom frameworkfine-grained management

“L’homme est libre au moment qu’il veut l’être”

“Man is free at the instant he wants to be”

—Voltaire

Cluster configuration example #1

AWS, DigitalOcean, GCE

Mesos

Hardware

Kernel

Apps

API

DockerApp

DockerApp

Services

Services REST API“Marathon”

DockerApp

AppDocker

AppApp

Cluster configuration example #2

AWS, DigitalOcean, GCE

Mesos

Hardware

Kernel

Apps

API

Services

Services REST API“Marathon”

Kubernetes

DockerApp

DockerApp

DockerApp

AppDocker

AppApp

Docker

Cluster configuration example #3

Serv Serv Serv Serv

Mesos

Spark, MPI, Hadoop, Storm

ServServServServ

Mesos SDKJava, Python, C++, Go

Services REST API“Marathon” (init)

Batch REST API“Chronos” (cron)

Serv

App Recurring Jobs(ETL, backups)

Hardware

Native Long running Batch

Apps

API

Kernel

AppDocker

App

AWS, DO, GCE

Docker

Cluster configuration example #4

Mesos

Spark, MPI, Hadoop, Storm

ServServ

Mesos SDKJava, Python, C++, Go

Services REST API“Marathon” (init)

Batch REST API“Chronos” (cron)

Serv

App Recurring Jobs(ETL, backups)

Hardware

Native Long running Batch

Apps

API

Kernel

AppDocker

App

Serv

© Reuters

Mesos is not only for “big” players!

Tiny clusters for everybody

● Mesosphere on GCE

● Mesosphere on DigitalOcean

● Mesosphere on AWS

● Portable Mesosphere on a USB stick

google.mesosphere.com

digitalocean.mesosphere.com

elastic.mesosphere.io

Demo: Mesos-on-Mesosphere

TaskTaskTask

Mesos-on-Mesosphere

Zookeeper

MesosSlaveMarathon

MesosMaster

MesosMasterMesos

MasterMesosMaster

MesosSlave

...

Task

Docker

DockerDockerDocker

MesosSlave

Mesos-on-Mesosphere

Zookeeper

MesosSlaveMarathon

MesosMaster

MesosMasterMesos

MasterMesosMaster

MesosSlave

...

MesosMaster

Docker

MesosSlave

MesosSlave

Docker

MesosSlave

MesosMaster

Summary

● Complete stack for large (and small) distributed apps

● Multi-tenancy

● Resource optimizations

● Easy to deploy

● No vendor lock-in

Thank You.

alex@mesosphere.io#mesos on irc.freenode.net