dockerday2015: docker orchestration for sysadmin
TRANSCRIPT
Orchestration for SysadminsPhu Nguyen Anh – Einsights
Orchestration for Sysadmins
Andrea Luzzardi / Victor VieuxSoftware Engineers @ docker
Source: Recap from DockerCon US 2015
Agenda
• Introduction to Docker Swarm
• Swarm Fault Tolerance: Replication
• Integrating Swarm with Mesos
• Provisioning Swarm with Docker Machine
Source: Recap from DockerCon US 2015
“Running containers on multiple hosts.”
Source: Recap from DockerCon US 2015
Swarm in a nutshell
• Exposes several Docker Engines as a single virtual Engine
• Serves the standard Docker API
• Extremely easy to get started
• Batteries includes but swappable
(PhuNA edited)
Source: Recap from DockerCon US 2015
(Added by Phu)
TimelineOct
Nov
Dec
Jan
Feb
Jun
Proof of Concept
DockerCon EU Open Repository
First Release Candidate
Swarm Beta Release
Global Hack Day Open Proposal
0.2.0 Release
0.3.0 Release
Apr
Source: Recap from DockerCon US 2015
Setting up Swarm
• Create a cluster: - $ swarm create
• Add nodes to a cluster: - $ swarm join --advertise=<node_ip> token://<token>
• Start Swarm: - $ swarm manage -H <swarm_ip> token://<token>
Or you can use your own etcd, zookeeper or consul
Source: Recap from DockerCon US 2015
Swarm Scheduler
• Resource Management - Memory, CPU, Network
• Fit containers - Apply filters to exclude nodes - Use a strategy to rank and pick the best node
• Scheduling Knobs - Constraints - Affinities
$ docker run -m 1g -c 2 -p 80:80 ...
$ docker run -e constraint:storage==ssd ...
$ docker run -e affinities:container!=redis* ...
(Constraint, Affifinity, Port, Dependency, Health)
(spread, binpack, random)
(PhuNA edited)
(PhuNA edited)
Source: Recap from DockerCon US 2015
(Added by Phu)
(Added by Phu)
(Added by Phu)
“Batteries Included but Swappable”
Source: Recap from DockerCon US 2015
Docker Swarm + Mesos
• Experimental • Docker Swarm act as a Mesos Framework • In collaboration with
$> swarm manage --cluster mesos-experimental <mesos_master_url>
Source: Recap from DockerCon US 2015
Mesos Cluster
+
Mesos CLI
Mesos Slaves
+
+
+
+
+
+
+
+
+
+
+
+
+
Mesos Masters
Marathon
Source: Recap from DockerCon US 2015
Mesos Cluster + Docker Swarm
+
Mesos CLI
Mesos Slaves
+
+
+
+
+
+
+
+
+
+
+
+
+
Mesos Masters
Marathon
DockerCLI
Source: Recap from DockerCon US 2015
Mesos Cluster + Docker Swarm
+
Mesos CLI
Mesos Slaves
+
+
+
+
+
+
+
+
+
+
+
+
+
Mesos Masters
Marathon
DockerCLI
shipyard
kitematic compose
Source: Recap from DockerCon US 2015
BY
Source: Recap from DockerCon US 2015
Swarm Manager Replication
Source: Recap from DockerCon US 2015
Replication
DockerCLI
Swarm Cluster
• Swarm Manager is a Single Point of Failure
API Request
Manage
Source: Recap from DockerCon US 2015
Replication
DockerCLI
Swarm Cluster
• Upon failure it becomes impossible to manage the cluster
• Cluster continues to run un-managed
API Request
Manage
Source: Recap from DockerCon US 2015
Replica
Replication
Primary
DockerCLI
Swarm Cluster
Forward
API Request
Manage
Replica
• One primary, many replicas
• Leader Election built on top of Consul, etcd, ZooKeeper
• Replicas forward API calls to primary
• CLI can talk to any instance
Source: Recap from DockerCon US 2015
Replica
Replication
Primary
DockerCLI
Swarm Cluster
Forward
API Request
Manage
Replica
• Replicas monitor health of primary instance
• Leader gets re-elected upon primary failure
Source: Recap from DockerCon US 2015
PrimaryReplica
Replication
Replica
DockerCLI
Swarm Cluster
Forward
API Request
Manage
• One replica promoted to primary
• Other replicas notified of leadership update
• CLI can continue talking to same instance
Source: Recap from DockerCon US 2015
Demo
Source: Recap from DockerCon US 2015
“From Zero to Docker”
Source: Recap from DockerCon US 2015
Machine in a nutshell
• Host management for a container centric world
• Provisions Virtual Machines
• Installs & Manages Docker Engine
• Swarm Cluster Provisioning
Source: Recap from DockerCon US 2015
Machine Provisioning
5 Local Providers
9 Cloud Providers
7 Operating Systems
Source: Recap from DockerCon US 2015
Machine Management
• Install, Configure & Upgrade Docker Engine - docker-machine upgrade node-{1,2,3}
• PKI Management - TLS Encryption & Authentication by default - CA generation, Key signing and Re-keying - Seamless integration: Auto-configure the CLI
• Bring Your Own Machine - Generic Driver
Source: Recap from DockerCon US 2015
Machine Operations
• Operate - List, Start, Stop and Kill machines
• Debug - docker-machine ssh node-2
• Script - docker-machine toolkit: ip, url, …
Source: Recap from DockerCon US 2015
Machine Clustering
• Integration with Docker Swarm
• Provision Swarm management nodes
• Pre-configure machines to join a Swarm cluster
• Tweak Swarm options
Source: Recap from DockerCon US 2015
Demo
Source: Recap from DockerCon US 2015
Thank you, Questions?Andrea Luzzardi - @aluzzardi
Victor Vieux - @vieux
Source: Recap from DockerCon US 2015