container orchestration on_aws
TRANSCRIPT
Container orchestration on AWS
by Kasper Nissen @phennex
Hi! My name is Kasper
@phennex
kubeclo
ud.io
foto: Lars Kruse, Aarhus UniversitetPervasive Systems group, Section of Electrical and Computer Engineering, Department of Engineering, Aarhus University
@phennex
@phennex
AGENDA1. Containers
2. Container Orchestration
3. Options?
4. Implementations?
5. Demo (rancher & kops)
CONTAINERS
@phennex
what? why?
@phennex
“… The value of this utilitarian object lies not in what it is, but in how it is used. The
container is at the core of a highly automated system for moving goods from anywhere, to anywhere, with a minimum of
cost and complication on the way.”–Marc Levinson
@phennex
TEXT
A
B
illustrations from:
@phennexillustrations from:
@phennexillustrations from:
@phennex
Multiplicity of goods
Multiplicity of methods for transport/storing
illustrations from:
@phennexillustrations from:
@phennex
Multiplicity of stacks
Multiplicity of hardware environments
illustrations from:
Static websitenginx 1.5 + mod security + openssl + bootstrap 3
User DBpostgresql + pgv8 + v8
Background workersPython 3.0 + celery + pyredis + libcurl + ffmpeg +
libopencv + nodejs + phantomjs
API endpointPython 2.7 + Flask + pyredis + celery + psycopg + postgresql-client
Public cloud
Production cluster
Customer Data Center
Development Laptop
Contributor’s desktop
Web frontendRuby + Rails + sass + Unicorn
Host OS
OS
Runtime
App
Container
@phennexillustrations from:
Dev Ops
@phennex
Host OS
OS
Runtime
App
Container
OS
Runtime
App
Container
OS
Runtime
App
Container
Multiple containers on the same host
@phennex
prometheus: image: quay.io/prometheus/prometheus links: - node-exporter:nodeexporter grafana: image: grafana/grafana links: - prometheus:prometheus node-exporter: ports: - 9100:9100/tcp image: prom/node-exporter
docker-compose
@phennex
Host OS
Container
Elastic Load Balancer
Container
Container
Host OS
Container Container
Container
Host OS
Container Container
Container
Replicated setup
@phennex
WHAT DO WE NEED?
@phennex
Scheduling
@phennex
Resource optimization
@phennex
Consensus
@phennex
Resilience
@phennex
Scalability
@phennex
CONTAINER ORCHESTRATION
@phennex
Node Node Node Node Node Node Node
Node NodeNode Node Node Node Node
big dataapp Bapp A database
… maybe a node we forgot about…
The reality?
@phennex
Node Node Node Node Node
Node Node Node Node Node
big dataapp Bapp A database
The dream
datacenter
Cluster Manager
@phennex
CATTLE VS PETS
@phennex
WHAT OPTIONS DO WE HAVE?
@phennex
Docker Swarm Kit
@phennex
Master
Pluggable Executor
Agent
Pluggable Executor
Agent
Pluggable Executor
Raft Consensus
(Leader election)
Docker Swarm Kit
Docker Engine
TaskService
Submit a task
Execute the task
@phennex
Docker Swarm Kit
docker service create --replicas 1 --name helloworld alpine
docker service ls
@phennex
You like using the docker CLI, and ecosystem tools
Get the native Docker API experience and compatibility
Easy to setup
Why Docker Swarm Kit?
docker swarm init
docker swarm join
@phennex
Controlled by Docker Inc.
Fairly new (Launched at DockerCon 2016)
Why NOT Docker Swarm Kit?
@phennex
APACHE MESOS
@phennex
ZooKeeperZooKeeper
ZooKeeper
Service 1Service 1Mesos Master
Service 1Service 1MarathonCoordination
& Configurations
Long running tasks
JobsMesos Slaves
Apache Mesos
@phennex
You are a Big Data house
You have a lot of job oriented or task oriented workloads
You have an infrastructure team
Your IT department employs a team of distributed systems specialists
You want to schedule multiple giant workload types concurrently
Maybe you will run Hadoop, Marathon, and Chronos all together
You have a 10.000+ node cluster
Cluster design is extremely scalable, and easily supports thousands of nodes
Battleprooved (Twitter, AirBnB, Uber)
Why Apache Mesos?
@phennex
Requires a lot of management (2 layers, Mesos and the frameworks)
Too big for many use cases?
Why NOT Apache Mesos?
@phennex
KUBERNETES
@phennex
KubernetesBased on Google’s 10 years of experience with Borg
Designed for containers
Declarative definition of desired state
Modular
Built with fault tolerance and resilience in mind
Govern by CNCF (Cloud Native Computing Foundation)
@phennex
Kubernetes
Scheduler
API Server
Kubernetes master
Kubelet Kubelet Kubelet
Application containers
Nodes
@phennex
Pods
Pod
Web server
Container
Pod
Web server
Container
Git sync
Container
Git repoConsumers Consumers
@phennex
Services
Pod
Web server
Container
Pod
Web server
Container
Pod
Web server
Container
type=FE type=FE type=FE
Service
VIP
Label selector: type=FE
Consumers
@phennex
Labels
Pod
Container
type=FE
Pod
Container
type=FE
Pod
Container
version=2
version=2
Dashboard Dashboard
show: type=FE show: version=v2
@phennex
Deployment (Replica Set)
Pod
Container
type=FE
Pod
Container
type=FE
Pod
Container
type=FE
Current: 3 Desired: 3
version=2 version=2 version=2
Deployment
@phennex
Deployment (Replica Set)
Pod
Container
type=FE
Pod
Container
type=FE
Pod
Container
type=FE
Current: 3 Desired: 3
version=2 version=2 version=2
Deployment
Pod
Container
type=FE
version=3
New version
@phennex
Deployment (Replica Set)
Pod
Container
type=FE
Pod
Container
type=FE
Pod
Container
type=FE
Current: 3 Desired: 3
version=3 version=2 version=2
Deployment
@phennex
Deployment (Replica Set)
Pod
Container
type=FE
Pod
Container
type=FE
Pod
Container
type=FE
Current: 3 Desired: 3
version=3 version=3 version=2
Deployment
@phennex
Deployment (Replica Set)
Pod
Container
type=FE
Pod
Container
type=FE
Pod
Container
type=FE
Current: 3 Desired: 3
version=3 version=3 version=3
Deployment
@phennex
Canary (AB testing)
Pod
Container
type=BE
Pod
Container
type=BE
Pod
Container
type=BE
Deployment
version=1 type=BE
Current: 2 Desired: 2
Deployment
version=3version=2version=2
version=2 type=BE
Current: 1 Desired: 1
Service
VIP
name=backend Label selector: type=BE
Consumers
@phennex
Governed by the Cloud Native Computing Foundation
Built by many years of experience with containers at Google
Topmost projects on Github
Large and engaged community
Why Kubernetes?
@phennex
Too small application for setting up a cluster?
Why NOT Kubernetes?
@phennex
HOW TO SET THIS UP ON AWS?
@phennex
@phennex
@phennex
@phennex
@phennex
Rancher Catalog
User Management
Container Orchestration and Scheduling
Infrastructure Services (Storage, Networking, Load Balancer, DNS, etc.)
Ops Management
Rancher overview
LDAP
Leverage existing tools: - CI/CD - Monitoring - etc.
@phennex
DEMO: KUBERNETES ON AWS WITH RANCHER
@phennex
OTHER SOLUTIONS
@phennex
DEMO: KUBERNETES ON AWS WITH KOPS
@phennex
@phennex
@phennex
@phennex
@phennex
WHAT WOULD YOU CHOOSE?
@phennex
? Questions
Resources
@phennex
https://apprenda.com/blog/introducing-kismatic-enterprise-toolkit-fully-automated-path-production-kubernetes/
https://www.openshift.com/
http://rancher.com/
http://research.google.com/pubs/pub44843.html
http://research.google.com/pubs/pub43438.html
https://docs.docker.com/swarm/overview/
http://kubernetes.io/
Graphic credits
@phennex
http://www.freepik.com
Many of the used illustrations in this presentation is designed by