container orchestration on_aws

Container orchestration on AWS

by Kasper Nissen @phennex

Hi! My name is Kasper

@phennex

kubeclo

ud.io

foto: Lars Kruse, Aarhus UniversitetPervasive Systems group, Section of Electrical and Computer Engineering, Department of Engineering, Aarhus University

@phennex

@phennex

AGENDA1. Containers

2. Container Orchestration

3. Options?

4. Implementations?

5. Demo (rancher & kops)

CONTAINERS

@phennex

what? why?

@phennex

“… The value of this utilitarian object lies not in what it is, but in how it is used. The

container is at the core of a highly automated system for moving goods from anywhere, to anywhere, with a minimum of

cost and complication on the way.”–Marc Levinson

@phennex

TEXT

A

B

illustrations from:

@phennexillustrations from:

@phennexillustrations from:

@phennex

Multiplicity of goods

Multiplicity of methods for transport/storing

illustrations from:

@phennexillustrations from:

@phennex

Multiplicity of stacks

Multiplicity of hardware environments

illustrations from:

Static websitenginx 1.5 + mod security + openssl + bootstrap 3

User DBpostgresql + pgv8 + v8

Background workersPython 3.0 + celery + pyredis + libcurl + ffmpeg +

libopencv + nodejs + phantomjs

API endpointPython 2.7 + Flask + pyredis + celery + psycopg + postgresql-client

Public cloud

Production cluster

Customer Data Center

Development Laptop

Contributor’s desktop

Web frontendRuby + Rails + sass + Unicorn

Host OS

OS

Runtime

App

Container

@phennexillustrations from:

Dev Ops

@phennex

Host OS

OS

Runtime

App

Container

OS

Runtime

App

Container

OS

Runtime

App

Container

Multiple containers on the same host

@phennex

prometheus: image: quay.io/prometheus/prometheus links: - node-exporter:nodeexporter grafana: image: grafana/grafana links: - prometheus:prometheus node-exporter: ports: - 9100:9100/tcp image: prom/node-exporter

docker-compose

@phennex

Host OS

Container

Elastic Load Balancer

Container

Container

Host OS

Container Container

Container

Host OS

Container Container

Container

Replicated setup

@phennex

WHAT DO WE NEED?

@phennex

Scheduling

@phennex

Resource optimization

@phennex

Consensus

@phennex

Resilience

@phennex

Scalability

@phennex

CONTAINER ORCHESTRATION

@phennex

Node Node Node Node Node Node Node

Node NodeNode Node Node Node Node

big dataapp Bapp A database

… maybe a node we forgot about…

The reality?

@phennex

Node Node Node Node Node

Node Node Node Node Node

big dataapp Bapp A database

The dream

datacenter

Cluster Manager

@phennex

CATTLE VS PETS

@phennex

WHAT OPTIONS DO WE HAVE?

@phennex

Docker Swarm Kit

@phennex

Master

Pluggable Executor

Agent

Pluggable Executor

Agent

Pluggable Executor

Raft Consensus

(Leader election)

Docker Swarm Kit

Docker Engine

TaskService

Submit a task

Execute the task

@phennex

Docker Swarm Kit

docker service create --replicas 1 --name helloworld alpine

docker service ls

@phennex

You like using the docker CLI, and ecosystem tools

Get the native Docker API experience and compatibility

Easy to setup

Why Docker Swarm Kit?

docker swarm init

docker swarm join

@phennex

Controlled by Docker Inc.

Fairly new (Launched at DockerCon 2016)

Why NOT Docker Swarm Kit?

@phennex

APACHE MESOS

@phennex

ZooKeeperZooKeeper

ZooKeeper

Service 1Service 1Mesos Master

Service 1Service 1MarathonCoordination

& Configurations

Long running tasks

JobsMesos Slaves

Apache Mesos

@phennex

You are a Big Data house

You have a lot of job oriented or task oriented workloads

You have an infrastructure team

Your IT department employs a team of distributed systems specialists

You want to schedule multiple giant workload types concurrently

Maybe you will run Hadoop, Marathon, and Chronos all together

You have a 10.000+ node cluster

Cluster design is extremely scalable, and easily supports thousands of nodes

Battleprooved (Twitter, AirBnB, Uber)

Why Apache Mesos?

@phennex

Requires a lot of management (2 layers, Mesos and the frameworks)

Too big for many use cases?

Why NOT Apache Mesos?

@phennex

KUBERNETES

@phennex

KubernetesBased on Google’s 10 years of experience with Borg

Designed for containers

Declarative definition of desired state

Modular

Built with fault tolerance and resilience in mind

Govern by CNCF (Cloud Native Computing Foundation)

@phennex

Kubernetes

Scheduler

API Server

Kubernetes master

Kubelet Kubelet Kubelet

Application containers

Nodes

@phennex

Pods

Pod

Web server

Container

Pod

Web server

Container

Git sync

Container

Git repoConsumers Consumers

@phennex

Services

Pod

Web server

Container

Pod

Web server

Container

Pod

Web server

Container

type=FE type=FE type=FE

Service

VIP

Label selector: type=FE

Consumers

@phennex

Labels

Pod

Container

type=FE

Pod

Container

type=FE

Pod

Container

version=2

version=2

Dashboard Dashboard

show: type=FE show: version=v2

@phennex

Deployment (Replica Set)

Pod

Container

type=FE

Pod

Container

type=FE

Pod

Container

type=FE

Current: 3 Desired: 3

version=2 version=2 version=2

Deployment

@phennex

Deployment (Replica Set)

Pod

Container

type=FE

Pod

Container

type=FE

Pod

Container

type=FE

Current: 3 Desired: 3

version=2 version=2 version=2

Deployment

Pod

Container

type=FE

version=3

New version

@phennex

Deployment (Replica Set)

Pod

Container

type=FE

Pod

Container

type=FE

Pod

Container

type=FE

Current: 3 Desired: 3

version=3 version=2 version=2

Deployment

@phennex

Deployment (Replica Set)

Pod

Container

type=FE

Pod

Container

type=FE

Pod

Container

type=FE

Current: 3 Desired: 3

version=3 version=3 version=2

Deployment

@phennex

Deployment (Replica Set)

Pod

Container

type=FE

Pod

Container

type=FE

Pod

Container

type=FE

Current: 3 Desired: 3

version=3 version=3 version=3

Deployment

@phennex

Canary (AB testing)

Pod

Container

type=BE

Pod

Container

type=BE

Pod

Container

type=BE

Deployment

version=1 type=BE

Current: 2 Desired: 2

Deployment

version=3version=2version=2

version=2 type=BE

Current: 1 Desired: 1

Service

VIP

name=backend Label selector: type=BE

Consumers

@phennex

Governed by the Cloud Native Computing Foundation

Built by many years of experience with containers at Google

Topmost projects on Github

Large and engaged community

Why Kubernetes?

@phennex

Too small application for setting up a cluster?

Why NOT Kubernetes?

@phennex

HOW TO SET THIS UP ON AWS?

@phennex

@phennex

@phennex

@phennex

@phennex

Rancher Catalog

User Management

Container Orchestration and Scheduling

Infrastructure Services (Storage, Networking, Load Balancer, DNS, etc.)

Ops Management

Rancher overview

LDAP

Leverage existing tools: - CI/CD - Monitoring - etc.

@phennex

DEMO: KUBERNETES ON AWS WITH RANCHER

@phennex

OTHER SOLUTIONS

@phennex https://stackpoint.io/#/

@phennex

https://github.com/kubernetes/kops

@phennex

DEMO: KUBERNETES ON AWS WITH KOPS

@phennex

@phennex

@phennex

@phennex

@phennex

WHAT WOULD YOU CHOOSE?

@phennex

? Questions

Thank you! @phennex

kaspernissen@gmail.com

@phennex

Resources

@phennex

https://apprenda.com/blog/introducing-kismatic-enterprise-toolkit-fully-automated-path-production-kubernetes/

https://www.openshift.com/

http://rancher.com/

http://research.google.com/pubs/pub44843.html

http://research.google.com/pubs/pub43438.html

https://docs.docker.com/swarm/overview/

http://kubernetes.io/

Graphic credits

@phennex

http://www.freepik.com

Many of the used illustrations in this presentation is designed by