containing chaos with kubernetes - terrence ryan, google - devopsdays tel aviv 2015

‹#›@tpryan

Terrence Ryan Developer Advocate

Containing Chaos with Kubernetes

‹#›@tpryan

Who are you?

‹#›@tpryan

01 IntroductionWhy Kubernetes?

‹#›@tpryan

What problem are we trying to solve?

‹#›@tpryan

# FRONTEND AND SERVICES FROM nginx-php-fpm COPY nginx.conf /etc/nginx/nginx.conf ADD www /var/www/

‹#›@tpryan

# BACKEND FROM ubuntu:12.04

ADD ./mysql-setup.sh /tmp/mysql-setup.sh RUN /bin/sh /tmp/mysql-setup.sh

EXPOSE 3306

CMD ["/usr/sbin/mysqld"]

‹#›@tpryan

‹#›@tpryan

‹#›@tpryan

‹#›@tpryan

# FRONTEND AND SERVICES FROM nginx-php-fpm COPY nginx.conf /etc/nginx/nginx.conf ADD www /var/www/

‹#›@tpryan

# FRONTEND AND SERVICES FROM nginx-php-fpm COPY nginx.conf /etc/nginx/nginx.conf ADD www /var/www/

# JUST SERVICES FROM nginx-php-fpm COPY nginx.conf /etc/nginx/nginx.conf ADD www /var/www/

‹#›@tpryan

# FRONTEND AND SERVICES FROM nginx-php-fpm COPY nginx.conf /etc/nginx/nginx.conf ADD www /var/www/

‹#›@tpryan

# FRONTEND AND SERVICES FROM nginx-php-fpm COPY nginx.conf /etc/nginx/nginx.conf ADD www /var/www/

# FRONTEND FROM nginx COPY nginx.conf /etc/nginx/nginx.conf ADD www /var/www/

‹#›@tpryan

‹#›@tpryan

‹#›@tpryan

# BACKEND FROM ubuntu:12.04

ADD ./mysql-setup.sh /tmp/mysql-setup.sh RUN /bin/sh /tmp/mysql-setup.sh

EXPOSE 3306

CMD ["/usr/sbin/mysqld"]

‹#›@tpryan

# BACKEND FROM ubuntu:12.04

ADD ./mysql-setup.sh /tmp/mysql-setup.sh RUN /bin/sh /tmp/mysql-setup.sh

EXPOSE 3306

CMD ["/usr/sbin/mysqld"]

# BACKEND FROM ubuntu:12.04

ADD ./mysql-setup.sh /tmp/mysql-setup.sh RUN /bin/sh /tmp/mysql-setup.sh

EXPOSE 3306 VOLUME ["/etc/mysql", "/var/lib/mysql"]

CMD ["/usr/sbin/mysqld"]

‹#›@tpryan

‹#›@tpryan

‹#›@tpryan

‹#›@tpryan

‹#›@tpryan

‹#›@tpryan

‹#›@tpryan

‹#›@tpryan

‹#›@tpryan

‹#›@tpryan

‹#›@tpryan

That’s a lot to manage.

‹#›@tpryan

‹#›@tpryan

4 3 2

‹#›@tpryan

02 ConceptsWhat is Kubernetes?

‹#›@tpryan

Desired State

‹#›@tpryan

Build Script

‹#›@tpryan

Build Script

./create_docker_images.sh

‹#›@tpryan

Build Script

./create_docker_images.sh

./launch_frontend.sh x 3

‹#›@tpryan

Build Script

./create_docker_images.sh

./launch_frontend.sh x 3

./launch_services.sh x 2

‹#›@tpryan

Build Script

./create_docker_images.sh

./launch_frontend.sh x 3

./launch_services.sh x 2

./launch_backend.sh x 1

‹#›@tpryan

Build Script

./create_docker_images.sh

./launch_frontend.sh x 3

./launch_services.sh x 2

./launch_backend.sh x 1

‹#›@tpryan

Build Script

./create_docker_images.sh

./launch_frontend.sh x 3

./launch_services.sh x 2

./launch_backend.sh x 1

‹#›@tpryan

Build Script

./create_docker_images.sh

./launch_frontend.sh x 3

./launch_services.sh x 2

./launch_backend.sh x 1

‹#›@tpryan

Desired State

There should be: 3 Frontends 2 Services 1 Backend

‹#›@tpryan

Desired State

There should be: 3 Frontends 2 Services 1 Backend

‹#›@tpryan

Desired State

There should be: 3 Frontends 2 Services 1 Backend

‹#›@tpryan

Pods

• Atomic component of Kubernetes • Made from one or more containers • It’s okay to have just one container • Share

• IP Address • Namespace

• Examples • Webserver + CMS • All Web available services • Converting an all in one box

‹#›@tpryan

Containers

• Subatomic particles of Kubernetes • Dockerfiles just like you are used to.

‹#›@tpryan

Controllers

• Handle turning current state into desired state • Example

• Replication Controllers

‹#›@tpryan

Controllers

• Handle turning current state into desired state • Example

• Replication Controllers

Observe

‹#›@tpryan

Controllers

• Handle turning current state into desired state • Example

• Replication Controllers

Observe

Act

‹#›@tpryan

Controllers

• Handle turning current state into desired state • Example

• Replication Controllers

Observe

Diff

Act

‹#›@tpryan

Service

• Defines a set of pods that work together for a common purpose

• Gets a virtual IP address • Used for exposing an application to non

Kubernetes clients

Virtual IP address

‹#›@tpryan

Labels & Selectors

• Metadata for Objects • Select sections of your infrastructure

App Tier Env

todo frontend stage

App Tier Env

todo frontend prod

App Tier Env

todo frontend dev

App Tier Env

todo frontend test

App Tier Env

todo api prod

App Tier Env

todo backend prod

App Tier Env

todo api stage

App Tier Env

todo api test

App Tier Env

todo api dev

App Tier Env

todo backend stage

App Tier Env

todo backend test

App Tier Env

todo backend dev

‹#›@tpryan

Labels & Selectors

• Metadata for Objects • Select sections of your infrastructure

App Tier Env

todo frontend stage

App Tier Env

todo frontend prod

App Tier Env

todo frontend dev

App Tier Env

todo frontend test

App Tier Env

todo api prod

App Tier Env

todo backend prod

App Tier Env

todo api stage

App Tier Env

todo api test

App Tier Env

todo api dev

App Tier Env

todo backend stage

App Tier Env

todo backend test

App Tier Env

todo backend dev

‹#›@tpryan

Labels & Selectors

• Metadata for Objects • Select sections of your infrastructure

App Tier Env

todo frontend stage

App Tier Env

todo frontend prod

App Tier Env

todo frontend dev

App Tier Env

todo frontend test

App Tier Env

todo api prod

App Tier Env

todo backend prod

App Tier Env

todo api stage

App Tier Env

todo api test

App Tier Env

todo api dev

App Tier Env

todo backend stage

App Tier Env

todo backend test

App Tier Env

todo backend dev

‹#›@tpryan

Labels & Selectors

• Metadata for Objects • Select sections of your infrastructure

App Tier Env

todo frontend stage

App Tier Env

todo frontend prod

App Tier Env

todo frontend dev

App Tier Env

todo frontend test

App Tier Env

todo api prod

App Tier Env

todo backend prod

App Tier Env

todo api stage

App Tier Env

todo api test

App Tier Env

todo api dev

App Tier Env

todo backend stage

App Tier Env

todo backend test

App Tier Env

todo backend dev

‹#›@tpryan

Demo: Kubernetes in Action

‹#›@tpryan

There’s More

• Secrets • Rolling Updates • Persistent Volumes

‹#›@tpryan

03 Container EngineHosted Kubernetes

‹#›@tpryan

I’ve mostly talked about developing on Kubernetes

‹#›@tpryan

Setting up a cluster

‹#›@tpryan

Setting up a cluster

• Choose a cloud: • GCE, AWS, Azure, Rackspace, on-premises, …

‹#›@tpryan

Setting up a cluster

• Choose a cloud: • GCE, AWS, Azure, Rackspace, on-premises, …

• Choose a node OS: • CoreOS, Atomic, RHEL, Debian, CentOS, Ubuntu, ...

‹#›@tpryan

Setting up a cluster

• Choose a cloud: • GCE, AWS, Azure, Rackspace, on-premises, …

• Choose a node OS: • CoreOS, Atomic, RHEL, Debian, CentOS, Ubuntu, ...

• Provision machines: • Boot VMs, install and run kube components, ...

‹#›@tpryan

Setting up a cluster

• Choose a cloud: • GCE, AWS, Azure, Rackspace, on-premises, …

• Choose a node OS: • CoreOS, Atomic, RHEL, Debian, CentOS, Ubuntu, ...

• Provision machines: • Boot VMs, install and run kube components, ...

• Configure networking: • IP ranges for Pods, Services, SDN, ...

‹#›@tpryan

Setting up a cluster

• Choose a cloud: • GCE, AWS, Azure, Rackspace, on-premises, …

• Choose a node OS: • CoreOS, Atomic, RHEL, Debian, CentOS, Ubuntu, ...

• Provision machines: • Boot VMs, install and run kube components, ...

• Configure networking: • IP ranges for Pods, Services, SDN, ...

• Start cluster services: • DNS, logging, monitoring, ...

‹#›@tpryan

Setting up a cluster

• Choose a cloud: • GCE, AWS, Azure, Rackspace, on-premises, …

• Choose a node OS: • CoreOS, Atomic, RHEL, Debian, CentOS, Ubuntu, ...

• Provision machines: • Boot VMs, install and run kube components, ...

• Configure networking: • IP ranges for Pods, Services, SDN, ...

• Start cluster services: • DNS, logging, monitoring, ...

• Manage nodes: • kernel upgrades, OS updates, hardware failures...

‹#›@tpryan

Or…

‹#›@tpryan

‹#›@tpryan

‹#›@tpryan

Container Engine

‹#›@tpryan

Container Engine

• Hosted Kubernetes

‹#›@tpryan

Container Engine

• Hosted Kubernetes• A few smart defaults set

‹#›@tpryan

Container Engine

• Hosted Kubernetes• A few smart defaults set• Allow for dipping your feet in

‹#›@tpryan

05 ConclusionsBring it home

Google has been developing and using containers to manage our applications for over 10 years.

‹#›@tpryan

‹#›@tpryan

Everything at Google runs on Containers: • Gmail, Web Search, Maps, ... • MapReduce, batch, ... • GFS, Colossus, ... • Even Google’s Cloud Platform: VMs run

in containers!

‹#›@tpryan

Everything at Google runs on Containers: • Gmail, Web Search, Maps, ... • MapReduce, batch, ... • GFS, Colossus, ... • Even Google’s Cloud Platform: VMs run

in containers!

We launch 2 Billion Containers a week

‹#›@tpryan

We think containers are the way to manage scale.

‹#›@tpryan

You should carefully consider whether running everything on

containers is right for you.

‹#›@tpryan

You should run everything on containers.

Just in case it isn’t clear here. this is not what I am recommending. I’m saying don’t Cherry Pick from the last statements to come to the conclusion that you have to go this way.

‹#›@tpryan

Container Engine can make dipping your toes in a little

easier.

‹#›@tpryan

Thank Youterrenceryan.com @tpryan This preso: http://bit.ly/tpryan-chaos