tectonic summit 2016: the origins of kubernetes

The origins of KubernetesBrendan Burns

Microsoft Azure Container Service

The origins of KubernetesBrendan Burns

Microsoft Azure Container Service

8

Pre-history

1979 - chroot

1982 - BSD

2000 - FreeBSD jail

2005 - Solaris Zones/Containers

2006-2007 - Linux CGroups and Namespaces

2006 - AWS begins selling VMs

2008 - I join Google websearch infrastructure teams

My Ops Experience

Dev. and On-call for Google Web-Search

No SSH

No imperative configuration

No room for errors

My Ops Experience

Dev. and On-call for Google Web-Search

No SSH

No imperative configuration

No room for errors

My Ops Experience

Dev. and On-call for Google Web-Search

No SSH

No imperative configuration

But lots of thoughts about Borg.

A starting point: Borg

The merge of two systems:

Global Work Queue

Babysitter

Lots of organic growth, spread out over time

Utilization is the major problem to be solved

A step forward: Seattle 2012

WebSearch leaves Seattle :(

But public cloud is growing :)

But public cloud is using VMs :(

But we can try to make it better :)

Declarative configuration

Server side deployment

Health maintainance

The door flies open: Docker

2013 - Docker begins

Image format and build tools

Image distribution infrastructure

Container runtime

The original idea: Seattle, Nov. 2013

A minimally viable, open source Docker orchestrator

Only two challenges:

func orchestrate(...) error { …}

The “minimally viable” borg: simplify ops

O(N) = O(1)

The “minimally viable” borg: decouple all the things

The “minimally viable” borg: naming

The “minimally viable” borg: discovery

Lessons learned: Be consistent

Lessons learned: Everyone comes in the front door

https://www.flickr.com/photos/16210667@N02/13504685914

Lessons learned: There is no hierarchyfrontend

bburns

Node - 1

backend

julia

Node - 1

frontend

julia

Node - 2

backend

bburns

Node - 2

Lessons learned: There is no hierarchyfrontend

bburns

Node - 1

backend

julia

Node - 1

frontend

julia

Node - 2

backend

bburns

Node - 2

Lessons learned: There is no hierarchyfrontend

bburns

Node - 1

backend

julia

Node - 1

frontend

julia

Node - 2

backend

bburns

Node - 2

Lessons learned: There is no hierarchyfrontend

bburns

Node - 1

backend

julia

Node - 1

frontend

julia

Node - 2

backend

bburns

Node - 2

Differences: It had to be public cloud

Differences: It had to be open

Differences: Build a community & ecosystem

Three years later

Many, many clusters in production

40,000+ commits

1000+ contributors

191 Meetups around the world

3 extraordinarily humbled co-founders

Unsolved problems: Configuration

App App

App

Unsolved problems: Cloud development

Unsolved problems: Scaling our community

Thank You!