![Page 1: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/1.jpg)
Laura Frank Director of Engineering, Codeship
Scalable and Available Services with $CONTAINER_TOOL
![Page 2: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/2.jpg)
![Page 3: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/3.jpg)
R A I N I N G O N Y O U R PA R A D E
Highly-available applications existed before containers
![Page 4: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/4.jpg)
We love to think we’re solving new problems in new ways
![Page 5: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/5.jpg)
We shouldn’t confuse new tools with new problems
![Page 6: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/6.jpg)
![Page 7: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/7.jpg)
Container toolinghas changed the way we design, build, run, and ship applications.
![Page 8: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/8.jpg)
is a new solution for a longstanding problem.
Container tooling
![Page 9: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/9.jpg)
Containers aren’t the point
We reason about services
![Page 10: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/10.jpg)
![Page 11: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/11.jpg)
Before the late 1980s
![Page 12: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/12.jpg)
1990s-ish
![Page 13: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/13.jpg)
3:00am when you’re on call
![Page 14: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/14.jpg)
How can we guarantee availability in an environment
that will definitely fail?
![Page 15: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/15.jpg)
!
DISTRIBUTED APPLICATIONS ENGINEERING, 1998
“Redundancy and recovery are the two main approaches to
solve this problem.”
![Page 16: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/16.jpg)
An Imprecise Guideline ignoring many system constraints
redu
ndan
cy re
quire
d (n
umbe
r of r
eplic
as)
time to recover from failure (generic time units)
![Page 17: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/17.jpg)
Container tools have some pretty sweet ways to deal with both redundancy and recovery.
![Page 18: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/18.jpg)
Recovery
Control Theory FTW
![Page 19: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/19.jpg)
Your orchestration platform is continuously trying to reconcile actual state with declared state.
![Page 20: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/20.jpg)
Desired State
-
ClusterOrch
actions to conve
rge state
Actual State at time T
![Page 21: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/21.jpg)
An Observability Problem
If a system can’t be observed, it can’t be controlled.
![Page 22: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/22.jpg)
An Observability Problem
Failure Process State User Input
![Page 23: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/23.jpg)
Desired State
-
ClusterMe!
Actual State at time T
![Page 24: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/24.jpg)
An Observability Problem
Offloading the responsibility of observability to an orchestrator improves the level of controllability in your system
![Page 25: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/25.jpg)
Atomic Scheduling Units
Scheduler
Orchestrator
taskN
task0
task1
Service Spec desired state
Service Object actual state
![Page 26: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/26.jpg)
Kubernetes MasterDesired State
Scheduler Controllers
API Server
task0
task1
etcd
![Page 27: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/27.jpg)
Kubernetes MasterDesired State
etcd
converged!
Scheduler Controllers
API Server
![Page 28: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/28.jpg)
Using an orchestration tool, your system never fails…
it just doesn’t converge
![Page 29: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/29.jpg)
Redundancy
Replicating and scheduling for high availability
![Page 30: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/30.jpg)
HA application problems
scheduling problems
task scheduling problems
![Page 31: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/31.jpg)
binpack
![Page 32: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/32.jpg)
binpack
![Page 33: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/33.jpg)
spread
![Page 34: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/34.jpg)
spread (optimized for HA apps)
![Page 35: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/35.jpg)
Most modern orchestration systems use an optimized scheduling algorithm for
dispatching services across a set of nodes.
G R E AT N E W S
![Page 36: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/36.jpg)
It is not your tool’s responsibility to know about your system and business constraints
• topology* (some schedulers are topology aware) • specifics like OS, kernel, instance family • PII and other compliance
Y O U S T I L L H AV E T O D O W O R K
![Page 37: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/37.jpg)
These tools work on the service level, not the infrastructure level
R E M I N D E R
![Page 38: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/38.jpg)
Scheduling Constraints
Restrict services to specific nodes, such as specific architectures, security levels, or types, first apply a label to the nodes
docker service create \ --constraint 'node.labels.type==web' my-app
in Docker
![Page 39: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/39.jpg)
nodeSelector has been around since 1.0, but there are alternatives which are more expressive
nodeAffinity has been around since 1.2 (still in beta).
nodeAntiAffinity does the opposite — you can repel things from one another.
in KubernetesScheduling Constraints
![Page 40: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/40.jpg)
requiredDuringSchedulingIgnoredDuringExecution: - weight: 1 preference: matchExpressions: - key: some-node-label-key operator: Exists
in KubernetesScheduling Constraints
![Page 41: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/41.jpg)
requiredDuringSchedulingIgnoredDuringExecution
in KubernetesScheduling Constraints
requiredDuringSchedulingRequiredDuringExecution
This allows labels to change while the pod is
running and won’t result in eviction
![Page 42: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/42.jpg)
Implements a spread strategy over nodes that belong to a certain category.
This is a “soft” preference
--placement-pref ‘spread=node.labels.key’
in DockerPlacement Preferences
![Page 43: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/43.jpg)
preferredDuringSchedulingIgnoredDuringExecution
in KubernetesPlacement Preferences
![Page 44: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/44.jpg)
Topology-aware Scheduling
us-east-1
us-east-2
us-east-1
us-west-1
![Page 45: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/45.jpg)
Topology-aware Scheduling
us-east-1
us-east-2
us-east-1
us-west-1
![Page 46: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/46.jpg)
Topology-aware Scheduling
Kubernetes has a topology-aware scheduler! Read the docs.
In Docker, apply labels to your nodes, and use a placement preference like:
--placement-pref ‘spread=node.labels.region’
![Page 47: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/47.jpg)
An Imprecise Guideline ignoring most constraints
redu
ndan
cy re
quire
d (n
umbe
r of r
eplic
as)
time to recover from failure (hypothetical time units)
![Page 48: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/48.jpg)
The Future of Orchestration
Warning: opinions
![Page 49: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/49.jpg)
A Framework for Evaluation
Genesis Custom Built Product Commodity
Visible (Lots of Management) Invisible (No Management)
![Page 50: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/50.jpg)
Genesis Custom Built Product Commodity
Wardley Maps (simplified)
Time
Invis
ible
Visib
le
![Page 51: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/51.jpg)
Genesis Custom Built Product Commodity
Invis
ible
Visib
le
Electricity 18th Century
Electricity 19th Century
Electricity now
![Page 52: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/52.jpg)
Genesis Custom Built Product Commodity
Electricity
Compute
Invis
ible
Visib
le
![Page 53: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/53.jpg)
Genesis Custom Built Product Commodity
Container Runtime 2000s Container Runtime
2014-2015
Container Runtime now
Invis
ible
Visib
le
![Page 54: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/54.jpg)
Genesis Custom Built Product Commodity
Container Orchestrator
Container RuntimeInvis
ible
Visib
le
![Page 55: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/55.jpg)
Genesis Custom Built Product Commodity
Container Orchestrator
Container RuntimeInvis
ible
Visib
le
?
?
?
![Page 56: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/56.jpg)
Orchestration is becoming commoditized. Orchestrators will not be able to differentiate easily.
![Page 57: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/57.jpg)
C O M M O D I T I Z AT I O N
If you have a hand-rolled solution for running apps with containers, it’s safe
to migrate to an orchestration platform.
![Page 58: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/58.jpg)
I N N O VAT I O N
Solutions to old problems get commoditized, but it leaves room
for genesis elsewhere
![Page 59: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/59.jpg)
Genesis Custom Built Product Commodity
Container Orchestrator
Container RuntimeInvis
ible
Visib
le
?
?
?
Istio & service mesh tools
Whatever Heptio is building
Storage!
![Page 60: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/60.jpg)
Closing Thoughts
![Page 61: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/61.jpg)
How can we guarantee availability in an environment
that will definitely fail?
![Page 62: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/62.jpg)
!
DISTRIBUTED APPLICATIONS ENGINEERING, 1998
“Redundancy and recovery are the two main approaches to
solve this problem.”
Google became a company in 1998!
![Page 63: Scalable and Available Services with Docker and Kubernetes](https://reader033.vdocument.in/reader033/viewer/2022052116/5a64774b7f8b9afc4d8b4741/html5/thumbnails/63.jpg)
Laura Frank Director of Engineering, Codeship
@rhein_wein
Thanks!