scheduling, priority, and...
TRANSCRIPT
![Page 1: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/1.jpg)
Scheduling, Priority, and QoS
Ralph McNeal
Principal Engineer, CenturyLink Cloud & Portals
ralphmcneal.com
![Page 2: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/2.jpg)
Discussion Topics
Pod Eviction
Quality Of Service (QoS)
Managing Compute Resources
Pod Priority
Preemption
Pod Assignment
![Page 3: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/3.jpg)
Architecture Overview
![Page 4: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/4.jpg)
Pod Eviction
![Page 5: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/5.jpg)
Pod Eviction
Resource Eviction
Scheduling Eviction
![Page 6: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/6.jpg)
Quality of Service (QoS) Classes
Guaranteed
Burstable
Best-Effort
![Page 7: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/7.jpg)
given by Kubernetes
QoS Level Assignment
![Page 8: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/8.jpg)
Container Resource Management
Requests
Limits
![Page 9: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/9.jpg)
Container Resource Management
containers:
name: haproxy:2.0.4
resources:
requests:
cpu: 100m
memory: 1Gi
limits:
cpu: 100m
memory: 1Gi
![Page 10: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/10.jpg)
Container Resource Management
containers:
name: clc-web-app:3.0.0
resources:
requests:
cpu: 100m
memory: 1Gi
limits:
cpu: 500m
memory: 2Gi
![Page 11: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/11.jpg)
Container Resource Management
containers:
name: clc-poc:1.0.1
![Page 12: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/12.jpg)
demo
Quality of Service (QoS)
![Page 13: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/13.jpg)
Container Resource Management
Scheduling?
Eviction?
Compressible Resources
Vs.
Incompressible Resources
![Page 14: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/14.jpg)
Don’t forget the kubelet
--kube-reserved and –system-reserved
![Page 15: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/15.jpg)
Determining Resource Specs
docker stats command
![Page 16: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/16.jpg)
Pod Priority
Priority indicates the importance of a Pod relative to other Pods.
![Page 17: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/17.jpg)
Pod Priority
apiVersion: kind: PriorityClassmetadata: name: high-priority
value: 10000000 globalDefault: falsedescription: ”For mission critical pods"
![Page 18: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/18.jpg)
Preemption
When a pod evicts a pod based on priority
![Page 19: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/19.jpg)
Non-preempting PriorityClass
apiVersion:
kind: PriorityClass
metadata:
name: high-priority-nonpreempting
value: 10000000
preemptionPolicy: Never
globalDefault: false
description: "This priority class will not cause other pods to be preempted."
![Page 20: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/20.jpg)
demo
Pod Priority and Preemption
![Page 21: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/21.jpg)
Interactions of Pod priority and QoS
The scheduler’s preemption logic does not consider QoS
The kublet considers both QoS and Pod priority during out-of-
resource eviction
![Page 22: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/22.jpg)
Pod Assignment
nodeSelector
Affinity
![Page 23: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/23.jpg)
Labels
• Key/value pairs on nodes, pods, and other objects
• Allows pod assignment via nodeSelector and affinity rules
![Page 24: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/24.jpg)
Node Selector
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx
nodeSelector:
disktype: ssd
![Page 25: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/25.jpg)
Affinity/Anti-Affinity
• More than just “and”
• Allows requirements to be “hard” or ”soft”
• Allows constraints against labels on other pods vs. the node
![Page 26: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/26.jpg)
Node Affinity
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: disktype
operator: In
values:
- ssd
![Page 27: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/27.jpg)
Pod Affinity
affinity:
podAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- cache
topologyKey: "kubernetes.io/hostname"
![Page 28: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/28.jpg)
Pod Anti-Affinity
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- web
topologyKey: "kubernetes.io/hostname"
![Page 29: Scheduling, Priority, and QoSralphmcneal.com/wp-content/uploads/2019/08/scheduling-priority-qo… · Scheduling, Priority, and QoS Ralph McNeal Principal Engineer, CenturyLink Cloud](https://reader035.vdocument.in/reader035/viewer/2022070816/5f0f716a7e708231d4442fb4/html5/thumbnails/29.jpg)
Questions?http://kubernetes.io
http://ralphmcneal.com