lessons from netflix mesos clusters

44

Upload: sharma-podila

Post on 19-Feb-2017

719 views

Category:

Software


2 download

TRANSCRIPT

Page 1: Lessons from Netflix Mesos Clusters
Page 2: Lessons from Netflix Mesos Clusters
Page 3: Lessons from Netflix Mesos Clusters
Page 4: Lessons from Netflix Mesos Clusters
Page 5: Lessons from Netflix Mesos Clusters
Page 6: Lessons from Netflix Mesos Clusters

81 Million subscribers worldwide and growing

Scale

Page 7: Lessons from Netflix Mesos Clusters

Page 8: Lessons from Netflix Mesos Clusters
Page 9: Lessons from Netflix Mesos Clusters
Page 10: Lessons from Netflix Mesos Clusters

○○

Page 11: Lessons from Netflix Mesos Clusters

○○

Page 12: Lessons from Netflix Mesos Clusters
Page 13: Lessons from Netflix Mesos Clusters
Page 14: Lessons from Netflix Mesos Clusters
Page 15: Lessons from Netflix Mesos Clusters
Page 16: Lessons from Netflix Mesos Clusters
Page 17: Lessons from Netflix Mesos Clusters

vs.

For long running stateful services

Page 18: Lessons from Netflix Mesos Clusters
Page 19: Lessons from Netflix Mesos Clusters
Page 20: Lessons from Netflix Mesos Clusters
Page 22: Lessons from Netflix Mesos Clusters

Fitness

Pending

Assigned

Urg

ency

N tasks to assign from M possible agents

Page 23: Lessons from Netflix Mesos Clusters

Page 24: Lessons from Netflix Mesos Clusters

MasterMasterMaster

Agent

Page 25: Lessons from Netflix Mesos Clusters
Page 26: Lessons from Netflix Mesos Clusters

ENI 1

ENI 2

Page 27: Lessons from Netflix Mesos Clusters

Compute focused Agent GroupASG-11: Active

Compute focused Agent GroupASG-10: Inactive

Network focused Agent GroupASG-21: Active

Network focused Agent GroupASG-20: Inactive

Fenzo autoscaling sets ASG’s “desired” count, honors “min” and “max”

AWS EC2 can rate limit API calls

Page 28: Lessons from Netflix Mesos Clusters
Page 29: Lessons from Netflix Mesos Clusters
Page 30: Lessons from Netflix Mesos Clusters

○○

○○

Page 31: Lessons from Netflix Mesos Clusters

○○○

Page 32: Lessons from Netflix Mesos Clusters
Page 33: Lessons from Netflix Mesos Clusters

+ <

Page 34: Lessons from Netflix Mesos Clusters

○○

○○

Page 35: Lessons from Netflix Mesos Clusters

Page 36: Lessons from Netflix Mesos Clusters
Page 37: Lessons from Netflix Mesos Clusters

○○

Page 38: Lessons from Netflix Mesos Clusters
Page 39: Lessons from Netflix Mesos Clusters
Page 40: Lessons from Netflix Mesos Clusters

“user”: … ,

“application”: … ,

“labels”: … ,

“Containers”: {

“C1”: { … }

},

“engine”: {

“jobspec”: { … }

}

Page 41: Lessons from Netflix Mesos Clusters
Page 42: Lessons from Netflix Mesos Clusters

○○

Page 43: Lessons from Netflix Mesos Clusters

○○

Page 44: Lessons from Netflix Mesos Clusters