santa cloud: how netflix does holiday capacity planning - south bay sre meetup aug-9-2016
TRANSCRIPT
![Page 1: Santa Cloud: How Netflix Does Holiday Capacity Planning - South Bay SRE Meetup Aug-9-2016](https://reader031.vdocument.in/reader031/viewer/2022020301/58ef99c31a28abd0768b46b5/html5/thumbnails/1.jpg)
Netflix Confidential
Santa Cloud: How Netflix Does Holiday Capacity Planning
August 9th, 2016
![Page 2: Santa Cloud: How Netflix Does Holiday Capacity Planning - South Bay SRE Meetup Aug-9-2016](https://reader031.vdocument.in/reader031/viewer/2022020301/58ef99c31a28abd0768b46b5/html5/thumbnails/2.jpg)
Netflix Confidential
![Page 3: Santa Cloud: How Netflix Does Holiday Capacity Planning - South Bay SRE Meetup Aug-9-2016](https://reader031.vdocument.in/reader031/viewer/2022020301/58ef99c31a28abd0768b46b5/html5/thumbnails/3.jpg)
Netflix Confidential
![Page 4: Santa Cloud: How Netflix Does Holiday Capacity Planning - South Bay SRE Meetup Aug-9-2016](https://reader031.vdocument.in/reader031/viewer/2022020301/58ef99c31a28abd0768b46b5/html5/thumbnails/4.jpg)
Netflix Confidential
● Additional complexities
○ Volatile customer traffic
○ Option to do regional failover (Chaos Kong)
○ No budgeting process
● Can’t run too lean or too wasteful
Capacity Tightrope Walking
● Holiday season = “let’s not get fired” season
![Page 5: Santa Cloud: How Netflix Does Holiday Capacity Planning - South Bay SRE Meetup Aug-9-2016](https://reader031.vdocument.in/reader031/viewer/2022020301/58ef99c31a28abd0768b46b5/html5/thumbnails/5.jpg)
Netflix Confidential
● Charter: “Ensure availability of cloud capacity in
an efficient manner, allowing engineering
organizations to further prioritize innovation
and availability.”
● Cross-functional team amongst:
○ Engineering
○ Data Science
○ Finance
Cloud Capacity Planning
![Page 6: Santa Cloud: How Netflix Does Holiday Capacity Planning - South Bay SRE Meetup Aug-9-2016](https://reader031.vdocument.in/reader031/viewer/2022020301/58ef99c31a28abd0768b46b5/html5/thumbnails/6.jpg)
Netflix Confidential
● Match methodology for the environment
○ Bottoms-up
○ Tops-down
○ Highly iterative
● Engagement with the largest service teams
● Evaluate migrations and coordinate changes
Holiday Preparation
![Page 7: Santa Cloud: How Netflix Does Holiday Capacity Planning - South Bay SRE Meetup Aug-9-2016](https://reader031.vdocument.in/reader031/viewer/2022020301/58ef99c31a28abd0768b46b5/html5/thumbnails/7.jpg)
Netflix Confidential
● One of our largest services changing hardware type
● Service profile:
○ ~20% of our total footprint
○ Autoscales ~65% in the course of a day
○ Runs in all regions
○ CPU bound, memory intensive
Case Study: API Service
![Page 8: Santa Cloud: How Netflix Does Holiday Capacity Planning - South Bay SRE Meetup Aug-9-2016](https://reader031.vdocument.in/reader031/viewer/2022020301/58ef99c31a28abd0768b46b5/html5/thumbnails/8.jpg)
Netflix Confidential
Changing The Trough
Reservation line
![Page 9: Santa Cloud: How Netflix Does Holiday Capacity Planning - South Bay SRE Meetup Aug-9-2016](https://reader031.vdocument.in/reader031/viewer/2022020301/58ef99c31a28abd0768b46b5/html5/thumbnails/9.jpg)
Netflix Confidential
Managing Failover Capacity
● Evolution of the Chaos Kong
● Cascading failovers
● Who gets capacity?
![Page 10: Santa Cloud: How Netflix Does Holiday Capacity Planning - South Bay SRE Meetup Aug-9-2016](https://reader031.vdocument.in/reader031/viewer/2022020301/58ef99c31a28abd0768b46b5/html5/thumbnails/10.jpg)
Netflix Confidential
● Through detailed planning:
○ Maximize trough borrowing
○ Highest chance of capacity availability
● Charter: “Ensure availability of cloud capacity in
an efficient manner, allowing engineering
organizations to further prioritize innovation
and availability.”
Our Role In The API Migration
● “Rinse-repeat” for the other large service teams
![Page 11: Santa Cloud: How Netflix Does Holiday Capacity Planning - South Bay SRE Meetup Aug-9-2016](https://reader031.vdocument.in/reader031/viewer/2022020301/58ef99c31a28abd0768b46b5/html5/thumbnails/11.jpg)
Netflix Confidential
![Page 12: Santa Cloud: How Netflix Does Holiday Capacity Planning - South Bay SRE Meetup Aug-9-2016](https://reader031.vdocument.in/reader031/viewer/2022020301/58ef99c31a28abd0768b46b5/html5/thumbnails/12.jpg)
Netflix Confidential
Questions