should a load-balancer choose the path as well as the server?
DESCRIPTION
Should a load-balancer choose the path as well as the server?. Nikhil Handigol Stanford University Joint work with Nick McKeown and Ramesh Johari. Datacenter. Wide-area. Enterprise. Client. LOAD-BALANCER. Can’t choose path :’(. Servers. Outline and goals. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/1.jpg)
Should a load-balancer choose the path
as well as the server?
Nikhil HandigolStanford University
Joint work with Nick McKeown and Ramesh Johari
![Page 2: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/2.jpg)
Datacenter
Wide-area Enterprise
![Page 3: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/3.jpg)
Can’t choose path :’(
LOAD-BALANCER
Client
Servers
![Page 4: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/4.jpg)
Outline and goals A new architecture for
distributed load-balancing joint (server, path) selection
Demonstrate a nation-wide prototype
Interesting preliminary results
![Page 5: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/5.jpg)
I’m here to ask for your help!
![Page 6: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/6.jpg)
Data Path (Hardware)
Control Path OpenFlow
OpenFlow Controller
OpenFlow Protocol (SSL)
Control Path
![Page 7: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/7.jpg)
Custom Hardware
Custom Hardware
Custom Hardware
Custom Hardware
Custom Hardware
OS
OS
OS
OS
OS
Network OS
Feature Feature
Software Defined Networking
Feature Feature
Feature Feature
Feature Feature
Feature Feature
Feature Feature
7
![Page 8: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/8.jpg)
Load Balancing is just
Smart Routing
![Page 9: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/9.jpg)
Custom Hardware
Custom Hardware
Custom Hardware
Custom Hardware
Custom Hardware
Network OS
Load-balancing logic
Load-balancing as a network primitive
Load-balancing decision
Load-balancing decision
Load-balancing decision
Load-balancing decision
Load-balancing decision
9
![Page 10: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/10.jpg)
Aster*x Controller
![Page 11: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/11.jpg)
![Page 12: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/12.jpg)
http://www.openflow.org/videos
![Page 13: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/13.jpg)
So far… A new architecture for distributed
load-balancing joint (server, path) selection
Aster*x – a nation-wide prototype Promising results that joint
(server, path) selection might have great benefits
![Page 14: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/14.jpg)
What next?
![Page 15: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/15.jpg)
How big is the pie?
Characterizing and quantifying the performance of joint (server, path) selection
![Page 16: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/16.jpg)
Load-balancing Controller
MININET-RT
![Page 17: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/17.jpg)
Load-balancing Controller
![Page 18: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/18.jpg)
Clients CDNISP
Model
![Page 19: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/19.jpg)
ParametersTopology
Intra-AS topologies
BRITE (2000 topologies)
CAIDA (1000 topologies)
Rocketfuel (~100 topos.)
20-50 nodesUniform link capacity
![Page 20: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/20.jpg)
ParametersServers
5-10 serversRandom placement
ServiceSimple HTTP serviceServing 1 MB fileAdditional server-side computation
![Page 21: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/21.jpg)
ParametersClients
3-5 client locationsRandom placement
Request patternPoisson process
Mean rate: 5-10 req/sec
![Page 22: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/22.jpg)
Load-balancing strategies?
![Page 23: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/23.jpg)
Simple but suboptimal
Complex but optimal
Design spaceDisjoint-Shortest-Path
Joint
Disjoint-Traffic-Engineering
![Page 24: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/24.jpg)
Anatomy of a request-response
Client Load-Balancer ServerRe
spon
se T
ime
Deliver
Retrieve
Choose
Request
Response 1st byte
Response last byte
Last byte ack
![Page 25: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/25.jpg)
Disjoint-Shortest-Path
CDN selects the least loaded server
Load = retrieve + deliverISP independently selects the shortest path
![Page 26: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/26.jpg)
Disjoint-Traffic-Engineering
CDN selects the least loaded server
Load = retrieve + deliverISP independently selects path to minimize max load
Max bandwidth headroom
![Page 27: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/27.jpg)
Joint
Single controller jointly selects the best (server, path) pair
Total latency = retrieve + estimated deliver
![Page 28: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/28.jpg)
Disjoint-Shortest-Path vs Joint
Disjoint-Shortest-Path performs ~2x worse than Joint
![Page 29: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/29.jpg)
Disjoint-Traffic-Engg. vs Joint
Disjoint-Traffic-Engineering performs almost as well as Joint
![Page 30: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/30.jpg)
Is Disjoint truly disjoint?Client Load-Balancer Server
Resp
onse
Tim
e
Deliver
Retrieve
Choose
Request
Response 1st byte
Response last byte
Last byte ack
Server response time contains network information
![Page 31: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/31.jpg)
The bottleneck effect
A single bottleneck resource along the path determines the performance.
![Page 32: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/32.jpg)
Clients CDNISP
The CDN-ISP game
![Page 33: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/33.jpg)
The CDN-ISP game System load monotonically
decreases Both push system in the same
direction
![Page 34: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/34.jpg)
Summary of observations
Disjoint-SP is ~2x worse than Joint Disjoint-TE performs almost as well
as Joint (despite decoupling of server selection
and traffic engineering) Game theoretic analysis supports
the empirical observation
![Page 35: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/35.jpg)
How we could collaborate
Netflix video - ~30% Internet traffic Important to efficiently utilize the available resources I want to apply my research work to Netflix’s service
“How can we jointly optimize (server, path) selection to achieve near-optimal performance?”
How can we work together on this?
![Page 36: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/36.jpg)
Can you share video streaming data? How can I model the “Netflix network”? Topology? B/W? Where is the bottleneck? Servers? Network? Where are the servers located? How many? What is the client request pattern? What is the video stream size distribution? Duration?
Bandwidth? How do you choose a server for a given request? How do you choose a path for a given request?
Questions – Video Streaming
![Page 37: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/37.jpg)
Can you share video streaming data? Cost structure – What is the cost model?
Why do you outsource video streaming to CDNs?
How do you deal with non-streaming part of the service (UI)?
Questions – Video Streaming
![Page 38: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/38.jpg)
Questions – AWS Deployment
Can we work together to characterize the AWS deployment? E.g., Size of deployment, incoming request pattern, inter-VM traffic
Are there web-level SLAs? Does AWS pose challenges?
What are the scaling bottlenecks? CPU? Network? Other?
![Page 39: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/39.jpg)
Let’s chat more!
![Page 40: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/40.jpg)
Conclusion A new architecture for
distributed load-balancing joint (server, path) selection
Aster*x - a nation-wide prototype Interesting preliminary results Future – application to streaming
media services
![Page 41: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/41.jpg)
Extra slides…
![Page 42: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/42.jpg)
Questions – AWS Deployment
Can you share Netflix AWS deployment data? How many VMs? What size? What is the service structure? How many tiers of services? Do you have any SLAs to meet? Any problems there? Would joint VM placement + routing help? What is the avg. NIC/CPU utilization on the VMs? Is the network ever a bottleneck? Do you do any MapReduce-style computation?
![Page 43: Should a load-balancer choose the path as well as the server?](https://reader036.vdocument.in/reader036/viewer/2022081511/56816390550346895dd48535/html5/thumbnails/43.jpg)
Sample topologies
BRITE CAIDA