web server load balancing/scheduling
DESCRIPTION
Web Server Load Balancing/Scheduling. Asima Silva Tim Sutherland. Outline. Web Server Introduction Information Management Basics Load Sharing Policies FLEX WARD EquiLoad AdaptLoad Summary Conclusions Future Work. Request enters a router - PowerPoint PPT PresentationTRANSCRIPT
Web Server Load Web Server Load Balancing/SchedulingBalancing/Scheduling
Asima SilvaAsima Silva
Tim SutherlandTim Sutherland
OutlineOutline
Web Server IntroductionWeb Server IntroductionInformation Management BasicsInformation Management BasicsLoad Sharing PoliciesLoad Sharing Policies– FLEXFLEX– WARDWARD– EquiLoadEquiLoad– AdaptLoadAdaptLoad
SummarySummaryConclusionsConclusionsFuture WorkFuture Work
Introduction to Web Server Load Introduction to Web Server Load BalancingBalancing
Request enters a Request enters a router router
Load balancing server Load balancing server determines which determines which web server should web server should serve the requestserve the request
Sends the request to Sends the request to the appropriate web the appropriate web serverserver
InternetInternet
Router
Load-Balancing Server
Web ServersRequestResponse
Traditional Web Cluster
How do we split up information?How do we split up information?
ContentContent Server FarmServer Farm
?
Load Balancing ApproachesLoad Balancing Approaches
File DistributionFile Distribution RoutingRouting
Content/Locality Content/Locality AwareAware
DNS ServerDNS Server
Size AwareSize Aware Centralized Centralized RouterRouter
Workload AwareWorkload Aware Distributed Distributed DispatcherDispatcher
IssuesIssues
Efficiently processing requests with Efficiently processing requests with optimizations for load balancingoptimizations for load balancing– Send and process requests to a web server Send and process requests to a web server
that has files in cachethat has files in cache– Send and process requests to a web server Send and process requests to a web server
with the least amount of requestswith the least amount of requests– Send and process requests to a web server Send and process requests to a web server
determined by the size of the requestdetermined by the size of the request
FLEXFLEX
Locality awareLocality aware load-balancing strategy load-balancing strategy based on two factors:based on two factors:– Accessed files, memory requirementsAccessed files, memory requirements– Access rates (working set), load requirementsAccess rates (working set), load requirements
PartitionsPartitions all servers into equally all servers into equally balanced groupsbalanced groups
Each server Each server transferstransfers the response to the the response to the browser to reduce bottleneck through the browser to reduce bottleneck through the router (TCP Handoff)router (TCP Handoff)
File DistributionFile Distribution RoutingRouting
Content/Locality Content/Locality AwareAware
DNS ServerDNS Server
Size AwareSize Aware Centralized RouterCentralized Router
Workload AwareWorkload Aware Distributed Distributed DispatcherDispatcher
Flex DiagramFlex Diagram
DNS DNS ServerServer
Requests
To Client Browser
ForwardsRequest
S1 S2
S3 S4
S5 S6
W(S1) ≈ W(S2) ≈ W(S3) ≈ … ≈ W(S6)
Ar(S1) ≈ Ar(S2) ≈ Ar(S3) ≈ … ≈ Ar(S6)
FLEX Cont.FLEX Cont.Advantages:Advantages:– Highly scalableHighly scalable– Reduces bottleneck by the load balancerReduces bottleneck by the load balancer– No software is requiredNo software is required– Reduces number of cache missesReduces number of cache misses
FLEX Cont. IIFLEX Cont. II
Disadvantages:Disadvantages:– Not dynamic, routing tale must be recreated Not dynamic, routing tale must be recreated – Only compared to RROnly compared to RR– Number of access logs required on each Number of access logs required on each
server could be tremendousserver could be tremendous– Responsibility of load-balancing and Responsibility of load-balancing and
transferring response is given to web servers transferring response is given to web servers – unorganized responsibility– unorganized responsibility
– How often to update access rates and working How often to update access rates and working sets? Monitor?sets? Monitor?
WARDWARD
Workload-Aware Request Distribution StrategyWorkload-Aware Request Distribution StrategyServer coreServer core are essential files that represent are essential files that represent majority of expected requests majority of expected requests Server core is Server core is replicatedreplicated at every server at every serverWard-analysis computes the nearly optimal core Ward-analysis computes the nearly optimal core size determined by size determined by workloadworkload access patterns access patterns– Number of nodesNumber of nodes– Node RAMNode RAM– TCP handoff overheadTCP handoff overhead– Disk access overheadDisk access overhead
File DistributionFile Distribution RoutingRouting
Content/Locality Content/Locality AwareAware
DNSDNS Server Server
Size AwareSize Aware Centralized RouterCentralized Router
Workload AwareWorkload Aware Distributed Distributed DispatcherDispatcher
WARD Cont.WARD Cont.Three components: dispatcher (load Three components: dispatcher (load balancer), distributor (router), web serverbalancer), distributor (router), web server
Three progressive architectures:Three progressive architectures:
Dispatcher
Distributor
Front End
LAN
Server
Server
Server
DispatcherDistributor
Front End
LAN
Server
ServerDistributor
SwitchDispatcher
Distributor
Front End
LAN
Server
ServerDistributor
Switch
Dispatcher
Single front-end distributor, centralized dispatcher
Co-located distributor and server
Co-located distributor, server, and dispatcher
LARD CARD WARD
WARD DiagramWARD Diagram
Requests
S1S2
S3
S4
S5S6
Queue:Queue:
Queue:
Queue:
Queue:Queue:
SwitchSwitch
•Each computer is a distributor and a dispatcher
WARD Cont. IIWARD Cont. II
Similar to FLEX, sends response directly to Similar to FLEX, sends response directly to clientclient
Minimizes forwarding overhead from handoffs Minimizes forwarding overhead from handoffs for the most frequent filesfor the most frequent files
Optimizes the overall cluster RAM usageOptimizes the overall cluster RAM usage
““by mapping a small set of most frequent files to by mapping a small set of most frequent files to be served by multiple number of nodes, we can be served by multiple number of nodes, we can improve both locality of accesses and the cluster improve both locality of accesses and the cluster performance significantly”performance significantly”
WARD Cont. IIIWARD Cont. III
Advantages:Advantages:– No decision making, core files are replicated No decision making, core files are replicated
on every serveron every server– Minimizes transfer of requests and disk reads, Minimizes transfer of requests and disk reads,
both are “equally bad”both are “equally bad”– Outperforms Round RobinOutperforms Round Robin– Efficient use of RAMEfficient use of RAM– Performance gain with increased number of Performance gain with increased number of
nodesnodes
WARD Cont. IVWARD Cont. IV
Disadvantages:Disadvantages:– Core files are created on past day’s data, Core files are created on past day’s data,
could decrease performance up to 15%could decrease performance up to 15%– Distributed dispatcher increases the number Distributed dispatcher increases the number
of TCP requests transfersof TCP requests transfers– If core files not selected correctly, higher If core files not selected correctly, higher
cache miss rate and increased disk accessescache miss rate and increased disk accesses
EquiLoadEquiLoad
Determines which server will process a Determines which server will process a request determined by the size of the request determined by the size of the requested filerequested file
Splits the content on each server by file Splits the content on each server by file size, forcing the queues sizes to be size, forcing the queues sizes to be consistent.consistent.
File DistributionFile Distribution RoutingRouting
Content/Locality Content/Locality AwareAware
DNSDNS Server Server
Size AwareSize Aware Centralized RouterCentralized Router
Workload AwareWorkload Aware Distributed Distributed DispatcherDispatcher
EquiLoad Solves Queue Length EquiLoad Solves Queue Length ProblemsProblems
This is badThis is bad
QueueQueue
QueueQueue
This is This is betterbetter
QueueQueue
QueueQueue
1k1k 1k1k 1000k1000k
1k1k 1k1k 1k1k 2k2k 2k2k 1k1k 1k1k 2k2k 1k1k
100k100k 100k100k
1k1k 1k1k 1k1k 2k2k 100k100k
EquiLoad DiagramEquiLoad Diagram
DistributorDistributor
Requests
To Client Browser
ForwardsRequest
S1 S2
S3 S4
S5 S6
1k-2k 2k-3k
3k-10k 10k-20k
20k-100k >100k
DispatcherDispatcher(periodically calculates (periodically calculates partitions)partitions)
EquiLoadEquiLoad
AdvantagesAdvantages– Dynamic repartitioningDynamic repartitioning– Can be implemented at various levelsCan be implemented at various levels
DNSDNS
DispatcherDispatcher
ServerServer
– Minimum queue buildupMinimum queue buildup– Performs well under variable workload and Performs well under variable workload and
high system loadhigh system load
EquiLoadEquiLoad
DisadvantagesDisadvantages– Cache affinity is neglectedCache affinity is neglected– Requires a front end dispatcherRequires a front end dispatcher– Distributor must communicate with serversDistributor must communicate with servers– Thresholds of parameter adjustmentThresholds of parameter adjustment
EquiLoad EquiLoad AdaptLoad AdaptLoad
AdaptLoad improves upon EquiLoad using AdaptLoad improves upon EquiLoad using “fuzzy boundaries”“fuzzy boundaries”– Allows for multiple servers to process a Allows for multiple servers to process a
requestrequest– Behaves better in situations where server Behaves better in situations where server
partitions are very close in sizepartitions are very close in size
AdaptLoad DiagramAdaptLoad Diagram
DispatcherDispatcher
Requests
To Client Browser
ForwardsRequest
S1 S2
S3 S4
S5 S6
1k-3k 2k-4k
3k-10k 8k-20k
15k-100k >80k
DistributorDistributor(periodically calculates (periodically calculates partitions)partitions)
SummarySummary
File DistributionFile Distribution RoutingRouting
Content/Locality Content/Locality AwareAware
DNS ServerDNS Server
Size AwareSize Aware Centralized Centralized RouterRouter
Workload AwareWorkload Aware Distributed Distributed DispatcherDispatcher
FLEX
EquiLoad, AdaptLoad
WARD
ConclusionsConclusions
There is no “best” way to distribute content There is no “best” way to distribute content among servers.among servers.
There is no optimal policy for all website There is no optimal policy for all website applications.applications.
Certain strategies are geared towards a Certain strategies are geared towards a particular website application.particular website application.
Future WorkFuture Work
Compare and contrast the three policiesCompare and contrast the three policies
Figure out how often nodes should be Figure out how often nodes should be repartitionedrepartitioned
Compare each policy to a standard Compare each policy to a standard benchmarkbenchmark
Figure out which policy works in a Figure out which policy works in a particular environmentparticular environment