18 june 2001 optimizing distributed system performance via adaptive middleware load balancing ossama...
TRANSCRIPT
18 June 2001
Optimizing Distributed System Performance via Adaptive
Middleware Load BalancingOssama [email protected]
Douglas C. [email protected]
Department of Electrical and Computer Engineering
University of California, Irvine
Irvine, California 92697 USA
University of California, Irvine
O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing
Introduction
Network
Clients
Req
uest
s
Rep
lies
IDC
Servers • Motivation– Given a resource
intensive distributed application
• Clients typically greatly out number servers
• Some servers can be more loaded than others
• Requests generated by clients are often “bursty” and unpredictable
• Solution– Adaptive load
balancing
University of California, Irvine
O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing
Basic Scenario and Concepts
LoadBalancer
Clients
Req
uest
s
Rep
lies
ReplicasReplica Groups
• Load balancing goals– Use load balancing to
distribute client requests equitably among several replicas, within a replica group
– Ensure differences in replica loads are kept to a minimum
• Common Problem– Load balancing
algorithms in use may be very good but underlying mechanism is often inefficient
University of California, Irvine
O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing
Load Balancing Strategies• Client binding granularity
– Per-session• Client permanently
forwarded to a replica– Per-request
• Requests forward on client’s behalf
– On-demand• Client can be rebound
to another replica whenever necessary
• Balancing policy– Non-adaptive
• No load feedback used when binding clients
– Adaptive• Load feedback taken in
to account
: Client : Replica
: Load Balancer
1. send_re
quest() 2. send_request()
: Client : Replica
: Load Balancer
1. send_re
quest()
2. L
OCATION_FORW
ARD()
3. send_request()
Per-session / On-Demand
`
Per-request
: Load Balancer
load_advisory()report_load()
: Replica
Adaptive
University of California, Irvine
O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing
Load Balancing Architectures• Load balancing
architecture comprised of a combination of client binding granularity and balancing policy
• Given the strategies just described, there are six possible architectures
• Three common architectures– Non-adaptive per-
session– Adaptive per-request– Adaptive on-demand
: Client : Server Replica
: Load Balancer
1. send_request()
2. LOCATIO
N_FORWARD()
3. send_request()
: Client : Server Replica
: Load Balancer
1. send_request()
2. send_request()
load_advisory()
report_load()
The front-endserver proxy.
: Client : Server Replica
: Load Balancer
1. send_request()
2. LOCATIO
N_FORWARD() load_advisory()
report_load()
3. send_request()
Non-adaptive Per-Session
Adaptive Per-request
Adaptive On-demand
University of California, Irvine
O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing
Load Balancing Experiment Testbed
Dual CPUReplica Host
Dual CPULoad Balancer
HostQuad CPUClient Host
Dual CPUReplica Host
100 MBpsNetwork Switch
• Testbed hardware– Client host (1)
• Quad CPU 400 MHz Pentium II Xeon, 1GB RAM
– Replica hosts (2), and load balancer host (1)
• Dual CPU 733 MHz Pentium III, 512MB RAM
• Testbed software– Debian GNU/Linux 2.1
“potato” (glibc 2.1, kernel 2.2.16)
– TAO “Latency” performance test
University of California, Irvine
O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing
Latency OverheadLatency Comparison
0
100
200
300
400
500
600
700
Classic LatencyPerformance Test
Latency Test w/Per-Session Load
Balancer
Latency Test w/Per-Request Load
Balancer
Latency Test w/TAOOn-Demand Load
Balancer
Lat
ency
(u
secs
)
Latency - Jitter
Average Latency
Latency + Jitter
University of California, Irvine
O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing
Throughput Overhead
Throughput Comparison
0500
10001500200025003000350040004500
Classic LatencyPerformance
Test
Latency Testw/Per-SessionLoad Balancer
Latency Testw/Per-RequestLoad Balancer
Latency Testw/TAO On-
Demand LoadBalancer
Th
rou
gh
pu
t (e
ven
ts p
er s
eco
nd
)
University of California, Irvine
O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing
Non-adaptive Per-session Effectiveness
Loads Under Non-Adaptive Per-Session Strategy
0
5
10
15
20
251 40 79 118
157
196
235
274
313
352
391
430
Elapsed Time (seconds)
Lo
ad (
req
ues
ts/s
eco
nd
)
Replica 1Replica 2Replica 3Replica 4
University of California, Irvine
O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing
Adaptive On-demand Effectiveness
Loads Under Adaptive On-Demand Strategy
020406080
100120140160180200
1 52 103
154
205
256
307
358
409
460
511
562
613
664
715
766
Elapsed Time (seconds)
Lo
ad (
req
ues
ts/s
eco
nd
)
Replica 1
Replica 2
Replica 3
Replica 4
University of California, Irvine
O. Othman, D.C. Schmidt Optimization via Adaptive Load Balancing
Conclusion• Load balancing can be performed at several levels
– The network level– The operating system level– The middleware level
• Network-based and OS-based suffer from several limitations– Inability to support application-defined load metrics
at run-time– Lack of adaptability due to absence of load feedback,
and lack of control over replicas• Middleware-based load balancing has a clear advantage
since it suffers from neither of these limitations