![Page 1: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/1.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 1
Case Study:Load Balancing
Thursday, 01 June 2006
![Page 2: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/2.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 2
IntroductionOptimize the utilization of resources to reduce the user response time In telecom load balancing goal is: search the optimal routing given a performance index valueIn computer networks lb goal is: find the optimalallocation of processes given distributed resources (e.g. minimize response time, maximize throughput)
![Page 3: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/3.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 3
Load Balancing categoriesStatic
Computed during capacity planning process (batch mode)
DynamicModified at runtime using dedicated hardware or software to collect performance indices (on-line mode)
![Page 4: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/4.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 4
Network topology – Example A
open single-class modelexponential arrival rate distribution with μ=1FIFO queue strategyservice time distribution const(1), exp(1), hyperexp(1,2)
![Page 5: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/5.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 5
Model definition - JMODELSee example Load_Balancing_Single_Class.jmodel
![Page 6: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/6.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 6
Performance indicesResponse Time at each stationSystem Response Time
![Page 7: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/7.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 7
1 – Random routingStatic routing strategyAll stations have the same probability to be selected
![Page 8: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/8.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 8
2 – Round Robin routingStatic routing strategySelect stations according to a cyclic algorithm
![Page 9: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/9.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 9
3 – Shortest Queue Length routingDynamic routing strategySelect station with the smallest queue length
![Page 10: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/10.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 10
4 – Least Utilization routingDynamic routing strategySelect station with the smallest average utilization
![Page 11: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/11.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 11
Results – Example A
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
Syst
em R
espo
nse
Tim
e
const(1) exp(1) hyper(1,2)Service Time Distribution
Random Round Robin Least Utilization Shortest Queue Length
![Page 12: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/12.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 12
Network topology – Example B
open single-class modelexponential arrival rate distribution with μ=1FIFO queue strategyhyperexponential service time distribution with μ=1 and increasing coefficient of variation c
![Page 13: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/13.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 13
Results – Example B
0
10
20
30
40
50
60
1 3 5 7 9 11 13 15 17 19
Coefficient of variation c
Syst
em R
espo
nse
Tim
e
Random Round Robin Least Utilization Shortest Q Length
![Page 14: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/14.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 14
QuestionIf we are forced to use a random routing strategy, how can we improve system response time in the case of hyperexponential service times with high values of c?
![Page 15: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/15.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 15
Network topology – Example C
exponential arrival rate distribution with μ=4hyperexponential service time distribution:
Two servers with μ = (1 + δ) and c = 4Two servers with μ = (1 - δ) and c = 4One server with μ = 1 and c = 4
from Asymptotic analysis we expect saturation with Random and Round Robin algorithms:
λi = λ / 5 = 4 / 5λi < 1 / DMAX 4 / 5 < 1 / (1 + δ)δ < 0.25
μ = (1 + δ)
μ = (1 - δ)
![Page 16: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/16.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 16
Results – Example C
0
5
10
15
20
25
30
0% 5% 10% 15% 20% 25% 30%δ
Syst
em R
espo
nse
Tim
e
Random Round Robin Least Utilization Shortest Q Length
![Page 17: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/17.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 17
Problem definitionFind optimal load balancing in a replicated web server farm
n serversRequests from a Poisson process with mean rate λService times exponentially distributed
Mean response time of server i is:Ri = 1 / (μi- λi) i = 1,2,…,n
μi and λi are the service rate and the arrival rate of device i
Mean response time of the subsystem is:
Compute optimal λ*i that minimize R given:
0 ≤ λi < μi
( ) ∑= −
=n
i ii
inR
121
1,,,λμ
λλ
λλλ K
∑=
=n
ii
1λλ
![Page 18: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/18.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 18
Chen algorithm (1)
∑
∑ ∑
=
= =
=+
=−=
n
jj
k
j
k
jjkj
nL
nkkL
1
1 1
)1(
,,2,1)(
μ
μμμ K
Order servers by increasing service times:μ1 ≥ μ2 ≥ … ≥ μn
Compute quantities L(k), given by:
![Page 19: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/19.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 19
Chen algorithm (2)Compute the value of k such that:
L(k + 1) > λ ≥ L(k)
The optimal values are given by:
ni
i
i
pp
pp
iii
,,10
,,2,1
*1
1*
K
K
+==
=−
−=
∑
∑
=
=
k
kk
k
λ
μ
λμμμλ
![Page 20: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/20.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 20
Chen algorithm – Example (1)
Find optimal load subdivision for replicated web servers:
two new servers (S1,2 = 1/μ1,2 = 8ms) four old servers (S3,4,5,6 = 1/μ3,4,5,6 = 12ms)
System arrival rate is λ = 88 req/s
![Page 21: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/21.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 21
Chen algorithm – Example (2)We compute L(k):
∑
∑ ∑
=
= =
=+
=−=
n
jj
k
j
k
jjkj
nL
nkkL
1
1 1
)1(
,,2,1)(
μ
μμμ K
L(1) = L(2) = 0L(3) = L(4) = L(5) = L(6) = 45.87L(7) = 585.33
Since λ = 88 req/sL(7) > 88 > L(6)k = 6
![Page 22: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/22.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 22
Chen algorithm – Example (3)Now we apply:
λ*1=λ*
2= 30.9 req/sλ*
3=λ*4= λ*
5=λ*6= 6.55 req/s
System response time R = 11.34msU1,2 = 24.72%U3,4,5,6= 7.84%
ni
i
i
pp
pp
iii
,,10
,,2,1
*1
1*
K
K
+==
=−
−=
∑
∑
=
=
k
kk
k
λ
μ
λμμμλ
![Page 23: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/23.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 23
Questions
What happen if arrival rate is λ = 44 req/s ?
What happen if arrival rate is λ = 586 req/s ?
![Page 24: Case Study: Load Balancing...exponential arrival rate distribution with μ=4 hyperexponential service time distribution: Two servers with μ= (1 + δ) and c = 4 Two servers with μ=](https://reader031.vdocument.in/reader031/viewer/2022011915/5fd3136fc9e6ec2da52c5eb2/html5/thumbnails/24.jpg)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 24
ReferencesBERTOLI, M. jSIM: Un Tool per la Simulazione di Modelli a Reti di Code. Tesi presso il Politecnico di Milano, 2005.BUZEN, J.P., AND CHEN, P.P. Optimal load balancing in memory hierarchies. In Information Processing 74. North-Holland, New York, 1974, pp. 271-275. CHEN, P.P. Optimal file allocation in multi-level storage systems. In AFIPS Conference Proceedings (New York, June 1973), vol. 42. AFIPS Press, Reston, Va., 1973, pp. 277-282. SERAZZI G., "The dynamic behavior of computer systems", in Experimental Computer Performance Evaluation, pp. 127-163, Elsevier, 1981.