energy-aware autonomic resource allocation in multitier virtualized environment … ·...
TRANSCRIPT
Energy-Aware Autonomic Resource Allocation in Multitier Virtualized Environment (in very Large Service Centers)
Danilo Ardagna
Dipartimento di Elettronica e Informazione, Politecnico di Milano [email protected]
ICEP - London 1st June 2012
This talk is based on….
1. D. Ardagna, B. Panicucci, M. Trubian, L. Zhang. Energy-aware autonomic resource allocation in multi-tier virtualized environments. IEEE Transactions on Services Computing, 5(1), 2-19, 2012.
2. B. Addis, D. Ardagna, B. Panicucci, L. Zhang. Autonomic management of cloud service centers with availability guarantees. Cloud 2010 Proceedings, 220-227, 2010.
3. M. S. Squillante, T. Nowicki, C. Wah Wu. Fundamentals of dynamic decentralized optimization in autonomic computing systems. Self-star Properties in Complex Information Systems, 2005.
2
Motivations
• Modern cloud infrastructures live in an open world, characterized by continuous changes in the environment
• Continuous changes occur autonomously and unpredictably, and they are out of control of the cloud provider
• Cloud providers main challenges • Costs (energy!) reduction • Performance levels improvement • Availability and dependability enhancement
3
Microsoft Service Centers
Google Service Centers
Microsoft: • Windows LiveID: >1Mld aut/day • MSN: 550M visitors per month • Hotmail: 400M users • Messenger: 320M users • Bing: >2Bill query/month Overall:
• 210K m2
• >200,000 servers
Google: • Number of searches July 2008: 48.7
billions • Peak number searches: 65 mln/h
Overall:
• 36 Service Centers
• >500,000 servers
4
Increasing complexity and size of Service Centers
Microsoft Service Centers
Google Service Centers
Microsoft: • Windows LiveID: >1Mld aut/day • MSN: 550M visitors per month • Hotmail: 400M users • Messenger: 320M users • Bing: >2Bill query/month Overall:
• 210K m2
• >200,000 servers
Google: • Number of searches July 2008: 48.7
billions • Peak number searches: 65 mln/h
Overall:
• 36 Service Centers
• >500,000 servers
Quincy: Ø Area: 43,600 m2 (10 football fields) Ø Consumption/Area: 54W/foot Ø Chilled water pipes: 4.8 km
Largest Google Service Center Ø Largest service center world wide Ø 45 containers, > 45,000 servers
5
Increasing complexity and size of Service Centers
Microsoft Service Centers
Google Service Centers
Microsoft: • Windows LiveID: >1Mld aut/day • MSN: 550M visitors per month • Hotmail: 400M users • Messenger: 320M users • Bing: >2Bill query/month Overall:
• 210K m2
• >200,000 servers
Google: • Number of searches July 2008: 48.7
billions • Peak number searches: 65 mln/h
Overall:
• 36 Service Centers
• >500,000 servers
Quincy: Ø Area: 43,600 m2 (10 football fields) Ø Consumption/Area: 54W/foot Ø Chilled water pipes: 4.8 km
Largest Google Service Center Ø Largest service center world wide Ø 45 containers, > 45,000 servers
6
Increasing complexity and size of Service Centers
Internet
Web Server2
Web Server1
Web Server3 Physical servers
Autonomic Self-Managment
7
Autonomic Self-Managment
Resource Allocation:
• Server ON/OFF
• Applications placement
• Frequency Scaling (DVFS)
• Load balancing
• Capacity allocation
8
Resource Allocation System
Our work
• Perspective of a Platform as a Service provider which has to manage the applications of its end customers:
• Providing response time and availability guarantees
• Minimizing its energy costs
• We propose a distributed hierarchical solution based on mixed integer non-linear optimization for the management of resources of very large cloud service centres, acting at multiple time-scales
9
Resource Allocation System: Hierarchical Structure
10
• Central Manager (CM)
• Partitions service applications and resources into clusters on the basis of workload forecasts for the next day (24h)
• Application Manager (AM)
• Determines the optimal resource allocation for a single cluster
• Server ON/OFF
• Application placement
• Frequency Scaling
• Load Balancing
• Capacity allocation
• Actuation mechanisms are raked according to their overhead
Fine grained multi-time scale resource
allocation
...
...
...
Free server pool
Central Manager
ApplicationManager
... ......
Virtual Machine Monitor
OperatingSystem
App
OperatingSystem
App
VM VM
- Load balancing- Capacity allocation - Frequency scaling
- Servers switching - Application placement
- Classes partitioning - Servers partitioning
Application Manager
T1
T2
T3
Cluster Cluster
• Open queueing network model: heterogeneous service centers and a delay center
• VMM modelling: Generalized Processor Sharing (GPS) scheduling
• VMs running the same tiers are replicated on multiple physical servers and work in load-sharing
Hexogenous arrival rate
Session modelling
Service centers model physical servers which support VMs execution
A class k request becomes a request k’ with probability πk,k’ or terminates
Service Center Performance and Availability Models
π
11
Evaluation of request response time
• GPS bounding technique to approximate the queueing system (Towsley et al. Statistical Analysis of the GPS Scheduling Discipline. IEEE J. Selected Areas in Comm. 1995)
• The server i capacity devoted to class k requests for tier j at time t is:
• The average response time for the execution of the process at tier j of class k requests at server i can be evaluated as:
• The average response time for class k requests is:
12
Interrelationship among the time scales T1, T2, and T3
0"
50"
100"
150"
200"
250"
300"
350"
T1
T2
T3
Time [min]
Req
uest
arr
ival
[req
/sec
]
⇤tk(T2)
⇤tk(T3)
13
Revenues are a function of average response times
Average response time soft-constraint
SLA – Service Level Agreement
⌥k
RkRk
↵k
vk
14
Servers power model
Dispatcher
HTTP servers tier
!k
Think time
"k,k’
DBMS servers tier
Application servers tier
-ak
vk
Rk Rk
Vk
bi,h
(a) (b) (c)Ui
pi,h+ci
15
R. Raghavendra, P. Ranganathan, V. Talwar, Z. Wang, X. Zhu. No ‘Power’ Struggles: Coordinated Multi-Level Power Management for the Data Center. SIGARCH Computer Architecture News, 36(1), 48-59, 2008.
System parameters
K Set of request classes
Nk Set of applications involved in the execution of class k request
I Set of servers available at the cloud service center
C Set of clusters
Hi Set of operating frequencies for server iCi,h Capacity of server i when it is working at frequency h 2 Hi
Ui Server i CPU utilization
RAM i RAM available at server iRAMk,j RAM required for the execution of the application tier j for class kRk Class k average response time
Ri,k,j Average response time of server i while executing the application at tier j for class k requests
T1 Long-term control time horizon
T2 Medium-term control time horizon
T3 Short-term control time horizon
nt2 Number of time bands T2 within the period T1
nt3 Number of time bands T3 within the period T2
16
SLA and Costs Parameters
Ak Class k availability threshold
Rk Class k revenue/penalty region threshold
�k Request class k maximum utility function value
↵k Request class k utility function slope
ai Server i availabilityci Time unit cost for server i when it is in low power sleep state
ci+pi,h Time unit static cost for server i when it is working at frequency h 2 Hi
bi,h Time unit dynamic cost for server i when it is working at frequency h 2 Hi
csi Cost for switching server i from low power sleep to active state
cm Cost associated with moving VMs to a di↵erent server
17
Workload Prediction and Monitoring Parameters
⇤tk(T2) Overall incoming workload for class k request in the time band t at time scale T2
⇤tk(T3) Overall incoming workload for class k request in the time band t at time scale T3
µk,j Maximum service rate of a capacity 1 server for executing application at tier j for class krequests
18
Optimization problems formulation
• Maximize profits:
• Revenues - Costs
• The CM determines one day ahead:
• The class partitions
• Server assignment to each cluster c and for every
time band
• We will consider first the AM solution acting at time scale T2 (medium-term problem)
19
{Kc}Itc
t 2 [1..nt2 ]
Decision variables – T2
20
x
ti 1 if server i is running at the time band t, 0 otherwise
y
ti,h 1 if server i is working at the time band t with frequency h, 0 otherwise
z
ti,k,j 1 if the application tier j for class k is assigned to server i at the time band t, 0 otherwise
�
ti,k,j Rate of execution for class k at tier j on server i at the time band t
�
ti,k,j Fraction of capacity devoted for executing VM at tier j for class k at server i at the time band
t
w
ti,k 1 if at least an application tier j for class k is assigned to server i at the time band t, 0 otherwise
Class k average response time
Utility function evaluation
Static servers cost
Servers switiching cost
VMs migrations cost
Optimization Problem – Objective function
2
66664
X
k2Kc
0
BBBB@�↵k
X
i2Itc,j2Nk
�ti,k,j
Ph2Hi
Ci,h yti,h
!µk,j �t
i,k,j � �ti,k,j
1
CCCCA+X
k2Kc
�k⇤tk(T2)�
X
i2Itc
ci+
�X
i2Itc,h2Hi
0
B@pi,hyti,h + bi,h y
ti,h
Pj2Nk,k2Kc
�
ti,k,j
Ph2Hi
Ci,h µk,j yti,h
1
CA
3
75 T2 �X
i2Itc
csi max(0, x
ti � x
t�1i ) +
�X
i2Itc,k2Kc,j2Nk
cmmax(0, z
ti,k,j � z
t�1i,k,j)
Dynamic servers cost
21
X
i2Itc
�
ti,k,j = ⇤t
k(T2) = ⇤tk,j(T2) 8k 2 Kc, j 2 Nk,
X
k2Kc,j2Nk
�
ti,k,j 1 8i 2 It
c,
X
h2Hi
y
ti,h = x
ti 8i 2 It
c,
z
ti,k,j x
ti 8i 2 It
c, k 2 Kc, j 2 Nk,
�
ti,k,j ⇤k(T2)
tz
ti,k,j 8i 2 It
c, k 2 Kc, j 2 Nk,
All incoming requests have to be served
At most 100% server capacity can be used
Application can be allocated only on servers ON Allocate workload were the corresponding application is running
If a server is running only one operating frequency can be selected
Optimization Problem – Constraints
22
�
ti,k,j
X
h2Hi
Ci,hyti,h
!µk,j�
ti,k,j � ✏ 8i 2 It
c, k 2 Kc, j 2 Nk
X
j2Nk
RAMk,jzti,k,j RAM i 8i 2 It
c, k 2 Kc,
Y
j2Nk
0
@1�Y
i2Itc
(1� ai)wt
i,k
1
A � Ak 8k 2 Kc
NkX
j=1
z
ti,k,j Nkw
ti,k 8i 2 It
c, k 2 Kc
�
ti,k,j ,�
ti,k,j � 0, x
ti, y
ti,h, z
ti,k,j , w
ti,k 2 {0, 1} 8i 2 It
c, k 2 Kc, j 2 Nk.
Queue equilibrium
Optimization Problem – Constraints
Servers RAM is not saturated
Availability is guaranteed
w is raised to 1 if at least one VM of the class k is allocated on server i
23
Optimization Model Analysis
• The optimization problem is a NP-hard non linear Mixed Integer Programming problem
• Even if the set of server ON is fixed, the joint capacity allocation and load balancig problem is difficult since the objective function is neither concave and neither convex
24
• Problem decomposition:
• Initial solution: Assign VMs to physical servers (problem equivalent to a special case of a CFLP, Capacitated Facility Location Problem)
• Fixed Point Iteration: Optimum load balancing and capacity allocation
• A local search iteratively improves the solution
Heuristic solution
25
• The set of servers ON and frequency of operations are held fixed
• Iteratively identifies the optimum value of a set of variables (λ-s or φ-s), while the value of the other one (alternatively φ-s or λ-s) is held fixed
• Based on Karush-Kuhn-Tucker (KKT) conditions
Fixed Point Iteration
Initial solution • Greedy algorithm which identifies a new solution from the solution of the
previous control time (T2) horizon:
• Reduce the number of servers switches
• Reduce the number of VM migrations
• Improve system stability
• Reduced optimization time
Heuristic solution
• Identifies a local optimum with respect to the integer variables x, y, z, and w, while the continuous variables (λ and φ) are updated in a greedy way
• Diversification of the Neighborhood search is obtained by the relaxation of availability constraints
Local search
26
AM optimization algorithm (T2 time scale)
27
12
Obj 4 pages
In this section we discuss the optimization technique used for solving the medium-term (P T2c ) and the
short (P T3c ) resource allocation problems. Computational results show that only small instances of these
problems can be solved to optimality with commercial non-linear optimization packages. To deal with
realistic problems of reasonable size we propose a heuristic approach (see Algorithm 1).
1 (x,y,z, �,�)=Initial Solution();2 FPI(�,�);3 STOP=false;4 while not STOP do5 while improve do6 while improve do7 Local Search Availability(x,y,z, �,�);
end8 FPI(�,�);
end9 (x,y,z,ˆ�,ˆ�)=Save Best Solution();
10 while iteration limit do11 Local Search withoutAvailability(x,y,z, �,�);
end12 Remove Violations(x,y,z, �,�);13 if not feasible solution then14 STOP=true;15 (x,y,z, �,�)=(x,y,z, ˆ�,ˆ�);
endendreturn (x,y,z, �,�);
Algorithm 1: Resource allocation optimization procedure
The basics of the overall procedure are:
(i) Finding an initial solution: For the medium-term problem we perform an heuristic method, called
Initial Solution() whose output is a set of servers in active state x, their corresponding operating frequencies
y, the assignment of tiers to servers z, the initial capacity allocation � = (�i,k,j)i2I,k2K,j2Nkand the load
balancing � = (�i,k,j)i2I,k2K,j2Nk. The details of the heuristic are in Section V-A1. For the short-term
problem the initial solution is the current one.
(ii) Improving the current solution with a local-search method, which consists on two two main phases:
First, a fixed point iteration procedure (FPI) searches for a local improvement of the capacity allocation and
load balancing decisions. The FPI changes variables �, while keeping � fixed, and vice-versa, by solving
the load balancing and capacity allocation problems, as described in Section V-A2. The current solution is
then enhanced iteratively with a local search procedure, that is a process upon which an exhaustive search
is performed in the neighborhood of the current solution in order to derive a local optima. A neighbour
of the current solution is constructed through changing the current one by using the so called “moves,” as
itemized in Section V-A3.
DRAFT
Initial solution
28
Servers are ordered a.t. non-increasing perf./cost ratio
⇤tk,j(T2)
�ti,k,j
Ui = U
�ti0,k,j
�ti0,k0,j
⇤tk0,j(T2)
⇤tk,j(T2)
Pj2Nk
1µk,j
Applications tiers are ordered a.t. non-increasing value
Fixed Point Iteration Technique
• The Fixed Point Iteration iteratively solves the load balancing and capacity allocation problems
• Load balancing and capacity allocation problems are convex
• We can not guarantee that the procedure converges to a global optimum but the procedure will always converge
29
FPI - The capacity allocation sub-problem
• The problem is separable and capacity allocation sub-problems (one for each server in each cluster) can be solved independently
• The objective function is convex, the optimum solution of each sub-problem can be identified by the KKT conditions
minPk2K
↵kP
i2Iactive,j2Nk
�i,k,j
Ciµk,j�i,k,j��i,k,j
Pk2K,j2Nk
�i,k,j 1 8i 2 Iactive,
�i,k,j < Ciµk,j�i,k,j , 8i 2 Iactive, k 2 K, j 2 Nk,
�i,k,j � 0 8i 2 Iactive, k 2 K, j 2 Nk.
|Iactive|
30
φ-iteration based on KKT
(P-CA)
31
minPk2K
↵kP
j2Nk
�k,j
Cµk,j�k,j��k,j
X
k2K,j2Nk
�k,j 1,
�k,j >�k,j
Cµk,j8k 2 K, j 2 Nk.
φ-iteration based on KKT
• Theorem 1: In the optimal solution of problem (P-CA) let
be the set of indexes (k,j) s.t. , and let an arbitrarily element in . Then:
where
• The overall complexity for the φ-iteration is:
NJ�k,j > 0 (k, j)
�k,j =1
Cµk,j
✓rakµk,j�k,j
akµk,j�k,j
(Cµk,j�k,j � �k,j) + �k,j
◆8(k, j) 2 NJ � {(k, j)},
�k,j =�k,j
Cµk,j
+1�
Pk2K
Pj2Nk
�k,jCµk,j
Pk2K
Pj2Nk
sakµ
k,j�k,j
akµk,j�k,j
NJ
O
✓|Iactive|
Pk2K
|Nk|◆
32
Theorem 1 Proof
• The Lagrangian of each sub-problem is:
33
L(�, L, Lk,j) =X
k2K
↵k
X
j2Nk
�k,j
⇥k,j�k,j � �k,j
� L
0
@1�X
k2K
X
j2Nk
�k,j
1
A+
�X
k2K
X
j2Nk
Lk,j
�⇥k,j�k,j � �k,j
�
Theorem 1 Proof
• The KKT conditions of optimality are:
34
L
0
@1�X
k2K
X
j2Nk
�k,j
1
A = 0
�k,j � 1 0 8k 2 K, j 2 Nk
�k,j < �k,j ⇥k,j 8k 2 K, j 2 Nk
Lk,j
�⇥k,j�k,j � �k,j
�= 0 8k 2 K, j 2 Nk
L � 0 8k 2 K, j 2 Nk
Lk,j � 0 8k 2 K, j 2 Nk
@L(�, L, Lk,j)
@�k,j=
�↵k⇥k,j�k,j
(⇥k,j�k,j � �k,j)2+
+L� Lk,j⇥k,j = 0 8k 2 K, j 2 Nk,
Theorem 1 Proof
• Since multipliers are zero
• If L=0, it follows that
and there is no solution
• Then L>0, and hence
• For any given (k1,j1), (k2,j2):
35
�k,j < ⇥k,j�k,j , Lk,j
�↵kUk,j�k,j
(Uk,j�k,j��k,j)2= 0
Pk2K,j2Nk
�k,j = 1
�↵k1⇥k1,j1�k1,j1
(⇥k1,j1�k1,j1 � �k1,j1)2=
�↵k2⇥k2,j2�k2,j2
(⇥k2,j2�k2,j2 � �k2,j2)2.
Theorem 1 Proof
• Then we get:
• and:
36
�k,j =1
⇥k,j
s↵k⇥k,j�k,j
↵k⇥k,j�k,j
(⇥k,j�k,j � �k,j) + �k,j
!
X
k2K
X
j2Nk
1
⇥k,j
s↵k⇥k,j�k,j
↵k⇥k,j�k,j
(⇥k,j�k,j � �k,j) + �k,j
!= 1
�k,j =�k,j
⇥k,j
+
1�Pk2K
Pj2Nk
�k,j
⇥k,j
Pk2K
Pj2Nk
r↵k⇥k,j�k,j
↵k⇥k,j�k,j
FPI – The load balancing sub-problem
• The solution of the load balancing sub-problem can be determined similarly
• The problem is separable and load balancing sub-
problems (one for every tier of every class) can be solved
independently
• The overall complexity for the λ-iteration is:
37
Pk2K
|Nk|
O
✓|Iactive|2
Pk2K
|Nk|◆
Local search
• Given a feasible solution the Local Search tries to improve it:
• Starts with an initial feasible solution: the current solution
• A set of solutions close to the current one, the neighborhood, is generated
• An improving solution is searched among the neighbor solutions
• Remarks:
• Local search does not build one alternative solution, but a set of solutions
• The set of neighbor solutions is built by partially modifying the current solution
• Local search moves from feasible solution to feasible solution
38
Local search
LocalSearch(InitialSolution){
x:= InitialSolution; build N(x); x:=σ(x); //the best solution of the neighborhood σ(x) is selected
while (x!=σ(x)) {
build N(x);
x:=σ(x);
}
39
• x is the current solution
• σ(x) is a function which selects the best solution in the neighborhood
The Local Search Algorithm
• The FPI allows to obtain a local optimum for the joint load balancing and capacity allocation problems, when a service center configuration in terms of servers ON, operating frequencies, and allocation of application tiers to servers is given
• The neighborhood is defined by moves based on the switching ON or OFF of servers, frequency scaling, and the re-allocation of application tiers on servers
• We can not run the FPI for every neighborhood candidate solution, it is too time consuming
40
The Local Search Algorithm
• To overcome this difficulty, during the neighborhood exploration we overestimate the value of each candidate solution by updating the values of a subset of the variables λ-s and φ-s
• The neighborhood exploration is obtained by applying four different moves:
• Turning ON a server
• Turning OFF a server
• Frequency scaling (increasing and decreasing)
• Servers swapping
• Re-allocation of tiers to servers
• When the local-search stops at a local optimum, then we execute the fixed point iteration again (try to escape from the local optima)
41
AM optimization algorithm (T2 time scale)
42
12
Obj 4 pages
In this section we discuss the optimization technique used for solving the medium-term (P T2c ) and the
short (P T3c ) resource allocation problems. Computational results show that only small instances of these
problems can be solved to optimality with commercial non-linear optimization packages. To deal with
realistic problems of reasonable size we propose a heuristic approach (see Algorithm 1).
1 (x,y,z, �,�)=Initial Solution();2 FPI(�,�);3 STOP=false;4 while not STOP do5 while improve do6 while improve do7 Local Search Availability(x,y,z, �,�);
end8 FPI(�,�);
end9 (x,y,z,ˆ�,ˆ�)=Save Best Solution();
10 while iteration limit do11 Local Search withoutAvailability(x,y,z, �,�);
end12 Remove Violations(x,y,z, �,�);13 if not feasible solution then14 STOP=true;15 (x,y,z, �,�)=(x,y,z, ˆ�,ˆ�);
endendreturn (x,y,z, �,�);
Algorithm 1: Resource allocation optimization procedure
The basics of the overall procedure are:
(i) Finding an initial solution: For the medium-term problem we perform an heuristic method, called
Initial Solution() whose output is a set of servers in active state x, their corresponding operating frequencies
y, the assignment of tiers to servers z, the initial capacity allocation � = (�i,k,j)i2I,k2K,j2Nkand the load
balancing � = (�i,k,j)i2I,k2K,j2Nk. The details of the heuristic are in Section V-A1. For the short-term
problem the initial solution is the current one.
(ii) Improving the current solution with a local-search method, which consists on two two main phases:
First, a fixed point iteration procedure (FPI) searches for a local improvement of the capacity allocation and
load balancing decisions. The FPI changes variables �, while keeping � fixed, and vice-versa, by solving
the load balancing and capacity allocation problems, as described in Section V-A2. The current solution is
then enhanced iteratively with a local search procedure, that is a process upon which an exhaustive search
is performed in the neighborhood of the current solution in order to derive a local optima. A neighbour
of the current solution is constructed through changing the current one by using the so called “moves,” as
itemized in Section V-A3.
DRAFT
Preserving availability constraints limits the solution space exploration
The current solution stabilizes, no improving moves exist
Local search implementation
43
If a local optima is obtained, availability contraints are relaxed and the current solution is unfeasible
The availability is forced again and the move can worsen the current solution value
The fixed point iteration performes load balancing and capacity allocation
If the current solution is improved, the search continues preserving availability constraints until a local optima is identified
Local search implementation
44
Short-term problem solution (T3)
• Based on the algorithm discussed before
• The only move allowed in the Local Seach is the frequency scaling
• This move does not introduce any availability violation
• Only the Local_Search_Availability() and FPI() procedures are executed (decision variables: y, λ, φ)
45
Experimental Results
• Instances randomly generated: the number of servers has been varied between 80 and 400. Clusters up to 200 requests classes have been considered and the number of tiers has been varied between 2 and 5
• Service times were randomly generated and for each test case the load was increased in a way that the utilization the cluster resources varied between 0.2 and 0.8
• Nk, αk and Rk values have been randomly generated, Rk is proportional to the number of tiers Nk and to the overall demanding time at various tiers of class k
• Per hour CPU/software revenues between 1 and 10$
• Cooling overhead 0.7 Time in sec
46
• Real log traces (Politecnico di Milano Web site), 10 requests classes
• Comparison with IBM Tivoli resource allocation policies
Scenario 1: users come from the same time zone
Scenario 2: users come from different time zones
IBM Tivoli comparison
47
IBM Tivoli comparison
• Incoming workload is evenly shared among physical servers and the scheduling policy applied at each server is processor sharing
• The application placement problem is reduced to a variant of the class constrained multiple-knapsack problem and a very efficient heuristic procedure is proposed (reduce the number of applications starts and stops, while equally balancing the load of physical machines)
• The number of servers to be adopted at each physical tier is determined by a local search algorithm which moves (homogeneous) servers from one tier to another with the aim at improving bottleneck request response time
48
SCENARIO 1
IBM Tivoli comparison
49
SCENARIO 2
IBM Tivoli comparison
50
Test on a real prototype environment
• SPECweb2005 JSP implementation and RUBBoS benchmarks
• The system is based on three physical servers based on a Intel Nehalem dual socket quad-core CPUs with 24GB of RAM which run Xen 3.4.1-rc10 (default credit scheduler). Each core has five P-states, corresponding to frequencies of 2.4, 2.27, 2.13, 2.00, and 1.6 GHz
• The first sever hosts the load generators, the remaining two are the System Under Tests (SUTs)target of our analyses and hosts the Web and DBMS server instances
• The Xen hypervisor is assigned to a dedicated core on the first socket of each server, while the VMs running the Web and DBMS instances are assigned to the cores of the second socket by setting CPU affinity
• The SPECweb2005 back-end simulators are assigned to a dedicated core on the second socket, they are always overprovisioned and their performance will be neglected
51
Test on a real prototype environment
• The validation in the real test bed is based on two case studies:
• Case a: Two classes single tier system based on SPECweb2005 e-commerce and banking workloads
• We set α1=α2=1, ,
• Case b: Three request classes: Class 1 and 2 run SPECweb2005 e-commerce and banking respectively, while class 3 executes RUBBoS. We set α1=α3=5, α2=10,
52
R1 = 1.4s R2 = 0.7s
R1 = 0.85s, R2 = 0.40s, R3 = 1.20s
Case a
5 10 15 20 2510
12
14
16
18
20
22
24
26
Time [minutes]
Thro
ughput [r
eq/s
ec]
Lambda1
Lambda2
53
Case a
5 10 15 20 250.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Time [minutes]
Resp
onse
tim
e [s]
R1
R2
R1
R2
54
• The two VMs share a single core running at the maximum frequency
• The medium-term algorithm is run
Case a
5 10 15 20 250.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Time [minutes]
Resp
onse
tim
e [s]
R1
R2
R1
R2
55
• Another VM hosting class 1 Web server is assigned to a dedicated core of the first SUT server which run also at the maximum available frequency
Case a
5 10 15 20 250.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Time [minutes]
Resp
onse
tim
e [s]
R1
R2
R1
R2
56
• φ and λ are updated every 5 minutes running the short-term algorithm
Case a
5 10 15 20 250.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Time [minutes]
Resp
onse
tim
e [s]
R1
R2
R1
R2
57
• In the last time interval, the incoming workload is further reduced, the two cores are set to the minimum frequency
Case b
0 50 100 150 200 250 300 350 400 450 5000
100
200
300
400
500
600
700
800
Time [minutes]
Nu
mb
er
of
Co
ncu
rre
nt
Use
rs
N1
N2
N3
58
Case b
0 50 100 150 200 250 300 350 400 450 5000.2
0.4
0.6
0.8
1
1.2
1.4
1.6
Time [minutes]
Resp
onse
tim
e [s]
R1R2R3RT1RT2RT3R1!ModelR2!ModelR3!Model
59
• Server 1 hosts: • On a first core three VMs running the Web server for the
three classes • On a second core a VM running class 3 database • The two cores are set to the maximum frequency P0
• Server 2 hosts: • On two dedicated cores two additional instances of
the Web server for class 1 and 2 which are set to P1
• φ and λ are updated according to the short-term algorithm
Case b
0 50 100 150 200 250 300 350 400 450 5000.2
0.4
0.6
0.8
1
1.2
1.4
1.6
Time [minutes]
Resp
onse
tim
e [s]
R1R2R3RT1RT2RT3R1!ModelR2!ModelR3!Model
60
• An additional Web servers is assigned to a dedicated core of server 1, for class 3
• An additional instance of the Web server is instantiated for class 2 which is assigned to a dedicated core on server 2
Case b
0 50 100 150 200 250 300 350 400 450 5000.2
0.4
0.6
0.8
1
1.2
1.4
1.6
Time [minutes]
Resp
onse
tim
e [s]
R1R2R3RT1RT2RT3R1!ModelR2!ModelR3!Model
61
• The cores of the two servers are set to P3 and P4
• The frequency of operation is raised again to P1 and P2
The long-term problem (T1)
• Each AM represents a subsystem of the global Cloud Service Center, operating on a subset of applications and resources
• Each AM has to assign the available physical servers to the given subset of applications in an optimal way at time scales T2 and T3
• The CM determines the different partitions of applications and servers to be assigned to each AM:
• At the beginning of each day, the whole application classes are partitioned using a heuristic procedure, considering the workload forecast for the entire day
• The assignment of computing capacity to the application partition is done on hourly basis, considering the workload of the current period
Ø Mark’s Theorem
62
The long-term problem (T1)
• Objective function:
• The aggregation function (the sum) is order preserving, the decentralized hierarchical optimal solution is as good as the centralized optimal solution
63
nT2Pt=1
Pc2C
f tc
• Include in the same partition applications whose peaks and valleys overlap
Hyerarchical Resource Allocation: Application partitioning
• Evaluate area of the workload predictions
64
Area ratio criteria example
65
0
0.1
0.2
0.3
0.4
0.5
0.6
1 3 5 7 9 11 13 15 17 19 21 23
Class 1
0
0.1
0.2
0.3
0.4
0.5
0.6
1 3 5 7 9 11 13 15 17 19 21 23
Class 2
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
1 3 5 7 9 11 13 15 17 19 21 23
Class 3
0
0.1
0.2
0.3
0.4
0.5
0.6
1 3 5 7 9 11 13 15 17 19 21 23
Class 1-2
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1 3 5 7 9 11 13 15 17 19 21 23
Class 1-3
AR=1 AR=0.5
Hyerarchical Resource Allocation: Initial solutions
Ø Initial assignment: greedy algorithm
Central Manager
Application Manager A
Application Manager B
Application Manager C
Free Server Pool
VM1 VM2
VM3 VM4
F.O. C: 0.3 F.O. B: 0.2 F.O. A: 0.1
VM1 VM2
VM4 VM3
66
Hyerarchical Resource Allocation: Initial solutions
Ø Local Search
Ø Implements 3 moves
Increase Cluster Capacity : New servers are added to a cluster
Central Manager
Application Manager A
Application Manager B
Application Manager C
VM1 VM2
VM3 VM4
Delta F.O. C: 0.1 Delta F.O. B: 0.2 Delta F.O. A: 0.1
67
Hyerarchical Resource Allocation: Initial solutions
Ø Local Search
Ø Implements 3 moves
Decrease Cluster Capacity : Servers are moved back to the free server pool
Central Manager
Application Manager A
Application Manager B
Application Manager C
VM1 VM2
VM3 VM4
Delta F.O. C: 0 Delta F.O. B: -0.2 Delta F.O. A: -0.1
68
Variazione F.O. A: +0.1 Variazione F.O. C: +0.2 Delta F.O. C: 0
Hyerarchical Resource Allocation: Initial solutions
Ø Local Search
Ø Implements 3 moves
Ø Capacity migration: servers are moved between two different clusters Ø Decrease Capacity: the best servers are removed from a cluster Ø Increase Capacity: the worst servers are added to a cluster
Central Manager
Application Manager A
Application Manager B
Application Manager C
VM1 VM2
VM3 VM4
Variazione F.O. B: -0.2 Delta F.O. A: -0.1 Delta F.O. B: +0.3
69
Experimental Analysis
70
Aims: Analyze the behavior of the hierarchical algorithm working at different timescales with incoming workloads characterized by increasing variability Synthetic traffic generation Test campaign on Large Scale Service Centers
• 2400/4800/7200 servers and 240/480/720 service applications respectively • 5/10/15 minutes timescale • Spread out parameter equals to [0.1, 0.2, 0.3]
Experimental analysis • Cluster dimension evaluation • Analysis of multiple timescale optimization
• Injection of reference workload (e.g., workload varying at 5 minutes time scale) into Service Center allocated at 1 hour or 15/10/5 minutes
h
h
h
h
Experimental results: Centralized solver comparison
80 classes, 800 servers 100 classes, 1000 servers
71
Hierarchical-10 Hierarchical-10
Hierarchical-10 Hierarchical-10
Hierarchical-20 Hierarchical-20
Hierarchical-20 Hierarchical-20
Hierarchical-30 Hierarchical-30
Hierarchical-30 Hierarchical-30
Centralized Centralized
Centralized Centralized
Centralized
Ø Physical machine: Ubuntu 10.4, 2 CPU Intel Nehalem 2.4 GHz quad-core, 24 GB RAM
Ø Percentage error and speedup (average over 10 runs)
Ø Speedup around 8
Ø Feasible solutions can be identified under critical workload conditions
Ø Overhead of the hierarchical algorithm can be neglected for instances larger than 400 servers
Experimental results: Centralized solver comparison
72
(|K|,|I|) % Gap centralized Speedup
(20, 20) 11.00 0.24
(40, 400) 11.00 0.49
(80, 800) 10.81 4.45
(100, 1000) 9.98 6.97
(120, 1200) 7.75 7.61
Experimental Analysis – Low variabiility
73
Hour 5 min 10 min 15 min
Revenues 0.34 0.28 0.31 0.38
Server Switch Mean 150 33 61 85
Mean per time unit(15 min)
- 99 91.5 85
VM Migration Mean 950 160 330 480
Mean per time unit(15 min)
- 480 495 480
Response Time [0.46 5.65] [0.21 15,6] [0.20 8.2] [0.21 8.2]
Experimental Analysis – High variabiility
74
Hour 5 min 10 min 15 min
Revenues 0.32 0.25 0.32 0.23
Server Switch Mean 150 76 119 150
Mean per time unit(15 min)
- 228 178 150
VM Migration Mean 950 500 760 930
Mean per time unit(15 min)
- 1500 1140 930
Response Time [0.63 6.17] [0.30 14,6] [0.27 8.8] [0.31 9.4]
Conclusions
• This work proposes resource allocation policies for the management of multi-tier virtualized cloud systems maximizing the cloud provider profits
• A hierarchical heuristic solution based on local-search which provides also availability guarantees for the running applications has been developed
• Current work is considering the resource allocation problem: • At very fine-grained time scales (sec) by adopting control theory models
• At larger time scales allocating resources of multiple service centers exploiting also the availability of green energy sources
75
Additional references
• G. Pacifici, M. Spreitzer, A. Tantawi, A. Youssef. Performance management for cluster-based web services. IEEE Journal on Selected Areas in Communications 2005.
• B. Urgaonkar, G. Pacifici, P. Shenoy, M. Spreitzer, A. Tantawi. An analytical model for multi-tier internet services and its applications. SIGMETRICS 2005.
• B. Urgaonkar, G. Pacifici, P. Shenoy, M. Spreitzer, A. Tantawi. Analytical modeling of multi-tier internet applications. TWEB 2007.
• M. Tanelli, D. Ardagna, M. Lovera. Identification of LPV state space models for Autonomic Web service systems. IEEE Transactions on Control Systems Technology. 19(1), 93-103, 2011.
76
Thanks !
Any questions ?
77