eedc intelligent placement of datacenters
TRANSCRIPT
Execution Environments for Distributed Computing
Intelligent Placement of Datacenters for Internet
Services
EEDC
343
30
Master in Computer Architecture, Networks and Systems - CANS
Homework number: 6
Members: Roger Rafanell [email protected]
2
Outline
Introduction Motivation Placing Datacenters Evaluation Conclusion
2
3
Introduction: Datacenter construction costs
Each datacenter costs >$100M to construct– The smaller datacenters are rated at ~25MW
Examples:– Microsoft DCs in Virginia & Chicago: $500M each
3
4
Outline
Introduction Motivation Placing Datacenters Evaluation Conclusion
4
5
Motivation
Internet services require thousands of servers
Use multiple “mirror” datacenters– High availability and fault tolerance
– Low response time
Spend millions building and operating datacenters
Consume enormous amounts of brown energy!!
6
Outline
Introduction Motivation Placing Datacenters Evaluation Conclusion
6
7
Intelligent placement of datacenters
Goal: Manage the monetary and environmental costs
Define framework Model costs and datacenter characteristics Create solution approaches Collect cost and location-related data Create placement tool
7
8
Selecting datacenter locations
Model datacenter placement– Network latencies
– Availability
8
9
Selecting datacenter locations
Model datacenter placement– Network latencies
– Availability
CAPEX costs– Distance to electricity and networking infrastructure
– Land and construction (maximum PUE)
– Power delivery, cooling, backup equipment
– Servers and networking equipment
9
10
Selecting datacenter locations
Model datacenter placement– Network latencies
– Availability
CAPEX costs– Distance to electricity and networking infrastructure
– Land and construction (maximum PUE)
– Power delivery, cooling, backup equipment
– Servers and networking equipment
OPEX costs– Maintenance and administration
– Electricity and water prices (average PUE)
10
11
Selecting datacenter locations
Model datacenter placement– Network latencies
– Availability
CAPEX costs– Distance to electricity and networking infrastructure
– Land and construction (maximum PUE)
– Power delivery, cooling, backup equipment
– Servers and networking equipment
OPEX costs– Maintenance and administration
– Electricity and water prices (average PUE)
Incentives (taxes)
11
12
Selecting datacenter locations
Model datacenter placement– Network latencies
– Availability
CAPEX costs– Distance to electricity and networking infrastructure
– Land and construction (maximum PUE)
– Power delivery, cooling, backup equipment
– Servers and networking equipment
OPEX costs– Maintenance and administration
– Electricity and water prices (average PUE)
Incentives (taxes)
12
13
The problem formulation
Goal– Minimize CAPEX and OPEX
Constraints– Response times < MAX LATENCY for all users– Min consistency delay between 2 DCs < MAX DELAY– Min system availability > MIN AVAILABILITY
Output– Number of servers at each location– Minimum cost
13
14
Solving the (non-linear) problem
Linear Programming– Does not support non-linear costs
Brute force– Too slow
Simple heuristics– May not produce accurate results efficiently
14
15
Our approach for solving the problem
Evaluate each potential solution– Quickly via Linear Programming (LP)
Consider neighboring configurations– Simulated annealing (SA)
Cost optimization process– Combine SA and LP
15
Current solution Near neighbor
LP
SA
LP
16
Our approach for solving the problem
16
LP
SA
LP
LP
SA
LP
SA
$13.8M/month
$9.2M/month $10.7M/month
$10.3M/month
17
Summary of our approach
1) Generate a grid of tentative locations
2) Collect data about each location
3) Define datacenter characteristics
4) Instantiate optimization problem
5) Solve optimization problem
17
18
Outline
Introduction Motivation Placing Datacenters Evaluation Conclusion
18
19
Comparing locations for 60k-server DC
0100020003000400050006000700080009000
Austin Bismarck Los Angeles
New York Orlando Seattle St. LouisCost
(tho
usan
d do
llars
per
mon
th)
Servers Land Building Connection Energy Water Staff Networking
19
20
Interesting questions
How much does…… lower latency cost?
… higher availability cost?
… faster consistency cost?
… a green DC network cost?
… a chiller-less DC network cost?
20
21
Cost of 60k-server green DC network
21
Green DC network costs $100k/month more, except when latency <70ms
22
Cost of a 60k-server chiller-less DC network
0
2
4
6
8
10
12
14
30 50 70 90 110
Cost
(in
mill
ion
dolla
rs)
Maximum latency (milliseconds)
Chiller-less
Traditional
22
Chiller-less DC network is cheaper but it cannot achieve low latencies
23
Conclusions
First scientific work on smart datacenter placement– Proposed framework and optimization problem
– Proposed solution approach
– Characterized many locations across the US
– Built a tool to automate the process
– Answered many interesting questions
Results show that smart placement can save millions Work enables smaller companies to reap the benefits
23
24
Future work
To extend with data from Europe
Include tax incentives
Test the tool with data from real services
24
25
Questions
26
Specially thanks to the real authors of the work …
Íñigo Goiri, Kien Le, Jordi Guitart,Jordi Torres, and Ricardo Bianchini
26
27
Energy costs and carbon emissions
Company #ServersEnergy/year
(MWh)Energy
cost/yearCO2/year
(Metric tons)
eBay 16K 0.6 x 105 $3.7M 0.4 x 105
Akamai 40K 1.7 x 105 $10M 1.0 x 105
Rackspace 50K 2 x 105 $12M 1.2 x 105
Microsoft >200K >6 x 105 >$36M >3.6 x 105
Google >500K >6.3 x 105 >$38M >3.8 x 105
Sources: [Qureshi’09], EPA
27