chapter 6 warehouse-scale computers to exploit request-level …harmanani.github.io › classes ›...

The University of Adelaide, School of Computer Science 22 November 2018

Chapter 2 — Instructions: Language of the Computer 1

1Copyright © 2019, Elsevier Inc. All rights Reserved

Chapter 6

Warehouse-Scale Computers to Exploit Request-Level and Data-Level Parallelism

Computer ArchitectureA Quantitative Approach, Sixth Edition


Introductionn Warehouse-scale computer (WSC)

n Provides Internet servicesn Search, social networking, online maps, video sharing, online

shopping, email, cloud computing, etc.n Differences with HPC “clusters”:

n Clusters have higher performance processors and networkn Clusters emphasize thread-level parallelism, WSCs

emphasize request-level parallelismn Differences with datacenters:

n Datacenters consolidate different machines and software into one location

n Datacenters emphasize virtual machines and hardware heterogeneity in order to serve varied customers

Introduction




Introductionn Important design factors for WSC:

n Cost-performancen Small savings add up

n Energy efficiencyn Affects power distribution and coolingn Work per joule

n Dependability via redundancyn Network I/On Interactive and batch processing workloads

Introduction


Introductionn Ample computational parallelism is not important

n Most jobs are totally independentn “Request-level parallelism”

n Operational costs countn Power consumption is a primary, not secondary, constraint when

designing systemn Scale and its opportunities and problems

n Can afford to build customized systems since WSC require volume purchase

n Location countsn Real estate, power cost; Internet, end-user, and workforce availability

n Computing efficiently at low utilizationn Scale and the opportunities/problems associated with scale

n Unique challenges: custom hardware, failuresn Unique opportunities: bulk discounts

Introduction




Efficiency and Cost of WSC

n Location of WSCn Proximity to Internet backbones, electricity cost,

property tax rates, low risk from earthquakes, floods, and hurricanes

n Power distribution

Efficiency and Cost of W

SC


Prgrm’g Models and Workloadsn Batch processing framework: MapReduce

n Map: applies a programmer-supplied function to each logical input record

n Runs on thousands of computersn Provides new set of key-value pairs as intermediate values

n Reduce: collapses values using another programmer-supplied function

Programm

ing Models and W

orkloads for WSC

s




Prgrm’g Models and Workloads

n Example:n map (String key, String value):

n // key: document namen // value: document contentsn for each word w in value

n EmitIntermediate(w,”1”); // Produce list of all words

n reduce (String key, Iterator values):n // key: a wordn // value: a list of countsn int result = 0;n for each v in values:

n result += ParseInt(v); // get integer from key-value pairn Emit(AsString(result));

Programm

ing Models and W

orkloads for WSC

s



n Availability:n Use replicas of data across different serversn Use relaxed consistency:

n No need for all replicas to always agree

n File systems: GFS and Colossusn Databases: Dynamo and BigTable

Programm

ing Models and W

orkloads for WSC

s




Prgrm’g Models and Workloadsn MapReduce runtime environment

schedules map and reduce task to WSC nodesn Workload demands often vary considerablyn Scheduler assigns tasks based on completion of

prior tasksn Tail latency/execution time variability: single

slow task can hold up large MapReduce jobn Runtime libraries replicate tasks near end of job

Programm

ing Models and W

orkloads for WSC

s



Programm

ing Models and W

orkloads for WSC

s




Computer Architecture of WSC

n WSC often use a hierarchy of networks for interconnection

n Each 19” rack holds 48 1U servers connected to a rack switch

n Rack switches are uplinked to switch higher in hierarchyn Uplink has 6-24X times lower bandwidthGoal is to

maximize locality of communication relative to the rack

Com

puter Ar4chitecture of WSC


Storagen Storage options:

n Use disks inside the servers, orn Network attached storage through Infiniband

n WSCs generally rely on local disksn Google File System (GFS) uses local disks and

maintains at least three relicas

Com





Array Switch

n Switch that connects an array of racksn Array switch should have 10 X the bisection

bandwidth of rack switchn Cost of n-port switch grows as n2

n Often utilize content addressible memory chips and FPGAs

Com



WSC Memory Hierarchyn Servers can access DRAM and disks on other

servers using a NUMA-style interface

Com





WSC Memory HierarchyC

omputer Ar4chitecture of W

SC


WSC Memory Hierarchy

Com





Infrastructure and Costs of WSCn Cooling

n Air conditioning used to cool server roomn 64 F – 71 F

n Keep temperature higher (closer to 71 F)n Cooling towers can also be used

n Minimum temperature is “wet bulb temperature”

PhyscicalInfrastrcutureand C

osts of WSC


Infrastructure and Costs of WSCn Cooling system also uses water (evaporation and

spills)n E.g. 70,000 to 200,000 gallons per day for an 8 MW facility

n Power cost breakdown:n Chillers: 30-50% of the power used by the IT equipmentn Air conditioning: 10-20% of the IT power, mostly due to fans

n How man servers can a WSC support?n Each server:

n “Nameplate power rating” gives maximum power consumptionn To get actual, measure power under actual workloads

n Oversubscribe cumulative server power by 40%, but monitor power closely


osts of WSC




Infrastructure and Costs of WSCn Determining the maximum server capacity

n Nameplate power rating: maximum power that a server can draw

n Better approach: measure under various workloadsn Oversubscribe by 40%

n Typical power usage by component:n Processors: 42%n DRAM: 12%n Disks: 14%n Networking: 5%n Cooling: 15%n Power overhead: 8%n Miscellaneous: 4%


osts of WSC


Measuring Efficiency of a WSCn Power Utilization Effectiveness (PEU)

n = Total facility power / IT equipment powern Median PUE on 2006 study was 1.69

n Performancen Latency is important metric because it is seen by

usersn Bing study: users will use search less as

response time increasesn Service Level Objectives (SLOs)/Service Level

Agreements (SLAs)n E.g. 99% of requests be below 100 ms


osts of WSC




Measuring Efficiency of a WSCPhyscicalInfrastrcuture

and Costs of W

SC


Cost of a WSCn Capital expenditures (CAPEX)

n Cost to build a WSCn $9 to 13/watt

n Operational expenditures (OPEX)n Cost to operate a WSC


osts of WSC




Cloud Computingn Amazon Web Services

n Virtual Machines: Linux/Xenn Low costn Open source softwaren Initially no guarantee of servicen No contract

Cloud C

omputing


Cloud Computingn Cloud Computing Growth

Cloud C

omputing



25

Fallacies and Pitfallsn Cloud computing providers are losing money

n AWS has a margin of 25%, Amazon retail 3%

n Focusing on average performance instead of 99th percentile performance

n Using too wimpy a processor when trying to improve WSC cost-performance

n Inconsistent Measure of PUE by different companies

n Capital costs of the WSC facility are higher than for the servers that it houses

Copyright © 2019, Elsevier Inc. All rights Reserved

Fallcies and Pitfalls

26

Fallacies and Pitfallsn Trying to save power with inactive low power

modes versus active low power modesn Given improvements in DRAM dependability and

the fault tolerance of WSC systems software, there is no need to spend extra for ECC memory in a WSC

n Coping effectively with microsecond (e.g. Flash and 100 GbE) delays as opposed to nansecond or millisecond delays

n Turning off hardware during periods of low activity improves the cost-performance of a WSC

Copyright © 2019, Elsevier Inc. All rights Reserved

Fallcies and Pitfalls

chapter 6 warehouse-scale computers to exploit request-level …harmanani.github.io › classes ›...

Documents