netapp data center design - svlg...netapp data center design presentation for svlg data center...

NetApp Data Center Design Presentation for

SVLG Data Center Efficiency Summit

November 5, 2014

Mark Skiff, PE, CEM, CFM

Director, IT Data Center Solutions

© 2014 NetApp, Inc. All rights reserved. – Limited Use

Data Center Design Evolution

© 2014 NetApp, Inc. All rights reserved. – Limited Use 2

Capacity Build

Cost Configuration

Density

kW/rack PUE

mW $/kW year power & cooling >1

Generation 1

Multiple

Smaller

1-3

n/a

Hot / Cold Aisle Prior to 2006 <4 1.65

Generation 2 3 1.8 Cold Aisle

Containment 2006

8 1.55

Generation 3 25 2 / 8 Gen 2 w/

Economizer 2007 (Delivered 2009)

12 1.16

Generation 4 25

(scalable) 2.6

Gen 3 Plus

Refinement 2014

4 to 12 1.12

* Estimated annual

Data Center Design Evolution

© 2014 NetApp, Inc. All rights reserved. – Limited Use 3

Capacity Build

Cost Configuration

Density

kW/rack PUE

mW $/kW year power & cooling >1

Generation 1

Multiple

Smaller

1-3

n/a

Hot / Cold Aisle Prior to 2006 <4 1.65

Generation 2 3 1.8 Cold Aisle

Containment 2006

8 1.55

Generation 3 25 2 / 8 Gen 2 w/

Economizer 2007 (Delivered 2009)

12 1.16

Generation 4 25

(scalable) 2.6

Gen 3 Plus

Refinement 2014

4 to 12 1.12

* Estimated annual

Measuring Performance with PUE

© 2014 NetApp, Inc. All rights reserved. – Limited Use 4

0.0

0.5

1.0

1.5

2.0

Typical 2.0 PUE

Chiller Plant

AHU

UPS

Humidity Control

Transformer

Lights

IT

Power Usage Effectiveness (PUE) ratio cited as the data center infrastructure efficiency metric

Total power / IT equipment power

Infrastructure systems account for half of total energy in data centers

Typical data center PUE = 2.0

Lower the number the more efficient

Opportunity + Innovation

© 2014 NetApp, Inc. All rights reserved. – Limited Use 5

Typical Hot Aisle / Cold Aisle Configuration

© 2014 NetApp, Inc. All rights reserved. – Limited Use 6

Supply Air Supply Air

© 2014 NetApp, Inc. All rights reserved. – Limited Use 7

Hot/Cold Aisle w/ VAV at 4 kW/Rack (150 watts/Square Foot)

Control Damper

Temp.

Sensor

C

R

A

H

R

A

C

K

Return Air

© 2014 NetApp, Inc. All rights reserved. – Limited Use 8

Operational January 2007

– 360 racks @ 8kW / Rack (high

density; 770 watts/sf)

CRAHs Supply Constant Temp

Neutral Cold Room Pressure

– Modulating Dampers

Eliminates Air Mixing

Uniform temperatures

– Infrared shows warm areas blanking

gaps

Control Damper

Press.

Sensor C

R

A

H

R

A

C

K

Return Air

Contained Cold Aisle w/ Pressure Controlled Cooling

© 2014 NetApp, Inc. All rights reserved. – Limited Use 9

Plan View & Cross Section

Cold Room Hot Aisle

Co

ld R

oo

m

Co

ld R

oo

m

Co

ld R

oo

m

Co

ld R

oo

m

Co

ld R

oo

m

Co

ld R

oo

m

Co

ld R

oo

m

Co

ld R

oo

m

Co

ld R

oo

m

Co

ld R

oo

m

Co

ld R

oo

m

Co

ld R

oo

m

AHU AHU AHU AHU AHU AHU

AHU AHU AHU AHU AHU AHU

© 2014 NetApp, Inc. All rights reserved. – Limited Use 10

Air Handlers Main Aisle

End of Aisle Air Handling and Center Aisle

© 2014 NetApp, Inc. All rights reserved. – Limited Use 11

Contained Cold Aisle (Cold Room) and Hot Aisle

Cold Room Hot Aisle

© 2014 NetApp, Inc. All rights reserved. – Limited Use 12

NetApp Global Dynamic Lab (GDL) & IT Site 2

Opened March, 2009

2,166 racks, 12 kW per rack average load

Predicted PUE = 1.2 (annualized)

FIRST EVER ENERGY STAR RATED

© 2014 NetApp, Inc. All rights reserved. – Limited Use 13

Plan View – 36 Cold Rooms x 60 cabinets

Corporate IT

R&D

25 mW = 770 watts/sf

© 2014 NetApp, Inc. All rights reserved. – Limited Use 14

GDL1

Speed Control

Outside

Air

CRAH

RA

CK Cold

Room

Mixing Dampers

Cooling Coil

Hot Air

Return

Pressure

Sensor

Fan

Outside

Relief

Dampers

• 360 IT racks

– 6kW/rack

– Dual path UPS &

generators

• 1800 Engineering racks

– 12 kW/rack

– Utility power only

• 30 Engineering Network

racks

– 3 kW/rack

– Single path UPS &

generator

• Cold Room with Pressure

Controlled Cooling

• Airside economizer

• Elevated Air and CHW

Temps

© 2014 NetApp, Inc. All rights reserved. – Limited Use 15

Main Hall; 1st Floor

© 2014 NetApp, Inc. All rights reserved. – Limited Use 16

Cold Room; 1st Floor

© 2014 NetApp, Inc. All rights reserved. – Limited Use 17

Measuring Efficiency with PUE

0.0

0.5

1.0

1.5

2.0

Typical 2.0 PUE

Chiller Plant

AHU

UPS

Humidity Control

Transformer

Lights

IT

0.0

0.5

1.0

1.5

2.0

1.2 PUE (Annualized)

Chiller Plant

AHU

UPS

Humidity Control

Transformer

Lights

IT

Power Usage Effectiveness (PUE) ratio cited as the data center infrastructure efficiency metric

Total power / IT equipment power

Infrastructure systems account for half of total energy in data centers

Typical data center PUE = 2.0

Lower the number the more efficient

NetApp RTP Building 4, predicted PUE = 1.2

State-of-the-Art Data Center 2009 2010 2011

EPA assumed PUE ratio 1.67 1.56 1.45

© 2014 NetApp, Inc. All rights reserved. – Limited Use 18

Full FREE Cooling PUE = 1.07 Outside Air Temp = 73oF (23oC)

Cold Room 75oF (24oC) Return Air

91oF (33oC)

90%

10%

Speed Control

CRAH

Cold

Room

Mixing Dampers

Cooling Coil

Hot Air

Return

Pressure

Sensor

Fan

Outside

Relief

Dampers

90%

90%

10%

RA

CK

RA

CK

© 2014 NetApp, Inc. All rights reserved. – Limited Use 19

Full PAY Cooling PUE = 1.30 Outside Air Temp > 91OF (33oC)

Return Air

91o (33OC)

90%

10%

Speed Control

CRAH

Cold

Room

Mixing Dampers

Cooling Coil

Hot Air

Return

Pressure

Sensor

Fan

Outside

Relief

Dampers

90% 100%

Cold Room 75oF (24oC) RA

CK

PARTIAL Free Cooling PUE = 1.07 – 1.30

20

Outside Air Temp >75oF (24oC) to <91oF (33oC)

90%

10%

Speed Control

CRAH

Cold

Room

Mixing Dampers

Cooling Coil

Hot Air

Return

Pressure

Sensor

Fan

Outside

Relief

Dampers

90% 100%

Return Air

91o (33OC) Cold Room 75oF (24oC) RA

CK

100%

20 © 2014 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use

© 2014 NetApp, Inc. All rights reserved. – Limited Use 21

TEMP (F) #HRS/YR % of YR

<75o (24oC) 5481 62%

75-80o 2043 23%

80-85o 585 7%

85-90o 426 5%

>90o (32oC) 225 3%

Raleigh Weather Data

“FREE”

“PAY”

Actual Performance: 2013 Free Cooling

© 2014 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only 22

Actual Performance: 4Year Average Free Cooling

© 2014 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only 23

Actual Performance: 4Year Monthly Free Cooling

© 2014 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only 24

Actual Performance: PUE

© 2014 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only 25

Current PUE = 1.16

© 2014 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only 26

Lessons Learned and a Few “Ah-Ha’s”

PUE Range = 1.07 – 1.30 (average annual 1.16)

4 kW / rack average after 5 years

Construction Costs 20-33% of Industry

60 Degree Chilled Water

Temp Rise 16 degrees

TCOE 1/3 Site 1, 1/5 HQ, 1/7 Colo

© 2014 NetApp, Inc. All rights reserved. – Limited Use 27

Global Dynamic Lab 2

Project Goals – Bigger, Better, Cheaper

Project Completion; May, 2014

Predicted PUE 1.12

© 2014 NetApp, Inc. All rights reserved. – Limited Use 28

Data Center Space

© 2014 NetApp, Inc. All rights reserved. – Limited Use 29

Center Aisle GDL2

© 2014 NetApp, Inc. All rights reserved. – Limited Use 30

Mechanical System Improvements

• Increased Air Handling Capacity

• Incorporated Washable Outside Air Pre-filters

• Improved Air Distribution

• Eliminated Cooling Systems for Electrical Distribution Rooms

• Eliminated Evaporative Humidification Systems

• Selected More Efficient Fan Wheel

• Changed to Operable Window System Vs. Louver/Damper System

© 2014 NetApp, Inc. All rights reserved. – Limited Use 31

Air Handling Capacity

© 2014 NetApp, Inc. All rights reserved. – Limited Use 32

Washable Pre-Filters

Actual Performance: 2013 Free Cooling

© 2014 NetApp, Inc. All rights reserved. NetApp Proprietary – Limited Use Only 33

© 2014 NetApp, Inc. All rights reserved. – Limited Use 34

Improved Air Distribution

© 2014 NetApp, Inc. All rights reserved. – Limited Use 35

Air Flow Through End of Hot Aisles

© 2014 NetApp, Inc. All rights reserved. – Limited Use 36

Eliminated Electrical Room Air Conditioning

© 2014 NetApp, Inc. All rights reserved. – Limited Use 37

Operable Window System for Exhaust

© 2014 NetApp, Inc. All rights reserved. – Limited Use 38

Exterior View (open)

© 2014 NetApp, Inc. All rights reserved. – Limited Use 39 3

9

Interior View (closed)

© 2014 NetApp, Inc. All rights reserved. – Limited Use 40

Interior View (open)

© 2014 NetApp, Inc. All rights reserved. – Limited Use 41

Summary and Recommendations

Containment w/ Pressure Control Allows for High Density

(and low construction cost)

Air Side Free Cooling Required for Low PUE

Sustainable Data Centers

• Low Energy Use

• Recycled Materials

• Non Potable for Cooling Tower Make-up

• Low Construction Costs

Supplier (A/E, GC) Selection is Critical Success Factor

Phased Construction Provides Flexibilty

(first cost and purchasing)

Thank you

© 2014 NetApp, Inc. All rights reserved. NetApp Confidential – Limited Use 42