APC InRow™ Cooler Prototype
Initial Report - Test Setup Verification
H. Coles, S. Greenberg
Contributing: J. Bean (APC)
November 18, 2011
1
Contents
• Introduction
• Technology Description/Attributes
• Installation
• Analysis Metrics, Methods, Metering, Test Description
• Results
• Next Steps
• Conclusions
2 SVLG 2011 Data Center Efficiency Summit - LBNL
Introduction
• Project funding – California Energy Commission (CEC)
– APC: In-kind Funding
• LBNL Role - Identify and demonstrate new data center energy saving technologies
• Goal - Reduction or Elimination of Compressor Cooling
• Technology Selected: APC dual heat exchanger InRow™ Cooler
• Status: Initial trial setup and data collection completed
3 SVLG 2011 Data Center Efficiency Summit - LBNL
APC Unit Description
4 SVLG 2011 Data Center Efficiency Summit - LBNL
IT
Rack
IT
Rack
IT
Rack
IT
RackIT
Rack
IT
Rack
IT
Rack
IT
Rack
IT
Rack
IT
RackIT
Rack
IT
Rack
Cold Aisle
Hot Aisle
Hot Aisle
Air to Water Hex (chilled)
Air to Water Hex (treated)
Fans
(3 EC type)
Air to Water
Heat Exchanger
(chilled water)
Air to Water
Heat Exchanger
(“treated” water)
Chilled
Water
Connection
“treated”
Water
Connection
Hot
Air
Entering
Cooled
Air
Leaving
Chilled
Water
Modulating
Valve
Treated
Water
Modulating
Valve
filter
Product Attributes
• Redundancy: dual water cooling loops and heat exchangers
• Self contained: additional CRAH/CRAC units not required
• Use min. chilled water – use only where/when needed for data center hot spots
• Some additional plumbing required
5 SVLG 2011 Data Center Efficiency Summit - LBNL
APC Unit Installation Building 50 Data Center Rm 1275
Lawrence Berkeley National Laboratory
Front – Air Exit
Cold Aisle
Rear – Air Entry
Hot Aisle Front Door
Display/Control
Filter
6 SVLG 2011 Data Center Efficiency Summit - LBNL
COP Metric Definition
COP [ kWthermal / kWelec. ] = cooling provided / power needed
power needed (kW) = (kW/ton * tons) + (kW/ton * tons) + APC Unit Power
7
APC Unit
Power
cooling provided (kW) = treated water cooling + chilled water cooling – APC Unit Power
y = 0.0000051561x3
- 0.0008596432x2
+ 0.0327788257x
+ 0.3552353121
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
40 45 50 55 60 65 70 75 80 85 90
Ele
ctri
cal
Po
we
r N
ee
de
d (k
W/t
on
)
Cooling Water Temperature (°F)
kW/ton vs. Chilled Water Temperature (CWT)distribution pumping included
Taylor Engineering
Santa Clara CAYear Average
Plant Model ASHRAE 90.1 Code Minimum
Water-Side Economizer Equipped – Santa Clara CA Climate Year Average (Taylor Engineering)
Heavy Use of Chiller
Little or No Chiller Use (water side economizer)
aka “treated”
8 SVLG 2011 Data Center Efficiency Summit - LBNL
(CHWT)
Chilled Water Temperature (°F)
Test Methods
• Short duration preliminary tests – Manual Mode • Test 1: cooling tower only (single coil)
• Test 2: chilled water only (single coil)
• Test 3: chilled and treated water combined (both coils)
• Key Metric – Coefficient of Performance (COP)
9 SVLG 2011 Data Center Efficiency Summit - LBNL
Metering
Cooling Tower
(“treated”)
Chilled
Cooling
Water
Flow
Sensor
Temperature
Sensors
Flow
Sensor
Power
Meter
Chilled Water
Btu Meter
Treated Water
Btu Meter
Air Temp. Sensors
Air Temp.
Sensors
10 SVLG 2011 Data Center Efficiency Summit - LBNL
Onicon Btu Meter
(treated water)
Onicon Btu Meter
(chilled water)
Ion Power Meter
(APC Unit)
Installed Power and Btu Meters
Synapsense Wireless Air Temperatures
Preliminary Results: Compare Cooling Modes (manual operation, valves 100% open, fans @ slow speed)
SVLG 2011 Data Center Efficiency Summit - LBNL 11
Chilled
Water
Chilled+
Treated
Water
Treated
Water
Ne
t C
oo
ling
Treated
+ Chilled
Cooling
Chilled
Water Only
Cooling
“Treated”
Water
Only
Cooling
Tre
ate
d H
2O
Co
olin
gC
hill
ed
Wa
ter
Co
olin
g
CO
PCh
ille
d W
ate
r C
oo
ling
Ne
t C
oo
ling
CO
P
Ne
t C
oo
ling
CO
P
Tre
ate
d H
2O
Co
olin
g
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
En
erg
y R
ate
(kW
th
erm
al, k
W e
lectr
ica
l)
CO
P (
kW
/kW
)
0.5
8 k
W/ton
0.4
6 k
W/ton
0.2
8 k
W/ton
AP
C P
ow
er
Re
qu
ire
d
AP
C P
ow
er
Re
qu
ire
d
AP
C P
ow
er
Re
qu
ire
d
Compare Exit Temperature with Similar Inlet Temperature
(manual mode)
SVLG 2011 Data Center Efficiency Summit - LBNL
12
657075808590
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36
Air Entering/Leaving APC InRow Unit
Air from Hot Aisle Entering APC Unit (°F)
Air to Cold Aisle Leaving APC Unit (°F)
405060708090
0 2 4 6 8 10 12 14 16 18 20 22 24
Air Entering/Leaving APC InRow Unit
Air from Hot Aisle Entering APC Unit (°F)
Air to Cold Aisle Leaving APC Unit (°F)
405060708090
0 2 4 6 8 10 12 14 16 18 20 22 24
Air Entering/Leaving APC InRow Unit
Air from Hot Aisle Entering APC Unit (°F)
Air to Cold Aisle Leaving APC Unit (°F)
Test #1
69.1°F
Test #2
50.2°F
Test #3
48.9°F
Treated
Chilled
Treated and Chilled
Treated Water Cooling Metering Test
SVLG 2011 Data Center Efficiency Summit - LBNL
13
0.250
0.310
0.370
0.430
0.490
0.550
0.610
0.670
0.730
0.790
0.8504.0
5.0
6.0
7.0
8.0
9.0
10.0
11.0
12.0
13.0
14.0
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36
Pe
rfo
rman
ce (k
W e
lec
/ to
n )
CO
P (k
W c
oo
lin
g /
kW e
lect
rica
l)COP Treated Water Cooling Only
COP [kW/kW] = cooling / total electrical (plant + APC unit) needed
Treated Water Cooling Performance (COP)
150
170
190
210
230
250
0 5 10 15 20 25 30 35
CFM
/ k
W C
oo
led
Treated Only Coil Test - CFM per Net Cooled kW
cfm per kW
Next Steps
• Test control system – temperature modulation, fan speed (data presented not typical in some cases – manual mode)
• Test at various output set point temperatures
• Monitor cfm/kW supported
• Investigate power used and COP change at high fan speeds
• Longer term test
• Test performance when both coils use treated water
• Test with warmer air inlet
• Re-check air temperature sensor calibration
14 SVLG 2011 Data Center Efficiency Summit - LBNL
Conclusions
• Test setup working
• Efficiency Points – Chilled water ~14kW cooling (4 tons) at COP =6 (0.58kW/ton)
– Treated water ~14kW cooling (4 tons) at COP =12 (0.28kW/ton)
• Unit appears to support typical server cfm/kW – E.g. treated water test = ~1000cfm for ~5kW cooling (170-230cfm/kW)
• Compare to others – Chilloff 2 Results (Row Coolers) SCOPc = ~6 using ~45°F cooling water
– COP this test = ~6 using 47°F cooling water
15 SVLG 2011 Data Center Efficiency Summit - LBNL
End
Questions?
16 SVLG 2011 Data Center Efficiency Summit - LBNL
Appendix
SVLG 2011 Data Center Efficiency Summit - LBNL 17
Test 1 – Treated Water Only Cooling
50
52
54
56
58
60
62
64
66
68
70
72
74
3.0
4.0
5.0
6.0
7.0
8.0
9.0
10.0
11.0
12.0
13.0
14.0
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36
Wat
er
Sup
ply
Te
mp
. (°F
)
The
rmal
Co
oli
ng
(kW
)
Time (hours)
Cooling vs. Water Supply TemperatureTreated Water- Manual Control, APC Fans at Low Speed
Treated Water Supply Temperature (°F)
Treated Water Cooling (kW)
midnight noon midnight noon
657075808590
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36
Air Entering/Leaving APC InRow Unit
Air from Hot Aisle Entering APC Unit (°F)
Air to Cold Aisle Leaving APC Unit (°F)
22.0
22.2
22.4
22.6
22.8
23.0
23.2
23.4
23.6
23.8
24.0
-3.0
-2.0
-1.0
0.0
1.0
2.0
3.0
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36
Flo
w (g
pm
)
De
lta
Tem
p. (
F)
Treated Coil Only Cooling - water delta temp. (°F), flow (gpm)
Treated Water Flow (gpm)
Treated Water dt (°F)
18 SVLG 2011 Data Center Efficiency Summit - LBNL
Test 1 – Treated Water Cooling Only (continued)
900
1000
1100
1200
0 5 10 15 20 25 30 35
Air
Flo
w (c
fm)
Treated Only Coil Test - Calculated air flow (cfm)
calculated air flow
150
170
190
210
230
250
0 5 10 15 20 25 30 35
CFM
/ k
W C
oo
led
Treated Only Coil Test - CFM per Net Cooled kW
cfm per kW
19 SVLG 2011 Data Center Efficiency Summit - LBNL
4.0
5.0
6.0
7.0
8.0
9.0
10.0
11.0
12.0
13.0
14.0
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36
CO
P (
kW c
oo
ling
/ kW
ele
ctri
cal)
COP Treated Water Cooling OnlyCOP [kW/kW] = cooling / total electrical (plant + APC unit) needed
Treated Water Cooling Performance (COP)
.251
.271
.293
.320
.352
.391
.440
.502
.586
.703
.879
Test 2 – Chilled Water Only Cooling
38
39
40
41
42
43
44
45
46
47
48
12
13
14
15
16
17
18
19
20
0 2 4 6 8 10 12 14 16 18 20 22 24
Wat
er
Sup
ply
Te
mp
. (°F
)
The
rmal
Co
oli
ng
(kW
)
Time (hours)
Cooling vs. Water Supply TemperatureChilled Coil Only - Manual Mode, CRAH Fans at Low Speed
Chilled Water Cooling (kW)
Chilled Water Supply Temperature (°F)
midnightnoon
405060708090
0 2 4 6 8 10 12 14 16 18 20 22 24
Air Entering/Leaving APC InRow Unit
Air from Hot Aisle Entering APC Unit (°F)
Air to Cold Aisle Leaving APC Unit (°F)
22
23
24
25
26
27
28
29
30
-3.00
-2.00
-1.00
0.00
1.00
2.00
3.00
4.00
5.00
0 2 4 6 8 10 12 14 16 18 20 22 24
Flo
w (
gp
m)
De
lta
Tem
p. (
F)
Chilled Coil Cooling - water delta temp. (°F), flow (gpm)
Chilled Water dt (°F)
Chilled Water Flow (gpm)
20 SVLG 2011 Data Center Efficiency Summit - LBNL
Test 2 – Chilled Water Only Cooling (continued)
1000
1100
1200
1300
1400
0 2 4 6 8 10 12 14 16 18 20 22 24
cfm
Chilled Coil Only Test - Calculated Total CFM
cfm
80
85
90
95
100
0 2 4 6 8 10 12 14 16 18 20 22 24
cfm
/ k
W
Chilled Coil Only Test - CFM / kW Cooled
cfm per kW
21 SVLG 2011 Data Center Efficiency Summit - LBNL
4.0
5.0
6.0
7.0
8.0
9.0
10.0
11.0
12.0
13.0
14.0
0 2 4 6 8 10 12 14 16 18 20 22 24
CO
P (
kW c
oo
ling
/ kW
ele
ctri
cal)
Time (hours)
COP Chilled Water Cooling (manual mode) COP [kW/kW] = cooling / total electrical (plant + APC unit) needed
Chilled Water Cooling Performance (COP)
midnightnoon
.251
.271
.293
.320
.352
.391
.440
.502
.586
.703
.879
Test 3 – Chilled and Treated Cooling Combined (24 hour test, both coils at full water flow rate)
-20-15-10-505101520253035404550556065707580
3456789
1011121314151617181920
0 2 4 6 8 10 12 14 16 18 20 22 24
Wat
er
Sup
ply
Te
mp
. (°F
)
The
rmal
Co
oli
ng
(kW
)
Time (hours)
Cooling vs. Water Supply TemperatureTreated and Chilled Water Combined - CRAH Fans at Low Speed
Chilled Water Cooling (kW)
Chilled Water Supply Temperature (°F)
Treated Water Supply Temperature (°F)
Treated Water Cooling (kW)
midnight noon
405060708090
0 2 4 6 8 10 12 14 16 18 20 22 24
Air Entering/Leaving APC InRow Unit
Air from Hot Aisle Entering APC Unit (°F)
Air to Cold Aisle Leaving APC Unit (°F)
22
23
24
25
26
27
28
29
30
-3.00
-2.00
-1.00
0.00
1.00
2.00
3.00
0 2 4 6 8 10 12 14 16 18 20 22 24
Flo
w (
gp
m)
De
lta
Tem
p. (
F)
Both Coils Cooling - water delta temp. (°F), flow (gpm)Chilled Water dt (°F)
Chilled Water Flow (gpm)
Treated Water Flow (gpm)
Treated Water dt (°F)
22 SVLG 2011 Data Center Efficiency Summit - LBNL
Test 3 (both coil) continued Energy Use Efficiency – COP and kW/ton
Notes: both coils full flow ( ~ 23-25 each), APC fans at low speed
Time (hours)
10/04/2011 -17:00 10/05/2011 -17:00
1050
1150
1250
1350
1450
0 2 4 6 8 10 12 14 16 18 20 22 24
cfm
Treated and Chilled Coils Used - Total Cfm
cfm
75
80
85
90
95
100
0 2 4 6 8 10 12 14 16 18 20 22 24
cfm
/ k
W
Treated and Chilled Coils Used - Total Cfm / kW Cooled
cfm / kW
23 SVLG 2011 Data Center Efficiency Summit - LBNL
4.0
5.0
6.0
7.0
8.0
9.0
10.0
11.0
12.0
13.0
14.0
0 2 4 6 8 10 12 14 16 18 20 22 24
CO
P (
kW c
oo
ling
/ kW
ele
ctri
cal)
COP Treated , Chilled Water Cooling and CombinationCOP [kW/kW] = cooling (kW) / total electrical (plant + APC unit) needed (kW)
Treated Water Cooling Performance (COP)
Chilled Water Cooling Performance (COP)
Combined Cooling Performance (COP)
.251
.271
.293
.320
.352
.391
.440
.502
.586
.703
.879
Initial Check Air Flow Using Energy Balance Cubic Feet per Minute
SVLG 2011 Data Center Efficiency Summit - LBNL 24
900
1000
1100
1200
0 5 10 15 20 25 30 35
Air
Flo
w (c
fm)
Treated Only Coil Test - Calculated air flow (cfm)
calculated air flow
1000
1100
1200
1300
1400
0 2 4 6 8 10 12 14 16 18 20 22 24
cfm
Chilled Coil Only Test - Calculated Total CFM
cfm
1050
1150
1250
1350
1450
0 2 4 6 8 10 12 14 16 18 20 22 24
cfm
Treated and Chilled Coils Used - Total Cfm
cfm
Test #1
69.1°F
Test #2
50.2°F
Test #3
48.9°F
Ave. Air Temp.
@ Fans
Air CFM = f (net cooling, air delta temp.)
low?