transient reduced-order convective heat transfer modeling for a data center rajat ghosh g.w....

TRANSIENT REDUCED-ORDER CONVECTIVE HEAT TRANSFER MODELING FOR A DATA CENTER

Rajat Ghosh

G.W. Woodruff School of Mechanical Engineering

Georgia Institute of Technology

Atlanta, GA 30332-0405

September 25, 2012

Ph.D. Proposal Presentation


OUTLINE

• Introduction• Problem Statement• Representative Case Studies

– Case study-1– Case Study-2– Case Study-3

• Remaining Deliverable• Dissertation Timeline• ClosureIntroduction 2/35


INCREASING ENERGY CONSUMPTION

• Need to improve energy efficiency in data centers (DCs).Introduction 3/35

(Based on data reported by J. Koomey in the New York Times, July 31, 2012.)


INCREASING POWER DENSITY

• Need of high-resolution monitoring and feedback control– Both in temporal and spatial dimensions.

Introduction 4/35

Datacom Equipment Power Trends and Cooling Applications (2005), ASHARE TC 9.9

http://download.intel.com/technology/eep/data-center-efficiency/state-of-date-center-cooling.pdf


DYNAMIC DATA CENTERS

Introduction 5/35

(Liu, J.,Terzis, A., "Sensing data centers for energy efficiency,“Phil. Trans. R. Soc. A (2012))

(CRAC supply temperature data from CEETHERM: Sept. 9-10, 2012, 11 pm -11pm )

• Rapidly changing server load leads to dynamic thermal environment

-Dynamic thermal analysis requires fast (near-real-time) modeling algorithm.

-State-of-the-art CFD/HT frameworks are too sluggish.• Need of fast surrogate modeling algorithm


DATA CENTER COOLING

Introduction 6/35

• 1/3 of energy spent in a DC is dedicated to its cooling systems.

http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/white_paper_c11-627731_ps10280_Products_White_Paper.html

• Various airflow schemes exist:

- Underfloor plenum supply and ceiling return airflow scheme.

• Forced convective air cooling:

- Heat generated at chips dissipates via cooling airflow propelled by fans in the computer room air conditioning (CRAC) units.


MULTISCALE THERMAL SYSTEM

Introduction 7/35

• Involvement of several decades of length and time scales- Spatial: 5 decades (mm to Dm).- Temporal: 4 decades (10-2 s to 10 s).

Turbulent Convection

Turbulent Convection Turbulent Convection + Conduction

Conduction


CURRENT APPROACHES FOR TRANSIENT THERMAL MODELING

Introduction 8/35

Computational Time

Mo

del

Acc

ura

cy

Lumped System Modeling

Computational fluid dynamics/ Heat Transfer (CFD/ HT) Modeling

Reduced-order Modeling

Involves iterative solution of non-

linear conservation equations.

Involves posing zero

local gradient

condition.

Optimal and controllable

Trade-off.


TYPES OF REDUCED-ORDER MODEL (ROM)

Introduction 9/35

ROM

Statistical Response Surface Model

Simplified Physics-based Model

Proper Orthogonal Decomposition

(POD/ PCA)

Nonlinear Volterra Theory

Modal Reduction-based Low- Dimensional Model

Harmonic Balance Approximation

Laplacian Model

Thermal Zone Model


COMPUTATION FOR A TRANSIENT CFD SIMULATION

• DC Modeling Requirement– m spatial nodes and n time steps.

• Restriction on temporal discretization:• The dependent variables for the turbulent convective

temperature field: u, v, w, T, ε, k.• Computational step~ O(n(m3 + 4m))

– m3: For solving momentum equations together.– 4m: For solving pressure correction (continuity)+Temperature

+ Turbulence– n: Number of time steps

• For a rack-level simulation: – m~ 1.4 millions, n~1– t~2 hours in a 5.6 GHz machineIntroduction 10/35

CFL

./scale

tU x


COMPUTATION FOR A REDUCED ORDER MODELING

• DC Modeling Requirement: m dimensional temperature field with n transient observations.

• Initial data is collected via measurements or CFD.• POD/Interpolation-based reduced-order modeling

– Computational step~ O(3mn+log(n)+kn+n2+k2))• 3mn: Row-wise average + Generation of parameter-dependent

component+ Generation of covariance matrix • log(n): Proper orthogonal decomposition of covariance matrix (Power

algorithm).• kn: Finding POD coefficients for the input parameter space• n2: Interpolation• k2: Computation of new data (k=principal component number).

• No higher power of m.Introduction 11/35


LIMITATION OF EXISTING MODELING ALGORITHM

• Computational fluid dynamic and heat transfer (CFD/HT) modeling – Too sluggish to be fit for a near-real-time modeling algorithm.– Stochastic nature does not warrant expensive CFD

simulations.

• Reduced-order modeling – A few studies exist with time as the parameter.– No study exists with spatial location as the parameter.– No multi-parameter model exists.– Few studies use experimental data as model input: use of

CFD defeats the purpose of using ROM.– Need an alternative to Galerkine projection-based POD

coefficient determination. Problem Statement 12/35


SCOPE OF DISSERTATION

• Development of measurement-based parametric modeling framework – One parameter model

• For improving temporal resolution.• For improving spatial resolution.

– Multi-parameter model• For improving resolution in an additional dimension like

rack heat load.

• Development of interconnected multiscale model

- Hybrid reduced-order modeling approach.

Problem Statement 13/35


SINGLE PARAMETER (TIME)REDUCED-ORDER ALGORITHM

• Proper Orthogonal Decomposition (POD)-based modal reduction.

• Time is the modeling parameter.– Reduces the sampling

rate

• Use interpolation/ extrapolation to determine POD coefficients– Avoid computationally-

prohibitive Galerkin projection.

Representative Case Study

14/35


CASE STUDY FOR SINGLE PARAMETER (TIME) POD MODEL

• After remaining shut down for 2 minutes, the CRAC unit is turned on at t=0.


15/35


OPTIMALITY OF POD MODES

• First 10 POD modes capture more than 90% characteristics of the temperature field.


16/35


PRINCIPAL COMPONENT NUMBER

• As captured energy percentage increases, the corresponding principal component number increases.


17/35

; .i j i j

1

.ii n

ii

E

1

1

C.E.P. min( ) ,

k

iin

ii

k


ERROR FORMULATION


18/35

Prediction Experiment POD .E T T MeasurementPrediction Scale ,E f T

Analytical Exact POD .E T T POD/InterpolationAnalytical 0

1

,n

ii k

E c

1

2POD/Extrapolation 1Analytical 1 2 3 1

exp( ).

1

mt pm nn d

lE c t k h c c k

Prediction Analytical

Prediction

( ).

abs E Ee

abs E

0 0min ( ) , R.e c c

1 2 3 1 2 3min ( , , ) , , , R.e c c c c c c .dim( )

ii

e

e

1

221.

dim( ) 1 ii

ee

1 .I


TEMPERATURE MEASUREMENT


19/35

• Grid: 21 T-type copper-constantan thermocouples made from 28 gauge (0.9 mm diameter) wire.

• Response time– 20 ms.

• Measurement Frequency: – 1 Hz.

• x-axis: Parallel to rack width.• y-axis: Parallel to tiles.• z-axis: parallel to rack height.

S. Ravindran, Error Estimates for Reduced Order POD Models of Navier-Stokes Equations, ASME IMECE, 2008, pp. 652-657.


POD/ INTERPOLATION FRAMEWORK• Temperature map at the rack inlet at t=92 s.


20/35

A posterior measurement,

t~100 s

An extra step of interpolation,

t~10 s

Deviation~ O(1%)

POD model is efficient in improving parametric resolution of transient

temperature data

Accuracy of POD model prediction is identical to

experimental data.


CALIBRATION OF ANALYTICAL ERROR


21/35

0 20 40 60 80 100 120 140 160 180 200-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

Time (s)

Err

or

(0 C)

Prediction Error

Analytical Error

• Calibrated analytical error obviates the necessity of determining a posteriori prediction error .


POD/ EXTRAPOLATION FRAMEWORK• Temperature map at the rack inlet at t=207 s.


22/35

A posterior measurement,

t~207 s

An extra step of interpolation,

t~10 s

Deviation~ O(5%)

POD model is efficient in improving parametric resolution of transient

temperature data

Accuracy of POD model prediction is identical to

experimental data.


CALIBRATION OF ANALYTICAL ERROR


23/35

200 205 210 215 220 225-3.5

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

Time (s)

Err

or

(0 C)

Prediction Error

Analytical Error

• Calibrated analytical error obviates the necessity of determining a posteriori prediction error .


SINGLE PARAMETER (SPACE)REDUCED-ORDER ALGORITHM

• Proper Orthogonal Decomposition (POD)-based modal reduction.

• Coordinates of spatial location are the modeling parameters.– Improves the granularity

of experimental data.

• Reduction in sensor density.


24/35


CASE STUDY FOR SINGLE PARAMETER (SPACE) POD MODEL


25/35

• Sudden shut down of the CRAC unit and power back after 100 s.

(Photo courtesy to IBM)


PREDICTION FOR DOF-1 POINTS


26/35

• Improves spatial resolution between (70, 51, -1) and (70, 50, -1).


PREDICTION FOR DOF-2 POINTS


27/35

• Improves spatial resolution between (56, 31, 2.5) and (56,30,5.5).


TWO PARAMETER REDUCED-ORDER ALGORITHM

• POD-based modal decomposition.• Time and rack heat load as the modeling parameters.Representative Case Study

28/35


CASE STUDY FOR MULTI-PARAMETER (TIME, RACK HEAT LOAD) POD MODEL


29/35

• Sudden shut down of the CRAC unit and power back after 100 s (t=0 in the plot).


COMPARISON FOR EXTRAPOLATION AT Q=1500 W


30/35

Extrapolation in time and interpolation in heat load.


INTERCONNECTED MULTISCALE MODELING

• Experimentally validated CFD/HT modeling for a selected part of the CEETHERM data center laboratory.

• Development of the hybrid modeling framework combining finite network modeling (FNM) and POD for simulating a selected part of the CEETHERM data center laboratory.

• Comparison and validation. Deliverable 31/35


DISSERTATION TIMELINE

May 2010

-Dec. 2011

• Single-parameter POD framework development with time as the parameter.

Jan. 2012

-June 2012

• Single-parameter POD framework development with spatial location(s) as the parameter(s).

June 2012-Oct. 2012

• Multi-parameter POD framework development with spatial location and time as the parameters.

Nov. 2012

- April 2013

• Interconnected multi-scale modeling.

Planning 32/35


PUBLICATIONSREFEREED JOURNAL PUBLICATION• Rajat Ghosh, and Yogendra Joshi, “Error Estimate in POD-based Dynamic Reduced-order Thermal Modeling of

Data Centers,” International Journal of Heat and Mass Transfer (Revised version submitted).

REFEREED CONFERENCE PUBLICATIONS• Rajat Ghosh, Levente Klein, Yogendra Joshi, and Hendrik Hamann, “Reduced-order Modeling Framework for

Improving Spatial Resolution of the Temperature Data Measured in an Air-cool Data Center,” Semi-Therm, San Jose, California, March 17-21, 2013.

• Rajat Ghosh, Vikneshan Sundaralingam, and Yogendra Joshi, “Effect of Rack Server Population on Temperatures in Data Centers,” Intersociety Thermal Conference (ITherm), San Diego, California, May 30-June 1, 2012.

• Rajat Ghosh, Vikneshan Sundaralingam, Steven Isaacs, Pramod Kumar and Yogendra Joshi, “Transient Air Temperature Measurements in a Data Center,” Indian Society of Heat and Mass Transfer Conference, Chennai, India, Dec. 27-30, 2011.

• Rajat Ghosh, Pramod Kumar, Vikneshan Sundaralingam, and Yogendra Joshi, “Experimental Characterization of Transient Temperature Evolution in a Data Center Facility,” International Symposium on Transport Phenomena, Delft, the Netherlands, Nov. 8-11, 2011.

• Rajat Ghosh and Yogendra Joshi, “Dynamic Reduced-order Thermal Modeling of Data Center Air Temperatures,” InterPACK, Portland, Oregon, July 6-8, 2011.

PLANNED PUBLICATIONS• Rajat Ghosh, and Yogendra Joshi, “Error Estimate in POD-based Dynamic Reduced-order Thermal Modeling of Data

Centers,” International Journal of Heat and Mass Transfer (Revised version submitted).

• Rajat Ghosh, and Yogendra Joshi, “Error Estimate in POD-based Dynamic Reduced-order Thermal Modeling of Data Centers,” International Journal of Heat and Mass Transfer (Revised version submitted).

Closure 33/35

Ph.D. Proposal Presentation 34/35

ACKNOWLEDGEMENT

Closure

The support for this work from IBM Corporation, with Dr. Hendrik Hamann as the Technical Monitor, is acknowledged . Acknowledgements are also due to the United States Department of Energy as the source of primary funds. Additional support from the National Science Foundation award CRI 0958514 enabled the acquisition of some of the test equipment utilized.

The support from the G.W. Woodruff School of Mechanical Engineering as a Graduate Teaching Assistant is acknowledged.

The collaboration, goodwill, and help received from all CEETHERM and METTL members (particularly Vikneshan Sundaralingam, Vaibhav Arghode, Pramod Kumar, Steven Isaacs) are highly appreciated.


Thank YOU!

closure 35/35


APPENDIX

Introduction

Ph.D. Proposal Presentation 37/35Introduction

10

( , , ; )( , , ) .

n

i ii

T x y z tT x y z

n

*0( , , ; ) ( , , ; ) ( , , ).i i i iT x y z t T x y z t T x y z

2 2*( ) , ( 1).J T

0[ ] 0.dJ

d

.R

* *1.

TrR T T

m

Ph.D. Proposal Presentation 38Introduction

*( ) .n n m n m nb inv T

; .i j i j

1

.ii n

ii

E

1

1

C.E.P. min( ) ,

k

iin

ii

k

01

( , , , ) ( , , ) ( , , ). ( )k

i

T x y z t T x y z x y z b t


INTRODUCTION• Impact of proliferated cloud computing-based

e-commerce services on data Centers:– Increasing dynamic characteristics.– Increasing energy consumption.– Increasing power densities of racks.

• Effect on cooling – 30%-40% energy consumed by cooling systems.– Importance of local thermal characteristics.

• Need – High resolution (space/time) temperature

monitoring.– Near-real-time feedback control for temperature.

Introduction


TEMPERATURE MEASUREMENT

Introduction

• Grid: 21 T-type copper-constantan thermocouples made from 28 gauge (0.9 mm diameter) wire.

• Response time– 20 ms.

• Measurement Frequency: – 1 Hz.

• x-axis: Parallel to rack width.• y-axis: Parallel to tiles.• z-axis: parallel to rack height.


DISSERTATION TIME LINE• Development of a single-parameter POD-based framework for

transient convective heat transfer modeling for an air-cool data center (May 2010-Dec. 2011).

• Development of a grid-based thermocouple network for transient air temperature measurements (Nov. 2010-Nov. 2011).

• Development of design protocol for filling out an empty rack (Nov. 2011-Dec. 2012).

• Development of a single-parameter POD-based framework capable of improving spatial resolution of transient temperature data (Mar. 2012-June 2012).

• Development of a two-parameter POD-based framework for transient convective heat transfer modeling for an air-cool data center (May 2012-Dec. 2012).

• Development of a scale-linking across various length-scales in a data center (Oct. 2012-Mar. 2013).

• Ph.D. dissertation defense (Mar. 2013).Introduction

Ph.D. Proposal Presentation 42/35Introduction

May 2010-Dec. 2010

Jan. 2011-June 2011

July 2011-Dec. 2011

Jan. 2012-June 2012

June 2012-Dec. 2012

Jan. 2013-May 2013

Single-parameter POD Framework Development


LITERATURE REVIEW

Introduction

Paper/ Thesis Comments

S. V. Patankar, Airflow and Cooling in a Data Center, Journal of Heat Transfer 132 (2010)

73001-1-73001-17.

CFD/HT simulation for a steady data center.

A.H. Beitelmal, C.D. Patel, Thermo-Fluids Provisioning of a High Performance High Density Data Center, Distributed and Parallel Databases,

21, 227–238, 2007.

CFD/HT simulation for a transient data center.

Shawn Shields, Dynamic Thermal Response of the Data Center to Cooling Loss during Facility Power Failure, Masters Thesis, Georgia Tech,

2009.

Measurement-based transient characterization of a data center.

J. D. Rambo, Reduced-order Modeling of Multiscale Turbulent Convection: Application to

Data Center Thermal Management, Ph.D. Dissertation, Georgia Tech, 2007.

Reduced-order modeling of data center and a posterior error

analysis.


LITERATURE REVIEW CONTD.

Introduction


Q. Nie, Experimentally Validated Multiscale Thermal Modeling of Electronic Cabinets , Ph.D.

Dissertation, Georgia Tech, 2008.

Interconnected multiscale modeling for a rack.

Graham Nelson, Development of an Experimentally-Validated Compact Model of a Server Rack, Masters Thesis, Georgia Tech,

2009.

Development of grid-based temperature measurement

system and compact modeling of a server.

E. Samadiani, Energy Efficient Thermal Management of Data Centers via Open Multi-

scale Design, Ph.D. Dissertation, Georgia Tech, 2009.

Reduced-order open design for a multi-scale data center.

V. López and H. F. Hamann, Heat transfer modeling in data centers, International journal of

heat and mass transfer 54 (2011) 5306-5318.

Simplified Physics-based Laplacian Model for a data

center.


LITERATURE REVIEW CONTD.

Introduction


S. Ravindran, Error Estimates for Reduced Order POD Models of Navier-Stokes Equations, ASME

IMECE, 2008, pp. 652-657.

A priori error estimate for a proper orthogonal

decomposition (POD)-based reduced order model for the

Navier-Stokes equations

transient reduced-order convective heat transfer modeling for a data center rajat ghosh g.w....

Documents

proposal presentation235

proposal presentation

order modeling

order model rom introductionph

proposal presentation635

proposal presentation335

time modeling algorithm

cooling airflow