Building a „Real‟ Data

Center During the Great

Recession

Steven Carlini Sr Director, Data Center Global Solution Offer Management

APC by Schneider Electric

Schneider Electric at a glanceglobal, innovative and responsible

Listed on the Paris

Stock Market – CAC40

billion euros sales in 2009

people in 100+ countries

in Fortune 500 ranking

Sales by geography – 2009

North

America

27%Asia

Pacific21%

RoW11%

Western

Europe

35%

EasternEurope

6%

of salesin new economies

Specialist in

Energy ManagementGlobal

of sales devoted to R&D

Innovative

Responsible • Environment

• Ethics

• Business

• Access to energy

Schneider Electric at a glancecovering all end markets

Buildings

Data Centers Energy &

Infrastructures

Industry

Residential

31%Healthcare

Offices

Retail

18%

25%

Electrical power

Oil & gas

Marine

10%

16%

Machine

construction

Mining, minerals

& metals

Water

Industrial

% of 2009 sales

IT & Finance

Individual &

collective housing

The energy dilemma is here to stay

vsEnergy demandBy 2050Electricity by 2030

CO2 emissions to

avoid dramatic climate

changes by 2050

The facts The need

Source: IEA 2007

Source: IPCC 2007, figure (vs. 1990 level)

Energy management is a key

to address the dilemma

From production to consumption the

energy train

Coal100 units 35 units 33 units

unit

saved at the data center

( physical infrastructure)

units

saved at the power plant

unit

saved at the server

31

1units

saved at the power plant

6

Companies should focus on end of the chain

to save energy

Challenges converging on data centers

●IT density

Increased stress on power and cooling due to …

● Centralization / consolidation of distributed data centers

● High density technology – Virtualization, on-demand

computing, multi-core processors

●Energy efficiency

A growing a global awareness resulting in …

● Data centers in the spotlight due to high energy

usage that is increasing rapidly

● Opportunity to redirect cost savings toward the business

●Design / build cycle time

Need for agility, speed, predictability in response to

escalating pace of change in …

● Business climate

● Service quality level and business continuity standards

● Compliance requirements

… all putting the squeeze on AVAILABILITY

AVAILABILITY is the ever-present constraint in

data center planning

Why build a new data center

Data centers

Go green

• Leveraged more

energy efficient IT

equipment

• Energy and

management

measurement software

•Reduction of carbon

footprint

Go fast

• Shorten design build

cycle time

• Design with

consideration of

preferences vs.

constraints

• Use tools to evaluate

design impacts

Go dense

• Data center

consolidation

program

• Virtualization of

servers & storage

• Consolidation of

applications

Overheard in planning meetings…

“We‟d like to have multiple data center

proposals ready to present to management, but

that‟s cost prohibitive”

“We redesigned our facility several times

before our plan got approved”

“We organized our resources to support an

$8 million project and then were given a $1

million budget”

The Root Cause of These Problems is PLANNING

Why projects succeed or fail

Source: The Standish Group, 1995

User involvement

Executive mgmt support

Clear statement of requirements

Realistic expectations

Smaller project milestones

Competent staff

Ownership

Clear vision and objectives

Proper planning

0% 5% 10% 15%

Lack of user involvement

Lack of resources

Lack of executive support

Changing requirements and specs

Lack of planning

Don’t need any longer

Lack of IT management

Unrealistic expectations

Incomplete requirements

0% 5% 10%

Why projects succeed Why projects are impaired

Lessons From LEED

PLANNING - Better Buildings, Same Cost

Surprise lesson learned from LEED projects:

LEED-certified buildings cost the same as “normal” buildings

“ The only effective way to budget for sustainable features within buildings

is to identify the goals, and build an appropriate cost model for them.

… It is possible to establish goals and budgets from the very beginning of

the project. Other methods are ineffective and unnecessary.”

● Establish team goals, expectations, and expertise

● Include specific goals in the program

● Align budget with program

● Stay on track through design and construction

How?

Source: The Cost of Green Revisted - Davis Langdon report, 2007

St. Louis data center – Planning timeline

Decision to build

state-of-the-art,

highly-efficient

data center

Data Centers

burdened by

Power, Cooling,

Space challenges

in West Kingston

Reduction of

original budget by

50%

Review of the

original design

due to new

constraints

Data center to be

co-located with

Schneider

Technology

Center in St. Louis

APC acquired by

Schneider Electric

Schneider Electric

is comprised of

well-known brands

including TAC,

Square D,

PowerLogic and

more

=S= was

operating 330

data rooms &

number of data

centers globally

Budget revision

caused a two week

deadline on creation

of new plan

Build completedConsolidation

project in final

stages

Over 500 servers

moved

67% servers

virtualized as of

April 2010

St. Louis data center – moving fast

Technical decisions:

First Floor of SETC

2N UPS Redundancy

4500sq feet

415V, Modular, Row-

based Data Center

Core SME Group

Engaged – pulled for

specialized Data

Center design build

knowledge

The end result – Implemented July, 2009

DP-UPS1

DP

-UP

S2

DP-UPS3

HP-CDC

LP-1D

T1D

CDC-HP-C1

TC1

500A CDC-HP-C2

TC2

500A CDC-HP-C3

TC3

500A

SAN

8300

Front

FrontFront / cold aisle

250 kW Odin - A1 250 kW Odin - A2

RACK 18 RACK 19 RACK 20 RACK 21 RACK 22 RACK 23 RACK 24 RACK 25 RACK 26 RACK 27

RACK 28 RACK 29 RACK 30 RACK 31 RACK 32 RACK 33 RACK 34 RACK 35 RACK 36 RACK 37

R

P

P

A2

R

P

P

B2

Design allows for future

expansion without shutdown

Extra HACs

Extra HACs

• 4,500 ft2 (400m2)

• Average density 5.8 kW/rack

• Zone based approach

● PUE 2.86 1.52 (with a target of 1.35 @ fully loaded)

● Build with scalable, energy-efficient technologies● UPS rightsizing, In-row cooling

● Hot aisle containment

● Energy smart servers

● Future ready● Connections to economizer and solar panels in place

Budget cuts didn’t stop us from going efficient

... Nor compromise services delivered

Capacity

Availability

Security

160 kW 450 kW(expand up to 1.3 MW)

• Multiple SPOF

• Minimal redundancy

Data center availability

~ 99.98%

• Limited access control• Biometric access control

• Rack level access control

2,500 sq ft 4,500 sq ft

Old data center St. Louis

LeanVirtualization

~ 55%Virtualization

~ 67%

Trade-offs lead to gains

ORIGINALDesign

ADJUSTEDDesign

Capacity• 1.3 MW capacity

• 600 m2 (6500 ft2)

2008/2009 economic

crisis

Budget cut

by 50%

Capacity• 700 KW capacity

• 400 m2 (4500 ft2)

Efficiency• Top priority

• PUE 1.34 at full load

Efficiency• Still top priority

• PUE 1.35 at full load

Redundancy• 2N dedicated chiller

• 2N piping, 2N air handlers

• 2N power path (1 onsite substation)

• 2N standby generators

Redundancy• Existing N+1 building chiller

• Existing N piping, N+1 air handlers

• 2N power path (1 onsite substation)

• Existing N building generator

Energy management• Real-time efficiency and energy

usage measurement

Energy management• Real-time efficiency and energy

usage measurement

Changed

Changed

Trade-offs – Planning for the future

Economizer

• Reduce cooling costs

aka “free’ cooling

• Reduce PUE

• Require air filtration

• Humidity an issue

• Estimated Cost -

$230,000

Solar/Alternative

Energy

• Going green

• Internal energy

generation

• Currently costly to

generate power

• Estimated Cost -

100,000 panels needed

to generate 1MW of

power

High Efficiency

Chiller

• Reduce data center

electrical

consumption

• Reduce PUE (when

used with an

economizer)

• Reduce

maintenance costs

• Estimated Cost -

$370,000

2008/2009

economic crisis

We balanced preferences and constraintsPreference:

Result:

• Dedicated chillers

• Dedicated diesel generator

• Water side Economizer

• Dedicated electrical substation

• Executive conference room overlooking data center

• Unfinished space on 1st floor

Constraint:

• Using existing 2MW stand-by diesel generator

• Using “base building” chiller plant

• Capacity to meet 48 month window – 2N - 400kW UPS

• Growth to 750kW 2N UPS and N+2 Cooling without shutdown

• Ultimate growth 1500kW 2N UPS and N+2 Cooling

• Criticality – Equal to Tier III

• Efficiency – PUE of 1.52 as built / 1.35 with future economizer

Evaluate

Preferences

& Constraints

Trade-Off

Analysis

Good planning with use of reference design enabled the success of the project

IT equipment at a glance

• 4,500 ft2 (400m2)

• 500+ servers

• 50% of servers virtualized

Servers

Storage

Network

Dell PowerEdge (rack and blade)

HP Integrity (RISC and Itanium)

9000

IBM Bladecenter L&H

p595 Power 6 / AIX HACMP

p520 TSM

HS21 blades / Novell SUSE Linux

Intel standalone servers

EMC

IBM Quantum tape library

3584 LTO tape backup

DS8300 SAN

Cisco Catalyst 6500 switches

Design considerations at a glance

415/240V distribution throughout the data center

Row-based cooling equipment architecture

Hot Aisle containment system

Metered and controlled chilled water loop

Modular equipment architecture

2 Megawatt existing onsite generator

34KV 12.5KV existing onsite substation

Schneider Electric products usedSupplier Subsystem

Pelco Whole-room video security

Management

NetBotz Rack-level video security

TAC Building management system

APCManagement system for physical infrastructure

Ion E Power monitoring

PowerLogic Metering and instrumentation

Power

Square D Switchgear and transformers

APC UPS

Defem Cable trays

Caterpillar Generator

APC Row-based cooling

Cooling

APC Hot-aisle containment

Veris Metering

Armstrong Int‟l Humidifier

Carrier Chillers

APC Racks

OtherNotifier Fire detection

Front / cold aisle

RACK 18 RACK 19 RACK 20 RACK 21 RACK 22 RACK 23 RACK 24 RACK 25 RACK 26 RACK 27

RACK 28 RACK 29 RACK 30 RACK 31 RACK 32 RACK 33 RACK 34 RACK 35 RACK 36 RACK 37

R

P

P

A2

R

P

P

B2

A data center “zone”

Hot aisle

Cold aisle

Cold aisle

6 Zones organized by density not application

88768136

Actual PUE is not staticSpot interventions done

in the data center

Long-term trend

as IT load grows

Questions?

1.52

Thank You!