Download - DATA CENTER TRENDS TODAY & TOMORROW
www.eypmcfinc.com Follow us on:
DATA CENTER TRENDSTODAY & TOMORROW
STEVEN SHAPIRO, PE ATD
MEET YOUR PRESENTERS
Steve ShapiroPartner, EYP Mission Critical Facilities
Jeff Margolin
Principal, Ramboll (Moderator)
www.eypmcfinc.com Follow us on:
AGENDA
• IT Trends
• Standards/Guidelines/Codes
• Cooling Design Trends
• Power Design Trends
Follow us on:
DATA CENTERIT TRENDS
www.eypmcfinc.com Follow us on:
IT TRENDS
• Density
• Density vs. reliability profile
• Physical footprint
www.eypmcfinc.com Follow us on:
IT EQUIPMENT TRENDS
Manufacturers of IT equipment recognize this demand for increased power.
• Footprint reduction• Decreased outlet temperature• State-of-the-art microchip technology• Open compute architecture
As design engineers, we must support this shift in paradigm…Both electrically and mechanically!
www.eypmcfinc.com Follow us on:
DENSITY
• For every 18F rise in temperature hardware reliability decreased by 50%
• Footprint reduction = 30% per year
• Since 2000, power consumption for chips doubled
• 17% annual increase in load density in past 10 years
www.eypmcfinc.com Follow us on:
IT TRENDS
Chip manufacturers are investigating:
• New sink architectures
• Water cooled sinks
• Larger cooling fans (up to 70 cfm per server)
ADVANCED COOLING TRENDS
www.eypmcfinc.com Follow us on:
PHYSICAL FOOTPRINT
• The rack is getting crowded
• Multiple PDUs, copper and fiber connections are making wire management a mess
• Power to the rack has many options
• Scalability is an issue
• Allocation of power is an issue
• Power metering is an issue
• How do the power cords really get utilized with the new equipment?
• How do we keep the SLAs?
www.eypmcfinc.com Follow us on:
DENSITY AND RELIABILITY PROFILE
300
TIE
RIV
III
II
25020015010050
W/SF
ENTERPRISE
Hyperscale
COLO
Design Criteria
I
OLD MAINFRAME
www.eypmcfinc.com Follow us on:
DENSITY AND RELIABILITY PROFILE
• Enterprise vs. Colo vs. public cloud
• Hybrid IT strategy – on premises/private cloud
• Multi-cloud – private and public cloud (AWS/AZURE/Google)
Follow us on:
DATA CENTER STANDARDS/GUIDELINES/CODES
www.eypmcfinc.com Follow us on:
GENERAL STANDARDS/GUIDELINES/CODES
• Uptime Institute Tier Certification: Tier I to IV• ANSI/BICSI 002-2014 – Class F0 to F4• TIA 942 – Tier 1-4• ICREA• UL• EN 50600 series 1, 2-1 to 2-6 – Availability Class 1-4• EN 50173-5• ISO/IEC 24764• HIPPA – Health Insurance Portability and Accountability Act• SOX – Sarbanes Oxley – 2002• SAS 70 – Type I or II• Gramm-Leach Bliley Act (GLBA)• FedRamp & NIST
UPTIME/RELIABILITY DESIGN AND INFRASTRUCTURE STANDARDS
www.eypmcfinc.com Follow us on:
GENERAL GUIDELINES/REQUIREMENTS
• Uptime Institute: Operational Sustainability
• ISO 9000 – Quality System
• ISO14000 – Environmental Management System
• IS0 27001 – Information Security
• PCI – Payment Card Industry Security Standard
• SAS70 & ISAE 3402 or SSAE16 (USA) – Assurance controls
• AMS-IX – Data Centre Business Continuity Standard
• EN50600-2-6 Management and Operational Information
OPERATIONS STANDARDS
www.eypmcfinc.com Follow us on:
GENERAL GUIDELINES/REQUIREMENTS
Energy Efficiency Standards
• LEED
• Energy Star
• Green Globes
• CEEDA
Sustainability Standards?
UPTIME/RELIABILITY DESIGN AND INFRASTRUCTURE STANDARDS
www.eypmcfinc.com Follow us on:
GENERAL GUIDELINES/REQUIREMENTS
NFPA – National Fire Protection Association
• National Electrical Code – NFPA70
• NFPA 855 Standard For The Installation Of Stationary Energy Storage Systems
• NFPA 70E – Standard for Electrical Safety in the Workplace
• NFPA 72 – National Fire Alarm Code
IBC – International Building Codes
CODES
Follow us on:
DATA CENTER MECHANICAL TRENDS
www.eypmcfinc.com Follow us on:
Cooling Evolution/ Trends
Indirect / Direct
Evaporative Systems
Chip / Server Cooling
Immersion Cooling
Kyoto Cooling
www.eypmcfinc.com Follow us on:
COOLING DESIGN TRENDS
Fighting density and increasing energy efficiency
• In the data center
• For the data center• Systems selection process
• Economization alternatives
www.eypmcfinc.com Follow us on:
MECHANICAL TRENDS
Customer trends require a change in the way mechanical design is approached.
Electrical Engineers must adapt to these changes as well!
• High density IT racks require intensive cooling in localized areas
• Broader acceptable temperature ranges at the rack level
• Concurrent maintainability requirements mean multiple power feeds
www.eypmcfinc.com Follow us on:
MECHANICAL TRENDS
Packaged units are on the rise
• Water reduction techniques are being implemented
Free air cooling
• Direct – outside air does enter the data center
• Indirect – outside air does not enter the data center
New innovation – Kyoto Wheel
www.eypmcfinc.com Follow us on:
FOR THE DATA CENTERSYSTEM SELECTION PROCESS
www.eypmcfinc.com Follow us on:
FOR THE DATA CENTERSYSTEM SELECTION PROCESS
www.eypmcfinc.com Follow us on:
FOR THE DATA CENTER
• Chiller/cooling tower
• Air cooled chiller
• Adiabatic cooling/swamp cooler (roof, ground, CRAH)
• Outside air
• Air to air (Munthers, Schneider, Hunt Aire, Kyoto)
• Water side economizer • (Heat exchanger)
• DX backup/full-partial
• Drycoolers
SYSTEM ALTERNATIVES
www.eypmcfinc.com Follow us on:
FOR THE DATA CENTERSYSTEM ALTERNATIVES
Direct Air Economizer System
Outside air does not enter
the data center
Indirect Air Economizer System
Outside air does enter
the data center
www.eypmcfinc.com Follow us on:
FOR THE DATA CENTERSYSTEM ALTERNATIVES
Direct Evaporative Air Economizer
“Swamp” cooler without outside air
entering the data center
Indirect Evaporative Air Economizer
“Swamp” cooler with outside air
entering the data center
www.eypmcfinc.com Follow us on:
FOR THE DATA CENTERSYSTEM ALTERNATIVES – KYOTO WHEEL
Indirect Air-side Economizer
www.eypmcfinc.com Follow us on:
IN THE DATA CENTERFAN POWERED CABINET EXAMPLE
Cabinet Fans
Hot Air Plenum
Conditioned Air
is drawn through the
front of the cabinet
Hot Air
is discharged out of
the top via ductwork
Raised Floor
www.eypmcfinc.com Follow us on:
IN THE DATA CENTERWATER COOLED CABINET EXAMPLE
Conditioned Air
is drawn through the
front of the cabinet
Fan Coil
on rear door
cools servers
Cabinet Fans
Cooling Coil
Supply Return Piping Raised Floor
Discharge Air
High Pressure
flex hose and quick-connect
fittings to chilled water mains
www.eypmcfinc.com Follow us on:
IN THE DATA CENTERHOT AISLE COLD AISLE CONFIGURATION
Conditioned Air
is drawn through the
front of the cabinet
Hot Air
is discharged out of
the back of the cabinet
Conditioned Air
is drawn through the
front of the cabinet
Return Air
Cold Hot ColdAisle Aisle Aisle
www.eypmcfinc.com Follow us on:
IN THE DATA CENTER
• Immersion cooling – bathtub
• Liquid cooling – to the server
• Liquid cooling – to the chip
• Rear door cooling
• Issues in existing facilities
• Issues with space vs. power requirements
LIQUID COOLING
www.eypmcfinc.com Follow us on:
FOR THE DATA CENTERRaised Floor No Raised Floor
Free movement of cabinets Cost savings
Flexible placement of venting tiles Overhead cable plant
Air flow control – easy to deal with high density cabinets
Increased floor to ceiling height
Cable concealment No weight concerns
EMC shielding/SRG No under floor fire protection
Concealment of plumbing Appearance?
Appearance Air control – may need containment
Additional cost Containment – added cost
Reduced floor to ceiling height
Underfloor fire detection/protection required
Clogged air paths
Issues with removal of too many tiles at a time
Not durable - damaged flooring over time
www.eypmcfinc.com Follow us on:
COOLING TECHNOLOGIES
• Data centers must meet an ISO Class 8 Clean Room Standard
• Containment
• Chimneys
• Cold aisle
• Hot aisle
• CFD analysis
www.eypmcfinc.com Follow us on:
MECHANICAL SYSTEMS
• CBRN – chemical, biological, radiological, nuclear filtration/threat
• Water storage
• Redundant piping, loop with dual risers, redundant coils
• Water treatment
• Fuel storage – 48, 72, 96 hours, above or below, day tanks, belly tanks
• Fuel treatment – polishing
• Fuel delivery/testing on delivery
• Fuel sampling
www.eypmcfinc.com Follow us on:
ELECTRO-MECHANICAL TRENDS
Power Usage Effectiveness (PUE) – the litmus test between the trades
How do we calculate PUE?
Follow us on:
DATA CENTER ELECTRICAL TRENDS
www.eypmcfinc.com Follow us on:
ELECTRICAL TRENDS
• Increased critical power requirements leads to more electrical design options
• Reliability and availability
• Maintainability
• Modularization and flexibility
• Energy efficiency
• Infrastructure utilization
www.eypmcfinc.com Follow us on:
ELECTRICAL TRENDS
• Green and energy savings – sustainability
• Solar
• Fuel cells
• Micro-turbines
• Wind
• Distributed generation and load curtailment
• Emissions – prime/standby/scrubbers
• Demand peak shaving
• Lighting controls
www.eypmcfinc.com Follow us on:
ELECTRICAL TRENDS
• Energy storage
• Lithium-ion batteries
• Sodium/nickel zinc, etc.
• Natural gas/hydrogen generation
• Compressed air storage
• Hydraulic storage
• Microgrid
• Other renewables
www.eypmcfinc.com Follow us on:
ELECTRICAL TRENDS
• The desire to scale the facility as the business grows allows the customer to minimize up front costs
• Initial investment
• We as engineers need to allow for this growthpotential when considering topology and floor plan layouts
• Many vendors have added self contained pods to their lines which cater to this need
MODULARIZATION & SCALABILITY
www.eypmcfinc.com Follow us on:
ELECTRICAL TRENDS
UPS Manufacturers recognize these customer driven demands and have implemented them into their latest products
• Distributed paralleling are trending up!
• Central static switches are getting unpopular!
• Higher infrastructure cost
• Modularization/scalability
• Transformer free modules are trending up!
• Increased efficiency with mitigated heat loss
• Less cooling required!
• Eco mode
• Running the module on bypass during normal voltage conditions
• Increase in efficiency!
www.eypmcfinc.com Follow us on:
ELECTRICAL TRENDS
Efficiency is the key to success… and UPS manufacturers know this!
• Silicon Controlled Rectifiers (SCR) based modules ~80% efficient
• Transistors (IGBT) replace SCR ~92% efficient
• Transformer free ~97% efficient
• Eco mode introduced ~99% efficient
www.eypmcfinc.com Follow us on:
ELECTRICAL TRENDS
400V distribution system
Eliminate the need for PDUs and deliver critical power directly from the UPS!
• Still maintain use of standard server equipment
• Reduces HVAC requirements
• Increases overall electrical efficiency
• Decreases footprint requirements for infrastructure
• Reduces maintenance costs
THINKING OUTSIDE THE “UPS BOX”
www.eypmcfinc.com Follow us on:
ELECTRICAL TRENDS
DC distribution system!• Still maintain use of standard server equipment
• More efficient than AC
• Less cable loses over long distances
• Lower cost
• No conversion losses
• Compatible with many alternative energy sources
THINKING OUTSIDE THE “UPS BOX”
Doesn’t the power ultimately get converted into DC at the server level?
www.eypmcfinc.com Follow us on:
ELECTRICAL TRENDS
Unfortunately, many customers are reluctant to try such radical ideas…
…so let’s stick with the UPS.
www.eypmcfinc.com Follow us on:
ELECTRICAL TRENDSUPTIME INSTITUTE TIER CERTIFICATIONS
Tier I Tier II Tier III Tier IV
Active capacity components to
support the IT load
N N+1 N+1 N after any failure
Distribution Paths 1 1 1 active
1 alternate
2 simultaneously
active
Concurrently Maintainable No No Yes Yes
Fault Tolerant No No No Yes
Compartmentalization No No No Yes
Continues Cooling No No No Yes
What are the customer’s goals?
www.eypmcfinc.com Follow us on:
CONCURRENT MAINTAINABILITY
• “Change tires, transmission and engine at 60mph… and don’t lose speed.”
• No IT downtime for preventative and corrective maintenance
• Scalability
GOALS
www.eypmcfinc.com Follow us on:
ELECTRICAL TRENDSTHE CUSTOMER’S GOALS DRIVE CRITICAL TOPOLOGY
Traditional:• N• N+1
N+N
www.eypmcfinc.com Follow us on:
RELIABILITY/AVAILABILITYTYPICAL UPS CONFIGURATIONS
PARALLEL
REDUNDANT
2N
www.eypmcfinc.com Follow us on:
RELIABILITY/AVAILABILITYTYPICAL UPS CONFIGURATIONS
DISTRIBUTED
REDUNDANT
3 to Make 2
www.eypmcfinc.com Follow us on:
RELIABILITY/AVAILABILITYTYPICAL UPS CONFIGURATIONS
BLOCK REDUNDANT 3+1
www.eypmcfinc.com Follow us on:
RELIABILITY/AVAILABILITY
Description of RBDMTBF
(Hours)
Inherent Availability
(Ai)
Probability of Failure
in 5 years
N + 1 UPS system – dual cord loads 32,509 0.99981626 58.16%
Distributed Redundant (3 to make 2) UPS system –
dual cord loads161,646 0.99997994 7.43%
2N UPS system – dual cord loads 214,182 0.99998723 6.56%
2(N + 1) UPS system – dual cord loads 305,251 0.9999868 6.49%
Utility and N + 1 UPS system, ASTSs – dual cord
loads65,056 0.99999821 8.02%
Block Reserve (2-3) UPS System, ASTSs – dual
cord loads257,459 0.99999058 2.58%
Distributed Redundant (3 to make 2) UPS system,
ASTSs – dual cord loads256,674 0.99999046 2.72%
2N UPS system, ASTSs – dual cord loads 445,691 0.99999845 1.12%
2(N + 1) UPS system, ASTSs – dual cord loads 989,960 0.99999839 0.88%
www.eypmcfinc.com Follow us on:
RELIABILITY/AVAILABILITY
• Load density (the major issue)
• 99.9999% availability (“Six Sigma”)
• 2(N+1), 2N, N+2 redundancy not so common
• N or (N+1) systems not good enough
• Fault tolerant configurations – eliminate SPOFs
• Full concurrent maintenance
www.eypmcfinc.com Follow us on:
THE ECONOMICAL SOLUTION
• Dual/diverse utility services
• Dual active distribution paths
• N+2 generator plant
• “Self-healing” automation and control
• Physical separation/compartmentalization
• Distributed redundant UPS
• Dual electrical cords to IT cabinets
www.eypmcfinc.com Follow us on:
KEY TAKEAWAYS
1. IT trends
2. Codes and standards
3. Mechanical systems trends
4. Electrical systems trends
www.eypmcfinc.com Follow us on:
THANK YOU AND PLEASE FEEL FREE TO CONTACT ME
Steven Shapiro, PE, ATD914.420.3213
http://www.linkedin.com/in/stevenshapirope
@stevenshapirope
UPCOMING WEBINARSamericas.ramboll.com/webinar/data-centers
Trade-offs and Co-benefits: Balancing Long-Term Sustainability Goals with Time-to-Market NeedsMarch 31 – Amy Malick, Ramboll
Integration – Toward Net PositiveApril 21 – Expert panel from Ramboll, EYP Mission Critical Facilities and PSC
Questions?Steve Shapiro | Partner | [email protected]