Agenda• Data Centre Power
– UPS– Power Distribution– Rack Mount PDU’s
• UPS– Power problems– Configuration– Capacity
• Power Distribution– Types of Distribution– Pros and Cons– Efficiency
Agenda Cont…• Power Density in Cabinet
– Single vs. Three Phase– 120V vs. 208V vs. 400V– High Density Applications (Blades, Switches)– Temperature Inside Cabinets
• Importance of PDU Design– Dimensions, Flexibility, Venting, Interface Location– Cord Retention, Obstruction– Alternate DesignsAlternate Designs
• Power Monitoring Software and Interface– How to save money with intelligent PDUs– Out of the box software– Integration to BMS and UPS Software (PDU doesn’t dictate what software to use)– PDU level and outlet level information, PUE Level 2 / 3, Accuracy – Switched PDUs – turn off servers, turn off outlets
Power Components in the Data CenterCenter
UPS• UPS• Floor Mount PDU• Distribution• Rack Mount PDURack Mount PDU
What is a UPS?• An uninterruptible power supply / source (UPS) or
battery/flywheel backup is an electrical apparatusbattery/flywheel backup, is an electrical apparatus that provides emergency power to a load when the input power source typically mains power failsinput power source, typically mains power, fails.
• A UPS is typically used to protect computers data• A UPS is typically used to protect computers, data centers, telecommunication equipment or other electrical equipment where an unexpected powerelectrical equipment where an unexpected power disruption could cause injuries, fatalities, serious business disruption or data lossbusiness disruption or data loss.
Most Important UPS Features
• High Efficiency• Service/Warranty / y• Battery Management• Scalable Design• Scalable Design• Redundancy Options• Remote Monitoring• Reduced Footprint and Weightp g
Voltage/Capacity
120V Si l Ph 300VA 3kVA• 120V Single Phase – 300VA - 3kVA• 240V Single Phase – 1500VA - 18kVA• 208V Three Phase – 5kVA - 225kVA• 480V Three Phase – 30kVA - 1MW480V Three Phase 30kVA 1MW• 600V Three Phase – 30kVA - 1MW
UPS Style
Th b i UPS d lThree basic UPS models
• 1) Off-line UPS• 2) Line Interactive UPS2) Line Interactive UPS• 3) Double Online Conversion
UPS Power Solutions(3) (5)Line- (9)True
Off-line Interactive On-LineYES
YES
Power Failure Power Sag Power Surges
YES
YES
YES
YES
YES
NO
NO
Power Surges Undervoltage Overvoltage
YES
YES
YES
YES
YES
YESNO
NO
NO
g Electrical Line Noise Frequency Variation
YES
NO
NO
YES
YES
YES
NO
NO
Switching Transient Harmonic Distortion
CLEAN POWER
NO
NO
YES
YES
??? CLEAN POWER ??? YES
OFF-LINE UPS
NORMAL MODE Utility Power
SurgeSuppressionEMI/RFIEMI/RFIFilters
InverterBatteryCharger UPS Load
Battery
ChargerOutlets
Power flows through the UPS
Battery & Inverter are OFF-LINEBattery & Inverter are OFF LINE
LINE-INTERACTIVE UPS
Utility NORMAL MODE
SurgeSuppressionEMI/RFI Fillters
Power BUCK
BOOST
NORMAL
InverterBattery
BOOST
UPS Load
Battery
InverterCharger Outlets
Power flows through the UPS
Battery & Inverter are OFF LINEBattery & Inverter are OFF LINE
ON-LINE UPS DOUBLE CONVERSION
NORMAL MODE BypassUtility
PowerSurgeSuppression InverterRectifier
OutputTo Load
EMI/RFIFilters DC Link
UPS Load
Battery
UPS Load Outlets
Power is converted to DC and reinverted to ACContinuous Clean, Filtered,
Regulated power to the load
What is Distribution?
H t f th UPS t th iti l l dHow power gets from the UPS to the critical load• Consists of:
– Floor mount PDU– Breaker outputs to individual cabinets– Rack/Cabinet mount PDU’s
Floor Distribution
T diti l t l hi f l• Traditional style – power whips from a panel board, run overhead or under floor to the
bi tcabinets.• Benefits
– power can be routed to exactly where it is needed.
• Cons– Expensive– Long cable runsong cable runs– Limited flexibility (adds moves changes)
Floor Mount PDU/RPP• Close coupled system – usually end of row distribution• PDU transforms voltage from 600 480 to 208/120V• PDU – transforms voltage from 600, 480 to 208/120V
– Usable distribution voltage to the cabinet– Built in panel boards for distribution to a specific area in the DC
• RPP – End of row distribution panel– Usable distribution voltage to the cabinet– Built in panel boards for distribution to a specific area in the DC
• Benefits Shorter cable runs to cabinets– Shorter cable runs, to cabinets
– Moderate flexibility
• ConsCons – Expensive, need an electrician for adds moves changes
Overhead Power Busway: Pros / ConsCons
• PROS– Lower voltage drop per feet
• CONS– More expensive material upfrontLower voltage drop per feet
(improved efficiency)– 40% fewer man hours to install
vs. cabling
p p– Requires plenty of space above racks
for installation (not always available)– Becomes costly if too many “turn”
(– Does not require panelboards– Feed directly from large floor
PDUs (300-500kVA)
components are used (i.e. tees, crosses, elbows)
– Monitoring can be challenging, but vendors are becoming creative and
– Delivers high density power (400A 3phase)
– Cleaner look, doesn’t block cooling better appearance
grelying more on PDU
– Once a vendor is chosen, system is not compatible with other busway in the
k t ( it t t i l d )cooling, better appearance– Most beneficial for relocation,
pods, containerized solutions
market (commitment to single vendor)
WHAT IS A PDU SUPPOSED TO DO?DO?
• Power servers reliably (last piece of the chain)P id t / t d t d it i• Provide accurate / easy to understand monitoring information
• Assist with capacity planning load balancing and PUEAssist with capacity planning, load balancing and PUE• Work effectively under high density loads
• WHAT IS NOT SUPPOSED TO DO? – Obstruct air flow and equipment accessObstruct air flow and equipment access– Fail under high temperature environment– Difficult to install and configureg
Where is all this power going to? • Lower power density racks
(< 5kW) can be powered by Si l Ph PDUSingle Phase PDUs
• For load densities > 5kW, customers should be looking at Three Phase power to cabinets
• Three Phase power is a must to power Blades, provide long term room for growth and allow for virtualizationvirtualization
• 415V can dramatically increase power density
Choosing the Right Voltage
120/208V Three Phase • Most available power (up to
28kW)
• Single 3PH circuit reduces bli t
• Higher power capacity (up to
cabling costs
• Metered PDUs facilitate load balancing
L b f i it
230V Single Phase Higher power capacity (up to 5kW)
• Less current draw• Lower losses • More efficient
• Lower number of circuits improves airflow
• Provides scalability for growth without bringing down power120V Single Phase
• Limited at 20A
• 30A impractical & expensive
More efficient • More security since requires
208V equipment• Locking input plugs • Most servers accept 120-
g g p
• Support 120V & 208V
• Allows for Blades, Virtualization and Cloud
g
• Widely available and preferred by small business
Most servers accept 0240V Computing
Maximizing Panel and Busway Power Usagewith Three Phase 30A PDUs
• 3PH 30A PDUsB t b l b t it
with Three Phase 30A PDUs
– Best balance between power capacity and number of circuits
– 98% power capacity from a 400A feed – 93% usage of poles or power drops
il blavailable
• Delivers 8.7kW
• Meets the need for most server racks (up to 2 Blades)
• Does NOT require exotic cooling solutions
• Lower cabling costs
• Local breakers provide additional protectionprotection
400/415V PDU
• Increase Data Center efficiency and deliver more powerIncrease Data Center efficiency and deliver more power • Reduce power losses and number of transformations• Proven power distribution worldwideProven power distribution worldwide• Becoming a new standard in many Data Centers
208V Three Phase 415V Three Phase
Power @ 30A (kW) 8.6 17.2
Output Voltage 208V 240V
Input Connection Line to Line Line to Neutral
Input Plug L15 / L21 / IEC L22 / IEC
Efficiency Gains None 6-8%
Powering Blade Servers / SwitchesSwitches
• 2 PDUs per cabinet• 2 PDUs per cabinet• Min 12 C19s or L6-20s
per PDU p• Each Blade requires 4-6
outlets each • Load Balancing is a must
since each outlet can have a different power prequirement
• 4 Blades = >20kW 5 6kW h i
BA• 5-6kW per chassis• 6 power supplies (3+3 redundancy )
Overall Approach
• PDUs designed to support Blade Servers and Switches i e CiscoPDUs designed to support Blade Servers and Switches, i.e. Cisco Catalyst 6500 and Nexus 7000
• Vertical and Horizontal PDUs can be used• Low / Mid / High Power Densities, depending on:
– Style of server/switch/– Number of servers/switches
• Propose 2 PDUs per rack or cabinet offering for redundancy• Locking C19 and C13s are used to replace L6 20• Locking C19 and C13s are used to replace L6-20
Recommendation
h h ( bl d / h )• 60-80A 208V Three Phase PDU (up to 4 blades/switches)– Input cord or Terminal Block
• 6 x 20A circuit Breakers• 6 x 20A circuit Breakers • 2-3 outlets per Breaker• Local and Remote load monitoringoca a d e ote oad o to g• Horizontals or Verticals
– Depends on type of cabinet, air constraints and mounting preference
Sample PDUs for Blades/Switches
I t• Inputs– 30A Single Phase– 30A and 60A Three Phase
2U Horizontals 0U Verticals
• Outputs– L6-30– Locking C13 and C19s
• Functionalityl d i i– Local and Remote Monitoring
• FormHorizontal OR Vertical– Horizontal OR Vertical
PDU DESIGN• Keys to a good PDU design
– Low profileLow profile– Flexible mounting
Interface location and accessibility– Interface location and accessibility – Cord retention features
High temperature operation– High temperature operation– Outlet distribution and density
I t d fl ibilit– Input cord flexibility – Chassis consistency– Custom Designs
Branch Breaker Area
• Low Profile UL489 Breaker Doghouse• Low Profile UL489 Breaker Doghouse• Molded to Number of Breakers• Clearly Labeled & DefinedClearly Labeled & Defined• Reduce Obstruction Inside Cabinet
2.8”
Cord Retention Feature
Cord Retention MechanismCord Retention Mechanism• Individual outlet retention
system• Eliminates need for cable bar • Built-in anchors on the PDU• Locking tethers attach to PDU• Locking tethers attach to PDU
anchor points• Tie wraps are required to
attach tethers to plugs • Tethers can be removed and
reused
Locking C19 / C13 Outlets
Release Button
- Same electrical property as standard C13/C19 outlets- No special input cord neededNo special input cord needed - Built-in button must be pressed to release cord- Prevents accidental OR intentional cord removal- Alternative to locking NEMA L6-20 outlets and other cable retention options- Saves installation time and provides cord security in high density cabinetsSaves installation time and provides cord security in high density cabinets
Higher Operational Temperature
Venting SystemI i li d• Improve convection cooling and airflow through vents positioned at the end and sides of the PDU
• Allows for higher tempAllows for higher temp environments (65C or 149F)
• Reduces failure due to component overheating
• Increasing server exhaust temperature(45-50C)
• Containment solutions generate more heat – Hot aisle containment– Vertical Exhaust Duct
C20 Input & Input Cord Retention
• IEC C20 is an international input for PDUs pwhich gives the customer the flexibility to plug in multiple power cords from 110-240V
• Power Cords– 110V Nema: 5-15, 5-20, L5-15, L5-20– 240V Nema: 6-15, 6-20, L6-15, L6-20– 240V IEC: C20, IEC 16A– 240V (others): German, British, Italian
plugs• Double Cord Retention system is provided to
k i t d f i lkeep input cords from coming loose
Alternative Designs• Hardwired Terminal Block inputs
– Permanent connection– Cost savings by not having to purchase mating connectors
• Dual Input PDU– 2 PDUs in 1 redundancy– Space saving
• Horizontals• Horizontals– 1U or 2U designs– Customers that cannot install vertical units
• Half Height– Mount up to 4 PDUs in cabinet
Ob t t l t f th bi t– Obstruct only part of the cabinet
Power Monitoring Software and InterfaceInterface
• How to save money with intelligent PDUs– Informed capacity decisions– Informed capacity decisions– Load balancing of phases / breakers– Reduce risks of overloading and outages
Daisy chaining– Daisy chaining
• Out of the box softwareI t ti t BMS d UPS S ft• Integration to BMS and UPS Software – PDU usually doesn’t dictate what software to be used
• PDU level and outlet level information for most• PDU level and outlet level information for most accurate PUE calculation S i h d PDU ff ff• Switched PDUs – turn off servers, turn off outlets, save power, increase security
PDU Local Interface Features
• Digital Local DisplayReset– Power, Amps, Voltage, PF
• Per Breaker or Phase
– IP Setup
• Buttons for Easy On-screen Menu
Large LCD Display
Status LED
Navigation• Temp/Humidity Sensors• Ethernet & Serial Communication
Ports
Easy to Navigate Soft Buttons
Ethernet Port Ports• IP Consolidation Feature
– Connect up to 20 PDUs
• USB PortEnvironmental
Sensor PortPDU Linking Ports US o t
– FW update & Data Download
• Rotates 180º to be viewed from either direction
USB Port
Ports
Serial Communication
PortPort
Product Metering Benefits
• Avoid outlet / breaker overloading • Avoid tripping breakers at PDU or upstream• Make informed capacity planning decisions• Balance loads across breakers or phases • Efficiently utilize power resources y p• Measure IT equipment power consumption to save
power and money• Calculate Green Grid’s PUE Level 2 and Level 3• Drive green data center initiativesg
IP Consolidation
• Setup, Control and Monitor lti l PDU i d tmultiple PDUs in your data
center with a Single IP Port • PDU Linking Feature allows you
to reduce the number of IPto reduce the number of IP connections in your data center, thus saving you time and money
• Link up to 30 PDUs• Only Master PDU requires
network connection• Linked units are connected
h h i l RS485through serial RS485• Auto PDU detection• PDU cloning feature
Why link multiple PDUs?
• Save MoneySave Money– Deploy fewer IP addresses– Reduce installation time IN OUT– Reduce Ethernet cable length
• Enjoy the following features– Outlet grouping across multiple PDUs (A/B) – Unified access to all linked PDUs
B bl l i f PDU h– Be able to clone settings from one PDU to another• Manage every PDU in the chain from a single screen
PDU Outlet Power Control • Remotely Control Individual Outlets
– On/Off, Cycle/ , y– Via https or SNMP– Local status LED
• Group Outlets from Single or Multiple PDUs– Control and Manage all of them as ONE
S O tl t• Sequence Outlets–In rush current mitigation on power up
• Outlet Level Scheduling• Outlet Level Scheduling
• Shut Off outlets to increase security or reduce power usage
Accuracy & Billing
Billi G d• Billing Grade– Data Centers can report and charge customers based
tion power consumption
• PDU must have:– +/- 1% Accuracy – Real Time Metering– Outlet Level Monitoring
Built-in Web Interface
• Single Monitoring Interface g g
• HTTP / SNMP v2/v3• IP v4 / v6
Fi U d• Firmware Upgrade• Alarm Settings
– PDU / Outlet / Temperature• Outlet Grouping• Data Logging & Exporting • Email NotificationEmail Notification• Admin and User Levels• Radius / LDAP Authentication
PUE Accuracy Categories
PUE Category 0 PUE Category 1 PUE Category 2 PUE Category 3g y g y g y g y
IT Energy Measurement
LocationUPS output UPS output Floor PDU output IT equipment input
(Rack PDU output)
Definition of IT Peak IT electricDefinition of IT Energy
Peak IT electric demand IT annual energy IT annual energy IT annual energy
Definition of Total Energy
Peak Total Electric demand (kW)
Total annual energy (kWh)
Total annual energy (kWh)
Total annual energy (kWh)
Accuracy Level Basic Intermediate High HighestAccuracy Level Basic Intermediate High Highest
• The closer to loads, the more accurate PUE calculations become
• Energy (kWh) is also more accurate than Power (kW) alone
• Annual measurement vs. peak demand increases accuracy
Main Takeawaysy
U d t d d it d d d ith• Understand power density and redundancy with three phase and high voltage PDUs
• Importance of PDU design installation time• Importance of PDU design installation time, equipment / air obstruction and increase mounting flexibilityflexibility
• How to save money and improve power monitoring and control with intelligent PDUs. g