reliable data centres - stulz

57
Reliable Data Centres Guide Version 2

Upload: others

Post on 29-Apr-2022

16 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Reliable Data Centres - STULZ

Reliable Data CentresGuide

Version 2

Page 2: Reliable Data Centres - STULZ

�� Impressum

Published�by:� BITKOMBundesverband�Informationswirtschaft,Telekommunikation�und�neue�Medien�e.�V.Albrechtstraße�10�A10117�Berlin-MitteTel.:�+49�30.27576-0Fax:�[email protected]

Contact: Holger�Skurk�Tel.:�+49.30.27576-250�[email protected]

Responsible�BITKOM-committee:�

AK�Data�Centre�&�IT-Infrastructure

Editorial�team: Holger�Skurk

Assistant�editor: Biliana�Schönberg

Design�/�Layout: Design�Bureau�kokliko�/�Anna�Müller-Rosenberger�(BITKOM)

Copyright:� BITKOM�2010

Titelbild: Alejandro�Mendoza,�istockphoto.com

The�contents�of�this�guide�have�been�carefully�researched,�and�reflect�BITKOM’s�understanding�at�the�time�of�publication.�We�do�not�make�any�claims�regarding�the�completeness�of�this�publication,�however.�Despite�taking�the�greatest�possible�care,�we�do�not�assume�any�liability�for�the�contents.�The�latest�version�of�the�guide�can�be�obtained�free�of�charge�from�www.bitkom.org/publikationen.�BITKOM�reserves�all�rights,�including�those�governing�the�reproduction�of�parts�of�this�document.

Page 3: Reliable Data Centres - STULZ

Reliable Data Centres

Reliable Data CentresGuide

Version 2

Page 4: Reliable Data Centres - STULZ

2

Inhaltsverzeichnis

1� Introduction�� � 32� The�availability�of�a�data�centre��� � 43� The�influence�of�safety�standards�on�the�design�of�data�centres�� � 7

3.1� ISO�27001�/�ISO�27002:2008�� � 73.2� ITIL�� � 83.3� Sarbanes-Oxley�Act�and�SAS�70�� � 83.4� Assessment�of�standards�� � 8

4� The�basis�of�IT�infrastructure:�the�rack�� � 94.1� Secure�server�cabinets�� � 94.2� Network�technology�� �104.3� Reliable�data�centre�� � 11

5� Energy,�air�conditioning�and�cooling�� � 125.1� Power�supply�companies�(PSCs)�–�electricity�feed-in�and�distribution�in�the�company�� � 125.2� Power�distribution�in�the�company�� �145.3� Uninterruptible�power�supply�(UPS)�� �165.4� Backup�power�� �225.5� Service/maintenance�� �275.6� Air�conditioning�� �28

6� Fire�safety�� �346.1� Technical�fire�protection�� �346.2� Structural�fire�safety�measures�� �38

7� Design�of�premises�and�safety�zones�for�data�centres�� �418� Wiring�� �43

8.1� Current�situation�� �438.2� Basic�standards�� �438.4� Structure�� �448.5� Redundancy�and�security�� �458.6� Installation�� �468.7� Documentation�and�labelling�� �46

9� Certification�of�a�reliable�data�centre�� �479.1� The�management�system�� �479.2� Certification�of�a�management�system�� �47

10� Annex�� �5011� Glossary�� �5312� Acknowledgements�� �54

Page 5: Reliable Data Centres - STULZ

3

Reliable Data Centres

1 Introduction

Planning,�developing�and�operating�IT�infrastructures�for�companies’�vital�applications�in�reliable�data�centres�is�a�real�challenge.�Not�only�do�you�have�to�select�the�right�IT�equipment,�the�design�of�the�data�centre�and�the�resul-ting�requirements�for�the�size�and�type�of�construction,�the�electrical�power,�heat�dissipation,�wiring,�safety�and�availability�–�not�to�mention�purchase�and�operating�costs�–�all�these�are�decisive�factors.

This�guide�offers�assistance�with�planning�and�implemen-ting�a�data�centre.�It�supplements�existing�standards�and�specifications,�which�can�also�be�referred�to�for�support.�These�are�often�extremely�general,�however,�whereas�this�guide�is�more�in-depth�and�provides�concrete�instructions�on�how�to�design�a�data�centre.�It�also�supplements�the�

“Planning�Guide�for�a�Reliable�Data�Centre”�matrix�which,�like�the�guide,�is�available�to�download�free�of�charge�on�the�BITKOM�website.�The�contents�of�the�matrix�are�con-tained�in�extracts�in�the�sections�of�the�guide.�The�guide�and�the�planning�aid�are�no�substitute�for�expert�advice�and�support�from�experienced�advisors�and�specialist�planners,�however.1

1� This�guide�will�be�continually�revised�by�the�BITKOM�Reliable�Data�Centre�&�IT�Infrastructure�working�group.�As�part�of�this�process,�it�will�be�adapted�in�line�with�the�latest�technical�developments�and�other�boundary�conditions,�and�extended�to�include�topics�such�as�the�requirements�of�the�insurance�industry,�risk�management,�processes�and�services,�for�example.

Page 6: Reliable Data Centres - STULZ

4

2 The availability of a data centre

The�rapid�pace�of�development�and�the�integration�of�information�technology�in�all�areas�of�business�means�that�today,�no�company�can�afford�this�technology�to�fail.�Just�a�few�years�ago,�many�companies�could�“survive”�the�failure�of�their�IT�infrastructure,�even�for�several�hours.�Today,�the�number�of�those�for�whom�continuous�availa-bility�of�their�IT�is�indispensable�is�growing�dramatically.�According�to�a�study�by�the�Meta�Group,�a�10-day�failure�of�key�IT�systems�can�permanently�damage�a�company�to�such�an�extent�that�there�is�a�50�%�probability�that�it�will�disappear�from�the�market�within�the�next�three�to�five�years.

Availability�tiers�–�US�Uptime�Institute�

Tier classification

Intro-duction

Explanation

Tier�I 1960s Single�power�supply�channel,�single�air-conditioning�supply,�no�redundant�components�99.671�%�availability

Tier�II 1970s Single�power�supply,�single�air-conditioning�supply,�redundant�components�99.741�%�availability

Tier�III End�of�1980s

Several�power�supply�channels�available�but�only�one�active,�red-undant�components,�maintenance�possible�without�interruption��99.982�%�availability

Tier�IV 1994 Several�active�electri-city�and�chilled�water�supply�channels,�redundant�compo-nents,�error-tolerant�99.995�%�availability

Source:�US�Uptime�Institute:�Industry�Standards�Tier�Classification

These�days,�when�creating�and�expanding�an�IT�concept,�or�examining�an�existing�one,�it�is�vital�to�find�out�how�important�the�availability�of�a�company’s�IT�infrastructure�is.�Consequently,�the�question�to�be�asked�is�this:

“What�maximum�IT�downtimes�can�the�company�tolerate?”

As�the�demand�for�availability�of�the�IT�infrastructure�grows,�the�requirements�become�increasingly�stringent�–�not�just�for�the�IT�systems�themselves�but,�above�all,�for�continually�ensuring�the�ambient�conditions�and�the�power�supply.�As�a�result,�redundancy�in�the�air�conditio-ning�and�electricity�supply,�dual�feed-ins,�and�interrup-tion-free�maintenance�of�systems�have�established�them-selves�as�the�norm�for�high-availability�IT�infrastructures.�

But�before�you�start�work�on�the�planning�and�layout�of�technical�components�for�achieving�the�availability�you�desire,�additional�reflection�based�on�risk�analysis�and�the�choice�of�site�is�imperative.�This�should�cover�possible�risks�associated�with�the�area,�which�could�influence�the�probability�of�potential�failure�geographically�(air�traffic,�flooding,�etc.),�politically�(war,�conflict,�terrorism,�etc.)�or�in�the�form�of�neighbouring�sites�(premises�at�particular�risk�of�fire�such�as�service�stations,�chemical�storage�facilities,�etc.).�Furthermore,�potential�wilful�damage�by�one’s�own�employees�and�from�outside�the�company�must�also�be�taken�into�overall�consideration.��

However,�the�demand�for�high�availability�does�not�only�entail�getting�to�grips�with�possible�technical�solutions,�but�requires�the�owner�to�design�and�set�up�a�compre-hensive�organisational�structure.�This�includes�the�pro-vision�of�trained�service�personnel,�for�example,�of�spare�parts�or�a�service�agreement.�Precise�instructions�on�the�procedure�in�the�event�of�failure�or�emergency�must�also�be�defined.�In�addition,�this�structure�must�enable�rapid,�precise�and�targeted�communication,�with�traceable�documentation�of�events.

Page 7: Reliable Data Centres - STULZ

5

Reliable Data Centres

The�term�“availability”�refers�to�the�probability�that�a�sys-tem�can�actually�be�used�as�planned�at�a�given�moment.�Availability�is�therefore�a�measure�that�can�be�recorded�and�determined�quantitatively.�On�the�other�hand,�there�exist�qualitative�availability�classes,�as�shown�in�the�table�below,�“Availability�classes�according�to�the�HV-Kompen-dium�(High�Availability�Compendium)�of�the�BSI“.�Here,�the�availability�class�of�a�service�is�a�measure�of�its�quality�in�terms�of�availability�based�on�hours�per�year.

A�system�is�regarded�as�available�if�it�is�capable�of�fulfilling�the�tasks�for�which�it�is�intended.�Availability�is�measured�as�the�ratio�of�failure-induced�downtime�to�the�overall�time�of�a�system.

If�we�calculate�availability�over�the�period�of�a�year,�using�the�above�formula,�an�availability�of�99.99�%,�for�example,�would�correspond�to�a�downtime�of�52.6�minutes.�

99�%�*�87.66�hours�per�year�99.9�%�*�8.76�hours�per�year�99.99�%�*�52.6�minutes�per�year�99.999�%�*�5.26�minutes�per�year�99.9999�%�*�0.5265�minutes�per�year

The�German�Federal�Office�for�Information�Security�(BSI)�has�defined�the�following�availability�classes:

Availability(in percent)

DowntimeUptime+ Downtime

100= 1 -

Availability class

Description Cumulated, probable downtime per year

Effects

AC�0�~95�% No�requirements�for�availability

approx.�2–3�weeks No�measures�are�necessary�as�regards�availability.��Implementing�the�IT�Grundschutz�(formerly�known�as�the�IT�Baseline�Protection�Manual)�for�the�other�basic�values�will�have�a�beneficial�effect�on�availability.

AC�1�99,0�% Normal�availability

Less�than�90�hrs. Availability�requirements�are�satisfied�by�the�simple�application�of�the�IT�Grundschutz�(BSI�100-1�and�BSI�100-2)

AC�2�99.9�% High�availability Less�than�9�hrs. The�simple�application�of�the�IT�Grundschutz�has�to�be�supplemented�by�the�implementation�of�modules�recommended�for�high�availability�requirements,�e.g.�modules�B�1.3�Contingency�Planning�Concept�and�B�1.8�Handling�Security�Incidents,�and�a�risk�analysis�on�the�basis�of�the�IT�Grundschutz�(BSI�100-3).

AC�3�99.99�% Very�high�availability

Less�than�1�hr. Implementation�of�the�measures�recommended�for�selected�objects�in�accordance�with�the�IT�Grund-schutz,�with�particular�emphasis�on�basic�availability,�e.g.�measure�M�1.28�UPS�in�the�server�room�or�M�1.56�Secondary�Power�Supply�in�the�data�centre,�supple-mented�by�HA�(high�availability)�measures�from�the�HA�Compendium

AC�4�99.999�% Maximum�availability

Approx.�5�min. IT�Grundschutz�with�additional�modelling�on�the�basis�of�the�HA�Compendium.The�IT�Grundschutz�as�the�basis�is�increasingly�supple-mented�and�replaced�by�HA�measures.

AC�5�100�% Disaster-tolerant – Modelling�according�to�the�HA�Compendium.The�IT�Grundschutz�continues�to�serve�as�a�basis�for�the�above�areas�and�other�safety/security�values�such�as�integrity�and�confidentiality.

Table�1:�BSI�availability�classes

Page 8: Reliable Data Centres - STULZ

6

The�Federal�Office�for�Information�Security�(BSI)�has�deve-loped�a�rating�system�for�data�centres:�VAIR�(Verfügbar-keitsanalyse�der�Infrastruktur�in�Rechenzentren�–�Avai-lability�Analysis�of�Data�Centre�Infrastructure).�At�www.vair-check.de,�data�centre�operators�can�enter�the�details�of�their�data�centre�infrastructure�and�check�the�reliability�of�their�data�centre�anonymously�and�free�of�charge.

Page 9: Reliable Data Centres - STULZ

7

Reliable Data Centres

3 The influence of safety standards on the design of data centres

The�planning�and�design�of�data�centres�is�governed�by�a�large�number�of�safety�standards.�They�provide�planners�and�designers�with�assistance�on�the�one�hand,�but�also�set�out�various�requirements.

Here,�we�will�present�the�most�important�standards�in�the�field�of�ISMS�(Information�Security�Management�Systems)�and�ITIL�(IT�Infrastructure�Library),�as�well�as�the�Sarbanes-Oxley�Act.

�� 3.1� ISO�27001�/�ISO�27002:2008

The�ISO/IEC�27001�standards�that�came�into�force�in�October�2005�cover�the�protection�of�confidential�busi-ness�information.�It�is�becoming�increasingly�important,�as�it�provides�a�foundation�on�which�companies�are�in�a�position�to�satisfy�the�requirements�of�third�parties.�These�can�be�legal�requirements�(such�as�KonTraG,�HGB�and�GoB,�GoBS,�GDPdU,�BDSG,�TMG,�TKG�and�StGB),�contractual�requirements�(e.g.�from�customers)�or�other�requirements.�This�standard�replaces�the�previous�British�standard�BS�7799-2,�which�was�annulled�without�replace-ment�in�February�2006.

The�specialist�business�term�used�is�compliance,�and�denotes�both�conformity�to�laws�and�guidelines�and�voluntary�codes�of�practice�by�business.

The�ISO/IEC�27001�assists�the�creation�of�a�process�for�building�and�operating�an�Information�Security�Manage-ment�System.�This�process�of�continuous�improvement�takes�place�in�four�known�steps:�“Plan,�Do,�Check,�Act”,�with�which�you�will�be�familiar�from�ISO�9001�(Quality�Management).�

Major�assistance�is�also�provided�by�the�Grundschutz�manuals�(catalogues),�which�have�been�developed�by�the�

BSI�(Federal�Office�for�Information�Security)�over�many�years,�and�conforms�to�“ISO�27001,�based�on�IT�Grund-schutz”.�The�modules�in�the�catalogues�are�extremely�valuable�in�the�implementation�of�an�Information�Secu-rity�Management�System.�

The�PLAN�phase�entails�the�planning�of�the�ISMS.�Above�all,�the�area�of�application�and�boundaries�of�the�ISMS�are�defined�here,�and�then�approved�by�the�management.�A�risk�analysis�is�also�carried�out.�This�ascertains�which�systems�and�applications�are�important�for�maintaining�a�company’s�business�operations,�and�how�great�depen-dency�on�them�is.�Based�on�the�results�of�this�analysis,�conclusions�are�drawn�about�the�required�level�of�protec-tion,�and�the�desired�availability�of�systems�and�applica-tions�is�determined.�

The�DO�phase�incorporates�concrete�measures�to�mini-mise�and�detect�risks�with�the�aid�of�a�risk�handling�plan.�The�ISO�27002:2008�(formerly�17799)�is�a�“Code�of�Practice�for�Information�Security�Management”�and,�as�such,�provides�valuable�tips�for�complying�with�the�“Controls/Measures”�contained�in�ISO�27001.�It�is�more�or�less�a�guide�for�putting�the�ISO�27001�into�practice.�Section�9�“Physical�and�Environmental�Protection”�also�sets�out�measures�and�suggestions�for�rooms�and�infrastructures.�Certification�is�only�granted�on�the�basis�of�ISO�27001�or�BSI�ISO�27001,�based�on�the�IT�Grundschutz.

During�the�CHECK�phase,�regular�monitoring�and�periodic�audits�ensure�that�implemented�measures�are�checked�on�a�regular�basis,�in�order�that�potential�improvements�can�be�flagged�up�(e.g.�fire�safety�monitoring�mechanisms,�fire�safety�tests).

In�the�fourth�phase�(ACT),�measures�defined�as�improve-ments�are�put�into�practice.

Page 10: Reliable Data Centres - STULZ

8

�� 3.2� ITIL

“IT�Service�Management”�is�an�important�consideration�during�the�planning�and�operation�of�a�“reliable�data�centre”.�Best�practice�recommendations�for�IT�service�management�have�been�in�existence�since�the�end�of�the�1980s,�when�the�British�government’s�Central�Computer�and�Telecommunications�Agency�(CCTA,�now�part�of�the�Office�of�Government�Commerce�–�OGC),�published�the�first�elements�of�the�IT�Infrastructure�Library�(ITIL).�These�guidelines,�which�have�been�set�out�in�writing,�range�from�detailed�advice�to�individual�processes�within�the�ITIL,�from�rules�of�procedure�to�the�newly�published�standard�ISO�20000�(formerly�BS�15000).�

For�existing�data�centres,�customers�also�use�a�service�management�system�in�accordance�with�ITIL�as�orienta-tion.�Computer�service�centres�are�often�confronted�by�invitations�to�tender�that�require�the�participating�compa-nies�to�have�ITIL.�Two�key�areas�are�always�included:

�� Service-Support�� Service-Delivery

This�set�of�rules�applies�to�all�IT�organisations�in�all�com-panies�–�no�matter�what�their�size.

�� 3.3� Sarbanes-Oxley�Act�and�SAS�70

The�Sarbanes-Oxley�Act�(SOX)�is�a�US�law�intended�to�improve�company�reporting,�and�was�issued�in�response�to�the�balance�sheet�scandals�involving�companies�such�as�Enron�and�Worldcom.�This�law,�which�has�been�in�force�since�30�July�2002,�does�not�only�apply�to�financial�data,�it�also�demands�security�in�IT.

Firstly,�the�law�apples�to�all�companies�listed�on�the�American�stock�exchange.�Then�it�also�applies�to�non-US�companies�that�have�a�parent�company�or�subsidiary�listed�on�the�American�stock�exchange.

The�Sarbanes-Oxley�Act�stipulates�that�company�proces-ses�must�be�described�and�defined�and�monitoring�proce-dures�established,�with�the�aim�of�minimising�the�risk�of�

incorrect�financial�statements.�Companies�are�monitored�by�approved�auditors�in�accordance�with�the�“SAS�70”�audit.�This�in�turn�is�largely�based�on�the�“Cobit�4.1”�rules�from�the�ISACA�(USA).��If�a�company�that�must�comply�with�SOX�has�outsourced�individual�systems�or�its�entire�IT,�for�example,�the�SAS�70�audit�must�also�be�applied�to�the�provider�in�question,�although�the�responsibility�always�remains�with�the�contractor.�In�this�case,�either�the�customer’s�auditors�can�carry�out�checks�to�SAS�70�in�the�computer�service�centre,�or�the�centre�itself�can�arrange�to�have�controls�carried�out.�The�auditor’s�report�must�not�exceed�the�time�of�the�customer’s�annual�financial�statement�by�more�than�six�months.�For�this�reason,�SOX�controls�must�basically�be�conducted�twice�a�year,�which�entails�considerable�expenditure.

At�international�level,�possible�conflicts�between�the�Sarbanes-Oxley�Act�and�national�regulations�have�been�discussed.�Any�solution�to�these�conflicts�is�still�far�from�clear.�A�“Euro�SOX”�is�in�progress,�however.�In�addition,�the�IDW�(Institut�der�Wirtschaftsprüfer�–�German�Institute�of�Public�Auditors)�is�currently�basing�its�instructions�for�compliance�with�test�requirements�on�Cobit�4.1.

�� 3.4� Assessment�of�standards

The�standards�described�above�are�frequently�checked�by�customers,�certification�societies,�auditors�and�other�institutions.�Whether�the�Sarbanes-Oxley�Act�and�SAS�70�make�a�data�centre�more�reliable�may�be�debatable�–�however,�the�general�requirements�contained�in�ISO/IEC�27002:2008�and�ISO/IEC�27001:2005�governing�measures�to�improve�security�are�thoroughly�justified�and�sensible.�ITIL�and�ISO�20000�demonstrably�safeguard�and�improve�processes�in�a�data�centre.�Public�sector�contractors�often�require�certification�to�BSI�–�here,�however,�the�expen-diture�for�documenting�and�operating�the�ISMS�is�very�high.�The�best�combination�is�ISO�27001�with�reference�to�the�IT�Grundschutz�(where�applicable),�not�certification�by�the�BSI�in�Bonn.�

Page 11: Reliable Data Centres - STULZ

9

Reliable Data Centres

4 The basis of IT infrastructure: the rack

Whether�you�have�a�separate�data�centre�or�an�individual�server�cabinet,�the�basis�for�secure�accommodation�of�IT�systems�is�always�the�rack.�Since�in�most�companies�IT�systems�consist�of�standardised�19-inch�components,�scalable,�flexible�rack�systems�in�this�design�offer�the�best�choice�when�putting�together�a�stable�and�resilient�IT�infrastructure.�It�ensures�perfect�interaction�between�sys-tem�and�support�components,�such�as�the�power�supply�and�air�conditioning.�A�company’s�decision�as�to�whether�to�house�its�IT�systems�in�a�dedicated�data�centre�or�to�have�a�standalone�solution�in�individual�server�cabinets�depends�on�the�IT�and�the�structural�requirements.�In�both�cases,�however,�the�same�fire�safety�and�other�secu-rity�standards�apply,�for�they�are�intended�to�protect�ICT�systems�and�data�from�the�inside.

�� 4.1� Secure�server�cabinets

A�secure�server�cabinet�should,�if�possible,�be�modular�in�design.�It�provides�the�company�with�reasonable�security�at�a�manageable�cost.�A�modular�cabinet�can�be�dismantled,�converted�or�moved�to�another�location�if�necessary.�This�flexibility�also�offers�advantages�in�terms�of�transport�and�re-erection�if�a�company�relocates.

Modularity�is�also�advantageous�when�wishing�to�expand�housing�solutions�or�air-conditioning�concepts.

When�planning�a�secure�server�cabinet�–�as�with�a�reliable�data�centre�–�the�following�characteristics�are�necessary�for�ensuring�that�systems�have�comprehensive�security�and�availability:

�� Ensuring�constant�temperature�and�air�humidity�through�precision�air�conditioning

�� Safeguarding�the�electricity�supply�by�means�of�an�uninterruptible�power�supply�(UPS)�and�an�additional�backup�power�supply�if�necessary

�� Protection�against�access�by�third�parties�(locking�systems,�network-monitored�rack�access,�biometric�data�acquisition)

�� Fire�prevention�and�reaction�� Modules�can�be�integrated�in�a�central�monitoring�

and�management�system

The�subject�of�stability�is�a�vital�aspect�in�all�rack�solu-tions.�Due�to�the�high�packing�density�of�modern�server�systems�and�storage�solutions,�as�well�as�the�power�supply�units�they�contain,�server�racks�with�a�load-bearing�capacity�of�up�to�1,000�kg�may�be�required,�depending�on�the�application.�Consequently,�the�bases�of�units�and�sliding�rails�must�also�be�designed�for�high�loads.�Bases�and�rails�can�currently�withstand�a�weight�of�up�to�150�kg�each.�

Another�important�aspect�is�the�routing�of�cables.�As�increasingly�fast�networks�are�used�in�combination�with�copper�wiring,�it�is�essential�that�power�and�data�cables�are�routed�separately,�to�prevent�them�from�causing�one�another�interference.�When�selecting�a�rack,�it�is�also�vital�to�ensure�that�the�power�distribution�system�can�be�easily�integrated,�for�in�the�end�a�reliable�power�supply�is�a�prerequisite�for�available�IT.�Fuse-protected�low-voltage�sub-distribution�must�also�be�available,�as�well�as�a�flexible�power�distribution�system�in�the�rack,�which�can�be�connected�both�to�the�mains�power�supply�and�an�uninterruptible�power�supply�(UPS).�Modern�solutions�have�up�to�88kW�in�a�rack.�This�is�made�possible�by�four�independent,�three-phase�incoming�feeders,�which�gua-rantee�a�reliable�power�supply�even�in�the�face�of�more�exacting�requirements.

As�server�capacity�and�the�packing�density�in�the�rack�increase,�the�importance�of�ventilation�concepts�such�as�perforated�doors�and�partitions�between�hot�and�cold�areas�of�the�rack�has�grown�enormously.�Other�energy-optimised,�performance-enhancing�solutions�can�be�achieved�by�hot�or�cold�housing�concepts�that�form�part�of�the�rack.�In�the�case�of�extreme�power�losses�in�the�rack,�liquid-cooled�solutions�in�the�form�of�air/water�heat�exchangers�are�indispensable.

Page 12: Reliable Data Centres - STULZ

10

Both�elements,�safeguarding�the�power�supply�and�the�air�conditioning,�can�be�monitored�by�sensors�integrated�in�the�infrastructure.�Whether�wired�or�wireless,�these�sen-sors�record�the�servers’�humidity�and�temperature,�and�also�their�power�input.�A�modern,�sensor-based�monito-ring�system�may�also�take�on�the�task�of�access�control.

4.1.1� Making�an�inventory�of�the�server�cabinet

In�data�centres�–�especially�those�of�a�certain�size�–�it�is�hard�to�maintain�an�overview�of�all�installed�hardware�components.�Although�it�is�possible�to�communicate�with�any�intelligent�IT�device�today,�the�equipment’s�physical�connection�to�the�rack�and�the�relevant�height�unit�is�another�matter.�The�structure�of�equipment�in�the�individual�cabinets,�with�servers,�fans,�UPS,�etc.�is�often�not�transparent.�As�a�result,�making�inventories�and�continually�updating�data�about�the�arrangement�of�components�in�the�data�centre�is�expensive�and,�in�most�cases,�time-consuming.�In�many�cases,�existing�manu-ally�written�documentation�is�not�checked�for�accuracy.�However,�this�information�is�vital�for�making�decisions�in�the�event�of�failure.�

Another�problem�is�the�half-life�of�the�acquired�informa-tion,�for�these�records�or�updates�still�only�ever�represent�a�snapshot�of�the�data�centre�inventory.�Yet�efficient�rack�assignment�and�transparent�component�management�require�continual,�up-to-date�and�therefore�reliable�data.

To�ensure�constant�access�to�up-to-date�inventory�data,�modern�inventory�systems�exist�directly�in�the�rack,�which�register�the�equipment�of�the�19”�level�completely�and�without�contact.

On�the�one�hand,�the�rack�configurations�are�visually�dis-played�on�a�website�belonging�to�the�monitoring�system�in�question;�on�the�other�hand,�the�user�can�open�a�list�or�import/export�data�using�the�XML�file�format.�These�data�records�can�then�be�further�processed�in�external�data-bases,�for�example,�and�considerably�facilitate�everyday�work�during�data�centre�optimisation.

�� 4.2� Network�technology

A�complete�examination�of�data�centres,�with�special�consideration�of�security�aspects,�cannot�fail�to�include�the�subject�of�network�technology.�Many�companies�have�already�switched�their�telephone�systems�to�Voice�over�IP�(VoIP).�Virtualised�clients�are�the�next�step.�This�enables�an�increasing�number�of�basic�services�critical�to�business�to�take�place�via�data�lines�which,�with�Power�over�Ether-net�(PoE),�also�supply�power�to�the�terminal�devices.�As�network�technology�becomes�increasingly�important�for�problem-free�business�operations,�security�requirements�are�tightening�up�in�this�area,�too.

As�with�the�servers,�for�network�technology�the�rack�provides�the�basic�housing.�Since�these�active�compo-nents�are�also�standardised�to�19�inches,�network�cabinets�are�generally�based�on�the�same�platform.�Requirements�are�also�comparable�in�terms�of�stability,�fire�safety�and�access�control.�However,�since�the�network�infrastructure�installed�in�the�building�tends�to�be�designed�to�last�more�than�10�years,�long-term�planning�is�recommended�when�purchasing�network�cabinets,�and�flexibility�of�accesso-ries�is�desirable.�This�way,�future�developments�will�also�be�taken�into�account.�It�must�be�borne�in�mind�that�the�interior�structure�of�racks�differs�greatly�from�one�to�the�next.�

The�frequent�switching�between�the�various�connections�of�the�network�components�means�that�cables�in�network�cabinets�have�to�be�rerouted�considerably�more�often�than�is�the�case�in�server�cabinets.�These�movements,�also�known�as�MACs�(Move,�Add,�Change),�as�well�as�incre-asing�port�density,�lend�special�importance�to�the�task�of�cable�management.�This�starts�with�the�roof�panels�and�bases.�Simple�cable�entry�at�these�points�facilitates�retrofitting�and�keeps�cable�runs�short.�Routing�ducts�and�guide�panels�ensure�orderly�fine�distribution�in�the�rack.�At�the�same�time,�cable�management,�in�particular,�must�place�great�importance�on�the�stability�of�components.�For�modern�current-carrying�network�cables�are�much�more�expensive�and�rigid�than�their�Cat-5�predecessors.�

Page 13: Reliable Data Centres - STULZ

11

Reliable Data Centres

One�subject�that�is�currently�gaining�in�importance�when�it�comes�to�network�cabinets�is�air�conditioning.�Swit-ches�and�routers�are�becoming�ever�more�powerful,�and�produce�more�and�more�waste�heat.�For�this�reason,�here,�too,�possibilities�for�expansion�have�to�be�considered.�The�spectrum�ranges�from�passive�air�conditioning�via�roof�panels,�ventilation�attachments�and�dual-walled�housings�to�fans,�right�up�to�roof�cooling�units.�

�� 4.3� Reliable�data�centre

In�addition�to�the�basic�requirements�for�a�reliable�data�centre�mentioned�above,�numerous�project�details�have�to�be�clarified�in�relation�to�construction.�

First�of�all,�a�precise�analysis�of�risks�and�weak�points�in�the�company�must�be�compiled,�flagging�up�possible�risks�for�IT�systems.�Responsibility�for�planning�and�building�a�data�centre,�persons�with�access�authorisation�and�regular�safety�checks�by�independent�auditors�must�be�covered.

Many�people�are�responsible�for�the�planning,�construc-tion�and�operation�of�a�data�centre.�In�addition�to�IT�spe-cialists,�these�include�building�experts�such�as�architects�and�construction�engineers,�specialist�planners�for�air�conditioning,�energy�and�active�defence,�the�department�in�charge�of�organisation�and,�last�but�not�least,�the�management�board.�

The�physical�requirements�facing�a�data�centre�do�not�consist�only�of�pure�IT-related�matters,�such�as�the�num-ber�and�type�of�servers�and�network�and�storage�devices,�but�also�active�and�passive�defence.�

A�modular�(and�hence�extendible/modifiable)�fireproof�security�room,�certified�as�far�as�possible,�may�form�part�of�the�equipment�of�the�data�centre.�The�use�of�a�stable,�multilayered�fire�safety�door�with�the�same�protection�values�as�the�security�room�is�also�compulsory.�Other�constructions�such�as�a�hermetically�sealed�ceiling,�wall�and�floor�system�to�protect�against�the�ingress�of�smoke�or�water,�for�example,�or�a�multistage�very�early�fire�detec-tion�system�with�multiple�sampling�points,�including�in�the�raise�floor,�are�now�the�state�of�the�art.�These�are�joined�by�an�appropriately�dimensioned�autonomous�fire�extinguishing�system�with�pressure�relief�and�ventilation�duct�dampers,�personal�access�control�using�card�readers�or�biometric�technology,�and�data�centre�periphery�moni-toring�via�LAN�video�technology.

To�ensure�the�flexible�expansion�of�data�centres,�it�is�a�good�idea�to�work�with�planners�and�suppliers�who�can�ensure�the�long-term�availability�of�the�required�products.

Page 14: Reliable Data Centres - STULZ

12

5 Energy, air conditioning and cooling

�� 5.1� Power�supply�companies�(PSCs)�–�electricity�feed-in�and�distribution�in�the�company

5.1.1� Current�situation

The�electricity�supply�is�of�major�importance�for�the�ope-ration�of�server�cabinets�or�entire�data�centres.�

The�power�supply�chain�begins�at�the�power�supply�companies,�which�generate�and�supply�primary�energy.�The�primary�energy�generators�transport�the�electricity�through�cables�via�high-voltage�masts�to�medium-voltage�substations.�From�the�medium-voltage�substations,�the�electricity�is�conveyed�via�underground�cables�to�trans-former�stations.�Transformer�stations�are�mostly�situated�in�large�buildings�or�at�the�edge�of�the�road�on�specially�designed�premises.�

Large�data�centres�with�an�area�of�several�thousand�square�metres�often�feature�additional�feed-ins�via�a�second�medium-voltage�substation,�ensuring�full�redun-dancy�–�i.e.�several�channels�to�increase�availability�–�even�as�far�as�the�power�plants�themselves.�

Possible�causes�for�an�interruption�to�the�power�supply�could�be:

�� Technical�faults�in�equipment�(e.g.�servers�and�their�components)

�� Technical�faults�in�the�power�distribution�(e.g.�cables,�sub-distribution�boards)

�� Faults�in�alternative�power�supply�solutions�(e.g.�standby�generating�systems,�also�known�as�emer-gency�diesel�generators,�battery-backed�uninterrupti-ble�power�supply�(UPS)�systems)

Faults�due�to�the�process�(e.g.�design�faults�in�the�power�supply,�logistical�errors)

Past�examples�show�how�dramatically�situations�can�escalate�if�the�power�supply�fails�for�longer�periods�and�no�backup�solution�is�available.�The�general�power�supply�can�succumb�to�failure�for�several�days�in�large�areas.�From�reports�of�the�damage�caused�in�such�cases,�it�is�easy�to�see�how�essential�an�autonomous�power�supply�is,�particularly�in�areas�of�critical�importance�to�compa-nies,�such�as�IT.�

When�building�a�data�centre,�ready-made�power�supply�solutions�cannot�simply�be�pulled�out�of�a�hat.�Howe-ver,�basic�principles�have�been�set�out,�which�must�be�adapted�to�individual�circumstances.�The�challenge�facing�the�planner�lies�in�putting�these�principles�into�practice�in�accordance�with�the�customer,�his�requirements�and�wishes�and,�last�but�not�least,�his�budget.

5.1.2� How�the�infrastructure�functions

The�power�supply�functions�by�means�of�ring�mains,�the�electricity�from�which�is�transformed�to�400�V�in�the�transformer�stations�and�which�is�conveyed�to�the�data�centre�through�cables�or�busbars�via�the�low-voltage�main�distribution�and�utility�power�supply�network.�The�normal�mains�sub-distribution�also�supplies�the�uninter-ruptible�power�supply�(UPS)�systems�with�electricity.

The�output�from�the�UPS�systems�is�routed�via�the�UPS�sub-distributors,�and�from�there�to�the�individual�server�cabinets.�Junction�boxes�or�distribution�boxes�are�installed�in�the�raised�floor�for�this�purpose.�From�these�branches�or�junctions,�more�cables�feed�the�electricity�to�the�power�supply�units�of�the�servers�in�the�cabinet.�If�only�one�UPS�system�is�installed,�power�supply�units�A�and�B�have�a�joint�supply;�if�there�are�two�UPS�systems,�their�supply�is�separate.�A�2�x�N�supply�therefore�increases�availability.

Page 15: Reliable Data Centres - STULZ

13

Reliable Data Centres

Table�2�from�the�BITKOM�matrix�“Planning�Guide�for�a�Reliable�Data�Centre”�–�Electricity�feed-in�from�PSC

Categories�A�and�B�are�currently�in�place�in�many�small�to�medium-sized�businesses,�often�even�without�any�pos-sibility�of�feed-in�for�a�mobile�standby�generator.�When�examined�more�closely,�however,�this�version�does�not�provide�any�real�security,�and�places�its�trust�solely�in�the�energy�suppliers.�We�are�always�hearing�the�old�saying�“It�won’t�happen�to�me.”�And�it�may�not�have�happened…yet.�But�if�a�single�link�in�the�supply�chain�fails,�this�interrupts�the�entire�supply�from�the�PSC,�and�electricity�has�to�be�supplied�by�the�UPS�system.�Moreover,�the�backup�time�of�a�UPS�system�is�generally�severely�limited.�It�depends�on�the�number�of�batteries�installed,�and�the�power�that�has�to�be�provided.�A�UPS�system�is�generally�unable�to�bridge�a�power�cut�lasting�more�than�30�minutes.�In�this�case,�a�functioning�computer�shutdown�routine�should�automatically�be�initiated,�messages�transmitted,�data�saved,�applications�closed�and,�finally,�computers�properly�shut�down.�

Planners�must�therefore�take�particular�care�to�ensure�that�the�backup�time�of�the�UPS�system�is�longer�than�the�time�required�to�transport�and�activate�a�mobile�standby�generator.�As�a�rule,�batteries�are�used�for�the�above�constellations.

Greater�security�is�offered�by�Category�C.�Here,�the�power�supply�is�redundant�even�from�the�low-voltage�main�distribution.�If�a�supply�channel�upstream�of�the�low-voltage�main�distribution�fails,�electricity�is�automatically�supplied�via�the�second,�redundant�channel.�If�the�primary�energy�supplier�suffers�failure,�the�power�supply�is�still�assured�by�the�mobile�standby�generator.

Unlike�Categories�A�and�B,�Category�C�features�a�second�UPS�with�two�dedicated�UPS�sub-distributors.�This�ensures�a�redundant�supply�from�the�UPS�systems�to�the�server�power�supply�units.

Category�D�differs�from�Category�C�in�that�the�supply�is�ensured�by�two�different�transformer�stations,�each�with�their�own�downstream,�redundant�infrastructure.�Moreo-ver,�in�the�event�of�a�power�cut�at�the�energy�supplier’s�end,�electricity�is�supplied�by�the�stationary�standby�gene-rator.�This�category�promises�a�very�high�level�of�security.�

Category�E�represents�the�“gold�standard”.�Not�only�does�it�offer�additional�redundancy�via�a�second�standby�gene-rator,�it�also�provides�an�additional�feed-in�from�another�independent�medium-voltage�substation.�However,�in�almost�all�cases�the�second�cable�feed�from�another�medium-voltage�station�first�has�to�be�produced�by�the�responsible�primary�energy�supplier.�This�can�mean�that�several�kilometres�of�new�cable�must�be�routed�to�the�

5.1.3� Recommended�equipment�for�different�downtimes

Dat

a ce

ntre

ca

tego

ry Electricity feed-in from PSC

Permitted

data centre dow

ntime*

Servers cabinet Servers cabinet Data centre / server room

up to 5 kW from 5 kW to 30 kW 500 to 2500 W/m2A Standard 72�h

B Standard 24�h

C Redundant�feed-ins 1�h

D Redundant�feed-ins 10�min

E Redundant�feed-ins�from�different�transformer�stations 0�min

Page 16: Reliable Data Centres - STULZ

14

data�centre�premises,�which�is�extremely�cost-intensive�and�should�be�taken�into�account�during�costing.

Regular�inspection�and�maintenance�of�the�entire�infra-structure�by�qualified�personnel�and�compliance�with�regulations�and�guidelines�on�operation�of�the�systems�are�all�indispensable�for�maintaining�availability.

�� 5.2� Power�distribution�in�the�company

5.2.1� Current�situation

Power�distributors�convey�the�power�of�the�utility�power�supply�network,�the�generator�and�the�UPS�to�the�equip-ment,�systems�and�lighting.�Two�distribution�systems�can�be�employed�to�ensure�higher�availability.

5.2.2� How�the�infrastructure�functions

In�the�power�distribution�system,�the�normal�mains�sup-plies�the�building�infrastructure,�including�lifts,�lighting�–�but�not�emergency�lighting�systems�to�VDE0108�–�com-pressors�in�DX�air�conditioners�(DX�=�direct�expansion),�chillers�and�other�installations.�In�the�event�of�power�failure,�this�power�supply�is�interrupted�until�an�available�generator�starts�up�and�an�automatic�switch�causes�it�to�restore�the�electricity�supply.

All�power�distribution�systems�must�be�equipped�with�an�input�fuse.�The�size�and�type�of�power�distribution�system�depends�upon�the�power�to�be�distributed,�the�desired�number�of�circuits�and�the�power�per�circuit.�Please�refer�to�the�table�below.

�Ideally,�fuse�protection�is�effected�selectively�within�the�power�strip,�i.e.�the�outputs�are�monitored�not�by�an�overall�fuse�but�by�several�fuses,�either�separately�or�as�a�group.�As�a�result,�in�the�event�of�failure�only�the�affected�output�or�group�is�disconnected�from�the�mains,�instead�of�the�entire�power�strip.�Either�fuses�or�automatic�

cut-outs�may�be�used.�A�cabinet�typically�contains�two�separate�power�strips,�which�permit�redundant�operation�of�IT�systems.

Overview�of�power�output�classes:�

Phases Maximum amperage

Maximum power output

One 16�A 3.6�kW

One 32�A 7.2�kW

Three 16�A 11�kW

Three 32�A 22�kW

(Further�combinations�with�two�phases�are�also�possible�but�are�uncommon�in�Germany)

Modern�power�distribution�units�(PDUs)�also�feature�measuring�or�switching�functions,�as�well�as�a�network�connection�for�extended�energy�management.�In�addi-tion,�various�models�offer�environmental�monitoring�with�a�range�of�sensors�for�measuring�the�temperature�and�air�humidity,�for�example.

Since�most�IT�units�in�data�centres�are�installed�in�19”�cabi-nets,�the�question�arises�as�to�where�the�power�distribu-tors�should�be�situated�and�how�the�power�cables�should�be�routed�to�the�19”�cabinets.�PDUs�are�either�installed�in�the�wall�or�surface-mounted,�in�separate�cabinets�or�inte-grated�in�a�19”�version.�The�cables�are�often�routed�under�the�raised�floor,�which�is�also�used�as�a�conduit�for�cold�air.�The�passage�of�air�can�be�impaired,�while�access�to�cables�is�difficult.�This�problem�can�be�remedied�by�routing�cables�into�the�19”�cabinet�from�below�via�a�raised�floor�plate.�Alternatively,�cables�can�be�routed�along�the�ceiling�or�walls�in�cable�runs�or�via�power�distribution�systems,�so�that�they�have�to�enter�the�19”�cabinet�from�above.�Integ-rated�PDUs�have�an�advantage�in�that�they�are�located�in�the�vicinity�of�the�point�of�use,�and�are�therefore�within�easy�reach�of�the�19”�cabinets.�Cables�can�be�routed�on�the�roof�of�the�19”�cabinets,�provided�that�power�and�data�cables�are�routed�separately.

Page 17: Reliable Data Centres - STULZ

15

Reliable Data Centres

Special�attention�must�be�paid�to�the�PDU�strips�in�the�cabinets.�Thanks�to�the�units’�modern,�compact�design,�these�days�many�systems�can�be�installed�in�one�cabinet.�In�extreme�cases,�a�cabinet�may�have�42�height�units�with�42�servers�per�height�unit�and�two�power�supply�units�per�server,�for�example.�A�total�of�84�sockets�would�have�to�be�available�for�this�cabinet.�

5.2.3.�Intelligent�multiple�outlet�strips

For�management�at�rack�level,�transparency,�order�and�ease�of�handling�are�vital.�Ideally,�the�multiple�outlet�strips�used�in�a�data�centre�will�have�different�plug-in�modules�for�country-specific�systems,�for�example,�which�can�be�used�by�untrained�staff.�In�this�case,�international�organisations�have�the�option�of�using�the�same�type�of�multiple�outlet�strip�in�all�their�branches,�without�having�

to�assign�specialist�personnel�to�convert�their�systems.�Today’s�multiple�outlet�strips�allow�modules�to�be�repla-ced�during�ongoing�operation.�High-end�systems�such�as�this�generally�also�feature�HTTP�or�SNMP�monitoring�and�management�options�and�user�administration,�which�guarantees�that�only�authorised�personnel�can�configure�the�outlet�strip.�These�modular�systems�permit�basic�equipment�of�the�racks�by�means�of�a�vertical�mounting�rail�with�three-phase�feed-in.�The�various�plug-in�modules�can�simply�be�inserted�in�this�rail,�considerably�reducing�wiring�and�assembly�time�and�expenditure.

Finally,�hosting�companies,�for�example,�which�must�demonstrate�a�high�degree�of�accuracy�in�the�distribution�of�energy�costs�per�server�(in�one�rack),�can�now�benefit�from�officially�calibrated�socket�modules.�Calibrating�measuring�instruments�of�this�kind�are�also�available�for�the�PDU.

5.2.4� Recommended�equipment�for�different�downtimes

Dat

a ce

ntre

ca

tego

ry Distribution

Permitted

data centre dow

ntime*

Servers cabinet Servers cabinet Data centre / server room

up to 5 kW from 5 kW to 30 kW 500 to 2500 W/m2A Standard,�

connection�of�servers�via�UPS�and�utility�power�supply�network�is�recommended72�h

B Standard,�connection�of�servers�via�UPS�and�utility�power�supply�network�is�recommended

24�h

C Redundant�setup�(A�and�B) 1�h

D Redundant�setup�(A�and�B) 10�min

E Redundant�setup 0�min

Table�3�from�the�BITKOM�matrix�“Planning�Guide�for�a�Reliable�Data�Centre”�–�Distribution

Page 18: Reliable Data Centres - STULZ

16

The�form�redundancy�takes�depends�on�the�number�of�power�supply�units�in�the�IT�devices.�A�good�basis�for�high�availability�would�be�two�power�supply�units�per�device,�which�are�redundant.�Then,�in�the�event�of�failure�of�a�power�supply�unit,�the�remaining�power�supply�unit�is�able�to�continue�supplying�the�IT�device�normally.�These�two�power�supply�units�per�device�should�be�supplied�by�the�PDU�via�two�separate�PDU�strips�to�two�separate�power�circuits.�Availability�can�be�further�increased�by�using�two�separate�PDUs,�which�receive�power�from�two�separate�UPS�systems�via�two�separate�transformers�and�two�separate�generators.�

5.2.5� Protective�measures�and�high�availability

Data�centres�are�subject�to�maximum�requirements�regarding�availability.�The�energy�supply�must�be�assured�permanently�to�reflect�this.�It�therefore�goes�without�saying�that�the�power�supply�to�the�data�centre�itself�and�all�areas�in�the�same�building�to�which�data�cables�are�routed�must�take�the�form�of�a�TN-S�system.�Permanent�monitoring�of�a�“clean”�TN-S�system�(e.g.�with�RCMs)�and�forwarding�messages�to�a�constantly�manned�point,�e.g.�the�control�centre,�is�absolutely�essential�for�reliable�operation.�Messages�inform�the�electrical�engineer�about�what�action�needs�to�be�taken,�so�that�he�can�take�appro-priate�service�measures�to�prevent�damage.�

All�PDUs�must�be�equipped�with�an�input�fuse�to�protect�the�lines.�The�size�and�type�of�power�distribution�system�depends�upon�the�power�to�be�distributed,�the�desired�number�of�circuits�and�the�power�per�circuit.�Power�circuits�have�either�one�or�three�phases.�Typical�power�supply�sizes�are�16�A,�single-phase�(approx.�3.5�kW),�32�A,�single-phase�(approx.�7�kW)�or�32�A,�three-phase�(22�kW)�for�high-performance�cabinets.�A�particularly�difficult�subject�is�that�of�“selective�fuse�co-ordination”,�which�enables�a�short�circuit�or�short�to�ground�in�a�cabinet�to�

be�safely�isolated�without�any�ill�effects�on�other�cabinets�and�IT�units.�

Where�personal�protection�is�concerned,�new�require-ments�exist�for�additional�protection�for�final�circuits�with�sockets.�Since�01.06.2007,�DIN�VDE�0100-410:2007-06,�“Protection�Against�Electric�Shock”�applies�to�newly�const-ructed�systems.�Modifications�and�extensions�to�existing�systems�must�also�be�carried�out�in�accordance�with�this�standard.

The�above�standard�stipulates�additional�protection�by�means�of�residual�current�devices�(RCDs)�for�all�sockets�in�a.c.�voltage�systems,�if�the�system�in�question�is�to�be�used�by�laypeople�or�is�for�general�use.�Care�must�be�taken�to�ensure�that�faults/damages�are�eliminated�immediately�by�an�electrical�engineer,�including�in�con-nected�electrical�devices/consumables/equipment.�This�requires�a�permanent�monitoring�system,�plus�organisati-onal�measures�to�ensure�rapid�troubleshooting.�

A�continuous�residual�current�monitor�satisfies�the�current�standard�governing�protective�measures�and�also�offers�increased�fire�protection,�even�without�switch-off�by�an�RCD.�

�� 5.3� Uninterruptible�power�supply�(UPS)

5.3.1� Current�situation

The�causes�of�power�failure�are�often�quite�banal:�even�simple�voltage�fluctuations�or�transient�failures�in�the�electricity�grid�can�be�enough�to�damage�hardware�or�software�or�to�interfere�with�it�to�such�an�extent�that�severe�errors�occur�in�IT�processes.�Irregularities�in�the�grid�may�be�rare,�but�they�are�more�frequent�than�many�people�assume.

Page 19: Reliable Data Centres - STULZ

17

Reliable Data Centres

UPS�systems�are�employed�to�prevent�the�possible�negative�consequences�of�brief�power�failures.�They�filter�out�disturbances,�such�as�surges�and�dips�in�voltage,�and�bridge�interruptions�in�the�mains�supply.�This�reduces�transmission�errors,�computer�crashes,�bugs�and�loss�of�data.�

Figure�1:�Frequency�of�mains�faults�in�relation�to�their�average�duration

Since�the�1980s,�investment�in�German�power�supply�systems�has�fallen�by�approximately�40�%.�In�recent�years,�average�downtime�has�risen�from�15�minutes�to�23�minu-tes�per�year.�And�the�growing�use�of�regenerative�energy�means�that�electricity�grids�are�increasingly�reaching�the�limits�of�their�capacity.�The�risk�of�bottlenecks�and�black-outs�is�rising.�These�are�the�conclusions�of�the�new�VDE�study�on�power�supply�quality.

5.3.2� UPS�system�technology

UPS�systems�differ�in�their�different�technology.�The�most�frequently�used�technology�is�the�static�UPS�system.�Rechargeable�(secondary)�cells�(accumulators)�are�used�to�store�energy.�When�two�or�more�connected�cells�are�linked�in�a�circuit,�this�is�known�as�a�secondary�battery,�or�just�a�rechargeable�battery.�In�the�event�of�a�power�failure,�the�energy�from�the�accumulator�is�kept�ready�for�critical�loads�by�a�static�transformer�(inverter)�at�the�output�of�the�UPS�system.�The�backup�time�is�determined�by�the�load�and�the�capacity�of�the�accumulator�batteries,�but�is�typically�within�the�range�of�10�to�max.�30�minutes.�

The�second�type�of�technology�used�is�the�dynamic�UPS�system�either�with�or�without�piston�engine.�Depen-ding�on�the�design,�the�energy�is�stored�by�a�kinetic�bulk�storage�device�or,�as�above,�a�rechargeable�battery�system.�The�dynamic�UPS�system�makes�the�battery’s�energy�available�to�critical�loads�by�means�of�a�rotating�transfor-mer�(generator)�at�the�output�of�the�UPS�system.�With�kinetic�storage,�the�backup�time�depends�on�the�load�of�the�IT�units�and�the�kinetic�energy�of�the�storage�device�(mass�and�speed),�and�is�measured�in�seconds.

The�dynamic�UPS�system�with�combustion�engine�combines�a�UPS�system�with�a�standby�generator,�and�is�therefore�capable�of�bridging�power�failures�over�a�longer�period.�

5.3.3� Method�of�operation

Static�UPS�versions�are�divided�into�3�categories.�The�classification�and�associated�methods�for�determining�static�UPS�systems�are�defined�and�described�in�European�standard�EN62040-3.�

Number of power failures and brief power losses p.a

Most dangerous lurks here:in the range up to 1 second

0-10 ms 10-20 ms 20 ms-1s 1 s-1h > 1h

60

50

40

30

20

10

0

Duration ofpower failures

Page 20: Reliable Data Centres - STULZ

18

Dynamic�UPS�systems�with�and�without�combustion�engine�are�subject�to�DIN�6280-12.�

In�data�centres,�static�UPS�systems�with�the�classification�“VFI”�to�EN64040-3�or�diesel�UPS�systems�to�DIN�6280-12�should�always�be�used.�

Static�UPS�systems�with�this�classification�are�available�with�an�output�range�of�10�kVA�to�1600�kVA,�and�can�be�

connected�in�parallel�up�to�an�output�of�4800�kVA,�depen-ding�on�the�make.�

Diesel�UPS�systems�are�available�with�an�output�of�200�to�1750�kVA.�They�are�able�to�cover�the�low�and�medium-vol-tage�range.�Numerous�units�can�be�connected�in�parallel.

Mains failures Time EN 62040-3 UPS-solution Arresto solution1.� Power�failure >�10�ms VFD

Voltage�+�Frequency��Dependent

Classification�3Passive�stand-by-opera-tion�(offline) .............................

2.� Voltage�fluctuations

<�16�ms

.............................

3.� Voltage�peaks 4�...�16�ms

.............................

4.� Undervoltage continuous VI�*)Voltage�Independent

Classification�2Line�interactive�operation .............................

5.� Overvoltage continuous

.............................

6.� Surge <�4�ms VFIVoltage�+�Frequency�Independent

Classification�3Double�conversion��operation�(online)��Delta�converter

.............................

7.� Effects�of��lightning

sporadic Lightning�and�over�voltage�protection��IEC�60364-5-534

8.� Voltage�distortion�(burst)

periodic

.............................

9.� Voltage�harmonics continuous

.............................

10.�Frequency�fluctuations

sporadic

.............................

*)�Alternative�technologies�are�capable�of�overcoming�mains�disturbances�1�to�9.

Table�4:�Types�of�mains�faults�and�suitable�UPS�solutions�according�to�EN62040-3�(ref.:�“Uninterruptible�Power�Supplies,�European�Guide“;�Hsgr.�ZVEI�2004)

Page 21: Reliable Data Centres - STULZ

19

Reliable Data Centres

5.3.4� UPS�redundancy

Redundancy�is�takes�the�following�form�when�UPS�sys-tems�are�employed.

Figure�2:�Redundancy�in�UPS�solutions

5.3.5� Electronic/manual�bypass

The�electronic�bypass�has�the�task�of�switching�the�loads�without�interruption�from�the�mains�to�the�inverter�of�the�UPS�system�(safe�busbar)�and�back.�In�the�event�of�faults�during�inverter�operation�or�with�large�amounts�of�overload,�the�electronic�bypass�switches�the�load�back�to�the�mains�without�a�break.�Depending�on�design,�the�electronic�bypass�can�either�be�integrated�in�the�UPS�sys-tem�(single�block�and�modular�block)�or�take�the�form�of�an�external�component�(parallel�block�with�external�elect-ronic�bypass).�Another�electronic�bypass�can�be�connected�in�parallel�to�create�redundancy�(N+1).

Every�UPS�system�should�be�equipped�with�a�manual�bypass.�This�switches�the�voltage�of�the�UPS�system�off�for�the�purpose�of�service�and�maintenance�work.�If�the�manual�bypass�is�integrated�in�the�system,�voltage�is�

present�in�the�input�and�output�terminals�of�the�UPS�sys-tem�even�in�bypass�mode.�The�system�cannot�be�replaced�without�cutting�off�the�loads.�However,�when�an�external�manual�bypass�is�used,�the�UPS�system�can�be�replaced�without�the�loads�having�to�be�switched�off.�If�the�manual�bypass�takes�the�form�of�modular�blocks�connected�in�parallel,�or�parallel�blocks,�it�must�be�designed�for�maxi-mum�consumer�load.�

5.3.6� Energy�storage�units�

Kinetic�energy�storage�units�are�designed�or�dimensioned�almost�exclusively�by�the�manufacturers�of�UPS�systems.�The�achievable�backup�times�are�restricted�to�seconds,�so�that�their�use�is�limited�to�diesel�UPS�systems�or�combina-tions�with�fast-start�standby�generators.

The�electrochemical�storage�units�used�in�connection�with�UPS�systems�include�lead�and�nickel�cadmium�batteries.�Use�of�lithium�ion�batteries�has�not�become�widespread.�Nickel�cadmium�rechargeable�batteries�are�relatively�insensitive�to�increased�ambient�temperatures,�but�their�use�is�contentious�because�of�environmental�pollution.

The�energy�storage�unit�most�commonly�used�in�UPS�sys-tems�is�the�lead�battery.�Lead�batteries�are�very�sensitive�to�temperature.�Low�temperatures�reduce�the�battery�capacity�and�therefore�the�backup�time�or�power,�while�high�temperatures�reduce�the�battery’s�useful�life.�The�optimum�ambient�temperature�is�20°C.

The�useful�life�of�battery�systems�varies�depending�on�the�technology,�the�materials�used�and�other�factors.�Accor-ding�to�Eurobat,�useful�life�is�based�on�an�ambient�tem-perature�of�20�°C�and�laboratory�conditions.�The�following�useful�life�figures�are�specified:3�–�5�years�–�standard�commercial6�–�9�years�–�general�purpose10�–�12�years�–�high�performance12�years�or�more�–�longlife

N

N+1

N+1

100%

100%

100%

50%

50%

50%

2N

2(N+1)

100%

100%

50%

50%

50%

50%

50%

50%

Page 22: Reliable Data Centres - STULZ

20

To�ensure�a�reliable�power�supply,�the�battery�system�must�be�inspected�regularly�and�replaced�before�the�ser-vice�life�has�elapsed.�In�addition,�it�must�be�borne�in�mind�that�the�battery�loses�capacity�over�its�life.�Setting�the�system�up�for�very�short�backup�times�carries�the�risk�that�the�already�aging�system�may�no�longer�be�able�to�pro-vide�the�required�power�and�may�cut�off�the�UPS�system.�In�areas�of�importance�to�safety,�overdimensioning�(factor�1.25)�is�required,�to�ensure�that�sufficiently�high�capacity�is�still�available�at�the�end�of�the�battery’s�service�life.

If�the�operator�decides�to�dispense�with�redundancy�in�his�UPS�system,�the�battery�system�should�at�least�be�divided�into�two�sections.�The�achievable�backup�time�of�a�section�is�just�one�part�of�the�planned�backup�time.�This�ensures�that�power�outages,�at�least,�are�backed�up�for�a�few�seconds.�This�is�not�a�suitable�approach�for�highly�available�data�centres,�however.

5.3.7� Recommended�equipment�for�different�downtimes

The�most�important�factors�to�bear�in�mind�when�desi-gning�a�UPS�system�is�the�required�electrical�power�of�connected,�critical�loads�and�the�installation�conditions.�To�provide�backup�power�during�outages,�an�energy�storage�unit,�such�as�a�battery�system�(cabinet�or�rack�with�dis-connectors�and�fuses),�or�a�flywheel�storage�device�must�be�planned�that�suits�the�power�supply�environment.�

Furthermore,�redundancy�arrangements�and�the�possi-bilities�of�input�and�output�supply�are�other�important�considerations.�

A�whole�range�of�different�concepts�are�available�for�selection�for�constructing�the�UPS�system.�Smaller,�indivi-dual�UPS�devices�are�popular�for�safeguarding�the�supply�of�a�few�servers�and�IT�storage�systems.�Different�versions�are�the�UPS�cabinet�or�tower�units�with�integrated�bat-tery,�the�external�battery�pack,�and�the�rack�version�for�installation�in�a�19”�cabinet.�Larger�UPS�systems�as�single�block�or�parallel�systems,�mostly�with�external�battery�cabinets,�battery�racks�or�flywheel�accumulator�systems,�are�mostly�set�up�and�operated�in�dedicated�operating�rooms.�Here,�a�modern,�liquid-cooled�UPS�system�offers�low-cost,�efficient�and�direct�UPS�air�conditioning�without�any�special�room�air�conditioning.�Further�advantages�of�operating�rooms�dedicated�to�UPS�are�that�thick�power�cables�in�computer�rooms�are�avoided,�as�are�batteries,�which�also�represent�a�fire�risk.�Modular�UPS�systems�combine�service�friendliness�with�rapid�adaptability�to�frequently�changing�maximum�power�requirements.�However,�the�number�of�modules�used�has�to�be�con-sidered,�as�availability�decreases�the�more�complex�the�system�becomes.�When�UPS�systems�are�installed�in�server�cabinets�or�take�the�form�of�a�dedicated�UPS�rack�in�rooms�shared�with�IT�equipment,�the�additional�fire�risk�posed�by�the�rechargeable�batteries�must�be�taken�into�consideration�in�the�design�of�alarm�and�fire�safety�devices.

Page 23: Reliable Data Centres - STULZ

21

Reliable Data Centres

Dat

a ce

ntre

ca

tego

ry UPS

Permitted

data centre dow

ntime*

Servers cabinet Servers cabinet Data centre/ server room

up to 5 kW from 5 kW to 30 kW 500 to 2500 W/m2A Standard,��

min.�backup�time�10�minutes,�maximum�dura-tion�dependent�on�controlled�shutdown�time�

of�servers

Standard,��min.�backup�time�1�h,�maximum�duration�

dependent�on�controlled�shutdown�time�of�servers

72�h

B Standard,��min.�backup�time�(incl.�ventilation)�10�minutes,�maximum�duration�dependent�on�controlled�

shutdown�time�of�servers

Standard,��min.�backup�time�1�h,�maximum�duration�

dependent�on�controlled�shutdown�time�of�servers

24�h

C Redundant�(N�+�1),��backup�time�10�–�20�min

1�h

D Redundant�(N�+�1),��backup�time�10�–�20�min

10�min

E Redundant,��backup�time�10�–�30�min

0�min

Table�5:�From�the�BITKOM�matrix�“Planning�Guide�for�a�Reliable�Data�Centre”�–�UPS

Ventilation�units,�chilled�water�pumps�and�cooling�units/compressors�may�need�to�be�supplied�via�a�UPS�system,�depending�on�the�energy�density�and�selected�backup�time.�The�amount�of�energy�required�for�cooling�can�also�be�made�available�via�a�storage�unit,�instead�of�cooling�units/compressors.�No�cooling�at�high�power�densities�results�in�overheating,�causing�IT�devices�to�shut�down�without�use�being�made�of�the�established�backup�time�for�any�planned�shutdown.

5.3.8� Special�features

Important�planning�features�for�dimensioning�and�instal-ling�a�UPS�system�are:

�� Output�power�rating�at�the�required�load�power�factor�(these�days�min.�0.95)

�� Connection�values�such�as�input/output�voltage�and�frequency

�� Current,�wire�cross�sections�and�possibilities�for�con-necting�UPS�inputs�and�outputs

�� Efficiency�and�power�loss�for�the�various�load�ratios�during�typical�operating�cycles�(e.g.�day/night,�week-day/weekend),�examination�of�energy�efficiency

�� Details�on�UPS�fuse�protection�for�the�various�opera-ting�modes

�� Effects�on�the�mains�input�and�input�power�factor.�However,�the�effects�of�the�connected�load�when�the�UPS�is�in�bypass�mode�must�also�be�taken�into�consideration

�� Available�backup�time�of�a�battery�system�or�flywheel�accumulator�at�actual�load

�� Maximum�available�backup�time�of�a�battery�system�or�flywheel�accumulator�at�actual�load

�� Information�on�the�energy�storage�unit�and�charge/discharge�behaviour

�� Permitted�ambient�parameters,�such�as�operating�temperature�and�humidity;�implemented�degree�of�protection,�fire�safety�and�air-conditioning�requirements

�� Noise�� Electromagnetic�compatibility�(EMC)�� Dimensions�and�weights

Page 24: Reliable Data Centres - STULZ

22

This�guide�cannot�offer�a�precise�analysis�of�these�indi-vidual�features,�because�the�circumstances�of�the�data�centre�power�supply�will�always�require�detailed�planning.�Some�interdependencies�to�be�considered�are�mentioned�here�as�examples:�

�� The�importance�of�the�connected�battery/flywheel�accumulator�for�the�backup�time�on�power�failure,�if�an�emergency�generator�is�available.

�� Consideration�of�the�input�power�factor�when�dimen-sioning�an�emergency�generator.�Operation�via�UPS�power�electronics�and�operation�via�the�bypass�must�also�be�taken�into�account�here.

�� The�influence�of�the�UPS�output�power�factor�on�the�possibility�of�supplying�modern�switching�power�supply�units�with�electricity,�even�at�full�load

�� The�limitation�of�power�during�operation�at�high�altitudes.

�� The�importance�of�considering�efficiency�over�a�typical�operating�cycle�(fluctuations�in�capacity�utilisation),�in�order�to�obtain�realistic�estimates�of�running�costs

The�price�of�a�UPS�depends�on�equipment�details�such�as�filters,�transformers,�fans,�electronic�bypass,�integra-ted�or�external�manual�bypass�and�different�switching�concepts.�Calculating�the�price�of�best-practice�solutions�is�extremely�complex�for�UPS�systems,�and�requires�time-consuming�analysis�of�the�circumstances,�boundary�con-ditions�and�dependencies�and�consideration�of�numerous�individual�parameters.�

�� 5.4� Backup�power

5.3.1� Generating�sets�for�supplying�backup�power�(emergency�power)�in�the�event�of�power�failure

Electricity�suppliers�cannot�guarantee�an�undisturbed�power�supply�at�all�times�and�all�places,�and�power�supply�companies�(PSCs)�always�waive�any�liability�in�their�stan-dard�contracts.�Therefore,�brief�interruptions�or�sustained�power�failures�must�be�bridged�by�backup�power�systems,�

in�order�to�maintain�the�operation�of�a�data�centre�and�its�technical�systems,�such�as�air�conditioning,�electricity�and�safety.�

The�amount�of�permitted�downtime�has�maximum�prio-rity�when�planning�emergency�power�systems.�Standby�generators�are�divided�into�various�groups,�accordingly:

�� Generators�without�a�required�time�for�taking�on�the�load.�Systems�are�put�into�operation�manually.�These�systems�are�unsuitable�for�automatic�operation�in�data�centres.

�� Generators�with�a�required�time�for�taking�on�the�load.�Here,�an�interruption�must�be�less�than�15�seconds�long�before�the�generator�assumes�the�task�of�supplying�power�after�automatically�being�brought�into�operation.�A�DIN�standard�sets�out�the�require-ments�for�generating�sets�with�combustion�engines�for�safety�power�supplies�in�hospitals�and�in�buildings�and�premises�intended�for�gatherings�of�people.�This�standard�should�also�be�regarded�as�a�minimum�requirement�for�generating�sets�in�the�data�centre�sector.

�� Generators�with�auto-reclosing�in�the�form�of�standby�power�supply�units.�Here,�the�interruption�time�must�be�less�than�one�second.�These�systems�are�no�longer�used�in�data�centres,�as�an�interruption�time�of�less�than�one�second�is�not�necessary.

�� Generators�for�uninterruptible�power�supply�in�the�form�of�diesel�UPS�systems.�Here,�on�power�failure�the�load�is�taken�on�without�interruption.

5.4.2� Emergency�power�supplies

In�the�two�last�cases,�special�versions�of�generating�sets�are�required,�which�are�standby�units�with�an�energy�storage�unit.�The�latter�must�be�continuously�fed.�The�resulting�operating�costs�mean�that�the�consumer�pays�for�his�more�reliable�supply.�

Various�versions�of�standby�units�exist,�as�combinations�made�up�of�diesel�engine,�flywheel,�electric�machine�and�the�necessary�couplings.�

Page 25: Reliable Data Centres - STULZ

23

Reliable Data Centres

Standby�power�supply�units�are�required�whenever�an�interruption�time,�such�as�that�caused�by�the�use�of�simple�emergency�generators,�would�not�be�a�feasible�option�for�reliably�continuing�ongoing�operation�at�the�consumer’s�business.�

The�most�commonly�used�systems�in�the�data�centre�are�those�mentioned�second.�The�information�below�refers�to�these�systems

5.4.3� Designing�the�emergency�power�system

The�factors�below�are�decisive�when�designing�the�output�power�of�the�generator:

�� Sum�total�of�connected�loads��� Demand�factor��� Starting�currents�and�starting�cos�phi�of�loads��� Circuit�feedback�of�loads�(rectifier�technology�of�UPS�

systems�or�frequency�converters)�� Permitted�dynamic�behaviour��� Reserve�for�expansions�� Supplement�for�different�ambient�conditions

Consumer power

When�calculating�the�consumer�power,�bear�in�mind�that�the�apparent�power�and�real�power�have�to�be�stated.

Demand factor

In�data�centres,�the�generator�output�power�must�be�set�up�with�a�demand�factor�of�1,�as�operation�of�the�data�centre�must�be�maintained�in�both�summer�and�winter.�

Switch-on behaviour

The�starting�and�switch-on�behaviour�of�electric�motors,�transformers�and�large�lighting�systems�with�bulbs�has�an�influence�on�generator�output�power.�Where�asyn-chronous�motors�are�used,�the�apparent�power�can�reach�6�times,�the�real�power�2�–�3�times�nominal�power.�The�required�generator�output�power�can�be�considerably�reduced�by�phased�switch-on.�All�available�measures�to�limit�starting�power�should�be�used�to�the�full.

Dynamic behaviour

The�dynamic�behaviour�of�the�generator�when�the�full�load�is�switched�on�and�during�expected�load�changes�during�operation�must�be�adapted�to�the�permitted�values�of�the�consumers.�

The�motor,�generator�or�both�may�have�to�be�overdimen-sioned�in�order�to�satisfy�the�required�values.

Ambient conditions

According�to�DIN�6271,�the�motor�reference�temperature�is�27°�C.�If�operating�temperatures�exceed�this,�the�motor�must�be�of�larger�dimensions.�The�motors’�reduction�factors�must�be�established.

Page 26: Reliable Data Centres - STULZ

24

5.4.4�Recommended�emergency�power�supply�as�a�function�of�the�permitted�downtimes:

Dat

a ce

ntre

ca

tego

ry Emergency power

Permitted

data centre dow

ntime*

Servers cabinet Servers cabinet Data centre/ server room

up to 5 kW from 5 kW to 30 kW 500 to 2500 W/m2A Optional 72�h

B Optional 24�h

C Redundant,��availability�in�15�s,�fuel�reserve:�24�hours

1�h

D Redundant,��availability�in�15�s,�fuel�reserve:�72�hours

10�min

E Redundant,��availability�in�15�s,�fuel�reserve:�72�hours

0�min

Table�6:�From�the�BITKOM�matrix�“Planning�Guide�for�a�Reliable�Data�Centre”�–�Emergency�Power

It�is�possible�to�hire�generating�sets�from�your�power�supply�company,�which�guarantee�the�emergency�power�supply�during�maintenance�and�repairs�via�an�external�connection.�But�hired�generators�are�no�answer�to�unfore-seen�power�outages,�as�you�cannot�be�certain�whether�or�not�hired�generators�are�actually�available�at�the�time�of�occurrence.

Room planning/detailed planning for emergency generators

The�following�details�must�be�considered�during�room�planning/detailed�planning:

�� Compliance�with�the�following�regulations�(DIN�VDE,�VDS,�WHG,�TA�Noise,�TA�Air,�VAws,�TRbF,�VDN…)

�� Basic�construction/version�of�the�generator�(statio-nary�installation,�container�or�hood-type�generator)

�� Design�of�the�tank�system�(day�tank�and�storage�tank)��� Design�of�the�exhaust�system�� Engine�cooling�(front-mounted�radiator,�table�cooler�

and�the�use�of�heat�exchangers)�� Backup�power�control/switching�systems�� Immission�control�

Basic room requirements

The�room�in�which�an�emergency�generator�is�erected�is�an�electrotechnical�operating�room.�It�must�be�protected�to�F90�quality,�and�is�a�fire�zone�in�its�own�right.�Venti-lation�apertures�must�be�provided�to�supply�the�cooling�and�combustion�air,�and�for�removing�heated�cooling�air.�These�apertures�must�lead�directly�to�the�outside.�Rooms�without�exterior�walls�are�unsuitable�because�of�the�cross�sections�required�for�ventilation.�If�necessary,�ventilation�ducts�in�F90�quality�must�be�built,�which�lead�directly�to�the�outside.�In�order�to�avoid�short�circuits�of�air,�supply�and�extraction�apertures�must�not�be�situated�immedia-tely�next�to�one�another.�The�generator�room�must�take�the�form�of�a�catch�basin,�with�a�circumferential�barrier�of�10�cm�with�3�coats�of�oil-resistant�paint,�for�protection�against�flooding�and�for�environmental�protection.�This�basin�must�be�monitored�for�leaks.�The�size�of�the�room�must�allow�an�escape�route�1m�wide.�The�doors�of�the�room�must�be�of�at�least�T30�quality�with�a�panic�lock.�

Page 27: Reliable Data Centres - STULZ

25

Reliable Data Centres

Applicable regulations

The�regulations�and�laws�listed�here�ensure�the�cor-rect�function�of�the�system,�operational�reliability�and�protection�of�the�environment.�Approval�authorities�may�impose�further�restrictions�and�requirements.�In�all�cases,�dialogue�with�the�authorities�should�be�initiated�at�an�early�stage�of�the�planning�phase.

Noise�protection�is�an�especially�important�consideration.�Below�is�a�list�of�continuous�emission�reference�values�for�emissions�sites�outside�buildings.

Industrial�estate 70�dB(A)

Business�park daytime��65�dB(A)

night-time��50�dB(A)

Core�regions,��villages�and�mixed�regions

daytime��60�dB(A)

night-time��45�dB(A)

Residential�areas�and�small�housing�estates

daytime��55�dB(A)

night-time��40�dB(A)

Purely�residential�areas daytime��50�dB(A)

night-time��35�dB(A)

Spa/rehabilitation�areas�for�hospitals/care�homes

daytime��45�dB(A)

night-time��35�dB(A)

The�residual�noise�level�is�assessed�at�an�appropriate�distance,�not�at�the�site�of�emission.

Basic generator construction/version

There�are�three�possible�generator�constructions/ver-sions.�With�a�built-in�generator,�the�entire�system�is�installed�in�the�building.�Interfaces�to�the�outside�are�the�air�supply�and�extraction�apertures,�the�exhaust�system�and,�if�necessary,�an�external�table�cooler.�In�this�version,�power�can�range�from�a�few�kVA�to�many�MVA.�Container�generators�are�frequently�used�when�insufficient�space�is�

available�in�the�building,�or�installation�in�the�building�is�unsuitable�for�other�reasons.�As�with�a�stationary�built-in�generator,�power�can�range�from�a�few�kVA�to�many�MVA.�The�third�type�is�the�hood-type�generator.�This�is�mostly�employed�when�power�of�a�few�kVA�to�several�hundred�kVA�is�required.�Its�advantage�lies�in�its�space-saving�design.�A�disadvantage�is�the�difficulty�of�access�to�all�system�components�in�the�event�of�service�or�repair.�The�diagrams�below�show�standby�generators�in�a�building�and�in�a�container.

Figure�3:�Standby�generator�in�a�building

Machine unit

Exhaust gas system

Batteries

Switchgear

Supply air section

Dischargeair section

Page 28: Reliable Data Centres - STULZ

26

Figure�4:�Standby�generator�in�a�container

Design of the tank system

The�key�factor�that�determines�tank�size�is�the�necessary�operating�time�and�the�system�output�power.�A�quan-tity�of�fuel�of�less�than�5000�litres�can�be�stored�in�the�generator�room.�If�more�than�5000�litres�are�required,�a�separate�storage�room�in�F90�quality,�a�tank�for�above-ground�storage�outside�the�building�or�an�underground�tank�must�be�provided.�The�day�tank�is�a�single-walled�tank�with�catch�basin.�It�must�be�fitted�in�such�a�way�that�static�pressure�is�applied�to�the�engine’s�injection�system.�The�storage�tank�must�have�double�walls,�or�the�storage�room�must�be�designed�as�a�catch�basin�for�the�

entire�tank�contents.�If�fuel�lines�are�in�place�between�the�day�tank�and�the�storage�tank,�the�entire�length�of�which�cannot�be�seen,�they�must�have�double�walls.�These�double-walled�pipes,�the�catch�basins�and�the�jacket�of�double-walled�tanks�must�be�monitored�for�leaks.�The�components�of�the�tank�system�are�described�below:

Design of the exhaust system

The�nominal�width�of�the�exhaust�system�is�based�on�the�nominal�power�of�the�emergency�generator,�the�planned�pipe�length,�the�number�and�type�of�changes�in�direction�and�the�required�sound�absorption.�Exhaust�systems�of�emergency�generators�are�pressure�systems,�and�reach�temperatures�of�up�to�500�°C.�They�must�be�restricted�in�such�a�way�as�to�exclude�any�danger�to�people�and�valua-bles.�The�exhaust�system�is�designed�as�described�below:

Design of the engine cooling

Engine�cooling�using�a�front-mounted�radiator�is�possible�up�to�an�output�range�of�approx.�1150�kVA.�This�means�that�all�the�cooling�air�has�to�be�conveyed�through�the�gene-rator�room.�At�an�output�of�approx.�800�kVA�and�above,�part�of�the�heat�from�the�engine�can�be�dissipated�via�a�table�cooler.�This�reduces�the�amount�of�cooling�air�that�has�to�be�conveyed�through�the�generator�room.�If�there�is�a�difference�in�height�between�the�diesel�engine�and�the�table�cooler�in�excess�of�10�m,�a�heat�exchanger�must�be�used�to�reduce�the�pressure�on�the�engine’s�cooling�circuit.�The�design�of�the�engine�cooling�and�external�cooling�sys-tems�based�on�the�difference�in�height�between�the�diesel�engine�and�the�cooling�circuit�is�described�below:

Design of backup power control/switching systems

�� Each�generating�unit�must�at�least�be�equipped�with�a�backup�power�controller,�which�assumes�the�following�tasks:�

�� Monitoring�the�mains�power�for�compliance�with�the�permitted�tolerances

�� Communication�with�the�engine�management�system

�� Starting�and�stopping�the�diesel�engine

Page 29: Reliable Data Centres - STULZ

27

Reliable Data Centres

�� Monitoring�the�generator�network�for�compliance�with�the�permitted�tolerances

�� Monitoring�the�engine�parameters�and�controlling�the�necessary�parameters

�� Managing�and�controlling�the�required�auxiliary�drives�(motor-driven�ventilation�flaps,�supply�and�extractor�fans,�fuel�pumps,�solenoid�valves,�leakage�sensors,�pipe�heaters,�coolant�preheating,�starter�bat-tery�charging,�control�battery�charging,�etc.

�� Managing�the�necessary�mains�and�generator�tie�switches�for�automatic�mode

�� Charging�and�monitoring�the�battery�

The�following�possibilities�exist�for�the�power�unit:�� The�mains�and�generator�switches�are�located�in�the�

backup�power�controller�� The�mains�switch�is�located�in�the�low-voltage�main�

distributor,�the�generator�switch�in�the�backup�power�controller.

�� The�mains�and�generator�switches�are�located�in�the�low-voltage�main�distributor,�the�generator�power�is�monitored�by�means�of�external�voltage�taps,�and�the�generator�is�protected�by�a�star-point�current�transformer.

Here�is�an�example�power�supply�diagram:

Figure�5:�Power�system�monitoring/switchover

�� 5.5� Service/maintenance

5.5.1� Service/maintenance�of�UPS�systems

Annual�maintenance�in�accordance�with�the�manufacturer’s�instructions�by�specialist�personnel�authorised�by�the�manufacturer�is�a�prerequisite�for�maintaining�proper�function.�Wearing�parts�must�be�replaced�before�their�service�life�expires,�in�accordance�with�the�manufacturer’s�instructions.�

Less�emphasis�is�placed�on�maintenance�of�commonly�used�sealed�lead�batteries,�as�these�are�maintenance-free.�The�term�“maintenance-free”�refers�to�the�interior�of�the�battery,�however,�and�means�that�distilled�water�does�not�have�to�be�topped�up.�On�the�other�hand,�all�connections�and�battery�connection�screws�have�to�be�checked�to�ensure�they�have�the�correct�torque.�The�voltages�of�the�individual�batteries�must�be�recorded�and�logged�in�the�charge�holding�and�discharging�phases.�The�condition�of�the�battery�can�only�be�evaluated�on�the�basis�of�this�data.�Just�as�important�is�regular�cleaning�of�the�battery�system,�to�prevent�leakage�current�or�short�circuits.

The�availability�of�appropriate�specialist�personnel�to�remedy�problems�is�another�safety�aspect�that�should�not�be�ignored.

5.5.2� Service/maintenance/test�runs�of�the�emergency�generator

Annual�maintenance�in�accordance�with�the�manufacturer’s�instructions�by�specialist�personnel�authorised�by�the�manufacturer,�and�monthly�test�runs,�are�prerequisites�for�maintaining�proper�function.�To�ensure�correct�function,�these�monthly�test�runs�must�last�at�least�one�hour�and�operate�at�50�%�of�rated�load.�They�may�be�performed�by�the�plant�operator�himself�if�he�has�received�adequate�instruction�on�the�system.�During�a�test�run,�the�system�must�reach�its�operating�tempe-rature.�A�fixed,�installed�resistor,�the�consumers�to�be�supplied�by�backup�power,�or�the�existing�power�system,�

G

Load60% 40%

U <

A E

Switchgear room Generator room

Generator-switch

Mainsswitch

Page 30: Reliable Data Centres - STULZ

28

via�mains�parallel�operation,�may�be�employed�as�a�load.�The�latter�requires�agreement�and�acceptance�testing�on�the�part�of�the�power�supply�company,�however.�

As�with�the�UPS�system,�the�availability�of�appropriate�specialist�personnel�to�remedy�problems�must�also�be�considered.

5.5.3� Maintenance/testing�of�the�electrical�installation

Electrical�systems�must�be�inspected�and�maintained�at�regular�intervals�in�accordance�with�the�applicable�regu-lations�(VDE�0105)�and�stipulations�of�the�professional�association.�To�this�aim,�the�systems�must�be�off-load,�and�appropriate�repeat�measurements�and�tests�conducted.�If�necessary,�an�A/B�supply�must�already�be�incorporated�in�planning�of�the�infrastructure.�This�enables�the�necessary�isolation�and�testing�to�take�place.

�� 5.6� Air�conditioning

5.6.1� Current�situation

Each�kilowatt�(kW)�of�electrical�power�that�is�consumed�by�ICT�devices�is�released�as�heat.�This�heat�must�be�conducted�out�the�device,�the�cabinet�and�the�room,�in�order�to�keep�the�operating�temperature�constant.�Air-conditioning�systems�with�different�methods�of�operation�and�performance�ratings�are�employed�to�dissipate�the�heat.

5.6.2� The�challenge�of�air�conditioning

The�air�conditioning�of�ICT�systems�is�a�cru-cial�factor�in�their�availability�and�security.�The�increasing�integration�and�packing�density�of�proces-sors�and�computer/server�systems�produces�quantities�

of�heat�which,�just�a�few�years�ago,�would�have�been�unimaginable�in�such�a�restricted�space.�

A�range�of�air-conditioning�systems�is�available�on�the�market,�to�suit�the�output�and�power�loss�–�i.e.�waste�heat�of�the�ICT�components.�With�more�than�130�W/cm²�per�CPU�–�equivalent�to�two�standard�light�bulbs�per�square�metre�–�the�huge�task�of�data�centre�air�conditioning�is�becoming�clear:�today,�this�power�density�results�in�heat�loads�far�in�excess�of�1kW�per�square�metre.

The�air�conditioning�of�data�centres�gives�rise�to�further�challenges.�Measurements�and�experience�from�the�field�show�that�power�losses�of�up�to�8�kW�in�a�rack�or�housing�can�still�be�tackled�with�traditional�air�conditioning�using�cooling�air�in�the�raised�floor,�still�commonly�used�in�many�data�centres.�However,�air�conduction�through�the�raised�floor�in�the�classic�mainframe�data�centre�cannot�always�keep�pace�with�the�sometimes�high�demands�that�prevail�today.�

After�decades�during�which�a�cooling�capacity�of�1�to�3�kW�per�19”�cabinet�was�fully�adequate,�today�it�has�to�be�possible�to�considerably�increase�the�cooling�capacity�per�rack.�Modern�IT�devices�in�a�19”�cabinet�with�42�height�units�can�consume�over�30�kW�of�electricity�and�emit�over�30�kW�of�heat.�A�further�increase�can�be�anticipated�due�to�demands�for�ever�increasing�performance�in�a�smaller�installation�space.�

Figure�6:�Basic�sketch�of�a�cold�aisle/hot�aisle�solution

Page 31: Reliable Data Centres - STULZ

29

Reliable Data Centres

In�order�to�improve�the�performance�of�existing�air-conditioning�solutions�with�a�raised�floor,�today�ICT�components�are�equipped�with�a�partition�and�arranged�according�to�the�so-called�cold�aisle/hot�aisle�principle.�Sections�of�the�cold�or�hot�aisles�are�enclosed,�to�enable�more�heat�per�rack�to�be�emitted.

Some�of�the�key�factors�influencing�the�choice�of�an�air-conditioning�solution�are�the�expected�maximum�power�loss,�the�running�costs,�purchase�cost,�installation�conditions,�costs�of�expansion,�future�proofness,�cost�of�downtimes�and�physical�safety.

5.6.3� Method�of�operation�of�closed-circuit�air�conditioning

In�the�past,�the�room�conditions�for�data�centre�air�con-ditioning�were�a�room�temperature�of�approx.�22�°C�and�roughly�50�%�relative�humidity�(r.h.).�Today,�due�to�the�prevalence�of�the�cold�aisle/hot�aisle�principle,�we�tend�to�refer�instead�to�supply�air�and�exhaust�air�conditions,�as�the�room�conditions�as�such�no�longer�apply�to�all�areas�of�the�room�in�question.�

Depending�on�the�application,�the�supply�air�conditions�in�the�cold�aisle�should�lie�between�20�and�up�to�27�°C�–�the�relative�humidity�of�the�supply�air�between�40�and�60�%�r.h.�Lower�humidity�would�result�in�electrostatic�charging,�high�humidity�to�corrosion�of�electrical�and�electronic�components.�It�is�a�basic�prerequisite�that�operating�conditions�with�very�cold�temperatures�below�18�°C�and�a�high�humidity�that�results�in�the�formation�of�condensa-tion�on�IT�devices�must�be�avoided.�

Air-conditioning�systems�in�ICT�rooms�operate�with�a�very�high�proportion�of�circulating�air.�In�the�closed-circuit�air-conditioning�system,�supply�air�cooled�by�the�air-conditio-ning�system�circulates�to�the�ICT�components�and�absorbs�the�heat.�This�heated�air�returns�to�the�air-conditioning�system�for�renewed�cooling.�Only�a�small�proportion�of�ambient�air�is�brought�into�the�room,�and�this�is�used�for�air�exchange.�

Optimum�conditions�in�terms�of�temperature�and�relative�humidity�can�only�be�achieved�with�closed-circuit�air-con-ditioning�units�–�so-called�precision�air-conditioning�units.�In�these�systems�the�energy�used�is�put�to�better�use,�i.e.�lowering�the�temperature�of�the�return�air�is�of�primary�importance.�In�contrast�to�these,�comfort�air-conditioning�units�for�homes�and�offices�exist,�such�as�split�air-condi-tioning�units,�for�example,�which�permanently�use�a�large�proportion�of�the�energy�to�dehumidify�the�circulating�air.�This�gives�rise�to�critical�room�conditions,�but�also�to�considerably�higher�running�costs,�which�is�why�the�use�of�these�systems�in�data�centres�is�uneconomical.

5.6.4�Design�of�closed-circuit�air-conditioning�systems

Closed-circuit�air-conditioning�systems�differ�considerably�in�their�design.�Very�roughly,�they�can�be�classed�as�DX�systems�(Direct�Expansion�systems)�and�CW�systems�(Chilled�Water�systems).�Both�systems�can�additionally�be�equipped�with�Indirect�Free�Cooling.

DX systems

Cold�air�is�generated�in�the�closed-circuit�air-conditioning�cabinet,�and�as�such�is�situated�in�the�vicinity�of�the�ICT�room.�The�circulating�air�is�cooled�directly�by�expanding�refrigerant,�and�the�absorbed�heat�is�conveyed�to�an�out-door�unit,�the�air-cooled�condenser.�

When�Indirect�Free�Cooling�is�used,�an�additional�heat�exchanger�for�Free�Cooling�is�installed�in�the�closed-circuit�air-conditioning�cabinet.�At�low�ambient�temperatu-res,�only�a�water/glycol�mixture�circulates�between�the�closed-circuit�air-conditioning�unit�and�the�outdoor�unit,�or�dry�cooler.�This�mixture�is�heated�by�the�circulating�air�and�cooled�again�outside�by�the�dry�cooler.�This�system�is�known�as�Indirect�Free�Cooling�because�the�water/glycol�mixture�is�interconnected�between�the�circulating�air�cir-cuit�in�the�ICT�room�and�the�ambient�air.�When�ambient�temperatures�are�high,�the�system�switches�to�DX�mode,�and�the�heat�of�the�circulating�air�is�first�transferred�to�a�liquid-cooled�condenser�in�the�air-conditioning�cabinet,�

Page 32: Reliable Data Centres - STULZ

30

before�the�heat�is�emitted�into�the�ambient�air�by�the�dry�cooler.

DX�systems�with�or�without�Indirect�Free�Cooling�are�employed�for�small�to�medium-sized�installations�with�a�heat�load�up�to�approx.�500�kW.

CW systems

Here,�cold�is�generated�in�chillers,�which�are�generally�installed�outdoors.�A�water/glycol�mixture�circulates�in�the�building.�The�heat�from�the�return�air�is�transferred�to�the�cold�water/glycol�mixture�in�the�closed-circuit�air-conditioning�cabinet,�which�is�cooled�by�chilled�water.�The�heated�water/glycol�mixture�is�cooled�once�more�in�the�chiller,�and�returns�to�the�closed-circuit�air-conditioning�cabinet.

In�this�system,�too,�Indirect�Free�Cooling�is�possible.�To�this�aim,�an�additional�free�cooling�coil�must�be�installed�outside�on�the�chiller.�At�low�ambient�temperatures,�the�water/glycol�mixture�circulates�between�the�chilled�water-cooled�air-conditioning�units�and�the�free�cooling�coil.�During�this�process,�the�heat�of�the�circulating�air�is�absorbed�in�the�air-conditioning�unit�and�emitted�outside�by�the�free�cooling�coil.�At�high�ambient�temperatures,�the�water/glycol�mixture�is�cooled�by�the�chiller.

CW�systems�with�or�without�Indirect�Free�Cooling�are�employed�for�larger�installations�with�a�heat�load�of�approx.�500kW�or�more.

5.6.5� Rack�arrangement�and�circulating�air�conduction

The�rack�arrangement�and�air�conduction�are�further�factors�influencing�the�performance�of�closed-circuit�air-conditioning�systems.�

These�days,�19”�racks,�in�particular,�are�arranged�according�to�the�so-called�hot�aisle/cold�aisle�principle,�in�order�to�permit�as�well�as�possible�the�required�horizontal�flow�of�air�around�the�ICT�components.�In�such�an�arrangement,�

the�flow�of�air�is�more�or�less�forced�to�absorb�heat�from�the�ICT�components�on�its�way�from�the�raised�floor�back�to�the�air-conditioning�unit.�

Performance�is�increased�still�further�by�enclosing�the�cold�aisle�or�the�hot�aisle.�The�height�of�the�raised�floor,�via�which�the�data�centre�can�be�supplied�with�cold�sup-ply�air,�is�also�an�extremely�important�factor.�The�supply�air�is�guided�into�the�cold�aisle�through�louvered�panels�or�grilles.�After�it�has�been�heated�by�the�ICT�equipment,�the�return�air�is�conveyed�back�to�the�air-conditioning�system�for�renewed�cooling.

5.6.6�Direct�cooling�of�racks�

When�heat�loads�exceed�10�–�15kW�per�rack,�direct�cooling�of�the�racks�is�necessary.�Direct�rack�cooling�is�achieved�by�heat�exchangers�in�the�immediate�vicinity�of�the�servers.�As�a�rule,�these�are�liquid-cooled,�and�are�situated�either�below�or�next�to�the�19”�racks.�Up�to�40kW�per�rack�can�be�dissipated�with�this�method.�

Figure�7:�Basic�sketch�of�a�liquid-cooled�server�rack

Page 33: Reliable Data Centres - STULZ

31

Reliable Data Centres

A�chilled�water�infrastructure�must�be�established�around�the�racks�for�this�purpose.�Liquid-cooled�racks�ensure�the�right�climatic�conditions�for�each�server�cabinet,�and�are�therefore�autonomous�in�terms�of�room�air�conditioning.

In�existing�buildings�with�a�low�ceiling�height,�liquid-cooled�server�racks�represent�a�good�method�of�reliably�dissipating�high�heat�loads�without�the�use�of�a�raised�floor.

5.6.7� Recommended�equipment�for�different�downtimes

Dat

a ce

ntre

ca

tego

ry Air conditioning

Permitted

data centre dow

ntime*

Servers cabinet Servers cabinet Data centre/ server room

up to 5 kW from 5 kW to 30 kW 500 to 2500 W/m2A

Precision�coolingHigh-performance�cooling��

or�liquid�coolingComplete�cold/hot�separation

Precision�cooling72�h

B

Precision�cooling

High-performance�or�liquid�coo-ling,�on-call�service,�redundancy�required�for�cabinets�with�high�

power�density.��UPS�backup�for�ventilation,�

Complete�cold/hot�separation

Precision�cooling,��on-call�service

24�h

CPrecision�cooling�in�redundant�

arrangement

High-performance�or�liquid�coo-ling,�Redundant�arrangement,��

UPS�backup�for�ventilation,�Complete�cold/hot�separation

Precision�cooling�in�redundant�arrangement

1�h

DPrecision�cooling�in�redundant�

arrangement,�UPS�backup

High-performance�or�liquid�coo-ling,�Redundant�arrangement,��

UPS�backup�for�ventilation,�Complete�cold/hot�separation

Precision�cooling�in�redundant�arrangement,��

UPS�backup�for�ventilation

10�min

E

Precision�cooling�in�redundant�arrangement,�UPS�backup

High-performance�or�liquid�coo-ling,�Redundant�arrangement,��

UPS�backup�for�ventilation,�Complete�cold/hot�separation

Precision�cooling�in�redundant�arrangement,��

UPS�backup�for�ventilation,�Emergency�cooling�func-

tions�via�an�additional�air-conditioning�system�(e.g.�well�water,�mains�water,�ventilation�

system)

0�min

Table�7:�From�the�BITKOM�matrix�“Planning�Guide�for�a�Reliable�Data�Centre”�–�Air�Conditioning

Page 34: Reliable Data Centres - STULZ

32

Figure�8:�Air-conditioning�solutions�for�different�cooling�loads

The�high-performance�cooling�required�at�higher�power�outputs�also�incorporates�the�need�for�cold�or�hot�aisle�enclosures.

As�the�cooling�capacity�increases,�the�air-conditioning�systems�must�be�adapted�accordingly.�The�diagram�below�shows�the�power�densities�that�can�currently�be�handled�by�the�different�air-conditioning�systems.

Air�conditioning�via�the�raised�floor,�racks�not�arranged��to�ensure�best�possible�ventilation

Air�conditioning�liquid-cooled��with�enclosed�warm�aisles

Air�conditioning�via�the�raised�floor,�racks�arranged�in�cold/hot�aisles

Air�conditioning�liquid-cooled�with�enclosed�cold�aisles

Air�conditioning�via�the�raised�floor,��with�enclosed�cold�aisles

Air�conditioning�with�liquid-cooled�racks��(closed�system)

Page 35: Reliable Data Centres - STULZ

33

Reliable Data Centres

5.6.8�Further�recommendations

Energy efficiency

As�energy�costs�continue�to�rise,�the�question�of�energy�efficiency�is�of�particular�importance�during�the�planning�phase�of�an�air-conditioning�system.�Here,�the�overall�costs�–�i.e.�capital�investment�in�the�new�system�and�anti-cipated�running�costs�throughout�the�running�time,�plus�maintenance�costs�–�must�be�calculated�and�evaluated.�A�system�with�free�cooling�generally�entails�higher�capital�investment,�but�the�additional�expense�is�offset�over�a�short�to�medium�period�of�time�due�to�the�considerably�reduced�running�and�maintenance�costs.

Scalability

In�many�data�centres,�ICT�systems�reach�their�maximum�final�stage�only�after�a�few�years.�Air-conditioning�sys-tems�must�therefore�be�scalable�to�reflect�this.�Conse-quently,�a�chilled�water�system�operating�at�partial�load�may�run�with�poor�efficiency�for�many�years,�until�the�planned�efficiency�is�at�least�reached�in�the�final�stage.�Here,�solutions�that�grow�to�match�requirements,�i.e.�modular�systems,�are�always�advantageous.

Redundancy

A�failure�probability�must�always�be�taken�into�account,�due�to�the�numerous�mechanical�components�in�the�air-conditioning�system.�Redundant�air-conditioning�units�assume�the�task�of�generating�the�required�cooling�power�in�the�event�of�failure,�and�offer�almost�100%�operational�reliability.�During�failure,�redundancy�in�the�air-conditioning�system�is�generally�no�longer�available,�and�corrective�action�must�be�initiated�without�delay.�–�otherwise,�there�is�a�risk�that�it�will�no�longer�be�possible�to�air�condition�the�room�in�the�event�of�further�failure.

5.6.9�Service�concept

In�an�air-conditioning�system,�wearing�parts,�e.g.�filter�mats�and�steam�cylinders,�are�used,�but�so�are�numerous�mechanically�moved�components.�Therefore,�preventive�maintenance�at�regular�intervals�is�a�must.�The�tasks�to�be�undertaken�are�described�in�DIN31051�and�VDMA�24186,�among�others.�

Suitable�types�of�service�contracts�are�available,�depen-ding�on�individual�requirements�for�air-conditioning�availability.�These�contracts�differ�in�terms�of�the�services�offered:

Repair�contract� Comes�into�force�after�a�failure�or�fault.�System�operability�is�restored�by�downstream�correc-tive�service�measures

Service�contract� Regular�work�that�ensures�system�availability�through�pre-ventive�service�measures.

Maintenance�contract�

Combination�of�repair�and�ser-vice,�which�combines�preventive�and�corrective�service�work

Full�maintenance�contract�

Complete�maintenance,�with�constant�costs�throughout�the�term�of�the�contract�offering�budget�security

These�contracts�can�sometimes�also�be�combined�with�a�24/7�emergency�service,�and�offer�contractually�fixed�on�site�time.�This�ensures�that�corrective�measures�are�initiated�immediately,�and�the�availability�of�the�system�is�restored�as�quickly�as�possible.

Page 36: Reliable Data Centres - STULZ

34

6 Fire safety

“Experience�shows�that�we�must�expect�the�outbreak�of�fire�at�practically�any�time.�The�fact�that�no�fires�have�occurred�in�many�buildings�for�decades�does�not�prove�that�no�risk�exists,�but�is�rather�a�piece�of�luck�for�those�concerned,�which�can�be�expected�to�come�to�an�end�at�any�time.”�Even�today,�nothing�needs�to�be�added�to�this�pronounce-ment�by�a�Higher�Administrative�Court�from�1987.�This�is�why�a�reliable,�fast�fire�detection�and�extinguishing�system�or�fire�prevention�system�is�an�indispensable�pre-requisite�for�the�reliable�operation�of�a�data�centre.

Water�as�an�extinguishing�agent�is�inappropriate�in�the�data�centre.�Today,�specialist�firms�in�the�industry�offer�suitable�extinguishing�systems�for�every�requirement.�When�constructing�a�new�data�centre�or�retrofitting�fire�safety�measures,�however,�precise�planning�and�design�of�these�systems�is�vital.

�� 6.1� Technical�fire�protection

Fire,�smoke,�aggressive�fumes�and�extinguishing�water�represent�a�latent�danger�for�data�centres.�To�ensure�safety,�a�technically�high-quality�very�early�fire�detec-tion�system�combined�with�very�high-quality�extingu-ishing�technology�is�necessary.�One�alternative�is�fire�prevention�by�lowering�the�amount�of�oxygen�in�the�air.�Here,�nitrogen�is�introduced,�which�reduces�the�oxygen�concentration�precisely�to�a�preset�value.�Despite�this,�rooms�protected�in�this�way�remain�accessible�to�persons�without�jeopardising�their�health.�

Extinguishing�foam,�on�the�other�hand,�cannot�be�used�as�it�would�damage�IT�systems�and�their�power�supply�units.�Powder�extinguishing�systems�are�also�unsuitable�for�data�centres�with�very�sensitive�equipment,�as�the�extinguishing�process�may�cause�more�damage�than�the�fire�itself.�Today,�therefore,�gas�extinguishing�agent�is�used�almost�exclusively.

6.1.1� Method�of�operation�of�the�infrastructure

Fire alarms

Smoke�alarms�that�work�on�the�principle�of�scattered�light�are�primarily�used�for�fire�detection�in�IT�rooms.�Here,�the�scatter�of�a�ray�of�light�on�any�smoke�particles�present�provides�a�measure�of�the�smoke�density.�This�technique�is�employed�both�in�conventional,�point�smoke�detectors�(optical�smoke�detectors,�fixed�point�detectors)�and�in�highly�sensitive�smoke�extraction�systems�(aspirating�detectors,�active�detectors).�In�contrast,�the�radioactive�ionisation�detector�used�years�ago�has�now�almost�com-pletely�disappeared�from�the�European�market.�

The�choice�of�the�most�suitable�fire�detector�depends�on�the�requirements�of�the�intended�site�of�use.�In�areas�without�particular�detection�requirements,�fixed�point�detectors�are�generally�used.�In�data�centres,�on�the�other�hand,�a�cushion�of�heated�air�up�to�1�metre�thick�can�form�under�the�ceiling.�This�is�produced�as�hot�air�from�air-con-ditioning�systems,�power�pack�fans�and�rack�ventilation�systems�is�driven�upwards,�and�may�prevent�smoke�from�reaching�fixed�point�detectors�in�sufficient�quantities�for�early�fire�detection.�In�areas�such�as�this,�in�rooms�with�a�lot�of�air�conditioning�and�in�buildings�with�high�ceilings,�fixed�point�detectors�quickly�come�up�against�their�limits.�We�therefore�recommend�instead�the�use�of�highly�sensi-tive�aspirating�systems.

For�the�protection�of�ventilated�IT�equipment,�the�VdS�(Verband�der�Sachversicherer�–�German�Association�of�Property�Insurers)�points�out�that�early�fire�detection�using�fixed�point�smoke�detectors�is�difficult,�to�say�the�least,�or�even�impossible.�Here,�too,�the�use�of�a�highly�sensitive�smoke�aspirating�system�is�recommended,�as�this�offers�the�possibility�of�an�“early,�localised�reaction”.�

Page 37: Reliable Data Centres - STULZ

35

Reliable Data Centres

Modern�server�racks,�which�have�their�own�cooling�system�and�are�encapsulated�from�the�environment,�represent�a�new�challenge�for�fire�safety�in�IT�rooms.�Smoke�produced�inside�the�racks�only�escapes�after�a�long�delay,�which�renders�normal�room�monitoring�with�smoke�detectors�totally�ineffectual.�Furthermore,�extinguishing�agent�released�in�the�event�of�a�fire�can�only�penetrate�the�server�cabinet�after�a�delay.�For�racks�such�as�this,�manu-facturers�offer�fire�detection�and�extinguishing�systems�for�equipment�protection�in�the�form�of�19”�plug-in�units.�One�of�these�devices�can�protect�up�to�5�racks.�For�this�application,�too,�in�which�major�dilution�of�the�smoke�can�be�expected�because�of�the�high�level�of�air�conditioning,�highly�sensitive�19”�aspirating�systems�are�now�used�to�an�ever�greater�extent.�The�latter�are�also�available�with�integrated�extinguishing�system,�external�extinguisher�activation�and�integrated�dual�detector�interdependency.�

Where�the�activation�of�automatic�extinguishing�systems�is�concerned,�false�alarms�must�be�ruled�out�as�far�as�possible.�To�this�aim,�fire�detectors�for�room�monitoring�must�be�installed�according�to�the�principle�of�dual�detec-tor�interdependency;�in�other�words,�two�detectors�must�sound�the�alarm�before�it�is�escalated.�Single�detectors�trigger�only�technical,�internal�alarms.�

For�some�years�now,�manufacturers�of�fire�detection�and�extinguishing�systems�have�been�producing�devices�that�are�capable�of�independently�monitoring�individual�racks�or�entire�cabinet�units�for�fire�aerosols,�and�extinguishing�them.�An�outbreak�of�fire�in�live�cabinets�can�be�triggered�by�scorching,�smouldering�or�glowing,�often�caused�by�overloaded�components�and�faulty�contacts.�A�fire�on�a�printed�circuit�board,�for�example,�covers�large�areas�of�hardware�with�soot�and�causes�widespread�corrosion.�A�further�difficulty�arises�through�the�fact�that�modern,�air-conditioned�racks�work�with�extremely�high�air�change�rates,�which�immediately�dilute�any�smoke,�rendering�it�virtually�undetectable�by�conventional�smoke�detectors�while�the�fire�is�in�its�initial�stages.�

As�with�very�early�fire�detection�for�IT�rooms,�here,�too,�very�early�fire�detection�systems�with�aspirating�detec-tors�offer�clear�advantages,�because�in�this�case�the�air�samples�can�be�taken�directly�in�the�cabinet�and�the�inlets�of�the�circulating�air�or�air�conditioning.�These�days,�the�extensive�modularity�of�modern�systems�means�they�are�best�equipped�for�enhancing�all�stages�of�fire�detection�and�fire�fighting.�They�are�ideally�suited�for�new-gene-ration�EDP,�server�and�control�cabinets,�which�are�sealed�with�integrated�cooling�systems.�They�take�the�form�of�19”�plug-in�units�for�highly�sensitive�fire�detection,�emit�control�signals�for�soft�shutdown�and�can�independently�extinguish�a�fire�on�an�object.�An�extinguishing�unit�can�either�be�integrated�in�the�unit�or�activated�externally.�Preferred�extinguishing�agents�are�Novec�1230�and�nitro-gen.�Nitrogen�has�an�advantage�in�that�it�distributes�itself�extremely�well�and�has�a�longer�life,�while�Novec�1230�is�very�compact�and�space-saving�to�keep�in�stock.�Devices�in�19”�design�are�available�on�the�market�that�have�been�approved�by�the�VdS�for�their�fire�detection�ability,�and�already�satisfy�the�latest�“A”�and�“B”�sensitivity�classes�of�DIN�standard�EN�54-20.

Fire extinguishing systems

The�effectiveness�and�reliability�of�a�fire�safety�system�depends�upon�planning�that�takes�account�of�all�the�risks,�dimensioning�and�evaluation�of�the�required�volume.

Gas�extinguishing�systems�are�divided�into�inert�gases�and�chemical�gases.

Inert gases:

�� Extinguishing�fires�by�oxygen�removal

The�gas�extinguishing�method�is�suitable�for�data�centres�and�their�equipment.�It�uses�the�technique�of�oxygen�removal.�Here,�the�extinguishing�agent�reduces�the�pro-portion�of�oxygen�in�the�room�air�to�such�an�extent�that�the�combustion�process�is�suppressed.�The�extinguishing�gas�used�is�an�inert�or�chemical�gas.

Page 38: Reliable Data Centres - STULZ

36

�� Carbon�dioxide�(CO2)�

Carbon�dioxide�is�present�in�the�atmosphere,�but�is�no�longer�used�in�new�systems�because�of�the�risk�it�poses�to�people.

�� Argon�(Ar)

Argon�is�an�inert�gas�and�can�be�extracted�from�the�ambi-ent�air.�Argon�itself�is�not�toxic,�but�can�lead�to�oxygen�deprivation�or�a�risk�of�fire�effluent�at�the�concentration�required�to�extinguish�a�fire.�

�� Nitrogen�(N2)

Nitrogen�is�also�contained�in�the�atmosphere.�It�is�colour-less,�odourless�and�tasteless.�Nitrogen�is�not�toxic,�but�can�pose�a�threat�due�to�fire�effluent�and�oxygen�deprivation.

�� Extinguishing�gas�FM-200�(HFC�227ea),�Novec�1230

These�extinguishing�gases�take�effect�by�absorbing�the�heat�in�the�flame.�A�physical�component�and,�to�a�lesser�extent,�a�chemical�component,�are�involved�in�this�process.�These�gases�are�approved�for�storage�in�extin-guishing�agent�containers�in�a�dedicated�fire�extingu-ishing�zone,�under�certain�preconditions,�which�may�be�a�decisive�factor�when�retrofitting�an�existing�data�centre�without�a�fire�extinguishing�system.�

What�is�of�overriding�importance�is�that�the�gas�extin-guishing�technology�offers�the�type�of�object�and�room�protection�needed.�There�is�no�water�damage,�and�powder�and�foam�residues�are�also�avoided.�Flooding�takes�place�within�120�seconds�in�the�case�of�inert�gases,�and�within�10�seconds�in�the�case�of�chemical�gases.�Fire�extingu-ishing�gases�are�not�electrically�conductive,�and�short�circuits�during�or�after�extinction�are�avoided.

With�gas�extinguishing�systems,�it�is�important�to�bear�in�mind�that�secure�and�reliable�pressure�relief�via�pressure�relief�dampers�is�required�to�deal�with�the�overpressure�that�results�when�the�gases�are�triggered.�The�applicable�regulations�stipulate�that�the�calculated�maintained�concentration�must�be�upheld�for�10�minutes.�The�power�supply�must�be�switched�off�throughout�the�data�centre�in�order�to�prevent�re-ignition.

Fire prevention system

This�version�does�not�just�react�in�the�event�of�a�fire,�but�prevents�it.�The�oxygen�reduction�is�kept�at�an�appropriate�level�by�very�precise�controls.�At�the�same�time,�nitrogen�is�introduced�into�the�rooms.�In�such�an�atmosphere,�no�open�flames�can�develop.�The�protected�areas�remain�accessible�to�people.�

Page 39: Reliable Data Centres - STULZ

37

Reliable Data Centres

6.1.2� Recommended�equipment�for�different�downtimes

Dat

a ce

ntre

ca

tego

ry

Technical fire protection Permitted

data centre dow

ntime*

Servers cabinet Servers cabinet Data centre/ server room

up to 5 kW from 5 kW to 30 kW 500 to 2500 W/m2

A Monitoring�unit�with�fire�detection Monitoring�unit�with�early�fire�detection 72�h

B Monitoring�unit�with�early�fire�detection�and�fire�extinguishing�technology

Fire�alarm�system,�monitoring�unit��with�very�early�fire�detection�and�autonomous�

fire�extinguishing�technology24�h

C Fire�alarm�system,�monitoring�unit�with�very�early�fire�detection��and�autonomous�fire�extinguishing�technology 1�h

DFire�alarm�system,�monitoring�unit��

with�very�early�fire�detection�and�autonomous�fire�extinguishing�technology

Fire�alarm�system,�monitoring�unit�with�very�early�fire�detection�and�autonomous�fire�extin-guishing�technology�/�oxygen�reducing�system

10�min

E Fire�alarm�system,�monitoring�unit�with�very�early�fire�detection�and�autonomous�fire�extinguishing�technology�/�oxygen�reducing�system,�each�in�redundant�arrangement 0�min

Table�8�from�the�BITKOM�matrix�“Planning�Guide�for�a�Reliable�Data�Centre”�–�Technical�fire�protection

Data centres

If�the�tolerated�downtimes�are�24�hours�maximum,�detec-tion�and�monitoring�units�without�downstream�extingu-ishing�systems�are�sufficient.�If�maximum�downtimes�of�less�than�one�hour�are�required,�a�downstream�fire�extin-guishing�system�–�with�gas�as�the�extinguishing�agent�is�indispensable.�If�even�shorter�or�no�downtimes�due�to�fire�can�be�tolerated,�fire�prevention�using�the�technique�of�oxygen�reduction,�combined�with�active�very�early�fire�detection�and�a�gas�extinguishing�system�with�Novec1230�is�a�possible�alternative.

The�decision�in�favour�of�gas�extinguishing�systems�or�fire�prevention�systems�can�only�be�taken�following�thorough�analysis�by�experts.�Aspects�to�be�taken�into�consideration�when�dimensioning�and�installing�a�fire�extinguishing�system�with�control�are:

�� Requirements�for�the�system(s),�tolerable�downtimes,�etc.

�� Type�of�detectors�(fixed�point�detectors,�aspirating�detectors)

�� Type�of�fire�extinguishing�system�and�extinguishing�agent

�� Possible�planning�of�a�fire�prevention�system

Server cabinets

A�monitoring�unit�with�fire�detection�(very�early�fire�detection)�detects�extremely�tiny�quantities�of�smoke�aerosols�that�are�released�in�the�earliest�phase�of�emer-gence�of�fire.�This�gains�valuable�time,�which�can�be�utilised�for�organisational�measures�(automatic�phone�callers,�paging,�etc.)�and�the�initiation�of�counter�mea-sures�such�as�a�soft�shutdown,�external�data�backup,�selective�switch-off�or�possibly�targeted�fire�extinction�of�particular�areas�or�objects.�

In�the�event�of�a�fire,�unit�switch-off�is�the�safest�alterna-tive�to�prevent�further�spreading�of�the�fire�or�of�corrosive�fumes.�Modern,�soft�shutdown�in�no�way�immediately�shuts�off�the�power�supply,�however.�If�soft�shutdown�occurs,�the�very�early�fire�detection�system�activates�an�intelligent�server�manager,�which�quickly�diverts�valuable�

Page 40: Reliable Data Centres - STULZ

38

data�to�neighbouring�data�or�server�cabinets.�However,�this�can�only�be�achieved�if�the�appropriate�software/hardware�environment�is�combined�with�a�high-perfor-mance�very�early�fire�detection�system.�The�definitive�cut-off�of�the�power�supply�then�takes�place�when�the�data�transfer�is�complete.�

�� 6.2� Structural�fire�safety�measures

The�aim�of�structural�fire�safety�measures�is�to�save�human�lives.�This�demands�maximum�quality�of�materials�and�workmanship,�and�strict�adherence�to�regulations�and�procedures.

The�basics�of�structural�fire�safety�are�set�out�in�national�building�regulations,�and�provisions�governing�tech-nical�fire�safety�equipment,�fire�safety�plans,�firewalls�and�emergency�escape�routes.�The�fire�characteristics�of�building�materials�and�components�is�regulated�by�DIN�4102,�which�does�not,�however,�cover�the�necessary�objectives�of�fire�safety�measures,�such�as�are�vital�for�IT�data�centres.�

Further�aspects�to�be�considered�carefully�are�the�fire�resistance�ratings�of�load-bearing�structures�and�fire�protection�in�electrical�installations�and�power�supply�systems.�When�planning�a�data�centre,�the�fire�resistance�rating�and�escape�routes�must�also�be�considered�with�a�view�to�fire�fighter�access�and�safety.�Fire�fighting�lifts�and�safety�stairwells�must�be�included.�Moreover,�data�centres�are�governed�by�fire�safety�regulations�aimed�specifically�at�businesses,�such�as�the�regulations�by�the�VdS.�

Fire�fighting,�extinguishing�agents�and�smoke�extraction�must�also�be�incorporated�in�planning.�This�would�include�mobile�fire�extinguishers,�the�possibility�of�extinguishing�agent�retention�features,�etc.

6.2.1� Fire�safety�objectives

When�planning�a�data�centre,�it�is�of�the�utmost�impor-tance�to�define�the�fire�safety�objectives.�During�the�

planning�phase�you�must�clarify�whether�the�regulations,�guidelines�and�fire�safety�objectives�themselves�can�be�put�into�practice.�The�use�of�experienced�planners�is�recommended,�because�structural�and�technical�fire�safety�measures�must�be�brought�into�balance�with�interruption-free�data�centre�operation.�Subsequent�installations�and�conversions�devour�huge�sums�of�money�or�lead�to�a�spectacular�rise�in�premiums�for�insurance�relating�to�fire�and�electronic�systems.

6.2.2� Method�of�operation�and�room�requirements

Structural�elements�are�categorised�in�fire�resistance�clas-ses�according�to�their�fire�behaviour.�The�fire�resistance�stages�are�mostly�set�at�30,�60,�90�and�120�minutes.�So,�F�30�means,�for�example,�that�during�a�fire�test,�at�least�30�minutes�must�pass�before�the�wall�gives�way.�The�building�authorities�denote�these�classes�as�F�60�for�“fire-retar-dant”�and�F�90�for�“fire-resistant”.

Walls,�floors�and�ceilings�must�be�constructed�with�at�least�fire�resistance�class�F90.�Doors�must�conform�to�T90�quality�as�a�minimum,�i.e.�doors�must�be�resistant�to�fire�for�90�minutes.�Protection�against�fumes�and�spray�water�is�also�indispensable.

Cable�and�installation�ducts�from�and�to�the�data�centre�must�be�effectively�protected.�Flame-retardant�cable�ducts�can�have�protection�with�E30�or�even�E90�qua-lity.�Installation�ducts�must�be�in�I30�or�I90�quality,�and�independent�ventilation�ducts�in�L90.�If�electric�cables�are�routed�through�fire-resistant�ceilings�and�walls,�the�cable�bushings�must�also�be�fire�and�fume�resistant�–�in�other�words,�sealed�off.�This�isolation�may�in�some�cases�be�achieved�by�means�of�intumescent�fire�pillows.�

Cable�runs�constitute�a�very�high�risk�in�the�event�of�a�fire,�and�should�be�coated�or�designed�to�be�water�and�moisture�resistant.�This�will�make�them�intumescent,�capable�of�reliably�preventing�fire�from�spreading�along�the�cables.�The�cables�themselves�should�consist�of�

Page 41: Reliable Data Centres - STULZ

39

Reliable Data Centres

flame-retardant�material,�which�also�does�not�produce�aggressive�fumes.

Fire�spreads�rapidly�and�uncontrollably�through�(flamma-ble)�pipes,�which�are�laid�on�and�in�ceilings�and�walls.�Pipe�shielding�provides�a�fire-resistant�and�fume-proof�barrier.

However,�simple�testing�of�structural�elements�is�in�no�way�adequate�for�complex,�high-availability�data�centres.�When�high�availability�is�required,�the�rooms�or�modular�

safety�cells�to�be�constructed�must�undergo�a�standar-dised�system�test�to�EN�1047-2,�as�must�the�structural�elements�of�the�ceiling-wall�and�wall-floor�connections,�the�cable�entries,�overpressure�relief�and�the�doors�and�surrounding�areas.�The�European�standard�for�the�data�centre�infrastructure�specifies�both�the�intensity�and�the�duration�of�precisely�defined�loads.�The�user�is�thus�safe�in�the�knowledge�that�his�entire�system�–�not�just�one�wall�or�the�door�–�is�fire-resistant.

6.2.3� Recommended�equipment�for�different�downtimes

Dat

a ce

ntre

ca

tego

ry

Structural fire safety Permitted

data centre dow

ntime*

Servers cabinet Servers cabinet Data centre/ server room

up to 5 kW from 5 kW to 30 kW 500 to 2500 W/m2

A Walls,�floors,�ceiling:�Fire�resistance�class�min.�F90,�protection�against�fumes�and�spray�water,�doors�min.�T90,�protection�of�equal�quality�for�

cable�walls

Walls,�floors,�ceiling:�Fire�resistance�class�min.�F90,�protection�against�fumes�and�water,�doors�min.�T90,�protection�of�equal�quality�for�cable�walls

72�h

B Walls,�floors,�ceiling:�Fire�resistance�class�min.�F90,�protection�against�fumes�and�spray�water,�doors�min.�T90,�protection�of�equal�quality�for�

cable�walls

Walls,�floors,�ceiling:�Fire�resistance�class�min.�F90,�protection�against�fumes�and�water,�doors�min.�T90,�protection�of�equal�quality�for�cable�walls

24�h

C System�test�of�structural�fire�safetyWalls,�floors,�ceiling,�doors:�as�per�European�

Standard�EN�1047-2,�protection�against�fumes�and�spray�water�for�60�min,�protection�of�equal�

quality�for�cable�walls

System�test�of�structural�fire�safetyWalls,�floors,�ceiling,�doors:�as�per�European�

Standard�EN�1047-2,�protection�against�fumes�and�water�for�60�min,�protection�of�equal�quality�for�

cable�walls

1�h

D System�test�of�structural�fire�safetyWalls,�floors,�ceiling,�doors:�as�per�European�

Standard�EN�1047-2,�protection�against�fumes�and�spray�water�for�60�min,�protection�of�equal�

quality�for�cable�walls

System�test�of�structural�fire�safetyWalls,�floors,�ceiling,�doors:�as�per�European�

Standard�EN�1047-2,�protection�against�fumes�and�water�for�60�min,�protection�of�equal�quality�for�

cable�walls

10�min

E System�test�of�structural�fire�safetyWalls,�floors,�ceiling,�doors:�as�per�European�Standard�EN�1047-2,�protection�against�fumes��

and�spray�water�for�60�min,�protection�of�equal�quality�for�cable�walls

0�min

Table�9:�From�the�BITKOM�matrix�“Planning�Guide�for�a�Reliable�Data�Centre”�–�Structural�fire�safety�measures

Page 42: Reliable Data Centres - STULZ

40

6.2.4�Special�features

Aspects�to�be�taken�into�consideration�during�project�planning�are:

�� Determination�of�fire�safety�objectives,�under�consideration�of�the�special�requirements�of�the�IT�infrastructure

�� Determination�of�structural�characteristics�� Planning�the�construction�–�by�professional�planners,�

if�possible�� Compilation�of�specifications�for�the�individual�ele-

ments�to�be�offered�for�tender�� Collection�of�incoming�quotations,�comparison,�

evaluation�� Drawing�up�a�contract-awarding�proposal�for�the�

decision-makers

Page 43: Reliable Data Centres - STULZ

41

Reliable Data Centres

7 Design of premises and safety zones for data centres

Security�of�information�technology�is�a�sweeping�term�that�covers�logistical�data�security,�the�physical�safety�of�systems�and�the�organisational�reliability�of�the�proces-ses.�The�objective�of�a�comprehensive�security�concept�is�to�take�all�areas�into�consideration,�to�recognise�and�assess�risks�early�on�and�to�initiate�measures�in�such�a�way�that�a�company’s�competitiveness�on�the�market�is�not�jeopardised.�

Through�an�overall�examination�of�the�IT�infrastructure�and�various�functional�areas�of�the�IT,�a�well�thought-out�plan�can�reduce�or�even�eliminate�major�risks�to�physical�safety.�A�decisive�role�is�played�by�the�premises�in�which�the�IT�is�located�on�the�one�hand,�and�the�spatial�arrange-ment�of�the�various�functions�in�relation�to�one�another�on�the�other�hand.

Location of IT rooms

The�design�of�an�IT�infrastructure�and�thus�the�selection�of�a�site�for�a�data�centre�must�be�based�on�the�company’s�data�security�principles,�which�reflect�availability�require-ments�and�the�company’s�strategic�direction.

The�following�criteria�should�be�taken�into�consideration�when�examining�the�physical�safety�of�a�site:

�� Low�risk�potential�from�neighbouring�usage,�adjacent�parts�of�the�building�or�functions

�� Avoidance�of�risks�from�media�and�power�supply�lines,�tremors�and�chemicals,�which�would�compromise�the�physical�safety�of�the�IT�systems

�� Avoidance�of�possible�dangers�posed�by�elementary�risks�(water,�gales,�lightning�strikes,�earthquakes),�risk�assessment�of�aspects�peculiar�to�the�region

�� The�data�centre�as�a�separate,�independent�functional�zone

�� A�“protected”�location�to�ensure�protection�against�sabotage

�� Assessment�of�potential�threats�on�the�grounds�of�the�company’s�social�policy

If�all�risk�factors�and�the�constraints�specific�to�the�company�are�taken�into�consideration,�potential�danger�and�the�resulting�time�and�expense�can�be�eradicated�in�advance�during�the�design�of�the�IT�infrastructure.�

Structure of a data centre

When�designing�and�planning�a�data�centre,�the�vari-ous�functional�areas�are�arranged�in�accordance�with�the�requirements�for�their�safety�and�security,�and�their�importance�for�maintaining�the�function�of�the�informa-tion�technology.

The�various�functional�areas�can�be�set�out�as�follows:

Safety zones

Function Identifi-cation (example)

1 Plot White

2 Semi-public�area,�neighbouring�offices

Green

3 Operating�areas,�rooms�adjoining�IT

Yellow

4 Technical�systems�for�operating�IT

Blue

5 IT�and�network�infrastructure

Red

Table�10:�Functional�areas�of�a�data�centre

Page 44: Reliable Data Centres - STULZ

42

Layout of safety zones

A�schematic�representation�of�the�various�safety�zones�produces�something�like�the�example�illustrated�here:�the�IT�area�(red)�is�situated�in�the�interior�and�is�protected�by�the�adjacent�zones�3�and�4�(yellow/blue).�Safety�zones�1�and�2�(white/green)�form�the�outer�layers.�The�individual�safety�zones�are�divided�by�safety�lines.

The�safety�lines�represent�the�monitored,�safe�transition�from�one�zone�to�another,�and�are�set�up�in�conformity�with�the�company’s�safety�requirements.

A�fitting�solution�for�avoiding�possible�sabotage�is�to�separate�the�functional�areas�by�ensuring�limited�access�to�sensitive�areas.�For�example,�a�maintenance�technician�for�the�air-conditioning�systems�or�UPS�has�access�only�to�the�technical�zones�(blue),�not�to�the�company’s�IT�zone�(red).�

The�locations�of�the�different�functional�areas�and�the�division�of�the�safety�zones�or�safety�lines�are�impor-tant�for�guaranteeing�the�safety�of�the�IT�infrastructure.�However,�continuous�IT�availability�can�only�be�achie-ved�through�a�comprehensive�safety�plan,�which�takes�account�of�all�aspects�of�IT�security�and�safety.�

Page 45: Reliable Data Centres - STULZ

43

Reliable Data Centres

8 Wiring

�� 8.1� Current�situation

The�primary,�original�task�of�data�centres�is�to�run�IT�appli-cations�on�mainframes�and�servers,�to�maintain�data�and�save�it�on�storage�systems.

From�an�IT�point�of�view,�the�principal�requirement�is�availability,�i.e.�the�ability�of�IT�applications�that�are�gene-rally�critical�for�companies�to�remain�operational�possibly�without�any�interruption.�These�typically�include�ERP�systems,�production�applications�in�industrial�companies,�databases,�office�applications�and�their�operating�sys-tems,�and�also�access�to�provider�networks�(MAN,�WAN)�and�to�the�internet.

The�ISO-OSI�7�Layer�Reference�Model�applies�to�the�IT,�and�this�defines�the�application�as�the�uppermost�layer,�and�the�physical�infrastructure�required�for�transporting�the�data,�the�IT�wiring�and�data�transport�devices�as�the�lowest,�or�first,�layer,�e.g.�layer�1�switches.

Therefore,�the�IT�wiring�is�of�fundamental�importance�for�the�availability�of�IT�applications�in�a�data�centre:�without�functioning�IT�wiring,�IT�devices�such�as�servers,�switches�and�storage�devices�cannot�communicate�with�each�other�or�exchange,�process,�hold�or�save�data.

It�is�frequently�the�case,�however,�that�IT�wiring�has�simply�grown�over�time,�and�only�satisfies�with�difficulty�current�requirements�such�as

�� high�duct�densities�� high�transmission�speeds�� interruption-free�hardware�modifications�� service�support�� ventilation�issues

The�structuring�of�IT�wiring�and�meticulous,�anticipatory�planning�are�therefore�key�tasks�for�a�data�centre�opera-tor.�Legal�principles�such�as�Basel�II�or�SOX�also�demand�rigorous�and�comprehensive�transparency.

�� 8.2� Basic�standards

What�all�standards�to�ISO/IEC,�EN�and�TIA�have�in�com-mon�is�that�they�demand�or�stipulate�structured,�appli-cation-neutral�IT�wiring.�In�addition,�they�express�clear�recommendations�for�setting�up�redundant�IT�wiring,�for�ensuring�high-level�reliability�of�the�data�centre.

The�planning,�installation�and�acceptance�testing�of�IT�wiring�in�data�centres�is�described�in�the�DIN�EN�50174�series�of�standards.�Important�contents�include�the�qua-lity�plan,�potential�equalisation,�safety�distances�between�copper�IT�wiring�and�other�electrical�sources,�and�the�documentation�and�acceptance�testing�of�the�entire�data�centre.�

8.3� Quality/selection�of�components/systems

Requirements�for�maximum�availability�and�ever�faster�data�transmission�rates�mean�that�the�quality�require-ments�facing�IT�wiring�components�for�data�centres�are�many�times�more�exacting�than�those�for�products�used�in�LANs.�When�it�comes�to�selecting�systems,�quality�principles�should�already�be�incorporated�in�the�very�early�planning�phase,�with�a�view�to�satisfying�performance�requirements�such�as�

�� Cable�design�for�copper�and�fibre-optics�� Bandwidths�for�copper�systems�and�fibre-optic�cables�� Insertion�and�return�loss�budgets�for�fibre-optics�� EMC�immunity�for�copper�systems�� Update�capability�to�higher�speed�classes�� 19”�cabinet�design

Whether�fibre-optic�or�copper�cables�are�used,�the�IT�wiring�components�may�take�the�form�of�factory-assem-bled,�turnkey�systems�for�plug�&�play�installations.

Preassembled�systems�have�the�best�possible,�reproduci-ble�quality,�and�therefore�promise�very�good�transmission�

Page 46: Reliable Data Centres - STULZ

44

characteristics�and�high�reliability.�Only�screened�systems�may�be�used�in�copper�systems,�due�to�the�demanding�availability�requirements.�Global�standards�require�copper�wiring�with�at�least�class�EA.

Sufficient�priority�must�also�be�lent�to�the�selection�of�suppliers�of�IT�wiring.�Besides�the�quality�of�the�wiring�components,�the�chief�requirements�for�a�reliable�supplier�are�specialist�data�centre�expertise,�experience�with�data�centre�IT�wiring�and�lasting�ability�to�supply.�Ideally,�the�supplier�should�also�offer�comprehensive�planning,�instal-lation�and�maintenance�services.

�� 8.4� Structure

Data�centres�are�the�company’s�central�nervous�system.�They�are�therefore�subject�to�continuous�change,�driven�by�the�short�lifecycles�of�the�active�components.�In�order�to�avoid�fundamental�or�far-reaching�changes�to�the�IT�wiring�with�every�new�piece�of�equipment,�a�physical�

IT�wiring�structure�that�is�clearly�arranged,�transparent�and�isolated�from�the�surrounding�“device�fleet”�is�highly�recommended.�

It�should�connect�the�respective�device�locations�to�a�uniform,�consistent�IT�wiring�structure.�

Standards�DIN�EN�50�173-5�and�ISO/IEC�24764�divide�this�equipment�wiring�into�the�segments�area�and�device�connection�wiring,�with�the�device�connection�interface�in�between.�The�active�devices�are�connected�to�the�“device-neutral”�area�wiring�via�the�device�connection�interface�by�means�of�device-specific�connecting�cables�that�are�as�short�as�possible.�Consequently,�when�a�device�is�replaced�–�which�often�requires�replacing�the�connection�face�on�the�device�–�only�the�cable�specific�to�this�connection�has�to�be�replaced�–�with�no�need�to�modify�or�strip�the�area�wiring.�

Areas�with�a�high�packing�density�are�worthy�of�particular�attention�in�this�context.�

Office

Telecommuni-cation room

Area distributor

Localdistributor

Device connection

Area distributor

Device connection

Interface to external netwrok

Main distributor

Area distributor

Device connection

Data Center...

backbone wiring horizontal wiring

Figure�9:�Schematic�diagram�to�EN�50173-5�/�TIA�942,�wiring�layout

Page 47: Reliable Data Centres - STULZ

45

Reliable Data Centres

The�above�technique�ensures�that�any�rewiring�involved�during�equipment�replacements�are�reduced�to�a�mini-mum�both�financially�and�in�terms�of�time�–�while�the�defined�structure�is�fully�preserved.

Where�necessary,�the�horizontal�wiring�must�be�made�of�copper�and�fibre-optic�cables,�in�order�that�various�devices�can�be�connected.�The�backbone�wiring�should�be�in�fibre-optic�and�copper�cables,�and�offer�redundancy.

Figure�10:�Representation�of�horizontal�wiring�(Cu�and�fibre-optic)�with�area�distributor�and�server/storage�cabinets�with�device�connection

Figure�11:�Representation�of�backbone�wiring�(fibre-optic)�with�main�distributor�and�connection�to�horizontal�wiring�(Cu�and�fibre-optic)�with�area�distributor�and�server/storage�cabinets�with�device�connection

Suitable�plug-in�systems�must�be�selected�for�the�device�connection�interfaces,�in�accordance�with�the�required�packing�density�of�the�connected�devices.�Suitable�plug-in�systems�are�named�in�standards�DIN�EN�50�173-5�and�ISO/IEC�24764.�

�� 8.5� Redundancy�and�security

The�requirement�for�high�availability�necessitates�redun-dancy�of�connections�and�components:�Thus,�it�must�be�possible�to�replace�hardware�during�ongoing�operation,�and�an�alternative�component�must�take�on�the�running�of�the�application�without�interruption�in�the�event�of�cable�failure.

It�is�therefore�obvious�that�an�appropriate,�all-inclusive�IT�wiring�platform�must�be�provided�that�ensures�the�correct�bending�radii,�safeguards�performance�and�can�be�fitted�quickly�and�reliably�during�operation.�

The�availability�of�applications�can�be�increased�through�the�use�of�preassembled�IT�wiring�systems.�This�reduces�the�time�that�installation�personnel�have�to�spend�in�the�safety�zone�of�the�data�centre�to�a�minimum,�both�during�initial�installation�and�eventual�hardware�modifications,�and�also�promises�additional�operational�reliability.�In�addition,�care�must�be�taken�to�ensure�that�all�pro-ducts�are�inspected�and�documented�as�part�of�quality�management.

For�the�mutual�connection�of�data�centres,�e.g.�redundant�data�centres,�backup�data�centres,�or�simply�for�backing�up�and�saving�data�at�a�different�site,�the�incorporation�and�security�of�MAN�and�WAN�provider�networks�(data�transport�services�or�so-called�dark�fibres)�or�dedicated�fibre-optic�cable�runs�is�of�enormous�importance�where�reliability�and�availability�are�concerned�and,�like�the�IT�wiring�inside�the�data�centre,�must�be�redundant.

Page 48: Reliable Data Centres - STULZ

46

�� 8.6� Installation

Engineers�and�technicians�must�be�trained�in�the�specifi-cations�of�the�systems�in�order�to�ensure�the�safe,�reliable�operation�of�fibre-optic�IT�wiring�in�the�data�centre,�particularly�in�the�matter�of�installation�and�work�with�patches.�Due�to�the�required�wiring,�when�selecting�the�19”�server�or�IT�wiring�cabinet�and�with�reference�to�item�“4.1�Secure�server�cabinets”,�the�use�of�cabinet�systems�at�least�800�mm�wide�is�recommended.�These�enable�the�installation�of�an�integral�cable�management�system�in�the�vertical�and�horizontal�direction.�As�a�rule,�the�depth�of�the�cabinet�is�determined�by�the�passive�and�active�components�to�be�installed�inside.�Cabinet�systems�with�a�minimum�depth�of�800�mm�have�also�proven�their�worth�for�passive�distributors.�A�depth�of�1000�to�1200�mm�is�recommended�for�cabinet�systems�destined�to�accommo-date�active�components.

The�possible�advantage�of�factory-preassembled�IT�wiring�systems�already�mentioned�in�the�section�on�safety�principles�comes�to�the�fore�during�installation�in�the�form�of�time�savings.�When�these�systems�are�employed,�it�is�worth�mentioning�that�when�data�centre�capacity�is�expanded�due�to�an�increase�in�IT�equipment,�these�devices�and�hence�the�IT�applications�themselves�can�be�wired�to�one�another�and�brought�into�operation�at�the�greatest�possible�speed�–�and�the�same�applies�to�hard-ware�modifications.

�� 8.7� Documentation�and�labelling

Painstaking,�continually�updated�documentation�is�an�important�means�of�ensuring�the�simple�management�of�IT�wiring�and�reliable�planning�of�conversions�and�upgrades.�A�large�range�of�possibilities�exist�in�this�respect,�from�“individual”�Excel�worksheets�to�proven,�software-based�documentation�tools.�What�is�important�is�that�the�documentation�is�always�kept�up-to-date,�and�reflects�the�actual,�installed�IT�wiring.�The�choice�of�tool�is�at�the�user’s�discretion.

Closely�connected�to�documentation�is�the�clear�and�–�even�in�limited�light�–�easily�legible�labelling�of�cables.�Here,�too,�numerous�identification�methods�are�available,�e.g.�from�cable�tags�with�replaceable�labels�to�barcode�labels.�The�type�selected�will�depend�on�individual�requirements.�It�is�important�to�ensure�that�identical�nomenclature�is�used�throughout�the�company.�Central�management�of�the�data�is�recommended�to�ensure�clear,�unambiguous�labelling�of�cables.

Page 49: Reliable Data Centres - STULZ

47

Reliable Data Centres

9 Certification of a reliable data centre

�� 9.1� The�management�system

9.1.1� The�design�of�a�management�system

For�all�companies,�customer�orientation,�cost�efficiency�and�competitiveness�are�in�the�foreground.�Decisive�fac-tors�are�resource�planning,�fault�prevention�and�a�process-oriented�approach�to�the�performance�of�services.�

International�standards�such�as�ISO/IEC�27001:2005,�etc.�provide�concrete�assistance�and�a�structure�for�imple-menting�a�management�system,�and�can�be�certified�by�neutral�associations.�

A�management�system�forms�the�basis�for��� process�orientation�� continuous�improvement�of�business�processes�(CIP)�� greater�customer�and�supplier�orientation�� interpretation�of�interactions�between�business�

processes�� fault�prevention�(preventive�and�corrective�measures),�

and��� compliance�with�legal�requirements�

9.1.2� Advantages�of�a�management�system

A�successful�management�system�brings�the�company�decisive�advantages:

�� Business�processes�become�traceable,�products�and�services�are�improved�

�� Weak�points�in�the�system�are�flagged�up�� Risks�are�recognised,�analysed,�minimised�or�averted�� The�company�is�prepared�for�emergencies��� Continuous�monitoring�lowers�the�error�rate�and�

therefore�improves�the�services�provided.�This�results�in�reduced�costs�and�higher�quality

�� Employees�are�motivated�by�the�clear�delineation�of�responsibilities

�� The�documentation�of�processes�creates�transparency�� Customers’�trust�in�the�company’s�ability�to�deliver�

quality�is�boosted�� The�company’s�image�is�enhanced�� Competitiveness�is�strengthened�� Liability�is�minimised

9.1.3� Combination�of�different�standards

The�uniform�structure�of�ISO�management�systems�basi-cally�permits�several�standards�to�be�combined.�If�such�a�combination�is�possible,�it�will�be�easy�to�incorporate�new�standards�and�exploit�synergistic�effects�in�different�areas�of�the�company.�

Examples�of�synergies�are�the�merging�of�the�following�elements�

�� Corporate�policy�and�goals�� Management�review�� Existing�documentation�(assignment�of�names,�revi-

sion,�approval,�etc.)�� Internal�audits�� Evaluation�of�error�frequency�and�sources�� Evaluation�of�customer�and�employee�satisfaction��� Evaluation�of�suppliers�� Procurement�process

�� 9.2� Certification�of�a�management�system

Once�the�quality�management�system�has�been�docu-mented�and�effectively�introduced�in�the�company,�it�can�be�certified�by�an�independent,�neutral�company�that�is�authorised�to�perform�certification.�First�of�all,�the�certi-fication�company�inspects�the�documentation�and�then�the�system�on�site.�The�auditor�possesses�the�necessary�

Page 50: Reliable Data Centres - STULZ

48

qualifications�and�professional�experience.�A�positive�result�leads�to�certification,�which�is�generally�valid�for�3�years.�

After�the�certification�audit,�the�management�system�is�checked�for�its�effectiveness�in�annual�monitoring�audits.�

9.2.1� The�advantages�of�certification

Certification�is�neutral�proof�of�the�effectiveness�of�the�management�system,�and�can�offer�the�following�advantages:

�� Acquisition�of�new�customers,�opening�the�door�to�new�markets�

�� Strengthening�competitiveness�� Fosters�the�customers’�trust�in�the�company�� Improves�the�company’s�ranking�and�creditworthiness�� Reduces�the�time�and�cost�of�demonstrating�quality�� International�recognition�and�acceptance

9.2.1� A�typical�certification�procedure

An�informative�discussion�should�be�held�before�a�certifi-cation�company�is�selected.

The informative discussion

The�subjects�under�discussion�basically�include�questions�about�certification�and�auditing,�organisational�procedure�(e.g.�scope�and�schedule)�and�costs.�

The certification order

By�placing�an�order�for�certification,�the�contracting�company�places�itself�under�obligation�to�provide�the�certification�company�with�the�necessary�QM�documen-tation.�Alternatively,�the�QM�documentation�may�also�be�inspected�on�site.�A�preliminary�audit�may�be�conducted,�if�the�company�so�desires.

Performing a preliminary audit

The�aim�of�the�preliminary�audit�is�to�ascertain�whether�the�basic�prerequisites�for�certification�of�the�quality�management�system�are�met.�It�determines�whether�the�certification�audit�can�be�carried�out�on�the�planned�date�with�success�as�the�likely�outcome.

Alternatively,�the�check�performed�during�the�preliminary�audit�may�incorporate�the�inspection�of�the�QM�manual�and�associated�documents.�The�preliminary�audit�is�basically�a�random�sample�test,�and�does�not�claim�to�be�complete.

The certification process

During�the�certification�audit,�the�auditors�check�whether�the�documented�procedures�and�routines�satisfy�the�requirements�of�the�relevant�rules,�and�whether�the�pro-cesses�and�agreements�defined�by�the�company�match�the�QM�documentation.

The�certification�audit�consists�of�a�comprehensive�assess-ment.�In�detail,�it�covers�inspection�of�the�following:

�� QM�system�documentation�� Structural�organisation�� Implementation�of�the�documented�processes�and�

their�effects�in�the�company

First-time�certification�takes�the�form�of�a�2-stage�process.�

In�stage�1,�important�key�elements�are�checked�and�an�assessment�ascertains�to�what�extent�the�company�is�prepared�for�certification.�The�company’s�QM�documen-tation�is�also�checked�for�conformity�to�standards�at�this�point.�

In�stage�2,�the�implementation�and�effectiveness�of�the�management�system�throughout�the�company�is�checked.�If�deviations�are�disclosed�during�the�assessment�of�the�quality�management�system,�a�post�audit�may�

Page 51: Reliable Data Centres - STULZ

49

Reliable Data Centres

need�to�be�performed.�The�certificate�is�then�only�granted�after�all�such�deviations�have�been�eliminated�and�when�the�negative�findings�fall�below�the�maximum�number�recommended�by�the�auditor.

The�result�is�documented�in�an�audit�report.�This�is�the�basis�upon�which�the�certification�committee�of�the�certification�company�decides�whether�or�not�to�grant�a�certificate.

The monitoring audit

Annual�monitoring�audits�take�place�during�the�three-year�period�of�validity�of�the�certificate.

The�1st�and�2nd�monitoring�audits�constitute�a�random�sample�check�as�to�whether:

�� Negative�finding(s)�from�the�previous�audit�has/have�been�remedied

�� Organisational�changes�have�taken�place�in�the�company

�� The�QM�system�has�changed�� The�certificate�and�certification�logo�are�being�used�

correctly�� Recent�changes�to�relevant�standards,�laws�and�regu-

lations�have�been�taken�into�consideration�� The�management�system�continues�to�satisfy�the�

requirements,�and�� The�quality�management�system�has�been�adhered�to�

and�continues�to�be�effective.�

If�the�result�of�the�monitoring�audits�is�successful,�a�complete�inspection�of�the�QM�system�will�take�place�in�a�new�process�in�three�years’�time,�beginning�with�a�recertification�audit.

The recertification audit

A�recertification�audit�is�necessary�for�ensuring�a�seam-less�extension�of�the�period�of�validity�of�a�certificate�by�

another�three�years.�The�recertification�audit�must�be�carried�out�before�the�certificate’s�validity�expires.�The�requirements�of�the�recertification�audit�are�virtually�identical�to�those�of�the�certification�audit.�Here,�however,�the�audit�may�be�performed�in�a�single�stage.

9.2.3� Selecting�the�right�certification�partner

The�selection�of�the�right�certification�partner�is�decisive�for�the�success�of�the�process.�As�with�any�service,�a�price�margin�applies.�It�is�therefore�advisable�to�obtain�several�quotations,�or�to�extend�the�existing�system�with�a�trus-ted�certification�partner�who�is�already�familiar�with�the�system.�

In�certain�cases,�the�certification�company’s�international�organisation�may�constitute�a�major�cost�factor�if,�for�example,�company�sites�in�other�countries�are�to�be�inclu-ded�in�the�certification�process.�

Auditors�with�different�certification�bodies�are�qualified�to�a�similarly�high�level,�as�the�approval�of�audits�is�regu-lated�centrally�by�accreditation�bodies�and�ISO�standards.

Page 52: Reliable Data Centres - STULZ

50

10 Annex

Selection�of�important�rules�and�regulations

DIN�6280,�Part�1-15 Generating�sets�with�reciprocating�internal�combustion�engines

Part�1� General�definitions

Part�2� Power�ratings�and�power�rating�plates

Part�3� Limiting�values�for�the�operating�behaviour�of�the�engine,�the�generator�and�the�generating�set

Part�4 Speed�governing�and�speed�behaviour�of�reciprocating�internal�combustion�engines;�definitions�

Part�5 Operational�behaviour�relating�to�synchronous�alternators��for�generating�sets

Part�6� Operational�behaviour�relating�to�asynchronous�alternators��for�generating�sets

Part�7� Controlling�and�governing�equipment�for�generator�operation

Part�8� Operational�behaviour�relating�to�generating�sets;�definitions

Part�9� Acceptance�test�

Part�10 Small�power�generating�sets;�requirements�and�tests�

Part�11� Measurement�and�assessment�of�mechanical�vibrations�in�generating�sets�with�reciprocating�internal�combustion�engines�

Part�12� Generating�sets�–�Uninterruptible�power�supply�–�Dynamic�UPS�systems�with�and�without�reciproca-ting�internal�combustion�engine�

Part�13� Generating�sets�–�Generating�sets�with�reciprocating�internal�combustion�engine�for�emergency�power�supply�in�hospitals�and�public�buildings�

Part�14� Combined�heat�and�power�system�(CHPS)�with�reciprocating�internal�combustion�engine�–�Basics,��requirements,�components�and�application�

Part�15� Combined�heat�and�power�system�(CHPS)�with�reciprocating�internal�combustion�engine�–�Tests

Page 53: Reliable Data Centres - STULZ

51

Reliable Data Centres

ISO�8528 Reciprocating�internal�combustion�engine�driven�alternating�current�generating�sets.�

German�Federal�Control�of�Pollution�Act

4.� Regulation�on�the�implementation��of�the�Federal�Control�of�Pollution�Act�

Regulation�on�plants�and�systems�requiring�approval.

9.� Regulation�on�the�implementation��of�the�Federal�Control�of�Pollution�Act�-

Basic�principles�of�the�approval�procedure

TA� Air�German�Technical�Guidelines�on�Air�Quality�Control�

TA� Noise�German�Technical�Guidelines�on�Noise�Prevention�

DIN�/�VDE�0107 Electrical�installations�in�hospitals�and�locations�for�medical�use�outside�hospitals

Supplement�1� Extracts�from�building�and�occupational�health�and�safety�regulations�

Supplement�2 Interpretation,�explanation�

DIN�/�VDE�0108� Power�installations�and�safety�power�supply�in�communal�facilities

Supplement�1 Building�regulations�

Part�2� Places�of�assembly�

Part�3 Stores,�shops�and�exhibition�rooms�

Part�4� Multi-storey�buildings

Part�5 Restaurants

Part�6 Closed�car�parks�

Part�7 Working�premises�

Part�8 Temporary�structures,�etc.

DIN�/�VDE�0100��Part�728

Standby�power�installations

Page 54: Reliable Data Centres - STULZ

52

PSC Connection�conditions�of�PSCs�

VDEW Guidelines�for�emergency�generating�sets�by�the�German�electricity�association�(VDEW)�

VDEW Parallel�operation�with�the�low-voltage�network�by�the�German�electricity�association�(VDEW)�

EltBauVO� Ordinance�governing�the�construction�of�operating�rooms�for�electrical�installations�

VDS Regulations�from�the�German�Association�of�Property�Insurers�

WHG German�Water�Resources�Act�

Fuel�Oil�Taxation�Act (operation�of�stationary�systems�with�fuel�oil)

DIN�31051� Maintenance

Page 55: Reliable Data Centres - STULZ

53

Reliable Data Centres

11 Glossary

19”�cabinet Rack�with�approximately�40�height�units,�overall�height�approximately�2�metres,�installation�width�483�mm,�installation�height�is�measured�in�height�units�(HU),�1�HU�=�44.45�mm

CW Chilled�Water;�air-conditioning�systems�with�chilled�water

Data�centre Server�room�and/or�data�centre

DX Direct�eXpansion;�air-conditioning�systems�with�refrigerant

Emission Influences�emitted�by�a�device�that�affect�the�environment

EMC ElectroMagnetic�Compatibility

Immission Influences�originating�from�the�environment�that�affect�a�particular�location

IT Information�Technology�(formerly�EDP�–�Electronic�Data�Processing)

Modular Describes�a�system�that�is�constructed�of�several�modules�(assemblies)

Parallel�operation Two�or�more�installations�jointly�supply�connected�consumers

PDU Power�Distribution�Unit�or�low-voltage�distribution�system

Precision�air-conditioning�system

Air-conditioning�system�that�is�able�to�keep�both�the�temperature�and�the�air�humidity�cons-tant.�The�parameters�of�the�air�at�the�inlet�openings�of�the�IT�units�should�be�between�22�and�27°C�and�40�and�60%�r.h.

PSC Power�Supply�Company

Redundant Duplicate�arrangement�to�increase�availability�(error�tolerance)

Scalable Step-by-step�adaptability�to�requirements

SG Standby�generator�(mostly�an�emergency�diesel�generator)

UPS Uninterruptible�Power�Supply

Page 56: Reliable Data Centres - STULZ

54

12 Acknowledgements

This�“Reliable�Data�Centre”�guide�was�compiled�in�consul-tation�with�the�BITKOM�„Data�Centre�&�IT�Infrastructure”�working�group.�

�� We�wish�to�express�our�sincere�thanks�to�all�members�of�the�working�group�for�the�valuable�discussions�we�had�with�them,�and�our�particular�thanks�go�to�the�following�for�their�involvement:

�� Silvia�Bader�(DEKRA�certification�GmbH)�� Harald�Becker�(Rosenberger-OSI�GmbH�&�Co.�OHG)�� Dr.�Gerald�Berg�(Rosenberger-OSI�GmbH�&�Co.�OHG)�� Klaus�Clasen�(Notstromtechnik�Clasen�GmbH)�� Peter�Clauss�(Wagner�Group�GmbH)�� Aykut�Güven�(DEKRA�certification�GmbH)�� Frank�Hauser�(Server�Technology�International)�� Dieter�Henze�(Rittal�GmbH�&�Co.�KG)�� Dr.�Siegbert�Hopf�(Masterguard�GmbH)�� Knut�Krabbes�(QMK�IT-Security+Quality)�� Stephan�Lang�(Weiss�Klimatechnik�GmbH)�� Helmut�Muhm�(Dipl.-Ing.�W.�Bender�GmbH�&�Co.KG)�� Hans-Jürgen�Niethammer�(Tyco�Electronics�AMP�

GmbH)�� Torsten�Ped�(Notstromtechnik�Clasen�GmbH)�� Achim�Pfleiderer�(Stulz�GmbH)�� Thorsten�Punke�(Tyco�Electronics�AMP�GmbH)�� Zeynep�Sakalli�(euromicron�solutions�GmbH)�� Michael�Schumacher�(APC�Deutschland�GmbH)�� Karlheinz�Volkert�(Orange�Business�Germany�GmbH)�� Peter�Wäsch,�SCHÄFER�Ausstattungs-Systeme�GmbH�� Ralph�Wölpert�(Rittal�GmbH�&�Co.�KG)

The�following�people�assisted�with�Version�1�from�Novem-ber�2006:

�� Dieter�Henze�(Rittal�GmbH�&�Co.�KG)�� Siegbert�Hopf�(Masterguard�GmbH)�� Peter�Koch�(Knürr�AG),�Helmut�Göhl�(O2�GmbH)�� Knut�Krabbes�(QMK�IT-Security+Quality)�� Matthias�Lohmann�(TÜV�Secure)�� Ingo�Lojewski�(Emerson�Network�Power�GmbH)�� Achim�Pfleiderer�(Stulz�GmbH)�� Jörg�Richter�(I.T.E.N.O.S�GmbH)�� Harry�Schnabel�(Harry�Schnabel�Consult)�� Michael�Schumacher�(APC�Deutschland�GmbH)�� Jürgen�Strate�(IBM�Deutschland�GmbH)�� Judith�Wagener�(Bull�GmbH)�� Ralph�Wölpert�(Lampertz�GmbH�&�Co.�KG)�� Eckhard�Wolf�(AEG�Power�Solutions�GmbH)�� Sandra�Schulz�(Giesecke�&�Devrient�GmbH)

Page 57: Reliable Data Centres - STULZ

The�Federal�Association�for�Information�Technology,�Telecommunications�and�New�Media�(BITKOM)�represents�more�than�1,300�companies�in�Germany.�Its�950�direct�members�generate�a�sales�volume�of�135�billion�euros�annually�and�employ�700,000�people.�They�include�providers�of�software,�IT�and�telecommunication�services,�manufacturers�of�hardware�and�consumer�electronics�as�well�as�digital�media�businesses.�BITKOM�is�working,�in�particular,�to�improve�the�regulatory�framework�in�Germany,�for�modernization�of�the�education�system�and�for�an�economic�policy�which�encourages�innovation.

Bundesverband�Informationswirtschaft,Telekommunikation�und�neue�Medien�e.�V.

Albrechtstraße�10�A10117�Berlin-MitteTel.:�03o.27576-0Fax:�[email protected]

This�translation�was�kindly�sponsored�by�the�following�companies: