Data Center Global Telecommunications Cabling Standards Landscape and Latest

Developments

Jonathan Jew - J&M Consultants, Inc.Chair TIA TR-42.6 Telecom Administration

Vice-chair TIA TR-42.1 Commercial BuildingsCo-chair BICSI Data Center Subcommittee

Vice-chair USTAG ISO/IEC JTC 1 SC 25 WG 3 Interconnection of IT Equipment

www.j-and-m.com

WHY USE DATA CENTER TELECOM STANDARDS?

Point-to-Point Non-Standard CablingUnorganized, Single-Use

Structured Standards Based Cabling- Organized, Reusable, Flexible

Standard/Structured vs. Proprietary/Point-to-Point Cabling

• Better availability• Saves money• Faster deployment • Support for multiple current &

future protocols

What are the standards?

• ANSI/TIA-942-A Telecommunications Infrastructure Standard for Data Centers (US, approved April 2012, published Aug 2012; Addendum 1 approved Feb 2013, available now)

• CENELEC EN 50173-5 Information Technology – Generic Cabling Systems Part 5: Data Centres (2007, European Union). Addendum 2 approved for publication in 2012

• ISO/IEC 24764 Information Technology – Generic Cabling for Data Centre Premises (2010, international). Addendum 1 approved for publication in 2014

• ANSI/BICSI-002 Data Center Design and Implementation Best Practices (2011, international). Revision in progress (~2014).

New standards & updates• BICSI, ISO/IEC, CENELEC, and TIA are all very

busy revising and developing new standards • They work in conjunction with each other and

organizations such as IEEE

BICSI-002 Data Center Best Practices• ANSI/BICSI-002 Data Center Design and

Implementation Best Practices Standard• Developed by 150+ committee members worldwide

(mostly users, engineers, consultants)• Meant to supplement, not replace existing data center

cabling standards by ISO/IEC, CENENLEC, TIA, and others

• Under revision expected late 2014/early 2015• Large document (~400 pages) covering a wide range of

subjects - electrical, mechanical, architectural, fire protection, security, structural, commissioning, telecom, etc.

TIA-942-A-1 Data center fabrics• Switch fabrics provide high-bandwidth any-to-any

connectivity required for cloud computing• Fabrics can and should be built using the standards

based cabling • More cabling and higher bandwidth needed for

fabrics than traditional switch architecture

Traditional 3-Tier Fat-Tree / Leaf & SpineSwitch Fabric

Balanced Twisted Pair Cabling• Cat 5e & 6 for 1 Gigabit Ethernet to 100 m• Cat 6A, 7, or 7A for 10G Ethernet to 100 m• ISO/IEC requires and other standards

recommend Cat 6A minimum • Cat 8 in development for 40GBase-T up to 30 m

(adequate for 80% of server connections); 25Gbase-T also under consideration

• From Flatman_01a_1112 ngbt link lengths report to IEEE 802.3 NGBase-T study Group Nov 2012

Switch-to-Server Link Length % of data center links30 meters 80%

40 meters 90%

100 meters 100%

Optical Fiber Cabling• Laser optimized 50/125 um multimode fiber

(OM3 or OM4) less expensive than single-mode fiber for lengths <=150 m

• 40G uses 8 MM fibers and MPO connector• 100G uses 20 MM fibers and either two 12

fiber MPOs or one 24 fiber MPO connector• 4-lane/8-fiber 100G (Mar 2015) will cost less,

use less power, and is expected to support at least 70m over OM3, 100m over OM4.

Optical Fiber Cabling• 16-fiber and 32-fiber MPOs being developed

by TIA (TIA-604.18 FOCIS-18)• More efficient use of fibers for 40G & 4-lane

100G which use 8 fibers (12-fiber MPOs have 4-unused fibers)

• 400G Ethernet will probably use 32 multimode fibers

Summary• Standards-based structured cabling optimizes cost,

flexibility, and availability• Standards are continuously updated:

– CENELEC: European Union– TIA: US, Canada, and US-based companies outside US– ISO/IEC: most other countries

• BICSI standards are international best practices that work with other standards

• Consider future needs to support 10/40/100G and data center fabrics needed for clouds– Cat 6A, 7, or 7A for balanced-pair cable – OM4 for multimode fiber terminated on MPOs (with multiples of 8 fibers)

• New 4-lane 100G Ethernet (Mar 2015)• New Cat 8 / 40GBase-T (Feb 2016)

Thank youJonathan JewPresident J&M Consultants, Inc.Email: jew@j-and-m.com

Co-chair BICSI data center subcommittee – www.bicsi.orgCo-editor TIA-942-A, Editor TIA-942-A-1Chair TIA TR-42.6 telecom administration subcommittee Vice-Chair TIA TR-42.1 commercial building cablingEditor ISO/IEC TR 14763-2-1 telecom administration identifiersUS National Committee Project Manager ISO/IEC 24764 data

center standardData Center & Administration Section Editor –

ISO/IEC 14763-2 cabling planning & installation

Supplementary Information

Differences between standardsTIA ISO/IEC CENELEC

Balanced Pairbackbone

Cat 3 minCat 6A recommended

Cat 6A, 7, 7A Cat 6, 6A, 7, 7A

Balanced PairHorizontal

Cat 6 minCat 6A recommended

Cat 6A, 7, 7A Cat 6, 6A, 7, 7A

Multimode Fiber

OM3 minOM4 recommended

OM3 min OM2 min

Coaxial cable 734/735 coax None None

MMF connector LC for 1 or 2/portMPO for 3+/port

LC for 1 or 2/portMPO for 3+/port

LC or MPO

SMF connector at EDA

LC for 1 or 2/portMPO for 3+/port

LC for 1 or 2/portMPO for 3+/port

LC-PC

SMF connector at Entrance Rm

LC for 1 or 2/portMPO for 3+/port

Angled-LC Angled-LC

ISO/IEC Category 8• ISO/IEC TR 11801-99-1 – Balanced cabling for

40Gbps channels – (2014-2015) meant for IEEE• Characterizes 30 meter channels (26 m with 2x2m

patch cords) to support 40 Gbps using existing Cat 6A, 7, 7A, and new Cat 8.1 and Cat 8.2 components

• Cat 8.1 for Class I channels is backward compatible with Cat 6A

• Cat 8.2 for Class II channels compatible with Cat 7/7A• IEEE 40GBase-T target is 30 m for Cat 8, unknown for

Cat 6A, 7, 7A• Parallel effort in TIA underway

Ethernet channel lengths over multimode fiber

Fiber Type

1G 10G 40G 100G 100G

# fibers 2 2 8 20 8

OM1 275 m 26 m - - -

OM2 550 m 82 m - - -

OM3 800 m 300 m 100 m 100 m 70 m*

OM4 1040 m 550 m 150 m 150 m 100 m*

Distances in red are specified by manufacturers but not in IEEE standards. *IEEE 802.3bm Standard for 4-lane 100G target 1Q2015

BICSI-002 Subjects Covered• Site Selection• Architectural and Structural Design• Electrical Systems• Mechanical Systems (i.e., HVAC)• Fire Protection and Security• Building Automation Systems• Commissioning• Maintenance

BICSI-002 Subjects Covered• Telecommunications

– Access Providers & Entrance Facilities– Telecom Spaces– Cabinets & Racks– Cabling Pathways– Telecom Cabling– Field Testing– Telecom Administration

• Information Technology