the battle continues single-mode versus …...cisco global cloud index forecast and methodology 2015...
TRANSCRIPT
The Battle Continues
Single-Mode Versus Multimode Fibresin Data Centres
BICSICanberra, ACT
11 October, 2017
Aungwin Tin - General Manager Sales, AFL
Outline
• Why choose multimode or single-mode fibres for data centres?
– MMF (multimode fibre) concerns
– SMF (single-mode fibre) concerns
• Keeping up with data rate demands
– Using Standards
– Activity via MSA (multiple source agreement) consortiums
• Fibre in Enterprise and Hyperscale data centres
• Summary
Why Choose MMF for Data Centres?
MMF (multimode fibre) has presented a strong argument for lower CapEx and OpEx in Data Centers over SMF (single-mode fibre)
– Lower cost multimode transceivers using VCSEL (vertical cavity surface emitting laser) versus expensive single-mode transceivers using edge-emitting lasers (such as Fabry–Pérot)
– Large core diameter minimizes impact of core misalignment or dirt
– Lower MMF transceiver power consumption
– Capability to scale up to 100GbE on 150 metre link length
– Simple field terminations with mechanical or fusion splice on connectors
Concerns with MMF in Data Centres
Modal dispersion in MMF affects:
• Maximum distance and bit rate, for example OM4 limits:
– 10GBASE-SR 400 metres
– 40GBASE-SR4 150 metres
– 100GBASE-SR4 100 metres
• Loss budgets are reduced as bit rates are increased:
– 3.5 dB@1GbE
– 2.6 dB@ 10GbE
– 1.5 dB@40/100GbE
• Connector cleaning becomes more critical as loss budgets shrink
Source: AFL
Why Choose SMF for Data Centres?
SMF is not affected by modal dispersion, and has very low attentuation over distance
• SMF has virtually unlimited bit rates for data centredistances, only limited by transceivers
• SMF cabling trunks can be narrowed down to a single pair of fibres in most cases, reducing cable costs, patch panel costs, complexities, and increasing air flow
• SMF would not need to be replaced for the foreseeable future – future proofed
Concerns with SMF in Data Centres
• High SMF transceiver costs engineered for traditional long haul networks discourage SMF as a viable option
• Connector end face cleanliness is critical
– 1 micron dust particle can block up to 10% of the light, resulting in up to a 0.05 dB loss
Source: AFL
Increasing Data Bit Rate with MMF
• Increase bit rate per channel
– Decreases length
– Decreases link budget• 3.5 dB for 1 GbE
• 2.6 dB for 10 GbE
• Use more laneways with parallel optics
– More fibres
• Use more wavelengths e.g. SWDM (Short Wave Division Multiplexing) with OM5
Source: Mellanox Technologies
Source: Mellanox Technologies
• From 1 GbE
– OM3/OM4 VCSEL 1,000 metres
• To 10 GbE
– OM3 VCSEL 300 metres
– OM4 VCSEL 400 metres
Replace transceivers, existing OM3/OM4 fibre can remain (but OM2 fibre would need to be replaced)
Multimode Migration Path in Data Centres1 GbE to 10 GbE Migration Using 2 fibres
Tx Rx
Rx Tx
Duplex LC/SC
Multimode Migration Path in Data Centres10 GbE to 40 GbE (using 10 GbE laneways)
• To 40 GbE (4 x 10 GbE parallel optics)
– OM3 VCSEL 100 metres 8 fibres
– OM4 VCSEL 150 metres 8 fibres
Replace transceivers and cable assemblies (12 fibre MPO)
Tx Rx
Rx Tx
12 fibre MPO
• 100 GbE (10 x 10 GbE parallel optics) – 1st generation
– OM3 VCSEL 100 metres 20 fibres
– OM4 VCSEL 150 metres 20 fibres
Replace transceivers and cable assemblies
24 fibre MPO
Multimode Migration Path in Data Centres40 GbE to 100 GbE (using 10 GbE Laneways)
Tx Rx
TxRx
Tx Rx
TxRx
2 x 12 fibre MPO
Multimode Migration Path in Data Centers40 GbE to 100 GbE (using 25 GbE Laneways)
• 100 GbE (4 x 25 GbE parallel optics) – 2nd generation
– OM3 VCSEL 70 metres 8 fibres
– OM4 VCSEL 100 metres 8 fibres
Replace transceivers, but may not need to replace cable assemblies
Tx Rx
Rx Tx
12 fibre MPO
Standards Based Data Rate Migration Path to 400GbE on MMF
* Future
** OM5 WBMMF (wideband multimode fibre) ANSI/TIA-492AAAE .
Multimode Fibre Type
10GbE 40GbE 100GbE 400GbE*
OM1 33 m
OM2 82 m
OM3 300 m 100 m (SR4 4x10G) 100 m (SR10 10x10G)70 m (SR4 4 x 25G)
70 m (SR16: 16x25G)*
OM4 400 m 150 m (SR4 4x10G) 150 m (SR10 10x10G)100 m (SR4 4 x 25G)100 m (SR2 2 x 50G)*
100 m (SR16: 16x25G)*? (SR4: 4x100G)*
OM5 ** 400 m 150 m (SR4 4x10G) 150 m (SR10 10x10G)100 m (SR4 4 x 25G)
100 m (SR16: 16x25G)*? (SR4: 4x100G)*
IEEE 802.3 Link Distance (metres) Most likelyNot required
Change cable assemblies?
The Buzz About OM5
• Larger operating window between 850 to 955 nm
• SWDM (short wave division multiplexing) with 30 nm spacing (850, 880, 910, 940 nm)
• Requires fewer fibres
• Premium cost over OM4 (about 30% to 50%)
• No additional link length advantage over OM4
• SWDM does not support breakout like parallel optics:
• Unable to break out 100GbE into 25GbE, just as 25GbE optics to server market is maturing
• Recently developed
– 25G (June 2016)
– 100G (4 x 25GbE)
Ethernet Direction
Source: 2016 Ethernet Roadmap - Ethernet Alliance
• In development
– 50G
– 200G (4 x 50GbE)
– 400G (16 x 25GbE)
• Future
– 100G serial
– 400G (4 x 100GbE)
– 1T (10 x 100 GbE)
OM4 Distance Limit as Bit Rate Increase
Source: 2016 Ethernet Roadmap - Ethernet Alliance
As data rates approach 40/100GbE or 25GbE per channel, link distances on OM4 MMF are limited to 100 metres
Next step is SMF up to 2 km
Migration Path and Future Needs
• Migration path with MMF can involve changing
– Transceivers
– Cabling from OM2 to OM3 to OM4 to OM5• Possible change in connectors from duplex to MPO
– Still using MMF (albeit another one), but not quite seamless
• Larger “hyperscale” data centres emerge with larger footprints and need to scale quickly
– Link distances up to 500 metres and beyond
– Increasing data rates to Tbps (1012)?
– Not practical with MMF
Data Rate Challenges for
Hyperscale Data Centers
Cisco Global Cloud Index Forecast and Methodology 2015 - 2020
“Hyperscale data centers already account for 34 percent of total traffic within all data centers and will account for 53 percent by 2020. “
“By 2020, the total global installed data storage capacity in cloud data centers will account for 88 percent share of total DC storage capacity, up from 64.9 percent in 2015.”
Source: Data Center Frontier
Google Hyperscale data center, Council Bluffs, Iowa, USA
Other Observations from Cisco
Source: Cisco Global Cloud Index Forecast and Methodology 2015 - 2020
1 GB = 109 bytes
1 EB = 1018 bytes
Large volume of
East – West traffic
Growth rate Cloud
Data Rate Expectations from Hyperscale Data Centers
Seamless and rapid migration to higher data rates on existing cabling as new technologies come on line
• Single-mode fibre is only practical option for migration to Terabit/sec and link length requirements
• IEEE 802.3ba defines 100 GBASE-LR4 for up to 10 km on single-mode fibre
• Several MSA (Multiple Source Agreements) consortiums have formed to address some of the gap - non standards approach
Applying Long Range Single-Mode Fibre Technology to Short Range Applications
• MSA (multiple source agreements) companies have taken the long range (10+ km and 40+ km) single-mode systems and adapted to short range data centre needs (up to 2 km)
• MSAs have generated enough volume from a small group of large “hyperscale cloud operator” such as:– eBay
– AWS
– Oracle
– Salesforce
Key Parameter
100 G PSM-4 MSA CWDM4 MSA 100G CLR4 Alliance Open Optics MSA
Reach 500 m 2 km 2 km 2 km
Fibre Single-mode Single-mode Single-mode Single-mode
Connector MPO 12 Duplex LC Duplex LC Duplex LC
Operating Window
1310 nm 1310 nm 1310 nm 1550 nm
100GbE 4 x 25GbE parallel 4 x 25GbE CWDM * 4 x 25GbE CWDM **4 x 25GbE DWDM
Supporters Component vendors, system vendorsMicrosoft
Component vendors, system vendorsFacebook developedCWDM4 OCP (Open Compute Project)
Component vendors, system vendorsebay, Dell, Oracle
Component vendors, system vendors
** Up to 32 channels* Non-FEC option
for latency
sensitive
applications
MSAs Approach to 100GbE
100 G PSM-4
• Parallel optics
• Uses 8 single-mode fibres: – 4 fibres for Tx and 4 fibres for Rx
• MPO connectors
• 4 channels x 25 GbE = 100 GbE
Tx Rx
Rx Tx
CWDM4, 100G CLR4, Open Optics
• WDM
• Uses 2 single-mode fibres: Tx and Rx
• Typically LC connectors
• 4 wavelengths 25 GbE each = 100 GbE
LC4 x 25G
Single-mode fibre
LC
Single-mode fibres
LC LC4 x 25G
Single-mode fibre
25 G
25 G
25 G
25 G
MUX
Tx
25 G
25 G
25 G
25 G
DeMUX
Rx
DeMUX
Rx
MUX
Tx
MPO MPO
25 G
25 G
25 G
25 G
25 G
25 G
25 G
25 G
MSAs Approach to 100GbE
The Open Compute Project
• In 2009, Facebook started a project “to design the world’smost energy efficient data centre, one that could handleunprecedented scale at lowest possible cost.”
• In 2011, Facebook created the Open Compute Project toshare its designs with the public and to spur on additionalcollaboration and advancement covering 9 areas:
• Rack and Power
• Server
• Storage
• Telco
• Compliance and Interoperability
• Data Center
• Hardware Management
• HPS (High Performance Computing)
• Networking
The Open Compute Members (partial list)
Facebook’s Approach to “Networking”
Facebook did their own analysis and drove theirrequirements through the Open Compute Project
Katharine Schmidtke, Head of Optical Technology Strategy, Facebook
“The analysis showed that the cost saving in using less fiber andpatch panels could more than offset the increased cost of the single-mode optical transceivers.
We saw an opportunity to reduce that cost even further byoptimizing the specification to fit data center requirements andbenefit from innovation in new technologies.”
Facebook’s Collaboration and Results
Used CWDM4 MSA as starting point and created CWDM4 OCP, withkey changes:• Reduced 2 km to 500 metres
• Relaxed specification operating specification from 0-70 deg C to 15-55 deg C(as data centres are temperature controlled)
• Removed active cooling and hermetic packaging to withstand potentialprolonged harsh environmental conditions
• Reduced transceiver life from 20 year to 5 years
With advancements in silicon photonics, developed low cost 100G transceiver, shown with “green” to distinguish from other versions.
25 Gigabit Ethernet Consortium
• Formed July 2014 by founding members above specifically for data centres
• 25GbE approved by IEEE in June 2016
• Consortium focused development of 25GbE instead of 10GbE
• Allows scale at 2.5 times the “standard” 10GbE, but at similar costs
• Changes migration path to 10GbE –> 25GbE –>100 GbE
Fibre in Enterprise vs Hyperscale Data Centers
Enterprise
• 1GbE to 10GbE to 40GbE
• Up to 150 metres
• Multimode meets most needs
• Parallel optics to meet increased bit rates
– MPO connectors
• SWDM with OM5 presents new growth path
Hyperscale
• 25GbE to 100GbE and beyond
• 500 metres to 2 km
• Single-mode meets the current needs, and can meet future requirements
• Increase serial speed to 100GbE and parallel speed to 1TbE
– MPO
• CWDM and DWDM
– Duplex LCs
Multimode vs single-mode fibre cabling is dependent on the type of data centre, link lengths, and expected bit rates
Summary
• Selecting between MMF and SMF depends on distance, data bit rate, overall system costs, and future direction
• Enterprise data centres primarily use multimode systems
• Hyperscale data centres (requiring longer distance and higher data bit rates) primarily use single-mode systems
• Prices continue to decline for single-mode photonics, fine tuned for data centre applications
– Expect increase in SMF in Enterprise data centres to solve data rate and distance challenges
• Enterprise data centres may choose to install SMF now to future proof their networks
Contact Information
Aungwin Tin
General Manager, Sales
Australia and New Zealand
AFL
Mobile: +61 400 598 089
93-97 Merrindale Drive 100 Olympia StreetCroydon South, VIC 3136 Tottenham, VIC 3012Australia Australia