the development of 40 gbe and 100 gbe: an update on ieee p802

Copyright © 2009 Force10 Networks, Inc. All rights reserved.

The Development of 40 GbE and 100 GbE: An Update on IEEE P802.3ba

John D’AmbrosiaChair, IEEE P802.3ba Task Force Scientist Sr., Components Technology

Force10 Networks


Per IEEE-SA Standards Board Operations Manual, January 2005

At lectures, symposia, seminars, or educational courses, an individual presenting information on IEEE standards shall make it clear that his or her views should be considered the personal views of that individual rather than the formal position, explanation, or interpretation of the IEEE.


Why Higher Speed Ethernet?

Fundamental bottlenecks are happening everywhereIncreased #

of usersIncreased

accessrates and methods

Increased services++ =Bandwidth

explosioneverywhere

As demonstrated by the number of ISPs: Comcast, AOL, YahooBB, NTT, Cox, EasyNet, Rogers, BT, ...

EFM, xDSL, WiMax, xPON, Cable, WiFi,3G/4G…

YouTube, BitTorrent, VOD, Facebook, Kazaa, Netflix, iTunes, 2nd

life, Gaming…

Source HSSG Tutorial, Nov 07


The Ethernet Ecosystem

Research, Education

and Government Facilities

ResearchNetworks

Broadband Access

BroadbandAccess Networks

Data Centers and Enterprise

EnterpriseNetworks

Content Providers

ContentNetworks

Internet BackboneNetworks


Internet eXchange andInterconnection Points



40GbE and 100GbE: Computing and Networking

100

1,000

10,000

100,000

1,000,000

1995 2000 2005 2010 2015 2020Date

Rat

e M

b/s

CoreNetworkingDoubling≈18 mos

ServerI/O

Doubling≈24 mos

Gigabit Ethernet

10 Gigabit Ethernet

100 Gigabit Ethernet

40 Gigabit Ethernet


Copyright © 2009 Force10 Networks, Inc. All rights reserved.Copyright © 2009 Force10 Networks, Inc. All rights reserved.

Industry Efforts

IEEE P802.3ba 40 Gb/s and 100 Gb/sEthernet

Ethernet Alliance: HSE Marketing / Technical SubcommitteesOIF – CEI-25G (Backplane)– CEI-28G (Chip-to-chip / module)

ITU-T Study Group 15– Transcoding (40 Gb/s Ethernet)– Development of 100G serial solution

Industry MSA Efforts


IEEE P802.3ba Objectives

Support full-duplex operation onlyPreserve the 802.3 / Ethernet frame format utilizing the 802.3 MACPreserve minimum and maximum FrameSize of current 802.3 standardSupport a BER better than or equal to 10-12 at the MAC/PLS service interfaceProvide appropriate support for OTNSupport a MAC data rate of 40 Gb/sProvide Physical Layer specifications which support 40 Gb/s operation over:

– at least 10km on SMF– at least 100m on OM3 MMF– at least 10m over a copper cable assembly– at least 1m over a backplane

Support a MAC data rate of 100 Gb/sProvide Physical Layer specifications which support 100 Gb/s operation over:

– at least 40km on SMF– at least 10km on SMF– at least 100m on OM3 MMF– at least 10m over a copper cable assembly

Adopted by IEEE P802.3ba and approved by 802.3 at Mar 2008 Plenary


The Dilemma

Challenges:– Multiple rates – 40 Gb/s and 100 Gb/s– Multiple physical layer specifications– Multiple possible solutions

– 40 Gb/s: 4 x 10 Gb/s, 2 x 20 Gb/s, 1 x 40 Gb/s– 100 Gb/s: 10 x 10 Gb/s, 5 x 20 Gb/s, 4 x 25Gb/s, 2

x 50 Gb/s, 1 x 100 Gb/s– Technology development / maturity for electrical and

optical signaling– Market Need – Cost Targets

Problem Statement: Develop an architecture that can enable implementations for today and tomorrow


Overview of Architecture

Consistent with previous Ethernet rates, extension to 40Gb/s & 100Gb/s data rates– Frame format; Services; Management

attributesMedia Access Control (MAC)– No changes to the MAC operation

Physical Coding Sublayer (PCS)Physical Medium Attachment Sublayer (PMA)Physical Medium Dependent Sublayer (PMD)Interface DefinitionsProvide appropriate support for OTN

PCS

PMA

PMD

LLC

MACReconciliation

Medium

Generalized LANCSMA/CD Layers


Physical Coding Sublayer (PCS)

10GBASE-R 64B/66B based PCS– Run at 100Gbps or 40Gbps serial rate– Includes 66 bit block encoding and scrambling

Multi-Lane Distribution– Data is distributed across “n” PCS lanes 66 bit blocks at a time– Round robin distribution– Periodic alignment blocks are added to each virtual lane to allow deskew in the

rx PCS

Alignment and static skew compensation is done in the Rx PCS only

Based on: gustlin_01_0508.pdf

Example – 100GbECSMA/CD Layers

Simple 66b word round robin

PCS Lane 1#1#n+1#2n+1


PCS Lane n#n#2n#3n M

M1

M2

Mn

Lane markers

Aggregate Stream of 64/66b words

= 66-bit word

M

M1

M2

Mn

#1#2#n#n+1#n+2#2n+1 #2n

Simple 66b word round robin



PCS Lane n#n#2n#3n M

M1

M2

Mn

Lane markers

Aggregate Stream of 64/66b words

= 66-bit word

M

M1

M2

Mn

#1#2#n#n+1#n+2#2n+1 #2n

Source: D’Ambrosia, Law, Nowell, “40 Gigabit Ethernet and 100 Gigabit Ethernet Technology Overview,” Ethernet Alliance White Paper, http://www.ethernetalliance.org/images/40G_100G_Tech_overview(2).pdf. , November 2008.


Physical Medium Attachment (PMA)

Parameterized PMA

Source: gustlin_01_0508.pdf

100GbECSMA/CD Layers

40GbECSMA/CD Layers

Enables multiple physical layer specifications– 10 x10 Gb/s– 4 x 25 Gb/s– 2 x 50 Gb/s– 1 x 100 Gb/s


Appropriate Support for OTN

Key Elements– Lane Independent PCS– 40 GbE must fit into OPU3 payload

– Transcoding (to be specified by ITU-T SG15)

– Coordination between ITU-T SG15 and IEEE on control block types.

– Lane marker transparency for 40 GbE– ITU-T decision, but maintain spare

value of control block types for encoding lane markers

– 100 GbE may be carried over current OTN networks using virtual concatenation:

– ODU3-3v, 3 bonded wavelengths of 40Gbit/s

– ODU2-11v, 11 bonded wavelengths of 10Gbit/s

– Link fault signaling for 802.3ba Ethernet over OTN can use mechanisms in Clause 46

Based on: trowbridge_01_0508.pdf

ResearchNetworks

BroadbandAccess

Networks

EnterpriseNetworks

ContentNetworks



ResearchNetworksResearchNetworks

BroadbandAccess

Networks

EnterpriseNetworksEnterpriseNetworks

ContentNetworksContent

Networks



10 Gbps λ DWDM

40 Gbps λ DWDM

Ethernet Ecosystem

Future –100 Gbps λ DWDM


Nomenclature

40 = 40Gb/s100 = 100Gb/s

Speed

Prefix Suffix

n=1 is not required as serial is implied

n = 4 or 10Copper

K = Backplane C = Cable Assembly

Coppern = Number of Lanes or Wavelengths

R = 64B/66B Block Coding

S = Short Reach (100m)L = Long Reach (10km)E = Extended Long Reach (40km)

OpticalOptical

LanesCoding Scheme

Medium


Summary: Physical Layer Specifications

100GBASE-ER4

100GBASE-LR4

40GBASE-LR4

40GBASE-SR4100GBASE-SR10

40GBASE-CR4100GBASE-CR10

40GBASE-KR4

Port Type

“n” x 10 Gb/sRe-use of 10GBASE-KR

√√At least 10m cu (twin-ax) cable

4 x 10 Gb/s√At least 10km SMF



“n” x 10 Gb/s√√At least 100m OM3 MMF

100GbE

4 x 10 Gb/sRe-use of 10GBASE-KR

√At least 1m backplane

40GbE Solution SpaceDescription


IEEE P802.3ba Task Force Timeline

20092008

J M M J S N J M M J S N J M M JN

Task ForceFormation

TFReview

WGBallot

LMSCBallot

Standard

TF Reviews WG Ballots LMSC Ballots

2010

D F A J A O D J M J S NF A J A O D M JM JF A J

Proposal Selection

YouAreHere


Higher Speeds Drive Density –Everyone Benefits!

You may not need 100Gigabit Ethernet today, but the Eco-System does….Even if you don’t need 100 Gigabit Ethernet or (40 Gigabit Ethernet), you still benefitNew technologies will drive 10 GbE port density up and cost down– Just as 10 GbE did for GbE

Assuming switch/routers have the switching capacity, these line-rate combinations on a single line card are possible for 100 GbE– 1 x 100 GbE port– 10 x 10 GbE ports– 100 x 1 GbE ports– And even more oversubscribed port density…


Industry System Port Count Cycle

2002 2004 2006 2010*2008

100 GE Standard In Development

GE 100’s Ports > 1000 Ports

10 GE 10’s Ports > 100’s Ports100’s Ports

10’s Ports


Summary

Baseline Proposals for all objectives adopted

Task Force Review Phase Goal – Goal: Generate technically complete document

Key Milestones to come– Working Group Ballot – March 2009– Sponsor Ballot – Nov 2009– Standard approved – June 2010

An architecture is in place for the next jump in speed!Ethernet continues to evolve to meet the needs of the industry


John’s Predictions for Future

A 40G serial optical PMD will be developedA duplex fiber OM3 MMF PMD will be developed ≈25G signaling will be used for 100G Backplane and Cu Cable Assembly SolutionsA 100G serial optical PMD will be developedSomething beyond 10GBASE-TTerabit Ethernet


Recent Data from AMS-IX

582 Gb/s

Data provided by Henk Steenman, AMS-IX


Thank You

the development of 40 gbe and 100 gbe: an update on ieee p802

Documents