10 gigabit ethernet - hsn

10 10 Gigabit EthernetGigabit Ethernet

Sungsoo Kang

Network Technology DivisionETRI

2ETRI Proprietary

차차 례례

Background & reviewFrom 10Mb/s to 1 Gb/s Ethernet

10GbEBackground & activityTechnical detail

10GbE prospective

3ETRI Proprietary

Ethernet history Ethernet history -- originoriginALOHA

Based on contention of broadcast (air) channelPure ALOHASlotted ALOHA

Pack

et PacketPack

etPacket

4ETRI Proprietary

Pure ALOHA collisionPure ALOHA collision

Extremely inefficient, since the worst-case period of vulnerability is the time to transmit two frames.

Collides withthe start ofthe shaded

frame

Collides withthe end ofthe shaded

frame

Vulnerabletimet0 + 3tt0 + 2tt0 + tt0

t

5ETRI Proprietary

Robert Metcalf’s Idea – borrow ALOHA’s idea to LANWith adding collision detection by carrier sensing

Now, let see MAC protocol

• Invented by Metcalf at Xerox in 1973 and patented in 1976

• Xerox, Digital and Intel jointly made DIX.xx

• IEEE standard in 1989

Invention of EthernetInvention of Ethernet

of the first Ethernet designof the first Ethernet design

6ETRI Proprietary

Ethernet History: CSMA/CDEthernet History: CSMA/CD

Take Slotted ALOHA to the next level, use the slots as “contention periods”.

If no collision occurs before the end of the period, then complete transmission of the frame.

CSMA/CD can be in one of three states: contention, transmission, or idleVariable length of packet

FrameFrameFrame Frame

transmissionperiod

contentionperiod

idleperiod

contentionslots

time

t0

7ETRI Proprietary

Ethernet LAN Ethernet LAN 발전발전

1973

1980

1990

2000

2002

10 Mbps

100 Mbps

1,000 Mbps

10,000 Mbps

Xerox Ethernet2.94 Mbps

Ethernet10BASE5(80) : Thick Coaxial802.3a&b(85) : Thin Coaxial 802.3i(90) : UTP3(or better)802.3j(93) : Fiber

Fast Ethernet802.3u(95) : UTP3&5, Fiber 802.3y(97) : UTP3

Gigabit Ethernet802.3z(98) : Fiber, STP802.3ab(99) : UTP5

10Gigabit Ethernet802.3ae(02) : Fiber Only

Ethernet Standards Approval Timeline

1995

1998

FDX/

Flow

con

trol

,A

uto-

nego

tiatio

n

Car

rier 확장

,Pa

cket

bur

stin

g

FDX

only

8ETRI Proprietary

Change of Market DriversChange of Market Drivers

6000

5000

4000

3000

2000

1000

0

voice data97 98 99 00 02 03 04 05 06 0701 08

Global Voice / Data Crossover

Gbp

s

Source: A.D. Little Source: The Yankee Group

100

50

200019991998 2001 2002B

illio

ns o

f Dol

lars

150

200

0

Voice and Data Revenue Growth

voice data

Source: NGN Ventures 2000

9ETRI Proprietary

Circuit traffic : increase 8~10 % per yearPacket traffic : increase 200 % per yearHow do we efficiently collect traffic from Access and Metro Area Networks ??

10 Gig Ethernet might be best candidate

Issues Issues ……

10ETRI Proprietary

Gigabit BW vs. ASPGigabit BW vs. ASP

Ethernet이 SONET 보다 85%정도 저렴할 것으로 예상− 10Gigabit Ethernet : $950(2001), $485(2004)− OC-192 SONET : $7,653(2001), $3,143(2004)

ASP(Average Selling Price), Normalized to 1Gbps Service.

Ethernet SONET

Source : Dell’Oro Group

11ETRI Proprietary

What is 10Gig Ethernet?What is 10Gig Ethernet?

Most simple way to expand enterprise & service provider’s networkCan use already installed more than 300 million Ethernet switch portSupport all data serviceSupport not only LAN but also MAN, WANSimple, cheap and fast comparing to competitive technologiesMatch to MAN/WAN backbone speed(OC-192)Support all link without protocol conversion !!

12ETRI Proprietary

802.3802.3ae. schedule & plausible scenarioae. schedule & plausible scenario

1999 2000 2001 2002

HSSGformed

PARdraft

PARapproved

802.3aeformed

WGballot

Firstdraft

LMSGballot standard

Dra

ft 1

Dra

ft 2

Dra

ft 3

Dra

ft 4

Dra

ft 5

10 GbE Standard draft near complete.Vendors build pre-standardproducts based on draft

Commodity10 GbETransceiversHit the market

Commodity10 GbE Solutions

in market

2003

Committee is waiting for enough prototypes

13ETRI Proprietary

IEEE P802.3ae ObjectivesIEEE P802.3ae Objectives

Preserve 802.3 Ethernet frame formatPreserve 802.3 min/max frame sizeFull duplex operation onlyFiber cabling only10.0 Gbps at MAC-PHY interfaceLAN PHY data rate of 10 GbpsWAN PHY data rate of ~9.29 Gbps

14ETRI Proprietary

10GBASE-W is subset of 10GBASE-R

10GBASE-X− 10GBASE-LX4

10GBASE-R− 10GBASE-SR− 10GBASE-LR− 10GbASE-ER

10GBAE-W− 10GBASE-SW− 10GBASE-LW− 10GBASE-EW

1010Gig Ethernet Reference ModelGig Ethernet Reference Model

LLC-LOGICAL LINK CONTROL

MAC CONTROL (OPTIONAL)

MAC-MEDIA ACCESS CONTROL

RECONCILIATION

64B/66B PCS

WIS

PMA

PMD

64B/66B PCS

PMA

PMD

8B/10B PCS

PMA

PMD

XGMIIXGMIIXGMII

MEDIUMMEDIUM MEDIUM

MDIMDIMDI

10BASE-W 10BASE-X10BASE-R

PHYSICAL

DATA LINK

NETWORK

TRANSPORT

SESSION

PRESENTATION

APPLICATION

OSI

Ref.

model

Higher layers

MDI = Medium Dependent Interface PMD = Physical Medium Dependent

PCS = Physical Coding Sub-layer WIS = WAN Interface Sub-layer

PHY = Physical Layer Device XGMII = 10 Gigabit Media Independent Interface

PMA = Physical Medium Attachment

PHYPHY

15ETRI Proprietary

802.3802.3ae/z/ab Distance Chartae/z/ab Distance Chart

Campusbackbones

Buildingbackbones

Wiringcloset

1000BaseCXCopper

9 µm SMF

50 µm / 62.5 µm MMF

50 µm MMF

62.5 µm MMF

4 pairUTP 5

1000BaseLX~ 1300 nm

1000BaseSX~ 850 nm

1000BaseT1999 (802.3ab)

25 m 275 m 550 m 5 km

GbE

300 m65 m

MAN / WAN

> 40 km

62.5 µm MMF, WWDM

50 µm MMF, serial

1310 nm

850 nm

9 µm SMF, serial

9 µm SMF, serial & WWDM

1550 nm

1310 nm

> 10 km

10 G

bE

16ETRI Proprietary

Nomenclature Nomenclature && Clause(2)Clause(2)

Nomenclature

clause

48

8b/10bPCS &PMA

10GBASE-SR

10GBASE-SW

10GBASE-LX4

10GBASE-LR

10GBASE-LW

10GBASE-ER

10GBASE-EW

49

64b/66bPCS

50

WIS

51

SerialPMA

54

1310 nmWWDMPMA

850 nmserialPMA

52

1310 nmserialPMA

1550 nmserialPMA

Ref: IEEE Draft P802.3ae/D 2.3, March 2001

17ETRI Proprietary

10 10 Gig Ethernet PHYGig Ethernet PHY

PMA – Physical Media AttachmentSerialization/Deserialization(SerDes)Octet delineation[SONET framing][Scrambling function for SONET frame (x7 + x6

+ 1)]PCS – Physical Coding Sublayer

MAC packet delineation & Header control64B/66B CodingScrambling function for data (x43 + 1)

PMD – Physical Media Dependent interfaces

OpticsClock Recovery

MAC

PHY

PMD

PMA

PCS

18ETRI Proprietary

1010GBASEGBASE--XX

Ethernet Packet + min . IPG

8b

10b

8b/10bencoder

SERDES

8b

10b

8b/10bencoder

SERDES

8b

10b

8b/10bencoder

SERDES

8b

10b

8b/10bencoder

SERDESPMA

8b/10b PCS

MAC 10 Gbps

XGMII

10 Gbps

12.5 Gbps,4 @ 3.125 Gbps

12.5 Gbps,4 @ 3.125 Gbps

19ETRI Proprietary


SERDESPMA

64b/66b PCS

MAC 10 Gbps

XGMII or XAUI

10 Gbps

10.3 Gbps

10.3 Gbps

XSBI

64 bit scrambler

64bSync. Bits (2)

1010GBASEGBASE--RR

64b 64b 64b 64b64b 64b

20ETRI Proprietary

PMA

64b/66b PCS

MAC 10 Gbps

XGMII or XAUI9.29 Gbps

9.58 Gbps

9.95 Gbps

XSBI

Extra IPG

Extra IPG dumpedExtra IPG dumped

simplified SONET framersimplified SONET framerWIS 9.95 Gbps

1010GBASEGBASE--W(1)W(1)


SERDES

64 bit scrambler

64b 64b 64b 64b64b 64b

64bSync. Bits (2)

21ETRI Proprietary

RSOH

MSOH

Transport

Overhead

(STS-192c) Envelope Capacity

IDLE packetPCS data stream

Path Overheadcolumn

WIS Frame = STS-192c Frame

9 ro

ws

576 octets 17280 octets

(STS-192c) SPE

116704 octets

1664063

9 ro

ws

Payload Capacity

IDLE packet

IDLE

IDLE

Fixedstuff

1010GBASEGBASE--W(2)W(2)WIS is to allow to 10GBASE-W equipment to generate Ethernet data streams that may be mapped directly to STS-192c or VC-4-64c streams at the PHY levels, without requiring MAC or higher-layer processing.

STS-192c = Synchronous Transport Signal level 192, c=concatenatedSPE=Synchronous Payload Envelope

22ETRI Proprietary

Structure of SONET OverheadStructure of SONET Overhead

pointer

MSOH

RSOH

POH

J1

B3

C2

G1

F2

H4

Z3

Z4

Z5

E1 F1

D1 D2

B2 K1

S1 M1

J0A1 A2 Z0A1 A1 A2A2

B1 ∆ ∆ ∆ X X

∆ ∆ ∆ D3

H1 H1 H1 H2 H2 H2 H3 H3 H3

B2 B2 K2

D6D5D4

D9D8D7

D12D11D10

E2 ∆∆

Z0A1 …

∆ …

∆ …

B2 …

H1 …

D4

D7

D10

A2 …

∆ …

∆ …

…

H2 …

D5

D8

D11

Z0 …

∆ …

∆ …

…

H3 …

D5

D8

D11

321 193 385 576

23ETRI Proprietary

WIS for SONET OHWIS for SONET OH

Only minimum SOH process for OC-192c interfacePointer process (H1-H2, H3)Automatic protection (K1,K2)Framing & regenerator (A1, A2, J0, Z0)Error check, Mux error monitor & remote error indicator (B1, B2, M1, S1)Others : un-supported (DCC, order wire, sync. etc.)

Only 1349 octets are defined in WIStotal:5184 octets Overhead(SOH, LOH) are defined in SONET

Also minimum POH process for OC-192c interfaceSTS path signal label (C2)STS path trace (J1)Others : un-supported (H4, F2, Z3, … etc.)

24ETRI Proprietary

1010GBASEGBASE--W is SONET FriendlyW is SONET Friendly

SONET friendly does NOT mean SONET compliant…

SONET frame (bits) are SONET compliantNo layer 2 bridging requiredOverhead will be interoperable with existing equipment

Does NOT meet SONET jitter requirementsmatch the ITU Grid

Does NEED a PHYSICAL layer conversion

25ETRI Proprietary

Hardware implementation issueHardware implementation issue

XGMII is interface between MAC and PHYShort reach interface: XGMII is ~ 7 cmNeed to extend reach for many 10GbE port

Reconciliation sublayer PHY

Station management

XGMII = 10 Gigabit Media Independent Interface

36

1

XGMII signals

(serial 2 wire interface)

TXC (transmit clock)

TXD (32 data bits, 4 control bits)

RXC (receive clock)

RXD (32 data bits, 4 control bits)

MDIO

MDC

36

1

1

1

26ETRI Proprietary

Extending reach for implementationExtending reach for implementation

LLC-Logical Link Control

MEDIUM

Physical Medium Dependent (PMD)

Physical Coding Sublayer (PCS)

Physical Medium Attachment (PMA)

MAC Control (Optional)

XGMII

XAUIXGXS

XGXS

Reconciliation

Media Access Control (MAC)

XGMII

PHYSICAL

DATA LINK

NETWORK

TRANSPORT

SESSION

PRESENTATION

APPLICATION

OSI

Ref.

modelOptional XGMIIeXtender

PHY

Higher Layer

27ETRI Proprietary

InterfacesInterfaces

XGMII (10G Media Independent I/F)4 byte-wide lanes with 1 control bit per lane

XAUI (10G Attachment Unit I/F)Extends XGMII reach (3” vs. 20”)4 differential lanes at 3.125 Gbps

XSBI (10G Sixteen-Bit Interface)Based on the OIF SFI-4 interface16 differential signals at 622-645 Mbps

28ETRI Proprietary

XAUIXAUI

XAUI : 10 Gigabit eXtended Attachment Unit InterfaceXGXS : XGMII eXtender SublayerIncreased reach

XGMII is ~ 7 cmXAUI is ~ 50 cm

Lower connection countXGMII is 74 wires (2 sets of 32 data, 4 control & 1 clock)XAUI is 16 wires (2 sets of 4 differential pairs)

3.125 Gbaud, 8B/10B encoded over 50cm FR-4 PCB tracesCDR-based, 4-lane serial, self-timed interfaceMay be implemented in CMOS, BiCMOS, SiGeDirect mapping of RS/XGMII data to/from PCS

http:// grouper.ieee.org/groups/802/3/ae/public/may00/taborek_2_0500.pdf

29ETRI Proprietary

XGXS functionXGXS function

8B/10B transmission codeColumn stripping across 4 independent serial lanes

Identified as lane 0, lane 1, lane 2, lane 3Perform XAUI lane and interface(lane) synchronizationPerform lane-to-lane deskewProvide packet delimitersPerform error control to prevent error propagation

30ETRI Proprietary

Issues on Optical moduleIssues on Optical module

10 GbE PMD and distances

300 meter, 850 nm – VCSEL

2 Km, 1310 nm – Fabry-Perot laser, singlemode

10 Km, 1310 nm – DFB laser or VCSEL, singlemode

40 Km, 1310 nm – DFB laser, cooled, singlemode

40 Km, 1550 nm – DFB laser, singlemode

Source: Business Communication Review, April,2000

31ETRI Proprietary

Two type of LASERTwo type of LASER

SIOAlGaAsGaAs MQWAlGaAs

p-n junction boxbase oftransistorcan

Mounting postLaser beamoutput

Semiconductorsample(GaAs)

fine wirecontact

Conventional LD

Gainregion

Oxidelayer

current

Top layer(99.9% reflective)

Bottom layer(99.9% reflective)

Laser cavity(length)

VCSEL

32ETRI Proprietary

Prospective on VSCELProspective on VSCEL

Comparison with Edge-emit LDVertical : longitudinal structureCircular : elliptical coupling efficiency 80% : 10%Low threshold currents (typically less than 1 mA)Very stable over temperature performanceYield improvement

On wafer : after wafer & individually

Trends850 nm commercially available now1310 nm just announced ~ 10 Km with range1550 nm by end of 2001, be announced

33ETRI Proprietary

Optical Fibers

Interface +Tx/Rx

Electronics

XGP Transceiver

4Lanes

FromUpper

Layer IC

ToUpper

Layer IC

O/E

4Lanes

..

..

..

..

..

..

Interface +Tx/Rx

Electronics

XGP Transceiver

4Lanes

ToUpper

Layer IC

FromUpper

Layer IC

O/E

4Lanes

..

..

..

..

..

..

transceiver

Connector plugCage assembly

Bezel

Connector Receptacle

XGP MSA GroupXGP MSA Group

Muti Source Agreement(MSA)SFP(Small Form-Factor Pluggable)

14 Participating Transceiver CompaniesTarget IEEE 802.3ae, other 10Giga standardSupport 4 lane electrical interfacewww.xgpmsa.org

34ETRI Proprietary

GbEGbE: Typical Implementation: Typical Implementation

Logical Link Control

Physical Medium Dependent

Physical Coding SublayerPhysical Medium Attachment

MAC Control (Opt)

(XG)MII

Reconciliation SublayerMedia Access Control

Media

802.3 Layer Model

MDI

35ETRI Proprietary

Typical Gigabit Optical Typical Gigabit Optical XCVRsXCVRs

1x9 GBIC

SFF

SFPPluggable

Pin in Hole

36ETRI Proprietary

Early 10 Gigabit Optical XCVRSEarly 10 Gigabit Optical XCVRS

XGXSFTRX

XENPACK

37ETRI Proprietary

Summary of 10 Gigabit EthernetSummary of 10 Gigabit Ethernet

MACIt’s just Ethernet

Maintains 802.3 Frame Format and SizeFull Duplex operation only

PHYLAN PHY uses simple encoding mechanisms to transmit data on dark fiber & dark wavelengthsWAN PHY adds a SONET framing sublayer to enable transmission of Ethernet on SONET transport infrastructure

PMDSupport distances from 65m on installed MMF to 40km on SMF

No copper solution proposed

38ETRI Proprietary

10 10 GbE in the LANGbE in the LAN

10 GbE in service providerdata centers & enterprise LANsSwitch to switchSwitch to serverData centersBetween buildingsServer

Farm

10GbE

Data Center

Campus ACampus B

10GbE

10GbE10GbE

InternetExtranet

10 GbE SMF

10 GbE MMF100 to 300 m

2 to 40 Km2 to 40 Km2 to 40 Km

39ETRI Proprietary

Location B

Location A

10GbE

10GbE

MetroMetro

Remote Servers

Location C

10GbE

10GbE

10GbE

10GbE

10 10 GbE in the MANGbE in the MAN

MetropolitanNetworks

Enterprises:Enables dark fiber to become a 10 Gbps fat pipe for metro networks

Service Providers: enables Gigabit services at costs less than T3 or OC-3

40ETRI Proprietary

10 10 GbE in the MAN over DWDMGbE in the MAN over DWDM

Enterprises:10 GbE enables serverless buildings,remote backup, disaster recovery

Service Providers: 10 GbE enables dark wavelengthGigabit services at costs less than T3 or OC-3MAN DWDM Optical

Network

Location B

Location A

10GbE

Remote Servers

Location C

10GbE

10GbE

DWDM mux

Distance: 100 kmDistance: 100 km

41ETRI Proprietary

Carrier DWDM device collocated with SP 10 GbE Switch

Core DWDM Core DWDM Optical NetworkOptical Network

10GbE

Service Provider Point of Presence

(PoP)CarrierCentral

Office (CO)

10GbEOpticalTransport

OpticalTransport

CarrierCentral

Office (CO)

Service Provider Point of Presence

(PoP)

10 10 GbE in the WANGbE in the WAN

Attachment to the optical cloud with WAN physical layerCompatibility with the installed base of SONET OC-192 Interface & link between SP to IXC network can be co-locatedNo need for protocol conversion, traffic remains IP/Ethernet

NationalBackbone

42ETRI Proprietary

Issues for 10GbEIssues for 10GbE

WAN interconnection

Market issue

Industry issue

What’s our next step ?

43ETRI Proprietary

‘‘PathPath’’, , ‘‘LineLine’’, , ‘‘SectionSection’’

Line TerminatingEquipment

(LTE)

Section Section

Line

Path(s)

Stratum Clock

Regenerator(STE)

LocalClock

LocalClock

Note: A Line can be longer than two Sections

Path TerminatingEquipment

(PTE)


(PTE)


(PTE)


(PTE)Path

TerminatingEquipment

(PTE)


(PTE)


(PTE)


(PTE)

Line TerminatingEquipment

(LTE)

44ETRI Proprietary

Layer Diagram of ELTELayer Diagram of ELTE

Layer 1 Path Relay

SONETPhysical Media Adaptation

SONETSection Termination

SONETLine Termination

802.3 Link Segment SONET OC-192c

Ethernet Line Terminating Equipment (ELTE)

The client-side link (private network) uses Ethernet optics and implements part of the WAN Interface Sublayer (terminates section and line overhead)

The system-side link (to public network) contains an OC-192c or OC-768 SONET LTE

A Layer-1 Path Relay transports the SPE from the client side to the system side and vice versa

Source: PMC-Sierra, T1X1.5,2001, May

802.3 WANPhysical Medium Attachment

802.3 WANPhysical Medium Dependent

PartialWAN Interface Sublayer

λ A λ B

802.3 MAC

64B/66BPhysical Coding Sublayer

WAN Interface Sublayer

802.3 WANPhysical Medium Attachment

802.3 WANPhysical Medium Dependent

Higher Layers

Reconciliation Sublayer

45ETRI Proprietary

Cost prediction onCost prediction on GbEGbE & 10GbE& 10GbE

동향 예측GbE 동향 예측

스위치: $1,400 (1999) $550 (2004)NIC : $ 570 (1999) $ 80 (2004)

10GbE 동향 예측

스위치: $2,800 (2000) $2,000 (2004)NIC : $3,000 (2001) $1,200 (2004)

출처: Linley Group, June 2001

46ETRI Proprietary

Trend on 10GbE industriesTrend on 10GbE industries

Chip set trend장미빛사진 ? (10GbE Transceiver가 Agere의경우 2003년대량출하시 WWDM XAUI가 $1000수준으로예상(uncooled DFB LD)

메트로응용을위한 10GEA 결성및 MEF, RPR표준화추세10GEA: 2001년1월결성, ETRI포함 111개회사/기관메트로이더넷포럼: 2001년10월결성, 60여개제조사및통신사업자IEEE802.17 결성, 2003.3 까지 RPR표준화목표

10GbE카드출시공고시스코: 비표준의 1550nm, 50Km 공고(CISCO12000시리즈용), $80,000/cardAvaya: 850nm/1310nm/1550nm($20,000 ~ $60,000)을 2002.01 출시공고Ethernet system itself is still not yet

10G급 NPU, Switch개발동향EZchip: 2000.10까지 $28M fundingXelerated: $12M funding (2001.10)Silicon Access: $85 M fundingIBM의경우, 자사의 10G급 switch fabric은 Ezchip과 interoperability

IEEE802.3ae동향:정식표준확정의지연예상(wait for enough prototype)Source: Network World High-speed LANs news letter, Aug., 2001, by Jeff Caruso

47ETRI Proprietary

Next, Next, 40GbE40GbE or or 100GbE100GbE ????

20062002

1998

1995

1990198310

100

1,000

10,000

100,000

1980 1985 1990 1995 2000 2005 2010

Meg

abits

per

sec

ond

10BASE-T

Fast Ethernet

Gigabit Ethernet

10 Gigabit Ethernet

Moore’s L

aw

10BASE-T

Fast Ethernet

Gigabit Ethernet

10 Gigabit Ethernet

48ETRI Proprietary

Next Speed Increase? Next Speed Increase?

50% of 802.3ae thinks next speed should be 40 GbE

People are already talking about combining 10 GbE Serial with 4 Color WDM to provide 40 GbEEthernet Utilizing 802.3ad (link aggregation)Or, does this mean OC-768-like?

But, 100 GbE is not unlikely (year 2006) ??

49ETRI Proprietary

referencesreferences

TextbookJ. Kadambi, et. al, “Gigabit ethernet”, prentice hall, 2000

Tanenbaun, “Computer Network”, 5th edition, 2000

윤종호, “고속 이더넷”, 1996, Ohm사

강철희외 3인 공역, “그림으로 보는 기가비트 이더넷,” 2000, 교보문고

Other useful articles …IEEE 802.3ae, http://rouper.ieee.org/groups/802/3/ae/index.html

10GEA White Papers, http://www.10gea.org

C.Simoneaux & J. Jewell, “MSA Group overview and status report”, IEEE 802.3ae interim meeting, Sept., 2000

H. Kaplan & B. Noseworthy, “The Ethernet: Evolution from 10 to 10,000 Mbps – How it all works!”, N+I 2000 Atlanta workshop

Optical Ethernet, http://www.optical-ethernet.com/optical/index.htm

IEEE 802.3, http://www.ieee802.org/3/efm/index.html

10 gigabit ethernet - hsn

Documents