Optics in Internet Routers

Mark Horowitz, Nick McKeown, Olav Solgaard, David Miller

Stanford University http://klamath.stanford.edu/or

2

0,1

1

10

100

1000

10000

1985 1990 1995 2000

Spec

95In

t CPU

resu

lts

0,1

1

10

100

1000

10000

1985 1990 1995 2000

Fib

er

Ca

pa

cit

y (

Gb

it/s

)

TDM DWDM

Packet processing Power Link Speed

2x / 18 months 2x / 7 months

Source: SPEC95Int & David Miller, Stanford

Why We Need Faster Routers

To prevent routers from becoming the bottleneck

3

POP with smaller routersPOP with large routers

Interfaces: Price >$100k, Power > 400WIt is common for 50-60% of interfaces to be for interconnection within the POPIndustry trend is towards large, single router per POP

Fast (Large) Routers

Big POPs need big routers

4

A router is a packet-switch, and therefore requiresA switch fabricPer-packet address lookupLarge buffers for times of congestion

Address lookup and buffering are infeasible using optics presentlyA typical 10 Gb/s router linecard has 30 Mgates and 2.5 Gbits of memory

Research ProblemHow to optimize the architecture of a router that uses an optical switch fabric?

All optical IP routers are infeasible today

5

100 Tb/s Optical Router100 Tb/s Optical Router

Collaboration

4 Stanford professors (M. Horowitz, N. McKeown, D. Miller and O. Solgaard), and their groups

ObjectiveTo determine the best way to incorporate optics into routersPush technology hard to expose new issues

Photonics, Electronics, System designMotivating example: The design of a 100 Tb/s Internet router

Challenging but not impossible (~100x current systems)It identifies some interesting research problems

6

Arbitration

160Gb/s

40Gb/s

40Gb/s

40Gb/s

40Gb/s

OpticalSwitch

• Line termination

• IP packet processing

• Packet buffering

• Line termination

• IP packet processing

• Packet buffering

160-320Gb/s

160-320Gb/s

Electronic

Linecard #1ElectronicLinecard #625

Request

Grant

(100Tb/s = 625 * 160Gb/s)

100 Tb/s Router

7

Research Problems

LinecardMemory bottleneck: Address lookup and packet buffering

ArchitectureArbitration: Computation complexity

Switch FabricOptics: Fabric scalability and speedElectronics: Switch control and link electronicsPackaging: Three surface problem

8

160Gb/s Linecard: Packet Buffering

ProblemPacket buffer needs density of DRAM (40 Gbits) and speed of SRAM (2ns per packet)

SolutionHybrid solution uses on-chip SRAM and off-chip DRAMIdentified optimal algorithms that minimize size of SRAM (12 Mbits)Precisely emulates behavior of 40 Gbit, 2ns SRAM

DRAM DRAM DRAM

160 Gb/s 160 Gb/s

Queue Manager

[klamath.stanford.edu/~sundaes/Papers/ieeehpsr2001.pdf]

SRAM

9

Architecture: The Arbitration Problem

A packet switch fabric is reconfigured for every packet transfer

At 160Gb/s, a new IP packet can arrive every 2ns

The configuration is picked to maximize throughput and not waste capacity

Known algorithms are too slow

Our solution is to eliminate the arbitration

10

Two-Stage Switch

External Outputs

Internal Inputs

1

N

ExternalInputs

Spanning Set of Permutations

Spanning Set of Permutations

1

N

1

N

Recently shown to maximize throughput

[C.S.Chang et al.: http://www.ee.nthu.edu.tw/~cschang/PartI.pdf]

11

Problem: Unbounded Mis-sequencingExternal Outputs

Internal Inputs

1

N

ExternalInputs

Spanning Set of Permutations

Spanning Set of Permutations

1

N

1

N

11

2

2

We have developed an algorithm toKeep packets ordered andGuarantee a delay bound within the optimum

[Infocom’02: klamath.stanford.edu/~keslassy/download/infocom02_two_stage.pdf]

12

1

2

3

Phase 2

Phase 1

Idea: use a single-stage twice

An Optical Two-stage Switch

Lookup

Buffer

Lookup

Buffer

Lookup

Buffer

Linecards

13

Cascaded Wavelength Switches

mn x mn switching fabric2n building blocksSupports spanning set of permutations

1Input m

Input 2Input 1

2Input m

Input 2Input 1

nInput m

Input 2Input 1

1Output m

Output 2Output 1

2Output m

Output 2Output 1

nOutput m

Output 2Output 1

14

Building Block: Wavelength Switch

m-Input and n-OutputSwitching by wavelength selection of tunable lasersOptical amplifier (EDFA) can be included to reduce loss

Input 1

Input 2

Input m

1, 2, …, n1, 2, …, n

1, 2, …, n

Output 1

Output 2

Output n

12

n

PowerCombiner

WavelengthDemultiplexer

15

Wavelength Switch:Receiver Side

n-Input and m-OutputTunable optical filters are key components

Output 1

Output 2

Output m

Input 1

Input 2

Input n

12

n

PowerDivider

WavelengthMultiplexer

TunableFilters

16

Arrays of Optoelectronic Transceivers

CMOS Optical Receiver

A 1.6Gb/s, 3mW Integrating CMOS Optical Receiver with AlGaAs Photo-Detectors

Standard CMOS Electronics with flip-chip bonded optical devicesRemoves the trans-impedance amplifier to reduce power and improve bit-rate.Enables dense arrays of receivers and transmitters on chip