high-speed backplane interconnect

7/28/2019 High-Speed Backplane Interconnect

1/49

High-speed backplane interconnect

Vladimir Stojanovic

(with slides from J. Zerbe, P. Desai, R. Kollipara)


2/49

Outline

Inside the router

Backplane channel problem

What can backplane designer do about it

What can IC designer do about it

Scaling the system to 10-100Tb/s


3/49

Inside the Router

MAC

TM/

Fabric

IF

NPU

SerDes

Optics

SerDes

SerDes

Crossbar

Line Cards:8 to 16 per System

Switch Cards:2 to 4 per System

Passive

Backplane

MEM

MEM

MEM

MEM

Past OC-12

622 MHz LVDSparallel

GigE

1.25 Gbps serial

Present OC48

2.5 Gbps serial

10GigE

XAUI (3.125 Gbps) serial


4/49

OpticsMAC/

Framer

NPU/

TM

Switch

Fabric IF

XAUI

4, 3.125 Gbps

Serial Links

SPI4.2 CSIX

ProprietaryBackplane

8 to 16

of 1-3.2Gbps

Serial Links

Line Card:

Switch Card:32 to 64

Backplane

Serial Links

(1-3.2 Gbps)

Switch

Crossbar

IC

Serial Links in Networking Systems


5/49

Backplane interconnect path

Backplane via

Backplane connector

Line card

trace

PackageChip

Line card

viaBackplane trace

Packageto board

transition

There are many components on the signal path,

potential source of problems


6/49

RaSer X Link Features

PLL

Serializer Tx Link

20 bit

ParallelInterface

1-10 Gbps

RefClk

Tx

Eq

Deserializer and

CDR

Rx Link

1-10 Gbps

Rx

Eq

20 bit

Parallel

Interface

TX

RX

PLL

Process 0.13m CMOSPower 40mW / Gb

Area 1mm2

2-PAM Range 2 6.4 Gb/s

4-PAM Range 5 10 Gb/s


7/49

Impedance-controlled (CML) I/Os

Integrated terminations

Adjustable output-voltage/common mode

I/O Driver Scheme (Example)

50 W 50 WVtt

Zo = 50W

Zo = 50WRx

Tx

Vtt


8/49

System Issues

Goal Increase Router Throughput

Limitations

Backplane channel

Power

Mechanical/Physical density constraints

Backplane and linecard routing density

Connector pin density

Package I/O density


9/49

Outline

Inside the router






10/49

Backplane Component Effects

PCB only

PCB + Connectors

PCB, Connectors,

Via stubs & Devices


11/49

Deterministic Noise

0 2 4 6 8 10

-60

-50

-40

-30

-20

-10

0

frequency [GHz]

Attenuation[d

B]

FEXT

NEXT

THROUGH

0 1 2 3

0

0.2

0.4

0.6

0.8

1

ns

pu

se

response

Tsymbol=160ps

Inter-symbol interference Dispersion (skin-effect, dielectric loss) - short

latency

Reflections (impedance mismatches connectors,

via stubs, device parasitics, package) long latency


12/49

XTALK and reflections

Far-end XTALK (FEXT)

Desired signal

Near-end XTALK (NEXT)

Reflections

Primary reflection sources are at the

connector/backplane transition

Grouped in time as a function of backplane length


13/49


14/49

Test Backplane Example

Dielectric material FR-4 Nelco Roger

6000 4350

Dielectric constant 4.2 4 3.6

Loss tangent (1 MHz) 0.016 0.005 0.0035

Loss tangent (1 GHz) 0.017 0.007 0.0035

Thickness 0.295" 0.299" 0.297"

FR4 Cross Section

Trace lengths: 1.5,9, 14, 20 and 32

Effective number of

signal layers: 13

Effective number of

total layers: 28


15/49

Backdrilling - A Solution to the Stub

Effect


16/49

Stub Effect Eye Pattern Analysis(2.5 Gbits/sec FR-4)

MAX STUB MIN STUB


17/49

MAX STUB MIN STUB



18/49

MAX STUB MIN STUB



19/49

Connector design

GBX

Teradyne


20/49

Connector Density

Teradynes GbX Connector

Differentia

l

Pairs/inch

Card

Pitch

Bandwidth/linear inch

(at 6.25 Gbps)

5 pair 69 1.25" min.(30 mm)

431 Gb

4 pair 551.00" min.

(24.7mm)343 Gb

3 pair 41.80" min.

(20 mm)256 Gb

2 pair 27.5.575" min.

(14 mm)171 Gb


21/49

Reducing Crosstalk within the

Connector

Cross talk is

reduced in the

mating interfaceby surrounding

each pair with a

ground shield

D/C Shield

B/P ShieldMated pair


22/49

Backplane Connector Considerations

Many connector types: Teradyne: VHDM, HSD, GbX,

Tyco: HS3, HMZd,

FCI: Metral 2000, 3000, 4000,

3M/Harting: HSHM,

ERNI: ERmetZd, ErmetXT,

Issues

Loss, impedance profile, crosstalk, skew Foot print: routability, pin density, via impedance

Single-ended and differential

Press-fit and SMT


23/49

10Gbps Test Package Design Example

Ceramic BGA

Wire-bonded

4-Layer

1 mm pitch

Source: Designed for Rambus by Kyocera


24/49


25/49

Outline

Inside the router






26/49

Loss : Equalize to Flatten Response

Channel is band-limited

Equalization : boost high-frequencies relative to lowerfrequencies

+

=


27/49

Receiver Linear Equalizer

Amplifies high-frequencies

attenuated by the channel

Digital or Analog FIR filter

Issues

Also amplifies noise!

Precision

Tuning delays (if analog)

Setting coefficients

Adaptive algorithms such asLMS

WL-1

DDD

WLW

1

+

H(s)

freq


28/49

Transmitter Linear Equalizer

Attenuates low-frequencies Need to be careful about output

amplitude : limited output power

If you could make bigger swingsyou would

EQ really attenuates low-

frequencies to match highfrequencies

Also FIR filter : D/A converter

Can get better precision than Rx

Issues How to set EQ weights?

Doesnt help loss at f

H(s)

freq


29/49

Transmit Linear Equalizer :

Single Bit Operation

0.0 0.3 0.6 0.9 1.2-0.3

-0.1

0.1

0.3

0.5

0.7

UnequalizedEqualization PulseEnd of Line

time (ns)

Volta

ge


30/49

Decision Feedback Equalization (DFE)

Dont invert channeljust remove ISI

Know ISI because already

received symbols

Doesnt amplify noise

Requires a feed-forward

equalizer for precursor

ISI

Reshapes pulse to

eliminate precursor

-

FIR filter

Decision (slicer)

FIR filter

Feed-forward EQ

Feed-back EQ


31/49

DFE Example


32/49

Transmit and Receive Equalization

Transmit and receive equalizers are

combined to make a range restricted DFE Tx equalizer functions as the feed-forward filter

Rx equalizer restricted in performance of loop

TAP SEL

LOGIC

TX

DATA

3

RX

DATA


33/49

Tx & Rx Equalization Ranges

TX Driver/Equalizer : 5 taps

1(pre)+1(main)+3(post)

RX Equalizer

5-17 taps after main

Pick any 5 taps


34/49

Pulse Amplitude Modulation

Binary (NRZ) is 2-PAM 2-PAM uses 2-levels to send

one bit per symbol

Signaling rate = 2 x Nyquist

4-PAM uses 4-levels to send2 bits per symbol

Each level has 2 bit value

Signaling rate = 4 x Nyquist

00

01

11

10

1

0

1

0


35/49

When Does 4-PAM Make Sense?

First order : slope of S21

3 eyes : 1 eye = 10db

loss > 10db/octave : 4-PAM

should be considered

0.0 1.0 2.0 3.0 4.0

Nyquist Frequency (GHz)

|H(f)|

-20db

-40db

-60db


36/49

Example : 5Gbps Over 26 FR4

With No Equalization


37/49

Example : 5Gbps Over26 FR4

Correct Tx Equalization


38/49


Under Equalized


39/49


Over Equalized


40/49

26 FR4 Bot 3.125Gbps, 2P noEQ


41/49

26 FR4 Bot 3.125Gbps, 2P w/EQ


42/49

26 FR4 Bot 6.4Gbps, 2P w/3G EQ


43/49

26 FR4 Bot 6.4Gbps, 2P w/EQ


44/49

26 FR4 Top 6.4Gbps, 2P w/EQ


45/49

26 FR4 Top 6.4Gbps, 4P w/EQ


46/49

26 Nelco6k-cb Top 10Gbps, 4P


47/49

26 Nelco6k-cb Top 6.4Gbps, 2P

S li h h h


48/49

Scaling the router throughput

System (Tot. throughput ~2.5Tb/s) 8-16 Line Cards


49/49

Scaling the router throughput

System (Tot. throughput ~100Tb/s) 100 Line Cards

1Tbs / LC 10Gbs links

4mW/Gbs Link power in 0.065um

Speedup ~1x

#links at switch card ~ 10k

Limitations #diff pairs at switch card 20k

Switch card power from links in 0.13um~10k*10Gbs*4mW/Gbs ~ 400W

Connector density 50diff pairs/inch: (tot length=20k/50= 400)

BP/LC routing pitch 0.050 Num. Layers (BP=13, LC=4) 5k diff pairs/layer = 250LC

routing width

Package ball pitch (1mm/200um)

high-speed backplane interconnect

Documents