energy efficient bandwidth for electrical backplanes and

Energy Efficient Bandwidth for Electrical Backplanes and Copper Interconnects

Dr. Jeffrey H. Sinsky

Optical Subsystems and Advanced Photonics Department, Bell Labs, Alcatel-Lucent

[email protected]

2 | Energy Efficient Backplanes | April 2010 |J. Sinsky All Rights Reserved © Alcatel-Lucent 2010

Contributors

David Neilson, Bell Labs, Alcatel-Lucent

Andrew Adamiecki, Bell Labs, Alcatel-Lucent


Overview

The limitations of copper – how far can we go with copper… when do we need optics

Understanding the fundamental challenges of electrical backplanes and copper interconnects

Thinking about electrical transmission lines from a “Shannon” perspective

Using higher order modulation formats – the good and the bad

Examples of multilevel signaling performance over backplanes and cables

Current state of the industry

Where we need to go

Conclusion


The Limitations of Copper – Experimental Findings To Date

D.T.Neilson, D. Stiliadis, and P. Bernasconi, “Ultra-High Capacity…" ECOC 2005, Glasgow, UK, September 2005


Applications that will require optical interconnects

- Distance bandwidth products of over 100 Gb/s ⋅m are possible candidates

- Distance bandwidth products of over 1Tb/s ⋅m will almost certainly be optical!

EXAMPLES -distance bandwidth products between 100 Gb/s ⋅m and 1Tb/s ⋅m

Between Racks: 100m+ @ 1Gb/s-10Gb/s rates Backplanes: 1m @ 100Gb/s-1Tb/s rates Linecards: 0.3m @ 300Gb/s-3Tb/s rates On chip: 0.03m @ 3Tb/s-30Tb/s rates

Rates are per waveguide!

D.T.Neilson, Challenges for Optical and Electrical Interconnects,” WTM 2010 IEEE Photonics Society Winter Topical Meeting on Photonics for Routing and Interconnects, Majorca, Spain, January 12 , 2010.

Understanding the fundamental challenges of electrical backplanes and copper interconnects


Electrical Transmission Line Losses

impedance sticcharacteri widthtraceW

resistance surface2

where

lengthdB/unit 686.8

0

0

0

==

==

=

Z

R

WZR

s

sc

σωμ

α

tangentloss tan

h wavelengtguidedconstant dielectric effective

constant dielectric relative

lengthdB/unit tan11686.8

=

===

⎟⎟⎠

⎞⎜⎜⎝

⎛−−

=

δ

λεε

λδ

εε

εεπα

g

re

r

gre

r

r

red

loss leakagel

lossradiation r

loss dielectricd

lossconductor

=

=

=

=

+++=

α

α

α

α

ααααα

c

lrdc

Jia-sheng Hong, M.J. Lancaster, Microwave Filters for RF/Microwave Applications, Wiley, New York, pg. 83.

f∝ f∝


Backplane Design Challenges at High Speed

Backplane Via

Backplane Trace

Backplane PCB

ConnectorLine Card Via

Line Card Trace

Tx/Rx ASIC (includes via)

AirMaxTM

steep roll offfrom lossy trace

narrow resonancefrom stub effect

0 dB

-20 dB

-40 dB

-60 dB

12.5 GHz

-33 dB

~ 14 GHz

severeattenuation

FCI Airmax Demo24” Trace

Transfer Function

•Controlled impedance must be maintained•Cross-talk between pins is difficult to reduce

•The viahole looks like an inductor•Unless removed, the unterminated part of the viahole acts as a resonant stub


The impact of backplane connectors and dielectric loss

Jri Lee, Ming-Shuan Chen, and Huai-De Wang, “Design and Comparison of 20-Gb/s Backplane Transceivers for Duobinary, PAM4, and NRZ Data, IEEE Journal of Solid-State Circuits, Vol. 43, No. 9, September 2008.


Backplane Losses – Examples

FR4 Backplanes : 5dB/GHz/m

10Gb/s data rate (f=5GHz) 0.8m length : 20dB loss

Improved Backplane materials : 1dB/GHz/m

50Gb/s data rate (f=25GHz) 0.8m length : 20dB loss

Coax Low loss dielectric : 0.05dB/GHz/m

40Gb/s data rate (f=20GHz) 20m length : 20dB loss

Note: Losses calculated at Nyquist frequency (Bitrate/2 for NRZ)

D.T.Neilson, Challenges for Optical and Electrical Interconnects,” WTM 2010 IEEE Photonics Society Winter Topical Meeting on Photonics for Routing and Interconnects, Majorca, Spain, January 12 , 2010.


Cable Design Challenges at High Speed

The cable dilemna•Larger diameter less loss, worse high frequency performance•Smaller diameter better high frequency performance, more loss

The cable dilemna•Larger diameter less loss, worse high frequency performance•Smaller diameter better high frequency performance, more loss

{ }

( )

2 42 1 (1/ 6)( / 2 ) (7 /120)( / 2 ) ... ***

where/ 2

c m m m

m o i

o i

r t r t r

r r rt r r

λ π= − − −

= +

= −***Green, H., “Determination of the Cutoff of the First Higher Order Mode in a Coaxial Line by the Transverse Resonance Technique,” IEEE MTT Trans., Vol. 37, No.10, Oct. 1989, pp. 1652-1653.

( )11.811

whereinner radius in inchesouter radius in inches

GHzcr o i

i

o

fr r

rr

π ε≅

+

==

ro

ri

A non-TEM mode will start to propagate at λc, or fcGHz

A non-TEM mode will start to propagate at λc, or fcGHz


Materials

For cables

Lower loss dielectric materials

Clever use of metals that try to equalize the frequency response roll off

Example: Gore “eye opener” cable

For Backplanes

Lower loss dielectric materials – must be cheap and applicable to multilayer assembly

Remove woven nature of the dielectric material

Example: Dupont is developing dielectric materials that do not exhibit the periodic mismatch of a typical FR4 which results from its woven structure

Aramid reinforcement -More Uniform and Homogenous

106 glass

1080 glass

Woven-Glass-Non Uniform

Thinking about electrical transmission lines from a “Shannon” perspective


Interconnect Channels: Shannon C: Channel capacity (bits/s)

B: Channel bandwidth (Hz)

SNR: Signal energy / noise energy

Shannon formula assumes

optimum constellation and optimum coding of data

the best you can do

C = B log2 (1 + SNR)

Shannon capacity:

SNR (dB)

spec

tral

eff

icie

ncy

(bit

s/s/

Hz)

-5 0 5 10 15 20 25 300123456789

10

Shannon

’s lim

it

“Error-fr

ee”

achievab

le“Erro

r-free”

not achievab

le


Spectral Efficiency (b/s/Hz)

Electrical InterconnectsUsing Higher order Formats

Possible to use higher order signaling to reduce max frequency but requires higher SNR (e.g. 2b/s/Hz requires 5dB improved SNR but only half frequency so lower losses)

If we take the FR4 example above 5dB/GHz/m for 0.8m backplane

Look at required SNR at launch: this accounts for losses and ShannonAbove 2.5Gb/s optimum format is not NRZ

Operation at 20Gb/s and 40Gb/s may be

possible

NRZ D.T.Neilson, Challenges for Optical and Electrical Interconnects,” WTM 2010 IEEE Photonics Society Winter Topical Meeting on Photonics for Routing and Interconnects, Majorca, Spain, January 12 , 2010.

backchllossa =

D. T. Neilson, "Photonics for switching and routing," IEEE J. Sel. Topics Quantum Electron. 12, 669-678 (2006).


Choosing a Signaling Format for Energy Efficient Transmission - Tradeoffs

“Fitting” signal bandwidth into optimal portion of the channel response

Intelligent transmission of signal energy

Sensitivity of format to “spectral anomalies”

Impact on power requirements

Increased SNR requirement for higher order constellations

Efficiency of required electronics

Implementation complexity

Cost


Signal Bandwidth Compression – Spectrally Efficient Formats for Cables and Backplanes

Formats

Duobinary

PAM-4

polybinary

PAM-X

Pattern dependence issues must be kept in mind

Multilevel signaling relocates signal energy to the frequency range with the lowest insertion loss

This makes sense for electrical backplanes and cables!

•H.-J. Goetz, J.H. Sinsky, “The Duobinary Format: A New Application for an Idea Published Long Ago,” DesignCon 2006, Santa Clara, CA, invited talk.


Considerations for Optimal Transmission

distortion function

Certain formats are more sensitive to the fine structure found in the preferred region.


Compare Signaling Architecture Hardware Complexity

NRZ

Duobinary



Compare Signaling Architecture Hardware Complexity (continued)

PAM-4



Power Consumption of a Typical 90nm CMOS Buffer as a Function of Bandwidth


It is important that the hardware required to transmit and receive the hardware is not too complex!!

(simulation)


Using Better Materials and Connectors Makes Sense Megtron 6 vs. Taconic Substrates

Backplane thickness – 250 milsDaughter card thickness - 150 mils

Channel bounds – OIF CEI-25G-LR

•1 dB/GHz/m

2 dB/GHz/m------ Taconic------ Megtron6

STRADA Whisper

Daughtercards have 10” of a 6-9-6 pair and backplane has16 total inches of 6-9-6 diff pair. Each daughtercard interface has a STRADA Whisper mated connector pair. (total length = 1 meter)

(simulation)

Courtesy of Tyco Electronics

Examples of Backplane and Copper Interconnect Signaling Techniques


Examples

PAM-4 backplane (5 Gb/s)

Duobinary backplane (25 Gb/s)

Duobinary cable (40 Gb/s)

NRZ with equalization (25 Gb/s)


PAM-4 Example

Stonick, J.T.; Gu-Yeon Wei; Sonntag, J.L.; Weinlader, D.K.; , "An adaptive PAM-4 5-Gb/s backplane transceiver in 0.25-μm CMOS," Solid-State Circuits, IEEE Journal of , vol.38, no.3, pp. 436- 443, Mar 2003.


Electrical Duobinary over Backplanes and Cables

How it works

We take advantage of the natural spectral roll off response of the backplane channel to convert standard NRZ data to duobinary data, effectively using the channel as part of the transmitter.

At the receiver, a quasi-digital duobinary-to-binary converter is used to regenerate NRZ data.

We were the first to demonstrate GHz-rate duobinary data transmission over FR4 backplanes and the first to demonstrate 25 Gb/s data transmission over an FR4 backplane

)()()()( ωωωω duoEQChFIR HHHH =⋅⋅

25 Gb/s Example

Avoids resonances

Sinsky, J.H.; Duelk, M.; Adamiecki, A., "High-speed electrical backplane transmission using duobinarysignaling," Microwave Theory and Techniques, IEEE Transactions on , vol.53, no.1pp. 152- 160, Jan. 2005.


10 Gb/s Backplane Transmission Results

Tyco Quadroute Backplane (34” trace)

Pre-emphasized Signal Into the backplane

34” backplanechannel

10 Gb/s Duobinary backplane outputPRBS 23 Measurement

4,6,4 (mils)

Trace Geometry (width, space, width)

HM-ZD4000-6Quadroute

Connector TypeNelco DielectricBoard Name

Duobinarydecision thresholds

BER <10-13

Xaui Reference Backplane

First demonstration of GHz-

speed duobinarysignaling over a

backplane

J.H. Sinsky, A. Adamiecki, M. Duelk, “10-Gb/s Electrical Backplane Transmission using DuobinarySignaling,” IMS-2004, Fort Worth, TX, June 2004.


25 Gb/s Backplane Transmission Results (4 x 25 Gb/s)

2

20

Layers

AirMax VS®7.5cm, 25cm 50cm, 75cm

Nelco 4000-6

AIRMAX Backplane

AirMax VS®5cm eachRogers RO4350

2 Line Cards

Connector Type

Line Lengths

PCB Material

Board Name

Line Card

0 5 10 15 20 25

-50

-40

-30

-20

-10

0 35cm (14'') 60cm (24'')

S21

Mag

nitu

de [d

B]

Frequency [GHz]

25 Gb/s Duobinary backplane outputPRBS 7 Measurement

10ps/DIV65mV/DIV

Pre-emphasized Signal Into the backplane

24” backplanechannel

BER <10-13

0.2V/DIV 10ps/DIV

PRBS 7

First demonstration of 25 Gb/s data

transmission over an FR4 backplane

Adamiecki, A.; Duelk, M.; Sinsky, J.H., "25 Gbit/s electrical duobinarytransmission over FR-4 backplanes," Electronics Letters , vol.41, no.14pp. 826- 827, 7 July 2005.


40 Gb/s over an 24.4 meters of SMA Coaxial CablePre-emphasized Signal Into the cable

40 Gb/s Duobinary cable outputPRBS 7 Measurement

24.4 meter coaxcable channelDC-24 GHz

DC-18 GHz

Operating Bandwidth

MegaPhase

MIDISCO

Manufacturer

Micro-porousPTFE Tape

Air/PTFE

Dielectric

40 feet0.200”

0.205”

Outer Diameter

40 feet

Length

BER <10-14

First demonstration

of 40 Gb/sduobinary data transmission over a 24.4 m

coax cable

J.H. Sinsky, A. Konczykowska, A. Adamiecki, F. Jorge, M. Duelk, “39.4 Gb/s Duobinary Transmission over 24.4 meters of Coaxial Cable using a Custom Indium Phosphide Duobinary-to-Binary Converter Chip,” accept IEEE Trans. On Microwave Theory and Techniques, Dec. 2008 or Jan. 2009.


NRZ with Equalization Example (25 Gb/s) – Tyco Designcon 2010

18” of Megtron 6 (including daughtercards) with Avago silicon

Both daughtercards use Tyco’s STRADA Whisper connectors on them

•25 Gb/s

Courtesy of Tyco Electronics


Current State of the Industry

OIF

Looking at 25 Gb/s Long Reach over backplane (CEI-25G-LR)

Have been exploring NRZ signaling with pre-emphasis and equalization

IEEE 802.3ba - 40Gb/s and 100Gb/s Ethernet Task Force

Provide Physical Layer specifications which support 40 Gb/s operation over

at least 10km on SMF

at least 100m on OM3 MMF

at least 7m over a copper cable assemblyat least 1m over a backplane

Provide Physical Layer specifications which support 100 Gb/s operation over:

at least 40km on SMFat least 10km on SMFat least 100m on OM3 MMFat least 7m over a copper cable assembly


Where we need to go

Need to focus more on energy efficiency

Intelligent use of channel

Use better channels! They exist!

Lower loss dielectrics

Tighter more random weaves

Improve metallization roughness spec

Once materials are improved, backplane connectors will become the bottleneck!

Innovative designs are needed – very challenging

Consider “low” frequency equalization carefully for multi-level formats

There may be more sensitivity to gain and phase variations


Conclusion

Electrical backplanes and interconnects are not dead yet!!

Electrical backplane transmission at 25 Gb/s should be possible up to 1 meter

An optimal combination of multi-level formats, improved dielectric material, and improved connectors should pave the way for progress

For coaxial cables, rates to 40 Gb/s should be possible in a commercial setting over many meters

An exciting challenge would be to push backplane line rates to 40 Gb/s

Would require innovation in connectors and connection techniques between backplane layers

energy efficient bandwidth for electrical backplanes and

Documents