©hdp user group international, inc.1 optical interconnect project definition stage project may 20,...

©HDP User Group International, Inc. 1

Optical Interconnect Project

Definition Stage Project

May 20, 2010 – HDPUG European Meeting

Bruchsal, Germany


PurposePurpose

• To address performance limitations encounted in high-speed electrical backplanes (15-20+ Gbps) by use of optical signal transmission

• To demonstrate that optical waveguides within a backplane can benefit the system’s interconnect topology by providing:

– Higher data rates

– Additional I/O

– Better interconnect architectures


ObjectivesObjectives

• Review available technologies and components for an optical backplane

• Define an optical backplane prototype (“HDPUG Emulator”)

• Compare selected technologies with experimental qualification or proof-of-concept demos

• Assemble and test HDPUG optical backplane demonstrator

• Realize hardware based on as many “off-the-shelf” components as possible


Execution PlanExecution Plan


PHASE 1

Benchmark, Specification, Selections of mtrls + comp’s

PHASE 2

Proof-of-concept demos for verification of parts/sub-units

PHASE 3

Optical BP Demonstrator – Design, Fabrication & Test

PHASE 4

Reporting,Practice guidelines

~ 18 Months

Q1/10 Q3/11

NOW

Q2/10 Q3/10 Q2/11

Definition Implementation

Status UpdateStatus Update

©HDP User Group International, Inc. Proprietary

5

• Activities– Project devided into 4 sub-teams

• Waveguides (M.Immonen, Meadville, B.Booth, Optical Interlinks)

• Connectors and components (R.Pitwon, Xyratex)

• Architecture (D.Smith, NGC, D.Rolston, Reflex Photonics)

• Specifications (S.Xiang, Huawei)

– Demonstrator concept ”Optically Enchanced Backplane” proposed by Reflex Photonics

• Fiber-based data links from line-cards to BP

• Waveguide-based optical routing layer add-on in a conventional electrical backplane

• Medium-term solution

• Extensive use of off-shelf components and mature technologies

Optical Backplane ConceptOptical Backplane Concept

• Optical Electrical hybrid backplane– Optical data buses for high speed data bus

– Electrical data buses for slow speed signals and power

• Optical paths based on fibers and/or waveguides


Optical connector

Electrical connector

High-speed Optical data bus (optical waveguide )

Slow-speed electrical data bus

Optical module

line card

switch cardoptical waveguide or fiber O/E module

SpecificationsSpecifications


Specification – Phase 1 Prototype Phase 2-3 Objective

Parameter Target Value Expected Value

Dimensions

Dimension of Backplane 50cm x 50cm TBD

Optical Waveguide Layers 1 ≥ 2

Backplane Waveguide Channel Length ≥30cm ≥75cm

Date Rate Gbps per channel 10Gbps 10~20Gbps

Density

Channels per Tx/Rx 12 24

Channels per Connector 1 x 12 2×12

Waveguide Core Pitch 250µm 125um or 62.5um

Core Width x Height 50µm x 50µm TBD

Connector Footprint 2cm x 2cm TBD

Loss

Waveguide Transmission Loss ≤0.1 dB/cm ≤0.06 dB/cm

Connector Insertion Loss ≤3 dB ≤2 dB

Total Loss <10dB <8dB

Cost (volume production)

Cost of optical passive channel (including connector) ≤$ 1 per Gigabit ≤$ 0.5 per Gigabit

Cost of optical device (including Tx and Rx) ≤$ 2 per Gigabit ≤$ 1 per Gigabit

Phase 1: Prototype; Phase 2: Engineering Solution; Phase 3: System Solution/Product

Prior Art

• Optical Backplanes with intra-card optical connectors have been demonstrated

• Very dense optical connectors can be introduced into the future while continuing to evaluate their price, cost and complexity.

8©HDP User Group International, Inc.

Prior Art

• Most of the techniques used to demonstrate optical backplane concepts involve the right-angle turn.

• Either electrical bending or optical reflection.

• These optical/electrical “connectors” are introduced at the back of the line-card and are aligned with optical waveguides within cut-out holes in the backplane.

http://www.zurich.ibm.com/pdf/sys/Optical_Interconnects@IBM-ZRL_2005-04_E.pdf©HDP User Group International, Inc.

THE OPTICAL BACKPLANE PROPOSAL


10

Optically Enhanced Backplane

• Optical interconnects are added to the chassis – not the middle of the backplane.

• The optical signals are directed out the front of the line-card and back towards the optical waveguide layer.

• The electrical connections and the line-card’s general form-factor are not affected.

©HDP User Group International, Inc.

Optical PathOptical Path

12

Optical cable path(Jumper cable)

Optical waveguide (inside backplane)

Light Source(module)

• The chassis is configured to use waveguides within the backplane, but route each line-card’s optical i/o to the front panel.

• The line-card has a jumper cable to the top of the chassis

Optical conduit(Extension cable)


Optical BP FeaturesOptical BP Features

13

• Optical– No 90 bends / no active

alignments– No EMI or EM susceptibility– Optical interconnects within

chassis are semi-permanent– Standard MTO or MT type

optical connectors– Easy cleaning– Standard glass fiber used

outside of chassis– No risk of damage to optical

parts within the backplane– Chassis to chassis

interconnect extensions

• Electrical– Backwards compatible line-

cards

– Optics and/or electrical line-cards used simultaneously

– Legacy or standard electrical backplane architectures possible

– Standard power-supplies and controllers can be used


ArchitectureArchitecture


• The primary reason for using optical waveguides in the system is that it provides higher data through-put over existing solutions.

• The crossbar offers extremely good interconnect topology, but is typically expensive to implement, optical waveguides offer a solution…

D.Rolston; Ex: http://www.mcs.anl.gov/~itf/dbpp/text/node33.html

N1(I)

N2(O) N3(O) N4(O)

N2(I)

N3(I)

N4(I)

FIFO Buffers

N1(O)

4

4

4

4

Tx

Rx

Rx

Rx

Rx

Tx

Rx

Rx

Rx

Rx

Tx

Rx

Rx

Rx

Rx

Tx

Tx:

Rx:

Fully Connected Crossbar Interconnect (4x4)

ApplicationApplication


• The physical layer of the optical backplane will be essentially ‘protocol agnostic’

• However, as a technology targeted towards computing and switching for data systems, the architecture used will be based on an Ethernet packet-switched architecture with broadcasting.

• Custom chip-sets / FPGAs with memory buffering can be used

• Input from OEM partners is also anticipated, and adaptations of current system line-cards will also be considered.

• Architectural modeling of latency, fault-tolerance, and packet overflow can also be investigated.

D. Rolston, http://cs.uccs.edu/~cs522/msgformat/format.htm

OPTICAL BACKPLANE CONSTRUCTION


16

• The backplane contain electrical connectors identical to a standard backplane, where the optical connections along the top.

• This allows backwards compatibility for line-cards that are not fitted with optical i/o.

Optical BackplaneOptical Backplane


Optical Waveguide LayerOptical Waveguide Layer


• To implement the crossbar, the transmit of a node is split numerous times to broadcast to all other nodes.

• This is normally impossible for electrical interconnects with high data rates without active switches, due to impedance control issues.

• Optical waveguides can be split and still carry high-speed optical data.

Node “R” Node 1 Node 2 Node 3 Node 4 Node 12

WG Diffusion Process

• GuideLink™ waveguide process:

• Monomer + Polymer Formulations pre-coated on Mylar


WG Diffusion Structures

• Polymer waveguide layout will require numerous splitters, bends and crossovers.

• Power estimates:– 4 splitters, plus material

attenuation/dispersion...

• If 1mW transmitted

• Estimate 20uW received at each line-card.

20

GuideLink™ 1x2 splitter

W/G Distribution

GuideLink™ Crossovers


WG PhotoImage Process

21

Core

Flexible Substrate

Bottom Clad

Flexible Substrate

Bottom Clad

Flexible Substrate

Bottom CladCore

Top Clad

3) Top Clad• Coating • Soft bake• UV exposure• Post-exposure bake• Hard bake

2) Core• Coating• Soft bake• UV exposure• Post-exposure bake• Development• Hard bake

1) Bottom Clad• Coating • Soft bake• UV exposure• Post-exposure bake• Hard bake


WG Photoimage Structures

• Cross-section– Mask Exposure:

– Target Width = 35 µm

– Laser Writing:

– Target Width = 50 µm

• Crossings– Routing made easier for

optical interconnects

– Not possible with electrical interconnects

22Courtesy of ©HDP User Group International, Inc.

WG Layer IntegrationWG Layer Integration


12 channels (12.5 Gb/s/ch) > 150 Gb/s

1.2cm

Courtesy of

• Challenges in fabrication and integration– Highly sensitive process – defects

vs. performance vs. yield

– Must be PCB process compatible: pressing, chemical processes, LF-soldering,…

– High-layer count boards: alignment, long optical path,…

Optical Connections

• The optical cable runs within the chassis rails and connects “semi-permanently” with the side of the backplane motherboard.

• The electrical connectors can be placed with the backplane with the same general structure and position of electrical backplanes


Optical Connector Dimensions

• Optical connections can be made on the top and bottom edges of the backplane.

• Given a ~35-mm line-card pitch, a total of 4 connections can be made per line-card, with optical arrays of either 12 or 24 waveguides given standard sizes for MT connectors.

35-mm


Optically Enhanced Line Card

• Optical modules with either 12 or 24-channels each are mounted ANYWHERE on the line card.

• The optical modules are placed to critical “high-bandwidth” devices (processors, switches, memory arrays, etc…)

• The optical port out the front of the line-card is the aggregate of all the optical modules.

Optical modules

MPOadaptor


Surface-Mount Transceivers

• Reflex LightABLE® Optical Modules– Module allows for 4+4,

12, and 24 optical channels

– Form factors under 16-mm x 8-mm x 3.5-mm

– Surface-mount and wirebond compatible

– Standard MT-style optical interface

– Data rates from 3.5-Gbps to 12.5-Gbps


Execution Plan + ScheduleExecution Plan + Schedule


28

Project Task Date Started Date Completed

Phase 1: Benchmark survey and specification definition

Benchmark survey of available technologies Mar-10 May-30

Choose feasible candidates of piece-part technologies Apr-10 Jun-10

Specification definition of demonstrator and sub-units May-10 Jun-10

Definition of test vehicles, standards and methods Jun-10 Jul-10

Phase 2: Piece-part technologies research

Optical interface Jun-10 Aug-10

Waveguide material and components Jun-10 Aug-10

Opto-electric PCB Jul-10 Jan-11

Optoelectronic device and transceiver (Tx/Rx) modules Jul-10 Nov-10

Optical modeling & simulation Jul-10 Aug-10

Optical coupling device and optical connector Jul-10 Aug-10

Phase 3: Demonstrator integration and test

Demonstrator architecture definitions Aug-10 Feb-11

Subassemblies fabrication and test Oct-10 Mar-11

Demonstrator integration and test Feb-11 Apr-11

Phase 4: Project Report

Establish optical backplane Interconnect practice guidelines Dec-10 Jun-11

Complete project report Jun-11 Aug-11

Issue project report Sep-11 Sep-11

Interested ParticipantsInterested Participants

29

• 3M, Denny G. Aeschliman• Alcatel-Lucent, Joe Smetana• Albemarle, Guillaume Artois• Celestica, Thilo Sack• Cisco, Wei Xie• Cisco, Li Li• Cisco, John Duffy• Cisco, Jie Xue• Conpart, Helge Kristiansen• Dow, Tom Sutter• Ericsson, Alessandro Alquati• Fujitsu, Tetsuro Yamada• Huawei, Danny Tu• Huawei, Sang Liu• Huawei, Xi Jin• Huawei, Shaoyong Xiang• Juniper Networks, Mark Marino

• Meadville, Chris Katzko• Meadville, Marika Immonen• Meadville, Tarja Rapala• Northrop Grumman, Daniel J. Smith• NSN, Dietmar Breisacher• National Semiconductor, Hau Nguyen• Optical Interlinks, Bruce Booth• Promex, Dick Otte• Rogers Corporation, Diana Williams• Reflex Photonics, David Rolston• Xyratex, Richard Pitwon


ContactsContacts

30

• Jack Fisher (HDP User Group)– Project Facilitator

– [email protected]

• Marika Immonen (Meadville Group, Finland)– Project Leader


• Marshall Andrews (HDP User Group)– Executive Director


• David Rolston (Reflex Photonics, Canada)– Contributor of the Optically Enhanced Backplane



APPENDIX: SUB-TEAM PLANS


31

Optical Interconnect ProjectOptical Interconnect Project

Subteam: Waveguides

Members: Jack Fisher, Shaoyong Xiang, Richard Pitwon, Marika Immonen, Bruce Booth, Tom Sutter, Denny Schilman

12 Apr 2010 32

Scope/ Subteam: WGsScope/ Subteam: WGs

• Scope– Definition of Emulator Specs

• Link budget, data rate, technology options, wavelength, channel no...=> spec list + reqs for channels

• => Required optical components/ functions (P2P, P2MP, cross-connects, curves,...)

• Key metrics (<= priorities) – performance (loss, power, density..), reliability, cost, maturity, availability,...?

– Reference design• Ref. design = > Physical specs + layout

– Review + benchmark• Available waveguide materials vs. Spec. List (MMF ref.)

• Maturity analysis (field data/ref. users, cost, ...)

© HDP User Group International, Inc.

33

Scope/ Subteam: WGsScope/ Subteam: WGs

• Scope, cont.– Qualification

• Selection of 1-3 mtrls for evaluation (MMF-ref.?)

• TV(s) + test plan– Variables (e.g.)

» Builds: layer count (ele), surface/inner optical

» Designs: Linears, bends, crossings, 1xN’s,..

» ....

– Outcome • Feasible mtrls base (vs. Spec list)

• Feasible optical board builds

• Design rules + boundary cond’s – optical, opto/mech.,..


34

Action Plan + ScheduleAction Plan + Schedule


35

Task Target Schedule Actual

Emulator Specs 30/4/2010

Reference Design

Review + Benchmark 16/5/2010

TV Board design 6/6/2010

Materials supply 20/6/2010

Board manufacturing 11/7/2010

Testing + Qualification

5/9/2010

Optical Interconnect ProjectOptical Interconnect Project

Subteam: Connectors and Components

Members: Jack Fisher, Richard Pitwon, Marika Immonen, Tarja Rapala, Shaoyong Xiang

20 Apr 2010 36

Scope / Subteam: CCsScope / Subteam: CCs

• Scope– Review of optical PCB connector research

• Identify application spaces e.g. Telecom, IT, HPC, Military, Aerospace

• Summarise activities by major connector companies including Tyco, Molex, FCI and end users e.g. IBM Research, Intel

– Connector survey• Collate different connector types for different applications – passive,

active (embedded transceiver), in-plane, out-of-plane, waveguide to fibre, waveguide to waveguide

• Identify relevant metrics: coupling loss, mating cycles, repeatablility, reliability, cost, availability, ...


37


• Scope continued– Component survey – optical engines

• Sources: Reflex Photonics, Avago, Zarlink, Conjunct, MergeOptics (now part of FCI), ...

• Metrics: Bandwidth (data rate), channel number, power consumption per channel, interface (e.g. lens array, free space)

– Component survey – materials• Low cost materials to meet repeatable high precision robust

alignment and connection requirements e.g. Liquid crystal polymer

• Metrics: Cost, tolerance (required prob 1 μm - 4 μm), strength, cTe, …


38


• Scope continued– Reference Design

• Passive parallel optical connector e.g. MTP allowing MT patchcords to connect to optical PCB in-plane or out-of-plane (embedded mirrors), tbd

• Active connector e.g. Optical engine based solution connecting directly via lens array or short length of fibre to optical PCB, tbd

– Test and Qualification• Coupling loss profile vs mating cycles vs applied strain

• Shock / vibration testing of optical connectors


39


• Outcome / Deliverables– Characterisation / qualification report

• For passive and active connectors under test

– Cost breakdown• Connector cost

• Possibly to form the basis of copper optical intercept calculation (this was an iNemi goal, but wasn’t reliably realised)

– Standardisation activities• New standards or upgrade to current standards for testing of

optical PCBs and optical PCB connectors e.g. IEC


40

Action Plan + ScheduleAction Plan + Schedule


41

Task Target Schedule Actual

Review of optical PCB connector research

30/05/2010

Connector survey 30/06/2010

Component survey 30/06/2010

Reference design 31/09/2010

Materials/component supply

31/11/2010

Manufacture of connector prototypes

31/01/2011

Testing + Qualification complete

31/03/2011

©hdp user group international, inc.1 optical interconnect project definition stage project may 20,...

Documents