www.laser focusworld.com October 2018

Photonics technologies target health monitoring PAGE 32

Simulation aids ultrafast laser source design PAGE 35

Asphere polishing achieves sub-100 nm form error PAGE 43

Vibration control for superresolution microscopesPAGE 48

®

Optical tweezers trick the brain; Single-cell metabolic imaging PAGE 39

Photonics Technologies & Solutions for Technical Professionals Worldwide

Silicon photonics speeds datacenter networking PAGE 27

1810LFW_c1-c5.indd 1 1/16/19 10:03 AM

Switch

Optics

Relative bandwidth increase since 2010 (= 1×)

2018

200G

12.8T

2016

100G

6.4T

2014

100G3.2T

2012

40G1.28T

2010

40G0.64T

25×

20×

15×

10×

5×

0×

Reprinted with revisions to format, from the October 2018 edition of Laser Focus World. Copyright 2018 by PennWell Corporation

Integrated optics permeate datacenter networksCYRIEL MINKENBERG, GREG FINN, and NICK KUCHAREWSKI

Datacenter networks are facing a key in-flection point: while the reach of electri-cal interconnects keeps shrinking with increasing signaling rates, the capabil-ities of merchant switch silicon—de-fined as off-the-shelf, silicon-chip-based switches—are outpacing those of plug-gable optical modules. This has created an optical bandwidth crunch that pre-vents optimal use of the most valuable assets—the servers.

Emerging technologies promise to ad-dress these issues by co-packaging fi-ber-optics links directly with the switch application-specific integrated circuit (ASIC). With production deployments planned in the near future, integrat-ed optics will free datacenter networks from the cost, power, density, and ar-chitectural constraints limiting conven-tional optical interconnect solutions.

The state of datacenter networksOver the past 10 years, the Ethernet ecosystem has come a long way. Networking gear was highly vertically integrat-ed, based on proprietary hardware and software, leading to a poor price/performance ratio. The resulting lack of global

network bandwidth im-posed strong constraints on workload placement and was a poor match for data-intensive work-

loads. In effect, the network gated ag-gregate performance, posing a major obstacle to improving server utilization.

Many of the changes necessary to dismantle vertical integration in the networking ecosystem have material-ized.1 This has been accomplished by far-reaching disaggregation (decoupling hardware from software) and commod-itization, as exemplified by open-source-design networking boxes built around merchant switch silicon, open-source networking software stacks, and the advent of software-defined networking (SDN) and programmable forwarding and data planes.

Despite these improvements, the fun-damental problem of global perfor-mance being gated by scarce networking resources still exists. Disaggregation has uncovered a new bottleneck—that is, the burden of cost has shifted from the switch and router boxes to the optical

links that are needed to interconnect them in a warehouse-scale datacenter.

Merchant silicon switch ASICs have been on an exponential growth curve. Since 2010, single-chip capacity has doubled every two years through a combination of increasing channel count and rate. Optical modules, on the contrary, have been on a much more gradual incline, creating a gap-ing chasm in their respective levels of integration (see Fig. 1).

Currently, a single-switch ASIC provides as many as 256 channels at 50 Gbit/s each.2 Optical modules, by contrast, are severely under-integrat-ed, currently providing a maximum of four channels. This discrepancy is also reflected in the relative cost per capac-ity, which has dropped much faster for switch capacity, to the point where in some cases the cost of the optics to pop-ulate the interfaces of a 1RU (1 rack-unit) switch exceeds the cost of the unit itself.

This lack of integration can be at-tributed to several factors. First, opti-cal modules are complex micro-optical systems, comprising an array of discrete

FIGURE 1. Over the past 8 years, switch ASIC capacity

growth has far outpaced that of optical modules; this widening gap is also reflected in relative cost per capacity (cost per Gbit/s), which for single-mode fiber-optic modules is now higher than for Ethernet switches.

Photonic integrated circuits (PICs) and silicon-photonics-based optical engines will expand datacenter network potential over the next decade.

P H O T O N I C S F O R D A T A C E N T E R S

In-Package-Optics

P H O T O N I C S F O R D A T A C E N T E R S

components from often widely differing technologies, typically hand-assembled into optical circuits. And although sili-con photonics has brought about improve-ments in transceiver manufacturing, the deployment model based on discrete plug-gable modules has not changed, preclud-ing more substantial integration benefits.

Another compounding factor is a splin-tering of the market and the associated plethora of optical interfaces defined by both standards and multi-source agree-ments (MSAs), with its negative conse-quences for economies of scale of each individual interface.

In addition to cost, faceplate density and power consumption have also become constrained by pluggable transceiver op-tics. Density is limited by the number of modules (of a given form factor) that can fit on a 1RU faceplate. This arrangement concentrates a lot of power dissipation in a small area and obstructs the airflow needed for cooling. A substantial amount of energy is consumed for electrical signal-ing from switch ASIC to faceplate-mount-ed modules across a PCB. As per-chan-nel signaling rates keep increasing, this requires more pre- and post-condition-ing to maintain signal integrity.

Enter integrated opticsTo ease the pluggable module bottle-neck, a high degree of integration can be achieved by co-packaging optical engines along with the main ASIC on a common substrate (see Fig. 2). This reduces input/output (I/O) power by limiting electrical signaling to intra-package distances, re-duces cost by increasing channel count per optical sub-assembly and by elimi-nating discrete transceiver packaging as well as high-speed PCB traces, and enables high-density optical faceplate connectors.

The practical realization of such an op-toASIC requires 1) high-density, low-cost photonic integrated circuits (PICs) along with energy-efficient driver/receiver circuits to implement the optical engines, 2) pack-aging technology to accommodate a sub-stantial number of such engines in a sin-gle package with a high-capacity switch ASIC, and 3) low-power electrical inter-faces to connect engines and ASIC.

Although there is little doubt about the appeal and technical feasibility of inte-grated optics, we also need to consider how this technology fits market needs, and how it slots into the existing supply chain and business model of datacenter networking gear.3-7

Silicon photonics platformsSilicon photonics allows the integration of many different optical devices onto a sin-gle silicon die.7 However, there are a va-riety of silicon photonics platforms and integration approaches. These are not all created equal when it comes to suitabili-ty for integrated optics solutions.

Ease of manufacturing should drive the choice of the optimal silicon photon-ics platform. A key observation is that get-ting light into and out of the silicon is the main driver of manufacturing cost and complexity. This leads to a conclusion that runs counter to the intuition learned from decades of scaling down device fea-tures in CMOS devices. In photonics, scaling waveguide geometries down to the deep sub-micron range turns out to be counterproductive to the core objec-tive of achieving high-throughput, high-yield manufacturing.

A multi-micron-waveguide platform has several crucial advantages over sub-micron platforms. A large-waveguide platform is much more tolerant to pro-cess variations, leading to higher yield. Large waveguides enable passive-align-ment structures for low-loss, high-through-put fiber attachment. They also exhibit lower optical loss and polarization-inde-pendent performance, can be designed to be single-mode, and can handle higher optical power. These characteristics also enable wavelength-division multiplexing (WDM), which greatly improves band-width density.

Another core advantage is the abili-ty to integrate devices from other mate-rials systems—for instance, III-V-based light sources—by means of low-loss, high-throughput passive alignment.

Next, despite the large geometry, tight waveguide bends can be manufactured, enabling compact structures to match the density (pitch) of both ASIC SerDes (seri-alizers/deserializers that convert data be-tween serial and parallel interfaces in each direction) and fiber arrays.

Another crucial component is the mod-ulator. A larger mode size enables the use of smaller and more energy-efficient elec-tro-absorption modulators (EAMs), as op-posed to the Mach-Zehnder-type mod-ulators needed in submicron platforms.

FIGURE 2. A conventional 1RU switch solution has faceplate-mounted, single-port pluggable optical transceiver modules. Integrated optics enable a solution in which a switch ASIC die and several multiport optical engines are assembled on a common substrate. Low-power electrical I/O optimized for intra-package reach drives the optical engines. All high-speed I/O is transported on and off the package on fibers attached directly to the optical engines and to faceplate-mounted high-density passive optical connectors.

FIGURE 3. Rockley Photonics’ prototype optoASIC comprises a Layer-3 routing ASIC with 12 100 Gbit/s Ethernet ports, analog driver and receiver circuits, and silicon PICs with optical modulators and photodetectors supporting parallel single-mode fiber. Light sources are external to the device; optical power is delivered to the transmit PICs via additional input fibers. The total power per 100G port, including lasers, is under 3 W—lower than the typical power of a QSFP28 transceiver alone, yet including the switching functionality.

Package

Fiber ribbons

Short-reachelectrical links

Analog CMOSchiplets & silicon

photonic PICs

Organic substrate

SwitchASIC

The Silicon Photonics platform for the Age of AI, Mobility and Autonomy.

For more information please contact us: contact@rockleyphotonics.comor visit our website: www.rockleyphotonics.com

P H O T O N I C S F O R D A T A C E N T E R S

This increases bandwidth density and low-ers power consumption. Moreover, these modulators can be driven by single-stage, low-voltage CMOS drive circuits, further increasing the energy-efficiency advantage, and have been shown to be amenable to operation beyond 100 Gbit/s.

Rockley Photonics has developed a tech-nology platform for In-Package-Optics solu-tions exhibiting all of these characteristics and has implemented a prototype optoA-SIC that demonstrates technical feasibili-ty and power-saving potential (see Fig. 3).8

To scale to higher channel counts and enable third-party integration, we envi-sion another optoASIC-based architecture that supports packaging a CMOS die—for instance, from a merchant switch sil-icon vendor, along with multiple optical engines from a silicon photonics vendor (see Fig. 4).

Implications for datacenter network architectureIntegrated optics will drive down the cost of optical links by giving rise to high-er-channel-count optical engines, lower total component cost (fewer parts, less packaging), fewer assembly steps, and high compound yield. Integration also reduc-es energy consumption by minimizing the energy spent on electrical I/O. In addition, the elimination of faceplate-mounted mod-ules improves airflow and increases the number of fibers that can be accommo-dated through dense optical connectors.

Perhaps even more important than im-proving cost, power, and density metrics are the potential implications for datacen-ter network architecture as a whole. This

technology marks a fundamental shift in the economics of reach—the very same economics that underpinned the archi-tectural decisions that have led to today’s network architectures.9 As soon as band-width-distance becomes much less of a constraining factor than it traditionally has been, architects will be free to de-sign towards more productive optimiza-tion criteria.

Datacenter networks will finally be lib-erated from the bandwidth-delay-product constraints that have so long dictated ar-chitectural decisions. With these limita-tions out of the way, the full potential of a datacenter’s computing power can be unlocked. The imminent era of integrat-ed optics will provide a sea of possibili-ties for repartitioning existing designs and inventing completely new ones, from the

chip and box level all the way up to the entire datacenter.10 The optoASIC revo-lution has only just begun.

ACKNOWLEDGEMENTIn-Package-Optics is a trademark of Rockley Photonics Limited.

REFERENCES 1. J. Hamilton, “Data center networks are in

my way,” Stanford Clean Slate CTO Summit, Stanford, CA (2009).

2. T. Prickett Morgan, “Feeding the insatiable bandwidth beast,” nextplatform.com (Apr. 30, 2018).

3. B. Arimilli et al., “The PERCS high-performance interconnect,” IEEE Hot Interconnects 18, Mountain View, CA, 75–83 (Aug. 2010).

4. K. Hasharoni et al., “A high end routing platform for core and edge applications based on chip to chip optical interconnect,” Proc. OFC, paper OTu3H.2, Anaheim, CA (2013).

5. A. V. Krishnamoorthy et al., J. Lightwave Technol., 35, 15, 3103–3115 (Aug. 2017).

6. M. Akhter et al., “WaveLight: A monolithic low latency silicon-photonics communication platform for the next-generation disaggregated cloud data centers,” IEEE Hot Interconnects 25, Santa Clara, CA, 25–28 (Aug. 2017).

7. C. Minkenberg et al., J. Opt. Commun. Netw., 10, 7 (Jul. 2018).

8. G. Reed and A. Knights, Silicon Photonics: An Introduction, Wiley (Mar. 2004).

9. C. Minkenberg et al., “Redefining the economics of reach with integrated optics,” IEEE HPSR ‘18, Bucharest, Romania (Jun. 2018).

10. N. Kucharewski et al., “Network architecture in the era of integrated optics,” Proc. OFC, paper M3J.7, San Diego, CA (2018).

Cyriel Minkenberg is a system architect, Greg Finn is vice president of business devel-opment, and Nick Kucharewski is chief com-mercial officer located in the San Jose, CA of-fice, all at Rockley Photonics, Oxford, England; e-mail: cyriel.minkenberg@rockleyphotonics.com; www.rockleyphotonics.com.

FIGURE 4. To enable third-party integration, optoASICs must support packaging CMOS dies along with optical engines from different vendors; this requires close collaboration and/or standardization to ensure interoperability in terms of high-speed electrical interfaces, optics management interfaces, substrate material, thermal management, and assembly flow.