topologies and technologies for optically interconnected...

Topologies and Technologies for Optically Interconnected Multicomputers Using Inverted Graphs

Roger D. Chamberlain and Robert R. Krchnavek Department of Electrical Engineering

Washington University, St. Louis, MO 63130

Abstract

To successfully exploit the benefits of optical technology in a tightly-coupled multicomputer, the architectural design must reflect both the advantages of optics and the limitations of optics. This paper describes a class of such architectures, based upon inverted graph topologies. Two instances of this class (an inverted hypercube and an inverted mesh) are further explored to illustrate their properties. W e then consider the physical construction of these systems, demonstrating the relevant technological components necessary to manufacture a working system.

1 Introduction

The role of optics in computing has been de- bated for many years. The high bandwidth available with optical technologies has become pervasive in the telecommunications industry, with optical fiber the medium of choice for large digital networks. Optical interconnections have also entered the domain of local area networks with the FDDI standard. What has yet to happen, to any significant degree, is the use of optical interconnections within a single, tightly-coupled multicomputer.

The advantages of an optical interconnect are significant. The bandwidth available in optical media is significantly higher than what is currently available electrically, with Gbps data rates supported easily. This implies that bit-serial interconnects are a clear alternative, rather than the multi-conductor solutions used currently to increase data rates in electrically connected systems.

Since even Gbps data rates do not saturate the optical bandwidth, multiple sources can share the same physical “link” using wavelength division multiplexing (WDM), and all of the sources on the link can transmit simultaneously at their maximum data rate. Effectively, the optical media have a high signal fanin.

Figure 1: Fanin illustration

This is illustrated in Figure 1, where three processors are connected to a common optical link. Each of the processors transmits at a distinct wavelength (Pz transmits using wavelength Xi) , and receives on the other wavelengths. In contrast, in the electrical domain the sharing of a single link (e.g., a bus) requires that only one transmitter be enabled at any one time. Additionally, optical media do not incur capacitive loading effects, providing for a high fanout capability, and optical media have greater noise im- munity than electrical systems.

The lack of penetration of optical technologies into the design and construction of multicomputers can be attributed to a number of causes. First, the complexity of transmitters and receivers is greater when using optics. Signals must be converted between the electrical and optical domains, and devices to perform these tasks are complex, bulky, and (historically) very power hungry. For example, a few years ago, laser threshold currents were typically > 50 mA and actual drive currents even higher. This required a separate driver circuit which added significantly to the complexity of the system.

Second, the complexity of the transmission medium is greater when using optics. In the electrical do-

0-8186-5830-4/94 $3.00 0 1994 IEEE

T

255

main, traces on printed wiring boards (PWB) are a mature technology. Thousands of point-to-point connections between ICs on a PWB can be manufac- tured both easily and reliably. In addition, high density board-to-board connectors (e.g., 25 mil pitch) can be used to interconnect PWBs with negligible loss in electrical signal integrity at current data rates. On the other hand, optical interconnection technology has been plagued by excessively lossy waveguides and limited to material systems incompatible with the large areas associated with the PWBs in multicomputer systems (e.g., LiNb03). Furthermore, optical coupling between lasers/detectors and waveguides continues to be a problem.

Finally, until recently electrical alternatives have been a viable approach to multicomputer interconnection designs. While data rate requirements were low, there was insufficient motivation to take on the complexity of an optical interconnection scheme. However, as bandwidth requirements grow, and integrated circuit performance levels increase, optical alternatives are increasingly viewed as necessary to maintain bal- anced performance between computation and communication.

The high bandwidth of optical interconnects, in conjunction with the complexity of working in the optical domain, suggests that optical interconnects not be a simple reimplementation of existing electrical designs. In particular, the system architecture should in- corporate knowledge of the strengths and weaknesses of the implementation technology. The challenge then is to design alternative multicomputer architectures that take advantage of the benefits of the optical media and diminish the effects of the media’s disadvantages.

2 Inverted graphs

The architectural design for an optically interconnected multicomputer must consider the strengths and weaknesses inherent in the optical technology. The strengths include:

High bandwidth - Gbps data rates are easily supported.

Multiple signal sources - WDM allows for a number of high bandwidth sources per link.

0 High fanout - A large number of receivers can be supported without capacitive loading effects.

The weaknesses can be summarized as follows:

(a) traditional hypercube

(b) inverted hypercube

Figure 2: 3-dimensional hypercube topologies

0 Complex transmitters and receivers - Signals must be converted between the optical and electrical domains.

0 Complexity within the optical medium ~ The manufacturing processes for optical interconnects are far behind those for electrical PWBs.

These strengths and weaknesses imply several distinc- tions between an architecture based on optical media and one based on electrical interconnections. First, high bandwidth allows one to restrict consideration to bit-serial data pathways without sacrificing effective data rate. Second, high fanin and fanout implies that the interconnection topology need not be limited to point-to-point links between pairs of processors. Since a single optical link can support numerous senders and receivers simultaneously, each link can serve as a shared resource common to more than two processors. Third, the complexity of the transmitters and receivers implies that the number of ports per processor must be limited.

Current massively parallel processing (MPP) system designs are often built around a scalable graph topology (e.g., nCUBE 2 hypercube, Intel Paragon mesh), where the processors are associated with vertices in the graph and the edges of the graph identify point-to-point communications links between processors. Figure 2(a) shows a 3-dimensional, 8-processor traditional hypercube topology. The boxes in the figure correspond to processor-memory pairs, and the interconnecting lines represent communications links.

256

n n

Y Y

(a) traditional mesh

(b) inverted mesh

Figure 3: Mesh and inverted mesh topologies

We propose a class of interconnection topologies where one reverses the role of the graph vertices and edges in defining the processors and links. We call the proposed topologies inverted graphs [3]. In inverted graph topologies, edges represent processor-memory pairs and vertices represent interprocessor communications links. This is shown pictorially in Figure 2(b), which illustrates the same 3-dimensional hypercube graph, now implementing a 12-processor inverted hypercube. Figure 3 illustrates both a traditional 2- dimensional mesh topology and an inverted mesh.

The inverted graph topologies have been proposed primarily with the goal of exploiting the benefits and minimizing the costs associated with an optical interconnection scheme. The interprocessor communication links are bit-serial, multipoint connections, tak- ing advantage of the high bandwidth, high fanin, and high fanout capabilities of optical media. In addition, there are only two ports per processor, reflecting the complexity of the signal translation devices.

One of the clear advantages of inverted graph topologies is how various parameters scale as the number of processors is increased. In a hypercube graph, the number of vertices v = 2d, where d is the dimension of the hypercube, and the number of edges e = d2d-1. In the traditional hypercube topology, the number of processor-memory pairs P = v , the number of links L = e , and each processor has d ports. In an inverted hypercube, the number of processors P = e = d2d-', the number of links L = U = 2d, and

each processor has 2 ports. As the number of processors in the system grows, the total number of links required to construct a traditional hypercube system becomes quite large. This is not as much of a concern with electrical interconnections, as the cost for each connection is relatively small. However, with optical links, the penalty (in terms of system complexity) is quite high. This is contrasted with the total number of links required to construct an inverted hypercube system. Here, the number of links is actually smaller than the number of processors. For example, in a 1024-processor hypercube (d = 10) the number of links L = 5120, while in a 1024-processor inverted hypercube (d = 8) the number of links L = 256. Total communications bandwidth does not suffer, because as the inverted hypercube scales to larger processor populations, the bandwidth utilized in each physical link goes up. Although not as dramatic in the inverted mesh topology, the total number of links required to construct an inverted mesh system grows slower than what is required in a traditional mesh.

Another important parameter associated with optically interconnected systems is the number of ports required at each processor, Since the electrical-to- optical transmitters and optical-to-electrical receivers are significantly more complex than their purely electrical counterparts, we wish to limit the number needed for system implementation. In a traditional hypercube the ports per processor grows logarithmi- cally with the number of processors, while in the inverted hypercube the ports per processor is fixed at two (independent of system size). A traditional mesh topology has four ports per processor, again larger than the two needed for an inverted mesh.

3 Using inverted graph topologies

For an inverted graph topology to be useful, its properties must first be well understood. Traditional graph topologies (e.g., mesh, hypercube) have been extensively studied, resulting in elegant processor labeling schemes, minimal message routing algorithms, embeddings of one graph onto another, and a host of other useful properties. Here, we briefly examine some of the properties of an inverted hypercube and an inverted mesh, showing processor labeling schemes for each. In addition, we show an embedding for the inverted mesh that makes it particularly attractive for further consideration.

257

(a) traditional hypercube

(b) inverted hypercube

Figure 4: Hypercube and inverted hypercube labeling

3.1 Inverted hypercube

Traditional hypercube processor labeling is illustrated in Figure 4(a) for a 3-dimensional hypercube. For an n-dimensional hypercube, processors labels are n-bit binary numbers where the ith bit locates the processor in the ith dimension of the hypercube graph. This labeling scheme has a number of very useful properties [16]: nearest neighbor processors are easy to identify, minimal routing algorithms are well known and simple to implement, etc. The labeling scheme proposed here for the inverted hypercube will lever- age many of these properties.

The inverted hypercube labeling scheme is illustrated in Figure 4(b) for a 3-dimensional inverted hypercube. Using a three symbol alphabet (O, l , - ) , processor labels are derived from the traditional hypercube graph labels that correspond to the two communications links to which the processor is attached.

At the symbol position where the two link labels differ, the processor label is given the “-” symbol. At the remaining symbol positions, the processor label matches the two link labels. This scheme was pre- sented by Dowd and Jabbour [6] for labeling links in general k-ary n-cubes. Consider the topmost processor in Figure 4(b). The two communication links that it connects to are labeled 110 and 111. This results in a processor label of 11-.

Each processor has two ports, labeled “0” and “1.” The port labels are determined by examining the symbol position (in the two communications links) where the processor label is “-.” For example, processor 00- is connected to links 000 and 001, with port 0 attached to link 000 and port 1 attached to link 001.

Using this labeling scheme, nearest neighbor processors (those directly connected via a communications link) can be identified by comparing the two processor labels. Ignoring the symbol positions where either label contains a “-” symbol (i.e., treating that symbol as a “don’t care” condition), if the remaining portion of both processor labels are equal the processors are adjacent (they have a common communications link). In a similar fashion, the number of intermediate hops required to communicate between any two processors can be determined from the Ham- ming distance between their processor labels when the “-” symbols are treated as don’t care conditions. The number of hops is the Hamming distance plus one. For example, messages between processor 00- and processor 11- must traverse three hops (their labels differ in the first two symbol positions), and messages between 10- and 0-1 must traverse two hops (their labels differ in only the first symbol position).

3.2 Inverted mesh

The labeling scheme for an inverted mesh depends upon the ability to embed a traditional mesh into an inverted mesh. Consider the inverted mesh of Fig- ure 5 . Here, a 3 x 3 graph is used to implement a 16-processor inverted mesh. This system becomes more interesting if it is rotated clockwise 45”, as in Figure 6. The processor labels are derived from the traditional mesh labeling scheme, with the first digit indicating the processor location in the x dimension and the second digit indicating the location in the y dimension. Note that a traditional mesh is embedded in the graph, with the ability for each processor to im- mediately communicate t o its neighbor to the north, south, east, and west. Each processor also has diagonal connections, in addition to the traditional mesh message pathways.

258

Figure 5: Inverted mesh topology Figure 7: Inverted mesh logical connections

the diagonal connections are northwest-southeast. Since a traditional mesh can be embedded in the

inverted mesh, algorithms and applications that currently take advantage of a mesh interconnection can be executed unaltered on an inverted mesh. This is an important advantage, since a significant amount of algorithm design and effort has been expended devel- oping applications for systems interconnected with a traditional mesh topology.

4 Implementation of an inverted mesh using optical interconnections

Figure 6: Inverted mesh interconnect and labeling 4.1 Physical layout

The diagonal connections are examined in Fig- ure 7, which shows the logical point-to-point connections supported by the inverted mesh of Figure 6. In this figure, the physical multipoint optical connections have been replaced by lines that indicate the nearest neighbor processors. The embedded mesh can now be clearly seen, along with the additional diagonal connections. Using the diagonal connections, the maximum length path in a 4 x 4 traditional mesh topology (6 hops from processor 00 to processor 33) is half that value (3 hops) in a 4 x 4 inverted mesh.

The inverted mesh topology is closely related to the X-net mesh used in the MasPar MP-1 [9]. However, the X-net mesh has diagonal connections across every square, while the inverted mesh has diagonal connections across every other square. Each processor can determine whether its local diagonal connections are northeast-southwest or northwest-southeast by examining the 2 and y coordinates of its processor label. If both coordinates are even or both coordinates are odd, the diagonal connections are northeast-southwest. If one coordinate is even and the other coordinate is odd,

Before we describe a proposed physical layout, we first consider some of the quantitative factors associated with the processors. Given a current high- performance microprocessor as the computational resource (e.g., something with the power of the DEC Al- pha chip), 400 MIPS is a reasonable instruction rate. The Alpha chip occupies about 60 cm2 of board area and consumes over 20 W of power. Memory capac- ity is primarily limited by the board area available, but 64 Mbyte or larger memory sizes are likely to be required. Using currently available 16 Mbit memory chips, a 64 Mbyte memory will require 80 cm2 of board area and consume approximately 15 W. Allot- ting 10 cm2 and 15 W for a communications controller, transmitters, and receivers to route, send, and receive messages, the processor-link interface will require a relatively small fraction of the area and power budget for the processor. The example processor modules described above require approximately 150 cm2 of real estate. In addition, the system can take advantage of further advances in microprocessor and VLSI technology as it becomes available.

259

We next consider transmitters (electrical-to- optical) and receivers (optical-to-electrical). Previ- ously, laser diodes had threshold currents of approximately 50 mA and typical operating currents of 60- 100 mA. This requires complex drive circuitry usually consisting of high power, high speed GaAs ICs. Re- cently, we have seen a steady decrease in the threshold current [ 5 ] . Lasers with threshold currents of < 1 mA have been fabricated and we have reached a point where direct laser diode driving by CMOS circuitry is expected. Furthermore, with CMOS circuitry capa- ble of operating at GHz rates expected in the near fu- ture and integrated detectors for feedback power con- trol becoming more common, we can expect the high speed modulation of laser diodes to be not significantly more complicated than driving a conventional electrical transmission line. Similar advances have been seen in receiver design. Recent work [4] has demonstrated a 5 Gbps integrated, differential optical receiver with simulation indicating > 10 Gbps operation is possi- ble with modifications to the biasing and packaging. We can expect this trend in optical transmitters and receivers to continue. We are also beginning to see commercial ventures in this area. For example, Hon- eywell [8] has recently introduced a monolithic optical receiver currently a t 400 Mbps using technology that can be extended to > 1 Gbps (Honeywell’s descrip- tion). However, while these advances in laser transmitter and receiver technology are being driven by fiber-based SONET systems, board-level optical technology such as waveguides, passive coupling schemes, and devices still require considerable effort.

As an example, we will consider a 1024-processor multicomputer using an inverted mesh topology. This system could be arranged as a stack of 16 multiprocessor boards as shown in Figure 8. What is not shown, however, is the interconnection of all of the individual processor modules.

Note that the distance from one processor module to another can be several cm while the furthest distance can be over a meter. Although fiber optics and fiber related devices have the lowest optical loss, there is considerable difficulty in implementing a tightly- coupled system using exclusively fiber optic technology. For example, routing optical fiber to each processor would increase the manufacturing complexity, reduce the density, potentially produce cooling problems, and severely limit the incorporation of optical devices. Instead of utilizing optical fiber, we propose a combination of optical waveguides, optical fiber, and free space interconnects - each type of interconnect being used where appropriate. In particular, optical

waveguides would be used to connect individual processors, while optical fiber and/or free space interconnects could be used to interconnect the boards.

The success of an optical interconnection scheme will depend heavily on the physical layout dictated by the interconnection diagram (Figure 5 or 6). For example, in electrical interconnections, crossovers are a well established technology. Multi-layer PWBs with 10 or more electrical interconnection layers are common. On the other hand, multi-layer optical waveguide technology is essentially nonexistent. Therefore, if the physical optical interconnection requires optical paths to crossover, this significantly complicates the technology.

Figure 9 is a drawing of a section of a proposed physical layout for the optical interconnection of the processors in the inverted mesh topology. More specifically, Figure 9 shows a repeat unit for the topology. Proceeding with our example of a 1024 processor system on 16 boards, each board would consist of 64 processors. The physical layout shown in Figure 9 would be copied and translated (with appropriate processor renumbering) to layout all 64 processors on one of the 16 boards.

Several important points can be made regarding Figure 9. First is the observation that if the optical interconnects are allowed to go through the individual processor boards, no optical crossovers are required. This significantly reduces the complexity of implementing the optical interconnects. To enable the optical interconnects to go through the individual processor boards, we propose individual processor boards be fabricated as daughter boards to the larger, optical interconnect, mother board. In this way, the optical path goes below the processor board. This is indicated in the figure by the dashed lines showing waveguides below processor modules.

It should be noted that the inverted mesh topology has a significant advantage over the inverted hypercube topology with resect to eliminating optical crossovers. In the inverted hypercube, optical crossovers are required even if the optical path is allowed to pass below individual processor boards.

The motherldaughter scheme presents a question regarding the placement of optical sourcesldetectors. Should the sources/detectors be placed on the optical interconnect board or the individual processor boards? If the optical sources/detectors are placed on the individual processor boards, this facilitates easy replacement if an optical source fails (optical sources have considerably lower reliability than conventional electronic components and are a likely failure mechanism).

260

/ l m

Figure 8: Tightly-coupled, 1024 processor system

Figure 9: Physical layout

Replacing a failed source merely requires unplugging the processor board and replacing it with a new one. However, placing the optical source on the processor board presents an extremely difficult optical alignment/coupling problem. The plug-in processor board must provide a means for the optical signal to align to a coupling device to within a few micrometers. In addition, because different processor boards use different optical wavelengths to communicate, several different spare processor boards would have to be available. Alternatively, if the optical sources/detectors are part of the optical interconnect board, processor board replacement does not require accurate optical alignment and only one type of processor board is required since only electrical signals travel on the board. The difficulty with this scheme is replacing failed optical sources. We describe briefly in Section 4.3 a scheme for easily replacing failed optical sources while main- taining high coupling efficiency.

Another observation from Figure 9 is the connection node for the processors. In an electrical interconnect system, a connection node is simply a multipoint connection of PWB copper traces. In an optical system, such a node must be carefully designed to uniformly distribute the optical signal without los- ing light. This 4 x 4 optical coupler is described in Section 4.4.

Finally, the optical waveguides need to be fabricated on a board that supplies power and physical support to the individual processors. The area of this board is approximately 1 m2. This places several requirements on the optical waveguide material system.

4.2 Material considerations

Material selection is a critical concern for the fabrication of the waveguides described above. The cri- teria used to evaluate dielectric materials for optical waveguides consists of optical properties, electri-

261

cal properties, processing properties, and the physical and mechanical properties. Previous work in low loss acrylic polymer waveguides has shown that organic- based optical waveguides have several advantages over glass, semiconductors, or LiNb03. First and foremost, acrylic waveguides have been demonstrated at low loss (M 0.1 dB/cm) [lo]. They can be fabricated over large areas, are generally low in cost, are compatible with a wide range of electronic materials including the boards used in PWB technology, can be processed at low tem- peratures, and can be custom formulated to meet the needs of a particular system [11, 121. They are also easily photopatterned allowing for the fabrication of the passive device structures described in Sections 4.3 and 4.4.

In addition to acrylic based technology, several other organic polymers are potential candidates for board-level optical interconnects. Two of the more common material systems that are seeing widespread use as dielectric layers in multi-chip modules are poly- imide and benzocyclobutene (BCB). In comparison to acrylic technology, these material systems generally exhibit higher temperature stability allowing for higher temperature post processing. However, in the mother/daughter physical layout described, post- processing on the optical interconnect board is minimal and 350 - 400°C temperature stability may not warrant the higher cost of these material systems over the more common acrylic-based technology.

We have fabricated low loss optical waveguides using acrylics as well as BCB. The losses are approximately 0.1 dB/cm at 633 nm for the acrylic- based waveguides. BCB has loss minima at 850 and 1300 nm. Loss measurements for the BCB waveguides are underway. Preliminary results indicate that BCB waveguides will also exhibit losses approximately 0.1 dB/cm.

Consider the waveguide runs shown in Figure 9. Assuming a square processor board, the length of one side is approximately 13 cm. The longest distance for a waveguide run is approximately 40 cm. This translates into an optical loss about 4 dB (assuming perfect optical coupling). This is well within the range of modern, high-speed, optical receivers.

4.3 Optical coupling

Lack of high-efficiency passive coupling schemes represents one area which prevents optics from play- ing a broader role in high speed electronic systems. Many current approaches to the coupling of optical fibers to optical waveguides on substrates are confined to the perimeter regions of the substrates using vari-

ous so-called butt-coupling approaches [17]. Further- more, it is common to use active alignment wherein light carried by the illuminated fiber and waveguide is monitored and optimized continuously during the mechanical coupling process [13]. Obviously, this is a time-consuming and costly technique that does not lend itself to low-cost manufacturing techniques.

In the proposed multicomputer, a method of optically interconnecting the mother boards is required. Two technologies come to mind: optical fiber and free space interconnects. For this work, we propose using optical fibers. In addition to being immune to dust (a serious concern for free space interconnects), optical fiber interconnects between boards provide a natural point for connecting the multicomputer into a larger optical network and effectively increase the processor count for the multicomputer. In employing optical fiber to interconnect the various mother boards, it is critical to identify techniques to passively couple the optical fibers to the ridge waveguides on the mother boards. The challenge is to maximize the coupling efficiency between the optical fiber and the waveguide. This in turn highlights the need for a technique to mechanically guide the fiber exit face precisely to the waveguide entrance aperture with ease.

We have developed a novel coupling concept wherein the same polymer waveguide technology which is used to fabricate the optical waveguides is simultaneously used to form self-aligned mechanical alignment ways for properly positioning the optical fibers [2]. See Figure 10. These alignment ways require that optical fibers be fabricated with D-shaped transition regions near their coupling ends. As the fibers, called D-fibers, are inserted into the alignment ways they become precisely positioned in con- tact with the ridge polymer optical waveguides. This novel approach allows coupling of standard multimode graded-index fiber to ridge waveguides, with little or no active optimization, anywhere on the substrate sur- face - not just near the substrate perimeter. This technique has recently been extended to single-mode fibers/waveguides. Coupling measurements are underway.

A second coupling issue involves coupling the optical waveguides on the mother board to the optical sources and detectors. This coupling issue is more complex than the optical fiber/waveguide coupling described above because of the need to easily replace a defective optical source. The laser/fiber coupling problem has been the subject of considerable research [l, 141. Typically, an anisotropically etched sil- icon substrate is used to hold an optical fiber. Further

262

I alignment way ) polymer 71 9 f waveguide

optical fiber I (

(a) A top-view drawing of the passive coupling scheme that relies on alignment ways fabricated at the same time the polymer waveguides are formed.

index matching

cladding layer

waveguide area of encapsulation

/ D-fiber

(b) A side-view drawing of the fiber-to- waveguide coupling scheme using D-fiber.

core n

(c) A view of the D-fiber showing the lapped region.

(d) A scanning electron micrograph of the alignment ways.

Figure 10: Fiber to waveguide coupling using alignment ways

etching is used to produce alignment pads for the laser diode. The laser is then soldered in place producing a compact optical fiber coupled source. Similar techniques have been used to fabricate compact, multi- fiber, precision connectors. These devices generally use V-grooves to precisely hold the fibers in position. By combining these two techniques, one can achieve a manufacturable and repairable method of mounting optical sources into the mother board. This, in effect, produces a connectorized source. This technique has several advantages. The critical laser diode/optical fiber alignment is done during the manufacturing of the submodule and is separate from the assembly of the mother board. The assembly of these submodules into the optical interconnect board then becomes a

simple mechanical connection. This simple mechanical connection is also the method by which a defective laser source can be replaced.

4.4 Key passive elements

The architecture described here requires the simul- taneous transmission of multiple wavelengths on the same waveguide. It is clear that the bandwidth of the optical media is high enough to support multiple wavelength signals. However, in light of the schematic representation shown in Figure 9, the implementation of the connection node needs further explanation. The architecture requires multiple sources, each operating at a different wavelength, a means of coupling these

263

Figure 11: 10 x 10 optical coupler

multiple sources into a single optical waveguide, and then a method of separating the wavelengths at the receiving end. For the inverted mesh topology, we require 4 different wavelengths regardless of the number of processors. An ideal component to achieve the multiplexing of N wavelengths and subsequently distributing the multiplexed signals onto N optical waveguides is an N x N coupler. Conventional N x N couplers, consisting of stringing together 2 x 2 couplers, have several disadvantages. They are bulky, lossy, and produce very high power densities in the center element, which can lead to non-linear effects. Although the 4 x 4 optical couplers required for the inverted mesh topology can be fabricated using optical fiber and can be relatively compact, using optical fiber instead of waveguides would still present numerous manufacturing problems. For example, using optical fiber couplers would require numerous fiberlwaveguide in- terfaces which are difficult to manufacture, generally require active alignment, and can become a reliability problem. Recent experiments using Fourier-optics designs [7] based on earlier microwave antenna work [15] show that highly efficient N x N optical couplers can be fabricated that are wavelength insensitive. A scanning electron micrograph of a 10 x 10 optical coupler is shown in Figure 11. The actual coupling region measures approximately 100 pm x 750 pm and is fabricated using the same polymer resins used to fab-

ricate the optical waveguides. In fact, the polymer waveguides, the D-fiber alignment ways, and the 4 x 4 couplers are patterned simultaneously, in a single pho- tolithographic step.

The second most import passive element in the optical interconnection of this inverted mesh topology is a wavelength demultiplexing device. Several schemes have been proposed in the literature. In general, discrete micro-optical elements exhibit high performance but potentially suffer from poor manufacturability primarily due to the difficulty associated with integrat- ing the devices into a system. Integrated optical elements, on the other hand, are easily integrated into the system but usually do not perform to the level of a discrete micro-optical element. In this work, we anticipate using an integrated grating to separate the wavelengths at the receiver end. Because the inverted mesh topology only requires 4 wavelengths, they can be widely separated. This eases the requirements on the grating performance.

5 Conclusions

Replacing electrical wires with optical fibers in existing toplogies is not the way to exploit optics in large, tightly-coupled, multiprocessor systems. We must re- think the system architecture from top to bottom, tak- ing advantage of the benefits and limiting the penalties associated with optical technologies.

These advantages include high bandwidth, fanin, and fanout. The disadvantages include complex transmitters, receivers, and interconnection media. The inverted graph topology has been designed specifically to exploit these advantages and limit these drawbacks. Wavelength division multiplexing is used to share single links across a large number of processors, and the number of transmitters and receivers per processor is fixed at a low value (2). In short, the fanin and fanout requirements of an interconnection system are handled by the interconnection media rather than the processors themselves.

We must also choose the appropriate optical technology, and not indiscriminately use fiber when an alternative is better suited to the job at hand. Poly- mer waveguides can now be routinely constructed with low loss ( 5 1 dB/cm), support large processing areas (printed wiring board sizes), and use materials that are compatible with existing printed wiring board technology.

In conclusion, we believe that optical interconnection technology has matured to the point where it is

264

usable in the construction of tightly-coupled digital systems. The pieces of the puzzle are all in place to build systems that exploit the advantages of optics.

Acknowledgements

This work was supported in part by the National Science Foundation under grant number MIP-9309658 and the Department of Defense under grant number F30602-93-C-0004. The authors wish to thank the following individuals for their inputs into this work: Christopher Phelps, Timothy Barry, Kenichi Naka- gawa, Marcello Cordaro, and Dr. Daniel Rode.

References

C. A. Armiento, M. Tabasky, C. Jagannath, T. W. Fitzgerald, C. L. Shieh, V. Barry, M. Rothman, A. Negri, P. 0. Haugsjaa, and H. F. Lockwood. Passive Coupling of InGaAsP/InP Laser Array and Singlemode Fibers Using Sili- con Waferboard. Electronics Letters, 27(12):1109 (1991).

T. S. Barry, M. H. Cordara, R. R. Krchnavek, K. Nakagawa, C. W. Phelps, and D. L. Rode. Efficient Optical Fiber-To-Waveguide Coupling Suitable For Passive Alignment. In Proc. of the 43rd Electronic Components and Technology Conf., June 1993.

R. D. Chamberlain and R. R. Krchnavek. Archi- tectures for Optically Interconnected Multicom- puters. In Proc. of GLOBECOM93, IEEE, De- cember 1993.

G.-K. Chang, T . P. Liu, J . L. Gimlett, H. Shirokmann, M. Z. Iqbal, J . R. Hayes, and K. C. Wang. IEEE Photonics Technology Letters, 4(4):384-386, April 1992.

T . R. Chen, B. Zhao, Y. H. Zhuang, A. Yariv, J. E. Ungar, and S. Oh. Appl. Phys. Lett., 60(15):1782-1784, April 1992.

P. W. Dowd and K. Jabbour. Spanning Multiac- cess Channel Hypercube Computer Interconnec- tion. IEEE Trans. on Comp., 37(9):1137-1142, September 1988.

C . Dragone. IEEE Photonics Technology Letters, 3:812, 1991.

[8] Honeywell, Inc. Monolithic Optical Receiver, OEIC RX-400. Honeywell Systems and Research Center, Bloomington, MN.

[9] K. Hwang. Advanced Computer Architecture: Parallelism, Scalability, Programmability. Mc- Graw Hill, New York, NY, 1993.

[lo] R. R. Krchnavek, G. R. Lalk, and R. Denton. In Proc. of the Symp. on Materials for Optical Information Processing, pages 95-100, Materials Research Soc., 1992.

[ll] R. R. Krchnavek, G. R. Lalk, and D. H. Hart- man. Laser Direct Writing of Channel Waveg- uides using Spin-on Polymers. J . of App. Phys., 66(11):5156, 1989.

[12] G. R. Lalk, R. R. Krchnavek, D. H. Hartman, and R. Denton. Processing Techniques for Poly- mer Optical Waveguides. In Proc. of ANTEC91, pages 1606-1609, Soc. of Plastics Engineers, May 1991.

[13] E. J. Murphy, T. C. Rice, L. McCaughan, G. T. Harvey and P. H. Read. Permanent Attachment of Single-Mode Fiber Arrays to Waveguides. J . of Lightwave Technology, LT-3(4):795-799, August 1985.

[14] L. A. Reith, J . W. Mann, G. R. Lalk, R. R. Krch- navek, N. Andreadakis, and C.-E. Zah. Relaxed- Tolerance Optoelectronic Device Packaging. J . Lightwave Technology, 9(4), 1991.

[15] W. Rotman and R. F. Turner. Wide-Angle Mi- crowave Lens for Line Source Applications. IEEE Trans. Ant. Prop., 11:623, 1963.

[16] Y. Saad and M. H. Shultz. Topological Prop- IEEE Trans. on Comp., erties of Hypercubes.

37(7):867-872, July 1988.

[17] Y. Yamada, F. Hanawa, T . Kitoh and T. Maruno. Low-Loss and Stable Fiber-to-Waveguide Con- nection Utilizing UV Curable Adhesive. IEEE Photonics Technology Letters, 4(8):906-908, Au- gust 1992.

265

I

topologies and technologies for optically interconnected...

Documents