incrementally scalable optical interconnection network with a constant degree and constant diameter...

Incrementally scalable opticalinterconnection network with a constant degreeand constant diameter for parallel computing

Ahmed Louri and Costas Neocleous

A new scalable interconnection topology called the spanning-bus connected hypercube ~SBCH! that issuitable for massively parallel systems is proposed. The SBCH uses the hypercube topology as a basicbuilding block and connects such building blocks by use of multidimensional spanning buses. In doingso, the SBCH combines positive features of both the hypercube ~small diameter, high connectivity,symmetry, simple routing, and fault tolerance! and the spanning-bus hypercube ~SBH! ~constant nodedegree, scalability, and ease of physical implementation!, while at the same time circumventing theirdisadvantages. The SBCH topology permits the efficient support of many communication patternsfound in different classes of computation, such as bus-based, mesh-based, and tree-based problems, aswell as hypercube-based problems. A very attractive feature of the SBCH network is its ability tosupport a large number of processors while maintaining a constant degree and a constant diameter.Other positive features include symmetry, incremental scalability, and fault tolerance. An opticalimplementation methodology is proposed for the SBCH. The implementation methodology combines theadvantages of free-space optics with those of wavelength-division multiplexing techniques. An analysisof the feasibility of the proposed network is also presented. © 1997 Optical Society of America

Key words: Interconnection networks, scalability, massively parallel processing, optical intercon-nects, wavelength division multiplexing, product networks.

1. Introduction

The interconnection network, not the processing ele-ments ~PE’s! or their speed, is proving to be the de-cisive and determining factor in terms of the cost andperformance of parallel-processing systems.1–4 Sev-eral topologies have been proposed to fit differentstyles of computation. Examples include crossbars,multiple buses, multistage interconnection networks,and hypercubes, to name a few. Among these, thehypercube has received considerable attentionmainly because of its good topological characteristics~small diameter, regularity, high connectivity, simplecontrol and routing, symmetry, and fault tolerance!and its ability to permit the efficient embedding ofnumerous topologies, such as rings, trees, meshes,and shuffle exchanges, among others.5

The authors are with the Department of Electrical and Com-puter Engineering, The University of Arizona, Tucson, Arizona85721.

Received 13 September 1996; revised manuscript received 22April 1997.

0003-6935y97y266594-11$10.00y0© 1997 Optical Society of America

6594 APPLIED OPTICS y Vol. 36, No. 26 y 10 September 1997

However, a drawback to the use of the hypercube isits lack of scalability, which limits its use in buildinglarge-size systems out of smaller-size systems. Thelack of scalability of the hypercube stems from thefact that the node degree is not bounded and varies aslog2 N. This property makes the hypercube cost pro-hibitive for large N. Most hypercube-based inter-connection networks proposed in the literature6–8

suffer from similar size-scalability problems.Recently some networks have been introduced that

are products of hypercube topology with some fixed-degree networks such as the mesh, the tree, and thede Bruijn4,7,9 in the quest to preserve the properties ofthe hypercube while improving its scalability charac-teristics. Notable among these is the optical mul-timesh hypercube ~OMMH!.10–12 The OMMH is anetwork that combines the positive features of thehypercube ~small diameter, regularity, high connec-tivity, simple control and routing, symmetry, andfault tolerance! with those of a mesh ~constant nodedegree and size scalability!. The OMMH can beviewed as a two-level system: a local-connectionlevel representing a set of hypercube modules and aglobal-connection level representing the mesh net-work that connects the hypercube modules.

The OMMH network has been demonstrated phys-ically by use of a combination of free-space and fiber-optics technologies and has shown good performancecharacteristics for a reasonably sized network.However, for very large networks ~greater than 1000PE’s!, the OMMH experiences a logarithmic increasein terms of diameter and requires a large number offibers that make implementation complicated and ex-pensive.

In this paper, we propose a novel network thatimproves the topological characteristics as well as theimplementation and performance aspects of theOMMH network. The new network topology pro-posed is called the spanning-bus connected hyper-cube ~SBCH! and possesses a constant degree and aconstant diameter while preserving all the propertiesof the hypercube. The SBCH, which is similar to theOMMH, employs the hypercube topology at the local-connection level. The global-connection level con-necting the hypercube modules is a spanning-bushypercube ~SBH! network.13

The SBH is a D-dimensional lattice of width w ineach dimension. Each node is connected to D buses,one in each of the orthogonal dimensions; w nodesshare a bus in each dimension. The SBH offerssmall node degree, small diameter, low cost, and scal-ability. It can be scaled up by expansion of the sizeof the spanning buses.13 However, expanding thesize of the buses leads to a O~w! increase in trafficdensity,13 which in turn leads to bus-congestion prob-lems.14

The advantage of the SBCH network is that itutilizes the hypercube local-interconnection level todecrease traffic density, thereby alleviating the bus-congestion problems encountered in pure SBH net-works. This feature allows the SBCH buses tosupport a larger number of processors than theSBH network, thus allowing larger systems to bebuilt. As such, the SBCH is incrementally scal-able, with a high degree of connectivity and a smalldiameter. Additionally, we also propose an opticalimplementation of such a network. Optical inter-connects offer many desirable features, such as verylarge communication bandwidth, reduced crosstalk, immunity to electromagnetic interference, andlow power requirements.3,4,15,16

2. Structure of the Spanning-Bus ConnectedHypercube Network

In this section, we formally define the structure of theSBCH network and discuss its properties.

A. Topology of the Spanning-Bus Connected HypercubeInterconnection Network

The size of the SBCH is characterized by a three-tuple ~w, n, D!, where w, n, and D are positive inte-gers. The first parameter, w, defines the number ofnodes attached to a bus. The second parameter, n, isthe degree of the point-to-point n cube ~hypercube!.The third parameter, D, identifies the number ofbuses spanned by a PE in the network.

For a SBCH~w, n, D! the number of nodes uVu is

equal to wD2n. A node address in the SBCH is de-noted by a ~D 1 1!-tuple ~a1, a2, . . . , aD, an! by use ofa mixed-radix system, where, for i 5 1 to i 5 D, 0 #ai # ~D 2 1! and 0 # an # ~2n 2 1!.

Given the set of nodes ~V!, the set of edges ~E! isconstructed as follows: For two nodes ~a1, a2, . . . ,aD, an! and ~b1, b2, . . . , bD, bn!, where, for i 5 1 to i 5D, 0 # ai , D for j 5 1 to j 5 D, we have 0 # bi , D,0 # an , 2n, and 0 # bn , 2n:

1. The two nodes span the same bus ~i! if an 5 bnand ~ii! if, for i 5 1 to i 5 D, there are only twocomponents ai and bi that are identical, while allother components are different.

2. There is a link ~called a hypercube link! betweentwo nodes if and only if, for i 5 1 to i 5 D, ~i! ai 5 biand ~ii! an and bn differ by one bit position in theirbinary representation ~Hamming distance of 1!.

In this paper, we consider only SBCH networks withD 5 2. Therefore, in the notation the third parame-ter, D, is dropped. Consequently, a SBCH~w, n, 2!network is referred to as SBCH~w, n!. Figure 1~a!shows a SBCH~2, 3! interconnection; the solid linesrepresent point-to-point hypercube links, and the darkthick lines represent buses. Small filled circles rep-resent nodes of the SBCH network, which are, in thispaper, abstractions of PE’s or memory modules orswitches. Note that, because D 5 2, each node spanstwo buses, one bus along each dimension. Further-more, there are three bidirectional point-to-point links,attached to a node, that correspond to the hypercubelinks. Careful observation of Fig. 1~a! shows that thenode addresses satisfy the connection rules outlinedabove.

As can be seen from Fig. 1~a!, the SBCH~2, 3! con-sists of 22 3 23 5 32 nodes. It can be viewed as eightconcurrent two-dimensional ~2-D! SBH’s. Note thatw horizontal buses and w vertical buses are needed toform one w 3 w 2-D SBH network. Figure 1~b!shows one such 2-D SBH formed by nodes with thesame hypercube addresses and belonging to differenthypercube modules. Similar considerations apply tothe other seven 2-D SBH’s shown in Fig. 1~a!. TheSBCH~2, 3! network can also be viewed as four con-current three-dimensional ~3-D! hypercubes in whichfour nodes having identical hypercube addressesform a 2 3 2 SBH. The SBCH~2, 3! shown in Fig.1~a! looks like a hypercube-clustered SBCH. In gen-eral, there are 2n 2-D SBH’s and w2 hypercube mod-ules.

The choice of two parameters, w and n, completelydetermines the size of the network, the resources andimplementation requirements, and the scaling com-plexity. The parameter w determines the size of thebuses, whereas the parameter n defines the size ofthe hypercubes. From a scaling viewpoint, two scal-ing rules can be applied to a SBCH~w, n! network.The first rule, which we call the fixed-w rule, keepsthe size of the buses constant and increases the sizeof the network by increasing n. The second rule,which we call the fixed-n rule, keeps the size of the

10 September 1997 y Vol. 36, No. 26 y APPLIED OPTICS 6595

hypercube constant and increases the size of the net-work by increasing w. Clearly, the advantage of theSBCH~w, n! network is its flexibility to scale up byuse of either or a combination of the two scaling rules.

For instance, the size of the SBCH can grow with-out altering the number of links per node by expan-sion of the size of the buses. For example, 3-Dhypercubes can be added on the perimeter of the 2-DSBH’s of Fig. 1. Figure 2 illustrates a SBCH~3, 3!that is constructed by expansion of the SBCH~2, 3!

Fig. 1. ~a! Example of the SBCH network: A SBCH~2, 3! ~32nodes! interconnection is shown. The solid thick lines representbus connections, while the bold thin lines represent point-to-pointhypercube connections. ~b! Example of a 2-D SBH networkwithin a SBCH~2, 3! network. Note that the nodes that constructthe 2-D SBH belong to different hypercube modules but that theyposses the same binary hypercube-address representation withintheir corresponding hypercube modules. Eight such 2-D SBH’scoexist in the SBCH~2, 3! interconnection.


network by addition of the hypercube modules alongan outer row and an outer column. The existingconfiguration of the nodes of the SBCH~2, 3! networkdid not change because each node still spans twobuses and still has three bidirectional point-to-pointlinks for the hypercube connections. This option al-lows the SBCH to be truly size scalable.

B. Properties of the Spanning-Bus ConnectedHypercube Interconnection Network

1. Diameter and Link ComplexityThe diameter of a network is defined as the maxi-mum distance between any two processors in thenetwork. Thus, the diameter determines the maxi-mum number of hops that a message may have totake. Bearing in mind that D 5 2, the diameter of a2-D SBH is 2. The diameter of a hypercube with Nnodes is n 5 log2 N; therefore the diameter ofSBCH~w, n! is ~n 1 2!. For the SBCH~w, n! net-work, N 5 w22n, therefore n 5 log2~Nyw2!. Conse-quently the diameter of the SBCH~w, n! network canbe written as log2~Nyw2! 1 2. Using the fixed-wscaling rule shows that the diameter of the SBCH~w,n! network experiences a logarithmic increase @O~log2N!# when the network size increases. However, us-ing the fixed-n scaling rule would make the diameterconstant for any network size. The constant value isn 1 2.

Link complexity or node degree is defined as thenumber of physical links per node. For a regularnetwork in which all nodes have the same number oflinks, the node degree of the network is that of a node.The node-link complexity, or degree, of a hypercubewith N nodes is n 5 log2 N and that of a 2-D SBH is2. A node of a SBCH~w, n! network possesses linksfor both the hypercube connections and the bus con-nections. Consequently, the node degree of theSBCH network is ~n 1 2! or log2~Nyw2! 1 2. Again,when the fixed-w scaling rule is used the SBCH net-work experiences a logarithmic increase in degree@O~log2N!#; however, when the network is expandedby use of the fixed-n scaling rule, the degree becomesconstant, i.e., ~n 1 2!.

2. Bisection WidthThe bisection width of a network is defined as theminimum number of links that have to be removed topartition the network into two equal halves.17 Thebisection width indicates the volume of communica-tion allowed between any two halves of the networkwith an equal number of nodes. The bisection widthof an n-dimensional hypercube is 2~n21! 5 Ny2, sincethat is the number of links that are connected be-tween two ~n 2 1!-dimensional hypercubes to form ann-dimensional hypercube. Since there are w2 suchn-dimensional hypercubes connecting 2n 2-D SBH’s,the bisection width of a SBCH~w, n! is equal to w2 32n21 5 Ny2.

Fig. 2. SBCH~3, 3! ~72 nodes! interconnection. This SBCH network can be constructed by the addition of hypercube modules along a rowand a column to the network SBCH ~2, 3! of Fig. 1.

3. Granularity of Size ScalingIdeally it should be possible to create larger and morepowerful networks by the simple addition of morenodes to the existing network. For a 2-D SBH thegranularity of size scaling is only 2w 1 1 since at aminimum one bus per dimension could be added tothe network to increase its size. Therefore the gran-ularity of the size scaling in a w 3 w 2-D SBH of N 5w2 nodes is 2N1y2 1 1. However, we can increasethe size of a hypercube only by doubling the numberof nodes; that is, the granularity of size scaling in ann-dimensional hypercube is 2n.

In Subsection 2.A, we explained how the SBCH~w,n! network can be scaled up by use of two differentscaling rules. When the fixed-w scaling rule is ap-plied, the granularity of size scaling follows the hy-percube size scaling. Therefore, the granularity ofsize scaling by use of the fixed-w rule is w2 3 2n 5 N.When the fixed-n scaling rule is used, the granularityof size scaling follows that of the SBH. Therefore,the granularity of size scaling with the fixed-n rule is2n~2w 1 1! 5 2~Nyw! 1 2n. Note that the granular-

ity of size scaling for the fixed-w rule is O~N!, whereasfor the fixed-n rule it is O~Nyw!.

4. Topological Comparisons of the Spanning-BusConnected Hypercube with Other Known NetworksIn this subsection we compare the SBCH networkwith existing well-known topologies. These includethe Boolean hypercube ~BHC!,5 the Torus network,18

the SBH,13 and the OMMH.10 The comparison pa-rameters include diameter, degree, number of links,and average traffic density. More details of the der-ivation of average traffic density and other parame-ters can be found in Ref. 19. The topologicalcharacteristics of the results of the comparison areshown in Fig. 3.

In Fig. 3 the notation SBCH~16, n! denotes that thenetwork is expanded following the fixedw516 rule;that is, the size of the buses is kept constant ~16 PE’sper bus! and the size of the hypercube module ischanged to have the same network size for compari-son purposes. The notation SBCH~w, 4! denotesthat the network is expanded following the fixedn54


Fig. 3. Network comparisons for ~a! diameter, ~b! degree, ~c! number of links, and ~d! average traffic density.

rule, which means that the size of the hypercubemodule is kept fixed ~n 5 4! and the size of the busesis increased. Note that, when expanding the SBCHnetwork, some mathematical constraints exist. Thenotation ~16, 16, n!-OMMH denotes that the size ofthe mesh network in the OMMH is fixed, whereas thesize of the hypercube is varied. Similarly the ~l, l,4!-OMMH notation denotes that the size of the hy-percube is fixed, and the mesh size is varied. Fi-nally, the notation SBH~D 5 3! means that thedimension of the SBH network is kept constant, andthe size of the buses ~w! is changed.

Figures 3~a! and 3~b! show the graph comparisonsin terms of diameter and degree as the network size


is increased. At the key mark of 10,000 nodes ~de-sirable for massively parallel processing!, SBCH~w,4! and SBCH~16, n! exhibit very good performancesin terms of diameter and degree, with values of 6 and7, respectively. Note that SBCH~w, 4! is more de-sirable than SBCH~16, n! because it possesses con-stant degree and diameter, features that allow it to bescalable. Figure 3~c! shows that large SBCH net-works are feasible with a small number of links.Conversely, the OMMH network experiences fairlylarge diameters, high numbers of links, and high to-pological cost. However the ~l, l, 4!-OMMH pos-sesses constant degree ~8!, a feature that makes italso size scalable. Nevertheless, the graphs indicate

that the SBCH network improves the OMMH net-work drastically in terms of every topological charac-teristic. The SBCH network possesses very smalldiameter and low degree, while it offers size scalabil-ity and regularity.

Figures 3~a!–3~c! suggest that the SBH networkexperiences the best topological characteristics.Figure 3~d! illustrates a graphic comparison amongthe SBCH, the SBH, and the BHC in terms of averagetraffic density. The average traffic density is de-fined as the product of the average distance and thetotal number of nodes, divided by the total number ofcommunication links.15 The BHC has low trafficdensity, and it is insensitive to variations in networksize. The SBCH network demonstrates the capacityfor a higher traffic density than does the BHC, but fora larger network size it also exhibits no sensitivity tovariations in network size. On the other hand, theSBH network shows the capacity for an increase intraffic density; therefore, for larger networks theSBH network most likely would experience severebus-congestion problems that would lead to long mes-sage delays. Hence, even though the SBH networkdemonstrates better topological characteristics thandoes the SBCH network, the latter is more efficientbecause it can utilize a larger number of PE’s withfewer communication delays ~Ref. 19!.

3. Optical Implementation of the Scanning-BusConnected Hypercube Network

Obviously an electronic implementation of the pro-posed SBCH network is feasible. One methodologywould be to use multiprocessor-board technology~e.g., multichip-module technology! for the hypercubesubnetwork connections and backplanes for the busconnections. For limiting the number of boards re-quired, k hypercube modules can be clustered to-gether on a single multiprocessor board. However,for a large number of PE’s and a greater bandwidthand interconnect density, conventional backplaneshave major limitations.3,4,20 These include signalskew, wave reflection, impedance mismatch, skin ef-fects, and interference, among many others.

A possible alternative is the use of optical intercon-nects. Optical interconnects offer many communi-cation advantages over electronics, includinggigahertz transfer rates in an environment free fromcapacitive loading effects and electromagnetic inter-ference, high interconnection density, low power re-quirements, and possibly a significant reduction indesign complexity through the use of multiple-accesstechniques and the third dimension of free-space op-tics. The effectiveness of optical interconnects hasbeen examined extensively.3,4,15,16 In the followingsubsections we propose an all-optical implementationof the SBCH~w, n! network in which the hypercubemodules are implemented by use of free-space space-invariant optics,10 and the bus modules are imple-mented by use of wavelength-division multiple-access ~WDMA! techniques.

A. Optical Implementation of Space-invariant Hypercubesby Use of Holographic Optical Elements

The free-space optical implementation of the hyper-cube network has been studied and analyzed rigor-ously in Refs. 4, 10, and 11. The main objective wasto exploit the third dimension and the communica-tion advantages of free-space optics to provide effi-cient and adequate implementation of the hypercubenetwork. The design methodology is based on anobservation that PE’s in an interconnection networkcan be partitioned into two different sets such thatany two PE’s in a set do not have a direct link ~exceptfor completely connected networks!. This is a well-known problem of bipartitioning a graph if the inter-connection network is represented as a graph. For abinary n cube, PE’s whose addresses differ by morethan 1 in the Hamming distance can be in the samepartition, since no link exists between two PE’s iftheir Hamming distance is greater than 1.

Besides bipartitioning the graph, we arrange thePE’s in each partition in the plane such that inter-connection between two planes becomes space invari-ant ~the connection pattern is identical for every PEin the plane!. This self-imposed requirement re-duces the design complexity of the optical setup.The two partitions of PE’s are called planel andplaner. Optical sources and detectors are assumedto be resident on processor–memory boards located inplanel and planer. Free-space holographic opticalelements ~HOE’s! are used to implement the connec-tion patterns required among the PE’s of the twoplanes.4,10,11

B. Implementation of the Spanning-Bus Hypercube byUse of Wavelength-Division Multiple-Access Techniques

In this subsection we present the implementation ofthe SBH subnetwork using WDMA techniques. Forexploiting the large communication bandwidth of op-tics, WDMA techniques that permit multiple multi-access channels to be realized on a single physicalchannel can be utilized. Optical passive star cou-plers can be utilized for the WDMA channels. Thepurpose of an N 3 N star coupler is to couple lightfrom one of its N input guides to all the N outputguides uniformly. Star couplers with 128 3 128ports and the capacity to handle more than 100 dif-ferent wavelengths are feasible with the currentlyavailable technology. An experimental integrated-services digital network ~ISDN! switch architecturethat uses eight 128 3 128 multiple star couplers tohandle more than 10,000 input-port lines has beenreported.21

The SBCH~w, n! network consists of w2 hypercubemodules and 2n, w 3 w 2-D SBH’s. From the dis-cussion of Subsection 3.A, every hypercube module isbipartitioned into two planes, called planel andplaner. In the SBCH network all planes of planelare grouped together to form a plane called PLANEL,while all planes of planer are grouped to form anotherplane called PLANER. The SBH buses can be imple-mented by interconnection of the individual planes of


Fig. 4. Optical implementation of PLANEL of a SBCH~4, 3! network by use of optical star couplers. We need two tunable transmitters–receivers for every node. Similar connections exist for PLANER. For clarity of the figure only a few buses are shown. Note that each SCr

implements two rowwise buses, while every SCc implements two columnwise buses.

planel of PLANEL and planes of planer of PLANER. Thehypercube modules are implemented by use of free-space optics to provide the connectivity between theplanes of planel and planer. Additionally, 2n21 2-DSBH subnetworks per plane ~PLANEL or PLANER! needto be implemented. Each 2-D SBH consists of 2wbuses, therefore a total of 2w 3 2n21 buses per PLANE

are required.A trivial implementation of the SBH subnetwork

is to assign a distinct wavelength for every PE inPLANEL and PLANER and then to perform WDMA tech-niques to implement the buses. However, such astraightforward method requires a prohibitivelylarge number of different wavelengths and fibers.For example, for a SBCH~4, 3! consisting of 128PE’s, a total of 64 wavelengths would be necessary.A wavelength-assignment technique10,16 can be em-


ployed to reduce the number of wavelengths used inthe system.

Let us take a running example, a SBCH~4, 3!.Figure 4 shows how wavelengths are assigned foreach PE of PLANEL. The following wavelengths areassigned to the first row: l1, l2, . . . , l8. Then,l2, . . . , l7, l1 are assigned as wavelengths in thesecond row. In general, wavelength assignment ina row is achieved by rotation of the wavelengthassignment of the previous row by one column.This wavelength assignment results in no two PE’sin the same row or column of PLANEL that have anidentical wavelength. Similar considerations takeplace for PE’s of PLANER. With this wavelength-assignment technique, the total number of wave-lengths required to implement the SBCH~4, 3!network is reduced from 64 to 8. In general, for a

SBCH~w, n! the following wavelength assignmentfor the first row must be performed: l1, l2, . . . ,lw2~n21!y2, and then l2, . . . , lw2~n21!y2, l1, are assignedto the PE’s of the second row, and so on. Thus animplementation of a SBCH~w, n! with the abovewavelength assignment requires no more than w 32~n21!y2 wavelengths.

With reference to Fig. 4, the wavelengths assignedto the PE’s of the first row are divided into two groupsof four wavelengths each. The groups are ~l1, l3, l5,l7! and ~l2, l4, l6, l8!. Each of these groups corre-sponds to the implementation of a rowwise bus. Ev-ery PE in the group should be able to tune in to anyof the wavelengths assigned to that group. For ex-ample, the node of group 1 with wavelength l1 mustbe able to tune to wavelengths l3, l5, and l7, whichcorrespond to wavelengths that were assigned to theother PE’s of that group. Rotating the wavelengthassignments of the previous rows will form the newwavelength groups that correspond to every row.

Similarly, each column of Fig. 4 must also be di-vided into two groups of four wavelengths each. Forexample, for the second column of Fig. 4 the followinggroups are formed: ~l2, l4, l6, l8! and ~l3, l5, l7, l1!,Each of these wavelength groups corresponds to theimplementation of a columnwise bus. Again, rota-tion of the columnwise wavelength assignment willresult in the formation of the wavelength groups forthe other columns.

We now consider the overall optical implementa-tion of a SBCH~w, n!. For simplicity and withoutloss of generality we consider the implementation ofan example network of size SBCH~4, 3!. Figure 4shows an example PLANEL of the SBCH~4, 3! network.We assume that each PE has three light sources:one fixed source Sh that illuminates the HOE to gen-erate the required hypercube links and two others Srand Sc that are relatively tunable sources and arecoupled into optical fibers to implement the two span-ning buses. It should be noted that full tunability isnot required, as explained above. Furthermore,each PE is equipped with three receivers. One fixedreceiver, Rh, receives light from the free-space opticsimplementing the hypercube, and the other two re-ceivers, Rr and Rc, receive light from fibers comingfrom star couplers. The key component that pro-vides bus connectivity here is the tunable-transmitter–fixed-receiver scheme. The wavelengthassignment shown in Fig. 4 corresponds to the receiverwavelength assignment of every PE. Other PE’s cancommunicate with a particular PE by simply tuning into the wavelength assigned to that PE. Therefore, itis important that tunable devices with sufficient tun-ing range, as well as tuning time, be available. Rapidprogress is being made in the development of tunabledevices, both in the range over which they are tunableand in their tuning times.21,22 Current tuning rangesare of the order of 4–10 nm, and the tuning times varyfrom nanoseconds to milliseconds.21

With reference to Fig. 4, each node utilizes twostar couplers, one for each spanning bus. Let each

star coupler that implements the rowwise buses bedenoted by SCr and each star coupler that imple-ments the columnwise buses by SCc. In a givenSBCH network, a SCr multiplexes light from Srsources coming from nodes lying in the same row ofthe plane, while SCc multiplexes light from Scsources coming from nodes lying in the same col-umn of the plane. Note that, instead of using astar coupler for every rowwise or columnwise bus,every star coupler implements 2~n21!y2 5 2 buses ofa w 5 4 number of nodes. Which rowwise orcolumnwise buses are implemented is dictated bythe wavelength assignment and wavelength group-ing, as explained above. For example, the SCr ofthe first row of Fig. 4 implements the buses withwavelengths ~l1, l3, l5, l7! and ~l2, l4, l6, l8!. Us-ing a single WDMA channel reduces the number ofstar couplers required for implementation by a fac-tor of 2~n21!y2. In general, a single star couplerimplements 2~n21!y2 buses.

Figure 5 shows a top view of both planes of theSBCH~4, 3! network. In the middle of the figure theHOE’s that implement the hypercube modules areshown. Only two star couplers are visible. The topSCr implements the two rowwise buses of the firstrow of PLANER, while the bottom SCr implements therespective buses of the first row of PLANEL. A total of2w2~n21!y2 star couplersyplane are required. TheSBCH~4, 3! network shown in Fig. 4 requires 2 3 4 32~321!y2 5 16 star couplersyplane, resulting in a totalof 32 star couplers for both planes. For the case inwhich each and every bus were to be implemented byuse of separate star couplers, the total number for the

Fig. 5. Top view of a SBCH~4, 3! network. The figure shows theimplementation of the first rowwise buses of PLANEL and PLANER.Each star coupler implements two buses of size 4. Similar con-nections exist for the other rows and columns of the SBCH~4, 3!.


complete implementation would rise to 64 star cou-plers.

For alleviating bus collisions ~e.g., different mes-sages destined to the same PE at the same time!, thetime domain along each subchannel can be utilized.Time-division multiple-access techniques can be com-bined with the proposed WDMA scheme. However,this issue is beyond the scope of this paper.

4. Comparisons with Other Networks that EmployOptical Star Couplers for Their Implementation

When optical star couplers are used to implement anoptical network the implementation cost is domi-nated basically by the number of star couplers andtunable transmitters–receivers needed for the imple-mentation. Recently two star-coupler-based opticalinterconnection networks have been proposed.Dowd15 proposed the wavelength-division multiple-access channel hypercube, WMCH~k, n!. AWMCH~k, n! network has k PE’s along each of ndimensions for a total of kn PE’s. One tunable trans-mitter and one tunable receiver per PE and per di-mension are required. All PE’s along eachdimension are connected by means of a passive opti-cal star coupler with the WDMA technique.

The WMCH network is essentially an optical ver-sion of the conventional SBH network. Thereforethey have similar network characteristics and per-formance. Liu et al.16 proposed an optical inter-connection network called a dBus-array~k, n! as ahardware improvement to the WMCH. A dBus-array~k, n! consists of an n-dimensional array withkn PE’s and kn21 unidirectional buses that are alsoimplemented by use of optical passive star couplers.The dBus-array~k, n! required one tunable trans-mitter and one tunable receiver per PE, comparedwith n tunable transmitters and n tunable receiversper PE for the WMCH. In what follows, we com-pare the hardware costs of these two networks withthat of the SBCH.

Figure 6~a! compares the SBCH~8, n! network withthe WMCH~8, n! and the dBus-array~8, n! networksin terms of the number of star couplers as the net-works grow in size. The SBCH~8, n! grows by use ofthe fixedw58 rule, while the other two networks growby use of the fixedk58 rule. Figure 6~b! shows a com-parison of the SBCH~w, 3!, WMCH~k, 3!, and dBus-array~k, 3! networks as they grow in size by use of thefixedn53 scaling rule. Figures 6~c! and 6~d! show thecomparison in terms of the number of tunabletransmitters–receivers used. As the figures show,the SBCH network requires fewer optical star cou-plers and a smaller number of tunable transmitters–receivers than do the WMCH and the dBus-array~8,n! networks. Hence the SBCH network offers re-duced implementation costs compared with theWMCH and the dBus-array~8, n! while providingperformance advantages. We note that Szyman-ski describes a star-based optical network calledHypermesh.23 This research was not available to usat the time of this study. Comparison of the SBCH


and the Hypermesh networks is under way and is thesubject of a separate paper.

5. Power-Loss Analysis of the Optical Implementation

In this section we present some system-noise calcu-lations to investigate the bit-error rate ~BER! char-acteristics of the proposed optical implementation ofthe SBCH network. Calculation of the BER of anoptical system requires the estimation of the signal-to-noise ratio ~SNR!. Estimation of total powerlosses, leading into receiver-sensitivity calculations isrequired to obtain the SNR. In what follows theoptical power loss of the implementation methodol-ogy is calculated. Then receiver sensitivity is esti-mated, and consequently the BER of the proposedimplementation is evaluated.

The number of PE’s that an optical system cansupport is determined by the emitting power of thetransmitter, the required receiver sensitivity, and thelosses occurring between the transmitter and the re-ceiver. Let Lsf be the source-to-fiber coupling loss,Lfd be the fiber-to-detector coupling loss, and Lf be thefiber-attenuation loss. We also assume that all PE’sare equidistant. Let Le be the excess loss of theoptical star coupler. To estimate the star-couplersplitting loss, it is necessary to know the input powerto the coupler and the fan-out. Let Pin be the powercoming into the coupler from an input channel andPout be the power coming out from an output channel;then Lsp 5 10 log ~PoutyPin!.

The total transmission loss is then

Ltotal 5 Le 1 Lsp 1 dLf 1 Lsf 1 Lfd. (1)

Pout is equal to Pinyk, where k is the fan-out of the starcoupler. For the SBCH, k is equal to w 3 2~n21!y2

~number of PE’s in a row or column of PLANEL orPLANER!. Consequently, Eq. ~1! can be rewritten as

Ltotal 5 Le 2 10 log k 1 dLf 1 Lsf 1 Lfd 2 3

5 Le 2 10 log w 232~n 2 1! 1 dLf 1 Lsf

1 Lfd 2 3. (2)

To estimate the total loss of the optical system, weconsider values from commercially available compo-nents. We assume laser diode sources with a char-acteristic 17 dBm. Also, the insertion loss for acommercially available fiber coupler is taken to be 21dB, while fiber-to-detector losses are Lfd 5 20.46 dB.The fiber loss is taken to be Lf 5 0.3 dBykm, but sinced is of the order of centimeters the total fiber loss isnegligible. In addition, a 23-dB loss is added forengineering errors. Rearranging Eq. ~2! and usingthe above values determine the number of PE’s sup-ported by the star couplers, given a desirable BER.For a desired 10217 BER the required receiver sensi-tivity of the GaAs metal-Semiconductor field-effect-transistor transimpedance24 can be calculated to be219.2 dBm. For laser diodes of 7.0-dBm power thetotal loss in the optical system should be 226.2 dB,

Fig. 6. Comparison of the number of optical star couplers and number of tunable transmitters or receivers among the SBCH, dBus-array~k, n!, and WMCH networks. ~a! and ~b! show the results for optical star couplers, and ~c! and ~d! show the results for the numberof tunable transmitters or receivers.

yielding a star-coupler fan-out of k 5 118. Thisvalue is within the capabilities of current star-couplertechnology. The optical fan-out of star couplers re-ported to date is 128 3 128.21 For k 5 118, largeSBCH networks are feasible. For example, aSBCH~30, 5! network supporting ;28,800 PE’s couldbe implemented.

6. Conclusions

In this paper, we proposed a novel hybrid networkthat significantly improves hypercube-based topol-

ogies in general and the SBH and OMMH networksin particular. The key attractive features of theproposed network include the possibility of aconstant-diameter and a constant-degree network,while it is feasible to interconnect thousands of pro-cessors at a reasonable cost. Additionally, the net-work is incrementally scalable and fault tolerant.Theses features make the SBCH very suitable formassively parallel systems. A WDMA techniquehas been proposed for the optical implementation ofthe SBCH network. Analysis of the implementa-


tion reveals that a greater than 20,000-PE SBCHnetwork with a BER of less than 10217 is currentlyfeasible.

This research is supported by National ScienceFoundation grant MIP-9310082.

References1. K. Hwang, Advanced Computer Architecture: Parallelism,

Scalability, Programmability ~McGraw-Hill, New York, 1993!.2. H. J. Siegel, Interconnection Networks for Large-scale Parallel

Processing ~McGraw-Hill, New York, 1990!.3. H. S. Stone and J. Cocke, “Computer architecture in the

1990s,” Computer 24, 30–38 ~September 1991!.4. A. Louri and H. Sung, “3D optical interconnects for high-speed

interchip and interboard communications,” Computer 27,27–37 ~October 1994!.

5. Y. Saad and M. H. Schultz, “Topological properties of hyper-cubes,” IEEE Trans. Comput. 37, 867–872 ~1988!.

6. Q. M. Malluhi and M. A. Bayoumi, “The hierarchical hyper-cube: a new interconnection topology for massively parallelsystems,” IEEE Trans. Parallel Distrib. Sys. 5, 17–30 ~1994!.

7. C. Chen, D. P. Agrawal, and J. R. Burke, “dBCube: a newclass of hierarchical multiprocessor interconnection networkswith area efficient layout,” IEEE Trans. Parallel Distrib. Sys.4, 1332–1344 ~1993!.

8. K. Efe, “The crossed cube architecture for parallel computa-tion,” IEEE Trans. Parallel Distrib. Sys. 3, 513–524 ~1992!.

9. J. R. Goodman and C. H. Sequin, “Hypertree: a multiproces-sor interconnection topology,” IEEE Trans. Comput. C-30,923–933 ~1981!.

10. A. Louri and H. Sung, “Scalable optical hypercube-based in-terconnection network for massively parallel computing,”Appl. Opt. 33, 7588–7598 ~1994!.

11. A. Louri and H. Sung, “An optical multi-mesh hypercube: ascalable optical interconnection network for massively parallelcomputing,” J. Lightwave Technol. 12, 704–716 ~1994!.


12. A. Louri and S. Furlonge, “Feasibility study of a scalable op-tical interconnection network for massively parallel processingsystems,” Appl. Opt. 35, 1296–1308 ~1996!.

13. L. D. Wittie, “Communication structures for large networks ofmicrocomputers,” IEEE Trans. Comput. C-30, 264–273~1981!.

14. G. Lerman and L. Rudolph, Parallel Evolution of Parallel Pro-cessors ~Plenum, New York, 1993!.

15. P. W. Dowd, “Wavelength division multiple access channelhypercube processor interconnection,” IEEE Trans. Comput.41, 1223–1241 ~1992!.

16. Y. Li, A. W. Lohmann, and S. B. Rao, “Free-space opticalmesh-connected bus networks using wavelength-division mul-tiple access,” Appl. Opt. 32, 6425–6437 ~1993!.

17. V. Kumar, A. Grama, A. Gupta, and G. Karypis, Introductionto Parallel Computing: Design and Analysis of Algorithms~Benjamin Cummings, New York, 1994!.

18. D. A. Reed and R. M. Fujimoto, Multicomputer Networks:Message-Based Parallel Processing ~MIT Press, Cambridge,Mass., 1987!.

19. A. Louri and C. Neocleous, “A spanning bus connected hyper-cube: a new scalable optical interconnection network for mul-tiprocessors and massively parallel systems,” IEEE J.Lightwave Technol. 15, 1241–1253 ~1997!.

20. P. Sweazey, “Limits of performance of backplane buses,” inDigital Bus Handbook ~McGraw-Hill, New York, 1990!.

21. A. Cisneros and C. A. Brackett, “A large ATM switch based onmemory switches and optical star couplers,” IEEE J. Select.Areas Commun. 9, 1348–1360 ~1991!.

22. P. F. Moulton, “Tunable solid state lasers,” Proc. IEEE 80,348–364 ~1992!.

23. T. Szymanski, “Hypermeshes: optical interconnection net-works for parallel computing,” J. Parallel Distributed Comput.26, 1–23 ~1995!.

24. T. V. Moui, “Receiver design for high-speed optical fiber sys-tems,” IEEE J. Lightwave Technol. LT-2, 243–267 ~1984!.

incrementally scalable optical interconnection network with a constant degree and constant diameter...

Documents