SUPERHIGHWAYS TECHNOLOGY AND BROADBAND VPN

Partitioning techniques for reducing computationaleffort of routing in large networks

M.E. Woodward and M.M.M. Al-Fawaz

Abstract: A new scheme is presented for partitioning a network having a specific number of nodesand degree of connectivity such that the number of operations required to find a constrained pathbetween a source node and destination node, averaged over all source–destination pairs, isminimised. The scheme can speed up the routing function, possibly by orders of magnitude underfavourable conditions, at the cost of a sub-optimal solution.

1 Introduction

A communication network is a collection of devices wishingto communicate and a set of nodes providing a connectionbetween such devices. Since, in general, there is no directconnection between devices, the network must route thetraffic from node to node until the information is deliveredto the destination node. A path in the network is thendefined as a collection of sequential communication linksultimately connecting two devices to each other. In general,there is more than one path connecting any two devices inthe network. The process of selecting and finding the pathsin the network is referred to as the routing function. [1–5].Paths may or may not be subject to constraints on suchthings as delay, bandwidth or packet loss. Routing that hasto satisfy such constraints is known as quality of service(QoS) routing [6–8]. What follows applies to any specificrouting function employed in packet switched networks.

Partitioning is a significant topic in the field of networkrouting [4, 5, 9–12]. A partitioned routing scheme withoptimally selected partitioning structures can yield anenormous reduction in routing time and hence in routingcost. A new scheme for an optimal way of partitioningnetwork nodes into blocks is proposed to reduce thenumber of operations required in selecting a path, in anetwork having a specific number of nodes N and a specificdegree of connectivity C, such that the number ofoperations in finding the shortest path between a specifiedsource node and destination node is minimised. This schemeprepares the network topology based on an optimalpartitioning of nodes to speed up the routing function,possibly by orders of magnitude, at the cost of a possiblesub-optimal solution.

2 The objective function

The objective is to find a partition on the network thatminimises the number of operations required to find aconstrained path between a source and destination averaged

over all source–destination pairs, using some specifiedrouting algorithm. Then if both nodes are in the sameblock of the partition, we need only consider the nodes inthat block in deriving the paths and we can ignore the restof the network, provided that the partition is selected in aspecific way so that all nodes in a block have at least onepath between them.

Let Pij be the conditional probability that, given aconnection request is made, this is from source node i todestination node j, and let nij (t) be the number of timessource node i has requested a connection to destinationnode j in the last t time units. Measurement of nij (t) can beused to estimate Pij as follows:

PijðtÞ ¼nijðtÞPN�1

i¼1

PNj¼iþ1 nijðtÞ

; t ¼ t2 � t1 ð1Þ

where t is the progressing time of the measurement period.As t increases, Pij (t) becomes more accurate since:

limt!1

PijðtÞ ¼ Pij ð2Þ

This is illustrated in Fig. 1. Now let L denote the index of aspecific partition of N nodes into K blocks, where block k,k¼ 1,y,K, contains NkL nodes and has connectivity CkL

and let Pij (k, L) be the probability that nodes i and j areassigned to block k of partition L. The value of Pij (k, L)will depend on the assignment strategy and the partitionstructure. Now let F(N, C) be a function representing theaverage computational effort (in number of operations)required to find a path between a source and destination,subject to some constraints, in a network of N nodes andconnectivity C. The form of F(N, C) will depend on therouting algorithm used but is usually a monotonicallyincreasing function of its argument. In particular, QoSrouting algorithms may require to select paths subject toseveral constraints [13–15], which requires considerablymore effort than, say, an algorithm based on minimum hopcount.

nij (�) t2t1 t

Pij (�)�

Fig. 1 Estimation of conditional probability of connection requestThe authors are with the Department of Computing, University of Bradford,Bradford, West Yorkshire BD7 1DP, UK

r IEE, 2004

IEE Proceedings online no. 20040150

doi:10.1049/ip-com:20040150

Paper first received 24th January 2003 and in revised form 5th January 2004

IEE Proc.-Commun., Vol. 151, No. 1, February 2004 95

The partition that satisfies the following equation willthen be optimum in the sense that it minimises the numberof operations to generate all paths from source i todestination j averaged over all i, j, iaj, denoted Bp.

Bp ¼minL

1

NðN � 1ÞXi;j; 6¼j

PijXk

Pijðk; LÞF ðNkL; CkLÞ !

þ 1 �Xk

Pijðk; LÞ !

F ðN ; CÞ!

ð3Þ

On the right hand side of (3), the first of the twosummed terms represents the computational effort, aver-aged over all source–destination pairs, when the source anddestination are located in the same block of the partition.The function F(NkL, CkL) represents the average computa-tional effort in finding a path from source to destinationwhen these are both located in block k of partition L. This issummed over all blocks in the partition indexed by k. Thesecond of the two summed terms represents the computa-tional effort, averaged over all source–destination pairs,when the source and destination are in different blocks ofthe partition. In this case the routing search defaults to thewhole network and so the function F(N, C) is used here.The addition of the two terms is summed over all possibleconnection requests (all i, j) and the average taken bydividing by N(N�1), which is the total number of possibleconnection requests. The partition corresponding to theminimum of this function is selected over all partitionsindexed by L.

3 Statistically identical nodes

We first consider the case where all nodes are statisticallyidentical; that is, all node-pairs are associated with an equalprobability of connection request. For evaluation purposeswe consider F(N, C) to be a function that finds all possiblepaths between a specified source and destination applied toa symmetrical network with C¼N�1; that is, the networktopology is in the form of a strongly connected graph. Sucha network has a maximum number of paths between anysource–destination pair and, in this respect, the resultsobtained represent a worst-case situation, or upper boundon computational effort. It can be shown that for such anetwork F(N, C) has the form [16]:

F ðN ; CÞ ¼ ðN2 � 5N þ 8ÞðC � 1Þ þ 1 ð4Þ

Equation (4) effectively represents the number of commu-nications (or messages passed) between nodes to find allpaths between any source node and any destination node ina symmetrical network of N nodes and connectivity C.Since this function applies to a symmetrical network it is thesame for all source–destination pairs and so also representsthe number of operations averaged over all source-destination pairs. Specific algorithms, such as Dijkstra’sshortest path algorithm, would have a different form forF(N, C).

If all connection requests are equally likely then Pij ¼1=ðNðN � 1ÞÞ for 8i, j and

Pijðk; LÞ ¼NkL

N NkL�1

N � 1; 8i; j; k; L ð5Þ

Then using (5) and noting that the factor 1=ðNðN � 1ÞÞ in(3) has no effect on the choice of a minimum and so can beeliminated, (3) reduces to

Bp ¼minL

Xk

NkLðNkL�1ÞNðN � 1Þ ðF ðNkL; CkLÞÞ

þ 1 �Xk

NkLðNkL�1ÞNðN � 1Þ

!F ðN ; CÞ ð6Þ

In Fig. 2 we have plotted the minimum values of thisobjective function for two cases having different numbers ofnodes and for partitions having a specific number of blocks.It can be seen that the objective function exhibits aminimum which, in all cases examined, has been foundindependent of connectivity.

For a number of nodes up to 2000, the optimum numberof blocks for a given number of nodes is shown in Fig. 3from which it can be seen that as the number of nodesincreases, the optimal number of blocks tends to saturate.From the figure, it is clear that 23 is the optimal number ofblocks for networks in the region of 2000 nodes, whereasfor 1000 nodes the optimal number of blocks has already

−101030507090

110130150170190210230250

0 4321 5 6number of blocks

num

ber

of o

pera

tions

per

conn

ectio

n re

ques

t

c=6c=8c=10

nodes=11

nodes=30

04080

120160200240280320360400440480520

num

ber

of o

pera

tions

per

conn

ectio

n re

ques

t

c=6c=8c=10

a

0 4321 5 6 7number of blocks

b

Fig. 2 Number of operationsa Nodes¼ 11b Nodes¼ 30

0

5

10

15

20

25

0 200 400 600 800 1000 1200 1400 1600 1800number of nodes

num

ber

of b

ocks

Fig. 3 Optimal number of operations

96 IEE Proc.-Commun., Vol. 151, No. 1, February 2004

reached 17. It is suggested from this that the ratio of blocksto nodes in the optimal partitioning structure decreases asthe network becomes large (the number of nodes increases).

4 Heterogeneous nodes

In most practical situations, the network nodes would notbe statistically identical. In this case we need to assign nodesto the blocks of a partition in such a way that the objectivefunction is minimised. We propose two algorithms asfollows.

4.1 Enumerative algorithmWe begin this process by starting with the 2-block partitionsand evaluating the objective function for all possibleassignments of nodes to the blocks. The partition structureand node assignment is chosen that gives the lowest value ofthe objective function. This process is then repeated for all3-block partitions, and so on until the lowest value of theobjective function for k+1 block partitions, kZ2, is greaterthan the lowest value for k-block partitions. Then theassignment that minimises the objective function for thek-block partitions is the optimum one.

The nodes are assigned to a given block at the first stepbased on some initial criterion (connectivity and Pij). Theweights for each block (sum of Pij for nodes assigned to thatblock) are calculated and the objective function isrecalculated. The next step re-shuffles the nodes betweenblocks, assigning nodes to blocks based on the nodessimilarity (large Pij) to the current weight of that block. Theweights of the blocks are recalculated at the end of this step.This process continues recursively until no nodes changeblocks. This method works well if the number of blocks isnear to the optimum for the number of network nodes andthe initial solution is reasonably close to the final solution,but it can take a long time if the initial solution is far fromthe optimal solution.

The basic algorithm is as follows:

� Partition the network of N nodes into k initial blocks,where the following method for selecting the optimumpartition structure is proposed:

Index all partitions having k blocks with L and let BkL bethe value of the objective function for the partition L havingk blocks, starting with each block having 2 nodes and just 1block having the remaining nodes. Also, define BkLðminÞ ¼minL

BkL then:

1. let B1L¼N, 8L;

2. k¼ 1;

3. repeat

3. k¼ k+1;

4. for all partition structures L having the same number ofblocks k

5. Evaluate the BkL for all possible assignments of nodes toblocks and select BkL(min);

6. end for

8. until

9. BkLðminÞ4Bk�1;LðminÞ or k ¼ N=2b c;

The partition structure L with k�1 blocks correspondingto Bk�1, L(min) is optimum.

Clearly, to evaluate BkL for all possible node assignmentswould be computationally intractable in the case of largenetworks. We therefore propose a more efficient method

that avoids complete enumeration but may result in a sub-optimal solution.

4.2 Minimum weight cut algorithm(MWCA)This algorithm attempts to determine the cut (or cuts)between blocks with the lowest possible weight (probabilityof selecting nodes in different blocks). The lowest possibleweight cut may or may not satisfy the best optimum nodepartitioning but should result in a good solution. Weproceed as follows:

� The same basic method as for the Enumerative Algo-rithm is adopted except at step 5 Bk,L is evaluated only oncerather than for all possible assignments of nodes to blocks.This single node assignment is carried out as follows:

� Obtain a table of Pij+Pji 8i, j, and arrange in ascendingorder.

� Starting from the first table entry (lowest Pij+Pji) assignnode i to one block and node j to a different block.

� Continue down the table until the smallest block is full.

� If k¼ 2 the remaining (unassigned) nodes all go in theblock with spare capacity and the assignment is complete.

� If k42 continue assigning nodes to pairs of remainingblocks until a block is full.

� Continue until only one block with spare capacity is left.The remaining nodes are assigned to this block.

This process is supplemented by considering the con-nectivity matrix for the specified topology considered andrejecting any partitions that result in blocks with dis-connected nodes.

To illustrate the above, consider the 8-node network withconnectivity matrix shown in Table 1 and assume themeasured Pij values are also as shown in the matrix table ofTable 1. This table also shows whether a pair of nodes havea direct connection between them (indicated by a 1) or not(indicated by a 0). From this the Pij+Pji are sorted intoascending order in Table 2. The possible different partitionstructures and the respective values of the objective functionare then shown in Table 3. This indicates that the partitionwith block structure (4, 4) is best. The node assignment thatgives this result is obtained as ((1, 3, 4, 7), (2, 5, 6, 8)).

Using this method only one node assignment needs to becarried out per partition structure, although the effort insorting the Pij+Pji can be considerable.

5 Computation results

5.1 Effects of partitioning upon scalabilityTo test the scalability of the proposed algorithm, we run asimulation for a large number of nodes, (1000–4000 nodes),and attempt to give more path alternatives to the algorithmsby requesting low quality paths. The result is shown inFig. 4.

The experiment shows that the algorithm takes about50 106 operations to run over a flat network that has 1000nodes, while the same number of operations are required bya partitioning scheme for 2690 nodes where 50 106 isassumed to be the accepted search time in both cases. Thisgives a 2.69 times scalability using the proposed partitioningscheme. That is, a path can be found in a partitionednetwork having 2.69 times the number of nodes than in theoriginal un-partitioned network, with the same averagecomputational effort. This is sometimes called the scalefactor [17]. Note that the partitioning overhead has little

IEE Proc.-Commun., Vol. 151, No. 1, February 2004 97

effect if the time t is large since the effort in partitioning isthen distributed over a correspondingly large number ofconnection requests (Partitioning is assumed to take placeonce every t time units).

5.2 Effects of QoS parameters uponcomputational effortAnother factor that can affect the scalability is the quality ofthe required path. Using the same example as before but

Table 1: Matrix of Pij values and connectivity

N1 N2 N3 N4 N5 N6 N7 N8

N1 0 1 1 1 0 0 0 0

0 0.00554 0.01961 0.01824 0.01767 0.010177 0.01902 0.01355

N2 1 0 1 0 1 0 0 0

0.00206 0 0.00027 .00309 0.02228 0.00271 0.02158 0.00648

N3 1 1 0 1 0 1 1 0

0.01392 0.01796 0 0.01734 0.00252 0.00068 0.01595 0.00873

N4 1 0 1 0 0 0 1 0

0.02195 0.00341 0.0228 0 0.00723 0.00168 0.02118 0.01673

N5 0 1 0 0 0 1 0 1

0.02239 0.00917 0.36604 0.0154 0 0.00178 0.0046 0.00619

N6 0 0 1 0 1 0 0 1

0.00519 &0.01549 &0.01526 &0.0173 &0.00722 0 &0&0.0182 &0.01605

N7 0 0 1 1 0 0 0 1

0.00391 0.00062 0.02047& 0.01201 0.00158 0.01994 0 0.01142

N8 0 0 0 0 1 1 1 0

0.00754 0.01214 0.00491 0.02002 0.00015 0.0089 0.02147 0

Table 2: Pij+Pji sorted in ascending order

No. Link Pij Link Pji Pij+Pji SortedPij+Pji

Link

1 P12 0.005544 P21 0.002063 0.00760678 0.0061929 P57, 75

2 P13 0.01962 P31 0.13926 0.03354621 0.0063528 P58, 85

3 P14 0.018245 P41 0.21955 0.04020267 0.0065076 P24, 42

4 P15 0.01767 P51 0.022391 0.04006507 0.0076068 P12, 21

5 P16 0.010177 P61 0.00519 0.01536804 0.0090121 P56, 65

6 P17 0.01902 P71 0.003919 0.02294093 0.0136534 P38, 83

7 P18 0.01355 P81 0.00755 0.02109991 0.015368 P16, 61

8 P23 0.00027 P32 0.017964 0.01824068 0.0159484 P36, 63

9 P24 0.00309 P42 0.003411 0.00650758 0.0182092 P26, 62

10 P25 0.02228 P52 0.009175 0.03145693 0.0182407 P23, 32

11 P26 0.00271 P62 0.015497 0.01820918 0.0186244 P28, 82

12 P27 0.02158 P72 0.000629 0.02221008 0.0189973 P46, 64

13 P28 0.00648 P82 0.012143 0.01862436 0.0210999 P18, 81

14 P34 0.01734 P34 0.01734 0.04015176 0.0222101 P27, 72

15 P35 0.00252 P53 0.366047 0.368568 0.022646 P45, 54

16 P36 0.00068 P63 0.015267 0.01594842 0.0229409 P17, 71

17 P37 0.01595 P73 0.020475 0.03643293 0.0249632 P68, 86

18 P38 0.00873 P83 0.004916 0.01365336 0.0314569 P25, 52

19 P45 0.00723 P54 0.015408 0.02264603 0.0328928 P78, 87

20 P46 0.00168 P64 0.017309 0.01899733 0.0332008 P47, 74

21 P47 0.02118 P74 0.012017 0.03320083 0.0335462 P13, 31

22 P48 0.01673 P84 0.020026 0.03676246 0.0364329 P37, 73

23 P56 0.00178 P65 0.007226 0.00901213 0.0367625 P48, 84

24 P57 0.0046 P75 0.001584 0.00619289 0.0381467 P67, 76

25 P58 0.00619 P85 0.000154 0.00635278 0.0400651 P15, 51

26 P67 0.0182 P76 0.019941 0.03814669 0.0401518 P34, 43

27 P68 0.01605 P86 0.008904 0.02496316 0.0402027 P14, 41

28 P78 0.01142 P87 0.021472 0.03289283 0.368568 P35, 53

98 IEE Proc.-Commun., Vol. 151, No. 1, February 2004

with tighter QoS constraints gives the results shown inFig. 5.

This indicates that the average number of operationsrequired to find a requested path is 60 106 in a flatnetwork with 1790 nodes and is enough to obtain therequested path in a partitioned network with 3730 nodes, ascaling of 2.08. Thus, tightening the QoS requirements hasreduced the scalability.

5.3 Relating the computational effort andthe blocking probabilityAnother factor that can affect scalability is blockingprobability, which is the probability of failing to find apath that satisfies the requested QoS. An importantconsideration is the trade-off between the computationaleffort (average number of operations per connectionrequest) and blocking probability.

Figure 6 shows a graph for the relationship betweenblocking probability and computational effort for bothpartitioned and non-partitioned routing.

This shows that there is a reduction in routing overheadfor the partitioned network. However, it also shows anincrease in blocking probability for the partitioned networkwhich can be significant if a minimal blocking probability isrequired.

6 Conclusions and future work

In summary, we have a method for partitioning aset of nodes in a way that minimises the number ofoperations required to find a constrained path between asource and destination node. The number of operations inthis study are based on a worst case situation for a specificnumber of nodes and connectivity in the sense that thefunction F(N, C) finds all possible paths between asource and destination. The study thus effectively gives anupper bound on the number of operations and isindependent of the routing algorithm used. An algorithmhas been presented that can be used to directly find anoptimal (or near optimal) partition that minimises (orreduces) the number of operations required in using anyrouting algorithm. Therefore, the partitioning and routingcan be integrated into a single algorithm that greatlyreduces the amount of computation at the cost of a possiblesub-optimal solution and an increase in blockingprobability.

An adaptive version of the partitioning algorithm wouldbe able to optimise the routing algorithm to be used astraffic flows in the network change. The probability ofselecting a source and destination node in the same block ofthe partition would depend on the traffic flows through thenodes (high traffic¼ high probability). Thus, as traffic flowschange with time, the node probabilities also change which,in turn, may change the optimum partition structure (theone that minimises the number of operations). The ideamight be that when traffic flows change by more than aspecified amount then the partitioning algorithm is used todetermine a new (optimum) partition structure. A challen-ging problem is therefore to answer the question: whatshould the traffic levels be so that the partitioning algorithmis initiated at the optimum times? If the levels are set too lowthen the algorithm is run too often, creating unnecessaryoverhead, since most of the time the optimum partition willnot change. If levels are set too high then the algorithm isrun too infrequently, causing the routing to be selectedusing a non-optimum partition structure for long periods oftime.

Finally, it should be mentioned that the partitioningmethod presented is very different from the well knowntechnique inherent in hierarchical routing [4, 5] which relieson partitioning a network into sub-networks at successivelevels in a hierarchy and routing between the sub-networksuntil the destination node is located. A comparison of thetwo methods would be interesting and there may be apossibility of combining the two methods. That is, start bypartitioning a network into blocks (sub-networks) so thatthe probability of locating a source and destination node inthe same block is maximised. Then, apply the sameprocedure to each block and partition these into smallerblocks.

Table 3: Partitioning structures for 8 nodes using theminimum weight cut algorithm

L Structure Numberof blocks

MWCAvalue

Exactminimum

1 8 1 74 74

2 6, 2 2 62 61

3 5, 3 2 60 60

4 4, 4 2 55 54

5 4, 2, 2 3 59 57

6 3, 3, 2 3 63 60

7 2, 2, 2, 2 4 65 63

1030507090

110130150170190210

0 500 1000 1500 2000 2500 3000 3500 4000 4500number of nodes

num

ber

of o

pera

tions

per

conn

ectio

n re

ques

ts (

×106 )

nonpartitioned

plus overhead

partitioned

Fig. 4 Scalability with partitioning

10

30

50

70

90

0 500 1000 1500 2000 2500 3000 3500 4000 4500number of nodes

num

ber

of o

pera

tions

per

conn

ectio

n re

ques

ts (

×106 ) nonpartitioned

plus overheadpartitioned

Fig. 5 Scalability with partitioning and tighter QoS constraints

1030507090

110130150170190210

0 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16

num

ber

of o

pera

tions

per

conn

ectio

n re

ques

ts (

×106 )

nonpartitioned

partitioned

blocking probability

Fig. 6 Blocking probability against average number of operationsper connection request

IEE Proc.-Commun., Vol. 151, No. 1, February 2004 99

Thus, when a connection is requested, one might thenmove down the levels in the hierarchy until the smallestblock that contains the requested source and destination islocated and this can then be used for the routing. Thiswould appear a promising avenue for developing the workwith the aim of reducing computational effort of routing inlarge networks still further.

7 References

1 Layuan, L., and Chunlin, L.: ‘A routing protocol for dynamic andlarge computer networks with clustering topology’, Comput. Com-mun., 2000, 23, pp. 171–176

2 Onvural, R.O.: ‘Asynchronous transfer mode networks: performanceissues’ (Artech House Inc., Boston, London, 1995, 2nd edn.)

3 Balakrishnan, V.: ‘Network optimization’ (Chapman & Hall, 1995)4 Kleirock, L., and Kamoun, F.: ‘Hierarchical routing for large

networks, performance evauation and optimization’, Comput. Netw.,1977, 1, (3), pp. 155–174

5 Kleirock, L., and Kamoun, F.: ‘Stochastic performance evaluation ofhierarchical routing for large networks’, Comput. Netw., 1979, 3, (5),pp. 337–353

6 Chen, S., and Nahrstedt, K.: ‘An overview of quality-of-servicerouting for the next generation high-speed networks: Problems andsolutions’, IEEE Netw. Mag., 1998, 12, (6), pp. 64–79

7 Bhatti, S.N., and Crowcroft, J.: ‘QoS-sensitive flows: Issues in IPpacket handling’, IEEE Internet Comput., 2000, 4, pp. 48–57

8 Halabi, S., and McPherson, D.: ‘Internet routing architectures’ (CiscoPress, 2000, 2nd edn.)

9 Hansen, P., and Jaumard, B.: ‘Cluster analysis and mathematicalprogramming’, Math. Program., 1997, pp. 191–215

10 Handel, R., and Schroder, S.: ‘ATM networks, concepts,protocols, applications’ (Addison-Wesley, Wokingham, 1998,3rd edn.)

11 Halsall, F.: ‘Data communications, computer networks and opensystems’ (Addison-Wesley, Wokingham, 1995, 4th edn.)

12 Mukherjee, A., and Bandyopadyay, A.K.: ‘A minimal cost design fora large hierachically clustered computer communication networkconsidering a desired index of service’, Comput. Commun., 1997, 20,pp. 507–511

13 Layuan, L., and Chunlin, L.: ‘QoS-based routing algorithm for ATMnetworks’, Comput. Commun., 2001, 24, pp. 416–421

14 Wang, Z., and Crowcroft, J.: ‘Quality-of-service routing for support-ing multimedia applications’, IEEE J. Sel. Areas Commun., 1996, 14,(7), pp. 1228–1234

15 Bagchi, A.: ‘Route selection with multiple metrics’, Inf. Process. Lett.,1997, 64, (4), pp. 203–205

16 Al-Fawaz, M.M., and Woodward, M.E.: ‘Unordered partitioning foran optimal clustering of nodes in PNNI routing protocols’. EighteenthUK Performance Engineering Workshop, Glasgow, July 2002

17 Woodward, M.E., and Sinha Deb, S.: ‘Limiting performance ofstochastic partitioning in network routing’, Electron. Lett., 2003, 39,(3), pp. 336–337

100 IEE Proc.-Commun., Vol. 151, No. 1, February 2004