A Globally Optimised Multipath Routing AlgorithmUsing SDN

Noel Farrugia, Victor Buttigieg and Johann A. BriffaDepartment of Communications and Computer Engineering,

University of Malta,Msida MSD 2080, Malta

Email: noel.farrugia.09@um.edu.mt, victor.buttigieg@um.edu.mt, johann.briffa@um.edu.mt

Abstract—The Software Defined Network (SDN) model

promises the delivery of a network architecture capable of

handling the demands of current and future applications more

efficiently than the current distributed network architecture.

SDN gains this ability by moving to a logically centralised

control plane. The centralised network controller has up to

date information on both the network topology and the flows

using the network. In this work we use the information already

available at the network controller to create a globally optimised

Per-Packet multipath routing algorithm. An optimal Per-Packet

multipath routing algorithm may split a flow unequally between

different paths. Current SDN capable hardware is not able to

split flows in an unequal fashion while still remaining scalable.

To overcome this limitation, we also propose a stochastic flow

splitting mechanism capable of dividing a flow into any number

of fractional components while offering better scalability than

current methods. To analyse the performance of our proposed

system, the GEANT network topology is used. Under heavy traffic

load, our system shows a 90% improvement in a flow’s received

throughput when compared to OSPF and ECMP.

I. INTRODUCTION

Next generation networks need to be capable of handling amultitude of applications, each having their own requirementsand constraints on delay, jitter, throughput and reliability. Thedemand for more robust, resilient and efficient networks led tothe development of Software Defined Networks (SDNs) [1].SDN is a new networking paradigm that shifts the distributedcontrol plane to a logically centralised server. This architec-tural change gives the network controller a top down view ofthe network, facilitating network management and innovation.A testimonial for the success of the SDN concept is Google’sown globally deployed network [2]. This work exploits thenetwork information available at the SDN network controllerto implement a globally optimised routing algorithm and henceincreasing the overall network performance and efficiency.

To utilise a network to its maximum potential, the use ofmultipath technology is a must [3], [4]. However, multipath isnot a universal solution, as implementing multipath techniquesin dense network topologies, or lightly loaded networks mightnot be the best solution [5]. Multipath routing can be splitinto two main classes: Per-Flow and Per-Packet. Per-Flow

multipath routes packets related to the same flow over thesame path; however, different flows between the same source-destination pair can make use of different paths. Equal CostMultipath Routing (ECMP) is an example of a Per-Flow

multipath technique [6] that load balances flows between pathsof equal cost. ECMP determines the path a flow must takebased on a hash value of the packet headers. Apart frombeing unable to load balance between links of unequal cost,ECMP routed data can still face congestion if two demandingflows are routed over the same path. Conversely, Per-Packet

multipath can route packets originating from the same flowthrough different paths. Although Per-Packet multipath can po-tentially offer better throughput performance when comparedto Per-Flow multipath due to its finer granularity [3], Per-Flow

multipath is more commonly used. This is because connectionoriented protocols, such as TCP, will suffer performancepenalties due to packet re-ordering [7], [8], a problem thatPer-Flow multipath does not have.

An optimal multipath routing algorithm requires knowledgeof the network topology and the flows traversing it [3].Traditionally, distributed networks do not have this informationavailable, which is why the SDN architecture is chosen in thiswork. OpenFlow [9], the most widely used SDN southboundinterface protocol, implements flow splitting with the use ofgroups and hash-based splitting [10]. Groups are sufficient forcoarse, equal traffic splitting but will quickly run into scala-bility issues when faced with unequal, fine grained traffic splitratios. Because of these limitations, and the fact that hashingfunctions do not always achieve the desired splitting accuracy,Tuncer et al. [11] developed a traffic splitting mechanismbased on IP addresses. The accuracy of the work developedin [11] relies on the IP address distribution and uses Per-Flow

multipath to avoid out of order packets. In light of the lackof a suitable flow splitting mechanism, capable of handlingunequal split ratios while remaining scalable, we propose astochastic Per-Packet Flow Splitting (PPFS) algorithm thatresides on the network switch. This is combined with aGlobally Optimised Multipath Routing (GOMR) algorithmbased on Linear Programming to increase the total throughputof a network. The performance of our proposed GOMR-PPFS architecture is tested using simulations and is comparedagainst Open Shortest Path First (OSPF) and ECMP routingusing both UDP and TCP traffic. As this work is focusingon routing algorithms at the network layer, Multipath TCP(MPTCP) [12] will not be taken into consideration.

The rest of this paper is organised as follows: Section IIdescribes the GOMR-PPFS architecture, Section III gives the

simulation setup and results, and Section IV concludes thispaper and highlights areas of future work.

II. GOMR-PPFS ARCHITECTURE

The GOMR-PPFS architecture is made up of two parts: theglobally optimized multipath routing algorithm, and the flowsplitting mechanism used by a switch. This section will lookat these in turn.

A. The GOMR Algorithm

Given a set of flows, each with different requirements,and a fixed network topology, the routing algorithm needsto find the best path, or set of paths, that a data streamshould follow to reach its intended destination. To offer thebest service to all the flows using the network, the pathselection mechanism needs to take into consideration all theother flows that are concurrently making use of the network.The problem described above is also known as the Multi-Commodity Flow (MCF) problem [13]. The optimal solutionfor the MCF problem can be found using Linear Programming(LP) techniques [13].

Let G = (V,E) be a directed graph representing thenetwork topology, where V and E are the set of verticesand edges respectively. V represents the set of terminals andswitches in the network, while E represents the links thatinterconnect them. Let K represent the set of flows, and krepresent a single flow. Let cij and �ij represent the capacityand cost of link (i, j) 2 E respectively, where i, j 2 V definethe source and sink respectively. The decision variable fk

ij

represents the data rate corresponding to flow k transmittedover link (i, j) 2 E. The total data rate corresponding to flowk is represented by bk � 0 with s and d representing the flow’ssource and destination respectively. Using these definitions, therouting problem can be written in LP format as follows:

minfkij

X

k2K

X

(i,j)2E

�ijfkij , (1)

such thatX

k2K

fkij cij , 8 (i, j) 2 E, (2)

fkij � 0, 8 (i, j) 2 E, k 2 K, (3)

X

j:(i,j)2E

fkij �

X

j:(j,i)2E

fkji =

8><

>:

bk if i = s

�bk if i = d

0 otherwise8 k, i. (4)

The solution implies 0 fkij bk. The capacity constraint (2)

ensures that no link is being operated beyond its capacity,while constraint (3) ensures that no negative flow can beassigned. The balance constraint (4) states that for flow k,the total outgoing flow is equal to bk for the source node andthe total incoming flow is equal to bk for the sink node. Forall other nodes, the total incoming flow must be equal to thetotal outgoing flow. In this work, the link cost �ij is taken tobe equal to the link’s delay attribute.

TCP flows are inherently bi-directional because a data flowfrom a source node to a destination node automatically gener-ates a flow in the opposite direction due to acknowledgementtransmission. The LP based routing algorithm needs to takethese acknowledgement flows into account to allocate therequired network resources. Given a flow with a requested datarate of �, the data rate for the TCP acknowledgement flow inthe GOMR algorithm is set to (5), where ↵ is the size of theacknowledgement packet and ⌧ is the size of the TCP datapacket. This calculation assumes no packets are lost duringtransmission and an ACK packet is sent for every data packetreceived. Under these circumstances, this may be consideredas the worst case scenario because under normal workingconditions an ACK packet is sent every two data packets [14].We opt to reserve additional bandwidth for the ACK flows toavoid throttling data transmission due to insufficient networkresources to transmit the ACK flow.

Ack Rate =↵

⌧⇥ � (5)

The GOMR algorithm can choose to split the ACK flow overmultiple paths in a similar fashion to the data flow. This reversemultipath will also impact TCP performance [15].

Note that LP algorithms generally do not scale well as thenumber of flows or network complexity increases [2], andbecause of this they are rarely used in deployed networks.Interested readers can refer to [16], [2] for techniques on howto improve the running time of an LP algorithm.

B. The PPFS Algorithm

As indicated in Section I, real valued, unequal split ratiosare very difficult to implement in current SDN capable hard-ware [11]. This section describes our stochastic solution thatcan handle any split ratio with better scalability than currenthash-based alternatives [10].

To implement this new functionality, additional informationneeds to be stored in the routing table. For each flow that willpass through the switch, the routing table needs to include alist of the output port numbers and their respective cumulativesplit ratios. Upon packet reception, the switch will identifythe flow, generate a uniform random number in the range [0�1), and select which port to forward this packet on based onthis number. The forwarding port is selected such that thecorresponding cumulative split ratio is the first value greaterthan the generated random number.

Consider as an example the partial routing table shown inTable I. In this example, the switch needs to forward 30% ofFlow A’s packets through port 1, 15% of Flow A’s packetsthrough port 2, and the remaining 55% of Flow A’s packetsthrough port 3. Similarly, the switch needs to forward 10% ofFlow B’s packets through port 1 and the residual 90% over port3. When the switch receives a packet from flow A it generatesa random number, for example 0.4, and forwards the packetbased on this value; in this case, port 2.

OpenFlow switches support unequal flow splitting by meansof hash-based splitting [10], [11]. To achieve the desired flowsplit ratios while using hash-based splitting, entries are added

TABLE IPARTIAL ROUTING TABLE WITH SPLIT FLOW

Entry Number Flow Cumulative Split Ratio Port Number

0 Flow A0.3 10.45 2

1 3

1 Flow B 0.1 11 3

in the routing table until the split ratio is met. For example,to install Flow B’s entry shown in Table I, a total of tenentries would be required; one entry mapped to port 1 andthe remaining nine mapped to port 3. Comparing the numberof rules required by PPFS and hash-based splitting in thisexample, PPFS requires 80% less rules to achieve the samesplit ratio. As the split ratios become more intricate, ourtechnique scales much better than hash-based splitting. Formore details about unequal hash-based splitting, the readeris referred to [10]. The proposed Per-Packet flow splittingmechanism is inherently more complex than round robin [17];however, our technique allows for finer splitting accuracy thattranslates directly into better performance. The proposed flowsplitting mechanism requires some minor changes to both theOpenFlow protocol and the OpenFlow switch design.

C. The GOMR-PPFS Architecture

Conceptually, using SDN nomenclature, the GOMR algo-rithm is installed on the network controller while the PPFSmodule is embedded on SDN capable switches. The GOMRrouting module is given as input the network topology in-cluding the link’s properties such as data rates and delayvalues together with a set of flows and their requested datarate demands. The GOMR takes all of this into considerationand calculates the optimal routing solution as defined by (1)– (4). Once a routing solution is found, the routing table foreach switch is calculated using the PPFS method explained inSection II.

III. RESULTS

A. Simulation Setup

The performance of the three routing algorithms, GOMR-PPFS, OSPF, and ECMP are simulated using the NetworkSimulator version 3.26 (ns-3) [18]. A custom switch device isdeveloped to handle the flow splitting mechanism described inSection II-B. The random numbers are generated by using ns-3’s own Uniform Random Number generator. The formulation(1) – (4) is solved using the GNU Linear Programming Kit(GLPK) [19] via the LEMON [20] library interface. TheLEMON library is used because it provides a common inter-face for various LP solvers while providing graph traversal op-erations. Packets are transmitted at a Constant Bit Rate (CBR)using ns-3’s OnOff application. All switches considered inthis work have unlimited buffers to eliminate packet dropscaused by buffer overflow. Although this is not an accurate

54

3

2

10 10/1

5/1

5Mbps/1ms

5/1

10/1

5/1

Fig. 1. Diamond network topology. Edge labels X/Y denote the link datarate X in Mbps and the link delay Y in ms. Diamonds represent terminalsand circles represent switches.

representation of real network devices, this assumption allowsus to measure the overall network utilisation in a simplerway. The routing tables used to obtain the OSPF results aregenerated using the Ipv4GlobalRoutingHelper func-tion PopulateRoutingTables() that ships as stan-dard with ns-3 [21]. The Per-Flow ECMP results weregenerated by enabling the FlowEcmpRouting flag inthe Ipv4GlobalRouting class. This functionality is notshipped as standard with ns-3 and the patch given in [22]is required. The results presented in this work are retrievedusing the ns3 FlowMonitor module [23]. In all scenarios, therouting tables are populated before packet transmission starts,eliminating the routing protocol overhead.

All the definitions described below are as given in [23].Throughput is defined as the average number of bits receivedper second at the receiver, between the time the receiverreceives the first and last bit for a given flow. Delay is definedas the time taken for a packet to get from the source to thedestination, while Jitter is the delay variation between thecurrent and the previous packet of the same flow. For packetPx, where x > 1, jitter is given by:

Jitter(Px) = Delay(Px)�Delay(Px�1) (6)

The results and topologies presented in Section III-B andSection III-C illustrate the potential of GOMR-PPFS and SDN.For simplicity, both examples use UDP and a packet sizeof 1 500 bytes including headers. ECMP results are omittedfrom the results presented in Section III-B and III-C as theyare identical to OSPF. The results are identical because eachsource node is only transmitting a single flow. Finally, inSection III-D we show results on a more complex topology,for both UDP and TCP traffic.

B. Diamond Network

To demonstrate the benefits of using all the available net-work paths to meet the flow’s demand, the diamond topologyshown in Fig. 1 is used. Results are generated by transmitting100 000 UDP packets of size 1 500 bytes from source node0 to sink node 5. Two simulations were run with the flowthroughput set to 5 Mbps and 10 Mbps, with the results givenin Table II. When the flow has a data rate of 5 Mbps theGOMR algorithm can choose to transmit the flow over the

TABLE IIDIAMOND NETWORK RESULTS

DataRate

(Mbps)

Throughput (Mbps) Avg. Jitter (ms) Avg. Delay (ms)

OSPF GOMR-PPFS

OSPF GOMR-PPFS

OSPF GOMR-PPFS

5 5 5 0 0 11.2 11.210 5 9.99 1.2 106 60 000 152

9

8

7

6

54

3

2

1

0 5Mbps/1ms

30/5

30/1 30/1

30/130/1

30/1

10/1

30/5

30/5

30/1

Fig. 2. Butterfly network topology. Edge labels X/Y denote the link datarate X in Mbps and the link delay Y in ms. Diamonds represent terminalsand circles represent switches.

path 0 ! 1 ! 2 ! 4 ! 5 or 0 ! 1 ! 3 ! 4 ! 5. The twopaths are equally valid because they both have the necessarycapacity to meet the flow’s demands and both have the sametotal aggregate delay. On the other hand, when routing the10 Mbps flow, neither of the paths have the capacity to caterfor the flow requirements single handedly; thus, the GOMRalgorithm will split the flow equally between the two paths.

If there exists a single path between the source and des-tination that is capable of meeting the data rate requestedby the flow, OSPF and GOMR-PPFS will have identicalperformance. The advantage of multipath becomes evidentwhen the requested flow data rate is larger than what a singlepath can handle, as is the case with the 10 Mbps flow. In thiscase, GOMR-PPFS achieved better throughput and delay at thecost of an increase in jitter. OSPF transmits all of the data overone single path creating a bottleneck resulting in an averagepacket delay of ⇡ 60 s. Dividing the flow between all theavailable paths reduces the delay to 152.07 ms. The increasein jitter is expected as using multiple paths and random flowsplitting will disrupt the order packets are received [24].

C. Butterfly Network

One of the key advantages of SDN over traditional routingalgorithms is the increased amount of information available atthe network controller about the current network state. Thisadvantage was not needed in the diamond network becauseonly a single flow is present. The Butterfly network, shownin Fig. 2, is used to showcase how the extra informationavailable at an SDN network controller can be used to improveperformance. Flow information will be used by LP to find theoptimal solution that minimises the routing cost while meetingthe flows’ throughput demands.

The butterfly topology is chosen as it features a bottlenecklink and multiple paths between source and destination. To cre-ate a scenario where flow splitting is present, shorter paths are

TABLE IIIBUTTERFLY NETWORK SIMULATION SETUP

Scenario Flow ID (Source ! Sink) Data Rate (Mbps)

1 0 0 ! 8 51 1 ! 9 10

2 0 0 ! 8 101 1 ! 9 20

TABLE IVBUTTERFLY NETWORK RESULTS

Scenario Flow ID Throughput (Mbps) Avg. Jitter (ms)

OSPF GOMR-PPFS OSPF GOMR-PPFS

1 0 5 5 0 01 10 10 0 0

2 0 5 9.95 1.2 191 20 20 0 7.57

allocated higher delay values than their counterparts given that(1) is minimising delay. This highlights the benefits achievedby splitting the flow over multiple paths, giving preference topaths with minimal delay, without limiting the generality ofGOMR-PPFS. Unlike ECMP [6], which is capable of loadbalancing when equal cost paths are present, the proposedsystem is capable of dividing a flow into any number offractional components to transmit as many packets as possibleon the best path while making use of other available paths tomeet the flow’s requested throughput.

The flow setup shown in Table III is used to generate theresults presented in Table IV. These results are obtained bytransmitting 10 000 UDP packets of size 1 500 bytes. As in thediamond topology, when a single path has enough capacity tomeet the flow’s demand, OSPF and GOMR-PPFS have nearidentical performance. Considering scenario 1, GOMR-PPFStransmitted flow 1 over path (3 ! 4 ! 5 ! 7) instead ofthe shorter path (3 ! 7). The path with the smallest delayis chosen because (1) is minimising the link cost; equal tothe link’s delay value. This path choice resulted in a delayreduction of 5% when compared to OSPF.

A multipath solution will achieve better throughput thanOSPF when the only way to meet a flow’s demand is throughlink aggregation. This situation is represented by scenario 2,where GOMR-PPFS achieves a 50% increase in throughputwhen compared to OSPF for flow 0. OSPF transmits flow 0over the path (2 ! 6) which is only capable of transmittingat a maximum data rate of 5 Mbps. This link over-utilisationresults in network congestion that will negatively impact delayand would translate into packet drops if finite buffers werebeing considered. Analogous to the diamond network, splittingthe flow over multiple paths will increase jitter.

Under scenario 2, flow 1 is being effected by jitter eventhough the link (3 ! 7) has enough bandwidth to satisfy the20 Mbps flow request. To help better understand the reason

9

8

7

6

54

3

2

1

0 5Mbps

5

10 10

2020

5

5,5

15

5

5

Flow 0

Flow 1

Fig. 3. Butterfly network optimal solution. Edge labels represent the datarates of each flow passing through that link, fk

ij when using GOMR-PPFS.

why, the optimal routing solution using GOMR-PPFS is shownin Fig. 3. Flow 0 is requesting a data rate of 10 Mbpswhich can only be satisfied by aggregating the throughputof the two available paths that connect source node 0 withsink node 8. Flow 1 is requesting a data rate of 20 Mbpswhich the single path (3 ! 7) is fully capable of handling.However, because (1) is minimising delay, the optimal solutionwill transmit as much data as it can on the path with thesmallest aggregate delay. Due to the fact that the optimalsolution is taking into consideration all the present flows inthe network, only 25% of flow 1 will be transmitted over thepath (3 ! 4 ! 5 ! 7). Without the knowledge of all thecurrent flows using the network, flow 1 could have transmitted50% of its data over the link (3 ! 4 ! 5 ! 7) resultingin performance degradation for flow 0. GOMR-PPFS avoidsthis situation because of the use of SDN and the networkinformation available at the network controller.

D. GEANT Network

In this section, the GOMR-PPFS architecture is tested ona more realistic network topology, modeled on the 2017GEANT [25] network of Fig. 4. In our model, switches andterminals are connected together using bi-directional, full-duplex links with a delay of 1 ms per link. A 1 Gbps linkconnects the terminals with the switches such that the resultsare not distorted due to a bottleneck in this link. The modelednetwork link capacities are different from the ones shown inFig. 4; specifically, the red, light blue, and dark blue linkswere set to data rates of 1 Mbps, 10 Mbps, and 100 Mbpsrespectively. We chose to scale down the link capacities toavoid simulating a large number of flows, observing thatstandard TCP flows are unable to reach gigabit speeds.

Ns-3 simulations are used to compare the performance ofOSPF, ECMP and GOMR-PPFS as the network load increases.The number of concurrent flows simulated is linearly increasedfrom 50 to 300 in steps of 50. Each flow transmits at aCBR of 0.9 Mbps between randomly selected source anddestination terminals with the simulation being terminatedafter 700 seconds are simulated. A data rate of 0.9 Mbps isselected because when adding the TCP acknowledgement flow,the total data rate is just below 1 Mbps. This is equivalentto the smallest link’s data rate. This value is used to reachnetwork congestion even when the number of concurrent flows

using the network is relatively small. For every scenario, 25iterations are simulated, except for the case with 300 flowswhere 8 iterations are simulated due to timing constraints.Multiple iterations were run to minimise the effect of any onespecific random flow set out of the many possible variations.For every iteration, the network connections were built in arandom fashion to avoid biasing the results when OSPF isfaced with equal length paths that may have different aggregatedelay values. In each case, the flows were generated such thatthe network capacity is not exceeded.

Two different sets of simulations were run, using UDPand TCP. For TCP simulations we use a maximum packetsize of 590 bytes including headers, which corresponds tothe TCP default segment size of 536 bytes [26], [27]. TheNewReno congestion control mechanism is used as it is themost commonly used algorithm [28]. For consistency, UDPpackets also have a size of 590 bytes including headers. TheUDP results show the raw performance of our proposed systemas UDP does not throttle its transmission rate in the face ofeither packet re-ordering, network congestion, or both. On theother hand, TCP results allow us to analyse the impact networkcongestion has over a connection oriented protocol.

The results presented in Fig. 5 show the average percentageof flows that have failed to reach more than 95% of theirrequested throughput. As expected, ECMP offers slightlybetter performance than OSPF, particularly for UDP traffic.However, both routing algorithms deteriorate quickly as thenetwork traffic increases. Considering the 300 flow results,it can be observed that more than a third of the flows arereceiving less than 95% of the data rate requested under bothUDP and TCP.

The box plots shown in Fig. 6 are used to analyse theflows’ received throughput distribution. It can be noted thateven though all the routing algorithms report a similar medianvalue, the range and values of the outliers varies dramaticallybetween routing algorithms. From the throughput distributionresults it can be seen that GOMR-PPFS is capable of providingan increase in a flow’s received throughput of up to 90%when compared to both ECMP and OSPF. For TCP traffic,some of the outliers for GOMR-PPFS report a higher receivedthroughput than what was requested. This anomaly happensbecause throughput is being calculated between the times thefirst and last packets arrive at the receiver; thus, if there issignificant delay for the first packet to arrive while subsequentpackets arrive quickly thereafter, this would appear as a higherthan transmitted data rate.

It can also be noted that while GOMR-PPFS suffers from aslight dip in performance when used under TCP comparedto UDP, both OSPF and ECMP report better performanceunder light traffic load when using TCP compared to UDP.The OSPF and ECMP improvement under TCP is attributedto TCP’s congestion avoidance technique, while the dip inperformance for GOMR-PPFS is caused by packet re-ordering.Due to their lack of global network knowledge, ECMP andOSPF algorithms can over provision links that in turn causescongestion, resulting in severe performance degradation. The

GE

AR

1-9 Gbpsmultiples of 10 Gbpsmultiples of 100 Gbps

Fig. 4. 2017 GEANT network topology [25]. Copyright 2017 GEANT, all rights reserved. Used with permission.

performance improvement reported by GOMR-PPFS is mainlyattributed to the exploitation of the already available networkknowledge on the SDN network controller.

In our simulation model every switch is equipped with aninfinite buffer; therefore, no packet drops will be reported asa result of network congestion. As packets need to wait forlonger periods of time in the switch’s buffer before they areforwarded, network congestion has a direct impact on packetdelay. Referring to the results presented in Fig. 7, it can be seenthat GOMR-PPFS will always outperform OSPF and ECMPwith respect to the average packet delay sustained. What isinteresting to note is the magnitude in the delay differencebetween UDP and TCP. This is attributed directly to the factthat TCP has a congestion control mechanism which can limit

the transmission rate while UDP has no such functionality.The delay plots shown in Fig. 7 serve to prove that GOMR-PPFS succeeds in avoiding congestion; decreasing the numberof dropped packets should finite buffers be implemented.

As a final note, we observe that GOMR-PPFS seems toshow similar performance for TCP and UDP traffic. Thiswould appear to go contrary to the commonly held beliefthat TCP deteriorates rapidly when packets are received outof order, as is likely to happen when packets from the sameflow are routed differently. However, we observe that in oursimulations only about 3% of flows have split routes, lesseningthe overall impact of this problem to only a small fraction oftraffic. Issues related to out of order packets are still present,as evidenced by the outliers for TCP flows in Fig. 6, with

Fig. 5. The average percentage of flows that have received less than 95%of their requested throughput when using the GEANT network topology. Theerror bars are displaying ±�, where � is the standard deviation.

some TCP flows only receiving around 60% of their requestedthroughput.

IV. CONCLUSION

This work presented a Per-Packet multipath routing al-gorithm capable of unequal flow splitting and real valuedsplit ratios that uses the network information available atthe SDN network controller to improve network performance.Current SDN capable hardware is unable to perform unequalflow splitting while still remaining scalable; therefore, wepresented a stochastic flow splitting module that can handleunequal flow splitting while remaining scalable. Using theGEANT network topology we have demonstrated how usingthe network information available at the network controller,performance improvements of up to 90% in the flow’s receivedthroughput were reported when comparing GOMR-PPFS withECMP and OSPF. In addition, the proposed system guaranteesthat the majority of the flows traversing the network will reachtheir requested throughput, assuming that network capacity isnot exceeded.

All of the results presented in this work assume switches areequipped with infinite buffers; thus, eliminating packets dropscaused by buffer overflow. We conjecture that as the networkdevices start to drop packets due to congestion, TCP will reporta much higher throughput deficit for OSPF and ECMP whencompared to GOMR-PPFS. Investigating the effect of finitebuffers on network performance is something we plan to lookat in the future. Not much work has been done in the literaturethat investigates the impact of both forward and reverse pathPer-Packet multipath on TCP streams. From the presentedresults it is clear that TCP performance plummets when facedwith packet re-ordering. Improving the performance of TCPflows that are subjected to packet re-ordering is something weare currently looking into. At present, all the flows are beingassumed to have the same priority. This is rarely the casein real networks as different flows have different priorities.Adding support for flow prioritisation is left for future work.

(a) UDP

(b) TCP

Fig. 6. Box plot showing the distribution of the flows’ received throughputwhen using the GEANT network topology. The whiskers represented by thedashed lines represent the data range (max and min values). The ‘+’ symbolsrepresent the data outliers (data points that have a value ±1.5⇥ the inter-quartile range [29]).

ACKNOWLEDGEMENT

The research work disclosed in this publication is partiallyfunded by the ENDEAVOUR Scholarships Scheme (Group B).

This research has been carried out using computational fa-cilities procured through the European Regional DevelopmentFund, Project ERDF-076 Refurbishing the Signal ProcessingLaboratory within the Department of CCE, University ofMalta.

Fig. 7. The average packet delay when using the GEANT network topology.

REFERENCES

[1] D. Kreutz, F. M. V. Ramos, P. E. Verssimo, C. E. Rothenberg, S. Azodol-molky, and S. Uhlig, “Software-Defined Networking: A ComprehensiveSurvey,” Proceedings of the IEEE, vol. 103, no. 1, pp. 14–76, Jan. 2015.

[2] S. Jain, A. Kumar, S. Mandal, J. Ong, L. Poutievski, A. Singh,S. Venkata, J. Wanderer, J. Zhou, M. Zhu, J. Zolla, U. Hlzle, S. Stuart,and A. Vahdat, “B4: Experience with a globally-deployed softwaredefined WAN,” in Proceedings of the ACM SIGCOMM 2013 Conference

on SIGCOMM, ser. SIGCOMM ’13. ACM, pp. 3–14.[3] S. K. Singh, T. Das, and A. Jukan, “A Survey on Internet Multipath

Routing and Provisioning,” IEEE Communications Surveys Tutorials,vol. 17, no. 4, pp. 2157–2175, 2015.

[4] S. Habib, J. Qadir, A. Ali, D. Habib, M. Li, and A. Sathiasee-lan, “The Past, Present, and Future of Transport-Layer Multipath,”arXiv:1601.06043 [cs], Jan. 2016, arXiv: 1601.06043.

[5] X. Liu, S. Mohanraj, M. Pioro, and D. Medhi, “Multipath Routing froma Traffic Engineering Perspective: How Beneficial Is It?” in 2014 IEEE

22nd International Conference on Network Protocols. IEEE, Oct. 2014,pp. 143–154.

[6] C. E. Hopps, “Analysis of an equal-cost multi-path algorithm,” InternetRequests for Comments, RFC 2992, November 2000.

[7] M. Li, A. Lukyanenko, Z. Ou, A. Yla-Jaaski, S. Tarkoma, M. Coudron,and S. Secci, “Multipath Transmission for the Internet: A Survey,” IEEE

Communications Surveys Tutorials, vol. 18, no. 4, pp. 2887–2925, 2016.[8] M. Laor and L. Gendel, “The effect of packet reordering in a backbone

link on application throughput,” IEEE Network, vol. 16, no. 5, pp. 28–36, Sep. 2002.

[9] “OpenFlow - Open Networking Foundation,” (2017, Mar 16). [Online].Available: https://www.opennetworking.org/sdn-resources/openflow

[10] P. Medagliani, J. Leguay, M. Abdullah, M. Leconte, and S. Paris,“Global Optimization for Hash-Based Splitting,” in 2016 IEEE Global

Communications Conference (GLOBECOM), pp. 1–6.

[11] D. Tuncer, M. Charalambides, S. Clayman, and G. Pavlou, “FlexibleTraffic Splitting in OpenFlow Networks,” IEEE Transactions on Network

and Service Management, vol. 13, no. 3, pp. 407–420, Sep. 2016.[12] A. Ford, C. Raiciu, M. Handley, S. Barre, and J. Iyengar, “Architectural

guidelines for multipath TCP development,” Tech. Rep. 6182, (2017,Sep 13). [Online]. Available: https://tools.ietf.org/html/rfc6182

[13] W. Dai, X. Sun, and S. Wandelt, “Finding feasible solutions for multi-commodity flow problems,” in 2016 35th Chinese Control Conference

(CCC). IEEE, Jul. 2016, pp. 2878–2883.[14] B. Forouzan, Data Communications and Networking, 5th ed. McGraw-

Hill Higher Education, 2012.[15] J. Karlsson, P. Hurtig, A. Brunstrom, A. Kassler, and G. D. Stasi,

“Impact of Multi-path Routing on TCP Performance,” in 2012 IEEE

International Symposium on a World of Wireless, Mobile and Multimedia

Networks (WoWMoM), jun 2012, pp. 1–3.[16] T. H. Szymanski, “Max-Flow Min-Cost Routing in a Future-Internet

with Improved QoS Guarantees,” IEEE Transactions on Communica-

tions, vol. 61, no. 4, pp. 1485–1497, April 2013.[17] “Per Packet Load Balancing,” (2017, Mar 30). [Online]. Available:

http://www.cisco.com/c/en/us/td/docs/ios/12 0s/feature/guide/pplb.html[18] “ns-3,” (2017, Mar 04). [Online]. Available: https://www.nsnam.org/[19] “GLPK - (GNU Linear Programming Kit),” (2017, Mar 04). [Online].

Available: https://www.gnu.org/software/glpk/[20] “LEMON Graph Library,” (2017, Mar 04). [Online]. Available:

http://lemon.cs.elte.hu/trac/lemon[21] “Routing overview,” (2017, Mar 04). [Online]. Available: https:

//www.nsnam.org/docs/release/3.26/models/html/routing-overview.html[22] “ECMP operation in global routing,” (2017, Sep 13). [Online].

Available: https://www.nsnam.org/bugzilla/show bug.cgi?id=667[23] G. Carneiro, P. Fortuna, and M. Ricardo, “Flowmonitor: A network mon-

itoring framework for the network simulator 3 (ns-3),” in Proceedings of

the Fourth International ICST Conference on Performance Evaluation

Methodologies and Tools, ser. VALUETOOLS ’09. ICST, Brussels,Belgium: ICST (Institute for Computer Sciences, Social-Informatics andTelecommunications Engineering), 2009, pp. 1:1–1:10.

[24] N. M. Piratla and A. P. Jayasumana, “Metrics for packet reordering - Acomparative analysis,” International Journal of Communication Systems,vol. 21, no. 1, pp. 99–113, Jan. 2008.

[25] GEANT topology map. (2017, Mar 08). [Online]. Avail-able: http://www.geant.org/Networks/Pan-European network/Pages/GEANT topology map.aspx

[26] J. Postel, “The TCP Maximum Segment Size and Related Topics,” RFC,IETF, Tech. Rep., 1983.

[27] “ns3::TcpSocketBase Class Reference,” (2017, Sep 13). [Online].Available: https://www.nsnam.org/doxygen/classns3 1 1 tcp socketbase.html

[28] A. El-Sayed, S. HAGGAG, and N. EL-FESHAWY, “A Survey ofMechanisms for TCP Congestion Control,” International Journal of

Research and Reviews in Computer Science (IJRRCS), vol. 02, pp. 676–682, 01 2011.

[29] “Box Plot: Display of Distribution,” (2017, Dec 05). [Online]. Available:http://www.physics.csbsju.edu/stats/box2.html