Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 1

Exploiting Overlay Network Features to Enhance the Performance of the File Mover

C. Anglano, M. CanonicoDipartimento di Informatica

Università del Piemonte Orientale, Alessandria (Italy)email:{cosimo.anglano,massimo.canonico}@unipmn.it

INGRID 07

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 2

Outline

● Motivation and background● The File Mover

– Transfer from cached copies– Multipath file transfers– Striped file transfers

● Experimental evaluation– Scenarios– Results

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 3

Motivation and background● A rather large set of scientific disciplines

requires to manage massive data collections

● One of the crucial component of Data Grids is its file transfer infrastructure– Exe Time = FileTransfer Time + Computational Time

● For 30% to 80% of the network path chosen by the IP routing, it was possible to find an alternative path with better performance [Savage]

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 4

The File Mover (FM)

● The File Mover – is a file transfer infrastructure – based on the overlay network paradigm – specifically conceived for Data Grids

● The File Mover architecture includes– File server– File relays– File proxy– Information system

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 5

The FM Architecture

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 6

The FM features● Monitoring of the traversal throughput of

each virtual link– forecast computation by NWS statistical alg.

● Computation of the best path– variant of the Dijkstra's shortest path

algorithm

● Virtual path transfer– UDP-based Data Transfer (UDT) protocol

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 7

The FM overlay network

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 8

The IP path

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 9

The best path

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 10

The virtual path

Virtual link

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 11

Computation of thebest virtual path

● Best virtual path consists in the sequence of virtual links that provides the highest achievable throughput among all possible virtual paths between a Server and a Proxy

Set of virtual links Set of File Relays

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 12

1st extension: transfer from cached copies

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 13

2nd extension: Multi path transfer

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 14

Balancing of the work

● The size of the chuck sent on each virtual path is proportional to its achievable throughput:

– File size 1GB– AT(VP1)=70Mbps AT(VP2)=30Mbps– VP1 will transfer 700MB– VP2 will transfer 300MB

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 15

3rd extension: striped file transfer

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 16

Experimental evaluation

● The network testbed– Specification and enforcement of the capacity

of virtual links

● Traffic Control (TC) mechanism available in Linux kernel

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 17

Background traffic parameters

● Injection of background traffic

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 18

Background traffic workloads

● We defined 3 different background traffic workloads corresponding to situation of low, intermediate and high intensity:

– AvgCap=63Mbps– MaxPacketSize=1500 byte– BgTrafIntensity=80%

Packet Per Seconds (PPS)=4200● Distributed Internet Traffic Generator (D-ITG)

20%50%80%

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 19

Overlay configuration

● The machines in our testbed, being relatively old, could sustain the generation of at most 6 D-ITG streams

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 20

Evaluation of transfer from cached copies

● Cached copies greatly enhances performance for all background traffic intensity

● The ability of the FM to adapt to variations of bg traffic intensity is demonstrated by the results in Num. Paths column

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 21

Evaluation of multipath transfers

● Multipath transfers result in significantly better performance w.r.t. the plain FM

● The reduction of the achievable throughput on any virtual path increases with the number of used virtual paths

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 22

Evaluation of striped transfers

● We observer performance improvementsover the plain FM

● Replacing one of the File Relay with a Server results in significantly reduction of the number of alternative paths

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 23

Conclusions and Future work

● The techniques we considered result in significant performance improvements– these results are preliminary due to the

limited size of our overlay network

● New techniques– Multipath + Cache– Striped + Cache – ...

 Università del Piemonte Orientale – INGRID 07 – S.Margherita Ligure ­ Italy 24

Conclusions and Future work (II)

● Source code available on http://dcs.mfn.unipmn.it

● Investigation of better algorithms concerning virtual path selection

● Investigation of new approach for collecting throughput measurements – Scalability– Accuracy