Introducing EC-GIN:Introducing EC-GIN:Europe-China Grid InterNetworkingEurope-China Grid InterNetworking

Europe-China Grid InterNetworkingEurope-China Grid InterNetworkingEuropean Sixth Framework STREP FP6-2006-IST-045256European Sixth Framework STREP FP6-2006-IST-045256

Michael Welzl, Dan HeMichael Welzl, Dan He

OGF 27, 15 oct 2009OGF 27, 15 oct 2009

Enriched with customisedEnriched with customisednetwork mechanismsnetwork mechanisms

Original Internet technologyOriginal Internet technology

What is EC-GIN about?What is EC-GIN about?

Traditional Internet Traditional Internet applicationsapplications(web browser, ftp, ..)(web browser, ftp, ..)

Real-time multimedia Real-time multimedia applications (VoIP, applications (VoIP, video conference, ..)video conference, ..)

Today‘s Grid Today‘s Grid applicationsapplications

Driving a racing carDriving a racing caron a public roadon a public road

Applications with specialApplications with specialnetwork properties andnetwork properties andrequirementsrequirements

Bringing the Grid to its full potential !Bringing the Grid to its full potential !

EC-GINEC-GIN EC-GINEC-GIN enabled enabledGrid applicationsGrid applications

Project factsProject facts

• STREP in FP6 Call 6, total funding € 2.2 MSTREP in FP6 Call 6, total funding € 2.2 M

• Start date: 1 November 2006; duration: 3 yearsStart date: 1 November 2006; duration: 3 years

• Consortium: 11 partnersConsortium: 11 partners 7 European partners

Universität Innsbruck, UIBK, Austria (coordinator) University of Zürich, UniZH, Switzerland Institut National de Recherche en Informatique et Automatique, INRIA,

France Lancaster University, ULANC, U.K. Justinmind, JIM, Spain EXIS IT, Greece University of Surrey, UniS, U.K.

4 Chinese partners Beijing University of Posts and Telecommunications, BUPT Institute of Software, Chinese Academy of Sciences, ISCAS China Telecommunication Technology Labs, CTTL China Mobile Group Design Institute Co., Ltd, CMDI

Research Objectives and ChallengesResearch Objectives and Challenges

• ObjectivesObjectives The EC-GIN project, based on a number of properties that make Grids unique

from the network perspective, aims at developing a tailored network technology in dedicated support of Grid applications. These technical solutions will be supplemented with a secure and incentive-based Grid Services network traffic management system, which will balance the conflicting performance demand and the economic use of resources in the network and within the Grid.”

• Research ChallengesResearch Challenges How to model Grid traffic and simulate a Grid-network?

Much is known about web traffic (e.g. self-similarity) - but the Grid is different

Up to now, Grid-Sim / Net-Sim were two separate worlds(different goals, assumptions, tools, people

How to specify and manage QoS? Grids need file transfer deadlines, not sustained bit rates

How to align network and Grid economics? Grid service model, charging model for grid services Network Mgmt mechanisms in support of those areas in an integrated

fashion

Example Scenario: Large File TransferExample Scenario: Large File Transfer

Multipath file transfer

(AB + ACB) beneficial

Multipath file transfer not beneficial

due to shared bottleneck

Also consider: Also consider: B wants to send to A; obtains prediction; does not know thatB wants to send to A; obtains prediction; does not know thatC sends to A at the same time C sends to A at the same time prediction is wrong in right diagram prediction is wrong in right diagram

Large File Transfer Scenarios: QuestionsLarge File Transfer Scenarios: Questions

• When does use of multiple paths make sense?When does use of multiple paths make sense? To increase overall throughput

• How could this functionality be exposed?How could this functionality be exposed? As a transport service?

• What constraints can be considered?What constraints can be considered? To allow the delivery of a certain level of QoS

• How could large file transfer be authenticated and authorized?How could large file transfer be authenticated and authorized? By intermediate nodes?

• How to align incentives?How to align incentives? Who should be charged how much?

Some answers (EC-GIN results)Some answers (EC-GIN results)

• Incentives: Private Shared History - PSH2Incentives: Private Shared History - PSH2 tested in file sharing scenario; closely related to LFT PSH2 simulation with data from ThePirateBay Scaled down by 1000 for simulation (70 GB

scaled to 70 MB); used 29 files with over 1000 nodes

• PricingPricing Resource allocation model based on combinatorial double

auction Four entities involved: User broker (UB), Grid service provider

(GSP), Grid market auctioneer (GMA), Combinatorial Double Auctioneer, Grid information service (GIS)

More results: models and simulationMore results: models and simulation

• Various Grid traffic measurements and analysesVarious Grid traffic measurements and analysesATLAS (EGEE), GridFTP,

IPTV CDN, PPLive

• ns-2 code for Grid trafficns-2 code for Grid trafficsimulationsimulationTraffic generators for applications

above available from Sourceforgecan be tuned for a wide variety

of network conditions andstreaming parameters

QoS for Grids:Bulk Data Transfer Service (BDTS)

Real networkReal network

Control Plane: Control Plane: jBDTSjBDTS Data Plane: Data Plane: FLOCFLOC

Contribution to OGF- NSI: Network Service InterfaceContribution to OGF- NSI: Network Service Interface

Weighted fairness for data transfersWeighted fairness for data transfers

• Based on model of multiple TCP Based on model of multiple TCP flows (extension of Padhye eqn)flows (extension of Padhye eqn) based on RTT, loss and parameter n Easy to calculate; n QQ>0

• MulTFRC: emulate MulTFRC: emulate nn TCP flows TCP flows Based on TFRC (Equation Based

TCP-friendy Rate Control protocol) Equation exchanged with ours +

some minor other changes

• Applications Assign priorities to flows belonging

to one user Assign priorities to users

(e.g. M-IPTV)

... and many other things, available as tools...... and many other things, available as tools...

• GridMAP Passive measurement system

• High-Speed SOAP Engine Transparently deployable speed improvement

• Dynamic optimum server selection Mobile IPTV

• Peer Awareness(for fast file transfer with limited number of nodes) Topology discovery and shared bottleneck detection

• Parallel Transfer Grid(for fast file transfer with large number of nodes) Heuristic based scheme to find and use the best Forwarding Peers

... and integrated in GINTONIC... and integrated in GINTONIC

• Grid InterNetworking TOolbox Nestled in the CoreGrid InterNetworking TOolbox Nestled in the CoreSeparate daemons which communicate via “Base”Architecture somewhat similar to Globus ToolkitYou install the “Base” plus what you need, not

everything!

MECHANISM XMECHANISM X

Daemonization codeDaemonization code

X daemon based on WP3/WP4 prototypeX daemon based on WP3/WP4 prototype

MECHANISM YMECHANISM Y

Daemonization codeDaemonization code

Y daemon based on WP3/WP4 prototypeY daemon based on WP3/WP4 prototype

BASEBASEBASEBASE

11 22

Host 1Host 1Host 2Host 2

GINTONIC libraryGINTONIC library

GINTONIC libraryGINTONIC library GINTONIC libraryGINTONIC library

GINTONIC libraryGINTONIC library

Thanks for your attention!Thanks for your attention!

More information,More information,references,references,code:code:http://www.ec-gin.euhttp://www.ec-gin.eu