the globus extensible input/output system (xio): a protocol independent io system for the grid

23
The Globus eXtensible Input/Output System (XIO): A protocol independent IO system for the Grid Bill Allcock, John Bresnahan, Raj Kettimuthu and Joe Link Argonne National Laboratory

Upload: odell

Post on 05-Jan-2016

31 views

Category:

Documents


1 download

DESCRIPTION

The Globus eXtensible Input/Output System (XIO): A protocol independent IO system for the Grid. Bill Allcock, John Bresnahan, Raj Kettimuthu and Joe Link Argonne National Laboratory. Grid IO. Geographically disperse resources Supercomputers/clusters Large data store - PowerPoint PPT Presentation

TRANSCRIPT

  • The Globus eXtensible Input/Output System (XIO):A protocol independent IO system for the GridBill Allcock, John Bresnahan, Raj Kettimuthu and Joe LinkArgonne National Laboratory

  • Grid IOGeographically disperse resourcesSupercomputers/clustersLarge data storeSpecialized scientific devicesTelescopes, environmental sensorsCollaborative sessionsAccess GridVarying network and file transfer protocolsTCP, UDP, XCP, GridFTP etc

  • Typical Approach

  • Example Application

  • ProblemsDevelopment timeApplication must learn to use many different APIsEach API has its own semanticsAsynchronous/SynchronousThreaded/non-threadedScalability and compatibilityApplication must be modified to work with new protocols/devicesApplication must keep up with orthogonal research issues

  • ObservationsData stream IO abstractionMany Grid IO needs can be treated as a stream of bytesOpen/close/read/write functionality satisfies most requirementsProtocol detailsRarely does the application need to deal with protocol detailsMost needs can be satisfied at initialization time

  • SolutionGlobus XIO user APISingle API/single set of semanticsSimple open/close/read/write APIDriver abstractionHides protocol detailsAllows for extensibilityDrivers can be selected at runtime

  • Globus XIO Approach

  • Example Application with XIO

  • DriversMake single API do many types of IOSpecific drivers for specific protocols/devicesTransportMoves data out of process spaceTCP, UDP, File etcTransformManipulate or examine dataDo not move data outside of process spaceCompression, security etc

  • StackTransportExactly one per stackMust be on the bottomTransformZero or many per stackControl flows from user to the top of the stack, to the transport driver

  • Driver developmentA set of function signaturesImplemented by the driverRegistered with Globus XIOSemanticsXIO makes calls to these functions expecting specific behaviorRead() interface function should produce data and write() should consume data

  • Attributes and controlsWay to expose functionalities specific to a protocolUser API to control the attributes and special features of a driverAn opportunity to tweak parameters specific to a driverAttributes and controls support is optional and a driver may choose not to have it

  • Globus XIO FrameworkMoves data from user to driver stackManages the interactions between driversAssist in the creation of driversAsynchronous supportClose and EOF barriersError checkingInternal API for passing operations down the stack

  • Existing driversTransportTCP, UDP, File, GridFTPUDT - UDP based reliable data transport protocolMulti-stream - uses multiple transport protocol streams for data transferTransformGSI, HTTP, Ordering

  • Performance analysisCompare with raw sockets and Globus IOLatency testsPing-Pong tests, 1000 times, average of 10 runsLoopback, LAN and WANBandwidth testsSend 1000 back-to-back messages and wait for reply, average of 10 runsLAN and WAN

  • Latency over Loopback Interface

    Chart1

    0.020.0370.031

    0.0520.0690.062

    0.3960.4290.432

    6.9617.3887.391

    78.52880.39180.425

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Latency in ms

    Loopback

    Sheet1

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO0Raw socketsGlobus XIOGlobus IO over XIOGlobus IO

    100020373931100085955510.020.0370.0390.031

    10000526971621000032.692307692336.538461538519.2307692308100.0520.0690.0710.062

    1000003964294364321000008.333333333310.1010101019.09090909091000.3960.4290.4360.432

    1000000696173887452739110000006.13417612417.05358425516.177273380310006.9617.3887.4527.391

    1000000078528803918081380425100000002.37240220052.90979013852.4156988591000078.52880.39180.81380.425

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO0Raw socketsGlobus XIOGlobus IO over XIOGlobus IO

    100021723523722710008.29493087569.21658986184.608294930910.2170.2350.2370.227

    100001042105210561032100000.95969289831.34357005760.0959692898101.0421.0521.0561.032

    10000086878715872186991000000.32232070910.39138943250.13813744681008.6878.7158.7218.699

    10000008519385263852748521310000000.08216637520.09507823410.0234761072100085.19385.26385.27485.213

    10000000857711857738858139857614100000000.00314791350.04990025780.00151566210000857.711857.738858.139857.614

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO

    100029.36641528.27328727.63552528.86641510003.72237469235.89411407561.7026252609

    1000090.73114190.42309790.37168590.131141100000.33951297930.39617709650.6612944502

    10000092.6272492.32253592.22773392.727241000000.32895830640.4313061687-0.1079596024

    100000092.94322192.87292892.97736592.64322110000000.0756300451-0.03673640710.3227777096

    1000000093.39905393.33095593.50619693.199053100000000.0729108035-0.11471529590.2141349335

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO

    100025.91526.0526.01125.97510000.52093382210.37044182910.2315261432

    1000028.21728.42228.43728.429100000.72651238620.7796718290.7513201262

    10000040.05840.54240.68440.31000001.20824804031.56273403560.6041240202

    1000000220.114221.782222.28221.1110000000.75778914560.98403554520.4524927992

    100000002460.1062468.0192470.5322465.23100000000.32165280680.42380287680.2082837081

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO

    10003.4886353.4722373.4850153.46863510000.47004057460.10376551290.5732901264

    1000013.66896813.20405813.44748913.468968100003.40120775761.62030520521.4631682509

    10000016.55792416.20258116.19775316.3579241000002.14606009792.17521834261.2078808913

    100000016.63011816.423916.40334416.53011810000001.84134592431.3636343410.6013186437

    1000000016.58454916.51167516.43258516.544549100000000.43940899450.91629865850.2411883495

    Sheet3

    Sheet3

    0.020.0370.031

    0.0520.0690.062

    0.3960.4290.432

    6.9617.3887.391

    78.52880.39180.425

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Latency in ms

    Loopback

    Sheet2

    0.2170.2350.227

    1.0421.0521.032

    8.6878.7158.699

    85.19385.26385.213

    857.711857.738857.614

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Latency in ms

    LAN

    29.36641528.27328728.866415

    90.73114190.42309790.131141

    92.6272492.32253592.72724

    92.94322192.87292892.643221

    93.39905393.33095593.199053

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Bandwidth in Mbps

    LAN

    25.91526.0525.975

    28.21728.42228.429

    40.05840.54240.3

    220.114221.782221.11

    2460.1062468.0192465.23

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Latency in ms

    LAN

    3.4886353.4722373.468635

    13.66896813.20405813.468968

    16.55792416.20258116.357924

    16.63011816.423916.530118

    16.58454916.51167516.544549

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Bandwidth in Mbps

    WAN

    29.36641528.27328727.63552528.866415

    90.73114190.42309790.37168590.131141

    92.6272492.32253592.22773392.72724

    92.94322192.87292892.97736592.643221

    93.39905393.33095593.50619693.199053

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Bandwidth in MBps

    LAN

    20373931

    52697162

    396429436432

    6961738874527391

    78528803918081380425

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Latency in us

    Loopback

    217235237227

    1042105210561032

    8687871587218699

    85193852638527485213

    857711857738858139857614

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Latency in us

    LAN

    25.91526.0525.975

    28.21728.42228.429

    40.05840.54240.3

    220.114221.782221.11

    2460.1062468.0192465.23

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in Bytes

    Latency in ms

    WAN

    3.4886353.4722373.4850153.468635

    13.66896813.20405813.44748913.468968

    16.55792416.20258116.19775316.357924

    16.63011816.423916.40334416.530118

    16.58454916.51167516.43258516.544549

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Bandwidth in MBps

    WAN

    859555

    32.692307692336.538461538519.2307692308

    8.333333333310.1010101019.0909090909

    6.13417612417.05358425516.1772733803

    2.37240220052.90979013852.415698859

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    % increase in latency w.r.to raw sockets

    Loopback

    8.29493087569.21658986184.6082949309

    0.95969289831.34357005760.0959692898

    0.32232070910.39138943250.1381374468

    0.08216637520.09507823410.0234761072

    0.00314791350.04990025780.001515662

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    % increase in latency w.r.to raw sockets

    LAN

    3.72237469235.89411407561.7026252609

    0.33951297930.39617709650.6612944502

    0.32895830640.4313061687-0.1079596024

    0.0756300451-0.03673640710.3227777096

    0.0729108035-0.11471529590.2141349335

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    % decrease in bandwidth w.r.to raw sockets

    LAN

    0.52093382210.37044182910.2315261432

    0.72651238620.7796718290.7513201262

    1.20824804031.56273403560.6041240202

    0.75778914560.98403554520.4524927992

    0.32165280680.42380287680.2082837081

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    % increase in latency w.r.to raw sockets

    WAN

    0.47004057460.10376551290.5732901264

    3.40120775761.62030520521.4631682509

    2.14606009792.17521834261.2078808913

    1.84134592431.3636343410.6013186437

    0.43940899450.91629865850.2411883495

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size of Bytes

    % decrease in bandwidth w.r.to raw sockets

    WAN

  • Latency comparison on LAN

    Chart2

    0.2170.2350.227

    1.0421.0521.032

    8.6878.7158.699

    85.19385.26385.213

    857.711857.738857.614

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Latency in ms

    LAN

    Sheet1

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO0Raw socketsGlobus XIOGlobus IO over XIOGlobus IO

    100020373931100085955510.020.0370.0390.031

    10000526971621000032.692307692336.538461538519.2307692308100.0520.0690.0710.062

    1000003964294364321000008.333333333310.1010101019.09090909091000.3960.4290.4360.432

    1000000696173887452739110000006.13417612417.05358425516.177273380310006.9617.3887.4527.391

    1000000078528803918081380425100000002.37240220052.90979013852.4156988591000078.52880.39180.81380.425

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO0Raw socketsGlobus XIOGlobus IO over XIOGlobus IO

    100021723523722710008.29493087569.21658986184.608294930910.2170.2350.2370.227

    100001042105210561032100000.95969289831.34357005760.0959692898101.0421.0521.0561.032

    10000086878715872186991000000.32232070910.39138943250.13813744681008.6878.7158.7218.699

    10000008519385263852748521310000000.08216637520.09507823410.0234761072100085.19385.26385.27485.213

    10000000857711857738858139857614100000000.00314791350.04990025780.00151566210000857.711857.738858.139857.614

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO

    100029.36641528.27328727.63552528.86641510003.72237469235.89411407561.7026252609

    1000090.73114190.42309790.37168590.131141100000.33951297930.39617709650.6612944502

    10000092.6272492.32253592.22773392.727241000000.32895830640.4313061687-0.1079596024

    100000092.94322192.87292892.97736592.64322110000000.0756300451-0.03673640710.3227777096

    1000000093.39905393.33095593.50619693.199053100000000.0729108035-0.11471529590.2141349335

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO

    100025.91526.0526.01125.97510000.52093382210.37044182910.2315261432

    1000028.21728.42228.43728.429100000.72651238620.7796718290.7513201262

    10000040.05840.54240.68440.31000001.20824804031.56273403560.6041240202

    1000000220.114221.782222.28221.1110000000.75778914560.98403554520.4524927992

    100000002460.1062468.0192470.5322465.23100000000.32165280680.42380287680.2082837081

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO

    10003.4886353.4722373.4850153.46863510000.47004057460.10376551290.5732901264

    1000013.66896813.20405813.44748913.468968100003.40120775761.62030520521.4631682509

    10000016.55792416.20258116.19775316.3579241000002.14606009792.17521834261.2078808913

    100000016.63011816.423916.40334416.53011810000001.84134592431.3636343410.6013186437

    1000000016.58454916.51167516.43258516.544549100000000.43940899450.91629865850.2411883495

    Sheet3

    Sheet3

    0.020.0370.031

    0.0520.0690.062

    0.3960.4290.432

    6.9617.3887.391

    78.52880.39180.425

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Latency in ms

    Loopback

    Sheet2

    0.2170.2350.227

    1.0421.0521.032

    8.6878.7158.699

    85.19385.26385.213

    857.711857.738857.614

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Latency in ms

    LAN

    29.36641528.27328728.866415

    90.73114190.42309790.131141

    92.6272492.32253592.72724

    92.94322192.87292892.643221

    93.39905393.33095593.199053

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Bandwidth in Mbps

    LAN

    25.91526.0525.975

    28.21728.42228.429

    40.05840.54240.3

    220.114221.782221.11

    2460.1062468.0192465.23

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Latency in ms

    LAN

    3.4886353.4722373.468635

    13.66896813.20405813.468968

    16.55792416.20258116.357924

    16.63011816.423916.530118

    16.58454916.51167516.544549

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Bandwidth in Mbps

    WAN

    29.36641528.27328727.63552528.866415

    90.73114190.42309790.37168590.131141

    92.6272492.32253592.22773392.72724

    92.94322192.87292892.97736592.643221

    93.39905393.33095593.50619693.199053

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Bandwidth in MBps

    LAN

    20373931

    52697162

    396429436432

    6961738874527391

    78528803918081380425

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Latency in us

    Loopback

    217235237227

    1042105210561032

    8687871587218699

    85193852638527485213

    857711857738858139857614

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Latency in us

    LAN

    25.91526.0525.975

    28.21728.42228.429

    40.05840.54240.3

    220.114221.782221.11

    2460.1062468.0192465.23

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in Bytes

    Latency in ms

    WAN

    3.4886353.4722373.4850153.468635

    13.66896813.20405813.44748913.468968

    16.55792416.20258116.19775316.357924

    16.63011816.423916.40334416.530118

    16.58454916.51167516.43258516.544549

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Bandwidth in MBps

    WAN

    859555

    32.692307692336.538461538519.2307692308

    8.333333333310.1010101019.0909090909

    6.13417612417.05358425516.1772733803

    2.37240220052.90979013852.415698859

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    % increase in latency w.r.to raw sockets

    Loopback

    8.29493087569.21658986184.6082949309

    0.95969289831.34357005760.0959692898

    0.32232070910.39138943250.1381374468

    0.08216637520.09507823410.0234761072

    0.00314791350.04990025780.001515662

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    % increase in latency w.r.to raw sockets

    LAN

    3.72237469235.89411407561.7026252609

    0.33951297930.39617709650.6612944502

    0.32895830640.4313061687-0.1079596024

    0.0756300451-0.03673640710.3227777096

    0.0729108035-0.11471529590.2141349335

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    % decrease in bandwidth w.r.to raw sockets

    LAN

    0.52093382210.37044182910.2315261432

    0.72651238620.7796718290.7513201262

    1.20824804031.56273403560.6041240202

    0.75778914560.98403554520.4524927992

    0.32165280680.42380287680.2082837081

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    % increase in latency w.r.to raw sockets

    WAN

    0.47004057460.10376551290.5732901264

    3.40120775761.62030520521.4631682509

    2.14606009792.17521834261.2078808913

    1.84134592431.3636343410.6013186437

    0.43940899450.91629865850.2411883495

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size of Bytes

    % decrease in bandwidth w.r.to raw sockets

    WAN

  • Latency comparison on a WAN

    Chart3

    25.91526.0525.975

    28.21728.42228.429

    40.05840.54240.3

    220.114221.782221.11

    2460.1062468.0192465.23

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Latency in ms

    WAN

    Sheet1

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO0Raw socketsGlobus XIOGlobus IO over XIOGlobus IO

    100020373931100085955510.020.0370.0390.031

    10000526971621000032.692307692336.538461538519.2307692308100.0520.0690.0710.062

    1000003964294364321000008.333333333310.1010101019.09090909091000.3960.4290.4360.432

    1000000696173887452739110000006.13417612417.05358425516.177273380310006.9617.3887.4527.391

    1000000078528803918081380425100000002.37240220052.90979013852.4156988591000078.52880.39180.81380.425

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO0Raw socketsGlobus XIOGlobus IO over XIOGlobus IO

    100021723523722710008.29493087569.21658986184.608294930910.2170.2350.2370.227

    100001042105210561032100000.95969289831.34357005760.0959692898101.0421.0521.0561.032

    10000086878715872186991000000.32232070910.39138943250.13813744681008.6878.7158.7218.699

    10000008519385263852748521310000000.08216637520.09507823410.0234761072100085.19385.26385.27485.213

    10000000857711857738858139857614100000000.00314791350.04990025780.00151566210000857.711857.738858.139857.614

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO

    100029.36641528.27328727.63552528.86641510003.72237469235.89411407561.7026252609

    1000090.73114190.42309790.37168590.131141100000.33951297930.39617709650.6612944502

    10000092.6272492.32253592.22773392.727241000000.32895830640.4313061687-0.1079596024

    100000092.94322192.87292892.97736592.64322110000000.0756300451-0.03673640710.3227777096

    1000000093.39905393.33095593.50619693.199053100000000.0729108035-0.11471529590.2141349335

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO

    100025.91526.0526.01125.97510000.52093382210.37044182910.2315261432

    1000028.21728.42228.43728.429100000.72651238620.7796718290.7513201262

    10000040.05840.54240.68440.31000001.20824804031.56273403560.6041240202

    1000000220.114221.782222.28221.1110000000.75778914560.98403554520.4524927992

    100000002460.1062468.0192470.5322465.23100000000.32165280680.42380287680.2082837081

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO

    10003.4886353.4722373.4850153.46863510000.47004057460.10376551290.5732901264

    1000013.66896813.20405813.44748913.468968100003.40120775761.62030520521.4631682509

    10000016.55792416.20258116.19775316.3579241000002.14606009792.17521834261.2078808913

    100000016.63011816.423916.40334416.53011810000001.84134592431.3636343410.6013186437

    1000000016.58454916.51167516.43258516.544549100000000.43940899450.91629865850.2411883495

    Sheet3

    Sheet3

    0.020.0370.031

    0.0520.0690.062

    0.3960.4290.432

    6.9617.3887.391

    78.52880.39180.425

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Latency in ms

    Loopback

    Sheet2

    0.2170.2350.227

    1.0421.0521.032

    8.6878.7158.699

    85.19385.26385.213

    857.711857.738857.614

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Latency in ms

    LAN

    29.36641528.27328728.866415

    90.73114190.42309790.131141

    92.6272492.32253592.72724

    92.94322192.87292892.643221

    93.39905393.33095593.199053

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Bandwidth in Mbps

    LAN

    25.91526.0525.975

    28.21728.42228.429

    40.05840.54240.3

    220.114221.782221.11

    2460.1062468.0192465.23

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Latency in ms

    WAN

    3.4886353.4722373.468635

    13.66896813.20405813.468968

    16.55792416.20258116.357924

    16.63011816.423916.530118

    16.58454916.51167516.544549

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in KBytes

    Bandwidth in Mbps

    WAN

    29.36641528.27328727.63552528.866415

    90.73114190.42309790.37168590.131141

    92.6272492.32253592.22773392.72724

    92.94322192.87292892.97736592.643221

    93.39905393.33095593.50619693.199053

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Bandwidth in MBps

    LAN

    20373931

    52697162

    396429436432

    6961738874527391

    78528803918081380425

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Latency in us

    Loopback

    217235237227

    1042105210561032

    8687871587218699

    85193852638527485213

    857711857738858139857614

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Latency in us

    LAN

    25.91526.0525.975

    28.21728.42228.429

    40.05840.54240.3

    220.114221.782221.11

    2460.1062468.0192465.23

    Raw sockets

    Globus XIO

    Globus IO

    Message Size in Bytes

    Latency in ms

    WAN

    3.4886353.4722373.4850153.468635

    13.66896813.20405813.44748913.468968

    16.55792416.20258116.19775316.357924

    16.63011816.423916.40334416.530118

    16.58454916.51167516.43258516.544549

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Bandwidth in MBps

    WAN

    859555

    32.692307692336.538461538519.2307692308

    8.333333333310.1010101019.0909090909

    6.13417612417.05358425516.1772733803

    2.37240220052.90979013852.415698859

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    % increase in latency w.r.to raw sockets

    Loopback

    8.29493087569.21658986184.6082949309

    0.95969289831.34357005760.0959692898

    0.32232070910.39138943250.1381374468

    0.08216637520.09507823410.0234761072

    0.00314791350.04990025780.001515662

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    % increase in latency w.r.to raw sockets

    LAN

    3.72237469235.89411407561.7026252609

    0.33951297930.39617709650.6612944502

    0.32895830640.4313061687-0.1079596024

    0.0756300451-0.03673640710.3227777096

    0.0729108035-0.11471529590.2141349335

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    % decrease in bandwidth w.r.to raw sockets

    LAN

    0.52093382210.37044182910.2315261432

    0.72651238620.7796718290.7513201262

    1.20824804031.56273403560.6041240202

    0.75778914560.98403554520.4524927992

    0.32165280680.42380287680.2082837081

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    % increase in latency w.r.to raw sockets

    WAN

    0.47004057460.10376551290.5732901264

    3.40120775761.62030520521.4631682509

    2.14606009792.17521834261.2078808913

    1.84134592431.3636343410.6013186437

    0.43940899450.91629865850.2411883495

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size of Bytes

    % decrease in bandwidth w.r.to raw sockets

    WAN

  • Bandwidth on a LAN

  • Bandwidth on a WAN

  • Multiple drivers on a stack

    Chart2

    0.1160.106

    0.5540.565

    5.7835.68

    62.00858.982

    637.996607.974

    Globus XIO with GSI

    Globus IO with GSI

    Message Size in KBytes

    Latency in ms

    Loopback

    Sheet1

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO

    1183435270.0000290.00003434

    10355354440.0000440.00005353

    100173435280.0000290.00003434

    1000203739310.0000320.00003737

    10000526971620.0000650.00006969

    1000003964294364320.0004190.000429429

    100000069617388745273910.0074070.0073887388

    10000000785288039180813804250.08039180391

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO

    1566973810.000065650.00006969

    10547073680.000064640.0000770

    100728588830.00008800.00008585

    10002172352372270.000232300.000235235

    1000010421052105610320.0010510500.0010521052

    10000086878715872186990.00870187010.0087158715

    1000000851938526385274852130.085214852140.08526385263

    100000008577118577388581398576140.857738857738

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO0.00003535

    10.0393170.0664260.055670.0733930.02644326.4430.00005454

    100.2745180.193820.1922580.2188680.02572825.7280.00003535

    1002.0237124.6414291.8679721.8612640.02575725.7570.00003939

    100011.17434311.15030411.13962411.1419660.02597525.9750.00007171

    1000011.21551511.19868411.20120911.2013410.02842928.4290.000436436

    10000011.21894211.2066211.20720111.2077370.040340.30.0074527452

    100000011.21943711.20655211.20652911.2077180.22311223.110.08081380813

    1000000011.21948811.20621211.20665511.207613

    0.03413834.138

    0.02575525.755

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO0.02582425.824

    134.22234.13834.17226.4430.0260526.05

    1025.67625.75525.69625.7280.02842228.422

    10025.70325.82425.72825.7570.04074240.742

    100025.91526.0526.01125.9750.229782229.782

    1000028.61728.42228.43728.4290.0518451.842.4680192468.019

    10000040.05840.74243.88440.30.0520152.01

    1000000220.114223.782225.28223.110.0520252.020.03417234.172

    100000002464.1062468.0192467.5322465.230.05220452.2040.02569625.696

    0.05611856.1180.02572825.728

    0.07276672.7660.02601126.011

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO0.671603671.6030.02843728.437

    10.0005050.000520.0004470.0051340.04388443.884

    100.0624880.0606270.0603910.31034851844.26551.8442650.22528225.28

    1000.3302480.3304130.3295353.02433152033.0252.033022.4675322467.532

    10003.3310313.3316723.3313813.41237252004.19552.004195

    1000010.17634710.0641779.90050510.12042752196.6852.196680.00007373

    10000010.55887110.16270710.16538710.14766356073.28556.0732850.00007373

    100000010.63610510.50094510.45151210.5523775803.99575.8039950.00008888

    1000000010.89985610.72244210.52713710.780650.000237237

    0.05184351.8430.0010561056

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO0.05203752.0370.0087218721

    151.79451.8451.84426551.8430.05207452.0740.08527485274

    1051.98752.0152.0330252.0370.05222352.2230.858139858139

    10052.03152.0252.00419552.0740.08651386.513

    100052.1852.20452.1966852.2230.595538595.538

    1000078.27582.11881.07328586.5131.3613461361.346

    100000446.146448.766450.803995443.538

    10000001369.6771371.6031372.540941361.346

    0.000027270.00008181

    0.000044440.00006868

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO0.000028280.00008383

    10.0609990.0473490.0476180.000031310.000227227

    100.3446920.4061160.4049450.000062620.0010321032

    1002.596942.2881692.2582260.0004324320.0086998699

    10004.3007773.7013824.2840.00739173910.08521385213

    100006.5414526.1045676.3360380.080425804250.857614857614

    1000007.1477737.2487527.128835

    10000008.6445558.6366248.343988

    1000000011.51399411.50757110.984320.00005656

    0.03422234.2220.00005454

    0.02567625.6760.00007272

    0.0001160.0000910.02570325.7030.000217217

    0.0000990.0001070.02591525.9150.0010421042

    0.0001070.0000970.02961729.6170.0086878687

    0.000240.0002030.04105841.0580.08519385193

    0.0013080.0012070.220114220.1140.857711857711

    0.0130280.0123692.4641062464.106

    0.1212010.116925

    0.00001818

    Globus XIOGlobus IOGlobus XIOGlobus IO0.00003535

    10.0830.078134270.00001717

    100.0660.2291053440.0000220

    1000.070.07810034280.00005252

    10000.1160.126100037310.000396396

    100000.5540.5651000069620.0069616961

    1000005.7835.681000004294320.07852878528

    100000062.00858.982100000073887391

    10000000637.996607.974100000008039180425

    0.0000780.0780.0000830.083

    0.0002290.2290.0000660.066

    0.0000780.0780.000070.07

    0.0001260.1260.0001160.116

    0.0005650.5650.0005540.554

    0.005685.680.0057835.783

    0.05898258.9820.06000860.008

    0.607974607.9740.627996627.996

    Globus XIOGlobus IOGlobus XIOGlobus IO

    10000.1160.10610003731

    100000.5540.565100006962

    1000005.7835.68100000429432

    100000062.00858.982100000073887391

    10000000637.996607.974100000008039180425

    Globus XIO with GSIGlobus IO with GSIGlobus XIO with GSIGlobus IO with GSI

    100011610610.1160.106

    10000554565100.5540.565

    100000578356801005.7835.68

    10000006200858982100062.00858.982

    1000000063799660797410000637.996607.974

    Sheet2

    Sheet2

    18343527

    35535444

    17343528

    20373931

    52697162

    396429436432

    6961738874527391

    78528803918081380425

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Latency in us

    Loopback

    Sheet3

    56697381

    54707368

    72858883

    217235237227

    1042105210561032

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Latency in us

    LAN

    Sheet4

    0.03931710.055670.073393

    0.274518100.1922580.218868

    2.0237121001.8679721.861264

    11.174343100011.13962411.141966

    11.2155151000011.20120911.201341

    11.21894210000011.20720111.207737

    11.219437100000011.20652911.207718

    11.2194881000000011.20665511.207613

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Bandwidth in MBps

    LAN

    0.03422234.13834.17226.443

    0.02567625.75525.69625.728

    0.02570325.82425.72825.757

    0.02591526.0526.01125.975

    0.02961728.42228.43728.429

    0.04105840.74243.88440.3

    0.220114223.782225.28223.11

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Latency in ms

    WAN

    0.0005050.000520.0004470.005134

    0.0624880.0606270.0603910.310348

    0.3302480.3304130.3295353.024331

    3.3310313.3316723.3313813.412372

    10.17634710.0641779.90050510.120427

    10.55887110.16270710.16538710.147663

    100000010.50094510.45151210.55237

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Bandwidth in MBps

    WAN

    51.79451.8451.84426551.843

    51.98752.0152.0330252.037

    52.03152.0252.00419552.074

    52.1852.20452.1966852.223

    78.27582.11881.07328586.513

    446.146448.766450.803995443.538

    1369.6771371.6031372.540941361.346

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Latency in ms

    Super WAN

    0.0609990.0473490.047618

    0.3446920.4061160.404945

    2.596942.2881692.258226

    4.3007773.7013824.284

    6.5414526.1045676.336038

    7.1477737.2487527.128835

    8.6445558.6366248.343988

    11.51399411.50757110.98432

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Message Size in Bytes

    Bandwidth in MBps

    Super WAN

    3731

    6962

    429432

    73887391

    8039180425

    Globus XIO

    Globus IO

    Message Size in Bytes

    Latency in us

    Loopback

    0.1160.106

    0.5540.565

    5.7835.68

    62.00858.982

    637.996607.974

    Globus XIO with GSI

    Globus IO with GSI

    Message Size in KBytes

    Latency in ms

    Loopback

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IORaw socketsGlobus XIOGlobus IO over XIOGlobus IO

    118343527156697363

    10355354441054707368

    1001734352810072858883

    1000203739311000217235237227

    1000052697162100001042105210561043

    1000003964294364321000008687871587218699

    10000006961738874527391100000085193852638527485213

    100000007852880391808138042510000000857711857738858139857724

    0.88888888890.94444444440.50.23214285710.30357142860.125

    0.51428571430.54285714290.25714285710.29629629630.35185185190.2592592593

    11.05882352940.64705882350.18055555560.22222222220.1527777778

    0.850.950.550.08294930880.09216589860.0460829493

    0.32692307690.36538461540.19230769230.0095969290.01343570060.0009596929

    0.08333333330.1010101010.09090909090.00322320710.00391389430.0013813745

    0.06134176120.07053584260.06177273380.00082166380.00095078230.0002347611

    0.0237240220.02909790140.02415698860.00003147910.00049900260.0000151566

    Globus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO

    188.888888888994.444444444450123.214285714330.357142857112.5

    1051.428571428654.285714285725.71428571431029.629629629635.185185185225.9259259259

    100100105.882352941264.705882352910018.055555555622.222222222215.2777777778

    100085955510008.29493087569.21658986184.6082949309

    1000032.692307692336.538461538519.2307692308100000.95969289831.34357005760.0959692898

    1000008.333333333310.1010101019.09090909091000000.32232070910.39138943250.1381374468

    10000006.13417612417.05358425516.177273380310000000.08216637520.09507823410.0234761072

    100000002.37240220052.90979013852.415698859100000000.00314791350.04990025780.001515662

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IORaw socketsGlobus XIOGlobus IO over XIOGlobus IO

    10.0693170.0664260.055670.067393125.62225.73825.77225.643

    100.2745180.193820.1922580.2188681025.67625.75525.69625.728

    1002.0237122.0114291.8679721.86126410025.70325.82425.72825.757

    100011.17434311.15030411.13962411.141966100025.91526.0526.01125.975

    1000011.21551511.19868411.20120911.2013411000028.21728.42228.43728.429

    10000011.21894211.2066211.20720111.20773710000040.05840.54240.68440.3

    100000011.21943711.20655211.20652911.2077181000000220.114221.782222.28221.11

    1000000011.21948811.20621211.20665511.207613100000002460.1062468.0192470.5322465.23

    0.04170694060.19687811070.02775653880.00452735930.00585434390.0008196081

    0.29396250880.29965248180.20271894740.00307680320.00077893750.0020252376

    0.00606953950.07695759080.08027229170.00470762170.00097264910.0021009221

    0.00215126740.00310702830.0028974410.00520933820.00370441830.0023152614

    0.0015006890.00127555440.0012637850.00726512390.00779671830.0075132013

    0.0010983210.00104653360.00099875730.01208248040.01562734040.0060412402

    0.00114845330.00115050340.00104452660.00757789150.00984035550.004524928

    0.00118329820.00114381330.00105842620.00321652810.00423802880.0020828371

    Globus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO

    14.170694057719.68781107092.7756538810.45273593010.58543439230.0819608149

    1029.396250883429.965248180420.2718947391100.3076803240.07789375290.2025237576

    1000.60695395397.69575908038.02722917091000.47076216780.09726491070.2100922071

    10000.21512674170.31070283060.28974410410000.52093382210.37044182910.2315261432

    100000.15006890010.12755544440.1263785034100000.72651238620.7796718290.7513201262

    1000000.10983210360.10465336210.0998757281000001.20824804031.56273403560.6041240202

    10000000.11484533490.11505033630.104452656610000000.75778914560.98403554520.4524927992

    100000000.11832982040.11438133360.1058426196100000000.32165280680.42380287680.2082837081

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IORaw socketsGlobus XIOGlobus IO over XIOGlobus IO

    10.0005250.000520.0005170.00052410.0007140.0007040.0006930.000698

    100.0624880.0606270.0603910.061048100.1934740.1655660.1792320.183474

    1000.3302480.3301130.3295350.3243311002.0358522.0322921.9280952.015852

    10003.3310313.3306723.313813.322372100029.36641528.27328727.63552528.866415

    1000010.07634710.0641779.9985510.0204271000090.73114190.42309790.37168590.131141

    10000010.55887110.36270710.33538710.44766310000092.6272492.32253592.22773392.72724

    100000010.63610510.50094510.45151210.55237100000092.94322192.87292892.97736592.643221

    1000000010.89985610.72244210.67713710.780651000000093.39905393.33095593.50619693.199053

    10.00952380950.01523809520.00190476190.01400560220.02941176470.0224089636

    100.02978171810.03355844320.02304442450.14424677220.07361195820.0516865315

    1000.00040878370.00215898360.01791683830.00174865360.05292968250.0098238968

    10000.00010777440.00516987080.00259949550.03722374690.05894114080.0170262526

    100000.0012077790.00772075440.00554963020.00339512980.0039617710.0066129445

    1000000.01857812260.02116552040.01053218660.00328958310.0043130617-0.001079596

    10000000.01270765940.01735531950.00787271280.0007563005-0.00036736410.0032277771

    100000000.01627672880.02043320570.01093647480.000729108-0.0011471530.0021413493

    Globus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO

    10.95238095241.52380952380.190476190511.40056022412.94117647062.2408963585

    102.9781718093.35584432212.3044424531014.42467721767.36119581965.1686531524

    1000.04087837020.21589835521.79168382551000.17486536355.29296825110.9823896826

    10000.0107774440.5169870830.259949547210003.72237469235.89411407561.7026252609

    100000.1207778970.77207543570.5549630238100000.33951297930.39617709650.6612944502

    1000001.85781226042.11655204421.05321866321000000.32895830640.4313061687-0.1079596024

    10000001.2707659431.73553194520.787271280210000000.0756300451-0.03673640710.3227777096

    100000001.62767287932.04332057231.0936474757100000000.0729108035-0.11471529590.2141349335

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO

    10.0003390.0003360.0003370.000331

    100.0625910.0602510.0597240.061591

    1000.3309420.3224250.320170.329942

    10003.4886353.4722373.4850153.468635

    1000013.66896813.20405813.44748913.468968

    10000016.55792416.20258116.19775316.357924

    100000016.63011816.323916.40334416.530118

    1000000016.58454916.51167516.43258516.544549

    0.00884955750.0058997050.0235988201

    0.03738556660.04580530750.0159767379

    0.02573562740.03254951020.0030216775

    0.00470040570.00103765510.0057329013

    0.03401207760.01620305210.0146316825

    0.0214606010.02175218340.0120788089

    0.01841345920.01363634340.0060131864

    0.00439408990.00916298660.0024118835

    Globus XIOGlobus IO over XIOGlobus IO

    10.88495575220.58997050152.3598820059

    103.73855666154.58053074721.597673787

    1002.57356273913.25495101860.3021677514

    10000.47004057460.10376551290.5732901264

    100003.40120775761.62030520521.4631682509

    1000002.14606009792.17521834261.2078808913

    10000001.84134592431.3636343410.6013186437

    100000000.43940899450.91629865850.2411883495

    88.888888888994.444444444450

    51.428571428654.285714285725.7142857143

    100105.882352941264.7058823529

    859555

    32.692307692336.538461538519.2307692308

    8.333333333310.1010101019.0909090909

    6.13417612417.05358425516.1772733803

    2.37240220052.90979013852.415698859

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    % increase in latency w.r.to raw sockets

    Loopback

    MBD0002E9CF.xls

    Chart3

    116106

    554565

    57835680

    6200858982

    637996607974

    Globus XIO with GSI

    Globus IO with GSI

    Message Size in Bytes

    Latency in us

    Loopback

    Sheet1

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO

    1183435270.0000290.00003434

    10355354440.0000440.00005353

    100173435280.0000290.00003434

    1000203739310.0000320.00003737

    10000526971620.0000650.00006969

    1000003964294364320.0004190.000429429

    100000069617388745273910.0074070.0073887388

    10000000785288039180813804250.08039180391

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO

    1566973810.000065650.00006969

    10547073680.000064640.0000770

    100728588830.00008800.00008585

    10002172352372270.000232300.000235235

    1000010421052105610320.0010510500.0010521052

    10000086878715872186990.00870187010.0087158715

    1000000851938526385274852130.085214852140.08526385263

    100000008577118577388581398576140.857738857738

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO0.00003535

    10.0393170.0664260.055670.0733930.02644326.4430.00005454

    100.2745180.193820.1922580.2188680.02572825.7280.00003535

    1002.0237124.6414291.8679721.8612640.02575725.7570.00003939

    100011.17434311.15030411.13962411.1419660.02597525.9750.00007171

    1000011.21551511.19868411.20120911.2013410.02842928.4290.000436436

    10000011.21894211.2066211.20720111.2077370.040340.30.0074527452

    100000011.21943711.20655211.20652911.2077180.22311223.110.08081380813

    1000000011.21948811.20621211.20665511.207613

    0.03413834.138

    0.02575525.755

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO0.02582425.824

    134.22234.13834.17226.4430.0260526.05

    1025.67625.75525.69625.7280.02842228.422

    10025.70325.82425.72825.7570.04074240.742

    100025.91526.0526.01125.9750.229782229.782

    1000028.61728.42228.43728.4290.0518451.842.4680192468.019

    10000040.05840.74243.88440.30.0520152.01

    1000000220.114223.782225.28223.110.0520252.020.03417234.172

    100000002464.1062468.0192467.5322465.230.05220452.2040.02569625.696

    0.05611856.1180.02572825.728

    0.07276672.7660.02601126.011

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO0.671603671.6030.02843728.437

    10.0005050.000520.0004470.0051340.04388443.884

    100.0624880.0606270.0603910.31034851844.26551.8442650.22528225.28

    1000.3302480.3304130.3295353.02433152033.0252.033022.4675322467.532

    10003.3310313.3316723.3313813.41237252004.19552.004195

    1000010.17634710.0641779.90050510.12042752196.6852.196680.00007373

    10000010.55887110.16270710.16538710.14766356073.28556.0732850.00007373

    100000010.63610510.50094510.45151210.5523775803.99575.8039950.00008888

    1000000010.89985610.72244210.52713710.780650.000237237

    0.05184351.8430.0010561056

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO0.05203752.0370.0087218721

    151.79451.8451.84426551.8430.05207452.0740.08527485274

    1051.98752.0152.0330252.0370.05222352.2230.858139858139

    10052.03152.0252.00419552.0740.08651386.513

    100052.1852.20452.1966852.2230.595538595.538

    1000078.27582.11881.07328586.5131.3613461361.346

    100000446.146448.766450.803995443.538

    10000001369.6771371.6031372.540941361.346

    0.000027270.00008181

    0.000044440.00006868

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO0.000028280.00008383

    10.0609990.0473490.0476180.000031310.000227227

    100.3446920.4061160.4049450.000062620.0010321032

    1002.596942.2881692.2582260.0004324320.0086998699

    10004.3007773.7013824.2840.00739173910.08521385213

    100006.5414526.1045676.3360380.080425804250.857614857614

    1000007.1477737.2487527.128835

    10000008.6445558.6366248.343988

    1000000011.51399411.50757110.984320.00005656

    0.03422234.2220.00005454

    0.02567625.6760.00007272

    0.0001160.0000910.02570325.7030.000217217

    0.0000990.0001070.02591525.9150.0010421042

    0.0001070.0000970.02961729.6170.0086878687

    0.000240.0002030.04105841.0580.08519385193

    0.0013080.0012070.220114220.1140.857711857711

    0.0130280.0123692.4641062464.106

    0.1212010.116925

    0.00001818

    Globus XIOGlobus IOGlobus XIOGlobus IO0.00003535

    10.0830.078134270.00001717

    100.0660.2291053440.0000220

    1000.070.07810034280.00005252

    10000.1160.126100037310.000396396

    100000.5540.5651000069620.0069616961

    1000005.7835.681000004294320.07852878528

    100000062.00858.982100000073887391

    10000000637.996607.974100000008039180425

    0.0000780.0780.0000830.083

    0.0002290.2290.0000660.066

    0.0000780.0780.000070.07

    0.0001260.1260.0001160.116

    0.0005650.5650.0005540.554

    0.005685.680.0057835.783

    0.05898258.9820.06000860.008

    0.607974607.9740.627996627.996

    Globus XIOGlobus IOGlobus XIOGlobus IO

    10000.1160.10610003731

    100000.5540.565100006962

    1000005.7835.68100000429432

    100000062.00858.982100000073887391

    10000000637.996607.974100000008039180425

    Globus XIO with GSIGlobus IO with GSI

    1000116106

    10000554565

    10000057835680

    10000006200858982

    10000000637996607974

    Sheet2

    Sheet2

    18343527

    35535444

    17343528

    20373931

    52697162

    396429436432

    6961738874527391

    78528803918081380425

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Latency in us

    Loopback

    Sheet3

    56697381

    54707368

    72858883

    217235237227

    1042105210561032

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Latency in us

    LAN

    Sheet4

    0.03931710.055670.073393

    0.274518100.1922580.218868

    2.0237121001.8679721.861264

    11.174343100011.13962411.141966

    11.2155151000011.20120911.201341

    11.21894210000011.20720111.207737

    11.219437100000011.20652911.207718

    11.2194881000000011.20665511.207613

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Bandwidth in MBps

    LAN

    0.03422234.13834.17226.443

    0.02567625.75525.69625.728

    0.02570325.82425.72825.757

    0.02591526.0526.01125.975

    0.02961728.42228.43728.429

    0.04105840.74243.88440.3

    0.220114223.782225.28223.11

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Latency in ms

    WAN

    0.0005050.000520.0004470.005134

    0.0624880.0606270.0603910.310348

    0.3302480.3304130.3295353.024331

    3.3310313.3316723.3313813.412372

    10.17634710.0641779.90050510.120427

    10.55887110.16270710.16538710.147663

    100000010.50094510.45151210.55237

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Bandwidth in MBps

    WAN

    51.79451.8451.84426551.843

    51.98752.0152.0330252.037

    52.03152.0252.00419552.074

    52.1852.20452.1966852.223

    78.27582.11881.07328586.513

    446.146448.766450.803995443.538

    1369.6771371.6031372.540941361.346

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    Latency in ms

    Super WAN

    0.0609990.0473490.047618

    0.3446920.4061160.404945

    2.596942.2881692.258226

    4.3007773.7013824.284

    6.5414526.1045676.336038

    7.1477737.2487527.128835

    8.6445558.6366248.343988

    11.51399411.50757110.98432

    Raw sockets

    Globus XIO

    Globus IO over XIO

    Message Size in Bytes

    Bandwidth in MBps

    Super WAN

    3731

    6962

    429432

    73887391

    8039180425

    Globus XIO

    Globus IO

    Message Size in Bytes

    Latency in us

    Loopback

    116106

    554565

    57835680

    6200858982

    637996607974

    Globus XIO with GSI

    Globus IO with GSI

    Message Size in Bytes

    Latency in us

    Loopback

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IORaw socketsGlobus XIOGlobus IO over XIOGlobus IO

    118343527156697363

    10355354441054707368

    1001734352810072858883

    1000203739311000217235237227

    1000052697162100001042105210561043

    1000003964294364321000008687871587218699

    10000006961738874527391100000085193852638527485213

    100000007852880391808138042510000000857711857738858139857724

    0.88888888890.94444444440.50.23214285710.30357142860.125

    0.51428571430.54285714290.25714285710.29629629630.35185185190.2592592593

    11.05882352940.64705882350.18055555560.22222222220.1527777778

    0.850.950.550.08294930880.09216589860.0460829493

    0.32692307690.36538461540.19230769230.0095969290.01343570060.0009596929

    0.08333333330.1010101010.09090909090.00322320710.00391389430.0013813745

    0.06134176120.07053584260.06177273380.00082166380.00095078230.0002347611

    0.0237240220.02909790140.02415698860.00003147910.00049900260.0000151566

    Globus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO

    188.888888888994.444444444450123.214285714330.357142857112.5

    1051.428571428654.285714285725.71428571431029.629629629635.185185185225.9259259259

    100100105.882352941264.705882352910018.055555555622.222222222215.2777777778

    100085955510008.29493087569.21658986184.6082949309

    1000032.692307692336.538461538519.2307692308100000.95969289831.34357005760.0959692898

    1000008.333333333310.1010101019.09090909091000000.32232070910.39138943250.1381374468

    10000006.13417612417.05358425516.177273380310000000.08216637520.09507823410.0234761072

    100000002.37240220052.90979013852.415698859100000000.00314791350.04990025780.001515662

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IORaw socketsGlobus XIOGlobus IO over XIOGlobus IO

    10.0693170.0664260.055670.067393125.62225.73825.77225.643

    100.2745180.193820.1922580.2188681025.67625.75525.69625.728

    1002.0237122.0114291.8679721.86126410025.70325.82425.72825.757

    100011.17434311.15030411.13962411.141966100025.91526.0526.01125.975

    1000011.21551511.19868411.20120911.2013411000028.21728.42228.43728.429

    10000011.21894211.2066211.20720111.20773710000040.05840.54240.68440.3

    100000011.21943711.20655211.20652911.2077181000000220.114221.782222.28221.11

    1000000011.21948811.20621211.20665511.207613100000002460.1062468.0192470.5322465.23

    0.04170694060.19687811070.02775653880.00452735930.00585434390.0008196081

    0.29396250880.29965248180.20271894740.00307680320.00077893750.0020252376

    0.00606953950.07695759080.08027229170.00470762170.00097264910.0021009221

    0.00215126740.00310702830.0028974410.00520933820.00370441830.0023152614

    0.0015006890.00127555440.0012637850.00726512390.00779671830.0075132013

    0.0010983210.00104653360.00099875730.01208248040.01562734040.0060412402

    0.00114845330.00115050340.00104452660.00757789150.00984035550.004524928

    0.00118329820.00114381330.00105842620.00321652810.00423802880.0020828371

    Globus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO

    14.170694057719.68781107092.7756538810.45273593010.58543439230.0819608149

    1029.396250883429.965248180420.2718947391100.3076803240.07789375290.2025237576

    1000.60695395397.69575908038.02722917091000.47076216780.09726491070.2100922071

    10000.21512674170.31070283060.28974410410000.52093382210.37044182910.2315261432

    100000.15006890010.12755544440.1263785034100000.72651238620.7796718290.7513201262

    1000000.10983210360.10465336210.0998757281000001.20824804031.56273403560.6041240202

    10000000.11484533490.11505033630.104452656610000000.75778914560.98403554520.4524927992

    100000000.11832982040.11438133360.1058426196100000000.32165280680.42380287680.2082837081

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IORaw socketsGlobus XIOGlobus IO over XIOGlobus IO

    10.0005250.000520.0005170.00052410.0007140.0007040.0006930.000698

    100.0624880.0606270.0603910.061048100.1934740.1655660.1792320.183474

    1000.3302480.3301130.3295350.3243311002.0358522.0322921.9280952.015852

    10003.3310313.3306723.313813.322372100029.36641528.27328727.63552528.866415

    1000010.07634710.0641779.9985510.0204271000090.73114190.42309790.37168590.131141

    10000010.55887110.36270710.33538710.44766310000092.6272492.32253592.22773392.72724

    100000010.63610510.50094510.45151210.55237100000092.94322192.87292892.97736592.643221

    1000000010.89985610.72244210.67713710.780651000000093.39905393.33095593.50619693.199053

    10.00952380950.01523809520.00190476190.01400560220.02941176470.0224089636

    100.02978171810.03355844320.02304442450.14424677220.07361195820.0516865315

    1000.00040878370.00215898360.01791683830.00174865360.05292968250.0098238968

    10000.00010777440.00516987080.00259949550.03722374690.05894114080.0170262526

    100000.0012077790.00772075440.00554963020.00339512980.0039617710.0066129445

    1000000.01857812260.02116552040.01053218660.00328958310.0043130617-0.001079596

    10000000.01270765940.01735531950.00787271280.0007563005-0.00036736410.0032277771

    100000000.01627672880.02043320570.01093647480.000729108-0.0011471530.0021413493

    Globus XIOGlobus IO over XIOGlobus IOGlobus XIOGlobus IO over XIOGlobus IO

    10.95238095241.52380952380.190476190511.40056022412.94117647062.2408963585

    102.9781718093.35584432212.3044424531014.42467721767.36119581965.1686531524

    1000.04087837020.21589835521.79168382551000.17486536355.29296825110.9823896826

    10000.0107774440.5169870830.259949547210003.72237469235.89411407561.7026252609

    100000.1207778970.77207543570.5549630238100000.33951297930.39617709650.6612944502

    1000001.85781226042.11655204421.05321866321000000.32895830640.4313061687-0.1079596024

    10000001.2707659431.73553194520.787271280210000000.0756300451-0.03673640710.3227777096

    100000001.62767287932.04332057231.0936474757100000000.0729108035-0.11471529590.2141349335

    Raw socketsGlobus XIOGlobus IO over XIOGlobus IO

    10.0003390.0003360.0003370.000331

    100.0625910.0602510.0597240.061591

    1000.3309420.3224250.320170.329942

    10003.4886353.4722373.4850153.468635

    1000013.66896813.20405813.44748913.468968

    10000016.55792416.20258116.19775316.357924

    100000016.63011816.323916.40334416.530118

    1000000016.58454916.51167516.43258516.544549

    0.00884955750.0058997050.0235988201

    0.03738556660.04580530750.0159767379

    0.02573562740.03254951020.0030216775

    0.00470040570.00103765510.0057329013

    0.03401207760.01620305210.0146316825

    0.0214606010.02175218340.0120788089

    0.01841345920.01363634340.0060131864

    0.00439408990.00916298660.0024118835

    Globus XIOGlobus IO over XIOGlobus IO

    10.88495575220.58997050152.3598820059

    103.73855666154.58053074721.597673787

    1002.57356273913.25495101860.3021677514

    10000.47004057460.10376551290.5732901264

    100003.40120775761.62030520521.4631682509

    1000002.14606009792.17521834261.2078808913

    10000001.84134592431.3636343410.6013186437

    100000000.43940899450.91629865850.2411883495

    88.888888888994.444444444450

    51.428571428654.285714285725.7142857143

    100105.882352941264.7058823529

    859555

    32.692307692336.538461538519.2307692308

    8.333333333310.1010101019.0909090909

    6.13417612417.05358425516.1772733803

    2.37240220052.90979013852.415698859

    Globus XIO

    Globus IO over XIO

    Globus IO

    Message Size in Bytes

    % increase in latency w.r.to raw sockets

    Loopback

  • SummaryProvides a simple and uniform IO APIAppropriate for most byte stream oriented applicationsAbility to quickly adapt to Different transport protocolsDifferent data access mechanismsProvides a convenient framework for research and development