hybrid reliable multicast with tcp-xm jon crowcroft marinho p barcellos, unisinos stefano pettini,...
TRANSCRIPT
![Page 1: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/1.jpg)
Hybrid Reliable Multicast with TCP-XM
Jon CrowcroftMarinho P Barcellos, UNISINOS
Stefano Pettini, ESA
Karl Jeacle
![Page 2: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/2.jpg)
CoNEXT 2005
Rationale
Would like to achieve high-speed bulk data delivery to multiple sites
Multicasting would make sense Existing multicast research has focused on
sending to a large number of receivers But Grid is an applied example where sending to
a moderate number of receivers would be extremely beneficial
![Page 3: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/3.jpg)
CoNEXT 2005
Multicast availability
Deployment is a problem!Protocols have been defined and implementedValid concerns about scalability; much FUD “chicken & egg” means limited coverage
Clouds of native multicastBut can’t reach all destinations via multicastSo applications abandon in favour of unicast
What if we could multicast when possible……but fall back to unicast when necessary?
![Page 4: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/4.jpg)
CoNEXT 2005
Multicast TCP?
TCP “single reliable stream between two hosts”
Multicast TCP “multiple reliable streams from one to n hosts”
May seem a little odd, but there is precedent…TCP-XMO – Liang & CheritonM-TCP – Mysore & VargheseM/TCP – Visoottiviseth et alPRMP – Barcellos et alSCE – Talpade & Ammar
![Page 5: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/5.jpg)
CoNEXT 2005
ACK implosion
![Page 6: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/6.jpg)
CoNEXT 2005
Building Multicast TCP
Want to test multicast/unicast TCP approachBut new protocol == kernel changeWidespread test deployment difficult
Build new TCP-like engineEncapsulate packets in UDPRun in userspacePerformance is sacrificed……but widespread testing now possible
![Page 7: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/7.jpg)
CoNEXT 2005
TCP/IP/UDP/IP
SendingApplication
TCP
IP
UDP
IP
ReceivingApplication
TCP
IP
UDP
IP
If natively implemented
test deployment
![Page 8: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/8.jpg)
CoNEXT 2005
TCP engine
Where does initial TCP come from? Could use BSD or Linux
Extracting from kernel could be problematic
More compact alternative lwIP = Lightweight IPSmall but fully RFC-compliant TCP/IP stack
lwIP + multicast extensions = “TCP-XM”
![Page 9: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/9.jpg)
CoNEXT 2005
TCP-XM overview
Primarily aimed at push applications Sender initiated – advance knowledge of receivers Opens sessions to n destination hosts simultaneously Unicast is used when multicast not available Options headers used to exchange multicast info API changes
Sender incorporates multiple destination and group addresses Receiver requires no changes
TCP friendly, by definition
![Page 10: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/10.jpg)
CoNEXT 2005
TCP
SYN
SYNACK
ACK
FIN
FIN
ACK
ACK
ACK
DATASender Receiver
![Page 11: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/11.jpg)
CoNEXT 2005
TCP-XM
Sender
Receiver 1
Receiver 2
Receiver 3
![Page 12: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/12.jpg)
CoNEXT 2005
TCP-XM connection
ConnectionUser connects to multiple unicast destinationsMultiple TCP PCBs created Independent 3-way handshakes take placeSSM or random ASM group address allocated
(if not specified in advance by user/application)
Group address sent as TCP optionAbility to multicast depends on TCP option
![Page 13: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/13.jpg)
CoNEXT 2005
TCP Group Option
New group option sent in all TCP-XM SYN packets Non TCP-XM hosts will ignore (no option in SYNACK) Presence implies multicast capability Sender will automatically revert to unicast
1 byte 1 byte 4 bytes
kind=50 len=6 Multicast Group Address
![Page 14: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/14.jpg)
CoNEXT 2005
TCP-XM transmission
Data transferData replicated/enqueued on all send queuesPCB variables dictate transmission modeData packets are multicast (if possible)Retransmissions are unicastAuto fall back/forward to unicast/multicast
CloseConnections closed as per unicast TCP
![Page 15: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/15.jpg)
CoNEXT 2005
TCP-XM protocol states
![Page 16: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/16.jpg)
CoNEXT 2005
Fall back / fall forward
TCP-XM principle “Multicast if possible, unicast when necessary”
Initial transmission mode is group unicastEnsures successful initial data transfer
Fall forward to multicast on positive feedbackTypically after ~75K unicast data
Fall back to unicast on repeated mcast failure
![Page 17: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/17.jpg)
CoNEXT 2005
Multicast feedback Option
New feedback option sent in ACKs from receiver Only used between TCP-XM hosts Indicates % of last n packets received via multicast Used by sender to fall forward to multicast transmission
1 byte 1 byte 1 byte
kind=51 len=3 % Multicast Packets Received
![Page 18: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/18.jpg)
CoNEXT 2005
TCP-XM reception
ReceiverNo API-level changesNormal TCP listenAuto-IGMP join on TCP-XM connectAccepts data on both unicast/multicast ports tcp_input() accepts:
packets addressed to existing unicast destination……but now also those addressed to multicast group
Tracks how last n segs received (u/m)
![Page 19: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/19.jpg)
CoNEXT 2005
API changes
Only relevant if natively implemented!
Sender API changesNew connection typeConnect to port on array of destinationsSingle write sends data to all hosts
TCP-XM in use:conn = netconn_new(NETCONN_TCPXM);
netconn_connectxm(conn, remotedest, numdests, group, port);
netconn_write(conn, data, len, …);
![Page 20: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/20.jpg)
CoNEXT 2005
PCB changes
Every TCP connection has an associated Protocol Control Block (PCB)
TCP-XM adds:struct tcp_pcb {
…
struct tcp_pcb *firstpcb;/* first of the mpcbs */
struct tcp_pcb *nextm; /* next tcpxm pcb */
enum tx_mode txmode; /* unicasting or multicasting */
u8_t nrtxm; /* number of retransmits for multicast */
u32_t nrtxmtime; /* time since last retransmit */
u32_t mbytessent; /* total bytes sent via multicast */
u32_t mbytesrcvd; /* total bytes received via multicast */
u32_t ubytessent; /* total bytes sent via unicast */
u32_t ubytesrcvd; /* total bytes received via unicast */
struct segrcv msegrcv[128]; /* ismcast boolean for last n segs */
u8_t msegrcvper; /* % of last segs received via mcast */
u8_t msegrcvcnt; /* counter for segs recvd via mcast */
u8_t msegsntper; /* % of last segs delivered via mcast */
}
![Page 21: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/21.jpg)
CoNEXT 2005
Linking PCBs
*next points to the next TCP session *nextm points to the next TCP session that’s part of a
particular TCP-XM connection Minimal timer and state machine changes
next
nextm
M1
next
nextm
U1
next
nextm
M2
next
nextm
M3
next
nextm
U2
![Page 22: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/22.jpg)
CoNEXT 2005
What happens to the cwin?
Multiple receiversMultiple PCBsMultiple congestion windows
Default to min(cwin) i.e. send at rate of slowest receiver
Is this really so bad?Compare to time taken for n unicast transfers
![Page 23: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/23.jpg)
CoNEXT 2005
LAN speed
![Page 24: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/24.jpg)
CoNEXT 2005
LAN efficiency
![Page 25: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/25.jpg)
CoNEXT 2005
WAN speed
![Page 26: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/26.jpg)
CoNEXT 2005
WAN efficiency
![Page 27: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/27.jpg)
CoNEXT 2005
Multiple TCP-XM flows
![Page 28: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/28.jpg)
CoNEXT 2005
TCP-XM vs TCP flows
![Page 29: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/29.jpg)
CoNEXT 2005
Protocol performance
![Page 30: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/30.jpg)
CoNEXT 2005
Protocol efficiency
![Page 31: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/31.jpg)
CoNEXT 2005
Grid multicast?
How can multicast be used in Grid environment?
TCP-XM is new multicast-capable protocol
Globus is de-facto Grid middleware
Would like TCP-XM support in Globus…
![Page 32: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/32.jpg)
CoNEXT 2005
Globus XIO
eXtensible Input Output libraryAllows “i/o plugins” to Globus
APISingle POSIX-like API / set of semanticsSimple open/close/read/write API
Driver abstractionHides protocol details / Allows for extensibilityStack of 1 transport & n transform driversDrivers can be selected at runtime
![Page 33: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/33.jpg)
CoNEXT 2005
XIO architecture
![Page 34: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/34.jpg)
CoNEXT 2005
XIO implementation
![Page 35: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/35.jpg)
CoNEXT 2005
XIO/XM driver specifics
Two important XIO data structures
1. Handle Returned to user when XIO framework ready Used for all open/close/read/write calls lwIP netconn connection structure used
2. Attribute Used to set XIO driver-specific parameters… … and TCP-XM protocol-specific options List of destination addresses
![Page 36: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/36.jpg)
CoNEXT 2005
XIO code example// init stackglobus_xio_stack_init(&stack, NULL);
// load drivers onto stackglobus_xio_driver_load("tcpxm", &txdriver);globus_xio_stack_push_driver(stack, txdriver);
// init attributesglobus_xio_attr_init(&attr);globus_xio_attr_cntl(attr, txdriver, GLOBUS_XIO_TCPXM_SET_REMOTE_HOSTS,
hosts, numhosts);
// create handleglobus_xio_handle_create(&handle, stack);
// send dataglobus_xio_open(&handle, NULL, target);globus_xio_write(handle, "hello\n", 6, 1, &nbytes, NULL);globus_xio_close(handle, NULL);
![Page 37: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/37.jpg)
CoNEXT 2005
One-to-many issues
Stack assumes one-to-one connectionsXIO user interface requires modificationNeeds support for one-to-many protocolsMinimal user API changesFramework changes more significant
GSI is one-to-oneAuthentication with peer on connection setupBut cannot authenticate with n peersNeed some form of “GSI-M”
![Page 38: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/38.jpg)
CoNEXT 2005
Driver availability
Multicast transport driver for Globus XIORequires Globus 3.2 or later
Source code onlineSample clientSample serverDriver installation instructions
http://www.cl.cam.ac.uk/~kj234/xio/
![Page 39: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/39.jpg)
CoNEXT 2005
mcp & mcpd
Multicast file transfer application using TCP-XM
‘mcpd &’ on servers ‘mcp file host1 host2… hostN’ on client
http://www.cl.cam.ac.uk/~kj234/mcp/Full source code onlineFreeBSD, Linux, Solaris
![Page 40: Hybrid Reliable Multicast with TCP-XM Jon Crowcroft Marinho P Barcellos, UNISINOS Stefano Pettini, ESA Karl Jeacle](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d045503460f949d820d/html5/thumbnails/40.jpg)
CoNEXT 2005
All done!
Thanks for listening!
Questions?