march 5, 2008 kangho kim system software team internet server group vee ‘08
DESCRIPTION
Inter-domain Socket Communications Supporting High Performance and Full Binary Compatibility on Xen. March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08. Outline. Motivation Design & Implementation Socket creation Connect Accept Send/recv Close - PowerPoint PPT PresentationTRANSCRIPT
Inter-domain Socket Communications Supporting High Performance
and Full Binary Compatibility on Xen
March 5, 2008
Kangho KimSystem Software TeamInternet Server Group
VEE ‘08
Outline
Motivation Design & Implementation
– Socket creation– Connect– Accept– Send/recv– Close– Live migration
Evaluation results Conclusion
Motivation
Low bandwidth (Xen 3.0.3)
We need XWAY!!
Motivation(cont’d)
Low latency (Xen 3.0.3)
Existing socket applications
Design & Implementation
Shared memory
Direct communication TCP/IP bypass Direct data copy
Shared memory
Hardware
VM VM VM
Ker
nel
Use
r
Domain 0 Domain U Domain U
XWAY Shared memory
TCP
XWAY channel
Head Tail
SQ
Head Tail
RQ
Head Tail
RQ
Event channel
Head Tail
SQ
Shared memoryfor one XWAY channel
Domain A Domain B
XWAY architecture
socket application
library
VFS
INET
UNIX X25…XWAY switch
TCP
IP
XWAYprotocol
Network device driverXWAY driver
BSD socket
user
kernel
UDP
Dual channel
XWAY channel for data TCP/IP channel for socket options and socket state
socket application
XWAY socket
socket application
XWAY socket
TCP/IP channel
XWAY channel
xway_sock
Socket data structure for XWAY
llseek: no_llseekpoll: sock_pollopen: sock_no_openrelease: sock_closereadv: sock_readvwritev: sock_writevsendpage: sock_sendpage
release: inet_releasebind: inet_bindconnect: xsw_stream_connectaccept: xsw_acceptpoll: xsw_polllisten: inet_listenshutdown: inet_shutdownsendmsg: inet_sendmsgrecvmsg: sock_common_recvmsgsendpage: inet_sendpage
close: xsw_closeconnect: xsw_v4_connectdisconnect: xsw_disconnectaccept: xsw_csk_acceptioctl: tcp_ioctlinit: xsw_v4_init_sockshutdown: xsw_shutdownsetsockopt: tcp_setsockoptgetsockopt: tcp_getsockopt
remote_dom_ideventchn_porteventchn_irqxring_sendxring_recvxring_send_hxring_send_txring_recv_hxring_recv_t…
inode
socket
sock
Create
Sock
xway_sock
llseek: no_llseekpoll: sock_pollopen: sock_no_openrelease: sock_closereadv: sock_readvwritev: sock_writevsendpage: sock_sendpage
release: inet_releasebind: inet_bindconnect: inet_stream_connectaccept: inet_acceptpoll: tcp_polllisten: inet_listenshutdown: inet_shutdownsendmsg: inet_sendmsgrecvmsg: sock_common_recvmsgsendpage: inet_sendpage
close: xs_closeconnect: xs_v4_connectdisconnect: xs_disconnectaccept: xs_csk_acceptioctl: xs_ioctlinit: xs_v4_init_sockshutdown: xs_shutdownsetsockopt: xs_setsockoptgetsockopt: xs_getsockopt
inode
socket
sock
Connect
TCP channel
Sock Sock
Connectionhelper
XWAY channelSQ
RQ
SQ
RQ
Accept
TCP channel
XWAY channel
Sock Sock
XWAY channel
XWAY channel
XWAY channel
Send/Recv
TCP channel
Sock Sock
XWAY channelSQ
RQ
SQ
RQ
CloseSock Sock
TCP channelTCP channel
XWAY channelSQ
RQ
SQ
RQ
XWAY channel
Poll
TCP session
XWAY session
Sock Sock
Live migration
Cut XWAY channels underneath XWAY socket– Redirect all data send/receive calls to TCP channels
Ready to live migration
Hardware
VM VM VM
Ker
nel
Use
r
Domain 0 Domain U Domain U
TCP/IP
Hardware
VM
Domain 0
Live migration (cont’d)
TCP channel
Sock Sock
XWAY channelSQ
RQ
SQ
RQ
Evaluation
Bandwidth
Netperf TCP_STREAM
Latency
Netperf (TCP_RR)
Application performance
DBT-1 performance
Execution time of API
Binary compatibility
Scp Ssh Apache – TCP_DEFER_ACCEPT Vsftpd – connect, accept, MSG_PEEK Proftpd – shutdown, connect, SO_LINGER Telnet MySQL Netperf
Conclusion
XWAY design and implementation– Full binary compatibility for socket interface
– High performance
– Live migration support Remaining work
– UDP, …
– Domain crash resiliency
– Reducing CPU usage Code release
– April 20, 2007: xway ver 0.6
– June 4, 2007: xway ver 0.6.1
– By the end of March, 2008: xway ver 0.7 (live migration included) Download
– http://sourceforge.net/projects/xway
Questions or Comments
Kangho Kim– [email protected]
Cheiyol Kim– [email protected]
Hyun-Sup Shin– [email protected]