![Page 1: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/1.jpg)
RDMA ENABLED WEB SERVER
Rajat Sharma
![Page 2: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/2.jpg)
Objective
To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP data transfer model.
RDMA reduces the CPU overhead at the server side leading to high performance data transfer , Computing and increased Scalabilityin terms of Clients
![Page 3: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/3.jpg)
Known Implementation
Researchers from Ohio Supercomputing Center have implemented what is known as
“ RDMA enabled Apache ”having a special hardware RDMA NIC card as 10 Gbit iWarp (RDMA over TCP) network card.
Summarized the details into a paper
![Page 4: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/4.jpg)
“Accelerating Web Protocols Using RDMA”
“Dennis Dalessandro and Pete WyckoffOhio Supercomputing Center”
Topics of Discussion :
AbstractLot of applications challenge the capacities of server. Increased capacity is achieved through module mod_rdma with unmodified Apache code
![Page 5: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/5.jpg)
Novel Technology
• RDMA vastly improves Performance and Scalability for clients
• Information sent over the network as TCP/IP packets includes multiple copies of the same data resulting in large CPU overhead limiting the server capacity to serve more clients
• TCP cannot serve the need of Supercomputing
![Page 6: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/6.jpg)
Examples : RDMA real life need
• Large data centers : Medical Research Institutes - Access of remote data bases for Researchers -> High Throughput
• Grid Chem Client: Monitoring of Parallel calculations through CGI scripts need large files compressing , sensing or receiving consumes a lot of processing power
![Page 7: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/7.jpg)
• Todays client Common to have 1 MB web content .
• HPC will need high data transfers for dynamic content
Online gaming servers and Streaming Servers will not function well without efficient high speed data transfer
![Page 8: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/8.jpg)
Performance Bottlenecks
• Most of the performance bottlenecks are often found at the server side
• Several measures to remove themRound Robin DNSActive monitoring for load balancing
But data transfer is a biggest limitation -> Nature Of HTTP over TCP
![Page 9: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/9.jpg)
Contd..
Protocol acceptance : Tough Enabling clients with Iwarp software ->No CPU improvement
![Page 10: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/10.jpg)
Why APACHE ??
• Most popular web server• Wide acceptance• Allow third party modules to affect the server
behavior • “Hooks” feature -> used by the Module to
access the web server
![Page 11: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/11.jpg)
Need for RDMA
• Traditional TCP Packet processing• Multiple Copies of data
![Page 12: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/12.jpg)
Direct Data transfer
• Allows adapters to move data from Memory to Memory without CPU
• Advantages– Prevents system call overhead– Hardware interrupts– Latency– Data movement
![Page 13: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/13.jpg)
RDMA and IWARP
![Page 14: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/14.jpg)
RDMA enabled hardware
• RDMA Needs special hardware to be deployed• Knows as RNIC• Main features – Zero Copy
– OS Bypass
![Page 15: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/15.jpg)
IWARP software
• RNICS cards are expensive• NOT viable to have an RNIC card on every client• RNIC enabled Server -----------------
Normal NIC enabled client
HOW ?Solution : IWARP software Protocol
OR IWARP emulator
![Page 16: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/16.jpg)
Efficient Server • RDMA software – not as efficient in Client side
-- No improvement in CPU overhead on client side since no hardware.
RNIC RDMA RDMA Iwarp hardware software
SERVER CLIENT
![Page 17: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/17.jpg)
Soft Iwarp implementations
• User space implementation can be used by applications in user space linking
to library to generate RDMA packets
• Kernel space implementation RDMA calls in kernel space
Soft IWARP : -- Web browser plug in module ..
![Page 18: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/18.jpg)
Design• Web server decision :
other than Apache , all servers lack the third party interaction freedom• Backward Compatibilty
Header Field
RDMA header
RNIC Port Number
Stag
Toffset
Max Len
![Page 19: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/19.jpg)
RDMA data transfer Schemes• GET request and POST request
GET REQUEST Server Writes and Client Reads
Server writes ->
Server ----------------------------------- ClientRDMA write
Client Reads Client ------------------------------------ Server
RDMA read
![Page 20: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/20.jpg)
GET: Server-writes
![Page 21: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/21.jpg)
GET : Client reads
![Page 22: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/22.jpg)
POST Request
• POST requests work in the same way
: Writing data to the server
– Post Request will have
• Client – Writes
• Server – Reads
![Page 23: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/23.jpg)
RDMA connection• Server should be ready to start before the client initiates• Client ---------------------------- Server
– RTT TimeCase Server –Writes
If client GET request Max_ buffer_length < Resource_LenServer ERROR mess
Case client – reads
Max_buffer_len < Resource _client_needsClient Error: Stop RDMA GET request
![Page 24: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/24.jpg)
Implementation
• RDMA mod_rdma module interacts with Apache Server using “hook”
• Child Initrdma_child_init( )
• Pre – connection rdma_pre_connection_handler( )
• Connection Cleanuprdma_connection_cleanup( )
![Page 25: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/25.jpg)
Contd..
• Insert Filterrdma_insert_filter( )
Output Filterrdma_output_filter( )
Ship_file( )
Ship_heap( )
![Page 26: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/26.jpg)
Limitations• RDMA connection cost
• Memory Registration
– Static Registration– Pre –registering size of memory.Mmap () and then register the memory in case of file from disk How ???
Difference b/w memory allocated and memory registered?
– Dynamic Registration
Placing the dynamic content requires memcopy every time to copy the data to RDMA buffer, thus mod_rdma causes one more memory copy
![Page 27: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/27.jpg)
Experimental Results
• Test our web server using 2 processes value in the background , simulating it as an heavily
loaded CPU by using nice function.
15 nodesMain Memory = 2GB10Gbit Iwarp adapterResults shows some hidden costs in terms of
memory
![Page 28: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/28.jpg)
Hidden Costs
![Page 29: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/29.jpg)
Single Client Performance
Highly loaded CPU costs more memory registration as mem_reg = CPU task
![Page 30: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/30.jpg)
Multiple client Performance
![Page 31: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/31.jpg)
Alternate Approach
• Inclusion of Proxy server to implement a Web Server.
![Page 32: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/32.jpg)
Proposed areas in future work
• More requests like PUT • Kernal based implementation of the Web
Server.– Resulting in change of path of data from kernel to
NIC card– Fast implementation
![Page 33: RDMA ENABLED WEB SERVER Rajat Sharma. Objective To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d5d5503460f94a3be83/html5/thumbnails/33.jpg)