ibm research © 2009 ibm corporation highly scalable web applications with zero-copy data transfer...

IBM Research

© 2009 IBM Corporation

Highly Scalable Web Applications with Zero-Copy Data Transfer

Toyotaro Suzumura, Michiaki Tatsubori, Scott Trent, Akihiko Tozawa, and Tamiya OnoderaTokyo Research LaboratoryIBM Research

2009/4/22 17:00-17:30 Web-Eng 3 - Web Architecture Aspect

IBM Research

© 2009 IBM Corporation2

Executive Summary

Scale-up approach still matters for ultimate scalability– The performance of server-side applications is becoming increasingly

important as more applications exploit the Web application model in the Web 2.0 era

Our Main Contribution– A novel approach to boost Web application performance with what we

call “Zero Copy Data Transfer” that reduces redundant memory copying and context switch overhead between user space and kernel space

Achievements 126% performance improvement with micro-benchmarks 31% performance improvement for standard Web

benchmark, SPECweb2005

IBM Research


Outline of this talk

1. Motivation

2. Our Solution: Zero Copy Data Transfer

3. Performance Evaluation

4. Related Work, Future Direction and Conclusions

IBM Research


Background : Web Server Performance Does Matter

In recent years, software applications are increasingly being developed to adopt the Web application model via HTTP protocol in the Web 2.0 era.

This rapidly growing use is dramatically increasing the performance requirements for Web application servers.

Gigantic internet companies such as Microsoft, Google, Amazon.com, and Yahoo! have introduced ten thousands of physical hosts to maintain QoS and scalability

Total Sites Across All Domains August 1995 - March 2009 (Netcraft)

http://news.netcraft.com/archives/web_server_survey.html

IBM Research


Optimizing Web Server Performance

Static Web Server (Only serving static files) – In kernel web servers [Almol, TOCS ‘04] [King,USENIX ‘01]– Zero-copy Approach [Lighttpd 1.5 beta, ’08]

Dynamic Web Server (including business logic) = HTTP Server + Connector (SAPI)

+ Script Execution Runtime – Scale-Out Approach with Load Balancing – Optimizing Script Execution Runtime with Just-in-time compiler

[Tozawa, PHP ’08] [YARV] – Dynamic Web Server Comparison : PHP vs. JSP [Trent,

Middleware 2008] • Lighttpd / FastCGI / PHP was the best !

How can we make web sites more scalable to support users ?

We want more.

Any other optimization ?

IBM Research


Profiling Dynamic Web Server with SPECweb2005

Dynamic web server consists of Lighttpd Web Server, PHP Runtime (P9), and FastCGI as SAPI

The profiling result shows that significant time is spent on memory copying both in Web server and PHP runtime

37%

21%

5%

13%

24%

runtime librarylibc- 2.6.socompiled codeextension libraryother

CPU Usage in PHP (P9) (SPECweb Banking)

53999 9.4771 libc-2.6.so lighttpd memcpy39009 6.8463 libc-2.6.so phoebe-fcgi memcpy34101 5.9849 e1000 lighttpd (no symbols)23828 4.1819 libcrypto.so.0.9.8b lighttpd bn_mul_add_words20247 3.5534 libp9rtsvc24.so phoebe-fcgi storeGenericAux19645 3.4478 libcrypto.so.0.9.8b lighttpd bn_sqr_comba812944 2.2717 libcrypto.so.0.9.8b lighttpd BN_from_montgomery11547 2.0266 libc-2.6.so lighttpd _int_malloc10737 1.8844 libp9rtsvc24.so phoebe-fcgi loadIndex

CPU Usage (%) Memory

copy

IBM Research


Sniffing FastCGI Packets in SPECweb Banking

\0x01\0x01\0\0x01\0\0x08\0\0\0\0x01\0\0\0\0\0\0\0x01\0x04\0\0x01\0x05T\0\0\0x0F\0x0FSERVER_SOFTWARElighttpd/1.4.18\0x0B\0x19SERVER_NAMEmichis3.trl.ibm.com:8099\0x11\0x07GATEWAY_INTERFACECGI/1.1\0x0B\0x04SERVER_PORT8099\0x0B\0x0CSERVER_ADDR9.116.14.105\0x0B\0x04REMOTE_PORT3456\0x0B\0x0BREMOTE_ADDR9.116.14.91\0x0B\0x19SCRIPT_NAME/bank/account_summary.php\0x09\0PATH_INFO\0x0FFSCRIPT_FILENAME/home/suzumura/software/lighttpd/var/www/html/bank/account_summary.php\r.DOCUMENT_ROOT/home/suzumura/software/lighttpd/var/www/html/\0x0B\0x19REQUEST_URI/bank/account_summary.php\0x0C\0QUERY_STRING\0x0E\0x03REQUEST_METHODGET\0x0F\0x03REDIRECT_STATUS200\0x0F\0x08SERVER_PROTOCOLHTTP/1.1\0x09\0x19HTTP_HOSTmich-is3.trl.ibm.com:8099\0x0FYHTTP_USER_AGENTMozilla/5.0 (Windows; U; Windows NT 5.1; ja; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11\0x0BcHTTP_ACCEPTtext/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5\0x14\0x17HTTP_ACCEPT_LANGUAGEja,en-us;q=0.7,en;q=0.3\0x14\0x0CHTTP_ACCEPT_ENCODINGgzip,deflate\0x13\0x1DHTTP_ACCEPT_CHARSETShift_JIS,utf-8;q=0.7,*;q=0.7\0x0F\0x03HTTP_KEEP_ALIVE300\0x0F\nHTTP_CONNECTIONkeep-alive\0x0CFHTTP_REFERERhttp://mich-is3.trl.ibm.com:8099/bank/check_detail_html.php?check_no=1\0x0B\0x80\0\0x01VHTTP_COOKIECoreID6=46881926841911901663357; w3ibmProfile=2005081020414106874663086|gASP|760|488|null; sauidp=U162807841190166314000; s_nr=1190095558906; ibmSurvey=1211076852290; CoreID6=46881926841911901663357; w3ibmProfile=2005081020414106874663086|gASP|760|488|null; sauidp=U162807841190166314000; s_nr=1190095558906; ibmSurvey=1211076852290;userid=1\0x01\0x04\0\0x01\0\0\0\0\0x01\0x05\0\0x01\0\0\0\0

\0x01\0x06\0\0x01\0x1F\0xF8\0\0Set-Cookie: CoreID6=46881926841911901663357;p

ath=/\r\nSet-Cookie: w3ibmProfile=2005081020414106874663086|gASP|760|488|null;pa

th=/\r\nSet-Cookie: sauidp=U162807841190166314000;path=/\r\nSet-Cookie: s_nr=119

0095558906;path=/\r\nSet-Cookie: ibmSurvey=1211076852290;path=/\r\nSet-Cookie: u

serid=1;path=/\r\nContent-Type: text/html\r\n\r\n<!DOCTYPE html PUBLIC "-//W3C//

DTD HTML 4.01 Transitional//EN" \r\n"http://www.w3.org/TR/html4/loose.dtd">\r\n<

html>\r\n <head>\r\n <title>SPECweb2005: Account Summary</title>\r\n </head

>\r\n <body bgcolor="white">\r\n \r\n <

table summary="SPECweb2005_User_Id">\r\n <tr><th>User ID</th></tr>\r\n <t

r><td>1</td></tr> \r\n </table>\r\n <table summary="SPECweb2005_Acct_Summary"

cellpadding=3 border=1>\r\n <tr>\r\n <th>Account</th>\r\n <th>Type<

/th>\r\n <th>Current Balance</th>\r\n <th>Total Deposits</th>\r\n

<th>Average Deposit</th>\r\n <th>Total Withdraws</th>\r\n <th>Average

Withdraws</th>\r\n </tr>\r\n <tr>\r\n <td>0000000251</td>\r\n

<td> Saving\r\n </td>\r\n <td>9373.01</td>\r\n

<td>73.01</td>\r\n <td>73.01</td>\r\n <td>67.01</td>\r\n <td>6

7.01</td>\r\n\0x09</tr>\r\n <tr>\r\n <td>0000000252</td>\r\n

<td> Other\r\n </td>\r\n <td>9373.01</td>\r\n <td

>173.01</td>\r\n <td>73.01</td>\r\n <td>117.01</td>\r\n <td>67.01

</td>\r\n\0x09</tr>\r\n </table> \r\n \r\n <table>\r\n <tr>\r\n <td><b><font color="#9400d3">SPECwe

b2005: Account Summary</font></b></td>\r\n </tr>\r\n </table>\r\n \r\n \0x09<ul>\r\n\0x09 <li><a href="account_summa

ry.php">Account Summary</a>\r\n\0x09 <li><a href="check_detail_input.php">Check

Detail</a>\r\n <li><a href="profile.php">Change Profile</a>\r\n <li><

a href="transfer.php">Transfer Money</a>\r\n\0x09 <li>Bill Pay\r\n <ul>\

r\n <li><a href="add_payee.php">Add Payee</a>\r\n <li><a href=

"bill_pay.php">Quick Pay</a>\r\n <li><a href="bill_pay_status_input.php

">Check Status</a>\r\n </ul>\r\n\0x09 <li><a href="order_check.php">Orde

r Check</a>\r\n <li><a href="logout.php">Logout</a>\r\n\0x09</ul>\r \r\n \r\n<pre>

HTTP server

PHP Runtime

account_summary.phpin SPECweb Banking

Response: 19352 bytes (19KB)

Request: 1381 bytes

FastCGI

IBM Research


Relatively Static Part(Cached File)

Simplified Dynamic Web Page

Semi-Static Part(Cached File, Long Characters, etc)

Footer Part

DB

Header Part

File System

Dynanmic Part

IBM Research


Interaction between HTTP Server and PHP Runtime

HTTP Server

PHP Runtime

Kernel Space

User Space

socket

Buffer

PHP Appheader_processing();

echo file_get_contents(‘fileA’);

footer_processing();

File

FastCGI

File System

Buffer

BufferFile

Content

IBM Research



1. Background & Motivation




IBM Research


Our Solution: Zero Copy Data Transfer

Reduces inter-process communication overhead between web server and PHP runtime

Normally, PHP reads file contents, converts it to string buffer, and forwards it as a FastCGI packet.

Instead, do the following– PHP runtime passes the “file name” within a FastCGI packet.

– Web server then uses the information to invoke a zero copy system call such as sendfile, which is supported by major operating systems

Note that PHP developers need not modify their scripts to use our optimization

IBM Research


Proposed ApproachHTTP Server

PHP Runtime

Kernel Space

User Space

socket

Buffer

PHP Appheader_processing();

echo file_get_contents(‘fileA’);

footer_processing();

File

FastCGI

File System

Buffer

BufferFile URI

X-ZeroCopy

X-ZeroCopyHandling

sendfile

IBM Research


File Processing : FTCS (File-Type Character String) Object

A new type of character string object that only holds the file name (URI) of a file without reading the entire content of a file as an ordinary character string object

abcdefghijklmnopqrstuvwxyz

abcdefghijklmnopqrstuvwxyz

PHP Runtime

abcdefghijklmnopqrstuvwxyz URI: fileAPHP

Runtime

Ordinal Character

String Object File(fileA)

File(fileA)

FTCS Object

IBM Research


X-ZeroCopy: Enhancing FastCGI Protocol

.X-ZeroCopy : New HTTP Header (HTTP Extension) – File name and location are included in the body portion of a

FastCGI message– The header is recursively defined to allow the transmission of

multiple files

Our PHP runtime automatically generates an X-ZeroCopy header and body content in a transparent manner from unmodified PHP scripts

X-ZeroCopy = “X-ZeroCopy” “:” # (offset “/” length )

IBM Research


Example

X-ZeroCopy: 5/10

hello/tmp/A.htmworld

<?php

echo “hello”; $a = file_get_contents(‘/tmp/A.html’); echo $a; echo “world”;?>

FTCS(A.html)

PHPRuntime

IBM Research


Transparency to Applications

PHP developers need not modify their applications to leverage our proposed approach

If there is a side effect on FTCS object, our PHP runtime loads the content of a while when needed (Lazy I/O Processing)– In this case, performance improvement can not be obtained

<?php $a = file_get_content(“fileA”);

$b = trim($a); // remove white space?>

IBM Research







IBM Research


Performance Evaluation

Micro-benchmark– Theoretically our approach should be effective when the

FastCGI communication overhead between the PHP runtime and the Web server is a major performance bottleneck.

– To find the threshold where file size becomes a bottleneck, we prepared a simple PHP script micro-benchmark that simply displays a file (ranging from 10 KB to 200 KB) via the file_get_contents PHP extension.

SPECweb2005– A standard web benchmark consisting of 3 web representative

scenarios: Banking, Ecommerce, and Support

IBM Research


Comparative PHP Runtimes: P9 vs. P9ZC vs. Zend

P9 – Our research PHP runtime with Just-in-time compiler – Single Thread + 8 FastCGI Processes

P9ZC– P9 with our proposed zero-copy data transfer – Single Thread + 8 FastCGI Processes

Zend (PHP 5.2.5) (Only used for SPECweb) – A major PHP runtime available from www.php.net– APC (Alternative PHP Cache) is turned on to allow a

PHP intermediate code to be cached in shared memory– Single Thread + 8 FastCGI Processes

IBM Research


Micro-benchmark: Throughput and Speedup (%) of P9ZC over P9 with varying file sizes.

File Size large

Speedup

P9ZC

•The speedup of P9ZC over P9 increases from 1.26 for a 10 K file up to 2.26 for a 60 K file.

•After 60K, the speedup gradually decreases but P9ZC remains roughly twice as fast as P9

P9

Apache Bench (ab) with 1 process, 100 concurrent requests, and a 60 second run, measured after sufficient warm-up

SUT (3GB, 3.4GHz Xeon, Fedora Core7), Prime Client (3.4 Ghz Xeon), P9 as of 2008/08/08, Zend (PHP 5.2.5) with APC enabled,

Throughput

IBM Research


Micro-benchmark: CPU usage for the memcpy function.

CPU Usage of memcpy

0

10

20

30

40

50

60

70

File Size (KB)

CP

U U

sage

of m

emcp

y (%

)

P9

P9FS

P9 22.8 33.6 46 50.4 55.5 55.7 59.7 60.4 61.5 62.5

P9FS 4.32 4.12 3.93 3.32 2.9 2.8 2.64 2.43 2.56 2.46

20 40 60 80 100 120 140 160 180 200

P9ZC

P9ZC

CPU Usage of memcpy

0

10

20

30

40

50

60

70

File Size (KB)

CP

U U

sage

of m

emcp

y (%

)

P9

P9FS

P9 22.8 33.6 46 50.4 55.5 55.7 59.7 60.4 61.5 62.5

P9FS 4.32 4.12 3.93 3.32 2.9 2.8 2.64 2.43 2.56 2.46

20 40 60 80 100 120 140 160 180 200

P9ZC

P9ZC

File Size large

P9ZC significantly reduces CPU usage for the memcpy function while P9 needs uses approximately from 20% to 60% on them of CPU time to perform memory copying

Lower is better

% of memcpy

IBM Research


Performance Evaluation with SPECweb2005

Linux (kernel 2.6.17)

PHPProcess

BESIM(databaseSimulator)

HTTP server

Application Server Backend (Business Logic / DB) Simulator

6 Clients

Named pipe / TCPsocket

tcp/ipsocket

Client(Emulator)

FastCGIProtocol

HTTPApache

HTTP server(FastCGI)

Linux 2.6.18 2GB RAM, Xeon 2.4GH, 2CPU

Fedora Core 7, kernel 2.6.17, Pentium 4 3.4GHz, 2GB RAM)

Linux 2.6.17, Xeon 2.4GHz, 1GB)

Client(Emulator)Client(s)

6 clients

(Linux 2.6.2, Xeon 3.0GHz, 3GB)

Lighttpd1.4.19

mod_fcgi

8 processes

IBM Research


SPECweb2005 Banking

Comparison by Good+Tolerable Requests

0

5000

10000

15000

20000

25000

200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 3000

sessions

# of

req

uest

s

Zend

P9

P9 ZC

No performance improvement is observed due to the fact that sendfile is ineffective with SSL communication and

SUT (3GB, 3.4GHz Xeon, Fedora Core7), 3 Clients and Prime Client (3.4 Ghz Xeon), BESIM: Apache 2.2.26, IBM J2RE 1.5.0 Linux build, P9 as of 2008/08/08, Zend (PHP 5.2.5) with APC enabled, SPECweb 3 minute run

Average Data Transfer Size: 34.8 KB

Higher is better

Throughput

IBM Research


SPECweb2005 Ecommerce

Comparison by Good Requests (Ecommerce)

0

5000

10000

15000

20000

25000

10020

030

040

050

060

070

080

090

010

0011

0012

0013

0014

0015

0016

0017

0018

0019

0020

0021

0022

0023

0024

0025

0026

0027

0028

0029

0030

00

sessions

# o

f re

quest

s

Zend

P9

P9 ZC

P9 Level 1


Our approach outperforms original P9 by 22%, and Zend by 57%


Higher is better

Throughput

Sessions

IBM Research


SPECweb2005 Support

Comparison by Good+Tolerable Requests in Support

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

10020

030

040

050

060

070

080

090

010

0011

0012

0013

0014

0015

0016

0017

0018

0019

0020

0021

0022

0023

0024

0025

00

sessions

# of

req

uest

s

Zend

P9

P9ZC


Our approach outperforms original P9 by 31%, and Zend by 61%


Higher is better

Throughput

Sessions

IBM Research


CPU Usage of memcpy in 3 scenarios

CPU Usage of memcpy

3.13

9.04

12.48

12.6

2.1931

7.22

15.64

12.43

1.5534

5.05

3.43

4.42

0 2 4 6 8 10 12 14 16 18

Banking

Banking (No SSL)

Ecommerce

Support

Percentage when each runtime reaches up to the peak throughput

P9FS

P9

Zend

% CPU used for memcpy

P9ZC

P9

Zend

P9ZC dramatically decreases CPU time used for memory copying

Support : 78.5 KB Ecommerce: 143.9 KB Banking : 34.8 KB


IBM Research






4. Related Work, Future Direction, and Conclusions

IBM Research


Related Work

Zero Copy Approach – Evaluation of sendfile [Nahum, TON ‘02] – Faster FastCGI in Ligthttpd 1.5 beta [‘08]

• Implemented as P9 Level 1

SSL-enabled sendfile [Keromytis, TOCS ‘06]– Could allow our approach to be used for SPECweb

Banking Scenario

In kernel Web Servers [Armol, TOCS ‘04] [King, USENIX ‘01]–Their focus is on sending only static files

IBM Research


Future Directions

Apply our approach to other programming languages such as Java, Ruby, and Python

Performance evaluation with more applications such as SugarCRM, MediaWiki, and phpBB

Extend the proposed approach to general cases where file processing is not explicitly required– Constant and long character sequences can be

dynamically and/or statically stored in a flat file – A web server sends the file to web clients via the

sendfile system call – Challenge: runtime overhead

IBM Research


Conclusions

Proposed a novel approach that improves Web applications performance by a zero-copy approach

Showed promising performance improvement over our original PHP runtime with SPECweb2005 126% performance improvement with micro-

benchmarks 31% performance improvement for standard Web

benchmark, SPECweb2005

ibm research © 2009 ibm corporation highly scalable web applications with zero-copy data transfer...

Documents

rn otherrn rn

rn savingrn rn

rn htmlrn rn specweb2005

rn change profile rn

lighttpd web server

web sites

ibm corporation

specweb2005 dynamic