������������� � ��� ������������� �������

��� �"!$#&%(')!+*-,.�/1032547698:6<;>= ?A@CBED>F�G&@&GH= IHJCKMLNGCDO?PG&IQCR9SUTWV X)Y.Q[ZA\A]^T`_ V aUb�_ cd]`b

e � fhgji$kl,min�"o/1032547698:6<;>= ?A@CBED>F�G&@&GH= IHJCKMLNGCDO?PG&ITCQUpUSWbqX)YmQdZA\A]^T`_ V aUbr_ cd]`b

�s!+')'utsvwi",(fAx7i/1032547698:6<;>= ?A@CBED>F�G&@&GH= IHJCKMLNGCDO?PG&IQCQ[\yp&QWV zCpWX)Y.Q[ZA\A]^T`_ V aUb�_ cd]`b

{ 'u!7�}|<':~�,m� f/1032���� ��=j��GCD�F�G&@&GH=EIHJ[K�LwG&D)?PG&IZ��HYmQCR9V R�XmSC\H_ V aUb�_ cd]`b

AbstractThis paperprovides an overview of techniquesfor improving Web perfor-

mance.For improving server performance,multiple Webserverscanbeusedincombinationwith efficient loadbalancingtechniques. We alsodiscusshow thechoiceof server architectureaffectsperformance. We examinecontentdistri-bution networks (CDN’s) andtherouting techniques that they use.While Webperformancecanbeimprovedusingcaching, akey problemwith cachingis con-sistency. We presentdifferenttechniquesfor achieving varying forms of cacheconsistency.

Keywords: cacheconsistency, contentdistribution networks, Web caching,Web perfor-mance,Webservers

�

�7���N�N�>�����q�N�W���

The World Wide Web hasemerged asoneof the mostsignificant applica-tions over the past decade. The infrastructure required to support Webtrafficis significant, and demands continue to increaseat a rapid rate. Highly ac-cessed Websitesmayneedto serve over a million hits perminute. Additionaldemands arecreated by theneedto serve dynamicandpersonalizeddata.

This paperpresentsan overview of techniques andcomponents needed tosupport high volumeWebtraffic. Theseincludemultiple servers at Websiteswhich canbescaled to accommodatehigh requestrates. Variousload balanc-ing techniqueshave beendevelopedto efficiently route requests to multipleservers. Web sites may also be dispersedor replicatedacross multiple geo-graphic locations.

Webserverscanuseseveral differentapproachesto handling concurrentre-quests includingprocesses,threads,event-drivenarchitecturesin which a sin-gleprocessis usedwith non-blocking I/O, andin-kernelservers.Eachof thesearchitectural choiceshascertain advantagesand disadvantages. We discusshow thesedifferentapproachesaffect performance.

Over the pastfew years, a numberof content distribution networks (CDN)have beendevelopedto aid Webperformance. A CDN is a sharednetwork ofservers or cachesthat deliver content to users on behalf of content providers.The intent of a CDN is to serve content to a client from a CDN server so thatresponsetime is decreasedover contacting the origin server directly. CDN’salso reduce the load on origin servers. This paper examinesseveral issuesrelated to CDN’sincludingtheir overall architectureandtechniquesfor routingrequests.Wealsoprovideinsight into theperformanceimprovementstypicallyachievedby CDN’s.

Cachingis a critical technique for improving performance. Caching cantake placeat several points within the network including clients, servers, andin intermediate proxies betweenthe client and server. A key problem withcaching within the Web is maintaining cacheconsistency. Web objectsmayhave expiration timesassociatedwith themindicating whenthey becomeob-solete. The problem with expiration times is that it is often not possible totell in advancewhenWebdatawill becomeobsolete.Expiration timesarenotsufficient for applications which have strong consistency requirements.Stalecached dataandtheinability in many casesto cachedynamicandpersonalizeddatalimits theeffectivenessof caching.

Theremainder of this paper is organizedasfoll ows. Section1 providesanoverview of techniques usedfor improving performanceat a Web site. Sec-tion 2 discussesdifferentserver architectures.Section3 presentsanoverviewof content distribution networks. Section4 discussesWebcaching andcacheconsistency techniques.

��� �7���l�N� �s�^�s¡£¢}¤<�N¥E���"�§¦����3¤�¦ �¨¦5©ª¤¬«­>�&�w¤

Highly accessedWebsites mayneedto handle peakrequestrates of over amillion hits perminute. Webserving lends itself well to concurrency becausetransactions from different clients canbe handled in parallel. A single Webserver canachieve parallelism by multithreading or multitasking between dif-ferentrequests.Additionalparallelismandhigherthroughputscanbeachievedby using multiple servers andloadbalancing requestsamongtheservers.

Figure1 showsanexampleof a scalableWebsite. Requestsaredistributedto multiple serversvia a load balancer. The Web serversmay access oneormoredatabasesfor creating content. TheWebserverswould typically containreplicatedcontentsothatarequestcouldbedirectedto any serverin thecluster.For storingstaticfiles,onewayto sharethemacrossmultiple serversis to useadistributedfile systemsuchasAFSor DFS[42]. Copiesof filesmaybecachedin oneor moreservers. Thisapproachworksfine if thenumberof Webserversis not too large anddatadoesn’t change very frequently. For large numbersof serversfor which dataupdatesarefrequent, distributedfile systemscanbehighly inefficient. Part of the reason for this is the strong consistency modelimposedby distributed file systems. Sharedfile systemsrequire all copiesof files to be completely consistent. In order to update a file in one server,all other copiesof the file need to be invalidated before the updatecan takeplace. Theseinvalidation messagesaddoverheadandlatency. At someWebsites,thenumber of objectsupdatedin temporal proximity to eachothercanbequite large. During periodsof peakupdates,thesystemmight fail to performadequately.

Another methodof distributing contentwhich avoids someof theproblemsof distributedfile systemsis to propagateupdatesto serverswithout requiringthe strict consistency guaranteesof distributed file systems. Using this ap-proach, updatesarepropagatedto serverswithout first invalidating all existingcopies. Thismeansthatat thetimeanupdateis made,datamaybeinconsistentbetweenserversfor a little while. For many Web sites, theseinconsistenciesarenot a problem,andtheperformancebenefitsfrom relaxing theconsistencyrequirementscanbesignificant.

���^��� ® ��¦��°¯±¦�²W¦����r�^��¡

The load balancerin Figure1 distributesrequestsamongthe servers. Onemethodof loadbalancing requeststo serversis via DNSservers.DNSserversprovide clients with the IP addressof oneof thesite’s content delivery nodes.Whena request is madeto a Web site such ashttp://www.research.ibm.com/compsci/, “www.research.ibm.com”mustbetranslatedto anIPaddress,and DNS servers perform this translation. A nameassociatedwitha Website canmapto multiple IP addresses,eachassociatedwith a different

Load Balancer

WebServer

WebServer

WebServer

Database Database

³+´¶µE·:IHG¹¸E6Architectureof a scalableWebsite. Requestsaredirectedfrom theloadbalancer

to oneof severalWebservers.TheWebserversmayaccessoneor moredatabasesfor creatingcontent.

Webserver. DNSserverscanselect oneof theseserversusingapolicy suchasround robin [10].

Thereareotherapproacheswhichcanbeusedfor DNSload balanceswhichoffer someadvantages over simple round robin [13]. The DNS server canuseinformation about thenumberof requestsperunit time sentto a Websiteaswell asgeographic information. The Internet2DistributedStorage Infras-tructureProjectproposeda DNS that implementsaddressresolution basedonnetwork proximity information,suchasround-trip delays [8].

Oneof theproblemswith load balancingusing DNSis that name-to-IPmap-pings resulting from a DNS lookup may be cachedanywherealong the pathbetweenaclient andaserver. Thiscancauseload imbalancebecauseclient re-questscanthenbypasstheDNSserverentirely andgodirectly to aserver [19].Name-to-IP addressmappings have time-to-live attributes(TTL) associatedwith themwhich indicatewhenthey areno longervalid. UsingsmallTTL val-uescanlimit loadimbalancesdueto caching. Theproblemwith this approachis thatit canincreaseresponsetimes[59]. Anotherproblemwith thisapproachis thatnot all entitiescaching name-to-IPaddressmappingsobey TTL’swhicharetoo short.

Adaptive TTL algorithms have beenproposedin which the DNS assignsdifferentTTL valuesfor different clients [12]. A requestcomingfrom a client

with ahigh requestratewould typically receiveaname-to-IP addressmappingwith a shorter lifetime thanthat assignedto a client with a low request rate.This prevents a proxy with many clients from directing requeststo the sameserver for too long a period of time.

Another approachto loadbalancing is using a connectionrouter in front ofseveralback-endservers. ConnectionroutershidetheIP addressesof theback-end servers. That way, IP addressesof individual servers won’t be cached,eliminating the problem experienced with DNS load balancing. Connectionrouting canbeusedin combination with DNSrouting for handling largenum-bersof requests. A DNS server can route requests to multiple connectionrouters. The DNS server provides coarse grained load balancing, while theconnection routersprovide finer grained load balancing. Connection routersalsosimplify the managementof a Website becauseback-endservers canbeaddedandremovedtransparently.

IBM’ sNetwork Dispatcher[32] is oneexampleof aconnectionrouterwhichhidesthe IP addressof back-endservers. Network Dispatcher usesWeightedRoundRobin for load balancing requests. Using this algorithm, servers areassignedweights.All serverswith thesameweight receive a new connectionbefore any server with a lesser weight receives a new connection. Serverswith higher weights getmoreconnections thanthose with lower weights, andserverswith equal weightsgetanequal distribution of new connections.

With Network Dispatcher, requestsfrom the back-endservers go directlybackto theclient. This reducesoverheadat theconnection router. By contrast,someconnectionroutersfunction asproxies betweenthe client andserver inwhich all responsesfrom servers go throughtheconnection router to clients.

Network Dispatcher hasspecial features for handling client affinity to se-lectedservers.Thesefeaturesareuseful for handling requestsencryptedusingthe SecureSocketsLayer protocol (SSL).Whenan SSLconnection is made,a session key mustbe negotiatedandexchanged. Sessionkeys areexpensiveto generate. Therefore, they have a lifetime, typically 100seconds,for whichthey exist aftertheinitial connection is made.SubsequentSSLrequestswithinthekey lifetime reusethekey.

Network dispatcher recognizesSSLrequestsby the port number (443). Itallows certain ports to be designatedas“sticky”. Network Dispatcherkeepsrecordsof old connectionsonsuchportsfor adesignated affinity life span(e.g.100secondsfor SSL).If a request for a new connection from thesameclientonthesameport arrivesbeforetheaffinity life spanfor thepreviousconnectionexpires, thenew connection is sentto thesameserver that theold connectionutilized.

Usingthis approach,SSLrequestsfrom thesameclient will go to thesameserver for the lifetime of a session key, obviating the needto negotiate newsession keys for each SSLrequest. This cancausesomeload imbalance,par-

ticularly sincetheclient addressseenby Network Dispatcher mayactually beaproxy representing severalclientsandnot just theclient corresponding to theSSLrequest. However, the reduction in overhead dueto reduced session keygeneration is usually worth the load imbalancecreated. This is particularlytrue for siteswhich make gratuitoususeof SSL.For example, somesites willencrypt all of the imagefiles associatedwith anHTML pageandnot just theHTML pageitself.

Connection routing is oftendone at layer 4 of the OSI model in which theconnectionrouterdoesnotknowthecontentsof therequest.Anotherapproachis to perform routing at layer 7. In layer7 routing, alsoknownascontent-basedrouting, therouter examinesrequestsandmakesits routing decisionsbasedonthe contents of requests[55]. This allows more sophisticated routing tech-niques. For example, dynamic requests could be sentto one set of servers,while static requestscould be sentto another set. Different quality of servicepoliciescould beassignedto different URL’sin whichthecontent-basedroutersendstherequestto anappropriateserverbased onthequality of servicecorre-sponding to therequestedURL. Content-based routing allows theserversat aWebsiteto beassymetrical. For example,informationcould bedistributedataWebsitesothat frequently requestedobjectsarestoredonmany or all servers,while infrequently requestedobjects are only storedon a few servers. Thisreducesthestorage overheadof replicatingall information on all servers.Thecontent-basedrouter canthenuseinformation on how objects aredistributedto make correct routing decisions.

Thekey problemwith content-basedroutingis thattheoverheadwhichis in-curred canbehigh[60]. In order to examinethecontentsof arequest,theroutermustterminatetheconnection with theclient. In a straightforwardimplemen-tation of content-basedrouting, the router actsasa proxy betweenthe clientandserver, andall dataexchangedbetweenthe client andserver go throughtherouter. Betterperformanceis achievedby usinga TCPhandoff protocol inwhich theclient connection is transferred from therouter to aback-endserver;this canbedone in a manner which is transparent to theclient.

A number of client-based techniqueshavebeen proposedfor loadbalancing.A few yearsago, Netscapeimplementeda schemefor doingloadbalancing atthe Netscape Web site (before they were purchasedby AOL) in which theNetscape browserwasconfiguredto pick theappropriate server [49]. Whenauseraccessedthe Web site www.netscape.com,the browserwould randomlypick a numberº between 1 andthenumberof serversanddirect therequest towww º .netscape.com.

Anotherclient-based technique is to usethe client’s DNS [23, 58]. Whena client wishesto accessa URL, it issues a query to its DNS to get the IPaddressof the site. The Web site’s DNS returns a list of IP addressesof theservers instead of a single IP address.The client DNS selects an appropriate

server for the client. An alternative strategy is for the client to obtainthe listof IP addressesfrom its DNS anddo theselection itself. An advantageto theclientmakingtheselection itself is thattheclientcancollect informationabouttheperformanceof differentserversat thesiteandmake an intelligentchoicebasedon this. The disadvantagesof client-based techniques is that the Website losescontrol over how requestsarerouted, andsuchtechniques generallyrequire modificationsto theclient (or at leasttheclient’s DNSserver).

���&»�� ­¼¤¬�N�s�^�s¡�½�¾��s¦����U�¿©À¤<«ÂÁÃ�����w¤<���

Web servers satisfy two types of requests,static and dynamic. Static re-quests arefor files that exist at the time a requestis made.Dynamicrequestsarefor content thathasto begeneratedby aserverprogramexecuted at requesttime. A key differencebetweensatisfying static anddynamicrequestsis theprocessingoverhead. The overhead of serving static pagesis relatively low.A Web server running on a uniprocessor cantypically serve several hundredstatic requestsper second. Of course, this number is dependenton the databeingserved;for large files, thethroughputis lower.

Theoverhead for satisfying a dynamic request maybeordersof magnitudemorethantheoverheadfor satisfying a staticrequest. Dynamicrequestsofteninvolveextensiveback-endprocessing.Many Websitesmakeuseof databases,anda dynamicrequestmay invoke several databaseaccesses.Thesedatabaseaccessescanconsumesignificant CPUcycles. Theback-end softwarefor cre-ating dynamic pages may be complex. While the functionality performedbysuchsoftwaremaynot appear to becompute-intensive, such middlewaresys-temsare often not designed efficiently; commercial products for generatingdynamicdatacanbehighly inefficient.

Onesourceof overheadin accessingdatabasesis connectingto thedatabase.Many databasesystems require a client to first establish a connectionwith adatabasebeforeperformingatransaction in which theclient typically providesauthentication information. Establishingaconnection is oftenquiteexpensive.A naive implementation of a Web site would establish a new connection foreachdatabaseaccess. This approachcould overload the databasewith rela-tively low traffic levels.

A significantly more efficient approach is to maintain one or more long-running processeswith open connections to the database. Accesses to thedatabasearethenmadewith oneof these long-running processes. That way,multiple accessesto thedatabasecanbemadeover a single connection.

Another source of overhead is the interfacefor invoking server programsin order to generate dynamic data.Thetraditional method for invoking serverprogramsfor Webrequestsis via theCommonGateway Interface(CGI). CGIforks off a new processto handle eachdynamicrequest;this incurs significant

overhead.Thereareanumber of fasterinterfacesavailable for invoking serverprograms[34]. Thesefasterinterfacesuseoneof two approaches. The firstapproachis for theWebserver to provide an interfaceto allow a programforgeneratingdynamicdatato beinvokedaspartof theWebserver processitself.IBM’ s GO Webserver API (GWAPI) is anexampleof suchaninterface.Thesecond approachis to establish long-running processesto which a Webserverpasses requests.While this approachincurssomeinterprocesscommunicationoverhead,theoverheadis considerably lessthanthatincurredby CGI.FastCGIis anexampleof thesecondapproach[53].

In order to reducetheoverheadfor generatingdynamic data,it is oftenfea-sibleto generatedatacorresponding to adynamicobject once, storetheobjectin a cache,andsubsequently serve requests to the object from cache insteadof invoking theserver program again[33]. Usingthis approach,dynamicdatacanbeservedat about thesamerateasstaticdata.

However, therearetypesof dynamicdata that cannot beprecomputed andservedfrom a cache. For instance,dynamic requeststhatcausea sideeffect attheserver suchasa databaseupdate cannot besatisfied merelyby returning acached page.As anexample,consideraWebsitethatallowsclientsto purchaseitems using credit cards. At the point at which a client commits to buyingsomething, that information hasto be recorded at the Web site; the requestcannot besolelyservicedfrom a cache.

Personalized Web pagescanalsonegatively affect the cacheability of dy-namicpages. A personalizedWeb pagecontainscontent specificto a client,suchastheclient’sname.SuchaWebpagecould notbeusedfor anotherclient.Therefore, caching the pageis of limited utili ty sinceonly a singleclient canuseit. Eachclient would needa differentversion of thepage.

Onemethod which canreducethe overhead for generating dynamic pagesandenablecaching of somepartsof personalizedpagesis to definethesepagesasbeing composedof multiple fragments [15]. In this approach,a complexWeb pageis constructed from several simpler fragments. A fragment mayrecursively embedother fragments. This is efficient becausetheoverheadforassembling a Webpagefrom simpler fragments is usually minor comparedtotheoverhead for constructing thepagefrom scratch,which canbequitehigh.

Thefragment-basedapproachalsomakesit easierto designWebsites. Com-moninformationthatneedsto beincludedon multiple Webpages canbecre-atedasa fragment. In order to change the informationon all pages, only thefragment needsto bechanged.

In order to usefragmentsto allow partial caching of personalized pages,the personalized informationon a Web pageis encapsulated by oneor morefragments that are not cacheable, but the other fragments in the page are.When serving a request, a cache composes pagesfrom its constituent frag-ments,many of which arelocally available. Only personalized fragmentshave

to be created by the server. As personalized fragments typically constitute asmall fraction of the entire page, generating only them would require loweroverhead thangeneratingall of thefragments in thepage.

Generating Webpagesfrom fragmentsprovidesother benefits aswell. Frag-mentscanbeconstructedto represententitiesthathavesimilar lifetimes.Whena particular fragment changesbut the rest of the Web pagestaysthe same,only the fragmentneedsto be invalidatedor updatedin thecache, not theen-tire page. Fragmentscanalsoreduce the amount of cachespacetaken up bymultiple pageswith commoncontent. Suppose that a particular fragment iscontainedin 2000popular Webpageswhich should becached.Usingthecon-ventionalapproach,thecachewouldcontainaseparateversionof thefragmentfor eachpageresulting in asmany as2000copies.By contrast,if thefragment-basedmethodof pagecomposition is used,only a single copy of thefragmentneedsto bemaintained.

A key problem with caching dynamic content is maintaining consistentcaches. It is advantageous for the cacheto provide a mechanism, suchasan API, allowing the server to explicitly invalidateor updatecached objectsso that they don’t become obsolete. Web objects may be assignedexpirationtimesthat indicate whenthey should beconsideredobsolete. Suchexpirationtimesaregenerally notsufficient for allowing dynamicdatato becachedprop-erly becauseit is oftennot possible to predict accurately whenadynamic pagewill change.

»�� ­�¤<�N��¤¬�¨¢Ä¤¬�N¥j�¹�"�§¦����3¤Å�nÆwÆN��¤<Æ

A central component of the responsetime seenby Webusersis, of course,the performanceof the origin server that providesthe content. Thereis greatinterest,then, understandingtheperformanceof Webservers: How quickly canthey respondto requests?How well do they scalewith load?Are they capableof operatingunderoverload, i.e.,canthey maintain somelevel of service evenwhentherequestedloadfar outstrips thecapacity of theserver?

A Web server is an unusualpieceof software in that it mustcommunicatewith potentially thousands of remoteclients simultaneously. The server thusmustbeableto dealwith a large degree of concurrency. A server cannot sim-ply respond to eachclient in a non-preemptive, first-comefirst-serve manner,for several reasons. Clients aretypically locatedfar away over the wide-areaInternet,andthusconnection lifetimescanlastmany secondsor evenminutes.Particularly with HTTP1.1,aclient connection maybeopenbut idle for sometime beforea new request is submitted. Thusa server canhave many concur-rent connections open, andshould be abledo work for oneconnectionwhenanother is quiescent.Anotherreason is thataclient mayrequestafile which isnot residentin memory. While theserverCPUwaitsfor thedisk to retrieve the

file, it canwork on responding to anotherclient. For these andother reasons,a server mustbeableto multiplex thework it hasto do throughsomeform ofconcurrency.

A fundamental factor which affects theperformanceof a Webserver is thearchitectural modelthatit usesto implement that concurrency. Generally, Webserverscanbeimplementedusingoneof four architectures: processes, threads,event-driven, andin-kernel. Eachapproachhasits advantagesanddisadvan-tageswhich we go into moredetail below. A central issuein this decision ofwhich model to useis what sort of performanceoptimizations areavailableunder that model. Another is how well that modelscales with the workload,i.e.,how efficiently it canhandle growing numbers of clients.

»��^��� ¢Ç�N�>�3¤¬ÆwÆ-Èɯ±¦�Æ7¤¬�­¼¤<�N�¹¤<�+ÆProcesses are perhapsthe most commonform of providing concurrency.

The original NCSA server andthe widely-known Apacheserver [2] usepro-cesses as the mechanism to handle large numbers of connections. In thismodel,a processis created for eachnew request,which canblock whennec-essary, for examplewaiting for datato becomeavailable on asocket or for fileI/O to beavailable from thedisk. Theserver handlesconcurrency by creatingmultiple processes.

Processes have two mainadvantages. First, they areconsistent with a pro-grammers’way of thinking, allowing the developer to proceedin a step-by-stepfashion withoutworryingabout managingconcurrency. Second, they pro-vide isolation andprotection betweendifferentclients. If oneprocesshangsorcrashes,theotherprocessesshould beunaffected.

The main drawback to processesis performance. Processes are relativelyheavyweight abstractions in mostoperating systems,andthuscreating them,deleting them,andswitching context betweenthemis expensive. Apache,forexample, tries to amelioratethesecosts by pre-forking a number of processesandonly destroys themif the load falls below a certain threshold. However,thecosts arestill significant,aseachprocess requiresmemoryto beallocatedto them.As thenumberof processesgrow, large amounts of memoryareusedwhich putspressureon thevirtual memorysystem,which couldusethemem-ory for other purposes,suchascaching frequently-accesseddata. In addition,sharing information, suchasa cached file, acrossprocessescanbedifficult.

»��&»�� ÊÌË �N¤�¦���Èɯ±¦�Æ7¤¬��­�¤<�N��¤¬�+ÆThreadsarethe next mostcommonform of concurrency. Serversthat use

threadsincludeJAWS [31] andSun’s Java WebServer [64]. Threadsaresim-ilar to processesbut areconsideredlighter-weight. Unlike processes,threadssharethe sameaddressspace andtypically only provide a separatestackfor

eachthread. Thus,creation costsandcontext-switching costsareusually muchlower thanfor processes.In addition, sharing between threads is mucheasier.Threadsalso maintain the abstraction of an isolated environment much likeprocesses,although the analogy is not exact since programmersmust worrymoreabout issueslikesynchronizationandlocking to protect shareddatastruc-tures.

Threads have several disadvantages as well. Since the addressspaceisshared, threadsarenotprotectedfrom oneanotherthewayprocessesare.Thus,a poorly programmedthreadcancrashthewholeserver. Threadsalsorequireproper operating systemsupport,otherwisewhenathreadblocksonsomethinglike a file I/O, thewholeaddressspacewill bestopped.

»��[ͼ� Î ��¤<���nÈɽÏ�"�W��¤<�Э¼¤¬�N��¤¬�"ÆThe third form of concurrency is known as the event-driven architecture.

Serversthat usethis methodincludeFlash[56] andZeus[72]. With this ar-chitecture, a single processis usedwith non-blocking I/O. Non-blocking I/Ois a way of doingasynchronousreadsandwriteson a socket or file descriptor.For example, instead of a processreading a file descriptor andblocking untildatais available,an event-drivenserver will returnimmediately if thereis nodata.In turn, theO.S.will let theserverprocessknowwhenasocketor file de-scriptor is readyfor reading or writing throughanotification mechanism. Thisnotification mechanismcanbeanactiveonesuchasasignal handler, or a pas-sive onerequiring theprocessto asktheO.S.suchastheselect() systemcall. Throughthesemechanismstheserver processwill essentially respond toeventsandis typically guaranteed to never block.

Event-drivenservershaveseveraladvantages.First, they arevery fast.Zeusis frequently usedby hardware vendors to generate high Web server num-berswith the SPECWeb99benchmark [61]. Sharing is inherent, sincethereis only oneprocess,andno locking or synchronization is needed. Thereareno context-switch costs or extra memoryconsumption that arethe casewiththreads or processes.Maximizing concurrency is thusmucheasier thanwiththepreviousapproaches.

Event-driven servers have downsides as well. Like threads,a failure canhalt thewholeserver. Event-drivenserverscantax operating systemresourcelimits, suchas the numberof openfile descriptors. Different operating sys-temshave varying levels of support for asynchronous I/O, so a full y event-driven server may not be possible on a particular platform. Finally, event-driven servers require a different way of thinking from the programmer, whomustunderstandandaccount for the waysin which multiple requestscanbein varying stagesof progresssimultaneously. In this approach,the degree of

concurrency is fully exposedto thedeveloper, with all theattendantadvantagesanddisadvantages.

»��HÑ�� �7�>ÈÉÒÓ¤¬�"�s¤<²$­¼¤¬�N��¤¬�+Æ

Thefourth andfinal form of server architecturesis the in-kernel approach.Serversthatusethismethod includeAFPA [36] andTux [66]. All of theprevi-ousarchitecturesplacetheWebserver softwarein userspace; in this approachthe HTTP server is in kernel space, tightly integratedwith the host TCP/IPstack.

Thein-kernelarchitecture hastheadvantagesthat it is extremely fast,sincepotentially expensive transitions to userspace arecompletely avoided. Simi-larly, nodataneedsto becopied acrosstheuser-kernel boundary, anothercostlyoperation.

Thedisadvantagesfor in-kernel approachesareseveral. First, it is lessro-bust to programmingerrors; a server fault cancrashthe whole machine, notjust the server! Development is much harder, since kernel programming ismoredifficult andmuch lessportable thanprogramminguser-space applica-tions. Kernel internalsof Linux, FreeBSD,andWindows vary considerably,makingdeploymentacrossplatformsmorework. Thesocket andthread APIs,on theother hand, arerelatively stable andportableacross operating systems.

Dynamiccontentposesanevengreater challengefor in-kernelservers, sinceanarbitraryprogrammaybeinvokedin responseto arequestfor dynamiccon-tent.A full -featuredin-kernelwebserver would needto have aPHPengine orJava runtimeinterpreterloaded in with the kernel! Theway current in-kernelserversdealwith this issueis to restrict their activitiesto thestaticcontent com-ponent of Webserving,andpassdynamic contentrequeststo acompleteserverin userspace,such asApache. For example,many entries in theSPECWeb99site [61] that usethe Linux operating system usethis hybrid approach,withTux serving staticcontent in thekernel andApachehandling dynamic requestsin user space.

»��&Ô�� ­¼¤¬�N��¤¬��¢}¤¬�+¥E���"�§¦��s�3¤£ÁÃ������¦��"�UÆ7���

Sincewe areconcernedwith performance,it is thusinteresting to seehowwell the different server architectures perform. To evaluate them, we tooka experimental testbed setup andevaluate the performanceusing a syntheticworkloadgenerator[51] to saturatetheserverswith requestsfor arangeof webdocuments.Theclients wereeight 500 MHz PC’s running FreeBSD,andtheserver wasa 400MHz PCrunning Linux 2.4.16.Eachclient hada 100mbpsEthernet connectedto a gigabit switch, and the server wasconnectedto theswitchusing GigabitEthernet. Threeserverswereevaluatedasrepresentatives

0

500

1000

1500

2000

2500

3000

Ser

ver

Thr

ough

put i

n H

TT

P o

ps/s

ec

Tux 2.0FlashApache 1.3.20

³"´¶µ ·:IdG Õ:6Server Throughput

of their architecture: Apacheas a process-basedserver, Flashas an event-drivenserver, andTux asanin-kernel server.

Figure2 showsthe server throughput in HTTP operations/secof the threeservers. As canbeseen, Tux, thein-kernelserver, is thefastestat2193ops/sec.However, Flashis only 10percent slowerat2075ops/sec,despite beingimple-mentedin userspace.Apache,on theotherhand,is significantly slower at875ops/sec. Figure3 shows theserver responsetime for thethree servers.Again,Tux is thefastest,at3 msec,Flashsecondat5 msec,andApacheslowestat10msec.

Sincemultiple examplesof eachtype of server architecture exist, thereisclearly no consensus for what is the best model. Instead,it may be that dif-ferentapproaches arebetter suitedfor different scenarios. For example, thein-kernel approachmay be mostappropriate for dedicatedserver appliances,or asCDN nodes,whereasa back-enddynamic contentserver will rely on thefull generality of a process-basedserver like Apache. Still, web site opera-tors should beawareof how thechoice of architecture will affect Webserverperformance.

ͼ� Á̽ÏÖÓÆq×Ø�7���l�N� �¹¤<��©ª¤¬«°¢Ä¤¬�N¥j�¹�Ù�£¦����3¤Ó� Ë �N����¡ ˽¿�UÆ-�N�"�U«����N�W���

End-to-end Web performanceis influenced by numerous factors suchasclient and server network connectivity, network loss and delay, server load,HTTP protocol version, andnameresolution delays. The content-serving ar-chitecture hasa significant impact on someof these factors, as well factors

0

2

4

6

8

10

12

Ser

ver

Res

pons

e T

ime

in m

sec

Tux 2.0FlashApache 1.3.20

³+´¶µE·:IHG�Új6ServerResponseTime

not related to performancesuch ascost, reliability, andease of management.In a traditional content-serving architecture all clients requestcontent from asingle location, asshown in Figure4. In this architecture, scalability andper-formance areimproved by adding servers,without the ability to addresspoorperformancedueto problemsin thenetwork. Moreover, this approachcanbeexpensive sincethesitemustbeoverprovisionedto handle unexpectedsurgesin demand.

Oneway to addresspoor performancedueto network congestion, or flashcrowdsat servers, is to distributecontent to servers or caches locatedclosertotheedges of thenetwork, asshown in Figure5. Sucha distributednetwork ofservers comprisesa content distribution network (CDN). A CDN is simply anetwork of servers or cachesthatdeliver content to users on behalf of contentproviders. The intent of a CDN is to serve content to a client from a CDNserver suchthat the response-time performanceis improved over contactingtheorigin serverdirectly. CDN serversaretypically shared, delivering contentbelongingto multipleWebsitesthough all serversmaynotbeusedfor all sites.

CDNs have several advantages over traditional centralized content-servingarchitectures,including [67]:

improving client-perceived responsetime by bringing content closer tothenetwork edge,andthuscloser to end-users

off-loadingwork from origin serversby serving larger objects,suchasimagesandmultimedia, from multiple CDN servers

origin servers

client

³"´ µE·:IdG�Û)6Traditionalcentralizedcontent-serving architecture

reducing content provider costsby reducing the needto invest in morepowerful serversor morebandwidth asuser populationincreases

improvingsiteavailabili ty by replicatingcontent in many distributed lo-cations

CDN serversmay be configured in tree-like hierarchies[71] or clusters ofcooperatingproxies thatemploy content-basedrouting to exchangedata[28].CommercialCDNs alsovary significantly in their sizeandservice offerings.CDN deploymentsrangefrom afew tensof servers(or serverclusters), to overten thousandserversplaced in hundredsof ISP networks. A large footprintallows a CDN service provider (CDSP) to reachthe majority of clients withvery low latency andpathlength.

Content providersuseCDNsprimarily for serving staticcontentlike imagesor large stored multimedia objects(e.g., movie trailers and audio clips). Arecent study of CDN-servedcontent found that96%of theobjectsservedwereimages[41]. However, the remaining few objectsaccountedfor 40–60% ofthebytesserved,indicating asmallnumberof very largeobjects. Increasingly,CDSPs offer services to deliver streaming mediaand dynamic datasuchaslocalizedcontent or targetedadvertising.

CDN server

origin server

client

³+´¶µE·:IHG�ÜE6DistributedCDN architecture

ͼ�^��� Á̽ÏÖ Ý��"� Ë �&�7¤<�q�N���N¦�² Î ²W¤<�§¤ ���wÆ

As illustratedin Figure 6, CDNs have threekey architectural elements inaddition to the CDN servers themselves: a distribution system, an account-ing/billing system, and a request-routing system [18]. The distribution sys-tem is responsible for moving content from origin servers into CDN serversandensuring dataconsistency. Section4.4 describessometechniquesusedtomaintain consistency in CDNs.Theaccounting/billing systemcollects logsofclient accessesandkeepstracksCDN server usagefor useprimarily in admin-istrative tasks. Finally, the request-routing systemis responsible for directingclient requeststo appropriateCDN servers. It mayalsointeract with the dis-tribution system to keepanup-to-dateview of which content resideson whichCDN servers.

The request-routing system operatesasshown in Figure7. Clients accesscontent from the CDN servers by first contacting a request router (step 1).The requestrouter makesa server selection decision andreturns a server as-signment to the client (step2). Finally, the client retrievescontent from thespecified CDN server (step3).

CDN server

origin server

client

distributionsystem

requestrouter accounting

and billing

measure/track

measure/track

³"´¶µE·:IHG�ÞE6CDN architecturalelements

ͼ�&»�� Á̽ÏÖ ß�¤¬às��¤<Æn�-ÈÉß������+�^�s¡

Clearly, the request-routing systemhasa direct impacton theperformanceof the CDN. A poor server selection decision can defeatthe purposeof theCDN,namely to improveclient responsetimeoveraccessingtheorigin server.Thus,CDNstypically rely onacombinationof static anddynamicinformationwhenchoosing thebestserver. Severalcriteria areusedin therequest-routingdecision, includingthecontent being requested,CDN server andnetwork con-ditions,andclient proximity to thecandidateservers.

The mostobviousrequestrouting strategy is to direct the client to a CDNserver that hoststhe content being requested. This is complicated, however,if the request router doesnot know the content beingrequested, for exampleif request-routing is done in the context of nameresolution. In this casetherequest containsonly a server name(e.g.,www.service.com) asopposedto thefull HTTPURL.

For goodperformancetheclient should bedirectedto a relatively unloadedCDN server. This requires that the request router actively monitor the stateof CDN servers. If eachCDN location consists of a cluster of servers andlocal load-balancer, it maybepossible to query a server-sideagent for serverload information, asshownin Figure8. After theclient makesits request,the

origin server

client

CDN server

1

request router

2

3

³+´¶µE·:IHG�á`6CDN request-routing

request routerconsults an agent at eachCDN site load-balancer(step 2), andreturnsanappropriateanswer backto theclient.

As Web responsetime is heavily influenced by network conditions, it isimportant to choose a CDN server to which the client hasgood connectiv-ity. Uponreceivinga client request,therequest router canaskcandidateCDNservers to measure network latency to the client usingICMP echo (i.e., ping)andreport themeasuredvalues.Therequest router thenrespondsto theclientrequest with the CDN server reporting the lowest delay. Since thesemea-surements aredone on-line, this technique hasthe advantageof adapting therequest-routing decision to the mostcurrent network network. On the otherhand, it introducesadditional latency for theclient asthe requestrouterwaitsfor responsesfrom theCDN servers.

A commonstrategy usedin CDNrequest-routing is to chooseaserver“near”the client, where proximity is definedin terms of network topology, geo-graphic distance,or network latency. Examplesof proximity metricsincludeautonomoussystem (AS) hops or network hops. Thesemetricsarerelativelystaticcompared with server load or network performance,andarealsoeasierto measure.

client

CDN server cluster

1

request router2

3

load balancer

4

³"´¶µE·:IHG â�6InteractionbetweenrequestrouterandCDN servers

Note that it is unlikely that any oneof these metricswill be suitable in allcases. Most request routers usea combination of proximity andnetwork orserver load to make server selection decisions. For example,client proximitymetricscanbeused to assign aclientto a“default” CDNserver, whichprovidesgoodperformancemostof thetime. Theselection canbetemporarily changedif loadmonitoring indicatesthatthedefault server is overloaded.

Request-routing techniquesfall into three main categories: transport-layermechanisms, application-layer redirection, and DNS-basedapproaches[6].Transport-layer requestrouters useinformation in the transport-layer headersto determinewhich CDN server should serve the client. For example, the re-questrouter canexaminetheclient IP addressandport number in a TCPSYNpacket andforward thepacket to anappropriate CDN server. Thetarget CDNserverestablishestheTCPconnection andproceedsto serve therequestedcon-tent.Forwardtraffic (including TCPacknowledgements)from theclient to thetarget server continuesto be sent to the request router and forwarded to theCDN server. Thebulk of traffic (i.e., the requestedcontent) will travel on thedirectpath from theCDN server to theclient.

Application-layerrequest-routing hasaccessto muchmoreinformationaboutthe content being requested. For example,the request-router canuseHTTP

headerslike the URL, HTTP cookies, andLanguage. A simple implementa-tion of anapplication-layer requestrouter is a Web server that receivesclientrequestsandreturns an HTTP redirect (e.g.,return code302) to theclient in-dicating theappropriate CDN server. Theflexibilit y affordedby this approachcomesat theexpenseof addedlatency andoverhead, however, sinceit requiresTCPconnection establishment andHTTP header parsing.

With request-routingbasedontheDomainNameSystem(DNS),clientsaredirected to the nearestCDN server during the nameresolution phaseof Webaccess. Typically, the authoritative DNS server for the domainor subdomainis controlled by the CDSP. In this scheme,a specializedDNS server receivesnameresolution requests,determinesthe location of theclient andreturnstheaddressof a nearby CDN server or a referral to another nameserver. The an-swermayonly becached at theclient-sidefor a shorttime sothat therequestrouter canadapt quickly to changesin network or server load. This is achievedby setting theassociated time-to-live (TTL) field in theanswer to a very smallvalue(e.g.,20 seconds).

DNS-basedrequestrouting maybeimplementedwith either full - or partial-site content delivery [41]. In full-site delivery, the content provider dele-gatesauthority for its domainto the CDSPor modifiesits own DNS serversto return a referral (CNAME record) to the CDSPs DNS servers. In thisway, all requestsfor www.company.com, for example, are resolved to aCDN server which then delivers all of the content. With partial-site deliv-ery, the content provider modifiesits content so that links to specificobjectshave hostnamesin a domain for which the CDSPis authoritative. For ex-ample,links tohttp://www.company.com/image.gif arechangedtohttp://cdsp.net/company.com/image.gif. In thisway, theclientretrievesthe baseHTML page from the origin server but retrievesembeddedimagesfrom CDN serversto improve performance.

The appeal of DNS-basedserver selection lies in both its simplicity – itrequiresno changeto existing protocols, andits generality – it works acrossany IP-basedapplication regardlessof thetransport-layer protocol being used.Thishasled to adoption of DNS-basedrequestroutingasthedefacto standardmethodby many CDSPs andequipmentvendors. Using theDNS for request-routingdoeshavesomefundamental drawbacks,however, someof whichhavebeenrecently studiedandevaluated[59, 45,6].

ͼ�[ͼ� Á̽ÏÖ ¢}¤¬�+¥E���"�§¦��s�3¤£­��N�����W¤¬Æ

Several research studieshave recently tried to quantify theextent to whichCDNsareableto improveresponse-timeperformance.An early studyby John-sonetal. focusedonthequality of therequest-routing decision[35]. Thestudycomparedtwo CDSPsthatuseDNS-basedrequest-routing. Themethodology

wasto measure the responsetime to download a single object from the CDNserverassignedby therequest router andthetimeto downloadit from all otherCDN servers that could be identified. The findings suggestedthat the serverselection did not alwayschoosethe bestCDN server, but it waseffective inavoiding poorly performing servers, andcertainly betterthanchoosinga CDNserver randomly. Thescopeof thestudy waslimited, however, sinceonly threeclient locationswereconsidered, performancewascomparedfor downloadingonly onesmall object, andtherewasno comparison with downloading fromtheorigin server.

A study donein the context of developing the request mirroring MedusaWeb proxy, evaluated the performanceof oneCDN (Akamai) by download-ing the sameobjects from CDN servers and origin servers [37]. The studywasdoneonly for a single-user workload,but showedsignificantperformanceimprovement for thoseobjectsthat wereservedby theCDN, whencomparedwith theorigin server.

More recently, Krishnamurthyet al. studied the performanceof a numberof commercial CDNsfrom thevantage point of approximately20 clients [41].The authors conclude that CDN servers generally offer much better perfor-mancethan origin servers, though the gainsweredependent on the level ofcaching and the HTTP protocol options. Therewerealso significant differ-encesin downloadtimesfrom differentCDNs. Thestudyfindsthat,for someCDNs,DNS-basedrequestrouting significantly hampers performanceduetomultiple namelookups.

Ñ�� Áæ�� Ë ¤5ÁÃ���sÆN�UÆn�w¤<���q¾Caching hasproven to beaneffective andpractical solution for improving

thescalability andperformanceof Webservers. StaticWebpagecaching hasbeenapplied both at browsers at the client, or at intermediaries that includeisolatedproxy cachesor multiple cachesor serverswithin aCDN network. Aswith caching in any system, maintaining cacheconsistency is oneof themainissues that a Webcaching architecture needs to address.As moreof the dataon theWebis dynamicallyassembled,personalized,andconstantly changing,thechallengesof efficientconsistency managementbecomemorepronounced.To preventstaleinformation from beingtransmittedto clients,anintermediarycachemustensure thatthelocally cached datais consistentwith thatstoredonservers. Theexactcacheconsistency mechanismandthedegreeof consistencyemployedby anintermediarydependson thenatureof thecached data;not alltypesof data needthe samelevel of consistency guarantees. Consider thefollowing example.

Example 1 Online auctions: Consider a Web server that offers online auc-tions over the Internet. For each item being sold, the server maintains in-

formation such as its latest bid price (which changes every few minutes)aswell as other informationsuch as photographs and reviews for the item (allof which change lessfrequently). Consider an intermediary that cachesthisinformation.Clearly, thebid price returnedby the intermediary cacheshouldalwaysbeconsistentwith that at theserver. In contrast,reviewsof itemsneednot alwaysbeup-to-date, sincea usermaybewilling to receiveslightly staleinformation.

The above exampleshows that an intermediary cache will needto providedifferentdegreesof consistency for differenttypes of data.Thedegree of con-sistency selectedalsodeterminesthemechanismsused to maintain it, andtheoverheadsincurredby boththeserver andtheintermediary.

Ñ��^��� ½�¤�¡��N¤�¤<Æ���¥ãÁÃ����ÆN�UÆn�7¤ ���q¾In general thedegreesof consistency thatanintermediary cachecansupport

fall into thefollowing four categories.

strong consistency: A cache consistency level that always returns theresults of thelatest(committed) write at theserver is saidto bestronglyconsistent. Due to the unbounded message delays in the Internet, nocacheconsistency mechanismcanbestrongly consistentin this idealizedsense. Strongconsistency is typically implementedusinga two-phasemessage exchangealongwith timeoutsto handle unbounded delays.

delta consistency: A consistency level that returnsdatathatis never out-dated by morethan ä time units, where ä is a configurable parameter,with thelastcommittedwrite at theserver is saidto bedeltaconsistent.In practice the value of deltashould be larger than å which is the net-work delay betweentheserver andthe intermediaryat that instant, i.e.,å�æçä�è�é .

weakconsistency: For this level of consistency, a readat the intermedi-ary doesnotnecessarily reflectthelastcommittedwrite at theserver butsomecorrect previousvalue.

mutualconsistency: A consistency guaranteein whichagroupof objectsare mutually consistentwith respect to eachother. In this casesomeobjects in the group cannot be more current than the others. Mutualconsistency canco-exist with theother levelsof consistency.

Strongconsistency is useful for mirror sitesthatneed to reflectthe currentstateat theserver. Someapplications basedon financial transactionsmayalsorequirestrong consistency. Certaintypesof applicationscantoleratestaledataaslongasit is within someknown timebound. For suchapplicationsdeltacon-sistency is recommended. Delta consistency assumesthat thereis a bounded

Overheads Polling Periodicpolling Invalidates Leases TTL

File Transfer W’ êãëOìMí W’ W’ W’ControlMsgs. 2R-W’ 2R/t - îPêïë)ì�íjð 2W’ 2W’ W’Staleness 0 t 0 0 0Write delay 0 0 notify(all) min(t, notify( ñ)ò�ò�ó )) 0Server State None None All ô<ò�ò�ó None

4)=�õ&�¶Gs¸E6Overheads of DifferentConsistency Mechanisms. Key: ö is the periodin periodic

polling or theleasedurationin theleasesapproach. W’ is thenumberof non-consecutivewrites.All consecutive writeswith no interleaving readsis countedasa singlewrite. R is thenumberof reads. í is the numberof writes that were not notified to the intermediaryasonly weakconsistency wasprovided.

communicationdelay betweentheserver andthe intermediary cache. Mutualconsistency is useful whenacertainsetof objectsat theintermediary(e.g.,thefragments within a sports score page, or within a financial page) need to beconsistentwith respect to eachother. To maintain mutualconsistency theob-jectsneedto beatomically invalidated suchthat they all either reflectthenewversion or maintain theearlier staleversion.

Most intermediariesdeployedin theInternet todayprovideonly weakcon-sistency guarantees[29,62]. Until recently, mostobjectsstored onWebserverswererelatively staticandchangedinfrequently. Moreover, this data wasac-cessed primarily by humans using browsers. Sincehumanscan tolerate re-ceiving staledata (andmanually correct it usingbrowserreloads),weakcacheconsistency mechanismswereadequatefor thispurpose.In contrast,many ob-jectsstored on Web serverstoday change frequently andsomeobjects (suchasnews storiesor stockquotes) areupdatedevery few minutes [7]. Moreover,theWebis rapidly evolving from a predominantly read-only informationsys-tem to a system wherecollaborative applications andprogram-driven agentsfrequently readas well as write data. Suchapplications are lesstolerant ofstaledatathan humans accessing information using browsers. Thesetrendsargue for augmentingtheweakconsistency mechanismsemployed by today’sproxieswith those thatprovidestrongconsistency guaranteesin order to makecaching moreeffective. In theabsenceof suchstrong consistency guarantees,servers resort to markingdataasuncacheable, andthereby reduce the effec-tiveness of proxy caching.

Ñ��&»�� ÁÃ����ÆN�UÆn�7¤ �s�q¾�÷ø¤<� Ë ¦��l�UÆw��Æ

Themechanismsusedby an intermediaryandtheserver to provide thede-greesof consistency describedearlier fall into 3 categories: i) client-driven, ii)server-driven, andiii) explicit mechanisms.

Server-drivenmechanisms,referred to asserver-basedinvalidation, canbeusedto provide strongor delta consistency guarantees [69]. Server-basedin-validation, requiresthe server to notify proxies whenthe datachanges. Thisapproachsubstantially reducesthenumber of control messagesexchangedbe-tweenthe server andthe intermediary(sincemessagesaresentonly whenanobject is modified). However, it requires the server to maintain per-objectstateconsisting of a list of all proxies that cachethe object; the amountofstatemaintainedcanbe significant especially at popular Web servers. More-over, whenan intermediary is unreachable dueto network failures,the servermusteither delaywrite requestsuntil it receivesall the acknowledgments ora timeoutoccurs,or risk violating consistency guarantees. Severalnew proto-cols have beenproposedrecently to provide delta andstrongconsistency us-ing server-basedinvalidations.Webcacheinvalidationprotocol (WCIP) is onesuchproposalfor propagating server invalidationsusingapplication-level mul-ticast while providingdeltaconsistency [43]. Webcontentdistribution protocol(WCDP)is anotherproposalthatsupports multiple consistency levelsusingarequest-responseprotocol thatcanbescaled to support distribution hierarchies[65].

The client-driven approach,alsoreferred to asclient polling, requiresthatintermediariespoll theserveroneveryreadto determineif thedatahaschanged[69]. Frequent polling imposesalargemessageoverheadandalsoincreasestheresponsetime (sincethe intermediary mustawait the result of its poll beforeresponding to a readrequest). The advantage, though, is that it does not re-quire any state to be maintainedat the server, nor doesthe server ever needto delay write requests(since the onus of maintaining consistency is on theintermediary).

Most existing proxies provide only weakconsistency by (i) explicitly pro-viding a server specified lifetime of an object (referred to as the time-to-live(TTL) value), or (ii) by periodic polling of the the server to verify that thecached datais not stale [14, 29, 62]. The TTL value is sentas part of theHTTP responsein anExpires tagor using theCache-Control headers.However, a priori knowledge of whenan objectwill be modified is difficultin practice andthe degreeof consistency is dependent on the clock skew be-tweenthe server andthe intermediaries. With periodic polling the length oftheperiod determinestheextent of theobject staleness.In either case,modi-fications to theobjectbefore its TTL expires or betweentwo successive pollscauses theintermediaryto return staledata.Thusbothmechanismsareheuris-tics andprovide only weakconsistency guarantees. Hybrid approacheswherethe server specifies a time-to-live value for eachobject andthe intermediarypolls theserver only whentheTTL expires alsosuffer from thesedrawbacks.

Server-basedinvalidation andclient polling form two endsof a spectrum.Whereas theformer minimizesthenumberof control messagesexchangedbut

DEC Boston Univ. BerkeleyTraceù

0

50000

100000

150000

200000

250000

Stat

e sp

ace

over

head

ú

SI û

SI

SI û

CP ü CP ü CP ü DEC ý Boston Univ.þ Berkeleyÿ

Trace�

0

500000

1000000

1500000

Num

ber

of m

essa

ges

�

SI �

SI

SI �

CP �

CP �

CP

(a) StateSpaceoverhead (b) ControlMessages

³+´¶µE·:IHG��j6Efficacy of server-basedinvalidationandclient polling for threedifferent trace

workloads(DEC,Berkeley, BostonUniversity).Thefigureshows thatserver-basedinvalidationhasthelargeststatespaceoverhead;client polling hasthehighestcontrolmessageoverhead

mayrequire a significant amountof stateto be maintained, the latter is state-lessbut canimposea large control messageoverhead. Figure9 quantitativelycompares these two approaches with respect to (i) the server overhead, (ii)the network overhead, and (iii) the client responsetime. Due to their largeoverheads,neitherapproachis appealing for Webenvironments. A strong con-sistency mechanismsuitablefor theWebmustnot only reduce client responsetime,but alsobalanceboth network andserver overheads.

Oneapproachthat providesstrong consistency, while providing a smoothtradeoff between thestatespaceoverhead andthenumber of control messagesexchanged,is leases [27]. In this approach,the server grants a lease to eachrequest from an intermediary. The lease duration denotesthe interval of timeduring which theserver agrees to notify theintermediary if theobjectis mod-ified. After theexpiration of the lease, the intermediarymustsenda messagerequestingrenewal of thelease.Theduration of theleasedeterminestheserverandnetwork overhead.A smaller leaseduration reducestheserver state spaceoverhead, but increasesthe numberof control (lease renewal) messages ex-changedandviceversa.In fact,aninfinite leaseduration reducestheapproachto server-basedinvalidation, whereas a zerolease duration reduces it to client-polling. Thus,the leasesapproachspans theentire spectrumbetweenthe twoextremesof server-basedinvalidation andclient-polling.

Theconceptof aleasewasfirst proposedin thecontext of cacheconsistencyin distributed file systems [27]. Recentlysomeresearch groups have beguninvestigating theuseof leasesfor maintainingconsistency in Webintermediarycaches. Theuseof leasesfor Webproxy cacheswasfirst alluded to in [11] andwassubsequently investigated in detail in [69]. The latter effort focused onthedesign of volumeleases– leasesgranted to a collectionof objects– soasto reduce (i) the leaserenewal overheadand(ii) the blocking overheadat theserver due to unreachable proxies. Other efforts have focusedon extendingleasesto hierarchical proxy cache architectures[70, 71]. Theadaptive leases

effort describedanalytical andquantitativeresultsonhow to select theoptimalleasedurationbased on theserver andmessageexchangeoverheads[21].

A qualitativecomparisonof theoverheadsof thedifferentconsistency mech-anismsis shown in Table1. The messageoverheadsof an invalidation-basedor lease-based approachis smaller thanthat of polling especially whenreadsdominate writes,asin theWebenvironment.

Ñ��[ͼ� �7��� ¦�²^�U��¦ �w¤¬Æ ¦������Ï� ��¦ �w¤¬Æ

With server-driven consistency mechanisms,when an object is modified,the origin server notifies each“subscribing” intermediary. The notificationconsistsof either aninvalidatemessageor anupdated(new) version of theob-ject. Sendinganinvalidatemessage causesanintermediaryto marktheobjectas invalid; a subsequentrequest requiresthe intermediaryto fetch the objectfrom the server (or from a designatedsite). Thus,eachrequest after a cacheinvalidate incurs anadditional delay dueto this remotefetch. An invalidationaddsto 2 control messagesandadata transfer (aninvalidation message,a readrequest on a miss,anda new datatransfer) along with the extra latency. Nosuchdelay is incurred if the server sends out the new version of the objectuponmodification. In an update-basedscenario, subsequentrequestscanbeservicedusinglocally cached data. A drawback,however, is thatsending up-datesincursalargernetwork overhead(especially for largeobjects).Thisextraeffort is wasted if theobject is never subsequently requestedat the intermedi-ary. Consequently, cacheinvalidatesarebetter suitedfor lesspopular objects,while updatescanyield better performancefor frequently requestedsmallob-jects. Deltaencoding techniqueshave been designedto reducethesizeof thedatatransferred in an update by sending only the changes to the object[40].Notethatdeltaencoding is not relatedto delta consistency. Updates,however,require better security guarantees andmake strongconsistency managementmore complex. Nevertheless,updatesareuseful for mirror sites wheredataneeds to be”pushed” to thereplicaswhenit changes.Updates arealsousefulfor pre-loadingcaches with content that is expected to becomepopular in thenearfuture.

A server candynamically decide between invalidatesandupdatesbased onthe characteristics of an object. Onepolicy could be to sendupdatesfor ob-jectswhosepopularity exceedsa thresholdandto sendinvalidatesfor all otherobjects. A morecomplex policy is to take bothpopularity andobject sizeintoaccount. Sincelarge objects imposea larger network transfer overhead, theserver canuseprogressively larger thresholds for such objects (the larger anobject, the more popular it needsto be before the server starts sending up-dates).

The choice betweeninvalidation andupdatesalsoaffects the implementa-tion of a strong consistency mechanism. For invalidationsonly, with a strongconsistency guarantee,theserver needsto wait for all acknowledgmentsof theinvalidation message(or a timeout) to committhewrite at theserver. With up-dates, on theother hand, theserver updatesarenot immediately committedattheintermediary. Only after theserver receivesall theacknowledgments (or atimeout) andthensends a commitmessage to all the intermediariesis thenewupdate versioncommitted at the intermediary. Suchtwo-phasemessage ex-changesareexpensive in practice andarenot required for weaker consistencyguarantees.

Ñ��HÑ�� ÁÃ����ÆN�UÆn�7¤ �s�q¾�÷ø¦��s¦ ¡�¤ �§¤<����¥j�¹�ÓÁ̽ÏÖÓÆ

An important issue thatmustbeaddressedin a CDN is that of consistencymaintenance. The problem of consistency maintenancein the context of asingleproxy usedseveraltechniquessuchastime-to-live (TTL) values,client-polling, server-based invalidation, adaptive refresh [63], and leases [68]. Inthesimplestcase,aCDN canemploy thesetechniquesateachindividualCDNserver or proxy – eachproxy assumes responsibility for maintaining consis-tency of datastored in its cacheand interacts with the server to do so inde-pendently of otherproxies in the CDN. Sincea typical CDN may consist ofhundredsor thousandsof proxies (e.g., Akamai currently hasa footprint ofmore than14,000 servers), requiring eachproxy to maintain consistency in-dependently of otherproxies is not scalable from theperspective of theoriginservers(sincetheserver will needto individually interactwith a large numberof proxies).Further, consistency mechanismsdesignedfrom theperspectiveofa single proxy (or a small groupof proxies)do not scalewell to large CDNs.The leasesapproach, for instance,requires the origin server to maintain per-proxy statefor eachcached object. This state space canbecomeexcessive ifproxiescache a large numberof objectsor someobjects arecached by a largenumberof proxieswithin a CDN.

A cacheconsistency mechanismfor hierarchical proxycacheswasdiscussedin [71]. Theapproachdoes not proposea new consistency mechanism,ratherit examinesissuesin instantiating existing approachesinto ahierarchical proxycacheusing mechanismssuchasmulticast. They argue for a fixed hierarchy(i.e., a fixed parent-child relationship betweenproxies). In addition to con-sistency, they alsoconsiderpushing of content from origin serversto proxies.Mechanismsfor scaling leasesarestudiedin [68]. Theapproachassumesvol-umeleases,whereeachleaserepresents multiple objects cached by a stand-aloneproxy. They examineissues suchasdelaying invalidationsuntil leaserenewalsanddiscussprefetchingandpushing leaserenewals.

Anothereffort describescooperativeconsistency along with a mechanism,called cooperative leases,to achieve it [52]. Cooperative consistency enablesproxies to cooperatewith oneanother to reduce the overheadsof consistencymaintenance.By supporting deltaconsistency semantics andby using asingleleasefor multipleproxies,thecooperative leasesmechanismallowsthenotionof leasesto beapplied in ascalablemannerto CDNs.Anotheradvantageof theapproachis that it employs application-level multicastto propagateserver no-tificationsof modificationsto objects,which reducesserveroverheads.Exper-imental results show that cooperative leasescanreduce the number of servermessages by a factor of 3.2 and the server stateby 20% whencompared tooriginal leases,albeit at anincreasedproxy-proxy communicationoverhead.

Finally, numerous studies have focusedon specific aspects of cache con-sistency for content distribution. For instance,piggybacking of invalidations[40], theuseof deltasfor sending updates[48], anapplication-level multicastframework for Internet distribution [26] and the efficacy of sending updatesversusinvalidates[22].

ß礬¥E¤<�N¤<���3¤¬Æ[1] V. Almeida, A. Bestavros, M. Crovella, and A. de Oliveira. Characterizingreference

locality in theWWW. In Proceedingsof PDIS’96:TheIEEEConferenceonParallel andDistributedInformationSystems, Miami Beach,Florida,December1996.

[2] TheApacheProject.TheApacheWWW server. http://httpd.apache.org.

[3] M. F. Arlitt andT. Jin. Workloadcharacterizationof the1998world cupwebsite. IEEENetwork, 14(3):30–37,May/June 2000.

[4] M. F. Arlitt andC. L. Williamson. InternetWebservers:Workloadcharacterizationandperformanceimplications. IEEE/ACM Transactionson Networking, 5(5):631–646,Oct1997.

[5] G. Banga,J.Mogul, andP. Druschel.A scalableandexplicit eventdelivery mechanismfor UNIX. In Proceedingsof the USENIX1999Technical Conference, Monterey, CA,June1999.

[6] A. Barbir, B. Cain, F. Douglis, M. Green,M. Hofmann, R. Nair, D. Potter and O.Spatscheck.Known CDN request-routingmechanisms.IETF Internet-Draft,February2002.

[7] P. Barford,A. Bestavros,A. Bradley, andM. E. Crovella. Changesin WebClientAccessPatterns:CharacteristicsandCachingImplications.World Wide WebJournal, 1999.

[8] M. Beck and T. Moore. The Internet2Distributed StorageInfrastructureProject:AnArchitecturefor InternetContentChannels.In Proceedingsof the3rd International WebCaching Workshop, 1998.

[9] T. Berners-Lee,R. Fielding,andH. Frystyk. Hypertext transferprotocol– HTTP/1.0.IETF RFC1945, May 1996.

[10] T. Brisco. DNSSupportfor LoadBalancing.IETF RFC1794, April 1995.

[11] P. Cao and C. Liu. Maintaining StrongCacheConsistency in the World-Wide Web.In Proceedingsof the SeventeenthInternational Conference on DistributedComputingSystems, May 1997.

[12] V. Cardellini,M. Colajanni,andP. Yu. DNS DispatchingAlgorithmswith StateEstima-torsfor ScalableWebServerClusters.World Wide Web, 2(2),July 1999.

[13] V. Cardellini,M. Colajanni,andP. Yu. DynamicLoadBalancingonWeb-ServerSystems.IEEEInternetComputing, pages28–39, May/June1999.

[14] V. Cate. Alex: A GlobalFile System.In Proceedingsof the1992USENIXFile SystemWorkshop, pages1–12,May 1992.

[15] J. Challenger, A. Iyengar, K. Witting, C. Ferstat,andP. Reed.A PublishingSystemforEfficiently CreatingDynamicWeb Content. In Proceedingsof IEEE INFOCOM2000,March2000.

[16] M. Crovella andA. Bestavros. Self-similarity in World Wide Webtraffic: Evidenceandpossiblecauses.IEEE/ACM Transactionson Networking, 5(6):835–846,Nov 1997.

[17] C. R. Cunha,A. Bestavros, andM. E. Crovella. Characteristicsof www client-basedtraces.TechnicalReportCS95-010, BostonUniversityComputerScienceDepartment,Boston,MA, June1995.

[18] M. Day, B. Cain,G. Tomlinson,andP. Rzewski. A model for contentinternetworking(CDI). InternetDraft (draft-ietf-cdi-model-01.txt),February2002.

[19] D. Dias,W. Kish, R. Mukherjee,andR. Tewari. A ScalableandHighly AvailableWebServer. In Proceedingsof the1996IEEEComputerConference(COMPCON), February1996.

[20] A. Downey. Thestructuralcauseof file sizedistributions.In Proceedingsof theNinth In-ternationalSymposiumonModeling, AnalysisandSimulationof ComputerandTelecom-municationSystems(MASCOTS), Cincinnati,OH, Aug 2001.

[21] V. Duvvuri, P. Shenoy, andR. Tewari. Adaptive Leases:A StrongConsistency Mecha-nismfor theWorld Wide Web. In Proceedingsof theIEEE Infocom’00,Tel Aviv, Israel,March2000.

[22] Z. Fei. A Novel Approachto Managing Consistency in ContentDistribution Networks.In Proceedings of the6th Workshop on WebCaching and ContentDistribution, Boston,MA, June2001.

[23] Z. Fei, S. Bhattacharjee,E. Zegura,and M. Ammar. A Novel Server SelectionTech-niquefor Improving theResponseTimeof aReplicatedService.In Proceedingsof IEEEINFOCOM’98, 1998.

[24] R. Fielding, J. Gettys,J. Mogul, H. Frystyk, and T. Berners-Lee. Hypertext transferprotocol– HTTP/1.1.IETF RFC2068, January1997.

[25] R. Fielding,J. Gettys,J. Mogul, H. Frystyk,L. Masinter, P. Leach,andT. Berners-Lee.Hypertext transferprotocol– HTTP/1.1. IETF RFC2616,June1999.

[26] P. Francis.Yoid: ExtendingtheInternetMulticastArchitecture.Technicalreport,AT&TCenterfor InternetResearchat ICSI (ACIRI), April 2000.

[27] C. GrayandD. Cheriton.Leases:An EfficientFault-TolerantMechanismfor DistributedFile CacheConsistency. In Proceedingsof the Twelfth ACM Symposiumon OperatingSystemsPrinciples, pages202–210,1989.

[28] M. GritterandD R. Cheriton.An Architecturefor ContentRoutingSupportin theInter-net. In Proceedingsof theUSENIXSymposiumon InternetTechnologies,SanFrancisco,CA, March2001.

[29] J. GwertzmanandM. Seltzer. World-Wide WebCacheConsistency. In Proceedingsofthe1996USENIXTechnical Conference, January 1996.

[30] J. C. Hu, S. Mungee,and D. C. Schmidt. Techniquesfor developing and measuringhigh-performance Web serversover ATM networks. In Proceedingsof the Conferenceon ComputerCommunications(IEEEInfocom), SanFrancisco,CA, Mar 1998.

[31] J. C. Hu, I. Pyarali,andD. C. Schmidt. Measuringthe impactof eventdispatching andconcurrency modelson Webserver performance over high-speednetworks. In Proceed-ingsof the2ndGlobal InternetConference(heldaspart of GLOBECOM ’97), Phoenix,AZ, Nov 1997.

[32] G.Hunt,G.Goldszmidt,R.King, andR.Mukherjee.Network Dispatcher:A ConnectionRouterfor ScalableInternetServices.In Proceedingsof the7thInternational World WideWebConference, April 1998.

[33] A. IyengarandJ.Challenger. Improving WebServer Performanceby CachingDynamicData. In Proceedingsof theUSENIXSymposium on InternetTechnologiesandSystems,December1997.

[34] A. Iyengar, J. Challenger, D. Dias,andP. Dantzig. High-PerformanceWebSiteDesignTechniques.IEEEInternetComputing, 4(2),March/April 2000.

[35] K. L. Johnson, J. F. Carr, M. S. Day, andM. F. Kaashoek.The measuredperformanceof contentdistribution networks. In InternationalWeb Caching and ContentDeliveryWorkshop (WCW), Lisbon,Portugal,May 2000. http://www.terena.nl/conf/wcw/Proceedings/S4/S4-1.pdf.

[36] P. Joubert,R. King, R. Neves, M. Russinovich, and J. Tracey. High-performancememory-based web servers:Kernelanduser-spaceperformance.In Proceedingsof theUSENIXAnnualTechnical Conference, Boston,MA, June2001.

[37] M. KoletsouandG. M. Voelker. TheMedusaproxy: A tool for exploring user-perceivedweb performance.In Proceedingsof InternationalWeb Caching and ContentDeliveryWorkshop (WCW), Boston,MA, June2001.Elsevier.

[38] B. KrishnamurthyandJ.Rexford. WebProtocolsandPractice. AddisonWesley, 2001.

[39] B. KrishnamurthyandC. Wills. ProxyCacheCoherency andReplacement—TowardsaMore CompletePicture. In Proceedingsof the 19th International Conferenceon Dis-tributedComputingSystems(ICDCS), June1999.

[40] B. KrishnamurthyandC. Wills. Studyof PiggybackCacheValidationfor ProxyCachesin the WWW. In Proceedingsof the 1997USENIXSymposiumon InternetTechnologyandSystems,Monterey, CA, pages1–12,December 1997.

[41] B. Krishnamurthy, C. Wills, andY. Zhang.On theuseandperformanceof contentdistri-bution networks. In Proceedingsof ACM SIGCOMMInternetMeasurementWorkshop,November2001.

[42] T. T. Kwan,R. E. McGrath,andD. A. Reed.NCSA’s World Wide WebServer: DesignandPerformance.IEEEComputer, 28(11):68–74,November1995.

[43] D. Li, P. Cao,andM. Dahlin. WCIP: Web CacheInvalidationProtocol. IETF InternetDraft, November2000.

[44] B. Mah. An empiricalmodelof HTTPnetwork traffic. In Proceedingsof theConferenceon ComputerCommunications(IEEEInfocom), Kobe,Japan,Apr 1997.

[45] Z. Morley Mao, C. D. Cranor, F. Douglis,M. Rabinovich, O. Spatscheck, andJ. Wang.A preciseandefficient evaluationof the proximity betweenweb clientsandtheir localDNSservers. In Proceedingsof USENIXAnnual Technical Conference, June2002.

[46] J. C. Mogul. Clarifying the fundamentalsof HTTP. In Proceedingsof WWW2002Conference, Honolulu,HA, May 2002.

[47] J. C. Mogul. Network behavior of a busy Webserver andits clients. TechnicalReport95/5, Digital EquipmentCorporationWesternResearchLab, Palo Alto, CA, October1995.

[48] JC. Mogul, F. Douglis,A. Feldmann,andB. Krishnamurthy. PotentialBenefitsof DeltaEncodingandDataCompressionfor HTTP. In Proceedingsof ACM SIGCOMMConfer-ence, 1997.

[49] D. Mosedale,W. Foss,andR. McCool. LessonsLearnedAdministeringNetscape’s In-ternetSite. IEEEInternetComputing, 1(2):28–35, March/April 1997.

[50] E. M. Nahum, T. Barzilai, and D. Kandlur. Performanceissuesin WWW servers.IEEE/ACM Transactionson Networking, 10(2):2–11, Feb2002.

[51] E. M. Nahum,M. Rosu,S.Seshan,andJ.Almeida. Theeffectsof wide-areaconditionson WWW server performance. In Proceedingsof the ACM SigmetricsConferenceonMeasurement andModelingof ComputerSystems, Cambridge,MA, June2001.

[52] A. Ninan,P. Kulkarni,P. Shenoy, K. Ramamritham,andR. Tewari. Cooperative Leases:ScalableConsistency Maintenancein ContentDistribution Networks. In ProceedingsoftheWorld Wide Webconference(WWW2002), May 2002.

[53] OpenMarket. FastCGI.http://www.fastcgi.com/.

[54] V. N. Padmanabhan andL. Qui. The contentandaccessdynamicsof a busy web site:findingsandimplications.In SIGCOMM, pages111–123,2000.

[55] V. Pai,M. Aron,G.Banga,M. Svendsen, P. Druschel,W. Zwaenepoel, andE.M. Nahum.Locality-AwareRequestDistribution in Cluster-basedNetwork Services.In Proceedingsof ASPLOS-VIII, October1998.

[56] V. Pai, P. Druschel,andW. Zwaenepoel. Flash:An efficient andportableWebserver. InUSENIXAnnualTechnical Conference, Monterey, CA, June1999.

[57] V. S.Pai,P. Druschel,andW. Zwaenepoel. I/O Lite: A copy-freeUNIX I/O system.In 3rdUSENIXSymposium on Operating SystemsDesignand Implementation, New Orleans,LA, February1999.

[58] M. Rabinovich and O. Spatscheck.Web Caching and Replication. Addison-Wesley,2002.

[59] A. Shaikh,R. Tewari, andM. Agrawal. On the Effectivenessof DNS-basedServer Se-lection. In Proceedingsof IEEEINFOCOM2001, 2001.

[60] J. Song,A. Iyengar, E. Levy, andD. Dias. Architectureof a Web Server Accelerator.ComputerNetworks, 38(1),2002.

[61] The Standard Performance Evaluation Corporation. SpecWeb99.http://www.spec.org/osg/web99,1999.

[62] Squid Internet Object Cache Users Guide. Available on-line at http://squid.nlanr.net,1997.

[63] R. Srinivasan,C. Liang, and K. Ramamritham. Maintaining TemporalCoherency ofVirtual Warehouses.In Proceedingsof the 19th IEEE Real-Time SystemsSymposium(RTSS98),Madrid, Spain, December1998.

[64] Sun Microsystems Inc. The Java Web server.http://wwws.sun.com/software/jwebserver/index.html.

[65] R. Tewari, T. Niranjan,andS.Ramamurthy. WCDP:WebContentDistributionProtocol.IETF InternetDraft, March2002.

[66] RedHat Inc. TheTux WWW server. http://people.redhat.com/mingo/TUX-patches/.

[67] D. C. Verma.ContentDistribution Networks:An EngineeringApproach. JohnWiley &Sons,2002.

[68] J. Yin, L. Alvisi, M. Dahlin, and A. Iyengar. EngineeringServer-driven Consistencyfor Large-scaleDynamic Web Services. In Proceedingsof the 10th World Wide WebConference, HongKong, May 2001.

[69] J.Yin, L. Alvisi, M. Dahlin,andC. Lin. VolumeLeasesfor Consistency in Large-ScaleSystems.IEEETransactionson Knowledge andData Engineering, January1999.

[70] J. Yin, L. Alvisi, M. Dahlin, and C. Lin. HierarchicalCacheConsistency in a WAN.In Proceedingsof theUsenixSymposiumon InternetTechnologies(USITS’99),Boulder,CO, October1999.

[71] H. Yu, L. Breslau,andS.Shenker. A ScalableWebCacheConsistency Architecture.InProceedingsof theACM SIGCOMM’99,Boston,MA, September1999.

[72] ZeusInc. TheZeusWWW server. http://www.zeus.co.uk.