1

Provisioning On-line Games: A TrafficAnalysis of a Busy Counter-Strike Server

Wu-chang Feng Francis Chang Wu-chi Feng Jonathan Walpole

Department of Computer Science EngineeringOGI School of Science and Engineering at OHSU

Beaverton, OR 97006{wuchang,francis,wuchi,walpole}@cse.ogi.edu

Abstract—This paper describes the results of a 500million packet trace of a popular on-line, multi-player,game server. The results show that the traffic behaviorof this heavily loaded game server is highly predictableand can be attributed to the fact that current game de-signs target the saturation of the narrowest, last-milelink. Specifically, in order to maximize the interac-tivity of the game itself and to provide relatively uni-form experiences between players playing over differ-ent network speeds, on-line games typically fix theirusage requirements in such a way as to saturate thenetwork link of their lowest speed players. While thetraffic observed is highly predictable, the trace alsoindicates that these on-line games provide significantchallenges to current network infrastructure. As a re-sult of synchronous game logic requiring an extremeamount of interactivity, a close look at the trace re-veals the presence of large, highly periodic, bursts ofsmall packets. With such stringent demands on inter-activity, routers must be designed with enough capac-ity to quickly route such bursts without delay. As cur-rent routers are designed for bulk data transfers withlarger packets, a significant, concentrated deploymentof on-line game servers will have the potential for over-whelming current networking equipment.

This material is based upon work supported by the NationalScience Foundation under Grant EIA-0130344 and the gener-ous donations of Intel Corporation. Any opinions, findings, andconclusions or recommendations expressed in this material arethose of the author(s) and do not necessarily reflect the views ofthe National Science Foundation or Intel.

I. I NTRODUCTION

Due the confluence of advances in graphics hard-ware, in network bandwidth on the backbone andto the home, in client and server hardware, and ininnovative software game design, on-line interac-tive games across all mediums has seen explosivegrowth. Forrester Research estimates that in 2001,there were 18 million on-line gamers in a 1.6 bil-lion dollar industry. Recent measurement studieshave shown that this is consistent with the observedgrowth in gaming traffic as Internet games now ac-count for a growing fraction of aggregate load [1].

With the upcoming launches of Microsoft’s Xboxon-line game network and Sony’s Playstation 2 on-line game network, along with the development ofemerging massively multi-player on-line games thatallow for thousands of simultaneous players [2], itis clear that gaming traffic will soon grow to scalesfar beyond what is being observed today. While notindicative of all on-line games, the class of gamesknown as “first-person shooters” has clearly domi-nated much of the observed gaming traffic. With alarge list of popular games such as Doom, Quake,Descent, Duke Nukem 3D, Half-Life, Counter-Strike, Unreal Tournament, Tribes 2, Day of Defeat,Command & Conquer Renegade, etc., these gamesare representative of what on-line games of the fu-ture are moving towards: large-scale, highly interac-tive, virtual worlds [3], [4], [5].

With the recent global explosion of on-line multi-player gaming, it is becoming more important to un-derstand its network behavior and usage in order to

2

provision and design future network infrastructure.By nature, traffic generated in support of this type ofapplication will be much different than web or TCP-based traffic which has received most of the atten-tion in the network research community [6], [7], [8],[9], [10], [11], [12], [13], [14], [15]. In particular,on-line gaming requires low-latency point-to-pointcommunication as well as directed broadcast chan-nels to facilitate its real-time game logic. In addition,such traffic tends to employ small, highly periodic,UDP packets. Packets are small since the applica-tion requires extremely low latencies which makesmessage aggregation impractical. Packets are highlyperiodic as a result of the game’s dynamic require-ment of frequent, predictable state updates amongstclients and servers. Finally, packets are typically sentvia UDP since clients typically send packets at an in-terval that is much shorter than the time it would taketo retransmit lost packets. In addition, the latency in-duced via socket buffers [16] and delayed acknowl-edgements is often too large to support meaningfulinteractivity.

In this paper, we take a closer look at a class ofapplications that look to become a major componentin the traffic mix for the foreseeable future. Specif-ically, we study the behavior of an extremely busy,on-line game server over the course of a week. Sec-tion II describes the on-line game we have examined.Section III describes the results of a one-week traceof the game server. Section IV analyzes the implica-tions that gaming traffic will have on network infras-tructure of the future and Section V concludes.

II. BACKGROUND

In order to understand the characteristics of In-ternet gaming, we examined the behavior of an ex-tremely popular Counter-Strike server [5]. Counter-Strike is a modification to the popular Half-Life [4]game and has become one of the most popular andmost network-intensive games played over the In-ternet as of this writing. Counter-Strike is a partof a large class of multi-player, on-line, first-personshooters that has dominated network gaming trafficover the last several years. As of May 2002, therewere more than 20,000 Counter-Strike servers ac-tive [5]. The game is architected as a client-serverapplication with multiple clients communicating andcoordinating with a central server that keeps track of

the global state of the game.

In the game itself, two teams continuously playback-to-back rounds of several minutes in duration.Each team attempts to complete their objectives andfoil those of the other team during the round. Theround ends when one team manages to completetheir mission, when one team eliminates the otherteam entirely, or when time runs out. When playersare eliminated from a round, they become spectatorsuntil the next round. During this time, the player canshadow another player that has not been eliminated.The game itself is played on a variety of maps whichrotate based on how the server is configured. Typ-ically, maps are rotated every 30 minutes, allowingfor over 10 rounds to be played per map. Dependingon the hardware and configuration, a Counter-Strikeserver can support up to 32 simultaneous players.

Traffic generated by the game can be attributed toa number of sources. The most dominant source isthe real-time action and coordinate information sentback and forth between clients and server. This in-formation is periodically sent from all of the clientsto the server. The server then takes this informationand performs a periodic broadcast to each client, ef-fectively distributing the global state of the game. Inaddition to game physics datum, the game engineallows for broadcast text-messaging and broadcastvoice communication amongst players all throughthe centralized server. The game server also sup-ports the upload and download of customized logosthat can be seen by everyone on a per-client basis.Each client is able to customize a texture map thatmay be placed on the surfaces of the current map.These images are uploaded and downloaded whenusers join the game and when a new map starts sothat each client can properly display the custom de-cals of the other users. Finally, the game server sup-ports downloads of entire maps, which may consistof sounds, texture libraries and a compiled BinarySpace Partitioning tree [17]. In order to prevent theserver from becoming overwhelmed by concurrentdownloads, these downloads are rate-limited at theserver.

3

Start Time Thu Apr 11 08:55:04 2002Stop Time Thu Apr 18 14:56:21 2002Total Time of Trace 7 d, 6 h, 1 m, 17.03 s

(626,477 sec)Maps Played 339Established Connections 16030Unique Clients Establishing 5886Attempted Connections 24004Unique Clients Attempting 8207

TABLE I

GENERAL TRACE INFORMATION

III. E VALUATION

A. Trace summary

To properly evaluate the traffic generated by thisrepresentative application, we hosted the Counter-Strike (version 1.3) server of one of the most pop-ular on-line gaming communities in the North-west region: Olygamer.com [18]. Because of thelarge Olygamer following, our exceptional Internetconnectivity (OC-3 to Internet2, dual T3s to non-Internet2 sites), the speed of the server used (DellDimension 4300, Pentium 4, 1.8GHz, 512MB), andthe superiority of our game configuration (whichincludes modules that eliminate cheating and deterteam killing [19], [20], [21], [22], [23]), this serverquickly became heavily utilized with connections ar-riving from all parts of the world irrespective of thetime of day. The server itself was configured witha maximum capacity of 22 players. After a briefwarm-up period, we recorded the traffic to and fromthe server over the course of a week (April 11-18).The trace collected consisted of a half billion pack-ets. Note that while we are able to effectively an-alyze this single server, the results in this study donot directly apply to overall aggregate load behav-ior of the entire collection of Counter-Strike servers.In particular, it is expected that active user popula-tions will not, in general, exhibit the predictabilityof the server studied in this paper and that the globalusage pattern itself may exhibit a high degree of self-similarity [1], [24], [25].

Table I summarizes the trace itself. The trace cov-

Total Packets 500,000,000Total Packets In 273,846,081Total Packets Out 226,153,919Total Bytes 64.42 GBTotal Bytes In 24.92 GBTotal Bytes Out 39.49 GBMean Packet Load 798.11 pkts/secMean Packet Load In 437.12 pkts/secMean Packet Load Out 360.99 pkts/secMean Bandwidth 883 kbsMean Bandwidth In 341 kbsMean Bandwidth Out 542 kbs

TABLE II

NETWORK USAGE INFORMATION

Total Bytes 37.41 GBTotal Bytes In 10.13 GBTotal Bytes Out 27.28 GBMean Packet Size 80.33 bytesMean Packet Size In 39.72 bytesMean Packet Size Out 129.51 bytes

TABLE III

APPLICATION INFORMATION

ers over a week of continuous operation of the gameserver. Over 300 maps were played during this time

4

0�

2000 4000 6000 8000 10000�

Time (min)

0

100

200

300

400

500

600

700

800

900

1000

1100

1200

1300B

andw

idth

(ki

lobi

ts/s

ec)

4/11�

4/12�

4/13�

4/14�

4/15�

4/16�

4/17�

4/18�

Fig. 1. Per-minute bandwidth of server for entire trace

frame and more than 16000 user sessions were estab-lished. Due to the popularity of the server, each useraveraged almost 3 sessions for the week and morethan 8000 connections were refused due to the lackof open slots on the server. In addition, each playerwas connected to the game an average of approxi-mately 15 minutes. Table II summarizes the networkload generated by the server. Over the 500 millionpacket trace, more than 60GB of data were sent in-cluding both network headers and application data.Overall, the bandwidth consumed approached1Mbsfor the duration of the trace. The table also showsthat even though the application received more pack-ets than it sent, its outgoing bandwidth exceeded itsincoming bandwidth. Table III shows the reason whythis was the case. The mean size of outgoing applica-tion data packets was more than three times the sizeof incoming application data packets.

To understand the dynamics of the trace, Figure 1and Figure 2 plot the per-minute average bandwidthand packet load observed at the server. As the fig-ures show, while there is a lot of short-term varia-tion in the trace, the trace exhibits fairly predictablebehavior over the long-term. Aggregate bandwidthconsumed by the server hovers around 800-900 kilo-bits per second (kbs) while the server sees a packetrate of around 700-800 packets per second (pps). Inaddition, as Figure 3 shows, the number of activeplayers also shows a lot of short-term variation withfairly predictable long-term behavior. The number

0�

2000 4000 6000 8000 10000�

Time (min)

0

200

400

600

800

1000

1200

Pac

ket l

oad

(pac

kets

/sec

)

�

4/11�

4/12�

4/13�

4/14�

4/15�

4/16�

4/17�

4/18�

Fig. 2. Per-minute packet load of server for entire trace

0�

200000 400000 600000Time (sec)

0.0

5.0

10.0

15.0

20.0

25.0

30.0

Pla

yers

�

4/11�

4/12�

4/13�

4/14�

4/15�

4/16�

4/17�

4/18�

Fig. 3. Per-minute number of players for entire trace

of players sometimes exceeds the maximum numberof slots of 22 as multiple clients can come and goduring an interval. The trace itself also encompassesseveral brief network outages on April 12, April 14,and April 17. These outages caused minor disrup-tions in user numbers as the client and server discon-nect after not hearing from each other over a periodof several seconds. In all three cases, all of the play-ers or a majority of players were disconnected fromthe server at identical points in time. While someof the players, having recorded the server’s IP ad-dress, immediately reconnected, a significant num-ber did not as they relied on dynamic server auto-discovery and auto-connecting to find this particu-

5

0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0log (interval size)

−8.0

−7.0

−6.0

−5.0

−4.0

−3.0

−2.0

−1.0

0.0lo

g (n

orm

aliz

ed v

aria

nce)

H=1/2Server trace

50ms 30min

Fig. 5. Variance time plot for total server packet load

lar game server [26]. Thus, the user population andnetwork traffic observed around these outages showsignificant dips on the order of minutes even thoughthe actual outage was on the order of seconds. Dueto the size of the trace, however, these brief outagesdid not have a significant impact on the traffic char-acterization.

Figure 4 shows a further breakdown of the tracebetween incoming traffic generated by clients andsent to the server and outgoing traffic generated bythe server and sent to the clients. As the figureshows, the incoming packet load exceeds the outgo-ing packet load while the outgoing bandwidth ex-ceeds the incoming bandwidth. Thus, it is clearthat while the server sends less packets, the pack-ets it sends are significantly larger than the incomingpackets. This is not surprising as the game serveritself is logically playing the role of a broadcaster:taking state information from each client and broad-casting it out to all other clients [27]. Section III-Cgives a more detailed analysis of packet sizes in thetrace.

B. Periodicity and predictability

While visually it appears that the server’s net-work load is relatively stable, the true measure ofvariability is its associated Hurst parameter [10],[28]. In order to measure variability across multipletime scales, we use the standard aggregated variancemethod to estimate the Hurst parameter (H) of the

0�

50 100�

150� 200�

Interval number

0

500

1000

1500

2000

2500

Pac

ket l

oad

(pac

kets

/sec

)

�

Fig. 6. Total packet load for them = 10ms interval size

trace. In this method, the sequence is divided intomultiple, consecutive, equally-sized, blocks. Thevalues within the block are averaged and the vari-ance of the sequence of averages is calculated. For ashort-range dependent process, as the block size (m)is increased, the variance of the resulting sequenceconsistently decreases. In contrast, for long-rangedependent sequences, the sequence maintains highvariability across block sizes and time scales. To de-termine the degree of long-range dependence, the logof the normalized variance is plotted against the logof the block size. The normalized variance is cal-culated as the variance of the aggregated sequencedivided by the variance of the initial, unaggregatedsequence. The block size, in this case, is the numberof frames per block. The Hurst parameter (H) canbe estimated by taking the magnitude of the slope ofthe best-fit line through the data points (β) and cal-

culatingH via the relationH = 1 − β2 . The Hurst

parameter thus ranges between12 and 1. A short-

range or no dependence sequence will have a slopeof −1 which corresponds to anH of 1

2 while a long-range dependent sequence will have anH closer to1.

Figure 5 shows the variance-time plot of the trace.For this plot, a base interval size ofm = 10mswas used. The plot shows three distinct regions ofbehavior. For smallm (m < 50ms), there is ahigh degree of increased smoothness as the inter-val size is increased, as shown by the slope of the

6

0�

2000 4000 6000 8000 10000�

Time (min)

0.0

100.0

200.0

300.0

400.0

500.0

600.0

700.0

800.0

900.0B

andw

idth

(ki

lobi

ts/s

ec)

4/11�

4/12�

4/13�

4/14�

4/15�

4/16�

4/17�

4/18�

0�

2000 4000 6000 8000 10000�

Time (min)

0.0

100.0

200.0

300.0

400.0

500.0

600.0

700.0

800.0

900.0

Ban

dwid

th (

kilo

bits

/sec

)

4/11�

4/12�

4/13�

4/14�

4/15�

4/16�

4/17�

4/18�

(a) Incoming bandwidth (b) Outgoing bandwidth

0�

2000 4000 6000 8000 10000�

Time (min)

0

200

400

600

Pac

ket l

oad

(pac

kets

/sec

)

�

4/11�

4/12�

4/13�

4/14�

4/15�

4/16�

4/17�

4/18�

0�

2000 4000 6000 8000 10000�

Time (min)

0

200

400

600

Pac

ket l

oad

(pac

kets

/sec

)

�

4/11�

4/12�

4/13�

4/14�

4/15�

4/16�

4/17�

4/18�

(c) Incoming packet load (d) Outgoing packet load

Fig. 4. Per-minute incoming and outgoing bandwidth and packet load of server for entire trace

variance plot asH drops below12 . For larger inter-

val sizes (50ms < m < 30min), significant vari-ability and burstiness remains even as the intervalsize is increased. Finally, for large interval sizes(m > 30min), the plot shows typical behavior fora short-range dependent sequence as larger intervalsizes consistently produce reduced variability withan estimatedH of around1

2 .

To fully understand the behavior at varying time-scales, Figure 6 and Figure 7 plot the total packetload, the incoming packet load, and the outgoingpacket load observed at the server for the first 20010ms intervals of the trace. The figure exhibits an

extremely bursty, highly periodic pattern. When bro-ken down between incoming and outgoing packetload, it is clear that the periodicity comes fromthe game server deterministically flooding its clientswith state updates about every50ms. This syn-chronous behavior is due to the game server logic it-self. Since clients connect via diverse network paths,the incoming packet load is not highly synchronized.

Given this behavior, Figure 8 shows the plot ofthe first 20050ms intervals of the trace. As ex-pected, aggregating over this interval smooths outthe packet load considerably. Betweenm = 50msandm = 30min, however, the variance-time plot

7

0�

50 100�

150� 200�

Interval number

0

500

1000

1500

2000

2500P

acke

t loa

d (p

acke

ts/s

ec)

�

0�

50 100�

150� 200�

Interval number

0

500

1000

1500

2000

2500

Pac

ket l

oad

(pac

kets

/sec

)

�

(a) Incoming packet load (b) Outgoing packet load

Fig. 7. Incoming and outgoing packet load for them = 10ms interval size

exhibits a high degree of variability. This variabilitycan be attributed to the network disruptions causedby the30min map time of the server. As the serverloads a new map every half-hour, the network trafficdips significantly for a short period of time. Becausemost of the clients will have the maps stored locally,this down time is due completely to the server doinglocal tasks to perform the map change over. Fig-ure 9 shows this behavior with a plot of the first180001sec intervals. Noticeable dips appear every1800 (30min) intervals. Because map changing isa configuration-specific feature, this behavior is nota generic characteristic and can be affected by gameadministrators changing maps directly, players vot-ing to change maps or extending the current map,or a different map time limit setting. For this traceand server configuration, increasing the interval sizebeyond the default map time of30min removes thevariability. Figure 10 plots the first 20030min inter-vals of the trace. As the figure shows, the variabilityhas been eliminated.

The predictability of the aggregate leads us to ex-amine how predictable the resource consumption ofeach individual flow is in the trace. Perhaps the mostinteresting observation is that when the mean band-width of the server is divided by the total numberof players allowed by the game server itself (22),the bandwidth consumed per player is on average40kbps. This is no coincidence as the game is meant

0�

50 100�

150� 200�

Interval number

0

500

1000

1500P

acke

t loa

d (p

acke

ts/s

ec)

�

Fig. 8. Total packet load plot form = 50ms

to be played uniformly across a wide range of net-work speeds, down to and including the ubiquitous56kbps modem. As typical performance of56kbpsmodems range from40− 50kbs [29], it is clear thatthis particular game was designed tosaturate thenarrowest last-mile link. Going back to the traceitself, we measured the mean bandwidth consumedby each flow at the server in order to get a pictureof the bandwidth distribution across clients. Assum-ing minimal packet loss and a negligible differencein link-layer header overhead between the last-milelink and the server’s link, the bandwidth measured at

8

0�

3600 7200 10800� 14400� 18000�

Interval number

0

500

1000

1500P

acke

t loa

d (p

acke

ts/s

ec)

�

Fig. 9. Total packet load plot form = 1sec

0�

50 100�

150� 200�

Interval number

0

500

1000

1500

Pac

ket l

oad

(pac

kets

/sec

)

�

Fig. 10. Total packet load plot form = 30min

the server will be quite close to what is sent acrossthe last hop. Figure 11 shows a histogram of band-widths across all sessions in the trace that lastedlonger than30sec. The figure shows that the over-whelming majority of flows are pegged at modemrates or below even though connections arrived viadiverse network mediums. The figure also showsthat some flows do, in fact, exceed the56kbps bar-rier. This is due to the fact that the client can bespecially configured to crank up the update rate toand from the server in order to improve the inter-activity of gameplay even futher. As shown by thehistogram, only a handful of ”l337” players connect-ing via high speed links have taken advantage of the

0�

50000�

100000 150000Bandwidth (bits/sec)

0

20

40

60

80

100

Num

ber

of p

laye

rs

�

Fig. 11. Client bandwidth histogram

setting.

C. Tiny packets

While the good news in the trace is that for a fixedset of players, the traffic generated is highly stableand predictable, the bad news is that the traffic itselfis made up of large, periodic bursts of very smallpackets. Figure 12(a) plots the probability densityfunction (PDF) of packet sizes across the 500 millionpacket trace. The graph is truncated at 500 bytes asonly a negligible number of packets exceeded this.As the figure shows, almost all of the packets are un-der 200 bytes. Note that packet sizes include onlythe application data itself and not the UDP, IP, orEthernet headers. Figure 12(b) shows the PDF ofboth incoming and outgoing packets. While incom-ing packets have an extremely narrow distributioncentered around the mean size of 40 bytes, outgo-ing packets have a much wider distribution around asignificantly larger mean. This causes the outgoingbandwidth to exceed the incoming bandwidth eventhough the rate of incoming packets exceeds that ofoutgoing packets. Figure 13 shows the cumulativedistribution function (CDF) of the packet sizes. Asthe figure shows, almost all of the incoming pack-ets are smaller than 60 bytes while a large fractionof outgoing packets have sizes spread between 0 and300 bytes. This is significantly different than aggre-gate traffic seen within Internet exchange points [1]in which the mean packet size observed was above

9

0�

100�

200�

300�

400�

500�

Packet size (bytes)

0.00

0.01

0.02

0.03

0.04

0.05P

roba

bilit

y

�

0�

100�

200�

300�

400�

500�

Packet size (bytes)

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.10

Pro

babi

lity

�

InboundOutbound

Fig. 12. Packet size probability density functions

0.0 100.0�

200.0�

300.0�

400.0�

500.0�

Packet size (bytes)

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Pro

babi

lity

�

InboundOutboundTotal

�

Fig. 13. Packet size cumulative density functions

400 bytes.

IV. I MPLICATIONS ON ROUTING

INFRASTRUCTURE

A. The Bad News

Perhaps the most significant aspect of the trace isthe observation that the game traffic itself consistsof large, periodic bursts of short packets. While thetrace is of only a single game over a single week,we believe that this is a characteristic that will befundamental in all sufficiently loaded, highly inter-active, on-line games due to the nature of the appli-

Outgoing Traffic

Total Packets From Server to NAT 677,278Total Packets From NAT to Clients674,157Loss Rate 0.046%

Incoming Traffic

Total Packets From Clients to NAT853,035Total Packets From NAT to Server 841,960Loss Rate 1.3%

TABLE IV

NAT EXPERIMENT

cation and the underlying game logic. Short packetsare required for low latency while highly periodictraffic allows the game to provide uniform interac-tivity amongst all of its clients. Some evidence ofthis exists in aggregate measures of other game ap-plications [1].

Unfortunately for games, routers are not necessar-ily designed for this type of traffic. With the explo-sion of the web and peer-to-peer networks, the ma-jority of traffic being carried in today’s networks in-volve bulk data transfers using TCP. While TCP ac-knowledgements are typically small, data segmentscan be close to an order of magnitude larger thangame traffic. With the addition of delayed acknowl-edgments and piggy-backed acknowledgments, theaverage packet sizes of most bi-directional TCP con-

10

nections will exceed those for games. Router de-signers and vendors often make packet size assump-tions when building their gear, often expecting av-erage sizes in between 1000 and 2000 bits (125-250bytes) [30]. Thus, a significant shift in packet sizefrom the deployment of on-line games will make theroute lookup function the bottleneck versus the linkspeed [31]. Routing devices that are not designed tohandle small packets will then see significant packet-loss oreven worseconsistent packet delay and delayjitter when handling game traffic.

To demonstrate this problem, we inserted acommerical-off-the-shelf NAT device in between theserver and the rest of the Internet. The specific de-vice used was an SMC Barricade hardware NAT de-vice (SMC7004AWBR). The SMC device has thecapacity to switch at speeds of up to100Mbs and hasa listed packet routing capacity of 1000-1500pps onaverage [32]. The wireless interface and all otherphysical ports were disabled so that the device onlyhandled game traffic during the experiment.

After a brief warm-up period, we traced a single,30min map of our server. Throughout the experi-ment, players complained about a significant degra-dation in performance with noticeable amounts oflag and packet loss that did not exist before the NATdevice was in place. The results of the trace areshown in Table IV. The table shows that the NATdevice does, in fact, induce packet loss with over 1%loss for incoming packets and almost 0.5% loss foroutgoing packets. The losses occur at the most criti-cal points during gameplay when a lot of events areoccuring, magnifying the impact of the losses evenfuther. It is interesting that more incoming packetsare lost versus outgoing packets as Figure 7 showsthat the outgoing packet load is much more bursty.We believe that the loss on the incoming path is in-duced by and is a result of individual server packetbursts. Figure 14 and Figure 15 plot the packet loadthrough the device. In this case, the incoming packetload from the clients to the NAT device is relativelystable while the packet load from the NAT deviceto the server sees frequent drop-outs. Interestinglyenough, this disruption in service causes the gameapplication itself to freeze as well with outgoing traf-fic from the server to the NAT device and outgo-ing traffic from the NAT device to the clients show-ing drop-outs directly correlated with lost incoming

packets. While the loss rates overall were small, itwas enough to substantially degrade the game qual-ity. Due to the nature of the game applications, ob-served loss ratesself-tunethemselves at the worsttolerable level of performance. Any further degrada-tion caused by additional players and/or backgroundtraffic will simply cause players to quit playing, re-ducing the load back to the tolerable level. Giventhe level of dissatisfaction caused by this experimentwithin the game, we believe the worst tolerable lossrate for this game is not far from 1-2%.

The NAT experiment shows that when placed be-hind a device like this, a single instance of theserver running at about800kbps is enough to over-whelm the resources of a device designed to routeat significantly higher rates. For this application,adding buffers or combining packets does not nec-essarily help performance since delayed packets canbe worse than dropped packets. Delayed packets de-grade user experience [33] as well as consume ad-ditional processing resources at the server as it triesto keep its global state consistent. In this particu-lar case, buffering the50ms packet spikes will con-sume more than a quarter of the maximum tolera-ble latency for this type of interactive game. With-out the routing capacity in place, hosting a success-ful game server behind such a device is simply notfeasible. Extending this to on-line ventures fromMicrosoft and Sony and to massively multi-playergames, it is apparent that even mid-range routers orfirewalls within several hops of large hosted on-linegame servers will need to be carefully provisioned tominimize both the loss and delay induced by routingextremely small packets. This will almost certainlyrequire increasing the peak route lookup capacity ofintermediate routers as adding buffers will add an un-acceptable level of delay.

B. The Good News

The good news about the trace is that the pre-dictability in resource requirements makes the mod-eling, simulation, and provisioning on-line gamingtraffic a relatively simple task as the traffic does notexhibit fractal behavior when the number of activeplayers is relatively fixed. As a result of this pre-dictability, the traffic from an aggregation of all on-line Counter-Strike players is effectively linear to thenumber of active players. While this is the case, the

11

0�

500�

1000 1500Time (sec)

0

100

200

300

400

500

600P

acke

t loa

d (p

acke

ts/s

ec)

�

0�

500�

1000 1500Time (sec)

0

100

200

300

400

500

600

Pac

ket l

oad

(pac

kets

/sec

)

�

(a) Packet load from clients to NAT (b) Packet load from NAT to server

Fig. 14. Per-second incoming packet load for NAT experiment

0�

500�

1000 1500Time (sec)

0

100

200

300

400

500

600

Pac

ket l

oad

(pac

kets

/sec

)

�

0�

500�

1000 1500Time (sec)

0

100

200

300

400

500

600

Pac

ket l

oad

(pac

kets

/sec

)

�

(a) Packet load from server to NAT (b) Packet load from NAT to clients

Fig. 15. Per-second outgoing packet load for NAT experiment

actual number of players on-line over time may, infact, exhibit a high degree of variability and self-similarity. Self-similarity in aggregate game trafficin this case will be directly dependent on the self-similarity of user populations [24], [25]. Since thetrace itself can be used to more accurately developsource models for simulation [34], we hope to makethe trace and associated game log file publicly avail-able [35].

The other silver lining in this trace is that whilethe small packets of on-line games have the poten-

tial to wreak havoc on routing infrastructure, the pe-riodicity and predictability of packet sizes allowsfor meaningful performance optimizations withinrouters. For example, preferential route cachingstrategies based on packet size or packet frequencymay provide significant improvements in packetthroughput. We hope to explore these issues furtheron a network processor testbed [36].

12

V. CONCLUSION

With the explosion in on-line, multi-player net-work games, it is becoming imperative to character-ize this component of Internet traffic that will remaina sizable portion of overall usage. To this end, wehave collected and analyzed a one-week, 500 mil-lion packet trace of a popular Internet game server.The resulting traffic consists of large, highly periodicbursts of small packets with predictable long-termrates and can be attributed to the synchronous na-ture of current game designs, the need to uniformlydeliver a high degree of interactivity, and the phe-nomenon ofnarrowest last-mile link saturation. Thebad news is that because of these factors, on-linegame servers tend to generate heavy packet loadsthat may potentially overwhelm the route-lookup re-sources within the network. The good news is that,given a fixed number of players, these servers gen-erate highly predictable traffic, thus simplifying theprovisioning for and modeling of on-line games. Inaddition, the predictability and periodicity may befurther leveraged to improve route caching strategieswithin the network.

As part of future work, we plan on using the re-sults of this trace to explore the possibility of betterroute caching algorithms to support networks carry-ing this type of traffic. We also plan on examininga multitude of emerging on-line games. While weexpect the trends to change very little as long as thelast hop is dominated by slow modems, the gaminglandscape tends to change much more quickly thanany other class of Internet applications. Unlike webtraffic whose aggregate application behavior is rela-tively well-characterized and changes slowly, we ex-pect game traffic to be much more dynamic as gamestend to zoom to popularity in a short period of timewith new games replacing old games within veryshort time periods.

REFERENCES

[1] S. McCreary and k. claffy, “Trends in Wide Area IP TrafficPatterns: A View from Ames Internet Exchange,” inPro-ceedings of 13th ITC Specialist Seminar on Measurementand Modeling of IP Traffic, September 2000, pp. 1–11.

[2] GameSpy.com, “What’s This World Coming To? The Fu-ture of Massively Multiplayer Games,”http://www.

gamespy.com/gdc2002/mmog .

[3] id Software, “id Software,” http://www.

idsoftware.com/ .

[4] Half-Life, “The Official Half-Life Web Site,” http://

half-life.sierra.com/ .

[5] Counter-Strike, “Counter-Strike: A Counter-Terrorism Half-Life Modification,” http:

//www.counter-strike.net/ .

[6] P. Barford and M. Crovella, “Generating Representa-tive Web Workloads for Network and Server PerformanceEvaluation,” inProceedings of ACM SIGMETRICS, July1998, pp. 151–160.

[7] R. Caceres, N. Duffield, A. Feldmann, J. Friedmann,A. Greenberg, R. Greer, T. Johnson, C. Kalmanek, B. Kr-ishnamurthy, D. Lavelle, P. Mishra, K. Ramakrishnan,J. Rexford, F. True, and J. vanderMerwe, “Measurementand Analysis of IP Network Usage and Behavior,”IEEECommunication Magazine, May 2000.

[8] M. Crovella and A. Bestavros, “Self-similarity in WorldWide Web Traffic: Evidence and Possible Causes,” inPro-ceedings of ACM SIGMETRICS, May 1996.

[9] A. Feldmann, A. Gilbert, and W. Wilinger, “Data Net-works as Cascades: Investigating the Multifractal Natureof Internet WAN Traffic,” in Proceedings of ACM SIG-COMM, September 1998.

[10] W. Leland, M. Taqqu, W. Willinger, and D. Wilson, “Onthe Self-Similar Nature of Ethernet Traffic (Extended Ver-sion),” IEEE/ACM Transactions on Networking, vol. 2, no.1, February 1994.

[11] B. Mah, “An Empirical Model of HTTP Network Traffic,”in Proceedings of IEEE INFOCOM, April 1997.

[12] V. Paxson and S. Floyd, “Wide-Area Traffic: The Fail-ure of Poisson Modeling,” inProceedings of ACM SIG-COMM, August 1994, pp. 257–268.

[13] V. Paxson, “Empirically-derived Analytic Models ofWide-Area TCP Connections,”IEEE/ACM Transactionson Networking, 1994.

[14] V. Paxson, “End-to-End Internet Packet Dynamics,” inProceedings of ACM SIGCOMM, September 1997.

[15] A. Veres and M. Boda, “The Chaotic Nature of TCP Con-gestion Control,” inProceedings of IEEE INFOCOM,April 2000.

[16] Ashvin Goel, Charles Krasic, Kang Li, and JonathanWalpole, “Supporting Low Latency TCP-Based MediaStreams,” inProceedings of IWQoS, May 2002.

[17] H. Fuchs, Z. M. Kedem, and B. F. Naylor, “On VisibleSurface Generation by a Priori Tree Structures,”ComputerGraphics(SIGGRAPH ’80 Proceedings), vol. 14, pp. 124–133, July 1980.

[18] Olygamer.com, “Olygamer.com Gaming HQ,”http:

//www.olygamer.com/ .

[19] Half-Life Admin Mod Developers, “Half-Life AdminMod Home,” http://www.adminmod.org/ .

13

[20] Cheating-Death Developers, “Cheating-Death Home,”http://www.cheating-death.com/ .

[21] CSCop Developers, “Project Info - CSCop for Counter-Strike,” http://sourceforge.net/projects/

cscop .

[22] CSGuard Developers, “CSGuard Home,”http://www.

olo.counter-strike.pl/ .

[23] UDPSoft.com, “HLDS Ping Booster,”http://www.

udpsoft.com/ .

[24] T. Henderson and S. Bhatti, “Modelling User Behavior inNetworked Games,” inACM Multimedia, 2001, pp. 212–220.

[25] T. Henderson, “Latency and User Behaviour on a Multi-player Game Server,” inNetworked Group Communica-tion, 2001, pp. 1–13.

[26] T. Henderson, “Observations on Game Server DiscoveryMechanisms,” inNetGames, April 2002.

[27] D. LaPointe and J. Winslow, “Analyzing and SimulatingNetwork Game Traffic,”http://www.cs.wpi.edu/

˜claypool/mqp/net-game/game.pdf , December2001.

[28] H.E. Hurst, “Long-Term Storage Capacity of Reservoirs,”Proc. American Society of Civil Engineers, vol. 76, no. 11,1950.

[29] J. Kristoff, “Understanding 56Kbps Modems,”http://homepage.interaccess.com/

˜jkristof/56kmodem.html , 1997.

[30] C. Partridge, P. Carvey, E. Burgess, I. Castineyra,T. Clarke, L. Graham, M. Hathaway, P. Herman, A. King,S. Kohalmi, T. Ma, J. Mcallen, T. Mendez, W. Milliken,R. Pettyjohn, J. Rokosz, J. Seeger, M. Sollins, S. Storch,B. Tober, G. Troxel, D. Waitzman, and S. Winterble, “A50-Gb/s IP Router,”IEEE/ACM Transactions on Network-ing, vol. 6, no. 3, pp. 237–245, June 1998.

[31] Super Computer Gaming, “Super ComputerGaming: Jupiter Cluster,” http://www.

supercomputergaming.com/ , 2002.

[32] SMC FAQ, “SMC Barricade FAQ: How fastis the Barricade? What is the throughput?,”http://www.smc-europe.com/english/

support/faq.html .

[33] I. S. MacKenzie and S. Ware, “Lag as a Determinant ofHuman Performance in Interactive Systems,”Proceedingsof the ACM Conference on Human Factors in ComputingSystems - INTERCHI, pp. 488–493, 1993.

[34] M. Borella, “Source Models of Network Game Traffic,”Computer Communications, vol. 23, no. 4, pp. 403–410,February 2000.

[35] W. Feng, “cs.mshmro.com Counter-Strike Server Traf-fic Trace 4/11-4/18,” 40GB trace available upon request,April 2002.

[36] Intel, Inc., “Intel IXP 1200,” http://developer.

intel.com/design/network/products/

npfamily/ixp1200.ht%m .