Electronic copy available at: http://ssrn.com/abstract=1748022

STATISTICAL MODELING OF HIGH FREQUENCY FINANCIAL DATA:FACTS, MODELS AND CHALLENGES

Rama CONT

Laboratoire de Probabilites et Modeles Aleatoires,CNRS-Universite de Paris VI

&IEOR Dept, Columbia University, New York

The availability of high frequency data on transactions,quotes and order flow in electronic order-driven markets hasrevolutionized data processing and statistical modeling tech-niques in finance and brought up new theoretical and compu-tational challenges. Market dynamics at the transaction levelcannot be characterized solely in terms the dynamics of a sin-gle price and one must also take into account the interactionbetween buy and sell orders of different types by modelingthe order flow at the bid price, ask price and possibly otherlevels of the limit order book. We outline the empirical char-acteristics of high frequency financial time series and providean overview of stochastic models for the continuous-time dy-namics of a limit order book, focusing in particular on modelswhich describe the limit order book as a queuing system. Wedescribe some applications of such models and point to someopen problems.1

Keywords: high frequency financial data, limit orderbook, duration modeling, queueing systems, heavy trafficlimit, transaction data, tick data, bid-ask spread, Hawkesprocess, point process.

1This survey is based on joint work with Allan Andersen, Adrien deLarrard, Arseniy Kukanov, Sasha Stoikov, Rishi Talreja and EkaterinaVinkovskaya, whom I thank for numerous discussions and research assis-tance.

Contents1 Electronic order-driven markets 1

2 Trades, quotes and order flow 2

3 Dynamics of limit order books 4

4 Heavy-traffic limits and diffusion approximations 6

5 Price impact modeling 8

6 Connecting volatility across time scales 9

7 Perspectives and challenges 9

1. ELECTRONIC ORDER-DRIVEN MARKETS

In recent years, automated trading and dealing have largelyreplaced floor-based trading in equity markets. ElectronicCrossing Networks (ECN’s) such as Archipelago, Instinet,Brut and Tradebook providing order-driven trading systemshave captured a large share of the market. In contrast tomarkets where a market maker or specialist centralizes buyand sell orders and provides liquidity by setting bid and askquotes, these electronic platforms aggregate all outstandinglimit orders in a limit order book that is available to mar-ket participants and market orders are executed against thebest available prices, in a mechanical manner. As a resultof the ECN’s popularity, established exchanges such as theNYSE, Nasdaq, the Tokyo Stock Exchange, Toronto StockExchange, Vancouver Stock Exchange, Euronext (Paris, Am-sterdam, Brussels), and the London Stock Exchange havefully or partially adopted electronic order-driven platforms.

At the same time, the frequency of submission of ordershas increased and the time to execution of market orders onthese electronic markets has dropped from more than 25 mil-liseconds in 2000 to less than a millisecond in 2010. Asshown in Table 1, thousands of orders are submitted in a 10-second interval, resulting in frequent updates in bid and ask

Electronic copy available at: http://ssrn.com/abstract=1748022

quotes, up to hundred thousand times a day, as well as fre-quent changes in transaction prices.

Average no. of Price changesorders in 10s in 1 day

Citigroup 4469 12499General Electric 2356 7862General Motors 1275 9016

Table 1. Average number of orders in 10 seconds and numberof price changes (June 26th, 2008).

As a result, the evolution of supply, demand and price be-havior in equity markets is being increasingly recorded: thisdata is available to market participants in real time and for re-searchers in the form of high frequency databases. The anal-ysis of such high frequency data constitutes a challenge, notthe least because of their sheer volume and complexity. Thesedata provide us with a detailed view of the complex dynamicprocess through which the market ‘digests’ the inflow of sup-ply and demand to generate the price [14, 38].

The large volume of data available, the presence of sta-tistical regularities in the data and the mechanical nature ofexecution of orders makes order-driven markets interestingcandidates for statistical analysis and stochastic modelling.

Regime Time scale IssuesUltra-high ∼ 10−3 − 0.1 s Microstructure,frequency (UHF) LatencyHigh ∼ 1− 100 s TradeFrequency (HF) execution“Daily” ∼ 103 − 104 s Trading strategies,

Option hedging

Table 2. A hierarchy of time scales.

At a fundamental level, statistical analysis and modelingof high frequency data can provide insight into the interplaybetween order flow, liquidity and price dynamics [15, 55, 29]and might help bridge the gap between market microstructuretheory [12, 51, 30, 33, 45, 54]–which has provided useful in-sights by focusing on models of price formation mechanismin stylized equilibrium settings– and ‘black box’ stochasticmodels used in financial risk management, which representthe price as an exogenous random process.

At the level of applications, models of high frequency dataprovide a quantitative framework for market making [8] andoptimal execution of trades [2, 11, 50, 1]. Another obviousapplication is the development of statistical models in viewof predicting short term behavior of market variables such asprice, trading volume and order flow.

The study of high frequency market dynamics is also im-portant for risk management and regulation. Even though the

horizons traditionally considered by risk managers and regu-lators have been longer ones (typically, daily or longer), trad-ing strategies, at different frequencies may interact in a com-plex manner, leading to ripples across time scales which prop-agate from high frequency to low frequency and even leadingto possible market disruptions, as shown by the Flash Crashof May 2010 [16, 44].

2. TRADES, QUOTES AND ORDER FLOW

High frequency data in finance involve time series of pricesand quantities associated to these prices. However, when onelooks at transaction-level frequencies, a security is not char-acterized by a single price: one must distinguish betweentransaction data –which record quantities that were actuallytraded and the prices at which they were traded– and quotes–proposals to buy or sell a given quantity of an asset at a givenprice.

2.1. Transaction data

Transaction data record trades that occur on an exchange,each record being associated with a time stamp, a price andquantity (trade volume, in number of shares). Transactiondata present several particularities which differentiate themfrom financial time series sampled, say, at daily frequency[38, 39].

Price changes are discrete: they are multiples of the min-imal price change �–the ’tick’– and more than often equal toone, two or a few ticks. These price changes are not indepen-dent: the autocorrelation function of transaction price returnsis significantly negative at the first lag and then it rapidly de-creases to zero [17]. This is a typical microstructure effectwhich disappears when one considers returns at longer timescales.

Trades occur at irregular intervals: the durations betweensuccessive trades reflect the intensity of trading activity [39]:they are random and endogenous i.e. linked to the behaviorof the price and possibly to previous trading history. Withrespect to traditional time series models, this poses the addi-tional problem of modeling the durations. These durations areneither independent –they show significant autocorrelation–nor exponentially distributed: Figure 1 displays a quantile-quantile plot comparing the distribution of duration for Citi-group to an exponential distribution. The irregularity of ob-servations makes it much easier to work with continuous-timemodel, as opposed to time series models based on a fixed timestep. The sizes of these transactions –trading volume– areboth heterogeneous and strongly autocorrelated [25].

Another issue with high frequency data is the presence ofseasonality effects. Trading activity is far from uniform dur-ing the trading day and exhibits a U-shaped pattern: activityis typically highest at the market open and close, and lowestaround lunch time. Quantities related to price volatility and

trading volume also follow strong intraday seasonality pat-terns. As a result, intraday data cannot be treated as station-ary and should be first de-seasonalized before further analy-sis. This is done in practice by averaging across many days toobtain a seasonality profile or by using Fourier-based filteringmethods. Statistical models of transaction data need to jointly

Fig. 1. Quantiles of inter-event durations compared withquantiles of an exponential distribution with the same mean(Citigroup, June 2008). The dotted line represents the quan-tiles of an exponential distribution fitted to the observed dura-tions, which are found to be quite different from the empiricalquantiles.

model the durations and the price changes. It is thus naturalto model such data as a marked point process [22]. One ap-proach is to consider a stochastic model for durations and,conditional on the duration, model the price changes using aclassical ARMA or GARCH model [24, 32]. The duration �ibetween transactions i− 1 and i may be represented as

�i = i�i,

where (�i)i≥1 is a sequence of independent positive randomvariables with common distribution and E[�i] = 1 and theconditional duration i = E[�i∣ i−j , �i−j , j ≥ 1] is mod-eled as a function of past history of the process:

i = G( i−1, i−2, ..., ..; �i−1, �i−2, ..., ..).

Engle and Russell’s Autoregressive Conditional Durationmodel [24] proposes an ARMA(p, q) representation for G:

i = a0 +

p∑i=1

ak i−k +

q∑i=1

bq�j−k

where (a0, ..., ap) and (b1, ..., bq) are positive constants. TheACD-GARCH model of Ghysels and Jasiak [32] combines

Fig. 2. A limit buy order: Buy 2 at 69200.

this model with a GARCH model for the returns. Engle [25]proposes a GARCH-type model with random durations wherethe volatility of a price change may depend on the previ-ous durations. Variants and extensions are discussed in [10,39]. Such models, like ARMA or GARCH models defined onfixed time intervals, have easily computable likelihood func-tions so maximum likelihood estimation may be done numer-ically.

2.2. Quotes and order flow

Market participants can post two types of buy/sell orders. Alimit order is an order to trade a certain amount of a securityat a given price. Limit orders are posted to electronic trad-ing system and the state of outstanding limit orders posted bymarket participants can be summarized by stating the quanti-ties posted at each price level: this is known as the limit orderbook. An example of a limit order book is shown in Figure 2.The lowest price for which there is an outstanding limit sellorder is called the ask price and the highest buy price is calledthe bid price.

A market order is an order to buy/sell a certain quantityof the asset at the best available price in the limit order book.When a market order arrives it is matched with the best avail-able price in the limit order book and a trade occurs. Thequantities available at the bid/ask in the limit order book aredecreased by x when a market order of size x is executed.Figure 3 shows the evolution of a limit order book when amarket order is executed.

A limit order sits in the order book until it is either exe-cuted against a market order or it is canceled. A limit ordermay be executed very quickly if it corresponds to a price nearthe bid and the ask, but may take a long time if the marketprice moves away from the requested price or if the requestedprice is too far from the bid/ask. Alternatively, a limit ordercan be canceled at any time. On most electronic exchanges,most limit orders are cancelled shortly after being posted: formany liquid stocks on the NYSE and NASDAQ, up to 80% oflimit orders are canceled within less than a second of posting.

Fig. 3. A market sell order of 10.

Quote data contain the evolution of the bid and ask pricesand the quantities (number of shares) quoted at these prices.These quantities vary every time a limit order is posted, exe-cuted against a market order or cancelled. Figure 4 shows theintraday evolution of the ask price for Citigroup on June 26,2008.

Trades and quotes data are available for a wide rangeof stock exchanges and markets worldwide. The most wellknown trade and quote database is the TAQ database, main-tained by New York Stock Exchange (NYSE) since 1993.Trade and quote data allow to reconstruct the sequence oflimit and market orders and infer whether the transaction wasbuyer or seller initiated. The idea is to compare the tradeprice with the bid and ask price of the prevailing quote, whichis the most recent past quote. This is the basis of the Lee andReady algorithm [47] for assigning signs to trades (see also[20] for a discussion).

Most stocks are simultaneously quoted on many ex-changes and, at each point in time, there is a reference Na-tional Best Bid-Offer (NBBO) quote but the uniqueness ofthe NBBO should not hide the multiplicity of order books,with different depths and bid-ask spreads, which is the realityof this market.

Quote data are sometimes known as the ’Level I orderbook’, since they correspond to the first (i.e. best price) levelof the order book. More generally high frequency time seriesof order book data have become available in the recent years:Level II order book data give the prices and quantities forthe first 5 non-empty levels of the book on each side, whilecomplete order book data correspond to quantities at all pricelevels [21]. Order books for many stocks, including liquidones, have typically less than 5 active queues on each side ofthe order book, the rest being empty or sparsely filled [14].

3. DYNAMICS OF LIMIT ORDER BOOKS

In a limit order market, outstanding limit orders at any giventime are represented by the limit order book, which summa-

Fig. 4. Intraday evolution of the ask price: Citigroup, 26 June2008. Horizontal axis: number of events.

rizes the state of supply and demand. Not surprisingly, empir-ical studies [21, 35] show that the state of the order book con-tains information on short-term price movements so it is of in-terest to provide forecasts of various quantities conditional onthe state of the order book. Order books presents a rich paletteof statistical features [14, 29, 34, 41, 55] which are challeng-ing to incorporate in a statistical model, partly because ofthe high dimensionality of the order book and the complex-ity of its evolution. Providing analytically tractable modelswhich enable to compute and/or reproduce conditional quan-tities which are relevant for trading and intraday risk manage-ment has proven to be challenging, given the complex relationbetween order book dynamics and price behavior.

3.1. The limit order book as a queueing system

The state of the order book is modified by order book events:limit orders (at the bid or ask), market orders and cancelations[21, 20, 55]. A limit buy (resp. sell) order of size x increasesthe size of the bid (resp. ask) queue by x.

Market orders are executed against limit orders at the bestavailable price: a market order decreases of size x the cor-responding queue size by x. Limit orders placed at the bestavailable price are automatically executed against market or-ders according to a priority rule. In many electronic equitymarkets, priority is based on the time of arrival (first-come,first served). However, other priority rules exist: size-basedpriority grants priority to larger orders, while pro-rata match-ing, used in some fixed income futures markets such as EU-REX, allocates shares to limit orders proportionally to theirsize [46]. Cancellation of x orders in a given queue reducesthe queue size by x.

It is thus natural to represent a limit order book as a mul-ticlass queueing system [21, 55] i.e. a system of (interacting)

queues of buy and sell limit orders, executed against marketorders (’the server’) or cancelled before execution. Point pro-cesses and queueing analogies have been used early on formodeling order flows in securities markets [48] were con-sidered by Smith et al [55] for limit order book modeling.Queueing models allow for analytical computation of manyquantities of interest [21, 19, 4].

A queueing model of limit order book dynamics consistsin specifying the arrival rates of different types of order bookevents (limit buy, limit sell, market buy, market sell, buy/sellcancel) and the rules of execution of these orders. The evo-lution of the order book is then defined by the order flow andthe dynamics of durations, prices and bid-ask spreads are thenendogenous in these models. The arrival rates may be takenconstant –leading to a Poisson order flow [21]– or be stochas-tic processes affected by the state of the order book and byrecent order flow [4].

Cont, Stoikov and Talreja [21] study a Markovian queue-ing model in which arrivals of market orders and limit or-ders at each price level are independent Poisson processes andcompute analytically, using simple matrix computations andLaplace transform methods, the distribution of time to fill (ex-ecution time), the probability of ’making the spread’ and theconditional distribution of price moves given the state of theorder book [21]. Though the Poisson assumption is not veryrealistic –durations are neither exponentially distributed norindependent– the model is shown to capture effectively theshort-term dynamics of a limit order book.

In reality, the order flow is clustered in time: as shown inFigure 5, we observe periods with a lot of market buy ordersand periods with a lot of market sell orders [4, 40]. This ispartly due to the fact that many observed orders are in factcomponents of a larger ’parent’ order which is executed insmall blocks to minimize market impact [2]. This results in asignificant autocorrelation in durations, as well as a positivecross-correlation of arrival rates across event types (marketbuy, limit sell, etc.). These features, which are not capturedin models based on Poisson processes, may be adequatelyrepresented by a multidimensional self-exciting point process[40, 4]: in such models, the arrival rate �i(t) of an order oftype i is represented as a stochastic process whose value de-pends on the recent history of the order flow: each new orderincreases the rate of arrival for subsequent orders of the sametype (self-exciting property) and may also affect the rate ofarrival of other order types (mutually exciting property):

�i(t) = �i +

J∑j=1

�ij

∫ t

0

e−�i(t−s)dNj(s)

Here �ij measures the impact of events of type j on the rateof arrival of subsequent events of type i: as each event of typej occurs, �i increases by �ij . In between events, �i(t) decaysexponentially at rate �i. Maximum likelihood estimation ofthis model on TAQ data [4] shows evidence of self-exciting

and mutually exciting features in order flow: the coefficients�ij are all significantly different from zero and positive, with�ii > �ij for j ∕= i. Andersen et al [4] enhance this basicmodel by incorporating dependence of parameters �, � on thebid-ask spread.

Like models based on Poisson order flow, these modelsallows for analytical computations of many quantities, with-out recourse to simulation. The availability of such analyticalformulae is a key feature when using such models in real-timeapplications.

Fig. 5. Changes in the size of the ask queue: each data pointrepresents one order or cancellation. Positive values representsizes of limit orders, negative values represent market ordersor cancellation. Citigroup, June 26, 2008.

3.2. Queuing models for Level-1 order books

Empirical studies of limit order markets suggest that the ma-jor component of the order flow occurs at the (best) bid andask price levels (see e.g. [13]). Furthermore, studies on theprice impact of order book events show that the net effect oforders on the bid and ask queue sizes is the main factor drivingprice variations [20]. These observations motivate a reduced-form modeling approach in which, instead of modeling theorder queues at all price levels, one focuses on queue sizes atthe best bid and ask [19]. In this approach one focuses on thedynamics of the state variables (Sbt , S

at , q

bt , q

at ) where

∙ the bid price Sbt and the ask price Sat ,

∙ the size of the bid queue qbt representing the outstandinglimit buy orders at the bid, and

∙ the size of the ask queue qat representing the outstand-ing limit sell orders at the ask.

Figure 6 summarizes this representation. In principle, the

Fig. 6. Level-I representation of a limit order book: in thisrepresentation, one keeps track of the quantities of sharesavailable at the bid (best buy price) and the ask (best sellprice).

evolution of the size of the queues at the best quotes dependson the deeper levels of the order book: indeed, once the bid(resp. the ask) queue is depleted, the price will move to thequeue at the next level, which is queue at the next level be-low (resp. above) the current bid (ask). The new queue sizethus corresponds to what was previously the number of or-ders sitting at the second-best price. Instead of keeping trackof these queues (and the corresponding order flow) at all pricelevels (as in [21, 55]), Cont & Larrard [19] propose a model inwhich the bid/ask queue sizes (qbt , q

at ) are ’renewed’ at each

price change, i.e. every time one of the queues is depleted.More precisely,

∙ if the ask queue is depleted at t then (qbt , qat ) is a vari-

able with distribution f , where f(x, y) represents theprobability of observing (qbt , q

at ) = (x, y) right after a

price increase.

∙ if the bid queue is depleted at t then (qbt , qat ) is a vari-

able with distribution f , where f(x, y) represents theprobability of observing (qbt , q

at ) = (x, y) right after a

price decrease.

The distributions f, f may be estimated from empirical data:an example is shown in Figure 7. They summarize the interac-tion of the queues at the best bid/ask levels with the rest of theorder book, viewed here as a ’reservoir’ of limit orders. Thus(qbt , q

at ) is a random process in the positive orthant, ’renewed’

every time it hits the axes. This approach considerably sim-plifies the model, enhances tractability and makes it easier toestimate parameters.

Between two hitting times, one can specify various dy-namics for the queues. Cont & Larrard [19] consider the sim-plest specification, namely a Markovian model with Poisson

Fig. 7. Empirical (joint) distribution of bid and ask queuesizes after a price move (Citigroup, June 2008) [19]. Unit:blocks of 100 shares.

arrivals of orders. This model allows to obtain analytical ex-pressions for various quantities of interest such as the distri-bution of the duration between price changes, the distributionand autocorrelation of price changes, and the probability of anupward move in the price, conditional on the state of the or-der book. Interestingly, these results provide reasonable fits toempirical data [19], showing that such low-dimensional mod-els may be suitable for many applications. Figure 8 showsan example of such a result, comparing theoretical transi-tion probabilities for the price computed in this reduced-formmodel with empirical transition probabilities estimated fromhigh frequency data.

4. HEAVY-TRAFFIC LIMITS AND DIFFUSIONAPPROXIMATIONS

Analytically tractable models of order book dynamics maycontain assumptions which are not statistically realistic: du-rations between orders are neither independent nor exponen-tially distributed, and various order types exhibit strong de-pendence [4]. In addition, the order flow exhibits seasonalityas well as dependence with respect to price and spread behav-ior [14].

While a model incorporating all these features may be toocomplex to study analytically, two remarks show that the sit-uation is not as hopeless as it may seem at first glance.

First, as shown in Table 1, most applications involve thebehavior of prices over time scales an order of magnitudelarger than the typical inter-event duration: for example,in optimal trade execution the benchmark is the Volumeweighted average price (VWAP) computed over a day or aan hour: over such time scales much of the microstructuraldetails of the market are averaged out. Second, as notedin Table 1, in liquid equity markets the number of events

Fig. 8. Conditional probability of a price increase, as a func-tion of the bid and ask queue size, computed using a Marko-vian queuing model for the Level-1 limit order book [19](solid curve) compared with transition frequencies for Citi-group tick-by-tick data on June 26, 2008 (points).

affecting the state of the order book over such time scalesis quite large, of the order of hundreds or thousands. Thetypical duration �L = 1/� (resp. �M = 1/�) between limitorders (resp. market orders and cancelations) is typically0.001 − 0.01 ≪ 1 (in seconds). These observations showthat it is relevant to consider asymptotic methods in whichthe rate of arrival of orders is large, similar to those usedin the study of heavy-traffic limits in queueing theory [57].In this limit, the possibly complex discrete dynamics of thequeueing system is approximated by a simpler system witha continuous state space, which can be either described by asystem of ordinary differential equations (in the ’fluid limit’,where random fluctuations in queue size vanish) or a systemof stochastic differential equations (in the ’diffusion limit’where random fluctuations dominate). Intuitively, the fluidlimit corresponds to the regime of the law of large numbers,where random fluctuations average out and the limit is de-scribed by average queue size, whereas the diffusion limitcorresponds to the regime of the central limit theorem, wherefluctuations in queue size are asymptotically Gaussian. Therelevance of each of these asymptotic regimes is, of course,not a matter of taste but an empirical question which dependson the behavior of order flow in the system.

As argued in [18], for most liquid stocks, while the rateof arrival of market orders and limit orders is large, the im-balance between limit orders, which increase queue size, andmarket orders and cancels, which decrease queue size, is anorder of magnitude smaller: over, say, a 30 minute interval,one observes an imbalance ranging from 1 to 5 % of orderflow. In other words, over a time scale �1 ≫ �L, �M (say,1 minute) a large number N ∼ ��1 of events occur, but thebid/ask imbalance accumulating over the same interval is of

order√N ≪ N i.e. (� − �) is small, of the order 1/

√N .

This corresponds to a situation where limit order accumulateand disappear roughly at the same rate. In this regime, thequeues follow a diffusive behavior and it is relevant to con-sider the diffusion limit of the limit order book. When the se-quences of order sizes at the bid and the ask (V ai , V

bi , i ≥ 1)

and inter-event durations (�ai , �bi , i ≥ 1) are weakly depen-

dent covariance-stationary sequences, the rescaled order bookprocess converges weakly [18]

(qa(nt)√

n,qb(nt)√

n)t≥0

d⇒(Qt)t≥0

to a Markov process (Qt)t≥0 diffusing in the quarter-plane

{(x, y), x ≥ 0, y ≥ 0}

which is “renewed” every time it hits one of the axes. In thecase of a balanced order book where the arrival rate of limitorders matches on average their depletion rate by market or-ders and cancels and E[V ia ] = E[V ib ] = 0, the process Qt

∙ behaves like a planar Brownian motion with drift b andcovariance matrix(

�av2a �vavb

√�a�b

�vavb√�a�b �bv

2b

)(1)

on the interior of the orthant {(x, y), x > 0, y > 0},

∙ is reinitialized according to F each time it hits the x-axis and

∙ is reinitialized according to F each time it hits the y-axis.

Qt is an example of ’regulated Brownian motion’ [36], a no-tion introduced in the context of heavy traffic limits of multi-class queueing systems. Here � is the ’tick’ size,

∙ 1/�a = limn→∞(�a1 + ...+ �an)/n is the average dura-tion between events at the ask,

∙ v2a = E[(V a1 )2] + 2∑∞i=2 Cov(V a1 , V

ai ) represents the

variance of event sizes at the ask,

∙ 1/�b = limn→∞(� b1 + ...+ � bn)/n is the average dura-tion between events at the bid,

∙ v2b = E[(V b1 )2] + 2∑∞i=2 Cov(V b1 , V

bi ) represents the

variance of event sizes at the ask,

∙ F (resp. F ) is the (rescaled) distribution of (qa, qb)after a price increase (resp. decrease), and

∙ � =1

vavb

(cov[V a1 V

b1 ] + 2

∑∞i=1 cov[V a1 V

bi ] + cov[V b1 V

ai ])

measures the correlation between order sizes at the bidand at the ask. If order sizes at the bid and ask aresymmetric and uncorrelated then � = 0. Empiricallyones finds that � < 0 for equity order books [18].

This diffusion approximation is analytically tractable and al-lows to compute analytically [18] many quantities such asthe duration until the next price change, the probability ofan increase in the price, the distribution of the time to exe-cution,... conditionally on the state of the order book. Forexample, in the symmetric case �a = �b, va = vb, the proba-bility pup(x, y) that the next price move is an increase, givena queue of x shares on the bid side and y shares on the askside, has a simple expression [18]:

pup(x, y) =1

2−

arctan(√

1+�1−�

y−xy+x )

2 arctan(√

1+�1−� )

(2)

which only depends on the size x of the bid queue, the size yof the ask queue, and the correlation � between event sizes atthe bid and the ask.

Beyond their computational aspect, these analytical re-sults shed light on the relation between order flow and pricedynamics. A nice application is given by Avellaneda, Reedand Stoikov [9]: using formula (2) in the case � = −1, theyinvert the ’effective’ size (x, y) of the queues from the empir-ical transition probabilities of the price and compare it to thevisible queue size: the difference, which they call ‘hidden liq-uidity’, is shown to account for a significant portion of marketdepth for some US stocks.

5. PRICE IMPACT MODELING

A particularly important issue for applications is the impactof orders on prices: the optimal liquidation of a large block ofshares, given a fixed time horizon, crucially involves assump-tions on price impact (see Bertsimas and Lo [11], Almgrenand Chriss [2], Obizhaeva and Wang [50], Alfonsi and Schied[1]). Of course, any model for the dynamics of the (full) or-der book implies a model of price impact. However, even insimple models for order book dynamics the price impact maybe difficult to characterize. Hence, the typical approach is toinvestigate the relation between price changes and order flowdirectly using high frequency data.

Various models for price impact have been proposed inthe literature but there is little agreement on how to modelit: price impact has been described by various authors as lin-ear, non-linear, square root, temporary, instantaneous, per-manent or transient. The only consensus seems to be theintuitive notion that imbalance between supply and demandmoves prices.

The empirical literature on price impact has primarilyfocused on trades. Starting with Hasbrouck [37], empiri-cal studies on public data [28, 31, 37, 42, 43, 56, 53] haveinvestigated the relation between the direction and sizes oftrades and price changes and typically conclude that the priceimpact of trades is an increasing, concave (“square root”)function of their size.

Studies based on proprietary data [26, 3, 49] are able tomeasure impact of “parent orders” gradually executed overtime and give a different picture. Using large-cap US eq-uity orders executed by Citigroup Equity Trading, Almgrenet al [3] find evidence for a nonlinear dependence of tem-porary price impact ΔS as function of trade size V , of theform ΔS ∼ V 3/5 across the range of trade sizes consid-ered, but reject the ‘square root’ law. Obizhaeva [49] studiesthe short run price-volume relation using a proprietary dataset of portfolio transitions using a methodology which cor-rects for the endogeneity of trading decisions and finds that,in the short run, purchases of stocks induce permanent priceincreases, the magnitude of which is proportional to the sizeof the trade. Moreover, if volume is expressed as a percent-age of average daily volume and returns are expressed as apercentage of daily volatility, the proportionality constant isrelatively stable across stocks.

Fig. 9. Price impact of a market order.

This focus on trades leaves out the information in quotes,which provide a more detailed picture of price formation [27,23, 20]. In fact, knowledge of the limit order book allowsto estimate the short term price impact of incoming orders.This can be understood by considering the following exampleof an order book with constant number of shares D per level(see Figure 9). If, during a short time interval [t, t + Δ], thevolume of incoming limit orders (resp. market orders andcancellations) at the bid are L (resp. M,C) shares, then thenet change in the number of limit orders at the bid is L−M−C. If the depth of the order book is D shares (on average) ateach level then the change in the bid price will be

S(t+ Δ)− S(t) = �

[L−M − C

D

](3)

where � is the tick size and [.] denotes the integer part. Thisexample, illustrated in Figure 9, shows that the change in thebid (resp. ask) price is determined by the order flow imbal-ance L − M − C at the bid (resp. the ask), defined as thesum of limit order minus the market orders and cancellationsduring the interval [t, t+ Δ] [20]:

OFI(t, t+ Δ) = L−M − C (4)

Motivated by this example, Cont, Kukanov and Stoikov [20]investigate a model in which high frequency price changes are

driven by order flow imbalance:

S(t+ Δ)− S(t)

�= c

OFI(t, t+ Δ)

D(t)+ �(t), (5)

where � is the tick size, �(t) is a (white) noise term andD(t) isa measure of order book depth (number of limit orders at thebid/ask) . Empirical analysis of 1 second and 10 second mid-price changes using the TAQ database for US stocks showempirical evidence for the linear price impact model [20]: cis estimated to be typically between 0.1 and 1.

The linear model (5) is quite different from models ofprice impact that consider only the size of trades [31, 37, 43,56, 52, 53]. Instead of looking at price impact of trades as a(nonlinear) function of trade size, it shows that all order bookevents (including trades) have a linear price impact, equal toc/D(t) on average, and this impact can be summarized byaggregating all events into a single variable, the order flowimbalance. This representation also implies that price impacthas strong intraday seasonality, which follows the seasonalitypatterns in order book depth; this is indeed shown to be thecase empirically [20].

6. CONNECTING VOLATILITY ACROSS TIMESCALES

While most econometric models of price volatility were ini-tially developed with a focus on daily returns, with the adventof intraday and high frequency trading there is an increasingneed for developing realistic statistical models for intradayvolatility prediction and risk management. High-frequencydata are also of interest for the estimation of volatility at dailylevel: as shown by Andersen and Bollerslev [6, 5, 7], real-ized variance computed from high frequency data yields abetter estimator of daily volatility than estimates computedfrom daily returns.

One approach is to combine time series models -such asGARCH models - developed for daily returns, with a modelfor durations between trades–such as the ACD model [24]:this allows to incorporate the information on variable rates ofactivity, peaks in trading activity being represented by shorterdurations [25, 32]. This approach has been mostly applied totransaction data but similar approach may also be consideredfor quotes [27].

Another approach for establishing a link between volatil-ity and order flow is to start from a model for order book dy-namics, and compute the diffusion limit of the price processwhich describes the diffusive behavior of the price at longertime scales. This idea is implemented in [19, 18] by provinga functional central limit theorem for the order book modelsdescribed above. This result allows in particular to computethe variance of price changes over a time scale T ≫ �0 muchlarger than the typical inter-event duration. For example, in aMarkovian model for a symmetric Level 1 order book, Cont

& Larrard [19] derive the following relation between the stan-dard deviation � of price increments S(t+ T )−S(t) and thestatistics of order flow:

� =v �

D(F )

√��T

�0= v��

√�T

D(F ), (6)

where � = 1/�0 is the order flow in shares per unit time,F is the distribution of queue sizes after a price change and

D(F ) =√∫

x y dF (x, y) is a measure of order book depth:it is the geometric mean of the queue size at the bid and theask after a price change. Here � is the familiar ratio of acircle’s perimeter to its diameter. In particular, this relationsuggests that the high frequency properties of order flow con-tribute to price volatility mainly through � and D(F ): forstocks that trade at the same frequency, volatility increaseswith the ratio �/D(F ). This prediction is empirically verifiedfor major US stocks [19]. Similar relation may be obtainedfor non-Markovian and asymmetric models, see [18]

7. PERSPECTIVES AND CHALLENGES

Existing studies have only scratched the surface of high fre-quency financial data: much remains to be done towards abetter analysis and understanding of intraday dynamics of fi-nancial markets and the development of operational models.

These studies have mostly focused on equity marketsand, to a lesser extent, futures markets and foreign exchangemarkets. Further studies are necessary on other markets –in particular fixed income markets and markets for listedderivatives– which have their own features.

Also, almost all existing models focus on single-assetmarkets, whereas, in practice, most applications –in trading,risk management or regulation– involve multiple assets andportfolios. Extending univariate models to a multidimen-sional setting may not be straightforward: for example, inmodels where durations are random, keeping track of syn-chronicity across different assets is not trivial to implement.For this reason, whereas single-asset models go to a greatlength to model randomness in durations, multiasset model-ing has mostly focused on modeling of fixed-interval returns,the most well-known being Hasbrouck’s Vector AutoRegres-sive (VAR) [38] model. Yet, the connection between orderflows across different assets is particularly relevant whenconsidering indexes, futures and options, which cannot beconsidered in isolation and in which supply and demand aredriven by events occurring in another market -the market forthe components of the index or the underlying asset of thederivative. Such multi-asset limit order book models are veryrelevant for applications and their development represents achallenge.

More remains to be done to bridge the gap between mi-crostructure models –which hinge on various behavioral as-sumptions about market participants strategies– and stochas-tic models, which are more operational and data-driven. The

theoretical input from microstructure models may be used todesign new stochastic models whose ingredients will havea better economic interpretation. The study of heavy-trafficlimits and diffusion limits of discrete order book modelsseems to be a promising direction of research. This approachrestores analytical tractability into the models and allows toconnect microstructural features to statistical properties ofprices and order flow, which are observable and may be usedin the empirical validation of models.

We have focused in this survey on stochastic models andrelated statistical techniques for high frequency data. Othertechniques, though less present in the research literature –machine learning methods, wavelet-based methods, nonpara-metric pattern recognition methods, large scale data miningtechniques, methods from cryptography– have been exten-sively applied by practitioners for prediction and pattern de-tection in high frequency data. These methods have con-tributed in some cases to the development of successful trad-ing algorithms, which may or may not be generalizable toother data sets, but it remains to be seen whether they can alsoprovide interesting fundamental insights into the dynamics ofsupply, demand and price in financial markets.

REFERENCES

[1] A. ALFONSI, A. SCHIED, AND A. SCHULZ, Opti-mal execution strategies in limit order books with gen-eral shape function, Quantitative Finance, 10 (2010),pp. 143–157.

[2] R. ALMGREN AND N. CHRISS, Optimal execution ofportfolio transactions, Journal of Risk, 3 (2000), pp. 5–39.

[3] R. ALMGREN, C. THUM, E. HAUPTMANN, ANDH. LI, Market impact, Risk, 18 (2005), p. 57.

[4] A. ANDERSEN, R. CONT, AND E. VINKOVSKAYA, Apoint process model for the high-frequency dynamics ofa limit order book, Financial Engineering Report 2010-08, Columbia University, 2010.

[5] T. ANDERSEN AND T. BOLLERSLEV, Answering theskeptics: Yes, standard volatility models do provide ac-curate forecasts, International Economic Review, 39(1998), pp. 885–905.

[6] T. ANDERSEN AND T. BOLLERSLEV, Towards a uni-fied framework for high and low frequency returnvolatility modeling, Statistica Neerlandica, 52 (1998),pp. 273–302.

[7] T. ANDERSEN, T. BOLLERSLEV, AND N. MEDDAHI,Correcting the errors: Volatility forecast evaluation us-ing high-frequency data and realized volatilities, Econo-metrica, 73 (2005), pp. 279–296.

[8] M. AVELLANEDA AND S. STOIKOV, High-frequencytrading in a limit order book, Quantitative Finance, 8(2008), pp. 217–224.

[9] M. AVELLANEDA, S. STOIKOV, AND J. REED, Fore-casting prices from level-i quotes in the presence of hid-den liquidity, working paper, 2010.

[10] L. BAUWENS AND N. HAUTSCH, Modelling financialhigh frequency data using point processes, in Hand-book of Financial Time series, T.G. Andersen et al., ed.,Springer, 2009, pp. 953–979.

[11] D. BERTSIMAS AND A. LO, Optimal control of execu-tion costs, Journal of Financial Markets, 1 (1998), pp. 1–50.

[12] B. BIAIS, L. GLOSTEN, AND C. SPATT, Market mi-crostructure: A survey of microfoundations, empiricalresults and policy implications, Journal of FinancialMarkets, 8 (2005), pp. 217–264.

[13] B. BIAIS, P. HILLION, AND C. SPATT, An empiricalanalysis of the order flow and order book in the parisbourse, Journal of Finance, 50 (1995), pp. 1655–1689.

[14] J.-P. BOUCHAUD, D. FARMER, AND F. LILLO, Howmarkets slowly digest changes in supply and demand,in Handbook of Financial Markets: Dynamics and Evo-lution, T. Hens and K. Schenk-Hoppe, eds., Elsevier:Academic Press, 2008.

[15] J.-P. BOUCHAUD, M. MEZARD, AND M. POTTERS,Statistical properties of stock order books: empiricalresults and models, Quantitative Finance, 2 (2002),pp. 251–256.

[16] CFTC AND SEC, Findings regarding the market eventsof May 6, 2010, Report to the joint advisory committeeon emerging regulatory issues, CFTC, 2010.

[17] R. CONT, Empirical properties of asset returns: styl-ized facts and statistical issues, Quantitative Finance, 1(2001), pp. 223–236.

[18] R. CONT AND A. DE LARRARD, Linking volatility withorder flow: heavy traffic approximations and diffusionlimits of order book dynamics, working paper, 2010.

[19] R. CONT AND A. DE LARRARD, Price dy-namics in a markovian limit order market,http://ssrn.com/abstract=1735338, 2010.

[20] R. CONT, A. KUKANOV, AND S. STOIKOV, The priceimpact of order book events, working paper, ColumbiaUniversity, 2010.

[21] R. CONT, S. STOIKOV, AND R. TALREJA, A stochasticmodel for order book dynamics, Operations Research,58 (2010), pp. 549–563.

[22] R. CONT AND P. TANKOV, Financial Modelling withJump Processes, Chapman & Hall/CRC, 2003.

[23] Z. EISLER, J.-P. BOUCHAUD, AND J. KOCKELKOREN,The price impact of order book events: market orders,limit orders and cancellations, working paper, 2010.

[24] R. ENGLE AND J. RUSSELL, Autoregressive condi-tional duration: a new model for irregularly-spacedtransaction data., Econometrica, v66 (1998), pp. 1127–1162.

[25] R. F. ENGLE, The Econometrics of Ultra-High Fre-quency Data, Econometrica, 68 (2000), pp. 1–22.

[26] R. F. ENGLE, R. FERSTENBERG, AND J. RUSSELL,Measuring and modeling execution cost and risk. NYUWorking Paper No. FIN-06-044, 2006.

[27] R. F. ENGLE AND A. LUNDE, Trades and Quotes: ABivariate Point Process, Journal of Financial Economet-rics, 1 (2003), pp. 159–188.

[28] M. EVANS AND R. LYONS, Order flow and exchangerate dynamics, Journal of Political Economy, 110(2002), p. 170.

[29] J. D. FARMER, L. GILLEMOT, F. LILLO, S. MIKE,AND A. SEN, What really causes large price changes?,Quantitative Finance, 4 (2004), pp. 383–397.

[30] T. FOUCAULT, O. KADAN, AND E. KANDEL, Limit or-der book as a market for liquidity, Review of FinancialStudies, 18 (2005), pp. 1171–1217.

[31] X. GABAIX, P. GOPIKRISHNAN, V. PLEROU, ANDH. STANLEY, A theory of power-law distributions in fi-nancial market fluctuations, Nature, 423 (2003), p. 267.

[32] E. GHYSELS AND J. JASIAK, GARCH for irregularlyspaced financial data: the ACD-GARCH model, Stud-ies in Nonlinear Dynamics and Econometrics, 2 (1998),pp. 133–149.

[33] L. GLOSTEN, Is the limit order book inevitable?, Jour-nal of Finance, 49 (1994), pp. 1127–1161.

[34] C. GOURIEROUX, J. JASIAK, AND G. LE FOL, Intra-day market activity, Journal of Financial Markets, 2(1999), pp. 193–226.

[35] L. E. HARRIS AND V. PANCHAPAGESAN, The infor-mation content of the limit order book: evidence fromNYSE specialist trading decisions, Journal of FinancialMarkets, 8 (2005), pp. 25–67.

[36] J. HARRISON AND V. NGUYEN, Brownian models ofmulticlass queueing networks: Current status and openproblems, Queueing Systems, 13 (1993), pp. 5–40.

[37] J. HASBROUCK, Measuring the information content ofstock trades, Journal of Finance, 46 (1991), pp. 179–207.

[38] J. HASBROUCK, Empirical Market Microstructure, Ox-ford University Press, 2007.

[39] N. HAUTSCH, Modelling irregularly spaced financialdata, vol. 539 of Lecture Notes in Economics and Math-ematical Systems, Springer, 2004.

[40] P. HEWLETT, Clustering of order arrivals, price impactand trade path optimisation, working paper, OCIAM,2006.

[41] B. HOLLIFIELD, R. A. MILLER, AND P. SANDAS, Em-pirical analysis of limit order markets, Review of Eco-nomic Studies, 71 (2004), pp. 1027–1063.

[42] D. KEIM AND A. MADHAVAN, The upstairs marketfor large-block transactions: Analysis and measurementof price effects, Review of Financial Studies, 9 (1996),pp. 1–36.

[43] A. KEMPF AND O. KORN, Market depth and order size,Journal of Financial Markets, 2 (1999), p. 29.

[44] A. KIRILENKO, A. KYLE, M. SAMADI, ANDT. TUZUN, The flash crash: The impact of high fre-quency trading on an electronic market, ssrn workingpaper, 2010.

[45] A. KYLE, Continuous auctions and insider trading,Econometrica, 53 (1985), p. 1315.

[46] J. LARGE AND J. FIELD, Pro-Rata Matching in One-Tick Markets, tech. report, Oxford Man Institute.

[47] C. LEE AND M. READY, Inferring trade direction fromintraday data, Journal of Finance, 46 (1991), pp. 733–746.

[48] H. MENDELSON, Market behavior in a clearing house,Econometrica, (1982), pp. 1505–1524.

[49] A. OBIZHAEVA, Portfolio transitions and stock pricedynamics, working paper, University of Maryland,2009.

[50] A. OBIZHAEVA AND J. WANG, Optimal trading strat-egy and supply/demand dynamics, working paper, MIT,2006.

[51] C. PARLOUR, Price dynamics in limit order markets,Review of Financial Studies, 11 (1998), pp. 789–816.

[52] V. PLEROU, P. GOPIKRISHNAN, X. GABAIX, ANDH. STANLEY, Quantifying stock-price response to de-mand fluctuations, Physical Review E, 66 (2002),p. 027104.

[53] M. POTTERS AND J. BOUCHAUD, More statisticalproperties of order books and price impact, Physica A,324 (2003), p. 133 140.

[54] I. ROSU, A dynamic model of the limit order book, Re-view of Financial Studies, 22 (2009), pp. 4601–4641.

[55] E. SMITH, J. D. FARMER, L. GILLEMOT, ANDS. KRISHNAMURTHY, Statistical theory of the contin-uous double auction, Quantitative Finance, 3 (2003),pp. 481–514.

[56] N. TORRE AND M. FERRARI, The Market ImpactModel, BARRA, 1997.

[57] W. WHITT, Stochastic Process Limits, Springer Verlag,2002.