Electronic copy available at: http://ssrn.com/abstract=2044825

Backtesting

Peter Christoffersen∗

Desautels Faculty of Management, McGill University,

Copenhagen Business School and CREATES

1001 Sherbrooke Street West

Montreal, Canada H3A 1G5

peter.christoffersen@mcgill.ca

Tel: (514) 398-2869

Fax: (514) 398-3876

November 19, 2008

Abstract

This chapter surveys methods for backtesting risk models using the ex ante risk measure forecasts

from the model and the ex post realized portfolio profit or loss. The risk measure forecast can

take the form of a V aR, an Expected Shortfall, or a distribution forecast. The backtesting

surveyed in this chapter can be seen as a final diagnostic check on the aggregate risk model

carried out by the risk management team that constructed the risk model, or they can be used

by external model-evaluators such as bank supervisors. Common for the approaches suggested

is that they only require information on the daily ex ante risk model forecast and the daily ex

post corresponding profit and loss. In particular, knowledge about the assumptions behind the

risk model and its construction is not required.

Keywords: Value-at-Risk, expected shortfall, distribution, forecasting, model evaluation, test-

ing, historical simulation.

∗Prepared for the Encyclopedia of Quantitative Finace edited by Rama Cont and published by John Wiley &

Sons, Ltd. I am also affiliated with CIRANO and CIREQ. I am grateful for financial support from FQRSC, IFM2

and SSHRC and from the Center for Research in Econometric Analysis of Time Series, CREATES, funded by the

Danish National Research Foundation.

1

Electronic copy available at: http://ssrn.com/abstract=2044825

1 Introduction

The term “backtesting” is used in several different ways in finance. Most commonly backtesting

denotes either 1) an assessment of the hypothetical historical performance of a suggested trading

strategy, or 2) the evaluation of financial risk models using historical data on risk forecasts and

profit and loss realizations. This chapter is about the evaluation of risk models.

The objective is to consider the daily ex ante risk measure forecasts from a model and test it

against the daily ex post realized portfolio loss. The risk measure forecast could take the form

of a V aR, an Expected Shortfall, or a distributional forecast. The goal is to be able to backtest

any of these risk measures of interest. The backtesting procedures developed in this chapter can

be seen as a final diagnostic check on the risk model carried out by the risk management team

that constructed the risk model, or they can be used by external model-evaluators such as bank

supervisors.

Evidence on actual bank V aRs and their backtesting performance can be found in Berkowitz

and O’Brien (2002), Jorion (2002), Perignon, Deng and Wang (2006), Perignon and Smith (2006)

and Smith (2007). The regulatory considerations involved in backtesting are detailed by the Basle

Committee on Banking Supervision (1996a, 1996b, 2004). Lopez (1999) analyze the regulatory

approach to backtesting and Campbell (2007) provides a survey of backtesting that includes a

discussion of regulatory considerations. Backtesting of credit risk models is investigated in Lopez

and Saidenberg (2000).

This chapter first establishes procedures for backtesting the Value-at-Risk (V aR) metric (eqf15/004,

eqf15/008). Second, we consider increasing backtesting power by using explanatory variables to

backtest the V aR. Third, we consider increasing power by backtesting risk measures other than

V aR and we discuss various other issues in backtesting.

2 Backtesting V aR

By now Value-at-Risk (V aR) is by far the most popular portfolio risk measure used by risk man-

agement practitioners. The V aR revolution in risk management was triggered by JP Morgans

RiskMetrics approach launched in 1994. Supervisory authorities immediately recognized the need

for methods to backtest V aR and the first research on backtesting was published soon after in

Kupiec (1995) and Hendricks (1996). Christoffersen (1998) extended Kupiec’s test of unconditional

V aR coverage to tests of conditional V aR coverage. These concepts will be defined shortly.

2.1 Defining the Hit Sequence

First, define V aRpt+1 to be a number constructed on day t such that the portfolio losses on day

t+1 will only be larger than the V aRpt+1 forecast with probability p. If we observe a time series of

past ex-ante V aR forecasts and past ex-post losses, PL, we can define the “hit sequence” of V aR

2

violations as

It+1 =

½1, if PLt+1 > V aRp

t+1

0, if PLt+1 < V aRpt+1

. (1)

The hit sequence returns a 1 on day t + 1 if the loss on that day is larger than the V aR number

predicted in advance for that day. If the V aR is not exceeded (or violated) then the hit sequence

returns a 0. When backtesting the risk model, we construct a sequence {It+1}Tt=1 across T days

indicating when the past violations occurred.

2.2 The Null Hypothesis

If we are using the perfect V aR model then given all the information available to us at the time

the V aR forecast is made, we should not be able to predict whether the V aR will be violated. We

should be expecting a 1 in the hit sequence with probability p and we should be expecting a 0 with

probability 1− p and the occurrences of the hits should be random over time.

We will say that a risk model has correct unconditional V aR coverage if Pr (It+1 = 1) = p and

we will say that a risk model has correct conditional V aR coverage if Prt (It+1 = 1) = p. Roughly

speaking, correct unconditional V aR coverage just means that the risk model delivers V aR hits

with probability p on average across the days. Correct conditional V aR coverage means that the

risk model gives a V aR hit with probability p on every day given all the information available on

the day before. Note that correct conditional coverage implies correct unconditional coverage but

not vice versa.

We can think of V aR backtesting as testing the hypothesis

H0 : It+1 ∼ i.i.d. Bernoulli(p).

If p is one half, then the i.i.d. Bernoulli distribution describes the distribution of getting a a

head when tossing a fair coin. When backtesting risk models, p will not be one half but instead

on the order of 0.01 or 0.05 depending on the coverage rate of the V aR. The hit sequence from a

correctly specified risk model should look like a sequence of random tosses of a coin which comes

up heads 1% or 5% of the time depending on the V aR coverage rate.

Note that in general, the expected value of a binary sequence is simply the probability of getting

a 1

Et [It+1] = Prt (It+1 = 1) 1 + Prt (It+1 = 0) 0 = Prt (It+1 = 1) ≡ πt+1|t

We will denote this conditional hit probability by πt+1|t and its unconditional counterpart is defined

E [It+1] ≡ π.

We can therefore construct the following null hypothesis of correct conditional coverage for the

hit sequence from a V aR model

H0 : Et [It+1] = πt+1|t = p

namely that the conditionally expected value of the hit sequence at time t+ 1 given all the infor-

mation available at time t is the promised V aR coverage rate p.

3

2.3 Unconditional Coverage Testing

We first want to test if the unconditional probability of a violation in the risk model, π, is signifi-

cantly different from the promised probability, p.We call this the unconditional coverage hypothesis.

We can write H0 : E [It] ≡ π = p.

The expected value of the hit sequence can be estimated by the sample average, π̂ = 1T

PTt=1 It =

T1/T , where T1 is the number of 1s in the sample. Note that if the null hypothesis is true we have

that E [π̂] = 1T

PTt=1 It = p. Assuming that the observations on the hit sequence are independent

over time, the variance of the π̂ estimate is 1T V ar (It) where V ar (It) can be estimated as the sample

variance of the hit sequence. The hypothesis that E [It] = p can therefore be tested in a simple

means test

MT =√T

π̂ − ppV ar (It)

∼ N(0, 1) (2)

We can also implement the unconditional coverage test as a likelihood ratio test. For this we

write the likelihood of an i.i.d. Bernoulli(π) hit sequence

L (π) =TYt=1

(1− π)1−It+1πIt+1 = (1− π)T0 πT1

where T0 and T1 are the number of 0s and 1s in the sample. We can easily estimate π from

π̂ = T1/T , that is the observed fraction of violations in the sequence. Plugging the ML estimates

back into the likelihood function gives the optimized likelihood as

L (π̂) = (1− T1/T )T0 (T1/T )

T1 .

Under the unconditional coverage null hypothesis that π = p, where p is the known V aR

coverage rate, we have the likelihood

L (p) =TYt=1

(1− p)1−It+1pIt+1 = (1− p)T0 pT1 .

And we can check the unconditional coverage hypothesis using a likelihood ratio test

LRuc = −2 ln [L (p) /L (π̂)] ∼ χ21.

Asymptotically, that is as the number of observations, T, goes to infinity, the test will be distributed

as a χ2 with one degree of freedom.

The choice of significance level comes down to an assessment of the costs of making two types of

mistakes: We could reject a correct model (Type I error) or we could fail to reject (that is accept)

an incorrect model (Type II error). Increasing the significance level implies larger Type I errors

but smaller Type II errors and vice versa. In academic work, a significant level of 1%, 5% or 10%

is typically used. In risk management, the Type II errors may be very costly so that a significance

level of 10% may be appropriate.

4

Often we do not have a large number of observations available for backtesting, and we certainly

will typically not have a large number of violations, T1, which are the informative observations.

It is therefore often better to rely on Monte Carlo simulated P-values rather than those from the

χ2 distribution. Christoffersen and Pelletier (2004) discuss how to implement the Dufour (2006)

Monte Carlo P-values in a backtesting setting.

2.4 Independence Testing

There is strong evidence of time-varying volatility in daily asset returns as surveyed in Andersen

et al (2005). If the risk model ignores such dynamics then the V aR will react slowly to changing

market conditions and V aR violations will appear clustered in time. Pritsker (2001) illustrates this

problem using V aRs computed from Historical Simulation. If the V aR violations are clustered then

the risk manager can essentially predict that if today has a V aR hit, then there is a probability

larger than p of getting a hit tomorrow which violates that the V aR is based on an adequate model.

This is clearly not satisfactory. In such a situation the risk manager should increase the V aR in

order to lower the conditional probability of a violation to the promised p.

The most common way to test for dynamics in time series analysis is to rely on the autocor-

relation function and the associated Portmanteau or Ljung-Box type tests. We can implement

this approach for backtesting as well. Let γk be the autocorrelation at lag k for the hit sequence.

Plotting γk against k for k = 1, ...,m will give a visual impression of the correlation of between a

hit in one of the last m trading days and a hit today. The Ljung-Box test provides a formal check

of the null hypothesis that all of the first m autocorrelations are zero against the alternative that

any of them is not. The test is easily constructed as

LB(m) = T (T + 2)mXk=1

γ2kT − k

∼ χ2m (3)

where χ2m denotes the chi-squared distribution withm degrees of freedom. Berkowitz, Christoffersen

and Pelletier (2007) find that setting m = 5 gives good testing power in a realistic daily V aR

backtesting setting.

Independence testing can also be done using the likelihood approach. Assume that the hit

sequence is dependent over time and that it can be described as a first-order Markov sequence with

transition probability matrix

Π1 =

"1− π01 π01

1− π11 π11

#.

These transition probabilities simply mean that conditional on today being a non-violation (that is

It = 0) then the probability of tomorrow being a violation (that is It+1 = 1) is π01. The probability

of tomorrow being a violation given today is also a violation is π11 = Pr (It = 1 and It+1 = 1) .The

probability of a non-violation following a non-violation is 1 − π01 and the probability of a non-

violation following a violation is 1− π11.

5

If we observe a sample of T observations, then we can write the likelihood function of the

first-order Markov process as

L (Π1) = (1− π01)T00 πT0101 (1− π11)

T10 πT1111

where Tij , i, j = 0, 1 is the number of observations with a j following an i. Taking first derivatives

with respect to π01 and π11 and setting these derivatives to zero, one can solve for the Maximum

Likelihood estimates

π̂01 =T01

T00 + T01, π̂11 =

T11T10 + T11

.

Using then the fact that the probabilities have to sum to one we have π̂00 = 1− π̂01, π̂10 = 1− π̂11.

Allowing for dependence in the hit sequence corresponds to allowing π01 to be different from

π11. We are typically worried about positive dependence which amounts to the probability of a

violation following a violation (π11) being larger than the probability of a violation following a non-

violation (π01). If on the other hand the hits are independent over time, then the probability of a

violation tomorrow does not depend on today being a violation or not and we write π01 = π11 = π.

We can test the independence hypothesis that π01 = π11 using a likelihood ratio test

LRind = −2 lnhL (π̂) /L

³Π̂1

´i∼ χ21

where L (π̂) is the likelihood under the alternative hypothesis from the LRuc test.

Other methods for independence testing based on the duration of time between hits can be

found in Christoffersen and Pelletier (2004).

3 Backtesting with Information Variables

The preceding tests are quick and easy to implement. But as they only use information on past

V aR hits, they might not have much power to detect misspecified risk models. In order to increase

the testing power, we consider using the information in past market variables, such as interest

rate spreads or volatility measures. The basic idea is to test the model using information which

may explain when violations occur. The advantage of increasing the information set is not only to

increase power but also to help us understand the areas in which the risk model is misspecified.

This understanding is key in improving the risk models further.

If we define the q-dimensional vector of information variables available to the backtester at time

t as Xt, , then the null hypothesis of a correct risk model can be written as

H0 : Pr (It+1 = 1|Xt) = p⇔ E [It+1 − p|Xt] = 0.

The hypothesis says that the conditional probability of getting a V aR violation on day t + 1

should be independent of any variable observed at time t and it should simply be equal to the

promised V aR coverage rate, p. This hypothesis is equivalent to the conditional expectation of the

hit sequence less p being equal to 0.

6

Engle and Manganelli (2004) develop a conditional autoregressive V aR by regression quantiles

approach. For backtesting they suggest the following dynamic quantile test

DQ = (I − p)0X¡X 0X

¢−1X 0 (I − p) / (Tp (1− p)) ∼ χ2q (4)

where X is the T by q matrix of information variables, and (I − p) is the T by 1 vector of hits

where each element has been subtracted by the desired covered rate p.

Berkowitz, Christoffersen and Pelletier (2007) finds that implementing the DQ test using simply

the lagged V aR from a GARCH model as well as the lagged hit gives good power in a realistic daily

V aR experiment. Smith (2007) also finds good power when applying the DQ test. A regression-

based approach to backtesting is also used in Christoffersen and Diebold (2000) and Christoffersen

(2003). Christoffersen, Hahn and Inoue (2001) develop tests for comparing different V aR models.

Smith (2007) further suggest a probit approach where the potentially time-varying probability

of a hit is modeled using the normal cumulative density function and where Lagrange Multiplier

tests are used.

When backtesting with information variables the question of which variables to include in the

vector Xt of course immediately arises. It is difficult to give a general answer as it depends on the

particular portfolio at hand. It is likely that variables which are thought to be correlated with the

future volatility in the portfolio are good candidates. In equity portfolios option implied volatility

measures such as the VIX would be an obvious candidate. In FX portfolios option implied volatility

measures could be constructed as well. In bond portfolios variables such as term spreads and credit

spreads are likely candidates as are variables capturing the level, slope and curvature of the term

structure.

4 Other Issues in Backtesting

When correctly specified the one-day V aR measure tells the user that there is a probability p of

loosing more than the V aR over the next day. Importantly, it does not, for example, say how

much one can expect to loose on the days where the V aR is violated. Thus the V aR only contains

partial information about the distribution of losses. This limitation of the V aR is reflected in the

definition of the hit variable in (1) which in turn limits the possibilities for backtesting in the V aR

setting. We can only backtest on the hit occurrences and not for example on the magnitude of the

losses when the hits occur because the V aR does not promise hits of a certain magnitude.

4.1 Backtesting Expected Shortfall

The limitations of the V aR as a risk measure has to suggestions of alternative risk measures, most

prominently Expected Shortfall (ES), also referred to as Conditional VaR (CVAR) (eqf15/004,

eqf15/008), which denotes the expected loss on the days where the V aR is violated. We can define

7

it formally as

ESpt+1 = Et

£PLt+1|PLt+1 > V aRp

t+1

¤Note that ES provides information on the expected magnitude of the loss whenever the V aR is

violated. We now consider ways to backtest the ES risk measure.

Consider again a vector of variables, Xt, which are known to the risk manager, and which may

help explain potential portfolio losses beyond what is explained by the risk model. The ES risk

measure promises that whenever we violate the V aR, the expected value of the violation will be

equal to ESpt+1.We can therefore test the ES measure by checking if the vector Xt has any ability

to explain the deviation of the observed shortfall or loss, PLt+1, from the expected shortfall on the

days where the V aR was violated. Mathematically, we can write

PLt+1 −ESpt+1 = b0 + b01Xt + et+1, for t+ 1 where PLt+1 > V aRp

t+1

where t + 1 now refers only to days where the V aR was violated. The observations where the

V aR was not violated are simply removed from the sample. The error term et+1 is assumed to be

independent of the regressor, Xt.

In order to test the null hypothesis that the risk model from which the ES forecasts were made

uses all information optimally (b1 = 0), and that it is not biased (b0 = 0), we can jointly test that

b0 = b1 = 0.

Notice that now the magnitude of the violation shows up on the left hand side of the regression.

But notice that we can still only use information in the tail to back-test. The ES measure does

not reveal any particular properties about the remainder of the distribution and therefore we only

use the observations where the losses were larger than the V aR.

Further discussion of ES backtesting can be found in McNeil and Frey (2000).

4.2 Backtesting the Entire Distribution

Rather than focusing on particular risk measures from the loss distribution such as the Value at

Risk or the Expected Shortfall, we could instead decide to backtest the entire loss distribution from

the risk model. This would have the benefit of potentially increasing further the power to reject

bad risk models.

Assuming that the risk model produces a cumulative distribution of portfolio losses, call it

Ft(∗). Then at the end of every day, after having observed the actual portfolio loss (or profit) wecan calculate the risk model’s probability of observing a loss below the actual. We will denote this

probability by pt+1 ≡ Ft (PLt+1) .

If we are using the correct risk model to forecast the loss distribution, then we should not be

able to forecast the risk model’s probability of falling below the actual return. In other words,

the time series of observed probabilities pt+1 should be distributed independently over time as a

Uniform(0,1) variable. We therefore want to consider tests of the null hypothesis

H0 : pt+1 ∼ i.i.d. Uniform (0, 1) . (5)

8

The Uniform(0,1) distribution function is flat on the interval 0 to 1 and zero everywhere else. As

the pt+1 variable is a probability it is must lie in the zero to one interval. Constructing a histogram

and checking if it looks reasonably flat provides a useful visual diagnostic. If systematic deviations

from a flat line appear in the histogram, then we would conclude that the distribution from the

risk model is misspecified. Diebold, Gunther and Tay (1998) contains a detailed discussion of this

approach.

Unfortunately, testing the i.i.d. uniform distribution hypothesis is cumbersome due to the re-

stricted support of the uniform distribution. But we can transform the i.i.d. Uniform pt+1 to an

i.i.d. standard normal variable, zt+1 using the inverse cumulative distribution function, Φ−1 (pt+1) .

We are then left with a test of a variable conforming to the standard normal distribution, which

can easily be implemented. The i.i.d. property of zt+1can be assessed via the autocorrelation func-

tions and by using the LB(m) test in (3) on zt+1 and |zt+1| for example. The normal distributionproperty can be tested using the method of moments approach in Bontemps and Meddahi (2005).

Regression based tests using information variables can also be constructed. Further analysis of

distribution backtesting can be found in Berkowitz (2001) and Crnkovic and Drachman (1996).

4.3 Dirty Profits and Losses

The backtesting procedures surveyed in this chapter all in one way or another compare the ex

ante forecast from a risk model to the ex post profit and losses (P/L). It is therefore obviously

essential but unfortunately not always the case that the ex post recorded P/L arise directly from

the portfolio used to make the ex ante model predictions. The risk model will typically produce a

risk forecast for the loss distributions of a particular portfolio of assets. The ex post profits and

losses should only contain cash flows directly related to the portfolio of assets assumed in the risk

models. But the total P/L of a trading desk may contain trading commission revenues as well as

costs that are not directly related to holding the portfolio of assets assumed in the risk model. This

extraneous cash flows should be stripped from the P/L before backtesting.

The daily P/L may of course also include cash-flows from intraday transactions that is the P/L

from selling an asset that was bought the same day. Such cash-flows are not directly related to

the end-of-day portfolio entered into the risk model and should ideally be stripped away as well.

O’Brien and Berkowitz (2005) contain a detailed discussion of these issues.

4.4 Multiple-day Horizons and Changing Portfolio Weights

The backtesting procedures described above can be relatively easily adopted to the multi-day risk-

horizon setting. If for example a 10-day V aR forecast is observed daily along with the ex post

10-day P/L, then the hit sequence can be constructed daily as in equation (1). The 10-day V aR

horizons implies that 9-day autocorrelation is to be expected in the hit-sequence and that this

should be allowed for when constructing its variance in (2). Similarly, in the DQ test in (4) one

9

should use information variables available 10 days prior to the observed P/L. Smith (2007) discuss

backtesting with multi-day V aRs.

4.5 Allowing for Risk Model Parameter Estimation Error

In the discussion so far we have abstracted from the fact that the risk models in use most likely

contain parameters that are estimated with errors. This parameter estimation error will in turn

render the hit sequence observed with error. As a consequence when backtesting the risk model

we may reject the “true” risk model just because its parameters were estimated with error. As

discussed in Engle and Manganelli (2004) this is mainly an issue when the risk model is evaluated

in-sample that is when the model is evaluated on the same data in was estimated on. In typical

external back-testing applications, the backtesting procedures are used in a more realistic out-of-

sample fashion where the model is estimated on data up until day t and then used to forecast risk

for day t + 1. In this setting parameter estimation error is less likely to be critical. Parameter

estimation issues in relation to V aR modeling has been analyzed in detail in Escanciano and Olmo

(2007), and Giacomini and Komunjer (2005).

References

[1] Andersen, T., T. Bollerslev, P. Christoffersen, and F. Diebold (2005), Practical Volatility and

Correlation Modeling for Financial Market Risk Management, in M. Carey and R. Stulz (eds.),

Risks of Financial Institutions, University of Chicago Press for NBER, 513-548.

[2] Basle Committee on Banking Supervision (1996a), Amendment to the Capital Accord to In-

corporate Market Risks, Bank for International Settlements.

[3] Basel Committee on Banking Supervision, (1996b), Supervisory Framework for the Use of

‘Backtesting’ in Conjunction With the Internal Models Approach to Market Risk Capital

Requirements, Bank for International Settlements.

[4] Basle Committee on Banking Supervision (2004), International Convergence of Capital Mea-

surement and Capital Standards: a Revised Framework.

[5] Berkowitz, J. (2001), Testing Density Forecasts, Applications to Risk Management Journal of

Business and Economic Statistics, 19, 465-474.

[6] Berkowitz, J., P. Christoffersen, and D. Pelletier (2007), Evaluating Value-at-Risk Models with

Desk-Level Data, Manuscript North Carolina State University.

[7] Berkowitz, J. and J. O’Brien (2002), How Accurate are the Value-at-Risk Models at Commer-

cial Banks? Journal of Finance, 57, 1093-1112.

10

[8] Bontemps, C. and N. Meddahi (2005), Testing Normality: A GMM Approach, Journal of

Econometrics, 124, 149-186.

[9] Campbell, S. (2007), A Review of Backtesting and Backtesting Procedures, Journal of Risk,

9, 1-17.

[10] Christoffersen, P. (1998), Evaluating Interval Forecasts, International Economic Review, 39,

841-862.

[11] Christoffersen, P. (2003), Elements of Financial Risk Management, Academic Press.

[12] Christoffersen, P. and F. Diebold (2000), How Relevant is Volatility Forecasting for Financial

Risk Management?, Review of Economics and Statistics, 82, 1-11.

[13] Christoffersen, P., J. Hahn and A. Inoue (2001), Testing and Comparing Value-at-Risk Mea-

sures Journal of Empirical Finance, 8, 325-342.

[14] Christoffersen, P.F. and D. Pelletier (2004), Backtesting Value-at-Risk: A Duration-Based

Approach, Journal of Financial Econometrics, 2, 84-108.

[15] Crnkovic, C. and J. Drachman (1996), Quality Control, Risk, September, 138-143.

[16] Diebold, F.X., T. Gunther, and A. Tay (1998), Evaluating Density Forecasts, with Applications

to Financial Risk Management, International Economic Review, 39, 863-883 .

[17] Dufour, J.-M., (2006), Monte Carlo Tests with Nuisance Parameters : A General Approach

to Finite-Sample Inference and Nonstandard Asymptotics in Econometrics, Journal of Econo-

metrics, 133, 443-477.

[18] Engle, R.F. and S. Manganelli (2004), CAViaR: Conditional Autoregressive Value-at-Risk by

Regression Quantiles, Journal of Business and Economic Statistics, 22, 367-381.

[19] Escanciano, J. and J. Olmo (2007), Estimation Risk Effects on Backtesting For Parametric

Value-at-Risk Models, Manuscript, City University London.

[20] Giacomini, R., and I. Komunjer (2005), Evaluation and Combination of Conditional Quantile

Forecasts, Journal of Business and Economic Statistics, 23, 416-431.

[21] Hendricks, D. (1996), Evaluation of Value-at-Risk models using historical data, Economic

Policy Review, Federal Reserve Bank of New York, April, 39-69.

[22] Jorion, P. (2002), How informative are Value-at-Risk disclosures? Accounting Review, 77,

911-931.

[23] Kupiec, P. (1995), Techniques for Verifying the Accuracy of Risk Measurement Models, Journal

of Derivatives, 3, 73-84.

11

[24] Lopez, J. (1999), Regulatory Evaluation of Value-at-Risk Models, Journal of Risk, 1, 37-64

[25] Lopez, J. and M. Saidenberg, Evaluating Credit Risk Models, Journal of Banking and Finance,

24, 151-165.

[26] McNeil, A. and R. Frey (2000), Estimation of Tail-Related Risk Measures for Heteroskedastic

Financial Time Series: An Extreme Value Approach, Journal of Empirical Finance, 7, 271-300.

[27] O’Brien, J. and J. Berkowitz (2005), Bank Trading Revenues, VaR and Market Risk, in M.

Carey and R. Stulz (eds.), Risks of Financial Institutions, University of Chicago Press for

NBER.

[28] Perignon, C., Z. Deng and Z. Wang (2006), Do Banks Overstate their Value-at-Risk? Manu-

script, HEC Paris.

[29] Perignon, C. and D. Smith (2006), The Level and Quality of Value-at-Risk Disclosure by

Commercial Banks, Manuscript, Simon Fraser University.

[30] Pritsker, M. (2001), The Hidden Dangers of Historical Simulation, Finance and Economics

Discussion Series 2001-27. Board of Governors of the Federal Reserve System.

[31] Smith, D. (2007), Conditional Backtesting of Value-at-Risk Models, Manuscript, Simon Fraser

University.

12