IFRS 9: Does one model fit all?Lessons from the ashes of the Great Moderation

Hugo ChimAugust 2017

2

Introduction

Hugo Chim

Assistant Manager – Deloitte Financial Services Risk Advisory

Hugo Chim is an Assistant Manager in the Risk Modelling Team of Deloitte’s Financial Services Risk Advisory Practice. Hugo has been a key modeller and SME in Expected Credit Loss (ECL) models for major European banks with systemic importance.

He has also applied advanced Econometric techniques to both retail and non-retail credit risk problems such as forecasting multiple-scenario PD using Vector Auto-regression and Error Correction, Logistic Models, and EMV techniques. Before joining the FS Practice, he was an Economist in Deloitte Economic Consulting.

Hugo graduated from Warwick University with Distinction in MSc Economics.

Getting to know you…

3

AgendaTopics to cover today

1. Current modelling challenges 4

• Impact of IFRS 9 and model risks 5

• Challenge 1: Absence of cyclical behaviours 7

• Challenge 2: Good-time optimism bias 11

• Challenge 3: Paradigm shifts 16

2. Proposed Approach 25

3. Application showcase

• AR(1) MS-model estimation outcomes 28

4. In-sample assessment 30

5. Forecast performance benchmarking 34

6. Areas in further development 36

4

Current modelling challengesWhy should we care?

5

Why should we care?IFRS 9 is expected to have a significant impact on provision stock and pricing.

Revenue Source and portfolio composition

2468

101214161820

500 1,000 1,500 2,000

Rev

enu

e, £

m

Risk Weighted Assets, £m

2468

101214161820

500 1,000 1,500 2,000

Rev

enu

e, £

m

Risk Weighted Assets, £m

Bank A’ top 100 key clients, Today Bank A’ top 100 key clients, To target for

To Exit:Small revenue, low ROE and large RWA clients

To Focus :Large revenue, high ROE and low RWA clients

Expected significant increase in provisions

Source: Deloitte Research

Cost of product offerings and pricing impactPrice makers who think IFRS 9 will affect costs in each products:

Price takers who think IFRS 9 will affect costs in each products:

IFRS 9 cyclicality and capital impactIFRS 9 Pro-cyclicality could drive CET1 ratio below 7% in a downturn.

6

The starting pointThe common Merton-Vasicek Framework

• The Vasicek (2002) model decomposes default risks into systematic (𝑧𝑧𝑡𝑡) and obligor specific factors (𝜀𝜀𝑖𝑖).

• The standard practice is to model default risks as an unobserved latent asset return index (𝑅𝑅𝑖𝑖𝑡𝑡) that is positively correlated to a given obligor’s asset value on book. Intuitively, a lower expected future return (which leads to a lower asset value) is expected to increase default risks.

• When the index falls below a given threshold (default boundary), e.g. when asset values is below liability value, the obligor is expected to default. Credit: CCBS Handbook #34 - Modelling Credit Risk. Bank of

England

Vasicek Model (Yang 2013 Probit-linear formulation)

𝑃𝑃𝑃𝑃 𝑍𝑍𝑡𝑡 = 𝑃𝑃(𝑅𝑅𝑖𝑖𝑡𝑡 < 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑑𝑑𝑖𝑖 𝑆𝑆𝑘𝑘𝑡𝑡 = Φ(𝛼𝛼 + �𝑘𝑘=1

𝑚𝑚

𝛽𝛽𝑘𝑘𝑆𝑆𝑘𝑘𝑡𝑡 + 𝜏𝜏𝜀𝜀𝑖𝑖)

Where,

• ∑𝑘𝑘=1𝑚𝑚 𝛽𝛽𝑘𝑘𝑆𝑆𝑘𝑘𝑡𝑡 = 𝜌𝜌𝑍𝑍𝑡𝑡 , 𝒁𝒁𝒕𝒕 ~ 𝑵𝑵(𝟎𝟎,𝟏𝟏) is the systematic, and factor; and

• 𝜏𝜏𝜀𝜀𝑖𝑖 = 1 − 𝜌𝜌 𝜀𝜀𝑖𝑖 , 𝜀𝜀𝑖𝑖 ~ 𝑁𝑁(0, 1) is the idiosyncratic factor.

Obligor i asset value (returns)

index

Systematic risk factor (𝒁𝒁𝒕𝒕) Obligor

specific risk factor (𝜺𝜺𝒊𝒊)Macro factors

(𝑺𝑺𝒕𝒕)

Default Risks for Obligor i

Implicit mean-reversion

assumption

Merton-Vasicek model (2002)

7

Empirical Reality

-2.5-2

-1.5-1

-0.50

0.51

1.52

2.5

0 2 4 6 8 111315171921232527293234363840424446485053555759

Theoretical systematic factor

Z TTC

Current modelling challengesChallenges we have come across at our clients when they were modelling the systematic factor under the Vasicek framework.

Obligor i asset value (returns)

index

Systematic risk factor (𝒁𝒁𝒕𝒕)

Obligor specific risk factor (𝜺𝜺𝒊𝒊)

Default Risks for Obligor i

Cyclical (Stationary)

-6

-5

-4

-3

-2

-1

0

1

2

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61

Estimated systematic factor from default data

Acyclical (Non-stationary)

1

Theory

Optimism bias

2

Paradigm shifts

3

Overly optimistic trend forecasts

Abrupt change in trend

Long period of decline in risk

(Acyclical) 1

3

2

Illustration only

Estimated systematic risk from default data

8

Challenge 1: Absence of cyclical behavioursSince the last financial crisis, UK mortgage default rates have been decreasing. Cyclical behaviour has not been observed in the last 10 years.

Source: CML 2017

1.8

1.9

2

2.1

2.2

2.3

2.4

0.0%

0.5%

1.0%

1.5%

2.0%

2.5%

3.0%

2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Mortgage default risk post 2008-9 crisis in the UK

3plus arrears rate Implied returns

9

0

0.005

0.01

0.015

0.02

0.025

0.03

2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020

Forecasting results based on observed data post 2008-9 crisis

Observed 3plus arrears rate Predicted 3plus arrears rate

Challenge 1: Absence of cyclical behavioursStandard models tend to predict the downward trend to continue.

Source: CML 2017 and Deloitte analysis

Mean forecasts

Optimism bias

2

Acyclical (Non-stationary)

1

Q1

Can credit risk be trending down indefinitely? Q2

Is it tenable to argue for cyclical behaviour around a stable TTC PD?

Development data

Experimental Results

10

In July 2007, Charles Prince (Ex-Citi CEO) told the Financial Times that global liquidity was enormous and only a significant disruptive event could create difficulty in the leveraged buyout market.

"As long as the music is playing, you've got to get up and dance…We're still

dancing.”

Credit: The New York Times

Source: Reuters

11

Challenge 2: Good-time optimism biasDuring the Great Moderation between the end of 1991 recession and the 2008 financial crisis, the UK experienced 63 consecutive quarters of economic growth. During the UK’s (and the US’) Great Moderation, many argued that the old laws of economics had been abolished.

Source: CML 2017

-1%

0%

1%

1%

2%

2%

3%

3%

4%

4%

5%

5%

0

5

10

15

20

25

30

35

40

45

50

Macroeconomic conditions during the "Great Moderation" in the UK

Inflation-adjusted HPI

Inflation-adjusted earnings

0

0.01

0.02

0.03

0.04

0.05

0.06

Mortgage default risk (3+) during the "Great Moderation" in the UK

Source: BBC Nov 2006

Credit-Cycle?

12

Challenge 2: Good-time optimism biasThe Great Moderation however did end abruptly and dramatically.

Source: CML 2017

-8%

-6%

-4%

-2%

0%

2%

4%

6%

0

10

20

30

40

50

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009

Macroeconomic conditions during the "Great Moderation" in the UK

Inflation-adjusted HPI Inflation-adjusted GDP Inflation-adjusted earnings

0

0.01

0.02

0.03

0.04

0.05

0.06

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009

Mortgage default risk (3+) during the "Great Moderation" in the UK

Sudden break in trend

Credit: The Telegraph

Paradigm shifts3

13

Challenge 2: Good-time optimism biasPredictions made based on the data from the Great Moderation period exhibited a continuing downward trend. As such, the forecast errors after the 2008-9 crisis assuming the same trend will be significant and persistent.

Source: CML 2017 and Deloitte Analysis

0.0%

1.0%

2.0%

3.0%

4.0%

5.0%

6.0%

1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Forecasted default risk (3+) based on data from the "Great Moderation" period

Observed 3plus arrears rate Predicted 3plus arrears rate

Mean forecasts

The model is almost irrelevant for a long period of time during and after the crisis.

Model performance appears to improve after 10 years.

Paradigm shifts

3Acyclical (Non-stationary)

1

Optimism bias

2

Development data

Experimental Results

14

"If you live each day as if it was your last…”

Credit: Esquire

Steve Jobs’ Stanford University Commencement address in 2015.

“someday you’ll most certainly be right.”

15

"In the long run we are all dead. Economists set themselves too easy, too useless a task if in tempestuous seasons they can only tell us that

when the storm is long past the ocean is flat again.”

Credit: The Guardian

A Tract on Monetary Reform (1923) - John Maynard Keynes

16

Challenge 3: Paradigm shiftsBased on 48 years of data, we may estimate the long-run 3+ arrears rate to be circa 1.9%;however, over the entire period, it is difficult to argue for any meaningful cyclical behaviour around this long-run value. The 3+ data appears to be so persistent that non-stationarity test often indicates that 3+ data is a random-walk.

0%

1%

2%

3%

4%

5%

6%

7%19

69

1970

1971

1972

1973

1974

1975

1976

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

2016

Extrapolated 3+ Delinquency Rates

Extrapolated 3+ rates Overall mean

Source: CML 2017 and Deloitte AnalysisExperimental Results

17

48 years is a long time to have no paradigm shifts.

Paradigm shifts can lead to false non-stationarity test results.

18

Challenge 3: Paradigm shift detectionVisual inspection identifies three potential states of the mortgage market: Pre 90s crisis, period around the 90s crisis and periods around the 2008/9 crisis. The dividing line however can be arbitrary.

0%

1%

2%

3%

4%

5%

6%

7%19

69

1970

1971

1972

1973

1974

1975

1976

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

2016

Extrapolated 3+ Delinquency Rates

Extrapolated 3+ rates Overall mean Baseline overtime average PD Stressed average PD 1 Stressed average PD 2

Source: CML 2017 and Deloitte AnalysisExperimental Results

19

0%

1%

2%

3%

4%

5%

6%

7%

1969

1970

1971

1972

1973

1974

1975

1976

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

2016

Extrapolated 3+ Delinquency Rates

Extrapolated 3+ rates Overall mean

Challenge 3: Paradigm shift detectionHistorical events may also provide some insights.

Pre 1980s• Vast majority of mortgage

funding came from building societies, which borrowed retail funds from the household sector and lent them to households with secure incomes, a savings history, and a significant deposit.

• It was arguably almost riskless to lenders because interest rates were adjusted to enable flows of funds to be balanced.

• Mortgagors bore all the interest risk.

Late 70s-late 1980s• (Foreign) Banks and insurance

companies started to enter the market.

• Bank of England (1980) lifted the Supplementary Special Deposits regulations (“Corset”).

• 1984 changes in regulations meant building societies could then operate like banks.

• 1980s right to buy for council tenants, injecting 1 million households (lower income) into the mortgage market.

• Growth in use of mortgage backed securities by US banks.

1989-1992• By 1989, institutions were

lending at unprecedented income multiples and LTVs.

• Homeownership had spread down the income scale.

• 20% of new loans were at more than 100% LTV.

• Economy and housing market slowed down. Transactions halved by 1992.

• 2 million households faced negative equity and repossessions reached a peak in 1991.

1993-2007• Great Moderation started to

take effects.

2008-9• Global Financial Crisis

2010 - ?• Gradual recovery with low

interest rates, more robust regulations and stronger bank balance sheets.

High volatility

Low volatility

Period of stability Period of building up systematic risks

Period of prosperity Period of recovery

Known Crisis

Experimental Results

20

Challenge 3: Paradigm shifts and biasFor simplicity, if we only consider two possible paradigms: Baseline and Stressed, what can we say in general about each of these paradigms?

Possible Paradigms

Baseline Paradigm Stressed Paradigm

Baseline TTC PD

Baseline systematic factor volatility

Baseline sensitivity to macroeconomic factors

Higher TTC PD

Higher systematic factor volatility

Higher sensitivity to macroeconomic factors

Standard Models often predict the average of the

two.

Over estimate in Baseline paradigm

Under estimate in Stressed paradigm

21

Dummy variables model the instability of time series in hindsight• Step 1: Generate period dummies for high volatility periods (stressed), using low volatility period as baseline. Let:

• 𝑑𝑑𝑠𝑠𝑡𝑡𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 = �1, 𝑓𝑓𝑡𝑡𝑡𝑡 𝑦𝑦𝑡𝑡𝑦𝑦𝑡𝑡 𝑓𝑓𝑡𝑡𝑡𝑡𝑓𝑓 1980 𝑡𝑡𝑡𝑡 1992, 2008 𝑡𝑡𝑡𝑡 20090, 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑜𝑜𝑜𝑜𝑡𝑡𝑡𝑡

• Step 2: Run regression using both level and multiplicative dummies:

Φ−1 𝑂𝑂𝑃𝑃𝐹𝐹𝑡𝑡 = 𝑦𝑦𝑡𝑡 = 𝛼𝛼 + �𝑘𝑘=1

𝑚𝑚

𝛽𝛽𝑘𝑘𝑆𝑆𝑘𝑘𝑡𝑡 + 𝜔𝜔𝑑𝑑𝑠𝑠𝑡𝑡𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 + �𝑘𝑘=1

𝑚𝑚

𝛿𝛿𝑘𝑘𝑆𝑆𝑘𝑘𝑡𝑡 ∗ 𝑑𝑑𝑠𝑠𝑡𝑡𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 + 𝑣𝑣𝑡𝑡

• Essentially we are estimating two mean models:

𝑦𝑦𝑡𝑡 = 𝛼𝛼 + �𝑘𝑘=1

𝑚𝑚

𝛽𝛽𝑘𝑘𝑆𝑆𝑘𝑘𝑡𝑡 + 𝑣𝑣𝑡𝑡

Baseline Model

0102030405060708090

HPI

Baseline Stressed

𝑦𝑦𝑡𝑡 = 𝑦𝑦𝑡𝑡 = (𝛼𝛼 + 𝜔𝜔) + �𝑘𝑘=1

𝑚𝑚

(𝛿𝛿𝑘𝑘 + 𝛽𝛽𝑘𝑘)𝑆𝑆𝑘𝑘𝑡𝑡 + 𝑣𝑣𝑡𝑡

Stressed Model

Challenge 3: Conventional approach to paradigm shifts

Higher average default risk across all possible HPI values.

Higher sensitivity to HPI changes

(𝛿𝛿𝑘𝑘 + 𝛽𝛽𝑘𝑘)

(𝛼𝛼 + 𝜔𝜔)

Systematic Risk against HPI

Syst

emat

ic R

isk

22

0

0.2

0.4

0.6

0.8

1

1.2

Stressed Period Dummy

Challenge 3: Conventional approach to paradigm shiftsDummy variables model only works in hindsight

Stressed Dummy

1

0

StressedPeriod

Baseline Period

Future state is still unknown

1

0?

In reality, it is hard to say when exactly the Great Moderation actually started.

• This method is only useful in hindsight.• This is a crude approach with abrupt arbitrary changing points. • This method does not assign probabilistic weight to possible future paradigms.• IFRS 9 requires banks to make scenario forecasts and so information on future paradigms (a form of scenario) is

arguably essential.• Assigning wrong future paradigms to forecasts may lead to bias.

Methodology issues

23

"You only learn who has been swimming naked when the tide goes out,"

"And what we are witnessing at some of our largest financial institutions is an ugly sight."

Credit: Forbes.com

But perhaps we could assign some probabilities to this event happening?

24

Proposed approachIntroducing randomness in paradigm shifts

25

Proposed approachMortgage default in a Markov-Switching (MS) framework

• The dynamics of the CCI are modelled as a state-dependent process where the state(Stressed or Baseline) is unobserved by the modeller.

• Let 𝑦𝑦𝑡𝑡 denote the CCI in period t and suppose that its mean and variance are governed by an unobserved state variable 𝑆𝑆𝑡𝑡 =𝑺𝑺𝒕𝒕𝑺𝑺𝑺𝑺𝑺𝑺𝑺𝑺𝑺𝑺𝑺𝑺,𝑩𝑩𝑩𝑩𝑺𝑺𝑺𝑺𝑩𝑩𝒊𝒊𝑩𝑩𝑺𝑺 in our simple two-state example:

𝑦𝑦𝑡𝑡 = 𝜇𝜇 𝑆𝑆𝑡𝑡 + 𝜎𝜎 𝑆𝑆𝑡𝑡 𝑣𝑣𝑡𝑡 𝑜𝑜𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑣𝑣𝑡𝑡 ~ 𝑜𝑜. 𝑜𝑜.𝑑𝑑.𝑁𝑁(0,1)

Such that,

�𝑦𝑦𝑡𝑡 = 𝜇𝜇𝑠𝑠 + 𝜎𝜎𝑠𝑠𝑣𝑣𝑡𝑡 𝑜𝑜𝑓𝑓 𝑡𝑡𝑡𝑡 = 𝑺𝑺𝒕𝒕𝑺𝑺𝑺𝑺𝑺𝑺𝑺𝑺𝑺𝑺𝑺𝑺𝑦𝑦𝑡𝑡 = 𝜇𝜇𝑏𝑏 + 𝜎𝜎𝑏𝑏𝑣𝑣𝑡𝑡 𝑜𝑜𝑓𝑓 𝑡𝑡𝑡𝑡 = 𝐵𝐵𝑦𝑦𝑡𝑡𝑡𝑡𝑡𝑡𝑜𝑜𝐵𝐵𝑡𝑡

Note that the difference between this set up and the dummy-variable regression model is that the state-switching mechanism is a random process. Particularly, the state of the world is governed by an ergodic irreducible hidden Markov Process (mean-reverting without absorbing state):

𝑷𝑷 =𝑝𝑝𝑠𝑠𝑠𝑠 𝑝𝑝𝑠𝑠𝑏𝑏𝑝𝑝𝑏𝑏𝑠𝑠 𝑝𝑝𝑏𝑏𝑏𝑏

Where for example 𝑝𝑝𝑏𝑏𝑠𝑠 = 𝑃𝑃(𝑆𝑆𝑡𝑡 = 𝑆𝑆𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑑𝑑|𝑆𝑆𝑡𝑡−1 = 𝐵𝐵𝑦𝑦𝑡𝑡𝑡𝑡) is the probability that the process is in a Baseline state at time t given that it was in a Stressed state at time t-1.

MS-Model set up

26

Proposed approachMortgage default in a Markov-Switching framework

• Forecasts can be made using the key outputs of the MS-model. For a h-step ahead forecast based on information at time t,

𝐸𝐸 𝑦𝑦𝑡𝑡+ℎ 𝐼𝐼𝑡𝑡 = ξ̂ t t ∗ 𝐏𝐏𝐡𝐡 ∗ �𝛍𝛍

Forecasting with MS-Model

-6

-4

-2

0

2

4

6

8

10

12

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41

CCI @ State 1: Random Walk

S1 S1 Upper 95% S1 Lower 95%

0

0.05

0.1

0.15

0.2

0.25

0.3

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41

CCI @ State 2: Persistent Auto-Regressive Process

S2 S2 Upper 95% S2 Lower 95%

S B

T+2

S

B

T+1

70%

T+130%

60%

40%

𝑦𝑦𝑇𝑇

T+2

TT90%10%

Illustration only

27

Application showcaseThe UK mortgage market

28

AR(1) MS-model estimation outcomesCML 3+ Arrears data

Simulation for the Stressed state

Simulation for the Baseline state

-0.5

0

0.5

1

1.5

2

2.5

3

1 2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829303132333435363738394041

Stressed State Stressed State Upper 95%

Stressed State Lower 95% Stressed State Random Draw

0

0.5

1

1.5

2

2.5

3

1 2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829303132333435363738394041

Base State Base State Upper 95%

Base State Lower 95% Base State Random Draw

Experimental Results

Stressed state

Sensitivity to HPI 1% fall in HPI growth -> 0.8% fall in implied returns (R) (4x more sensitive)

Asymptotic TTC implied returns (R) 1.4 (46% less)

Asymptotic R volatility (𝜎𝜎𝐑𝐑2) 0.037 (>10x Baseline)

Average duration (Forced uniform volatility)

5 years (3 years)

Baseline state

Sensitivity to HPI 1% fall in HPI growth -> 0.2% fall in implied returns (R)

Asymptotic TTC implied returns (R) 2.6

Asymptotic R volatility (𝜎𝜎𝐑𝐑2) 0.003

Average duration (Forced uniform volatility)

4 years (9 years)

𝑷𝑷 = 𝑃𝑃(𝑆𝑆𝑡𝑡 = 𝑆𝑆𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑑𝑑|𝑆𝑆𝑡𝑡−1 = 𝑆𝑆𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑑𝑑) 𝑃𝑃(𝑆𝑆𝑡𝑡 = 𝑆𝑆𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑑𝑑|𝑆𝑆𝑡𝑡−1 = 𝐵𝐵𝑦𝑦𝑡𝑡𝑡𝑡)𝑃𝑃(𝑆𝑆𝑡𝑡 = 𝐵𝐵𝑦𝑦𝑡𝑡𝑡𝑡|𝑆𝑆𝑡𝑡−1 = 𝑆𝑆𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑑𝑑) 𝑃𝑃(𝑆𝑆𝑡𝑡 = 𝐵𝐵𝑦𝑦𝑡𝑡𝑡𝑡|𝑆𝑆𝑡𝑡−1 = 𝐵𝐵𝑦𝑦𝑡𝑡𝑡𝑡)

= 𝟎𝟎.𝟖𝟖𝟎𝟎 0.200.27 𝟎𝟎.𝟕𝟕𝟕𝟕

29

In-sample assessmentHow well does the model fit the data?

30

Assessment: actual vs. predicted AR(1) MS-Model prediction is the average between the Baseline and Stressed state predictions, weighted by the probabilities of being a given state.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0%

1%

2%

3%

4%

5%

6%

7%

8%

STA

TE P

RO

BABIL

ITY

3+ R

ATE

Historical Smooth State Probability and 3+ Delinquency Rates

P(Stressed State) P(Base State) Extrapolated 3+ Delinquency Rate

E(3+|Stressed) E(3+|Base) Fitted E(3+)

Experimental Results

31

Assessment: In-sample fit comparison with next-best benchmark modelAR(1) MS-model appears to be able to cope with abrupt turning points (paradigm shifts) better than the benchmark AR(1) model, resulting in a significantly better in-sample fit.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0%

1%

2%

3%

4%

5%

6%

7%

8%

STA

TE P

RO

BABIL

ITY

3+ R

ATE

Historical Smooth State Probability and 3+ Delinquency Rates

P(Stressed State) P(Base State) Extrapolated 3+ Delinquency Rate

Fitted E(3+) AR(1) Benchmark

MS-model appears to be able to cope with abruptly large turning points.

0.0%

2.0%

4.0%

6.0%

8.0%

10.0%

12.0%

14.0%

16.0%

18.0%

MS MAE AR(1) MAE

Average in-sample percentage prediction errors (MAE)

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

MS RMSE AR(1) RMSE

Average in-sample percentage prediction errors (RMSE)

Experimental Results

32

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0%

1%

2%

3%

4%

5%

6%

7%

8%

STA

TE P

RO

BABIL

ITY

3+ R

ATE

Historical Smooth State Probability and 3+ Delinquency Rates

P(Stressed State) P(Base State) Extrapolated 3+ Delinquency Rate Fitted E(3+)

Assessment: Sensitivity of state detection and sense-checking against real-life eventsOur AR(1) MS-model appears to be able to capture a large number of major macroeconomic events that might have led to shifts in macro paradigms.

1974 oil crisis

1980 start of deregulation 1989 Mortgage

Crisis

1988 Change in tax rules - Highest level of transactions ever recorded

Great Moderation and building up risks

2004 BOE raised rate

2008 Lehman Brothers Bankruptcy

1998 BOE rate peaked

2004 BOE cut rate

QE Era

Experimental Results

33

Forecast performance benchmarkingDoes it forecast better than the next-best benchmark model?

34

Performance benchmarkingFor short-run forecasts, forward-chaining 1-step ahead cross-validation indicates that the AR(1)MS-Model outperforms a AR(1) model significantly. For longer-run forecasts, similar conclusion can be reached for a 5 year hold-out sample test.

Experimental Results

0.0%

0.5%

1.0%

1.5%

2.0%

2.5%

MAE

Average out-of-sample percentage forecast errors over 20 1-step ahead forecasts

(MAE)

AR(1)MS AR(1)

0.0%0.5%1.0%1.5%2.0%2.5%3.0%

RMSE

Average out-of-sample percentage forecast errors over 20 1-step ahead forecasts

(RMSE)

AR(1)MS AR(1)

0.0%

1.0%

2.0%

3.0%

4.0%

5.0%

MAE

Average out-of-sample percentage forecast errors for 5 years hold-out sample (MAE)

AR(1)MS AR(1)

0.0%1.0%2.0%3.0%4.0%5.0%6.0%

RMSE

Average out-of-sample percentage forecast errors for 5 years hold-out sample (RMSE)

AR(1)MS AR(1)

35

Areas in further developmentWhat’s next?

36

Areas in further developmentLooking ahead

Development areas Benefits Drawbacks

Extend to VAR MS-model

• Capture interdependency among default risk and macro factors

Contagion dynamicsKey topic: does default risk in one market/industry today affects another tomorrow?• Capture shifts in paradigm of macro factors.• Self-sufficient forecasting system.

• Require even more data• More complex to implement• May not lead to improve forecast

precision

Extend to provide interval forecasts

• Align with BOE stress-testing format with multiple scenarios under both baseline and stressed setting.

• Provide a distributional perspective to forecast uncertainties allowing for deviations from normality assumptions.

• No close-form solutions and will require Monte Carlo simulation.

37

Questions?Thank you!

This publication has been written in general terms and we recommend that you obtain professional advice before acting or refraining from action on any of the contents of this publication. Deloitte LLP accepts no liability for any loss occasioned to any person acting or refraining from action as a result of any material in this publication.

Deloitte LLP is a limited liability partnership registered in England and Wales with registered number OC303675 and its registered office at 2 New Street Square, London, EC4A 3BZ, United Kingdom.

Deloitte LLP is the United Kingdom affiliate of Deloitte NWE LLP, a member firm of Deloitte Touche Tohmatsu Limited, a UK private company limited by guarantee (“DTTL”). DTTL and each of its member firms are legally separate and independent entities. DTTL and Deloitte NWE LLP do not provide services to clients. Please see www.deloitte.com/about to learn more about our global network of member firms.

© 2017 Deloitte LLP. All rights reserved.