PROJECT A2

OECD FORECASTS

DURING & AFTER THE

FINANCIAL CRISIS

NAEC Seminar, 20 November 2013

Speakers: • Sveinbjorn Blondal, ECO, OECD • Nigel Pain, ECO, OECD • Christine Lewis, ECO, OECD

A POST-MORTEM

Motivation and context

• NAEC: reviewing the crisis and identifying lessons for future forecasting performance.

• Regular forecast post-mortems by ECO.

• Three aspects of the project:

– Quantitative evaluation of the forecasts.

– What structural, financial and policy-related factors are related to the errors?

– Interviews on post-crisis changes with experts from other international organisations.

Ov

erv

iew

2

• Three sets of forecasts:

– May Economic Outlook for the next year;

– November Economic Outlook for the next year;

– May Economic Outlook for the same year

• Outturn data

– The published data in May after the year being

forecast

• Focus on calendar year GDP growth forecasts

– The project also considered inflation forecasts

and quarterly forecasts for the large economies

3

Data set and definitions T

he

da

ta

PROPERTIES OF THE RECENT FORECAST

ERRORS

4

The downturn was not foreseen

• Forecasts of (Q4/Q4) OECD GDP growth in 2008 and 2009 were revised down substantially.

Pro

per

ties

of

the

fore

cast

err

ors

5

-3

-2

-1

0

1

2

3

May-07 Nov-07 May-08 Nov-08 May-09

Forecasts of GDP growth in 2008

Outturn

%

-3

-2

-1

0

1

2

3

May-08 Nov-08 May-09 Nov-09 May-10

Forecasts of GDP growth in 2009

Outturn

%

The recovery has been mixed

• OECD GDP growth (Q4/Q4) rebounded more quickly than initially expected in 2010.

• But disappointments resumed in 2011 and 2012

Pro

per

ties

of

the

fore

cast

err

ors

6

0

1

2

3

4

May-09 Nov-09 May-10 Nov-10 May-11

Forecasts of GDP growth in 2010

Outturn

%

0

1

2

3

4

May-10 Nov-10 May-11 Nov-11 May-12

Forecasts of GDP growth in 2011

Outturn

%

The biggest error in the 18-month ahead

calendar year growth forecasts was 2009.

Pro

per

ties

of

the

fore

cast

err

ors

7

Errors are typically smaller at shorter

forecast horizons.

Pro

per

ties

of

the

fore

cast

err

ors

Forecast error = outturn less forecast 8

On average, the errors for the BRIICS are

similar to those for OECD countries.

Pro

per

ties

of

the

fore

cast

err

ors

Forecast error = outturn less forecast 9

Pro

per

ties

of

the

fore

cast

err

ors

Forecast errors were largest in the

vulnerable euro area countries

10

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

0.5

OECD Euro areacore

Euro areavulnerable

Other OECDEurope

Rest ofOECD

BRIICS

Average growth forecast error 2007-12

May current year

Nov forecast for following year

May forecast for following year

%pt

Pro

per

ties

of

the

fore

cast

err

ors

The pattern of forecast accuracy is

similar, though differences are smaller

11

0

1

2

3

4

5

OECD Euro areacore

Euro areavulnerable

Other OECDEurope

Rest ofOECD

BRIICS

RMSE of GDP growth forecasts 2007-12

May current year

Nov forecast for following year

May forecast for following year

%pt

The root mean squared error (RMSE) could be used to put

confidence intervals around a point forecast.

EXPLAINING FORECAST ERRORS

12

Global interconnectedness rose substantially

prior to the crisis, with rising global & national

imbalances.

Ex

pla

inin

g t

he

erro

rs

13

0

40

80

120

160

0

20

40

60

80

1995 2007 1995 2007 1995 2007

Trade openness(LHS, % of GDP)

Financial openness(RHS, % of GDP)

Foreign banks(LHS, % of total)

Global trade and financial linkages

Note: Trade openness is the average of all countries; the share of foreign banks and financial openness are the median OECD country.

• International trade and financial openness.

• Banking sector information.

• Economy-wide regulations.

• Pre-crisis imbalances.

• Survey information.

• Fiscal consolidation.

• The euro area crisis.

14

What factors could be correlated with

recent growth forecast errors?

Ex

pla

inin

g f

ore

cast

err

ors

In the downturn (May 2008 for 2008-09) Relationship

Trade openness -

Foreign bank penetration -

Factors correlated with forecast errors (+/- indicate growth stronger/weaker than expected)

Ex

pla

inin

g t

he

erro

rs

In the recovery (May 2010 for 2010-11 and May 2011 for 2011-12)

Relationship

Better capitalised banks pre-crisis +

Increases in non-performing loans -

Equity price growth +

Improved consumer sentiment +

15

• Forecast error = Outturn - Forecast

• This relationship is similar for average errors over the full period

The downturn was stronger than projected in

more open economies (negative spillovers)

Cumulative growth forecast errors for 2008-09, made in May 2008

Ex

pla

inin

g t

he

erro

rs

16

-16

-12

-8

-4

0

0 50 100

Foreign banks' assets (% total)F

ore

cast

err

or

(%pt)

-16

-12

-8

-4

0

0 100 200Trade openness

Fore

ca

st

err

or

(%p

t)

There were downside surprises in 2010-11 in

countries with lower pre-crisis bank capital

Ex

pla

inin

g t

he

erro

rs

17 Bank capital is the capital adequacy of deposit-takers, measured as

a ratio of total regulatory capital to risk-weighted assets.

-8

-4

0

4

8

8 12 16 20

Bank regulatory capital in 2007 (%)

Fo

reca

st

err

or

(%p

t)

Growth forecast errors for 2010-11, from May 2010

And also where the financial system

was weakening

Ex

pla

inin

g t

he

erro

rs

18

-12

-8

-4

0

4

-5 0 5 10 15

Change in non-performing loans, 2011-12 (%pt)

Fo

reca

st

err

or

(%p

t)

Growth forecast errors for 2011-12, from May 2011

Growth forecast errors over 2007-12 were

larger in more regulated economies

Ex

pla

inin

g t

he

erro

rs

19 Indicators are the OECD product market regulation index and the OECD

measure of the strictness of employment protection (for regular workers)

0

1

2

3

Least regulated Middle Most regulated

Group mean by product market

Group mean by labour market

Degree of regulation for market indicated (2008)

RMSEs of November projections for next year

• IMF: fiscal multipliers under-estimated in

recovery:

– growth weaker than expected in countries with

stronger projected fiscal consolidation.

• Alternatively, actual consolidation could have

been stronger than projected consolidation.

• Growth disappointments also coincided with

the euro area crisis.

• What does the OECD evidence say?

20

Forecast errors, fiscal consolidation and

fiscal multipliers

Ex

pla

inin

g f

ore

cast

err

ors

Ex

pla

inin

g t

he

erro

rs

Growth weaker than projected in countries with

more fiscal consolidation, but only in Europe.

21

Ex

pla

inin

g t

he

erro

rs

Incorrect assumptions about euro crisis and

govt. bond spreads also a source of error at the

same time.

22

• Yes, growth disappointments in countries with

stronger projected consolidation.

• Yes, growth disappointments in countries with

stronger consolidation than projected.

• But only in Europe, and only if Greece is

included.

• The bond spread errors are a more important

source of growth forecast errors

– confirmed by econometric evidence.

23

Forecast errors, fiscal consolidation and

fiscal multipliers: OECD evidence

Ex

pla

inin

g f

ore

cast

err

ors

PUTTING THE FORECAST ERRORS IN CONTEXT

24

An historical perspective: OECD growth forecast

errors in 2009 and 2010 are similar to the early 1970s

Pu

ttin

g t

he

erro

rs i

n c

on

tex

t

25

Recent errors were larger than in the “Great

Moderation” but smaller than the 1970s…

Pu

ttin

g t

he

erro

rs i

n c

on

tex

t

26

0

1

2

3

4

1971-1981 1982-1990 1991-2006 2007-2012

May projection for current year

November projection for next year

May projection for next year

%pt RMSEs of growth forecasts - G7 countries

Unweighted average of errors in G7 economies

…and are not that different if growth volatility

is accounted for

Pu

ttin

g t

he

erro

rs i

n c

on

tex

t

27

0.0

0.5

1.0

1.5

2.0

2.5

1971-1981 1982-1990 1991-2006 2007-2012

May projection for current year

November projection for next year

May projection for next year

RMSEs adjusted for outcomes - G7 countries

Ratio of GDP growth RMSE to standard deviation of GDP growth

Pu

ttin

g t

he

erro

rs i

n c

on

tex

t The forecasts were considerably worse

than usual for some countries

Dark blue and orange indicate statistical significance (at the 10% level) 28

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

CAN FRA DEU ITA JPN GBR USA

Additional effect in 2007-12

Average error 1982-2006

%ptGrowth forecast errors in 2007-12 compared to the past

From May projections for the next year

OECD countries: 2007-12 Accelerations Decelerations

Number in period 56 118

% correct: projections for same year 86 88

% correct: projections for next year 91 50

29

Directional accuracy is good for growth

accelerations, but less so for growth slowdowns

Pu

ttin

g t

he

erro

rs i

n c

on

tex

t

Directional accuracy of May growth projections

G7 countries: 1982-2006 Accelerations Decelerations

Number in period 82 78

% correct: projections for same year 79 83

% correct: projections for next year 74 45

Tools to help identify growth slowdowns are

needed.

However, forecast errors were strikingly

similar across forecasters (also pre-crisis)

Pu

ttin

g t

he

erro

rs i

n c

on

tex

t

30

-6

-4

-2

0

2

4

2007 2008 2009 2010 2011 2012

From the same year

OECD IMF

Consensus EC

%pt

Average errors in forecasts from May for the year

ahead - G7 countries

-6

-4

-2

0

2

4

2007 2008 2009 2010 2011 2012

From one year ahead%pt

Unweighted average for the G7 economies

• They are conditional on a set of assumptions:

– financial market variables, fiscal policy changes,

commodity prices.

• They can point to the need for policy changes

to tackle unsustainable developments:

– If implemented, projection errors may occur.

• Politically-sensitive issues can cause errors:

– A two year projection had to assume the euro area crisis

would diminish.

31

The OECD forecasts are projections rather

than pure forecasts

Pu

ttin

g t

he

erro

rs i

n c

on

tex

t

CHANGES IN FORECASTING PRACTICES

AND PROCEDURES

32

• Greater centralisation of the forecast process.

• Enhanced monitoring of near-term

developments.

• Increased attention on financial sector.

• Enhanced focus on risk assessments and

global spillovers.

33

Post-crisis changes in forecast practices and

procedures at the OECD and elsewhere

On

go

ing

ch

an

ges

• Reflects errors common to all forecasts and

the importance of global financial and trade

interconnections.

– Early view on global developments and risks

and their implications.

– Early guidance via “top down” projections

bringing together information from different

sources and models.

34

Increased centralisation of the forecast

process

On

go

ing

ch

an

ges

• Surveys and high frequency data can provide

early signals of big changes.

– Nowcasting (OECD indicator models).

– Composite leading indicators.

– Evidence from business contacts (forecast

discussions with BIAC and TUAC).

– Use of internet-based indicators (“big data”)

35

Enhanced monitoring of near-term

developments.

On

go

ing

ch

an

ges

– OECD aggregate financial conditions indices

(used forecasts and scenarios).

– Incorporation of broader range of financial

variables in projections.

– Enhanced discussions with internal/external

financial market specialists.

– Incorporating banking sector and global

interconnectedness in macro models.

36

Increased attention on financial sector O

ng

oin

g c

ha

ng

es

• Reflects uncertainty about basic assumptions

behind projections.

– More information about the risks around the main

projections.

– Greater use of quantitative scenario analysis to

illustrate possible outcomes.

– Horizon scanning to plan ahead for unlikely but

potentially costly events.

37

Enhanced focus on risk assessments and

global spillovers

On

go

ing

ch

an

ges

• Forecasting in recent years proved very challenging and

growth has been repeatedly over-estimated.

• Global interconnectedness, structural policy settings and

the health of the banking sector are all related to forecast

errors.

• Errors in assumptions about the speed at which the euro

crisis would ease have been an important source of growth

forecast errors.

• Important changes are now taking place to forecasting

practices and procedures.

38

Summary

SPARE SLIDES

Pro

per

ties

of

the

fore

cast

err

ors

Inflation was under-estimated on

average

Inflation = percentage change in the private consumption deflator 40

Pu

ttin

g t

he

erro

rs i

n c

on

tex

t The accuracy of G7 forecasts over the

long-run

41

Pu

ttin

g t

he

erro

rs i

n c

on

tex

t

The indicator models were helpful

-10

-8

-6

-4

-2

0

2

4

2007:Q1 2008:Q1 2009:Q1 2010:Q1 2011:Q1 2012:Q1

Estimate 1 quarter ahead

Current quarter estimate

Outturn

Quarterly indicator model estimates and GDP growthG7, weighted average%

42