RESEARCH & DEVELOPMENT STATISTICS (NBB)

Jean PalateDavid de Antonio Liedo

Christian-Albrechts-Universität zu KielInstitut für Statistik und Ökonometrie

July 2015

Nowcasting German GDPwith

Licensed under the EUPL (http://ec.europa.eu/idabc/eupl). The last updated version of the software can be downloaded herehttps://github.com/jdemetra/jdemetra-app/releases/tag/v2.0.0

- Humans have limited capacity to process information and interprete it.

- Confirmation bias , wishful thinking, and group think: pervasive in macroeconomic forecasting.

2000

Q1

2000

Q4

2001

Q3

2002

Q2

2003

Q1

2003

Q4

2004

Q3

2005

Q2

2006

Q1

2006

Q4

2007

Q3

2008

Q2

2009

Q1

2009

Q4

2010

Q3

2011

Q2

2012

Q1

2012

Q4

2013

Q3

2014

Q2

2,000,000

2,050,000

2,100,000

2,150,000

2,200,000

2,250,000

2,300,000

2,350,000

2,400,000

2,450,000

2011Q3

EA12 GDP Chain linked volumes (2010), million

euro

LINKING TECHNOLOGY IN A REAL-TIME FORECASTING ENVIRONMENT

Monitoring the macro economy in real time and detecting turning points requires certain skills and intuition

Technology can help …

LINKING TECHNOLOGY IN A REAL-TIME FORECASTING ENVIRONMENT

Monitoring the macro economy in real time and detecting turning points requires certain skills and intuition

Technology can help …

Red Bull Racing Chief Technical Officer Adrian Newey Source: Mark Thompson/Getty Images AsiaPac

Sebastian Vettel driving for Red Bull Racing in 2010.Photographer: Andrew Hoskins at British Grand Prix

LINKING TECHNOLOGY IN A REAL-TIME FORECASTING ENVIRONMENT

Monitoring the macro economy in real time and detecting turning points requires certain skills and intuition

Technology can help … TODAY: real-time simulation

Sebastian Vettel driving for Red Bull Racing in 2010.Photographer: Andrew Hoskins at British Grand Prix

Simulate real-time forecasts

Forecasting uncertainty as a function of the news-flow

REAL-TIME FORECASTING EVALUATION PLUG-IN

Simulate real-time forecasts

Forecasting uncertainty as a function of the news-flow

Replace the concept of “forecast horizon” by “information set”

REAL-TIME FORECASTING EVALUATION PLUG-IN

1. WHAT IS JDEMETRA (JD) +

A Real-Time Forecasting Evaluation Library

1. WHAT IS JDEMETRA (JD) +

2. MODELING THE REAL-TIME NEWSFLOW

A Real-Time Forecasting Evaluation Library

1. WHAT IS JDEMETRA (JD) +

2. MODELING THE REAL-TIME

NEWSFLOW

3. NEXT STEPS

A Real-Time Forecasting Evaluation Library

GERMAN GDP Defining the calendar Estimation In Sample analysis Out-of-Sample (Real Time simulation) News Analysis

JDEMETRA+ is Pure Java software• Mainly (>95%) based on libraries written by Research &

Development (NBB)• Complete control• High-performance (compared to Matlab…)• No economic cost for the user: Open Access software licensed under

the EUPL (http://ec.europa.eu/idabc/eupl)• It has been designed for extension (today you will see the proof)

JDEMETRA+ provides many useful services Primary goal remains seasonal adjustment (TRAMO-SEATS and X12). Externalities: temporal disaggregation (Chow-Lin, Fernandez,

Litterman), benchmarking (Denton, Cholette), Outliers detections, chain linking, etc…

On-going: Multivariate models (SUTSE, DFM, BVAR) Dynamic access to different sources: Excel, Txt, SAS, Databases… Rich graphical components Storage of current work through workspace… Graphical interface based on NetBeansInternational Cooperation Maintenance partly ensured by the Bundesbank (X11) Support of the SA Center of Excellence (INSEE, ONS, ISTAT, STATEC,

EUROSTAT…)

1.WHAT IS JD+

2. MODELING THE NEWSFLOW

EXPECTATIONSformation and

updating

Econometric & Statistical

tools

JD+ defines nowcasting in terms of the dynamic interactions of real world “things”:

A. The newsflow (potentially “Big Data”- V3: Volume/Variety/Velocity)

B. Technologies for signal extraction (e.g. short-term forecasting methods)

C. Interpretation of changes in expectations in terms of the news

Nowcasting model

A

B

C

2. MODELING THE NEWSFLOW

EXPECTATIONSformation and

updating

Econometric & Statistical

tools Rather than evaluating the properties of a given econometric tool ( ), our aim is to evaluate the “nowcasting model” as a whole.

Real-Time Forecasting Evaluation

A

B

C

B

SOME DATA ISSUES BEFORE WE START….

A quick look at Production Index Manufacturing (Germany)Before we start

(includes manufacturers, mines, and utilities)

Production Index Manufacturing vs Industrial Output

Note 1: Industrial production in manufacturing looks very much like industrial output

(includes manufacturers, mines, and utilities)

Production Index Manufacturing vs Industrial Output

Note 1: Industrial production in manufacturing looks very much like industrial outputNote 2: Industrial output “first release” is more volatile than the last available series

Production Index Manufacturing vs Industrial Output(includes manufacturers, mines, and utilities)

Note 1: Industrial production in manufacturing looks very much like industrial outputNote 2: Industrial output “first release” is more volatile than the last available series

Q1) Can the “prelim” data have a larger variance? It goes against the news hypothesis, but thera are ways to work out a rational explanationQ2) Does it imply that revisions are predictable? Not necesarilyQ3) Which series do we choose? I have no choice: only “first” is available in real-time

This periodogram of the revisions together with Note 2implies that a significant part of the variance of the “first release”seems to be removed in the revision process.

Production Index Manufacturing vs Industrial Output(includes manufacturers, mines, and utilities)

Industrial Output (MoM% Germany)

Advanced Release calendar and seasonally adjusted

Industrial Output (MoM% Germany)

Advanced Release calendar and seasonally adjusted

Similar performance to the market, even if not attempt has been made to exploit residual seasonality/calendar, which could be (not necesarily) predictable

Industrial Output (MoM% Germany)

Advanced Release calendar and seasonally adjusted

MODELING THE GERMAN REAL-TIME

DATAFLOW

Some examples for GDP Real-time publication schedule

Real-time data (instead of revised)

Camacho M. and G. Pérez-Quirós (2010)     Real-time Real-time

De Antonio Liedo (2014) «Nowcasting Belgium» Real-time Real-time

Banbura, Giannone, Modugno, Reichlin (2012) Real-time Real-timeGiannone, Reichlin and Simonelli (2009) Real-time Real-timeGDPnow Real-time Real-timeBarnett et al. (2014) « nowcasting Nominal GDP» Real-time Real-timeAngelini, Camba-Mendez, Giannone, Reichlin and Rünstler (2011)

Stylized Revided

Banbura and Modugno (2014) Stylized Revised

Kuzin, Marcelino and Schumacher (2011) Stylized Revised

Piette (2015) bridge with targeted predictors based on elastic-net

Stylized Revised

REAL-TIME FORCASTING EVALUATION“Small sample” of the literature: Simulating real-time

forecasts in macro since Giannone, Reichlin and Small (2008) and Evans (2005)

Analysis of data revisions:"A Real-Time Data Set for Macroeconomists," Dean Croushore and Tom Stark, Journal of Econometrics 105 (November 2001),

First real-time database for German GDP:Clausen and Meier (2003)

Some examples for GDP Real-time publication schedule

Real-time data (instead of revised)

Camacho M. and G. Pérez-Quirós (2010)  small model/calendar

Real-time Real-time

De Antonio Liedo (2014) «Nowcasting Belgium» Real-time Real-time

Banbura, Giannone, Modugno, Reichlin (2012) Real-time Real-timeGiannone, Reichlin and Simonelli (2009) Real-time Real-timeGDPnow Real-time Real-timeBarnett et al. (2014) « nowcasting Nominal GDP» Real-time Real-timeAngelini, Camba-Mendez, Giannone, Reichlin and Rünstler (2011)

Stylized Revided

Banbura and Modugno (2014) Stylized Revised

Kuzin, Marcelino and Schumacher (2011) Stylized Revised

Piette (2015) bridge with targeted predictors based on elastic-net

Stylized Revised

REAL-TIME FORCASTING EVALUATION“Small sample” of the literature: Simulating real-time

forecasts in macro since Giannone, Reichlin and Small (2008) and Evans (2005)

Surprisingly, it took time to formalize the other important dimension of real-time data. First papers to focus on the “real-time dataflow”:-Giannone, Reichlin and Small (2008) , Journal of Monetary Economics-Evans (2005), International Journal of Central Banking

Some examples for GDP Real-time publication schedule

Real-time data (instead of revised)

Camacho M. and G. Pérez-Quirós (2010) small model/calendar    

Real-time Real-time

De Antonio Liedo (2014) «Nowcasting Belgium» Real-time Real-time

Banbura, Giannone, Modugno, Reichlin (2012) Real-time Real-timeGiannone, Reichlin and Simonelli (2009) Real-time Real-timeGDPnow Real-time Real-timeBarnett et al. (2014) « nowcasting Nominal GDP» Real-time Real-timeAngelini, Camba-Mendez, Giannone, Reichlin and Rünstler (2011)

Stylized Revided

Banbura and Modugno (2014) Stylized Revised

Kuzin, Marcelino and Schumacher (2011) Stylized Revised

Piette (2015) bridge with targeted predictors based on elastic-net

Stylized Revised

The publication calendar is a key parameter in our forecasting evaluation set-up (GRS2008)

REAL-TIME FORCASTING EVALUATION“Small sample” of the literature: Simulating real-time

forecasts in macro since Giannone, Reichlin and Small (2008) and Evans (2005)

Some examples for GDP Real-time publication schedule

Real-time data (instead of revised)

Camacho M. and G. Pérez-Quirós (2010) small model/calendar    

Real-time Real-time

De Antonio Liedo (2014) «Nowcasting Belgium» Real-time Real-time

Banbura, Giannone, Modugno, Reichlin (2012) Real-time Real-timeGiannone, Reichlin and Simonelli (2009) Real-time Real-timeGDPnow Real-time Real-timeBarnett et al. (2014) « nowcasting Nominal GDP» Real-time Real-timeAngelini, Camba-Mendez, Giannone, Reichlin and Rünstler (2011)

Stylized Revided

Banbura and Modugno (2014) Stylized Revised

Kuzin, Marcelino and Schumacher (2011) Stylized Revised

Piette (2015) bridge with targeted predictors based on elastic-net

Stylized Revised

The publication calendar is a key parameter in our forecasting evaluation set-up (GRS2008)

Simplified “vintage-based estimation” only for key variables à la Jacobs and van Norden (2011) or Clements and Galvao (2013): “advanced” vs “last available”

REAL-TIME FORCASTING EVALUATION“Small sample” of the literature: Simulating real-time

forecasts in macro since Giannone, Reichlin and Small (2008) and Evans (2005)

The publication calendar is a key parameter in our forecasting evaluation set-up (GRS2008)

Simplified “vintage-based estimation” only for key variables à la Jacobs and van Norden (2011) or Clements and Galvao (2013): “advanced” vs “last available”

Our tool will save you a lot of time

REAL-TIME FORCASTING EVALUATION“Small sample” of the literature: Simulating real-time

forecasts in macro since Giannone, Reichlin and Small (2008) and Evans (2005)Some examples for GDP Real-time

publication scheduleReal-time data (instead of revised)

Camacho M. and G. Pérez-Quirós (2010) small model/calendar    

Real-time Real-time

De Antonio Liedo (2014) «Nowcasting Belgium» Real-time Real-time

Banbura, Giannone, Modugno, Reichlin (2012) Real-time Real-time

Giannone, Reichlin and Simonelli (2009) Real-time Real-time

GDPnow Real-time Real-time

Barnett et al. (2014) « nowcasting Nominal GDP» Real-time Real-time

Angelini, Camba-Mendez, Giannone, Reichlin and Rünstler (2011)

Stylized Revided

Banbura and Modugno (2014) Stylized Revised

Kuzin, Marcelino and Schumacher (2011) Stylized Revised

Piette (2015) bridge with targeted predictors based on elastic-net

Stylized Revised

1) Just introduce the publication delay for each series ...

2) Decide when to update your forecasts (e.g. in this example, the days when GDP flash, IFO Surveys and industrial production are released)

“vintagebased”

IFOIFO

α t β t

Q-ML under “weak” cross correlation patterns: Doz et al. (2012)

“vintagebased”

IFOIFO

α t β t

3) next, specify your state=space model: SUTSE, DFM, BVAR

Measurement Equation: y𝑡¿=Z α t+ Λ βt+ξ t ¿

(β tα t)=(T 111 T 12

1

T 211 T 22

1 )( β t−1

α t−1)+…+(T11

𝑝 T 12𝑝

T21𝑝 T 22

𝑝 )(β t −𝑝

αt −𝑝)+(uβ ,tuα ,t)State Equation:

Usual identification assumptions

Q-ML under “weak” cross correlation patterns: Doz et al. (2012)

“vintagebased”

IFOIFO

α t β t

3) next, specify your state=space model: SUTSE, DFM, BVAR

Measurement Equation: y𝑡¿=Z α t+ Λ βt+ξ t ¿

(β tα t)=(T 111 T 12

1

T 211 T 22

1 )( β t−1

α t−1)+…+(T11

𝑝 T 12𝑝

T21𝑝 T 22

𝑝 )(β t −𝑝

αt −𝑝)+(uβ ,tuα ,t)State Equation:

Usual identification assumptions

“vintagebased”

IFOIFO

α t β t

3) next, specify your state=space model: SUTSE, DFM, BVAR

Weighted averageof the MoM% factorsto approximate QoQ% rates

Cumulative sum over 12 months

Monthly growth rates

Data can be seasonallyadjusted in real-timeand transformed into growth rates

In this example, most data are alreadytransformed andsurveys can be leftuntransformed

“vintagebased”

IFOIFO

α t β t

3) next, specify your state=space model: SUTSE, DFM, BVAR

Weighted averageof the MoM% factorsto approximate QoQ% rates

Camacho M. and G. Pérez-Quirós (2010) use «YoY» instead of «Q» to link the PMI, IFO and NBB monthly Surveys in their «eurosting» model for the euro area.Their link also applies to the measurement error (à la Mariano-Murasawa); not in our case.

… and estimate it

Principal Components

… and estimate it

Principal Components

EM algorithmBanbura and Modugno (2010)

… and estimate it

Principal Components

EM algorithmBanbura and Modugno (2010)

Numerical Optimization Uses EM to initialize. Algorithms:- Levenberg-Marquardt- Broyden–Fletcher–Goldfarb–Shanno Options: - Simplified iterations - Iterations by blocks

Final EM algorithm

… and estimate it

Principal Components

EM algorithmBanbura and Modugno (2010)

Numerical Optimization Uses EM to initialize. Algorithms:- Levenberg-Marquardt- Broyden–Fletcher–Goldfarb–Shanno Options: - Simplified iterations - Iterations by blocks

Final EM algorithm

… and estimate it

IN-SAMPLE

Correlation of Measurement Errors

Business Expectations IFO

Markit PMI (Manufactures)

Correlation of Measurement Errors

GDP final

GDP flash

OUT-OF-SAMPLE

4) Define evaluation sample and dates at which model parameters must be re-estimatedFor univariate models, recursive estimation every month, while multivariate models may be re-estimated once or twice per year, depending on the application

5) Visualize results

Real GDP growth (flash)

Real GDP growth (final)

Real-time updates for GDP growth

Simulated release calendar

Simulated release calendar

Real-time updates for GDP growth

Simulated release calendar

Real-time updates for GDP growth

Simulated release calendar

Real-time updates for GDP growth

Theoretical Forecasting UncertaintyThe forthcoming (unpredictable) news flow determines the size ofthe RMSE as a function of the information set

Theoretical RMSE around nowcast for GDP (final)

days before (-)or after (+) the end of the quarter

Theoretical Forecasting UncertaintyThe forthcoming (unpredictable) news flow determines the size ofthe RMSE as a function of the information set

Theoretical RMSE around nowcast for GDP (final)

Empirical(2005-2014)

days before (-)or after (+) the end of the quarter

Theoretical Forecasting UncertaintyThe forthcoming (unpredictable) news flow determines the size ofthe RMSE as a function of the information set

Theoretical RMSE around nowcast for GDP (final)

Empirical(2005-2014)

!

days before (-)or after (+) the end of the Quarter(notice x-axis isnot scaled)

Forecasting Uncertainty for Real GDP%

• Toy model with only Industrial Production (preliminary) and IFO Business Expectations performs only a bit worse than the model with ten variables (Camacho M. and G. Pérez-Quirós (2010) advocate for small models)

• Surprising that the introduction of export expectations (Kiel10 X) doesn’t have a larger impact

days before (-)or after (+) the end of the quarter

Forecasting Uncertainty for Real GDP%

This makes sense(but revised IPI is notavailable in real-time)

• Toy model with only Industrial Production (preliminary) and IFO Business Expectations performs only a bit worse than the model with ten variables (Camacho M. and G. Pérez-Quirós (2010) advocate for small models)

• Surprising that the introduction of export expectations (Kiel10 X) doesn’t have a larger impact

days before (-)or after (+) the end of the quarter

Forecasting Uncertainty for Real GDP%

Counter Intuitivethat using revised IPIworsens it

This makes sense(but revised IPI is notavailable in real-time)

• Toy model with only Industrial Production (preliminary) and IFO Business Expectations performs only a bit worse than the model with ten variables (Camacho M. and G. Pérez-Quirós (2010) advocate for small models)

• Surprising that the introduction of export expectations (Kiel10 X) doesn’t have a larger impact

days before (-)or after (+) the end of the quarter

The paradox explainedDoes it make sense that the model is consistent with an inferior performancewhen the quality of the industrial production has improved?

Counter Intuitivethat using revised IPIworsens it

I believe it does

days before (-)or after (+) the end of the quarter

Hyndman, R. J. and Koehler A. B. (2006). "Another look at measures of forecast accuracy." Diebold, F.X. and R.S. Mariano (1995)Diebold, F.X. (2013), “Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective …”

6) Quantify Results

NEWS

Banbura and Modugno (2010)Journal of Applied Econometrics

“News” in the real-time dataflow

Definition: unexpected component of a given data release or revision Mathematically, the vector of news

Synonyms: innovation, surprise, shock

Note 1: This definition implies that news cannot be read if we do not have a prior expectation

Note 2: The vector of news can be large, specially if a given release incorporates historical data revisions

, 𝐼 𝑣+1 , 𝐼 𝑣+1

𝐼 𝑣+1𝐼 𝑣+1 𝐼 𝑣+1𝐼 𝑣+1

Updating the forecast on the basis of news

Assume only one indicator is released

*

CES-IFO

“Quality” mattersDefinition: quality is defined here as the correlation between the factor and the news

Assume only two indicators are released

CES-IFO

MARKIT

“Quality” mattersDefinition: quality is defined here as the correlation between the factor and the news

Assume one indicator was earlier

*

Markit-PMI is

earliest

“Timeliness” also matters Definition: timeliness refers to the habit of being available at the forecaster’s information set earlier than other indicators

<Weight is higher

Once Markit-PMI is published, the news content would be smaller (because of the correlation with CES-IFO), so the impact“wx news” will be smaller for the subsequent CES-IFO release

1

2

“Timeliness” also matters

• This simple mathematical expression has explained the importance of timeliness ( and )

• This larger “impact” coefficient is translated into tangible phenomena:

- more citations (FT, Bloomberg)- the ability to have an effect in market expectations- a higher economic value

• The obvious implication: survey data providers may have incetives to release their data as early as possible (without compromising on their quality, which can be objectively evaluated too)

1 2

Updating the forecast today

Today: 9 july 2015

15 december 2014

Updating the forecast today

Today: 9 july 2015

15 december 2014

IPI june CES-IFO CES-IFO

Markit

IPI march

IPI aprilFlash

Markit

CES-IFO

IPI june CES-IFO CES-IFO

Markit

IPI march

IPI aprilFlash

Markit

CES-IFO

Relative impacts can change if timeliness assumption is modified

C Getty Images

Photo: Urban Events

You are the pilot

• Think about the most suitable forecasting model

• Understand the data and assess model fit

• Before using your model out-of-sample , use our “simulator” to become aware of the risks

SUMMARY

Features

• Simulates forecasting scenarios using real-time data availability (users can define the release calendar in a simple manner)

• Check whether a new model yields statistically significant gains in forecasting accuracy with respect to alternatives

• Robust quantification of forecast accuracy as a function of the information available (ongoing: test release impacts)

• Many measures of forecast accuracy and possibility to perform analysis by subsamples

(ongoing: Giacomini and Rossi, 2010)

You are the pilot

• Think about the most suitable forecasting model

• Understand the data and assess model fit

• Before using your model out-of-sample , use our “simulator” to become aware of the risks

• Good luck!

SUMMARY

(α=5%)

Features

• Simulates forecasting scenarios using real-time data availability (users can define the release calendar in a simple manner)

• Check whether a new model yields statistically significant gains in forecasting accuracy with respect to alternatives

• Robust quantification of forecast accuracy as a function of the information available (ongoing: test release impacts)

• Many measures of forecast accuracy and possibility to perform analysis by subsamples

(ongoing: Giacomini and Rossi, 2010)

Supplementary material

PERIODOGRAMDecomposes the sum of squares of the

growth rates in terms of the Fourier coefficients

A quick look at Production Index Manufacturing (Germany)

(includes manufacturers, mines, and utilities)

(includes manufacturers, mines, and utilities)