Overview of Main Quality Diagnostics

Anu PeltolaEconomic Statistics Section, UNECE

UNECE Workshop on Seasonal AdjustmentUNECE Workshop on Seasonal Adjustment20 – 23 February 2012, Ankara, Turkey

February 2012 UNECE Statistical Division Slide 2

Overview

Purpose of quality diagnostics Main quality issues Main results First visual checks Pre-processing Decomposition Main quality diagnostics

February 2012 UNECE Statistical Division Slide 3

Purpose of Quality Diagnostics

Seasonality is identified based on hypotheses • Seasonal component is estimated = is uncertain

Diagnostics will reveal any essential weaknesses in seasonal adjustment

Help draw attention to problematic issues• They prevent the use of misleading results that could

lead to false signals The automatic procedure in Demetra+ is

reliable! • But diagnostics are especially important for analysing

in detail the aggregate series

February 2012 UNECE Statistical Division Slide 4

Main Quality Issues

Appropriateness of the identified model and components

Number and type of outliersStability of the seasonal componentAbsence of residual seasonality and

residual calendar effectsMagnitude of the possible phase

delay

February 2012 UNECE Statistical Division Slide 5

Main results inform you about…

Estimation time span used for identifying the seasonal pattern

Application of log-transformation If there working day, Easter or Leap year

effects were identified If outliers were found and when A summary quality diagnostics

February 2012 UNECE Statistical Division Slide 6

Visual checks

To find seasonal breaks and high variability Problematic with moving averages, fitting the ARIMA model and finding

effects

February 2012 UNECE Statistical Division Slide 7

Pre-processing

Statistical properties of the ARIMA model

Regression variables The pre-adjusted series Residuals

• should be independent and random and follow normal distribution

February 2012 UNECE Statistical Division Slide 8

Decomposition

Stochastic series presents the results Cross-correlation of results

• In theory, components should be uncorrelated• A green p-value in Demetra+ would indicate

insignificant cross-correlation Estimator

Estimate

P-Value

Trend/Seasonal -0.1250 -0.1504 0.8018

Trend/Irregular -0.0450 -0.0856 0.7311

Seasonal/Irregular

0.0446 0.0195 0.5900

February 2012 UNECE Statistical Division Slide 9

Quality Diagnostics

Presence of seasonality Spectral graphics Revision history Sliding spans Model stability analysis

February 2012 UNECE Statistical Division Slide 10

Presence of Seasonality

Friedman test & Kruskall-Wallis test• Is there stable seasonality?

Evolutive seasonality test• Is there moving seasonality?

Combined seasonality test• Is there identifiable seasonality?

Residual seasonality test• Is there seasonality left in residuals in the entire series

or in the last 3 years of data?

February 2012 UNECE Statistical Division Slide 11

Spectral Graphics

Periodogram Auto-regressive

spectrum• Analyse the

residuals, irregular component and seasonally adjusted series for remaining seasonal or trading day effects

Spectral graphics of the residuals

February 2012 UNECE Statistical Division Slide 12

Revision History

Analyses revisions that happen when new observations are added at the end of the series

February 2012 UNECE Statistical Division Slide 13

Sliding Spans

Analyses stability of • Seasonal component• Trading day effect (if present) • Seasonally adjusted series

Slidings spans of the seasonal component

February 2012 UNECE Statistical Division Slide 14

Model Stability Analysis

Calculates ARIMA parameters and coefficients of regression variables for different periods

Computes the results on a moving window of eight years which slides by one year

The points correspond to the successive estimations

Strong movement of values from negative to positive indicates instability

February 2012 UNECE Statistical Division Slide 15

Problematic Issues

Which are the most essential tests? How to read and understand the

diagnostics? When does a result signify bad quality? What to do to improve results? Which poor results of quality diagnostics

could be accepted? Which quality diagnostics could be published

to the users?