Paul A. Samuelson --If prediction is the ultimate aim of all science,

then we forecasters ought to award ourselves the palm for accomplishment, bravery or rashness.

… we (economists) are better than anything

480

… we (economists) are better than anything else in heaven and earth at forecasting aggregate business trends -- better than gypsy tea-leaf readers, Wall Street soothsayers and chartist technicians, hunch-playing heads of mail-order chains, or all-powerful heads of state.

Forecasting models have their limitations because they deal with human behavior and ever-changing institutions.

481

institutions.

Forecasting is an important part of the economic po licy-making process.

Forecasting Terminology

Forecast -- a quantitative estimate (or set of estim ates) about the likelihood of future events based on past and current information.

Point forecasts -- those which yield a single number

482

Point forecasts -- those which yield a single number

Interval forecasts -- indicated in each period the i nterval in which it is hoped the actual value will lie

“If you twist my arm, you can make me give a single number as a guess about next year’s GNP. But you will twi st hard. My scientific conscience would feel more comfortable giving you my subjective probability distribution for all the values of GNP.” -- Samuelson

1. Design Phase

2. Specification Phase

3. Evaluation Phase

Clearly & LevenbachThe Professional Forecaster

483

A forecast is a systematic process of decisions and actions performed in an effort to predict the future. A forecast is not an end product, but rather an input to the decision-making process.

� Conditional forecasts

conditional not only on the estimated structural parameters but also on the future values of the explanatory variables

� Illustration

484

� Illustration

� Point forecast ± c (standard error of the point forecast)

� c corresponds to the t-distribution with n-k-1 degrees of freedom

� To obtain a 95% confidence interval for the forecas t, choose α α α α = .05.

485

choose α α α α = .05.

� To obtain a 90% confidence interval for the forecas t, choose αααα = .10.

� To obtain a 99% confidence interval for the forecas t, choose αααα = .01.

� All probability statements conditional on a normal distribution of the forecast object.

Residual = dependent variable –predicted valueyxxxxy TT

ii1)(ˆ −=

486

A 95% confidence interval for the forecast of the dependent

variable. This band accounts for not only the variability due to estimating the parameters but

also the variability of the disturbance (error) term

Standard error of the predicted mean value

[ ] 21

12 ))(( Ti

Ti xxxxs −

� Predicted value ± t90,.025 (standard error of prediction)

� Standard error for individual forecast

[ ] 21

12 )))((1( Ti

Ti xxxxs −+

487

� For observation 1:

� predicted value = 1.9738

� t90,.025 = 1.987

� standard error of prediction = 4.2912

� 95% CI for prediction of observation 1

� 1.9738 ± (4.2912)(1.987)

Forecast Methods

QuantitativeQualitative

Formal Intuitive (Naïve)

488

Seasonal

TrendExtrapolation

Time Series Analysis

EconometricAnalysis

ARIMA VAR VEC

(1) Linear trend model

A time-series X t increases in constant absolute amounts each time period

(2) The series X t grows with constant percentage increases rather than constant absolute increase.

t10t e tbbX ++=

489

increases rather than constant absolute increase.

(3) Quadratic trend model

ttwrt

t wrtAXeAeX t ++== loglog

t2

210t utbtbbX +++=

(4) Cubic trend model

(5) Moving averages

(6) Exponential smoothing

t1nt1tt1t en/)X...XX(X ++++= +−++

t3

32

210t vtCtCtCCX ++++=

490

(6) Exponential smoothing

(7) Holt-Winters model

Use of ARIMA Models

t2t2

1tt1t e...X)1(X)1(XX ++α−α+α−α+α= −−+

Add factors -- take account of special circumstances and knowledge not embodied in the formal model.

Forecasts using econometric models are generally su perior

Forecasts explicitly conditional -- possible therefo re to investigate the sensitivity of the forecast to alte rnative assumptions, e.g., vary the forecasted level of X t+1 or to consider alternative add factors . 1tu +

491

Forecasts using econometric models are generally su perior to those based on simple extrapolation techniques.

Importance and value of add factors -- reflection of expert judgement on factors not included in the model.

Forecasts with subjective adjustments generally hav e been more accurate than those obtained from the purely m echanical application of the econometric model combination of model building and subjective expertise.

Backcasting “Historical” Ex-post Forecast Ex-anteSimulation Forecast

492

EstimationPeriod

Out-of-Sample Within-Sample Out-of-Sample Out-of-Sample

Source: Pindyck and Rubinfeld (1998)

T1 T2 T3(today)

Time, t

“In science and in real economic life, it is terrib ly important not to be wrong much.”

--- Samuelson (1965)

FORECAST EVALUATION

∑=

−

=M

ttt AF

M 1

2,)(1

Error SquaredMean MSE )1(

∑

∑

=

=

−

=

=

M

ttt

t

AFM

MSE

M

1

2/1

1

1

Error AbsoluteMean MAE (3)

)(

Error SquaredMean Root RMSE (2)

w where100, x w

Error Percent Absolute Weighted(5)

100 1

ErrorPercent AbsoluteMean MAPE )4(

1

t1

t

1

=−

−

=

∑∑

∑

=

=

=

M

tt

tM

t t

tt

M

t t

tt

A

A

A

AF

xA

AF

M

Points Turning (7)

)(

(1966)) gForecastin Economic (Applied statistic UTheil (6)

2

1

1

2

2

1

1

2

2

2

−=

∑

∑

=

=

M

tt

M

ttt

A

AF

U

Theil’s coefficient should be applied to the difference between predicted and actual changes. When models predict the levels of economic variables, there are two possible ways to obtain predicted changes:

(1) F - F

495

(1) Ft - Ft-1

(2) Ft - At-1

With (1),The difference between a forecasted and actual change is

(Ft - Ft-1) - (At - At-1) = (Ft - At) - (Ft-1 - At-1)

496

With (2),= (Ft - At-1) - (At - At-1) = Ft - At

Leuthold (1975)

Turning-PointErrors

Overestimation of Increase

Underestimation of Increase

1tt1tt

t

AF OR FFCHANGE FORECASTED F

−− −==

t

CHANGE ACTUAL A

−

=

497

changes PredictedAF (2)FF (1)

forecastsin changes predicted define toHow :KEY

1tt

1tt

−−

−

−

Turning-PointErrors

Overestimation of Increase

Underestimation of Increase

1tt AA −−

Turning point errors -- incorrect forecasts of the direction of change.

Generally found that most of points fall in the con e of underestimation of change.

Systematic underestimation of change is a general

498

Systematic underestimation of change is a general finding for most forecasts.

Conservative bias in forecasting -- forecasts often below true magnitudes; bias reinforced by hedging i n forecasts (adjusting add factors toward zero) so as to avoid taking extreme positions.

058352.0040141.0126877.0072308.0043621.0075339.0052519.0001612.0128294.0130402.0011684.0

04858.005098.005190.004895.004193.003768.003472.003528.003509.003029.002937.0

20120.178741.044453.247727.104038.199944.151244.104570.065571.330487.439785.0

31592.021733.068691.039148.023617.040789.028434.000873.069458.070600.006326.0

31592.027133.068691.0

039148.23617.040789.028434.000873.069458.070600.006326.0

98408.2581733.2741309.2710852.2646383.2280789.2008434.1909127.1930542.1810600.1783674.15

3.266.271.285.267.224.208.181.190.194.169.15

PRODUCTWEIGHTABSPCERABSERRORERRORIMPFIMP

−

−−

−

499

WAPEMAPE500775.1613042.1

111242.0083217.0063864.0124417.0191126.0072579.0113179.0

10454.009291.009051.007998.006834.006151.005744.0

06407.189570.070563.055565.179664.218000.197026.1

60227.045054.034576.067360.003476.139294.061275.0

60227.045054.034576.067360.003476.139294.061275.0

99773.5584946.4934576.4997360.4303476.3869294.3348725.30

6.563.500.493.430.373.331.31

−−−−

500

MSE

SS)r1(2U

MSE

)SS(U

MSE

)AF(U AF

C

2AF

S

2

m

−=−=−=

(A)

501

1UUU

proportion proportion proportion

covariance nce varia bias)(or mean

MSEMSEMSE

CSM =++

↑↑↑

With this decomposition,

UM Measure of bias - unequal central tendencies of the actual and forecasted changes

US Measure of unequal variation - squared difference between standard deviations

2)AF( −

2)SS( −

502

UC Measure of incomplete covariation - correlation coefficient r between actual and forecasted values

2AF )SS( −

1UUUMSE/SS)r1(2U

;MSE/)SS( U;MSE/)AF(U

SS)r1(2

CSM

AFC

2AFS

2M

AF

=++−=

−=−=−

UC - nonsystematic random error, cannot be avoided

UM, US - represent systematic errors that should be avoided

US 0 as SF = SAUS indicates ability of the model to replicate the degree of variability if U S large - actual series fluctuated considerably but simulated series shows little fluctuation

UM 0 as if U M large - average predicted changedeviates substantially from average realized change

AF =

503

considerably but simulated series shows little fluctuation or vice versa

UC 0 as r =1 can never hope that forecasters will be a ble to predict so that all points are located on the straight line of perfect forecasts

UC remaining error after deviations from average values and average variabilities have been accounted for

MSE/S)r1(U

MSE/)rSS(U

MSE/)AF(U

S)r1()rSS()AF(MSE

2A

2D

2AFR

2M

2A

22AF

2

−=−=

−=

−+−+−=

504

component eDisturbancU

component RegressionU

1UU U unity toclose be should U

zero frommuch differ not should U,U

D

R

DRM

D

RM

→

→

=++

Run the Auxiliary Regression

Consider this decomposition in relation to the regression

If residuals have zero mean, then

iii FA ε+β+α=

ResidualFA ii +=

505

If regression coefficient truly 1, then

MSE will then consist of only one term, the disturbance proportion.

.0UFA M =→=

.0USrS RFA =→=

Composite Forecasting -- often alternative forecasts of the same data are available, each of which conta ins information independent of others.

--- Bates and Granger (1969)--- Granger and Newbold (1977)--- Just and Rausser (1981)--- Bessler and Brandt (1981)

506

Bates and Granger suggest that if the objective is to make as good a forecast as possible, the analyst should attempt to combine the forecasts.

Composite forecasting can provide forecasts which a re preferred to the individual forecasts used to gener ate the composite.

2211 FMwFMwCF +=

Building composite forecasts requires that the anal yst select weights to assign the individual forecasts. How to weight?(1) Minimum variance weighting

minimize the variance of the forecast errors over the forecast period.

2122w

−= σρσσ

507

The forecast error variance associated with method i; is the correlation coefficient between the errors o f forecasts i and j.

12

2122

21

2121

1

2

ww

w

−=

−+−=

σρσσσσρσσ

→σ2i ρ

2

2FM1FMCF

averaging Simple (2)

+=

508

r

FM

generally, More

r

1ii∑

=

Run the Regression

with the restriction that w 1 + w2 = 1

w1, w2 turn out to be the same as the w 1, w2associated with minimum variance weighting

tt22t11t FMwFMwA ε++=

509

This regression technique allows “optimal” weights when there are more than two forecast methods

1w...ww

FMw...

FMwFMwFMwA

k21

tktk

t33t22t11t

=+++ε+++++=

5

1t1t1t YYe

ErrorForecast Ahead Step-1

+++ −=

511

hththt YYe

ErrorForecast Ahead Step-h

+++ −=

512

513

514

MODEL A MODEL B

515

14.937.911.468.37(%)MAPE

152341155874188307140833($)MAE

188717189827211982195696RMSE($)VarianceMinimumComposite

AverageComposite

B MODELA MODEL

0.4086872B MODELfor Weight 0.5913128A MODELfor Weight

==

516

tt A MODEL

(.2482)

*90105.0

(422200)

163750ACTUAL

0.98246890.017354110.94930070.0505220.000177

UDURUCUSUM

A MODELFor

+=

517

There is no efficacious substitute for economic analysis in business forecasting. Some maverick may hit a home run on occasion; but over the long season, batting averages tend to settle down to a sorry level when the more esoteric methods of soothsaying are relied upon.

Better to be wrong in good company than run the

518

risk of being wrong all alone.

Often a forecaster is forced to give a single point estimate because his boss or others cannot handle a more complicated concept. Then he must figure out for himself which point estimate will do them the m ost good, or the least harm.

Economic forecasters are like six Eskimos in one bed; the only thing you can be sure of is that they are all going to turn over together.

--- Roy Blough

Self-styled “prophets” who mislead us should be reminded that among the ancient Scythians, when prophets predicted thing that failed to come true, they were laid, shackled hand and foot, on a litter

519

they were laid, shackled hand and foot, on a litter cart filled with heather and drawn by oxen, on whic h they were burned to death.

--- Washington Post

Those who live by the crystal ball should learn to eat ground glass.

--- Ted Moskovitz