forecasting jy le boudec 1. contents 1.what is forecasting ? 2.linear regression 3.avoiding...
Post on 19-Dec-2015
219 views
TRANSCRIPT
Forecasting
JY Le Boudec
1
Contents
1. What is forecasting ?2. Linear Regression
3. Avoiding Overfitting4. Differencing
5. ARMA models6. Sparse ARMA models
7. Case Studies
2
1. What is forecasting ?
Assume you have been able to define the nature of the loadIt remains to have an idea about its intensity
It is impossible to forecast without error
The good engineer shouldForecast what can be forecast
Give uncertainty intervals
The rest is outside our control
3
4
2. Linear Regression
Simple, for simple casesBased on extrapolating the explanatory variables
5
6
7
8
Estimation and Forecasting
In practice we estimate from y, …, yt
When computing the forecast, we pretend is known, and thus make an estimation errorIt is hoped that the estimation error is much less than the confidence interval for forecast
In the case of linear regression, the theorem gives the global error exactlyIn general, we won’t have this luxury
9
10
We saw this alreadyA case where estimation error versus prediction uncertainty can be quantified
Prediction interval if model is knownPrediction interval accounting for estimation (t = 100 observed points)
11
3. The Overfitting Problem
The best model is not necessarily the one that fits best
12
Prediction for the better model
13
This is the overfitting problem
How to avoid overfittingMethod 1: use of test dataMethod 2: information criterion
14
15
16
Best Model for Internet Data, polynomial of degree up to 2
17
18
d = 1
Best Model for Internet Data, polynomial of degree up to 10
19
4. Differencing the Data
20
21
22
Point Predictions from Differenced Data
23
Background On Filters (Appendix B)
24
We need to understand how to use discrete filters.
Example: write the Matlab command for
25
26
A simple filter
Q: compute X back from Y
27
28
29
30
Impulse Response
31
32
33
A filter with stable inverse
How is this prediction done ?
This is all very intuitive
34
35
Prediction assuming differenced data is iid
36
Prediction Intervals
A prediction without prediction intervals is only a small part of the storyThe financial crisis might have been avoided if investors had been aware of prediction intervals
37
38
39
Compare the Two
40
Linear Regression with 3 parameters + variance Assuming differenced data is iid
41
5. Using ARMA Models
When the differenced data appears stationary but not iid
42
Test of iid-ness
43
44
ARMA Process
45
46
ARMA Processes are Gaussian (non iid)
47
48
49
ARIMA Process
50
Fitting an ARMA Process
Called the Box-Jenkins methodDifference the data until stationaryExamine ACF to get a feeling of order (p,q)Fit an ARMA model using maximum likelihood
51
Fitting an ARIMA ModelApply Scientific Method
1. make stationary and normal (how ?)2. bound orders p,q3. fit an ARMA model to Yt -
i.e. Yt - » ARMA 4. compute residuals and verify white noise and normal
Fitting an ARMA modelPb is :
given orders p,q given (x1, …xn) (transformed data) compute the parameters of an ARMA (p,q) model that maximizes
the likelihood
52
A: the mean , the polynomial coefficients k and k , the noise variance 2
Q:What are the parameters ?
This is a non-linear optimization problem
Maximizing the likelihood is a non-linear optimization problems
Usually solved by iterative, heuristic algorithms, may converge to a local maximummay not converge
Some simple, non MLE, heuristics exist for AR or MA modelsEx: fit the AR model that has the same theoretical ACF as the sample ACF
Common practice is to bootstrap the optimization procedure by starting with a “best guess”
AR or MA fit, using heuristic above
53
Fitting ARMA Model is Same as Minimizing One-Step ahead prediction error
54
Best Model Order
55
Check the Residuals
56
Example
57
58
59
Forecasting with ARMA
Assume Yt is fitted to an ARMA process
The prediction problem is: given Y1=y1,…,Yt=yt find the conditional distribution of Yt+h
We know it is normal, with a mean that depends on (y1,…,yt) and a variance that depends only on the fitted parameters of the ARMA process
There are many ways to compute this; it is readily done by Matlab
60
61
Forecasting Formulae for ARIMA
Y = original dataX = differenced data, fitted to an ARMA model1. Obtain point prediction for X using what we just saw2. Apply Proposition 6.4.1 to obtain point prediction for Y3. Apply formula for prediction interval
There are several other methods, but they may have numerical problems. See comments in the lecture notes after prop 6.5.2
62
63
Improve Confidence Interval If Residuals are not Gaussian (but appear to be iid)
Assume residuals are not gaussian but are iidHow can we get confidence intervals ?
Bootstrap by sampling from residuals
64
65
With bootstrap from residuals
With gaussian assumption
66
6. Sparse ARMA Models
Problem: avoid many parameters when the degree of the A and C polynomials is high, as in the previous example
Based on heuristicsMultiplicative ARIMA, constrained ARIMAHolt Winters
67
68
Holt Winters Model 1: EWMA
69
70
EWMA is OK when there is no trend and no periodicity
71
72
73
74
75
76
77
Sparse models give less accurate predictions but have much fewer parameters and are simple to fit.
78
Constrained ARIMA
(corrected or not)
79
7. Case Studies
80
81
82
83
84
85
86
87
88
h = 1h = 1
89
90
h = 2h = 2
91
92
logh = 1log
h = 1
Conclusion
Forecasting is useful when savings matter; for exampleSave money on server space rentalSave energy
Capturing determinism is perhaps most important and easiest
Prediction intervals are useful to avoid gross mistakes
Re-scaling the data may help
… à vous de jouer.
93
94
Joyeuses Pâques !