crude oil price forecastingthrough narx modelling … · price direction on the short term, up to...
TRANSCRIPT
1
CRUDE OIL PRICE FORECASTINGTHROUGH NARX MODELLING
AS A DYNAMIC ARTIFICIAL NEURAL NETWORKS
Isabelle Cristiani-d’Ornano,
Université de Nice Sophia-Antipolis
Ahmed Ksaier
Université de Nice Sophia-Antipolis
Jean-Etienne Carlotti
Université Paris-Sud – Université Paris-Saclay
Abstract
This paper presents a model based on multilayer feedforward neural network used to forecast crude oil
price direction on the short term, up to 20 days. After testing pre-processing data methods, we use a
dynamic Nonlinear Auto Regressive model with exogenous input (NARX) as a form of ANN in order
to take account for the time factor.
As a feedback dynamic neural model, we integrate the output from time series model to feed NARX
model. Our NARX model uses data running from 1 January 2002 to 31 December 2015. Results
obtained from this training offer a good understanding of the crude oil price dynamic and we observe
that predictive trend carried out in the fact.
Keywords: Crude oil price forecasting, Prediction models, Artificial Neural Networks, NARX
JEL Classification : C22 C32 C52 F31 Q43.
2
1. Introduction
As one of the most strategic resource of the world, crude oil is a key variable for global economy.
Crude oil prices variations are observed by both market actors and financial practitioners. In order to
reduce the negative effect of the price fluctuations, forecasting direction of crude oil price is
essentially not only for oil exporting and oil importing countries but also for minimizing shocks on
other energy markets. Forecasting its price is necessary but it is a challenging task because many
factors influence their trends and level price volatility, unforeseen events like weather conditions,
financial speculations and shocks, foreign exchange rates variations, OPEC oil policy, dollar index,
gold, heating oil spot price… wars, embargoes and political events. (Panas and Ninni, 2000).
Usually, time series analysis is a method of forecasting that focuses on the historical behavior of
dependent variable. Oil prices are assumed to be normally distributed in many studies but their
departure from normal distribution was disregarded due to misinterpretation of Central Limit
Theorem. In fact, crude oil prices are non-Gaussian. Forecasting crude oil prices through fundamental
method is a complex task due to uncertainty, noisiness and non-stationary inbuilt in indicators that
drive them. Therefore, time series models provide an alternative to analyze and predict future
movements based on past behavior of oil prices.
The most common models in time series forecasting, particularly in the parametric estimation method,
are the autoregressive integrated moving average (ARIMA) and generalized autoregressive conditional
heteroskedasticity (GARCH) models. ARIMA is a basic linear forecasting model, which uses a lagged
series. Because of its simplicity and good performance, ARIMA used to be applied to many time
series analyses. GARCH is based on the idea of non-consistent variance in a general time series, and
can be applied to the volatility analysis of a time series.
Recently, machine-learning methodologies, such as artificial neural networks (ANN), are used in
many forecasting studies, as the versatility of these models allows them to be applied to any time
series data. Because an ANN is not based on an asymptotic theory in econometrics, it has a wide and
increasing range of applications.
Indeed, Artificial Neural Networks have seen an explosion of interest over the last few years, and have
being successfully applied across an extraordinary range of problem domains, in areas as diverse as
finance, medicine, engineering, geology and physics.
In this contribution, we present a dynamic Artificial Neural Networks called a Nonlinear
Autoregressive model with external output (NARX), this one using a time factor, allowing a dynamic
model and should improve the ability and generate a greater prediction.
This paper proceeds as follows: section 2 presents a brief literature review, section 3 describes
methodology used, section 4 deals with data and analysis and section 5 concludes this paper.
3
2. Literature Review
Since the oil crisis in 1973, many studies focused on forecasting crude oil price. We present here a
brief review of the key papers, surveyed on the various techniques used to forecast crude oil price:
both traditional and statistical econometric models. Descriptions of static and dynamic artificial neural
networks displayed in the section 4.
Amano (1987) was one of the first to use an econometric model for forecasting oil price. Through
cointegration analysis, Gulen (1998) succeed to predict the WTI crude oil price. Morana (2001) used
a semi-parametric approach based on GARCH to forecast Brent crude oil price on short term.
In their early work, Ye et al. (2002) used a linear regression for forecasting WTI crude oil spot price
on short term by using OECD oil inventory levels and stocks. Few years later, Ye et al. (2006)
included in their modelling nonlinear variables such as low and high inventory variables to the linear
forecasting model.
Others methods were carried out. By utilizing the error correction models, Lanza et al. (2005) worked
on crude oil prices. Starting from OPEC behavior, Dees et al. (2007) modelled a linear model of the
oil market to forecast both oil demand, oil supply and prices.
At the same time, Moshiri et al.(2006) worked on the chaos and nonlinearity in crude oil prices. They
compared ARMA (Autoregressive Moving Average) and GARCH (Generalized Autoregressive
Conditional Heteroscedasticity) models for the daily forecasting of crude oil prices to FNN
(Feedforward Neural Network) and concluded that the latest is the best candidate for modelling and
forecasting.
It appears that both traditional statistical and econometric techniques are able to capture only linear
process in data time series. Because the oil price is characterized by both high nonlinearity and high
volatility, these models are inappropriate to forecast the oil price.
Studies on the relation between spot prices and futures prices have been very experienced by many
researchers. They displayed the importance of efficiency, prediction ability, lead and lags.
Among them, Narayan (2007) and Agnolucci (2009) used GARCH model to predict spot and futures
crude oil prices. Murat and Tokat (2009) worked on this relation. They showed the capacity of futures
prices to predict spot price variations through random walk model.
More recently, Mohammadi et al. (2010) compared issues from different GARCH models in order to
forecast crude oil price. In the same posture, Kang et al. (2009) tested CGARCH, FIGARCH and
IGARCH models to forecast volatility of crude oil markets in order to measure the best one.
Ksaier et al. (2010) used the Hurst exponent in order to determine the existence of a long memory
phenomenon in the daily oil return series (WTI), they concluded that unlike the GARCH and
IGARCH models, the selected FARIMA-FIGARCH model is able to capture persistence in the
volatility of crude oil price and FIGARCH model generates a best forecasting accuracy.
4
In this contribution, we use a dynamic ANN model for crude oil price forecasting on short term. It is a
nonparametric, nonlinear model. Because ANN lets the data speak for itself there is not a a priori
assumption and feedforward network with nonlinear function allows to approximate any function.
Shambora et al. (2007) experienced an ANN model to forecast the crude oil price. The target of the
model is the predicted prices. They concluded that ANN outperforms other techniques. For the same
purpose of forecasting, Yu et al. (2007) proposed a multiscale neural network to predict oil price. This
model performed better than a single-scale one.
Lackes et al. (2009) experienced a layer backpropagation FNN to predict crude oil price trend on short
term but also on mid-term and long term (i.e. 3 months). They concluded that prediction modelling
with 5 neurons was less rich than with 2 neurons on the long term.
ANN model was also tested by Haidar et al. (2009) to forecast crude oil prices on the short term (3
days). They displayed that the futures prices offer more information to the spot prices and increases
forecasting ability.
It appears that these empirical studies that display ANN performance have a better accuracy prediction
than the other models.
3. Methodology
Artificial Neural Network (ANN) is an information processing system developed as generalizations of
mathematical models of human neural biology (Figure 1). ANN is composed of nodes or units
connected by directed links. Each link has a numeric weight (W is the weight matrix). Note that in
Figure 1 we have included a bias b with the purpose of setting the actual threshold of the activation
function.
Figure 1. Mathematical model of the artificial neuron
mdmmm
d
d
d
j
jijii
d
j
jiji
wwww
wwww
wwww
w
xwfvfy
xwv
321
2232221
1131211
1
1
)()(
5
Where f is the activation function, xj is the input neuron j, yi is the output of the hidden neuron i, and
W is the weight matrix. The Neural Networks learns by adjusting the weight matrix. Therefore, the
general process responsible for training the network is mainly composed of three steps:
1. Feed forward the input signals
2. Back propagate the error
3. Adjust the weights
Training the Neural Networks is divided into four steps:
1- Data selection: Data selection step restricts subsets of data from larger databases and different
kinds of data sources. This phase involves sampling techniques, and database queries.
2- Data pre-processing: data preprocessing represents data coding, enrichment and clearing
which involves accounting for noise and dealing with missing information.
3- Data transformation: has the purpose to convert data into a form suitable to feed the NN.
4- Neural Networks selection and training.
3.1 Dynamic Neural Networks
Neural networks can be classified into dynamic and static categories, and dynamic neural networks
can also be classified into feedback and no feedback categories.
In no feedback dynamic neural networks, the output of the network depends not only on the current
input to the network, but also on the previous inputs to the network. In feedback dynamic neural
networks, the output of the network depends not only on the current input to the network, but also on
the previous inputs, outputs, or states of the network. One principal application of dynamic neural
networks is in time series prediction.
Time series prediction studies the trends of process time series of the predicted target.
Neural Networks time series tools in MATLAB provide three categories to solve three different kinds
of nonlinear time series problems :
- Nonlinear autoregressive with external input (NARX）
- Nonlinear autoregressive (NAR)
- Nonlinear input-output
NARX used in our paper is a feedback dynamic neural network, and it also can be seen as the Back
Propagation Neural Networks that depends on the previous inputs and outputs to the network. The
structure of NARX network used in the paper as shown in Figure 1.
3.2 Nonlinear Autoregressive Network with Exogenous Inputs (NARX):
The NARX models are commonly used in the system of identification area (Xie et al., 2009). All the
specific dynamic networks discussed so far have either been focused networks, with the dynamics
only at the input layer, or feed-forward networks. The nonlinear autoregressive network with
exogenous inputs (NARX) is a recurrent dynamic network, with feedback connections enclosing
several layers of the network. The NARX model is based on the linear ARX model, which is
commonly used in time series modelling.
6
Figure 2 : NARX Model
Figure 2 illustrates the standard NARX network. The standard NARX network used here is a two-
layer feedforward network, with a sigmoid transfer function in the hidden layer and a linear transfer
function in the output layer. This network also uses tapped delay lines (d) to store previous values of
the input, x(t) and output, y(t) sequences. First, load the training data and use tapped delay lines with
two delays for both the input and the output, so training begins with the third data point. There are two
inputs to the series-parallel network, the x (t) sequence and the y(t) sequence. Notice that the y(t)
sequence is considered as a feedback signal, which is an input that is also an output (target). The
model can be summary by the Eq. 1:
),,,,,( 11 dttdttt xxyyfy
where, yt is the output of the NARX network and also feedback to the input of the network and tapped
delay lines (d) that store the previous values of xt and yt sequences.
4. Data Selection and Case study
4.1 Data
Most governments and private sector players on the oil market consider the WTI as the benchmark for
the international crude oil price. The crude oil prices on other markets are influenced by fluctuations in
the WTI oil market (He K, Yu L, Lai KK 2012).
In this study, our data sample contains 5 daily (5-day / week) time series over the period from January
1st, 2002 to December 31th, 2015, namely crude oil price, Euro/Dollar exchange rate, stock price, US
10 Year Treasury and VIX (Volatility Index as a “fear index”). These data sets retrieved from
DATASTREAM sources.
- US interest rate: Decreasing interest rates level will make lendings more accessible and will
stimulate crude oil demand, which, in turn, will contribute to the oil prices increase.
- SP500 index : This index, characterizing state of financial markets, is included here to
represent general financial situation and its possible effects on oil prices. According to a
number of previous studies there is a strong relationship between crude oil prices and stock
markets.
- Euro-Dollar Exchange rate: is referred as a factor of daily changes in crude oil prices, there is
a negative correlation between crude oil prices and Euro-Dollar exchange rate, stronger U.S
Dollar leads to lower crude oil prices and weaker U.S. Dollar leads to higher crude oil prices
(see graph below).
7
050
10
015
0W
TI
01 Jan 02 01 Jan 04 01 Jan 06 01 Jan 08 01 Jan 10 01 Jan 12 01 Jan 14 01 Jan 16DATE
Crude Oil Prices
Daily evolution
.6.8
11.2
EU
RO
/DO
LL
AR
01 Jan 02 01 Jan 04 01 Jan 06 01 Jan 08 01 Jan 10 01 Jan 12 01 Jan 14 01 Jan 16DATE
Exchange Rate
Daily evolution
8
500
1000
1500
2000
2500
SP
500
01 Jan 02 01 Jan 04 01 Jan 06 01 Jan 08 01 Jan 10 01 Jan 12 01 Jan 14 01 Jan 16DATE
Stock Prices
Daily evolution
12
34
56
US
10
Ye
ar T
reas
ury
01 Jan 02 01 Jan 04 01 Jan 06 01 Jan 08 01 Jan 10 01 Jan 12 01 Jan 14 01 Jan 16DATE
Interest Rate
Daily evolution
9
020
4060
80
VIX
Inde
x
01 Jan 02 01 Jan 04 01 Jan 06 01 Jan 08 01 Jan 10 01 Jan 12 01 Jan 14 01 Jan 16Date
VIX
Daily evolution
4.2 Analysis Model
In this contribution, nonlinear autoregressive with exogenous input (NARX) model (Figure 3),
constructed by using MATLAB neural network toolbox, is used in order to obtain future prediction
values of a time series, y (t), from past values of this time series and past values from other time
series, xi (t). In these experiments, we conducted NARX with different numbers of hidden layers,
numbers of tapped delay lines (d) and one output neuron with two-layer feed-forward networks
(hyperbolic tangent transfer function in the hidden layer and linear transfer function in the output
layer) were used.
Training automatically stops when generalization improving, as indicated by an increase in the Mean
Square Error (MSE) of the validation samples.
Figure 3: NARX Model used
10
According to the Figure 4, all values of R are above 0.998 during training of the neural network. This
shows that the output produced by the neural network model is closely similar to the target and that the
model is adequate.
Figure 4: Regression Plots
The model specification is trained by experimenting number of nodes and number of tapped delay
lines. The number of neurons in the hidden layer is determined by trial and error method. First, the
trials initialize at error with 2 nodes. Then, the process is repeated until 15 nodes were used. On the
other hand, the number of tapped delay lines was initialized at 2 steps up until 4 steps. A comparison
of the MSE value for all networks was carried out. The lowest MSE value will be selected as optimum
network. Based on training method, the lowest MSE value is obtained with 12 nodes in hidden layers
and 2 tapped delay lines. This result is also supported by a high R value (0.99855) at 12 nodes that
means a close relationship between Output and Target. Hence, 12 hidden nodes with 2 tapped delay
lines are selected as optimum network.
11
4.3 Prediction Model
From ours results it appears that thanks to neural networks the direction of crude oil price can be
predicted (red curve), with a high accuracy. From the training regression plot displayed supra (Figure
4) we obtain the model fit (blue curve) which corresponds to the observed trend. Indeed, we observe a
high correlation and low mean squared error term. Dynamic Neural Network can be helpful for
investors and policy makers by giving information about the future direction of the oil price market.
5. Conclusion
In this paper, after surveyed a brief review of literature on forecasting crude oil price, we presented a
NARX models for forecasting crude oil prices on the short term.
Indeed, taking account the variation of crude oil price and nonlinearity of its time series, traditional
and statistical econometric models to forecast crude oil prices are inappropriate. So we chose a
dynamic ANN as a nonlinear artificial model - after a description of this approach - to predict crude oil
price on short term. In this intention, we test the ability of NARX method to make accurate prediction.
We conclude that artificial neural networks offer a greater predictive ability for crude oil price
forecasting. Our future research continues to investigate other techniques, which could lead to
improving the short term forecast.
12
References
Abdullah, S. N. and Zeng, X. (2010) “Machine learning approach for crude oil price prediction with
Artificial Neural Networks-Quantitative (ANN-Q) model‟‟ Proceedings of the
International Joint Conference on Neural Networks , pp. 1-8.
Agnolucci, P. (2009) “Volatility in crude oil futures: A comparison of the predictive ability of
GARCH and implied volatility models”, Energy Economics, 31, pp. 316-321.
Alizadeh, A. and Mafinezhad, K (2010) “Monthly Brent Oil Price Forecasting Using Artificial Neural
Networks and A Crisis Index”, Proceedings of the International Conference On Electronics And
Information Engineering, 2, pp. 465-468.
Amano A. (1987), “A Small Forecasting Model of the World Oil Market”, Journal of Policy Modeling
9(4), pp. 615-635.
Beidas-Strom, S. and Pescatori, A. (2014), “Oil price volatility and the role of speculation”, IMF
working paper ,14, 218.
Blanchard, O. J. and Gali, J. (2007), “The macroeconomic effects of oil shocks: Why are the 2000s so
different from 1970s?”, NBER working paper number 13368, pp. 1-77.
Cristiani-d’Ornano I., Ksaier A., Carlotti J.E. (2010) “Crude Oil Price: How To Anticipate Its Future
trajectory? A Specific Phenomenon Of Volatility Clustering, The Middle East Business and Economic
Review, Vol.22, No.1
Cheong, C.W. (2009), “Modelling and forecasting crude oil markets using ARCH-type models”,
Energy Policy, 37, pp. 2346-2355.
Dees, S., Karadeloglou, P., Kaufmann, R. K. and Sanchez, M. (2007), “Modelling the world oil
market: assessment of a quarterly econometric model”, Energy Policy 35, pp. 178-191.
Ghouri, S. S. (2006), “Assessment of the relationship between oil prices and US oil stocks”, Energy
Policy 34, pp. 3327-3333.
Gulen, S. G. (1998), “Efficiency in the Crude Oil Futures Markets”, Journal of Energy Finance and
Development, 3, pp. 13-21.
Haidar, I., Kulkarni, S. and Pan, H. (2009), “Forecasting model for crude oil prices based on artificial
neural networks”, International Journal of Computer Science and Information Security, issue Vol. 2,
N°1.
Hamilton, J. D. (1996) „„This is what happened to the Oil-Macroeconomy relationship‟‟Journal of
Monetary Economics 38(2), 215-220.
Hamilton, J. D. (2008), “Understanding crude oil prices”, The Energy Journal, International
Association for Energy Economics 30(2), pp. 179-206.
13
Hamilton, J. D. (2010), “Nonlinearities and the macroeconomic effects of oil prices”, NBER working
paper number, 16186, pp. 1-22.
He K, Yu L, Lai KK. Crude oil price analysis and forecasting using wavelet decomposed ensemble
model. Energy 2012;46:564–74.
Hornik, K., Stinchcombe, M. and White, H. (1989), “Multi-layer feedforward networks are universal
approximators”, Neural Networks, 2, pp. 359-366.
Huntington, H. G. (1994), “Oil Price Forecasting in the 1980s: What Went Wrong?”, The Energy
Journal 15(2), pp. 1-22.
Kang, S.H., Kang, S.M. and Yoon, S.M. (2009), “Forecasting Volatility of Crude Oil Markets”,
Energy Economics 31, pp. 119-125.
Kilian, L. (2008), “The Economic effects of energy price shocks”, Journal of Economic Literature,
46(4), pp. 871-909.
Lanza, A. Manear, M and Giovannini, M. (2005), “Modeling and forecasting cointegrated
relationships among heavy oil and product prices”, Energy Economics, 27, pp.831-848.
Lackes, R. Borgermann, C. and Dirkmorfeld, M. (2009), “Forecasting the price development of crude
oil price with artificial neural networks”, lecture Notes in computer Science, 5518, pp. 248-255.
Kilian, L., Fattouh, B. and Mahadeva, L. (2013), “The role of speculation in oil markets: What have
we learned so far?” The Energy Journal 34(3), pp. 7-33.
Kohonen, T. (1982), “Self-Organized Formation of Topologically Correct Feature Maps”, Biological
Cybernetics 43, pp. 59-69.
Kulkarni, S. and Haidar, I. (2009), “Forecasting model for crude oil price using artificial neural
networks and commodity futures prices”, International Journal of Computer Science &Information
Security 2(1), pp. 81-88.
Mohammadi, H. and S, L. (2010), “International Evidence on Crude Oil Price Dynamics: Applications
of ARIMA–GARCH Models”, Energy Economics 32, pp. 1001-1008.
Morana, C. (2001), “A semiparametric approach to short-term oil price forecasting”, Energy
Economics, 23, pp. 325-338
Moshiri, S. and Foroutan, F. (2006), “Forecasting Nonlinear Crude Oil Future Prices”, The Energy
Journal 27, pp. 83-97.
Murat, A. and Tokat, E. (2009), “Forecasting Oil Price Movements with Crack Spread Futures”,
Energy Economics 31(1), pp. 85-90.
14
Narayan, P. and Narayan, S. (2007), “Modelling Oil Price Volatility”, Energy Policy, 35, pp. 6549-
6553.
Sadorsky, P. (2006), “Modelling and Forecasting Petroleum Futures Volatility”, EnergyEconomics,
28, pp. 467-488.
Shambora, W. and Rossiter, R. (2007), “Are there exploitable inefficiencies in the futures market for
oil?”, Energy Economics, 29, pp. 18-27.
Wei, Y., Wang, Y. and Huang, D. (2010), “Forecasting Crude Oil Market Volatility: Further Evidence
Using GARCH-class Models”, Energy Economics 32, pp. 1477-1484.
Xie, W., Yu, L., Xu, S. and Wang, S. (2006), “A new method for crude oil price forecasting based on
support vector machines”, Lecture notes in computer science 3994, pp. 444-451.
Ye, M., Zyren, J. and Shore, J. (2002), “Forecasting crude oil spot price using OECD petroleum
inventory levels”, International Advances in Economic Research, 8, pp. 324-334.
Zhang, G., Patuwo, B. E., and Hu, M. Y. (1998), “Forecasting with artificial neural networks: The
state of the art”, International Journal of Forecasting 14, pp. 35-62.