Oil Forecasting Using Artificial Intelligence

Show more

1. Introduction

Numerous studies have documented that forecasting oil is not an easy task. Researchers have used a massive amount of linear and nonlinear models to capture the correct direction of oil contracts but this seems to hide an error which is quite high. Most probably this is coming from the market volatility or from non-linearity that models cannot catch. For overpassing the previous obstacles we decide to take in consideration two hi-tech tools named Deep Beliefs networks and adaptive neural fuzzy inference system in terms of forecasting crude oil closing prices. From literature aspect these models are giving better forecast accuracy and at the same time are easier and extremely fast to process. Karathanasopoulos, [1] was the first that used this method in terms of forecasting the crack spread. He was convinced that this new methodology is promising and outperforms all the traditional linear and non-linear models.

Theoretical Basis: Deep Belief Networks and adaptive Neurofuzzy Inference system.

Deep belief networks, (DBNs) have been presented for the first time by Hinton et al. [2] . The DBN model is based on multiple layers. The bottom layer is observable, while the remaining hidden layers are formed by stacking many restricted Boltzmann machines (RBMs), introduced by Ackley et al. [3] and Hinton and Sejnowski [4] , on the top of each other. When the system is fed with a large quantity of data, it is first processed through the visible layer units, and, subsequently, the hidden layers help to detect data features according to the connection weights. Importantly, there are no connections between the units of a single layer—all the connections of the RBM’s units are restricted to different layers. The procedures for data are the same for all the RBMs, constructing the DBN. Both DBNs and RBMs have been frequently employed as a forecasting tool in time-series analysis (Chao et al. [5] , Fagiani et al. [6] , He [7] , Hinton and Salakhutdinov [8] , Kang and Choi [9] , Kuremoto et al. [10] , Shen et al. [11] , and Karathanasopoulos and Osman [12] ). To sum up, as far as we are concerned, this is the first study to utilize the DBNs for oil market predictions From the other side ANFIS was first introduced from Jang [13] . ANFIS is putting together the advantages of neural networks (NN) and Fuzzy Inference Systems (FIS) in terms of improving the accuracy of forecasts. ANFIS has the capability of learning fast, the capability of seizing the nonlinear structure and the capability of adaptation, making itself easy to use. The ANFIS has been successfully implemented to a broad range of problems in a variety of fields including economics [14] , energy [15] , health [16] and the environment [17] for different purposes including diagnosis [18] , evaluation [19] , prediction [20] , and forecasting [21] .

2. Data Analysis for Dubai Crude Oil

We start our forecast research analysis on crude oil by downloading 18 years daily closing prices data from Thomson Reuter’s database. Moreover by using the log return equation we convert our data to daily returns. Instead of being confident with our data we are checking for stationarity. The stationarity test used this time is the Augmented Dickey-Fuller (ADF) test. The output proves that daily oil prices are stationary and indicates that further analysis can be done. In the next step we divide the data through optimizing the dataset in the below segmentation (Table 1).

Table 1. Total dataset divided in training and validation periods.

3. Models in Study

By using the below models we can compare their predictive ability and their statistical accuracy.

Naive strategy. The simplest strategy assumes that the return on day t + 1 is equal to the return on day t: (yesterday closing price is the today forecast)

${\stackrel{^}{Y}}_{t+1}={Y}_{t}$ (1)

where ${Y}_{t}$ is the actual return on day t, and ${\stackrel{^}{Y}}_{t+1}$ denotes the predicted return for day t + 1.

Moving Average Convergence Divergence (MACD). The MADC model reports that the return on day t + 1 equals the average return through a certain trailing window:

${M}_{t}=\frac{\left({Y}_{t}+{Y}_{t-1}+{Y}_{t-2}+\cdots +{Y}_{t-n+1}\right)}{n}$ (2)

where ${M}_{t}$ represents is the moving average on day t, and n denotes the length of the moving average window (the number of days).

Autoregressive moving average models (ARMA). The ARMA model predicts returns based on their previous values Koreisha and Fang, (the autoregressive part) and on their residuals (the moving average parts). The ARMA model can take the below form:

$\begin{array}{c}{Y}_{t}={\varphi}_{0}+{\varphi}_{1}{Y}_{t-1}+{\varphi}_{2}{Y}_{t-2}+\cdots +{\varphi}_{\rho}{Y}_{t-\rho}+{\epsilon}_{t}\\ \text{\hspace{0.17em}}-{w}_{1}{\epsilon}_{t-1}-{w}_{2}{\epsilon}_{t-2}-\cdots -{w}_{q}{\epsilon}_{t-q}\end{array}$ (3)

where ${Y}_{t}$ is the return on day t and ${Y}_{t-1}$ , ${Y}_{t-2}$ , and ${Y}_{t-\rho}$ are the lagged returns, ${\varphi}_{0}$ , ${\varphi}_{1}$ , ${\varphi}_{2}$ , and ${\varphi}_{\rho}$ denote the regression coefficients, ${\epsilon}_{t}$ indicates the residual term and ${\epsilon}_{t-1}$ , ${\epsilon}_{t-2}$ , and ${\epsilon}_{t-q}$ are its lagged values. Finally, ${w}_{1}$ , ${w}_{2}$ , and ${w}_{q}$ denotes the weights.

4. Deep Beliefs Networks

Karathanasopoulos and Osman [12] have used the Deep Belief Network with the below description to forecast successfully the Dubai financial market. Deep belief network is a very new probabilistic feed-forward forecasting tool with input layer, hidden and output layer (see Figure 1).

Karathanasopoulos and Osman [12] mentioned that the basic idea behind this new model is the use of a layer-by-layer unsupervised learning method in order to pre-train the initial values of the weights in the neural network. A layer-by-layer unsupervised training procedure implies that each layer captures the features of the previous one, and transferring them to next one. In this research paper each pair of layers is pre-trained by using the restricted Boltzmann machines. Restricted Boltzmann machines are composed from 2 different layers connected together. One layer has visible nodes/neurons and the other hidden nodes/neurons (see Figure 2).

The nodes on each layer have no connections between them only with units of

Figure 1. Deep belief network structure.

Figure 2. Restricted Boltzmann machine structure.

other layers. All these connections are symmetric and bidirectional. As Karathanasopoulos and Osman [12] mentioned, that restricted Boltzmann machines can be used in numerous tasks but in forecasting experiments are providing the learning training for the structure of the deep belief network. In few words restricted Boltzmann machines are a special type of generative energy based models that can learn a probability distribution over its set of inputs. Except from that the standard type of restricted Boltzmann machine has binary valued hidden and visible nodes.

5. Neural Fuzzy Inference System

The Adaptive neural fuzzy inference system ANFIS is a very common and widely used artificial intelligence technique in literature. This method covers advantages of Fuzzy Logic and Artificial Neural Networks in the same structure. Although Neural Networks have powerful learning ability, Fuzzy structures have strong inference systems and no learning ability. In contrary, ANFIS combines these both desirable characteristics in the same topology. ANFIS was presented by R. Yang in 1993 [22] . From the three type of ANFIS structure we chose the Sugeno ANFIS structure which is explained in the next five stages.

Stage 1: This layer can be called as fuzzification layer. In this stage the parameters are called premise parameters and can be re-arranged according to the output error in every loop. These parameters are the input parameters which show the membership grades on the fuzzy.

Stage 2: In the second stage is computed the fixed node which is the output of all the incoming signals. Every output of the second stage can affect the triggering level of rules in the next stage. The trigger level is called the firing strength of the fuzzy system.

Stage 3: In the third stage we have the layer which is called normalization layer. For this layer, all firing strengths are re-arranged again by considering their own weights.

Stage 4: In the fourth stage the Defuzzication layer reports the preliminary calculation of the output, this layer has adaptive nodes and is expressed in functions. Because are ANFIS model is Sugeno type if then calculation turns to linear approach [23] .

Stage 5: In the last stage we have the Summation neuro which is a fixed node that computes the overall output of all the incoming signals. ANFIS has Backwards and forward Learning Ability in one loop. This correction is processed though Least square estimator which arranges the parameters and minimize the squared error (Table 2).

Table 2. ANFIS parameters.

6. Trading Signals and Strategy

The trading strategy we follow is based on the directional price of the forecasts, Karathanasopoulos et al. [24] and Karathanasopoulos [25] [26] . We hypothetically assuming a long (buy) or short (sell) position in the crude oil if the given models predict positive or negative return for the next day. Further to that, when the forecast indicates consecutive positive or negative price changes, the short/long positions, are maintained, focusing on the decrease of the trading costs.

7. Statistical performance and Empirical Performance (Table 3 & Table 4)

Table 3. Out of sample statistical performance for the Dubai Crude Oil (excluding cost).

Table 4. Out of sample empirical performance for the Dubai Crude Oil (excluding cost).

8. Concluding Remarks

In this research paper, we propose a deep belief network and an adaptive neural fuzzy inference system for forecasting the Crude oil closing prices. More specifically we have used these novel methodologies for the first time in terms of forecasting one day ahead of the crude oil prices. In terms of feeding both networks we have used a pool of 5000 most correlated indices to the main oil index. In terms of benchmarking our forecasted returns we use three linear models such as naïve strategy MACD and ARMA model. Furthermore, evaluating the forecast with statistical and empirical measures we came to the conclusion that both new models outperform significantly all the other models and give promising results for further use in financial forecasting.

References

[1] Karathanasopoulos, A. (2017) Modelling and Trading Commodities with a New Deep Belief Network. Economics and Business Letters, 6, 28-34.

https://doi.org/10.17811/ebl.6.2.2017.28-34

[2] Hinton, G., Osindero, S. and Teh, Y.W. (2006) A Fast Learning Algorithm for Deep Belief Nets. Neural Computation, 18, 1527-1554.

https://doi.org/10.1162/neco.2006.18.7.1527

[3] Ackley, D.H., Hinton, G. and Sejnowski, T.J. (1985) A Learning Algorithm for Boltzmann Machines. Cognitive Science, 9, 147-169.

https://doi.org/10.1016/S0364-0213(85)80012-4

[4] Hinton, G. and Sejnowski, T.J. (1986) Learning and Relearning in Boltzmann Machines, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume1: Foundations. MIT Press, Cambridge, MA.

[5] Chao, J., Shen, F. and Zhao, J. (2011) Forecasting Exchange Rate with Deep Belief Networks. The 2011 International Joint Conference on Neural Networks, San Jose, CA, 31 July-5 August 2011, 1259-1266.

https://doi.org/10.1109/IJCNN.2011.6033368

[6] Fagiani, M., Squartini, S., Gabrielli, L., Spinsante, S. and Piazza, F. (2015) A Review of Datasets and Load Forecasting Techniques for Smart Natural Gas and Water Grids: Analysis and Experiments. Neurocomputing, 170, 448-465.

https://doi.org/10.1016/j.neucom.2015.04.098

[7] He, W. (2014) Deep Neural Network Based Load Forecast. Computer Modelling & New Technologies, 18, 258-262.

[8] Hinton, G. and Salakhutdinov, R. (2006) Reducing the Dimensionality of Data with Neural Networks. Science, 313, 504-507.

https://doi.org/10.1126/science.1127647

[9] Kang. Y. and Choi, S. (2011) Restricted Deep Belief Networks for Multi-View Learning. In: Gunopulos, D., Hofmann, T., Malerba, D. and Vazirgiannis, M., Eds., Machine Learning and Knowledge Discovery in Databases. Lecture Notes in Computer Science, Springer, Berlin, Heidelberg, 130-145.

https://doi.org/10.1007/978-3-642-23783-6_9

[10] Kuremoto, T., Kimura S, Kobayashi K, and Obayashi M. (2014) Time Series Forecasting Using a Deep Belief Network with Restricted Boltzmann Machines. Neurocomputing, 137, 47-56.

https://doi.org/10.1016/j.neucom.2013.03.047

[11] Shen, F., Chao, J. and Zhao, J. (2015) Forecasting Exchange Rate Using Deep Belief Networks and Conjugate Gradient Method. Neurocomputing, 167, 243-253.

https://doi.org/10.1016/j.neucom.2015.04.071

[12] Karathanasopoulos, A. and Osman, M. (2018) Forecasting the Dubai Financial Market with a Combination of Momentum Effect with a Deep Belief Network. Journal of Forecasting, 38, 346-353.

https://doi.org/10.1002/for.2560

[13] Jang, J.S.R. (1991) Fuzzy Modeling Using Generalized Neural Networks and Kalman Filter Algorithm. Proceedings of the AAAI, Anaheim, CA, 14-19 July 1991.

[14] Lotfi, E., Darini, M. and Karimi-T, M.R. (2016) Cost Estimation Using ANFIS. The Engineering Economist, 61, 144-154.

https://doi.org/10.1080/0013791X.2015.1104568

[15] Panapakidis, I.P. and Dagoumas, A.S. (2017) Day-Ahead Natural Gas Demand Forecasting Based on the Combination of Wavelet Transform and ANFIS/Genetic Algorithm/Neural Network Model. Energy, 118, 231-245.

https://doi.org/10.1016/j.energy.2016.12.033

[16] Appiah, R., Panford, J.K. and Riverson, K. (2015) Implementation of Adaptive Neuro Fuzzy Inference System for Malaria Diagnosis (Case Study: Kwesimintsim Polyclinic). International Journal of Computer Applications, 115, 33-37.

https://doi.org/10.5120/20166-2284

[17] Prasad, K., Gorai, A.K. and Goyal, P. (2016) Development of ANFIS Models for Air Quality Forecasting and Input Optimization for Reducing the Computational Cost and Time. Atmospheric Environment, 128, 246-262.

https://doi.org/10.1016/j.atmosenv.2016.01.007

[18] Akdemir, B., Oran, B., Günes, S. and Karaaslan, S. (2010) Prediction of Cardiac End-Systolic and End-Diastolic Diameters in M-Mode Values Using Adaptive Neural Fuzzy Inference System. Expert Systems with Applications, 37, 5720-5727.

https://doi.org/10.1016/j.eswa.2010.02.038

[19] Okic, D.A. and Jovic, S. (2017) Evaluation of Agriculture and Industry Effect on Economic Health by ANFIS Approach. Physica A: Statistical Mechanics and Its Applications, 479, 396-399.

https://doi.org/10.1016/j.physa.2017.03.022

[20] Bahram-Parvar, M., Salehi, F. and Razavi, S.M.A. (2017) Adaptive Neuro-Fuzzy Inference System (ANFIS) Simulation for Predicting Overall Acceptability of Ice Cream. Engineering in Agriculture, Environment and Food, 10, 79-86.

https://doi.org/10.1016/j.eaef.2016.11.001

[21] Barak, S. and Sadegh, S.S. (2016) Forecasting Energy Consumption Using Ensemble ARIMA-ANFIS Hybrid Algorithm. International Journal of Electrical Power and Energy Systems, 82, 92-104.

https://doi.org/10.1016/j.ijepes.2016.03.012

[22] Koreisha, S.G. and Fang, Y. (2004) Updating ARMA Predictions for Temporal Aggregates. Journal of Forecasting, 23, 275-296.

https://doi.org/10.1002/for.913

[23] Jang, J.-S.R. (1993) ANFIS: Adaptive-Network-Based Fuzzy Inference System. IEEE Transactions on System, Man, and Cybernetics, 23, 665-685.

https://doi.org/10.1109/21.256541

[24] Roger, J.S. and Sun, C.T. (1995) Neuro-Fuzzy Modeling and Control. Proceedings of the IEEE, 83, 378-406.

https://doi.org/10.1109/5.364486

[25] Karathanasopoulos, A., Mitra, S., Skindilias, K. and Lo, C.C. (2016) Modelling and Trading the English and German Stock Markets with Novelty Optimization Techniques. Journal of Forecasting, 36, 974-988.

https://doi.org/10.1002/for.2445

[26] Karathanasopoulos, A. (2016) Modelling and Trading the English Stock Market with Novelty Optimization Techniques. Economics and Business Letters, 5, 50-57.

https://doi.org/10.17811/ebl.5.2.2016.50-57