Developing a Data Mining Based Model to Extract Predictor Factors in Energy Systems: Application of Global Natural Gas Demand

Abstract

Recently, the natural gas (NG) global market attracted much attention as it is cleaner than oil and, simultaneously in most regions, is cheaper than renewable energy sources. However, price fluctuations, environmental concerns, technological development, emerging unconventional resources, energy security challenges, and shipment are some of the forces made the NG market more dynamic and complex. From a policy-making perspective, it is vital to uncover demand-side future trends. This paper proposed an intelligent forecasting model to forecast NG global demand, however investigating a multi-dimensional purified input vector. The model starts with a data mining (DM) step to purify input features, identify the best time lags, and pre-processing selected input vector. Then a hybrid artificial neural network (ANN) which is equipped with genetic optimizer is applied to set up ANN’s characteristics. Among 13 available input features, six features (e.g., Alternative and Nuclear Energy, CO₂ Emissions, GDP per Capita, Urban Population, Natural Gas Production, Oil Consumption) were selected as the most relevant feature via the DM step. Then, the hybrid learning prediction model is designed to extrapolate the consumption of future trends. The proposed model overcomes competitive models refer to different five error based evaluation statistics consist of R², MAE, MAPE, MBE, and RMSE. In addition, as the model proposed the best input feature set, results compared to the model which used the raw input set, with no DM purification process. The comparison showed that DmGNn overcame dramatically a simple GNn. Also, a looser prediction model, such as a generalized neural network with purified input features obtained a larger R² indicator (=0.9864) than the GNn (=0.9679).

1. Introduction

The world energy demand increased in the two past decades and even predictions implying the growing trends for the next decades [1,2,3,4]. Still, fossil fuels play a critical role in the energy supply chain due to economic feasibility. Refer to International Energy Agency’s (IEA) 2016 report, fossil fuels in the form of liquid fuels, natural gas, and coal contain more than 80% of the world energy consumption [5]. Easiness of utilization, higher performance, compared to traditional energy sources, ease of mobility via land or sea, and affordable extraction cost introduced oil and natural gas (NG) as strategic commodities [6,7,8]. However, emergent ecological concerns and rethinking of a more peaceful future (sustainable development goals) attracted attention toward climate change challenges (such as greenhouse gases emissions and global warming) [9]. The two non-aligned objectives, on one hand, development and increasing needs for energy supply and on the other hand global environmental concerns, attracted researchers to study energy systems and develop different plausible future perspectives.

World’s economic growth still strongly correlated with the energy and, as a result, the energy price influence potential future trends. The 1970’s energy crisis represented how energy shocks can cause chaotic behavior of economic systems especially in the case of developed countries which mostly play in energy markets as importers. Although new paradigms emerged to address the modern world energy challenges, such as evaluating and improving the energy security level, but policymakers are facing deep uncertainties in this area. Energy markets are known as complex systems [10,11], which consist of numerous players with conflict of interests and changing rules.

Despite successful efforts, the main problem is still existing, which is defined as “discovering reliable future trends and probable alternative futures in the field of energy systems and uncover the most influencing driving forces to aid energy management process”. To project a reliable future, it is crucial to investigating almost all informative input features which represent historical behavior of the targeted complex system and extrapolate future trends. Studying different input features means to peruse different aspects, while on the other hand, it means more efforts in term of cost, time, and complicated estimator. A traditional approach to address a complex problem is to simplify and decompose it to its main constituent elements. Feature selection methods are aimed to adjust the unnecessary complexity revealed to refer to the existence of multiple input features. A feature selection method determines the best subset of the initial features set as a representative of the solution space (for more information about feature selection approaches and methods see: [12,13,14,15]). Selected features interpreted as key driving forces which will help to extrapolate future trends. Moreover, pre-processing might also be performed in order to speed up computation [16]. Finally, an estimator is needed to predict future behaviors of the targeted variable. Despite there are many classical methods but artificial intelligence-based methods are extensively used during the last couple of decades [17,18] that attempt to uncover weak signals and pattern more reliable and accurate. As Kourou et al. noted these methods can discover and identify patterns and relationships between them, from complex datasets, while they are able to effectively predict future outcomes [19]. Technologies as Data Mining, Natural Language Processing, and Machine Learning can provide novel alternatives to explore and exploit potential retrieved knowledge from historical records [20], and help to decrease prediction errors effectively.

This paper is aimed to develop an intelligent learning-based prediction model which is equipped with data mining (DM) techniques to purify and the setup input vector. The DM step is used to select and organize the best input features that represent patterns of future global NG demand trends. Although many previous studies successfully addressed NG global demand prediction problem, we attempt to uncover the most effective driving forces as input features and analyzing how they will affect the objective function (NG global demand prediction). For example, the proposed model studies time relation between input variables and the target variable. So a less-dimension input set is available to policymakers to simplify and experience reliable decision-making process.

As it is impressed by a series of variables and oscillating time series, the NG forecasting problem is a very challenging [21]. These days, massive efforts have been investigated artificial intelligence (AI) models or integration of several models (hybrid models) for prediction problems to increase the accuracy and the model reliability [22,23].

2. Literature Review

Also, numerous notable studies have investigated demand prediction for the case of energy resources [24,25,26,27,28,29]. Among those, Baumeister and Kilian published a research paper to analyze how vector autoregression (VAR) models form policy-relevant forecasting scenarios in the case of an oil market. The model investigates the influence of scenario weights’ probability changes to the real-time oil price forecasting [30]. In addition, Dilaver et al. investigated NG consumption in Europe to support long-term investments and contracts [31]. They estimated an OECD-Europe NG demand trends with annual time series during the period from 1978 to 2011 by applying a structural time series model (STSM). Finally, three scenario streams developed based on business as usual, high, and low case scenarios.

Li et al. used system dynamic models to create possible outlooks to 2030 for the case of China’s NG consumption growth. Then to assess the results accuracy and propose policy recommendations on NG exploration and development of China’s NG industry, a scenario analysis step was applied [32]. Also, Suganthi and Samuel provided a comprehensive review of an energy model, which attempted to forecast demand function [33]. Authors classified prediction models and presented that most of the recent researches contained basically quantitative models that result in a single future prediction. Models used statistical error functions to estimate, accuracy compared with other comparative models. However, as mentioned above data-driven models may regret set of effective qualitative variables. In the other hand projecting alternative futures based on qualitative approaches are challenging, especially in the case of validation and, moreover, they are extremely affected by the expert group (number of experts and judgment validation).

Baumeister and Kilian [30] described how vector autoregression (VAR) models can generate policy-relevant forecast scenarios of the global oil market and how real-time oil price forecasting, influenced by possible changes in probability weights, corresponded to the scenarios. In addition, Dilaver et al. [31] investigated natural gas consumption in Europe to support long-term investments and contracts. They implemented a structural time series model (STSM) to estimate the OECD-Europe natural gas demand function with annual data from 1978 to 2011. Using that information, they developed three scenarios based on high-case, reference (business as usual), and low-case scenarios.

Suganthi and Samuel [33] provided a comprehensive review of energy models aimed at forecasting demand functions. The study classified prediction models and identified that most of the recent studies were comprised of quantitative models that provided only a single future prediction scenario. The models predominantly used statistical error functions to estimate accuracy compared with other models. To present a more universal review and to dedicate insights about prediction approaches used by previous studies,

Table 1. Analyzing previous studies, based on their approaches to address energy consumption prediction problem.

Approaches		References
Classical computational extrapolation	Time series	[24,25,31,34,35,36,37,38,39,40,41,42,43]
	Regression	[28,44,45,46,47]
Econometrics		[48,49,50,51,52]
Expert systems and learning models	Artificial neural network (ANN)	[21,40,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69]
	Genetic programming (GP)	[21,24,40,58,65,67,69,70,71,72,73,74,75,76]
	Ant colony optimization	[77]
	Particle swarm optimization (PSO)	[26,78,79]
	Support vector machine (SVM)	[25,40,59,64,80,81]
	Fuzzy inference system (FIS)	[21,44,62,73,82]
Others	Decomposition approach	[83,84]
	Input-output model	[85,86]
	Bottom-up model	[87,88,89]
	Grey method	[26,37,76,86,90,91]
	Logistic model	[92]

summarized models used to address energy consumption prediction problem.

As shown in Table 1 and noted before learning based techniques can lead to develop a more reliable, accurate estimator as they can represent a self-adjustment characteristic since they learn formerly signals via feedback loops. To dedicate a more detailed understanding of various existed forecasting models,

Table 2. The major pros and cons of main forecasting methods.

Type of Models	Pros & Cons
Classic price modeling/forecasting	Focus on historical data. Do not consider jumps and drips of the prices. Generally, these models were introduced for stock markets. Do not use unit root test for time series and econometric methods to estimate their parameters.
Time series models	Focus on historical data. Do not cope with extreme jumps and drips. Do not use feedback loops to dynamically upgrade model adjustment features.
Learning forecasting models	Focus on historical data. Generally being able to learn fluctuations and related formerly signals. Use feedback loops to upgrade model adjustment features dynamically.
Qualitative based forecasting models	Rarely depend on quantitative forecasting methods. Donate insights about long-run behaviors of a complex system. Mostly depend on experts’ evaluations, instead of historical time series. Are able to dynamically modify input features.

shows the pros and cons of main forecasting methods.

In this paper, we are aimed to propose a learning-based model, which is designed to present a more reliable and relevant input features (driving forces) to initialize a hybrid prediction ANN [93] to equip decision-making process with accurate and reliable forecasts. Next section investigates the proposed methodology and brief descriptions of various steps, then section three is dedicated to presenting the implementation phase and discussing results to show how the proposed methodology overcomes other benchmark models. Finally, section four provides summaries and conclusions.

3. The Methodology of Research

As noted previously, the following research is aimed to expand a data mining based prediction model. Three major goals are targeted: (1) determining features which effectively present trends for NG global demand (i.e., driving forces which can shape and define future trends), (2) identification of time lags to define time relations between input variables and the target variable (as inter-correlated features represents their influences with different time lags/delays), and (3) developing a learning intelligent prediction model that can extrapolate future trends for the global NG demands. Figure 1 conceptually shows designed data mining genetic-neural network (DmGNn) methodology to approach noted goals.

Following, the main phases and steps of the proposed methodology are discussed and corresponding outputs are mentioned:

PHASE 1.

Step 1. Data gathering: in this step previous studies reviewed to detect potential input features. Unlike most of the published researches, this paper pursues the maximum approach, means gathering and using maximum available features to ensure that no useful information will be neglected by the estimator. In simple words, the proposed methodology doesn’t immure the solution space due to the use of purified input vector. Output: input feature subset.

PHASE 2.

Step 2. Feature selection: this step is designed to select the most relevant subset of the extracted features. The main objective is to reduce problem dimensions while preserving all local and global optimal solutions (i.e., reliability of the results). Here, the correlation-based feature selection (CFS) [94,95] technique is used to define final input feature subset. Output: refined input feature sub-set.

Step 3. Time lag selection: it is investigated to study how different time lags for input features may affect forecasting accuracy. The theory behind it is that in a complex system elements are inter-correlated but with varying degrees, and sometimes react with different time lags [17]. This step will study time relation between input features and the target variable (natural gas demand). Information criteria [96] method is used to detect lag orders. Output: timed input features subset.

Step 4. Normalization: different scales of input features may cause in a biased final forecasting model [97]. This step is aimed at reproducing input features but in similar, uniform scales. Min-max method [98,99] is used to normalize the input features subset. Output: uniformed timed input features sub-set.

PHASE 3.

Step 5. Design of the forecasting model: in this step, an ANN [100] is equipped with a GA [101] in order to optimize the network’s characteristics and develop an accurate prediction model. Output: estimator.

PHASE 4.

Step 6. Implementation: finalized input features subset are applied to the developed estimator. In this step, the input set is divided into two main portions, one to train (about 80%) and other to test (about 20%) the performance of the prediction framework. Outputs: adjusted prediction model & resulted in extrapolated results.

PHASE 5.

Step 7. Validation: this step is dedicated to comparing the obtained results of the proposed prediction framework with other benchmark comparative models based on multiple error indicators. R², MAE, MAPE, MBE, and RMSE are used to compare prediction results and perform accuracy analysis. Output: accuracy analysis.

To model complex systems (like energy systems), selecting a robust model architecture is very challenging [17,102]. Data mining (DM) techniques are selected to handle the complexity of input variables. DM is defined as the process of extracting appealing patterns and deriving knowledge in massive datasets. Refer to Han et al.: "the principal dimensions are data, knowledge, applications, and technologies" [103]. Following sections are dedicated to present implementation process and obtained results in details.

3.1. Input Preparation

3.1.1. Data Gathering

Input data extremely affect the accuracy and quality of the obtained results. In the case of energy consumption, previous researches investigated different sets of input features to predict energy consumption’s upcoming trends. A significant limitation of a prediction model is that it cannot reflect the effects of variables, which did not exist in the input feature set (those have been neglected). To ensure robustness and the validity of the proposed prediction model, the paper proposes the maximal approach, which means to investigate all available input data and reduce dataset dimension through a DM technique. This approach has the advantage of retaining all signals and trends, while simultaneously the model faces an undeniable challenge that is the increased complexity level due to the large input set, which may negatively affect prediction efficiency. In another hand, it is a challenging process to set up strategic decisions based on a large collection of parameters/inputs. To handle the noted problem, a DM based data pre-processing step is proposed by this paper to examine and purify input features.

Table 3. Initialize input features.

Title	Unit	Reference(s)	Source
Alternative and Nuclear Energy	% of total energy use	Proposed by authors	World Bank
CO2 Emissions	metric tons per capita	[49]	World Bank
CO2 Emissions	Kt		World Bank
Energy Imports, Net	% of energy use	Proposed by authors	World Bank
Fossil Fuel Energy Consumption	% of total	[104]	World Bank
GDP Growth	annual %	[49,55,59,77,78,105,106,107,108,109,110,111,112,113]	World Bank
GDP per Capita	current US$		World Bank
Population Growth	annual %	[46,52,53,77,78,105,107,108,109,112,113,114,115,116,117]	World Bank
Urban Population	person	[105,111]	World Bank
Gold Price	10:30 A.M.in London Bullion Market, US$	Proposed by authors	World Bank
Natural Gas Production	billion cubic meters	Proposed by authors	British Petroleum
Oil Consumption	million tones	Proposed by authors	British Petroleum
Crude Oil Prices	US dollars per barrel ($2013)	[52,104,106,112,114,115,118]	British Petroleum

summarizes the most frequently used input features (by other researchers) and the features which were available/accessible online.

3.1.2. Data Preprocessing (Feature Selection, Lag Selection and Data Normalization)

In machine learning problems, it is very challenging to select a representative collection of features to build the model [94]. Studying more features (a larger feature set), helps to explore more problem dimensions and to reduce the threat of missing potential solutions, but at the same time it may conclude more computational complexity, learning algorithm confusing, and over learning.

DM, as a process, generally contains data cleaning, integration, selection and transformation to discover patterns, evaluate them, and present the extracted knowledge [103,119]. In knowledge discovery processes, such as DM, the feature subset selection is very crucial, not only for the insight achieved from determining variables, but also for the upgraded reprehensibility, scalability, and the validity of the constructed models [13]. This research uses a correlation-based feature selection (CFS) algorithm to determine the most relevant input features. CFS was initially proposed by Hall in 1999 [94]. A CFS examines and ranks feature subsets, unlike many much-used techniques which aimed to rank features individually, such as ReliefF [120]. CFS consists of two nested steps: (1) feature evaluation and (2) searching the feature subset [95].

The feature evaluation step is the heart of a CFS which aimed to heuristically measure the merit of a subset of features. This process studies the usefulness of every single feature to predict targeted variable (here natural gas consumption) while the inter-correlation level among subset is investigated. Equation (1) formulizes this heuristic, proposed by Ghiselli [121] which is a form of Pearson’s correlation, where all variables are standardized.

$$Meri{t}_S=\frac{k \overline{r_{cf}}}{\sqrt{k+\left(k-1\right)\overline{r_{ff}}}}$$

(1)

where the equation aimed to evaluate merit of a feature subset S composed of k features. $\overline{r_{cf}}$ represents the average feature-class correlation and $\overline{r_{ff}}$ refers to the average feature-feature inter-correlation.

To decide which of the features to include in the final subset and which to ignore one would try all possible subsets. Note that for k possible features there are 2^k possible subsets. Obviously, it is not a logical procedure to investigate all possible subsets one by one especially when k is relatively large. To address this challenge various heuristic search strategies are proposed which often converge to an optimal subset in reasonable time [95,122]. "Best First" [123] and "Greedy Stepwise" [124] searching methods were applied to the CFS to study input dataset using various searching paradigms. For instance, the best first starts with an empty set and develops all possible single feature subsets. The subset with the highest evaluation value, as noted above, selected to expand by adding another single feature. The stopping condition is obtained when expanding the subset results in no improvement. Both of searching methods resulted in the same feature subset which means they support each other. Finally, through 13 representative input features (presented in Table 3) six input features selected as the model’s input, contains: (1) alternative and nuclear energy, (2) CO₂ emissions, (3) GDP per capita, (4) urban population, (5) NG production, and (6) oil consumption.

Sometimes relevant features in a time series dataset show their influence with lags of time. Also, there would be time lags for a policy/decision in the complex energy market. Detecting related lags would assist a prediction model to accurately follow possible fluctuations [17]. At this step, the proposed DmGNn methodology attempts to determine time lags related to finalized feature subset correlated with the target attribute (i.e., NG global demand).

Numerous lag selection approaches exist that contain lag selection as a pre-processing, post-processing, or even as a part of the learning process [125]. Among popular statistical tests based on information criteria pre-processing lag selection methods, Akaike information criteria (AIC), Bayesian information criteria (BIC), and Schwarz Bayesian information criteria (SBIC), are well used [126,127]. Information criteria methods consider 1 lag (as the minimum number) to p which define intermediate lags. The main hypothesis is to define the lag order p to minimize the following equation:

$$IC\left(p\right)=Nln\widehat{{\sigma }^2}\left(p\right)+p\left[f\left(N\right)\right]$$

(2)

where $\widehat{{\sigma }^2}\left(p\right)$ is defined as the estimated regression variance, related to the sample size and order p of lag structure, and N is the number of observations [128]. $p\left[f\left(N\right)\right]$ is the penalty function to increase the order of model. Different choices of $f\left(N\right)$ cause in different information criteria.

A −20 to +20 time lags were implemented for each feature versus the target attribute using Matlab software. Figure 2 summarizes results of the time lag selection process for selected features, alternative and nuclear energy and CO₂ emissions. For each chart, the vertical axis shows the level of correlation between the correspondence feature and targeted variable while horizontal axis implies different time lags. The order p defines the effective time lag which possess the highest correlation level, according to the chart.

Now, optimum input features are detected. Six selected features are representatives of all 13 identified input features and also the selected subset has been reorganized based on detected time lags.

Although an optimum set of input features have been selected, still input features are asymmetric and the units are different in scales. Data normalization step is investigated to restrain the parameters range influence on the results and adapt values of different features with different domains and scales to a shared scale. The min-max normalization method is used to adjust dataset using the following equation:

$$Normalized Data=\frac{y\left(i\right)-\min \left\{y\right\}}{max\left\{y\right\}-\min \left\{y\right\}}$$

(3)

where y(i) is an ith element in the column and $min\left\{y\right\}$ minimum and $max\left\{y\right\}$ is the maximum of related column’s elements.

The next sub-section is dedicated to discussing the forecasting framework.

3.2. Designing the Forecasting Framework

3.2.1. Artificial Neural Network

Computational intelligence methods such as an artificial neural network (ANNs) [129] are modern paradigms to handle complex optimization problems [130,131,132]. ANN is organized as a simplified abstract of the biological nervous system to emulate neurons mechanism. A neuron is the computation unit of an ANN. Mathematically a neuron is a function, which aimed at dynamically reduce deviation cost. The mathematical description of a neuron presented as follows:

$$O_j\left(t\right)=f\left\{\left[{\displaystyle \sum }_{i=1}^n{w}_{ij}{x}_i\left(t-{\tau }_{ij}\right)\right]-T_j\right\}$$

(4)

where x_i and o_j respectively are the input and the output at time t, ${\tau }_{ij}$ defines the delay between x_i and o_j. T_j presents the threshold of the jth neuron, while w_ij is the connection coefficient from neuron i to neuron j.

An ANN consists of characteristics: the input layer, the hidden layer, the interconnection between different layers, the learning step to find the optimum values of interconnections weights, the transformer function which assigned to produce outputs refer to weighted inputs, the number of neurons performing in each layer and the output layer. Figure 3 schematically presents the architecture of an ANN with a single hidden layer.

As it has been shown in Figure 3, neurons are deployed in layers. Nodes of layers in a row are connected to show interactions and information flow in an ANN. The connection between node i and j defines by the weight w_ij and also a bias b_i parameter is assigned to each neuron [133]. To minimize the error at each step (which is known as epoch) an ANN compute and error function and uses an algorithm to reduce the error value.

An ANN has the ability to be trained in order to build a precise network and minimize the lost function by adjusting w_ij weight matrices [17]. So, the performance of the learning algorithm will define the performance of the ANN. In this paper, a genetic algorithm (GA) is used to equip ANN as the learning algorithm. In the next section, the GA procedure is explained briefly.

3.2.2. Genetic Algorithm

Training an ANN is very complex, which can directly influence outcomes’ quality. Recently, numerous academic studies are presented, which applied meta-heuristic and intelligent algorithms (i.e., GA) as learning algorithms [134].

GA is an evolutionary optimization approach developed by Holland in 1975 [101], which acts based on random search procedure. Compared to traditional optimization methods the GA has numerous advantages. For example, the algorithm converges to a good, feasible solution faster than other existing traditional methods [24]. Series of computational operators like selection, mutation, and crossover functions are used in a GA to achieve a reliable solution. Figure 4 briefly presents the GA procedure.

3.2.3. Genetic Neural Network

In this paper, weights and thresholds of the ANN are updated by a GA. For this purpose, input vectors transformed into a genetic gene in the format of the chromosome. Then, the initial population is formed from the randomly generated chromosome. Now values of the optimization algorithm such as selection, crossover, and mutation rates can be set to design the algorithm. The fitness function is the reciprocal of the quadratic sum of the difference between predicted and real values [135]. Roulette wheel selection is used to select a new individual, then two chromosomes are exchanged via crossover operation to generate a new individual. Finally, the mutation step is applied to avoid premature convergence.

Equipping an ANN with a GA could save training time and improve the precision of the forecasting model [135]. Figure 5 schematically shows the flowchart of the presented GNN.

The next section is dedicated to present the architecture of ANN, which is the basic framework of the developed forecasting model.

3.2.4. The Architecture of the ANN

This research targeted to present accurate NG demand predictions, so the selected features were inputted at the initiatory layer (input layer) of the designed ANN. A single hidden layer network was designed to perform the prediction, so the model contains a three-layer architecture. Figure 6 shows the performance of a three-layered NN for three, four, five, six, and seven neurons in the hidden layer. Four neurons were used for the hidden layer as it returns the best performance among other tested number of neurons (see Figure 6).

As it has been represented in Figure 6, based on the R² and root mean square error (RMSE) statistics, four number of neurons the proposed data mining genetic-neural network (DmGNn) model performs better than other examined set.

4. Outputs and Results

As mentioned before this paper is aimed at developing a forecasting model to accurately forecast global NG demand. Here, the historical behavior of the global NG demand from 1965 to 2013 period (billion cubic meters) is gathered via www.bp.com. Now, the model is designed, and it can be used to project future trends for NG global consumption. For this reason, 40 historical annual fundamental time series data (from 1665 to 2004) are investigated as a learning set. The forecasting period contains nine annual values for NG global demand prediction problem (from 2005 to 2013). Ten iterations have been investigated for the proposed DmGNn model. Figure 7 presents projections (average for 10 iterations) resulted from the DmGNn models.

Learning models were extensively applied in the case of NG demand predictions [62,136]. Some competitive prediction models were selected to compare outputs of the proposed model and analysis of the accuracy. Adaptive Neuro-Fuzzy Inference Systems (ANFIS) [137,138,139] and a set of classical well-known neural network-based techniques such as: Radial Basis Function Neural Network (RBF) [140,141], Multi-Layered Perceptron (MLP) [18,142], and Generalized Regression Neural Network (GRNN) [143,144,145] are nominated and optimized (through trial and error processes to minimize forecast errors) to prove the accuracy of the proposed DmGNn model through a comparison study.

To evaluate different models, a set of mathematical criteria organized to measure prediction performance. A relatively large set of validity indicators support the justification of model usage [21]. These statistics are summarized in

Table 4. Used mathematical criteria to evaluate forecast errors.

Error Title	Abbreviation	Formula
R-squared	R2	${\mathrm{R}}^2=1-\frac{S{S}_{res}}{S{S}_{tot}}$ *
Mean Absolute Error	MAE	$\mathrm{MAE}=\frac{1}{n}{\displaystyle {\displaystyle \sum }_i}\left|f_i-y_i\right|$
Mean Absolute Percentage Error	MAPE	$\mathrm{MAPE}=\frac{100}{n}{\displaystyle {\displaystyle \sum }_i}\left|\frac{f_i-y_i}{y_i}\right|$
Mean Bias Error	MBE	$\mathrm{MBE}=\frac{1}{n}{\displaystyle {\displaystyle \sum }_i}\left(f_i-y_i\right)$
Root Mean Square Error	RMSE	$\mathrm{RMSE}=\sqrt{\frac{{\displaystyle \sum }{\left(y_i-f_i\right)}^2}{n}}$

* Where, $S{S}_{tot}={{\displaystyle \sum }}_i{\left(y_i-\overline{y}\right)}^2$ and $S{S}_{res}={{\displaystyle \sum }}_i{\left(y_i-f_i\right)}^2={{\displaystyle \sum }}_i{\left(errors\right)}^2$.

(where y_i refers to real historical value and f_i presents forecasting value).

Each model ran for 10 times and the average of outputs was calculated.

Table 5. Statistical errors for each prediction model.

Models	Characteristics	R2	MAE	MAPE	MBE	RMSE
DmGNn	Number of Neurons = 4; Maximum generation = 100; Cross Over Probability = 0.8; Mutation Probability = 0.05;	0.9847	52.19	1.69	13.54	61.33
MLP	Maximum Epochs = 200; Train Parameter Goal = 1 × 10−7;	0.8241	115.59	3.80	−44.85	145.61
ANFIS	FIS Generation Approach: FCM; Number of Clusters = 10; Partition Matrix exponent = 2;	0.8494	63.45	1.89	21.31	84.31
RBF	Spread Value = 0.17;	0.0018	308.64	10.42	−308.64	366.51
GRNN	Spread Value = 1;	0.9864	127.63	4.17	−4.03	142.12

presents the performance of the proposed and competitive models refer to statistics introduced in Table 4.

As it is shown in Table 5, the proposed DmGNn significantly outperforms other competitive models. Although the GRNN presents a reasonable performance under R² (for R² both DmGNn and GRNN almost performed similar) and MBE criteria, however, it failed under MAE and RMSE error tests. Here multiple error tests have been implemented to show estimators’ accuracy comprehensively. Among them, MAE and RMSE are mentioned and used by many previous researchers and claimed that the combination of MAE and RMSE could be more beneficial [146]. The RMSE is more appropriate to represent model performance than the MAE when the error distribution is expected to be Gaussian [146]. Simultaneously, Willmott and Matsuura suggested that the RMSE is not a good indicator of average model performance and might be a misleading indicator of average error, and, thus, the MAE would be a better metric for that purpose [147]. The DmGNn overcame other competitive models under both RMSE and MAE.

The pattern of the absolute error for each model is shown in Figure 8, which represents how various forecasting models behave along the test period. As it is shown, the proposed DmGNn outperforms other benchmark forecasting models (with lower absolute error value for forecasting period) and resulted in a robust forecast series (unlike other forecasting models DmGNn’s forecast errors showed a low swing pattern).

Table 5 and Figure 8 showed how the proposed DmGNn model overcame other benchmark models. However, the initial objective was to propose a prediction model which is aimed to uncover main driving forces in order to approach to reliable extrapolations. ANN and GA were combined and implemented successfully by other previous researchers to predict future trends in the area of energy (for instance: [40,58,65,67,69]), but this paper presents a GNn with adjusted characteristics equipped with a purified input vector using data mining based pre-processing techniques. Data mining significantly helped to improve prediction accuracy and reliability. To show the efficiency of the data mining phases, both pre-processed and raw data were applied to the design prediction model.

Table 6. Statistical errors for different input vectors (raw versus processed).

Input Protocol	R2	MAE	MAPE	MBE	RMSE
Processed data	0.9847	52.19	1.69	13.54	61.33
Raw data	0.9679	79.96	2.61	2.66	94.21

compared the results under statistical errors criteria.

As it has been shown, even the GRNN with a purified input vector overcame the GNn in the case of R² indicator (0.9864 > 0.9679). Moreover, to dedicate a better understanding, Figure 9 shows absolute error for two different input protocols (purified using data mining and raw data) along the testing period. The line related to GNn represents more fluctuations than the green line (belongs to DmGNn), simultaneously shows a larger number for errors corresponding to each test point (years).

5. Conclusions

Energy is a major topic both in practice and theory, which many researchers investigated issues related to energy sectors and industries. The international energy supply system is characterized by a complicated combination of technological, social, economic, and political elements. Predicting and planning for the future global energy market is interesting and, simultaneously, a challenging subject in both research and practical investment projects. Thus the accurate prediction of energy demand is critical to developing future policies, modify current plans, and evaluate potential strategies since the energy market are complex and changing dynamically over time. This paper primary targeted to provide an accurate and robust prediction model to predict the global natural gas demands using a learning-based prediction model. On the other hand, authors aimed at introducing a process which reduces problem space dimensions to define the most relevant features subset which affects NG future consumption trends. The subset contains input features which shape the future, which are defined as driving forces. The estimator model can predict trends easier as the input is purified and the adjusted input vector has fewer dimensions than raw input vector, and also policymakers can monitor or in some cases manipulate NG market refers to extracted features.

In order to investigate maximum feasible solutions and to prevent missing any potential optimal option, input features were gathered based on the literature review and also related online dataset survey. Input features would define the model structure and support the accuracy of the output results. Although, increasing the number of input variables may cause computational complexity and reducing interpretability of the results. Instead, a large number of input features expands solution space and consequently reduces the probability of ignoring appropriate answers. A feature selection step is proposed and is implemented to reduce the dataset dimensions while guarantees that the prediction model will explore all optimal solutions. Finally, six input features were selected among 13 primary input features. The feature selection approach guarantees to investigate all solution space using a limited set of input features. Then possible time lags among input features versus the targeted attribute (NG global demand) were studied and subsequently applied to the refined input set. Investigating suitable time lags will cause in a more accurate and rational prediction model, which guarantees synchronization between input features and the target attribute at t time step. Finally, a neural network framework is developed, which equipped using a genetic algorithm to optimize the network’s characteristics aimed to predict future NG global demands.

Four benchmark models are investigated to study the performance of the proposed data mining genetic-neural network (DmGNn) model. The proposed DmGNn model outperforms other benchmark models refer to five different error statistics. Based on the R² statistic the DmGNn track real testing set fluctuations very well (only missed about 2%). Moreover, to distinct how the proposed pre-processing step affects the model accuracy, DmGNn model compared to a single GNn (without pre-processing phases). As shown the proposed pre-processing step improves predictions both in term of accuracy and reliability (robustness). Moreover, based on the interpretative capability index, the DmGNn dedicates a more clear vision about future trends since it uses a smaller input dataset. A limited input feature set enables decision-makers to design responsive policies/strategies/actions as they were aware of attributes affecting the global NG demands.

The proposed DmGNn is characterized by high flexibility, universal operation, learning ability, and low requirements for computation resources. As a result, it can be used by decision-makers and market participants who face a complex environment. Although results showed notably high-performance indexes for the proposed DmGNn model but such mathematical models are limited to conditions where appropriate and reliable data is provided. Moreover, in the conditions of availability of data mathematical estimators, even learning-based, are constrained to the historical time series and emerging signals with no significant background are neglected and subsequently and can cause deviation in the predictions. For further studies, a hybrid qualitative-quantitative model can be considered in order to cover a single quantitative prediction model’s weaknesses.