Next Article in Journal
How Does the Residential Complex Regulate Residents’ Behaviour? An Empirical Study to Identify Influential Components of Human Territoriality on Social Interaction
Previous Article in Journal
Composite Materials of Rice Husk and Reed Fibers for Thermal Insulation Plates Using Sodium Silicate as a Binder
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A New Insight for Daily Solar Radiation Prediction by Meteorological Data Using an Advanced Artificial Intelligence Algorithm: Deep Extreme Learning Machine Integrated with Variational Mode Decomposition Technique

1
Department of Water Engineering, Faculty of Agriculture, University of Tabriz, Tabriz 5166616471, Iran
2
Department of Civil Engineering, Faculty of Engineering, Tishk International University, Sulaimani 46001, Iraq
3
Department of Railroad Construction and Safety Engineering, Dongyang University, Yeongju 36040, Republic of Korea
4
Faculty of Science, Agronomy Department, Hydraulics Division, University 20 Août 1955, Route El Hadaik BP 26, Skikda 21000, Algeria
5
Department of Civil Engineering, Luebeck University of Applied Sciences, 23562 Lübeck, Germany
6
Department of Civil Engineering, Ilia State University, 0162 Tbilisi, Georgia
*
Author to whom correspondence should be addressed.
Sustainability 2023, 15(14), 11275; https://doi.org/10.3390/su151411275
Submission received: 6 June 2023 / Revised: 13 July 2023 / Accepted: 17 July 2023 / Published: 19 July 2023

Abstract

:
Reliable and precise estimation of solar energy as one of the green, clean, renewable and inexhaustible types of energies can play a vital role in energy management, especially in developing countries. Also, solar energy has less impact on the earth’s atmosphere and environment and can help to lessen the negative effects of climate change by lowering the level of emissions of greenhouse gas. This study developed thirteen different artificial intelligence models, including multivariate adaptive regression splines (MARS), extreme learning machine (ELM), Kernel extreme learning machine (KELM), online sequential extreme learning machine (OSELM), optimally pruned extreme learning machine (OPELM), outlier robust extreme learning machine (ORELM), deep extreme learning machine (DELM), and their versions combined with variational mode decomposition (VMD) as integrated models (VMD-DELM, VMD-ORELM, VMD-OPELM, VMD-OSELM, VMD-KELM, and VMD-ELM), for solar radiation estimation in Kurdistan region, Iraq. The daily meteorological data from 2017 to 2018 were used to implement suggested artificial models at Darbandikhan and Dukan stations, Iraq. The input parameters included daily data for maximum temperature (MAXTEMP), minimum temperature (MINTEMP), maximum relative humidity (MAXRH), minimum relative humidity (MINRH), sunshine duration (SUNDUR), wind speed (WINSPD), evaporation (EVAP), and cloud cover (CLOUDCOV). The results show that the proposed VMD-DELM algorithm considerably enhanced the simulation accuracy of standalone models’ daily solar radiation prediction, with average improvement in terms of RMSE of 13.3%, 20.36%, 25.1%, 27.1%, 34.17%, 38.64%, and 48.25% for Darbandikhan station and 5.22%, 10.01%, 10.26%, 21.01%, 29.7%, 35.8%, and 40.33% for Dukan station, respectively. The outcomes of this study reveal that the VMD-DELM two-stage model performed superiorly to the other approaches in predicting daily solar radiation by considering climatic predictors at both stations.

1. Introduction

Consumption of fossil fuels causes serious diseases, climate change, pollution in the environment, and change in the balance of ecology. Therefore, many developing countries have focused on clean and renewable energy sources in recent decades. Solar radiation is one of the most important clean and renewable energy resources, and its prediction is very necessary for agricultural, industrial, transport, and environmental applications, since it reduces greenhouse gases and thus is environmental friendly [1]. Temporal variation in solar radiation (SR) is the most important factor that provides the generation potential of a solar photovoltaic (PV) system. Therefore, predicting temporal variation in SR is essential for power generation in solar PV and for keeping balance between demand and real-time dispatch planning [2]. SR time series data have a non-linear nature and noise in their structure due to the effect of environmental parameters (e.g., air temperature, surface reflectivity, cloud cover, and aerosols). Furthermore, spatial SR measurements are not available. To cope with this issue, it is very essential to develop new methods considering data preprocessing approaches to precisely predict SR [1].
Machine learning (ML) methods are successfully used in predicting SR using different climatic parameters as input [3,4,5,6,7,8,9,10]. Fan et al. (2020) employed a hybrid support vector machine (SVM), combining it with metaheuristic algorithms for predicting daily SR and comparing it with multivariate adaptive regression splines (MARS) and extreme gradient boosting (XGBoost) models. They found that the hybrid SVM provided superior accuracy to the MARS and XGBoost [5]. Dong et al. (2020) applied three ML methods (i.e., XGBoost, SVM, and MARS) in estimating daily diffuse solar radiation. They found acceptable results from all ML methods they considered, and the XGBoost showed superior performance and stability compared to the other two methods [4]. Obiora et al. (2021) compared the long short-term memory (LSTM) network, convolutional LSTM (ConvLSTM), convolutional neural network (CNN), random forest (RF), SVM, and XGBoost in predicting solar radiation of Johannesburg using meteorological inputs, and ConvLSTM provided the best accuracy [9]. Huang et al. (2021) applied 12 ML methods to predict daily and monthly SR using meteorological inputs. The gradient boosting regression tree (GBRT), XGBoost, random forest, and Gaussian Process Regression (GPR) provided better prediction accuracy, and among them, the stacking model and XGBoost were selected as the best models for predicting SR [6]. Mbah et al. (2022) employed XGBoost to predict daily global SR on tilted surfaces using three inputs of time, day number, and horizontal SR. They found promising outcomes from the XGBoost model [11]. Solano et al. (2022) used SVM, categorical boosting (CatBoost), XGBoost, and voting average (VOA), which integrates SVM, XGBoost, and CatBoost in predicting SR 1 h, 2 h, and 3 h ahead in Salvador, Brazil. The ensemble model provided the best accuracy for all prediction horizons [12]. Namrata et al. (2023) investigated the accuracy of XGBoost by combining it with moth flame optimization (MFO) and grey wolf optimization (GWO) and found that the XGBoost-MFO performed superiorly to the others in predicting SR [13]. Goliatt and Yaseen (2023) applied hybrid covariance matrix adaptive evolution strategies (CMAES) with XGBoost and MARS models for the prediction of daily SR of Burkina Faso, Sub-Sahara Africa, and obtained promising outcomes [14]. Demir and Citakoglu (2023) predicted monthly SR using five ML methods, namely SVM, long short-term memory (LSTM), GPR, ELM, and K-nearest neighbors (KNN) methods, and they found that the LSTM and GPR methods provided the best accuracy for SR prediction in Turkey, which has arid and semi-arid climates [15].
The successful utilization of extreme learning machines (ELMs) in the prediction of SR has been achieved by employing diverse climatic input parameters [16,17,18,19,20]. Suyono et al. (2018) used an ELM for predicting solar radiation in Basel, Switzerland, the results of which are available to the public, and compared the results with multilinear regression (MLR). The ELM model improved the accuracy of MLR by 15.29% and 24.79% with respect to root mean square errors and mean absolute errors [18]. Hou et al. (2018) used an ELM model integrated with a variable forgetting factor (FOS-ELM) for predicting global SR at Bur Dedougou, Burkina Faso. They used wind speed, maximum and minimum temperature, maximum and minimum humidity, evaporation, and vapor pressure deficiency as inputs and found reliable prediction accuracy again compared to a standard ELM model [16]. Zhang and Wei (2019) developed a hybrid model with an ELM optimized by a bat algorithm (BA) and integrated it with wavelet transform and principal component analysis (PCA) for predicting daily SR data obtained by NASA. The hybrid model provided a promising prediction accuracy [20]. Feng et al. (2020) developed a hybrid ELM optimized by particle swarm optimization (PSO-ELM) for accurately predicting daily SR of the Loess Plateau of China and compared it with a standard ELM, an SVM, generalized regression neural networks, an M5 model tree, and an autoencoder. The PSO-ELM was found to be the best performing method for SR prediction [21]. Viscondi and Alves-Souza (2021) predicted SR of São Paulo, Brazil, using an artificial neural network (ANN), ELM, and SVM using meteorological parameters as inputs. SVM gave the lowest root mean square error, while ELM had the fastest training time [19]. Karaman et al. (2021) predicted the SR of Karaman province, Turkey, with an ELM and ANN and obtained superior accuracy from the ELM method [7]. Preeti et al. (2023) applied hybrid Kernel-based ELM for predicting incident SR in Australia using cloud properties obtained from the MODIS (Moderate Resolution Imaging Spectroradiometer) as input and obtained good accuracy [17]. Previous literature shows that the accuracy of the deep ELM method has not been investigated in SR prediction until now. Figure 1 was extracted from VOSViewer based on keywords that appeared in a literature review search. It indicates the importance and applicability of the artificial intelligence models for solar radiation prediction.
Guermoui et al. (2020) recently reviewed research works related to SR prediction using ML methods. They reported that the hybrid methods generally performed superiorly to the standalone methods using different climatic inputs and case studies, and they recommend the utilization of hybrid methods in predicting SR, even though they have more complex model structure and time-consuming procedures as disadvantages [22]. Data preprocessing methods are very useful in improving prediction accuracy, especially when the investigated phenomenon has complex behavior (noisy and chaotic data). Variational mode decomposition (VMD) is a promising data decomposition method [23,24,25,26] that can process exact mathematical information and is more sensitive to both noise and sampling compared to alternative methods such as empirical mode decomposition (EMD). VMD is a non-recursive variational method that searches a number of modes and their central frequencies in a band-limited manner and reconstructs the original data (signal) using the least-squares method [26]. VMD possesses a better denoising property and capability to separate similar frequencies compared to EMD. In the present study, VMD is utilized to improve the accuracy of the deep ELM method (VMD-DELM) in predicting SR. The outcomes of this method are compared with those of other ML methods including an outlier robust ELM (ORELM), optimally pruned ELM (OPELM), online sequential ELM (OSELM), kernel ELM (KLM), standalone ELM, and MARS. According to the best knowledge of the authors, there is no study in the related literature that employed this method in SR prediction. The main aims of this study are as follows: (i) to investigate the capability of the hybrid VMD-DELM method to predict SR using inputs of maximum and minimum temperature, maximum and minimum relative humidity, sunshine duration, wind speed, evaporation and cloud coverage; (ii) to compare the outcomes of VMD-DELM with those of VMD-ORELM, VMD-OPELM, VMD-OSELM, VMD-KELM, VMD-ELM, ORELM, OPELM, OSELM, KELM, ELM, and MARS. Therefore, this research provides a novel strategy for predicting solar radiation by hybridizing ELMs with the VMD technique using different meteorological parameters, which could have significant outcomes for a range of applications in energy management and climatic studies. In the next section, the methods implemented in this study are explained. The case study and data are provided in the third section. The fourth section provides the formulations and a brief explanation of the assessment criteria. The outcomes of the implemented models are provided in the fifth section, followed by the sixth section, which discusses the provided results. The conclusions derived from the applications are finally provided in the seventh section.

2. Models Descriptions

In this study, two different types of artificial intelligence models, namely standalone (i.e., DELM, ORELM, OPELM, OSELM, KELM, ELM, and MARS) and hybrid (i.e., VMD-DELM, VMD-ORELM, VMD-OPELM, VMD-OSELM, VMD-KELM, and VMD-ELM) models, were used for predicting SR at Darbandikhan and Dukan stations by applying daily data for maximum temperature, minimum temperature, maximum relative humidity, minimum relative humidity, sunshine duration, wind speed, evaporation, and cloud cover. The technique of VMD as one of the popular mathematical strategies that is applicable for breaking down a dataset into a different classes of intrinsic mode functions (IMFs) and extracting the effective features. The oscillatory patterns within the data can be identified using IMFs. These values are beneficial for constructing different artificial intelligence paradigms. During the workflow of decomposition, the VMD technique recognizes the most prominent frequency components and classifies them into separate modes from high-frequency noise to low-frequency trends. By carrying out the decomposing of the data in this manner, VMD is able to decrease noise and identify significant input parameters that are important for designing accurate artificial intelligence tools. The IMFs obtained from using the VMD strategy are the best candidates to be main features for a diverse range of artificial intelligence algorithms, including ORELM, OPELM, OSELM, KELM, ELM, and MARS algorithms.

2.1. Multivariate Adaptive Regression Splines (MARS)

Friedman (1991) introduces the multivariate adaptive regression splines (MARS) machine learning model [27]. The MARS algorithm uses an ensemble of basis spline functions (BFs) for predicting an output variable, and its building strategy can be achieved in three steps: (i) forward process, (ii) backward pruning, and (iii) selection using an efficient generalized cross-validation (GCV) algorithm (Figure 2). More precisely, the MARS algorithm is similar to the “stepwise linear regression”, as it combines an ensemble of “piecewise linear basis functions” having the following form [28]:
x t + = x t          i f   x > 0 0           o t h e r w i s e
t x + = t x          i f   x < 0 0           o t h e r w i s e
The output of the MARS model can be written as follows:
Y ^ x = δ 0 + m = 1 M δ m B F m x
where δ 0 is a constant value, δ m shows the coefficient linked to the m-th basis function, M indicates the number of basic functions, K denotes the number of piece-wise degree q polynomials, t is the knot, and B F m is the m-th basis function, which can be expressed as follows [28,29,30]:
B F m k x = k = 1 K x t + q

2.2. Extreme Learning Machine (ELM)

Huang et al. (2006) proposed an extreme learning machine algorithm (ELM) for training the single-layer feedforward neural network (SLFN) for which the hidden neurons between the input and the hidden layers are randomly generated, and the output weights between the hidden and the output layers are analytically calculated using the least-squares solutions [31,32]. The ELM algorithm can be summarized as follows. As illustrated in Figure 3, ELM consists of three layers: the input, hidden, and output layers. For a training dataset with N data couples (xi, yi), x i = x i 1 , x i 2 , x i 3 , , x i n T R n corresponds to the training input matrix (i.e., the predictors), and y i = x i 1 , x i 2 , x i 3 , , x i m T R m corresponds to the target value of the input variables. The response of the ELM model with L hidden neurons can be calculated as follows [33,34,35,36]:
f X = i = 1 L β i · Ø w i , b i , x j , j = 1 , 2 , 3 , , L
where Ø · is the activation function, wij is the weights between the input and the hidden layer, bi is the bias of the ith hidden node, and β i is the output weights. The previous equation can be reformulated as follows [37,38]:
H β = T
H = h x 1 h x 2 h x N = g w 1 , b 1 , x 1 g w 1 , b 1 , x 2 g w 1 , b 1 , x N g w L , b L , x 1 g w L , b L , x 2 g w L , b L , x N N × L
β = β 1 T β L T L × m        a n d           β = t 1 T t N T N × m
β = H + T
where H + corresponds to the Moore–Penrose generalized inverse of matrix H.
Using this kind of training algorithm, ELM is becoming a fast-learning algorithm [37,39]. The use of ELM has gained much popularity during the last few years, leading to the development of several improved version of the standalone ELM. In this study, the proposed ELM predictive model is based on one hidden layer of interconnected nodes, sigmoid activation function, and 15 hidden neurons, in order to improve performance of the ELM framework in terms of statistical parameters and gain more accurate results and generalization of both the training and testing stages. In the present study, four improved ELM versions were applied, namely: (i) kernel extreme learning machine (KELM), (ii) optimally pruned extreme learning machine (OPELM), (iii) online sequential extreme learning machine (OSELM), and (iv) outlier robust extreme learning machine (ORELM). The four algorithms are briefly described below.

2.3. Optimally Pruned Extreme Learning Machine (OPELM)

The optimally pruned extreme learning machine (OPELM) was developed by [40,41] to improve on the original ELM and to overcome some limitations of the original ELM. More precisely, the major advantage of the OPELM model is its ability and capability to prune irrelevant input variables available in the training dataset. The main steps of OPELM can be summarized as follows [42]: (i) based on a large number of neurons, an oversized ELM model is firstly presented; (ii) by applying the “multiresponse sparse regression (MRSR)”, the hidden neurons are ranked, taking into account their contribution to the ELM response; and (iii) the irrelevant neurons are pruned using the “leave-one-out (LOO)” validation (Figure 4). More details about OPELM can be found in [40,41].

2.4. Online Sequential Extreme Learning Machine (OSELM)

One of the most relevant improved versions of the original ELM is the online sequential extreme learning machine (OSELM) proposed by [43]. During the training of the ELM model, all training data should be available simultaneously, and if a new data point is available, the entire training process should be restarted. Liang et al. (2006) have introduced OSELM as a solution for the previous problem, and it is considered an incremental version of the batch ELM. OSELM has the ability and capability to learn the data “one by one” or “chunk by chunk”, and only the new data are trained, so the already used data are “discarded” [43]. OSELM can be achieved in two different steps: (i) the initialization and (ii) the sequential learning. During the first stage, only a small training dataset is presented to the OSELM model for an initialization process, and the initial output matrix is then calculated. However, during this initialization stage, it is recommended that the amount of data in the initial training sample be at least equal to if not greater than the number of hidden neurons. Finally, for the sequential stage, once new data are presented, the partial hidden layer output matrix is then calculated and the output weight is updated [43,44,45,46].

2.5. Outlier Robust Extreme Learning Machine (ORELM)

As stated by several researchers, the original ELM algorithm has several running limitations in the presence of outliers in the dataset, which were the most predominant reasons for developing the outlier robust extreme learning machine (ORELM). To overcome the presence of noise and outliers, ORELM was proposed by [47] based on the incorporation of an adjustment coefficient “C” called the “regularization parameter” as follows [47,48]:
m i n e 1 + 1 C β 2 2    s u b j e c t   t o   e = T H β
In the above equation, “e” is the output error, β is the output weight matrix, and T is the target variable. The corresponding augmented Lagrangian function of the above equation can be calculated as follows:
L μ   e , β , λ = e 1 + 1 C β 2 2 + λ T T H β e + μ 2 T H β e 2 2
where μ is a penalty coefficient, and λ is the Lagrange multiplier [47,48].

2.6. Kernel Extreme Learning Machine (KELM)

To improve the performance of the original ELM, Huang et al. (2012) introduced the kernel extreme learning machine (KELM) using several kernel functions [49]. Using Mercer’s conditions, KELM defines a kernel matrix as follows [50]:
Ω = H H T : Ω i j = h x i h x j = K x i , x j
where K (∙) is the kernel function. Typical kernel functions for KELM can be Gaussian kernel, hyperbolic tangent (sigmoid) kernel, wavelet kernel, and polynomial kernel. For example, the RBF kernel function can be written as follows [51,52]:
Ω = K x i , x j = e x p γ x i x j 2 ,   γ > 0
where γ is the kernel parameter.

2.7. Deep Extreme Learning Machine (DELM)

Based on the idea of deep learning architecture, a deep learning model (DELM) was firstly introduced using the same architecture as the autoencoder artificial neural network [53]. The basic structure of DELM is given in Figure 5, and it is composed of an ensemble of stacked layers of standalone ELM. For any given dataset having a training input and output variables (xi, yi), the mathematical connection between the output H(m−1) of hidden layer m − 1 and the output H(m) of hidden layer m can be written in the following manner [54]:
H m = f β m T , H M 1
Finally, the output weight β of the model is computed as follows:
β * = I C + H M T H M 1 H M T Y
where I is the unit matrix and C > 0 is the regularization factor [54,55].

2.8. Variational Mode Decomposition (VMD)

Variational mode decomposition (VMD) developed by [56] is a robust signal decomposition algorithm extensively applied in many area of scientific research. The VMD is an adaptive, non-recursive method of mode variation and signal processing for which the operation of calculating the decomposition components is conducted by iteratively finding the “frequency center” and “bandwidth” of each component based on the optimal solution of the variational model [57,58]. The decomposition using the VMD algorithm can be achieved as follows:
m i n u k , ω k k = 1 K t δ t + j π t u k t e j ω k t 2 2 s u b j e c t      t o k = 1 K u k = f t
In the above equation, u k = u 1 , u 2 , u 3 , , u k corresponds to the modal function, which is also called the intrinsic mode function (IMF), and ω k = ω 1 , ω 2 , ω 3 , , ω k is each center frequency. At the end of the decomposition process, the original signal is decomposed into an ensemble of IMF subcomponent, written as follows:
I M F i t = H i t cos λ i t
where IMFi (t) is the i-th IMF, and Hi and λi are the amplitude and the phase of the component, respectively [59]. More details about the VMD can be found in [56]. The flowchart of the proposed methodology is provided in Figure 6.

2.9. Hybrid Models Based on Variational Mode Decomposition

In the present study, variational mode decomposition (VMD) is applied to decompose the input variables into multiple subsequences of intrinsic mode functions (IMFs). First, the described models were applied and compared using the same input variables without decomposition. The second stage involves the prediction of the output variable using the IMFS obtained after decomposition: the obtained IMFs are used as new input variables. The third and final stage is the comparison of the obtained results with the measured data using several numerical indexes, i.e., the R, NSE, RMSE, and MAE, among others.

3. Case Study and Data Explanation

In this study, the performance of the suggested artificial intelligence models was assessed at Darbandikhan station, which is located nearby the multipurpose embankment Darbandikhan dam (Latitude 35.1109° N, Longitude 45.6953° E, Altitude = 513 m), and Dukan station, located nearby the multipurpose concrete arch dam (Latitude 35.9496° N, Longitude 44.9621° E, Altitude = 690 m). Table 1 shows the statistical characteristics of the dataset used in this study, including mean (Average), minimum (Min), maximum (Max), standard deviation (St. Dev.), and skewness (Skew) of maximum air temperature (MAXTEMP), minimum air temperature (MINTEMP), maximum relative humidity (MAXRH), minimum relative humidity (MINRH), sunshine duration (SUNDUR), wind speed (WINSPD), evaporation (EVAP), cloud cover (CLOUDCOV), and solar radiation (SR) calculated at Darbandikhan and Dukan meteorological stations in Kurdistan region, Iraq (Figure 7), for the period 1 January 2017 to 31 December 2018. In this study, the data were divided 80–20% for both training and testing stages for developing artificial intelligence models. In this study, the selection of Darbandikhan and Dukan meteorological stations for case studies was based on several factors. Firstly, both stations are representative of important drinking water sources in Kurdistan region, Iraq. Moreover, recently, both Darbandikhan and Dukan locations have encountered considerable growth and development, resulting in magnified problems in terms of sustainability of the environment.

4. Performance Metrics

In this study, four statistical metrics, namely, root mean square error (RMSE), mean absolute error (MAE), Nash–Sutcliffe Efficiency (NS), and correlation coefficient (r), were applied to assess model performance. The formulations of used indicators to evaluate prediction accuracy in each model are given below:
RMSE = i = 1 n ( SR ) io ( SR ) ip n
r = i = 1 n ( ( SR ) io ( SR ¯ ) io ) ( ( SR ) ip ( SR ¯ ) ip ) i = 1 n ( ( SR ) io ( SR ) io ¯ ) 2 i = 1 n ( ( SR ) ip ( SR ) ip ¯ ) 2
NS = 1 i = 1 n ( ( SR ) io ( SR ) ip ) 2 i = 1 n ( ( SR ) io ( SR ) io ¯ ) 2
MAE = i = 1 n ( SR ) io ( SR ) ip n
where (SR)ip denotes the models’ output values and (SR)io denotes the real values. Also, n is the total number of items.

5. Results

The current research utilized the different climatic variables for forecasting solar radiation (SR) at Darbandikhan and Dukan stations, Iraq. As explained previously, the appraisal of standalone (i.e., DELM, ORELM, OPELM, OSELM, KELM, ELM, and MARS) and hybrid (i.e., VMD-DELM, VMD-ORELM, VMD-OPELM, VMD-OSELM, VMD-KELM, and VMD-ELM) models for forecasting SR is the necessary feature of the current research scheme. Table 1 shows the basic statistical properties of climatic variables (i.e., maximum air temperature (MAXTEMP), minimum air temperature (MINTEMP), maximum relative humidity (MAXRH), minimum relative humidity (MINRH), sunshine duration (SUNDUR), wind speed (WINSPD), evaporation (EVAP), cloud cover (CLOUDCOV), and solar radiation) employed at Darbandikhan and Dukan stations. It can be found from Table 1 that both stations provide anaverage MAXRH over 60% based on the training and testing data. In case of MAXTEMP, two Iraq stations show high MAXTEMP values (e.g., higher than 43 °C). Figure 8a,b show the results of correlation matrix plots among individual input variables at Darbandikhan and Dukan stations. It can be imagined from Figure 8a,b that SUNDUR (CC = 0.91), MAXTEMP (CC = 0.81), EVAP (CC = 0.79), and MINTEMP (CC = 0.74) variables exhibited high correlation with SR among diverse input variables at Darbandikhan station. In addition, SUNDUR (CC = 0.86), EVAP (CC = 0.79), MAXTEMP (CC = 0.77), and MINTEMP (CC = 0.70) variables provided high correlation with SR among different input variables at Dukan station.
Wind rose diagrams [60] are visual charts that characterize the speed and direction of winds at a specific location and time. Provided in a circular pattern, the length of each spoke around the circle explains the amount of time that the wind blows from a particular direction. Colors along the spokes express categories of wind speed. The wind rose diagram from Figure 9a shows the wind blew from the east (E) for about 9.5% of the specific time interval, north/east/east (NEE) for about 8.1%, south/east/east (SEE) for about 8.0% of the specific time interval, and so on. Also, the longest spoke shows that the wind blew from the east (E) at speeds between 1−2 m/s for about 3.0% of the specific time interval, 2–3 m/s for about 5.0% of the specific time interval, and 3–4 m/s for about 1.4% of the specific time interval. Finally, 4–5 m/s for about 0.1% of the specific time interval can be estimated. In addition, Figure 9b shows that the wind blew from the west (W) for about 8.0% of the suggested time period, south (S) for about 6.5%, south/west/west (SWW) for about 7.2% of the suggested time period, etc. Furthermore, the longest spoke shows that the wind blew from the west (W) at speeds between 0–2 m/sec for about 1.2% of the suggested time period, 2–4 m/s for about 5.6% of the suggested time period, 4–6 m/s for about 0.8% of the suggested time period, and 6–8 m/s for about 0.4% of the suggested time period.

5.1. Forecasting Sunshine Duration at Darbandikhan Station

5.1.1. Implementation of Standalone Models

The forecasted arguments of standalone models based on three performance indexes (i.e., RMSE, NS, and MAE) for Darbandikhan station are assembled in Table 2, which shows that forecasted arguments of DELM (RMSE = 20.158 W/m2, NS = 0.944, and MAE = 13.974 W/m2) were better than those of ORELM, OPELM, OSELM, KELM, ELM, and MARS, conditional on the training phase. During the testing phase, DELM (RMSE = 28.186 W/m2, NS = 0.783, and MAE = 22.754 W/m2) performed better than the ORELM, OPELM, OSELM, KELM, ELM, and MARS models. Figure 10a–g provide the scatter-plots of forecasted and observed values of SR using standalone models during the testing phase. The fit line (blue color) and correlation coefficient values are located in the suitable scatter-plots. It can be inferred from Figure 10a–g that an apparent variation can be tracked among the developed standalone models. Also, DELM yielded the best correlation coefficient (R = 0.954) among all standalone models.

5.1.2. Implementation of Hybrid Models

The forecasted assessments of hybrid models dependent on three performance indexes (RMSE, NS, and MAE) for Darbandikhan station are listed in Table 3, which shows that forecasted assessments of the VMD-DELM (RMSE = 23.090 W/m2, NS = 0.927, and MAE = 16.425 W/m2) model were more exceptional than those of VMD-ORELM, VMD-OPELM, VMD-OSELM, VMD-KELM, and VMD-ELM based on the training phase. During the testing phase, the VMD-DELM (RMSE = 24.433 W/m2, NS = 0.837, and MAE = 20.255 W/m2) model was superior to the VMD-ORELM, VMD-OPELM, VMD-OSELM, VMD-KELM, and VMD-ELM models. Figure 11a–f present the scatter-plots of forecasted and observed values of SR utilizing hybrid models during the testing phase. The blue color line and correlation coefficient values are found in the appropriate scatter-plots. It can be imagined from Figure 11a–f that an obvious disparity can be tracked among the applied hybrid models.

5.1.3. Visual aids for Performances of Standalone and Hybrid Models

To verify the forecasted accuracy using the visual aids, Figure 12a,b provide the comparison of different models judged based on RMSE (W/m2) and NS indices during the testing phase for Darbandikhan station. As can be explained from Figure 12a, the comparison between standalone and hybrid models demonstrated that the hybrid models were clearly superior to standalone models based on the RMSE (W/m2) indices. In addition, Figure 12b shows that the hybrid models showed better performance in forecasting SR compared to the standalone models based on the NS indices. Also, VMD-DELM presented a first-rate model to forecast SR dependent on standalone and hybrid models. Additional descriptions can evaluate the performance of standalone and hybrid models using violin plots [61], error histograms, and Taylor diagrams [62]. The violin plot can be characterized as one means of ascertaining the distribution of defined numerical values. Figure 13a supplies a similar shape for DELM, ORELM, and OPELM regarding the maximum, minimum, mean, and median of observed values based on the standalone models. Also, Figure 13b provides an identical pattern for VMD-DELM, VMD-ORELM, and VMD-OSELM concerning statistical standards (i.e., maximum, minimum, mean, and median) of observed values based on the hybrid models. Figure 14a–h express the error histogram of different standalone and hybrid models, including special indices (i.e., the standard deviation (σ) and mean (μ)). It can be judged from Figure 14a–h that VMD-DELM achieved the best forecasting by furnishing the minimum values of standard deviation (σ) indices among the hybrid models (Figure 14a–d). Also, DELM, which yielded the best accuracy among the standalone models (Figure 14e–h), provided the lowest value of standard deviation (σ) indices during the testing phase. However, VMD-OSELM and OSELM supplied the worst SR forecasting, with the topmost value of standard deviation (σ) indices during the testing phase based on the different standalone and hybrid models. To confirm the models’ achievement, a Taylor diagram (Figure 15) employs the values of CC, NSD (normalized standard deviation), and RMSE. The leading function of the Taylor diagram can be clarified to find the closest model with the forecasted SR value dependent on standard deviation (polar axis) and correlation coefficient (radial axis). Figure 15, therefore, demonstrates the best accuracy of VMD-OPELM and VMD-DELM compared to the developed standalone and hybrid models.

5.2. Forecasting Sunshine Duration at Dukan Station

5.2.1. Employment of Standalone Models

The forecasted results of standalone models dependent on three performance indices (RMSE, NS, and MAE) for Dukan station are assembled in Table 4, which shows that forecasted results of ORELM (RMSE = 24.172 W/m2, NS = 0.886, and MAE = 18.355 W/m2) were better than those of DELM, OPELM, OSELM, KELM, ELM, and MARS based on the training phase. During the testing phase, DELM (RMSE = 23.974 W/m2, NS = 0.785, and MAE = 19.685 W/m2) accomplished better results than ORELM, OPELM, OSELM, KELM, ELM, and MARS. Figure 16a–g support the scatter-plots of observed values of SR and those forecasted using standalone models during the testing phase. The fit line (blue color) and correlation coefficient values are found in the proper scatter-plots. It can be inferred from Figure 16a–g that an obvious divergence can be tracked among the advanced standalone models. In addition, DELM maintained the leading correlation coefficient (R = 0.956) among all standalone models.

5.2.2. Employment of Hybrid Models

The forecasted evaluations of hybrid models conditional on three performance standards (RMSE, NS, and MAE) for Dukan station are organized in Table 5, which shows that forecasted evaluations of VMD-OPELM (RMSE = 23.534 W/m2, NS = 0.892, and MAE = 17.300 W/m2) were better than those of VMD-DELM, VMD-ORELM, VMD-OSELM, VMD-KELM, and VMD-ELM based on the training phase. During the testing phase, VMD-DELM (RMSE = 22.721 W/m2, NS = 0.807, and MAE = 18.392 W/m2) was more remarkable than VMD-ORELM, VMD-OPELM, VMD-OSELM, VMD-KELM, and VMD-ELM. Figure 17a–f demonstrate the scatter-plots of forecasted and observed values of SR utilizing hybrid models during the testing phase. The blue color line and correlation coefficient values are discovered in the relevant scatter-plots. It can be guessed from Figure 17a–f that a distinct distinction can be tracked among the enforced hybrid models. Also, VMD-DELM maintained the top correlation coefficient (R = 0.963) among the developed hybrid models.

5.2.3. Optical Services for Performances of Standalone and Hybrid Models

To validate the forecasted precision of utilizing the optical services, Figure 18a,b specify the comparison of different models judged based on RMSE (W/m2) and NS indices during the testing phase for Dukan station. As can be determined from Figure 18a, comparison between standalone and hybrid models displayed that the hybrid models were preferable to the standalone models, which was remarkably conditional on the RMSE (W/m2) indices. In addition, Figure 18b displays that the hybrid models yielded better achievements in forecasting SR compared to the standalone models, conditional on the NS indices. Also, VMD-DELM was the dominant model to forecast SR, conditional on standalone and hybrid models. The violin plot, error histogram, and Taylor diagram describe the performances of standalone and hybrid models using additional figures. Figure 19a supports an analogous aspect of the DELM, ORELM, and OSELM covering the maximum, minimum, mean, and median of observed values, conditional on the standalone models. Also, Figure 19b supplies a matching shape for VMD-DELM, VMD-ORELM, and VMD-OSELM conditional on the statistical standards (maximum, minimum, mean, and median) of observed values based on the hybrid models.
Figure 20a–h signify the error histogram of different standalone and hybrid models, including special indices (i.e., the standard deviation (σ) and mean (μ)). It can be indicated from Figure 20a–h that VMD-DELM accomplished the best forecasting by suggesting the minimum values of standard deviation (σ) indices among the hybrid models (Figure 20a–d). Also, DELM, which provided the best accuracy among the standalone models (Figure 20e–h), suggested the lowest value of standard deviation (σ) standard during the testing phase. However, VMD-OSELM and OSELM explained the worst SR forecasting with the topmost value of standard deviation (σ) standard during the testing phase. To approve the models’ performance, a Taylor diagram (Figure 21) uses the values of CC, NSD, and RMSE. The dominant approach of a Taylor diagram can be resolved by finding the closest model with the forecasted SR value conditional on standard deviation (polar axis) and correlation coefficient (radial axis). Figure 21, therefore, shows the dominant accuracy of VMD-DELM and VMD-OSELM based on the suggested standalone and hybrid models.

6. Discussion

The current research accomplished the forecasting of solar radiation (SR) by employing the standalone and hybrid models at Darbandikhan and Dukan stations, Iraq. Conditional on the statistical results of standalone models, DELM provided the best accuracy of solar radiation forecasting at Darbandikhan and Dukan stations during the testing phase. Also, VMD-DELM suggested the best accuracy of solar radiation forecasting at Darbandikhan and Dukan stations during the testing phase based on the statistical results of hybrid models. It is worth noting that all the developed hybrid models at Darbandikhan and Dukan stations provided better accuracy for forecasting solar radiation compared to corresponding standalone models (e.g., VMD-DELM vs. DELM). For example, NS values fluctuated from 0.393 to 0.783 for standalone models and from 0.728 to 0.837 for the hybrid models at Darbandikhan station, while the corresponding ranges were 0.458–0.785 and 0.682–0.807 for the standalone and hybrid models at Dukan station.
The essential aim of implementing the hybrid models can be defined as enhancing the forecasted precision of solar radiation compared with the corresponding standalone models. All the hybrid models could enhance the forecasted precision of corresponding standalone models dependent on the values of NS standard at Darbandikhan and Dukan stations. Contemplating the hybrid models, VMD-DELM (6.897% by DELM), VMD-ORELM (10.902% by ORELM), VMD-OPELM (10.000% by OPELM), VMD-OSELM (12.121% by OSELM), VMD-KELM (20.640% by KELM), and VMD-ELM (28.169% by ELM) improved the forecasted efficiency of solar radiation at Darbandikhan station. As well, VMD-DELM (2.803% by DELM), VMD-ORELM (4.731% by ORELM), VMD-OPELM (3.816% by OPELM), VMD-OSELM (12.174% by OSELM), VMD-KELM (20.033% by KELM), and VMD-ELM (28.437% by ELM) increased the forecasted precision of solar radiation at Dukan station. Proving to be the best model among the standalone and hybrid models, VMD-DELM, which supplied the best efficiency, boosted the forecasted efficiency of solar radiation by 1.578% (VMD-ORELM), 7.170% (VMD-OPELM), 7.722% (VMD-OSELM), 11.008% (VMD-KELM), and 14.973% (VMD-ELM) at Darbandikhan station, respectively. Also, recognizing the best model’s classification for standalone and hybrid models, VMD-DELM, which supplied the best precision, increased the forecasted precision of solar radiation by 1.255% (VMD-ORELM), 2.281% (VMD-OPELM), 4.264% (VMD-OSELM), 10.397% (VMD-KELM), and 18.328 (VMD-ELM) at Dukan station, respectively. In current research, the hybrid models could always build up the forecasted efficiency of appropriate standalone models, conditional on the values of NS standard on both stations. Also, noticing the similar articles on solar radiation prediction conditional on the standalone and hybrid models, Ghimire et al. (2019) provided the standalone (SVR) and hybrid (PSO-SVR and PSO-ODW-SVR) models to forecast solar radiation using the MODIS data in Australia. Results explained that PSO-ODW-SVR provided the best efficiency in forecasting solar radiation among the developed models [63]. Ali et al. (2021) performed solar radiation forecasting using standalone (MARS, Volterra, and RF) and hybrid (VMD combination) models in Australia. They found that VMD-SA-RF produced the best accuracy for solar radiation forecasting compared to other models [23]. Huynh et al. (2021) developed standalone (LSTM, MARS, and SVR) and hybrid (RLMD combination) models for solar radiation forecasting with 30-min interval in Vietnam. Results showed that RLMD-LSTM was the best model for forecasting solar radiation among the developed models [64]. In addition, Sivakumar et al. (2023) suggested standalone (GRU, LSTM, BiLSTM, CNN, DNN, ANN, and SVR) and hybrid (DWT and VMD combinations) models for solar radiation forecasting in India. They concluded that VMD-GRU provided the best precision for solar radiation forecasting compared to other models [65]. However, similar studies using current standalone (DELM, ORELM, OPELM, OSELM, KELM, and ELM) and hybrid (VMD combination) models have not been accomplished for predicting or forecasting solar radiation until now. Therefore, the current approach is judged to provide an original research contribution for predicting solar radiation. Dependent on the current research, because the solar radiation forecasting dependent on standalone and hybrid models has spotlighted the few data preprocessing and artificial intelligence techniques, the current research for solar radiation forecasting might be considered insignificant. Therefore, different studies employing diverse models are recommended to strengthen the forecasted precision of solar radiation by applying different meteorological variables.

7. Conclusions

The VMD data decomposition technique is beneficial for processing big datasets using AI models. By decomposing the data into smaller subsets and importing those subsets for training AI models, the training stage time reduction, efficiency improvement, and overfitting issue minimization can be investigated. Moreover, those tools can improve the performance of AI approaches by training them using diverse data subsets, yielding better achievements and generalization when working with new data. For this purpose, in the present study, the VMD-DELM model is developed by applying maximum air temperature, minimum air temperature, maximum relative humidity, minimum relative humidity, sunshine duration, wind speed, evaporation, and cloud cover as input parameters for prediction of solar radiation at Darbandikhan and Dukan stations. The meteorological data from January 2017 to December 2018 at daily scale were inserted to develop the VMD-DELM model in order to obtain a high level of generalization performance in solar radiation estimation. For comparison, VMD-ORELM, VMD-OPELM, VMD-OSELM, VMD-KELM, VMD-ELM, standalone DELM, ORELM, OPELM, OSELM, KELM, ELM, and MARS models were applied in this study. For accuracy evaluation of the proposed VMD-DELM model, several criteria including RMSE, NS, MAE, and R were adopted. As results indicate, the VMD-DELM model showed more accurate results at both sites, with RMSE of 24.433 (W/m2) (Darbandikhan) and 22.721 (W/m2) (Dukan), whereas the MAE was 20.255 (W/m2) (Darbandikhan) and 18.392 (W/m2) (Dukan), respectively. Also, R was 0.962 (Darbandikhan) and 0.963 (Dukan) for the testing phase, respectively.
This study provides a framework based on artificial intelligence methodology through combining the VMD technique and DELM by applying historical climatological parameters to predict solar radiation. Incorporation of other decomposition tools and application of remote sensing data could be investigated in future studies. Accurate prediction of solar radiation would assist government and policymakers in planning of future renewable energy systems and optimization of energy generation resources. Future studies could involve the implementation of new tools that can investigate more complex and non-linear datasets, as well as combining various data decomposition techniques to produce hybrid methods that can maximize the accuracy of each individual model to produce more accurate results.

Author Contributions

Conceptualization, M.A.; Software, M.A.; Data curation, K.O.A.; Writing—original draft, M.A., K.O.A., J.S., A.F.F., S.K., S.H. and O.K.; Writing—review & editing, M.A., K.O.A., J.S., A.F.F., S.K., S.H. and O.K.; Visualization, M.A.; Supervision, M.A.; Project administration, M.A. All authors have read and agreed to the published version of the manuscript.

Funding

This project has been supported by a research grant of the University of Tabriz (number 2216).

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

AIArtificial intelligence
ANNArtificial neural network
CLOUDCOVCloud cover
CNNConvolutional neural network
DELMDeep extreme learning machine
DLDeep learning
EVAPEvaporation
IMFIntrinsic mode functions
KELMKernel extreme learning machine
MAEMean absolute error
MARSMultivariate adaptive regression splines
MAXRHMaximum relative humidity
MAXTEMPMaximum temperature
MINRHMinimum relative humidity
MINTEMPMinimum temperature
MLMachine Learning
NSNash-Sutcliffe Efficiency
OPELMOptimally pruned extreme learning machine
ORELMOutlier robust extreme learning machine
OSELMOnline sequential extreme learning machine
RFRandom forest
RMSERoot mean square error
SRSolar radiation
SUNDURSunshine duration
SVMSupport vector machines
VMDVariational mode decomposition
WINSPDWind speed

References

  1. Attar, N.F.; Sattari, M.T.; Prasad, R.; Apaydin, H. Comprehensive review of solar radiation modeling based on artificial intelligence and optimization techniques: Future concerns and considerations. Clean Technol. Environ. Policy 2022, 25, 1079–1097. [Google Scholar] [CrossRef]
  2. Acikgoz, H. A novel approach based on integration of convolutional neural networks and deep feature selection for short-term solar radiation forecasting. Appl. Energy 2022, 305, 117912. [Google Scholar] [CrossRef]
  3. Alizamir, M.; Kim, S.; Kisi, O.; Zounemat-Kermani, M. A comparative study of several machine learning based non-linear regression methods in estimating solar radiation: Case studies of the USA and Turkey regions. Energy 2020, 197, 117239. [Google Scholar] [CrossRef]
  4. Dong, J.; Wu, L.; Liu, X.; Fan, C.; Leng, M.; Yang, Q. Simulation of daily diffuse solar radiation based on three machine learning models. Comput. Model. Eng. Sci. 2020, 123, 49–73. [Google Scholar] [CrossRef]
  5. Fan, J.; Wu, L.; Ma, X.; Zhou, H.; Zhang, F. Hybrid support vector machines with heuristic algorithms for prediction of daily diffuse solar radiation in air-polluted regions. Renew. Energy 2020, 145, 2034–2045. [Google Scholar] [CrossRef]
  6. Huang, L.; Kang, J.; Wan, M.; Fang, L.; Zhang, C.; Zeng, Z. Solar Radiation Prediction Using Different Machine Learning Algorithms and Implications for Extreme Climate Events. Front. Earth Sci. 2021, 9, 596860. [Google Scholar] [CrossRef]
  7. Karaman, Ö.A.; Tanyıldızı Ağır, T.; Arsel, İ. Estimation of solar radiation using modern methods. Alex. Eng. J. 2021, 60, 2447–2455. [Google Scholar] [CrossRef]
  8. Kisi, O.; Alizamir, M.; Trajkovic, S.; Shiri, J.; Kim, S. Solar radiation estimation in Mediterranean climate by weather variables using a novel Bayesian model averaging and machine learning methods. Neural Process. Lett. 2020, 52, 2297–2318. [Google Scholar] [CrossRef]
  9. Obiora, C.N.; Hasan, A.N.; Ali, A.; Alajarmeh, N. Forecasting Hourly Solar Radiation Using Artificial Intelligence Techniques Prévision du rayonnement solaire horaire à l’aide de techniques d’intelligence artificielle. IEEE Can. J. Electr. Comput. Eng. 2021, 44, 497–508. [Google Scholar] [CrossRef]
  10. Alizamir, M.; Shiri, J.; Fard, A.F.; Kim, S.; Gorgij, A.D.; Heddam, S.; Singh, V.P. Improving the accuracy of daily solar radiation prediction by climatic data using an efficient hybrid deep learning model: Long short-term memory (LSTM) network coupled with wavelet transform. Eng. Appl. Artif. Intell. 2023, 123, 106199. [Google Scholar] [CrossRef]
  11. Mbah, O.M.; Madueke, C.I.; Umunakwe, R.; Agba, M.N. Extreme Gradient Boosting: A Machine Learning Technique for Daily Global Solar Radiation Forecasting on Tilted Surfaces. J. Eng. Sci. 2022, 9, E1–E6. [Google Scholar] [CrossRef] [PubMed]
  12. Solano, E.S.; Dehghanian, P.; Affonso, C.M. Solar Radiation Forecasting Using Machine Learning and Ensemble Feature Selection. Energies 2022, 15, 7049. [Google Scholar] [CrossRef]
  13. Namrata, K.; Kumar, M.; Kumar, N. Data-Driven Hyperparameter Optimized Extreme Gradient Boosting Machine Learning Model for Solar Radiation Forecasting. Adv. Electr. Electron. Eng. 2023, 20, 549–559. [Google Scholar] [CrossRef]
  14. Goliatt, L.; Yaseen, Z.M. Development of a hybrid computational intelligent model for daily global solar radiation prediction. Expert Syst. Appl. 2023, 212, 120357. [Google Scholar] [CrossRef]
  15. Demir, V.; Citakoglu, H. Forecasting of solar radiation using different machine learning approaches. Neural Comput. Appl. 2023, 35, 887–906. [Google Scholar] [CrossRef]
  16. Hou, M.; Zhang, T.; Weng, F.; Ali, M.; Al-Ansari, N.; Yaseen, Z.M. Global solar radiation prediction using hybrid online sequential extreme learning machine model. Energies 2018, 11, 3415. [Google Scholar] [CrossRef] [Green Version]
  17. Preeti; Bala, R.; Singh, R.P. Prediction of Incident Solar Radiation Using a Hybrid Kernel Based Extreme Learning Machine. Int. J. Artif. Intell. Tools 2023, 32, 2350004. [Google Scholar] [CrossRef]
  18. Suyono, H.; Santoso, H.; Hasanah, R.N.; Wibawa, U.; Musirin, I. Prediction of solar radiation intensity using extreme learning machine. Indones. J. Electr. Eng. Comput. Sci. 2018, 12, 691–698. [Google Scholar] [CrossRef]
  19. Viscondi, G.d.F.; Alves-Souza, S.N. Solar irradiance prediction with machine learning algorithms: A Brazilian case study on photovoltaic electricity generation. Energies 2021, 14, 5657. [Google Scholar] [CrossRef]
  20. Zhang, X.; Wei, Z. A hybrid model based on principal component analysis, wavelet transform, and extreme learning machine optimized by bat algorithm for daily solar radiation forecasting. Sustainability 2019, 11, 4138. [Google Scholar] [CrossRef] [Green Version]
  21. Feng, Y.; Hao, W.; Li, H.; Cui, N.; Gong, D.; Gao, L. Machine learning models to quantify and map daily global solar radiation and photovoltaic power. Renew. Sustain. Energy Rev. 2020, 118, 109393. [Google Scholar] [CrossRef]
  22. Guermoui, M.; Melgani, F.; Gairaa, K.; Mekhalfi, M.L. A comprehensive review of hybrid models for solar radiation forecasting. J. Clean. Prod. 2020, 258, 120357. [Google Scholar] [CrossRef]
  23. Ali, M.; Prasad, R.; Xiang, Y.; Khan, M.; Farooque, A.A.; Zong, T.; Yaseen, Z.M. Variational mode decomposition based random forest model for solar radiation forecasting: New emerging machine learning technology. Energy Rep. 2021, 7, 6700–6717. [Google Scholar] [CrossRef]
  24. An, X.; Tang, Y. Application of variational mode decomposition energy distribution to bearing fault diagnosis in a wind turbine. Trans. Inst. Meas. Control 2017, 39, 1000–1006. [Google Scholar] [CrossRef]
  25. Hu, H.; Wang, L.; Tao, R. Wind speed forecasting based on variational mode decomposition and improved echo state network. Renew. Energy 2021, 164, 729–751. [Google Scholar] [CrossRef]
  26. Majumder, I.; Dash, P.K.; Bisoi, R. Variational mode decomposition based low rank robust kernel extreme learning machine for solar irradiation forecasting. Energy Convers. Manag. 2018, 171, 787–806. [Google Scholar] [CrossRef]
  27. Friedman, J.H. Multivariate adaptive regression splines. Ann. Stat. 1991, 19, 1–67. [Google Scholar] [CrossRef]
  28. Liu, B.; Vu-Bac, N.; Zhuang, X.; Fu, X.; Rabczuk, T. Stochastic integrated machine learning based multiscale approach for the prediction of the thermal conductivity in carbon nanotube reinforced polymeric composites. Compos. Sci. Technol. 2022, 224, 109425. [Google Scholar] [CrossRef]
  29. Xu, Q.S.; Daeyaert, F.; Lewi, P.J.; Massart, D.L. Studies of relationship between biological activities and HIV reverse transcriptase inhibitors by multivariate adaptive regression splines with curds and whey. Chemom. Intell. Lab. Syst. 2006, 82, 24–30. [Google Scholar] [CrossRef]
  30. Saporetti, C.M.; Fonseca, D.L.; Oliveira, L.C.; Pereira, E.; Goliatt, L. Hybrid machine learning models for estimating total organic carbon from mineral constituents in core samples of shale gas fields. Mar. Pet. Geol. 2022, 143, 105783. [Google Scholar] [CrossRef]
  31. Huang, G.B.; Chen, L.; Siew, C.K. Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Netw. 2006, 17, 879–892. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Huang, G.B.; Zhu, Q.Y.; Siew, C.K. Extreme learning machine: Theory and applications. Neurocomputing 2006, 70, 489–501. [Google Scholar] [CrossRef]
  33. Zhang, J.; Li, Y.; Xiao, W.; Zhang, Z. Non-iterative and fast deep learning: Multilayer extreme learning machines. J. Frankl. Inst. 2020, 357, 8925–8955. [Google Scholar] [CrossRef]
  34. Alizamir, M.; Heddam, S.; Kim, S.; Mehr, A.D. On the implementation of a novel data-intelligence model based on extreme learning machine optimized by bat algorithm for estimating daily chlorophyll-a concentration: Case studies of river and lake in USA. J. Clean. Prod. 2021, 285, 124868. [Google Scholar] [CrossRef]
  35. Alizamir, M.; Kim, S.; Zounemat-Kermani, M.; Heddam, S.; Shahrabadi, A.H.; Gharabaghi, B. Modelling daily soil temperature by hydro-meteorological data at different depths using a novel data-intelligence model: Deep echo state network model. Artif. Intell. Rev. 2021, 54, 2863–2890. [Google Scholar] [CrossRef]
  36. Wei, F.; Wu, Y.; Xu, S.; Wang, X. Accurate visible light positioning technique using extreme learning machine and meta-heuristic algorithm. Opt. Commun. 2023, 532, 129245. [Google Scholar] [CrossRef]
  37. Wong, P.K.; Yang, Z.; Vong, C.M.; Zhong, J. Real-time fault diagnosis for gas turbine generator systems using extreme learning machine. Neurocomputing 2014, 128, 249–257. [Google Scholar] [CrossRef]
  38. Alizamir, M.; Kisi, O.; Kim, S.; Heddam, S. A novel method for lake level prediction: Deep echo state network. Arab. J. Geosci. 2020, 13, 956. [Google Scholar] [CrossRef]
  39. Kisi, O.; Alizamir, M. Modelling reference evapotranspiration using a new wavelet conjunction heuristic method: Wavelet extreme learning machine vs wavelet neural networks. Agric. For. Meteorol. 2018, 263, 41–48. [Google Scholar] [CrossRef]
  40. Miche, Y.; Sorjamaa, A.; Lendasse, A. OP-ELM: Theory, experiments and a toolbox. In Proceedings of the International Conference on Artificial Neural Networks, Lecture Notes in Computer Science, Prague, Czech Republic, 3–6 September 2008; Volume 5163, pp. 145–154. [Google Scholar]
  41. Miche, Y.; Bas, P.; Jutten, C.; Simula, O.; Lendasse, A. A methodology for building regression models using extreme learning machine: OPELM. In Proceedings of the ESANN 2008, European Symposium on Artificial Neural Networks, Bruges, Belgium, 5–7 October 2008. [Google Scholar]
  42. Grigorievskiy, A.; Miche, Y.; Ventelä, A.M.; Séverin, E.; Lendasse, A. Long-term time series prediction using OP-ELM. Neural Netw. 2014, 51, 50–56. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  43. Liang, N.Y.; Huang, G.B.; Saratchandran, P.; Sundararajan, N. A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans. Neural Netw. 2006, 17, 1411–1423. [Google Scholar] [CrossRef] [PubMed]
  44. Mirza, B.; Kok, S.; Dong, F. Multi-layer online sequential extreme learning machine for image classification. In Proceedings of the ELM-2015 Volume 1: Theory, Algorithms and Applications (I), Hangzhou, China, 15–17 December 2015; Springer International Publishing: Berlin/Heidelberg, Germany, 2016; pp. 39–49. [Google Scholar]
  45. Zhang, S.; Liu, Y.; Wang, J.; Wang, C. Research on combined model based on multi-objective optimization and application in wind speed forecast. Appl. Sci. 2019, 9, 423. [Google Scholar] [CrossRef] [Green Version]
  46. Alizamir, M.; Heddam, S.; Kim, S.; Gorgij, A.D.; Li, P.; Ahmed, K.O.; Singh, V.P. Prediction of daily chlorophyll-a concentration in rivers by water quality parameters using an efficient data-driven model: Online sequential extreme learning machine. Acta Geophys. 2021, 69, 2339–2361. [Google Scholar] [CrossRef]
  47. Zhang, K.; Luo, M. Outlier-robust extreme learning machine for regression problems. Neurocomputing 2015, 151, 1519–1527. [Google Scholar] [CrossRef]
  48. Liu, H.; Chen, C. Spatial air quality index prediction model based on decomposition, adaptive boosting, and three-stage feature selection: A case study in China. J. Clean. Prod. 2020, 265, 121777. [Google Scholar] [CrossRef]
  49. Huang, G.; Zhou, H.; Ding, X.; Zhang, R. Extreme learning machine for regression and multiclass classification. IEEE Trans Syst. Man Cybern. Part B 2012, 42, 513–529. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  50. Zhu, X.; Guo, H.; Huang, J.J.; Tian, S.; Zhang, Z. A Hybrid Decomposition and Machine Learning Model for Forecasting Chlorophyll-a and Total Nitrogen Concentration in Coastal Waters. J. Hydrol. 2023, 619, 129207. [Google Scholar] [CrossRef]
  51. Song, C.; Yao, L.; Hua, C.; Ni, Q. Comprehensive water quality evaluation based on kernel extreme learning machine optimized with the sparrow search algorithm in Luoyang River Basin, China. Environ. Earth Sci. 2021, 80, 521. [Google Scholar] [CrossRef]
  52. Alizamir, M.; Kim, S.; Zounemat-Kermani, M.; Heddam, S.; Kim, N.W.; Singh, V.P. Kernel extreme learning machine: An efficient model for estimating daily dew point temperature using weather data. Water 2020, 12, 2600. [Google Scholar] [CrossRef]
  53. Kasun, L.L.C.; Zhou, H.; Huang, G.B.; Vong, C.M. Representational learning with extreme learning machine for big data. IEEE Intell. Syst. (SCI-E) 2013, 28, 31–34. [Google Scholar]
  54. Jia, Z.; Liu, Z.; Cai, Y. A novel fault diagnosis method for aircraft actuator based on ensemble model. Measurement 2021, 176, 109235. [Google Scholar] [CrossRef]
  55. Wu, F.; Li, J.; Geng, W.; Tang, W.; Chen, X.; Zhao, W. Predicting anisotropic parameters of strata by deep multiple triangular kernel extreme learning machine optimized by flower pollination algorithm. J. Appl. Geophys. 2023, 209, 104904. [Google Scholar] [CrossRef]
  56. Dragomiretskiy, K.; Zosso, D. Variational mode decomposition. IEEE T Signal Process. 2014, 62, 531–544. [Google Scholar] [CrossRef]
  57. Wang, H.; Kang, C.; Tian, Z.; Zhang, A.; Cao, Y. Vegetation periodic changes and relationships with climate in Inner Mongolia Based on the VMD method. Ecol. Indic. 2023, 146, 109764. [Google Scholar] [CrossRef]
  58. Zhang, C.; Zhang, Y.; Liu, D.; Zhang, X.; Wu, X.; Qian, Z.; Li, X. A novel two-level approach to defect detection in braided CFRP using air-coupled ultrasonic testing. Ultrasonics 2023, 128, 106884. [Google Scholar] [CrossRef]
  59. Zhu, C.; Yang, H.; Fan, Y.; Fan, B.; Xu, K. Online spatiotemporal modeling for time-varying distributed parameter systems using Kernel-based Multilayer Extreme Learning Machine. Nonlinear Dyn. 2022, 107, 761–780. [Google Scholar] [CrossRef]
  60. Graedel, T.E. The wind boxplot: An improved wind rose. J. Appl. Meteorol. 1977, 16, 448–450. [Google Scholar] [CrossRef]
  61. Hintze, J.L.; Nelson, R.D. Violin plots: A box plot-density trace synergism. Am. Stat. 1998, 52, 181–184. [Google Scholar]
  62. Taylor, K.E. Summarizing multiple aspects of model performance in a single diagram. J. Geophys. Res. Atmos. 2001, 106, 7183–7192. [Google Scholar] [CrossRef]
  63. Ghimire, S.; Deo, R.C.; Raj, N.; Mi, J. Wavelet-based 3-phase hybrid SVR model trained with satellite-derived predictors, particle swarm optimization and maximum overlap discrete wavelet transform for solar radiation prediction. Renew. Sustain. Energy Rev. 2019, 113, 109247. [Google Scholar] [CrossRef]
  64. Huynh, A.N.L.; Deo, R.C.; Ali, M.; Abdulla, S.; Raj, N. Novel short-term solar radiation hybrid model: Long short-term memory network integrated with robust local mean decomposition. Appl. Energy 2021, 298, 117193. [Google Scholar] [CrossRef]
  65. Sivakumar, M.; George, S.T.; Subathra, M.S.P.; Leebanon, R.; Kumar, N.M. Nine novel ensemble models for solar radiation forecasting in Indian cities based on VMD and DWT integration with the machine and deep learning algorithms. Comput. Electr. Eng. 2023, 108, 108691. [Google Scholar] [CrossRef]
Figure 1. The major keywords abstracted from the Scopus database for solar radiation estimation using artificial intelligence paradigms.
Figure 1. The major keywords abstracted from the Scopus database for solar radiation estimation using artificial intelligence paradigms.
Sustainability 15 11275 g001
Figure 2. The MARS algorithm structure.
Figure 2. The MARS algorithm structure.
Sustainability 15 11275 g002
Figure 3. The extreme learning machine structure.
Figure 3. The extreme learning machine structure.
Sustainability 15 11275 g003
Figure 4. The flowchart of the OPELM algorithm.
Figure 4. The flowchart of the OPELM algorithm.
Sustainability 15 11275 g004
Figure 5. Deep extreme learning machine architecture.
Figure 5. Deep extreme learning machine architecture.
Sustainability 15 11275 g005
Figure 6. The flowchart of the proposed methodology used in the study.
Figure 6. The flowchart of the proposed methodology used in the study.
Sustainability 15 11275 g006
Figure 7. Map of the study region (a). The locations of stations and weather towers: (b) Darbandikhan station, (c) Dukan station.
Figure 7. Map of the study region (a). The locations of stations and weather towers: (b) Darbandikhan station, (c) Dukan station.
Sustainability 15 11275 g007
Figure 8. Correlation matrix plots for the (a) Darbandikhan and (b) Dukan stations.
Figure 8. Correlation matrix plots for the (a) Darbandikhan and (b) Dukan stations.
Sustainability 15 11275 g008
Figure 9. Wind rose plots for the (a) Darbandikhan and (b) Dukan stations.
Figure 9. Wind rose plots for the (a) Darbandikhan and (b) Dukan stations.
Sustainability 15 11275 g009
Figure 10. Scatter-plots of forecasted and observed SR using standalone models during the testing phase for Darbandikhan station: (a) DELM, (b) ORELM, (c) OPELM, (d) OSELM, (e) KELM, (f) ELM, and (g) MARS.
Figure 10. Scatter-plots of forecasted and observed SR using standalone models during the testing phase for Darbandikhan station: (a) DELM, (b) ORELM, (c) OPELM, (d) OSELM, (e) KELM, (f) ELM, and (g) MARS.
Sustainability 15 11275 g010aSustainability 15 11275 g010b
Figure 11. Scatter-plots of forecasted and observed SR using hybrid models during the testing phase for Darbandikhan station: (a) VMD-DELM, (b) VMD-ORELM, (c) VMD-OPELM, (d) VMD-OSELM, (e) VMD-KELM, (f) VMD-ELM.
Figure 11. Scatter-plots of forecasted and observed SR using hybrid models during the testing phase for Darbandikhan station: (a) VMD-DELM, (b) VMD-ORELM, (c) VMD-OPELM, (d) VMD-OSELM, (e) VMD-KELM, (f) VMD-ELM.
Sustainability 15 11275 g011
Figure 12. Comparison of the different models evaluated in terms of (a) RMSE (W/m2) and (b) NS during the testing phase for Darbandikhan station.
Figure 12. Comparison of the different models evaluated in terms of (a) RMSE (W/m2) and (b) NS during the testing phase for Darbandikhan station.
Sustainability 15 11275 g012aSustainability 15 11275 g012b
Figure 13. Violin plots of forecasted and observed SR using (a) standalone and (b) hybrid models during the testing phase for Darbandikhan station.
Figure 13. Violin plots of forecasted and observed SR using (a) standalone and (b) hybrid models during the testing phase for Darbandikhan station.
Sustainability 15 11275 g013aSustainability 15 11275 g013b
Figure 14. Values of error histogram for SR forecasting by different models during the test phase for Darbandikhan station: (a) VMD-DELM, (b) VMD-ORELM, (c) VMD-OPELM, (d) VMD-OSELM, (e) DELM, (f) ORELM, (g) OPELM, and (h) OSELM.
Figure 14. Values of error histogram for SR forecasting by different models during the test phase for Darbandikhan station: (a) VMD-DELM, (b) VMD-ORELM, (c) VMD-OPELM, (d) VMD-OSELM, (e) DELM, (f) ORELM, (g) OPELM, and (h) OSELM.
Sustainability 15 11275 g014aSustainability 15 11275 g014bSustainability 15 11275 g014c
Figure 15. Taylor diagram of different models for SR forecasting during the test phase for Darbandikhan station.
Figure 15. Taylor diagram of different models for SR forecasting during the test phase for Darbandikhan station.
Sustainability 15 11275 g015
Figure 16. Scatter-plots of forecasted and observed SR using standalone models during the testing phase for Dukann station: (a) DELM, (b) ORELM, (c) OPELM, (d) OSELM, (e) KELM, (f) ELM, and (g) MARS.
Figure 16. Scatter-plots of forecasted and observed SR using standalone models during the testing phase for Dukann station: (a) DELM, (b) ORELM, (c) OPELM, (d) OSELM, (e) KELM, (f) ELM, and (g) MARS.
Sustainability 15 11275 g016aSustainability 15 11275 g016b
Figure 17. Scatter-plots of forecasted and observed SR using hybrid models during the testing phase for Dukan station: (a) VMD-DELM, (b) VMD-ORELM, (c) VMD-OPELM, (d) VMD-OSELM, (e) VMD-KELM, (f) VMD-ELM.
Figure 17. Scatter-plots of forecasted and observed SR using hybrid models during the testing phase for Dukan station: (a) VMD-DELM, (b) VMD-ORELM, (c) VMD-OPELM, (d) VMD-OSELM, (e) VMD-KELM, (f) VMD-ELM.
Sustainability 15 11275 g017aSustainability 15 11275 g017b
Figure 18. Comparison of the different models evaluated in terms of (a) RMSE (W/m2) and (b) NS during the testing phase for Dukan station.
Figure 18. Comparison of the different models evaluated in terms of (a) RMSE (W/m2) and (b) NS during the testing phase for Dukan station.
Sustainability 15 11275 g018
Figure 19. Violin plots of forecasted and observed SR using (a) standalone and (b) hybrid models during the testing phase for Dukan station.
Figure 19. Violin plots of forecasted and observed SR using (a) standalone and (b) hybrid models during the testing phase for Dukan station.
Sustainability 15 11275 g019
Figure 20. Values of error histogram for SR forecasting by different models during the test phase for Dukan station: (a) VMD-DELM, (b) VMD-ORELM, (c) VMD-OPELM, (d) VMD-OSELM, (e) DELM, (f) ORELM, (g) OPELM, (h) OSELM.
Figure 20. Values of error histogram for SR forecasting by different models during the test phase for Dukan station: (a) VMD-DELM, (b) VMD-ORELM, (c) VMD-OPELM, (d) VMD-OSELM, (e) DELM, (f) ORELM, (g) OPELM, (h) OSELM.
Sustainability 15 11275 g020aSustainability 15 11275 g020b
Figure 21. Taylor diagram of different models for SR forecasting during the test phase for Dukan station.
Figure 21. Taylor diagram of different models for SR forecasting during the test phase for Dukan station.
Sustainability 15 11275 g021
Table 1. Basic statistical properties of parameters used in the study.
Table 1. Basic statistical properties of parameters used in the study.
Station Data SetUnitAverageMin.Max.St. Dev.Skew.
DarbandikhanTraining dataMAXTEMP°C28.7553.8048.8012.16−0.034
MINTEMP°C17.647−2.9032.809.066−0.013
MAXRH%60.3640.4096.1020.0390.036
MINRH%22.5694.089.3017.3591.054
SUNDURmin515.700924228.132−0.931
WINSPDm/s2.35807.300.9570.286
EVAPmm5.5270.1017.303.8660.441
CLOUDCOVokta1.929083.0451.217
SRW/m2191.634.5033585.78−0.353
Testing dataMAXTEMP°C23.52710.4044.5010.0410.394
MINTEMP°C15.4115.5026.606.5180.182
MAXRH%69.16728.10093.8021.201−0.406
MINRH%37.7534.9085.8023.750.140
SUNDURmin317.770643225.996−0.125
WINSPDm/s2.5671.304.700.7480.783
EVAPmm3.5110.1010.702.5570.619
CLOUDCOVokta3.899083.4120.033
SRW/m2106.243.20228.9060.890.127
DukanTraining dataMAXTEMP°C28.6133.049.012.454−0.060
MINTEMP°C16.458−3.033.09.016−0.020
MAXRH%60.3516.095.021.973−0.005
MINRH%24.8353.089.019.0660.913
SUNDURmin485.5280843264.571−0.652
WINSPDm/s2.8490.7010.501.402.144
EVAPmm6.609020.04.4150.394
CLOUDCOVokta2.10708.02.7661.10
SRW/m2153.7370294.9171.688−0.156
Testing dataMAXTEMP°C23.65110.043.010.1510.377
MINTEMP°C14.6604.027.06.7070.192
MAXRH%70.23826.095.023.588−0.421
MINRH%40.3395.085.024.823−0.021
SUNDURmin324.6050686243.31−0.027
WINSPDm/s3.1261.09.21.6541.286
EVAPmm4.2040.2012.203.2470.455
CLOUDCOVokta4.05508.03.2310.029
SRW/m288.3505.230193.751.970.259
Table 2. The performance evaluation of standalone models applied in forecasting of SR during the training and testing phases for Darbandikhan station.
Table 2. The performance evaluation of standalone models applied in forecasting of SR during the training and testing phases for Darbandikhan station.
MethodsTraining PhaseTesting Phase
RMSE
(W/m2)
NSRMAE
(W/m2)
RMSE
(W/m2)
NSRMAE
(W/m2)
DELM20.1580.9440.97113.97428.1860.7830.95422.754
ORELM23.0750.9270.96316.72630.6800.7430.95023.145
OPELM24.5370.9180.95817.57432.6240.7100.94126.263
OSELM23.2910.9260.96217.43733.5480.6930.93623.431
KELM36.0590.8230.90727.71337.1170.6250.91530.757
ELM22.4150.9310.96516.54639.8410.5680.88525.146
MARS41.2580.7680.87731.67047.2170.3930.88038.574
Table 3. The performance evaluation of hybrid models applied in forecasting of SR during the training and testing phases for Darbandikhan station.
Table 3. The performance evaluation of hybrid models applied in forecasting of SR during the training and testing phases for Darbandikhan station.
MethodsTraining PhaseTesting Phase
RMSE
(W/m2)
NSRMAE
(W/m2)
RMSE
(W/m2)
NSRMAE
(W/m2)
VMD-DELM23.0900.9270.96316.42524.4330.8370.96220.255
VMD-ORELM23.2890.9260.96217.27825.4290.8240.96121.093
VMD-OPELM23.7920.9230.96016.64128.2000.7810.96123.660
VMD-OSELM21.2430.9380.96814.96928.3570.7770.96024.277
VMD-KELM25.4170.9120.95518.70830.0400.7540.94823.345
VMD-ELM20.8790.9400.96914.93731.6020.7280.93925.821
Table 4. The performance evaluation of standalone models applied in forecasting of SR during the training and testing phases for Dukan station.
Table 4. The performance evaluation of standalone models applied in forecasting of SR during the training and testing phases for Dukan station.
MethodsTraining PhaseTesting Phase
RMSE
(W/m2)
NSRMAE
(W/m2)
RMSE
(W/m2)
NSR
MAE
(W/m2)
DELM26.6170.8610.92820.38223.9740.7850.95619.685
ORELM24.1720.8860.94118.35525.2510.7610.95020.838
OPELM26.3860.8640.93020.50825.3200.7600.94921.539
OSELM28.0600.8460.92322.34228.7660.6900.94823.893
KELM31.3970.8070.90926.32232.3220.6090.94529.243
ELM31.7660.8030.90626.65735.3950.5310.93931.669
MARS32.5510.7930.89826.56638.0830.4580.90130.796
Table 5. The performance evaluation of hybrid models applied in forecasting of SR during the training and testing phases for Dukan station.
Table 5. The performance evaluation of hybrid models applied in forecasting of SR during the training and testing phases for Dukan station.
MethodsTraining PhaseTesting Phase
RMSE
(W/m2)
NSRMAE
(W/m2)
RMSE
(W/m2)
NSRMAE
(W/m2)
VMD-DELM25.8650.8690.93219.64322.7210.8070.96318.392
VMD-ORELM25.2590.8750.93519.21823.2980.7970.95518.925
VMD-OPELM23.5340.8920.94417.30023.7150.7890.95319.058
VMD-OSELM27.6630.8500.92220.93024.5640.7740.95020.087
VMD-KELM28.8520.8370.91622.82826.7870.7310.94722.647
VMD-ELM26.5160.8630.92819.84429.1620.6820.90023.374
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Alizamir, M.; Othman Ahmed, K.; Shiri, J.; Fakheri Fard, A.; Kim, S.; Heddam, S.; Kisi, O. A New Insight for Daily Solar Radiation Prediction by Meteorological Data Using an Advanced Artificial Intelligence Algorithm: Deep Extreme Learning Machine Integrated with Variational Mode Decomposition Technique. Sustainability 2023, 15, 11275. https://doi.org/10.3390/su151411275

AMA Style

Alizamir M, Othman Ahmed K, Shiri J, Fakheri Fard A, Kim S, Heddam S, Kisi O. A New Insight for Daily Solar Radiation Prediction by Meteorological Data Using an Advanced Artificial Intelligence Algorithm: Deep Extreme Learning Machine Integrated with Variational Mode Decomposition Technique. Sustainability. 2023; 15(14):11275. https://doi.org/10.3390/su151411275

Chicago/Turabian Style

Alizamir, Meysam, Kaywan Othman Ahmed, Jalal Shiri, Ahmad Fakheri Fard, Sungwon Kim, Salim Heddam, and Ozgur Kisi. 2023. "A New Insight for Daily Solar Radiation Prediction by Meteorological Data Using an Advanced Artificial Intelligence Algorithm: Deep Extreme Learning Machine Integrated with Variational Mode Decomposition Technique" Sustainability 15, no. 14: 11275. https://doi.org/10.3390/su151411275

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop