Solar Irradiance Forecasting Using Dynamic Ensemble Selection

Abstract

Solar irradiance forecasting has been an essential topic in renewable energy generation. Forecasting is an important task because it can improve the planning and operation of photovoltaic systems, resulting in economic advantages. Traditionally, single models are employed in this task. However, issues regarding the selection of an inappropriate model, misspecification, or the presence of random fluctuations in the solar irradiance series can result in this approach underperforming. This paper proposes a heterogeneous ensemble dynamic selection model, named HetDS, to forecast solar irradiance. For each unseen test pattern, HetDS chooses the most suitable forecasting model based on a pool of seven well-known literature methods: ARIMA, support vector regression (SVR), multilayer perceptron neural network (MLP), extreme learning machine (ELM), deep belief network (DBN), random forest (RF), and gradient boosting (GB). The experimental evaluation was performed with four data sets of hourly solar irradiance measurements in Brazil. The proposed model attained an overall accuracy that is superior to the single models in terms of five well-known error metrics.

1. Introduction

Energy production plays a major role in modern societies, impacting the economy and the development of several countries [1]. Over the past years, energy production has primarily relied on the employment of fossil fuels, which are still the main energy production source [2]. However, fossil fuels are one of the primary emission sources of CO2, which also contributes to greenhouse effects and global warming [3]. Moreover, it is a non-renewable energy source, and it is being consumed more than it is produced.

Alternatively, solar energy is an important renewable energy source that produces on average 1.74 × 10¹⁷ W in 1 year, which is meets energy demands worldwide [4]. Solar energy is considered a promising alternative to fossil fuels. However, it has intermittent and volatile characteristics due to factors, such as precipitation, temperature, wind velocity, and atmospheric pressure [5,6]. These volatile characteristics can produce voltage fluctuations, possibly leading to instability in the power grid if they are not taken into account [7]. Moreover, the integration of traditional power systems with renewable power sources requires a precise balance between the demand and supply of power [6,8]. If the energy demand surpasses the energy supply, destabilization of the power grid can occur, resulting in power quality degradation or even grid damage.

Conversely, if the demand is lower than the supply, energy will eventually be lost, and wastage can occur because of the possible high costs associated with the storage of solar energy [7]. Considering the prediction of the demand and supply, the strategic section of energy plants decides whether energy is purchased. Prediction errors can lead to extra charges for the final consumer [9]. Therefore, accurate forecasting of solar irradiance plays a major role in power grid systems, estimation of reserves, scheduling, congestion management, use of produced energy in the energy market, and reduction of energy production costs [5,9,10].

Traditional statistical methods, such as the autoregressive integrated moving average (ARIMA) and seasonal ARIMA (SARIMA), are popular due to their simplicity and a well-defined Box and Jenkins methodology is used to select a suitable configuration of the model [11]. However, ARIMA models assume a linear correlation structure between forecasts and past data, which may underperform in the presence of nonlinear patterns. Considering that real-world data is often composed of a combination of linear and nonlinear patterns [12], and solar irradiance data present intermittent and volatile characteristics, the ARIMA models may perform worse than other methods in the literature [5,13].

In contrast to linear statistical models, forecasting methods based on machine learning are more suitable for mapping nonlinear patterns [13] in data. Machine learning methods, such as artificial neural networks (ANNs) and support vector machines (SVMs), are flexible data-driven models and have been used in the irradiation forecasting context [13]. However, machine learning-based models can also present problems related to model misspecification, overfitting, and underfitting [14,15,16], which leads to a poor generalization capacity [17].

To overcome the limitations of single and machine learning models, one possible strategy is to combine the forecasts of several models through the employment of ensembles to improve the system’s accuracy [18]. This strategy is promising because it reduces the risk of selecting an inappropriate model. In addition, it has the flexibility of the use of different combination strategies and different model generation methodologies [19]. Ensemble learning strategies are often composed of three stages: generation, pruning, and combination [19,20]. The ensembles can be classified according to the base learners being homogeneous or heterogeneous. Homogeneous ensembles are composed of the same learning algorithm while heterogeneous ones are often composed of different ones [19]. The main issue is that ensembles must be as accurate and diverse as possible, and the diversity control must be performed in the generation stage. In this sense, heterogeneous ensemble approaches are expected to produce more diverse models, considering the different learning algorithms of the base models [19,21].

In the context of solar irradiation forecasting, several ensemble learning approaches have employed a static ensemble of forecasters through time [13,22,23,24,25]; however, considering the volatile and dynamic characteristics of solar irradiance, static ensemble approaches can present inaccurate results since the best model could change over time. This dynamic characteristic is expected because solar irradiance may be affected by weather conditions [4]. Therefore, a dynamic model selection strategy [14] could improve the results over static ones.

In this work, an ensemble based on dynamic selection is proposed for solar irradiance forecasting. The proposed model, referred to as heterogeneous ensemble dynamic selection (HetDS), chooses the most appropriate set of models from a pool of forecasters comprising ARIMA, support vector regression (SVR), multilayer perceptron neural network (MLP), extreme learning machine (ELM), deep belief network (DBN), random forest (RF), and gradient boosting (GB). The best model in the pool is selected based on a local accuracy procedure, which defines a region of competence of size k. Experiments were conducted using the base models and static ensemble approaches, and the results demonstrate that the proposed method achieves the overall best results, considering the root mean square error (RMSE) and mean absolute percentage error (MAPE), mean absolute error (MAE), average relative variance (ARV), and index of agreement (IA), based on evidence provided by Friedman–Nemenyi [26] hypothesis testing. Moreover, the proposed model presents the following advantages:

• Reduces the risk of selecting an inappropriate model;
• Dynamically searches for the most suitable forecaster to predict a given local pattern in a solar irradiance series;
• It is an agnostic model since other forecasting models can be explored in the pool;
• Increases the generalization capacity of the system.

The rest of the paper is divided as follows: the related works are discussed in Section 2; Section 3 describes the models used to perform solar irradiance forecasting; Section 4 presents our proposal to solve the same problem; Section 5 shows the details of the database used, computational results, and a discussion; and finally, Section 6 presents the main conclusions and future works.

2. Related Works

Considering the volatility of irradiation data and the importance of obtaining accurate results in power grid systems, several models have been employed to improve the accuracy of forecasts. In general, forecasting models for solar irradiance are classified into physical models, empirical, statistical, and machine learning models [27]; however, hybridization between these classes of models is also possible and tends to improve the results when compared with single model approaches [28,29].

Physical models, such as numerical weather prediction (NWP), are complex structures that consider several aspects of the environment and the irradiance data [30]. In contrast, empirical models are less complex and often employ linear and nonlinear regressions to perform estimates [31]. However, empirical models may present limited accuracy.

Statistical models, such as the autoregressive integrated and moving average model

(ARIMA) perform temporal mappings over past data and produce forecasts, and have been used in several irradiance forecasting applications. Shadab et al. [32] employed a multiplicative seasonal ARIMA model to forecast monthly average isolation data considering different sky conditions. Voyant et al. [33] employed an ARIMA to predict global irradiation. In this work, the model is used in conjunction with MLP networks, and the proposed approach is compared with single models.

Machine learning models are not only capable of performing nonlinear mappings in the data but also present some flexibility regarding noisy data and tend to achieve an improved performance since real-world data often composed of nonlinear patterns. The employment of SVM models in irradiation tasks has achieved promising results. Chen and Li [34] used the SVM models, considering exogenous variables, such as sunshine ratio, maximum and minimum air temperature, relative humidity, and atmospheric water vapor pressure. Moreover, Bendiek et al. [35] employed an optimization method for several models, such as SVM and MLP, and then performed experimental comparisons. The results indicated that SVM with the proposed optimization achieved the best results.

Random forests (RFs) are part of the ensemble class of models, where several regression trees are employed and combined to achieve the final result. This approach is interesting to solar radiation forecasting since the data may present high volatility, and it mitigates the chance of selecting an inappropriate model. Srivastava et al. performed several comparisons among machine learning models, such as classification and regression trees (CART) and random forests, using data from India, and the RF model obtained the best results. Huang et al. [36] compared several methods using exogenous data and concluded that the RF method achieved the smallest errors. Another algorithm employed in irradiation forecasting tasks is gradient boosting (GB), which is based on a tree-based ensemble learning technique. Park et al. [37] employed LightGBM to perform multi-step ahead forecasting in irradiation data. Moreover, Fan et al. [38] used an extreme gradient boosting algorithm and SVM to predict daily global solar radiation using temperature and precipitation in humid subtropical climates.

The work of Elminir et al. [39] was one of the pioneering works that applied neural networks when considering solar irradiance components. In this investigation, the authors predicted infrared, ultraviolet, and global insolation using an MLP trained by the Levenberg–Marquardt algorithm. The database addressed was from Egypt. They also applied the trained network considering a database from Helwan and Aswan. In both cases, the accuracy was up to 90%.

MLP was also applied to generate synthetic daily solar radiation series using five exogenous variables as inputs: daily clear sky global radiation, cloud cover, temperature, water vapor, and ozone. They proposed that the model can generate good predictions for locations in which there are no ground measurements. The results proved the generalization capability of MLPs.

The paper from Salcedo-Sanz et al. [40] considers the outputs of the WRF meso-scale model as inputs of an ELM to predict solar irradiation. They proposed the application of the coral reefs optimization algorithm with species (CRO-SP) as a feature selection procedure to minimize the number of input variables. The same author proposed the use of coral reefs optimization (CRO) hybridized with an ELM to predict solar irradiation [41]. A binary-encoded CRO is used as a variable selection model. A comparison was performed considering the performances of the genetic algorithm replacing the CRO, and the SVR, MARS, and MLR as the predictor. The results revealed the viability of the proposed method.

The deep belief network was applied to perform the monthly solar forecasting task considering Moderate Resolution Imaging Spectroradiometer (MODIS) satellite data in the work of Ghimire et al. [42]. The authors also evaluated the use of 15 feature selection algorithms among the filters, wrappers, and bio-inspired approaches. The computational results revealed that the model can overcome MLP, decision trees, RF, and GB. A similar architecture (functional DBN) was proposed together with embedding clustering for daily global solar radiation forecasting by Zang et al. [43]. The model could overcome the stand-alone DBN, functional DBN, and other ML-based models.

3. Background

This section describes the models used to perform solar irradiance forecasting.

3.1. Autoregressive and Moving Average Model

The autoregressive and moving average models (ARMA) belongs to the Box and Jenkins family of linear models [11]. While the AR part considers the lags of the series, the MA creates the output response addressing random shocks $a_{t-P-j}$ [44], which are weighted by ${\mathsf{\theta }}_j$ coefficients, as in Equation (1):

$$\widehat{x_t}={\varphi }_1{x}_{t-P}+\dots +{\varphi }_p{x}_{t-P-p+1}-{\theta }_1{a}_{t-P}-\dots -{\theta }_q{a}_{t-P-q+1}+a_t,$$

(1)

where ${\varphi }_i$ and ${\theta }_j$ are the autorregressive and moving average coefficients, respectively; P is the number of steps ahead considering the direct approach; and $i=1, 2,\dots ,p$ and $j=1, 2, \dots , q$, are free coefficients [44].

The most usual way to apply the model considers the random shocks a_t as equivalent to the residual set of the previous samples [45].

Finally, to adjust the free coefficients of the ARMA model, we used the maximum likelihood estimator (MLE), which is the usual way [11]. Note that this model can also be named the autoregressive and moving average model (ARMA) when no differentiations are performed in the time series [11].

3.2. Random Forest

The random forest (RF) algorithm was introduced by Breiman in 2001 [46]. It is a learning method used for classification and regression that operates by building multiple decision trees during the training and production of the final prediction.

The name came from random decision forests that were first proposed by Ho [47]. The method combines Breiman’s idea of bagging and the random selection of features, independently introduced by Ho [47] and Amit and Geman [48], to build a collection of decision trees with a controlled variation.

The trees used in RF are based on binary partitioning trees. These trees partition the predictor space using a sequence of binary divisions on individual variables. The root node of a tree comprises the entire predictor space. The nodes that are not split are called terminals and form the final partition of this space. Therefore, each non-terminal node is divided into two descending nodes, according to the value of the predictor variables. A division is determined by a division point or a continuous predictor variable. When the points for the predictor are less than the split point, the decision is made to go to the left, and the remainder goes to the right [49].

3.3. Gradient Boosting

Decision trees have several important characteristics for forecasting tasks. However, decision trees’ individual predictive power can be enhanced by forming a prediction committee, which is the central idea of boosting techniques.

Boosting is a generic strategy to improve the performance of any learning algorithm. The method was proposed to deal with pattern classification problems, with the introduction of the AdaBoost [50]. Subsequently, several generalizations emerged from the original strategy, among them, the gradient boosting algorithm [51], which can be applied to both classification and regression problems, using any differentiable objective function.

The so-called boosting methods are based on the principle of minimizing a cost function through the aggregation of multiple weak models (weak learners) to build a more robust model using the gradient method as a procedure, which comprises a systematic strategy for the construction of forecast committees [51].

In a boosting method, the performance evaluation of the models is performed sequentially. The criteria in the next step can be changed according to the evaluation in the previous step: if the first evaluator indicates a low score for a criterion, this score will be considered by the next evaluator. Thus, the final assessment is more accurate, and the process is more dynamic [52].

In the gradient boosting method, errors are minimized by the gradient descent algorithm. The proposal avoids local minimum points so that the second-best criterion receives the best rating by adjusting the criteria to capture the best performance [53].

3.4. Support Vector Regression

Support vector regression (SVR) [54] is based on the structural risk minimization, characterized by a convex optimization problem, generating a single global minimum. The objective is to find a function in the form of 2:

$$\left\{f\right|f\left(\mathrm{x}\right)=w^{T}x+b, w \in {\Re }^d, b \in \Re \}$$

(2)

where w is a weight vector with components determined by the regularized risk function, x is the input vector, and b is the bias.

In regression problems, the model provides continuous outputs to the training data, considering a maximum deviation of ε of the expected value [55]. However, the cost function is not computed for values inside the region limited by ε. In addition, the use of kernel functions makes it possible to perform nonlinear mappings in the data to a feature space in which a linear function is found.

3.5. Multilayer Perceptron

Multilayer perceptron (MLP) is a famous architecture of neural networks. It is a feedforward approach since the information present in the inputs flows in just one direction, from the input layer to the output layer. The MLP is a universal function approximator because it can map any continuous, nonlinear, limited, and differentiable function with a predetermined error threshold.

The most usual way to train the weights of MLP is the backpropagation algorithm [56], which uses the information of the gradient function to modify the values of the weights in order to reduce the output error. In this work, we address an MLP with three layers (input, hidden, and output layers), and a sigmoid function is employed as an activation function [57].

3.6. Extreme Learning Machines

The extreme learning machines (ELMs) are feedforward neural networks but contain just a single hidden layer [58]. The main difference between them lies in the training process. While the MLP uses an iterative process to adjust the weights of the network, the ELM just modifies the weights of the hidden layer.

To perform the training, a linear regression problem is considered. The proposers of the architecture, Huang et al. [59], used the Moore–Penrose generalized inverse operation since it simultaneously minimizes the norm of the output weight vector and the mean square error of the output response [60]. The authors also proved that the insertion of a new randomly generated neuron in the hidden layer led to a decrease in the output error.

3.7. Deep Belief Network

The deep belief network (DBN) is a deep learning model that was introduced by Hinton et al. in 2006 [61]. It is a probabilistic generative graphical model created to overcome the limitations of traditional models, such as slow convergence and convergence to a local minimum, among others.

A DBN is composed of restricted Boltzmann machines (RBMs), in which each one presents two layers, visible and hidden. In this case, each hidden layer acts as a visible layer to the next RBM. The first two layers present indirect connections that create an associative memory. On the other hand, the lower layers present direct connections [62].

The training process follows a greedy approach so that each layer is adjusted until the global optimum is found, starting from the visible layer of the RBM. Then, the hidden layer is tuned, and its outputs are used as inputs to the next RBM layer until the end of the network [63].

4. Proposed Method

The proposed model comprises two stages: (i) model generation and (ii) model selection. The model generation stage performs the training of all forecasting methods (ARIMA, GB, RF, SVR, DBN, ELM, MLP) to produce a pool of models. The model generation stage is crucial to the ensemble, considering that the models should be as accurate and diverse as possible. The accuracy is achieved through a search methodology over the space of the possible parameter configurations of all models, and diversity is achieved because different models with distinct learning algorithms are used in the pool. The model generation architecture is presented in Figure 1.

The dynamic model selection stage occurs as new test instances arrive at the proposed model, HetDS(m,k) (source code available at: https://github.com/domingos108/solar_forecasting, accessed on 5 March 2022), standing for a heterogeneous ensemble dynamic selection. Based on the local performance, the proposed method selects the best set models with a size m from the pool for each new test instance. It defines a region of competence of a size k, composed of the k nearest instances in the validation set. Therefore, all models in the pool are evaluated in this region employing the RMSE metric, and the m models with the lowest RMSE are selected to perform the forecast of the test instance.

The Euclidean distance metric is employed to determine the k nearest instances on the validation set. Therefore, different models can be selected for each test instance to improve the forecast accuracy. The architecture of the dynamic model selection is presented in Figure 2. The configuration of the number of models $m$ used in the dynamic ensemble and the number of nearest instances k plays a major role in the performance of the proposed system.

Lower and higher values of m increase the chance of selecting inappropriate model(s) since there is no guarantee that the best model(s) in the region of competence (validation set) will also demonstrate an improved generalization capacity on the test set. In the first scenario, when the value of m is near 1, the selected model may not achieve an improved performance on the test set, whereas in the second situation, when the value of near the size of the pool, it increases the chance of selecting inaccurate models for the ensemble combination.

Moreover, solar irradiation data presents dynamic characteristics due to exogenous variables. In addition, the value of k defines the size of the region of competence, which should represent the dynamics of the system. The models are combined through a median operator [64] since it is more robust to outliers. It is essential to mention that if the value of m = 1, then no combination is performed.

5. Experiments

The experiments were conducted on four datasets of solar plants from four cities in Brazil. All data correspond to hourly measures of solar irradiance, considering sunlight times, which is generally between 6:00 and 18:00 in Brazil. The data was split into three disjoint sets to perform training and model selection, and the last set was used for testing. To select the model parameters, a grid search procedure was conducted in the parameter space defined in

Table 1. Parameter configuration of the models and algorithms used.

Model	Parameters	Option
ARIMA	p,d,q	Hyndman Method [65]
MLP	Algorithm	Backpropagation
	Activation Function	Sigmoid
	Number of Hidden Layer Nodes	20, 50, 100
ELM	Algorithm	Moore-Penrose pseudo-inverse
	Activation Function	Hyperbolic Tangent
	Number of Hidden Layer Nodes	20, 50, 100, 200, 500
SVR	Kernel	RBF
	γ	0.1, 0.01, 0.001
	C	10, 100, 1000
	ε	0.1, 0.01, 0.001
GB	Number of estimators	50, 100, 200
	Max depth	5, 10, 15
	Max features	0.6, 0.8, 1
	Sub sample	0.6, 0.8, 1
	Learning rate	0.1, 0.3, 0.5
RF	Number of estimators	50, 100, 200
	Max depth	5, 10, 15
	Max features	0.6, 0.8, 1
DBN	Number of Hidden Layer Node	100, 200
	Learning rate RBM	0.01, 0.001
	Learning rate	0.01, 0.001

. Regarding the ARIMA, p and q are the AR order and the MA order, respectively. Concerning the SVR, γ is the kernel parameter, C is the regularization factor, and ε is the maximum deviation of the observed value.

The ARIMA model employed a stepwise methodology to select an appropriate model [65]. To perform the predictions, we used 4758 hourly samples of the solar irradiation series so that 60% were used for training (the first 2856 samples), 20% for validation (951 samples), and 20% for testing (the last 951 samples). Moreover, the data sets used in the experiments were scaled to the range [0.1, 0.9] [66].

Several configurations of the proposed model were tested, using the m best models varying in the set 1, 3, 5 and the size of the region of competence varying in the set 5, 10, 20. Moreover, static ensemble strategies were also tested using mean and median combination operators using the same models from the pool.

5.1. Data Description

The databases are related to four major cities in Brazil: Florianópolis, Fortaleza, Salvador, and São Paulo.

Florianópolis is the capital of Santa Catarina state, located in the south of the country. It is a coastal town. Fortaleza is the capital of Ceará state, in the Brazilian northeast. It is also a coastal town relatively close to the equator line. Salvador is the capital of Bahia state, located in the Brazilian northeast region. São Paulo city is the major city in Brazil. It is almost on the Tropic of Capricorn. The red marks in Figure 3 represent the locations of these cities on a map of Brazil. Considering the cyclic behavior of solar irradiation, where maximum values are often achieved at 12:00 and minimum values at 6:00 and 18:00, all inputs were analyzed in a window size of 12. Moreover, this value can also be achieved through an autocorrelation plot.

As can be seen, the longest distance is between Fortaleza and Florianópolis, almost 3400 km. Due to this, it is clear that the sunlight may present different magnitudes along the year considering all cities. All databases are composed of 4758 hourly samples of solar irradiation (kJ/m²). We considered the measures from 1 January 2020 to 31 December 2020, the whole year. However, we selected just the samples from 6:00 to 18:00 during each day.

Table 2. Location of the solar irradiation stations and descriptive statistics regarding the mean and deviation of solar irradiation.

Station	Coordinates	Altitude	Mean	STD	CV
Florianopolis	−27.0253; −48.620096	4.87	1230.7	1101.2	0.895
Fortaleza	−3.815701; −38.537792	29.55	1223.9	883.18	0.722
Salvador	−13.551500; −8.505760	47.56	1348.2	1122.6	0.833
São Paulo	−23.496294; −6.620088	785.64	1355.4	1204.6	0.889

presents the geographical location of the solar irradiation stations and the descriptive statistics of the corresponding series. Note the difference in the altitude of São Paulo among the others, the only city that is not a coastal town. The acronyms STD and CV represent the standard deviation and coefficient of variation, respectively.

5.2. Evaluation Metrics

In this work, we consider the following metrics to perform a comparative analysis among the forecasting models presented, the root mean squared error (RMSE), the mean absolute percentage error (MAPE), the mean absolute error (MAE) the average relative variance (ARV), and the index of agreement (IA) [67,68], which are given by Equations (3)–(7), respectively:

$$\mathrm{RMSE}=\frac{1}{N}{\sum }_{t=1}^N\left(\sqrt{{\left({\widehat{x}}_t-x_t\right)}^2}\right),$$

(3)

$$\mathrm{MAPE} =\frac{100}{N}{\sum }_{t=1}^N\left|\frac{\widehat{x}-x_t}{x_t}\right|,$$

(4)

$$\mathrm{MAE} =\frac{1}{N}{\displaystyle \sum }_{t=1}^N\left|{\widehat{x}}_t-x_t\right|,$$

(5)

$$\mathrm{ARV} =\frac{{\sum }_{t=1}^N{\left({\widehat{x}}_t-x_t\right)}^2}{{\sum }_{t=1}^N{\left( {\widehat{x}}_t-\overline{x}\right)}^2},$$

(6)

$$\mathrm{IA} =1-\frac{{\sum }_{t=1}^N{\left( {\widehat{x}}_t-x_t\right)}^2}{{\sum }_{t=1}^N{\left( \left|{\widehat{x}}_t-\overline{x}\right|-\left|x_t-\overline{x}\right|\right)}^2},$$

(7)

where N is the number of samples, $x_t$ is the observed value in time t, $x_t$ is the predicted value, and $\overline{x}$ is the mean of the series. The percentage difference (PD), given by Equation (8), is calculated to compare the performance of HetDS with the other approaches:

$$PD=\frac{Metri{c}_{model}-Metri{c}_{HetDS}}{Metri{c}_{model}} ·100,$$

(8)

where Metric_HetDS and Metric_model are the RMSE values attained by HetDS and the comparative models used in the experimental evaluation, respectively. The higher the PD value, the better the RMSE value obtained by HetDS with respect to the model under comparison.

5.3. Results

The performance of the forecasting models was assessed based on the test set (the last 951 samples) of each series. For the sake of simplicity, the proposed method is henceforth referred to as HetDS(m,k), standing for a heterogeneous ensemble dynamic selection with parameters m (best-selected models) and k (size of the region of competence).

Table 3. Performance of HetDS(m,k) and literature models in terms of RMSE, MAPE, MAE, ARV and IA for all data sets of solar irradiance. The best values attained in each data set are highlighted in bold.

Series	MODEL	RMSE	MAPE	MAE	ARV	IA
Fortaleza	ARIMA [32,33]	0.0824	21.21	0.0639	0.1181	0.9705
	RF [69,70]	0.0742	13.06	0.0563	0.1170	0.9739
	GB [37,38]	0.0746	13.63	0.0558	0.1155	0.9739
	SVR [34,71]	0.0629	12.87	0.0457	0.0695	0.9830
	MLP [39,72]	0.0676	14.65	0.0516	0.0853	0.9797
	ELM [40,41]	0.0718	12.95	0.0528	0.1037	0.9762
	DBN [42,43]	0.0660	11.89	0.0481	0.0839	0.9803
	HetDS(1,5)	0.0626	11.07	0.0447	0.0689	0.9831
	HetDS(1,10)	0.0617	10.73	0.0436	0.0672	0.9836
	HetDS(1,20)	0.0612	10.67	0.0434	0.0660	0.9838
	HetDS(3,5)	0.0607	10.77	0.0435	0.0665	0.9839
	HetDS(3,10)	0.0605	10.61	0.0429	0.0659	0.9841
	HetDS(3,20)	0.0600	10.57	0.0426	0.0646	0.9844
	HetDS(5,5)	0.0618	11.31	0.0448	0.0715	0.9830
	HetDS(5,10)	0.0616	11.23	0.0445	0.0710	0.9832
	HetDS(5,20)	0.0616	11.22	0.0445	0.0708	0.9832
	Hetmean	0.0644	12.80	0.0479	0.0807	0.9812
	Hetmedian	0.0647	12.30	0.0477	0.0807	0.9811
Florianópolis	ARIMA [32,33]	0.1024	24.74	0.0746	0.2135	0.9478
	RF [69,70]	0.0938	21.28	0.0671	0.1944	0.9547
	GB [37,38]	0.0942	20.36	0.0663	0.1876	0.9553
	SVR [34,71]	0.0962	20.43	0.0655	0.1754	0.9558
	MLP [39,72]	0.0956	20.76	0.0667	0.1850	0.9548
	ELM [40,41]	0.1002	22.44	0.0713	0.2067	0.9499
	DBN [42,43]	0.0962	22.00	0.0693	0.1970	0.9531
	HetDS(1,5)	0.0955	20.68	0.0665	0.1840	0.9550
	HetDS(1,10)	0.0955	20.82	0.0670	0.1839	0.9550
	HetDS(1,20)	0.0938	20.52	0.0655	0.1791	0.9565
	HetDS(3,5)	0.0937	20.03	0.0648	0.1800	0.9564
	HetDS(3,10)	0.0934	19.87	0.0645	0.1788	0.9567
	HetDS(3,20)	0.0931	19.87	0.0643	0.1780	0.9570
	HetDS(5,5)	0.0934	19.97	0.0646	0.1802	0.9566
	HetDS(5,10)	0.0933	19.97	0.0645	0.1801	0.9566
	HetDS(5,20)	0.0933	19.95	0.0646	0.1805	0.9566
	Hetmean	0.0933	20.28	0.0648	0.1819	0.9564
	Hetmedian	0.0936	20.30	0.0650	0.1820	0.9563
Salvador	ARIMA [32,33]	0.1051	27.15	0.0788	0.2739	0.9398
	RF [69,70]	0.0954	19.48	0.0657	0.2102	0.9524
	GB [37,38]	0.0973	20.18	0.0668	0.2190	0.9504
	SVR [34,71]	0.0902	17.12	0.0579	0.1581	0.9610
	MLP [39,72]	0.0900	17.95	0.0601	0.1678	0.9599
	ELM [40,41]	0.0972	20.48	0.0666	0.1866	0.9541
	DBN [42,43]	0.0918	19.38	0.0632	0.1822	0.9573
	HetDS(1,5)	0.0935	18.28	0.0619	0.1804	0.9568
	HetDS(1,10)	0.0945	18.37	0.0626	0.1855	0.9557
	HetDS(1,20)	0.0943	18.03	0.0619	0.1842	0.9559
	HetDS(3,5)	0.0907	17.60	0.0599	0.1743	0.9589
	HetDS(3,10)	0.0903	17.55	0.0597	0.1730	0.9592
	HetDS(3,20)	0.0902	17.30	0.0592	0.1714	0.9594
	HetDS(5,5)	0.0896	17.74	0.0596	0.1720	0.9597
	HetDS(5,10)	0.0894	17.64	0.0593	0.1705	0.9600
	HetDS(5,20)	0.0895	17.64	0.0593	0.1706	0.9599
	Hetmean	0.0903	18.53	0.0609	0.1792	0.9585
	Hetmedian	0.0898	18.05	0.0602	0.1740	0.9593
São Paulo	ARIMA [32,33]	0.2138	61.59	0.1456	1.2968	0.7355
	RF [69,70]	0.1746	55.63	0.1173	0.9224	0.8215
	GB [37,38]	0.1772	53.20	0.1161	0.8342	0.8260
	SVR [34,71]	0.1942	55.88	0.1211	0.8148	0.8083
	MLP [39,72]	0.1849	58.32	0.1257	0.8780	0.8115
	ELM [40,41]	0.1927	59.09	0.1304	0.9796	0.7919
	DBN [42,43]	0.1892	57.10	0.1263	1.0258	0.7917
	HetDS(1,5)	0.1864	54.57	0.1194	0.8251	0.8163
	HetDS(1,10)	0.1808	52.70	0.1156	0.7878	0.8267
	HetDS(1,20)	0.1797	52.28	0.1154	0.8031	0.8262
	HetDS(3,5)	0.1767	52.40	0.1155	0.8335	0.8261
	HetDS(3,10)	0.1726	51.90	0.1137	0.8118	0.8327
	HetDS(3,20)	0.1720	51.51	0.1132	0.8144	0.8332
	HetDS(5,5)	0.1777	53.80	0.1176	0.8814	0.8201
	HetDS(5,10)	0.1764	53.84	0.1171	0.8748	0.8224
	HetDS(5,20)	0.1762	53.85	0.1169	0.8730	0.8230
	Hetmean	0.1802	55.30	0.1205	0.9473	0.8113
	Hetmedian	0.1832	55.11	0.1208	0.9539	0.8070

presents the results achieved by the models described in Section 2, ARIMA, DBN, ELM, GB, MLP, RF, and SVR, and the variations of the proposed models. The values presented are the average of 10 independent simulations, and the best results are highlighted in bold.

In addition,

Table 4. Percentage difference (%) regarding RMSE between single and ensemble models with the best proposed version of HetDS.

MODEL		SERIES
	Fortaleza	Florianópolis	Salvador	São Paulo
ARIMA [32,33]	14.97	27.28	19.55	9.11
RF [69,70]	6.38	19.16	1.5	0.72
GB [37,38]	8.21	19.67	2.96	1.18
SVR [34,71]	0.89	4.67	11.45	3.18
MLP [39,72]	0.75	11.29	6.97	2.62
ELM [40,41]	8.11	16.55	10.74	7.03
DBN [42,43]	2.72	9.16	9.08	3.23
Hetmean	4.43	4.23	7.76	2.46
Hetmedian	5.43	2.88	4.86	2.47

presents the percentage difference (PD) regarding RMSE between single and ensemble models with the best proposed version of HetDS. It is possible to note that HetDS obtained a superior performance in terms of RMSE when compared to the other statistical and ML models used.

Figure 4, Figure 5, Figure 6 and Figure 7 show the dispersion of the results considering the 10 independent simulations by means of a boxplot graphic.

5.4. Discussion

Due to the number of computational results presented in Table 3, it is possible to discuss many aspects regarding the performance of the models. One can observe that the best RMSE value does not necessarily correspond to the best values for the other error metrics for a given method [73]. Previous works on time series forecasting have indicated such a behavior, which can occur when using distinct error metrics [56,74].

In two scenarios, a single model achieved the smallest error. Precisely, the smallest MAPE for Salvador, and the smallest MAPE, MAE, ARV, and IA for Florianópolis were achieved by SVR. In all other cases, the proposed model stood out. In terms of RMSE, Table 4 reveals the proposed approach overcame the single propositions, and Het_mean and Het_median.

After the application of the Friedman–Nemenyi [26] statistical test using 95% confidence, it is observed that the proposed method (HetDS(m,k)) achieved the best overall results in almost all comparisons, outperforming the single methods considering the f4our datasets, as highlighted in Figure 8.

The analysis of the boxplot graphics in Figure 4, Figure 5, Figure 6 and Figure 7 leads to some interesting remarks. As expected, the ARMA and SVR do not present dispersion since their training processes are performed using closed-form solutions. In general, the proposed approach achieved the best solutions and relatively small dispersion. Moreover, it did not find the best general solution for Salvador.

It is clear that the linear ARIMA model did not overcome any machine learning approach. One possibility of enhancing its prediction capability is to use bio-inspired metaheuristics [75]. Regarding the other single models, we observe high variability in terms of the performance or dispersion. It indicates how difficult it is to solve solar irradiation forecasting problems.

Finally, Figure 9, Figure 10, Figure 11 and Figure 12 present the predictions provided by the best models in comparison with the original series in the test set. It is important to observe the performance on the peaks of the solar irradiance, where most of the deviation occurs.

6. Conclusions

At present, solar energy has a significant role due to the current concerns about green energy around the world. Thus, to improve the quality of energy generation processes, accurate forecasting systems are desirable. Solar irradiation data can present dynamic characteristics due to weather conditions. Therefore, single models (linear or nonlinear) may not achieve the best results. In this work, a heterogeneous dynamic ensemble (HetDS) approach was used to perform solar irradiation forecasting. Static ensemble approaches employ a group of models to predict future values. However, since the characteristics of the data may change over time, a dynamic approach could improve the results. In this sense, a dynamic ensemble composed of a pool of heterogeneous models was selected for each test pattern.

As stand-alone approaches, we considered the linear ARIMA from the Box and Jenkins methodology, and a pool of machine learning models: DBN, ELM, GB, MLP, RF, and SVR. The databases considered were from four major Brazilian cities. The experimental evaluation showed that the proposal can overcome the single models in almost all scenarios with a small dispersion, revealing its high approximation capability. The dynamic selection ensemble of the forecasting models increased the accuracy of the system, showing its competitiveness.

Regarding future work, we highlight the possibility of using recurrent neural networks and deep learning approaches. In addition, deeper analysis of the selection of the best lags can be considered, and the use of exogenous variables, such as precipitation, pressure, and temperature, or synthetic data, aiming to attain more accurate forecasts.