Time Series Forecasting of Thermal Systems Dispatch in Legal Amazon Using Machine Learning

Abstract

This paper analyzes time series forecasting methods applied to thermal systems in Brazil, specifically focusing on diesel consumption as a key determinant. Recognizing the critical role of thermal systems in ensuring energy stability, especially during low rain seasons, this study employs bagged, boosted, and stacked ensemble learning methods for time series forecasting focusing on exploring consumption patterns and trends. By leveraging historical data, the research aims to predict future diesel consumption within Brazil’s thermal energy sector. Based on the bagged ensemble learning approach a mean absolute percentage error of 0.089% and a coefficient of determination of 0.9752 were achieved (average considering 50 experiments), showing it to be a promising model for the short-time forecasting of thermal dispatch for the electric power generation system. The bagged model results were better than for boosted and stacked ensemble learning methods, long short-term memory networks, and adaptive neuro-fuzzy inference systems. Since the thermal dispatch in Brazil is closely related to energy prices, the predictions presented here are an interesting way of planning and decision-making for energy power systems.

1. Introduction

Electricity generation and its costs in an isolated system off-grid are mostly related to fuel engines with diesel or natural gas consumption [1], mainly because these energy types are easily transported and they are non-intermittent energy sources in contrast to solar or wind power plants which can be integrated [2]. It is challenging to achieve accurate generation prediction of these cited renewable sources [3], while fossil fuel has significant disadvantages as it generates pollutant emissions and there is a need to refine the oil that is imported in part from foreign countries in the thermal plants of the Brazilian Amazon Region [4].

The interdependence between the nexus of greenhouse emissions, economic growth, financial development, and load demand-supply in the past with the future is important in managing policy recommendations and/or establishing strategies for the corporate sector to adopt business operations with better support technologies to improve profit with ecological efficiency [5]. Determining scenarios that could improve the reduction in carbon dioxide or coordinate carbon capture systems to reach targets considering uncertainties and promoting the best possible adaptability involves following the convergence results of these four parameters [6].

The extraction, conversion, and distribution of crude oil as diesel form part of a logistic value energy chain that is complex, and the implementation necessary for electricity generation in the power grid for a domestic load to supply isolated regions from this resource brings higher costs for the electrical system in a short time period [7]. In trying to avoid these disadvantages, the integration of renewable energy sources by applying an optimal dispatch algorithm can be evaluated and assist responsible companies in controlling greenhouse emissions from fossil fuels as well as reducing import dependence. To avoid these disadvantages, modeling the load capacity forecast is essential to plan the energy requirements of the analyzed region and the grid modifications needed [8].

Economic dispatch is the process of determining the optimal combination of power output from different generation units to meet the electrical load demand at the lowest cost while satisfying various system and operational constraints [9]. The key difference between the low-carbon form of economic dispatch and the common (or traditional) form lies in prioritizing environmental concerns, particularly carbon emissions, in the decision-making process [10]. While the traditional economic dispatch aims at cost minimization based on fuel and operational costs, the low-carbon form integrates environmental considerations, specifically targeting a reduction in carbon emissions, thus aligning the dispatch process with broader environmental and sustainability goals [11].

Low-carbon economic dispatch depends on energy conversion appliances which affect the exchange power and thermal load, storage, and carbon trading and require to be integrated with the demand response and scenarios of the predicted future load [12]. These parameters can help to determine losses and increase investment mechanisms to reduce costs and increase the profit-aggregating influence factors for carbon intensity, benefiting from the scale of technologies that could contribute to reaching lower emission targets in future scenarios [13].

A major concern about sustainability in the Amazon region is related to the use of diesel oil as the main source of energy. Figure 1 shows the used diesel oil for electricity generation in the legal Amazon by region states in 2021. According to this information, the further away from the industrialized center, the more fossil fuel consumption is observed. This is concerning since these regions are where there are the largest areas of preserved vegetation and the presence of large thermoelectric plants increases the level of pollution in the area, thus affecting the local ecosystem [14].

Traditional diesel engine power plants, compared to those based on new concepts, such as gasification with the Rankine cycle from biomass forest, are still cheaper to implement [15]. The new Rankine-gasification power plants have 1.2 times higher cost for implementation than diesel thermoelectric plants currently used despite heat wasted through exhaust gases in the general diesel operation model [16].

However, the annual operation and maintenance costs using diesel have doubled in price compared with biomass use based on research performed in an isolated village aiming for decarbonization. When natural gas pipelines are implemented, this could reduce costs by forty percent and carbon dioxide emissions by twenty-five percent in Amazonia isolated systems [17].

Fossil fuel load forecasting is important for the adaptability and stability of regions that have high dependence as evaluated by Li et al. [18], who applied diversified ensemble learning methods to reduce delayed dispatch and provide strategically scheduled storage for natural gas.

These methods can enhance hybrid generation using fossil fuels and renewable resources by integrating photovoltaic and diesel engines to reduce expenditure. The algorithms of Kanaan et al. [19] utilize improved strategies for storage, decreasing the daily carbon emission costs of a power plant by 29%, and have the potential for application in isolated regions of the legal Amazon by avoiding battery degradation.

Evaluating the consumption management and generation and adoption of technologies that relocate electrical load is a solution to increase energy sustainability [20], Within this context, forecasting time series using ensemble learning approaches is an alternative to improve the reliability of the electrical system to achieve a better environmental-economic coefficient [21]. Ensemble learning is the fusion of several predictive models to enhance collective performance [22]. It starts by assembling a varied array of foundational forecasting models, like decision trees, support vector machines, support vector regressors, and others. Each of these base models offers distinct perspectives on the data, which the ensemble method consolidates [23]. Specifically, for generation forecasting, ensemble learning leverages diverse algorithmic strengths to attain heightened accuracy and resilience in predictions [24].

There is a trend towards using deep learning models, given their ability to comprehend non-linear patterns [25]. However, these models can require more computational effort and are not always the best alternative. Considering the promising results of using ensemble models, this paper aims to present a comparison between ensemble learning methods and models based on deep learning, such as the long short-term memory (LSTM) model.

Considering that the use of thermal energy results in higher energy costs for consumers [26], especially when there is a shortage of raw materials, forecasting its use can help with energy planning. Based on this premise, the main theme of this paper is the forecasting of thermal systems dispatch. The analysis focuses on the legal Amazon since new energy sources are being explored and implementation of several insulated systems in this region and their more efficient use can improve energy supply capacity.

Given the major importance of thermal systems generation in northern Brazil, this paper proposes a thorough evaluation of ensemble learning methods for dispatch forecast for thermal systems in this region. The main contributions of this paper are:

• The time series forecasting of thermal dispatch is an indicator that can be used for evaluating energy prices, which is an important concern in insulated thermal systems since the prices are related.
• The bagging ensemble learning method proved to be an appropriate structure, having a lower computational demand compared to the LSTM network based on deep learning. The bagging approach proved to be a better approach than the boosting and stacking ensemble approaches.
• The ensemble learning approaches were shown to be stable having few differences when several experiments were performed considering random initialization.

The remainder of this paper is organized as follows: In Section 2, related works regarding time series forecasting applications are discussed. Section 3 explains the proposed method. In Section 4, the results of the application of the proposed method are analyzed, and finally, in Section 5, a conclusion is drawn, and directions for future work are suggested.

2. Related Work

The use of artificial intelligence to analyze electrical power systems is increasing due to the extent to which these models have to deal with non-linearities [27]. Nowadays, deep learning has been widely applied in several fields [28,29,30,31]. Regarding power systems [32], its application can focus on the identification of faults in the power grid based on classification [33] or the prediction of its condition based on time series analysis [34]. Given the advantages of having earlier knowledge about a system’s condition, time series forecasting of energy is a way to improve decision-making in electrical power management [35].

In the work by Pazikadin et al. [36], a review is presented about solar irradiance measurement instrumentation and power generation forecasting over the past five years, specifically focusing on the integration of artificial neural networks (ANNs). They discuss the pivotal role of ANNs in enhancing the accuracy of solar power forecasts, addressing their capabilities, limitations, and areas for further improvement. The synthesis of five years of research provides insights into the progress, emerging trends, and future directions in harnessing solar energy through instrumentation and predictive modeling, contributing to a broader understanding and development of renewable energy systems.

Ahmed and Khalid’s research [37] focuses on forecasting models applied to various renewable energy sources, including solar, wind, hydroelectric, and other emerging technologies. It involves a targeted analysis of ANN-based models and their practical implications in optimizing the integration and management of renewable energy systems. Their work proposes new advances in power system forecasting for greater efficiency and stability in energy systems.

Focused on improving the supply of electricity, Corso et al. [38] presented a study on the application of convolutional neural network (CNN)-based models for classifying faults in electrical insulators, an approach that was also applied in [39] through a hybrid approach that combines several models. In [40], CNNs are combined with LSTM for fault detection in electrical machines.

The CNN-LSTM can also be used for time series forecasting [41]. In [42], an interpretable method is proposed to improve fault classification. In [43], an object detection method based on deep learning is combined with the Quasi-ProtoPNet to have an improved architecture to classify faults in the power grid, enhancing the reliability of the electrical power system. Deep learning strategies have been successfully applied to time series forecasting [44].

In [45], ANNs are applied for forecasting electricity generation, and in the work of Akhter et al. [46], the landscape of forecasting photovoltaic power generation through machine learning and metaheuristic techniques is analyzed. They highlight the effectiveness, adaptability, and areas for improvement in machine learning and metaheuristic techniques, thus providing practical applications in optimizing the efficiency of photovoltaic energy systems. Nazir et al. [47] presented a review of five-year wind generation forecasting methodologies, focusing on ANN applications. In their paper, they explore the evolution and limitations of ANNs for energy forecasting, providing ideas on possible implementations of these models, and highlighting their potential for renewable energy forecasting.

Regarding hybrid methods, many authors are applying filters combined with forecasting models to enhance their capacities and generalization. For instance, in [48], the Hodrick–Prescott filter, and in [49,50,51], the wavelet transform, are applied for denoising. All these studies proved that the use of filters in pre-processing the time series is a promising strategy when trend forecasting is the major goal of the analysis.

Yamasaki et al. [52] explore the application of optimized hybrid ensemble learning approaches in the context of very short-term load forecasting. According to them, by leveraging the strengths of hybridization, the optimized ensemble learning models demonstrate superior performance in accurately predicting short-term electricity demand. Also using a hybrid model, Stefenon et al. [53] used a group method of data handling combined with the Christiano–Fitzgerald random walk filter. The results highlighted that a denoising strategy is a promising way to enhance the architecture of these models.

The methods used here could be applied to other fields, such as in [54], where the kernel representations are evaluated, making it possible to capture changing regimes. In [55], a regime-switching method with non-linear representation is proposed; the authors proved that the model can produce promising performance results in stock market forecasting.

According to Klaar et al. [56], the use of a wavelet to decompose the signal combined with an attention mechanism is an outstanding strategy to enhance the prediction of the LSTM network. This combination is also applied in [57], where the predictions concern the level of dams for hydroelectric power plants. In [58], the wavelet with the LSTM is evaluated without the attention mechanism; using this strategy, the results were better than for other models, showing that the combination of these two techniques is a good strategy.

The research by Stefenon et al. [59] proposes a hybrid forecasting approach for modeling and predicting the complex dynamics of Italian electricity spot prices. They combine the strengths of the Prophet forecasting model with seasonal trend decomposition using LOESS (locally estimated scatterplot smoothing), known for its ability to handle seasonality, holidays, and long-term trends. The integration of these two methodologies enhances the accuracy and robustness of electricity spot price forecasts.

The work of Klaar et al. [60] focuses on enhancing the accuracy of energy price forecasting in Latin America, with a specific case study conducted in Mexico. They propose a novel approach by integrating ensemble learning methods and seasonal decomposition techniques to optimize the structure of forecasting models. The motivation behind this research is the dynamic and often unpredictable nature of energy markets, particularly in the context of Latin American countries where energy pricing is influenced by various economic, environmental, and regulatory factors.

In the work of Júnior et al. [61], the authors explore the landscape of forest bioelectricity generation in Brazil, investigating its distribution patterns and temporal-spatial dependencies. The study combines geographical information systems and time-series data, to assess the distribution of forest bioelectricity plants across different regions of the country. Their research offers a foundation for policymakers, energy planners, and stakeholders to make informed decisions that promote the efficient and sustainable integration of this renewable energy source into Brazil’s broader energy landscape.

In [62], a hybrid approach for energy consumption forecasting within the context of smart grids is presented. The proposed hybrid model combines the strengths of machine learning and optimization techniques to enhance forecasting accuracy and efficiency. When evaluating the available forecasting models, it is a hard task to define the most appropriate structure to perform the predictions. According to Stefenon et al. [63], the ensemble learning methods may have better performance than deep learning since they are based on weak learners that usually require less computational effort to be computed.

Dataset

The data considered relate to the thermal generation due to dispatch in the northern region of Brazil and are provided by the national system operator (in Portuguese Operador Nacional do Sistema—ONS). The original dataset is available at: https://dados.ons.org.br/dataset/geracao-termica-despacho-2 (accessed on 25 November 2023). The total generation relates to 16 thermal power plants (Aparecida Parte I, Cristiano Rocha, Geramar I, Geramar II, Jaraqui, Manauara, Maranhão III, Maranhão IV, Maranhão V, Mauá 3, MC2 Nova Venécia 2, Parnaíba IV, Parnaíba V, Ponta Negra, Porto do Itaqui, and Tambaqui).

Scheduled and verified generation data for thermal power plants dispatched by the ONS on an hourly basis are provided. This time series is presented in Figure 2. Given an hourly basis, 2160 records are considered from 18 August (0 h) to 15 November (23 h) of 2023, resulting in a period of evaluation of 90 days. The time series forecasting presented here is one step ahead, meaning that the predictions would be presented one hour earlier.

The models presented in this paper can be applied to other datasets since the analysis is carried out in relation to the variation in the time series (see Figure 2). Especially concerning the analysis of variation in diesel consumption, it is necessary to understand whether non-linearities (noise in the time series) affect the applicability of the model. In these cases, it is necessary to use input filters to attenuate the noise.

3. Proposed Method

A common ensemble method for time series forecasting involves combining predictions from multiple base models to improve forecasting accuracy [64]. Let $\left\{y_t\right\}$ represent a univariate time series where $y_t$ denotes the value at time t. Suppose we have M base forecasting models denoted as $f_1,f_2,\dots ,f_M$, which can be any suitable time series forecasting models [65].

For a given time series dataset up to time T, the ensemble forecast ${\widehat{y}}_T$ at time T is computed by aggregating the predictions of the base models:

$${\widehat{y}}_T={\displaystyle \frac{1}{M}}\sum _{i=1}^M{f}_i\left(T\right)$$

where $f_i\left(T\right)$ represents the forecast generated by the i-th base model at time T. Various aggregation methods can be used for combining predictions, such as simple averaging, or more sophisticated techniques like stacking or boosting applied to time series forecasting [66]. Ensemble methods aim to leverage the diversity of individual models to create a more accurate and robust forecast by reducing potential biases or errors inherent in any single forecasting model [67]. In this paper, the stacking, bagging, and boosting ensemble learning models are considered.

3.1. Stacking Ensemble Learning Model

Let X be the training dataset with input features represented as $X=\{x_1,x_2,\dots ,x_n\}$ and corresponding labels $Y=\{y_1,y_2,\dots ,y_n\}$. Let M be the number of base models, and each base model $h_i$ is represented by:

$$h_i:X\to Y_i,i=1,2,\dots ,M$$

where $Y_i$ denotes the predictions made by the i-th base model [68].

The predictions of the base models are combined to form a new dataset $X^{\prime }$, where each instance $x_j^{\prime }$ is given by:

$$x_j^{\prime }=\{h_1\left(x_j\right),h_2\left(x_j\right),\dots ,h_M\left(x_j\right)\},j=1,2,\dots ,n.$$

A meta-learner g is then trained on the new dataset $X^{\prime }$ using the true labels Y:

$$g:X^{\prime }\to Y.$$

The final prediction $\widehat{y}$ for a new input $x_{\mathrm{new}}$ is obtained by applying the meta-learner g on the predictions of the base models for $x_{\mathrm{new}}$:

$${\widehat{y}}_{\mathrm{new}}=g\left(\{h_1\left(x_{\mathrm{new}}\right),h_2\left(x_{\mathrm{new}}\right),\dots ,h_M\left(x_{\mathrm{new}}\right)\}\right).$$

The architecture of the stacking ensemble learning approach is presented in Figure 3.

3.2. Boosting Ensemble Learning Model

The boosting model is an ensemble method where multiple weak learners are combined to create a strong learner [69]. Let T denote the number of iterations or base models. Initialize the weights of the training instances uniformly: $D_1\left(i\right)={\displaystyle \frac{1}{n}}$ for $i=1,2,\dots ,n$.

For each iteration $t=1,2,\dots ,T$: Train a weak learner $h_t$ on the weighted training data X with weights $D_t\left(i\right)$. Calculate the weighted error ${ϵ}_t$ of $h_t$:

$${ϵ}_t=\sum _{i=1}^n{D}_t\left(i\right)·⊮(h_t\left(x_i\right)\ne y_i)$$

where $⊮$ is the indicator function. Compute the importance of $h_t$:

$${\alpha }_t={\displaystyle \frac{1}{2}}ln\left({\displaystyle \frac{1-{ϵ}_t}{{ϵ}_t}}\right).$$

Update the weights $D_{t+1}\left(i\right)$ for the next iteration:

$$D_{t+1}\left(i\right)={\displaystyle \frac{D_t\left(i\right)·exp\left(-{\alpha }_t·y_i·h_t\left(x_i\right)\right)}{Z_t}}$$

where $Z_t$ is a normalization factor to ensure that the weights sum up to 1 [70]. The architecture of the boosting ensemble model is shown in Figure 4.

3.3. Bagging Ensemble Learning Model

Bagging (Bootstrap Aggregating) is an ensemble method where B base models are trained on different subsets of the training data. Let B denote the number of base models [71]. Each base model $h_i$ is trained on a bootstrap sample $X_i$ (a randomly sampled subset with replacement from X) and is represented by:

$$h_i:X_i\to Y_i,i=1,2,\dots ,B$$

where $Y_i$ denotes the predictions made by the i-th base model on its respective bootstrap sample $X_i$ [70].

The final prediction ${\widehat{y}}_{\mathrm{new}}$ for a new input $x_{\mathrm{new}}$ is obtained by aggregating the predictions of all base models:

$${\widehat{y}}_{\mathrm{new}}=\mathrm{Aggregation}(h_1\left(x_{\mathrm{new}}\right),h_2\left(x_{\mathrm{new}}\right),\dots ,h_B\left(x_{\mathrm{new}}\right)).$$

Common aggregation methods include averaging the predictions for regression problems or taking a majority vote (or averaging probabilities) for classification problems. In this paper, aggregation through averaging was applied to the forecasting of time series. The architecture of this model is shown in Figure 5.

3.4. Parameters

For the hyperparameters evaluation, the linear (LIN) radial basis function (RBF) [72], and polynomial (POLY) kernel functions are considered. The sequential minimal optimization (SMO), iterative single data algorithm (ISDA) [73], and L1 soft-margin minimization by quadratic programming (L1QP) solver are evaluated [74]. Besides the error measures, the total time (training and testing) for each experiment is presented.

The L1QP involves solving the optimization task in its entirety using traditional optimization methods, such as the interior-point method [75]. The Wolfe dual problem that is evaluated in the optimization process is calculated by the following equation:

$$\begin{array}{cc}\hfill & \underset{\alpha }{max}\sum _{i=1}^m{\alpha }_i-{\displaystyle \frac{1}{2}}\sum _{i=1}^m\sum _{j=1}^m{\alpha }_i{\alpha }_j{y}_i{y}_j{\mathbf{x}}_i,{\mathbf{x}}_j\hfill \end{array}$$

(1a)

$$\begin{array}{cc}\hfill & \mathrm{s}.\mathrm{t}.:0\le {\alpha }_i\le C,\forall i=1,\dots ,m\hfill \end{array}$$

(1b)

$$\begin{array}{cc}\hfill & \sum _{i=1}^m{\alpha }_i{y}_i=0\hfill \end{array}$$

(1c)

where ${\alpha }_i$ is the dual variable that lies within the box.

SMO improves the computational efficiency by solving a simpler sequence of optimization problems. The SMO algorithm works on the dual form of the problem, solving the dual problem for two points at a time, thereby updating two dual variables at each step [48]. Instead of solving the complete optimization problem, ISDA processes one data point at a time. This makes its functionality similar to SMO, but with the key difference of updating only one multiplier at a time. To determine which multiplier to update, ISDA selects the point that most significantly violates the Karush–Kuhn–Tucker (KKT) conditions [76].

The kernel functions (K) applied in the support vector regression, which are the weak learners used in the ensemble learning methods evaluated here are as follows: the linear (LIN) (2), radial basis function (RBF) (3), and polynomial (POLY) (4). It should be noted that the selection of the transformation to be applied is determined by the type of data and is often performed by experimentation [74]. In this paper, these functions are evaluated to obtain the best application of the compared methods.

$$K(x_j,x_k)={x_j}^{\prime }{x}_k.$$

(2)

$$K(x_j,x_k)=exp(-{∥x_j-x_k∥}^2).$$

(3)

$$K(x_j,x_k)={\left(1+{x_j}^{\prime }{x}_k\right)}^q.$$

(4)

3.5. Benchmarking

For the proposed method, the LSTM is considered. LSTM networks are a type of recurrent neural network architecture designed to model sequential data while addressing the vanishing or exploding gradient problem in traditional ANNs [77]. Given a sequence of input data, an LSTM unit maintains a hidden state vector ${\mathbf{h}}_t$ and a cell state vector ${\mathbf{c}}_t$ at time step t [25].

The LSTM unit consists of several gates that control the flow of information [78]. The input gate (${\mathbf{i}}_t$) controls the flow of new information into the cell state, the forget gate (${\mathbf{f}}_t$) controls the flow of old information to be forgotten from the cell state, and the output gate (${\mathbf{o}}_t$) controls the flow of information from the cell state to the hidden state for output [79]. The computations in an LSTM unit are described as follows:

$$\begin{array}{cc}\hfill {\mathbf{i}}_t& =\sigma ({\mathbf{W}}_i·[{\mathbf{h}}_{t-1},{\mathbf{x}}_t]+{\mathbf{b}}_i)\hfill \\ \hfill {\mathbf{f}}_t& =\sigma ({\mathbf{W}}_f·[{\mathbf{h}}_{t-1},{\mathbf{x}}_t]+{\mathbf{b}}_f)\hfill \\ \hfill {\mathbf{o}}_t& =\sigma ({\mathbf{W}}_o·[{\mathbf{h}}_{t-1},{\mathbf{x}}_t]+{\mathbf{b}}_o)\hfill \\ \hfill {\tilde{\mathbf{c}}}_t& =tanh({\mathbf{W}}_c·[{\mathbf{h}}_{t-1},{\mathbf{x}}_t]+{\mathbf{b}}_c)\hfill \\ \hfill {\mathbf{c}}_t& ={\mathbf{f}}_t\odot {\mathbf{c}}_{t-1}+{\mathbf{i}}_t\odot {\tilde{\mathbf{c}}}_t\hfill \\ \hfill {\mathbf{h}}_t& ={\mathbf{o}}_t\odot tanh\left({\mathbf{c}}_t\right)\hfill \end{array}$$

where $\mathbf{W}$ and $\mathbf{b}$ are weight matrices and bias vectors specific to each gate, ${\tilde{\mathbf{c}}}_t$ is the candidate cell state, and $\sigma$ and tanh are the sigmoid and hyperbolic tangent activation functions [80]. The LSTM network was employed for benchmarking using three optimizers as follows: root mean square propagation (RMSProp) [81], adaptive moment estimation (ADAM) [82], and stochastic gradient descent with momentum (SGDM) [83].

An adaptive neuro-fuzzy inference system (ANFIS) is also evaluated in comparison with the best method found in this work, considering the Sugeno FIS using fuzzy c-means (FCM), subtractive clustering, and grid partition. The ANFIS is a hybrid model that integrates the strengths of both ANNs and fuzzy logic principles, making it particularly effective for time series forecasting [84].

ANFIS harnesses the learning capabilities of neural networks to optimize the fuzzy inference system’s parameters, thereby enhancing the model’s ability to handle non-linear and complex patterns inherent in time series data [85]. ANFIS employs a Takagi–Sugeno-type fuzzy inference mechanism, which facilitates the generation of a fuzzy rule base from the input data, enabling a nuanced mapping of inputs to outputs. ANFIS stands out as a robust and versatile tool for time series forecasting, offering significant improvements in predictive performance and reliability [86].

The evaluation flowchart followed in this paper is presented in Figure 6. The proposed application is based on ensemble learning models that combine weak learners to create a meta-learner. For this, the bagging, boosting, and stacking approaches are used. A statistical analysis was carried out considering 50 experiments with each model to evaluate the variability in the results considering a random initialization of the network. The maximum, minimum, mean, standard deviation, and variance values are presented. The LSTM and ANFIS methods were used for comparative analysis (benchmarking).

3.6. Evaluation Setup

For comparison, the data are split using 70% for training and 30% for testing. All experiments were computed using an i5-7300HQ, with 20 GB of RAM, using the Matlab (2019a) software. For future comparisons, the ensemble learning models applied in this paper can be found at: https://github.com/vhrique/ELT (accessed on 24 October 2024). The stacked, boosted, and bagged ensemble learning models are analyzed. For model assessment, the mean squared error (MSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and coefficient of determination are used, given by:

$$MSE={\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2$$

(5)

$$MAE={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left(y_i-{\widehat{y}}_i\right)$$

(6)

$$error={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left|\begin{array}{c}{\displaystyle \frac{y_i-{\widehat{y}}_i}{y_i}}\end{array}\right|\times 100$$

(7)

$$R^2=1-{\displaystyle \frac{{\sum }_{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2}{{\sum }_{i=1}^n{\left(y_i-{\overline{y}}_i\right)}^2}}$$

(8)

where y is the observed value, $\widehat{y}$ is the predicted value, and $\overline{y}$ is the average of the observed values [48].

4. Results and Discussion

In this section, the results of the predictions using the ensemble learning methods are discussed and a final statistical evaluation is presented considering the best parameters of each structure.

4.1. Ensemble Learning Model Evaluation

The results of the stacked ensemble learning model are presented in

Table 1. Results of stacked ensemble learning model.

Solver	Kernel	MSE	MAE	MAPE (%)	R2	Time (s)
ISDA	LIN	1.49 $\times {10}^5$	1.76 $\mathbf{\times }{\mathbf{10}}^{\mathbf{2}}$	7.03	0.4198	1.28
	RBF	3.78 $\times {10}^5$	3.13 $\times {10}^2$	1.32 $\times {10}^1$	0.4735	0.65
	POLY	1.03 $\times {10}^{12}$	1.58 $\times {10}^5$	5.39 $\times {10}^3$	-	40.89
L1QP	LIN	1.49 $\times {10}^5$	1.76 $\mathbf{\times }{\mathbf{10}}^{\mathbf{2}}$	7.02	0.4202	10.53
	RBF	4.06 $\times {10}^5$	3.28 $\times {10}^2$	1.39 $\times {10}^1$	0.5812	12.62
	POLY	1.25 $\times {10}^9$	3.67 $\times {10}^3$	1.21 $\times {10}^2$	-	9.33
SMO	LIN	1.48 $\mathbf{\times }{\mathbf{10}}^{\mathbf{5}}$	1.76 $\mathbf{\times }{\mathbf{10}}^{\mathbf{2}}$	7.02	0.4218	2.70
	RBF	4.06 $\times {10}^5$	3.27 $\times {10}^2$	1.39 $\times {10}^1$	0.5809	10.57
	POLY	8.92 $\times {10}^9$	1.64 $\times {10}^4$	5.51 $\times {10}^2$	-	95.33

Best results are in bold.

. Using the linear kernel function, all compared solvers had better results of error (MSE, MAE, and MAPE). The results using the polynomial kernel function were much higher, and this kernel function should be avoided for the stacked ensemble model. The best solver for this model was the SMO, which was slightly better than the other solvers considering the linear kernel function.

Given the poor results of the stacked ensemble learning model, other approaches are evaluated. In

Table 2. Results of boosted ensemble learning model.

Solver	Kernel	MSE	MAE	MAPE (%)	R2	Time (s)
ISDA	LIN	1.25 $\times {10}^5$	1.20 $\times {10}^2$	5.50	0.5111	31.82
	RBF	4.48 $\times {10}^5$	4.06 $\times {10}^2$	1.94 $\times {10}^1$	0.7461	6.74
	POLY	4.43 $\times {10}^7$	2.06 $\times {10}^3$	7.56 $\times {10}^1$	-	1136.64
L1QP	LIN	5.19 $\mathbf{\times }{\mathbf{10}}^{\mathbf{4}}$	1.03 $\mathbf{\times }{\mathbf{10}}^{\mathbf{2}}$	4.96	0.7977	179.57
	RBF	4.66 $\times {10}^5$	4.15 $\times {10}^2$	1.98 $\times {10}^1$	0.8181	167.07
	POLY	5.03 $\times {10}^7$	1.86 $\times {10}^3$	6.86 $\times {10}^1$	-	214.36
SMO	LIN	8.51 $\times {10}^4$	1.54 $\times {10}^2$	7.88	0.6682	49.58
	RBF	4.66 $\times {10}^5$	4.17 $\times {10}^2$	2.00 $\times {10}^1$	0.8156	4.74
	POLY	6.83 $\times {10}^7$	2.30 $\times {10}^3$	8.70 $\times {10}^1$	-	1847.39

Best results are in bold.

, the results of the prediction performances of the boosted ensemble learning model are presented considering the same variation of structures used previously.

Regarding the hyperparameters, the results of the boosted model were similar to the stacked ensemble learning model. Here, the linear kernel function was the best selection considering the error measures. However, for this model, the L1QP is the best solver and this structure has been used further for the final assessments.

In

Table 3. Results of bagged ensemble learning model.

Solver	Kernel	MSE	MAE	MAPE (%)	R2	Time (s)
ISDA	LIN	6.39 $\times {10}^3$	4.20	1.02 $\times {10}^{-1}$	0.9751	9.71
	RBF	4.21 $\times {10}^5$	3.42 $\times {10}^2$	1.48 $\times {10}^1$	0.6403	1.91
	POLY	1.00 $\times {10}^6$	2.65 $\times {10}^2$	1.00 $\times {10}^1$	-	430.01
L1QP	LIN	6.39 $\times {10}^3$	4.23	9.71 $\times {10}^{-2}$	0.9751	36.75
	RBF	4.48 $\times {10}^5$	3.55 $\times {10}^2$	1.54 $\times {10}^1$	0.7482	41.035
	POLY	4.03 $\times {10}^5$	6.01 $\times {10}^1$	1.91	0.5721	37.813
SMO	LIN	6.36 $\mathbf{\times }{\mathbf{10}}^{\mathbf{3}}$	3.77	7.34 $\mathbf{\times }{\mathbf{10}}^{\mathbf{-}\mathbf{2}}$	0.9752	16.97
	RBF	4.49 $\times {10}^5$	3.56 $\times {10}^2$	1.54 $\times {10}^1$	0.7494	1.52
	POLY	4.43 $\times {10}^5$	6.99 $\times {10}^1$	2.24	-	865.87

Best results are in bold.

, the complete evaluation of the bagged model is presented. These results were more promising given that a MAPE of 0.073% and a coefficient of determination of 0.9752 were achieved. These results were achieved considering the SMO solver with the linear kernel function; additionally, the MSE and the MAE were better than the compared models and configurations.

In Figure 7, a comparison between the observed and the predicted values is presented for all the evaluated ensemble learning methods. In Figure 7A, major differences occurred when there were higher values of schedule generation, resulting in higher errors, making the stacked ensemble learning model unsuitable for this evaluation.

The boosted model (Figure 7B) was better than the previously evaluated staked model; however, there were still major differences between the predicted and the observed values. The bagged model, presented in Figure 7C, had promising results with predictions close to the desired output. The prediction results presented here consider one step ahead (1 h), which is adequate for a schedule generation problem. In the following, a statistical assessment of these ensemble learning models considering their best setup parameters is presented.

4.2. Statistical Evaluation

For the bagged and stacked ensemble learning approaches, the best solver was the SMO, and for the boosted, the best was the L1QP; all these results were considering the linear kernel function. Considering the best configuration for each model, a statistical analysis is presented in

Table 4. Statistical analysis of ensemble learning approaches.

		Stacking	Boosting	Bagging
MSE	Max	1.48 $\times {10}^5$	1.40 $\times {10}^5$	6.44 $\times {10}^3$
	Min	1.48 $\times {10}^5$	3.80 $\times {10}^4$	6.29 $\times {10}^3$
	Mean	1.48 $\times {10}^5$	8.57 $\times {10}^4$	6.38 $\times {10}^3$
	Std Deviation	8.82 $\times {10}^{-11}$	2.63 $\times {10}^4$	3.35 $\times {10}^1$
	Variance	7.78 $\times {10}^{-21}$	6.90 $\times {10}^8$	1.12 $\times {10}^3$
MAE	Max	1.76 $\times {10}^2$	2.14 $\times {10}^2$	4.57
	Min	1.76 $\times {10}^2$	7.90 $\times {10}^1$	3.7
	Mean	1.76 $\times {10}^2$	1.62 $\times {10}^2$	4.04
	Std Deviation	1.15 $\times {10}^{-13}$	3.25 $\times {10}^1$	1.83 $\times {10}^{-1}$
	Variance	1.32 $\times {10}^{-26}$	1.06 $\times {10}^3$	3.34 $\times {10}^{-2}$
MAPE	Max	7.02	1.19 $\times {10}^1$	1.22 $\times {10}^{-1}$
	Min	7.02	3.65	7.25 $\times {10}^{-2}$
	Mean	7.02	8.44	8.90 $\times {10}^{-2}$
	Std Deviation	3.59 $\times {10}^{-15}$	2.05	1.18 $\times {10}^{-2}$
	Variance	1.29 $\times {10}^{-29}$	4.19	1.39 $\times {10}^{-4}$
R2	Max	4.22 $\times {10}^{-1}$	8.52 $\times {10}^{-1}$	9.75 $\times {10}^{-1}$
	Min	4.22 $\times {10}^{-1}$	4.56 $\times {10}^{-1}$	9.75 $\times {10}^{-1}$
	Mean	4.22 $\times {10}^{-1}$	6.66 $\times {10}^{-1}$	9.75 $\times {10}^{-1}$
	Std Deviation	-	1.02 $\times {10}^{-1}$	1.31 $\times {10}^{-4}$
	Variance	-	1.05 $\times {10}^{-2}$	1.71 $\times {10}^{-8}$

. This evaluation considers 50 runs with random initialization of the weights.

The bagging ensemble learning method has proved that in addition to having a lower error compared to other models, it is stable since for the MSE, the variation in the maximum error and minimum error was 2.38%, considering 50 experiments with random initialization. For this model, the coefficient of determination remains at 0.97 in all experiments.

Given the error results of the bagging model, the standard deviation was also low, proving that this structure has an acceptable performance regarding its robustness. The more stable model was the stacking ensemble learning method; however, its performance regarding the error was inferior to the bagging, which was the more suitable model for the task covered in this paper.

4.3. Comparative Analysis

In this subsection, a comparative analysis with variations of the LSTM and ANFIS models is presented. Specifically, for the LSTM, the RMSprop, Adam, and SGDM optimizers are evaluated, and the evaluated parameter is the number of hidden units, with 10, 50, 100, and 500 hidden units. For the ANFIS, the FCM, grid partition, and subtractive clustering are considered; for the FCM, the parameter evaluated is the number of clusters.

Table 5. Comparison of ensemble bagged method to LSTM model.

Model	Method	Evaluated Parameter	MSE	MAE	MAPE (%)	R2	Time (s)
LSTM	RMSprop	10	1.65 $\times {10}^5$	1.81 $\times {10}^2$	7.31	0.3576	42.21
		50	6.06 $\times {10}^4$	8.31 $\times {10}^1$	2.89	0.7634	27.81
		100	5.11 $\times {10}^4$	8.21 $\times {10}^1$	3.11	0.8006	29.40
		500	2.88 $\times {10}^4$	6.33 $\times {10}^1$	2.87	0.8876	55.55
	Adam	10	1.22 $\times {10}^5$	1.55 $\times {10}^2$	6.33	0.5246	27.65
		50	3.31 $\times {10}^4$	5.29 $\times {10}^1$	2.07	0.8710	29.76
		100	1.06 $\times {10}^4$	2.00 $\times {10}^1$	0.71	0.9586	28.72
		500	1.75 $\times {10}^4$	5.38 $\times {10}^1$	2.19	0.9317	56.04
	SGDM	10	1.25 $\times {10}^5$	1.58 $\times {10}^2$	6.50	0.5133	25.97
		50	5.88 $\times {10}^4$	8.03 $\times {10}^1$	3.06	0.7705	26.16
		100	4.79 $\times {10}^4$	6.65 $\times {10}^1$	2.39	0.8131	26.72
		500	1.26 $\times {10}^4$	1.14 $\times {10}^1$	0.29	0.9510	53.84
ANFIS	FCM	10	1.71 $\times {10}^4$	6.00 $\times {10}^1$	2.66	0.9334	6.11
		20	2.59 $\times {10}^4$	7.87 $\times {10}^1$	3.49	0.8991	11.16
		30	1.20 $\times {10}^7$	1.53 $\times {10}^3$	6.48 $\times {10}^1$	-	17.15
		40	2.85 $\times {10}^9$	6.35 $\times {10}^2$	2.60 $\times {10}^1$	-	25.99
		50	2.16 $\times {10}^{10}$	5.82 $\times {10}^3$	2.51 $\times {10}^2$	-	37.936
Bagged Model	-	-	6.36 $\mathbf{\times }{\mathbf{10}}^{\mathbf{3}}$	4.04	8.95$\mathbf{\times }{\mathbf{10}}^{\mathbf{-}\mathbf{2}}$	0.9752	15.59

Best results are in bold.

shows the results of this comparison in relation to the bagged ensemble learning model.

The ANFIS model using the grid partition only converged using two membership functions (MFs) for the input variables; when using more MFs, the model did not converge. When subtractive clustering is evaluated, no combination of parameters results in the convergence of the model. For this reason, the grid partition and subtractive clustering are not presented in Table 5. This issue also occurred when using CNN-LSTM; models based on deep learning did not result in acceptable prediction values.

For the LSTM, the best results were achieved using 500 hidden units in both RMSprop and SGDM. An attempt was made to use more hidden units but the model did not converge when a greater number of hidden units was used. For the Adam optimizer, the best result was using 100 hidden units. Among the optimizers used for the LSTM, the best result was using Adam; however, this result was inferior in all the metrics evaluated compared to the bagged ensemble model, proving, in this comparative analysis, that the approach used in this article is the most suitable to be applied to this challenge.

4.4. Comparison to Other Research

Comparatively, Ribeiro et al. [23] had an MSE of 388.46 (RMSE was originally presented), an R² of 0.9544, and a symmetric MAPE of 0.0418 for three months ahead using a self-adaptive decomposition and heterogeneous ensemble learning method. The authors highlight that using more steps ahead results in greater difficulty in predicting future values. In this paper, a step ahead is considered, and this analysis is suitable for the proposed problem, resulting in better performance according to the results discussed here.

A very short-term load forecasting was also discussed by Yamasaki et al. [52]; with the promising results pointed out in their work, the use of ensemble learning methods, such as those applied in this paper. is proving to be a potentially valuable solution. The ensemble stacking model was used in [24]; this model was not the best solution here, emphasizing that a complete evaluation with several ensemble learning models is necessary for the best prediction strategy.

Noise reduction methods were applied by Stefenon et al. [53], Ribeiro et al. [87], and Moreno et al. [88], which may be interesting for improving the capabilities of the prediction model. However, if denoising is not used properly, it can result in unnecessary attenuation of the signal. Predicting time series relative to the original signal is a better strategy because no information is lost.

5. Conclusions

By employing time series forecasting techniques, this paper provides insights into the patterns, trends, and dependencies associated with diesel-powered thermal systems. Understanding these dynamics is pivotal for energy planners, policymakers, and industry stakeholders to make informed decisions regarding energy resource allocation, infrastructure development, and environmental sustainability.

Considering a MAPE of 0.089% and a coefficient of determination of 97.52% achieved by the bagged ensemble learning model (average considering 50 experiments), besides its robustness proved by statistical evaluation, this model is shown to be promising for short-term time series forecasting. By having more precise predictions about the use of thermal systems, it is possible to improve the management of the feedstock to be used for burning and generating electricity. This can prevent a shortage of the necessary inputs, resulting in a price spike in this type of generation.

This research lays the groundwork for informed decision-making, emphasizing the importance of balancing energy needs, environmental considerations, and long-term sustainability in Brazil’s energy landscape. Stakeholders can steer the nation towards a more resilient, diversified, and environmentally conscious energy future. Further research can be undertaken, including an evaluation of the relations between different regions since when there is a lack of rain and the hydropower dams have lower levels, generation using thermal systems is used in all regions of the country. For time series with higher noise, filters with higher intensity may be a solution that can be applied to related tasks.