1. Introduction
In recent years, global decarbonization efforts have intensified to mitigate climate change, with the transition toward renewable and smart energy systems becoming a key initiative in various regions. Regional smart energy systems require reliability and flexibility, integrating electricity, heating and transportation sectors to manage demand–supply fluctuations across various time scales [
1]. Short-term load forecasting (STLF) is a critical component in the effective management and coordination of regional smart energy systems [
2]. Precise STLF is crucial for maintaining an immediate equilibrium between electricity supply and consumption [
3] and enhancing the dependability and robustness of the smart energy system. Furthermore, the emergence of regional smart energy systems and sophisticated metering technology has significantly increased the granularity and volume of load data. This presents new opportunities and challenges for regional STLF [
4]. Electricity often exhibits nonlinear fluctuations. These fluctuations in load can be significantly influenced by unpredictable external factors such as weather conditions (temperature, humidity), economic activities and social events. Existing models may not incorporate these variables effectively, thus failing to account for sudden spikes or drops in demand. Therefore, developing robust and accurate regional STLF models is paramount for optimizing power system operations, reducing energy wastage and facilitating the incorporation of renewable energy options [
5].
Short-term load power forecasting techniques can be generally categorized into statistical fitting approaches and machine learning approaches [
6]. Statistical fitting approaches such as regression analysis [
7], exponential smoothing [
8] and the autoregressive integrated moving average (ARIMA) [
9] depend exclusively on past data and employ statistical frameworks to identify linear correlations within the data [
10]. However, electricity demand often exhibits nonlinear and dynamic patterns influenced by numerous factors such as time of day, day of the week and seasonality. Traditional methods may not adequately capture these complexities, leading to suboptimal forecasts. In addition, these methods are limited to capturing only linear relationships. As the complexity of the data and prediction step size increase, these methods often stabilize around a consistent mean value, resulting in reduced prediction precision. Conversely, machine learning approaches have gained significant popularity for analyzing time-series data and are extensively applied in regional STLF [
11]. Traditional machine learning techniques, including support vector machines [
12] and random forests [
13], along with deep learning techniques, leverage their powerful self-learning capabilities and robust nonlinear modeling abilities. Consequently, these machine learning approaches typically yield more accurate short-term load forecasts compared to statistical fitting methods [
14].
As a prominent method in machine learning, deep learning has attracted considerable interest for STLF. Commonly employed models encompass Long Short-Term Memory (LSTM) networks [
15], Convolutional Neural Networks (CNNs) [
16], etc. For time-series forecasting, feature extraction from historical data plays a pivotal role. CNNs are extensively employed to derive features from time-series data. Alhussein et al. utilized CNNs for feature extraction from historical load data, combined with LSTM for future load forecasting [
17]. Wang et al. integrated CNNs with the Informer, improving prediction precision and addressing the challenge of restricted time-series data on a singular temporal scale [
18]. However, the feature extraction ability of CNNs is still limited, resulting in limited prediction accuracy. To further improve their ability, temporal neural networks (TCNs) were introduced [
19]. TCNs overcome long-term dependency problems and performance declines by incorporating dilated causal convolutions and residual blocks, thereby exceeding CNNs in time-series feature extraction [
20]. Gong et al. applied TCNs for hidden temporal feature extraction and the Informer for accurate short-term forecasting [
20]. Zhou et al. proposed a load forecasting model incorporating Prophet and improved TCNs, using TCNs to extract the internal correlation in the input matrix obtained by Prophet. This helps to achieve high efficiency and prediction accuracy [
21].
To enhance internal correlations and focus on crucial information in historical data, multi-level attention networks are being increasingly utilized and refined. In 2017, Vaswani et al. proposed a Transformer model that excels in capturing long-range dependencies through self-attention mechanisms, surpassing traditional recurrent neural networks in capturing complex dependencies [
22]. Between the years of 2017 and 2024, many improvements have been made to enhance the capabilities of the Transformer. Zhou et al. proposed the Informer, which improved the Transformer by incorporating a hybrid architecture with temporal convolutional layers, enhancing its ability to model sequential data [
23]. However, its increased complexity due to hybrid architecture may lead to higher computational costs and training time. Liu et al. introduced the iTransformer, an innovative model that applies attention and feed-forward networks to inverted dimensions [
24]. This approach enhances the performance and generalization capabilities of the original Transformer, resulting in the best overall performance among all Transformer-based variants.
Electricity load sequences are intricate, typically comprising various elements like trends, seasonal variations and random fluctuations. Deep learning models have limitations in capturing these complexities effectively. Consequently, diverse data preprocessing techniques are utilized to dissect the multiple components within load data for robust feature extraction. Mounir et al. employed empirical mode decomposition (EMD) to segment input series into separate elements for independent prediction. They then amalgamated the predictions of each component to derive the final forecast [
25]. Li et al. utilized complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) to segment the original time series into several different components. Then, guided by sample entropy, they were restructured into a dual-component series for further processing [
26]. Yuan et al. leveraged variational mode decomposition (VMD) to decompose the input load series, effectively minimizing modal confusion and improving feature extraction capabilities [
27]. However, these decomposition methods often encounter high computational complexity and struggle to capture specific trends and seasonal changes within load data. Stefenon et al. aggregated Prophet and seasonal and trend decomposition utilizing LOESS (STL) for robust time-series analysis, achieving high prediction accuracy [
28]. Lin et al. devised a two-stage feature extraction method combining STL and time-lag autocorrelation to extract features from time series [
29]. Wang et al. employed Prophet to extract trends and seasonal variations within load data and further decomposed residual data to extract several stable data features. This method effectively captures features within the intricate original time series [
30]. Nonetheless, the feature extraction methods proposed in the aforementioned studies still lack the ability to extract features comprehensively. This is due to their oversight of significant events like holidays and limited capacity to extract periodic features from residual components.
To improve the extraction ability of periodic features from time series using decomposition methods, optimization techniques have been implemented to refine the parameters of the decomposition algorithms [
31]. Zhao et al. introduced an enhanced snake optimization algorithm (ISOA) to fine-tune VMD parameters, consequently improving the extraction of periodic features from load data [
32]. However, the improved snake optimization algorithm may suffer from slow convergence rates and sensitivity to parameter settings. Su et al. fine-tuned the VMD parameters using the sparrow algorithm to extract components with significant features, avoiding redundant and complex components [
33]. Nevertheless, the sparrow algorithm’s dependence on centralized coordination can lead to scalability issues and potential single points of failure. Wang et al. proposed an improved beluga whale optimization (IBWO) method, resulting in enhanced prediction accuracy [
34]. However, the IBWO method may exhibit limited exploration capabilities and susceptibility to premature convergence on suboptimal solutions. The aforementioned methods either overlook the significance of the fitness function or the optimization capability of the algorithm, leading to constrained optimization of the VMD parameters.
To tackle the challenges mentioned above, a short-term electricity load forecasting method for regional smart energy systems based on two-stage feature extraction and a hybrid inverted Transformer is proposed. This study’s primary contributions and innovations include the following:
A novel hybrid method is proposed in this study, integrating Prophet, IVYVMD, the iTransformer and a Multi-Perception Temporal Convolutional Network (MPTCN) to enhance the precision of short-term load forecasting in regional smart energy systems. Specifically, the hybrid method achieves forecasting accuracies exceeding 97% for a six-hour forecasting horizon.
A two-stage feature extraction (TFE) approach, utilizing Prophet and IVYVMD, is employed to construct a feature matrix encompassing trends, seasonality, events and multiple periodic components within residuals. This method generates ample features for subsequent model predictions, effectively reducing data volatility and complexity. Notably, this approach surpasses other feature extraction methods, demonstrating superior model prediction accuracy.
Leveraging the IVY algorithm to fine-tune VMD parameters can improve the feature extraction ability of VMD and optimize model performance in short-term load forecasting, thereby enhancing load forecasting precision. Specifically, the IVY algorithm surpasses other commonly used optimization algorithms in fine-tuning VMD parameters, resulting in superior prediction performance.
The experimental analysis in this study employs the dataset of the Elia grid in Belgium, utilizing four evaluation metrics and comparing 21 related models. The findings demonstrate that the proposed approach surpasses other models in both accuracy and consistency for load predictions.
In this paper,
Section 1 offers a detailed summary of the academic background, shedding light on the latest advancements in the research concerning short-term load forecasting.
Section 2 delineates the overarching framework of the TFE-iTransformer-MPTCN method proposed in this research, elucidating the six primary methods, namely Prophet, VMD, IVY algorithm, iTransformer, TCN and MLP, in intricate detail.
Section 3 conducts a comparative analysis of various benchmark models, demonstrating the overall outstanding performance of the short-term load forecasting model proposed.
4. Conclusions and Discussion
This paper introduces a novel short-term load forecasting method for regional smart energy systems. The approach leverages a two-stage feature extraction process and a hybrid iTransformer model to enhance forecasting accuracy. The model incorporates Prophet for initial feature extraction, IVYVMD for subsequent feature extraction and iTransformer-MPTCN for modeling. Experimental results using load data from the Elia grid in Belgium validate the effectiveness of the proposed method, achieving a remarkable prediction accuracy of 98.138% and outperforming 21 comparative models.
The two-stage feature extraction module effectively captures essential patterns such as trends, seasonality and holidays, thereby improving the model’s ability to learn significant features for accurate short-term load forecasting. The use of Prophet for the first-stage feature extraction demonstrates superiority over the commonly employed STL method, particularly in extracting holiday and specific periodic features at yearly, weekly and daily intervals. In addition, the application of the VMD method for the second-stage feature extraction outperforms other signal decomposition techniques by avoiding modal mixing and accurately extracting periodic patterns with varying frequencies from the residual component. The IVY algorithm employed for VMD parameter optimization surpasses other optimization methods, leading to enhanced decomposition of the residual component from the first-stage feature extraction. Moreover, the iTransformer-MPTCN model, with its robust feature extraction and pattern learning capabilities, effectively captures both short- and long-term dependencies within the input data matrix. This characteristic results in improved forecasting performance compared to the baseline iTransformer model. In summary, the comprehensive approach combining two-stage feature extraction and the hybrid iTransformer model outperforms existing Transformer-based deep learning models, showcasing a high level of accuracy in short-term load forecasting.
Despite the impressive performance of the proposed model, its structural complexity may present significant challenges for practical implementation, requiring substantial computational resources. Additionally, the parameter tuning for both the Prophet and IVY algorithms necessitates careful consideration. Future work will aim to refine the model, enhancing its efficiency while preserving high forecasting accuracy. We will also explore more effective feature extraction methods that demand minimal parameter tuning and exhibit greater adaptability to varying input load data. The proposed model will also be tested in different regions of smart energy systems to fully represent general applicability. These improvements will be crucial for enhancing the model’s practicality.