Hybrid LSTM–BPNN-to-BPNN Model Considering Multi-Source Information for Forecasting Medium- and Long-Term Electricity Peak Load

Abstract

Accurate medium- and long-term electricity peak load forecasting is critical for power system operation, planning, and electricity trading. However, peak load forecasting is challenging because of the complex and nonlinear relationship between peak load and related factors. Here, we propose a hybrid LSTM–BPNN-to-BPNN model combining a long short-term memory network (LSTM) and back propagation neural network (BPNN) to separately extract the features of the historical data and future information. Their outputs are then concatenated to a vector and inputted into the next BPNN model to obtain the final prediction. We further analyze the peak load characteristics for reducing prediction error. To overcome the problem of insufficient annual data for training the model, all the input variables distributed over various time scales are converted into a monthly time scale. The proposed model is then trained to predict the monthly peak load after one year and the maximum value of the monthly peak load is selected as the predicted annual peak load. The comparison results indicate that the proposed method achieves a predictive accuracy superior to that of benchmark models based on a real-world dataset.

1. Introduction

1.1. Background

Electricity peak load forecasting is critical for the safe and economic operation of power systems, electricity trading market, and power generation maintenance and expansion planning [1,2]. Balancing supply and demand during peak periods when electricity demand is the highest is critical. With the development of technology, many flexible resources including interruptible loads, electric vehicles, and energy storage systems allow independent system operators (ISOs) to develop demand response (DR) programs to mitigate the peak demand. DR programs ensure the security of power systems by curtailing peak loads or shifting some power demand from on-peak periods to valley periods. Customers participating in DR programs are compensated according to the amount of electricity demand reduction. However, a key challenge in the development of DR programs is forecasting an accurate baseline given by the short-term load forecast model. This baseline is an estimate of the power consumed by customers without implementation of a DR program [3]. Different from the purpose of short-term peak load forecasting, which is useful for controlling and scheduling power generation for generators [4,5], accurate medium- and long-term peak load forecasting allows ISOs to determine the generation capacity required to satisfy future demand, address various vulnerabilities in advance, and develop reliable and economic DR programs. Failure to accurately forecast peak loads can lead to over-investment for the construction of power equipment or cause instability in supply and demand [6]. Therefore, we focused on medium- and long-term load forecasting for accurate peak load prediction information in the coming year.

1.2. Literature Review

Numerous models proposed for forecasting peak load can be categorized into three categories [7], namely, (1) time series, (2) regression, (3) and machine- or deep-learning models based on the artificial intelligence algorithm. In most time-series models, including the Holt–Winters model based on exponential smoothing [8], autoregressive integrated moving average (ARIMA) [9,10], and seasonal autoregressive integrated moving average (SARIMA) [11,12], future load is predicted by considering only the variation pattern of the load itself without any exogenous variables based on the historical data. Future loads were formulated as a linear combination of their own values from previous time steps [6]. These models are well established and widely used because of their simplicity and low amount of historical load data required. However, if the order of the mathematical models, which is used to transform nonstationary data into stationary data through differencing, is not appropriately selected, the predictive performance of these models becomes poor.

Electricity load is not only related to its historical load data but also influenced by many exogenous variables, such as meteorological conditions, population information, and economic variables. Therefore, in various regression models, including linear regression [13], multiple regression [14,15,16], and exponential regression [17], these variables are considered for load forecasting. In these methods, future loads are predicted by establishing a mathematical relationship between affecting factors and load values. However, because the relationship between the power load and the influencing factors is complex and nonlinear, expressing a relationship between them with a precise relational equation is difficult.

Numerous studies based on machine-learning and deep-learning approaches have been conducted for forecasting the peak load. Park et al. [18] proposed an artificial neural network (ANN) for forecasting electricity load. The ANN [19,20] exhibits a superior ability to extract nonlinear characteristics of the peak load data and achieves excellent predictive performance compared with those of conventional regression models. In addition to the neural network, the support vector regression (SVR) model [21] aided by the kernel function can be used in learning the nonlinear mapping relationship between input and output data, which can improve predictive capacity for the peak load. Furthermore, the deep-learning algorithm, that is, long short-term memory (LSTM) [22], has been applied for predicting peak load, and the LSTM outperformed the conventional time series model because the model could capture the temporal dependencies of time-series data.

Hybrid models combining elements of multiple models have recently been applied to peak load forecasting. Ren et al. [23] proposed a CNN–LSTM hybrid model that applied the convolution layer of convolutional neural network (CNN) to capture the features of power load data and predicted the load using the unique cellular structures of LSTM. It was validated that the ARIMA–SVM hybrid model outperformed the ARIMA and SVM models individually in terms of predictive performance in the literature [24]. Jiang et al. [25] proposed a novel hybrid multitask multi-information fusion deep-learning framework (MFDL) to learn recent electricity usage behavior by LSTM and long-term regular electricity behavior features by CNN. It was validated that a hybrid forecasting method combining LSTM and neural prophet through an ANN had better predictive power than either individual model in the literature [26]. From the majority of previous research, it can be concluded that hybrid models have better predictive power than their constitutive models, because they can learn different types of data and combine the advantages of multiple models.

1.3. Contributions

The predictive accuracy performance of the aforementioned methods is hindered by some limitations. Xia et al. [27] reported the difficulty in accurate forecasting because factors are not stably random and the relationship between the peak load and other exogenous variables is complex and nonlinear. Thus, accurately predicting the future peak load using conventional time-series and regression models is difficult. Additionally, inappropriate parameters and kernel function could considerably reduce the predictive performance of the SVR model. Furthermore, a simple ANN model could not distinguish variables across time steps and extract available information from those time-series data. Annual data are insufficient for training a machine- or deep-learning model. Here, to achieve superior prediction of peak load in the coming year, we propose a new method. The primary contributions of this article are as follows.

(1)

We propose a hybrid model, named LSTM–BPNN-to-BPNN, that combines LSTM and a back propagation neural network (BPNN) model to extract the features of the historical data and future information, respectively. The output features of these models are concatenated to a vector and subsequently inputted into the next BPNN model to obtain the predicted peak load. In this method, we use targets set by the government for the pace of the economic development as future information to improve predictive performance.

(2)

To overcome the insufficient annual data problem for training the forecast model, we consider using an even smaller time scale of data; that is, we mine the correlation of peak load data on a monthly time scale. The monthly peak load is forecasted by the proposed model, and the maximum value selected as the final predicted annual peak load.

(3)

Moreover, we consider and analyze the characteristics of the peak load. According to the analysis, the indicators of the statistical monthly load characteristics were added to the input feature of the load forecast model and only the monthly peak load with large load magnitudes from April to October were predicted each year for superior predictive performance. The proposed model achieved superior predictive performance compared with other benchmark models in terms of the mean absolute percentage error based on the real-world dataset.

The remainder of this article is organized as follows. Section 2 and Section 3 introduce input feature extraction and data preparation, respectively. In Section 4, the hybrid LSTM–BPNN-to-BPNN model is introduced. Section 5 presents experiments and results to detail the predictive performance of the proposed model based on the real-world dataset. Finally, conclusions of this study are presented.

2. Input Feature Extraction

Numerous models with excellent predictive performance have been proposed to forecast medium- and long-term peak load. However, the quality of the model input data may considerably affect the predictive performance of the model. Therefore, analyzing the characteristics of the peak load and selecting appropriate features for accurately forecasting the load are critical.

2.1. Analysis of Peak Load Characteristics

Figure 1 displays the daily peak load curve of a city in China from 2012 to 2021. The peak load exhibits a similar periodicity from year to year and differs only in the magnitude of the loads, because people consume considerable electricity as part of their production activities and lifestyle. Furthermore, the daily peak load series not only exhibits a periodicity trend, but also seasonal characteristics: the daily peak load reaches local maximums in summer and winter, whereas the local minimum loads typically occur in spring and autumn.

We use the fast Fourier transform (FFT) algorithm as a periodic detection method to check and verify the periodicity and seasonal characteristic of peak load [28]. A total of 3653 daily peak loads from 2012 to 2021 were collected to construct time-series data. The time-series data consist of several signals with different amplitudes and periods. FFT was used to convert this time series from the time domain to the frequency domain and obtain the individual frequencies and the dominant frequency for signals. The frequency spectrum results reveal that the dominant frequency is 0.0027382 Hz, which means the corresponding dominant period is 365 days. This reflects the periodicity characteristic of peak loads. Furthermore, the next three most dominant frequencies are f1 = 0.0002738 Hz, f2 = 0.0005476 Hz, and f3 = 0.0109529 Hz, which correspond to periods of t1 = 3652 days, t2 = 1826 days, and t3 = 91 days, respectively. The period of 91 days reflects the seasonal characteristic of peak loads. This is consistent with the conclusions obtained from our visual observations.

We count the number of times the 3% peak load occurred in a year from 2012 to 2021. These loads are defined as loads that exceed 97% of the annual maximum load magnitude. As displayed in Figure 2, the load magnitude reached high levels from May to September each year and the annual peak load occurred in July or August every year.

The analysis reveals that the peak load is periodic and seasonal. Thus, peak loads are correlated not only between years but also between the months of the same year. Therefore, historical peak load data are essential input for peak load forecasting. However, because annual data are insufficient to train a forecast model, a smaller time scale of data should be used for mining the correlation of data on other time scales and improving load predictive performance. The periodic and seasonal characteristics of the peak load reveal that some correlation may exist between monthly data. Considering monthly data as input data for the medium- and long-term load forecast model is more appropriate than other smaller time scales. Monthly peak loads are first predicted by the load forecast model, and the maximum value of these predicted monthly peak loads is then selected as the predicted annual peak load.

To elaborate the characteristics of the monthly peak load, we introduce six indicators: (a) monthly peak load, which is defined as the maximum value of the daily load for a month; (b) monthly average daily load, which is the average of the daily load for a month and reflects the average level of electricity consumption; (c) monthly maximum peak–valley difference, which is the maximum value of the difference between the daily peak load and valley load for a month and reflects the peak load regulation capacity of the grid; (d) monthly average load rate, which is the average of the daily load rate (the ratio of daily average load to daily maximum load) for a month and reflects the degree of stability of load change; (e) monthly minimum load rate, which is the minimum value of the daily minimum load rate (the ratio of daily minimum load to daily maximum load) for a month and represents the magnitude of the load change; and (f) monthly average peak–valley difference rate, which is the average of the daily peak–valley difference rate (the ratio of daily peak–valley difference to daily maximum load) for a month and reflects the relative value of the peak load regulation capacity of the grid. As displayed in Figure 3, the pattern presented by the monthly load characteristics is consistent with the monthly peak load. Therefore, these monthly load characteristic indicators exhibit a high correlation with the monthly peak load. These indicators are essential input data for peak load forecasting.

2.2. Feature Selection

The machine-learning model is used to predict loads by learning the mapping relationship between model input and output data in a sample set. Therefore, forecast accuracy can be improved by selecting the critical variables by analyzing the correlation between input and output variables before load forecasting. According to the analysis of peak load characteristics in Section 2.1, in addition to the monthly peak load, five load characteristic indicators were selected as input variables, as listed in

Table 1. Input variables related to the peak load.

Variable Type	Name of Variable	Number of Input Variables
Load characteristic indicators	Monthly peak load Monthly maximum peak–valley difference	6
	Monthly average daily load
	Monthly average peak–valley load rate
	Monthly average daily load rate
	Monthly minimum load rate
Meteorological indicators	Monthly average maximum temperature	4
	Monthly maximum temperature
	Monthly maximum average temperature
	Monthly average minimum temperature
Economic indicators	Monthly GDP	4
	Monthly primary industrial added value
	Monthly secondary industrial added value
	Monthly tertiary industrial added value
Total		14

.

Moreover, the seasonal characteristics of the peak load reveals high correlation between electricity consumption and weather condition in all seasons. Meteorological factors including temperature, humidity, wind speed, precipitation, and solar radiation considerably influence electricity usage. The temperature can be easily measured and accessed compared with other weather variables. Cooling loads and heating loads increase loads in summer and winter, respectively, when the temperature becomes too high or low. The changes in the temperature considerably affect the peak load. Therefore, the temperature factor is an input feature of the peak load forecast model. Thus, four meteorological indicators were selected as input variables, as listed in Table 1.

The increase in the load is typically associated with the level of economic development for improving the quality of life. Economic information published quarterly or annually can be easily obtained from national and local statistics bureau websites. Therefore, economic variables should be the influential input features for accurate medium- and long-term load forecasting. Furthermore, governments typically set appropriate targets for the pace of the economic development in the coming years, which may considerably increase the load. Future information should be considered for peak load forecasting. Thus, four economic indicators were selected as input variables, as listed in Table 1.

The correlation between peak load and input variables is widely used to measure the importance of the features. The Pearson correlation coefficient is typically used to examine the linear correlation degree between two variables. The Pearson coefficient between each possible input variable and peak load is calculated and illustrated in Figure 4. The 14 variables in Figure 4 are in the same order as listed in Table 1. As displayed in Figure 4, the first row or column shows the correlation between monthly peak load and other input variables. The darker the green color of the cell, the greater the Pearson coefficient. Thus, variables in darker green exhibit a higher linear correlation relationship. In our paper, the criteria for how to use the Pearson correlation coefficients to select important features are the same as those used by Cho et al. [29]. That is, a Pearson correlation coefficient of 0.7 or more indicates a strong linear correlation, so variables with a Pearson correlation coefficient of 0.7 or higher are selected.

The greater Pearson coefficients between monthly peak load and load characteristic indicators show that the variation patterns are highly consistent (Figure 3). Furthermore, the economic variables are highly related to the monthly peak load. The smaller Pearson coefficients between monthly peak load and temperature variables exhibit weaker correlation compared with other variables. After Pearson correlation analysis, the load characteristic indicators and economic indicators are selected as input features. However, this does not mean temperature variables are not critical for peak load forecasting. Therefore, other criteria can be used to further assess the importance of features.

The extreme gradient boosting (XGBoost) algorithm is widely used to measure the importance of the features [30,31]. The candidate variables that affect peak load are considered as inputs, and the monthly peak load is the output of the XGBoost model. Figure 5 details the F score, which represents the feature importance, of the 14 input variables in Table 1. These scores are determined by the number of times that a feature is used to split the data across all trees. Features with higher F scores contribute more to predictive performance in the training course of the XGBoost algorithm. Variables whose F score is less than all the selected input features are considered redundant and are not selected as input features [32]. Therefore, in this study, variables whose F score is less than the F score of the variable “SIAV” (monthly secondary industrial added value) are not selected as input features.

The XGBoost feature importance results show that the meteorological indicators are more important than some of the economic indicators selected as input features. This means the input vector to peak load forecast model should include the meteorological indicators as features. Moreover, the F scores of all the meteorological indicators are higher than the cut-off F score of 4.0. Hence, the meteorological indicators are selected as input features. After correlation analysis and feature importance assessment, 14 variables (Table 1) are considered as the input features of the peak load forecast model.

3. Data Preparation

According to the aforementioned analysis, the magnitude of peak load is not only correlated with historical load value but also with external variables, namely, meteorological and economic variables. Therefore, these variables were considered as the input features of the medium- and long-term peak load forecast model. As the economic and meteorological variables always exhibit distinct units of time scales according to the information collected from public websites, these variables should be converted into the same unit of time scale before load forecasting. All collected data are converted into the monthly data.

The load and temperature data can easily be obtained on smaller time scales than the month. The monthly load and temperature data can be collected by statistically analyzing smaller time scales of data. For example, the monthly peak load is defined as the largest daily peak load in a month, and the monthly average highest temperature is obtained by the average of the daily highest temperature in a month. However, economic information, such as gross domestic product (GDP), is provided by season and year. Therefore, estimating monthly economic data based on quarterly or annual data should be investigated. A study [33] used the monthly industrial production index released by the Statistical Office to convert the quarterly GDP into monthly GDP. Subsequently, the weekly GDP was estimated by dividing the monthly GDP equally by the number of weeks in a month. Because the weekly GDP is affected by various factors, such as holidays, this average distribution method is not an accurate estimate of weekly GDP, which results in increasing the error of load forecasting. In one study [29], the nominal monthly GDP data were estimated from annual data by the curve fitting interpolation method. Although the curve fitted by this interpolation method is consistent with the trend in the annual data, the trend in the monthly data of various years may vary. This method did not fully utilize the advantage of economic data on smaller time scales for estimating monthly data. Therefore, to address the shortcomings of the aforementioned studies, the monthly industrial added value above designated size and the quarterly economic variable, such as GDP, released by the website of a Chinese city’s Statistics Bureau were used to estimate the monthly economic variable as follows:

$$mGD{P}_i=qGD{P}_j\times pGD{P}_{ij}(i=1,2,3\cdots \cdots 12;j=1,2,3,4)$$

(1)

$$pGD{P}_{ij}=mIA{V}_i/sIA{V}_j$$

(2)

where mGDP_i, qGDP_j, and pGDP_ij denote the GDP of month i, the quarterly GDP of season j, and the percentage of GDP of month i in season j, respectively; mIAV_i and sIAV_j denote the industrial added value above designated size of month i and the sum of the industrial added value above designated size of all months in season j, respectively.

Because economic growth is accompanied by industrial growth, the monthly GDP estimation is obtained by calculating the percentage of the monthly GDP from the monthly industrial added value above designated size.

4. Hybrid LSTM–BPNN-to-BPNN Model

4.1. Structure of the Proposed Model

Developing a model that can adequately extract the features of these data is critical for accurately predicting annual peak loads. Therefore, we propose a hybrid LSTM–BPNN-to-BPNN model combining LSTM and BPNN for predicting the annual peak load. The structure of the proposed model is displayed in Figure 6.

Analysis of the characteristics of peak load revealed that peak loads affected by the temperature and other factors are periodic and seasonal. For learning and forecasting such time-series data, the LSTM model can be applied to extract the evolution pattern of historical peak loads. However, because of the large time span of annual peak load in different years, accurately forecasting the future annual peak load only through learning the temporal correlation of historical data by using deep learning is not possible. Therefore, future information should be considered for medium- and long-term peak load forecasting.

National and local governments set targets for encouraging economic development in the coming years. These targets may affect the increase in the load and can be used to estimate economic data as the future information of the forecast months. As the peak load is related to the exogenous factors as discussed in Section 2, the BPNN can be applied to extract the feature of future information for improving the accuracy of peak load forecasting.

Therefore, using a hybrid model combining the advantages of LSTM and the BPNN model to learn the historical data and future information, respectively, will improve the predictive performance of the peak load forecasting model. The LSTM can be applied to learn the temporal correlation of historical data and extract the features of historical time-series data. Additionally, the BPNN exhibits superior ability to extract nonlinear characteristics of non-time-series data, which can be applied to extract the features of future information. Then, the output features of these models can be taken as the input features of another BPNN to predict the peak load. This is why we propose a hybrid LSTM–BPNN-to-BPNN model combining LSTM and BPNN for predicting annual peak load.

As displayed in Figure 6, the overall process of applying the proposed method to medium- and long-term peak load forecasting is as follows. (1) Collect the historical load, meteorological, and economic data, which are distributed over various time scales. Furthermore, future information including economic data and temperature data should be estimated or predicted. (2) Prepare data, analyze their characteristics, and convert all data into the same monthly time scale. (3) Apply the LSTM and BPNN to extract the features of the historical data and future information, respectively. (4) Concatenate the outputs of the LSTM and BPNN to a vector, and subsequently enter it into the next BPNN to obtain the predicted monthly peak load. (5) Determine the annual peak load according to predicted monthly peak loads.

4.2. LSTM

The LSTM model [34] proposed by Hochreiter and Schmidhuber is a special recurrent neural network (RNN) model that can be used to investigate the temporal relationship of time-series data. When the length of time-series data is long, the conventional RNN model exhibits gradient explosion or gradient vanishing during backpropagating, which reduces learning performance. Thus, the gate mechanism was introduced to address this shortcoming. The LSTM architecture is illustrated in Figure 7.

Unlike the typical RNN model, the cell-state and hidden-state are passed between adjacent LSTM blocks. The LSTM block consists of three gates, namely, input gate $i_t$, forget gate $f_t$, and output gate $o_t$, to determine previous information that should be updated, erased, or maintained. Each LSTM block takes the previous cell-state $c_{t-1}$, hidden-state $h_{t-1}$, and current input information $x_t$ as input data and subsequently outputs current cell-state $c_t$ and hidden-state $h_t$ to next LSTM block. Each LSTM internal node can be formulated as follows:

$$f_t=\sigma \left(W_{xf}{x}_t+W_{hf}{h}_{t-1}+b_f\right)$$

(3)

$$i_t=\sigma \left(W_{xi}{x}_t+W_{hi}{h}_{t-1}+b_i\right)$$

(4)

$$o_t=\sigma \left(W_{xo}{x}_t+W_{ho}{h}_{t-1}+b_o\right)$$

(5)

$$c_{-}{\mathrm{in}}_t=\tanh \left(W_{xc}{x}_t+W_{hc}{h}_{t-1}+b_{c_{-}\mathrm{in} }\right)$$

(6)

$$c_t=f_t\odot c_{t-1}+i_t\odot c_{-}{\mathrm{in}}_t$$

(7)

$$h_t=o_t\odot \tanh \left(c_t\right)$$

(8)

where W and b are the weight matrices and bias vector parameters between neurons, respectively, which should be learned during training; $\odot$ represents the Hadamard product (the corresponding elements of the matrix are multiplied together); $\sigma$ is the sigmoid activation function, which can compress the input value to the range (0, 1).

4.3. BPNN

BPNN is a multi-layer feed forward neural network trained by the backpropagation algorithm based on the gradient descent. A three-layer BPNN consisting of an input layer, a hidden layer, and an output layer are displayed in Figure 8. Each layer consists of a certain number of neuron nodes, and the nodes between adjacent layers are fully connected. The strength of connections between neuron nodes in each layer is represented by weights matrices and thresholds.

The BPNN model training involves two components, namely, the positive propagation of information and the reverse propagation of error. In positive propagation, information is passed layer-by-layer from the input layer to the output layer. If the error between the output value and the target value exceeds the goal setting range in terms of the mean square error, the weights and bias parameters are updated according to the reverse error. The output value of each neural node from the current layer is calculated by the following formulations:

$$I_j={\displaystyle \sum w_{ij}}\cdot x_i+b_j(i=1,2,3,\dots ,m;j=1,2,3,\dots ,n)$$

(9)

$$y_j=f\left(I_j\right)(j=1,2,3,\dots ,n)$$

(10)

where m and n denote the number of the neurons of the current and the previous layers, respectively; $w_{\mathrm{ij}}$ represents the connection weights of the previous layer and current layer and $b_j$ is the threshold value of the current layer; $x_i$ and $I_j$ denote the output value of neuron from the previous layer and the input value of neuron from the current layer, respectively; $y_j$ is the output value of neuron from the current layer; and f is an activation function enabling neural networks with the ability to learn the nonlinear mapping relationship between the input and output of the model.

4.4. Dataset Description

The proposed model is used to forecast the monthly peak load and, subsequently, the final predicted annual peak load is derived from predicted monthly peak loads. Therefore, the deep-learning model is trained using monthly data. Time-series data construction varies with the periodicity and seasonality of the peak load. The monthly peak load to be predicted is correlated with the same month data for years before or with the monthly data before the predicted month from nearest year. According to this analysis, the corresponding time-series samples can be constructed according to the time channels. Figure 9 displays the process of constructing time-series samples according to the annual and monthly time channels.

Figure 9a indicates that the data on the same month from one year before the monthly data to be predicted, including the historical peak load, temperature, and economic data, and the predicted monthly economic and temperature data are used as the input data. The predicted monthly peak load is the target value to the output of the model. In this case, the time-series length in the year time channel is one. The length of the year time series can be increased to determine the number of years of historical data for the same month, which can be used to better forecast the monthly peak load. Figure 9b displays that the historical data for the same month and the previous month of the year preceding the forecasting month and the predicted monthly economic and temperature data are used as the input data. The predicted monthly peak load is the label to the output of the model. In this case, the time-series length in the month time channel is two. The length of the month time series can be increased to determine the number of months of historical data of a year before, which can be used to predict the monthly peak load.

5. Experiments and Results

To verify the predictive performance of the proposed algorithm, the real-world data of a city in China from 2012 to 2021 were used to perform the numerical experiment. The data were categorized into training, validation, and test sets. The data from 2012 to 2019 were used for model learning, and the hyperparameters of the model were tuned by analyzing the accuracy of the model on the validation dataset in 2020. Evaluation experiments of the forecasting performance of the model were performed on the test dataset in 2021.

5.1. Experiment Setup

1

Simulation environment

The experiments were conducted in a Dell Precision 3650 Tower with Intel^® Core^TM i7-11700K [email protected] GHz and 32 GB RAM.

2

Implementation of the hybrid LSTM–BPNN-to-BPNN model

The proposed model was designed on the Pytorch platform, and the adaptive moment estimation [35] optimization algorithm was used to train the model. The hyperparameters of the hybrid model primarily include the number of layers of each model, the number of neurons per layer, and the dimension of the input layer and output layer. The dimension of the input layer is equal to the number of input variables; that is, the dimensions of the input layer of the LSTM and the BPNN models used for feature extraction were set to 14 and 8, respectively. The dimensions of their output layer were both set to 1. The dimension of the input layer of the BPNN model to be used for prediction was equal to the dimension of the concatenated vector and the number of neurons of its output layer was set to 1. The remaining parameters were determined using a grid search [6,36]. The main hyperparameters of the proposed hybrid model are shown in

Table 2. Hyperparameters of proposed hybrid model.

Hyperparameter	LSTM Part	BPNN Used for Feature Extraction	BPNN Used for Prediction
Number of hidden layers	3	2	2
Number of hidden units per layer	32	16	8
Learning rate	0.001
Number of epochs	200

.

3

Data preprocessing

Because forecast model performances are sensitive to the size of the input data, and the input variables consist of various variables, such as the raw temperature, economic data, and peak loads, normalization should be performed to improve predictive performance. The min–max normalization method [37] was applied to transform the size of the input data to the range [0, 1].

4

Evaluation metrics

The predictive performance of the proposed model was compared with some benchmark models using the mean absolute percentage error (MAPE), which is widely used to calculate prediction errors as follows:

$$MAPE(\%)=\left|\frac{y_i-{\widehat{y}}_i}{y_i}\right|\times 100\%$$

(11)

where $y_i$ is the true actual peak load value, and ${\widehat{y}}_i$ is the predicted peak load value.

5.2. Case Study and Performance Evaluation

Three case studies were performed to evaluate the predictive performance of the hybrid LSTM–BPNN-to-BPNN model. The details of each case study are presented in

Table 3. Details of each case study.

Case	Load Features		Meteorological Features	Economic Features	Time Range of Monthly Peak Load to Be Predicted
	Monthly Peak Load	Five Designed Load Characteristic Indicators
Case 1	√	×	√	√	January to December
Case 2	√	×	√	√	April to October
Case 3	√	√	√	√	April to October

. In Case 1, in terms of input load features, historical monthly peak load was considered to be the input load variable without adding the other five designed load characteristic indicators listed in Table 1 for predicting the monthly peak load for all months in the coming year. In Case 2, we used the same input variables as the model in Case 1, but only the monthly peak loads with large load magnitudes from April to October are predicted by the model in this case. In Case 3, in addition to the monthly peak load, meteorological, and economic features used in Case 2, the other five load characteristic indicators were introduced to the input load features for forecasting the monthly peak load for the same month in Case 2.

(1)

Case 1

Most medium- and long-term peak load forecasting methods only require historical peak load values as the input feature of the model without using other load characteristic indicators. Therefore, first, we disregard adding five load characteristic indicators as input variables for peak load forecasting. Thus, only historical monthly peak loads, temperature variables, economic variables, and the predicted monthly temperature and economic variables were considered as the input to the model. We set various annual time-series lengths and monthly time-series lengths to predict the peak load using the proposed model. The predictive performance of the model is listed in

Table 4. Predictive performance of the proposed model with various time-series lengths.

Time Channel	Time-Series Length	True Value of Annual Peak Load in 2021 (MW)	Predicted Value of Annual Peak Load in 2021 (MW)	MAPE (%)
Yearly	1	20,228	19,156.92	5.29%
	2		19,269.17	4.74%
	3		19,311.91	4.52%
	4		19,394.33	4.12%
Monthly	1		19,156.92	5.29%
	2		19,269.17	5.02%
	3		19,671.85	2.74%
	4		19,429.22	3.94%

. Figure 10 details the best forecast results in this case. The annual peak load forecasting value is derived from the results of the monthly forecast model.

The model trained based on data with a monthly time-series length of 3 achieved the best predictive performance. This model did not predict the monthly peak load for 12 months, because the length of the monthly time series is not 1. The monthly peak load forecast results reveal that the model exhibits a large error in predicting the monthly peak load with a smaller magnitude. These phenomena are marked with green boxes in Figure 10a. Because the maximum value is obtained from the predicted monthly peak load as the annual peak load, the prediction error of the model is small if the model is highly accurate in predicting the monthly peak load with larger magnitude.

(2)

Case 2

Analysis of the results in Case 1 revealed that the monthly peak load with smaller magnitude had a limited effect on the results of the annual peak load forecast. The analysis of characteristics of the peak load in Section 2 revealed that the load magnitude reached high levels from May to September each year. Therefore, the number of monthly peak loads should be reduced for the forecast. However, considering that a sufficient sample is required to train the model, in this case, the monthly peak load is predicted with large load magnitudes from April to October each year. The predictive performance of the model trained based on monthly data from April to October is listed in

Table 5. Predictive performance of the proposed model trained based on monthly data from April to October with various time-series lengths.

Time Channel	Time-Series Length	True Value of Annual Peak Load in 2021 (MW)	Predicted Value of Annual Peak Load in 2021 (MW)	MAPE (%)
Yearly	1	20,228	19,733.99	2.44%
	2		20,666.67	2.17%
	3		19,708.07	2.57%
	4		19,715.39	2.53%
Monthly	1		19,733.99	2.44%
	2		19,671.63	2.75%
	3		19,458.12	2.81%
	4		19,689.33	2.66%

, and Figure 11 details the best forecast results in this case.

The proposed model trained based on monthly data from April to October yielded superior predictive performance compared with results of training with all monthly data, because the distribution pattern of the sample data with higher load magnitude is closer to the variation pattern of the peak load, which improves the prediction accuracy of the peak load. Moreover, the proposed model trained based on time-series data constructed in the yearly time channel is more accurate than that in the monthly time channel in predicting peak load, which implied that the monthly peak load to be forecasted is highly correlated with data on the same month from years before.

(3)

Case 3

To investigate the effectiveness of adding the monthly load characteristic indicators as the input feature for peak load forecasting, the indicators to the input features were used to train the model. The performances of the two proposed models are listed in

Table 6. Comparison of the proposed model with or without using the designed feature.

Model	True Value of Annual Peak Load in 2021 (MW)	Predicted Value of Annual Peak Load in 2021 (MW)	MAPE (%)
Model A	20,228	20,666.67	2.17%
Model B		19,929.61	1.47%

. Model A represents the proposed model trained without adding the designed features as the input, whereas model B is trained considering the indicators as the input features. Each model is trained with optimal parameters. The results revealed that model B exhibits an improvement over model A, which details the importance of the designed load characteristic indicators for peak load forecasting.

5.3. Comparison between the Proposed Model and Other Benchmark Models

To verify the superiority of the predictive performance of the proposed method, we compared the model with other benchmark models. To ensure fairness, the most appropriate input and parameters were selected for the benchmark models. Python-based implementation was carried out with the statsmodels, sklearn, and Keras packages to construct benchmark models. The main hyperparameters for benchmark models are shown in

Table 7. Hyperparameters of the benchmark models.

No.	Model	Hyperparameters
1	ARIMA	Number of lags, the orders of autoregressive and moving average terms: (d, p, q) = (2, 1, 0)
2	SVR	Kernel = ’rbf’, regularization parameter C = 800, admissible error ε = 0.01, stopping criterion tol = 0.001
3	BPNN	Weight optimization = Adam, number of epochs = 200, number of hidden layers = 2, number of hidden units per layer = 32, learning rate = 0.005
4	LSTM	Weight optimization = Adam, number of epochs = 200, number of hidden layers = 3, number of hidden units per layer = 32, learning rate = 0.001

, except for MR, which does not possess a hyperparameter space to enhance its predictions.

A brief description of each benchmark model is as follows:

(1)

ARIMA: The autoregressive integrate moving average method is a conventional time-series analysis approach trained based on only the historical annual peak loads for predicting the future annual peak load.

(2)

MR: The multiple regression method forecasts the annual peak load by fitting the quantitative relationship between the external influencing factors and the annual peak load in the historical data. The external factors are primarily economic factors.

(3)

SVR: The support vector regression fits the relationship between external factors and the annual peak load by mapping the feature vector into the high-dimension space using the kernel function.

(4)

BPNN: The backpropagation neural network trains the parameters of the neural network through the backpropagation algorithm to learn the mapping relationship between the input and output of the model. Because the ANN does not distinguish time-series variables, it cannot capture temporal dependencies. Annual data are insufficient to train the forecast model. Therefore, the BPNN is trained based on the same monthly data as the proposed model without using future information.

(5)

LSTM: The LSTM considers temporal dependencies. To verify the effectiveness of future information for medium- and long-term peak load forecasting, the LSTM is trained based on the same monthly data as the proposed model without using future information.

Table 8. Comparison of predictive performance of six models in the forecasting annual peak load in 2021.

Model	True Value of Annual Peak Load in 2021 (MW)	Predicted Value of Annual Peak Load in 2021 (MW)	MAPE (%)
ARIMA	20,228	19,291.87	4.63%
MR		19,715.95	2.53%
SVR		20,640.12	2.04%
BPNN		19,400.18	4.09%
LSTM		19,510.49	3.55%
LSTM–BPNN-to-BPNN		19,929.61	1.47%

and Figure 12 detail a comparison between the proposed model and benchmark algorithm for annual peak load forecasting. The comparison of predictive performance reveals that LSTM–BPNN-to-BPNN outperformed other benchmark models in predicting the annual peak load with MAPE 1.47%. Because the selected city exhibits rapid economic development, which consequently considerably affects the growth in the electricity load, the ARIMA model exhibits the worst performance among all models because the model depends only on the historical peak data. Furthermore, learning time-series data is difficult using the BPNN model, which renders learning the peak load variation pattern difficult for accurately predicting future peak loads. The proposed model exhibits a distinct improvement over the LSTM, which emphasizes the importance of considering future information for medium- and long-term peak load forecasting. Moreover, the proposed model outperforms the SVR and MR model in terms of predictive performance, which is widely used for medium- and long-term load forecasting. Therefore, the proposed model is effective and accurate in medium- and long-term peak load forecasting.

6. Conclusions

A hybrid LSTM–BPNN-to-BPNN model was proposed for predicting annual peak load after a year. First, the peak load characteristics were analyzed in detail and appropriate input variables were selected before load forecasting. Unlike conventional medium- and long-term forecasting methods, in addition to historical load data, temperature, and economic variables, future information was considered for improving predictive performance. The LSTM and BPNN were applied separately to extract the features of the historical data and future information.

To verify the effectiveness of the proposed model, we compared the model with several benchmark models, namely, the ARIMA, MR, SVR, BPNN, and LSTM, based on a real-world dataset. The comparison results indicated that the proposed model outperformed the benchmark models in terms of the MAPE. Furthermore, the added load characteristic indicators based on the analysis of peak load characteristics reduced predictive error. Finally, future research could improve the forecasting performance of the LSTM–BPNN-to-BPNN model by determining more related predictive variables to apply to the model.