Robust Wavelet Transform Neural-Network-Based Short-Term Load Forecasting for Power Distribution Networks

Abstract

A precise short-term load-forecasting model is vital for energy companies to create accurate supply plans to reduce carbon dioxide production, causing our lives to be more environmentally friendly. A variety of high-voltage-level load-forecasting approaches, such as linear regression (LR), autoregressive integrated moving average (ARIMA), and artificial neural network (ANN) models, have been proposed in recent decades. However, unlike load forecasting in high-voltage transmission systems, load forecasting at the distribution network level is more challenging since distribution networks are more variable and nonstationary. Moreover, existing load-forecasting models only consider the features of the time domain, while the demand load is highly correlated to the frequency-domain information. This paper introduces a robust wavelet transform neural network load-forecasting model. The proposed model utilizes both time- and frequency-domain information to improve the model’s prediction accuracy. Firstly, three wavelet transform methods, variational mode decomposition (VMD), empirical mode decomposition (EMD), and empirical wavelet transformation (EWT), were introduced to transform the time-domain demand load data into frequency-domain data. Then, neural network models were trained to predict all components simultaneously. Finally, all the predicted data were aggregated to form the predicted demand load. Three cases were simulated in the case study stage to evaluate the prediction accuracy under different layer numbers, weather information, and neural network types. The simulation results showed that the proposed robust time–frequency load-forecasting model performed better than the traditional time-domain forecasting models based on the comparison of the performance metrics, including the mean absolute error (MAE), mean absolute percentage error (MAPE), and root mean squared error (RMSE).

1. Introduction

To deal with the severe climate change crisis, 193 countries agreed to adopt a set of global goals to end poverty, protect the planet, and ensure prosperity for all. Of particular interest is Goal 7, which aims to provide access to clean and affordable energy. The distribution network in a power system plays the role of an intermediate that links urban areas to the transmission/sub-transmission network. The distribution network is influenced by many factors, such as the power quality; the peak load, resistance, and reactance of the distribution lines; and the distance between the transformer and the consumers (including residential, commercial, and industrial consumers). Precise distribution-level short-term load forecasting (STLF) is a vital process that helps grid operators pursue this goal since a reliable STLF system provides critical input information for demand-side management (DSM), state estimation, maintenance scheduling, voltage support, etc. Moreover, providing the precise and rapid prediction of future demand is the foundation of hourly based applications, such as electricity market-clearing mechanisms and regulation bids. However, unlike high-voltage (HV) transmission networks, which are stable and remain unchanged for long periods, STLF for distribution networks have become more challenging in recent years. The reasons for this can be summarized as follows: (1) The original distribution networks are designed as passive systems, as the electricity can only be transmitted from power plants to consumers. However, the networks have turned from passive into active bidirectional systems since more and more distribution generators (DGs) have penetrated the grid. (2) High uncertainty surrounds the low-capacity load inside distribution networks and the diversity of users’ characteristics also influences load prediction accuracy. Compared to the well-developed load-forecasting procedures at the HV network level, distribution-level load-forecasting methods are still in the exploratory stage. Fortunately, a large amount of high-resolution electricity data has been collected with the widespread installation of smart meters. These aggregated smart meter data represent a useful data resource on the demand side to more accurately forecast the demand at the distribution level.

Load-forecasting methods have been well-discussed in the literature over the last few decades. Depending on the prediction duration, load forecasting can be further divided into STLF, mid-term load forecasting, and long-term forecasting. Of all the load-forecasting categories, STLF plays the most vital role in power systems due to its importance for power stability and economic performance. STLF methods can be classified into conventional statistical methods and machine-learning-based methods. Traditional statistical methods try to establish linear equations to predict the demand load, with linear regression (LR) and autoregressive integrated moving averages (ARIMA) being the most common statistical methods. Distribution-level LR-based STLF approaches are presented in [1,2,3,4]. As a naive forecasting approach, LR (or its modification, multivariable LR) aims to identify the linear correlations between the input features and the future demand load; the loss function employed by most LR methods is the least squares method, which minimizes the Euclidean distance between the predicted and true values. ARIMA is another statistical analysis model that uses time-series data to predict the future trends of datasets. It first applies lagged moving averages to smooth the demand load data. Then, it predicts the future demand load based on the assumption that the future trend will resemble the past trends [5]. The ARIMA algorithm is combined with other statistical methods to improve prediction accuracy. The methods that have been employed in the literature to enhance the accuracy of the ARIMA model include XGBoost [6], support vector machine (SVM) [7], decision trees (DTs) [8], and the hidden Markov model (HMM) [9]. However, the characteristics of the distribution-level demand load are nonlinear and nonstationary. Additionally, due to the high level of the incorporation of distributed renewable energy sources (such as solar panels and small wind turbines) and electric vehicles (EVs) into distribution networks in recent decades, the fluctuation of load curves has become increasingly extreme and unpredictable; hence, it is difficult to use a linear function to exactly estimate the demand load.

Machine-learning-based STLF methods have shown higher prediction accuracy than traditional statistical methods, including artificial neural networks (ANNs), random forests, and SVM. ANN-based methods have achieved huge success in power system applications, such as STLF [10], solar energy forecasting [11], energy management [12], and abnormal data detection [13]. The authors of [14] proposed a three-layer ANN STLF predictive model. However, the proposed STLF model was just a naive network that could only consider a single data point at a time and it could not apply the experiences learned from the historical training process. The improved ANN-based STLF methods can be further divided into feedforward neural networks [10,15,16], backpropagation neural networks [17,18,19], recurrent neural networks [10,16], the restricted Boltzmann machine [20], and convolutional neural networks [21,22,23]. RNNs have achieved particularly high accuracy in recent decades. RNN models contain memory units that can study present input information and data from the past. This characteristic is very useful for time-series tasks such as forecasting. RNN models can be further classified as long short-term memory (LSTM) and gated recurrent unit (GRU) models. In [24], a LSTM-based STLF model is utilized to predict short-term residential load. Firstly, Density-Based Spatial Clustering of Application with Noise (DBSCAN) is employed to cluster the load profiles into several groups and an LSTM model is developed to forecast the load of each group. Four input vectors—the sequence of energy consumptions, the series of the time day indices, the corresponding day of week indices, and the corresponding binary holiday marks—are fed into the model to improve the accuracy further. Although RNN can map nonlinear features such as conventional approaches, this approach also has drawbacks: (1) RNN can only analyze information at the time domain and is insensitive to the frequency information, while the demand load is a combination of electricity appliances operating with different frequencies. (2) Moreover, the relevant weather information should also be fed as an important input feature.

Decomposition methods include empirical mode decomposition (EMD) [25], variational mode decomposition (VMD) [26,27], seasonal and trend decomposition using Loess decomposition (STL decomposition) [28], and empirical wavelet transforms (EWT) [29,30]. EMD-based STLF methods are introduced in [25]. As an adaptive nonlinear decomposition method, EMD decomposes the original signal into a series of intrinsic mode functions (IMFs) using Hilbert–Huang transform and each IMF is an amplitude modulation–frequency modulation (AM-FM) signal [31]. However, as a purely data-driven method, EMD lacks a mathematical definition, so it is difficult to understand the decomposition results; secondly, the decomposed signals will diverge at the endpoints and are highly sensitive to noise [12]. VMD-based STLF methods are presented in [27,32]. As an alternative algorithm to EMD, VMD is a non-recursive, adaptive decomposition estimation method to decompose the original signal into several mode functions with specific bandwidths in the frequency domain [33]. The latest decomposition algorithm, EWT, combines the strength of the wavelet’s mathematical definition with the flexibility of EMD [29].

However, as discussed above, it is vital to create a precise prediction for the distribution network. The high uncertainty of the distribution network and the strict limitation of the traditional time-domain STLF model are the main barriers for researchers to improve the accuracy further. Hence, this paper proposes a hybrid STLF method that can extract both time-domain and frequency-domain features with high adaptivity. Time—frequency transformation approaches, such as the Fourier transform, wavelet transform (WT), and least-squares wavelet (LSW) [34], can effectively convert the data from the time domain into the frequency domain. The literature proposes a hybrid STLF approach that combines the WT and ANN [25,27,29,30,31]. In these works, the original time-series demand load is transferred into the frequency-domain data, then a series of neural networks are trained simultaneously to forecast each frequency component. Finally, the predicted values of all the components are combined to create the overall prediction of the demand load. However, there is a lack of work to take advantage of all the WT algorithms, which can improve the prediction accuracy further. As illustrated, a wealth of work is available in the literature and the existing STLF models still have some knowledge gaps that can be filled.

(1)

Although STLF has been fully investigated in transmission networks and at the household-level, distribution-level STLF is a relatively weak segment in current power systems.

(2)

A new hybrid STLF that takes advantage of Variational Mode Decomposition (VMD), Empirical Mode Decomposition (EMD), and Empirical Wavelet Transform (EWT) should be proposed.

The remaining paper is organized as follows: The relevant knowledge regarding datasets, wavelet transform, recurrent neural networks, and the model structure is introduced in Section 2. In Section 3, three case studies are implemented, which compare the proposed load forecasting algorithm and other methods and evaluate the parameters that achieve the best performance. Section 4 discusses the result obtained from Section 3. The conclusion and final discussion are provided in the last section.

2. Materials and Methods

2.1. Data Description

The dataset employed in this paper includes distribution-level electricity data, which is constructed by combining household-level smart meter data and weather and temporal data.

2.1.1. Distribution-Level Electricity Data

In this paper, the distribution network level data are obtained from the physical/informatic aggregator and the individual household-level smart meter data from Pecan Street Dataport (Dataport) are added up to match the capacity of the feeder model. The geographical location of the elasticity data is N 30°15′59.9976″, W 7°43′59.9880″ (Austin, TX, USA). The feeder models used for this research are selected from standard feeder models provided by GridLAB-D 4.3. In this work, 976 houses are aggregated to match the R5-12.47-2 feeder model, indicating a moderate suburban area (demand capacity is 4500 kW). To reach the defined aggregation size, household smart meter data are picked up randomly from the Dataport dataset; an example of the demand load is shown in Figure 1.

2.1.2. Weather and Temporal Information

The corresponding weather and temporal information at the same location are obtained from the National Solar Radiation Database (NSRDB). An example of the dataset is shown in

Table 1. Shows the weather and temporal dataset.

Timestamp	Holiday	HOD	DOW	MOY	Dew. Point (°C)	Temperature (°C)	Pressure (Pa)	Relative Humidity (%RH)
1 January 2014 00:00	1	0	3	1	−1.25931	1.801934814	1001.035	80.1306
1 January 2014 01:00	1	1	3	1	−1.25199	1.385064697	1000.496	82.60188
1 January 2014 02:00	1	2	3	1	−1.25886	1.022241211	999.9987	84.73964
1 January 2014 03:00	1	3	3	1	−1.26002	0.723382568	999.3622	86.57533
1 January 2014 04:00	1	4	3	1	−1.23658	0.513696289	998.8751	88.04627
1 January 2014 05:00	1	5	3	1	−1.19007	0.407220459	998.4365	89.02894
1 January 2014 06:00	1	6	3	1	−1.09016	0.479547119	998.138	89.215
1 January 2014 07:00	1	7	3	1	−0.80314	1.532342529	998.006	84.46244

and Figure 2; the weather parameters include Dew point (°C), Temperature (°C), Pressure (Pa), and Relative Humidity (%RH). As for the temporal information, four variables are introduced, which are: Holiday (1 for holiday days and 0 for non-holiday days), Hour of the Day (HOD) (index range from 0 to 23), Day of the Week (DOW) (index range from 0 to 6), and Month of the Year (MOY) (index ranges from 1 to 12). As categorical variables, DOY, HOD, DOW, and MOY should be pre-processed by one-hot encoding.

2.2. Methods

2.2.1. Empirical Mode Decomposition

EMD is a self-adoptive mode decomposition method and it can decompose the signal in temporal space directly without transferring it into frequency space (see Figure 3). The characteristic of EMD is that it does not rely on any mathematical functions but adopts the signal $f\left(t\right)$ accordingly; $f\left(t\right)$ is decomposed into $N+1$ Intrinsic Mode Functions (IMFs) $f_k\left(t\right)$ and a residuum $r\left(t\right)$, see (1).

$$f\left(t\right)={{\displaystyle \sum }}_{k=0}^N{f}_k\left(t\right)+r\left(t\right)$$

(1)

An IMF is an AM—FM function and can be expressed as follows:

$$f_k\left(t\right)=F_k\left(t\right)\cos \left({\phi }_k\left(t\right)\right) \mathit{where} F_k\left(t\right),{\phi }_k^{\prime }\left(t\right)>0 \forall t$$

(2)

The main assumption is that $F_k\left(t\right)$ and ${\phi }_k^{\prime }\left(t\right)$ varies much slower than ${\phi }_k\left(t\right)$. The detailed process is presented in Algorithm 1. An IMF should satisfy two conditions: (1) the number of extrema and the number of zero crossings must be the same or differ at most by one; (2) at any point, the mean value of upper (defined by local maxima) and lower envelope (defined by local minima) is zero. The detailed process of EMD is demonstrated in Algorithm 1.

Algorithm 1: Empirical Mode Decomposition (EMD).
Input: Real-world signal $f\left(t\right)$.
Output: IMFs $f_k$, where $k=1,2,\dots , N+1$.
Initialization: $n:=1$, $r_0\left(t\right)=f\left(t\right)$.
Step 1: Extract the $n$th IMF as follows:
	(a): Initialize $h_0\left(t\right) :=r_{n-1}\left(t\right)$ and $k :=1$.  (b): Detect the maxima and minima of $h_{k-1}\left(t\right)$.  (c): Compute the upper and lower envelope, $U_{k-1}\left(t\right)$ and $L_{k-1}\left(t\right)$ by a cubic spline interpolation from the maxima and minima (See Figure 3a).  (d): Compute the mean envelope: $m_{k-1}\left(t\right)=\frac{U_{k-1}\left(t\right)+L_{k-1}\left(t\right)}{2}$.  (e): Obtain the candidate component: $h_k\left(t\right):=h_{k-1}\left(t\right)-m_{k-1}\left(t\right)$ (See Figure 3b).  (f): If $r_1\left(t\right)$ satisfies conditions of an IMF:       (i): $x_n\left(t\right):=h_k\left(t\right)$ and $r_n\left(t\right):=r_{n-1}\left(t\right)-x_n\left(t\right)$.  (g): Else:       (i): $k:=k+1$.       (ii): Repeat steps b)-g) until $h_k\left(t\right)$ is an IMF.
Step 2: If $r_n\left(t\right)$ is a residuum, stop the process.       Else  $n:=n+1$ and start from Step 1.

2.2.2. Variational Mode Decomposition

VMD was proposed by K. Ragomiretskiy in 2014 [33]; it is a non-recursive, adaptive decomposition estimation method to decompose the original signal into $K$ mode functions $u_k\left(t\right)$ with specific bandwidth in the frequency domain. Additionally, each $u_k\left(t\right)$ is concentrated near the central frequency ${\omega }_k$. The nature of the VMD method is an optimal process to look for $K$ modes that cause the overall bandwidth to be the smallest, as shown in Algorithm 1:

• Already modified by Figure 3a,b.

$$\underset{\left\{u_k\right\},\left\{{\omega }_k\right\}}{min}\left\{{{\displaystyle \sum }}_{k=1}^K\Vert {\partial }_t\left[\left(\delta \left(t\right)+\frac{j}{\pi t}\right)*u_k\left(t\right)\right]e^{-j{\omega }_{k}t}{\Vert }^2\right\} \mathrm{s}.\mathrm{t}. {{\displaystyle \sum }}_{k=1}^K{u}_k=f\left(t\right)$$

(3)

where $f\left(t\right)$ is the original signal, $\delta \left(t\right)$ represents the Dirac distribution function; $\left(\delta \left(t\right)+\frac{j}{\pi t}\right)*u_k\left(t\right)$ is the corresponding unilateral spectrum of $u_k\left(t\right)$ by implementing the Hilbert transformation; $u_k$ and ${\omega }_k$ represents the $k$th mode and corresponding central frequency; and $e^{-j{\omega }_{k}t}$ is the exponent term to adjust the frequency spectrum to the corresponding base frequency band.

Then, by introducing a quadratic penalty $\alpha$ and Lagrange multiplier operator $\lambda \left(t\right)$, the constrained problem mentioned in (3) is transformed into a non-constrained problem, the augmented Lagrangian expression is expressed as:

$$\begin{array}{c}L\left(\left\{u_k\right\},\left\{{\omega }_k\right\},\lambda \right)|=\alpha {{\displaystyle \sum }}_{k=1}^K\Vert {\partial }_t\left[\left(\delta \left(t\right)+\frac{j}{\pi t}\right)*u_k\left(t\right)\right]e^{-j{\omega }_{k}t}{\Vert }_2^2\\ |+\Vert f(t)-{{\displaystyle \sum }}_{k=1}^K{u}_k{\left(t\right)\Vert }_2^2\\ |+⟨\lambda \left(t\right),f\left(t\right)-{{\displaystyle \sum }}_{k=1}^K{u}_k\left(t\right)⟩\end{array}$$

(4)

where $\alpha$ is adopted to ensure the accuracy of the reconstruction; $\lambda \left(t\right)$ is employed to tighten the constraint; and $\Vert f\left(t\right)-{{\displaystyle \sum }}_{k=1}^K{u}_k{\left(t\right)\Vert }_2^2$ is a quadratic penalty term to speed up the convergence. The expression (4) can be solved by employing the alternate direction method of multipliers (ADMM) to compute the saddle point of the equation. According to the ADMM optimization method, $u_k$ and are updated as follows:

$$u_k^{n+1}=\arg \underset{u_k}{min}L\left(\left\{u_{i<k}^{n+1}\right\},\left\{u_{i\ge k}^n\right\},\left\{{\omega }_i^n\right\},\left\{{\lambda }^n\right\}\right)$$

(5)

$${\omega }_k^{n+1}=\arg \underset{{\omega }_k}{min}L\left(\left\{u_i^{n+1}\right\},\left\{{\omega }_{i<k}^{n+1}\right\},\left\{{\omega }_{i\ge k}^n\right\},\left\{{\lambda }^n\right\}\right)$$

(6)

$${\lambda }^{n+1}={\lambda }^n+\tau \left(f\left(t\right)-{{\displaystyle \sum }}_{k=1}^K{u}_k^{n+1}\right)$$

(7)

$${{\displaystyle \sum }}_{k=1}^K{\Vert u}_k^{n+1}-u_k^n{\Vert }_2^2/u_k^n{\Vert }_2^2<\epsilon$$

(8)

Finally, $u_k^{n+1}$ and ${\omega }_k^{n+1}$ are solved as:

$$u_k^{n+1}\left(\omega \right)=\frac{f\left(\omega \right)-{{\displaystyle \sum }}_{i\ne k}{u}_i\left(\omega \right)+\frac{\lambda \left(\omega \right)}{2}}{1+2\alpha {\left(\omega -{\omega }_k\right)}^2}$$

(9)

$${\omega }_k^{n+1}=\frac{{{\displaystyle \int }}_0^{\infty }\omega {\left|u_k^{n+1}\left(\omega \right)\right|}^{2}d\omega }{{{\displaystyle \int }}_0^{\infty }{\left|u_k^{n+1}\left(\omega \right)\right|}^{2}d\omega }$$

(10)

where $f\left(\omega \right)$, $\lambda \left(\omega \right)$, $u_i\left(\omega \right)$, and $u_k^{n+1}\left(\omega \right)$ represent the Fourier transform of $f\left(t\right)$, $\lambda \left(t\right)$, $u_i\left(t\right)$, and $u_k^{n+1}\left(t\right)$.

2.2.3. Empirical Wavelet Transforms (EWT)

After the data are denoised via DWT, the denoised data $f\left(t\right)$ are decomposed into $N$ sub-layers via EWT. The EWT aims to extract multiple sub-layers by constructing adaptive wavelets. The EWT decomposition process is performed in the following steps. In EWT decomposition, the number of sub-layers $N$ is defined at the beginning.

Step 1: Apply fast Fourier transform (FFT) to the denoised data $f\left(t\right)$ to obtain the frequency spectrum $F\left(\omega \right)$.

Step 2: Search the $F\left(\omega \right)$ to find the $\mathrm{N}$ local maxima $\partial ={\left\{{\partial }_n\right\}}_{n=1,2\dots ,N}$ and the corresponding frequencies $\omega ={\left\{{\omega }_n\right\}}_{n=1,2\dots ,N}$ by using the magnitude threshold $\mathsf{\alpha }$ and frequency distance thresholds $\mathsf{\delta }$.$\mathsf{\alpha }$ is set as 3% of the fundamental magnitude to detect the significant frequencies and $\mathsf{\delta }$ is set as 8 Hz to avoid overestimation.

Step 3: Segment the frequency spectrum $\left[0,f_{sample}/2\right]$ into $N$ segments and the boundaries ${\Omega }_n$ is the center line between two neighboring local maxima (see Figure 4), which can be calculated as:

$${\Omega }_n=\frac{{\omega }_n+{\omega }_{n+1}}{2}$$

(11)

Step 4: Build $N$ wavelet filters, including one low-pass filter and $N-1$ band-pass filters based on the defined boundaries. The scaling and wavelet functions are defined in (12) and (13), respectively. A comparison between the demand load data at the time- and frequency-domain is shown in Figure 5; from the figure, it is observed that the power spectrums of the load demand not only contain time information but also indicate the frequency components.

2.2.4. Recurrent Neural Network

The LSTM model was first proposed in 1997. As shown in Figure 6, in LSTM, the hidden state in traditional RNN is replaced by the memory cell $C_t$ and three gates, i.e., the input gate $I_t$, the forget gate $F_t$, and the output gate $O_t$. The output of the previous time step $h_{t-1}$ and the input sequence of the current time step $X_t$ are adopted as the input of the gates. The sigmoid activation function 𝜎 controls these gates $(\cdot )$: the information is preserved when the activation output is close to 1 and the information is eliminated when the activation output approaches 0. As for the memory cell $C_t$, a candidate memory cell ${\tilde{C}}_t$ is computed at first. The only difference between ${\tilde{C}}_t$ and the gates are that ${\tilde{C}}_t$ utilizes a Tanh activation function $\tanh (\cdot )$ ranging from −1 to 1. Finally, the memory cell $C_t$ is generated by combining ${\tilde{C}}_t$ and $I_t$ and then the previous memory cell $C_{t-1}$ with $I_t$ and $F_t$, where $I_t$ decides how many data from ${\tilde{C}}_t$ are useful and $F_t$ decides how much information from the old memory cell is retained. The detailed formulas are presented as follows:

$$I_t=\sigma \left(W_{xi}{X}_t+W_{hi}{h}_{t-1}+b_i\right)$$

(12)

$$F_t=\sigma \left(W_{xf}{X}_t+W_{hf}{h}_{t-1}+b_f\right)$$

(13)

$$O_t=\sigma \left(W_{xo}{X}_t+W_{ho}{h}_{t-1}+b_o\right)$$

(14)

$${\tilde{C}}_t=\mathit{tan}h\left(W_{xc}{X}_t+W_{hc}{h}_{t-1}+b_c\right)$$

(15)

$$C_t=F_t\odot C_{t-1}+I_t\odot {\tilde{C}}_t$$

(16)

$$h_t=O_t\odot tanh\left(C_t\right)$$

(17)

where $X_t$,$\odot$ represents element-wise multiplication; $W_{xi}$, $W_{xf}$, $W_{xo}$, $W_{xc}$; $W_{hi}, W_{hf}, W_{ho}, W_{hc}$ are the weight matrices; and $b_i,b_f,b_o,b_c$ are the bias parameters.

2.2.5. Proposed Wavelet Transform-Based Forecasting System

As presented in Figure 7, the proposed method is divided into five steps and described as follows.

Step A: Data pre-processing is implemented in the first stage. The original input dataset, which includes data cleaning, is applied to the original dataset to populate the missing features. Then, a max–min scaling function is applied to the original dataset to limit the range of data between 0 and 1.

Step B: The denoised electric load is decomposed into $N$ sub-layers via the WT decomposition algorithm. An example with nine sub-layers shows the decomposed components from the original load curve.

Step C: Then $N$ LSTM prediction models are constructed and each BLSTM neural network model is trained for one sub-layer.

Step D: In the final step, the prediction results for all the sub-layers are reconstructed to present the final load forecasting results. Repeat Steps A–D until reaching the end of the testing dataset.

2.2.6. Performance Metrics

To assess the performance of the proposed predictor, four performance metrics are adopted, which are the mean absolute error (MAE), mean absolute percentage error (MAPE), rooted mean squared error (RMSE), and $R^2$. The detailed formulas are shown as follows:

(1) MAE:

$$MAE=\frac{{{\displaystyle \sum }}_{i=1}^N\left|y_i-{\widehat{y}}_i\right|}{N}$$

(18)

(2) MAPE:

$$MAPE=\frac{{{\displaystyle \sum }}_{i=1}^N\left|\left(y_i-{\widehat{y}}_i\right)/y_i\right|}{N}\times 100\%$$

(19)

(3) RMSE:

$$RMSE=\sqrt{\frac{\left({{\displaystyle \sum }}_{i=1}^N{[y_i-{\widehat{y}}_i]}^2\right)}{N}}$$

(20)

(4) $R^2$:

$$R^2=1-\frac{S{S}_{RES}}{S{S}_{TOT}}=1-\frac{{{\displaystyle \sum }}_i{\left(y_i-{\widehat{y}}_i\right)}^2}{{{\displaystyle \sum }}_i{\left(y_i-\overline{y}\right)}^2}$$

(21)

where ${\mathit{SS}}_{\mathit{RES}}$ is sum squared regression error and $S{S}_{TOT}$ is sum squared total error.

3. Results

This section evaluates the proposed robust wavelet transform-based STLF model by implementing three case studies. The first case study investigates the sub-layer number of the WT, the second case investigates the influence of weather information on forecasting accuracy, and the last compares different algorithms.

3.1. Sub-Layer Number

Referring to the EWT decomposition technique introduced in Section 2, the original time-varying load demand is decomposed into N sub-layers by the EWT, which is defined as $S_1–S_N$ in this study. The number of N has a significant impact on the final forecasting performance. In this study, the range of $N$ increases from 5 to 13. The performance of the proposed model with different numbers of $N$ is summarized in

Table 2. Day-ahead prediction performance of the proposed model with different sublayer numbers.

N	MAE (kW)	MAPE (%)	RMSE (kW)	R2
5	146.721	5.959	202.909	0.725
6	121.614	4.965	159.892	0.837
7	93.622	3.878	124.382	0.928
8	90.313	3.762	116.665	0.936
9	80.222	3.416	102.900	0.954
10	79.948	3.398	101.089	0.956
11	83.153	3.517	106.233	0.947
12	84.610	3.604	105.636	0.949
13	100.364	4.318	122.169	0.931

and Figure 8. From these tables, it is observed that the MAE, MAPE, and RMSE are relatively large when $N$ is too tiny (near 5) or too large (near 13) (see Figure 8). Among all $N$ values, the dominant value is $N=10$, followed by $N=9$, where the RMSE values are 101.089 kW and 102.900 kW, respectively.

Once the optimal number of the decomposition layers is determined, the load demand data are decomposed by EWT to obtain the sub-components. Then, $N$ LSTM predictors are trained simultaneously to predict each sub-component. The predictions for the decomposed sublayers regarding the validation set are shown in Figure 9. The load demand is decomposed into $N$ sub-layers by the EWT, which provides the best performance of the selected datasets ($N=10$). Sub-layers ($S_1–S_3$) capture the low-frequency oscillation of the baseline and the curves of these sub-layers vary smoothly and change steadily. The predicted curves of these four layers achieve higher accuracy from the prediction results. Because the Sub-layers ($S_8–S_{10}$) capture high-frequency components with a high fluctuation range and include the most noise, most of the prediction errors come from these components’ predictions.

3.2. Case Study 2: Influence of Weather/Temporal Information

The weather information, such as temperature, humidity, and pressure, has a significant impact on the prediction accuracy of the load forecasting model. The relevant work has demonstrated that the model with weather information as inputs achieves a higher accuracy than the model with such information. However, whether this conclusion still stands in the proposed robustness time—frequency model is unclear. In addition, the consumer’s electricity activity is strongly correlated with temporal information. For instance, the electricity consumption is high during peak times (7–10 am and 4–10 pm) every day, while the electricity usage is significantly low during the off-peak times (such as midnight). Hence, in this case study, the model with weather/temporal information input is compared with the model without weather/temporal information. In addition, relevant weather variables are employed as the input variables of the proposed STLF model. A comparison is created among STLF without external data, STLF with weather information, and STLF with both weather and temporal information, see

Table 3. Comparison of methods with/without weather information.

Method	MAE (kW)	MAPE (%)	RMSE (kW)	R2
Model + Weather Information	73.602	3.130	94.676	0.962
Model + Weather Information + Temporal Information	70.666	3.004	91.084	0.966
Model without External Information	79.948	3.398	101.089	0.956

. From the table, it is observed that the model without external information achieves the lowest prediction accuracy. When the weather information, e.g., temperature, humidity, and the dew point, is added as input variables, the prediction accuracy improves RMSE by 6.34%, MAE by 7.94%, and MAPE by 7.89%, respectively. While introducing the temporal information, e.g., DOY, HOD, DOW, and MOY, the prediction performance improves further, demonstrating that the relevant variables can enhance the prediction accuracy.

3.3. Case Study 3: Comparision of Different Algorithms

In this case, study, the one-step forecasting performance of the proposed method is compared with relevant forecasting approaches. A detailed description of the models adopted in this study is listed below: 1D CNN-LSTM STLF model; 1D CNN-GRU STLF model; EMD-LSTM STLF model; VMD-LSTM STLF model; and proposed model.

For models 1 and 2, the original time-varying load demand is adopted as the input of neural network models. However, for models 3, 4, and 5, the actual load demand data are decomposed via EMD/VMD/EWT, respectively, and then the neural network is trained for each sub-layer.

Table 4. Prediction performance of the proposed model and related works.

Method	MAE (kW)	MAPE (%)	RMSE (kW)	R2
1D CNN-LSTM	189.822	8.564	267.284	0.487
1D CNN-GRU	205.014	9.270	284.339	0.429
VMD-LSTM	122.899	5.010	171.473	0.803
EMD-LSTM	150.303	6.286	196.932	0.709
Proposed Method	70.666	3.004	91.084	0.966

shows the performance of five models considering the performance metrics, i.e., MAE, MAPE, RMSE, and $R^2$, of the predicted load demand regarding the distribution-level dataset. As shown in the table, the proposed ND-EWT-LSTM-BHO outperforms other models. Moreover, the spectral load forecasting methods, including ND-EWT-LSTM-BHO, EMD-LSTM, and VMD-LSTM, have better prediction accuracy than conventional deep learning methods, including 1D CNN-LSTM and 1D CNN-GRU. 1D CNN-LSTM and 1D CNN-GRU models have the worst estimation performance, with the highest MAE, MAPE, and RMSE in all the experiment groups. The prediction performance of VMD-LSTM and EMD-LSTM are similar, just below the proposed method. Figure 10 compares the predicted values with the testing set using the proposed and benchmark models. The results predicted by the proposed model are the closest to the ground truth measurements. Moreover, the results estimated by the CNN-LSTM/CNN-GRU model are the farthest from the ground truth curve, showing that CNN-LSTM and CNN-GRU perform worst among all the algorithms.

Figure 11 shows the scatter plot of different forecasting models’ ground truth and forecasting values. The scatter plot shows the correlation relationship between the two variables. The higher the $R^2$ value, the stronger the correlation between the predictions and ground truth, representing better accuracy achieved by the forecasting model. For the proposed model, the scatter about the line is relatively small and most of the points are on the regression line, with only several data values far from other data values. For other spectral methods, the $R^2$ of VMD-LSTM and EMD-LSTM models also show a strong correlation with the ground truth curve, with $R^2$ values over 0.70. CNN-GRU shows the worst correlation from the scatter plot, with $R^2$ values of 0.429.

4. Discussion

In Section 3, three case studies are implemented. From the first case study, it is observed that the layer number N has a significant impact on the final forecasting result. A small or large value of N will reduce the prediction accuracy. When N is too small (such as 2), the EWT is inefficient in separating all the frequency components, different frequency components are overlapping, and the LSTM model causes it to be difficult to produce a precise prediction for each element. When N is too large, there are too many LSTM predictors trained and the errors from all the predictors are added up, increasing the error of the predicted active power. As for case study 2, the two important variables that strongly influence human activity, namely weather and temporal information, are investigated. The result is obtained by comparing the models with/without the weather and temporal information. From the result of the case study, it is observed that the weather information, especially temperature T and humidity H, can improve the prediction accuracy and reduce the MAE by 3.98%. The temporal data, including DOY, HOD, DOW, and MOY, can reduce the prediction error further. The results indicate that it is essential to consider the weather and temporal variables for distribution-level load forecasting. In the last case study, the robustness of the time—frequency-domain STLF models (VMD-LSTM, EMD-LSTM, and VMD-LSTM) are compared with the conventional time-domain STLF models (including LSTM and GRU). The simulation result shows that all three time-frequency domain STLF models have lower prediction errors than LSTM and GRU models, especially the EWT model. The explanation for this result is: (1) The robustness of the time—frequency-domain STLF models can extract frequency and time information from the original dataset; this information is vital to the analysis of the trend of the demand load. (2) EWT takes advantage of both EMD (high adaptation) and VMD (a strong mathematical expression). Hence, EWT achieves the highest accuracy among the five models.

5. Conclusions

In this paper, a robustness frequency—time-domain STLF model was proposed to improve the prediction accuracy of the distribution network. The proposed model utilized the three wavelet transform approaches (EWT, EMD, and VMD) to decompose the original data to extract the inherent frequency components of the load and then several LSTM predictors are trained to predict each frequency component. Besides, this paper investigated the important variables that influence the prediction results. From the case study analysis, it was found that the decomposition layer number, weather, and temporal information have a significant impact on the load forecasting results. The results verified that the model that utilizes both time and frequency data could better estimate the trend of the load. However, there are also limitations to this work. The training time of the model is longer than the normal LSTM model and it is time-consuming to tune the hyperparameters. In future work, a fast parameter-tuning algorithm will be proposed to reduce the training time and improve the model’s efficiency.