Multihousehold Load Forecasting Based on a Convolutional Neural Network Using Moment Information and Data Augmentation

Abstract

Deep learning (DL) networks are a popular choice for short-term load forecasting (STLF) in the residential sector. Hybrid DL methodologies based on convolutional neural networks (CNNs) and long short-term memory networks (LSTMs) have a higher forecasting accuracy than conventional statistical STLF techniques for different types of single-household load series. However, existing load forecasting methodologies are often inefficient when a high load demand persists for a few hours in a day. Peak load consumption is explicitly depicted as a tail in the probability distribution function (PDF) of the load series. Due to the diverse and uncertain nature of peak load demands, DL methodologies have difficulty maintaining consistent forecasting results, particularly when the PDF of the load series has a longer tail. This paper proposes a multihousehold load forecasting strategy based on the collective moment measure (CMM) (which is obtained from the PDF of the load series), data augmentation, and a CNN. Each load series was compared and ordered through CMM indexing, which helped maintain a minimum or constant shifting variance in the dataset inputted to the CNN. Data augmentation was used to enlarge the input dataset and solve the existing data requirement issues of the CNN. With the ordered load series and data augmentation strategy, the simulation results demonstrated a significant improvement in the performance of both single-household and multihousehold load forecasting. The proposed method predicts day-ahead multihousehold loads simultaneously and compares the results based on a single household. The forecasting performance of the proposed method for six different household groups with 10, 20, 30, 50, 80, and 100 household load series was evaluated and compared with those of existing methodologies. The mean absolute percentage error of the prediction results for each multihousehold load series could be improved by more than 3%. This study can help advance the application of DL methods for household load prediction under high-load-demand conditions.

1. Introduction

Short-term load forecasting (STLF) facilitates the operation of power systems as an economical, secure, and reliable program. It is equally important in the residential sector. Recently, STLF methodologies based on statistical, machine learning, deep learning (DL), causal, econometric, and judgment-based algorithms have been developed [1]. For single households, these techniques must deal with granular load series collected from advanced metering infrastructure (AMI). Each series is composed of uncertain load consumption values, which have a high degree of stochasticity, making them difficult to define using a physical measure [2,3,4].

Although people today live and spend time in grouped areas with distinct lifestyles, such as in apartment blocks, department stores, and companies, each household produces a unique load consumption series. The major exogenous factors that bring differences in household load series include the use of automatic thermal devices, installation of electrical vehicles (EVs), penetration of renewable energy resources, and evolution of social demographics. The impact of these factors is expected to increase in the future, and the corresponding load series will become more complex for analysis and prediction. Hence, a new approach that addresses two main issues should be developed: (1) The vast number of customer load series at any given time, and (2) the increasing amount of uncertainty in single-household load series.

The load series from a single household has properties similar to nonstationary signals and has been mainly analyzed based on zero-mean, trend, and seasonal modeling [1]. Many statistical approaches can be used to extract hidden information from unusual load consumptions with the help of consecutive load observations during peak load demand hours [5,6,7]. Relevant customer load series has been explored using a Gaussian process for forecasting the load profiles of target households [8]. With the help of the correlation and L2 norm-based input selection methods, an appropriate sub-load series was extracted for each type of special day [9]. Clustering- and classification-based forecasting methodologies reduce the uncertainty level by batching homogenous customers, days, or weather parameters [9,10,11,12,13]. DL methodologies, such as support vector machine (SVM) [14], multilinear perceptron (MLP) [15], convolutional neural networks (CNNs) [16,17,18], and recurrent neural networks (RNNs) [19], have been used to forecast single-household loads. Pooling [16] and data augmentation [17] are popular algorithms for preparing a training dataset for the CNN from historical load series. A multiple household load forecasting model using Bayesian networks has been presented with various input features such as past consumption, temperature, and socioeconomic and electricity usage aspects [20]. A forecasting model for multiple individual households’ short-term electricity consumption has also been developed using two-level information extraction stages [21], where input datasets are structured in the low-level stage and a combined LSTM and CNN-based forecasting model is developed in the high-level stage. However, existing methodologies are associated with challenges in precisely batching and exploiting the diverse load series, and too weakly structured a dataset to achieve a high forecasting accuracy.

Advanced CNN-based architectures have been developed for image recognition, medical imaging, and natural language processing applications [22]. ImageNet [23] and VGGNet [24] are well-known CNN architectures in which a number of images can be taken as input for classifying multiple image-like objects. These architectures have been mainly developed for data classification problems that are closer to multi-series forecasting than to single-series forecasting. Many load series are recorded in ordered 1D architectures; however, CNN-based architectures operate by accepting three- or higher-dimensional datasets. Few have studied the local correlations between multiple time-series data. Convoluting multi-nonstationary signals can produce a better performance by exploiting the local minima, correlations, and dependencies [25]. Multisite PV profiles are predicted by developing a space–time matrix using a greedy adjoining algorithm [26]. For better accuracy, a large dataset is required for training with a minimum or constant shifting variance [27,28]. Existing pooling and augmentation techniques are popular for enlarging datasets and producing better forecasting results than their peers. However, the pooling strategy [16] has difficulty in delivering better forecasting results because of the large number of complex load series. Complex load series contain an abundance of tail information in their probability distribution functions (PDFs) because there may be higher load demand for a few hours in any given day. The data augmentation technique [17] has difficulty enlarging the appropriate dataset because of its shortcomings in generating a large number of similar residual load series. This makes it difficult to achieve a high forecasting accuracy for a single household; improving the performance continues to be a prerequisite. Hence, exploring multiple household load series for simultaneous load prediction is another approach to improve the forecasting accuracy of existing models.

This paper proposes a novel day-ahead multihousehold forecasting method based on a CNN architecture in which forecasting results are obtained simultaneously. The proposed method was tested on six different household groups containing 10, 20, 30, 50, 80, and 100 households with different load series from the database. Each load series was identified as unique by evaluating its collective moment measure (CMM). The CMM quantifies the hidden integrated statistics of the load series using moment parameter information, such as the mean and variance. The quantitative measurement of the moment parameters assists in learning the daily few-hour peak load from the load series of the household groups. The load series discovered based on the CMM index are more likely and placed in order for data augmentation. The data augmentation methodology for the ordered multihousehold load series maintains a minor or minimum shifting variance in the input dataset for the CNN. Although high load-demand hours are present each day in each load series, CMM-based ordering balances the entire uncertain peak load consumption, and the CNN can learn to predict accurate day-ahead load profiles for all input households. The simulation results showed a significant improvement of more than 8% and 3% in the forecasting accuracy in terms of the mean absolute percentage error (MAPE) over existing CNN methods with pooling [16] and augmentation [17], respectively.

The remainder of this paper is organized as follows. Section 2 discusses multihousehold load forecasting, the significance of the minimum shifting variance in the CNN, and reordering load series based on the tail PDF. Section 3 describes the estimation of the tail length, the data augmentation methodology for single-household load series, and the procedure of the proposed multihousehold forecasting algorithm. Section 4 presents the obtained multihousehold load forecasting simulation results and comparisons. Section 5 discuss the potential results and limitations of the paper. Finally, Section 6 concludes the paper.

2. Problem Description

2.1. Multihousehold Load Forecasting Based on CNN

The multihousehold load forecasting methodology predicts the number of day-ahead load profiles of different households simultaneously by sharing load-consuming characteristics from the nearest neighborhood. Many machine- and DL-based forecasting strategies, such as pooling [16] and k-nearest neighbor [29], can perform forecasting operations for single-household load series using the load-series data of other households. In comparison, a data augmentation strategy can predict all the household load series with a higher forecasting accuracy. Figure 1 shows the overall procedure of the proposed CNN-based multihousehold load forecasting methodology. The data augmentation method meets the data requirement of the CNN for multihousehold load forecasting, even though each household has a peak load consumption for a few hours in its daily load profiles.

2.2. Effectiveness of CNN with Low-Shifting Data

Shifting invariance is key to realizing a high prediction accuracy when using a CNN [30,31] and is useful for forecasting multihousehold load series. Shifting invariance means maintaining unbiased learning for small displacements in the input dataset [32], where the convolutional architecture is robust to learning the peak load difference in the input dataset. However, the CNN forecasting results vary significantly under high and low peak loads because of parameter sharing between the convolution layers and the partial effects from the pooling layers [28]. Hence, variance shifting should be minimized or set as a constant in the input dataset to achieve a high forecasting accuracy when using a CNN. To acknowledge the low shifting and evaluate the potential significant difference in the input series $L=[l_{\mathrm{1,1}}, l_{\mathrm{1,2}},l_{\mathrm{1,3}}, \dots \dots l_{d,1},\dots l_{d,t},\dots l_{d,T},\dots l_{D,1}\dots .l_{D,\mathrm{T}}]$, the average shifting variance ${\overline{\delta }}_{sf}$ is computed from n shifting variance ${\delta }_{sf}$, given by:

$${\delta }_{sf}(n)=\frac{1}{F}\sum _{i=n-F+1}^n{\left(l_i^{}-{\overline{l}}_F\right)}^2$$

(1)

$${\overline{\delta }}_{sf}=\frac{1}{D\times T-F}\sum _{n=F}^{D\times T}\delta \left(n\right)$$

(2)

where $l_i^{ }$ is the hourly electricity consumption with domain of $i=T\cdot \left(d-1\right)+t$, T represents the total hours in a day, t is the hour of the day d, D is the number of days, F is the specified length of the window series in $l_d$ to calculate the variance, and ${\overline{l}}_F$ is the mean load consumption of the window series.

2.3. Reordering Load Series Based on Tail Length for Reducing Shifting Variance

The tail length of a load series of a single household can be explicitly observed when there is unusual load consumption. All the load series exhibited either long or short tails in their PDFs [33,34]. Figure 2 shows two different load series with their corresponding tail characteristics. Figure 2a,b shows the short-tail load series and the corresponding PDF; Figure 2c,d shows the long-tail load series and corresponding PDF, respectively. The main reason for the occurrence of either short- or long-tail PDFs in the load series is the high demand for a few hours on a daily basis.

For example, air conditioners, heaters, ovens, washing machines, and microwaves are higher load-consuming utilities and are mostly used for specific durations only. Other major causes for different peak loads are the type of day, number of family members in a household, working hours, and whether EVs are used or not. The same peak demand may not exist at the same hour on a daily basis. This leads to a high uncertainty in day-ahead load prediction and affects the forecasting accuracy.

3. Proposed Multihousehold Load Forecasting Method

3.1. Tail Length Estimation of the PDF of Electricity Load

Electricity demand data from households with uncertain peaks can result in a long tail in the PDF. In this study, the occurrence of these uncertain peaks was estimated using the CMM index. Statistical moment metrics, such as the mean and variance, can be used to extract the appropriate tail information from each load series and help identify the uniqueness and learn how to predict the uncertain daily few-hour peak loads for each load series.

The overall uncertainty in the PDF of the load-series data is an ensemble of moments from different types of daily load profiles. Let the daily load profile $l_d=\left[l_{d,1}, l_{d,2},l_{d,3}, \dots \dots l_{d,t},\dots .l_{d,\mathrm{T}}\right]$, where $t$ represents the hourly load consumption on day $d$. The following statistical results can be obtained:

$$s_{d, 1}=\frac{1}{\mathrm{T}}\sum _{t=1}^{\mathrm{T}}{l}_{d,t}$$

$$s_{d, 2}=\frac{1}{\mathrm{T}}\sum _{t=1}^{\mathrm{T}}{\left(l_{d,t}-s_{d, 1}\right)}^2$$

$$s_{d, 3}=\sqrt{\frac{\sum _{t=1}^{\mathrm{T}}{\left(l_{d,t}-{\mathrm{s}}_{\mathrm{d},1}\right)}^2}{T}}$$

(3)

$$s_{d, 4}=\frac{1}{\mathrm{T}}\sum _{t=1}^{\mathrm{T}}{\left[\frac{\left(l_{d,t}-s_{d, 1}\right)}{s_{d, 3}}\right]}^3$$

$$s_{d, 5}=\frac{1}{\mathrm{T}}\sum _{t=1}^{\mathrm{T}}{\left[\frac{\left(l_{d,t}-s_{d, 1}\right)}{s_{d, 3}}\right]}^4$$

$$s_{d, 6}=\sum _{t=2}^{\mathrm{T}}\left|l_{d,t}-l_{d,t-1}\right|$$

where $l_{d,t}$ represents the single-household load consumption value at time t for day d, and $s_{d,1},{ s}_{d, 2}, s_{d,3}{, s}_{d,4}, s_{d,5},$and $s_{d,6}$are the statistical moment measuring tools for day d, such as the mean, variance, standard deviation, skewness, kurtosis, and cumulative difference, respectively. The cumulative difference is a newly introduced metric that measures the degree to which the daily load profile contains the sum of the hourly differences in the daily load consumption. Each statistical moment measures vary and represent the shift in the uncertainty on a daily basis. Hence, the overall metric difference vector $I_d$ can be computed on a daily basis as follows:

$$I_d=\left|s_d-s_{d-1}\right| \forall d$$

(4)

where $s_d=[s_{d,1},s_{d,2},s_{d,3},s_{d,4},s_{d,5},s_{d,6}]$ represents the overall change in the uncertainty values between the load profiles for days $d$ and $d-1$.

The CMM can serve as a valuable tool for defining characteristics such as the long-tail PDF of a single-household load series. Consequently, the CMM vector ${\mathrm{I}}_m=\left[I_1,I_2,I_3,\dots , I_d,\dots , I_D\right]$ for a single household $m$ can be obtained, where d represents D historical load profile days. Each load profile has a unique tail property, and the compositions of the different types of load profiles make each household load series unique. In this study, the L2 norm was used to calculate the overall magnitude of the CMM vector, which comprised the entire uncertainty in the load series. The comprehensive uncertainty presented in a CMM vector of the load series m is calculated as follows:

$$I\left(m\right)=\sqrt{\sum _{d=1}^D{\left|I_d\right|}^2}$$

(5)

where $I\left(m\right)$ is the L2-norm magnitude value of the CMM vector. Each load series $L_m^{ }$ can be rearranged based on the increasing values obtained from Equation (5).

3.2. Reordering Input Load Series

For each household group, each quantitative uncertainty measure $I\left(m\right)$ in a single-household load series can be calculated as follows:

$$I=\left[I\left(1\right),I\left(2\right),I\left(3\right),\dots \dots .I\left(m\right),\dots \dots .I\left(N\right)\right]$$

(6)

where N is the total number of load series in the input household group. The difference between $I\left(m\right)$ is measured by the integrated uncertainty between the two load series, and can help place each load series in the appropriate order. The proper placement of the load series may preserve the ordering of the shifting variance in the input load-series dataset. The lowest $I\left(m\right)$ indicates that the m-th load series has the lowest number of uncertain load profiles, and its position is calculated as follows:

$$I\left(m\right)=\underset{m}{\mathrm{argmin}}‖I\left(m\right)‖$$

(7)

Equation (7) is applied for all households where $N$ is replaced by $N-1$ each time. All the $N\times (N-1)/2$ iterations are eventually performed, and the ordered CMM values are finalized. The load series are placed according to the lowest $I\left(m\right)$ position given by Equation (7), and all the load series are placed in this order. The ordered dataset L is expressed as follows:

$${L=\left[\begin{array}{c}\begin{array}{ccc}{l}_{\mathrm{1,1}}& \begin{array}{ccc}{l}_{\mathrm{1,2}}& \dots & l_{1,d}\end{array}& \begin{array}{cc}\dots & l_{1,D}\end{array}\end{array}\\ \begin{array}{c}\begin{array}{ccc}{l}_{\mathrm{2,1}}& \begin{array}{ccc}{l}_{\mathrm{2,2}}& \dots & l_{2,d}\end{array}& \begin{array}{cc}\dots & l_{2,D}\end{array}\end{array}\\ \begin{array}{c}\begin{array}{ccc}⋮& \begin{array}{ccc}\dots & \ddots & \dots \end{array}& ⋮\end{array}\\ \begin{array}{ccc}{l}_{m,1}& \begin{array}{ccc}{l}_{m,2}& \dots & l_{m,d}\end{array}& \begin{array}{cc}\dots & l_{m,D}\end{array}\end{array}\end{array} \\ ⋮ \dots ⋮ \dots ⋮ \end{array}\\ \begin{array}{ccc}{l}_{N,1}& \begin{array}{ccc}{l}_{N,2}& \dots & l_{N,d}\end{array}& \begin{array}{cc}\dots & l_{N,D}\end{array}\end{array}\end{array}\right]}^T$$

(8)

where $L={\left[{L^{\prime }}_1,{L^{\prime }}_2,{L^{\prime }}_3,\dots {L^{\prime }}_m,\dots {L^{\prime }}_N\right]}^T$, and each ${L^{\prime }}_m=\left(\begin{array}{ccc}{l}_{m,1}& \begin{array}{ccc}{l}_{m,2}& \dots & l_{m,d}\end{array}& \begin{array}{cc}\dots & l_{m,D}\end{array}\end{array}\right)$ is the m-th-ordered single-household load series containing daily load profiles of D days.

3.3. Data Augmentation for Single-Household Load Series

The main purpose of using data augmentation in the residential sector is to learn new features from a single load series that is less volatile but still shows all the possible characteristics of the selected series [17]. Each ${L^{\prime }}_m$ is passed to the data-augmentation procedure, which creates $\mathcal{L}$ number of homogenous load series. Using multiple k-means clustering, each ${L^{\prime }}_m$ generates $\mathcal{L}$ different centroid load profiles $C_{\mathcal{l}}^m$that are less volatile and uncertain relative to each other. Each load profile $l_d^m$ of ${L^{\prime }}_m$ is decomposed into its centroid load profile $C_{\mathcal{l}}^m$ and daily residual load profile $r_d^m\left(\mathcal{l}\right)$ as follows:

$$l_{m,d}=C_{\mathcal{l}}^m+r_d^m\left(\mathcal{l}\right)$$

(9)

The identified $C_{\mathcal{l}}^m$ extract completes the seasonality of the load series, which is repetitive and easy to estimate using forecasting methods. By contrast, $r_{\mathcal{l},d}^m$ comprises a large amount of noisy information that implicitly contains the tail PDF property of the load series and is difficult to estimate. With $C_{\mathcal{l}}^m$, a new residual load series is generated that contains all the uncertain information of the load series, as given by:

$$R^m\left(\mathcal{l}\right)={L^{\prime }}_m-C_{\mathcal{l}}^m$$

(10)

where $R^m\left(\mathcal{l}\right)=\left(r_1^m\left(\mathcal{l}\right),r_2^m\left(\mathcal{l}\right),r_3^m\left(\mathcal{l}\right),\dots th r_d^m\left(\mathcal{l}\right),\dots .r_d^m\left(\mathcal{l}\right)\right)$ comprises new residual load profiles $r_d^m\left(\mathcal{l}\right)$ generated by $C_{\mathcal{l}}^m$ for all input days d. The efficacy of $R^m\left(\mathcal{l}\right)$ can be enhanced by maintaining k-input to the multiple k-means clustering. However, unwanted residual series are generated when the input k is greater than six. To explore only a homogenous residual load series and collect them in a single matrix $R_{in}$, the following equation was derived [17]:

$$R_{in}^m=\left\{r_{\mathcal{l}}|{\varnothing }_{\mathcal{l}}\le \frac{1}{\mathcal{L}}\sum _{\mathcal{l}=1}^{\mathcal{L}}{\varnothing }_{\mathcal{l}}\right\}$$

(11)

where ${\varnothing }_{\mathcal{l}}$ is the Frobenius norm of $R^m\left(\mathcal{l}\right)$, which selects only less volatile residual series. Each residual series shows similar properties to the inputted m-th load series; when their corresponding $C_{\mathcal{l}}^m$ is added, an augmented load series $L_{in}^m$ is generated as follows:

$$L_{in}^m=\left[{L^{\prime }}_m\left(1\right){{,L}^{\prime }}_m\left(2\right),{L^{\prime }}_m\left(3\right),\dots .{L^{\prime }}_m\left(\mathcal{l}\right)\dots {L^{\prime }}_m\left(\mathcal{L}\right)\right]$$

(12)

where ${L^{\prime }}_m\left(\mathcal{l}\right)=\left[l_{\mathcal{l}1}^m,l_{\mathcal{l},2}^m,l_{\mathcal{l},3}^m\dots l_{\mathcal{l},d}^m\dots .l_{\mathcal{l},D}^m\right]$ represents the extracted $\mathcal{l}$-th homogenous load series for household m. $L_{in}^m$ includes $\left(\mathcal{L}\times D\right)$ load profiles with $\mathcal{L}$ number of augmented load series for household m. Figure 3 shows the extracted $\mathcal{L}$ homogenous load series $L_{in}^m$from the augmentation CNN method for the two tested single-household load series.

3.4. Proposed Multihousehold Load Forecasting

Many CNN architectures are available in pretrained versions for image classification, segmentation, localization, and object detection [22,23,24]. The CNN architecture passes the input data to different layers: (a) input, (b) convolution layer, (c) pooling, and (d) fully connected layers. However, there are no precise rules for designing a CNN architecture. All the proposed CNN-based architectures require a large amount of input data to ensure better accuracy.

To perform multihousehold load forecasting operations, the proposed CNN forecasting model was designed in a manner similar to the existing CNN architecture [19,20,21]. Let all the augmented multihousehold load series collected in order using Equation (12) be given by:

$$X_{ }=\left[\begin{array}{c}{L^{\prime }}_1\left(1\right){{,L}^{\prime }}_1\left(2\right),{L^{\prime }}_1\left(3\right),\dots .{L^{\prime }}_1\left(\mathcal{l}\right)\dots {L^{\prime }}_1\left(\mathcal{L}\right)\\ {L^{\prime }}_2\left(1\right){{,L}^{\prime }}_2\left(2\right),{L^{\prime }}_2\left(3\right),\dots .{L^{\prime }}_2\left(\mathcal{l}\right)\dots {L^{\prime }}_2\left(\mathcal{L}\right)\\ \begin{array}{c}{L^{\prime }}_3\left(1\right){{,L}^{\prime }}_3\left(2\right),{L^{\prime }}_3\left(3\right),\dots .{L^{\prime }}_3\left(\mathcal{l}\right)\dots {L^{\prime }}_3\left(\mathcal{L}\right)\\ \begin{array}{c}\begin{array}{ccc}⋮& \begin{array}{ccc}⋮& \begin{array}{ccc}⋮& \begin{array}{ccc}\ddots & \ddots & \ddots \end{array}& ⋮\end{array}& \ddots \end{array}& ⋮\end{array}\\ {L^{\prime }}_m\left(1\right){{,L}^{\prime }}_m\left(2\right),{L^{\prime }}_m\left(3\right),\dots .{L^{\prime }}_m\left(\mathcal{l}\right)\dots {L^{\prime }}_m\left(\mathcal{L}\right)\end{array}\\ \begin{array}{c}\begin{array}{ccc}⋮& \begin{array}{ccc}⋮& \begin{array}{ccc}⋮& \begin{array}{ccc}\ddots & \ddots & \ddots \end{array}& ⋮\end{array}& \ddots \end{array}& ⋮\end{array}\\ {L^{\prime }}_N\left(1\right){{,L}^{\prime }}_N\left(2\right),{L^{\prime }}_N\left(3\right),\dots .{L^{\prime }}_N\left(\mathcal{l}\right)\dots {L^{\prime }}_N\left(\mathcal{L}\right)\end{array}\end{array}\end{array}\right]$$

(13)

where $X_{ }$ contains $N\times \left(\mathcal{L}\times D\right)$ load profiles with N input load series, $\mathcal{L}$ is the count of the augmented series, and D units of daily load profiles are presented in the load series.

The newly generated homogenous load series was treated as an image-like input and was convolutionally operated using CNN filters. Although CNN filters are resistant to learning over noisy information, $X_{ }$ provides the necessary and sufficient information to predict the high demand for a few hours. The convolved data are passed to the pooling layer, which is vital for selecting the maximum pointing information to update the weight and bias in the CNN layers. The ending of the CNN layers is attached to a fully connected layer that receives all the higher-dimensional learning weights in a one-dimensional space. Eventually, this layer performs the forecasting operation for a multihousehold load in one time-step. To perform the training operation, the proposed model provides multihousehold load profiles ${\widehat{Y}}_t$, which receive the actual multihousehold load profiles $Y_t$. The two profiles can be expressed as:

$${\widehat{Y}}_t=\left[\begin{array}{c}\begin{array}{ccc}{\widehat{l}}_{1,D, 1}& \begin{array}{ccc}{\widehat{l}}_{1,D, 2}& \dots & \begin{array}{cc}{\widehat{l}}_{1,D, t}& \dots \end{array}\end{array}& {\widehat{l}}_{1,D, T}\end{array}\\ \begin{array}{c}\begin{array}{ccc}{\widehat{l}}_{2,D, 1}& \begin{array}{ccc}{\widehat{l}}_{2,D, 2}& \dots & \begin{array}{cc}{\widehat{l}}_{2,D, t}& \dots \end{array}\end{array}& {\widehat{l}}_{2,D, T}\end{array}\\ \begin{array}{ccc}⋮& \begin{array}{ccc}⋮& \ddots & \begin{array}{cc} \ddots & ⋮\end{array}\end{array}& ⋮\end{array}\\ \begin{array}{c}\begin{array}{ccc}{\widehat{l}}_{m,D, 1}& \begin{array}{ccc}{\widehat{l}}_{m,D, 2}& \dots & \begin{array}{cc}{\widehat{l}}_{m,D, t}& \dots \end{array}\end{array}& {\widehat{l}}_{m,D, T}\end{array}\\ \begin{array}{ccc}⋮& \begin{array}{ccc}⋮& \ddots & \begin{array}{cc} \ddots & ⋮\end{array}\end{array}& ⋮\end{array}\end{array}\end{array}\\ \begin{array}{ccc}{\widehat{l}}_{N,D, 1}& \begin{array}{ccc}{\widehat{l}}_{N,D, 2}& \dots & \begin{array}{cc}{\widehat{l}}_{N,D, t}& \dots \end{array}\end{array}& {\widehat{l}}_{N,D, T}\end{array}\end{array}\right]$$

(14)

$$Y_t=\left[\begin{array}{c}\begin{array}{ccc}{l}_{1,D, 1}& \begin{array}{ccc}{l}_{1,D, 2}& \dots & \begin{array}{cc}{l}_{1,D, t}& \dots \end{array}\end{array}& l_{1,D, T}\end{array}\\ \begin{array}{c}\begin{array}{ccc}{l}_{2,D, 1}& \begin{array}{ccc}{l}_{2,D, 2}& \dots & \begin{array}{cc}{l}_{2,D, t}& \dots \end{array}\end{array}& l_{2,D, T}\end{array}\\ \begin{array}{ccc}⋮& \begin{array}{ccc}⋮& \ddots & \begin{array}{cc} \ddots & ⋮\end{array}\end{array}& ⋮\end{array}\\ \begin{array}{c}\begin{array}{ccc}{l}_{m,D, 1}& \begin{array}{ccc}{l}_{m,D, 2}& \dots & \begin{array}{cc}{l}_{m,D, t}& \dots \end{array}\end{array}& l_{m,D, T}\end{array}\\ \begin{array}{ccc}⋮& \begin{array}{ccc}⋮& \ddots & \begin{array}{cc} \ddots & ⋮\end{array}\end{array}& ⋮\end{array}\end{array}\end{array}\\ \begin{array}{ccc}{l}_{N,D, 1}& \begin{array}{ccc}{l}_{N,D, 2}& \dots & \begin{array}{cc}{l}_{N,D, t}& \dots \end{array}\end{array}& l_{N,D, T}\end{array}\end{array}\right]$$

(15)

where $l_{D+1, t}^m$ and ${\widehat{l}}_{D+1, t}^m$ are the actual and predicted hourly load consumptions at time t. The optimizing equation developed for the proposed multihousehold load forecasting algorithm can be written as:

$$Minimize\sqrt{\sum _j^{ }\sum _{m=1}^N\sum _{t=1}^T{\left(l_{m,D, t}^{ }-{\widehat{l}}_{m,D, t}^{ }\right)}^2}$$

(16)

where j denotes the number of predefined training epochs in the proposed CNN model.

Figure 4 shows the overall block diagram of the proposed multihousehold load forecasting method. The peak consumed loads in each load series generated a significant difference between the load profiles. The reordering of each load series based on the CMM was effective in maintaining a low shifting variance in the input multihousehold dataset. Maintaining this variance is useful for achieving a high forecasting accuracy for the multiple load series, where every household forecasting result within a different household group can be obtained simultaneously. The N-load series is maintained by ordering based on the tail PDFs for multihousehold load forecasting. With each ordered load series as the input, an augmented load series was generated and attached by maintaining the minimum shifting variance in the dataset. Consequently, the consecutive household load series from the ordering process presented the minimum difference in its tail characteristics.

Figure 5 shows the flowchart of the proposed methodology for multihousehold load forecasting, comprising three stages: (a) CMM load-series ordering, (b) data augmentation, and (c) the CNN framework. In the first stage, multiple load series are ordered using the lowest CMM values. Data augmentation is applied in the second stage to maintain a constant or minimum shifting variance in the input dataset. Finally, in the third stage, N load profiles are predicted for N multiple household load series using the CNN hyperparameters.

4. Simulation Results

4.1. Data Description and CNN Architecture Modeling

A large amount of multiload series data was collected from 1181 residential households with AMI installed, located in Seoul, South Korea, for the experiment. The AMI recording began in September 2016 and ended in August 2017.

Figure 6 shows the model of the proposed CNN architecture for the prepared dataset containing an input N household load series. The training dataset contained each load series data up to day $D$, whereas the testing dataset contained only multiload profiles of the next day, $D+1$.

Table 1. Hyper-parameter modeling for the tested core DL framework.

Method	Hidden Layers	Hidden Nodes/Filters	No. of Epochs(Iteration)	Activation Function	Optimizer
CNN	2	96	150	ReLU	RMSprop
LSTM	2	72	300	Sigmoid, tanh	RMSprop
CNN–LSTM	2	96	150	ReLU	RMSprop

lists the hyperparameters of the tested DL framework for performing multihousehold forecasting operations. The program was written in Python using the Keras and TensorFlow libraries [35,36,37]. The program was tested on an i7 CPU-based computer, and numerous forecasting results were obtained using Google Colab, which is a GPU-based cloud application. Three core DL architectures, namely CNN, LSTM, and CNN–LSTM, were employed at the start of the proposed method to determine which DL benchmark produced the best forecasting results.

4.2. Performance Evaluation Metrics

To generalize the proposed forecasting methodology, each input dataset containing 10, 20, 50, 80, and 100 household load series was used for performance evaluation. To compute the load forecasting results for the m-th household in each multiple-household group, the following performance metrics were used for each DL methodology for comparison:

$${\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}}_{\mathrm{m}}={\left(\frac{1}{T}\sum _{t=1}^T\frac{\left|l_{D+1, t}^{m\prime }-{\widehat{l}}_{D+1, t}^{m\prime }\right|}{l_{D+1, t}^{m\prime }}\times 100\mathrm{\%}\right)}_m$$

(17)

$${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{m}}={\left(\sqrt{\frac{1}{T}\sum _{t=1}^T{\left(\left(l_{D+1, t}^{m\prime }-{\widehat{l}}_{D+1, t}^{m\prime }\right)\right)}^2}\right)}_m$$

(18)

$${\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}}_{\mathrm{a}\mathrm{v}\mathrm{g}}=\frac{1}{N}\sum _{m=1}^N{\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}}_{\mathrm{m}}$$

(19)

$${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{a}\mathrm{v}\mathrm{g}}=\frac{1}{N}\sum _{m=1}^N{\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{m}}$$

(20)

where $l_{D+1, t}^{m\prime }$ and${\widehat{l}}_{D+1, t}^{m\prime }$ are the actual and predicted load consumptions of the next day D + 1 for hour t; ${\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}}_{\mathrm{m}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{m}}$ are the MAPE and root-mean-square error (RMSE) for testing household m. ${\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}}_{\mathrm{a}\mathrm{v}\mathrm{g}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{a}\mathrm{v}\mathrm{g}}$ represent the average MAPE and RMSE results, respectively, to generalize the forecasting results of the proposed multihousehold methodology.

4.3. Single-Household Load Forecasting

To evaluate the performance results for an individual household, the first ten household load series from the first group were used for forecasting using the CNN, LSTM, and CNN–LSTM methodologies. The input dataset was ordered, augmented, and prepared using the proposed methodology. Figure 7 shows the actual and predicted daily load profiles of the nine tested load series for the next day D + 1. In Figure 7, the forecasting results of the tested load profiles are arranged from lower-tail PDF load series to the longer-tail PDF load series. The CNN produced the most accurate prediction results, as shown in Figure 7g–i, which had a longer-tail PDF. Hence, the proposed CNN architecture predicted accurate results for the peak load consumption, despite the unusually high load demand observed for a few hours in a day.

Table 2. Comparison of the forecasting performance in terms of the MAPE (%) and RMSE (kWh) for the first household group.

Household	MAPE (%)			RMSE (kWh)
	LSTM	CNN	CNN–LSTM	LSTM	CNN	CNN–LSTM
1	18.45	22.55	23.18	0.071	0.093	0.090
2	17.43	16.49	18.32	0.106	0.105	0.122
3	21.11	27.72	27.14	0.078	0.104	0.103
4	32.18	23.67	26.10	0.109	0.108	0.142
5	32.81	20.66	28.10	0.164	0.136	0.162
6	18.56	26.10	25.74	0.125	0.155	0.152
7	45.10	32.82	37.85	0.130	0.111	0.116
8	29.37	27.53	28.23	0.137	0.135	0.179
9	35.27	31.15	32.06	0.256	0.208	0.213
10	22.79	22.63	22.83	0.208	0.207	0.208

presents the forecasting MAPE and RMSE results for the ten households. Among the ten load series, the CNN outperformed for six of them that had longer tail characteristics in their PDFs. In terms of the average forecasting results, the CNN enhanced the performance results, with the forecasting ${\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}}_{\mathrm{a}\mathrm{v}\mathrm{g}}$ being higher than 2.3% and forecasting ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{a}\mathrm{v}\mathrm{g}}$ being higher than 0.032 kWh compared with the LSTM and CNN–LSTM algorithms, respectively.

4.4. Average Single-Household Load Forecasting

The average single-household load forecasting results are effective for generalizing the performance of the proposed multihousehold load forecasting model. Figure 8 shows the box-plot results of the average forecasting performance result of four different household groups for a week. Each box plot represents the average MAPE results from the household groups containing 30, 50, 80, and 100 household load series. The box plot reveals that the performance of the CNN varies much less than those of LSTM and CNN–LSTM, despite the increase in the number of load series.

Table 3. Average forecasting performance results in terms of the MAPE (%) and RMSE (kWh) for the different multi-household groups in July 2017.

Multihousehold Groups	MAPE (%)			RMSE (kWh)
	LSTM	CNN	CNN–LSTM	LSTM	CNN	CNN–LSTM
N = 10	25.64	22.33	26.95	0.145	0.137	0.147
N = 20	31.83	27.32	30.27	0.151	0.157	0.177
N = 30	34.49	27.67	30.37	0.176	0.148	0.161
N = 50	37.74	31.16	35.69	0.237	0.172	0.209
N = 80	36.75	28.39	34.82	0.202	0.155	0.201
N = 100	38.36	32.43	36.16	0.171	0.131	0.162

shows the average MAPE performance results of all the multihousehold groups in July 2017. The month of July was considered for the observation because it contained mostly long-tail PDF load profiles in the load series compared with the other months in South Korea. Each result can be generalized to each household in the household group. As a result, the CNN-based model demonstrated a better performance, with the forecasting MAPE being higher than 3.5% and the forecasting RMSE being higher than 0.035 kWh for the general case.

4.5. Comparison with Pooled and Data Augmentation Methodologies

Pooled and data augmentation methodologies are efficient forecasting strategies for the electricity load of a single household. Therefore, they were compared with the proposed methodology for the same amount of input load series. For a robust comparison, four weeks from each season were selected to evaluate the forecasting performance of the 20, 50, 80, and 100 household load series.

Table 4. Comparison of the average forecasting MAPE (%) and RMSE (kWh) performance results between existing and proposed forecasting methodologies for all household groups across all seasons in 2017.

Multihousehold Groups	Season	MAPE (%)			RMSE (kWh)
		Pooled CNN	Augmented CNN	Proposed	Pooled CNN	Augmented CNN	Proposed
N = 20	Spring	36.23	29.28	25.46	0.236	0.198	0.122
	Summer	39.25	31.26	28.13	0.245	0.204	0.111
	Autumn	41.25	35.36	27.32	0.275	0.207	0.157
	Winter	42.63	33.23	26.63	0.277	0.213	0.118
	Average	39.84	32.28	26.82	0.258	0.205	0.127
N = 50	Spring	34.23	29.12	23.38	0.244	0.191	0.092
	Summer	36.15	30.93	27.92	0.251	0.201	0.107
	Autumn	39.37	33.72	31.16	0.297	0.218	0.172
	Winter	38.42	29.82	23.69	0.281	0.209	0.095
	Average	37.04	30.89	26.53	0.268	0.204	0.116
N = 80	Spring	33.15	28.12	22.26	0.256	0.181	0.075
	Summer	36.52	33.85	28.39	0.274	0.193	0.155
	Autumn	35.26	30.09	25.15	0.269	0.212	0.095
	Winter	34.29	27.06	22.15	0.264	0.195	0.088
	Average	34.80	29.78	24.48	0.265	0.195	0.103
N = 100	Spring	32.25	29.65	27.75	0.233	0.171	0.082
	Summer	34.68	31.17	27.99	0.241	0.182	0.081
	Autumn	36.16	33.46	31.11	0.278	0.196	0.121
	Winter	35.17	31.16	28.30	0.271	0.193	0.098
	Average	34.56	31.36	28.78	0.255	0.185	0.095

presents a comparison of the forecasting performances of the proposed methodology, pooled CNN, and augmented CNN in terms of ${MAPE}_{avg}$ and ${RMSE}_{avg}$. The proposed methodology shows noteworthy improvements of more than 8% and 0.108 kWh in ${MAPE}_{avg}$ and ${RMSE}_{avg}$, respectively. These results are significant for single-household day-ahead forecasting.

4.6. Augmentation vs. Ordered Strategies

Because the proposed method uses both augmentation techniques and ordered strategies, its forecasting evaluations were studied separately. Both the methods were found to be suitable for multihousehold load forecasting.

Table 5. Comparison of the forecasting performance results in terms of MAPE (%) and RMSE (kWh) between augmentation, ordered, and proposed methodologies.

Method	MAPE (%)				RMSE (kWh)
	N = 20	N = 50	N = 80	N = 100	N = 20	N = 50	N = 80	N = 100
Augmentation	30.09	28.92	26.15	29.31	0.217	0.117	0.096	0.087
Only ordered	33.25	32.48	31.36	34.21	0.184	0.164	0.144	0.126
Proposed	28.13	27.16	24.29	26.99	0.111	0.110	0.095	0.082

presents the average performance results in terms of ${MAPE}_{avg}$ and ${RMSE}_{avg}$ for the augmented, ordered, and proposed methods. The comparison shows that creating a hybrid configuration based on augmentation and reordering using the CMM for the CNN brings about an improvement of more than 2% in the MAPE. Hence, both data enlargement from augmentation and ordering based on the tail PDF of multiple load series are useful for producing robust forecasting results of the multihousehold load.

5. Discussion

In this study, we developed a methodology for identifying synergies in energy consumption patterns between households and presented simulation results demonstrating the effectiveness of the proposed approach.

As each household’s electricity consumption differed from others’, especially in peak load values for a few hours in a day, the difference was explicitly illustrated when analyzing the tail PDF of the load series. The statistics-based CMM index was able to order the load series by computing the overall moment measure of the PDF generated from the load series. The electricity demand data used consisted of data from groups of households with ensured similarity. The ordered dataset was enlarged by using a data augmentation methodology that is effective in fulfilling the data requirements of the CNN. As a result, the proposed approach showed a significant improvement of more than 8% and 3% in forecasting accuracy in terms of the MAPE over existing CNN methods for the typical individual household.

Despite efforts to deliver higher forecasting results, the proposed model exhibits some unaddressed aspects and limitations. While the proposed method demonstrated utility in a case study involving 100 households over a one-year period, it is limited by the availability of only one year’s worth of data. Implementing the proposed methodology requires access to comprehensive data, which may not always be possible. In cases where access is restricted or data availability is limited, applying the proposed approach to groups of households with slightly different characteristics remains unexplored territory. Moreover, alternative methods for identifying synergies could yield better results. Exploring these methods could enhance the robustness of the proposed approach.

In conclusion, the performance of small-scale electricity demand prediction could be improved by aligning input data and adjusting the learning sequence with CMM, along with the combination of data augmentation. However, the underlying principles behind the synergy of this combination have not been analyzed. This study did not explore alternative methods for data augmentation techniques for better synergy with rearranging input data; therefore, further research in this area will be necessary in the future.

6. Conclusions

Residential load series contain granular load consumption values. Therefore, their performance results are comparatively higher, particularly in terms of the MAPE. Existing DL-based forecasting methodologies for load series are often inefficient due to the high load demand observed for a few hours in a given day, which is explicitly depicted as a long tail in the PDF of the load series. A multihousehold forecasting method based on load-series ordering, data augmentation, and CNN seems to be most suitable.

The main results of this study can be summarized as follows:

• Multihousehold forecasting research is significant for residential living areas such as apartment blocks, community buildings, or even small towns where each household’s electricity consumption is different from others’;
• The electricity consumption characteristics of each household are explicitly demonstrated in the load series PDF. Based on tail information in each load series’ PDF, CMM ordering was developed;
• The proposed multihousehold load forecasting technique uses the CMM to index the order of the load series. The CMM produces an ordered load series by computing the overall moment measure of the PDF generated from the load series;
• The ordered multiple series maintain a minimum or constant shifting variance in the load-series dataset inputted to the CNN. A data augmentation method is used for each load series of the input multihousehold group to solve the existing data requirement issues of the CNN;
• A comparison of the average forecasting results for single households helped validate the efficacy of the proposed method;
• A higher load forecasting accuracy for both single and multiple households could be realized by utilizing the CMM, data augmentation, and CNN, which solves the peak load demand issue encountered in the residential sector.

In this manuscript, we have discussed multihousehold forecasting using moment information and data augmentation. In the future, we would like to explore deep statistical methodologies and incremental learning for the multihousehold electricity consumptions.