2.1. Power Demand Forecasting Using Deep Learning
Most of the techniques used to predict power demand have included Recurrent Neural Network (RNN)-based LSTMs, which have used on time series data and natural language processing [
4,
5]. In particular, CNNs have produced high classification and recognition performance in the field computer vision and pattern recognition [
6,
7,
8] and have also been demonstrated to be effective in various fields involving time series data such as language data, human behavior pattern data, energy load data etc. [
9,
10,
11].
An ensemble deep learning method using several deep learning networks was described in [
9]. In this paper, based on the observation that the output value changes when the number of epochs is changed, output values were obtained by using different epochs for each Deep Belief Network (DBN) over several DBNs. The authors then constructed an ensemble deep learning network using the output from a Support Vector Regression (SVR) as input and showed 4% and 15% better performance in predicting power demand than was obtained using SVR and DBN, respectively. In [
10,
11], time series data were processed using a multi-channel Deep CNN model which learns features from an individual univariate time series in each channel, and combines information from all channels to produce a feature representation at the final layer. This method was also applied to human behavior pattern data and ECG (electrocardiogram) data.
Most of the studies using artificial neural networks for power demand forecasting have used data from residential buildings, commercial or office buildings. Experiments on solar powered buildings have also been conducted [
12]. These experiments used power demand data from business days, non-business days, and seasonal data. The model was optimized by adjusting the numbers of features and neurons. A study using two types of artificial neural network models was described in [
13]. One model used a pre-trained Restricted Boltzmann machine (RBM), the other used a Rectified Linear Unit (ReLU) without pre-training. These models obtained better results in predicting the future 24 h than ARIMA or Shallow Neural Network (SNN). For small power systems with non-linear and non-critical characteristics, [
14] used an LSTM model to predict power demand. The amount of power used in residential areas was divided into smaller groups, down to individual households. A study using the LSTM model to forecast the power demand for each household was conducted [
15]. The authors forecasted the amount of power needed in the future based upon the current amount of power produced in a solar power plant. The LSTM, DBN, and Auto-LSTM were used in the experiment, and the Auto-LSTM had the best performance. Reference [
16] proposed the Augmented LSTM (ALSTM) network method, which enhances the Auto-LSTM network method used in [
17] by combining the AutoEncoder and LSTM. A study was carried out on forecasting power demand after 60 h by constructing the encoder and decoder using a Sequence to Sequence (S2S) structure-based on an LSTM. This work reported in [
18] (
Figure 1). This study mapped the date of the next day, not the current date, to power values. When power values and dates of the same day are used as inputs, the predicted values of the same day simply follow the pattern of the previous power values.
In order to improve the accuracy of the estimate of power demand for individual households, [
19] proposed Pooling-based Deep-RNN (PDRNN). This study used the power data from the target household as well as those from neighboring power areas. The root mean square error (RMSE) of PDRNN was much lower: 19.5%, 13.1% and 6.5% compared to results from the ARIMA, SVR and classical deep RNNs.
To predict the power demand for individual buildings, the network in [
20] was constructed using only a CNN, and was evaluated with only changing parameters. The data model used in [
20] differs from existing methods in that only the power data is input to the first CNN layer, and the final fully connected layer incorporates information such as date and temperature, to predict power demand. In order to evaluate its performance, the proposed network, a Support Vector machine (SVM) and RBM were compared [
20]. Experimental results showed better performance than previous methods, but the network model in [
20] was not better than the method described in [
18]. The use of CNN-based bagging techniques for smart grid load forecasting was reported in [
21]. In reference [
22], the USA District public consumption dataset and load dataset for 2016 provided by the Electric Reliability Council of Texas were processed using multiple CNNs to forecast power demand.
In the area of Natural Language Processing (NLP), RNNs, which are excellent for time series data processing, are primarily used. In order to improve an RNN’s performance, it is necessary to carefully select useful contextual information. Reference [
23] conducted a study to predict where users should move next by selecting time and space as the contextual information, in order to achieve better results than traditional RNN models. Similarly, reference [
24] introduced an RNN which is dependent on contextual information. In this study, when using input words to predict the next word, a feature layer with context information about the sentence topic was added.
2.2. Approaches Based on a Hybrid Network Model
One of the hybrid network structures for power demand forecasting is the CLDNN (a unified architecture of CNN, LSTM, and DNN) structure proposed in [
25]. In this model as shown in
Figure 2, LSTM layers were stacked on top of a CNN to create a hybrid network. This model was proposed for natural language processing, and the results on the power demand prediction problem showed limited prediction accuracy.
In reference [
26], CNNs and LSTMs were used together to construct a framework consisting of two phases, to estimate power demand (
Figure 3). The first function of the CNN layer is to extract the features of the power data, and the second function is to transform the one-dimensional power data into a multidimensional dataset by using the output of the CNN as input to the second phase, LSTM. The results are output through the dropout [
27] layer. In reference [
26], multi-step forecasting was performed, unlike traditional power demand forecasting methods based on a one-step forecasting.
Hybrid network studies for predicting power demand were also reported in [
28,
29]. In reference [
28], the authors transformed a dataset into 2D images and used those images as inputs to a CNN-RNN model. The accuracy of the CNN-RNN was 10% and 26% higher than that of an LSTM and an ANN [
30], respectively. Another study with a CNN-LSTM based hybrid framework was proposed in [
29]. In this study, CNN and LSTM were arranged horizontally and the characteristics of the input data were extracted separately. After feature extraction by the CNN and the LSTM, the outputs of the two networks were concatenated in the merge layer of a feature-fusion layer.
In this paper, we propose the (
c,
l)-LSTM+CNN hybrid prediction model. As discussed in
Section 3.2, we place multi-LSTM networks at the front to extract feature sets. Then, we create an ensemble by adding a CNN layer after the LSTMs in order to produce the final output.