Forecasting of Residential Energy Utilisation Based on Regression Machine Learning Schemes

Abstract

Energy utilisation in residential dwellings is stochastic and can worsen the issue of operational planning for energy provisioning. Additionally, planning with intermittent energy sources exacerbates the challenges posed by the uncertainties in energy utilisation. In this work, machine learning regression schemes (random forest and decision tree) are used to train a forecasting model. The model is based on a yearly dataset and its subset seasonal partitions. The dataset is first preprocessed to remove inconsistencies and outliers. The performance measures of mean absolute error (MAE), mean square error (MSE) and root mean square error (RMSE) are used to evaluate the accuracy of the model. The results show that the performance of the model can be enhanced with hyperparameter tuning. This is shown with an observed improvement of about 44% in accuracy after tuning the hyperparameters of the decision tree regressor. The results further show that the decision tree model can be more suitable for utilisation in forecasting the partitioned dataset.

1. Introduction

Energy forecast plays an important role in energy provisioning planning. Accurate forecasting can enable better power dispatch and emergency procurement of energy if the forecast energy utilisation exceeds the available capacity. Furthermore, if it is carried out on a long-term basis, it can provide valuable insights into capacity planning, which is becoming necessary in light of the increased deployment of variable energy sources [1]. If performed on a short-term basis (i.e., minutes to a day ahead), it can enable the proper coordination of available sources to meet the demand [2]. Given that energy supplies are becoming deregulated, forecasting their utilisation becomes more imperative.

Due to the importance of this task, scholars have proposed various schemes to deliver the most suitable method for energy utilisation prediction [3]. Predominately, machine learning has been used to forecast energy utilisation [2,3,4,5,6,7,8] followed by statistical analysis [9,10,11,12]. Various structures of artificial neural networks [13,14] have been used to forecast energy utilisation. In [15], a deep recurrent neural network scheme with attention was used on several buildings. In the scheme, it was shown that one month of data can give satisfactory results. A hybrid scheme composed of the grasshopper optimisation technique (a meta-heuristic algorithm) and a multilayer feed-forward neural network was used in [16] for the short-term forecasting of energy utilisation. An improvement on the classical neural network was proposed by Khwaja et al. [17]. The proposed scheme is based on an ensemble machine learning technique. The results of the study showed a reduction in bias and variance. Eskandari et al. [18] deployed convolution neural networks (CNN) to extract both the temperature and load features. The features are fed into a long short-term memory (LSTM) network and gated recurrent unit (GRU) for hourly load forecasting. The proposed method showed superiority in two different datasets. The same scheme was tested on a real-world dataset in [5,19], and it was found that the mean absolute percentage error (MAPE) and the root mean square error (RMSE) of the model are low as compared to back-propagation neural networks. Similar observations were made in [20,21,22] using an optimised LSTM, a GRU and a recurrent neural network (RNN).

A deep neural network (DNN)-based load forecasting model was proposed and used in a study conducted by Ryu et al. [23]. The dataset fed into the model was the individual’s energy utilisation and their local metrological data. Despite the success achieved by these neural network models, in [24], a scheme to further enhance the accuracy of their forecasting capabilities is given. Genetic algorithms and particle-swarm optimisation were used to learn the hyperparameters for load forecasting.

Although it seems that neural network-based schemes dominate the forecasting of short-term electricity [25], schemes based on genetic algorithms and particle-swarm optimisation are also widely used to learn the hyperparameters for load forecasting. Schemes such as support vector regression [26,27] and random forests [28] have also been used. However, these schemes have poor performance when compared with attention-based CNN-LSTM-bidirectional long short-term memory (BiLSTM) [29], and empirical wavelet transform and long short-term memory [30]. It is apparent that substantial attention has been given to the subject of energy demand forecasting. The motivation for this varies from the imperatives of the changing energy landscaping and adequate planning. The importance of this subject is underpinned by the constant improvement of the methods used to improve the accuracy of the forecasting. However, this has been limited to the tuning and adapting of various methods. There is no substantial attention given to changes that occur in the short term (e.g., seasonal), which can affect the energy utilisation considerably. This could result in inaccurate forecasting, which could be detrimental to the energy systems planning.

As a result, this paper presents a scheme based on regression for forecasting energy utilisation. The regression schemes used in this paper are based on the tree-like model and ensemble learning method. The proposed scheme is trained on datasets that are partitioned into annual and seasonal categories to take into account the variations in short-term forecasting. The outline of the paper is as follows: Section 2 outlines methods used in this paper. The data collection and simulation methods are discussed in this section. In Section 3, the results obtained in this study are presented and discussed. Finally, concluding remarks are given in Section 4.

2. Methods

2.1. Data Collection

The dataset used comprises energy utilisation (in kW) of a typical middle-class household in Pretoria (−25.724429, 28.175817), South Africa [31] collected hourly for a period of one year in 2019. The hourly aggregated data gives the total energy utilisation for that one hour. The data were collected through a single phase 80-amp meter with a PLC communication. The total energy used for the year was found to be $2315.21$ kW with a daily average consumption of $6.24$ kWh. The average hourly utilisation is presented in Figure 1a and the annual distribution of energy is shown in Figure 1b.

2.2. Data Preparation

The set used in this study consists of 8760 observations, representing an hourly sampling rate for a period of a year. For the purpose of this study, datasets were partitioned into two broad categories. The first partition consists of 365 partitions representing the daily category. The second consists of four partitions representing seasonal energy consumption categories. The South African seasonal calendar was used to partition the dataset in order to investigate the changes as a result of seasonal changes. To develop the model, the data are subdivided into training and test sets. The subdivision of the dataset was performed using the 70/30 principle. That is, 70% of the dataset was set to be the training data while the remaining 30% was set to be the testing set [32]. Following the partitions, the data were checked for outliers to ensure that they did not influence the performance of the models. The boxplot method was used to detect the outliers and for their subsequent removal. Figure 2 shows the boxplot of the dataset and it can easily be seen that there are outliers in the dataset, which are represented by the data points above the 75th percentile of the data.

Subsequent to the removal of the outliers, it can be seen in Figure 3 that the remaining data points are consistent and will therefore ensure that the forecasting model is not compromised.

2.3. Modelling and Simulation

Two machine learning regression models, the random forest (RF) and decision tree (DT) models, were trained and used to forecast the energy demand. The former averages the results of multiple trees to give a collaborative decision, making their outputs much stable. On the other hand, decision trees closely mirror the human decision-making process. The selection of these models was prompted by the fact that they have been found to outperform schemes such as autoregressive integrated moving average (ARIMA) [33] and seasonal auto-regressive integrated moving average (SARIMA) [34]. Additionally, these models have shown better performance [35] when compared to other regression schemes such as multiple linear regression [36]. To enhance the performance of the models, the hyperparameters of the models were tuned using a grid search algorithm. Algorithm 1 illustrates the grid search algorithm that was used. The computing was carried out using Python3 on a Windows platform with an 11th Gen IntelR Core (TM) i7 with an installed memory (RAM) of 16 GB.

Algorithm 1: Grid search for hyperparameter optimisation

The performance of machine learning regression schemes is usually assessed based on the mean absolute error (MAE), mean square error (MSE) and root mean square error (RMSE). Consequently, MAE, MSE and RMSE were used to assess the performance of the proposed models.

3. Results and Discussion

The total data points of energy consumption remaining in the dataset after outlier removal are presented in

Table 1. Data points after outlier removal.

Partition	Base Data Points	Remaining Data Points	Difference
Annual dataset	8760	7487	$1273\approx 14.5\%$
Summer dataset	2114	1808	$306\approx 14.5\%$
Autumn dataset	2185	1882	$303\approx 13.8\%$
Winter dataset	2185	1776	$409\approx 18.7\%$
Spring dataset	2161	1849	$312\approx 14.4\%$

. It can be seen that the highest percentage difference is observed during the winter months. This is a result of higher heating energy requirements during these months.

The Spring season comes second to the winter months in terms of the largest percentage difference. This is consistent with weather patterns in South Africa, as the first month of the spring season tends to be relatively cooler [37]. Table 1 further shows that the aggregate number of data points lost from the dataset based on the seasonal partition is higher as compared to the annual analysis. This shows that partitioning the dataset gives the opportunity to deal with each season’s peculiarity to avoid over-generalising.

The performance error of the machine learning models is shown in

Table 2. Model performance evaluation for annual dataset.

Partition	Model Description	$\mathit{MAE}$	$\mathit{MSE}$	$\mathit{RMSE}$
Annual dataset	RF—Base Model	$0.0679$	$0.0070$	$0.0840$
Annual dataset	DT—Base Model	$0.0903$	$0.0138$	$0.1174$
Annual dataset	RF—Tuned Model	$0.0697$	$0.0063$	$0.0855$
Annual dataset	DT—Tuned Model	$0.0644$	$0.0060$	$0.0782$

for the complete dataset (i.e., without outliers). It is evident that the performance of the decision tree marginally improved with hyperparameter optimisation as compared to the random forest model. This observation is made on all performance metrics that are applied in this work.

The comparison of the performances of these models is further presented illustratively in Figure 4a,b for the base models and Figure 5a,b for the tuned models.

It can easily be seen that the decision tree model exhibits the same consistent performance for both the base and the optimised models. However, it can be seen that the optimised random forest model suffers from generalisation/overfitting when optimised. This can be seen with the performance of the model worsening by about 3%. This can easily be a result of the candidate parameter selected to build the grid search. This can be improved through a randomised parameter selection, where the grid search is exposed to more parameters than a limited selection. It can further be seen that the performance of the decision tree model improves by at least 44%.

Table 3. Model performance evaluation for seasonal sub-datasets.

(a) Model performance evaluation for summer sub-dataset
Model description	$\mathit{MAE}$	$\mathit{MSE}$	$\mathit{RMSE}$
RF—Base Model	$0.0639$	$0.0072$	$0.0850$
DT—Base Model	$0.0850$	$0.0123$	$0.1110$
RF—Tuned Model	$0.0654$	$0.0082$	$0.0873$
DT—Tuned Model	$0.0694$	$0.0089$	$0.0851$
(b) Model performance evaluation for autumn sub-dataset
Model description	$\mathit{MAE}$	$\mathit{MSE}$	$\mathit{RMSE}$
RF—Base Model	$0.0683$	$0.0077$	$0.0876$
DT—Base Model	$0.0760$	$0.0105$	$0.1026$
RF—Tuned Model	$0.0709$	$0.0055$	$0.0896$
DT—Tuned Model	$0.0666$	$0.0054$	$0.0817$
(c) Model performance evaluation for winter sub-dataset
Model description	$\mathit{MAE}$	$\mathit{MSE}$	$\mathit{RMSE}$
RF—Base Model	$0.0552$	$0.0046$	$0.0681$
DT—Base Model	$0.0689$	$0.0071$	$0.0843$
RF—Tuned Model	$0.0566$	$0.0044$	$0.0690$
DT—Tuned Model	$0.0519$	$0.0046$	$0.0645$
(d) Model performance evaluation for spring sub-dataset
Model description	$\mathit{MAE}$	$\mathit{MSE}$	$\mathit{RMSE}$
RF—Base Model	$0.0547$	$0.0053$	$0.0726$
DT—Base Model	$0.0648$	$0.0088$	$0.0938$
RF—Tuned Model	$0.0550$	$0.0055$	$0.0728$
DT—Tuned Model	$0.0549$	$0.0054$	$0.0682$

presents the performance metrics for the models trained with the subsets of the dataset. It can be seen that the optimisation of hyperparameters still yields a substantial improvement in the performance of the random forest model. Contrary to the performance of the decision tree in the complete dataset, the optimisation of hyperparameters yielded a significant improvement in the performance of the model. This implies that the decision tree is suitable for forecasting the partitioned dataset.

4. Conclusions

Energy utilisation in residential areas has an inherent level of uncertainty, especially in recent times with hybrid working modes, which exacerbate the uncertainties and introduce inconsistencies in energy utilisation patterns. This makes the forecasting of energy utilisation important and challenging in the light of the existence of several inconsistencies. In this work, we used the boxplot method to deal with outliers (inconsistencies) in the dataset. Tree-based and ensemble learning regression schemes were trained on a dataset of yearly observation and also seasonal partitions of the same dataset. The results show an improvement in the forecasting capabilities of the random forest scheme on the yearly dataset with an optimised hyperparameter. By contrast, decision trees were only seen to improve when trained on seasonal partitions of the dataset and optimised hyperparameters. Future works will present the modelling and forecasting of energy utilisation from diverse households (i.e., rural, middle-class and upper-class households).