Using Industry 4.0’s Big Data and IoT to Perform Feature-Based and Past Data-Based Energy Consumption Predictions

Abstract

Industry 4.0 and its technologies allow advancements in communications, production and management efficiency across several segments. In smart grids, essential parts of smart cities, smart meters act as IoT devices that can gather data and help the management of the sustainable energy matrix, a challenge that is faced worldwide. This work aims to use smart meter data and household features data to seek the most appropriate methods of energy consumption prediction. Using the Cross-Industry Standard Process for Data Mining (CRISP-DM) method, Python Platform, and several prediction methods, prediction experiments were performed with household feature data and past consumption data of over 470 smart meters that gathered data for three years. Support vector machines, random forest regression, and neural networks were the best prediction methods among the ones tested in the sample. The results help utilities (companies that maintain the infrastructure for public services) to offer better contracts to new households and to manage their smart grid infrastructure based on the forecasted demand.

1. Introduction

The term “Industry 4.0” was first introduced at Hannover Fair in 2011. Subsequently, in 2013, the report entitled ‘Industrie 4.0′ developed by a research team recruited by the German government disseminated the term worldwide [1]. The term 4.0 refers to the previous three industrial revolutions. They were represented by advancements in coal and mechanical production in the late 18th century, the advent of electricity and the assembly line in the late 1800s, and finally, the computer and information technology revolution during and after the 1970s [1,2].

The fourth revolution describes many digital technologies and a decentralized production process, based mainly on the use of Radio-frequency identification (RFID), big data, cloud computing, smart sensors, machine learning, robotics, additive manufacturing, artificial intelligence, augmented reality, the Internet of things, material sciences and nanotechnologies [3,4]. These digital technologies enable a systematical deployment of Cyber-Physical Systems (CPS) and bring manufacturing into Industry 4.0 [5]. Industry 4.0’s objective is similar to previous revolutions and aims to increase productivity and enhance mass production using innovative technology [6] as well as advanced planning and control methods [7,8]. Although the “Industry 4.0” term is widely known and used, there are several denominations to describe similar industrial processes in the literature. Analogous concepts have been used with different denominations, such as advanced manufacturing [5], smart industry or factories [9,10], smart manufacturing [11], the industrial Internet of things [5,12], networking manufacturing [9], intelligent manufacturing [13,14], industrial Internet [15], some elements of the national plan carried out by the Chinese government entitled ‘Made in China 2025’ [4], and so on.

Digital technologies, present in Industry 4.0, are heavily used to improve smart cities’ services in terms of quality and efficiency and to help smart cities’ companies by supplying them with information for decision making [16]. A smart city may be described as a place where traditional connections and services are improved using digital technologies to create value for its inhabitants and businesses [17]. There are many opportunities for digital technologies in cities, such as transportation [18], healthcare [19] and buildings [20,21] as well as in energy [22]. Furthermore, the link between energy, sustainability, and smart cities is close because energy efficiency is crucial to increase sustainable urban development [23]. The use of digital technologies to improve traditional power distribution systems is also called a smart grid [24]. As a part of the smart grid, the Advanced Metering Infrastructure (AMI) is an essential component due to its capacity to collect, process, and transfer data through the Internet [24]. Smart meters are a vital component of AMI as a result of their role in the connection between the user and the smart grid [25]. Given the opportunity to apply Industry 4.0 technologies such as big data, the Internet of things, and artificial intelligence using smart meters as a data source, some research is being carried out to predict energy consumption. Such research uses many different approaches, such as clustering [26], conditional kernel density [27], neural networks [28], hierarchical probabilistic forecasts [29], deep learning, and support vector machines [30], among others.

Looking at the reliability, frequency, format, and availability of smart meters’ data [31], added to the smart meters’ importance for both the consumers and the AMI [32], and also taking into account the urgent need to manage resources’ supply and consumption efficiently, we can ask the following question: which is the most appropriate method to make energy consumption predictions based on household features and past energy consumption data? To answer this question, this article aims to test many forecasting methods to indicate an appropriate one to predict energy consumption using household smart meter data. The energy consumption forecast is applied in two different situations. The first situation aims to test different techniques to predict the energy consumption of new homes, for which there are no available previous consumption data. In this case, the energy consumption estimation is predicted based on home features and the appliances that will be available at home; this will be referred to from now on as feature-based prediction. The second one aims to identify an appropriate technique to predict energy consumption using previous consumption data. The approach intends to identify a proper technique to predict energy consumption using time-series data in this case. This will be referred to as past data-based prediction.

This paper is structured as follows: in Section 2, there is a brief review of regression and forecasting methods, Section 3 comprises the methods and materials, and results are shown in Section 4, with further discussion presented in Section 5. Finally, Section 6 offers conclusions and is followed by the References Section.

2. Literature Review

The literature shows that cyber-physical production networks operate automatically and smoothly with artificial intelligence-based decision-making algorithms in a sustainable manner, and Internet of Things-based manufacturing systems function in an automated, robust, and flexible manner [33]. So, we need to understand how IoT devices work along with decision-making algorithms in the context of energy data management.

2.1. Smart Meters

Internet of Things (IoT) technology is a formidable tool for industries, services, and users. This technology improves monitoring, control, customization, and prediction [34]. In the area of utilities (companies that maintain the infrastructure for public services), a successful case of using IoT-connected devices that add value to different stakeholders lies in smart meters [35,36]. These devices constitute an important part of the information technology in smart cities [37], being fundamental parts of smart water and electricity grids [31,38].

Smart meters represent an evolution of traditional, electromechanical meters [31,38]. Electromechanical meters feature manual reading and charging, physical connections, and analog displays. On the other hand, smart meters feature data storage and management, remote bidirectional metering and charging, end-to-end communication, blackout detection, diagnosis systems, In-House Displays (IHD), wireless connections, fast leak detection, and dynamic pricing according to supply and demand [31,38]. These features enable smart meters to become important data sources used in energy/water demand predictions, mainly alongside statistical prediction methods and machine learning.

Smart meters enable consumers to monitor their energy/water consumption in real-time and to produce their own energy, sending the surplus to the smart grid, among other benefits [39,40]. However, beyond the benefits for consumers, smart meters are essential for utilities to handle the increasing share of renewable sources in the energy matrix. It is necessary to adopt sustainable energy sources to mitigate the effects of global warming and climate change [41]. Nevertheless, sustainable sources such as wind, solar, and tidal are naturally generated with instability, and their peak production does not meet peak consumption [42]. Thus, smart meters are needed to gather data in real-time on what is generated, consumed, and stored to prevent and manage shortages [43]. Smart water meters have a similar role, detecting leaks and alerting the utilities as soon as they happen [38].

2.2. Energy Consumption Forecasting Methods

Many planning processes in the energy industry rely on forecasts of unknown future outcomes of relevant system variables. Besides forecasts of electricity prices [44,45], failure predictions for predictive maintenance [46,47], or sales forecasts of products [48,49], forecasts of resource and energy demand [50,51] are essential. This work aims to tackle two occasions on which utilities have to make predictions about energy consumption. The first is when someone moves to a new house and needs to sign an energy supply contract. The utility needs to evaluate how much energy that household is likely to consume, but there is no past consumption data to make a prediction. However, demographic and features data regarding the household may be used alongside some regression methods to predict monthly energy consumption.

Regression methods are supervised learning methods that predict continuous values based on characteristic features describing the behavior of a system. The simplest and most classic example of regression is linear regression. This work tests the accuracy of six regression methods to predict household energy consumption: linear regression, ridge regression, Bayesian ridge regression, support vector regression (scaled and unscaled), and random forest regression.

Ridge regression is commonly used when independent variables are highly correlated and is used in load forecasting as a comparison [52] or in improved versions [53]. Bayesian ridge is a special case of Bayesian linear regression, where the mean of a variable is a linear combination of other variables, and the posterior distribution can be approximated. It was used to reconstruct missing smart meter data [54] and to detect hacker attacks on smart grids [55].

Support vector machines are a class of machine learning methods developed in the United States at the end of the 20th century [56]. These models create vectors forming borders in a hyperspace based on a set of previously classified data to classify new data. The regression version, known as support vector regression (SVR) [57], creates a region in hyperspace where the data points are found. Recent examples of SVR are present in investigating the penetration of photovoltaic sources in microgrids [58] and the classification of household characteristics [59].

Random forest is an ensemble learning method that uses a large number of decision trees based on data characteristics to perform data classification and regression. In random forest regression, a special case of the random forest method, the mean or average prediction of the individual trees is returned [60]. Recent uses of this technique include helping small communities self-manage their electricity grid [61] and investigating energy disaggregation [62].

The other utility concern tackled by this work is when there is past energy consumption data when the utility needs to project future energy consumption for the following months. In this case, time-series forecasting methods can be used. Time-series forecasting methods are a special case of regression methods that try to predict future data values based on past values [63]. Here, we use five time-series forecasting methods: moving average, autoregressive integrated moving-average (ARIMA), exponential smoothing, neural network forecasting, and random forest forecasting. In addition, a naïve forecast (which replicates the last known data for the entire forecast period) is also used for baseline comparison.

Moving average is a common approach for modeling univariate time series, which specifies that the output variable depends linearly on a stochastic term’s current and various past values. ARIMA is an extended approach to moving average, which tries to transform non-stationary time-series into stationary ones and considers their previous values to predict the next ones. It is commonly used in recent studies about electric load prediction [64,65].

Exponential smoothing is a common technique for smoothing the time series using the exponential window function. Exponential functions assign exponentially decreasing weights over time instead of giving equal weights to past observations, as in the case of ARIMA. It is used as a benchmark for comparison in recent smart-meter data studies [27,66].

An artificial neural network, or simply a neural network, is a massively parallel distributed processor made up of simple processing units with a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in the sense that knowledge is acquired by the network from its environment through a learning process and that interneuron connection strengths, known as synaptic weights, are used to store the acquired knowledge [67]. The long short-term memory (LSTM) approach used in this work is also largely found in other recent studies for commercial building load forecasting, photovoltaic power generation, and residential load forecasting [66,68,69].

Random forest forecasting is a special case of random forest regression, where past data of a time series is given to forecasting future data. It is also commonly found in recent studies about load forecasting [70], peak prediction [71], and fault prediction [72].

3. Materials and Methods

The methodological procedure carried out in this work follows the Shearer (2000) proposal named Cross-Industry Standard Process for Data Mining (CRISP-DM) [73]. This procedure is a widely accepted neutral model, used in the data science literature, utilized, and reviewed in many different cases [74,75,76]. The CRISP-DM activities are organized in six stages: (i) business understanding, (ii) data understanding, (iii) data preparation, (iv) modeling, (v) evaluation, and (vi) deployment. The procedures are displayed following each of the first five steps proposed by CRISP-DM [73], and ways for implementing the sixth step are suggested in the discussion section.

3.1. Business Understanding

Utilities have the mission of generating, supplying, and distributing water, energy, or gas to their consumers. Utility infrastructure is the basis for other sectors of the economy, and utility networks represent the largest structures ever built by humanity [77]. However, the shift from fossil fuel production to renewable energy sources represents an immense challenge in terms of simultaneous management of what is produced, distributed, stored, and consumed [77]. Understanding energy consumption patterns and acting to decrease the infrastructure needed for peak usage will yield savings in the billions of dollars [78,79]. Thus, the need to understand how different residences with distinct characteristics consume natural resources, both in the present and in the future, is clear. Several Industry 4.0 technologies, such as big data, the Internet of things, and artificial intelligence, are used by utilities to manage energy consumption and to understand the consumption patterns of different configurations of households. In this sense, the smart meter is the main enabler of Industry 4.0 technologies for utilities.

In this work, the evaluation of how much energy a household will consume is completed with different regression methods based on the household’s features. The forecasting of how much energy a household will consume in the future was performed by time-series forecasting, using past energy consumed values of that household as a predictor for future values.

3.2. Data Understanding

After understanding the need for utilities, the next step was to find a database of resource consumption recorded by smart meters, which also brought information about the homes in which these meters are installed. The base chosen was the GEM HOUSE project [80], an open database available at the IEEE DataPort [80]. This project gathers the energy consumption of hundreds of German homes between 2018 and 2020. More precisely, the databases “all-users-daily-data.csv” and “household-information.csv” were used. The first database brings daily energy consumption data for each smart meter, with 829,521 rows of data in three columns, as illustrated in Figure 1.

The second base presents several characteristics of 552 households in which the smart meters are installed. These characteristics include data on the area of the residence, the number of people and children, as well as the heating and cooling systems of residences, and the existence and quantity of electric cars, washing machines, televisions, and other appliances. Figure 2 presents a sample of the data from this database, and Figure 3 presents a summary of all the characteristics and types of data.

Figure 4 presents a simplified schematic vision of the information contained in both databases and the link between them.

3.3. Data Preparation

After obtaining the databases, the data were managed using the Jupyter Notebook v. 6.4.5 available in Anaconda Navigator ^® v 2.2. Figure 5 presents a schematic vision of the data preparation, followed by a more detailed explanation to ensure reproducibility.

After inspecting a sample of the bases, we explored them to verify the existence of null data. In the database “all-users-daily-data.csv”, there were no null values. In the “household-information.csv” database, the “heatingTypeOther” and “waterHeaterTypeOther” columns were removed because they contain approximately 97% of null values in their structure. Other lines with null values were also removed, totaling a reduction from 552 to 472 lines; that is, 472 unique smart meters. Next, the two databases were merged, using the smart meters identifier (“userId”) as the connection key. Not all meters identified in the “all-users-daily-data.csv” database had household characteristics registered in the other database, so the data from these meters were removed from the merged database.

Some data reformatting had to be performed at the merged base (referred to as the GEM database). First, from the “date” column, an object type data which brings the period of energy consumption, the “dates” column was created with the same information but in the data format for time-series (datetime64[ns]). Then, several columns of data were transformed from “categories” to dummy variables, a format supported by machine learning algorithms. Another necessary transformation was in the reported energy consumption data. In the original format, the reported data is the accumulated amount registered in the smart meter since its installation. This data was transformed into the energy consumed on the recorded day by subtracting the previous value from the current day. As each meter’s first record date did not have the previous day for subtraction, those lines were removed. Thus, a sample of the transformed database is presented in Figure 6 and Figure 7; the whole database has 411,568 rows and 58 columns. Figure 8 shows the list with all data columns and their data type.

3.4. Modeling

3.4.1. Feature-Based Prediction

In the feature-based prediction experiment, the objective is to train an algorithm so that, based on the characteristics of a given residence, it can predict the residence’s energy demand. As all data from all smart meters will be used simultaneously, data from smart meters with less than 400 records each were discarded, totaling five smart meters. The energy consumption data were also transformed from a daily to a monthly basis. The database was randomly divided between a training split (80% of the database) and a test split (20% of the database).

For the feature-based experiment using random forest, a random forest regressor was used, imported from the sci-kit learn package. For a feature-based experiment using support vector machines, SVR was used, also imported from sci-kit learn. It is recommended that the values in the database are scaled (normalized) to use the support vector machine algorithm. The algorithm was trained on a scaled and unscaled basis for comparison purposes. The text version of the Jupyter Notebbok with these regression algorithms is in Appendix A.

3.4.2. Past Data-Based Prediction

The objective of the past data-based experiment is to forecast, based on past consumption data provided in a certain period, the energy consumption for a certain period in the future.

For the past data-based experiments, data on the features of the households are not necessary since the forecasting of the consumption of a given smart meter is performed only considering its own past data. Smart meters with less than 901 records were removed to guarantee a robust data history for each smart meter’s forecasting. A looping structure was created for all forecasting algorithms, in which the algorithm trains and forecasts the energy consumption for each smart meter. The division between the training and test sets was performed in temporal terms. The period with the oldest data is the training period, and the most recent period is the testing period. The moving average, ARIMA, and exponential smoothing algorithms were imported from the “statsmodels package”. The LSTM recurrent neural network method, imported from tensorflow, was used for the neural network. The random forest regressor, imported from the “sci-kit learn package”, was used for the random forest forecast. The text version of the Jupyter Notebbok with these regression algorithms is in Appendix B.

3.5. Evaluation

Both in the feature-based and past data-based experiments, the symmetric Mean Absolute Percentage Error (sMAPE) and the Root-Mean-Square Error (RMSE) were used to assess the accuracy of the predicted values. The sMAPE is typically defined according to Equation (1):

$$sMAPE=\frac{100\%}{n}{\displaystyle \sum }_{t=1}^n\frac{\left|F_t-A_t\right|}{\raisebox{1ex}{$\left(\left|A_t\right|+\left|F_t\right|\right)$}\!\left/ \!\raisebox{-1ex}{$2$}\right.},$$

(1)

where, for every t iteration from the first (t = 1) to the last (t = n) of the series, A_t is the actual value, and F_t is the predicted value. The “100%” term converts the result to a percentage basis.

The RMSE is defined by Equation (2):

$$RMSE=\sqrt{\frac{1}{n}}{\displaystyle \sum }_{j=1}^n\left(y_j-{\widehat{y}}_j\right)²,$$

(2)

where y_j is the actual value and ${\widehat{y}}_j$ is the predicted value.

For regression experiments, the evaluation is carried out using the mean values of sMAPE and RMSE of the train test split interactions. These values were compiled into a table for comparison. In the case of forecasting experiments, the sMAPE and RMSE values for each smart meter were added to a table and compared among all forecasting methods.

4. Results

4.1. Feature-Based Prediction

The following linear regression methods were used as baselines for comparison with Random Forest Regression and Support Vector Machines Regression: Linear Regression, Bayesian Ridge Linear Regression, and Ridge Linear Regression. The Support Vector Machines without scaling were also used for comparison, resulting in six different regression methods. The sMAPE and RMSE values of the predictions with their corresponding regression method are shown in

Table 1. Error values of the featured-based predictions.

Method	sMAPE (%)	RMSE (×1011)
Random Forest	17.92	7.656
Support Vector Machines	19.98	8.153
Support Vector Machines (unscaled)	43.74	15.729
Linear Regression	30.01	11.314
Bayesian Ridge	43.96	15.540
Ridge Linear Regression	29.79	11.323

.

The errors in the estimations indicate that the random forest method presented the best results in both error evaluations (sMAPE and RMSE), followed by the support vector machines method. These findings imply that random forest and support vector machines showed the greatest capacity to predict energy consumption with the data of 476 residences used in this experiment.

Linear regression and ridge linear regression presented very similar results, with error values worse than both random forest and support vector machines. Bayesian ridge linear regression and support vector machines without scaling appeared to be inadequate methods for predicting energy consumption in this experiment since their errors were worse than the baseline of the linear regression method.

4.2. Past Data-Based Prediction

Table 2. Error values and correspondent statistical measures for the past data-based predictions.

Method	Naïve Forecasting		Moving-Average		Arima		Expon. Smoothing		Random Forest		Neural Networks
Measures	sMAPE (%)	RMSE (×1010)	sMAPE (%)	RMSE (×1010)	sMAPE (%)	RMSE (×1010)	sMAPE (%)	RMSE (×1010)	sMAPE (%)	RMSE (×1010)	sMAPE (%)	RMSE (×1010)
Mean	31.53	3.748	29.43	3.370	28.47	3.339	28.58	3.372	25.63	3.248	23.67	2.812
Std Dev	16.24	2.837	15.72	3.339	14.37	2.508	14.09	2.601	12.06	2.431	9.95	1.949
Min	0.02	0.025	7.595	0.742	0.047	0.026	0.038	0.025	0.349	0.008	4.699	0.379
25% (Q1)	21.24	2.122	19.94	1.955	19.95	1.862	20.14	1.896	18.22	1.889	17.57	1.724
50% (Q2)	27.92	2.001	24.89	2.757	24.91	2.740	25.31	2.754	23.70	2.704	21.96	2.360
75% (Q3)	37.22	2.325	34.40	3.950	33.56	3.981	34.6	3.946	30.68	3.972	26.97	3.176
Max	122.80	22.042	148.47	18.374	127.20	20.424	123.57	24.165	80.88	2.291	78.16	17.996

summarizes sMAPE and RMSE values for each of the smart meters and forecasting methods.

As a way to evaluate the quality of the forecasting methods, Naïve forecasting was performed. In naïve forecasting, the last record of the training set is predicted in all records of the testing set. It is possible to see that all forecasting methods performed better than the naïve forecasting, which has the highest mean sMAPE, and the highest standard deviation. The same result can be seen if we evaluate the results with the RMSE.

It is possible to notice that the three traditional statistical methods (Moving-average, ARIMA, and Exponential Smoothing) presented very similar results. The three have an average sMAPE between 28.46% and 29.43%, a difference of less than 1% between the means of the three methods. The standard deviation between the three methods also shows a variation of less than 2%, hovering around 15%. The values between the quartiles are also very close. The results obtained with the Random Forest and Neural Network methods in the sample analyzed raised predictions more accurately when compared to the other three methods. Mean, standard deviation and quartile values are each smaller between Neural Network and Random Forest when compared to the Moving Average, ARIMA, and Exponential Smoothing. Based on the results obtained in the smart meter consumption time-series sample, the Neural Network was the more accurate forecasting method.

5. Discussion

The main novelty presented in this work is the comparison of hundreds of different smart meters data to achieve the best prediction methods, among the most standard present in the literature, in two different situations, namely, the feature-based predictions used to predict consumption for new households and the past data-based prediction to foresee long-term infrastructure needs.

5.1. Feature-Based Prediction

The results presented are very important for utilities, as they allow a quick and accurate forecast of energy consumption estimates for new customers. For example, one way to deploy the algorithms of this work (and thus reach the last step of the CRISP-DM method) would be to embed them into an application that quickly calculates values for a real-time pricing (RTP) contract. Using real-time consumption estimates provides the billing flexibility necessary for RTP contracts to be advantageous [81]. Another example of using these consumption prediction algorithms is setting reasonable resource-saving goals in gamification applications [82]. With achievable goals, it is possible that engagement will increase and the goals of rationalizing the use of resources will be achieved. It is important to mention that one of the challenges that utilities will face is how to gather feature data to predict energy consumption, with data disaggregation being one of the main tools to help this data collection without the need for anything external to the smart meters’ energy consumption data [83].

5.2. Past Data-Based Prediction

Predicting future energy use is a key need for utilities. The need to use renewable energy sources to meet sustainability goals [41] is a global reality. Such renewable sources are characterized by variable energy generation over short periods and the non-coincidence between generation peaks and consumption peaks [77]. In this context, it is necessary to use artificial intelligence and data from smart meters to manage energy supply, distribution, and demand.

The application of forecasting algorithms using data from a utility’s entire coverage region allows strategic decisions regarding infrastructure maintenance and expansion, supply diversion, blackout prevention, and storage for peak periods, among others. The daily frequency data presented in this work are more suitable for medium and long-term expansion and maintenance. However, applying the algorithms in smaller interval data, with the respective investigation about micro seasonality, enables utilities to reduce peak consumption and use of energy stored in capacitors, or even in electric cars, to stabilize generation and demand. A business plan for the resource management strategy of a utility that involves the results obtained with this work is also an example of the last step of the CRISP-DM method.

The results are valid for this smaller-scale set of households. Nevertheless, recent results in the literature are aligned with these in the sense that the Neural Networks method is an optimal starting point for developing more complex tools for long-term energy predictions [79,84].

5.3. Data Gathering and Privacy Concerns

Several surveys point to different factors regarding what can facilitate or hinder the acceptance of smart meters. Factors such as perceived usefulness, environmental concerns, and performance expectation are historically the main drivers of consumer acceptance [85,86,87]. Nevertheless, hedonistic motivation and social influence have been gaining strength as acceptance drivers [32]. However, insecurity regarding data protection, data misuse, and hacking can be a factor that drives consumers away from smart meters [83,88,89,90,91]. Gathering consumer data is essential for advancing smart grid management, so actions must be taken to prevent privacy damage and consumer resistance. Some options that apply to smart meters are anonymization technologies and privacy by design, using aggregation, encryption, and steganography techniques [83,92]. Other tools allow the user to adjust the utility’s control over the smart meter in legal terms [93] and in terms of application settings and communications [94,95]. Such options should be considered, so privacy concerns and a lack of control over data do not threaten data gathering.

6. Conclusions

Industry 4.0 and its related technologies are permanently changing the interactions between people and electronic devices. They are helping utilities and governments to better understand and manage all the sources and needs of smart cities, a crucial task in a century where climate change is most felt and scarce resources must be used in the most efficient ways. In this same context, smart meters act as IoT devices, gathering big data that should be used along with artificial intelligence and cloud computing to better manage energy supply and demand. This work sought to identify the more appropriate methods to use the data collected from these smart meters and household features to predict electricity consumption. Several methods were applied to a German dataset using the CRISP-DM framework, and the results were compared.

In theoretical terms, the results of this work offer a comparison between several prediction algorithms. The best prediction methods using feature-based data for the dataset are random forest and support vector machines. Neural networks and random forest showed the best results for past data-based prediction. Starting with these two approaches and developing more complex and comprehensive AI decision-making techniques will help improve the accuracy and robustness of the predictions.

In practical terms, the results of this work contribute to advancements in the management of smart grids by providing more tools and options that facilitate this management. Both the management of new customers, with an easy and quick forecast of energy consumption, and the strategic management of a sustainable future energy supply benefit from the results obtained here. Future challenges are presented in terms of how to gather consumption data without jeopardizing consumers’ privacy. Industries and governments should advance toward privacy by using design tools and smart meter acceptance policies to secure a safe and reliable data source for energy management.

This study provides a comparison between several prediction techniques and shows the best option in the database given. These techniques can be used for companies to predict energy consumption in different scenarios and should be improved by future research linking this data with local economic and meteorological data as well as with correlations between the features and energy consumed.

The 21st century presents immense challenges, including the management of smart cities. We are taking a crucial step towards the smart and sustainable management of natural resources by using the full potential of the data generated by IoT devices combined with machine learning techniques. Such advancements will be decisive for future generations’ quality of life in this century and beyond.