*2.3. Cloud Computing*

Cloud computing is the resources and services of the computer system accessed through a network without direct active managemen<sup>t</sup> by the user. Initially, the services were focused on data storage and computing power, but, today, the user's services and systems cover virtually any need.

Cloud computing offers advantages and disadvantages that must be assessed by the user when implementing or not implementing these systems. From the point of view of HEMS, we can highlight the following advantages:


On the contrary, some drawbacks can be very critical to the viability of the system:

	- - Downtime;
	- - Technical interruptions from suppliers, which are unavoidable and can occur at critical moments;
	- - More limited bargaining power, leading to limited customization;

Although a priori cloud computing is possibly an inevitable tool if we want to develop a truly competitive HEMS, we must be very aware of some of the limitations and implications that its use may entail, so we must adopt hybrid strategies between functional HEMS through local networks isolated from the Internet and HEMS based on cloud computing. Therefore, we are committed to systems that take full advantage of the functionality of the isolated network and to ensuring that the contracted services, which are based on or use the Internet, do not pose a risk or functional disruption to the system.

In this regard, the design of the HERMES system presents a dual functionality with communicating vessels between the own network (partially isolated) and the contracted cloud computing services. In addition, to protect the system's security, various levels of protection have been planned according to its exposure to the Internet. HERMES can maintain operational functionality if, for security reasons, it is decided or required to isolate the system from the Internet.

#### *2.4. Artificial Intelligence (AI), Expert System (ES), and Machine Learning (ML)*

AI is the ability of a man-made system to interpret and analyze data, learn from that data, and use that new knowledge to perform actions or tasks. This definition is an evolution of the one given by Andreas Kaplan and Michael Haenlein [41]. Another agent-based approach defines it as: "Computational intelligence is the study of the design of intelligent agents. An intelligent agen<sup>t</sup> is a system that acts intelligently: What it does is appropriate for its circumstances and its goal, it is flexible to changing environments and changing goals, it learns from experience, and it makes appropriate choices given perceptual limitations and finite computation" [42]. The definition is not trivial and has evolved over the years to encompass very diverse disciplines with applications in virtually all scientific fields [41–46], such as expert systems that emulate the behavior or responses that a human expert in an area of knowledge would give.

There is no doubt that IA is a fundamental tool in the present and future development of HEMS. IA encompasses a multitude of technologies, some of which are shown in Figure 2:

**Figure 2.** AI Technologies.

The HERMES system has been developed integrating ML, natural language processing, expert systems, and speech, without ruling out other technologies in the future such as image recognition (vision) for more advanced analysis of presence [47–49] with a higher level of personalization of interactions.

### *2.5. Virtual Assistant*

A Virtual Assistant [50] or Voice Assistant is a software agen<sup>t</sup> that can interpret human speech and certain commands and respond with synthesized voice, tasks, or services. Other definitions can be found in M. B. Hoy [51]. The development of natural dialogues between humans and machines is one of the goals of AI [50,52]. Voice assistants are here to stay [53], not only because of their benefits for people with specific needs or older adults [54–57] but also because they have been shown to bring benefits such as social cohesion [58] or improve comfort and allow the user to interact in a very natural way with machines, since speech is the main mode of communication for humans [59]. In our case, the last of the pillars that make up the HERMES system is precisely the assistant but endowed with greater intelligence, as we will indicate below.

From a HEMS perspective, voice assistants have several notable handicaps. First, their intelligence is limited in terms of energy efficiency, as verbal commands and functionality are focused on activating or deactivating devices. However, our HERMES system integrates a bidirectional communication channel to the virtual assistant (Figure 3) both with the system as a whole and with the residents, connecting the intelligence of the system with the user, becoming an "intelligent assistant" beyond the function and intelligence of these systems, complementing the functionality of the HEMS. This dual bidirectional channel represents a qualitative leap in the functionality and interaction of the HEMS system with the users.

**Figure 3.** HERMES: Bi-directional dual-channel formed between HEMS-Voice Assistant-Residents. Background: © 123fr.com.

Secondly, another important handicap is the vulnerability presented by these devices; for example, any user can issue verbal commands: "Open the door" or "buy this and send it to such address") [60–62]. In this case, the integration of the voice assistant in the HERMES system is done keeping in mind that this type of vulnerability cannot be fraudulently passed on, the system itself is the one that filters them. In this regard, HERMES detects the presence at the home of all the usual residents and identifies them so that certain commands can only be executed if at least one of them is at home or if the presence simulator has been activated, which can be activated remotely by the residents for a limited time. Other avenues that could be explored to avoid vulnerabilities could be the identification of users by smart cameras or by their voice profile [63,64]. In this way, all commands, or those that we consider critical to the system, can be filtered to prevent a local or network intruder from exploiting them.

### *2.6. Results from Knowledge*

The benefit of applying advanced and complex systems must be realized from the knowledge acquired from the collection of data and the application to balanced models, not forced, that allow the creation of precise and effective forms counting at all times on the users. Otherwise, the system will lack practical application, falling into the dynamics of a good theoretical study without a practical route. Therefore, the system has been developed in different phases, data collection being the first of them, from which practical solutions have been channeled. For this reason, the system has been developed in different phases, the first of which is data collection. Based on the data, practical solutions have been proposed, focusing on economics but adapting to the users, which allows long-term habits to be established, quality of life to be maintained, and ensures that the system is applied as it is beneficial.

From this dynamic of results from knowledge, the integration of all these systems in one (HEMS + IoT + Big Data + Cloud Computing + AI + Voice Assistant = HERMES) has led us to an ES with a multitude of possibilities in the field of energy efficiency and well-being of residents. This work shows the development of a new system with the following objective: "developing a comprehensive model for smart home consumption managemen<sup>t</sup> assisted by an ES (HERMES)".

The development of this objective was based on the following pillars: results from knowledge, energy savings, usability, user assistance, comfort, privacy, and security. The following lines of analysis and development were proposed:

	- - Actions: Programming household appliances and devices. Presence detection and habit analysis;
	- - Warnings: Advice and recommendations for savings based on detected habits or pre-established patterns;
	- -Alerts and maintenance of equipment and appliances;
	- - Integration with voice technology.

### **3. Materials and Methods**

It is not easy to make accurate predictions of electricity demand, microgeneration, or appliance usage in domestic environments. Factors such as the type of billing (five main energy billing approaches can be found in the literature [65–67]), weather conditions, or assumed habits and routines of users involve in themselves elements of uncertainty that are difficult to predict, so that deviations on forecasts of electricity consumption, microgeneration, or the operational needs of household appliances, can compromise the planning of HEMS. These uncertainties may result in situations where contracted power limits are required to be exceeded with consequent limitations or penalties, or the comfort level of residents may be affected. Therefore, in decision making, the value of past and present data must be prioritized over future data, with dynamic (stochastic) programming approaches [65,68,69].

If we add to this uncertainty the diversity of load types and their different scheduling possibilities, HEMS design strategies can be approached from multiple perspectives [65,68–71]. Before focusing in more depth on our development, we will review some of the discussed aspects to settle and show the fundamentals of the HERMES system presented in this paper.

### *3.1. Classification of Load Types*

There is no consensus on the classification of load types, so we propose a new model that will be useful for our work and is based on several classifications that focus on the characteristics of the loads [65,67], but to which we add the user's decision capability through the wizard or by programming so that some devices can change category based on the user's decision.

Classification of devices or systems according to their load scheduling (Figure 4):


through the user; in this regard, control is one of the contributions of the HERMES system. In turn, within this category, we can divide the loads into elastic or inelastic;

	- i. Uninterruptible loads. Once started, they must run a complete cycle continuously; only the corresponding start time can be programmed. In this category, we can find dishwashers, washing machines, or dryers, among other appliances;
	- ii. Interruptible loads. Once started, they can be interrupted but must be reconnected to complete the full cycle. These are usually constant-drain devices. Examples include plug-in hybrid electric vehicles and other rechargeable devices or external batteries, and the electric boiler;
	- i. Variable loads with alteration of comfort. Energy consumption can be adjusted in the middle of an operation but leads to loss of comfort and may require subsequent compensation. These are usually systems whose operation is maintained according to a reference defined by the residents, so their temporary variation by the HEMS may affect comfort. Ventilation, heating, or cooling are examples of this category;
	- ii. Variable loads without alteration of comfort. Energy consumption can be adjusted in the middle of an operation without significant loss of comfort or subsequent compensation. For example, dimming of artificial lighting by compensating with daylight.

**Figure 4.** Classification of equipment or systems based on the programming of their load.

In the HERMES system, the above examples of appliances could change category (temporarily or permanently) depending on the user's decision-making. An extreme example could be the refrigerator defined a priori as an uncontrollable load. The user can instruct the HEMS assistant to turn it off for a short period that does not jeopardize food preservation, making it controllable, inelastic, and interruptible. This example can be used to avoid a peak demand as long as there is no other controllable load to bridge the peak demand.

Based on the above structure, Table 1 is a classification of the main household appliances.

This classification is flexible and dynamic since the system can adjust specific parameters according to the characteristics of the residents or according to different scenarios. The system has general and appliance-specific parameters that it can readjust (see Table 2) to

adapt to a dynamic classification of appliances. In certain cases, this adjustment is shared by the system and the users, as could be the case for the air conditioning temperature. This behavior thus allows the system to adapt to the characteristics of different user groups and different scenarios (seasons of the year, vacation absences). Users can adjust these parameters within a range and even set the air-conditioning switch-on temperature by voice. The system acts accordingly to maintain comfort but reduce consumption, for example, after a period of operation, raising or lowering the cooling/heating temperature.

### *3.2. The Preamble of the HERMES System*

HERMES system scheduling is performed to manage a present and future time horizon based on past and present data. Both a continuous representation of time and a discretization into the minute, hourly, daily, weekly, and monthly intervals are used. For example, once a month, a heating cycle above 60 ◦C is completed in the electric boiler to eliminate possible Legionella outbreaks. This scheduling pursues the reduction of the consumption of household appliances and the shifting of loads (shifting to optimize expenditure and their optimal time of operation) to reduce electricity billing [72–74] and maintain or increase the comfort of residents [73,75]. Regarding billing optimization, the appliance scheduling technique based on mathematical optimization is suitable for small-sized problems such as individual dwellings instead of other less demanding techniques for larger problems, as we will discuss later. By contrast, in terms of comfort, the evaluation of resident comfort is a very complex task from a scheduling point of view due to how personal the perception and subjectivity of comfort can be, as well as the inconveniences of having to schedule appliances outside the preferred time window, maintain a certain order (washing machine before dryer) or accept unwanted elastic load modulations. As a step before implementing the HERMES system (as of 27 October 2019), daily usage profiles were recorded over an extended period (from 16 February 2019 to 26 October 2019) to characterize and minimize potential drawbacks that could affect comfort.


**Table 1.** Classification of the main household appliances according to their load.

In addition, given that household demand cannot be predicted with complete accuracy, we rely on a consumption profile characterized by minimizing the elements that introduce a certain degree of uncertainty, reinforced by two-way communication with residents to whom, on the one hand, electricity prices are provided a day in advance, as well as other statistics, and on the other, the system analyzes the use of household appliances and recommends their use based on history, coordination, and the use of appliances. This minimizes the problems of stochastic optimization, and although not all the elements that are a source of uncertainty and their derived problems (consumption peaks with penalties or loss of comfort) are avoided, they are reduced, and with experience, the residents themselves and the system learn and converge towards an increasingly optimal situation

in terms of billing and comfort. However, deviations of one from the other are allowed, although both are the ultimate goal, so that the system is constantly evolving around the optimal balance at all times, maintaining an MOP [76,77] that is very competitive with other techniques [65] of setting a single objective and the rest as constraints. MOP has allowed us to satisfy both consumer and system objectives [78].

**Table 2.** Programmable parameters associated with loads of each appliance. System managed control: "•" or "-". Resident-managed control: "-" or "".


•- Blender

**Appliance Parameters Description** Uninterruptible loads - Washing Machine - Power-on time. - Permanent consumption observation. - Manual or system-programmed ignition at the cheapest time between 7:00 and 11:00 AM. •- Dishwasher-Dishwasher. •- Power-on time. •- Permanent consumption observation. •- Manual or system-programmed ignition at the cheapest time for the next 12 h (24 h). - Dryer Machine - Power-on time. - Manual or system-programmed ignition. Note: Not applied or programmed to the study dwelling in the article. - Oven - Power-on time. •- Manual or system-programmed ignition Note: Not applied or programmed to the article study dwelling. Uncontrollable loads •- Television •- Sound equipment •- Computer. •- Refrigerator/Fridge-freezer. •- Light Spots/lighting. •- Microwave •- Vacuum Cleaner •- Iron •- Cooker pot •- Cooker Hood •- Hair dryer •- Toaster •- Kettle - On and off Observation of general consumption. Notification by the Assistant. • Rate information to residents on an hourly, strip, and daily basis. - Manual switching on and off by residents. • Warning of excessive consumption (via Assistant and Telegram) in the absence of residents or exceeding the contracted power limit. • Notification (via Wizard, Telegram, and control panel) of the electricity tariff.

> Using this bidirectional technique, which not only brings benefits but has also allowed us to limit uncertainties, it has been possible to implement stochastic dynamic programming with at most two levels of estimation: the target variable plus an additional level with stochastic variables, which greatly increases the accuracy of the predictions as will be seen in the results section. The following references show up to six different strategies for stochastic optimization: stochastic optimization, robust optimization, chanceconstrained optimization, stochastic dynamic programming, stochastic fuzzy optimization, and stochastic model, which generates synthetic consumption profiles [65,68,79–85]. For example, in [79] a stochastic energy consumption scheduling algorithm based on timevarying prices known in advance (similar to the one used in the HERMES system) is described as achieving a 24% to 41% reduction in simulations in billing costs. However, in HERMES, we have opted for a mixed model (with some elements with deterministic programming and others with stochastic programming), which has allowed us to obtain very similar reductions but with real data, not simulated, of up to 42% in absolute values (see Table 8) and 24% with counterbalanced data (see Table 6). Other techniques achieve reductions from 8% to 35% of the electricity bill [9], the optimization-based residential energy managemen<sup>t</sup> (OREM) technique being the most efficient [86] based on dividing the days into time slots, very similar to the time of use (ToU) scheme and the one proposed in this article, scheduling the operating time of the appliances in the minimum tariff time slot, but in our case minimizing the delays of the OREM technique by shifting the loads in a veryefficientwaycombiningthestrategywithothertechniques.

**Table 2.** *Cont.*

The various techniques employed in HEMS scheduling to find the optimal operating time of household appliances can be grouped into five categories [65,87]: mathematical optimization; heuristic and metaheuristic methods; model-based predictive control; ML; and game theory approaches. Each of these techniques has strengths for certain types of loads versus weaknesses for all other loads, and in almost all cases, the benefits provided drop drastically when uncertainties manifest themselves in a practical or worst-case form. The main source of uncertainty comes from the residents themselves, who are often influenced by external factors that are difficult to predict (changes in routines, illness, cancellation of a meeting) or varying perceptions and subjectivity. Based on this, HERMES decided to use a mixed model of techniques that would allow the residents to make their own decisions or let the system decide independently under MOP, using techniques such as mathematical optimization, heuristics, or ML.

The subsection "Equations" shows mathematical elements used and developed under a tree structure for decision making and resident assistance. Further on, the mutual learning process between the system and the residents will become evident by adapting the system to the residents' habits and the residents' system, which makes it possible to achieve the percentages of reductions indicated above. This feedback has allowed very significant improvements in the first year, which were further improved in the second year and again in the third year. This continuous improvement highlights the bidirectional interaction of the system with the residents, which would be difficult to achieve by applying a single technique and without the expert assistant to interact with and guide the residents.

As indicated, to highlight the potential of the wizard, HEMS has been developed on a mixed model of techniques [87] to improve energy use through load scheduling, in which uncertainties have been minimized so that these models must be able to admit the interaction of several agents that would become the elements of uncertainty as well as load scheduling. The objective of all HEMS is to optimize consumption, so they require scheduling over a future time horizon, for which household demands and electricity generation cannot be accurately predicted, requiring adequate consumption profiles, representative, and incorporating a certain degree of uncertainty management. For all these reasons, their efforts are focused precisely on optimizing consumption profile predictions. In our case, to highlight the assistant's potential, we have reduced the uncertainties to a scenario in which the development of HEMS is already considered sufficiently mature, with the assistant being a differentiating element and allowing us to show its potential. In this regard, to optimize consumption profile predictions, we will adopt a dark box model (modeling and forecasting frameworks based on data analysis schemes) as opposed to white-box models (classical and transparent modeling tools based on solving physical equations) and gray box models as a combination of white box and dark box [87,88]. We have limited the uncertainties to the demand area without incorporating electricity microgeneration and setting variable but known day-ahead prices. To obtain the prediction of household consumption, the element used for data analysis was based on ML techniques. Other data analysis techniques could have been applied (see [87]). A very accurate and robust model has been obtained using stochastic data of only two levels: The target or output variable plus an additional level on certain input variables of the ML itself. Since the HERMES system combines different techniques, e.g., deterministic programming for MOP or stochastic programming for consumption estimation, we have tried to simplify it, while trying not to harm the pursued objectives, as we will see in the next section.

#### *3.3. Deployment of the HERMES System and Involved Instruments*

The basis for developing the HERMES system to optimize savings and comfort is collecting past and present data and forecasting certain elements to create a robust and elastic energy use model. The only essential future data are the hourly kW price (€/kWh) 24 h in advance; thus, the chosen tariffs allow us to predict their value quickly; however, if they were not known, they could be obtained from prediction models with very accurate

approximations. The weather forecast and the presence of the residents in the home are also necessary but not essential future data.

Given the complexity of our system, and the need to obtain data, its implementation has been gradual, following "natural growth" towards the proposed objectives. Figure 5 shows the principal elements and services of the HERMES system.

**Figure 5.** The general framework of HERMES system elements and services.

Each of the elements shown in Figure 5 has been developed considering analysis, characterization, development of operating models, improvements achieved, deployment of the models and implementation, review of results, and return to the previous phase as necessary.

#### *3.4. Programming and Multi-Objective Optimization (MOP) of the HERMES System*

As discussed above, the HERMES system is based on MOP scheduling in order to (1) reduce electricity bills by reducing appliance consumption and shifting loads and (2) 'maintain or increase residents' comfort. The system sends the residents the next day's hourly rate by instant messaging; they can also consult it at any time through the HERMES user panel or consult it through the wizard. In Figures 6 and 7 we can see different data of the 2.0DHA electricity tariff. Although there are two well-defined time slots, there are significant daily variations in prices for each hour, so the system uses the daily prices in its programming, using any tariff as long as the prices are known or estimated with daily anticipation. For each day, the system selects the optimal time zone.

**Figure 6.** Average hourly prices with the range of fluctuations during 2020.

**Figure 7.** Example graph sent daily to the user with the next day's prices (example of 11 December 2020).
