3.4.1. Equations

HERMES optimal scheduling is modelled as an MOP problem [89]). In this model, the first objective (*f*1) is related to the minimization of the monthly bill, so the scheduling can shift loads either on time scales of minutes, hours, or even between days, with a monthly horizon for the optimization, instead of a daily horizon as most HEMS schedules usually present. Therefore, the shifting of loads is allowed even between days, as long as it does not impair the comfort of the residents, so that the first objective can be formulated as:

$$f\_1 = \sum\_{d=1}^{PT} \sum\_{h=1}^{HD} \left[ p\_{d,h}^{Tariff} E\_{d,h} \right] \tag{1}$$

where:


Equation (1) excludes fixed costs and taxes not associated with consumption. The choice of the tariff is important because it determines both the variable costs and part of the fixed costs, so, initially, a study was made to determine which tariff was the most suitable for the habits of the residents and the potential of the HERMES system. This choice led to a first saving in the monthly bill without affecting comfort, as reflected in the results (see the first part of Table 6).

To minimize the value of the function *f*1 several resources and constraints must be considered:

$$P\_a = P\_a^{LINC} + P\_a^{Con, Inc} + P\_a^{Con, Ela} \stackrel{\sim}{\leq} \begin{cases} \begin{array}{c} \text{P}\_{aMax}^{Turiff} \\ \text{P}\_{aMax, penalty}^{Turiff} \end{array} \tag{2}$$

Equation (2) establishes that the active power at any instant of time *Pa* expressed in kW cannot exceed the maximum contracted power *PTari f f aMax* nor the one higher than this one of penalty *PTari f f aMax*, *penalty* (in the case under study, this limit is set at 105% [90] of the maximum contracted power *PTari f f aMax* ). The *Pa* is the sum of all household loads, consisting of uncontrollable loads *PUNC a* and controllable inelastic *<sup>P</sup>Con*,*Ine a* and elastic *<sup>P</sup>Con*,*Ela a* loads. This constraint affects the HEMS scheduling, which is oriented to avoid reaching the maximum allowed and the penalty level. However, due to the freedom of the residents, in case of reaching the first level (5.5 kW), the assistant warns the residents, and in case of exceeding the penalty level (5.775 kW) the system can act by disconnecting elastic loads, and the warning of the assistant to the residents is of greater emphasis.

One relevant aspect is the ability of the system to schedule shiftable loads within the entire known price period *PTKnown* guaranteeing the comfort of the residents and, extending the scheduling horizon beyond the 24 h with which HEMS normally work:

$$PT\_{Known} \geq \text{ 24 hours} \tag{3}$$

This condition allows the system to increase consumption on days whose prices are lower than adjacent days, i.e., the system reschedules the loads when it obtains the prices for each hour of the following day, also taking into account the prices for the rest of the current day. After obtaining the prices, the system performs sorting by prioritizing the cheapest hours:

$$Array\left(p\_{d,d+1,h}^{Tiff}\right) : \left[ \begin{array}{ccccc} \text{Min}\left(p\_{d,d+1,h}^{Tiff}\right) & \dots & \text{Max}\left(p\_{d,d+1,h}^{Tiff}\right) \end{array} \right] = \left[ \begin{array}{ccccc} p\_1 & \dots & p\_m \end{array} \right] \tag{4}$$

where *p*1 ≤···≤ *pm*; and assigning specific controllable loads (such as the electric boiler *PE*.*Boiler a* , dishwasher *PDishwasher a* or batteries *PBaterias a* ) to the cheapest hours according to the energy required by each load (for example *EE*.*Boiler* > *Ed*,*<sup>h</sup>* > *EBatteries*) while maintaining the constraint (2) and comfort:

$$\begin{array}{l}p\_1 \stackrel{d,h}{\rightarrow} \mathfrak{a}\_{11} \stackrel{E,Boiler}{P\_a} + \mathfrak{a}\_{12} \stackrel{D \text{Dislawaker}}{P\_a} + \dots + \mathfrak{a}\_{1n} P\_a^{\text{Ratteries}} \le \mathcal{P}\_{a\text{Max}}^{\text{Tarnff}}\\p\_2 \stackrel{d,h}{\rightarrow} \mathfrak{a}\_{21} \stackrel{E,Boiler}{P\_a} + \mathfrak{a}\_{22} \stackrel{D \text{Dislawaker}}{P\_a} + \dots + \mathfrak{a}\_{2n} P\_a^{\text{Ratteries}} \le \mathcal{P}\_{a\text{Max}}^{\text{Tarnff}}\\p\_m \stackrel{d,h}{\rightarrow} \mathfrak{a}\_{m1} \stackrel{E,Boiler}{P\_a} + \mathfrak{a}\_{m2} P\_a^{\text{Diskousher}} + \dots + \mathfrak{a}\_{mn} P\_a^{\text{Ratteries}} \le \mathcal{P}\_{a\text{Max}}^{\text{Tarnff}}\end{array} \tag{5}$$

where *<sup>α</sup>ij* form a matrix *m* × *n* of binary coefficients associated with each controllable load as a function of the energy required by each load (*<sup>α</sup>*11 is associated with a load whose energy is greater than the equivalent for *α*12 and so on until the *α*1*n*; the next row corresponding to *p*2 represents loads whose work duration extends beyond the hour *h* associated with the minimum price *p*1. The constraint is given by *PTari f f aMax* forcing the HEMS scheduling to set to zero those loads whose energy is lower, i.e., to those coefficients *<sup>α</sup>ij* of higher columns, so that if the power *PTari f f aMax* is exceeded, they would not be activated until the next cheapest hour or once the appliances with higher loads have finished their operation (or the sum of the loads already allows incorporating a new lower load). An example matrix for three controllable appliances might look like the following:

$$\mathbf{a} = \begin{bmatrix} 1 & 1 & 0 \\ 1 & 0 & 1 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \\ \vdots & \vdots & \vdots \\ 0 & 0 & 0 \end{bmatrix} \tag{6}$$

The operation of the loads is not limited to whole hours, and they can be longer or shorter periods, so the coefficient *α*21 set to 1 in this example does not imply that the associated appliance is the second full hour working; it only indicates that it requires more than one hour to complete its cycle. In some cases, the order of the hours is not relevant, whereas in others it is, so the system also takes into account this limitation for each appliance. The matrix also shows that for the first cheapest hour, the system can only activate two controllable appliances to ensure that condition (2) is not violated or that the third appliance requires two consecutive hours to run its program, and that any other sum of consecutive hours would offer a higher sum price. *px* + *py* > *p*2 + *p*3. Similarly, if during that period residents activate any other loads (uncontrollable, controllable inelastic, or elastic) that compromise condition (2), the system will warn the residents and ultimately displace the elastic loads that can be displaced at that time. In the following results section, Figures 16–18 show how loads of the appliances are concentrated in the least cost hours. Three zones are distinguished corresponding to (1) the zone where the system works without interference from residents, usually night hours; (2) another optimal zone where both the system and the residents activate loads; (3) and a third one associated with residents' comfort where the system tries not to schedule loads because they correspond to the highest prices and informs the residents through the wizard.

One last remarkable resource to achieve condition (1) has been the development of a cumulative hourly consumption forecast for the next day *f Forecast* 1,*d*,*h* , understood as the expected consumption based on past consumption under similar conditions. It is based on the use of ML linear regression, offering a forecast based on historical consumption data of residents over a long period comprising a total of 9946 h (over 20,000 h for a second version). It uses data such as weather (both historical and forecast data), as well as the percentage of presence of residents in the house, the day of the week and month, and the price of the kWh. The project is developed in the subsection Consumption estimation (Machine

Learning), more information is provided in the Data Availability Statement. Based on this forecast, the following can be established: 

$$\begin{array}{l} \left( f\_{1,d,h} \le 0.75 f\_{1,d,h}^{Forecast} \right); \ f\_{1,d,h} \le 0.95 f\_{1,d,h}^{Forecast} \\\ 0.95 f\_{1,d,h}^{Forevision} \le f\_{1,d,h} \le 1.05 f\_{1,d,h}^{Forevision} \\\ f\_{1,d,h} \ge 1.05 f\_{1,d,h}^{Forecast}; \ (f\_{1,d,h} \ge 1.25 f\_{1,d,h}^{Forecast}) \end{array} \tag{7}$$

For each hour of the day, three levels are established that compare the actual accumulated consumption of the day *f*1,*d*,*<sup>h</sup>* and the accumulated forecast for that same hour *f Forecast* 1,*d*,*h* so that the user can consult through the wizard if their consumption is lower, higher, or close to the forecast. Lower and upper limits are also established in which the system informs the residents through the Wizard without waiting for the consultation; this would be in cases where the deviation is significant, set by default at a deviation of 25% of the expected amount. The residents can modify the margins established in Equation (7). In this way, a reinforcement message is established when consumption is lower than expected and a "warning" in cases where consumption is higher than expected.

The information provided by the ML could also be beneficial in cases where alternative or complementary energy sources or systems to the public power grid are used. We are referring to microgrids in which energy managemen<sup>t</sup> would be based on a different dynamic and in which generation and consumption forecasting through the ML would become much more important for the objective stated in (1).

### 3.4.2. Strategy for Comfort and *f*1 Optimization

If the first objective is related to the minimization of the monthly bill *f*1, the second objective of the MOP programming of the HERMES system is associated with the residents' comfort, trying to maintain a balance between both objectives because, on many occasions, they are opposed to each other. Since the HEMS does not have direct access to the uncontrollable loads, both the optimization of function *f*1 and comfort is usually focused on the controllable loads. However, in our case, thanks to the work of the Assistant, residents are more aware of the costs associated with the loads, so there is bi-directional feedback, and the system gains some influence over the uncontrollable loads, allowing optimization of both objectives, *f*1 and comfort more efficiently.

Comfort has a significant amount of resident subjectivity, and its programming can compromise the hardware resources of the system, so to avoid or alleviate these drawbacks, we chose to change the comfort penalty (discomfort) function typically used in HEMS [65,73,91,92] to a parameter approach adjustable by both residents and the system so that residents could vary these parameters to fit their conception of comfort and the system would balance them to optimize the *f*1 function within ranges that do not compromise comfort. However, this concept would only be practical if the parameters were associated with each appliance; it would not make sense if they were global, as we would return to the concept of a global comfort penalty function. At the same time, it would not be necessary to define individual comfort functions for each appliance because the residents are part of their programming through the parameter settings, so the programming must be very well calibrated, which requires more extended testing periods in the implementation of the system and a certain flexibility. This strategy avoids the two problems associated with comfort: subjectivity, as users can adjust the parameters within a range, and programming is simplified as it is customized for each appliance; however, it requires a longer testing period.

The following two examples (responsible for a large part of the electricity bill [9]) show the potential of this approach to comfort: The electric boiler and the air conditioning system:

• The two main problems of the electric water heater are that it runs out of hot water or that it consumes at very high or non-optimal cost hours. In a traditional HEMS, this situation should penalize the overall comfort function, although it might not anticipate the problem or optimize consumption to the maximum. In our case, three

groups of parameters have been created to optimize consumption and comfort, solving both problems. The first group selects the time slots in which the thermos flask is allowed to be turned on. The second group sets the temperature targets for each activation band. Finally, the third group adjusts the water heating curve. This third group is continuously adjusted thanks to the temperature sensor inside the tank and determines exactly how long it takes for the boiler to heat the water to the desired values. In this way, if very low temperatures are reached after use, the system responds by increasing the heating time and raising the maximum temperature of each range. The system adjusts these parameters automatically, ensuring the hot water supply and shifting the load to the optimal time slots (see Figure 8). However, residents can readjust the parameters to suit their comfort (maximum heating temperature and the number of heating hours). This set of parameters covers the complete characterization of the water heater, making it possible to cater for particular scenarios such as, for example, completely switching off the electric water heater during prolonged absences by disabling all operating slots, or from time to time run a heating cycle to 60–65 ◦C to eliminate possible Legionella outbreaks.

**Figure 8.** Heating of the water in the electric boiler during the most economical hours. The heating cycle of the boiler is displayed, and the heating hours are marked. Data from 21 June 2020 00:00:00 to 23 June 2020 23:59:59.

> • For air conditioning, the HERMES system controls several parameters and employs the following strategy to optimize consumption and maintain comfort: after a certain time after switching on the climate, the system automatically lowers or raises the

temperature to reduce consumption while maintaining comfort. The parameters used in this strategy are again three: initial temperature when the heating or cooling is turned on, the time in minutes until the system automatically applies the second temperature regulation (to reduce consumption and which could depend secondarily on other parameters such as outdoor temperature, indoor temperature or whether or not the residents come from outside), and finally, the third parameter would be the difference in degrees of the new temperature. The adjustment of the parameters is again dynamic depending on what the system requires and the residents' preferences.

In the programming of this strategy for the optimization of the comfort and *f*1 the following parameters associated with loads of each appliance have been used (Table 2):

In the previous paragraphs, the strategies and scheduling of the two main objectives of the HERMES system have been detailed. However, in this multi-objective structure, others could have been added, such as the reduction of *CO*2 emissions in tune with the reduction of the bill, demand response, and others. However, in terms of *CO*2 emissions, any system that minimizes consumption already contributes to reducing emissions, and if it also concentrates consumption in off-peak hours where renewable and less polluting energies tend to prevail, the reduction is even more significant. HERMES provides both benefits.

Therefore, this section can be concluded by indicating that the residents set the comfort levels they desire, and the system optimizes the objectives of minimizing the monthly bill and maximizing comfort, keeping the balance between the two.
