4.1. Results and Lessons Learned from PoC Pilot
This subsection provides the results from the defined trials as well as a list of lessons learned during the deployment and demonstration of the PHOENIX architecture in the PoC pilot site.
The first trials (DR—strategy for flexibility extraction) consisted of sending demand response events to the pilot building in order to test the possibility of shifting the load to reduce the grid charge at certain hours by changing the thermostat set point temperature during limited timeframes. In particular, Trial No1 (DR strategy for flexibility extraction—tariff scheme) was carried out to achieve a load shifting from high tariff to low tariff, aiming for a decrease of 20% on peak power loads and to an energy cost reduction of 18%. Trial No2 (DR strategy for flexibility extraction—renewable scheme) was carried out to achieve a 15% demand shifting from low renewable generation to high renewable generation. Both of them had a duration of 2 weeks in the winter period and two weeks in the summer period.
In Spain, where the PoC pilot site is located, all consumers have three or more periods daily with different energy prices. Moreover, there is the possibility of applying a dynamic tariff with an hourly price according to the actual market price. Trial No1 was performed to shift the load from high tariff hours to low tariff ones and its effect on efficiency was then analysed. The decision to choose the optimised hour for the intervention was carried out by forecasting the hourly price for the entire day ahead. The results that changed depending on the daily market prices consisted of the detection of two consecutive periods whose difference in the electricity price was maximum. To give an example of functioning for cooling loads, in the first period or low-price period, the setpoint temperature was lowered (the so-called ‘precooling phase’), while in the second period or high price period, the setpoint temperature was raised. In this way, the demand was shifted to the period in which the electricity was less expensive. In particular, in the offices participating in the experiment, a typical energy consumption of 23 kWh was expected; however, the actual consumption during the DR event was 19 kWh. Therefore, a reduction of approximately 17.4% in energy consumption was achieved when tariffs were prioritized in the demand response strategy.
Trial No2 is based on the same methodology, but the hour intervals are chosen depending on the renewable energy generation. The decision making was based on CO
2 emissions, considering the energy production of the different energy sources used in Spain and the carbon footprint of each of them. In this way, it was possible to identify periods with fewer emissions or high renewable generation periods, in which we performed the precooling phase, and periods with high emissions or low renewable generation periods, to perform the increase in the setpoint temperature. As a result, the objective of shifting 15% of the demand to hours in which the electricity is produced by more renewable sources was achieved through a flexibility engine, which is in charge of performing the flexibility services, while maintaining an acceptable internal air temperature for the occupants. In particular, the expected consumption for the involved offices during the two hours of the experiment was 9.97 kWh, 24% of which (2.4 kWh) were shifted to the timeframe of the precooling phase, i.e., the period of high renewable production. The final energy consumption during the hours of the experiment was 7.99 kWh, hence it also obtained an energy saving of 1.98 kWh as a consequence of the trial. Also in this case, the evaluation of the thermal comfort during the trial was studied in Trial No4. An important result from both trials is that the internal air temperature, which decreased because of the precooling (or increased because of the preheating in winter), returned to its original value much later after the end of the experiment, hence it is possible to take advantage of the thermal inertia of the building. This means that the building occupants should not have felt too warm (or too cool in the winter period) at any moment. This result confirmed the positive effects that envelope quality can have on energy efficiency and energy flexibility potential in buildings [
23]. In particular, through the combined effects of sufficient thermal mass and thermal insulation, it is possible to improve both the heat storage and heat saving of the building [
24,
25]. An appropriate building envelope can significantly improve the implementation of energy flexibility strategies, as it allows the use of the HVAC system to be shifted without compromising the adequacy of the thermal environment.
Trial No3 (DR strategy for energy saving) is related to the first trial. The DR events were the same; therefore, there are 6 weeks in total of data concerning demand response flexibility. In order to analyse energy savings in kWh, a predictive model based on Artificial Neural Networks (ANNs) [
16,
26] was created, which used the number of activated HVACs, setpoint and environmental conditions to estimate energy consumption. In order the ANN model to be trained and tested, baseline data consisting of past energy consumption measures and weather information, including air temperature, humidity and solar radiation, were used in order to create the inputs and the output of the model. Air temperature, humidity and solar radiation are variables that are commonly used in ANN models ensuring an improved quality of forecasts related to energy consumption in buildings [
27,
28,
29]. The obtained accuracy on the test was of 92% Coefficient of Variation of the Root Mean Square Error (CVRMSE) in order to achieve 15% savings in energy consumption.
For Trial No4 (Occupants’ feedback), users’ feedback within the demand response strategy was collected. The goal was to estimate the acceptance of different users toward the strategy and also to verify that the occupants’ thermal comfort was maintained during the experiments. The methodology was based on the distribution of two questionnaires: one was needed to create a baseline, i.e., to understand the general thermal preference of the occupants, and the other one was sent after each demand response event in order to test the reactions among the occupants. The questionnaires were created and distributed in English and in user-friendly language. The same questionnaire model used for the winter period was then slightly changed to be adapted to the summer season.
The thermal comfort is evaluated following the indication of the current regulations (ASHRAE [
30] and ISO 7730 [
31]). The method is widely used in the literature [
32,
33]. Occupants were asked:
Thermal sensation vote (TSV), with a seven-point Likert scale from ‘Much too cold’ to ‘Much too hot’.
Thermal preference vote (TPV), on a scale from ‘Much warmer’ to ‘Much cooler’.
Activity level in the previous 15 minutes.
Metabolic rate for food or beverages consumed in the last 20 minutes.
Current clothing to estimate clothing insulation.
Thermal acceptability vote (TAV) from ‘Totally acceptable’ to ‘Totally unacceptable.
The general acceptance of the strategy was evaluated through ad hoc questions about expected thermal sensation during the experiment, eventual actions taken to restore the comfort, perceived level of productivity during the experiment and opinion about the precooling phase. The latter questions do not have references in the literature due to the novelty of the topic; therefore, they are the results of previous studies by the University of Murcia research group [
34,
35].
The results from Trial No4 in the summer period, are divided into the thermal comfort part and the acceptance of the flexibility strategy. In
Table 6, the outputs concerning the occupants’ thermal comfort are presented.
To understand these outcomes, one should consider that during the demand response event, the set point temperature is raised, hence the risk is that the occupants should feel uncomfortably warm. To avoid that risk, a precooling phase is set before the actual demand response event. From the parameters of
Table 6, it can be deduced that the risk of overheating is avoided. Instead, the mean TSV is −1.11 (comfortably cool) and some users indicated they would prefer to feel a bit warmer. To verify whether this sensation is due to the precooling phase, the answers to the corresponding question were analysed in
Table 7. Out of nine respondents, four occupants declared they did not notice the precooling phase, two occupants stated the precooling phase was appropriate, one thought it was not needed, one that the room was too cool and one that the room was not cool enough. All the respondents considered that they did not need to take any action to restore their comfort. Overall, a good acceptance was shown through the experiment for the summer period.
As a last step, the comfort votes collected through the questionnaire are then compared with the standard predicted values, using the Predicted Mean Vote (PMV) method of the Fanger’s model [
36], which complies with AHSRAE Standard 55-2020 [
27]. From the questionnaire it was possible to deduce the occupants’ average clothing level (0.5 clo) and the average metabolic rate (1.1 met), while the physical characteristics of the environment were collected through sensors for each day of the trial.
The mean PMV obtained through the assessment was 0.35, while the actual mean TSV of the occupant was −1.11. The mean Predicted Percentage of Dissatisfied (PPD) was 8%, i.e., 92% of occupants should be thermally comfortable according to the standard predictions, while according to the questionnaire the percentage of respondents with −1≤TSV≤1 is 78%. In this case, the model slightly underestimated the actual discomfort of the occupants, as confirmed by other studies in the literature [
37,
38].
Results from Trial No5 (Ventilation control) consisted of a series of events where overly high CO
2 levels activated a ventilation system. CO
2 levels are related to many variables, such as activity in the room, number of occupants, ventilation rates and many others that are less dynamic, such as space volume, plants and building construction. CO
2 levels can serve as a “proxy” for the number of viruses in the air [
39]. It is well-known that good ventilation prevents the spread of viruses but continuous ventilation can result in the inefficient use of energy [
40]. The control system used at the PoC pilot site premises is able to initiate ventilation when CO
2 levels are high and allows the balancing of the occupant comfort and energy-savings, helping to improve indoor air quality.
Trial No6 (Crowdsensing) consisted of the design of a mechanism where the room temperature is adjusted in a dynamic way in real-time, according to the past and current votes a person has provided with regards to their comfort. This continuous voting system for thermal feedback only takes into account the current occupant’s past and current preferences, the latter having a greater influence. The acceptance of this real-time method is still under evaluation, where Cramer’s V association test [
41] is used to identify the strength of the association among vote types, and Spearman’s q correlation test [
42] is used to identify the direction of the association.
Based on these trials, the majority of KPIs were successfully demonstrated at the PoC pilot site, and
Table 8 below presents the results.
The results from the trials offer important information about the services provided in the PoC pilot premises and are used to validate the implemented architecture through the achievement of the set goals. Through the implementation of these trials, valuable lessons are also learned about the process itself that can be considered as guidelines when replicating the solution in new buildings. The most important lessons learned from the PoC pilot site are listed below:
It is possible to reduce energy costs by load shifting.
Energy consumption prediction using ML methods can help to estimate the energy savings in an accurate way.
For the success of a demand response strategy, sending a day-ahead notification to the occupants would be useful. From a beta test, we noticed that users tend to interrupt the demand response event, either intentionally in order to achieve comfort regarding the expense of DR aims or accidentally.
When designing a DR strategy, the benefits of the thermal inertia of the building should be taken into account for optimised results
The time needed to fill the feedback questionnaire decreases after the first time: in our specific case, the average time needed to fill the questionnaire after the first demand response event was 227 s, while the average time after the second one was 121 s and after the third one was 81 s. We believe this information can encourage the occupants to keep sending feedback in user-centric experiments, such as Trial No3.
The precooling phase should be adapted to the thermal preferences of the occupants, as some users stated that they would have preferred a higher temperature. Maintaining the same ventilation rate—designed according to average room occupancy and area—is suboptimal due to recent changes in work habits, such as flexible work hours and work-from-home schedules. Therefore, a dynamic ventilation strategy based on CO2 levels is more appropriate and helps on energy savings.
Thermal votes can be used to detect malfunctions and problems in the functional settings of devices in a very direct way.