Energy Portal Design and Evaluation for Consumer Active Participation in Energy Services: Seven-Month Field Study with 234 Slovenian Households

Round 1

Reviewer 1 Report

The manuscript is about Energy protal design and evaluation for consumer active participation in energy services: seven-month field study with 234 Slovenian households which is well developed and can be considered as the interest to readers. However, few issues are in the manuscript which needs to be addressed: 

1. The abstract of study is not up to journal standrad and must be improved. Some novel contributions of the study needs to be added in the manuscript which will help to improve the quality of abstract. 

2. Keywords are sufficient. But, it will be more good if authors can also add some journal specific keywords falls under the scope of journal.

3. Introduction is well written up to some extent. But, few latest contributions have been ignored. Authors can add few papers from last two years in this section. Figure 1 is not clear and lacks in clarity. The objectives have been well defined by authors. 

4. Theoretical background is well presented. But it will be more good if authors can also discuss about the literature review protocol strategy here. 

5. Section 3 of study is well discussed and presented by authors. 

7. I appreciate the authors effort in developement of GUI which can be considered as the novel contribution for readers. 

8. More discussion on Figure 9 and Figure 10 is required. 

9. Discussion and conclusion section should be seperated. 

10. Conclusion section should include future scopes of the study also.

In the end I would say manuscript is well developed and presented and can be considered for publication after a minor revision. 

 

Author Response

The manuscript is about Energy protal design and evaluation for consumer active participation in energy services: seven-month field study with 234 Slovenian households which is well developed and can be considered as the interest to readers. However, few issues are in the manuscript which needs to be addressed: 

1. The abstract of study is not up to journal standrad and must be improved. Some novel contributions of the study needs to be added in the manuscript which will help to improve the quality of abstract. 

Thank you for pointing out and suggestion. We restructured the abstract and added novel contributions. The abstract now scopes novel contributions to consumers as it was developed for utility customers as an innovative service that along with the consumption feedback, together with ambient feedback and energy community features. The experience and insight gained during the study can also serve as a foundation for new demand reduction and energy communities services and are novel contributions for the utility and the readers as well.

2. Keywords are sufficient. But, it will be more good if authors can also add some journal specific keywords falls under the scope of journal.

Thank you, we changed the keywords as we change some of the keywords with new and better suitable keywords that also fall under the scope of the journal: Smart metering, Demand Response, Energy Saving, ICT, Portal, Energy feedback, Ambient feedback, Energy Community.

3. Introduction is well written up to some extent. But, few latest contributions have been ignored. Authors can add few papers from last two years in this section. Figure 1 is not clear and lacks in clarity. The objectives have been well defined by authors. 

We agree on both points. Therefore, we added few latest contribution regarding the active role of the consumers, the benefits that energy communities bring to all of the participants, involving them in the process of decision-making as well as the benefits that technologies, with emphasis on mobile technologies, brings to both consumers and utilities regarding their mutual interaction, making it prompt, direct and affordable. The papers that we used are the following:

• https://www.mdpi.com/1996-1073/14/16/4873
• https://www.mdpi.com/1996-1073/15/8/2866
• https://www.sciencedirect.com/science/article/abs/pii/S030142072200071X

Figure 1 was updated, so it presents the evaluation in the communication between utilities and consumers, how paper bills that are later enriched with additional information and were (in some cases they still are) delivered in paper or electronically, are replaced with portals that consumers can access through personal computers, mobile devices or even wearables. Nowadays, with smart home technologies, ICT and IoT, the portals are furthermore advanced and serve consumers with enriched information by combining heterogeneous data sources. 

4. Theoretical background is well presented. But it will be more good if authors can also discuss about the literature review protocol strategy here. 

Thank you very much, we definitely agree. The related and relevant literature was examined in an effort to clarify the crucial role that consumers play in contributing to demand flexibility, highlighting the efforts to make electricity tangible, particularly through the use of consumption feedback, leading towards consumer-centric solutions such energy portals are. In the SoA of the paper we have systematically examined: types of feedback, factors that influence the effectiveness of feedback and feedback delivery channels with emphasis on portals. We also added this in the paper.

5. Section 3 of study is well discussed and presented by authors. 

Thank you very much.

7. I appreciate the authors effort in developement of GUI which can be considered as the novel contribution for readers. 

We definitely agree on this so we added it to the abstract as well as to the conclusion. It is very good point. We appreciate it.

8. More discussion on Figure 9 and Figure 10 is required. 

Very good point. We reinforced the discussion on Figure 9 because there is still work to be done in order to maintain active consumers’ participation over time. The fact that the average number of visits was higher at first and then began to decline indicates that consumers were excited to try the portal at first and then continued with their usual behavior of checking it once a month. This is also in line with the literature that consumer engagement is a challenging process.

We also reinforced the discussion on Figure 10, pointing out that for visits made using a PC, there are two peaks (morning and evening), however for visits made using a mobile device, there is only one peak (evening peak). Furthermore we additionally discuss the fact that 70% of the visits were from PCs, while only 30% were from mobile devices, despite the fact that consumers prefer to use mobile devices. In our opinion this could be due to the smaller screen resolution that mobile devices offer and the limitations regarding how much data could be displayed at the same time.

Additionally we added axis-description for both Figures in the Results section.

9. Discussion and conclusion section should be seperated. 

We have separated the conclusion, thank you very much.

10. Conclusion section should include future scopes of the study also.

Thank you, we added also future work in the conclusion section. Based on the knowledge gathered from this study, future work will involve further scaling up to additional household, as well as the creation of new energy services for energy communities, demand reduction and demand response services, and their introduction to customers.

In the end I would say manuscript is well developed and presented and can be considered for publication after a minor revision. 

Thank you very much for your review.

Reviewer 2 Report

This research presents the process of developing an energy web portal enriched with integrated ambient feedback to provide additional information at the point of consumption. The results provide insights from a seven-month field study in which 234 Slovenian households tested and evaluated the portal. So the article aim is very interesting and actual. The case study based on Slovenian case is really worth to be considered. Also the topic of the paper suties to Electronics journal. 

Additionaly I would like to notice that both abstract and keywords are adequate to article content.

 

But my recommendations to improve the paper:

-> paper discusses only one position form 2022 year. It's not sufficient to claim that the literature review is actual. Thus please add at least 3 positions form 2022 (at least one of them should be from the Electronics journal).

-> between 4. Results 4.1. Process of portal design and testing please add a general introduction to section 4. 

-> please reconsider the approximation for figure 8. Is way of point by point connection correct? The reviewer recommends revising the way of presentation of this figure to make it correct in point of scientific level.

 

But at the last stage of the review, I wanna confirm that the paper is worth to be published after the implementation of indicated remarks.

 

Author Response

This research presents the process of developing an energy web portal enriched with integrated ambient feedback to provide additional information at the point of consumption. The results provide insights from a seven-month field study in which 234 Slovenian households tested and evaluated the portal. So the article aim is very interesting and actual. The case study based on Slovenian case is really worth to be considered. Also the topic of the paper suties to Electronics journal. 

Additionaly I would like to notice that both abstract and keywords are adequate to article content.

Thank you very much, we appreciate it.  Based on other reviewers recommendation, we additionally improved the abstract by adding novel contribution for the consumer, readers and the utility. This paper offers novel contributions to consumers as it was developed for utility customers as an innovative service that along with the consumption feedback, offers also ambient feedback and energy community features. The experience and insight gained during the study can also serve as a foundation for new demand reduction and energy communities services and are novel contributions for the utility and the readers as well. We also changed some of the keywords with ones that fall under the scope of the journal: Smart metering, Demand Response, Energy Saving, ICT, Portal, Energy feedback, Ambient feedback, Energy Community.

But my recommendations to improve the paper:

-> paper discusses only one position form 2022 year. It's not sufficient to claim that the literature review is actual. Thus please add at least 3 positions form 2022 (at least one of them should be from the Electronics journal).

We appreciate and also agree. Therefore, we added one positions from 2022 from the Electronics journal, regarding consumers’ role and the fact that they are becoming more active thanks to the progress of Information and Communication technologies (ICT) and the Internet of Things (IoT).

We added one position from 2022 from the Sustainability Journal, regarding the categories of feedback where the author talks about explanatory, temporal and comparing category of energy related feedback which is in line with the types of feedback that we designed for the portal. 

We added also one contribution from 2022 from the Electronics Journal regarding the processing of the data and how it may raise privacy concerns at the consumers, if not regulated, and decrease their willingness to participate. The General Data Protection Regulation (GDPR) sets guidelines that ensure that the data is private, it belong to the consumers and they must consent in order for their data to be processed. In our study, we followed the data management plan for the collection, storage and processing of personal data that was based on GDPR. Each consumers consented to share their data. 

The papers that we used are the following:

• https://www.mdpi.com/2079-9292/11/7/1157
• https://www.mdpi.com/2071-1050/14/15/9115
• https://www.mdpi.com/2079-9292/11/6/897

 

-> between 4. Results 4.1. Process of portal design and testing please add a general introduction to section 4. 

Thank you very much for pointing this out. We added a general introduction regarding the results that follow. We described how the results are focused on two aspects: the portal design process and the analysis of the collected data. Furthermore, we described each of them. The portal design process presents the steps followed to design the portal, from the planning and market research step, to the maintenance step. Furthermore, if presents a summary and description of the types of designed feedback. We also briefly described the analysis of the collected data, focusing on the process of on-boarding and explain how much time it took consumers to register on the portal and to first visit the portal, followed by the analysis of their interaction with the portal over time in terms of frequency, devices, time of the month and day, etc. And finally, consumers' motivations for registering and visiting the portal, their evaluation of the portal in terms of design, features, and information, as well as satisfaction with the portal will be presented.

-> please reconsider the approximation for figure 8. Is way of point by point connection correct? The reviewer recommends revising the way of presentation of this figure to make it correct in point of scientific level.

Very good recommendation. We are really thankful. We upgraded the Figure that it now along with the cumulative number of users registered [%] also shows the actual number of users registered on each sequential day, from day 1 to day 81 so the whole sample is presented.

But at the last stage of the review, I wanna confirm that the paper is worth to be published after the implementation of indicated remarks.

Thank you very much for your review.

Reviewer 3 Report

To start with, I would like to thank authors for their work in terms of interesting topic and well written article. It was a pleasure to read it.

 

The paper under reviewing is devoted to the energy portal design and evaluation for consumer active participation in energy services. The Energy portal was tested on 234 consumers of the largest Slovenian electricity supplier during seven months. The structure of the paper is good and it contain all essential sections, the methodology and gotten results as well as discussion is clear described. References are sufficient and up-to-date, no self-citation is detected. Figures are clear, used English is in high level, text is easy to read, presented material is cohesive and coherence. The theme of article can be considered for SI topic “Smart Grids and Internet of Things with Special Emphasis on Demand Response” as implications of paper can have impact  on Demand Response.

 

The main problem of paper is the luck of novelty, what scientifically new does the article suggest? Such portals are existing and have more features. Connected to this concern, where is the review of existing portals?

The paper might be improved if it contained the data on energy use behavior after the implementation of portal to consumers. Although it would be good to get understanding how profitable it is for both an energy provider and consumers to use this portal, including taking into account the installation of smart meters.

 

Lastly, I highly appreciate the opportunities smart grids give unified by monitoring and control system with feedback. The use of smart meters allows not only to increase the power supply system reliability by means of demand response, but also by decreasing efforts and duration on fault localization (DOI: 10.1109/ACCESS.2020.3011836). I think authors could reinforce introduction by such kind of information.

 

In any case, I have a positive attitude towards the work written by the authors, and I believe that it can be accepted for publication due to the quality of the presented material. If the authors accept my comments, of course, it will also have a positive effect.

Author Response

To start with, I would like to thank authors for their work in terms of interesting topic and well written article. It was a pleasure to read it.

Thank you very much for your kind words.

 The paper under reviewing is devoted to the energy portal design and evaluation for consumer active participation in energy services. The Energy portal was tested on 234 consumers of the largest Slovenian electricity supplier during seven months. The structure of the paper is good and it contain all essential sections, the methodology and gotten results as well as discussion is clear described. References are sufficient and up-to-date, no self-citation is detected. Figures are clear, used English is in high level, text is easy to read, presented material is cohesive and coherence. The theme of article can be considered for SI topic “Smart Grids and Internet of Things with Special Emphasis on Demand Response” as implications of paper can have impact  on Demand Response.

The main problem of paper is the luck of novelty, what scientifically new does the article suggest? Such portals are existing and have more features. Connected to this concern, where is the review of existing portals?

During the planning step we analyzed existing portals such as Wattics, OPower, Arvato, FiWare, Solar, TIko, Eliq, WePower, etc., what we added in the paper. Other commercial portals with more features are available, but they are rarely described in the literature or they are also rarely accessible to those who are not registered. So, we followed (1) design guidelines from the literature in the theoretical background; (2) from the utility - corporate identity and business view; (3) the pilot setup – implemented only selected features that were in line with the research. Despite the fact that the presented portal is not a full product, it is an improvement for the existing customers and a step forward for the utility. This paper also offers unique nation-wide insights of the field study from several perspectives. It presents:

• an example of utility service development;
• the iterative development of UI;
• the establishment of monitoring system for interaction measurement;
• an example of ambient feedback integration – the smart shower head, with possibility of further upgrade with more sensors;
• a technical perspective of heterogeneous data sources integration and anonymization – electricity, water, survey.

The paper might be improved if it contained the data on energy use behavior after the implementation of portal to consumers. Although it would be good to get understanding how profitable it is for both an energy provider and consumers to use this portal, including taking into account the installation of smart meters.

The scope of the paper is to investigate how consumers interact with the portal. The energy consumption data was not part of the study. However, the full report regarding consumers’ behavior (water and electricity), prepared within the H2020 project is available at: https://www.newcomersh2020.eu/upload/files/Deliverable%205_2.pdf

Taking into consideration the extreme electricity prices during the energy crisis, we could say that it is very profitable for both energy providers and consumers to use this portal. Using the portal contributes to greater energy savings which leads towards lower bills for the consumers and lower costs for the energy providers.

Lastly, I highly appreciate the opportunities smart grids give unified by monitoring and control system with feedback. The use of smart meters allows not only to increase the power supply system reliability by means of demand response, but also by decreasing efforts and duration on fault localization (DOI: 10.1109/ACCESS.2020.3011836). I think authors could reinforce introduction by such kind of information.

Very good recommendation, we added this. It really reinforces the introduction. Smart meters enable users to actively engage, which not only increases power supply reliability but also reduces the time and effort required to locate issues and enhance power quality.

In any case, I have a positive attitude towards the work written by the authors, and I believe that it can be accepted for publication due to the quality of the presented material. If the authors accept my comments, of course, it will also have a positive effect.

Thank you very much for your review.