Predicting the Energy Consumption of Commercial Buildings Based on Deep Forest Model and Its Interpretability

Round 1

Reviewer 1 Report

This research compares performance of different machine learning methods on CBECS 2012 datasets. The research topic is important and worth investigating. The paper is well written but has some minor grammatical/formatting issues. However, before reaching a final decision, there are several concerns that need to be addressed first. Please find below my comments:

1- literature review section is weak. There are many similar research papers published in recent years, but they are not cited in the paper.

2- CBECS datasets were used by several studies to test prediction performance of different machine learning methods. it is essential to report their prediction performance. Including this information will enable readers to better evaluate the performance of proposed machine learning method in the paper.

3- The results should be statistically compared, considering that machine learning methods can have different performances in different runs due to different initializations and/or optimization algorithms. Addressing this issue can ensure the robustness of the conclusions.

4- Could you please provide information about the portion of test, validation, and training data used in the study?

5- To improve clarity, consider providing the settings of each machine learning method in a separate table, including details such as the number of hidden layers and optimization algorithms used.

6- Please provide a brief explanation of the data standardization process used in the paper.

7- Discuss the limitations of this research.

8- Some figures may need improvement in terms of clarity, specifically Fig. 10, 11, and 12 where numbers and texts can be enhanced.

Minor editing of English language required.

Author Response

Dear Reviewer
Thank you very much for reviewing the manuscript. Thanks for your patience and usefulness suggestion.
In the following, we respond to the reviewers' comments point by point. In the revised manuscript, all changes are highlighted in red. Please refer to the attachment for details.
Thank you again for your positive comments on our manuscript.

Author Response File: Author Response.pdf

Reviewer 2 Report

This paper proposes a deep learning energy assessment framework with interpretability to support building energy efficiency design. The proposed framework is validated using the Commercial Building Energy Consumption Survey dataset, and some results are provided.

In its current form, the paper is not suitable for publication. An extension of the literature review is required in order to accurately assess the novelties and innovations proposed in the paper. In addition, the materials and methods description must be improved. Finally, the results and discussion must be improved.

In the following, some notes are reported.

General notes:

-Please report at least a reference for each concept already present in the literature that is exploited in the paper.

-For each equation reported in the paper, please indicate if it is a continuous-time equation or a discrete-time equation.

-Please check if the description and the measurement unit of each single term used in the equations are reported.

-Please accurately motivate each design choice and assumption in the paper.

-Please fix typos in the paper, e.g., adjust the punctuation at line 54.

Keywords

-I suggest to the authors to increase the number of keywords.

Section 1 (Introduction)

-The introduction must contain the topic, the motivation of the work, the literature review, a sketch of the novelties proposed by the manuscript and the paper structure. Please check the introduction and ensure that each of these subparts is correctly and deeply assessed.

-Please extend the literature analysis in the introduction and accurately claim the novelties of the paper in the introduction and in the core of the paper. Reference must be accurately analyzed and described within the scope of the paper. Research gaps in the current literature must be emphasized with respect to the proposed novelties.

-Lines 44-46: please further assess the definitions. For example, grey box models are not mentioned.

-Please majorly focus the introduction on the cope of the paper. Some parts seem to be a list of concepts.

Section 2 (Materials and Methods)

-Please further describe Figures 1-2.

-Please further detail and describe Sections 2.1-2.4.

-Please accurately motivate the design choices of Section 2.5 on the adopted metrics.

-Please extend and reorganize Section 2.6.

-The model developed by the authors would to be used for prediction and/or for simulation purposes? -How feedback is taken into account in the proposed models?

Section 3 (Results and Discussion)

-Please further describe Table 1.

-Please report some additional qualitative results on the performance of the developed models.

-Please accurately assess the reliability of the proposed models in terms of potential use for monitoring, control, optimization and fault detection/diagnosis strategies.

Author Response

Dear Reviewer
Thank you very much for reviewing the manuscript. Thanks for your patience and usefulness suggestion.
In the following, we respond to the reviewers' comments point by point. In the revised manuscript, all changes are highlighted in red. Please refer to the attachment for details.
Thank you again for your positive comments on our manuscript.

Author Response File: Author Response.pdf

Reviewer 3 Report

The paper talks about the machine learning methods used to assess energy consumption in commercial buildings. The paper is well-written and is within the scope of the journal. After carefully reading the paper, here are my comments for the authors to make the paper more readable and understandable.

1. The title should remove the "Research on" and should start with a verb.

2. The abstract should include all the methods used in the study to provide a better understanding of the work a reader is going to read. 

3. In "introduction" section, the literature review could be more extensive. It should include studies related to the work being carried out and based on that, gaps should be mentioned. A more elaborate literature review is required. 

4. In many places, sentences are too long. Please shorten the sentence to improve the readability. For instance, lines 143-148, 409-414 and many more. Please give a thorough read to the paper and improve. 

5. In the data-preprocessing section, pre-processing steps are mentioned. In step 2, how do you reduce the uncertainty just by deleting the rows? Also, in step-3, how can replac9ng missing values with 0 have no significant impact on the regression result?  

6. In Tables 3 and 4, you provided the model performance values for Energy consumption and Total EUI. How do RMSE and R2 decrease when they are linked to energy consumption? Did you try hyperparameter tunning to improve the model performance? 

Author Response

Dear Reviewer
Thank you very much for reviewing the manuscript. Thanks for your patience and usefulness suggestion.
In the following, we respond to the reviewers' comments point by point. In the revised manuscript, all changes are highlighted in red. Please refer to the attachment for details.
Thank you again for your positive comments on our manuscript.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The paper has been improved significantly. But there are still some concerns. Please see my comments below:

1-While the RMSE of the training data holds significance, it's essential to consider the RMSE of the test data for model selection (for RF in Line 113).

2-Identifying the optimal hyperparameters for the network models is indeed crucial. However, my comment seems to have been misinterpreted. To accurately assess the performance of various ML models, it's recommended to run each model multiple times with different initial points and statistically evaluate the results. Could you please clarify whether the results are derived from a single run or multiple runs? In the case of multiple runs, has there been an assessment of the statistical significance of the results? If the results are based on a single run, kindly specify this.

3- Regarding MLP, typically one or two hidden layers are often adequate. Could you please elaborate on the reason behind selecting five hidden layers?

Author Response

Dear Reviewer
Thank you again for reviewing the manuscript. Thanks for your patience and usefulness suggestion.
In the following, we respond to the reviewers' comments point by point. In the revised manuscript, all changes are highlighted in red. Please refer to the attachment for details.
Thank you again for your positive comments on our manuscript.

Author Response File: Author Response.pdf

Reviewer 2 Report

This paper proposes a deep learning energy assessment framework with interpretability to support building energy efficiency design. The proposed framework is validated using the Commercial Building Energy Consumption Survey dataset, and some results are provided.

In its current form, the paper is not suitable for publication. In particular, the materials and methods description must be improved.

In the following, some notes are reported.

General notes:

-For each equation reported in the paper, please indicate if it is a continuous-time equation or a discrete-time equation. Please check if the indicated equation types in the manuscript are correct.

Section 2 (Materials and Methods)

-Please accurately motivate the design choices of Section 2.5 on the adopted metrics. In the revised version of the paper, the authors improved this aspect but, in my opinion, a further step is required.

-The model developed by the authors would to be used for prediction and/or for simulation purposes? How feedback is taken into account in the proposed models? In the first round of review, the authors answered clearly to these questions. In my opinion, the reported answers must be included in the revised version of the paper.

Author Response

Dear Reviewer,

Thanks again for reviewing the manuscript.

Thanks for your patience and helpful advice. In the following, we respond to the reviewers' comments point by point. In the revised manuscript, all changes are highlighted in red. For more details, please refer to the attachment. Thank you again for your positive comments on our manuscript.

Author Response File: Author Response.pdf

Round 3

Reviewer 1 Report

Accept.

Author Response

Thank you for your thoughtful and constructive comments, which help us improve our work.