Comparative Evaluation of Different Multi-Agent Reinforcement Learning Mechanisms in Condenser Water System Control

Round 1

Reviewer 1 Report

This is an interesting research that is important for decarbonization and energy efficiency of centrally controlled HVAC systems in buildings which may be applicable for large or commercial buildings. I would suggest the following:

1) As this research focuses on the optimization of HVAC by comparing the performance of three MARL controller. To avoid similarity to your previous article, I would suggest that section 1.1 to 3.1 of this manuscript should be refined and summarized the salient things to constitute the introduction /background of this study in a continuous write-up of 1-2 pages. This is because I can see very  similar expression and presentation in this manuscript with your recently published article in Energy and Building.

2) I observed that the result simulation in table 6 show little differences between the energy performance of Division and  the Interaction MARL controller, how can you substantiate the more?

3) Great Job!

 

 

Author Response

Thank you very much for your prompt review. The manuscript has been revised with changes marked. And details are addressed in the attached response letter.

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper presents a comparison of two specific model-free reinforcement learning techniques applied on a heating, ventilation, and cooling (HVAC) control system. The techniques are used to compare the difference between three categories of multi-agent reinforcement learning (MARL) techniques: division, multiplication, and interaction. Generally, this is an interesting work, with a few issues that need rectification.

1. The motivation for model-free control methods is lacking; there should be some motivating statements on the necessity/superiority of model-free methods at the start of subsection 1.2 to motivate the methods studied in this paper.

2. Some more explanation should be given for the three categories in the final paragraph of Section 1, to explain the differences between the methods, and their general applications. As it is, the differences are only given much later in the Section 3.

3. A clearer definition of `cooling season' needs to be provided. Furthermore, Figure 2 needs to be labelled for clarity.

4. Check the numbering of Section 4. Furthermore, how is the PID feedback control tuned in case #5? The explanation in the last paragraph before subsection 4.1 is not clear.

5. Fix the alignment and bulleting of points in Table 4.

6. Make sure to define abbreviations (e.g.,  HVAC, COP, PLR) before their first use both in the Abstract and also in the text.

Author Response

Thank you very much for your careful review. The article has been revised with changes marked. Details are listed in the response letter.

Author Response File: Author Response.pdf

Reviewer 3 Report

The authors dealt with high-dimensional control problems with multiple agents. To address this problem, several MARLs are available. The authors aim to perform a comparative analysis of Division, Multiplication, and Interaction. The simulations show interaction is more suitable for multi-equipment HVAC control problems.

In the abstract, HVAC – Write the abbreviation at its first occurrence. Avoid the word “good performance” Instead, the authors can mention the quantitative value for interaction’s performance.

In most sections, the authors started a subsection without a paragraph in that section itself. For instance, the authors started explaining the necessity of optimal control of the condenser water loop in central chiller plants without the proper motivation of the research. In section 2, the authors must mention the rationale for selecting the case system.

Figure 1: The labeling should be clear. Also, used outlines for the circles for a clear Venn diagram.

Figure 2: A legend for the different symbols will be helpful for the reader

Table 1: The third column is very confusing. A horizontal line will be helpful to differentiate between records.

Table 2: Hard to read. Do not center the text in the middle columns. It should be closer to the variable, especially when it has subscripts.

I did not understand the significance of Figure 3 and how the authors yielded the error-index values of the system model. The interpretation and significance of Figure 4 must be explained.

 The authors might have written the methodology in the manuscript. However, a separate research methodology section is required with a block diagram for better readability.

Author Response

Thank you for your careful review, the article has been revised with changes marked. Please check the attached response letter for more details.

Author Response File: Author Response.pdf