Aggregation–Decomposition-Based Multi-Agent Reinforcement Learning for Multi-Reservoir Operations Optimization

Round 1

Reviewer 1 Report

This is an excellent paper discussing a multi-agent RL approach for reducing the curse of dimensionality in multireservoir operations optimization problems. The proposed approach proves on par with or better than other stochastic optimization methods. Overall I think this is a nicely written manuscript and I don't have major concerns. Good job.

Author Response

We thank the respected Editor and reviewers for their constructive comments and efforts.

Reviewer 2 Report

The article is well written. I recommend you add the discussion section so that the reader can connect well from results and conclusion sections. This will bring up well your finding compared to other study of the similar. 

Please cross check the reference, eg ref 1 is missing the year of publication

Author Response

4) Adding a new short discussion section (Section 4) before Section Conclusions, connecting results to conclusions as requested by Reviewer 2 and the Editor.
5) English editing and making a number of required corrections
6) Double checking and correcting citations format in the text and references list such as reference 1 pointed out by Reviewer 2.
We

Reviewer 3 Report

The authors proposed a multi-agent RL approach combined with an aggregation/decomposition (AD-RL) method for reducing the curse of dimensionality in multireservoir operations optimization problems. Results show that the proposed method outperform the other stochastic optimization methods.

However, the proposed method is simple and general in this paper, the majority of outcomes are quite obvious, thus not very interesting for a reader. Apart from this main drawback, which should require an extremely deep revision of the manuscript, there are other main issues that the authors surely must resolve before the manuscript can be suitable for publication:

1. The title of this paper is “Dimensionality Reduced Multi-Agent Reinforcement Learning for Multireservoir Operations Optimization”. The AD-RL method is proposed and emphasized in this paper. Please double check the reasonable of this title.
2. Also elaborate explanation of Aggregate-Dynamic Programming (AD-DP), Multilevel Approximation Dynamic Programming (MAM-DP) and Reinforcement Learning (RL), are unnecessary, as they don't add any value to the water related journals.
3. How to calculate and modify the parameters of ANN model?
4. The authors are asked to clearly state which is the novelty of the paper for which the paper deserves publication.
5. What the authors are trying to achieve, and finally what are the primary conclusions to be beneficial for WATER? It is not clear from the conclusions.

Author Response

1) Modifying the title of the paper as asked by the Editor and Reviewer 3
2) Clarifying the definition of expressions and terminology used in the text such as the difference between FP1 and FP2 models which was not clear before
3) Shortening sections 2-3, 2-4, and 2-5 devoted to AD-DP, MAM-DP, and RL approaches, respectively, through removing unnecessary materials as requested by Reviewer 3 and Editor.

Round 2

Reviewer 2 Report

The manuscript is well revised

Reviewer 3 Report

All requested revisions and comments have been done by the authors. Therefore, in my view, it can be considered for publication as it is.