Controlling Fleets of Autonomous Mobile Robots with Reinforcement Learning: A Brief Survey

Round 1

Reviewer 1 Report

In this paper, the authors present a literature survey focussed specifically on the use of reinforcement learning as a solution for controlling fleets of autonomous mobile robots. I have attached a commented version of the paper but will summarise the outcomes of my evaluation below.

MAJOR COMMENTS

- The authors claim in the introductory text that "no paper exists yet which provides an overview of existing research about the control of fleets of AMR with RL". This is too strong a claim and should be rewritten as "to the best of our knowledge, no paper has been published yet..." Also, this is not strictly true (acknowledged in section 2). For example, see the following paper (which should be cited):

K. Zhu and T. Zhang, "Deep reinforcement learning based mobile robot navigation: A review," in Tsinghua Science and Technology, vol. 26, no. 5, pp. 674-691, Oct. 2021, doi: 10.26599/TST.2021.9010012.

Please rewrite this passage so that the paper's contribution is already framed more realistically here (and not just in Section 2). 

- Section 4 should include a definition of POMDP

MINOR COMMENTS

- There are a few minor issues with written English throughout that should be addressed (see a few examples in the attached commented version of the paper).

DECISION

In my opinion, this is a mostly well-written and relevant paper, albeit probably a bit specific and with parts covered elsewhere (e.g. reference [21]). For that reason, I am proposing that the paper is accepted after a minor revision taking into account the comments above and in the attached file.

Comments for author File: Comments.pdf

Author Response

Dear Reviewer 1

first of all, thank you for the constructive and helpful review. We have taken note of the comments and listed them in the word file below.

Yours Sincerely,

Mike Wesselhöft

Author Response File: Author Response.pdf

Reviewer 2 Report

The article provides a fairly large survey on Reinforcement Learning methods for сontrolling fleets of autonomous mobile robots over a long period of time. However, the presented analysis is more of a formal nature, containing a brief description of the methods contained in references. The article provides a detailed redundant description of selection process of references for review and reasons for exclusion of references. At the same time, the authors completely do not pay attention to the analysis and comparison of the scientific methods given in these references in terms of the effectiveness of their application for solving these problems. There are no quantitative assessments of the results. It should be agreed with the authors that the content of the article is a fairly complete document, which provides an overview of existing research about the control of fleets of AMR with RL. However, given such a fairly detailed survey, the authors did not actually present any of their own scientific research on this topic. In this connection, the scientific significance of the article for publication in this journal is extremely insufficient. The conclusion presented in the article also seems trivial. The opinion of the authors that "for industrial usability, it is necessary to generate more real-world data for training such approaches and to minimize the data effort for training" does not represent a value from a scientific point of view. Based on the foregoing, the reviewer recommends that the authors advance in their own scientific research on сontrolling fleets of autonomous mobile robots and recommend the materials presented in this article as an initial stage and a basis for further research.

Author Response

Dear Reviewer 2

first of all, thank you for the constructive and helpful review. We have taken note of the comments and listed them in the word file below.

Yours Sincerely,

Mike Wesselhöft

Author Response File: Author Response.pdf

Reviewer 3 Report

The application of Reinforcement Learning (RL) in controlling a fleet of autonomous mobile robots (AMR) is reviewed in this paper. Many approaches about routing, scheduling, collision avoidance and the orientation in dynamic environments are introduced and in-depth analysis about using RL for the control of fleets of AMR is provided.This survey is systematic,comprehensive and quite valuable to know about the latest developments and further some research in these fields.    

The main comments are as follows.
1. Structurally Section 5.3(Fleet scalability) is not suitable to be put in Section 5 and is introduced in parallel with the two control approaches of centralized control and decentralized control.

2.Section 6.5(Orientation in dynamic environments )should be placed in the previous section of Section 6.4(Combining different problems) , so that the four sub-problems are first introduced.

3.The sentence “we will focus on the combination of collision avoidance and path planning in Section VI-B.”should be “we will focus on the combination of collision avoidance and path planning in Section 6.4.” 

Author Response

Dear Reviewer 3

first of all, thank you for the constructive and helpful review. We have taken note of the comments and listed them in the word file below.

Yours Sincerely,

Mike Wesselhöft

Author Response File: Author Response.pdf