Autonomous Navigation Decision-Making Method for a Smart Marine Surface Vessel Based on an Improved Soft Actor–Critic Algorithm

Round 1

Reviewer 1 Report

Please find my comments in the attached file.

Comments for author File: Comments.pdf

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper is interesting, dealing with providing Decision-making Method for Smart Marine Surface Vessel using machine learning. The paper is well organized and easy to follow, few comments can be considered before acceptance as follows:

1. Please write the meaning of the abbreviation in the first appearance in abstract and manuscript such as SMASV, COLREG, please check the others.

2. Please write the sentences in passive form instead of using "We".

3. It is not clear if the collected data are from exp or just simulation, please elaborate.

4. If the data are coming from simulation, how did you ensure the accuracy of the results if it is not validated?

5. The conclusion must be improved, focusing on the results of the study.

6. Please write the references according to the journal format.

7. "Moreover, eight sea areas 19 are randomly selected to verify the generalization ability of the intelligent navigation system.", why eight seas, and what is the difference between each one?

8. Could you please provide more information about the SMASV. Dimensions, weight, speed, etc

Author Response

Please refer to the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

This is a well-written paper with excellent illustrations that applies a version of deep reinforcement learning regulated by maximum entropy to the problem of laser radar-based collision avoidance in a seaway. 

The introduction gives a comprehensive and informative introduction to the problem of collision avoidance for a vessel operating in a seaway with static and dynamic obstacles. 

Section 2 gives a clear description of the environmental model, the vessel model and the COLREGS scenarios that the control system must be trained to obey.

Section 3.1 gives a sketch of the SAC approach, which is somewhat difficult to follow because it relies heavily on notations introduced in Reference [26] by some of the authors.

Section 3.2 describes the formulation of the neural networks and components that comprise the SAC training framework. Some details are difficult to grasp - for example, do the critic evaluation and target Q-networks each contain two networks to allow the system to choose between two paths in each encounter? What are the consequences of choosing networks with either more, or less complexity?

Section 3.3 outlines the five-component reward function, which is reasonably intuitive but has several arbitrary parameters. Has a sensitivity analysis been conducted concerning the choices of these parameters?

Section 4 outlines how the system was trained and evaluated in Gazebo simulations. It shows the surprising result that training against static obstacles is sufficient for the simulated vessel to successfully survive complex scenarios involving moving vessels without suffering a collision and while obeying the COLREGS throughout.

In summary, this is a great paper that contains much of interest, particularly the application of maximum entropy to nautical navigation. If there is time, it would be nice to know how resilient this approach can be: for example, how does the speed and manoevrability of the SMASV compare to that of large commercial shipping? And for future work, how well does the system cope with obscuration and uncertainty in perception? 

 

 

 

Comments for author File: Comments.pdf

The quality of English expression is very good. A marked-up version of the manuscript is attached, with some grammatical comments and some questions about some places where the meaning is unclear.

Author Response

Please see the attachment

Author Response File: Author Response.pdf