Air Combat Maneuver Decision Method Based on A3C Deep Reinforcement Learning

Round 1

Reviewer 1 Report

The paper presents an interesting approach to deep reinforcement learning for an unmanned combat aircraft vehicle (UCAV) using the A3C (Asynchronous Advantage Actor-Critic) algorithm in decision-making scenarios.

In general, the paper is well written, but it requires a good language review and some key issues should be corrected.

    I would strongly recommend solving the following critical issues in the paper:

1. Line 261 is written: "the relevant parameters are shown in Table 2". Why are these parameters relevant? Are there more parameters used in the experiment? Why are these additional ones not relevant?

2. Concerning Algorithm 1:

>> I recommend the authors use a better representation for the algorithm; it is hard to keep track of the commands for on line 2, while on line 7 and for on line 12, where each command ends?

>> the representation of the algorithm is not defined correctly, for example line 9, what is written is a sort of comment instead of a proper line of command expected to be seen in an algorithm.

>> this same problem occurs in lines 14, 15, and 16, for example.

>> So, the authors should separate things. First of all, write down Algorithm 1 correctly as an algorithm. Secondly, explain in the paper how the algorithm works.

>> The way Algorithm 1 is presented in the paper makes it difficult to read and understand.

3. Concerning the simulation results. 

>> The blue UCAV never wins? The only results were a tie, or a red side won?

>> Section 4 is named "Experimental Simulation and Discussion". But, section 4 does not present discussions. The results are presented with some brief explanations.

Also, in conclusion, the results are not discussed.

4. The Conclusion section should be rewritten. It does not present a proper conclusion; there is no consideration of the obtained results, how they can be compared, how they can be improved, etc.

5. In the paper is mentioned (more than once) that the method presented in the paper is effective and feasible. In lines 345-347, the authors try to justify both effectiveness and feasibility, but in my opinion, these elements are not adequately justified. This should be better explained in the paper. How the method can be said as feasible? Is it compared with another method? The same can be asked for effectiveness.

6. Besides, the authors do not mention anything about future work.

-----------------------------------------------

As follows here are some specific comments and corrections:

1. Review English terms such as "hot field" (line 20), "a lot of study" (25); all sound informal for a scientific paper.

2. "decision-making" is repeated twice in the same line 26.

3. typo in line 75 -> are not considered

    typo in line 316 -> Figures 14 and 15 show

    typo in line 322 -> Figure 16

4. In the text, some elements shown in Fig. 1 are not defined, such as "d" and "v_b"; they are only defined later in the text, but not when Fig. 1 is first presented to the reader.

5. Equation 19 should not be R_i instead of R_p? In case it is R_p, what does it stand for?

Equation 26, what does R_s stands for?

6. In Table 2, the terms our UCAV and enemy UCAV are used, but at the beginning of section 4.1, the terms red and blue are used. I suggest keeping the same terms throughout the paper once they are defined.

7. Label of Fig. 9 is missing the case. Is it case 1?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors have done interesting work applying the A3C algorithm to the decision-making of UCAVs in air combat. However, the document must be improved before publication.

The introduction lists existing works on the topic but does not discuss the differences/similitudes, advantages/disadvantages of the current approach versus the existing ones. Also in the results section, there is no comment comparing them with other works. 

Also, it is not clear from the document which are the restrictions or limitations of the approach. Is it valid for any type of aircraft (fixed wing, quadrotor, etc)? Is it scalable to multiple UCAVS? And one of the main concerns: how is the UCAV supposed to have knowledge of the opponent parameters (pitch, speed, etc)?

The equations of the kinetic model could be better explained. For example, the meaning of tangential and normal overload can be included.

The DQN algorithm in case 3 is trained for the enemy UCAV to fight back, to try to escape, or to what? I can guess, but more detail should be given.

The same scenario is used for all the experiments. Wouldn't it be more appropriate to train the algorithm with a random initial state in the different episodes?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The authors have provided answers and corrections for all issues that I have described in my review.
Therefore, I should say the manuscript can be accepted.

Reviewer 2 Report