Robust Multiple Unmanned Aerial Vehicle Network Design in a Dense Obstacle Environment
Round 1
Reviewer 1 Report
This paper proposes a deep reinforcement learning (DQN) method to improve the robustness of a multi-UAV network while minimizing the number of links to avoid interference and consuming too many resources. This is done in an environment with obstacles, and it is studied what happen when a number of drones fail. The problem is divided in two: one the drone positioning and the other the link selection.
The paper is very well written and structured. The results are significative, and the proposed method works well enough. I am not sure about the dense obstacle scenario with lots of buildings, maybe the drones can fly to a higher altitude so there are less obstacles. Another thing to be considered is that the experiments move the drones is some kind of formation from point A to point B. I believe that in the event of a disaster the drones should cover and area as wide as possible and remain more or less static.
Author Response
Reviewer #1:
Comment:
This paper proposes a deep reinforcement learning (DQN) method to improve the robustness of a multi-UAV network while minimizing the number of links to avoid interference and consuming too many resources. This is done in an environment with obstacles, and it is studied what happen when a number of drones fail. The problem is divided in two: one the drone positioning and the other the link selection.
The paper is very well written and structured. The results are significative, and the proposed method works well enough.
Response:
Thanks for your appreciation of our work and these helpful comments have greatly improved the quality of this paper. A point-by-point statement of our responses to the reviewer’s comments is provided below.
Comment (1):
I am not sure about the dense obstacle scenario with lots of buildings, maybe the drones can fly to a higher altitude so there are less obstacles.
Response:
Thank you for your comment. In the “Airspace Management Regulations”, the flight altitude of UAV is limited. For example, the flight altitude of micro UAVs must be below 50 meters. Therefore, in urban environments, UAVs need to have the ability to avoid obstacles.
Comment (2):
Another thing to be considered is that the experiments move the drones is some kind of formation from point A to point B. I believe that in the event of a disaster the drones should cover and area as wide as possible and remain more or less static.
Response:
Thank you for your insightful comment. In practical missions, UAVs need to fly according to predetermined trajectory points. For example, UAVs move to point A, then to point B, and then to point C, etc. Affected by obstacles, UAVs may collide with obstacles when moving between two trajectory points. Therefore, an obstacle avoidance algorithm needs to be designed to ensure that UAVs can safely move between any trajectory point. In our work, we propose an obstacle avoidance method based on an artificial potential field, which enables UAVs to safely move to the next trajectory point.
As you mentioned, coverage problem is an important topic. Inspired by your comments, we will optimize formation control of UAVs by combining coverage issues in the future.
Thank you again for your comments.
Reviewer 2 Report
drones-2454624
This work presents a method combining formation control and link selection which can work in a distributed manner. For formation control, the proposed method keeps the UAVs compact in the obstacle environment through improved artificial potential field, and enable the UAVs to have many neighbors. For link selection, based on this compact formation, reinforcement learning is used to improve the robustness of network while reducing the number of network edges. The author’s work is timely new and interesting but to improve the quality of article, I have some suggestion as follows:
1. The current title of the article seems strange; therefore, I suggest a modified title as “Robust Multi-UAVs Network Design in Dense Obstacle Environment.”
2. A paper organization paragraph should be added at the end of the introduction section.
3. Section “4.3.1. The design of state” can be “4.3.1. State Design.” Similarly, revise the remaining sections and sub-sections.
4. Figure 7 caption can be “The number of UAVs neighbor.”
5. Throughout the paper, in every section & sub-section heading, and captions, the authors used the word “The” at the beginning of the heading which is completely useless. Therefore, revise all of them carefully and avoid such unnecessary words. Moreover, avoid all monotonous words throughout the article.
6. The English writing of thins article is completely of low quality. Just in the abstract, I have noticed more than 20 grammatical mistakes and typos. Therefore, this article needs a thorough proofreading to avoid such grammatical mistakes.
7. Comparison of the proposed method with other is weak. Therefore, I suggest the authors to add a separate table for the comparison of them with other state-of-the-art models. Because without a proper comparison, how the novel readers or researchers will understand that this work is novel or efficient as compared to the already existing methods?
8. For UAV communication, the author can refer to “A UAV-Swarm-Communication Model Using a Machine-Learning Approach for Search-and-Rescue Applications”, Drones, 2022” for more details.
9. Remove, the word “the” from figure 12 x and y-axis, also revise the remaining figures carefully. Similarly, the word “the” should be removed from Table 1 parameters.
10. What is the purpose of figure 12 (a)? I think no need to add this figure.
11. The contribution of this work needs to be further highlighted.
Extensive editing of English language required
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 3 Report
The paper deals with the design of a robust multi-UAVs network, ensuring high robustness in dense obstacle environment. In particular, the method proposed by the authors combines formation control and link selection, which can work in a distributed manner. For formation control, the proposed method keeps the UAVs compact in the obstacle environment through artificial potential field, enabling UAVs to have a large number of neighbors. For link selection, based on the generated compact formation, reinforcement learning is used to improve the robustness of network while reducing the number of network edges. Numerical results, considering three different network failure modes in a 3D urban scenario, are used to prove the effectiveness of the presented algorithm.
First of all, thanks to the authors for the work and for their interest and commitment in solving this kind of problem, since there are still several difficulties that must be faced and solved. The work is well organized, and the subdivision of paragraphs is well managed, but there are too many grammatical errors and strange expressions that prohibit fluent reading by the reader; for example, the use of the definite article with singular and plural nouns. Moreover, verbs must agree with subjects in number. English sentences and grammar should be revised and reframed, because the paper will be accessible to international readers.
The “Abstract” section well summarizes the contents of the presented papers.
The “Introduction” section presents the formation control problem poorly, with few bibliographical references. A starting point may be reading the following reference documents:
1. R. Olfati-Saber, “Flocking for multi-agent dynamic systems: Algorithms and theory”, IEEE Transactions on automatic control, vol. 51, no. 3, pp.401-420, 2006;
2. I. Maza, A. Ollero, E. Casado, and D. Scarlatti, “Classification of multi-uav architectures”, in Handbook of Unmanned Aerial Vehicles, K.P. Valavanis and G. J. Vachtsevanos, Eds. Dordrecht: Springer Netherlands, 2015, pp. 953-975, dOI: 10.1007/978-90-481-9707-1-119;
3. H. Su, X. Wang, and Z.Lin, “Flocking of multi-agents with a virtual leader”, IEEE transaction on automatic control, vol. 54, no. 2, pp. 293-307, 2009.
4. Consolini, L.; Morbidi, F.; Prattichizzo, D.; Tosques, M. “Leader-follower formation control of nonholonomic mobile robots with input constraints. Automatica 2008, 44, 1343-1349;
The “Problem Statement” section is well organized and well summarized the problem that has been addressed in the rest of the paper.
Moreover, there are some dark points that should be better clarified:
1. Specify the acronym “DQN” before using it on line 155. Specifying it several lines later and explaining its meaning may lead the reader into confusion.
2. “Adjt+1” represents the adjacency matrix at instant time t+1. Why is the time instant t specified in line 156?
3. Review the symbology used throughout the paper. For example, it would be preferable that the time is given as the argument of the function and not as a subscript, as in the case of the variable “Frep,i,t”. Another example is that of the symbol “
”, used both to represent the velocity vector and the row vectors that make up the “Dis” matrix.
4. When the equation (9) is introduced, the variable “l” is not specified. Furthermore, the variable “oi1” is called up again, used in equation (7) and introduced on line 176.
5. Specify the elements of the state vector “S” for each UAV of the swarm.
6. Clarify the elements of the “Dis” matrix. Do the elements for each row, and therefore for each UAV, represent the distance between the i-th UAV and the other UAV of the swarm?
7. Clarify equation (13) and lines 210 and 211.
8. Lack of units of measure throughout the document, including figures and tables.
9. In the “Numerical Simulation” section it would be interesting to verify the trend over time of the distance between the UAV of the swarm and evaluate the ability of the algorithm to maintain a desired distance between them.
For these reasons, I think that the paper is acceptable after a major revision.
There are too many grammatical errors and strange expressions that prohibit fluent reading by the reader; for example, the use of the definite article with singular and plural nouns. Moreover, verbs must agree with subjects in number. English sentences and grammar should be revised and reframed, because the paper will be accessible to international readers.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
NA
NA
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
The paper deals with the design of a robust multi-UAVs network, ensuring high robustness in dense obstacle environment. In particular, the method proposed by the authors combines formation control and link selection, which can work in a distributed manner. For formation control, the proposed method keeps the UAVs compact in the obstacle environment through artificial potential field, enabling UAVs to have a large number of neighbors. For link selection, based on the generated compact formation, reinforcement learning is used to improve the robustness of network while reducing the number of network edges. Numerical results, considering three different network failure modes in a 3D urban scenario, are used to prove the effectiveness of the presented algorithm.
First of all, thanks to the authors for the presented paper and for their effort in correcting the document. However, referring to the answers reported in the document “drones-2454624-coverletter.pdf”, some points are still to be clarified. In particular:
1. Comment (8): in my opinion, for a better understanding of the paper, it is necessary to specify which are the elements composing the state vector Si(t) for each UAV in terms of position, velocity, etc.
2. Comment (11): in my opinion, even if the experiments were performed on a simulation platform, since some of the variables used are dimensional quantities (e.g., maximum velocity of UAVs in Table 1, safe distance in Table 1, etc.), it is necessary for them to specify the units of measurement. Being an urban scenario, as defined in problem statement, assume that distances could be expressed in meters and velocities in m/s; therefore, it is possible and necessary to use the appropriate units of measurement throughout the paper, also for the variables specified on the axes of the presented graphs.
3. Comment (12): from the presented graph one can notice that the proposed algorithm, named as “our method”, presents oscillations once the desired distance between the UAVs is reached (the visualization of the graph could be improved by reducing the step size from 0.2 to 0.1. Again, it would be useful to specify the units of measurements on the y-axis). In my opinion, it would be appropriate to comment on this unwanted behavior of the algorithm and to provide, for a future release, a solution that can prevent oscillations. Indeed, in the literature this problem for potential methods is well known and, over the years, several researchers have tried to solve this intrinsic problem. I recommend reading the following reference documents:
a. Koren, Yoram, and Johann Borenstein. "Potential field methods and their inherent limitations for mobile robot navigation." Icra. Vol. 2. No. 1991. 1991.
b. Damas, Bruno D., Pedro U. Lima, and Luis M. Custodio. "A modified potential fields method for robot navigation applied to dribbling in robotic soccer." RoboCup 2002: Robot Soccer World Cup VI 6. Springer Berlin Heidelberg, 2003.
c. Howard, Andrew, Maja J. Matarić, and Gaurav S. Sukhatme. "Mobile sensor network deployment using potential fields: A distributed, scalable solution to the area coverage problem." Distributed autonomous robotic systems 5. Springer Japan, 2002.
d. Li, Feilong, et al. "Mobile robots path planning based on evolutionary artificial potential fields approach." Conference of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013). Atlantis Press, 2013.
4. Comment (13): I report below some of my suggestions regarding the correct revised sentences:
a. The vmax represents the upper limit of the velocity.
b. The lb is the lower bound on the number of nodes in the largest connected subset.
c. The is the link selection strategy.
For these reasons, I think that the paper is acceptable after a minor revision.
The English sentences and grammar have been revised and reformulated, in order to make it easier to read for an international audience.Only a few sentences remain to be corrected; see point 4 in the section "Comments and Suggestions for Authors".
Author Response
"Please see the attachment."
Author Response File: Author Response.pdf
