A Multi-Area Task Path-Planning Algorithm for Agricultural Drones Based on Improved Double Deep Q-Learning Net
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThe manuscript shows the test results of a flight planning algorithm for drone multitasking using Deep Reinforcement Learning (DRL).
The first part of the introduction does not seem related to the topic. It discusses the challenges of general agriculture but does not emphasize the specific topic of drone multitask planning and precision agriculture.
The introduction mentions several types of algorithms and some examples of algorithms used in similar works, among them the Double Deep Q-learning used in the manuscript. However, the title mentions that DRL was used. The justification and contribution of the work are poor and should be improved, especially the advantages or novel solutions proposed compared to other systems for planning drone flights in agriculture.
Section 2 mentions the use of sentinel-2 images over a frequency of 10 days with the GEE platform, but it is unclear which part of the methodology this section includes. It is not clear why a 10-day period when the GEE platform has data of 40 years or more and various types of satellite images and why use medium resolution images to plan drone flights.
Section 3 describes the methodology used, but the text has no structure, and it is unclear what steps are followed in the work. This section mentions DDQN as a decision algorithm, but the title mentions that it is DRL.
Figures (2, 3, 4, 4, 5, and 6) have missing information because the meanings of the abbreviations are not explained and there are no legends.
The results have methodology data in section 4.1. The description of the results is very ambiguous and not sufficiently detailed. There is no discussion or comparison of the results obtained with similar studies, so it is unclear what the contribution of the work is compared with other studies.
Discussion and comparison of results with similar studies are nonexistent.
The conclusions are weak; they are written as results and do not reflect what was obtained from the work.
The manuscript lacks structure and an unclear objective. It does not support the methodology of using satellite data to plan drone flights. Although it tries to explain the improvements introduced to other algorithms already tested, the work's innovative topic is unclear or unsupported.
Author Response
Please see the attachment
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsThe paper presents a deep reinforcement learning algorithm for multi-task regional path planning.
Comments:
1. The title has a space in reinforcement, please revise.
2. It may be helpful to define multi-task area specifically so it is understood by broader audience.
3. The innovation summaries look like smaller font size, please revise.
4. Is there a benefit to sampling during June and July? perhaps some elaboration here would be helpful.
5. What is PB-scale? define it.
6. On line 183, what is the square in the parenthesis of N? is this compile issue?
7. Line 186, what tensor are you referring to?
8. Figure 3, has typo for "select action"
9. Line 216, why the use of AAA, BBB, and CCC? how is this related to the other math symbols used? revise for more clarity.
10. Figure 5, should there be an arrow pointed to what is called the UAV agent? Also should this be renamed to something like UAV system that describes the multi-agent components inside?
11. Equation (6), has the square again. Same compile issue?
12. Equation (7), has square also.
13. In the Figures representing the results, what are the red boxes depicting. Please revise captions to reflect this.
14. What is the computational cost in time of each method? is there a possibility of deploying this system in real-time for future efforts? How does the improvements in path planning presented here impact the actual remote sensing data in the field? (i.e. is it worth the extra effort?)
15. I am not sure if it is just me, but all the mathmode symbols looked off and were not in line with the text. I suspect this would be corrected for published version but I would consider revising if not.
Author Response
Please see the attachment
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsDear authors, you have selected sufficiently interesting field of study to my mind. However, you need make strong improvements in paper structure and content description. At this moment, it looks on "spagetti" development, when different authors write some part of paper and all is mixed in one long document. Sorry, but it is bad practice. Why? I have stopped read document in page No. 9, because I have changed three times my view, because you explain something other, but each concept and solution require their own experiment design and explanation. For example, "multi-task" and "multi-area" problems are different, as well as, "multi-task area" and "multi-area tasks". Other example, you mention that Section 2 explains "CPP problem", but Section 2 presents data preprocessing using Sentinel-2 images and Otsu algorithm. In Section 3 you describe the framework, but in Section 2 you provide details about sub-element of framework breaking sequence of explanation.
So, what are my recommendations to improve paper?
Major: Use the standard structure of paper: Introduction, Material and Methods, Results and Discussion, Conclusions. Write the story structure, which can be transferred on paper structure. One author must rewrite content and other author - review paper (e.g. using "Duck" method). In your case "Material and methods" can start with experiment design (framework description) and then each sub-Section describes some part from this framework. There is lost information how error/quality is measured before Results.
Minor: these notes were identified within reading, but please primarily consider the Major recommendation, because all minor points are result of Major problem.
[Common] Text editing is required, e.g. there is superfluous space in title (word "reinforcement") or double points ".." in text line No. 16.
[Lines 13-18] There is contradiction in the logic: firstly you mention about multi-task CPP importance, then in the next sentence the fertilization path planning is developed. The sentence mention only one task "fertilization". (you have multi-area problem, that is written in Section 2/3)
[Abstract and Conclusion] Abstract and conclusions must mentioned achieved quantity results. There is not explanation how quality/accuracy must measured in your experiment. It is important thing for starting experiment development.
[Common] The abbreviations must be validated on their first definition before usage. Some abbreviations do not have definitions, or definition is provided later in the document.
[Line 126] "Innovations" are used mainly for economic and manufacturing fields, it will be "novelty" or "originality" in the case of scientific publication.
[Lines 133-135] To my mind, it is more impact neither originality of study, because it does not present some method or prototype, it is more fact, that something was done.
[Line 136-137] "Section 2 describes the CPP problem in 136 precision agriculture;", but the second section is about "Task Map Creation Based on GEE". And chapter is more related to case-study description.
[Line 183] There are lost mathematical symbols in math. expression.
[Fig. 2] Sorry, I have not understood the principle of sub-mission development. In the end 4 squares are depicted, which have geospatial split. So, are segments or these squares (grid cells) used to plan fly missions?
[Lines 175-187] If the very small region will be taken, which has noise fluctuations, the Otsu's algorithm will separate this area on two segments too, because it is main idea to divide on two groups independently on values. Therefore, there is lost some important description of data preprocessing before Otsu filter, how to validate interest regions using GNDVI. Without some validation, UAVs can process the healthy regions too.
[Fig. 3] "Edge extraction" - the Otsu's algorithm was mentioned in Section 2. It is threshold method for segmentation neither method based on edge extraction. The cloud mask usage is not described in Section 2. What did you do with empty cells?
[Fig. 3] Diagram does not depict input data for DDQL.
[Lines 204-210] The allocation of UAV (start points) must be explained. Are there drone for each sector (grid cell) or they fly from some central station?
It is very important information for path planning considering fly time restrictions.
[Lines 207-210] "collisions with walls", there was not mentioned something about walls in Section 2. How do you obtain this data (walls)? (they are not presented in Fig. 3).
[Lines 210-220] Explanation about States is not clear.
Author Response
Please see the attachment
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsThe version sent includes improvements to the preliminary version, especially regarding the title, introduction, and the manuscript's structure, but some comments still need to be corrected.
Lines 177 to 182 are part of the table as a table footnote. Or is it part of the text?
Although a table was added with the meaning of the abbreviations that appear in figures 2 to 6, I suggest that these be incorporated directly into the corresponding figures.
The discussion is still the weakest part of the manuscript; only two new references have been added, indicating that few papers are related to the topic. However, if this is the case, it represents an excellent opportunity to discuss and provide innovative ideas on the results. This section could be expanded and open to discuss topics that may interest the reader, improving the work's quality.
The conclusions are weak; they are written as results. Remember that the conclusions should summarize the results as the author's innovative ideas.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsThe structure of paper is sufficiently improved, it is well readable at this moment. However, there are problems with mathematical descriptions.
[Major] At this moment the mathematical description must be improved and validated with text description. Main detected problems: (1) the similar symbols of variables are used to different things; (2) there are errors in equations and suspicions that some information is lost.
[Minor] There are list of minor problems, which contains examples of problems with Major point too. Please, use this list as examples, but mathematical description must be checked for whole article.
[Fig. 2] Complete the sentence - what study area is depicted?
[Tab. 2] "The current area of UAV" - do you mean "current location of UAV"?
[Line 228-263] Please verify Equations and description of variables:
(1) For example, Fig. 4 depicts input {S, A} -> output R, but Fig. 5 - input {S, R} -> output A.
(2) If matrix M contains values [1; 4], than "a = (1, 2, 3, 4)" neither "i", and Eq. 3 does not require "i" index.
(3) Eq. 5 presents p(i, j) = 1, what do you mean with "1"?
(4) Eq. 4, please, do not use "i" for different variables "state" and "position". For UAV position I recommend to use (x, y) coordinates.
(5) Considering Eq. 4, you need have matrix U, which is not described.
(6) Describing using the form of Eq. 4, we depict firstly result, then condition. So, {EQ(2,0), if a = 0}, if correctly understood you wanted to write otherwise, if a = 0, then EQ(2,0)
(7) Eq. 5 is written incorrectly. Sorry I have not understood what you wanted to say.
[Line 261] 0-7 - please precise, that it is set of A.
[Eq. 6] EQ(2, 4) from Eq. 4 is not mentioned.
[Line 293] Do you really mean reward (R)? Considering to Eq.6 it can be negative. If it is simple random number, please, use other symbol for it.
[Line 304] Use some other symbol for step number neither S, which is defined as State in Eq. 5.
[Line 335] GELU - "activation function" neither "activations".
[Line 336] Is "A" a variable or English article?
[Tab. 3] Considering description, steps 1.-26. are Output results. There is lost header "Algorithm".
[Line 371] Windows 10 or 11?
[Line 371-372] Please, correct sentence, considering current description, Windows is written in Python 3.7.
[Line 370-372] 32 - RAM or GRAM?
[Ch. 3.1.] It belongs to "Material and Methods", but [Ch. 3.2.] will be "Results and Discussions".
[Line 423] There is lost part of sentence, you start "where .." without text before.
[Fig. 11] There are three colors, but legend presents only 2.
[Fig. 8-9 and Lines 437-439] You mention that BFS-BA algorithm worked worse. Meanwhile, Fig. 8 presents pictures where MLP-DDQN does not cover all locations, does it really complete the task? But Fig. 9 presents BFS-BA results with whole coverage. Please check text and figures for content consequence.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
