Round 1

Reviewer 1 Report

The paper presents an innovative method that uses a UAV for the detection and tracking of other ground vehicles, in order to be able to detect if they are in danger of collision with an EGV which is intended to be used in emergency situations. The paper is complete and well written: it explains the setup in which experiments took place, the hardware developed for these, and the methodology. Many results are presented, as a result of real-world experiments, and the different methodologies implemented were evaluated and compared.

I think that the work presented here is an interesting approach to the method proposed. It presents technical limitations in thinking about implementing such a system in a real life, as it is stated by the authors in the paper (for instance, the maximal speed at which the system operates and the maximal number of FPS that can be achieved), but it includes interesting and valuable advances in that direction.


I find the paper quite complete and suitable for publication. As a consequence, I have just some minor comments to do:

- On line 319 you reference [4] to justify the use of Yolov4 over Yolov5. However, in that paper just Yolov3 and Yolov4 are compared. In fact, there it says "Evaluation of a Yolov5 network and fine-tuning of the trained networks with further training data especially for real-life images is in progress".

- On the same line, is this comparison with respect to Yolov5s or any other size of the model?

- Line 225: Due to the dynamic limitations of DJI M600 (see [4,5]) the maximum speed of the maneuvers was 20-25 km/h. Higher speed maneuvers will be evaluated in the SIL simulation ([4]) of the system --> Is the 10Hz enhanced frequency of the RTK GNNS a limitation too in the maximal speed at which the system could perform?

- Line 264: This was carried out manually with a trial and error method. The system had instability unless the camera was mounted on it, so tuning was successful only when the payload was attached --> Is the PID stable enough between trials, or it needs to be adjusted just before each experiment?

- It is hard to interpret some figures. It would be nice to show the rules described in lines 709-714 directly on the legend of the figure. This applies to figures: 27, 28, 30, 31, 32,

- Some fixes that could be made to improve legibility:
     + Line 474: accurate data. Its position...
     + Lines 476-478: it is a bit difficult to read this sentence. Could you please rewrite it?
     + Line 822: The second is developed here --> rewrite to something more meaningful, like "The second is a self-development and it is the main contribution of the paper"

PS: I don't have access to the supplementary videos so I haven't been able to assess them.

Author Response

Reviewer 1:

- On line 319 you reference [4] to justify the use of Yolov4 overYolov5. However, in that paper just Yolov3 and Yolov4 are compared. In fact, there it says "Evaluation of a Yolov5 network and fine-tuning of the trained networks with further training data especially for real-life images is in progress".

- On the same line, is this comparison with respect to Yolov5s or

any other size of the model?

Thank you for the observation. Due to Nvidia Jetson NX's semi limited hardware capabilities only the small (s) version of Yolov3, Yolov4 and Yolov5 were considered as target neural networks. The larger networks proved to be too slow (below 10 FPS). The section was modified to better reflect this.  

- Line 225: Due to the dynamic limitations of DJI M600 (see[4,5]) the maximum speed of the maneuvers was 20-25 km/h.Higher speed maneuvers will be evaluated in the SIL simulation([4]) of the system --> Is the 10Hz enhanced frequency of theRTK GNNS a limitation too in the maximal speed at which the system could perform?

Thanks for the observation discussion of this topic is added at lines 496-497

- Line 264: This was carried out manually with a trial and error method. The system had instability unless the camera was mounted on it, so tuning was successful only when the payload was attached --> Is the PID stable enough between trials, or it needs to be adjusted just before each experiment?

The tuning had to be carried out only once, since we did not modify the camera mounting between experiments. And we experienced that the gimbal parameters are stable with temperature for example.

- It is hard to interpret some figures. It would be nice to show the rules described in lines 709-714 directly on the legend of the figure. This applies to figures: 27, 28, 30, 31, 32,

Thanks for the observation, now the rules shown in the figures are all listed below

- Some fixes that could be made to improve legibility:

+ Line 474: accurate data. Its position...

Detailed discussion is added now at lines 522-527

+ Lines 476-478: it is a bit difficult to read this sentence. Could you please rewrite it?

Rephrasing is provided now at lines 529-533

+ Line 822: The second is developed here --> rewrite to something more meaningful, like "The second is a self-development and it is the main contribution of the paper"

Rephrasing is provided now at lines 1116-1117

PS: I don't have access to the supplementary videos so I haven'tbeen able to assess them.

We have submitted them so please contact the editor!

Reviewer 2 Report

This paper described of system hardware and software components, test scenarios, object detection and tracking, and proposed development and evaluation of encounter risk decision methods.

Here are my concerns:

1.         Avoid lumping references as in [x-y], [x, y] and all other. It is not necessary to give several references that say exactly the same.

2.         As an object detection software system carried on UAV, the real-time requirement of object detection should also be high. Whether analysis of this can be reflected in the numerical results?

3.         In Figure 11, there are four legends, but there are only two curves in the figure. The author should explain the reason.

4.         It seems necessary to conduct parameter setting analysis and statistical experiments.

5.         The results should be discussed in-depth and with more insightful comments on various case studies.

6.         The writing of the paper needs to be further polished to make it publishable.

Author Response

1. Avoid lumping references as in [x-y], [x, y] and all other. It is not necessary to give several references that say exactly the same.

Thank you for the observation, we split the lumped references. Besides we note that there are multiple examples of lumping references in the MDPI template file. Regarding multiple references for the same topic they are usually detailed one by one just after the list and we find it practical to list all of the references related to a main topic upon introduction of that topic.

2. As an object detection software system carried on UAV, the real-time requirement of object detection should also be high. Whether analysis of this can be reflected in the numerical results?

The speed of different object detector neural networks on the Xavier NX on-board computer were listed in section 4 (Yolov5S @ 22 FPS), however, the reasoning of 10 FPS on-board operation was missing. New subsection (3.4) is added to Section 3 to give details on the 10 FPS operation speed. More complex object detectors give more robustness thus less tracking errors in complicated traffic situations, however, this lightweight setup was sufficient to solve the problem in our realistic scenarios with almost perfect tracks. The effect of different operation speeds to the decision capability is also an interesting question, but it is beyond the scope of this paper and is planned to be examined in an extensive simulation campaign.

3. In Figure 11, there are four legends, but there are only two curves in the figure. The author should explain the reason.

Thanks for the observation, the extra legends are removed.

4. It seems necessary to conduct parameter setting analysis and statistical experiments.

This is a really good observation, detailed parameter tuning was done as a new contribution and this observation inspired us to apply a more realistic evaluation criteria considering the stopped position of EGV after driver notification (including driver reaction time and EGV braking distance) and its minimum distance from the other vehicle track.

New Sections 7 and 8 are added about model parameterization and tuning and Section 9 (previously 7) was completely reformulated with the new results. That’s why all of these Sections are blue as they are either new or completely rewritten.

Two supplementary tables are provided including the results of parameter sweeps for tuning.

Statistical experiments were not possible as real demonstration data was utilized both for tuning and evaluation and it was collected in September 2022. There is no possibility for further experiments due to financial reasons however, a more detailed simulation campaign and evaluation of the tuned methods is planned as pointed out multiple times in the article. However, that should be the topic of another work considering the current length of this paper.

5. The results should be discussed in-depth and with more insightful comments on various case studies.

In depth analysis of tuned models is provided through tables and the detailed analysis of the plots of relevant scenarios.

6. The writing of the paper needs to be further polished to make it publishable.

We did our best to improve the language and readability doing several review rounds. This is shown by the enormous amount of blue texts in the article showing removed or added words and texts.