Integration of Unmanned Aircraft Systems into the National Airspace System-Efforts by the University of Alaska to Support the FAA/NASA UAS Traffic Management Program

Round 1

Reviewer 1 Report

The paper is about the FAA/NASA Unmanned Aircraft System (UAS) Traffic Management (UTM) research and the involvement of the University of Alaska (UAF) in the program. The authors describe conducted flight tests at the FAA UAS test sites, the results, the lessons learned, and technology gaps. They also describe the significance of unmanned aircraft (including many references to market analyses/forecasts), the need for a UTM, and provide a high-level description of the FAA/NASA UTM concepts and architecture and associated flight campaigns.

The paper is more a field report than a technical/scientific paper as it mainly describes experiments and summarizes the results without going much into details of underlying technology/science. Hence, I didn't provide an answer to "Scientific Soundness". Nevertheless, this paper is of great value for anyone operating unmanned aircraft in common airspace or working on airspace integration (especially in the US) and related technologies.

Some readers would benefit from more references to the underlying technologies of the systems used. Moreover, this paper doesn't refer much to UTM work in other countries or work related to unmanned aircraft integration into common airspace before the UTM term was coined.

I'm not sure if this paper is within the scope of this journal. It may fit under "Spaceborne, airborne and terrestrial platforms" but the paper is not so much about the use of unmanned aircraft in remote sensing (although many unmanned aircraft are used in remote sensing). A better fit may be in MDPI's "Drones" journal.

There are some minor issues: Inconsistent use of underlines and slanted font to indicate beginnings of sections (and missing underlines, lines 255, 261, 290). Page number of Fig. 3 ref. [9] seems to be wrong (found 454 instead of 11). Ref. [4] could be shortened by providing the direct link to the PDF file.

Author Response

Thank you very much for your time and effort in providing this review. The other authors and I appreciate your feedback and we hope to answer all questions to your satisfaction. We have provided responses following individual comments below.

The paper is about the FAA/NASA Unmanned Aircraft System (UAS) Traffic Management (UTM) research and the involvement of the University of Alaska (UAF) in the program. The authors describe conducted flight tests at the FAA UAS test sites, the results, the lessons learned, and technology gaps. They also describe the significance of unmanned aircraft (including many references to market analyses/forecasts), the need for a UTM, and provide a high-level description of the FAA/NASA UTM concepts and architecture and associated flight campaigns.

The paper is more a field report than a technical/scientific paper as it mainly describes experiments and summarizes the results without going much into details of underlying technology/science. Hence, I didn't provide an answer to "Scientific Soundness". Nevertheless, this paper is of great value for anyone operating unmanned aircraft in common airspace or working on airspace integration (especially in the US) and related technologies.

Thank you very much. We hope this will be of use to others working in the fields of integrating UAS into the NAS, and for planning/conducting BVLOS missions.

Some readers would benefit from more references to the underlying technologies of the systems used. Moreover, this paper doesn't refer much to UTM work in other countries or work related to unmanned aircraft integration into common airspace before the UTM term was coined.

Yes, we did make a decision to limit the scope of the paper to that of direct UTM work being accomplished in the US, and specifically that which UAF participated in. We hope that this is acceptable. With regard to the technologies used, we will attempt to better emphasize the specific UAS components and supporting sensors which enable BVLOS flights.

I'm not sure if this paper is within the scope of this journal. It may fit under "Spaceborne, airborne and terrestrial platforms" but the paper is not so much about the use of unmanned aircraft in remote sensing (although many unmanned aircraft are used in remote sensing). A better fit may be in MDPI's "Drones" journal.

We are unsure how to address this concern. We were specifically contacted by the editorial staff of this journal to submit an article on this particular subject for consideration in Remote Sensing Journal, Special Issue Trends in UAV Remote Sensing Applications. We hope that the content will be considered appropriate for the publication.

There are some minor issues: Inconsistent use of underlines and slanted font to indicate beginnings of sections (and missing underlines, lines 255, 261, 290). Page number of Fig. 3 ref. [9] seems to be wrong (found 454 instead of 11). Ref. [4] could be shortened by providing the direct link to the PDF file.

Thank you for pointing these issues out. We will correct the missing underlines which you have highlighted and resolve the issues with the reference [9] page number and the link for reference [4].

We will highlight changes in the updated manuscript for easy reference. Again, thank you very much for your time and consideration!

Mike Hatfield & UAF team

Reviewer 2 Report

Presented research is very important and interesting (as stated in paper introduction).

Perhapps clarity of the presentation could be improved by including overview of test and results in form of a table.

Also, a comment on possibke limitation if tests regarding use of only short range drones. Fixed wing drones were not used in tests.

Question about statement in line 91 & 92: The region segment was divided into North America, Asia-Pacific, Middle East & Africa, and Latin America. Where is Europe?

Also, in line 209, FIMS is written as abbreviation of flight management information system. Order should be changed.

Author Response

Thank you very much for your time and effort in providing this review. The other authors and I appreciate your feedback and we hope to answer all questions to your satisfaction. We have provided responses following individual comments below.

Presented research is very important and interesting (as stated in paper introduction).

Thank you very much. We hope this article is helpful to anyone working to integrate UAS into the NAS or in developing assets and plans supporting BVLOS missions.

Perhapps clarity of the presentation could be improved by including overview of test and results in form of a table.

Thank you for this idea. In the revised manuscript we have included a brief table summarizing the test information for the TCL1-3 campaigns at the beginning of Section 2, Materials and Methods.

Also, a comment on possibke limitation if tests regarding use of only short range drones. Fixed wing drones were not used in tests.

Yes, UAF flew almost entirely rotary-wing (and predominantly multirotor) sUAS for the purposes of the UTM TCL1-3 campaigns. Only in TCL1 did we fly a fixed-wing sUAS, the Aeromao Aeromapper. In the revised manuscript we have highlighted the sUAS types flown for each campaign in the table inserted at the beginning of the Materials and Methods section. Our missions were meant as a proof of concept, and we limited the UAS utilized to sUAS for purposes of practicality, manpower, and affordability for the campaigns. We have also made mention of this also at the beginning of the Materials & Methods section.

Question about statement in line 91 & 92: The region segment was divided into North America, Asia-Pacific, Middle East & Africa, and Latin America. Where is Europe?

Thank you for pointing out this omission. We have added Europe as one of the region segments in the study.

Also, in line 209, FIMS is written as abbreviation of flight management information system. Order should be changed.

Thank you for catching this. We have corrected this error.

We will highlight changes in the updated manuscript for easy reference. Again, thank you very much for your time and consideration!

Mike Hatfield & UAF team

Reviewer 3 Report

In the paper, the authors discuss the main campaign activities, results, and findings derived from the experimental activities in the integration of Unmanned Aircraft Systems (UAS) into the US National Airspace System.

The topic tackled by the authors is very interesting, as integrating manned an unmanned aircraft could be one of the ways to regulate the use of drones and improve people safety.

However, the paper presents many weaknesses, actually preventing its acceptance in the current form.

• The Introduction Section is very confusing. It does not allow to understand what is the problem tackled by the authors, what is the aim of the paper, and how the whole paper is organized. Conversely, it appears as a “collage” of several sources and images from other sources, and it does not contain a structured discussion of the paper.
• The Introduction includes a long discussion on the FAA/NASA UAS Traffic Management Program, that should be included in a dedicated section of the paper, and not in the Introduction.
• All the Figures included in the Introduction are very low quality, hard to be read on the printed paper, and taken from other sources. Are they really meaningful to the discussion? They only distract the reader from the objective of the paper. In addition, they are only attached to the paper, without any meaningful explanation and contextualization.
• The term Unmanned Aerial Vehicle (UAV) includes the Remotely-Piloted Aircraft Systems (RPAS), but they are not the same. An UAV can be also autonomous, not being an RPAS.
• The paper contains an overwhelming number of acronyms, very hard to remember while reading the paper. The authors should include an acronym list. Note that are also coincident acronyms (e.g. USS is used at least twice, for UAS System Supplier and UAS System Service).
• It seems that the whole system uses the ADS-B protocol for the communication between the UAS and the USS. However, it is well-known that ADS-B has severe reliability issues (see, for instance, [1]). Which was the reliability of the communication link? There was any interference/packet loss, especially in presence of close airplanes? How the proposed system is going to scale up?
• The paper appears to be more a technical report, than a scientific paper. To be a scientific paper, the paper needs to be organized better, with clearly-defined sections and a structured discussion of the experiments, the findings, and the results. In addition, the authors should make more effort to include new materials, and not to use already published figures.
• There are really little scientific references, and many of them are in the wrong format or miss important information (see, e.g., [4] and [14], just to name a few).
• Overall, the paper contains a lot of typos, grammatical mistakes, missing punctuation, and ill-formed sentences, frequently preventing the full understanding of the enclosed concepts. The authors are suggested to use even automatic correction tool to improve the quality and the style of the English language.

[1] Sciancalepore, Savio, Saeif Alhazbi, and Roberto Di Pietro. "Reliability of ADS-B communications: Novel insights based on an experimental assessment." Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing. 2019.

Author Response

Thank you very much for your time and effort in providing this review. The other authors and I appreciate your feedback and we hope to answer all questions to your satisfaction. We have provided responses following individual comments below.

In the paper, the authors discuss the main campaign activities, results, and findings derived from the experimental activities in the integration of Unmanned Aircraft Systems (UAS) into the US National Airspace System.

The topic tackled by the authors is very interesting, as integrating manned an unmanned aircraft could be one of the ways to regulate the use of drones and improve people safety.

Thank you very much. We hope that this information may be helpful to others who are working on issues relating to the integration of UAS into the NAS, or to those developing UAS/sensors or flight procedures necessary for supporting BVLOS missions.

However, the paper presents many weaknesses, actually preventing its acceptance in the current form.

• The Introduction Section is very confusing. It does not allow to understand what is the problem tackled by the authors, what is the aim of the paper, and how the whole paper is organized. Conversely, it appears as a “collage” of several sources and images from other sources, and it does not contain a structured discussion of the paper.

We have attempted to clarify the Introduction section by providing a concise motivation statement and  a roadmap to the rest of the document.

• The Introduction includes a long discussion on the FAA/NASA UAS Traffic Management Program, that should be included in a dedicated section of the paper, and not in the Introduction.

We have engaged MDPI to inquire on the flexibility of adding a dedicated section to contain this material. Tentatively, we have created a new Section 2, Background and moved the previous information into this new section (and renumbered subsequent sections). We hope this makes the Introduction and the rest of the paper flow more smoothly.

• All the Figures included in the Introduction are very low quality, hard to be read on the printed paper, and taken from other sources. Are they really meaningful to the discussion? They only distract the reader from the objective of the paper. In addition, they are only attached to the paper, without any meaningful explanation and contextualization.

We have attempted to include higher resolution pictures and diagrams for readability. We have included a number of these in the revised draft and will work to provide the higher quality pictures for the remainder in the final manuscript.

• The term Unmanned Aerial Vehicle (UAV) includes the Remotely-Piloted Aircraft Systems (RPAS), but they are not the same. An UAV can be also autonomous, not being an RPAS.

We have added some verbiage into Section 1 addressing this. The authors have backgrounds in the US military, DOD, aerospace industry, and small UAS ops. We understand the sensitivities various communities may have to particular terminologies and have attempted to acknowledge the terms and put them into proper context to avoid any confusion.

• The paper contains an overwhelming number of acronyms, very hard to remember while reading the paper. The authors should include an acronym list. Note that are also coincident acronyms (e.g. USS is used at least twice, for UAS System Supplier and UAS System Service).

Thank you for this input. A list of acronyms was provided in the appendix but may not have been noticed. We have added a note referencing this up front in the revised Section 1.

• It seems that the whole system uses the ADS-B protocol for the communication between the UAS and the USS. However, it is well-known that ADS-B has severe reliability issues (see, for instance, [1]). Which was the reliability of the communication link? There was any interference/packet loss, especially in presence of close airplanes? How the proposed system is going to scale up?

At the time of the UTM TCL1-3 efforts, NASA had directed that ADS-B be used as an additional safety precaution for the campaigns. The reliability of ADS-B was not a stated interest for the efforts and we did not pursue a study of this. As such, we used ADS-B for the flights for 2 reasons - so that NASA could track the signals for their purposes, and to demonstrate the feasibility of the signal being used to support certain SAA functions.

• The paper appears to be more a technical report, than a scientific paper. To be a scientific paper, the paper needs to be organized better, with clearly-defined sections and a structured discussion of the experiments, the findings, and the results. In addition, the authors should make more effort to include new materials, and not to use already published figures.

Yes, this is meant to be a technical report on the efforts undertaken by UAF to support the NASA/FAA field efforts in integrating UAS into the NAS. The purpose of this was to highlight the overall campaigns, how the tests were structured, major assets utilized, results, and lessons learned for future efforts.

Please note that based upon another reviewer's recommendations, we will be adding a short synopsis (in tabular form) at the beginning of Section 2 (now Section 3), Materials and Methods summarizing the UTM TCL flight campaigns, the UAS used in each, and overall results.

• There are really little scientific references, and many of them are in the wrong format or miss important information (see, e.g., [4] and [14], just to name a few).

We will review the references again and attempt to clean up any inconsistencies or formatting errors. We have corrected [4] and are presently reviewing the others for accuracy and consistency.

• Overall, the paper contains a lot of typos, grammatical mistakes, missing punctuation, and ill-formed sentences, frequently preventing the full understanding of the enclosed concepts. The authors are suggested to use even automatic correction tool to improve the quality and the style of the English language.

We have reviewed the document and attempted to revise the narrative sections with a mind towards the language and style used. We hope you will find these edits helpful.

We will highlight changes in the updated manuscript for easy reference. Again, thank you very much for your time and consideration!

Mike Hatfield & UAF team

Round 2

Reviewer 3 Report

The authors made significant efforts to address all the comments raised in the previous review round.

However, there are still some remarks on my side:

• In the Introduction, the acronyms should be introduced always between round parentheses. There are some acronyms, such as UAV, that are introduced between squared brackets, creating confusion with the References. 
• I personally find the Introduction very unusual, even for a Technical Note. In the Introduction I would like to find the main topic of the paper, any challenge or gap the paper fills, the contributions of the paper, and the organization of the paper. Honestly, I find only the main topic of the paper in the first lines.
• There are some figures, such as Figure 4 and Figure 5, that still are very low quality. 

Author Response

Thank you very much for taking the additional time to review our manuscript again. Your comments have been very helpful for guiding us in this process. Responses to this set of comments are provide below.

The authors made significant efforts to address all the comments raised in the previous review round.

However, there are still some remarks on my side:

• In the Introduction, the acronyms should be introduced always between round parentheses. There are some acronyms, such as UAV, that are introduced between squared brackets, creating confusion with the References.

We have replaced the "[UAV]" instance with the correct parentheses and found one other occurrence of an unintended '[' at the end of the document which we corrected.

• I personally find the Introduction very unusual, even for a Technical Note. In the Introduction I would like to find the main topic of the paper, any challenge or gap the paper fills, the contributions of the paper, and the organization of the paper. Honestly, I find only the main topic of the paper in the first lines.

We have added a summary at the beginning of the Introduction section which we hope captures this information succinctly.

There are some figures, such as Figure 4 and Figure 5, that still are very low quality.

We have replaced these figures with the best quality versions which we could find. We hope these will be satisfactory.

Again, thanks very much for the time and effort you have given towards making this a much better paper. Please don't hesitate if you have additional comments. Otherwise, best wishes!

Mike Hatfield & the UAF team