Exploring the Use of Geographic Information Systems to Identify Spatial Patterns of Remote UAS Pilots and Possible National Airspace Risk
Round 1
Reviewer 1 Report
The article presents a geospatial analysis of UAS operations in the United States, focusing on statistics related to remote pilot demographics and frequency of UAS sightings.
I believe the intention of the research is to use geospatial tools to identify correlations between remote pilot clusters and UAS risk. The concept is sound and interesting, and has the potential to yield fruitful insights. However, I found the analysis to be lacking in the results and discussion sections, and therefore recommend major revisions to be made to the paper.
My specific comments are as follows:
1. It will be useful to specify the problem to be addressed with either a 'Problem Statement' or a set of 'Research Questions' at the end of Section 2. The current problem statement (on lines 178-181) is too general.
2. In the methodology, there is no link between 'UAS Sightings' and 'UAS Risk', leaving the readers to make their own intellectual leaps and to draw their own conclusions. Indeed, while the terms 'risk' and 'safety' were introduced in Sections 1 and 2, these concepts were not present in the data analysis and discussion.
3. Results are currently presented as geospatial cluster and heat maps. These seem to be insufficient to draw concrete conclusions. I recommend supplementing the discussion with quantitative results as well.
Author Response
Please see attached response.
Thank You
Author Response File: 
Author Response.pdf
Reviewer 2 Report
In this study, the authors analyze publicly available data on pilot registration addresses (FAA registered airmen - manned and remote), UAS sightings, and the correlations between the two. Unsurprisingly, they find that their are more registered pilots closer to large airports, and that there are more UAS sightings in these areas. Given a limitation of the data, it is not possible to tell what proportion of these sightings lead to mishaps.
The motivation for this work is to help the FAA focus their attention on areas where the likelihood of hazards due to UAS intruding into is increased, and develop safety mitigation mechanisms that take geographical distribution into account.
For the former point, I don't find it credible that the FAA does not already do this. Perhaps the authors can provide evidence that this is not the case, or that this is seen to be a challenge. For the latter point, this seems to be the real motivation of the paper, and so I think it should be highlighted more prominently, and earlier on, but unfortunately, this is not developed at all, and the conclusions of the paper are rather unsurprising. If the authors were to suggest some concrete mitigations, show how they could be used within a safety management system, and use the data they analyzed to demonstrate that this would reduce risk levels, this would significantly strengthen the paper.
p1
It's unclear from the abstract what the point of the work is. The analysis "demonstrated implicit spatial relationships" but to what end?
p2
dropout of flight training - I'd say "dropout" is a noun, and "drop out" should be the verbal phrase here
The FAA provides two different methods - this should be moved after the list of three items at the top of p3
p3
part 61 vs Part 61 - pick one
unmanned technologies .. challenges the safety of the NAS - this is plausible but please provide evidence or a citation
inherent spatial nature of aviation - I guess so ... although everything physical is spatial; could you clarify a bit more?
a powerful tool -> with a ..
help the FAA better allocate its limited resources - doesn't it do that already?
around the airport operating area -> around airport operating areas
London-Gatwick - no hyphen
airport threats, airport disruption - what do these mean specifically?
graphically defined - do you mean categorized?
tracking UAS sightings - what is the definition of this term?
p4
is the FAA -> is that the FAA
not investigate nor confirm -> .. or confirm
small-time frame -> small time frame
whether the UAS was in fact an unknown hazard - right, so is it even worthwhile to look at "sightings" without this information?
faced with a record budget deficit - I don't see the relevance of this
safety oversight systems - is this the same as a safety management system?
FAA's culture shift - citation
FAA's RBDM strategic initiative - citation
developing risk-based models - this is an important point that motivates your work, and could be said earlier; unfortunately, you didn't follow up on this in the paper
p5
Researchers downloaded - who? you?
their associated airspace classifications - I'd suggest moving this bit into the next paragraph
the address of .. pilots - their home address? work address?
Tiger line files are .. - move this to previous paragraph, where this is first mentioned
The AVS Registry Database - you have two sections on this page where you talk about its format - combine them
also includes mechanics, repairmen, ... - this is a wider scope that was implied earlier in the paper; so your graphs consider all these individuals?
p6
each FAA Registered Airmen -> all .. Airmen
But why show all airmen? Don't you just care about remote pilots?
are only as precise -> is only ..
airspaces classes -> airspace classes
United State -> United States
This data layer allowed for later implicit geographic comparisons - where was this done? The airspace categories do not appear in any figures other than Fig 3.
spurious space after UAS sightings
p8
to determine if clustering and density patterns varied - why are you doing this?
p9
spatial analyst -> .. analysis?
hotspot .. hot spot - pick one
p10
square kilometers - does the FAA use km rather than miles?
urban areas and corridors - I don't see any corridors in Fig 5. Do you mean that the UAS fly these corridors between cities? Aren't the figures just showing registrations and not flight paths?
p12
suggests a significant amount of commercial activities - how can you tell that it's commercial?
p13
Fig 7 - why is the lower bound of the first density category 0.001 rather than 0?
have passed this test since 2016 - citation
but found that - citation
p14
implicitly compare - what is the word "implicitly" adding here? You're comparing. Aren't you doing that explicitly?
due to the imprecision of sighting location - do you have recommendations for data collection?
p15
The conclusions are fairly obvious. Even without the figures, would they be so hard to predict?
Author Response
Please find the attached response.
Thank You
Author Response File: Author Response.pdf
Reviewer 3 Report
Overall, I think this manuscript has good potential. It presents and interesting use case for GIS to help inform aviation safety decisions. Coming more from the aviation safety point of view, I think there are some areas that can be strengthened in the paper to better link the two areas. Below are the points that I consider need addressing to improve the contribution of the manuscript. They are presented in the order that I came across them rather than in order of importance.
1. Line 14 - airspace is one word not two
2. The authors should use numbered in-text citations as is journal formatting
3. Why is medical status relevant for Table 1? If including it, then you should also include number of theory exams and hours to complete the table.
4. The authors should note the applicability of Part 107 in the United States. For example, aircraft below 250g are exempt as they are considered harmless aircraft, see: https://doi.org/10.1177/1756829317691991
5. Huang et al. (https://doi.org/10.1016/j.techsoc.2021.101578) is certainly worth a mention, both in literature review and in discussion. In particular, they find non-compliance to be higher for UAS users in urban centres. That is a significant piece of information to feed into the discussion of how GIS data can help identify risk areas.
6. It is worth noting that the data gathered is coming from compliant users - those who have obtained a license and registered their drone. There will inevitably be non-compliant operators who choose not to as the chances of getting caught are not high. In line with this, rogue operators such as with the Gatwick and Newark examples are deliberately interfering with traffic. Intentionality separates them from users making genuine errors. Your sample is likely to fall into the latter category, and so they should be differentiated from those who carry out deliberate acts of interference.
7. It is also worth highlighting that airports themselves are increasingly using unmanned aircraft for tasks like inspections. They can clearly be operated safely within the operating area, and even over runways, taxiways and other critical infrastructure. Evidence suggests around 14% of airports are doing so https://doi.org/10.1139/dsa-2021-0048, with a number of useful applications within airport environments identified https://doi.org/10.1139/juvs-2016-0020
8. In line with the previous comment, the discussion of UAS in the airport environment is quite negative with the discussion being about "risks". That can be turned around to instead talk about how to ensure safety in airport environments. The two do relate to one another as safety involves minimising risk, but the goal is a safe operating environment for manned and unmanned aircraft.
9. Line 227 - the s is missing from the end of United States
10. The article skips straight from results to conclusions. It would be good to see some discussion, perhaps even changing the conclusion section to a discussion section. As things stand, the conclusion presents new ideas about how to use the GIS data and results, but these are not properly discussed. Topics like targeted safety promotion (and what this should look like), targeted enforcement action, and involvement of the airports and non-government organisations are all relevant. One thing that also appeared to be missing is the role of Community-Based Organisations (as the FAA calls them), which provide club environments and operating rules. Encouraging membership of such organisations within the higher risk areas may also be an interesting area of discussion, and something that many airports do as part of their own safety promotion efforts. Certainly lots can be fleshed out and I encourage the authors to do so.
11. There is a large quantum of sources that I would not consider as suitable for an academic journal. Particularly, there is a large number of news articles that have been cited. The authors should aim to reduce the number of such sources and try to incorporate more academic sources in the paper. Given the topic area, the high number of references to government agencies is entirely appropriate though.
12. Please be consistent in the terminology used for UAS - there are a few instances where other terms like "drone" have been used. The authors may like to clarify what term they will be using somewhere in the manuscript.
With the above changes made, I think the paper can make an interesting and useful contribution to the literature. Best of luck to the authors in revising the manuscript.
Author Response
Please find the attached response.
Thank You
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Thanks to the authors for addressing my comments. I see that minor edits have been made to the manuscript.
The authors have clarified that the article explores methods for utilising GIS information, but stops short of drawing explicit conclusions and suggestions to improve operational safety.
I am satisfied with their clarifications.
Author Response
Thank you. Please find attached our response.
Author Response File: Author Response.pdf
Reviewer 2 Report
p4
Why was the radius around Tampa airport chosen to be 10.78 miles?
p15
areas of higher risk - I guess it's implicit what you mean by this, but could you be more explicit? None of the figures indicate risk. Presumably it's a combination of pilot registration, airspace category, etc. Why not show this in a figure?
Author Response
Thank you. Please find attached our response.
Author Response File: Author Response.pdf
Reviewer 3 Report
While this manuscript has been improved upon the previous version, the authors have not addressed all of the points raised. Some have been rebutted, but others are important for improving the paper and its contribution to the literature.
Specifically, point 7 in the from the previous version is raised because the authors talk about operations near airports as always being risky and about sightings of unmanned aircraft within such spaces. The fact that around 14% of all airports are using unmanned aircraft themselves shows that operations in and around airports are routinely conducted in a safe manner. For this reason, sightings are also not reliable in of themselves as until there has been further investigation, it is unclear whether there was actually any safety threat posed by the unmanned aircraft. I think the authors may have misunderstood this point as the focus was not about the purpose of the UAV activity, but rather highlighting that operations around airports are not always more risky, as the risks can be easily managed.
In addition, point 10 has not been addressed. This significantly limits the contribution of the paper if the authors cannot explain how using GIS to identify spatial patterns would actually be useful. Applying new tools to see things can be interesting, but who is going to be interested in using them, and for what purposes. There needs to be some examples of how stakeholders like the FAA could leverage such data to help better target initiatives or similar. The suggestions of targeted safety promotion (and what this should look like), targeted enforcement action, and involvement of the airports and non-government organisations (such as community-based organisations) are useful for identifying such uses of the data. The authors do not need to properly investigate these as topics like safety promotion, enforcement, and stakeholder engagement are all well established in the literature. Rather it is illustrating how the combination of your paper with such established concepts may provide useful applications of your approach. The discussion section still doesn't do a lot of discussing, which makes it hard for readers to draw the 'so what' from the paper.
With these points properly addressed, then my comment still stands that this paper can make a useful contribution to the literature.
Author Response
Thank you, please find attached our response.
Author Response File: Author Response.pdf
Round 3
Reviewer 3 Report
Thank you for these changes, I do think they have improved the manuscript by making things clearer and explicitly providing areas for future research.
