2.1. Overview of the Approach
The RescueFly concept of operations plans to assist SAR missions at Lake Geierswalde and Lake Partwitz utilizing automated UAS based in decentralized hangars. We use this opportunity to develop an optimization model to solve the related FLP for UAV hangars, which shall provide minimum service time to hotspot areas where accidents are expected more frequently. These hotspots serve as demand for the FLP, so they must be identified based on their geographic characteristics and nearby amenities. Furthermore, positive and negative location factors must be determined to identify hangar candidate locations. We propose to derive the required information from open-source geographical data, so it is easily transferable to other regions or other FLP.
Furthermore, restricted areas and varying wind conditions affect the service times. This is considered in the FLP with the development of a standard SAR mission, which is optimized individually for each combination of hangar candidate and hotspot location in different scenarios. For the flight time to and from the hotspot area, we consider national and European regulations as well as external risk factors, such as potentially crowded areas with an A* algorithm [
39] on a binary occupancy grid to determine the shortest restriction-free flight path. Furthermore, we determine the search time required to find the person in distress in the derived hotspot area. Finally, the endurance of the UAV is also considered a constraint to the FLP.
A location candidate must provide solid ground, power supply, and reasonable access for installing and maintaining the system while being located outside restrictive areas (e.g., hazard areas, UAS geographical zones, natural reserves). In addition, vegetation shading must be avoided to permit reliable communication links to the emergency centers. Furthermore, the location shall grant swift access to designated beaches, recreation sites, hotel and camping facilities, grasslands, and other facilities where people are engaged in activities adjacent to lakes e.g., barbecue areas, boat slipways, and boat rentals. When planning a SAR mission, the risk to third parties, i.e., the air and ground risk, must be considered. SAR operations are excluded from the remit of Regulation (EU) 2018/1139 [
40], so the competent national authority is responsible for regulating SAR operations. According to § 21k LuftVO [
38], authorities conducting SAR operations are permitted to fly through UAS geographical zones [
2,
38]. As the UAV hangar locations shall provide a robust and optimum solution, complying with or flying through various UAS geographical zones can be selected operationally, e.g., depending on the urgency or the exposed crowd size. For this purpose, this paper considers five different scenarios:
- 1.
Restriction-free flight from the hangar to the hotspot by invoking the special rights of SAR authorities according to § 21k LuftVO [
38];
- 2.
Compliance with specified air risk relevant UAS geographical zones according to EU 2019/947 [
2] and § 21h LuftVO [
38], e.g., required distance to airfields;
- 3.
Compliance with UAS geographical zones relevant to air and ground risk;
- 4.
Compliance with all UAS geographical zones; and
- 5.
Compliance with all UAS geographical zones and avoidance of crowded areas.
Furthermore, the solution should be robust against the wind. Therefore, different wind cases are considered for each scenario based on the extensive study of Chu et al. [
13].
2.2. Acquisition of Open-Source Data
This section reviews open-source data to retrieve relevant information for identifying UAV hangar location candidates, which are divided into three georeferenced data-requirement groups, i.e., (1) positive and negative location factors for establishing UAV hangars. (2) Data are required to identify areas of high intrinsic risks for waterside accidents. (3) The UAS geographical zones according to EU 2019/947 [
2] and § 21h LuftVO [
38] must be identified for the mission planning with incrementally minimized external risks and environmental impacts according to our scenarios.
The primary source for the former two is the OpenStreetMap (OSM) (Available online:
https://www.openstreetmap.org/ (accessed on 22 November 2022)), a community-driven database for georeferenced data layers. OSM defines the georeferenced data with nodes, ways, and relations to describe the geometry, supplemented by tags (key-value principle) describing the object’s function. Using the Overpass API (Available online:
https://overpass-turbo.eu/ (accessed on 22 November 2022)), it is possible to define queries for extracting data based on region, layers, and tags. The location factors are based on the surface and its ability to accommodate UAV hangars. As listed in
Table 1, six tags for areas with grass and minimal vegetation are considered positive. Eight tags for areas with forests, large groups of trees, or wetlands will require additional construction work or shade the communication links, leading to negative location factors in
Table 2. Furthermore, each potential UAV hangar location requires road access for maintenance with a maximum permitted distance of 20 m from the road tags in
Table 3. In addition, the UAV hangar cannot be established on water surfaces, provided as Web Map Service (WMS) (Available online:
https://geoportal.brandenburg.de/de/cms/portal/start (accessed on 22 November 2022)). Finally, the power supply should be another positive location factor, but the required data are not public.
Table 4 lists 34 map features representing hotspot indicators. We assume that these features increase the probability of an accident on the water. For this purpose, we extrude the resulting map feature nodes and areas with a radius of 150 m in size, followed by an intersection with the water surfaces of the investigated lakes. If an area is a subset of the extruded hotspot indicators and a subset of either of the water areas at the same time, it is identified as a hotspot area. The radius of 150 m is based on the distance between buoys and shoreline of approximately 120 m plus an additional 30 m buffer since the OSM features may be georeferenced slightly outside of the shorelines. We assume that most swimmers tend to stay within the prescribed limits, increasing the risk of accidents in these areas.
For the planning of the standard SAR mission, the UAS geographical zones [
2,
38] are required. For UAV operations in Germany, the Digital Platform for Unmanned Aviation (dipul) provides a map tool (available online:
https://maptool-dpul-prod.dfs.de/ (accessed on 22 November 2022)) and web map service indicating the UAS geographical zones as separate layers. For data transmission, the availability of a sufficient broadband connection should be considered as well. However, the so-called Breitband monitor (available online:
https://www.breitband-monitor.de/mobilfunkmonitoring (accessed on 16 December 2022)) of the German Bundesnetzagentur provides only coverage at the ground level, so open-source data at cruising and search altitudes cannot be retrieved thus far.
2.3. Definition of the Standard SAR Mission
As the battery capacity limits the operation duration of the UAV, a restriction-free flight path between each potential UAV hangar location and each hotspot, and a standard search mission must be defined to assess the accessibility of the hotspot and feasibility of the search mission, given varying UAS geographical zones and wind scenarios. Each standard SAR mission consists of three phases: the approach, the search mission, and the return to the hangar. For consistency, we assume that the approach and return flights have the same paths and vertical profiles. Furthermore, as shown in
Figure 2, we assume a vertical climb at the UAV hangar to the approach altitude
, which is maintained until the UAV reaches the hotspot area. There, the UAV descends vertically to the search altitude
, which depends on the required resolution to detect a person in distress, and continues with a search pattern at
. After the search, the UAV climbs to maintain the return altitude
until reaching the hangar.
A maximum altitude of
100 m above ground is assumed for approach and return, leaving a safety buffer to the maximum permitted altitude of 120 m for the ‘specific’ category [
2].
depends on the characteristics and orientation of the camera and the required resolution for the automated detection of a person in distress. The camera of our UAV has an aspect ratio of 4:3 with a resolution of
Mpx, a lateral field of view
, a vertical field of view
, and a 1/2.3″ CMOS sensor.
The camera faces perpendicularly downward (i.e., nadir) to the water surface, guaranteeing the best coverage and detection [
24]. Furthermore, the larger
is perpendicular to the search direction so that the UAV is centered above the middle of the covered surface in a single camera frame, as shown in
Figure 3. Then, the achieved pixel density
D in [px m
−2] per frame is given with the search width
and length
in [m] according to:
Using the tangent of two assumed right-angle triangles,
and
can be determined at given
with
and
:
The required minimum pixel density
significantly drives the optimum
to detect persons in distress autonomously. A trade-off is necessary between a high-as-possible
for minimum-time coverage of the search area for the fastest SAR, and a sufficiently high pixel density to solve the detection and recognition task reliably, i.e., to distinguish persons in distress from all other swimmers. For estimating
, a set of test images of 35 swimming volunteers has been taken at Lake Partwitz under sunny and clear conditions without any significant wind. From the set, 96 images of different pixel densities between 5 to 3300 px m
−2 were generated, an example set is shown in
Figure 4. Those images were presented to 10 test persons (3 female, 7 male) aged 30 to 40 (
), thus equaling 960 samples. The test person’s tasks are (a) detecting objects in the image and (b) recognizing and describing the activity of the swimming persons. If all persons per image are detected, task (a) is classified as positive; if at least one person remains undetected, it is considered negative. If the test persons describe all activities of the swimmers correctly (e.g., breaststroke with drawn legs), task (b) is classified as positive.
The experiments show an average
= 9 px m
−2 with
= 8 px m
−2, and
= 503 px m
−2 with
= 493 px m
−2. The task complexity correlates strongly with the number and types of objects per test image resulting in high standard deviations. Thus, images containing few volunteers or volunteers on floating objects (e.g., surfboards) show significantly lower
due to contrast and size. Moreover, their activities are recognized more reliably than images with many volunteers swimming closely together. The obtained
from the test persons serve as estimates for the Deep Convolutional Neural Network (DCNN) intended to automatically detect a person in distress, assuming it will not perform significantly better or worse than humans. van Dyck et al. [
41] confirm this hypothesis, in which the DCNNs ResNet18 and vNet achieved
and
accuracy, respectively, compared to
of human observers. With
, Equations (1)–(3) are rearranged to solve for
:
Since the goal of the search mission is the reliable recognition of the person in distress, it is assumed that
= 1981 px m
−2 is required to avoid misdetection, which is a ground sampling distance of 0.0225 m. With Equation (
4),
82.92 m is determined to fulfill the task (b).
With the standard UAV mission, a set can be generated from the potential UAV hangar location
F (Facility), a hotspot
H as the demand, a wind scenario
W, and a UAS geographical zone scenario
Z. The corresponding total flight time
, for
,
,
constitutes the evaluation metric. The flight distance
results from the shortest restriction-free flight path from
to
considering
. For this, the approach and return flight are assumed identical, i.e.,
. Since various shapes of hotspot areas exist and the search time depends on the coverage algorithm, we simplify the resulting hotspot areas
[m
2] to rectangles with the identical area and the edge length
[m] of
Figure 3. Furthermore, an additional search detour factor
accounts for different coverage algorithms and shapes, resulting in a search distance
:
According to the UAV manufacturer, a reliable cruise speed during approach and return
= 10 m s
−1 and a vertical rate
= 2.5 m s
−1 is achieved. During the search phase, we assume a slower search speed
= 5 m s
−1 to provide suitable coverage and reduced motion blur. The total flight time for successive maneuvers (cf.
Figure 2) is:
in Equation (
6) represents the detour factor per wind scenario
. The actual values are derived from Chu et al. [
13], using wind speeds below 11 m s
−1 as recommended. We computed
for seven wind scenarios in
Table 5 with the battery use from Table 10 [
13], averaging over all wind directions and normalized on 1 s of flight time. Furthermore, the wind scenario
with 0 m s
−1 and turbulence index 0 is added as a baseline case with
.
Given the flight endurance
E = 22 min, each rescue mission from
i to
j is evaluated, so
from Equation (
6) are only classified as accessible:
Thus, is the binary accessibility variable from to , avoiding and considering , with the Big-M parameter M and an infinitesimally small positive quantity .
2.4. Optimization Model for UAV Hangar Positions
This section describes the FLP model to determine
optimum UAV hangar locations with maximum accessibility to all hotspots
across all wind scenarios
and all UAS geographical zone scenarios
while minimizing the respective total flight time. For that, we consider a finite set
H of hotspots and a finite set
F of potential facilities with the binary success variable
from Equation (
7) and total flight time
from Equation (
6), such that:
With the binary parameter
and
P facility locations to be established, while
ensures that each
i is connected to only one
j:
To this end, we process the data of
Section 2.2 in a georeferenced 5 m × 5 m grid inside
latitude and
longitude. Then, the location factors from
Table 1 and
Table 2 identify the solution space for candidate locations. To reduce the computational effort, a spacing of 50 m between the candidates for UAV hangar locations is chosen, resulting in
candidate locations. The hotspot areas from
Section 2.2 are processed with connected component labeling, resulting in 11 separate hotspot areas of varying extents across the two lakes. Each hotspot area is represented by hotspot centroids of a k-means clustering algorithm where k is the quotient of the respective hotspot area and the smallest hotspot area as an integer, resulting in a total of
hotspot locations. Consequently, a total of
path computations for one optimum facility location and
calculations for two hangar locations is required. Since the restriction-free pathfinding with the A* algorithm [
39] has a high computational effort, we reduce the number of paths by applying the two techniques illustrated in
Figure 5.
First, we label the discretized binary occupancy grid using a connected components algorithm removing all inaccessible candidates due to restrictions.
Figure 5a illustrates the method for
as an example. All candidate locations labeled ‘yellow’ are valid connections to the hotspots, removing invalid candidate locations (e.g., ‘purple’) from the later pathfinding.
Second, a fast ray occupancy intersection algorithm checks if straight paths from
j to
i exist that do not infringe on restricted areas. If this is the case, the shortest path is already found, and the A* pathfinding around restricted areas is not required for this particular combination.
Figure 5b illustrates the procedure using
as an example. Green-marked location candidates permit direct paths, given one example hotspot at Lake Geierswalde. Red borders show interruptions due to the occupancy envelope, respectively, invalid direct paths.
With these two steps, all UAV hangar candidates without valid connections were removed, and the shortest path to all candidates with unrestricted straight connections were successfully found. Accordingly, only the candidates with the same connected component label in
Figure 5a and intersecting with the occupancy envelope in
Figure 5b require calculating a restriction-free path with the A* algorithm [
39] in the two-dimensional discretized operation space. Horizontal, vertical, and diagonal movements are allowed. The cumulative great circle distance of the georeferenced path nodes along the shortest path from each source to each sink was subsequently calculated, considering UAS geographical zones for all
.