Are Modern Market-Available Multi-Rotor Drones Ready to Automatically Inspect Industrial Facilities?

Abstract

Industrial inspection is a well-known application area for unmanned aerial vehicles (UAVs), but are modern market-available drones fully suitable for inspections of larger-scale industrial facilities? This review summarizes the pros and cons of aerial large-scale facility inspection, distinguishing it from other inspection scenarios implemented with drones. Moreover, based on paper analysis and additionally performed experimental studies, it reveals specific issues related to modern commercial drone software and demonstrates that market-available UAVs (including DJI and Autel Robotics) more or less suffer from the same problems. The discovered issues include a Global Navigation Satellite System (GNSS) Real Time Kinematic (RTK) shift, an identification of multiple images captured from the same point, limitations of custom mission generation with external tools and mission length, an incorrect flight time prediction, an unpredictable time of reaching a waypoint with a small radius, deviation from the pre-planned route line between two waypoints, a high pitch angle during acceleration/deceleration, an automatic landing cancellation in a strong wind, and flight monitoring issues related to ground station software. Finally, on the basis of the paper review, we propose solutions to these issues, which helped us overcome them during the first autonomous inspection of a 2400 megawatts thermal power plant.

1. Introduction

Industrial inspection is a well-known application area for unmanned aerial vehicles (UAVs) [1,2,3]. They are widely used for inspection of pipelines and overhead powerlines. However, are modern market-available industrial drones fully suitable for inspections of larger-scale industrial facilities? Let us discuss the power plant inspection task, to better understand the challenges of larger-scale industrial facility inspection.

Industrial inspection with UAVs is a well-known approach. Moreover, large drone manufacturers like DJI or Autel Robotics have drones specifically designed for this task in their product range. Figure 1 shows some examples of inspection flights performed on large industrial facilities. As can be seen, these tasks require flying in an area with many obstacles, very close to the equipment being inspected and sometimes even touching it with the drone’s sensors. Not surprisingly, in most cases, the inspection drones are still used in tele-operated mode. They serve as flying cameras, helping diagnostic engineers watch and check the desired equipment from the best angle and a closer distance. However, manual control has multiple limitations. First, images of the same equipment captured on different days have low repeatability regarding angles and distance, making it difficult to process them automatically. Second, in large industrial facilities with many wires, buildings, pipes, and other obstacles, most pilots have to clearly see the drone with their eyes to guarantee safe flight, even if the drone is equipped with a first person view (FPV) camera. Thus, each inspection flight is limited to a small area around the pilot, where the drone can be clearly distinguished from obstacles. After each flight, the operator typically changes locations, adding time to the process. In addition, the images captured during such a manual inspection have low repeatability, making them difficult to interpret and compare to data collected during previous inspections. As a result, it can take several days or even weeks to inspect a large facility with a manually piloted drone.

Figure 1. Examples of aerial inspection on large-scale industrial facilities: (a) drone inspection tasks in a cement production factory [4]; (b) a set of inspection routes on a large power plant [5] and an industrial drone during non-contact inspection of insulators on the one of these routes [5]; (c) an industrial drone during contact inspection on a refinery [6].

Excluding the smallest ones, modern industrial drones have the functionality to automatically follow a route composed from waypoints with pre-defined geographical coordinates. Moreover, at each waypoint, most of them can orient a camera and capture images with it. Thanks to advances in Global Navigation Satellite System (GNSS) Real-Time Kinematic (RTK) positioning, drones can determine their position with decimetre accuracy [7]. The heading can be estimated using the so-called GNSS compass consisting of 2 GNSS receivers. With a distance between antennas of 1 m, it provides an error below $0.2$° [8] and is much less influenced by electromagnetic interference from high voltage equipment. The automatic flights are currently used to inspect overhead power lines [9], solar power plants [10], and even small power plants [11]. In all these cases, the drone flight is either around the inspection area or at the altitude above the highest obstacle. Inspecting large-scale industrial facilities, such as 2400 MW thermal power plants, requires flying inside the inspection area between many obstacles, whose height is often above the drone’s altitude. Moreover, in contrast to overhead line inspection, only a few areas are usually suitable for safe takeoff and landing. This task can be solved by multi-criteria mission planning, simultaneously considering obstacles’ position, radio communication availability, air turbulence, and flight time [5]. Still, the drone should be able to accurately and safely complete the planned missions.

The benefits of unmanned aerial inspection are undoubted. They have already been published in many research and review papers. This paper aims to summarize the challenges related to aerial inspection performed using market-available drones and connect them with specific solutions that will help overcome them or at least minimize related negative consequences. The review is specifically focused on market-ready products instead of research prototypes, because insurance companies avoid insuring the latter or provide insurance at much higher prices. At the same time, industrial facility owners usually strictly require insurance to secure UAV damage and third-party liability risk when performing aerial inspections.

The main contributions of this review are the following: (1) we summarized the pros and cons of aerial large-scale facility inspection, distinguishing it from other inspection scenarios implemented with drones; (2) we revealed specific issues related to modern commercial drone software and demonstrated that even the market leaders’ products suffer from them; (3) we proposed solutions to these issues, which helped us overcome them during the first autonomous inspection of a 2400 MW thermal power plant.

This review shows how drone developers can improve their products, making them more suitable for large-scale facility inspection. At the same time, it also demonstrates to researchers and inspection engineers what issues they will probably face while inspecting large-scale facilities using the industrial drones currently available on the market.

The rest of the paper is organized as follows. Section 2 describes the paper selection procedure. The analysis of the selected papers is provided in Section 3. Section 4 discusses the specific issues related to the market-available drone software and proposed solutions for each of them. The review results are discussed in Section 5. Findings and recommendations are summarized in Section 6.

2. Paper Selection

The paper selection procedure was performed using the Scopus database and the “inspection AND drone” prompt without any other limitations. The initial search resulted in 2025 papers published from 1985. The results included 729 research papers, 1168 conference papers and conference reviews, 65 reviews and short surveys, 49 books and book chapters, and 13 documents of other types, including erratum, notes, and retracted papers. Figure 2 demonstrates distribution of these papers relative to the year of their publication. It can be clearly seen that interest in inspection drones has constantly risen since 2014. Considering that the initial selection already includes 65 review papers and short surveys, most of which were published during the last ten years, it was decided to focus our analysis on these surveys. Review papers marked as books and book chapters in the Scopus database were also analyzed. We used two simple inclusion criteria: the paper should be dedicated to multi-rotor drones and inspection of local industrial facilities and buildings. The review is narrowed to multi-rotor systems because using other UAV types is significantly more complicated due to the large number of obstacles in large-scale industrial facilities.

Figure 2. Distribution of drone-based inspection papers by year of publication: (a) all papers; (b) journal research papers; (c) conference papers and conference reviews; (d) journal review papers; (e) books and book chapters; (f) other documents, including erratum, notes, and retracted papers.

The linear facilities, like overhead powerlines, pipelines, railways, and roads, were excluded because their monitoring requirements and complexity significantly differ from the inspection of large-scale industrial facilities [5]. Moreover, considering the long distances that should be traveled during such types of inspection, it is more promising to perform them by fixed-wing UAVs [12]. Three specific types of local facilities were also excluded for the same reason: agricultural fields, photovoltaic power plants, and bridges. Inspecting the first two types of facilities is significantly simpler than buildings, factories, or thermal power plants because it can be performed from a high altitude, making the problems of obstacles and interference negligible. Thus, it can be performed by any drone suitable for remote sensing [13].

On the contrary, the inspection of bridges is comparable and even sometimes more complicated than the inspection of the other types of local industrial facilities. At the same time, the set of problems in these cases differs from, e.g., power plant inspection. According to a technical report of the Minnesota Department of Transportation [14], one of the main challenges in bridge inspection is non-GPS navigation, which makes this case closer to indoor applications of drones. However, indoor inspection is out of the scope of this paper.

Another exclusion criterion is that the application area is different from industrial inspections. It was used to exclude papers on wild animal monitoring, military applications, counter-drone systems, and aerial object manipulation. Also, a set of criteria was used to focus our review on the capabilities of the drones as final market products. Thus, papers reviewing drone components and algorithms without relating them to any specific drones were also excluded. These papers were not used in the analysis of large-scale industrial facilities monitoring pros and cons, but some of them were further used to propose solutions to the revealed issues of the modern market-available UAVs.

Finally, we excluded papers focusing on the legal aspects of drone applications. Legal issues usually differ depending on the laws of specific countries, while our review aims to focus on engineering aspects of large-scale facility drone inspection.

The overall paper selection procedure is illustrated by the Figure 3. The initial selection of 2025 papers was narrowed to 114 by limiting the document types to review, short survey, book, and book chapter. Then, based on the chosen criteria and abstract analysis, another 75 papers were excluded from the list. Finally, after analyzing the remaining paper’s full texts, we excluded 19 documents. The final selection includes 18 papers published from 2016 to 2024 [1,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31], most of which were published in 2023. These papers were used to summarize the pros and cons of aerial large-scale facility inspection, distinguishing it from other inspection scenarios implemented with drones. The results of the performed analysis are presented in the next section.

Figure 3. Paper selection procedure.

3. The Analysis of the Selected Papers

The analysis of the selected papers revealed the following key advantages of aerial inspection performed with multi-rotor drones:

• Cost and speed: it is cheaper and faster than conventional inspection methods [1,17,21,26,27,29].
• Efficiency: it provides more meaningful results, which are easier to interpret [21]. Moreover, it allows the conducting of inspections more frequently, providing more accurate failure predictions [20,26,30].
• Safety: it provides a higher level of safety due to elimination or at least reduction of time and personnel spending in the areas with increased danger, such as open distribution devices or pipelines [15,25].
• Enhanced analysis capabilities: UAVs capture data from closer distances and in larger quantities with better repeatability. Thus, these data become suitable for analysis using neural networks and other machine-learning techniques [1,16,27,28]. Automated intelligent analysis speeds up the process and eliminates human factors, such as subjective opinions, providing better failure prediction accuracy.
• Non-stop operation of the industrial facility: in most cases, UAV inspection can be performed on active equipment without stopping production processes on industrial facilities [31]. Such capability indirectly affects the cost-efficiency of the aerial inspection by reducing losses caused by equipment downtime.

All the advantages can be summarized as better cost-efficiency, higher inspection quality, and better safety. The discussion section provides a critical analysis of them.

The main challenges of the aerial inspection on large-scale industrial facilities, according to the reviewed papers, are the following:

• Low battery life: modern multi-rotor drones’ maximum flight time rarely exceeds 1 h. In most cases, the actual flight time with payload is below 30 min. This challenge significantly increases the importance of optimal flight planning that minimizes the number of battery replacements required in order to complete all the inspection tasks [19,26,27,29].
• Limited payload is another challenge of aerial inspection with multi-rotor drones [15,26,29], which limits the amount of inspection equipment installed on a drone in each fight.
• Interference from surrounding objects in inspection data: surrounding objects may influence the diagnostic equipment of the drone. For example, they can create reflections that distort the images of thermal cameras. This challenge is mentioned in [1], where the authors suggest recording data from different fields of view as a solution. At the same time, it should be noted that this challenge is specific to the inspection of industrial facilities in general, not only to aerial inspection. Moreover, because multi-rotor drones can operate on short distances from the inspected equipment, the data they collect are less influenced than those obtained from the ground.
• High risk of collision due to many obstacles and a short distance to the inspected equipment during data acquisition [21].
• Weather limitations: all aerial vehicles have weather limitations, and UAVs are no exception. In some weather conditions, the inspection performed with the drone is not allowed due to safety reasons, while in other cases, such weather phenomena as air turbulence influence the inspection results [1,22,23,24,25].
• Operator competence: the importance of eliminating human errors and increasing the competence of UAV operators is declared in [18,22].
• Regulation issues: current regulations of airspace usage are not designed for frequent UAV usage, especially on industrial facilities, which are usually located in restricted areas with restricted airspaces, leading to a long legal process in order to gain flight permission [15,18,22,28].

After analysis of the above challenges of large-scale industrial facility inspection, it can be noted that all of them correspond to multi-rotor drones and industrial facility inspection in general. Thus, the reviewed surveys may create an illusion that that a complete set of the published technologies is already available on the market. The illusion is caused by the fact that, while preparing a review, researchers usually assume that if there is at least one paper that describes a solution to some problem, then the problem is solved, while the path from the research paper to a market product is rarely straightforward. At the same time, application engineers rarely use research prototypes in their solutions, preferring market-available products to provide end-customer service, predictable reliability, insurance availability, etc. To avoid such illusions in our review, we analyze issues related to modern commercial drone software in the next section. We show that all of them have a solution known from the state of the art, but in many cases, these solutions are not yet implemented in the market products.

4. The Issues Related to Modern Commercial Drone Software

It is relatively easy to summarize modern science and technology advances because they are the main focus of most research papers. It is much more difficult to reveal the drawbacks of market-available products because their manufacturers often try to mask them. Thus, in this part of our review, we mostly rely on the results of our own experimental studies performed on different drones. Whenever possible, we support our findings with results that can be found in the papers of other authors.

During the first autonomous inspection of a 2400 MW thermal power plant using a commercial UAV [5], we discovered multiple issues that were negligible for general remote sensing and monitoring applications but became critical in the case of large-scale industrial facility inspection. Further, we tested the market-available drones from other manufacturers and found that even the market leaders’ products suffer from the same problems, more or less. Thus, we understood that it is essential to focus the attention of manufacturers and application engineers on these issues. The results of our analysis are presented below.

The experimental studies reported in [5] were performed using a R.A.L. X6 hexacoper (RusAeroLab LLC, Moscow, Russia, https://rusaerolab.ru/products_ral_x6.html, accessed on 6 September 2024) (Figure 4). According to the datasheet provided by the manufacturer, this drone has following parameters: maximum flight distance—30 km; maximum reachable altitude—7 km; effective flight altitude—1 km; maximum flight time—1 h (no load)/45 min (1 kg load)/20 min (5 kg load); maximum wind gusts—25 m/s; temperature range—−30–60 °C; acceptable weather conditions—wind, snow, rain. The distance between opposite motors is 940 mm. The drone is equipped with Trimble MB-Two receivers that estimate position and heading. For this purpose, two GNSS antennas are mounted above the motors on opposite sides of the drone. Another Trimble MB-Two is connected to the ground station and serves as an RTK base station. The overall GNSS navigation system is similar to the one used on the mobile robot presented in [32].

Figure 4. R.A.L. X6 equipped with GIT UR-640D camera.

The drone and ground station communication is performed using three R.A.L. 01-002 mesh radio modems. One was installed on the ground station, the other was mounted on the roof of the main machinery building, and the third was manufactured in a light version integrated with the drone’s control system. According to their datasheet, these radio modems manage to download data from the drone with a 1.2–12  Mbps rate and upload with a data rate of 0.115–1  Mbps in a range of 20  km (line of sight). The radio modem mounted on the roof was used as a retransmitter to enlarge radio availability zones [5].

The R.A.L. X6 in the current research was equipped with a dual-specter RGB/IR camera (GIT UR-640D) mounted on a three-axis gyro-stabilized gimbal. This camera is connected to the drone’s control system with the Universal Asynchronous Receiver/Transmitter (UART) interface, making it possible to trigger image and video capturing. The camera has a resolution of $4608\times 3456$ and $640\times 480$ pixels for RGB and IR channels, respectively. The IR channel’s field of view is $32\times 24$°, which defines the effective image capturing distance in a range of 15–30 m.

The R.A.L X6 drone is based on proprietary software and hardware designed by its manufacturer. It is incompatible with open-source mission-planning software, like QGroundControl (http://qgroundcontrol.com/). Instead, it uses the proprietary RALTool application for configuration, mission planning, and control during the flight.

It is worth mentioning that the analysis performed before the research reported in [5] showed that the R.A.L. X6 should be suitable to perform missions on the reference power plant: it can fly in strong winds, supports automatic missions with gimbal and camera control commands, is equipped with precise GNSS RTK position and heading estimation hardware, and can communicate with the ground station through distributed mesh network, making it possible to maximize radio coverage in the power plant area. (Without radio coverage, the drone cannot receive GNSS RTK corrections from the base station, which are required for precise positioning). Moreover, it was successfully operated on a substation and used for overhead powerline inspection, which supported the manufacturer’s statement that the drone’s navigation system is resistant to electromagnetic interference from high-voltage equipment. At the same time, as mentioned above, the conditions in large-scale power plants are much more demanding than in the mentioned usage cases.

Figure 5 demonstrates other multi-rotor aerial systems discussed in this section. They include those produced by DJI and Autel Robotics and those based on ArduPilot, PX4, and INAV open autopilots.

Figure 5. Examples of multi-rotor aerial systems discussed in current research: (a) DJI Matrice 300 RTK with DJI Zenmuse L1 laser scanner [33]; (b) Autel Robotics EVO II Pro [34]; (c) Voljet M690 PRO drone, which is equipped with Pixhawk 2.1 ORANGE flight controller, compatible with ArduPilot/PX4 firmware [35]; (d) a 24″ heavy drone equipped with a controller running INAV [36].

DJI is a market leader [37] in both industrial and entertainment drones. Their drones are widely used in many industrial inspection applications [9,38,39]. The drones of the Mavic and Phantom series are usually operated manually with partial automation. In contrast, the Matrice series is often used automatically when a drone flies along the pre-programmed route. Also, for Matrice 30, DJI Mavic 3E, DJI Mavic 3T, Matrice 300 RTK, and Matrice 350 RTK drones, DJI provides advanced FlightHub 2 [40] mission planning and ground station software (FlightHub 2 2024-9-24 release version, SZ DJI Technology Co., Ltd., Shenzhen, China). It also provides DJI Fly software that is installed on DJI RC controllers (DJI Fly V 1.13.10 version, SZ DJI Technology Co., Ltd, Shenzhen, China).

Autel Robotics is another well-known manufacturer of industrial drones, whose products (especially EVO II, which was used in experimental studies, performed in current research, and its modifications) are also a popular choice for aerial inspection missions [38,41,42].

ArduPilot and PX4 are two well-known open-source firmware for multi-rotors [43]. They both are designed for the same hardware evolved from PixHawk flight controller [44]. Initially, this was designed for hobby projects. However, currently, CUAV and CubePilot produce industrial-grade controllers compatible with ArduPilot and PX4, which are used in many market-available drones manufactured by mid-size companies [45,46].

INAV Multicopter is one of the most recent open-source drone firmwares [47]. Having the same ancestor, it is close to BetaFlight but has improved navigation functionalities and ground station software supporting mission planning [48]. INAV firmware is widely used for fixed-wing and vertical takeoff and landing (VTOL) aircraft [49,50]. At the same time, it supports mulitcopters [51] and is used by several small companies around the world to manufacture low-cost thermal inspection industrial drones for local markets.

UgCS from SPH Engineering, a universal ground station software. In our research, we analyze its capabilities in addition to the described above multi-rotor systems because it is compatible with multiple drones from different vendors (DJI, Autel Robotics, ArduPilot/PX4-based, etc.) and widely used for industrial inspection [46,52,53,54].

The rest of the section is dedicated to the description of the revealed issues and corresponding solutions.

4.1. An RTK Shift

4.1.1. An Issue Description

GNSS RTK is widely used in industrial inspection to provide centimeter-range positioning precision. It requires a base receiver with a static antenna with known geographic coordinates to operate [55]. The receiver itself can determine the position of this antenna following a so-called “survey-in” procedure or a geodetic survey. The second approach is generally preferable because it provides the drone’s position in the same coordinate system used in construction plans and geo-referenced 3D models. Thus, it is possible to plan the routes offline using those plans and models, and then the drone will fly to the right points.

During the experimental studies, it was noticed that when the RTK base antenna position was defined using the geodetic survey results, the drone performed undesired maneuvers in case of radio connection loss. Specifically, it changed its altitude and shifted to the west by several meters. Then, when the connection was restored, the drone shifted back. Moreover, the maneuvers were very similar regardless of the place of communication failure.

We assumed that the absence of RTK correction caused the problem. At the same time, it needed to be clarified why the drone constantly shifts in the same direction and at the same distance. Everything became apparent after the position of the RTK base antenna was determined by a survey-in procedure. (The survey-in procedure was performed until 0.2  m accuracy was reached. The estimated position of the antenna lay approximately 2.5  m to the east relative to the coordinates provided by a geodetic engineer. We repeated the survey-in procedure multiple times in the following days, and all the results included similar additive errors  (Figure 6).

Figure 6. Position of the RTK base antenna determent by the receiver itself during survey-in procedures relative to the position provided by geodetic survey results: red circles—results of multiple survey-in procedures performed in different days; blue star—mean positions evaluated based on all the survey-in results.

This error describes the undesired behavior after connection loss: the drone stops receiving RTK corrections from the base receiver, and its reference system shifts 2.5 m, which causes movement aimed to compensate for this shift. Then, when communication is restored, the process repeats backward. At the same time, it is essential to understand the cause of the additive error to avoid it. Such a shift between geodetic and survey-in coordinates is hard to describe with known sources of the GNSS errors [56] as it stays close to constant during long periods and appears even if the survey-in is performed several minutes after the geodetic survey. The clue was found when the geodetic engineer was asked to complete a survey without receiving corrections from the local Networked Transport of RTCM via Internet Protocol (NTRIP) server: the difference from the receiver of the drone’s static base reduced to a centimeter level. Thus, the observed RTK shift was introduced by the NTRIP corrections during the geodetic survey.

As it recovered due to federal law, companies owning NTRIP stations in our country have to configure their coordinates with the ones defined using ellipsoids different from WGS84. The reason is that the results of geodetic surveys performed with these corrections should match existing geodetic plans. Due to historical reasons, in ex-USSR countries, geodetic plans were created in the SK-42 reference system based on Krasovsky 1940 ellipsoid. This ellipsoid is still actively used in many cartographic projections in Russia and ex-USSR countries. At the same time, most modern GNSS receivers use the WGS84 ellipsoid, which has different parameters from Krasovsky 1940. Thus, latitude and longitude parameters obtained using different ellipsoids should be appropriately converted into one coordinate system before use. This problem was mentioned multiple times in scientific papers on geography, geology, and geophysics [57,58,59,60]. Nevertheless, these papers were published in specialized journals, which drone or even geodetic engineers rarely read. Thus, there is no tool for RTK compensation in market-available inspection drones.

4.1.2. A Solution

The straightforward solution for the issue is configuring the RTK base location on the ground station using the survey-in procedure result or the geodetic survey performed without NTRIP corrections. However, in this case, the reference system of the drone will constantly shift relative to the existing construction plans and geo-referenced 3D models, making it impossible to use them while planning the routes. Meanwhile, the constant shift of the RTK base leads to the same shift of the rover position [61]. Thus, it can be compensated during the mission planning by adding difference between latitude and longitude determent by the receiver during the survey-in procedure and the ones provided as the result of the geodetic survey to the corresponding coordinates of each inspection route’s point. This approach is easy to implement for automatic mission generation but can be complicated during manual planning. Due to the RTK shift problem being relevant for most of the ex-USSR countries described above, we convinced the manufacturer to integrate the shift compensation into RALTool software. Thus, the mission can be planned according to standard geo-referenced models, and the additive corrections to the drone reference system are configured as two parameters on the ground station.

4.2. An Identification of Multiple Images Captured from the Same Point

4.2.1. An Issue Description

An industrial drone can take over a hundred images in a single flight. Moreover, these images often contain similar equipment that is difficult to distinguish from each other during post-processing (Figure 7). The base version of the R.A.L. X6 firmware was naming all the images sequentially. It was also able to store a timestamp (in seconds) and GNSS coordinates of the drone in the image metadata.

Figure 7. Similar IR images (raw) of different insulators captured during an inspection mission.

It was soon discovered that the camera may fail to make an image for different reasons, even if triggered. In such rare cases, the camera proceeded to name files from the identifier of the last successful capture, making it hard to match the planned and actually recorded data.

A typical task during the inspection of a large-scale facility is capturing many parts of equipment from a single point [5]. At the same time, as noted above, these equipment parts are very similar. Thus, GNSS metadata only partially solved the problem of robustly distinguishing images from each other. Finally, the drone takes multiple shots per second to minimize downtime during missions while its gimbal rotates at an angular speed of 0.7 rad/s. As a result, the 1-second accuracy is not sufficient to distinguish images of two closely aligned pieces of equipment from the multiple shots of a single object.

4.2.2. A Solution

The R.A.L. X6 drone had three previously reserved bytes in its camera triggering command. We suggested that the manufacturer use them to encode the unique identifier of the captured equipment and shooting angle among all the routes. This identifier is transferred to the camera and further used as a part of a filename. Thus, even if the missions were re-planned to fit new weather conditions [5], all the images will save their names, making it easier to process overall inspection results. This solution is currently available for the serial R.A.L. X6 drones delivered with a GIT UR-640D camera.

4.3. Limitations of Custom Mission Generation with External Tools

4.3.1. An Issue Description

The initial version of RALTool used the proprietary mission file format. It allowed the saving and loading of missions created using this software but did not provide any functionality to import sets of waypoints generated by external tools. At the same time, inspection planning heavily relies on digital models of the industrial facility, which cannot be imported into RALTool or many other mission planning programs. Also, general mission planning tools provided by drone manufacturers rarely provide extended route optimization functionality, which is required to minimize the number of batteries required to inspect all the equipment. Thus, the ability to load missions created using external application-specific software becomes critical for the inspection of complex industrial facilities.

4.3.2. A Solution

As a partial solution, the drone’s manufacturer provided us with the structure of the binary mission file used by RALTool (Table 1). Unfortunately, they decided not to make it open source and reserved the right to change it in the future, as well as for other versions of their drones. In the research presented in [5], this file format was used to generate the routes by mission planning and optimization software created by our team. These routes were then loaded into the drone and executed using RALTool.

Table 1. RALTool binary mission format.

Bytes	Preamble	Datatype
0–3	Fixed preamble	32-bit unsigned integer
4–7	Number of waypoints (N)	32-bit unsigned integer
$(8+40·(n-1\left)\right)$–$(8+40·(n-1)+3)$	Target latitude of n-th waypoint,	32-bit single float
$(8+40·(n-1)+4)$–$(8+40·(n-1)+7)$	Target longitude of n-th waypoint,	32-bit single float
$(8+40·(n-1)+8)$–$(8+40·(n-1)+11)$	Target altitude of n-th waypoint, m	32-bit single float
$(8+40·(n-1)+12)$	n-th waypoint radius, m	8-bit unsigned integer
$(8+40·(n-1)+13)$	Reserved field	8-bit unsigned integer
$(8+40·(n-1)+14)$–$(8+40·(n-1)+15)$	Identifier of the image captured in the n-th waypoint (added on request of the authors)	16-bit unsigned integer
$(8+40·(n-1)+16)$–$(8+40·(n-1)+19)$	Delay in the n-th waypoint, ms	32-bit unsigned integer
$(8+40·(n-1)+20)$	Maximum horizontal speed used to reach n-th waypoint, 0.1 m/s	8-bit unsigned integer
$(8+40·(n-1)+21)$	Maximum vertical speed used to reach n-th waypoint, 0.1 m/s	8-bit unsigned integer
$(8+40·(n-1)+22)$	RTK shift latitude correction specific for n-th waypoint, cm	8-bit signed integer
$(8+40·(n-1)+23)$	RTK shift longitude correction specific for n-th waypoint, cm	8-bit signed integer
$(8+40·(n-1)+24)$–$(8+40·(n-1)+27)$	Reserved field	32-bit unsigned integer
$(8+40·(n-1)+28)$–$(8+40·(n-1)+31)$	Target heading angle at n-th waypoint,	32-bit single float
$(8+40·(n-1)+32)$–$(8+40·(n-1)+33)$	Target gimbal pitch angle at n-th waypoint,	16-bit signed integer
$(8+40·(n-1)+34)$–$(8+40·(n-1)+35)$	Target gimbal yaw angle at n-th waypoint,	16-bit signed integer
$(8+40·(n-1)+36)$–$(8+40·(n-1)+37)$	Waypoint index	16-bit unsigned integer
$(8+40·(n-1)+38)$	additional actions at n-th waypoint (e.g., capture an image or start/stop a video)	8-bit unsigned integer
$(8+40·(n-1)+39)$	n-th waypoint command type (turn on/off motors, land, fly, etc.)	8-bit unsigned integer
$\dots$	Other waypoints	$\dots$
$(8+40·(N\left)\right)$–$(8+40·(N\left)\right)+71$	Pre-defined end sequence	72-byte unsigned integer array

As it is seen, all commands in the mission file (Table 1) have the same format, which includes spatial position, heading angle, camera targeting parameters, image identification, and commands to start capturing. Such a unified approach is more beneficial for inspecting complex industrial facilities in terms of fitting into waypoint limits because such missions have much more camera targeting and capturing commands than general waypoints. For example, the popular ArduPilot firmware uses three separate commands to move the drone, target the camera using a gimbal, and initiate image capturing. Thus, despite ArduPilot controllers supporting up to 1312 commands in one mission [62], the maximal number of the inspected parts of the equipment will be smaller than by R.A.L. X6, the missions of which can include up to 1000 commands.

4.4. An Incorrect Flight Time Prediction

4.4.1. An Issue Description

Our experimental studies discovered that a flight time predicted by RALTool software can be more than 1.5 times shorter than an actual flight time. Moreover, the estimations provided by popular UgCS (https://www.ugcs.com/) universal ground station software were even more incorrect [5]. As we assume, the main reason is that the models used in these software products to predict flight duration do not include the time required for heading change and gimbal rotation in their estimations, considering it incomparable to overall flight time. At the same time, while inspecting large-scale industrial facilities, the drone spends much time at each waypoint, rotating and targeting multiple parts of equipment with its camera.

4.4.2. A Solution

We proposed a new, more accurate model to predict flight time during industrial inspection missions (1).

$$T_f=\sum _{k=1}^K(\max (T_{h_k},T_{v_k},T_{g_k})+T_{y_k}+T_{d_k}),$$

(1)

where $T_f$—predicted flight duration, K—the number of waypoints in the route; $T_{h_k}$—the time required to perform horizontal flight from the $k-1$ to the k-th point; $T_{v_k}$—the time required to perform vertical flight from the $k-1$ to the k-th point; $T_{g_k}$—the time required to rotate the gimbal to the orientation corresponding to the k-th point from the one it was in the $k-1$-th point; $T_{y_k}$—the time required to change the drone’s heading to the direction corresponding to the k-th point from the one in the $k-1$-th point; $T_{d_k}$—the time delay configured for k-th point of the route.

$T_{h_k}$, $T_{v_k}$, $T_{g_k}$, and $T_{y_k}$ are evaluated using the $f_r$ function defined in (2). Generally, it evaluates the time required to change the value of the drone’s corresponding parameter d following a trapezoidal ramp with v maximum speed setpoint, $a_a$ acceleration, and $a_d$ deceleration desired rates.

$$\begin{array}{c}{f}_r(d,v,a_a,a_d)=\hfill \\ \left\{\begin{array}{cc}{\displaystyle \frac{d}{v}}+{\displaystyle \frac{v}{2a_a}}+{\displaystyle \frac{v}{2a_d}},\hfill & d-{\displaystyle \frac{v^2}{2a_a}}-{\displaystyle \frac{v^2}{2a_d}}>0\hfill \\ \sqrt{{\displaystyle \frac{2a_a{a}_{d}d}{a_a^2(a_a+a_d)}}}+\sqrt{{\displaystyle \frac{2a_a{a}_{d}d}{a_d^2(a_a+a_d)}}},\hfill & d-{\displaystyle \frac{v^2}{2a_a}}-{\displaystyle \frac{v^2}{2a_d}}\le 0\hfill \end{array}\right.\hfill \end{array}$$

(2)

The distribution between the time estimates used as arguments of maximum function in (1) and those placed outside represents the behavior of the R.A.L. X6 drone, which first reaches the desired heading and only then performs the flight and gimbal orientation. At the same time, it can be easily adapted for another drone. For example, if the drone changes heading simultaneously with the flight to the next waypoint, $T_{y_k}$ should be used as another argument for the maximum function.

The inspection flights performed on the reference power plant showed that the proposed model provides a forecasting error of less than 10% in most cases and significantly outperforms those implemented in commercial mission planning software. A detailed description of this model and experimental studies demonstrating its accuracy are presented in [5]. According to the drone’s manufacturer statements, the proposed flight time estimation model will be used in future versions of RALTool software.

4.5. An Unpredictable Time of Reaching Waypoint with a Small Radius

4.5.1. An Issue Description

Multiple inspection missions performed in strong wind conditions showed that inspection time could vary in a wide range and significantly deviate from the flight-time prediction model described above. These deviations are caused by the unpredictably long time the drone requires to reach waypoints with a relatively small radius. This issue most often occurs at waypoints in so-called Venturi zones near the edges of the buildings where air speed significantly rises and the air becomes turbulent.

Figure 8 demonstrated the distance to a waypoint with a 2 m radius during capturing of 12 photos in moderate (6 m/s, Figure 8a) and strong (11 m/s, Figure 8b) wind. The process takes around 120 s in moderate conditions, while the overall duration rises to 319 s in strong wind. As seen from Figure 8, additional time is required to return the drone inside the waypoint’s radius. It should be noticed that gimbal orientation and camera trigger commands in the R.A.L. X6 mission also have waypoint position and radius parameters, forcing the drone to reach the desired location within the radius before their execution.

Figure 8. The distance to a waypoint with 2 m radius during capturing of 12 photos in (a) moderated (6 m/s) and (b) strong (11 m/s) wind. The dashed lines represent the moment the drone’s camera captured the frame.

The above issue may cause a situation in which the drone’s battery capacity is not enough to complete the mission and safely return to the landing area.

4.5.2. A Solution

The issue is generally caused by the absence of the integral action in the drone’s position controller. The proportional and proportional-derivative controllers have a static error, which raises within the rise of the disturbance. Thus, in strong wind, the drone quickly reaches a 4 m radius, requires a lot of time to reach a 2 m zone, and may never get into a 1 m circle around the waypoint coordinates. Multiple papers propose the position controllers for multi-rotor drones, including integral action [63,64,65]. At the same time, tuning such controllers to guarantee overall control loop stability is a complex task. As a result, the proportional position controller remains one of the most popular solutions [66], widely used in many industrial drones, including the ones based on ArduPilot and PX4 firmware.

As a partial solution, [5] proposes to use a local turbulence map to avoid Venturi and air turbulence zones during mission planning, where the issue was most commonly observed. At the same time, this is inadequate from a safety point of view.

The significant changes in the position controller’s structure required much additional testing from the drone’s manufacturer. Considering that the R.A.L. X6 faced the issue of reaching small-radius waypoints only in strong winds (unsuitable for many other drones on the market), we proposed a simple, safety-oriented solution. The drone firmware was programmed to monitor an additional safety-timeout radius $r_{st}$ around each waypoint (Figure 9). This radius should be big enough so the drone could reach it in any acceptable wind conditions in the inspection area and, at the same time, small enough so that after reaching it, the drone can proceed with the mission to the next waypoint without hitting any obstacle. Generally, $r_{st}$ should be lower than $r_s$ safety distances defined in [5].

Figure 9. The timeout radius is larger than the waypoint radius, and if the drone stays inside of it for a certain amount of time without entering the waypoint radius, the point is skipped.

The proposed safety-related algorithm (Algorithm 1) is described below. The safety timer starts when the drones reach the $r_{st}$ radius. If the drone cannot reach the programmed waypoint radius before the timer exceeds $T_{st}$ safety timeout, the mission will switch to the next waypoint, which is distant from the current waypoint, at least on $r_{st}$. The last condition is used to skip all the commands related to multiple camera shots performed from one spatial area, assuming they all suffer from the same issue. In our research, $r_{st}$ was configured as 4 m and $T_{st}$ as 10 s.

Algorithm 1 Proposed safety-related algorithm

1: Start   2: while True do   3:     if $r_{st}$ is reached then   4:         Start timer   5:         while True do   6:            if $T_{st}$ is reached then   7:                Skip Waypoints closer than $r_{st}$   8:                Exit   9:            else if Waypoint is reached then 10:                Exit 11:            end if 12:         end while 13:     else 14:         Continue checking if $r_{st}$ is reached 15:     end if 16: end while

The proposed solution is based on the assumption that, from a safety point of view, it is better to skip the waypoints whose radius cannot be reached in a reasonable time, and proceed with the mission instead of wasting energy and risking ending it with an accident. It is currently available in the firmware of the serial R.A.L. X6 drones.

4.6. Deviation from the Pre-Planned Route Line between Two Waypoints

4.6.1. An Issue Description

Another issue found during the initial stages of aerial inspection with the R.A.L. X6 was that the drone had high deviations from the route line if the distance between waypoints was more than 50 m. Figure 10 shows the planned and actual path of the drone along the $\Pi$-shaped flight profile with the distance between waypoints equal to 782 m.

Figure 10. The planned (blue) and actual (red) path of the drone along the $\Pi$-shaped flight profile with a distance between waypoints equal to 782 m.

As seen, the deviation between the desired and actual locations of the drone was up to several dozen meters, which is unacceptable in terms of obstacle avoidance and flight safety in general. Such issues are rare in modern drones but can still be met in the ones based on INAV firmware [67].

4.6.2. A Solution

The absence of an interpolator in the trajectory planner of the initial R.A.L. X6 firmware caused the issue. The next waypoint coordinates were directly loaded into the position controller each time the drone reached the previous one. When any external factor shifted the drone (e.g., strong wind), it was trying to reach the waypoint following the shortest way in the direction of waypoint (Figure 11a) instead of following the initial line between the previous and current waypoints. Thus, the drone significantly shifted from the route line, as was observed in the flight demonstrated in Figure 10. The issue became more significant with the increase in distance between waypoints. The deviation is in direct ratio with the distance between waypoints so that it may grow from a couple of meters on a short distance to dozens of meters.

Figure 11. Drone flight to the waypoint: (a) following the shortest path to the waypoint; (b) using the carrot-chasing algorithm.

As a solution requiring minimal changes to the R.A.L X6 firmware, we proposed implementing the carrot-chasing algorithm [68]. This algorithm assumes that the drone’s position controller is dynamically loaded with a virtual waypoint, which follows the straight line between the consequent waypoints in the mission with the desired speed (Figure 11a). Thus, the drone “chases” the virtual waypoint and returns to the route line even in case of any deviations.

The “carrot-chasing” algorithm is well-known. The version of it described in [68] is implemented by two modules. The first one is the Path Generator. It interpolates a set of waypoints into the path, consisting of a more dense list of waypoints, which can then be approximated into a curve. The second module is the Path Follower. It receives the lookahead distance, desired velocity, and interpolated set of waypoints. The algorithm calculates the virtual target position that the drone is trying to reach. The Path Follower algorithm is straightforward and consists of 3 stages.

• At the first step, the argument ${\lambda }_p\left(t\right)$ is found as ${\lambda }_p\left(t\right)=arg{min}_{\lambda \in [0,L]}\left|\mathbf{p}\left(t\right)-\gamma \left(\lambda \right)\right|$, where $\lambda$—set of waypoints, $\gamma$—waypoint position, $p\left(t\right)$—drone’s position, ${\lambda }_p\left(t\right)$—closest waypoint to the current drone’s position.
• At the second step, the lookahead distance is added, and the virtual target pose is found as $p_t\left(t\right)=\gamma ({\lambda }_p\left(t\right)+d)$.
• A drone is navigated to the virtual target position.

This issue was discovered during our first flights and reported to the manufacturer. The suggested solution was rapidly implemented and integrated into the mainline of the R.A.L. X6 firmware. Thus, all the flights reported in [5] were performed using the abovementioned modification.

4.7. A High Pitch Angle during Acceleration/Deceleration

4.7.1. An Issue Description

The R.A.L. X6, being a drone designed to fly in strong winds, can perform pitch and roll movements with high amplitude and acceleration to compensate for air turbulence. Initially, the same maneuvers were used to perform accelerations and decelerations while flying along the route.

Figure 12 shows the transient processes of pitch/roll angles recorded in one of the inspection missions performed in moderate wind (5–7 m/s) conditions. The demonstrated part of the mission included the inspection of 6 pylons on the roof of the power plant, which were divided into groups of 2 and 4 entities. The distance between pylons in each group was 12 m, and the distance between the groups was approximately 55 m. The small peaks on the horizontal speed plot (Figure 12) represent the transitions between the pylons in one group, the high peak represents the transition between the groups, and the gaps when speed was close to zero represent the periods when the drone was targeting camera and captured the inspected equipment.

Figure 12. The horizontal speed and pitch/roll angles during the part of the inspection flight in moderate wind conditions.

As seen from Figure 12, to perform accelerations/decelerations before/after the waypoints, the drone’s pitch and roll angles reached and even exceeded 30°. Such amplitudes can be considered high for a drone with a 0.94 m frame. If the drone encounters high air turbulence during such accelerations/decelerations, it may reach critical roll/pitch angles, resulting in undesirable maneuvers accompanied by significant loss of altitude. Moreover, during such maneuvers, the drone will suffer from significant overloads, which may result in motor overheating and damage to mechanical components. Thus, from a safety point of view, high roll/pitch angles should be used only to compensate for external disturbances, but not to perform rapid acceleration/deceleration in waypoints of the inspection mission.

4.7.2. A Solution

According to our recommendation, the virtual waypoint in a carrot-chasing algorithm was programmed to accelerate/decelerate with a pre-configured rate significantly lower than the maximum values defined by the drone’s capabilities. In terms of control theory, this solution is similar to adding a second degree of freedom to the position controller [69], with the only difference being that less dynamic reaction on setpoint is provided by reduced acceleration rates in the path planner instead of introducing an additional reference controller to the design.

It is worth mentioning that lowering acceleration/deceleration rates does not significantly influence the overall flight duration, as the acceleration/deceleration phase is much shorter then the time required for long transitions, camera targeting, and image capturing. The proposed solution is currently integrated into the mainline version of the R.A.L. X6 firmware.

4.8. An Automatic Landing Cancellation in a Strong Wind

4.8.1. An Issue Description

Many research papers [70,71] are focused on different equipment and techniques to perform safe automatic landings. Moreover, most commercial drone platforms have in-built functions for automatic landing and take-off [72], and the R.A.L. X6 is no exception. Being different in complexity and relying on various sensors, most of those solutions focus on landing at a specific point with minimum tolerance. As a result, while landing in a strong wind gust, even at a low altitude above the ground, the drone starts to compensate for displacement, performing quite aggressive roll and pitch movements. These maneuvers create a significant risk of hitting the ground or landing zone infrastructure (RTK and communication antennas, lights, etc.). We observed such behavior multiple times, with the R.A.L. X6 during the power plant inspection reported in [5] and with drones based on ArduPilot and DJI flight controllers in other projects. To save the drone from a crash, the operator may cancel the landing sequence and wait until the wind gusts are over. At the same time, after the cancellation, the drone proceeds hovering around its last position. This low-altitude hovering has similar risks of hitting the ground, so many professional operators decide to land the drone manually in strong wind conditions.

Compared to many other drones, the R.A.L. X6 is often delivered without manual remote control, assuming it will be used only for automatic flying along the pre-programmed missions. Thus, the landing cancellation sequence becomes much more critical as it is the only reasonable action the operator can initiate if something goes wrong. In the initial version of the firmware, which was pre-loaded at the time of delivery, the stop command executed during the landing, similarly to many other commercial multicopters, switched the drone’s controller to the loiter mode. Moreover, it was soon revealed that if the stop command was received at the exact moment when the drone touched the ground, the controller proceeded to spin the motors without the ability to take off or proceed with the landing procedure.

4.8.2. A Solution

The issue was reported to the drone manufacturer with the recommendation to start climbing automatically to 5 m higher in case of landing cancellation. The reason for this recommendation was to give the drone enough volume to safely perform wind compensation maneuvers without the risk of hitting the ground. Hovering at a safe altitude gives the operator additional time to make the right decision: to repeat the landing sequence or change the landing zone to one more appropriate for the current weather conditions.

The manufacturer both fixed the bug related to the landing abort when touching the ground and implemented our recommendation to climb 5 m higher if a stop command was received during a landing procedure.

Another recommendation was to limit the roll and pitch angles during the landing procedure, assuming that in most cases during inspection missions, it is better to be blown by the wind and land several meters from the desired location than to hit the ground during the aggressive compensation maneuver. At the same time, following this recommendation required additional analysis of the influence of the roll/pitch limitation during the landing on the drone’s controller stability. Thus, it has yet to be implemented but has already been added to the manufacturer’s backlog.

4.9. Flight Monitoring Issues Related to Ground Station Software

4.9.1. An Issue Description

The last set of issues revealed during our experimental studies is dedicated to the user experience of the operator controlling the drone during the inspection mission. Many commercial products (e.g., DJI industrial drones) include remote control, which must be used to take over the drone in case of any abnormal situation during autonomous missions. Generally, the operator performs manual control using the drone’s cameras or just looking at the drone when it is in sight. This is why, in these products, the image from the camera takes the central part of the operator’s screen.

In the case of inspections of large industrial facilities, this approach does not provide operators with comfort or flight safety. Let us discuss an example of a power plant inspection reported in [5]. Due to the lack of suitable landing areas at the power plant, the distance between the operator and the drone most of the flight time ranged from 300 m to 850 m, making it impossible for the operator to estimate the drone’s orientation by his own eyes, even when the drone was not closed off by the buildings. At the same time, FPV piloting in a power plant is significantly more complicated because the inspection drone has to fly between the wires and buildings at distances below 20 m. Moreover, to safely control the drone in FPV mode, the last one should be equipped with a low-latency wide-angle camera in addition to an inspection one. Without performing an FPV flight, the only safe maneuver the operator can perform in an emergency is to gain altitude above all the buildings and fly back to the landing zone.

The complexity of drone manual control conditions in large industrial facilities critically influences the ground station interface requirements. The operator is no longer a pilot who flies the drone using the remote control. The main task of the inspection drone operator is to identify the potential accident and initiate returning to landing as soon as possible. The main dangerous situations that should be monitored are the following: (1) not enough battery to finish the mission; (2) hardware malfunctions; (3) significant change in weather conditions; (4) significant deviations from the pre-planned mission route.

The first two dangerous situations are controlled automatically by the R.A.L. X6 firmware and reported to the operator by sound notifications. Moreover, the drone can be programmed to initiate return-to-home sequences in case of them automatically. The change in the weather conditions is generally controlled by the operator and his assistants (if present), but the R.A.L. X6 ground station software can show the measurements of the remote meteo station on the main screen (Figure 13). The main dangerous situation the operator should visually control during the flight is the last one. Significant deviations from the pre-planned mission route can be caused by local wind gusts, air turbulence, navigation errors, mechanical damage to the drone parts, etc. The earlier the operator notices the deviations, the more chance the drone will be able to return to the landing safely. Thus, during the inspection flight, the operator is focused on the drone and a zone on a map around it (Figure 13, zone fenced with a blue line).

Figure 13. The main screen of the R.A.L X6 ground station software (RALTool). The main parameters received from the drone’s telemetry are fenced red. The widget corresponding to the meteo station data is marked with the green fence (the meteo station connection was not established at the moment of the screenshot). The blue line fences the drone position and a virtual waypoint on the map.

As seen from Figure 13, RALTool ground station software demonstrated the drone position on the map (yellow circle), the line of the route (yellow line), the virtual waypoint (white circle with the red crosshair), the horizontal distance from the drone to the virtual waypoint (green distance label), and the information about the drone position and altitude (yellow label beneath the drone circle).

The first ground station-related issue of the R.A.L. X6 is that in the initial version of RALTool software, there was no option to hide the yellow text with the drone’s coordinates. The operator cannot analyze these data in any way during the flight. At the same time, this text often overlaps the label with the distance to the virtual waypoint (Figure 14), which is essential for detecting the situations when the drone significantly deviates from the route.

Figure 14. Examples of the situations when distance is (a) non-overlapped and (b) overlapped by the drone’s coordinates in the RALTool interface.

The second issue is that RALTool initially did not provide any information about the vertical distance between the drone and the virtual waypoint. Thus, the only way for the operator to notice any vertical deviation on the three-dimensional inspection mission is to analyze the current altitude of the drone and compare it with the next waypoint’s altitude. This comparison process significantly overloads the operator on routes with dozens of waypoints. Finally, in case the yellow label with the drone’s coordinates is hidden, the operator will have to look at the altitude on the sidebar with other telemetry parameters (Figure 13, zone fenced with a red line), which is generally far from its primary focus of view. Thus, it was almost impossible to analyze vertical deviations from the route during the flight using the initial version of RALTool.

The third, but not the least important, issue is that in the ground station software delivered with the R.A.L. X6 drone, a mouse click on the map near the waypoint could move it. Moreover, such route modifications were automatically transferred to the drone during the flight without any additional confirmation. Thus, when the operator tried to pan the map, it could lead to an accident because the modified route went through the obstacle. The constant risk of such a critical mistake undoubtedly was a source of significant stress on the operator.

4.9.2. A Solution

All the issues were reported to the R.A.L. X6 manufacturer and were fixed in the following way: (1) an additional button was added to lock mission modification by any mouse clicks on the map (Figure 15a); (2) the horizontal distance to the virtual waypoint was accompanied with the vertical one (Figure 15b); (3) a checkbox was added in the RALTool main screen configuration to hide the drone’s coordinates (Figure 15b).

Figure 15. Modifications introduced into RALTool to overcome flight monitoring issues related to ground station software.

4.10. The Analysis of Other Market-Available Solutions

Table 2 demonstrates the presence of the revealed issues in different market available solutions. The table was compiled based on an analysis of scientific papers, drone manuals, and datasheets, as well as the results of experiments on real products. The first two columns represent the initial version of the R.A.L. X6 drone before our research and the one improved by the manufacturer according to our recommendations. The rest of the columns are dedicated to widespread market-available solutions. The details of their analysis are presented in the following subsections. Before discussing these solutions in detail, it is worth mentioning that, to the best of our knowledge, none of them provide any in-built solution for the RTK shift issue. Most likely, the absence of such functionality is caused by the widespread use of WGS84 worldwide, while the issue becomes critical only for countries that have historically used another reference system for a long time.

Table 2. The presence of the revealed issues in different market-available solutions *.

Issue	Initial R.A.L. X6	Improved R.A.L. X6	DJI	Autel Robotics	Ardu- Copter/ PX4	INAV	UgCS
An RTK shift	x	✓	x	x	x	x	x
An identification of multiple images captured from the same point	x	✓	✓	x	x	x	x
Limitations of custom mission generation with external tools and mission length	x	x/✓	✓	x/✓	x/✓	x	✓
An incorrect flight time prediction	x	✓	✓	x	x	n/a	x
An unpredictable time of reaching waypoint with a small radius	x	✓	n/a	n/a	x	x	n/a
Deviation from the route line	x	✓	✓	✓	✓	x	n/a
A high pitch angle during acceleration/deceleration	x	✓	x	x	✓	✓	x
An automatic landing cancellation in a strong wind	x	✓	✓	✓	x	x	n/a
Uploading custom missions	✓	✓	✓	x/✓	x/✓	x	✓
Flight monitoring issues related to ground station software	x	✓	✓	x	x	x	✓

* x—the issue is present; ✓—the issue is not observed (overcome); x/✓—the issue is generally resolved, but with some limitations; n/a—the issue is not applicable.

4.10.1. DJI

DJI drones, the leading solutions on the market, generally do not suffer from most of the issues we have faced with the R.A.L. X6. For example, the full name of each picture taken can be programmed during the mission planning through FlightHub 2, and the landing cancellation results in automatic altitude control by the obstacle avoidance sensors. Moreover, modern DJI drones have trajectory controller, which allows them to follow the exact pre-planned trajectory. Finally, in the older versions of DJI ground station software, such as DJI GS PRO, the flight time prediction algorithm was inaccurate and did not take into account hovering time in a waypoint, but in newer versions of DJI software, this drawback was eliminated, and now time accuracy of flight time prediction is 5–8%.

Finally, the FlightHub 2 software allows online control of the drone’s position in 3D anywhere on the route, and during mission planning, estimated flight time matches the real one with a tolerance within a few dozen seconds.

The main drawback of the DJI software is that neither FlightHub 2 nor DJI Fly software allow manually changing acceleration and deceleration at waypoints (Figure 16). Both of these software packages allow the change in only velocity along the trajectory. Also, there is no possibility of changing the waypoint radius, which makes the accuracy of the waypoint following unclear, especially when the drone is affected by strong wind or turbulence. At the same time, the targeting of narrow-angle inspection cameras is heavily influenced by any positioning errors [62].

Figure 16. DJI Fly waypoint configuration screen. The software provides the possibility to change only the waypoint speed (red fence), but not accelerations or waypoint radius.

For a long time, the mission planning capabilities of DJI drones were limited. For example, in GS PRO, the maximum number of waypoints in a single mission was 99, which is insufficient for large-scale industrial facility inspection. However, this number is significantly increased for modern DJI drones, reaching 950 waypoints on the Matrice 350 drone. However, even older versions of DJI software supported the import of Keyhole Markup Language (KML) files [73] as a template for the drone’s mission, making it easy to generate missions from external geographic information systems. At the same time, e.g., for the Matrice 100 drone, it was still impossible to define such parameters as flight speed or trigger camera using KML tags. Thus, after the KML was imported, a significant amount of work was required to turn it into the drone’s mission, especially in the large-scale inspection tasks, which include a lot of commands related to gimbal control in image capturing. The situation changed after DJI introduced WayPoint Markup Language (WPML)—an extension to KML designed to fully describe the drone’s mission, including heading, gimbal orientation, and camera parameters. Currently, WPML is actively used by researchers to create a mission with external planning software (e.g., [74]).

4.10.2. Autel Robotics

Two mainly used Autel software solutions for mission planning are cloud-based Autel SkyCommand Center and Autel Explorer, installed on Autel Start Controllers. Their maturity is very high, similar to DJI products, but even they have some issues previously revealed in our research for the R.A.L X6.

First of all, similarly to DJI, neither in Autel Explorer nor in Autel SkyCommand Center is it possible to introduce RTK shift and change acceleration and deceleration along the trajectory.

Second, in both of these Autel software packages, it is impossible to pre-determine the names of the pictures taken in the trajectory waypoints. The only way to distinguish the captured data is to use Exchangeable Image File Format (EFIX) data from the picture with GNSS coordinates. However, this approach is not a solution when multiple images are captured from a single waypoint.

Third, Autel software makes inaccurate flight time predictions. After six consecutive flights, the average difference between predicted and real flight time was 30%, which is significant, especially in comparison with modern DJI software flight time estimation or the model proposed in [5].

Finally, in Autel Explorer software, there is no possibility to change the accelerations of the drone or inclination angles (Figure 17). These parameters are set automatically by the autopilot software without considering local external factors, such as weather conditions.

Figure 17. Autel Explorer waypoint configuration screen. As can be seen, it is only possible to change the velocity of the drone in the point (red fence), but not accelerations of inclination angles.

Similarly to DJI, modern Autel drones support WMPL, which allows the import of missions created by external tools. The maximum number of waypoints, which can be loaded into the drone using Autel software development kit (SDK), is 500, enough for many inspection tasks. At the same time, for large facilities, it can lead to dividing missions into smaller ones to fit the limit, which will require more time to complete the inspection. However, if the mission is loaded using Autel Explorer without SDK, the maximum number of waypoints will be reduced to 99, which can be considered as a significant limitation.

4.10.3. ArduPilot/PX4

ArduPilot and PX4 firmware differ in terms of underlying real-time operation and software architecture. However, they implement very similar functionality, controller structure, interface with ground stations using MAVLink protocol, and are compatible with both Mission Planner and QGroundControl mission planning applications [75]. Finally, they share the same issues (Table 2).

Compared to the R.A.L. X6, ArduPilot/PX4-based drones follow the line of the route in a strong wind and can be configured to use acceleration/deceleration rates below the limits while flying between waypoints. At the same time, they suffer from the rest of the issues discussed in this paper. Their mission cannot store any specific identifiers for captured images or videos. The Mission Planner software does not provide any tools for flight time estimation. At the same time, according to our experiments, QGroundControl does not consider the time required for gimbal rotations, altitude, and heading changes, which makes its estimations inaccurate for typical inspection missions. Due to using a proportional position controller [66], ArduPilot and PX4 firmware have unpredictable times of reaching waypoints with a small radius in strong wind conditions. At the same time, in deference to the R.A.L. X6, their ground station software has a special command that allows them to skip part of the route and proceed with the mission from a specific waypoint, which can be considered as a partial solution for situations when the operator has a robust communication channel with the drone during all of the flight. According to our experiments and the description of the landing procedures for ArduPilot [76] and PX4 [77] (including the advanced ones), this firmware suffers from the same issue in case of landing cancellation as the initial version of the R.A.L. X6.

Finally, both Mission Planner and QGroundControl applications support only 2D demonstration of the mission plan. Thus, they do not provide information for the operator to monitor 3D deviations between the drone and the planned route.

Because ArduPilot and PX4 are open source firmware, their waypoint file format is well-known. At the same time, these firmwares have non-obvious limitations regarding waypoint numbers. Generally, a modern ArduPilot/PX4-compatible controller supports up to around 1300 commands in a single mission [62]. This number is comparable to waypoint limits of the R.A.L. X6 or enterprise DJI drones. At the same time, in deference to these drones, ArduPilot/PX4 firmware includes all waypoints, gimbal, and camera control commands in this limit. Thus, the maximum number of actual waypoints becomes comparable to or even smaller than that of Autel drones.

4.10.4. INAV

INAV drones are the least advanced compared to the other analyzed aerial systems (Table 2).

The most critical drawback of INAV firmware is the deviation from the planned route line, similar to the one revealed in the R.A.L. X6. As it can be seen from Figure 18, the actual trajectory (red) deviated from the planned (blue) line along all routes. Such behavior is caused by the INAV Multicopter navigation algorithm, which does not follow the line between two waypoints. Instead, it constantly recalculates bearing to the next waypoint and uses it for navigation.

Figure 18. The planned (blue) and actual (red) path of the drone, controlled by INAV. As can be seen, the drone deviates from the planned route line.

The next drawback is the inability to choose the name of the captured image. The INAV cannot control any camera parameters. Instead, it just sends the shutter trigger signal through the servo output. Thus, the naming of the captured pictures is determined only by the camera being used. However, this camera cannot obtain any information about the inspection mission from the flight controller.

Since INAV Multicopter uses a proportional controller for position control, it has the problem of reaching the waypoint with a small radius, similar to the R.A.L. X6 and ArduPilot/PX4. Also, in the case of landing abortion, the drone stays at the same height on which the landing was canceled, which may lead to undesired contact with the ground during the wind compensation movements if the altitude is small enough. However, the INAV Multicopter supports using a sonar or an optical flow sensor, which allows setting the minimum altitude above the terrain. This altitude will be gained if the landing is canceled.

The INAV has several issues relative to flight monitoring ergonomics. The main drawback is that INAV is designed to be monitored using On Screen Display (OSD) and FPV goggles. Thus, the INAV ground station lacks visual information about the current state of the drone and is mainly used for setting the drone up and mission uploading. The mission control screen shows only a limited amount of information about the drone’s behavior, for example, the drone’s position, waypoints, heading, ground speed, and altitude. These data are marked with a red fence on Figure 19. Also, the operator can see (Figure 19 the position of waypoints and the drone itself (blue fence) on a 2D map, and the mission height profile is marked with a green fence (Figure 19). Still, as in the initial version of RALTool, it is not enough to provide flight safety and operator comfort during an inspection of a large-scale industrial facility. There is not even information about battery usage, making the ground station unusable for constant flight monitoring without FPV goggles or a screen, where this information is shown via OSD.

Figure 19. INAV ground station mission control tab.

Finally, INAV has a limit of only 120 commands for most of the supported flight controllers. Similar to ArduPilot/PX4, this limit is shared by waypoints and other actions. Moreover, the mission file format lacks commands to control gimbal or camera behavior. Thus, INAV functionality is minimal in terms of automated large-scale industrial facility inspection requirements.

4.10.5. UgCS

Due to UgCS being only ground station firmware, all the issues related to the drone’s firmware are generally not applicable to it (Table 2).

UgCS provides an advanced 3D dimensional interface for mission planning and flight monitoring (Figure 20). Moreover, it imports missions in the easy-to-understand JavaScript Object Notation (JSON) format [78]. At the same time, it also suffers from several issues discussed in this paper. First, it does not provide any tool to define any identifier for collected camera data, even for DJI drones, which support assigning specific file names for captured images. Second, it has low accuracy of flight time prediction [5,79]. Finally, it does not allow the setting of desired accelerations and declarations of the drone, controlling its tilting angle (Figure 20).

Figure 20. UgCS ground station interface. The red fence marks the waypoint configuration window, the green fence marks the 3D mission view, and the blue fence marks the mission elevation profile.

5. Discussion

Aerial inspection using multi-rotor drones is a fast-growing technology with multiple advantages compared to traditional non-destructive inspection approaches. However, the attainability of these advantages varies between various application fields. Inspecting large-scale industrial facilities is one of the most complex tasks in aerial inspection. This section will focus on trade-offs that should be resolved in this application area.

All the advantages listed in Section 3 can be summarized as better cost-efficiency, higher inspection quality, and better safety. The cost-efficiency is generally achieved by performing inspections without stopping production processes, shorter inspection times compared to other methods, and reduction of work in hazardous areas, which are also quite expensive. At the same time, the estimates of aerial inspection costs rarely include all the indirect expenses, such as drone maintenance and diagnostics between flights, infrastructure (landing areas, equipment storage rooms, workshops, etc.), transportation costs, etc. They can look negligibly small, but this often does not seem right. For example, if the outsourcing company performs the inspection, this company can minimize costs on drone maintenance and infrastructure by sharing them between multiple clients. Simultaneously, the transportation cost for such a company will be higher because large industrial facilities are usually distributed around the region and situated in different towns. Moreover, the outsourcing company will also suffer from weather conditions limits because it should cover the salary and travel expenses of the operator, who waits for suitable weather in a remote location. Another example is when the employees of the industrial facilities perform an inspection. In this case, transportation costs and the effects of weather conditions will be minimal. However, the infrastructure and maintenance costs will significantly rise because the owners of the industrial facility should employ a qualified operator and prepare all necessary infrastructure for one or two drones, which will operate only a few days per month. Finally, even with qualified operators and all the precautions in both described-above cases, the risk of drone crash is not zero. At the same time, the crash can result in a stop of production and lead to losses incomparable to the inspection cost. Insurance can cover such risks, but according to our experience, its price is also relatively high and makes sense only if a single drone is used to inspect multiple facilities.

The higher quality of inspection data is a more straightforward benefit than cost-efficiency. Despite the reasons published in [1], the influence of the reflections and noises caused by surrounding objects on the data collected by multi-rotor drone is generally less compared to the other types of inspections. A comparison presented at the end of [5] clearly shows that thermal images captured from the air have better quality than the images of the same equipment shot from the ground (Figure 21). At the same time, to reach such quality it is not enough to make a shot from a short distance, which is alone not an easy task in automated mission planning. For accurate inspection, the images should be captured from different fields of view and out of the turbulent zones [1,5,26]. Moreover, during the angle of shot selection, one should consider the position of the sun and the presence of other interference that could influence the quality (like hot pipes on the image of the insulators during thermal inspections) [1,21,23,62]. Thus, efficient multi-criteria mission planning is crucial to achieving high inspection quality on large-scale industrial facilities.

Figure 21. Quality comparison between aerial (a,b,e,f) and ground (c,d,g,h) inspection [5]. The top row contains raw images and the bottom row contains post-processed images of the insulators cropped from the raw images.

The higher level of safety is another advantage of the inspection using multi-rotors that can lead to unnecessary illusions. From the point of view of the diagnostic engineer, it is much safer to capture the equipment located on top of a factory pipe using a drone than doing the same using manual inspection equipment after climbing that pipe. From another point of view, the drone itself can be considered a source of hazard, especially in the case of large industrial drones like the R.A.L. X6 or DJI Matrice 300. Moreover, all the areas beneath the drone’s route can be considered hazardous, and should be cleared of any other personnel. Thus, we can stress one more time the importance of the multi-criteria mission planning, which should not only consider factors influencing inspection data quality but also trace the inspection routes in such a way as to minimize flying above the areas critical for facility non-stop operation (from where personnel cannot be removed during the inspection).

After discussing the advantages, it is also worth discussing the challenges listed in Section 3. The limited battery life of modern drones is partially compensated by the relatively affordable prices of battery units for most industrial drones. Today, even small facilities are fully inspected only using several batteries. Moreover, the ability to charge multiple batteries simultaneously makes it possible to perform non-stop flights using a big enough battery pack. At the same time, while inspecting large facilities, the problem of low battery life becomes more complex. Due to the lack of landing area and large distances the drone has to fly between the equipment, it is necessary to always have enough spare battery time to return back to the landing in case of emergency or weather conditions changes [5], once again highlighting the importance of effective mission planning.

The limited payload, in many cases, is another aspect of low battery life. Modern industrial drones can simultaneously handle multiple cameras [80], but this option is rarely used because it is more effective to divide all the inspected equipment parts by the classes according to the drone payload that should be used for their diagnostic and then create a set of routes to inspect each class of equipment separately using only the necessary payload. Thus, all the main technical challenges and trade-offs of large-scale industrial facility aerial inspection using multi-rotor drones can be fully or partially resolved by tuning the criteria of the mission planning.

The rest of the listed challenges are operator competence and regulation issues. It is often assumed that more automated drones will require less competent operators. At the same time, it is true only if the complexity of the task stays constant. During industrial inspection using a manually controlled drone, the operator should be professional enough to keep the drone in the air, avoid collisions, and be able to reach the desired point of view to collect the inspection data. At the same time, such flights are usually performed in the area near equipment, where the drone is always kept in the operator’s line of sight. Moreover, only a few equipment parts are inspected in a single flight. Performing such flights in automated or fully automatic mode will require additional efforts related to mission planning, which make sense only if done for a relatively large area. In this case, flights are performed far from the operator in areas crowded with obstacles with a high risk of collisions [5,21]. Simple switching to manual control will not save the situation in case of emergency. Moreover, industrial drones, like the R.A.L. X6, can be purchased even without any manual control. At the same time, the competence of the operator for such automated flights should be even higher than that of manual ones. The operator should know the area deeply and constantly control for changes in weather conditions. Also, pre-flight checks and drone maintenance have a greater influence on safety than short manual flights. Thus, the difference between the operator of manually controlled drones and the automated one is similar to the difference between pilots of a small single-seater aircraft and of a large plane like the Airbus A380. The first one literally controls the plane on the tip of the fingers, while the second one relies on many automatic tools but should constantly analyze the situation and be ready to perform necessary actions in case of an emergency. There is no doubt that the minimal competence of the second one is usually higher than that of the first one.

As stated in the paper selection procedure description, the legal and regulation issues are out of the scope of this review. At the same time, despite it being one of the exclusion criteria, the papers from the final selection still mentioned them alongside other challenges. Modern airspace control rules in many countries were written decades ago and, after that, were only slightly adapted for the recent technology changes. At that time, people only imagined that one day, hundreds and thousands of UAVs would fly in the air indoors and outdoors. These rules were designed to control a relatively small amount of aerial vehicles controlled by professional pilots and guided by high-qualified air traffic controllers. Currently, in many countries, drone flights are either not properly integrated into the existing air traffic control mechanisms or integrated using the same rules as for regular human-controlled aircraft. There are currently many projects in this area [81,82,83,84,85]. However, until these projects are integrated into the legal system, the regulation issues will be a significant limiter for the implementation of any unmanned aerial inspection, especially inspection of large-scale industrial facilities.

After discussing the general pros and cons of aerial inspection of large scale industrial facilities, we now switch to the technical issues related to the market available multi-rotor drones. This topic is also very important, because if the drone lacks of any necessary capabilities, there will be no guarantee of safe and efficient inspection even in the case of the best mission planning.

As expected, the DJI and Autel Robotics products are the most mature of the analyzed ones. Furthermore, it is even more surprising that they share some of the issues we observed on the R.A.L. X6 during our research. At the same time, if the manufacturers pay attention to the discussed drawbacks with their resources, it will not take much time to overcome these issues.

ArduPilot and PX4 are extremely popular worldwide among mid- and small-size companies that produce drones. Thus, it is essential to show that both of these firmwares share key issues we revealed while using the R.A.L. X6 in the large-scale facility inspection application. Moreover, most of these issues can be considered safety related. At the same time, ArduPilot and PX4 are open-source projects with large communities worldwide, which makes it possible to rapidly improve the firmware based on solutions we proposed for the R.A.L. X6 in current research.

INAV is the least functional solution among the ones analyzed in this paper. It shares most of the issues we faced with the R.A.L. X6. Moreover, its overall functionality is much lower compared to open-source solutions such as ArduPilot and PX4. It is currently used mostly on manually controlled drones, but the presence of functions related to automated flights along the mission may create an illusion that it is suitable for more complex inspections, but, according to our review, it is not correct. Moreover, whether it requires any improvements or should just be considered unsuitable for complex inspection applications in large industrial facilities is questionable.

UgCS is universal and advanced mission planning software compatible with many industrial drones from different manufacturers. However, to be fully suitable for automatic inspection of industrial facilities, it requires improvement of its timing estimation algorithms, provision of functionality to specificity identifiers of the data collected during the flight, and implementation of the ability to configure lower acceleration/deceleration rates at waypoints.

Initially, the R.A.L. X6 drone was comparable to the ones running INAV Multicopter firmware, which made it unsuitable for large-scale industrial facilities inspection both in terms of safety and functionality. At the same time, as clearly shown above, all of its issues can be solved using state-of-the-art methods. We translated this information to the manufacturer, who released special patches for the drone’s firmware. After all the fixes, the drone successfully performed the first autonomous inspection of a 2400 MW thermal power plant [5]. Currently, the only claim left to this product is that its ground station software does not support to import missions in widely used file formats, like WPML. Opening the current version of their proprietary mission file format can be considered only as partial solution, especially in the case that they do not guarantee to support it in the future. We hope the manufacturer will change its opinion and will add WPML support in the next versions of the RALTool.

However, the modification of the industrial drone following the feedback of the research community can be considered a success story, and we hope it can motivate other drone manufacturers, especially those using flight controllers with open firmware, to adapt their products to fulfill the requirements of large-scale industrial facility inspection.

6. Conclusions

This review summarizes the pros and cons of aerial large-scale facility inspection, distinguishing it from other inspection scenarios implemented with drones. Based on their analysis, we reveal the main trade-offs and demonstrate the importance of highly efficient multi-criteria mission planning. At the same time, even the best planning will not result in safe and accurate inspection if the drone is not capable of completing the mission.

Industrial facility owners frequently require insurance to secure UAV damage and third-party liability risk when performing aerial inspections. In turn, insurance companies prefer to insure the flights of market-available drones rather than the customized ones created by the research teams. Most companies producing industrial drones claim their products are entirely suitable for industrial inspection. At the same time, during our research, which resulted in the first autonomous inspection of a large-scale thermal power plant, we had to overcome multiple issues with the R.A.L. X6 drone, influencing its safety and efficiency. Moreover, the analysis of the other popular market-available solutions showed that most of them suffer from the same problems, more or less. Thus, our research shows how drone manufacturers can improve their products. At the same time, it also demonstrates to the researchers and inspection engineers what issues they will probably face while inspecting large-scale facilities using the industrial drones currently available on the market.