*Article* **Safety Enhancement of UAVs from the Signal Processing's Perspectives: A Bird's Eye View**

**Chiman Kwan**

Signal Processing, Inc., Rockville, MD 20850, USA; chiman.kwan@signalpro.net

**Abstract:** Unmanned air vehicles (UAVs) or drones have gained popularity in recent years. However, the US Federal Aviation Administration (FAA) is still hesitant to open up the national air space (NAS) to UAVs due to safety concerns because UAVs have several orders of magnitude of more accidents than manned aircraft. To limit the scope in this paper, we focus on large, heavy, and expensive UAVs that can be used for cargo transfer and search and rescue operations, not small radio-controlled toy drones. We first present a general architecture for enhancing the safety of UAVs. We then illustrate how signal processing technologies can help enhance the safety of UAVs. In particular, we provide a bird's eye view of the application of signal processing algorithms on condition-based maintenance, structural health monitoring, fault diagnostics, and fault mitigation, which all play critical roles in UAV safety. Some practical applications are used to illustrate the importance of the various algorithms.

**Keywords:** UAVs; safety; drones; signal processing; condition-based maintenance; structural health monitoring; fault diagnostics; fault mitigation; contingency planning

#### **1. Introduction**

The use of unmanned aerial vehicles (UAV) in the military and industry today is becoming more and more widespread. However, perhaps due to lower manufacturing standards and budget limitations, the mishap rates in unmanned aerial vehicles (UAVs) are several orders of magnitude greater than manned aviation [1]. Considering these high mishap rates, the US Department of Transportation's Federal Aviation Administration (FAA) has initiated several programs and partnerships to enhance the safety and reliable operation of UAVs [2].

In this paper, we focus on large, heavy, and expensive (millions of dollars) UAVs such as cargo transport and search and rescue aircraft, not radio-controlled amateur drones. In general, UAV safety can be improved from the following perspectives. First, the UAV manufacturers need to use durable engines and communication equipment and strong structural materials. Reliable communication equipment will ensure drone safety, as a lost link between UAV and ground station is very dangerous [3]. Durable engines and strong materials will ensure reliable flight in rough weather conditions. Second, advanced conditioned-based maintenance (CBM) practice should be deployed [4–6]. Compared with traditional periodic preventive maintenance, CBM can be cost-effective but may require additional sensors to monitor some critical components such as the engine. Third, structural health monitoring (SHM) [7,8], especially the non-destructive type of SHM, should be frequently used to monitor cracks, loosened fasteners, etc.

The aforementioned ideas mostly require both hardware and software, and have been routinely used in UAVs to maintain reliability and safety. In addition to the above safety enhancement practices, we would like to emphasize that signal processing algorithms also play important roles in other critical areas. Figure 1 shows the general architecture of an integrated system that can enhance the safety of UAVs. Some monitoring operations such as SHM, CBM, are done off-line on the ground. The off-line process involves data

**Citation:** Kwan, C. Safety Enhancement of UAVs from the Signal Processing's Perspectives: A Bird's Eye View. *Drones* **2021**, *5*, 16. https://doi.org/10.3390/ drones5010016

Received: 20 January 2021 Accepted: 22 February 2021 Published: 26 February 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

collection and processing. Others such as sensor and actuator fault diagnostic, faulttolerant control are done online. The sensor's measurements (angles, speed, etc.) are all collected in real-time and processed in real-time. Contingency planning is done off-line. Moreover, as shown in Figure 1, we can clearly see the use of signal processing algorithms in fault detection, fault isolation, fault magnitude reconstruction, and fault mitigation. First, robust fault diagnosis algorithms [9–18] perform accurate fault detection and isolation. Failures in sensors and actuators can cause system instability. Diagnosing those faults will improve the overall safety of UAVs. Second, in fault mitigations, robust [19–23] and fault-tolerant controllers [24–31] are also critical for UAV safety. Robust controllers can tolerate some small perturbations in the UAVs due to aging or external disturbances. Fault-tolerant controllers can perform control reconfiguration to directly address sensor and actuator malfunctions during flight. Third, one critical aspect of fault mitigation is contingency planning for engine failures, which is the last line of defense for UAV safety. That is, all the aforementioned practices have been tried, but nothing works and the UAV is eventually on its course to crash due to engine loss. Under such an emergency, the goal of contingency planning is either to help UAV operators to glide the UAV or will make the UAV autonomously land itself to a crashing/ditching site or local airport runway if there is no reliable communication link available. In the case of an emergency due to full loss of thrust, wind plays a critical role with respect to reachability to the emergency landing site [32]. Upon full loss of thrust due to engine failure, because of the wind impact, the UAV may not reach the designated landing site and crash into populated areas causing loss of lives. Thus, the wind impact on reachability needs to be addressed in path planning for engine loss contingencies. It is also important that in the event of an emergency that might happen at high altitudes, the UAV should choose a forced landing path that does not violate no-fly zones or stormy weather air zones not to further complicate the situation.

**Figure 1.** Enhancing UAV safety from the perspectives of signal processing algorithms.

The contributions of our paper is as follows:


Our paper is organized as follows. In Sections 2 and 3, we briefly mention CBM and SHM practices. In Sections 4 and 5, we focus on fault diagnostics and fault magnitude estimation. In Section 6, we address fault mitigation using robust and fault-tolerant controllers. Section 7 discusses the last line of defense: contingency planning for an emergency landing. Finally, some concluding remarks, limitations, and future directions are given in Section 8.

#### **2. Condition Based Maintenance (CBM)**

Traditionally, mechanical components such as engines, bearings, and gearboxes are maintained by using preventive maintenance, which performs periodic checks on the components. For example, engine oil for cars is replaced with new oil every three thousand

miles or every six months. Preventive maintenance is certainly effective and has been widely used. However, one drawback is that it may not be cost-effective. When a car is not being used frequently, periodic maintenance may be wasteful because the oil quality may still be good.

In recent years, people have been advocating condition-based maintenance [33,34], which means some maintenance decisions should be made based on the condition of the component rather than time. Two survey papers [35,36] provided a good review of the relevant works in this area. Some papers have proposed the use of wireless sensor networks (WSN) for quantifying system conditions [37,38]. Figure 2 illustrates how WSN can be used for induction motor monitoring.

**Figure 2.** The use WSN for induction motor monitoring.

It should be noted the Internet of Things (IOT) is also a hot area that can be beneficial to CBM. In [39], some discussions talked about the various applications of IOT in CBM. IOT can also help online fault diagnosis as well.

#### **3. Structural Health Monitoring (SHM)**

As shown in Figure 3, Structural health monitoring (SHM) [7,8] is one way of nondestructive inspection (NDI). It can be done passively or actively. The purpose is to detect structural defects such as cracks on the wing, loosened bolt, and fasteners, etc.

**Figure 3.** Schematic view of the direct analog RF coupling approach for wireless aircraft wing inspection.

In our opinion, SHM is also one form of CBM. To illustrate the basic idea of SHM, one can refer to Figure 4, which shows an array of sensors/actuators to monitor an aircraft wing panel. Ultrasonic pulses are generated from each sensor, and the rest of the array is receiving the transmitted signals. Once all the sensors are scanned, the collected signals from all the elements will be fed into an algorithm for fault localization. In [8], a RAPID algorithm was developed, which has been widely used in many papers.

**Figure 4.** Application of ultrasonic sensor/actuator array to monitor aircraft wing panel.

The interrogation methods in [7,8] are bulky. Recently, there are new and recent advances in wireless sensing for SHM. As shown in Figure 5, passive low-power wireless node can be installed onto structures. Details can be found in [40].

**Figure 5.** Recent SHM using wireless sensors for active crack detection.

#### **4. Sensor and Actuator Fault Diagnostic Algorithms**

Sensors and actuators are critical components in complex systems. For instance, in an airplane, effective flight control is impossible if sensors and/or actuators are malfunctioning [41,42]. Sensors and actuators can fail, and their failures have a significant impact on the performance of a system. In the worst case, the failure even can affect the safe operation of the system, leading to a catastrophic event. Sensor failures may include precision degradation, drift, frozen reading, and complete failure [43]. Similarly, actuator failures may include limited range of motion, e.g., valve stiction and complete failure [44]. It is a challenging task to detect sensor and actuator failures [41,42] because sensor outputs contain information from a multitude of sources: normal system outputs, faulty sensor signals, and signals due to noise and external disturbances. Isolating different signals requires the utilization of the input-output relationship of the system. Conventional approaches to increasing the reliability of aircraft systems include installing redundant sensors, which will add more weights, costs, complexity, and most importantly, additional reliability problems.

Given a linear multi-input multi-output (MIMO) system, the inputs, internal states, and outputs can be described by

$$\begin{aligned} \mathbf{x}\_{k+1} &= \mathbf{A}\mathbf{x}\_k + \mathbf{B}\mathbf{u}\_k + \mathbf{B}\mathbf{M}\_{\boldsymbol{\upmu}}\boldsymbol{\Delta}\mathbf{u}\_k + \boldsymbol{\omega}\_k\\ \mathbf{y}\_k &= \mathbf{C}\mathbf{x}\_k + \mathbf{D}\mathbf{u}\_k + \mathbf{D}\mathbf{M}\_{\boldsymbol{\upmu}}\boldsymbol{\Delta}\mathbf{u}\_k + \mathbf{M}\_{\boldsymbol{\upmu}}\boldsymbol{\Delta}\mathbf{y}\_k + \boldsymbol{\varepsilon}\_k \end{aligned} \tag{1}$$

where **u***k*, **y***k*, and **x***k*, are system inputs, outputs, and internal states, respectively. Δ**u***k*, Δ**y***<sup>k</sup>* denote actuator and sensor faults. We would like to detect if the magnitude of Δ**u***<sup>k</sup>* or Δ**y***<sup>k</sup>* is non-zero. If the answer is positive, we have to estimate the magnitude and identify the failure direction matrix **M***<sup>u</sup>* or **M***y*. Finally, the magnitudes of failed sensors or actuators will be reconstructed.

Sensor fault detection and diagnosis have been a research topic for decades, and many articles have been published. Interested readers are referred to the survey paper by Frank [45]. While fault detection is relatively easy, isolation of multiple faults is still a challenge to many existing schemes. Recent advances in fault diagnosis use kernel partial least square [46] and nonlinear techniques [47].

#### **5. Sensor Magnitude Reconstruction**

After the identification of faulty sensors/actuators, one has to estimate the fault magnitudes. Detailed procedures can be found in [48]. With estimated fault magnitudes, one can correct measurements of the faulty sensors. The measurements in faulty sensors are affected both by the actuator fault Δ**u***<sup>k</sup>* and sensor faults Δ**y***k*. In order to get the corrected measurements in faulty sensors, one can design a Kalman filter,

$$\begin{aligned} \dot{\mathbf{x}}\_{k+1} &= \mathbf{A}\dot{\mathbf{x}}\_{k} + \mathbf{B}\mathbf{u}\_{k} + \mathbf{K}(\mathbf{y}\_{k} - \mathbf{M}\_{y}\Delta\mathbf{y}\_{k} - \mathbf{D}\mathbf{M}\_{u}\Delta\mathbf{\hat{u}}\_{k} - \mathbf{C}\mathbf{\hat{x}}\_{k} - \mathbf{D}\mathbf{u}\_{k}) \\ \dot{\mathbf{y}}\_{k} &= \mathbf{C}\mathbf{\hat{x}}\_{k} + \mathbf{D}\mathbf{u}\_{k} \end{aligned} \tag{2}$$

where **y**ˆ *<sup>k</sup>* is the corrected measurement of faulty sensors. **A**, **B**, **C**, **D** are the system matrices in the state-space model. **K** is the Kalman gain, **M***u* and **M***y* are the fault direction matrices [48].

Nowadays, UAV formation (swarm) control practice uses the Real-time Kinematic Global Navigation Satellite System-Inertial Navigation System (RTK-GNSS-INS) [49] for routing and navigation. Moreover, inter-UAV sensors such as Automatic Dependent Surveillance-Broadcast (ADS-B) [50], which provides altitude, aircraft flight ID, and vertical airspeed, have been used in advanced systems. ADS-B reduces the risk of runway collisions, even at night or during heavy rainfall. ADS-B applications being developed now will give pilots indications or alerts of potential collisions. Such systems may be used to dissolve the problem related to faulty sensors.

Here we describe some results of detecting GNSS sensor failure and how to use reconstructed sensor information to control the formation of UAVs. We will illustrate the effectiveness of a fault-tolerant formation control algorithm [30] using the rotary UAV developed by UC Berkeley. Two types of formations are considered: mesh and triangle. For each formation, we performed two types of flight: straight flight and 90-degree turning flight. In each case, we assume a fault occurs to the fifth UAV at 10 s. A GNSS/INS (Inertial Navigation System) combination is usually used in UAV navigation systems. In the presence of GNSS failure, the position information provided by the INS will diverge due to error accumulation. Motivated by this observation, the fault model under consideration simulates slowly divergent position measurements. A fault isolation scheme was designed to capture this GNSS failure. Then an observer was designed and used to estimate the positions of the UAV. Figure 6 shows the results, which clearly demonstrated that the reconstruction can still maintain the UAV formation.

**Figure 6.** Comparison of results with fault tolerance and without fault tolerance. (**a**) Without fault tolerance (one UAV is drifting away); (**b**) with fault tolerance.

#### **6. Robust and Fault-Tolerant Control**

As shown in Figure 7, there can be quite a few actuators for controlling the aircraft. For example, the rudder controls the yaw channel, elevators control the pitch channel, and flaps control the roll channel. From Table 1, we can see that some actuators can be used as secondary actuators for some channels. This shows that different combinations of actuators can serve as backup actuators for different roll, pitch, and yaw channels. In general, fault

tolerant control refers to the selection of substitute actuators when primary actuators fail to function.

**Figure 7.** Various actuators in a hypothetical aircraft.

**Table 1.** Primary, secondary and tertiary actuators for the roll, pitch, and yaw channels.


A robust controller [19–23] can deal with parametric uncertainties to some extent. However, when some faults such as sensor and actuator faults occur, robust controllers are not strong enough to handle such situations. A fault-tolerant controller [24–31] is extremely useful for guaranteeing closed-loop control performance.

Figure 8 shows a schematic diagram of how fault-tolerant control works. First, a fault diagnosis system that performs on-line fault detection and isolation. Second, a controller suite consists of a primary nominal control system used under normal operating conditions (without faults) and a secondary adaptive fault-tolerant control system engaged only after fault detection. Third, a reconfiguration supervisor makes decisions regarding control system reconfiguration and control reallocation using the fault information provided by the diagnostic module.

Here, we briefly summarize one application of our neural net (NN) controller. Our controller suite includes the nominal controller and the NN-based adaptive fault-tolerant controller. The nominal controller is used when the system is in a fault-free condition. The NN adaptive controller is activated after a fault is detected to compensate for the effect of the fault and to maintain acceptable control performance even in the presence of a fault. The fault-tolerant controller performance was demonstrated by using a wellknown RCAM (Research Civil Aircraft Model) developed by the Group for Aeronautical Research and technology in Europe (GARTEUR). The design of the nominal controller is based on standard approximate dynamic inversion. Figure 9 illustrates the proposed nonlinear adaptive control architecture: the aircraft (*f*), the command filter to provide desired handling qualities, approximate dynamic inverse ( ˆ *f* <sup>−</sup>1), a conventional linear tracking controller as described above, and an online learning neural network to correct for errors and uncertainty in association with the inversion model.

**Figure 8.** Illustration of fault-tolerant control strategy.

**Figure 9.** Neural Network Augmented Model Inversion Control Architecture.

One of the key advantages of the proposed fault-tolerant control scheme is its capability to handle any occurrences of new or unanticipated faults. The neural network-based adaptive controller activated after fault detection is still capable of compensating for the effect of the fault on-line and to maintain acceptable control performance before further pilot intervention.

Figures 10 and 11 give the control performances of the nominal controller and nonlinear adaptive controller engaged after a fault detection, respectively. We can clearly see the benefit of controller reconfiguration using on-line diagnostic information.

**Figure 10.** Tracking performance of the nominal controller when a aileron failure occurs at t = 10 s.

**Figure 11.** Tracking performance with controller reconfiguration after detection of a tailplane failure.

In recent years, there has been some new progress in fault-tolerant control. In [51], a hybrid fault-tolerant controller was proposed to handle control surface damages. Compared to the nonlinear control approach in [18], the controller in [51] was based on linearized models. In [52], a nonlinear sliding mode controller was proposed to deal with actuator failure in quadrotor UAVs. The inner loop was the attitude control and the outer loop was the position control. Sliding mode control was applied to both inner and outer loops. The control is challenging because if one or two control actuators fail, the system is underactuated. A special form of sliding mode control was proposed based on back-stepping. In [53], an observed-based fault-tolerant controller was proposed for carrier-based UAVs. Some states in the UAVs are assumed to be unmeasurable and hence the control problem is challenging. The controller was nonlinear and closed-loop stability was given.

#### **7. Contingency Planning**

This is the last defense for fault mitigation in UAVs. The UAV has lost its engine and it is on its way to crash. Emergency landing via the parachute systems is part of the contingency planning for some small-to-medium sized UAVs [54]. However, for big and heavy drones like Global Hawk (12 tons) [55], parachute is not feasible. Can we still do something to minimize the damage? In other words, for fixed wing UAVs, the hanging time of some UAVs such as Global Hawk can still be 30 min or more due to its high flying altitude and large wing span. Moreover, if one plans ahead, the UAVs can still glide to some safe landing places such as airports, non-populated places such as beaches, waterways, grassy areas, etc. A well-known example is the US Airways Flight 1549, which avoided a crash landing by gliding onto the Hudson River.

In our recent papers [56–60], we have provided detailed procedures for contingency planning for engine failures.

Figure 12 shows the workflow of contingency plan generation. It is an off-line process. Given a UAV and its associated flying capabilities (wingspan, gliding speed, descending rate, etc.) and also the theater of operations, we need the following two major steps: preprocessing and contingency plan generation. We will summarize those two steps in the next two sub-sections.

#### *7.1. Preprocessing*

There are multiple modules in the preprocessing step. First, landing place selection is needed. Based on the UAV's size and gliding speed, an appropriate landing site needs to have enough length. Potential landing sites include airport runways, beaches, waterways, etc. In [58], we developed a landing site selection algorithms based on Google maps. Figure 13 shows a landing site.

**Figure 12.** Contingency plan generation workflow [56].

**Figure 13.** Landing site candidate-1 (green dotted line passes through the center of the landing site along its length).

Second, for each landing site, we need to assign some waypoints such as Touchdown point (TDPT), Initial Approach Fix (IAF), and Final Approach Fix (FAF). Detailed procedures are provided in [56].

#### *7.2. Contingency Plan Generation*

Given a primary flight plan containing hundreds or even thousands of waypoints, we need to generate a contingency plan for each waypoint. An example of a contingency plan is shown in Figure 14.

**Figure 14.** Contingency plan example.

Suppose an engine failure occurs near a waypoint in the primary flight path, we will apply A\* path planning algorithm [56] to generate a contingency path between the contingency point (CP) and the IAF. Some no-fly zones need to be bypassed. From Figure 14, the red line section shows the A\* generated plan. In the contingency plan, when the UAV reaches the IAF, there may still be excessive altitude to lose. We have developed time-constrained path generation algorithm to lose excessive altitude [57]. Wind speed needs to be taken into account. Some extremal paths can be seen in Figure 15. RSR means right-straight-right; RSL means right-straight-left.

**Figure 15.** Two extremal paths. Other paths such as LSR, LSL, RSL, and RSR can be found in [57].

There are also some additional processing steps to deal with different excessive altitudes. Details can be found in [56].

One limitation of our contingency planning approach is that if the wind conditions deviate too much from the forecast conditions, which can happen in practice, then the pre-planned contingency paths may need to change on the fly. More research is needed in this direction.

#### *7.3. Emergency Landing in Hudson River*

Here, we demonstrate how we generate a contingency plan for US Airways Flight 1549, which lost both engines due to a bird strike. The plane was on route from New York City's LaGuardia Airport to Seattle, Washington and ditched into Hudson River on 15 January, 2009. In the climb phase right after its takeoff, it struck a flock of Canada geese and lost its engine power. The pilots Mr. Sullenberger and Mr. Skiles glided the plane to a ditching in the Hudson River. At the time of the bird strike, Flight 1549's airspeed was about 200 knots. The highest altitude right before the plane started sinking was 3034 feet (925 m). At this altitude, the plane was located at coordinates: Latitude: 40.861666 degrees, Longitude: −73.879722 degrees. Time was 3:27:29 p.m. The wind amplitude was around 13.4 knots. The wind direction was 320 degrees. Assuming the coordinates at the highest altitude corresponds to the coordinates of CP, we found the heading angle at CP using the waypoint coordinates at the highest altitude, the waypoint right before that, and the wind information.

We applied our contingency plan generation tool to this incident. Figures 16 and 17 show the top view and 3D view of the generated plan.

In the movie "Sully", various flight simulator runs also showed that it was possible to return to LaGuardia and Teterboro airports. Why did the pilots choose to land on Hudson river? The reason is that the pilots need to follow some procedures to make sure both engines were lost. During this period of checking, the plane has lost quite some altitude. Consequently, there was not enough altitude to glide it back to the airports. Based on his experience, the pilot, Sullenberger, made some rough calculations and decided to land on Hudson Bay; the decision saved over 150 people onboard.

**Figure 16.** Contingency plan for US Airways Flight 1549 to return to LaGuardia airport: Top down view.

**Figure 17.** Contingency plan for US Airways Flight 1549 to return to LaGuardia airport: 3D view.

#### **8. Conclusions**

UAVs are gaining popularity. However, the safety of UAVs is not on par with manned aircraft. In this paper, we present safety enhancements of UAVs using signal processing algorithms, which can help condition-based maintenance, structural health monitoring, sensor and actuator fault diagnostics, fault magnitude reconstruction, fault-tolerant control, and contingency plan generation. Some recent advances in the aforementioned areas are also highlighted.

It is important to emphasize that UAV safety requires an integrated approach that contains all of the above. More research and development effort is needed to produce an integrated safety system for UAVs. For instance, the contingency plans are generated off-line, and if wind conditions deviate a lot from the forecast data during actual flights, our current system may not be able to handle that scenario. One potential future direction in contingency planning is to deal with highly dynamic windy conditions. Another direction is to devise some online contingency planning strategies to complement the off-line generated contingency plans.

**Funding:** This project was supported in part by US government under the PPP program. The views, opinions and/or findings expressed are those of the author(s) and should not be interpreted as representing the official views or the US Government.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Conflicts of Interest:** The author declares no conflict of interest.

#### **References**

