Safe Local Aerial Manipulation for the Installation of Devices on Power Lines: AERIAL-CORE First Year Results and Designs

Abstract

The power grid is an essential infrastructure in any country, comprising thousands of kilometers of power lines that require periodic inspection and maintenance, carried out nowadays by human operators in risky conditions. To increase safety and reduce time and cost with respect to conventional solutions involving manned helicopters and heavy vehicles, the AERIAL-CORE project proposes the development of aerial robots capable of performing aerial manipulation operations to assist human operators in power lines inspection and maintenance, allowing the installation of devices, such as bird flight diverters or electrical spacers, and the fast delivery and retrieval of tools. This manuscript describes the goals and functionalities to be developed for safe local aerial manipulation, presenting the preliminary designs and experimental results obtained in the first year of the project.

1. Introduction

The power grid is one of the most extended and critical infrastructures in any country, comprising thousands of kilometers of power lines that require periodic inspection and maintenance (I&M) to ensure continuous service, despite the diverse factors that may affect correct operation. Adverse climatic conditions (i.e., rain, wind, snow), vegetation growth, and the impact on the bird species in the environment are some typical problems faced by companies responsible for its management and maintenance, involving a significant economic investment due to the vast extension of this infrastructure. Not only that, but the high altitude (15–50 m) and high voltage (15–500 kV) of the power lines make them a highly risky workspace for human operators, who must follow strict safety protocols [1] and use special clothes [2], which reduce the effective operation time and the feeling of comfort in the realization of manipulation tasks. The installation of devices, such as bird flight diverters or electrical spacers, such as those shown in Figure 1, is an illustrative example of an operation that requires the transportation and deployment of the worker on power lines. As indicated in [1], different methods and tools can be considered for positioning the personnel on the points of interest, including non-conductive ropes, bucket trucks, or manned helicopters.

The development of aerial manipulation robots in the last decade [3,4,5] has been motivated by the ability of these platforms to easily and quickly reach high altitude work spaces and conduct diverse operations involving physical interaction with the environment, demonstrating their application in contact-based inspection in chemical plants [6] and bridges [7], the installation of sensor devices [8,9], valve turning [10] and other torsional operations [11]. In this sense, the use of aerial manipulation robots are particularly useful to install devices, such as the ones shown in Figure 1, avoiding risk for human operators while reducing the time required to reach the points of interest. However, some new challenges arise. First, the electrical interaction between the high voltage power line and the aerial robot, considered a floating conductor [12], should be investigated to avoid possible malfunctions due to electrostatic discharge [13] or magnetic noise. Second, the risk of collision of the aerial robot with the transmission line conductors, especially outdoors, where the wind gusts may be significant, makes convenient the use of long-reach [8] or cable-suspended [14,15] configurations to increase the separation distance between the manipulator and the aerial platform. Third, the manipulator should provide a sufficient level of dexterity and force to install the mentioned devices, taking into account that these are intended for human operators.

This paper proposes the application of aerial manipulation robots for the installation of bird flight diverters and electrical spacers on power lines, presenting the preliminary designs and results derived from the first year of the AERIAL-CORE (AERIAL COgnitive integrated multi-task Robotic system with Extended operation range and safety) H2020 project [16]. The paper describes the technological solutions under development, including prototypes of arms and end effectors, methods for safe and accurate aerial manipulation, and augmented human–machine interfaces. The paper analyzes first the requirements and functionalities of the system from an illustrative scenario, identifying later the effects of the high voltage/current of the power line on the aerial robot. The paper aims to provide a wider perspective of the problem and the possible solutions, considering the different approaches followed by the authors.

The rest of the paper is organized as follows. Section 2 introduces the current procedures in the inspection and maintenance of power lines with a focus on the installation of devices based on the requirements and functionalities. Section 3 analyzes the electromagnetic effects of the power lines on the aerial manipulation robot. Section 4 presents the preliminary designs of arms and end effectors for the installation of the devices, whereas Section 5 is devoted to the methods for safe aerial manipulation. Section 6 describes the human–aerial robot interaction, comprising augmented cognitive human–machine interfaces intended to facilitate the aerial manipulation operation, and the use of long-reach aerial manipulators for fast and safe tool delivery to the operators. Finally, the conclusion and future works are summarized.

2. Procedures in the Inspection and Maintenance of Power Lines

In this section, an overview of the inspection and maintenance procedures is provided along with a set of system functionalities to be implemented by the aerial manipulation robots.

2.1. Power Lines Inspection

The inspection and maintenance of power line assets are of great importance to ensure that the infrastructure can deliver electricity to millions of inhabitants, while maximizing the operational life of the lines before replacement is required. The task of power line inspection is undertaken through various approaches [17]. The most common approach is to use a helicopter to transport a photographer within proximity of the line, taking high-resolution photos for further analysis. Unmanned Aerial Vehicles (UAVs) are also used by technicians to inspect faults and areas of importance. In both cases, pilots and technicians require special training and permissions to complete the inspection. Even though UAVs are faster, more accurate, and cost-effective, the working range for UAVs is limited to the pilot’s visual line of sight. To overcome this limitation, onboard sensors and navigation algorithms are embedded into the UAVs to enable the system autonomy to inspect cables, beyond visual line of sight (BVLOS) [18,19].

2.2. Installation of Devices on Power Lines

The aerial manipulators developed in the AERIAL-CORE project (see Section 4) are intended to conduct the installation of the three devices illustrated in Figure 1: the clip-type bird diverter, the helical bird diverter, and the electrical spacer. Although the particular devices employed in each country or region may vary according to the local regulation or available manufacturers, the mechanisms employed to attach these to the power lines are similar between the different models. Nowadays, the installation operation is carried out by human workers employing elevating work platforms or manned helicopters to reach the points of interest. Some solutions with rolling robots [20,21] and multirotors [22] have been already demonstrated. To identify the most suitable solution according to the features of each device,

Table 1. Procedure for the installation of the different devices.

Device	Operation	Manipulator	Main Requirement
Clip-Type	Insertion/pushing	Linear actuator	High force (>100 N)
Helical	Grasping and rolling	Dual arm	Dexterity, human-size
Electrical spacer	Screwing	Screwdriver	Support screw/nut

summarizes the associated operations and requirements for the corresponding manipulator. The clip-type bird diverter is installed, exerting a relatively high pushing force (around 200 N) over the device once the line is introduced in the slot up to the circular hole that crosses the plastic body of the device. The helical bird diverter requires dexterous bimanual manipulation capabilities, given the geometry of the device and the complex installation procedure [23]. In this case, a human-like and human-size dual-arm system results in a particularly suitable solution since it allows to replicate in an intuitive way the movements of the operators. As occurs with the clip-type bird diverter, the installation of the electrical spacer requires the design of a specific manipulator that holds the main body of the device along with the two clamps and the screws, which should be also screwed automatically with a screwdriver mechanism. The manipulator is attached to the multirotor platform, exploiting the yaw control to achieve an adequate orientation when it is introduced between the two cables.

2.3. Requirements and Functionalities

In order to perform the installation of devices on the power line using aerial robotic manipulators, the following main system requirements have been identified:

• Installation of clip-type bird diverters, helical bird diverters, and electrical spacers. These are the target devices on which the solution design is focused.
• Landing and takeoff from the power line. This allows an increment in the accuracy of the manipulation since the robot can operate with its base fixed on the cables.
• Locomotion along power lines using the cables. In this way, the installation of several devices along a segment of the power line can be simplified, avoiding continuous landings and take-offs and improving energy efficiency.
• Compatibility with the live-line operation. Cut-offs of the energy transmission during I&M tasks on power lines entail high costs due to the supply losses and should be planned to mitigate their effects. For this reason, these cut-offs should be avoided.
• Autonomous installation of devices. The installation of devices, such as clip-type bird diverters, can become repetitive and relatively easy. Consequently, it should be automated to speed up the process and to reduce the operator workload.
• Teleoperation capabilities for manipulation. In contrast to the previous requirement, the helical bird diverters require complex interaction with the power line that can be difficult to automate. In this case, teleoperation methods can be applied to achieve better performance by introducing the human in the loop.
• Cognitive safe aerial manipulation. During the approaching phase to the power line and when operating around it, potential obstacles, such as trees, electric towers, or cables can be present and the associated collision and entrapment risks should be detected and mitigated.

According to the previous requirements, the developed solution should incorporate the functionalities listed below:

• Task-oriented end effectors. The use of end effectors specially designed for the manipulation of a specific device leads to a more effective operation.
• Dual-arm manipulation capabilities. Devices such as helical bird diverters are relatively large and difficult to manipulate due to their elasticity, so a dual-arm system is more adequate to perform their installation.
• Manipulation with force interaction. The installation of devices involves the application of high forces to attach them in a reliable way to the power line. Since this can be critical, particularly if the robot is flying, the aerial manipulator and its control strategy need to be designed carefully.
• Aerial manipulation with extended locomotion. The aerial manipulator can integrate a rolling base that allows movement along the power line once the robot has landed on it. In this way, flight segments can be substituted by rolling segments on the line, whose energy consumption is lower.
• High-voltage-tolerant design. The electric and electromagnetic fields associated with the high voltage and current of the power line can disturb the robot electronics. Consequently, the robot design needs to deal with this difficulty.
• Planning system for the installation of devices on power lines. The efficient installation of many devices in terms of time and energy requires the use of mission plans computed according to the particular characteristics of both the robot and the environment.
• Autonomous navigation and manipulation. In order to autonomously install devices on power lines, the aerial manipulator needs to detect the power line, estimate its relative position, and apply the proper hybrid position/force control techniques.
• Augmented cognitive human–machine interface. The aerial manipulator will incorporate a module that allows an operator to easily operate it from a ground control station in order to perform and supervise the installation of complex devices.
• Obstacle detection and avoidance. Thanks to perception sensors installed onboard the aerial manipulator, the different obstacles in the operation area will be identified, and the robot trajectory will be locally modified to avoid them.

3. Effects of Power Lines on Aerial Manipulators

Power transmission lines generate magnetic and electric fields in their environment. The magnetic field strength depends on the current flowing through the conductor, while the electric field strength depends on the voltage. Both fields can affect the aerial manipulator operating near the power line.

3.1. Effects of the Magnetic Field

Power line can be seen as a thin straight long conductor, which generates the magnetic field proportional to the current flowing through it:

$$B=\frac{{\mu }_{0}I}{2\pi r}$$

(1)

where I is the current flowing through the conductor, B is the magnetic field strength, r is the distance from the conductor and ${\mu }_0$ is the permeability of vacuum.

The magnetic field isomagnetic lines of a single conductor are circular (Figure 2), centered on the conductor, and their direction is determined by a right-hand rule. With two or more conductors, the magnetic field becomes more complex.

The typical current flowing through the power line conductor is an alternating current with a frequency of 50 or 60 Hz and with the strength of several hundred amperes. It generates a strong alternating magnetic field with the same frequency as the current. In such a setup, at a distance of less than 1 m from the power line, the magnetic field strength is stronger than the earth’s magnetic field, rendering the information from the drone’s on-board compass useless.

On the other hand, since the idea is to perform operations on the power line, the magnetic field generated by the current flowing through the conductors can be used for precise localization with respect to the power line. We explored this approach in [24], where the position of the drone with respect to the two-wire power line was calculated, using measurements from magnetometers mounted on the drone. Here, we present an analytical solution derived from the nonlinear magnetic field equations that promises low computational time. To determine the orientation of the drone with respect to the power lines, only three magnetometers are needed, and neither the current nor the magnetic field equations need to be known. It was shown that an analytical solution for calculating the drone position can be determined by combining the magnetic field equation with the magnetic energy equation.

3.2. Electrostatic Discharge

Preliminary flight tests were conducted in the context of the AERIAL-CORE project [25] in order to identify possible malfunctions of an aerial manipulation robot interacting with a live power line, taking into account that the installation procedures described in Section 2.2 involve the potential risk of crash and entrapment due to the proximity with the cables. The experiments revealed that the multirotor platform can be severely affected by the electrostatic discharge (ESD) due to the high voltage difference between the power line and the robot, whose charge increases as it approaches to the power line due to the capacitive coupling [12]. The ESD event is triggered when the electric field between the two bodies (the power line and the aerial robot) exceeds the breakdown strength of the air, causing a charge transfer between them that equalizes the electric potential. This ESD current is introduced through the robotic manipulator and propagated through the wires and conductors of the different on-board electronic components, reaching the autopilot and electronic speed controllers (ESC). Although these components were not damaged in the realization of the tests, the ESD caused the crash of the platform, due to the perturbation introduced in the PWM (pulse width modulation) signal taken as input by the ESC of the propellers. This fault is particularly critical in platforms that integrate ESCs with active brakes, which is a common feature in many commercial multirotors, causing a sudden stop of the rotors when the PWM signal presents irregular pulses. In order to overcome this problem, two solutions were investigated in [25]: shielding the aerial manipulator to prevent the entry of the ESD, and insulating electrically the manipulator from the aerial platform to prevent its propagation.

4. Arms and End Effectors for Aerial Manipulation

4.1. Aerial Manipulation Platforms

According to the requirements and functionalities identified in Section 2.3, four different aerial robotic manipulators are developed to perform the installation of the devices shown in Figure 1. These are the Main Local-Manipulation Platform (MLMP), the Deployable Manipulator with Carriage (DM-C), the Linear-Actuator Platform (LAP), and the Dual-Arm Platform (DAP). Figure 3 depicts these platforms.

The MLMP platform is a multi-purpose aerial manipulator conceptually designed for the efficient installation of devices after landing on the power line and moving along it, using a rolling system inspired by [26]. This approach combines the benefits from rolling [20] and aerial solutions [22] into a compact system. To reach the stability required to operate on the cables, the mass is distributed in such a way that the center of mass is located below the cables. This aerial manipulator is endowed with a robotic arm with interchangeable end effectors that can be used for the teleoperated installation of clip-type bird diverters and electrical spacers. Nevertheless, opposite to other specialized solutions, such as [22], its multi-purpose design makes the MLMP platform suitable for other I&M tasks on power lines.

The DM-C platform was conceived as an alternative to the MLMP. The main idea is to keep similar capabilities while allowing the decoupling of the aerial platform to the manipulator and the rolling system. Thus, the rolling manipulator can be deployed on the power line for the installation of devices. Simultaneously, the aerial platform can be used to provide devices from a supply point and to assist the rolling manipulator in aerial transitions between different segments of the power line. With this novel approach, the robot complexity increases but the installation of many devices can be addressed faster and with more efficient use of the whole system.

The LAP platform is devoted to the fast and autonomous installation of clip-type bird diverters on flight. In contrast to the few state-of-the-art solutions available for the installation of bird diverters while flying, the LAP platform allows the application of high forces during installation, without the need to use a heavy aerial vehicle. Hence, the installation mechanism consists of a high force (up to 90 kg) linear actuator that exerts a pushing force directly on the device, which is held by a clamp mechanism that transmits the reaction forces to the power line, isolating in this way the aerial platform. This manipulator is attached to an aerial platform on its bottom part through a passive spherical joint that avoids the propagation of wrench disturbances from the mechanism to the aerial platform during the realization of the operation on flight.

The DAP platform is intended to install helical bird diverters for the first time on flight, using aerial robots. Due to the complexity in the manipulation of these large and flexible devices, this robot makes use of a teleoperated dual arm in the long-reach configuration [8,27], replicating the dexterous manipulation capabilities of human operators. Similarly to the LAP platform, the long-reach manipulator is attached to the aerial platform through a passive revolute joint that reduces the disturbance transmission. Moreover, the long-reach feature offers two main advantages. Firstly, it increases the safety distance between the manipulation area and the robot propellers, which reduces collision risks. Secondly, it acts as an electrical insulator for the aerial platform because the link is made of PVC. Thus, after placing the main system electronics on the aerial platform, high-voltage physical interaction with the power line is possible.

4.2. Design of End Effectors for Holding/Grabbing and Manipulation

The devices shown in Figure 1 are commercially available and intended for installation on the power lines by human workers. To overcome this, new types of end effectors need to be designed to enable the autonomous installation of these devices. Typically, in this scenario, there are two main functions of end effectors: holding/carrying the devices to be installed, and fixing them to the power line. The installation procedure depends on each particular device, as mentioned in Section 2.2. This can be done by applying sufficient force (clip-type), twisting the device around the power line (helical), or tightening the screw on the device (electrical spacer). After the device is fixed to the power line, the end effector must be released and detached from the installed device.

The two main functions of end effectors can be achieved by two individual end effectors, e.g., one end effector holds and carries the device to be installed, and the other end effector takes care of fixing it to the power line. This approach requires two different manipulators: one for each end effector, and a certain coordination between them. This increases the weight of the whole system and requires more precise manipulation. The advantage of this approach is that the individual end effectors are mechanically simpler and can be used for different purposes, e.g., one gripper type can be used to hold a clip-type bird diverter as well as a helical-type bird diverter.

The other approach integrates both functions on the same end effector, designed specifically for the device in question. These types of end effectors are more complex and are suited only for operation with one device type. The advantage of this approach is that only one end effector is needed to install (remove) the device to (from) the power line, using only one manipulator, which results in the need for less manipulation precision. The example of such an end effector for the installation of electrical spacers is shown in Figure 4, whereas Figure 5 shows the installation of the clip-type bird diverter.

4.3. Dual Arm Manipulation

As motivated in Section 2.2, the installation of the helical bird diverter will be conducted by a human-like dual arm system, considering the long-reach configuration [8] for the integration in the aerial platform to improve safety during the realization of the operation on flight. The manipulator employed in the preliminary tests shown in Figure 6 is the LiCAS A1 [28], a lightweight and compliant anthropomorphic dual-arm system, built with smart servos and a very low weight frame structure that protects the actuators against axial–radial loads and tilting moments and integrates a compact spring-lever transmission mechanism in all the joints to provide mechanical compliance [29]. The arms provide 4 DOF (degrees of freedom) for end effector positioning—three at the shoulder and one at the elbow—with the kinematic configuration described in [29] and depicted in Figure 6. The manipulator consists of a shoulder structure that supports the left/right arms in the usual upper arm/forearm link configuration. The main mechanical features of the arms are summarized in

Table 2. Main mechanical features of the LiCAS A1 dual-arm manipulator.

Weight (kg)	2.2	Payload (at elbow) (kg)	0.7
Forearm/Upper arm links length (mm)	250 250	Separation between arms (mm)	360
Max joint speed (deg/s)	300	Joint stiffness (Nm/rad)	5–20

.

The compliant mechanism integrated in the joints is a particular implementation of the series elastic actuators presented in [30,31], allowing the estimation and control of the torques and forces from the deflection of the manipulator, defined as the position deviation referred at the joints or at the end effector with respect to an equivalent stiff-joint manipulator [8,29,32]. Regardless of the sensor employed to measure the deflection, either a magnetic encoder (joint deflection) or a camera head (Cartesian deflection), the control problem can be formulated in the joint or Cartesian space, relying on the forward and inverse kinematic models of the manipulator. It is necessary to remark that most servo actuators do not provide torque feedback or control. At most, they can be controlled directly in PWM mode, although the friction of the gearbox, with reduction ratios in the order of 1:200, reduces significantly the accuracy in the estimation of the transmitted torque. The performance of a torque/force controller based on joint deflection measurement was evaluated on flight in [32], using a 3-DOF compliant dual arm, proposing the use of one of the arms (grabbing arm) for estimating the position of the aerial manipulator relative to a fixed point, a power line in this case, while the other arm performs the operation, exploiting the passive/active compliance of the manipulator.

5. Safe and Accurate Aerial Manipulation

5.1. Hybrid Position/Force Control of Aerial Manipulators

The force interaction of aerial manipulators can be accomplished, using a hybrid position/force controller; the motion part is helpful while approaching the force application point, and thereafter the force control is activated for the interaction task. Consider Cartesian-space dynamics of an aerial manipulator given by the following:

$$H\left(\mathbf{X}\right)\ddot{\mathbf{X}}+B(\mathbf{X},\dot{\mathbf{X}})+F_e=F$$

(2)

where $\mathbf{X}\in {\mathbb{R}}^6$ is the Cartesian-space position coordinates vector, $H\left(\mathbf{X}\right)\in {\mathbb{R}}^{6\times 6}$ is the inertial forces matrix, $b(\mathbf{X},\dot{\mathbf{X}})\in {\mathbb{R}}^6$ is a bias vector containing centripetal, Coriolis and gravity effects, $F_e\in {\mathbb{R}}^6$ is the external forces vector due to interaction with the environment and $F\in {\mathbb{R}}^6$ is the control wrench vector. The control target is to regulate the interaction force between the aerial manipulator and the environment. The following hybrid position/force controller is proposed [33]:

$$F=B(\mathbf{X},\dot{\mathbf{X}})+F_e+K_{f}S(F^d-F_e)+(I-S)K_p({\mathbf{X}}^d-\mathbf{X})+K_d({\dot{\mathbf{X}}}^d-\dot{\mathbf{X}})$$

(3)

where ${\mathbf{X}}^d,{\dot{\mathbf{X}}}^d\in {\mathbb{R}}^6$, are the desired position and velocity vectors, $F^d\in {\mathbb{R}}^6$ is the desired wrench, $K_f,K_p$ and $K_d$ are diagonal $6\times 6$ constant positive definite matrices, I is the $6\times 6$ identity and $S\in {\mathbb{R}}^{6\times 6}$ is a ones and zeros matrix that is useful to switch between position and force control depending on the task. The selected approach is the base of many other controllers found in the literature, which can also be enhanced, according to the requirements of the tasks performed by the aerial manipulators.

Case Study: Hybrid Position/Force Control of a 2-Link Aerial Manipulator

In this example, the control of a planar aerial manipulator is presented. The experiment consists of applying the wrench $F^d$ = (20 N, 0 N, 0 N, 0 Nm, 0 Nm, 0 Nm) at the point ${\mathbf{X}}^d$ = (0.2 m, 0 m, 1.7 m, 0${}^{\circ }$, 0${}^{\circ }$, 0${}^{\circ }$), with respect to the world reference frame, and considering the initial position of the end effector at $\mathbf{X}$(0) = (0 m, 0 m, 1.5 m, 0${}^{\circ }$, 0${}^{\circ }$, 0${}^{\circ }$). The controller used is (3), with a switching condition of $\left|\right|{\mathbf{X}}^d-\mathbf{X}\left|\right|<0.02$ m. In Figure 7, the controlled position of the end effector at $(r^d,z^d)\subset {\mathbf{X}}^d$ is presented, where $r^d=\left|\right|(x^d,y^d)\left|\right|$. Additionally, the forces applied by the end effector along the horizontal plane and vertical axis are also shown. Notice that the force errors and the z position error are 0; the $r-$plane error is not zero, due to the control switch, at the pointed time, when the corresponding condition was reached. The Figure 8 shows the trajectory of the aerial manipulator applying the force at the corresponding point.

5.2. Control Methods for Aerial Manipulation

Different methods exist to control aerial manipulators. Some of them consider decentralized approaches, developing control strategies to actuate the floating base and the manipulation tool as two separate systems [34,35]. Differently, aerial manipulators can also be considered as highly-coupled nonlinear systems. In this context, the dynamic model parameters are usually unknown or difficult to identify; in addition, wind gusts, the motion of the aerial manipulator, and its interaction with the environment produce external disturbances for the aerial floating base. Here, the control problem consists of stabilizing the UAV at a desired pose while the manipulator completes a task, minimizing the effects of external disturbances and the inaccurate representation of its dynamics [36,37]. In the following, some control methods to accomplish aerial manipulation tasks are discussed.

5.2.1. Dynamics and Actuation Properties of Multirotor UAVs

Control strategies for aerial manipulators strictly depend on their controllable space. In this work, we mainly focus on rotary-wings UAVs. The generalized dynamics of these platforms can be expressed as follows: [38]:

$$\left[\begin{array}{cc}m{\mathbf{I}}^{3\times 3}& {\mathbf{0}}^{3\times 3}\\ {\mathbf{0}}^{3\times 3}& \mathbf{J}\end{array}\right]\left[\begin{array}{c}\ddot{\mathbf{p}}\\ \dot{\omega }\end{array}\right]=-\left[\begin{array}{c}mg{\mathbf{e}}_3\\ \omega \times \mathbf{J}\omega \end{array}\right]+\left[\begin{array}{cc}{}^W{\mathbf{R}}_{\mathbf{B}}& {\mathbf{0}}^{3\times 3}\\ {\mathbf{0}}^{3\times 3}& {\mathbf{I}}^{3\times 3}\end{array}\right]u$$

(4)

where m is the mass of the UAV, $\mathbf{J}\in {\mathbb{R}}^{3\times 3}$ is the inertia tensor of the airframe, $\mathbf{p}\in {\mathbb{R}}^3$ is the position of the UAV with respect to the world frame, $\omega \in {\mathbb{R}}^3$ is the angular velocity of the UAV, ${}^W{\mathbf{R}}_{\mathbf{B}}\in \mathcal{SO}\left(3\right)$ is the orientation of the airframe with respect to the world frame, ${\mathbf{0}}^{3\times 3}$ is the zeros matrix of the corresponding dimensions, ${\mathbf{I}}^{3\times 3}$ is the identity matrix of the corresponding dimensions, g is the gravity acceleration, ${\mathbf{e}}_i$ is the canonical basis vector of $i=1,2,3$ and $u\in {\mathbb{R}}^6$ is the control vector in the generalized coordinates. For UAVs with passively or actively tilting rotors, the control vector is statically mapped from the rotors thrust, using the following:

$$u=\mathbf{F}(\alpha ,\varsigma ,L,\sigma )f_r$$

(5)

where $f_r\in {\mathbb{R}}^{2j}$ is the vector containing the vectorized thrusts supplied by the j rotors [39], and $\mathbf{F}(\alpha ,\varsigma ,L,\sigma )\in {\mathbb{R}}^{6\times 2j}$ is the allocation matrix that depends on the following:

• The vector containing the tilting angles of the j rotors about their respective x-axis $\alpha =\left[\begin{array}{c}{\alpha }_1,{\alpha }_2,...,{\alpha }_j\end{array}\right]$.
• The offset angle depending on the $x_B$ axis of the UAV (for example “X” or “+” quadrotor configurations) $\varsigma$.
• The length of the arms of the UAV L.
• The relation between the rotor torque constant $k_{\tau }$ and the rotor force constant $k_f$$\sigma =k_{\tau }/k_f$.

For fully actuated UAVs, i.e., those that are capable of moving in 6D with its number of actuators, there exists the allocation pseudoinverse $\mathbf{F}{(\alpha ,\varsigma ,L,\sigma )}^{†}$ such that $f_r=\mathbf{F}{(\alpha ,\varsigma ,L,\sigma )}^{†}u:{\mathbb{R}}^6\to {\mathbb{R}}^{2j}$. Therefore, the velocity of each rotor can be computed as follows [40]:

$${\omega }_{r_i}=\sqrt{{\displaystyle \frac{\sqrt{f_{r_i}+f_{r_{j+i}}}}{k_f}}},i=1,2,\dots ,j$$

(6)

The tilting angles to exert the desired thrust are given by the following [40]:

$${\alpha }_i=atan2(f_{j+i},|f_i\left|\right),i=1,2,\dots ,j$$

(7)

For the case when the rotors of the UAV are not able to tilt, i.e., $\alpha =0\forall t$, the UAV becomes underactuated since the thrust of the rotors can be only exerted, in a first stage, to control the orientation and the z axis of the airframe; hence, a cascade control is required to lead the UAV to a desired Cartesian position.

5.2.2. On Robust UAV Stabilization

In order to deal with both internal and external disturbances of aerial manipulators, appropriate control techniques must be implemented. First, consider the generalized multirotor UAV dynamics (4) in state-space representation, as follows:

$${\displaystyle \frac{d}{dt}}\left[\begin{array}{c}\mathbf{x}\\ \dot{\mathbf{x}}\end{array}\right]=\left[\begin{array}{c}\dot{\mathbf{x}}\\ -{\mathbf{M}}^{-1}b+{\mathbf{M}}^{-1}\mathbf{R}u+\mathcal{D}\left(t\right)\end{array}\right]$$

(8)

where $\mathbf{x}=\left[\begin{array}{cccc}{\mathbf{p}}^T& \varphi & \theta & \psi \end{array}\right]$, $\mathbf{M}=\left[\begin{array}{cc}m{\mathbf{I}}^{3\times 3}& {\mathbf{0}}^{3\times 3}\\ {\mathbf{0}}^{3\times 3}& \mathbf{J}\end{array}\right]$, $\mathbf{R}=\left[\begin{array}{cc}{}^W{\mathbf{R}}_{\mathbf{B}}& {\mathbf{0}}^{3\times 3}\\ {\mathbf{0}}^{3\times 3}& {\mathbf{I}}^{3\times 3}\end{array}\right]$, $b\in {\mathbb{R}}^6:b=\left[\begin{array}{c}mg{\mathbf{e}}_3\\ \omega \times \mathbf{J}\omega \end{array}\right]$ and $\mathcal{D}\left(t\right)\in {\mathbb{R}}^6$ is the external disturbances vector function, usually unknown but bounded and Lipschitz continuous, or at least with a removable discontinuity. In this context, there are some approaches that use higher order dynamics and the isolation of nonlinearities and disturbances [41]. To illustrate how robust control works on a UAV system, consider it as a 6-dimensional motion unit mass such as the following [42]:

$${\displaystyle \frac{d}{dt}}\left[\begin{array}{c}\mathbf{x}\\ \dot{\mathbf{x}}\end{array}\right]=\left[\begin{array}{c}\dot{\mathbf{x}}\\ u+\mathcal{F}(\mathbf{x},\dot{\mathbf{x}},t)\end{array}\right],$$

(9)

where $0<|\mathcal{F}(\mathbf{x},\dot{\mathbf{x}},t)|\le {\mathcal{F}}_{max}$ is a bounded disturbance term, including both the UAV dynamics and external disturbances. The target is to lead the UAV (9) to a displaced equilibrium ${\mathbf{x}}^d$. Below are presented some techniques to achieve such a target.

5.2.3. Active Disturbance Rejection Control

The main idea of this strategy is to estimate online $\mathcal{F}(\mathbf{x},\dot{\mathbf{x}},t)$ in some accurate manner [43]; thereafter, it is used as part of the feedback control action. The total disturbance can be estimated with a high-order observer such as the following:

$$\left[\begin{array}{c}{\dot{\hat{\mathbf{x}}}}_1\\ {\dot{\hat{\mathbf{x}}}}_2\\ {\dot{\hat{\mathbf{x}}}}_3\end{array}\right]=\left[\begin{array}{c}{\hat{\mathbf{x}}}_2+\epsilon {\Phi }_1\left({\displaystyle \frac{{\mathbf{x}}_1-{\hat{\mathbf{x}}}_1}{{\epsilon }^2}}\right)\\ {\hat{\mathbf{x}}}_3+{\Phi }_2\left({\displaystyle \frac{{\mathbf{x}}_1-{\hat{\mathbf{x}}}_1}{{\epsilon }^2}}\right)+u\\ {\displaystyle \frac{1}{\epsilon }}{\Phi }_3\left({\displaystyle \frac{{\mathbf{x}}_1-{\hat{\mathbf{x}}}_1}{{\epsilon }^2}}\right)\end{array}\right],$$

(10)

where ${\mathbf{x}}_1=\mathbf{x}$ is the measured pose of the UAV, ${\hat{\mathbf{x}}}_i\in {\mathbb{R}}^6$$i=1,2$, are the estimated states of the UAV, ${\hat{\mathbf{x}}}_3\in {\mathbb{R}}^6$ is an estimated augmented state vector that contain the estimated lumped disturbance ${\hat{\mathbf{x}}}_3\triangleq \hat{\mathcal{F}}(\mathbf{x},\dot{\mathbf{x}},t)$, ${\Phi }_i:{\mathbb{R}}^6\to {\mathbb{R}}^6$ are functions that guarantee $|{\mathbf{x}}_1-{\hat{\mathbf{x}}}_1|\to 0$ as $t\to \infty$ and $\epsilon$ is a positive scalar that the smaller it is, the faster (10) converges.

Thus, the feedback control that yields (9) despite disturbances and uncertainty affecting the UAV is the following:

$$u={\mathbf{R}}^{-1}\tilde{\mathbf{M}}\left(u_0-{\hat{\mathbf{x}}}_3\right),$$

(11)

where $\tilde{\mathbf{M}}$ is an approximation of $\mathbf{M}$. Note that, if the mass or the inertia tensor of the UAV are not accurately known, an approximation of such values are suitable to be proposed. If they are completely unknown, a unit value is also a good alternative; the estimation of the total disturbance is even capable of compensating them.

5.2.4. Variable Structure Control

The target is to lead the UAV (9) to a displaced equilibrium ${\mathbf{x}}^d$. Therefore, the following changes of variables are proposed [42]:

$$\begin{array}{c}\hfill \mathbf{z}={\mathbf{x}}^d-\mathbf{x}\end{array}$$

(12)

$$\begin{array}{c}\hfill \dot{\mathbf{z}}={\dot{\mathbf{x}}}^d-\dot{\mathbf{x}}\end{array}$$

(13)

$$\begin{array}{c}\hfill \sigma =\mathbf{C}\mathbf{z}+\dot{\mathbf{z}}\end{array}$$

(14)

where $\sigma \in {\mathbb{R}}^6$ is called the sliding surface and $\mathbf{C}\in {\mathbb{R}}^{6\times 6}$ is a constant diagonal matrix. Now the control target is to reach the origin (0,0). In this context, the following candidate Lyapunov function $V\left(\sigma \right)=\frac{1}{2}{\sigma }^T\sigma$ is proposed; its derivative is given by the following [42]:

$$\dot{V}\left(\sigma \right)={\sigma }^T\dot{\sigma }={\sigma }^T\left(\mathbf{C}\dot{\mathbf{z}}+{\ddot{\mathbf{x}}}^d-u-\mathcal{F}(\mathbf{x},\dot{\mathbf{x}},t)\right)$$

(15)

Using the following controller with acceleration feedforward,

$$u=C\dot{\mathbf{z}}+{\ddot{\mathbf{x}}}^d-K_s\mathsf{sign}\left(\sigma \right)$$

(16)

$\dot{V}$ can be rewritten as follows:

$$\dot{V}\left(\sigma \right)\le \left|\sigma \right|\left({\mathcal{F}}_{max}-K_s\right)$$

(17)

where it can be appreciated that $(\mathbf{z},\dot{\mathbf{z}})\to (0,0)$ as long as $K_s>{\mathcal{F}}_{max}$. The proposed controller (16) forces the motion of $(\mathbf{z},\dot{\mathbf{z}})$ to reach the surface and to remain on it despite the disturbance $\mathcal{F}(\mathbf{x},\dot{\mathbf{x}},t)$, by means of finite-amplitude high-frequency switching along (14) called chattering, which is important to avoid in systems such as DC motors or aircrafts [42] because it might be dangerous for the actuators. Different approaches have been proposed for chattering attenuation [42,44], which are suitable to be implemented for UAVs, specifically higher order SMC, since they conserve robustness and do not necessarily require the derivative of $\mathbf{z}$. In this context, the $u_0$ must be properly regulated (9).

5.2.5. Case Study: Full-Pose Regulation of Passively Tilted Hexarotor

In this example, a passively tilted hexarotor is controlled using a SMC. The proposed controller is as follows:

$$u=K_p\left(\mathbf{z}\right)+K_d\left(\dot{\mathbf{z}}\right)+K_s\mathsf{sign}\left(\sigma \right)$$

(18)

where $K_p,K_d$ and $K_s$ are diagonal matrices with positive constants. The target position is ${\mathbf{x}}^d$ = (1 m, 1 m, 2 m, 0${}^{\circ }$, 0${}^{\circ }$, 10${}^{\circ }$). In Figure 9 the position errors are presented. Notice that all error signals converge to 0; therefore, the full pose of the UAV is regulated.

5.3. Planning Methods for Aerial Manipulation

Devices such as bird diverters have to be installed every 5–10 m. Since the length of each segment of a power line is typically above 150 m, this implies the installation of a large number of devices. Moreover, taking into account the limited payload capacity of the aerial platform, it is expected that the aerial robot has to move frequently to a supply point where the devices are stored. Consequently, it is proposed the application of planning methods that generate the most efficient trajectories in terms of energy consumption for the aerial manipulator in order to maximize the number of installed devices. To the best of the authors’ knowledge, this paper proposes for the first time the use of planning methods for the efficient installation of devices on power lines.

Power lines are linear structures that admit a graph-based representation, where the nodes model the electric towers and the edges model the segments of the power line. These power lines are usually located in open environments with sparse obstacles, such as trees or buildings, but where weather factors, especially wind, play an important role when operating with aerial robots. Taking into account the features above, the planning problem is suitable to be addressed as a vehicle routing problem in which the obstacles are neglected. Later, during the robot operation, these obstacles can be avoided using a collision-avoidance system working at a local level.

Focusing on the formulation of the vehicle routing problem for the aerial manipulation platforms described in Section 4.1, the general approach proposed in this paper is based on covering all the segments of the power line while minimizing the energy consumption. For this, simplified but realistic models of energy consumption are required to define edge costs. These models should consider the hybrid locomotion capabilities of some of the presented aerial manipulators. Therefore, the graph representation of the power line is composed of two kinds of edges. Firstly, rolling edges are associated with the power line. Secondly, flying edges are associated with either aerial transitions between segments/points of the power line or connections of the power line with the supply point. Moreover, the consideration of effects, such as the wind, transforms the graph into a directed one since its edges have asymmetric costs. Finally, constraints such as the maximum robot payload or the battery capacity allow closing the definition of the problem.

Figure 10 shows some preliminary planning results for the installation of clip-type bird diverters on the represented power line, using both MLMP and the DM-C platforms. The problem was solved using linear programming after imposing the following requirements: (a) all the power lines should be covered rolling once, (b) minimizing the total energy consumption, (c) taking into account the maximum robot payload (10 devices for the MLMP and no limits for the DM-C because the aerial platform provides the devices), and (d) visiting the supply point to load the devices when it is needed. The number of diverters to be installed in each segment of the power line is indicated inside the circles in the figure. For these results, no wind was considered and the cost of rolling per unit of distance was fixed to be lower than the cost of flying. As can be seen in the figure, the proposed planning method computes efficient routes that are compatible with the imposed constraints.

6. Human–Aerial Robot Interaction

6.1. Augmented Cognitive Human–Machine Interfaces

The remote control of the aerial manipulator is carried out by the newly developed human–machine interface (HMI). The idea of the new augmented HMI is to create a method by which an operator can control the drone, not through the classic control that uses only hands and fingers, but through the full range of human body movement as well as voice commands. On the other hand, it is planned to provide the user with enhanced feedback that includes not only visual, but also acoustic and haptic information about the state of the vehicle. In addition to functionality, special attention is paid to the ergonomics of the newly developed HMI, which is personalized for each operator [45].

The user gives commands by the movement of the body (see Figure 11), which are detected based on measurements from IMUs (Inertial Measurement Units) placed at specific points on the body. This approach is the extension of the FlyJacket system presented in [46]. Another type of command output is based on the smart gloves worn by the user, which have the ability to measure the pose of the hand and the flexion of each finger. Finally, the voice recognition system is used for issuing direct commands to the drone.

The main feedback to the operator is visual information displayed on the smart glasses, allowing the user to see telemetry and camera information from the drone as well as the drone itself through the transparent lenses. Another feedback channel is based on the acoustic information, which can be used to inform the operator about hazards, the distance to the obstacle or to the ground, but also to follow a previously defined path or trajectory [47]. Tactile feedback is provided by the vibration motors placed at certain points on the operator’s body. A particular motor may start vibrating when the drone is in contact with the manipulated object, but it may also vibrate depending on whether the drone is performing a left–right or up–down movement [48].

6.2. Aerial Co-Worker for Fast Tool Delivery

One application in which the use of aerial manipulators can be particularly useful for the operators working on the power lines is the delivery of tools, such as screwdrivers and pliers, or devices, such as clip-type bird diverters. This is motivated by taking into account the time and effort required to climb to the power lines, and the possibility to lose accidentally part of the equipment during operations. As occurs with other human–robot collaborative tasks [49,50], the close interaction of the aerial robot with the human operator must be safe, especially taking into account the risk due to the high altitude and the difficulty in standing upright in this kind of workspace. Since the proximity of the propellers to the worker will result in an uncomfortable feeling and a sense of danger [51], it is preferable that the handover operation [8,52] is carried out in such a way that the manipulator is separated from the aerial platform. Thus, the proposed solution consists of employing an aerial manipulator in long-reach [8,27] or cable-suspended configurations [14,15], using a cable (around 1.5 m in length) to attach the manipulator to the multirotor base. Figure 12 illustrates the use of a dual-arm aerial manipulator in a long-reach configuration for the delivery of tools (a screwdriver and some pliers) to a human operator working on a line.

Four reference frames are considered in the definition of the operation: the Earth fixed frame $\left\{\mathbf{E}\right\}$ (inertial), the multirotor body frame $\left\{\mathbf{E}\right\}$, the manipulator frame $\left\{\mathbf{0}\right\}$, and the operators head/helmet frame $\left\{\mathbf{H}\right\}$. These are illustrated in Figure 12. In the navigation phase, the aerial robot takes off from the supply point and navigates to the proximity of the power line employing GNSS (Global Navigation Satellite System) sensors, such as GPS, to estimate the position relative to the Earth fixed frame. In the approaching phase, the aerial robot is positioned relative to the operators helmet, using, for example, visual markers on it for its detection and localization [53]. Two constraints should be imposed in the realization of the handover operation: (1) keep a minimum separation distance between the multirotor base and the operator’s helmet for safety, and (2) keep the end effector of the manipulator within the reach of the operator’s hand to ensure a comfortable interaction with the aerial robot.

7. Conclusions

This paper presented the first-year results of the AERIAL-CORE project, focusing on aerial manipulation operations. The AERIAL-CORE project is framed in the Horizon 2020 European research and innovation program and aims to develop new aerial robots able to autonomously perform or assist human operators in the inspection and maintenance of power grid lines. The goal of this work is to introduce the aerial manipulation problem in the context of power line maintenance, describing the general procedure for the installation of bird flight diverters and electrical spacers to outline the requirements in the design of the aerial manipulators. First prototypes of aerial manipulation platforms and end effectors satisfying such requirements were presented along with low-level control techniques and planning methods proposed to achieve safe operation on flight and energy efficiency, taking into account the vast extension of this kind of infrastructure. The paper also covered the interaction of the aerial manipulator with the human operators through the design of appropriate human–machine interfaces, as well as the application of aerial co-workers in aerial tool delivery tasks. Future directions of this work involve the integration of manipulation, control, perception, planning and navigation functionalities in the proposed platforms, and their experimental validation in a real power-line scenario.