Robust Vision-Based Control of a Rotorcraft UAV for Uncooperative Target Tracking

Abstract

This paper investigates the problem of using an unmanned aerial vehicle (UAV) to track and hover above an uncooperative target, such as an unvisited area or an object that is newly discovered. A vision-based strategy integrating the metrology and the control is employed to achieve target tracking and hovering observation. First, by introducing a virtual camera frame, the reprojected image features can change independently of the rotational motion of the vehicle. The image centroid and an optimal observation area on the virtual image plane are exploited to regulate the relative horizontal and vertical distance. Then, the optic flow and gyro measurements are utilized to estimate the relative UAV-to-target velocity. Further, a gain-switching proportional-derivative (PD) control scheme is proposed to compensate for the external interference and model uncertainties. The closed-loop system is proven to be exponentially stable, based on the Lyapunov method. Finally, simulation results are presented to demonstrate the effectiveness of the proposed vision-based strategy in both hovering and tracking scenarios.

1. Introduction

Unmanned aerial vehicles (UAVs) have received growing interest, due to their advantages of vertical take off and landing, rapid maneuverability, and low cost. With improvements in sensing devices, batteries, materials, and other technologies, UAVs have sufficient payload and flight endurance, supporting many applications such as transportation, real-time monitoring, search and rescue, and security and surveillance [1,2,3,4]. There have been a variety of studies related to missions using autonomous hovering and tracking technologies [5,6,7]. In [5], a finite-time controller was proposed to drive a quadrotor hovering above a target with a limited duration. In [6], a novel fuzzy PID-type iterative learning control was developed for trajectory tracking of a quadrotor under the effects of external disturbances and uncertain factors of the system. The problem of energy-efficient path planning and simultaneously anticipating disturbances has been addressed in [7]. However, the available studies have mainly focused on hovering above or tracking a target with a known trajectory and definite position and velocity information. Tracking passive, uncooperative, or even unknown targets, such as vehicles in traffic accidents and fire areas in groves or forests, is still a challenging problem for UAV control. These cases usually occur suddenly and detailed geometric information (e.g., dimensions and size) of the target is not available. On the other hand, rough information (e.g., shape and structure) or a simplified model of the target can be stored onboard, which can help the vehicle to identify and lock on to the target.

A large amount of state of the art sensors has been equipped on the airborne platform, and multi-sensor fusion is a perspective trend in UAV navigation and control [8]. Inertial measurement units (IMUs) and Global Positioning Systems (GPS) are commonly mounted on UAVs to provide position, orientation, and velocity information [9]. However, GPS cannot determine the relative position between the UAV and a target, unless the target’s position information is accurately known. Moreover, the accuracy of GPS is unreliable and ineffective in the presence of obstacles or external disturbances, such as urban canyon effects or electromagnetic interference [10,11]. On the other hand, a large number of UAVs, especially small-scale quadrotors, are equipped with cameras which can provide adequate visual information and act as an alternative option to GPS. The notion of introducing visual information into the control system is referred to as visual servoing [12]. According to different feedback information, visual servoing can be classified into two approaches: position-based visual servoing (PBVS) and image-based visual servoing (IBVS).

PBVS methods rely on relative pose estimation techniques, including the PnP algorithm, epipolar geometry, ICP algorithm, or the fusion of vision and other sensors. In [13], a monocular semi-direct visual odometry algorithm was applied to obtain position information and an adaptive sliding mode controller based on the backstepping technique was presented for quadrotor tracking control. In [14], visual feedback was exploited to estimate the position and attitude of a UAV based on the triangulation method, and a classical PID control was designed for indoor UAV flight. In [15], a vision-based control scheme was proposed for a quadrotor, in order to track a moving target. To compensate for the image dynamic uncertainties, a neural network controller based on a radial basis function was introduced into the closed-loop system. However, the methods mentioned above require a priori knowledge of the target’s geometric model, which does not apply to a newly discovered uncooperative target. Further, 3D reconstruction based on generic features (instead of specific artificial markers) is usually a challenging task, and the sophisticated image processing involved requires long computation times, which may result in delays in the actuation. The IBVS approach, on the other hand, allows for designing a control law directly in the 2D image space and omits the relative pose estimation process. Thus, the geometric information of the target is not required in this approach. The applications of IBVS for UAV control are largely found in recent studies, in which image features have been used as the position cue. To decouple the rotational motion of the vehicle from the change in the image features, several methods have been adopted in IBVS control of a UAV, including spherical projection [16,17], homography-based methods [18,19], virtual spring methods [20,21], and virtual camera methods [22,23]. It is worth noting that the reference image (or image trajectory) is usually designated ahead of the control process, which is typically referred to as the “teach-by-showing” framework. However, this process may not be practical in some specific missions, such as when a UAV is deployed to track a newly discovered target [24].

When tracking an uncooperative target without an accurate geometric model, one effective solution is Simultaneous Localization and Mapping (SLAM). SLAM is a real-time process by which a vehicle computes its localization and simultaneously builds a map of an unknown environment [25]. SLAM techniques are often employed to provide the pose estimation for the PBVS control of the UAV. In [26], a novel estimator based on an Extended Kalman Filter was proposed for multi-sensor fusion, including AHRS, GPS, and a monocular camera. Moreover, a visual SLAM-based scheme has been adopted to provide an accurate estimation of the translational position and velocity, as described in [27]. In [28], a strategy based on visual-inertial SLAM was employed for collision avoidance in a GPS-denied environment. In general, visual SLAM requires long computation times and large amounts of onboard resources, thus typically struggling to provide the rapid pose estimation needed by the UAV guidance system.

This paper presents a simple vision-based strategy to track an uncooperative target which is newly discovered by the UAV. The proposed strategy demonstrates a strict interaction between vision-based metrology and the resulting control. It is assumed that the target’s image is acquired and identified by matching the visual data extracted from the monocular camera with similar information stored on board. Then, a set of image features are well-defined by the image processing unit for the following process. Image centroid is employed for the metrology in the horizontal direction, in spite of the fact that the geometric model is unknown to the UAV. To eliminate the coupling effect of the UAV rotational motion and the change in the image features, we introduce a virtual camera frame. After reprojecting the image features onto the virtual image plane, we can directly estimate the relative horizontal distance between the UAV and the target, based on the perspective projection model. At the same time, the flying height of the UAV can be adjusted by specifying an optimal observation area on the virtual image plane. Motivated by the work in [29], the relative velocity is estimated by a Partial Velocity Evaluation method, which is reliable and computationally effective. Robust methods for the optic flow measurement have been reported in [30,31]. Considering the deviation of the measured mass and inertia matrix from the true values, as well as the disturbances resulting from the wind and other external factors, traditional PID algorithms may not be competent in the UAV controller design, as described in [32,33]. To compensate for the model and disturbance uncertainties and improve robust performance of the system, a gain-switching proportional-derivative (PD) control scheme is proposed in this paper. In this control strategy, the stability and robustness properties of the UAV nonlinear system are guaranteed based on the Lyapunov stability theory. Unlike the sliding mode control, which can be found in [34,35], the transient behavior of the system can be characterized analytically.

The remaining parts of this paper are organized as follows. The problem formulation is given in Section 2, the visual measurement for uncooperative target tracking is discussed thoroughly in Section 3, the gain switching PD controllers are designed in Section 4, the corresponding simulation results are depicted in Section 5 and, finally, the paper is concluded in Section 6.

2. Problem Formulation

The problem addressed in this paper corresponds to UAV tracking and hovering scenarios in which the target is newly discovered. The target, which is referred to as “uncooperative”, lacks accurate dimension/size information and real-time communication with the vehicle. It can be identified by the rough information of generic features stored onboard, such as shape and structure. The UAV studied here is a quadrotor equipped with multiple sensors, including an IMU, an ultrasonic sensor, and a monocular camera. A vision-based strategy integrating the metrology and the resulting control is employed to achieve target tracking and hovering observation. The quadrotor is a typical underactuated mechanism with more degrees of freedom than actuations. To analyze the tracking maneuvers, we first define several reference coordinate frames and present the equations of the motion of the quadrotor, then give an overall control framework of tracking and hovering observation.

2.1. Quadrotor Model

The quadrotor considered in this paper consists of a rigid cross-frame equipped with four rotors, as shown in Figure 1. The two rotors on the diagonal rotate in the same direction, while the adjacent rotors rotate in opposite directions. Six degrees of freedom of the quadrotor’s position and attitude can be achieved by adjusting the rotation speed of the four motors. Two coordinate frames are introduced to describe the equations of motion of the quadrotor. An inertial frame $\mathcal{I}$ is fixed to some point $O_i$ on the earth, with a basis $\{X_i,Y_i,Z_i\}$, whose elements are oriented north, east, and down, respectively. A body-fixed frame $\mathcal{B}$ is assumed to be attached to the center of mass $O_b$ of the quadrotor. The unit vectors of the body-fixed frame are represented by $\{X_b,Y_b,Z_b\}$,which are oriented forward, right, and down, respectively.

Consider a quadrotor with mass m and inertial matrix $\mathit{J}\in {\mathbb{R}}^{3\times 3}$. The translational dynamics in the inertial frame and the rotational dynamics in the body frame are given as follows.

For the translational dynamics,

$$\left\{\begin{array}{c}\dot{\mathit{\xi }}=\mathit{v}\hfill \\ m\dot{\mathit{v}}=-u_1\mathit{R}{\mathit{E}}_3+{\mathit{F}}_g+{\mathit{F}}_d\hfill \end{array}\right.,$$

(1)

and for the rotational dynamics,

$$\left\{\begin{array}{c}\dot{\mathsf{\Phi }}=W\left(\mathsf{\Phi }\right)·\mathit{\omega }\hfill \\ \mathit{J}\dot{\mathit{\omega }}=-\mathit{\omega }\times \mathit{J}\mathit{\omega }+\mathit{\tau }+{\mathit{\tau }}_d\hfill \end{array}\right.,$$

(2)

where $\mathit{\xi }={[x,y,z]}^T$ and $\mathit{v}={[v_x,v_y,v_z]}^T$ are position and linear velocities of the quadrotor, respectively, expressed in the inertial frame; ${\mathit{E}}_3={[0,0,1]}^T$ is the unit vector in the body frame; the attitude of the vehicle, $\mathsf{\Phi }$, is given by three Euler angles $\phi$, $\theta$, and $\psi$ denoting the roll, pitch, and yaw, respectively; and $\mathit{\omega }={[{\omega }_x,{\omega }_y,{\omega }_z]}^T$ is the quadrotor’s angular velocity expressed in the body frame. The corresponding rotation matrix from the body frame to the inertial frame is denoted by $\mathit{R}$, and the matrix associated with the Euler angles and the angular velocity can be written as

$$W\left(\mathsf{\Phi }\right)=\left[\begin{array}{ccc}1& sin\phi tan\theta & cos\phi tan\theta \\ 0& cos\phi & -sin\phi \\ 0& sin\phi /cos\theta & cos\phi /cos\theta \end{array}\right].$$

(3)

Assumption 1.

If the quadrotor does not perform maneuvers that are too aggressive, the roll and pitch angles will both be very small (< ${15}^{\circ }$). Then, the matrix $W\left(\mathsf{\Phi }\right)$ can be replaced with a unit matrix, and $\dot{\phi }$, $\dot{\theta }$, $\dot{\psi }$, $\ddot{\phi }$, $\ddot{\theta }$, $\ddot{\psi }$ can be regarded as approximately equal to ${\omega }_x$, ${\omega }_y$, ${\omega }_z$, ${\dot{\omega }}_x$, ${\dot{\omega }}_y$, ${\dot{\omega }}_z$, respectively.

According to the working principle of the quadrotor (see Figure 1), by adjusting the rotational speed of each group of rotors, the quadrotor can generate a thrust force $u_1$ and torque vectors $\mathit{\tau }={[u_2,u_3,u_4]}^T$, which can be described as

$$\left\{\begin{array}{c}{u}_1=b({\omega }_1^2+{\omega }_2^2+{\omega }_3^2+{\omega }_4^2)\hfill \\ u_2=bl({\omega }_4^2-{\omega }_2^2)\hfill \\ u_3=bl({\omega }_3^2-{\omega }_1^2)\hfill \\ u_4=d({\omega }_2^2+{\omega }_4^2-{\omega }_1^2+{\omega }_3^2)\hfill \end{array}\right.,$$

(4)

where ${\omega }_1$, ${\omega }_2$, ${\omega }_3$, and ${\omega }_4$ are the speeds of four motors, respectively; b and d represent lift and drag coefficients, respectively; and l is the distance from the center of each rotor to the center of mass of the quadrotor.

The gravity vector of the quadrotor is denoted by ${\mathit{F}}_g$. As the quadrotor may be disturbed by the wind and other external factors, some unstructured forces and moments of the translational and rotational dynamics, which are described as ${\mathit{F}}_d$ and ${\mathit{\tau }}_d$, are introduced into the system.

Assumption 2.

The external disturbances ${\mathit{F}}_d$ and ${\mathit{\tau }}_d$ are assumed to be bounded; that is, $∥{\mathit{F}}_d∥\le {\left(F_d\right)}_{max}$ and $∥{\mathit{\tau }}_d∥\le {\left({\tau }_d\right)}_{max}$, where ${\left(F_d\right)}_{max}$ and ${\left({\tau }_d\right)}_{max}$ are positive constants and $∥·∥$ denotes the standard Euclidean vector norm and induced matrix norm.

Assumption 3.

The mass is $m=\overline{m}+\mathsf{\Delta }m$, where $\overline{m}$ and $\mathsf{\Delta }m$ are the nominal and uncertain parts of the mass, respectively. The inertia matrix is $\mathit{J}=\overline{\mathit{J}}+\mathsf{\Delta }\mathit{J}$, where $\overline{\mathit{J}}$ and $\mathsf{\Delta }\mathit{J}$ are the nominal and uncertain parts of the inertia matrix, respectively. $\mathsf{\Delta }m$ and $\mathsf{\Delta }\mathit{J}$ satisfy the inequalities $∥\mathsf{\Delta }m∥\le c_m$ and $∥\mathsf{\Delta }\mathit{J}∥\le c_J$, where $c_m$ and $c_J$ are positive constants.

2.2. Control Framework of Tracking and Hovering Observation

A flowchart of the proposed vision-based UAV tracking strategy is presented as follows (see Figure 2). During the cruise of the UAV, when a sensitive target is captured and identified by the onboard camera, the tracking and hovering observation process starts. First, a set of feature points on the target are extracted. By using the center point of the feature points and their pixel velocities, we can determine the relative distance and velocity, respectively, between the UAV and target. Then, the relative distance and velocity are input into the controller as error states. Based on the gain-switching PD control, the target centroid is expected to coincide with the center of the image plane, and the target image is required to be within the optimal observation area.

3. Visual Measurement Using a Virtual Camera Frame

When a sensitive target is first detected and determined by the camera during the quadrotor flight, its projection on the image plane can be described by a set of parameters, including slope, curvature, area, image centroid, and some parameters related to the shape of the target [36]. The selected target for tracking is assumed to be horizontal to the X${}_i$–Y${}_i$ plane of the inertial frame, and its projection onto the image plane is a compact area with large values of shape parameters, such as sphericity or rectangularity.

3.1. Relative Distance Estimation in Horizontal Direction

As the target is uncooperative and cannot send any position information to the quadrotor, the relative distance measurement relies entirely on the vision-based method. However, the target does not have specific markers and its detailed geometry is unknown to the quadrotor, such that it is challenging to use PnP solvers or Template Matching approaches for relative pose acquisition. In this work, we utilize an effective and simple method based on the image error to estimate the relative horizontal distance between the quadrotor and the target. On the other hand, the quadrotor is an underactuated system with only four independently controllable degrees of freedom. Rolling and translational motions along the Y${}_b$ axis are coupled, as are pitching and translational motions along the X${}_b$ axis, which means that the vehicle will inevitably tilt when maneuvering horizontally. Thus, the image features of the target will change with not only the translational, but also the rotational, motion of the quadrotor, which makes it more complicated to estimate the relative distance and velocity.

Without loss of generality, the camera frame considered in this paper, $\mathcal{C}=\{X_c,Y_c,Z_c\}$, is assumed to coincide with the body frame $\mathcal{B}$. To solve the problem mentioned above, we introduce a virtual camera coordinate frame $\mathcal{V}$, which has the same origin as the frame $\mathcal{B}\left(\mathcal{C}\right)$. The corresponding virtual image plane X${}_v$–Y${}_v$ is always parallel to the X${}_i$–Y${}_i$ plane and with the same yaw angle as the frame $\mathcal{C}$; that is, its roll and pitch angles remain zero (see Figure 3).

The coordinates of a point P in the inertial frame and camera frame are denoted by ${}^i\mathit{P}={{[}^i{P}_x{,}^i{P}_y{,}^i{P}_z]}^T$ and ${}^c\mathit{P}={{[}^c{P}_x{,}^c{P}_y{,}^c{P}_z]}^T$, respectively. They have the following geometric relationship,

$${}^c\mathit{P}={\mathit{R}}^T({}^i\mathit{P}-{{}^i\mathit{O}}_c),$$

(5)

where ${{}^i\mathit{O}}_c$ denotes the coordinates of the origin ${\mathit{O}}_c$ in the inertial frame.

The pixel coordinates of the point P on the image plane are given by perspective projection equations [37]:

$$\left\{\begin{array}{c}u=f\frac{{}^c{P}_x}{{}^c{P}_z}+u_0\hfill \\ n=f\frac{{}^c{P}_y}{{}^c{P}_z}+n_0\hfill \end{array}\right.,$$

(6)

in which f is the focal length of the camera and ${[u_0,n_0]}^T$ is the coordinate of the image plane center.

Now, reproject the image coordinates ${[u,n]}^T$ onto the virtual image plane using a matrix, ${\mathit{R}}_{\theta \phi }$, associated with a rotation in the roll angle around X${}_i$ and in the pitch angle around Y${}_i$:

$$\left[\begin{array}{c}{}^{v}u\\ {}^{v}n\end{array}\right]=\frac{f}{{\overline{\mathit{R}}}_{\theta \phi }^3\mathit{s}}\left[\begin{array}{c}{\overline{\mathit{R}}}_{\theta \phi }^1\mathit{p}\\ {\overline{\mathit{R}}}_{\theta \phi }^2\mathit{p}\end{array}\right],$$

(7)

in which $\mathit{p}={[u,n,f]}^T$ and ${\overline{\mathit{R}}}_{\theta \phi }^1$, ${\overline{\mathit{R}}}_{\theta \phi }^2$, and ${\overline{\mathit{R}}}_{\theta \phi }^3$ are the row vectors of the matrix ${\mathit{R}}_{\theta \phi }$.

It is assumed that $N(N\ge 3)$ non-collinear points are fixed on the selected target. The image centroid of the target is computed as

$${}^v{u}_g=\frac{1}{N}·\sum _{k=1}^N{}^v{u}_k,{}^v{n}_g=\frac{1}{N}·\sum _{k=1}^N{}^v{n}_k.$$

(8)

In this paper, we need to drive the quadrotor directly above the target, such that the desired image feature is determined as the center of the virtual image plane, ${{[}^v{u}_0{,}^v{n}_0]}^T$, as shown in Figure 3. The reprojection of the feature points onto the virtual camera frame decouples the pitch and roll motion of the vehicle through the change of coordinates ${{[}^{v}u{,}^{v}n]}^T$. Therefore, the relative horizontal distance can be estimated directly, using the deviation of the image centroid of the target from the center of the image plane:

$$\mathsf{\Delta }x={}^{v}z\frac{a}{f},\mathsf{\Delta }y={}^{v}z\frac{b}{f},$$

(9)

where $a={}^v{u}_g-{}^v{u}_0$ and $b={}^v{n}_g-{}^v{n}_0$ are image errors defined in image space, and ${}^{v}z$ is the vertical distance of the virtual camera frame obtained by the ultrasonic sensor measurement.

3.2. Relative Distance Estimation in Vertical Direction

When tracking an uncooperative target, the relative horizontal distance estimated in the above section should converge to and remain at zero. The height control is relatively flexible, depending on different observation requirements. The easiest way which comes to our mind is to keep the quadrotor flying at a constant height; however, this cannot guarantee the target being observed in an optimal area on the image plane. Intuitively, the image size of the target varies with the height of the camera. The quadrotor is required to fly neither too high nor too low, avoiding making the target image invisible or leaving the field of view.

Now, introduce a circle as the optimal observation area on the virtual image plane centered at ${\mathit{O}}_1$ with radius $r_{opt}$. Then, construct a circle that passes through the farthest feature point from the image centroid of the target. The radius of the constructed circle, denoted by r, is defined as

$$r=max\left\{\sqrt{{{(}^v{u}_k-{}^v{u}_g)}^2+{{(}^v{n}_k-{}^v{n}_g)}^2}\right\},k=1,...,N.$$

(10)

According to Equation (10), the constructed circle will cover all of the feature points of the selected compact target. Given that the relative horizontal error converges to zero, the vertical distance of the quadrotor can be controlled by adjusting r to be equal to or less than $r_{opt}$, which guarantees the target image being kept in the optimal observation area. Therefore, the desired value of radius r satisfies the following conditions,

$$r_d=\sigma r_{opt},\sigma \le 1,$$

(11)

where $\sigma$ is the radius scaling factor and $\sigma =1$ indicates that the optimal observation area circumscribes the target image.

From Equations (10) and (11), the desired flying height of the quadrotor can be written as

$$z_d={}^{v}z·\frac{r}{r_d}={}^{v}z·\frac{r}{\sigma r_{opt}}.$$

(12)

Then, we can define the position error in the vertical direction as

$$\mathsf{\Delta }z={}^{v}z-z_d.$$

(13)

To control the translational motion of the quadrotor, we define the full position error as $\mathsf{\Delta }\mathit{\xi }={[\mathsf{\Delta }x,\mathsf{\Delta }y,\mathsf{\Delta }z]}^T$, which is desired to converge to ${[0,0,0]}^T$.

3.3. Relative Velocity Estimation

Generally, the control actions require knowledge of not only position error but also velocity error. The quadrotor is equipped with an IMU providing the angular velocity and acceleration of the vehicle. If the target is stationary, the relative velocity can be obtained directly by integrating the measured acceleration information. However, while the precision of the gyro is satisfactory for the needs of the vehicle maneuvers, UAV-equipped accelerometers are usually not accurate enough for evaluating the platform velocity [29]. Velocity error acquirement is more complicated when the uncooperative target is moving. In this paper, we use the optic flow and gyro measurements to estimate the relative quadrotor-to-target velocity, which is recorded by Partial Velocity Evaluation.

Taking the first derivative of Equation (7), we obtain the dynamics of an image feature ${}^v{\mathit{p}}_k={[{}^v{u}_k,{}^v{n}_k]}^T$ [22]:

$${}^v{\dot{\mathit{p}}}_k={\mathit{L}}_{vk}({}^v\mathit{v}-{}^v{\mathit{v}}_{pk})+{\mathit{L}}_{\psi k}\dot{\psi },$$

(14)

in which ${}^v\mathit{v}$ and ${}^v{\mathit{v}}_{pk}$ are the velocities of the point P and the quadrotor expressed in the virtual camera frame, and the matrices ${\mathit{L}}_{vk}$ and ${\mathit{L}}_{\psi k}$ are defined as

$${\mathit{L}}_{vk}=\left[\begin{array}{ccc}-\frac{f}{{}^v{P}_z}& 0& \frac{{}^v{u}_k}{{}^v{P}_z}\\ 0& -\frac{f}{{}^v{P}_z}& \frac{{}^v{n}_k}{{}^v{P}_z}\end{array}\right],{\mathit{L}}_{\psi k}=\left[\begin{array}{c}{}^v{n}_k\\ -{}^v{u}_k\end{array}\right].$$

Define a vector of image features, ${}^v\mathit{p}={[{}^v{\mathit{p}}_1^T,\cdots ,{}^v{\mathit{p}}_N^T]}^T\in {\mathbb{R}}^{2N}$, and assume that the velocities of N points are approximately equal to the target velocity, ${}^v{\mathit{v}}_{pk}={}^v{\mathit{v}}_t$. Then, Equation (14) can be extended as

$${}^v\dot{\mathit{p}}={\mathit{L}}_v({}^v\mathit{v}-{}^v{\mathit{v}}_p)+{\mathit{L}}_{\psi }\dot{\psi },$$

(15)

in which ${\mathit{L}}_v={[{\mathit{L}}_{v1}^T,\cdots ,{\mathit{L}}_{vN}^T]}^T\in {\mathbb{R}}^{2N\times 3}$ and ${\mathit{L}}_{\psi }={[{\mathit{L}}_{\psi 1}^T,\cdots ,{\mathit{L}}_{\psi N}^T]}^T\in {\mathbb{R}}^{2N}$.

The finite time difference of image features, ${}^v\dot{\mathit{p}}$, can be computed by directly measuring the optic flow of the visual features in images. Denoting by $\mathsf{\Delta }\mathit{v}$ the relative quadrotor-to-target velocity expressed in the inertial frame, we have

$$\mathsf{\Delta }\mathit{v}={\mathit{R}}_{\psi }{\mathit{L}}_v^{+}({}^v\dot{\mathit{p}}-{\mathit{L}}_{\psi }\dot{\psi }),$$

(16)

in which ${\mathit{L}}_v^{+}\in {\mathbb{R}}^{3\times 2N}$ is the Moore–Penrose pseudo-inverse of the matrix ${\mathit{L}}_v$ and ${\mathit{R}}_{\psi }$ is the rotation matrix corresponding to the yaw angle by the relation $\mathit{R}={\mathit{R}}_{\psi }{\mathit{R}}_{\theta \phi }$.

The relative translational velocity (expressed in the inertial frame) $\mathsf{\Delta }\mathit{v}$ and the pixel velocities ${}^v\dot{\mathit{p}}$ are related by the interaction matrix ${\mathit{L}}_{vk}$ and ${\mathit{L}}_{\psi k}$. At least two points on the target are required to determine the three components of vector $\mathsf{\Delta }\mathit{v}$. Due to the fact that the virtual image plane is introduced, the depth information of each feature point is not necessary, which is an advantage over traditional visual odometry.

4. Gain-Switching PD Controller Design

The control objective in this work is to drive a quadrotor tracking an uncooperative target at an appropriate height for optimal observation. The quadrotor, as an underactuated system, has six degrees of freedom with only four control inputs. Therefore, the control of a quadrotor is typically divided into an outer position loop and an inner attitude loop. The outer loop provides the reference attitude signal to the inner loop, while the inner loop tracks the orientation reference of the vehicle. A block diagram of the UAV control loop structure is shown in Figure 4. The objective is equivalent to designing a control input for translational motion based on visual feedback, such that the controlled underactuated system with model uncertainties can guarantee $\mathsf{\Delta }\mathit{\xi }\to \mathbf{0}$ and $\mathsf{\Delta }\mathit{v}\to \mathbf{0}$, then designing a control input for rotational motion with the knowledge of reference Euler angles derived from the outer loop.

PD controllers are commonly used in UAV control. Given the uncertainties in the dynamics of the vehicle, a traditional PD controller can produce large steady-state errors or even affect the overall stability of the system. To improve the robustness of the PD controller, we introduce a gain-switching term to deal with the uncertainties by avoiding using large gains.

4.1. Control of the Translational Motion

The translational dynamics (1) can be rewritten, in matrix form, as

$$\left[\begin{array}{c}\ddot{x}\\ \ddot{y}\\ \ddot{z}\end{array}\right]=-\frac{u_1}{m}\left[\begin{array}{c}cos\phi sin\theta cos\psi +sin\phi sin\psi \\ cos\phi sin\theta sin\psi -sin\phi cos\psi \\ cos\phi cos\theta \end{array}\right]+\left[\begin{array}{c}0\\ 0\\ g\end{array}\right]+\left[\begin{array}{c}{f}_{dx}\\ f_{dy}\\ f_{dz}\end{array}\right].$$

(17)

The translational motion of the quadrotor is actually accomplished by both the thrust and orientation of the body. Let us define a set of virtual control inputs for the translational motion,

$$\left\{\begin{array}{c}{u}_x=u_1(cos\phi sin\theta cos\psi +sin\phi sin\psi )\hfill \\ u_y=u_1(cos\phi sin\theta sin\psi -sin\phi cos\psi )\hfill \\ u_z=u_1(cos\phi cos\theta )\hfill \end{array}\right..$$

(18)

From Equations (17) and (18), we have

$$m\dot{\mathit{v}}=\mathit{u}+{\mathit{F}}_g+{\mathit{F}}_d.$$

(19)

The control law for the translational motion is proposed as

$$\mathit{u}={\mathit{u}}_R+{\mathit{u}}_P+{\mathit{u}}_D+{\mathit{u}}_{\epsilon }=-{\overline{\mathit{F}}}_g+(-{\mathit{K}}_P\overline{m}\mathsf{\Delta }\mathit{\xi })+(-{\mathit{K}}_D\overline{m}\mathsf{\Delta }\mathit{v})+(-{\mathit{K}}_{\epsilon }\overline{m}\mathit{n}(\epsilon ,\mathit{s})),$$

(20)

where quantities with overbar symbols indicate that a priori estimates are used, which may deviate from the true value; $\mathsf{\Delta }\mathit{\xi }$ and $\mathsf{\Delta }\mathit{v}$ are obtained by visual measurement, as detailed in Section 3; $-{\overline{\mathit{F}}}_g$ is the model compensation term designed to eliminate nonlinear elements of the dynamics; and ${\mathit{K}}_P$, ${\mathit{K}}_D$, and ${\mathit{K}}_{\epsilon }$ represent the proportional, differential, and gain-switching coefficient matrices, respectively. The switching function $\mathit{n}(\epsilon ,\mathit{s})$ can be any piecewise-continuous function with the following properties,

$$\left\{\begin{array}{c}∥\mathit{s}∥\mathit{n}(\epsilon ,\mathit{s})=∥\mathit{n}(\epsilon ,\mathit{s})∥\mathit{s}\hfill \\ ∥\mathit{n}(\epsilon ,\mathit{s})∥\ge 1-\epsilon /\phantom{\epsilon \mathit{s}}\mathit{s},\mathit{s}\ne 0\hfill \end{array}\right..$$

(21)

In this paper, the switching function is chosen as

$$\mathit{n}=\left\{\begin{array}{c}-\mathit{s}/\phantom{\mathit{r}\epsilon ,∥\mathit{r}∥<\epsilon }\epsilon ,∥\mathit{s}∥<\epsilon \hfill \\ -\mathit{s}/\phantom{\mathit{s}∥\mathit{s}∥}∥\mathit{s}∥,∥\mathit{s}∥\ge \epsilon ,\hfill \end{array}\right.$$

(22)

where $\epsilon$ is the gain switching threshold and $\mathit{s}$ is the error feedback term given by

$$\mathit{s}=k_P\mathsf{\Delta }\mathit{\xi }+k_D\mathsf{\Delta }\mathit{v}.$$

(23)

Substituting Equation (20) into (19) yields the error dynamics for the translational motion:

$$m\mathsf{\Delta }\dot{\mathit{v}}=-{\mathit{K}}_{P}m\mathsf{\Delta }\mathit{\xi }-{\mathit{K}}_{D}m\mathsf{\Delta }\mathit{v}-{\mathit{K}}_{\epsilon }\mathit{n}+\mathit{h},$$

(24)

where the disturbance function, denoted by $\mathit{h}$, can be written as:

$$\mathit{h}=-{\mathit{K}}_P\delta m\mathsf{\Delta }\mathit{\xi }-{\mathit{K}}_D\delta m\mathsf{\Delta }\mathit{v}-\delta {\mathit{F}}_g-m\dot{{\mathit{v}}_t}+{\mathit{F}}_d,$$

(25)

where $\delta m=\overline{m}-m$ and $\delta {\mathit{F}}_g={\overline{\mathit{F}}}_g-{\mathit{F}}_g$ denote the deviation between the measured values and the true values, $\dot{{\mathit{v}}_t}$ is the target acceleration, which is assumed to be unknown but has an upper bound.

Remark 1.

The disturbance function $\mathit{h}$ represents all sources of uncertainties in translational error dynamics, including model uncertainty resulting from the inaccurately measured mass of the quadrotor $-{\mathit{K}}_P\delta m\mathsf{\Delta }\mathit{\xi }$ and $-{\mathit{K}}_D\delta m\mathsf{\Delta }\mathit{v}$, gravitational error $-\delta {\mathit{F}}_g$, unknown target motion $-m\dot{{\mathit{v}}_t}$, and unstructured forces ${\mathit{F}}_d$. Under Assumption 2–3, the disturbance function $\mathit{h}$ is bounded.

Now, the properties of the gain-switching coefficient will be addressed. The matrix ${\mathit{K}}_{\epsilon }$ is symmetric and positive definite, and must satisfy the following conditions,

$$\left\{\begin{array}{c}∥{\mathit{K}}_{\epsilon }\mathit{n}∥/\phantom{∥{\mathit{K}}_{\epsilon }\mathit{n}∥(\overline{m}+c_m)}(\overline{m}+c_m)\ge k_{\epsilon }∥\mathit{n}∥\hfill \\ k_{\epsilon }\ge ∥\mathit{h}∥/\phantom{∥\mathit{h}∥(\overline{m}+c_m)}(\overline{m}+c_m)\hfill \end{array}\right.,$$

(26)

where $k_{\epsilon }$ is a positive bounding scalar designated to ensure that the term ${\mathit{u}}_{\epsilon }$ provides greater acceleration than that resulting from the disturbances in $\mathit{h}$.

The proportional and differential coefficient matrices ${\mathit{K}}_P$ and ${\mathit{K}}_D$ are selected to satisfy the following constraints with $k_P$ and $k_D$,

$$\left\{\begin{array}{c}∥{\mathit{K}}_P\overline{m}\mathsf{\Delta }\mathit{\xi }∥/\phantom{∥{\mathit{K}}_P\overline{m}\mathsf{\Delta }\mathit{\xi }∥(\overline{m}+c_m)}(\overline{m}+c_m)\ge k_P∥\mathsf{\Delta }\mathit{\xi }∥\hfill \\ ∥{\mathit{K}}_D\overline{m}\mathsf{\Delta }\mathit{v}∥/\phantom{∥{\mathit{K}}_D\overline{m}\mathsf{\Delta }\mathit{v}∥(\overline{m}+c_m)}(\overline{m}+c_m)\ge k_D∥\mathsf{\Delta }\mathit{v}∥\hfill \end{array}\right.,$$

(27)

in which $k_P$ and $k_D$ are feedback gains designated as the following functions of the desired rate of convergence, $\alpha$:

$$\left\{\begin{array}{c}{k}_P=2{\alpha }^2\hfill \\ k_D=2\alpha \hfill \end{array}\right.,\alpha >0.$$

(28)

These equations guarantee that the norm of ${\mathit{K}}_P$ and ${\mathit{K}}_D$ is large enough, such that the commanded force specified by ${\mathit{u}}_P$ and ${\mathit{u}}_D$ delivers—at a minimum—the acceleration specified by the vector ${\mathit{u}}_{\epsilon }$.

Remark 2.

Compared with traditional PD control, the proposed PD control consists of not only the proportional and differential terms ${\mathit{u}}_P$ and ${\mathit{u}}_D$, which can eliminate the position and velocity errors, but also a gain-switching term ${\mathit{u}}_{\epsilon }$, which acts as a robustifying term based on the switching function $\mathit{n}(\epsilon ,\mathit{s})$. By using Equation (26), appropriate robust control parameters can be selected to restrain the uncertainties $\mathit{h}$. In addition, the transient behavior of the system can be characterized analytically, based on the desired rate of convergence in feedback gains (28).

Referring to Equation (24) and defining the system state ${\mathit{e}}_t={\left[\mathsf{\Delta }{\mathit{\xi }}^T,{\mathsf{\Delta }\mathit{v}}^T\right]}^T$, the tracking dynamics model for translational motion can be written as

$$\left[\begin{array}{c}\mathsf{\Delta }\dot{\mathit{\xi }}\\ \mathsf{\Delta }\dot{\mathit{v}}\end{array}\right]=\left[\begin{array}{c}\mathsf{\Delta }\mathit{v}\\ \left(\frac{1}{m}\right)·(-{\mathit{K}}_{P}m\mathsf{\Delta }\mathit{\xi }-{\mathit{K}}_{D}m\mathsf{\Delta }\mathit{v}-{\mathit{K}}_{\epsilon }\mathit{n}(\epsilon ,\mathit{s})+\mathit{h})\end{array}\right].$$

(29)

Lemma 1.

Consider a system $\dot{\mathit{x}}\left(t\right)=f(t,\mathit{x}\left(t\right))$. If there exist a continuously differentiable function $V\left(\mathit{x}\right)$ and scalars $c_1,c_2>0$, which satisfy [38]

$$\begin{array}{c}\left(\mathrm{i}\right)c_1{∥\mathit{x}∥}^2\le V\left(\mathit{x}\right)\le c_2{∥\mathit{x}∥}^2\hfill \\ \left(\mathrm{ii}\right)\forall \mathit{x}\in \left\{\mathit{x}\right|V\left(\mathit{x}\right)>V^{*},V^{*}>0\},\hfill \\ \exists \dot{V}\left(\mathit{x}\right)\le -2\alpha [V\left(\mathit{x}\right)-V^{*}],\hfill \end{array}$$

then the system is (globally and uniformly) exponentially convergent to $\mathit{B}\left(q\right)$ with rate α, where

$$\begin{array}{c}\mathit{B}\left(q\right)\stackrel{\mathsf{\Delta }}{=}\{\mathit{x}\in {\mathbb{R}}^n:∥\mathit{x}∥\le q\}\hfill \\ q\stackrel{\mathsf{\Delta }}{=}{\left(V^{*}/\phantom{V^{*}{c}_1}{c}_1\right)}^{1/2}.\hfill \end{array}$$

Theorem 1.

Consider the closed-loop system for the translational motion in Equation (29) with the controller designed by Equation (20). If the corresponding parameters are assigned as in Equations (26)–(28), then the state ${\mathit{e}}_t$ will exponentially converge to zero.

Proof. 

The Lyapunov function candidate is designated as

$$V\left({\mathit{e}}_t\right)=\frac{1}{2}{\mathit{e}}_t^T\mathit{A}{\mathit{e}}_t,$$

(30)

where

$$\mathit{A}=\left[\begin{array}{cc}{k}_L\mathit{I}& k_P\mathit{I}\\ k_P\mathit{I}& k_D\mathit{I}\end{array}\right]$$

(31)

and $\mathit{I}$ is a unit matrix. The constant parameter $k_L$ is also a function of $\alpha$:

$$k_L=4{\alpha }^3.$$

(32)

By using the values $k_P$, $k_D$, and $k_L$ in Equations (28) and (32), the matrix $\mathit{A}$ is positive definite. Thus, $V\left({\mathit{e}}_{\mathit{t}}\right)$ is a positive definite function satisfying

$${\lambda }_{min}{∥{\mathit{e}}_{\mathit{t}}∥}^2\le V\left({\mathit{e}}_{\mathit{t}}\right)\le {\lambda }_{max}{∥{\mathit{e}}_{\mathit{t}}∥}^2.$$

(33)

Condition $\left(i\right)$ in Lemma 1 is satisfied with $c_1={\lambda }_{min}$ and $c_2={\lambda }_{max}$, where ${\lambda }_{min}$ and ${\lambda }_{max}$ are the smallest and largest eigenvalues of $\mathit{A}$.

Taking the time derivative of $V\left({\mathit{e}}_t\right)$ and substituting Equation (29) into it, we have

$$\begin{array}{cc}\hfill \dot{V}({\mathit{e}}_{\mathit{t}})=& k_L\mathsf{\Delta }{\mathit{\xi }}^T\mathsf{\Delta }\mathit{v}+k_P\mathsf{\Delta }{\mathit{v}}^T\mathsf{\Delta }\mathit{v}-k_P[\mathsf{\Delta }{\mathit{\xi }}^T{\mathit{K}}_P\mathsf{\Delta }\mathit{\xi }+\mathsf{\Delta }{\mathit{\xi }}^T{\mathit{K}}_D\mathsf{\Delta }\mathit{v}]-\hfill \\ \hfill & k_D[\mathsf{\Delta }{\mathit{v}}^T{\mathit{K}}_D\mathsf{\Delta }\mathit{v}+\mathsf{\Delta }{\mathit{v}}^T{\mathit{K}}_P\mathsf{\Delta }\mathit{\xi }]+\left(1/\phantom{1m}m\right){\mathit{s}}^T(\mathit{h}-{\mathit{K}}_{\epsilon }\mathit{n}(\epsilon ,\mathit{s})).\hfill \end{array}$$

(34)

Using the parameter constraints in Equations (26) and (27), it follows that

$$\begin{array}{cc}\hfill \dot{V}\left(\mathit{x}\right)\le & -k_P^2\mathsf{\Delta }{\mathit{\xi }}^T\mathsf{\Delta }\mathit{\xi }+(k_P-k_D^2)\mathsf{\Delta }{\mathit{v}}^T\mathsf{\Delta }\mathit{v}+(k_L-2k_P{k}_D)\mathsf{\Delta }{\mathit{\xi }}^T\mathsf{\Delta }\mathit{v}+\hfill \\ \hfill & {\mathit{s}}^T[\left(1/\phantom{1m}m\right)\mathit{h}-k_{\epsilon }\mathit{n}(\epsilon ,\mathit{s})].\hfill \end{array}$$

(35)

Substituting Equations (28) and (32) into (35) leads to the following,

$$\dot{V}\left({\mathit{e}}_t\right)\le -2\alpha V\left({\mathit{e}}_t\right)+E,$$

(36)

where

$$E={\mathit{s}}^T[\left(1/\phantom{1m}m\right)\mathit{h}-k_{\epsilon }\mathit{n}(\epsilon ,\mathit{s})].$$

(37)

By using the properties of $\mathit{n}$ in (21), Equation (37) yields the following expression,

$$E\le ∥\mathit{s}∥h^{*}-k_{\epsilon }∥\mathit{n}∥∥\mathit{s}∥,$$

(38)

where $h^{*}=∥\mathit{h}∥/\phantom{∥\mathit{h}∥m}m$.

Equation (22) requires that $\mathit{n}=\mathit{s}/\phantom{\mathit{s}\epsilon }\epsilon$ for $∥\mathit{s}∥<\epsilon$. Moreover, recalling (26), which states that $k_{\epsilon }\ge h^{*}$, Equation (38) shows, in this case, that

$$E\le k_{\epsilon }·\mathsf{\Delta },$$

(39)

where $\mathsf{\Delta }=({\epsilon }^2-{∥\mathit{s}∥}^2)/\phantom{({\epsilon }^2-{∥\mathit{w}∥}^2)\epsilon }\epsilon >0$.

When $∥\mathit{s}∥\ge \epsilon$, $\mathit{n}=-\mathit{s}/\phantom{\mathit{s}∥\mathit{s}∥}∥\mathit{s}∥$, it follows that

$$E\le 0.$$

(40)

Combining Equations (39) and (40), we can obtain a global upper bound

$$\overline{E}\le k_{\epsilon }·\mathsf{\Delta }.$$

(41)

Substituting Equation (41) into the time derivative of $\dot{V}\left({\mathit{e}}_t\right)$ in Equation (38) leads to

$$\dot{V}\left({\mathit{e}}_t\right)\le -2\alpha [V\left({\mathit{e}}_t\right)-V^{*}],$$

(42)

in which $V^{*}=\overline{E}/\phantom{\overline{E}2\alpha }2\alpha$.

Inequality (42) ensures that the state of the system ${\mathit{e}}_t$ can exponentially converge to a small ball around the origin defined by $V\left({\mathit{e}}_t\right)<V^{*}$. Thus, the condition $\left(ii\right)$ of Lemma 1 is satisfied. □

It is worth noting that the control law given by Equation (20) yields virtual control inputs for translational motion. Using Equation (18), we can compute the command thrust $u_1$, as well as the desired roll and pitch angles for the attitude controller:

$$\left\{\begin{array}{c}{u}_1=\sqrt{u_x^2+u_y^2+u_z^2}\hfill \\ {\phi }_d=arcsin\left(\frac{u_{x}sin{\psi }_d-u_{y}cos{\psi }_d}{u_1}\right)\hfill \\ {\theta }_d=arctan\left(\frac{u_{x}cos{\psi }_d+u_{y}sin{\psi }_d}{u_z}\right)\hfill \end{array}\right.,$$

(43)

where ${\psi }_d$ is the reference value of yaw. As the yaw angle is independent of the outer loop, we can prescribe it as its initial value or another constant.

4.2. Control of the Rotational Motion

In the inner attitude loop, a similar gain-switching PD control law is presented:

$$\begin{array}{cc}\hfill \mathit{\tau }=& {\mathit{\tau }}_R+{\mathit{\tau }}_P+{\mathit{\tau }}_D+{\mathit{\tau }}_{\epsilon }\hfill \\ \hfill =& \mathit{\omega }\times \overline{\mathit{J}}\mathit{\omega }+(-{\mathit{K}}_P^{\prime }\overline{\mathit{J}}\mathsf{\Delta }\mathsf{\Phi })+(-{\mathit{K}}_D^{\prime }\overline{\mathit{J}}\mathsf{\Delta }\mathit{\omega })+(-{\mathit{K}}_{\epsilon }^{\prime }\overline{\mathit{J}}{\mathit{n}}^{\prime }({\epsilon }^{\prime },{\mathit{s}}^{\prime })),\hfill \end{array}$$

(44)

where $\mathsf{\Delta }\mathsf{\Phi }=\mathsf{\Phi }-{\mathsf{\Phi }}_d$ and $\mathsf{\Delta }\mathit{\omega }=\mathit{\omega }-{\mathit{\omega }}_d$ denote the quadrotor attitude and the equivalent rate error, respectively, and ${\mathsf{\Phi }}_d={[{\phi }_d,{\theta }_d,{\psi }_d]}^T$ are the desired Euler angles output from the outer loop. The desired angular velocity ${\mathit{\omega }}_d$ is taken to be zero.

The switching function ${\mathit{n}}^{\prime }({\epsilon }^{\prime },{\mathit{s}}^{\prime })$ is chosen the same as before, and the error feedback term ${\mathit{s}}^{\prime }$ is given by

$${\mathit{s}}^{\prime }=-k_P^{\prime }\mathsf{\Delta }\mathsf{\Phi }-k_D^{\prime }\mathsf{\Delta }\mathit{\omega }.$$

(45)

Referring to Equation (2) and defining the system states ${\mathit{e}}_r={\left[\mathsf{\Delta }{\mathsf{\Phi }}^T,{\mathsf{\Delta }\mathit{\omega }}^T\right]}^T$, the tracking dynamics model for rotational motion, under Assumption 1, can be written as

$$\left[\begin{array}{c}\mathsf{\Delta }\dot{\mathsf{\Phi }}\\ \mathsf{\Delta }\dot{\mathit{\omega }}\end{array}\right]=\left[\begin{array}{c}\mathsf{\Delta }\mathit{\omega }\\ {\mathit{J}}^{-1}(-{\mathit{K}}_P^{\prime }\mathit{J}\mathsf{\Delta }\mathsf{\Phi }-{\mathit{K}}_D^{\prime }\mathit{J}\mathsf{\Delta }\mathit{\omega }-{\mathit{K}}_{\epsilon }^{\prime }\mathit{J}{\mathit{n}}^{\prime }({\epsilon }^{\prime },{\mathit{s}}^{\prime })+{\mathit{h}}^{\prime })\end{array}\right],$$

(46)

where the disturbance function ${\mathit{h}}^{\prime }$ is given by

$${\mathit{h}}^{\prime }=-{\mathit{K}}_P^{\prime }\delta \mathit{J}\mathsf{\Delta }\mathsf{\Phi }-{\mathit{K}}_D^{\prime }\delta \mathit{J}\mathsf{\Delta }\mathit{\omega }+(\mathit{\omega }\times \overline{\mathit{J}}\mathit{\omega }-\mathit{\omega }\times \mathit{J}\mathit{\omega })-\mathit{J}{\dot{\mathit{\omega }}}_t+{\mathit{\tau }}_d.$$

(47)

Theorem 2.

Consider the closed-loop system for the rotational motion in Equation (46) with the controller designed by Equation (44). If the corresponding parameters are assigned by

$$\left\{\begin{array}{c}∥{\mathit{K}}_P^{\prime }\overline{\mathit{J}}\mathsf{\Delta }\mathsf{\Phi }∥/\phantom{∥{\mathit{K}}_P^{\prime }\overline{\mathit{J}}\mathsf{\Delta }\mathsf{\Phi }∥{\lambda }_{max}\left(\mathit{J}\right)}{\lambda }_{max}\left(\mathit{J}\right)\ge k_P^{\prime }∥\mathsf{\Delta }\mathsf{\Phi }∥\hfill \\ ∥{\mathit{K}}_D^{\prime }\overline{\mathit{J}}\mathsf{\Delta }\mathit{\omega }∥/\phantom{∥{\mathit{K}}_D^{\prime }\overline{\mathit{J}}\mathsf{\Delta }\mathit{\omega }∥{\lambda }_{max}\left(\mathit{J}\right)}{\lambda }_{max}\left(\mathit{J}\right)\ge k_D^{\prime }∥\mathsf{\Delta }\mathit{\omega }∥\hfill \end{array}\right.,$$

(48)

$$\left\{\begin{array}{c}∥{{\mathit{K}}^{\prime }}_{\epsilon }∥/\phantom{∥{{\mathit{K}}^{\prime }}_{\epsilon }∥{\lambda }_{max}\left(\mathit{J}\right)}{\lambda }_{max}\left(\mathit{J}\right)\ge k_{\epsilon }^{\prime }∥{\mathit{n}}^{\prime }({\epsilon }^{\prime },{\mathit{s}}^{\prime })∥\hfill \\ k_{\epsilon }^{\prime }\ge ∥{\mathit{h}}^{\prime }∥/\phantom{∥{\mathit{h}}^{\prime }∥{\lambda }_{min}\left(\mathit{J}\right)}{\lambda }_{min}\left(\mathit{J}\right)\hfill \end{array}\right.,$$

(49)

$$\left\{\begin{array}{c}{k}_P^{\prime }=2{{\alpha }^{\prime }}^2\hfill \\ k_D^{\prime }=2{\alpha }^{\prime }\hfill \end{array}\right.,$$

(50)

then, the state ${\mathit{e}}_r$ will exponentially converge to zero.

Proof. 

The stability analysis of the system for rotational motion is similar to that for translational motion, so it is not described in detail here. □

5. Simulation Results

In this section, MATLAB simulations are presented to validate the performance of the proposed vision-based control scheme. We considered two scenarios in this work: hovering above a stationary target and tracking a moving target. In both scenarios, the target to be observed was assumed to be uncooperative and without detailed geometry information. The physical parameters of the simulated quadrotor were $m=2.1$ $\mathrm{kg}$ and $\mathit{J}=\mathrm{diag}\{0.0096,0.0098,0.016\}$ $\mathrm{kg}·{\mathrm{m}}^2/\mathrm{rad}$. The nominal part of the quadrotor’s mass and moment of inertia were $\overline{m}=2.1$ $\mathrm{kg}$ and $\overline{\mathit{J}}=\mathrm{diag}\{0.0081,0.0081,0.0142\}$ $\mathrm{kg}·{\mathrm{m}}^2/\mathrm{rad}$. The focal length divided by pixel size of the camera was set as 213, and the image resolution was $160\times 120$ pixels with principal point located at $[80,60]$. Considering that the quadrotor may be affected by wind disturbances in the environment, we applied some sinusoidal (cosinusoidal) forces and torques to the vehicle with the following values,

$$\begin{array}{c}{\mathit{F}}_{\mathrm{d}}=0.5·{\left[\begin{array}{ccc}sin(2\pi \mathrm{t})& cos(\pi \mathrm{t})& -sin(2\pi \mathrm{t})\end{array}\right]}^T\hfill \\ {\mathit{\tau }}_{\mathrm{d}}=0.01·{\left[\begin{array}{ccc}-sin(2\pi \mathrm{t})& -cos(\pi \mathrm{t})& sin(2\pi \mathrm{t})\end{array}\right]}^T\hfill \end{array}.$$

5.1. Scenario 1: Hovering Observation

In some cases, such as traffic and fire accidents, we usually drive a UAV equipped with optical sensors to approach the scene of the accident and hover over the damaged vehicle or burning object. To provide distinct images and effective air support for the subsequent rescue, it is required to adjust the height of the UAV, keeping the designated target within the optimal area of the camera’s field of view. The proposed vision-based control scheme in this work is applicable to the above missions.

In the simulation, the target was set as a rectangular object. Its visual features included its four vertexes with the following coordinates, $[0.2,0.25,0]$ m, $[0.2,0.25,0]$ m, $[-0.2,-0.25,0]$ m, and $[0.2,-0.25,0]$ m, all expressed in the inertial frame. The quadrotor initial position and attitude were $[1,0.8,-4]$ m and $[0,0,0.2]$ rad, respectively. For the optimal observation of the target, we defined a reference circle of radius 40 pixels on the virtual image plane, and the radius scaling factor $\sigma$ was designated to be 1. The corresponding desired height was 1.7048 m, which can be computed using Equation (12). The control gains used in the simulation are listed in

Table 1. Control gains in the simulation.

Gains	Values	Gains	Values
$k_P$	1	$k_P^{\prime }$	8
$k_D$	1.44	$k_D^{\prime }$	4
$k_{\epsilon }$	0.3	$k_{\epsilon }^{\prime }$	1
${\mathit{K}}_P$	diag{1.2,1.2,1.3}	${\mathit{K}}_P^{\prime }$	diag{10,10,10}
${\mathit{K}}_D$	diag{1.5,1.5,1.6}	${\mathit{K}}_D^{\prime }$	diag{5,5,5}
${\mathit{K}}_{\epsilon }$	$m·$ diag{0.3,0.3,0.5}	${\mathit{K}}_{\epsilon }^{\prime }$	$k_{\epsilon }^{\prime }·\mathit{J}$
$\epsilon$	0.1	${\epsilon }^{\prime }$	0.3

.

The transient response of the system mainly depends on the feedback gains $k_P$, $k_D$ and $k_P^{\prime }$, $k_D^{\prime }$, which are determined by the constraint Equations (28) and (50). Based on the trial and error method, the desired rates of convergence for translational and rotational motion were selected as $\sqrt{2}/2$ and 2, respectively. Then, the proportional and differential matrices ${\mathit{K}}_P$, ${\mathit{K}}_D$ and ${\mathit{K}}_P^{\prime }$, ${\mathit{K}}_D^{\prime }$ could be determined by using Equations (27) and (48). To guarantee the robustness properties of the system in the presence of wind and modelling errors, the control gains in the gain-switching term $k_{\epsilon }$, ${\mathit{K}}_{\epsilon }$ and $k_{\epsilon }^{\prime }$, ${\mathit{K}}_{\epsilon }^{\prime }$ must be selected such that they are larger than the effect resulting from the unstructured disturbances $\mathit{h}$, ${\mathit{h}}^{\prime }$, based on Equations (26) and (49). The gain switching thresholds $\epsilon$ and ${\epsilon }^{\prime }$ were tuned repeatedly to restrain the chattering phenomenon of the control efforts.

The simulation results are illustrated in Figure 5, Figure 6 and Figure 7. The translational motion of the quadrotor is shown in Figure 5. The horizontal position errors converged to less than 0.05 m with transient response time enduring at about 5 s. The error in the vertical direction was relatively large, about 0.12 m, mainly resulting from the uncertainty of the quadrotor mass. Figure 6 describes the rotational motion of the quadrotor. The Euler angles were kept within a small range, satisfying Assumption 1. The properties of the target centroid in image space are illustrated in Figure 7. It is shown that the vehicle could rapidly and smoothly approach the target, and the control accuracy of the target centroid on the image plane was kept within 5 pixels. The proposed system, therefore, is satisfactory for continuous hovering observation of the target.

For comparison, we ran a simulation of traditional PD control using the same initial conditions and gains. The results are reported in Figure 8 and Figure 9. The results indicate that the proposed gain-switching PD controller had better performance than the traditional PD controller in steady-state performance, with lower oscillation, and the proposed method had a smaller error limit of the target centroid in image space. Further, we relocated the quadrotor at two different initial positions to compare the system steady-state behavior between the gain-switching PD controller with the traditional PD controller. The mean, amplitude, and accuracy ($3\sigma$) of the position error were illustrated in

Table 2. The steady-state performance (position error) of the two methods.

	Traditional PD Control			Gain switching PD Control
	Mean	Amplitude	Accuracy	Mean	Amplitude	Accuracy
Initial Position (m) $[1,0.8,-4]$	8.9 ×${10}^{-4}$	0.1475	0.3209	3.7 ×${10}^{-4}$	0.0290	0.0628
	−0.0051	0.1360	0.2985	1.5 ×${10}^{-4}$	0.0345	0.0731
	−0.5211	0.0164	0.5495	−0.1323	0.0082	0.1484
Initial Position (m) $[1,2,-5]$	9.0 ×${10}^{-4}$	0.1475	0.3209	3.7 ×${10}^{-4}$	0.0291	0.0628
	−0.0051	0.1360	0.2984	1.6 ×${10}^{-4}$	0.0345	0.0731
	−0.5212	0.0165	0.5495	−0.1324	0.0083	0.1485
Initial Position (m) $[0.6,0.8,-1.5]$	8.8 ×${10}^{-4}$	0.1475	0.3208	3.7 ×${10}^{-4}$	0.0290	0.0627
	−0.0052	0.1360	0.2985	1.6 ×${10}^{-4}$	0.0345	0.0731
	−0.5211	0.0164	0.5494	−0.1322	0.0082	0.1483

. The results showed that the proposed controller was effective in different initial conditions. Compared with the traditional PD controller, the performance of the proposed controller was greatly improved, with the position-control accuracy increasing by a factor of 4–5.

To evaluate the imaging effects of the target and the corresponding flying height of the quadrotor under different observation requirements, we ran another simulation ignoring the external disturbance and model uncertainties. In the simulation, three criteria were set: flying at a constant height (i.e., $\mathsf{\Delta }z=0$), radius scaling factor $\sigma =1$, and $\sigma =0.8$. As plotted in Figure 10, the results showed that introducing a reference circle on the virtual image plane and adjusting the radius scaling factor can regulate the imaging effect, according to different observation requirements. To illustrate that the proposed strategy is applicable to different kinds of targets, we ran another simulation to hover above an irregularly shaped target. The Cartesian coordinates of the feature points that enclose the target were: $[0.25,-0.05,0]$ m, $[0.10,0.20,0]$ m, $[-0.20,0.15,0]$ m, $[-0.25,-0.10,0]$ m, $[-0.10,-0.20,0]$ m, $[0.15,-0.15,0]$ m. The imaging effect under different observation requirements was shown in Figure 11. Similar to the rectangular target, the proposed control strategy can meet the hovering and observation needs of the target.

5.2. Scenario 2: Tracking a Moving Target

We also considered some more challenging cases, such as when the target vehicle is moving or when a fire area changes in real-time. Therefore, not only positioning the UAV over the target is required, but also following its (unknown) trajectory. Compared with the traditional PD control, the robust controller proposed in this paper was more competent in this challenging mission. In this simulation, the target had the same geometric characteristics as in SCENARIO 1 but followed three different trajectories on the flat ground: circular movement, S-type movement, and linear movement. The quadrotor initial position was set to be $[1,1.5,-3]$ m. To evaluate the performance of the proposed controllers in a more realistic environment, we added some noise to the measurement information. White noise with covariances of 0.5 and 10${}^{-4}$ was augmented to the visual data (image features and their pixel velocities) and angular rates, respectively.

The tracking performance of the circular, S-type, and linear movements were illustrated in Figure 12, Figure 13, Figure 14, Figure 15, Figure 16 and Figure 17, respectively. Trajectories of motion of the quadrotor and the target in a 3D environment were plotted in Figure 12, Figure 14 and Figure 16, which showed satisfactory tracking performance in spite of the presence of disturbance and noise. Meanwhile, Figure 13, Figure 15 and Figure 17 depicted the position tracking error of the two controllers in different target movements. The results indicated that the proposed control strategy applied to different target maneuvers, and showed better robustness and higher tracking accuracy than the traditional PD controller. The only exception was that there existed some fluctuations during the linear tracking (Figure 16 and Figure 17). It was because of the vehicle’s reaction time when the target suddenly changed its direction.

6. Conclusions

In this paper, we have developed a vision-based control scheme of a quadrotor for target tracking in the absence of location information and geometric features of the target. After transforming the image features to a virtual camera frame, optical-based metrology is exploited to estimate the relative distance and velocity. At the same time, the height of the quadrotor and image size can be adjusted by regulating the optimal observation area and radius scaling factor. Considering the presence of external interference and model uncertainties, we presented a gain-switching proportional-derivative (PD) control strategy to improve the robustness of the system. Two case studies, corresponding to hovering and tracking scenarios, are presented in this work. The simulation results indicated that the proposed vision-based scheme performed better in both hovering and tracking missions, compared with the traditional PD control.

In future work, we are going to add a field of view constraint to the system, as the proposed algorithm can not guarantee that all visual features are always kept inside the field of view of the camera. We also plan to implement the proposed control scheme in a real quadrotor.