A Simple Learning Approach for Robust Tracking Control of a Class of Dynamical Systems

Abstract

This paper studies the robust tracking control problem for a class of uncertain nonlinear dynamical systems subject to unknown disturbances. A robust trajectory tracking control law is designed via a simple learning-based control strategy. In the developed design, the cost function based on the desired closed-loop error dynamics is minimized by means of gradient descent technique. A stability proof for the closed-loop nonlinear system is provided based on the pseudo-linear system theory. The learning capability of the developed robust trajectory tracking control law allows the system to mitigate the adverse effects of the uncertainties and disturbances. The numerical simulation results for a planar PPR robot are included to illustrate the effectiveness of the developed control law.

1. Introduction

Several robust control approaches have been proposed to deal with the issues associated with uncertainties, disturbances, and noise [1,2,3,4,5,6,7]. The sliding mode control (SMC) method represents a popular robust control approach [8,9,10,11,12,13]. SMC theory is based on using high controller gains to achieve robust performance despite the disturbances and uncertainties. However, high gains may cause unwanted chattering effects on the system response [14].

The use of learning-based model predictive controllers is an alternative approach to dealing with disturbances and uncertainties [15,16,17,18,19]. Neural network (NN)-based learning is a common control technique with various types of applications, including NN-based online learning [20], adaptive NN-based stabilization [21], and adaptive NN-based backstepping control [22,23]. The main concern in applying NN-based learning is the inherent difficulty in proving closed-loop system stability [24].

Another control approach combines a fuzzy neural network (FNN) and conventional PD controller to realize the so-called PD+FNN control [25,26,27]. This novel control approach employs a sliding mode control (SMC) based leaning technique that allows for robust control. Their approach does not rely on partial derivatives or matrix inversions. The work in [28] presents a robust adaptive learning control for a spacecraft in the presence of unknown disturbances and uncertainties. The asymptotic stability of the closed-loop systems with the proposed adaptive control law is proven based on the Lyapunov theory.

Type-2 neuro-fuzzy controllers (T2NFCs) have also found wide use, where type-2 fuzzy logic and artificial neural network controllers are fused together to achieve better performance [29,30,31,32]. The suggested online adaptive rules in T2NFCs help to lessen the need for detailed gain tuning by learning the disturbances and the system dynamics in an online fashion. The sliding mode learning control (SMLC) approach for controlling uncertain dynamical systems is addressed in [9]. In this work, a T2NFC learns the system behavior while a conventional control term provides the system stability. Then the T2NFC fully takes over the control of the system rapidly. This SMLC approach is shown to be capable of learning the system behavior even without the knowledge of any mathematical model so that the closed-loop system achieves robust control performance. The neuro-adaptive SMC approach has also been effectively employed in the control of robotic systems [33,34,35].

The feedback linearization control (FLC) technique is another widely used approach. However, the performance of the traditional FLC approach degrades under uncertainties and disturbances such that the closed-loop error fails to converge to zero [36,37]. In the literature, FLC with an integral action (FLC-I) has been proposed to ensure the convergence of the steady-state error. To overcome the drawbacks of the traditional FLC, several learning approaches have been proposed in conjunction with FLC [38,39,40]. In [40], an artificial NN with a nonlinear auto-regressive-moving average model has been designed to facilitate learning of the feedback linearized inputs for a nonlinear plant. In [41], a Gaussian process methodology is employed to determine a model for the FLC technique with minimal knowledge of the system a priori. In addition, in the literature, a dynamic linearization strategy has been designed for discrete-time nonlinear systems [42,43,44,45,46]. For systems that undergo varying working conditions such as time-varying uncertainty and delay [47,48], the aforementioned methods may not be good candidates. One approach to deal with this drawback is to incorporate disturbance observers for disturbance compensation [49,50,51,52,53,54,55]. For example, Ref. [49] proposes a gradient-descent-based learning gain as well as a disturbance observer of nonlinear systems.

Although all these learning-based and nonlinear methods have several benefits, such as handling uncertainties, unmodeled dynamics, and disturbances, the real-time implementations are challenging due to the fact that they are mostly computationally expensive. To address this, in this paper, we generalize the fundamental steps presented in our previous work [56,57] for the design of a simple learning (SL) control strategy for the trajectory tracking of a class of dynamical systems. The developed SL strategy utilizes the updates rules based on the desired closed-loop error dynamics to adjust its control gains and the disturbance estimates in the feedback control law. Here, the FLC method is used for designing the traditional feedforward control law based on the nominal model of the system. In the developed strategy, the gradient-descent method is used to minimize the desired closed-loop error function, therefore finding the adaptation rules for control gains and disturbance estimates. The stability of the closed-loop nonlinear system is mathematically proven based on the pseudo-linear system theory.

The main contributions of this paper are (a) the development of a general computationally efficient simple learning-based robust tracking control law and (b) the illustration of the effectiveness of the designed control law via computer simulation results using the planar PPR robot manipulator. Additionally, It is worth noting that the developed algorithm is computationally efficient for real-time implementation as presented in our previous work [56,57].

The paper is organized as follows: The mathematical model of a class of dynamical systems studied in this paper is presented in Section 2. In Section 3, an FLC-based control law is designed. In Section 3.1, the update rules for controller gains and disturbance estimates are derived. Proof of the closed-loop stability based on the pseudo-linear system theory is provided in Section 3.2. The satisfactory performance of the proposed method is illustrated by providing numerical simulations of the PPR robot manipulator in Section 4, and finally, Section 5 presents the conclusions and future research work.

2. Mathematical Model

Let $(Q,L)$ be a mechanical system with n degrees of freedom. Here $Q\subset {\mathbb{R}}^n$ is the configuration manifold (assumed to be a smooth manifold) and the smooth real-valued function $L=L(q,\dot{q})$ is the Lagrangian, where $q\in Q$ and $\dot{q}\in {\mathbb{R}}^n$ are referred to as n-vectors of generalized configuration variables and generalized velocity variables, respectively. In general, L can be expressed as

$$L=\frac{1}{2}{\dot{q}}^{T}M\left(q\right)\dot{q}-U\left(q\right)$$

(1)

where $\frac{1}{2}{\dot{q}}^{T}M\left(q\right)\dot{q}$ is the kinetic energy of the system and the smooth real-valued function $U\left(q\right)$ denotes the potential energy of the system. Here the $n\times n$ smooth matrix function $M\left(q\right)$ (called the generalized inertia matrix) is assumed to be symmetric and positive definite. Let $\tau \in {\mathbb{R}}^n$ denote the n-vector of generalized forces/torques acting on the system. Then, the Euler–Lagrange equations [58] are given by

$$\frac{d}{dt}\frac{\partial L}{\partial \dot{q}}-\frac{\partial L}{\partial q}=\tau ,$$

(2)

which can be expressed as Equation (3) considering all the model uncertainties and disturbances:

$$M\left(q\right)\ddot{q}+F(q,\dot{q})=B\left(q\right)u+\Delta (q,\dot{q})+w\left(t\right)$$

(3)

Here $\ddot{q}\in {\mathbb{R}}^n$ is referred to as an n-vector of generalized acceleration variables, $u\in {\mathbb{R}}^n$ is an n-vector of control input variables, $B\left(q\right)$ is an invertible $n\times n$ smooth matrix function, and $F(q,\dot{q})$ is an n-vector smooth function; in addition, $\Delta (q,\dot{q})\in {\mathbb{R}}^n$ and $w\left(t\right)\in {\mathbb{R}}^n$ represent n-vectors of modeling uncertainties and external disturbances, respectively.

Define by $x={[x_1^T,x_2^T]}^T={[q^T,{\dot{q}}^T]}^T$ the state. Then Equation (3) can be expressed in state space form as

$$\begin{array}{cc}\hfill & {\dot{x}}_1=x_2\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill & {\dot{x}}_2=f\left(x\right)+g\left(x_1\right)u+d\hfill \end{array}$$

(5)

where

$$f\left(x\right)=-M^{-1}\left(x_1\right)F\left(x\right),g\left(x_1\right)=M^{-1}\left(x_1\right)B\left(x_1\right),d=M^{-1}\left[\Delta \left(x\right)+w\right]$$

(6)

where $d\in {\mathbb{R}}^n$ represents a lumped disturbance vector that comprises modeling uncertainties and external disturbances. Here, the control design goal is to design a control input $u\in {\mathbb{R}}^n$ so that the system tracks a given reference trajectory $r\left(t\right)={[r_1^T\left(t\right),r_2^T\left(t\right)]}^T$, where $r_2={\dot{r}}_1$, accurately while the state and control variables remain bounded.

3. Control Design

To formulate the tracking problem for the given reference trajectory, define the tracking error vector $e\left(t\right)={[e_1^T\left(t\right),e_2^T\left(t\right)]}^T$, where $e_1=r_1-x_1$ and $e_2=r_2-x_2$. Consider the following control input

$$u=g^{-1}\left(x_1\right)\left(-f+K_1{e}_1+K_2{e}_2+{\dot{r}}_2-\widehat{d}\right)$$

(7)

where the $n\times n$ gain matrices $K_1$ and $K_2$ are symmetric and positive definite and $\widehat{d}\in {\mathbb{R}}^n$ represents the estimated disturbance vector. In what follows, for the simplicity of the development in this paper, we will assume $K_1=k_1{I}_n,K_2=k_2{I}_n$, where $k_1\in \mathbb{R}$ and $k_2\in \mathbb{R}$ are positive control gains and $I_n$ denotes the $n\times n$ identity matrix, so that Equation (7) can be rewritten as

$$u=g^{-1}\left(x_1\right)\left(-f+k_1{e}_1+k_2{e}_2+{\dot{r}}_2-\widehat{d}\right)$$

(8)

The closed-loop error dynamics can then be expressed as

$$\begin{array}{cc}\hfill & {\dot{e}}_1=e_2\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill & {\dot{e}}_2=-k_1{e}_1-k_2{e}_2-d+\widehat{d}\hfill \end{array}$$

(10)

3.1. Update Rules

Let $k_d={[k_{1d},k_{2d}]}^T$ denote the desired gain vector. Consider the following desired closed-loop error dynamics:

$$c(e,k_d)={\dot{e}}_2+k_{2d}{e}_2+k_{1d}{e}_1$$

(11)

The update rules are derived by using a gradient descent method to minimize the cost function and given by

$$C=\frac{1}{2}{c}^T(e,k_d)c(e,k_d)$$

(12)

The update rule for the controller gain is given by

$$\begin{array}{cc}\hfill {\dot{k}}_i& =-{\alpha }_i\frac{\partial C}{\partial k_i}\hfill \\ \hfill & =-{\alpha }_i{c}^T(e,k_d)\frac{\partial c}{\partial k_i}\hfill \\ \hfill & ={\alpha }_i{c}^T(e,k_d)e_i\hfill \end{array}$$

(13)

where the ith controller gain’s learning rate is denoted by ${\alpha }_i>0$. Similarly, the disturbance estimate can be updated using the rule below:

$$\begin{array}{cc}\hfill \dot{\widehat{d}}& =-{\alpha }_{\widehat{d}}\frac{\partial C}{\partial \widehat{d}}\hfill \\ \hfill & =-{\alpha }_{\widehat{d}}{c}^T(e,k_d)\frac{\partial c}{\partial \widehat{d}}\hfill \\ \hfill & =-{\alpha }_{\widehat{d}}c(e,k_d)\hfill \end{array}$$

(14)

where the learning rate ${\alpha }_{\widehat{d}}$ is for the disturbance estimate $\widehat{d}$. The disturbance estimates and the controller gains are updated until the cost function converges to zero (i.e., $C(e,k_d)\to 0$).

3.2. Stability Proof

The closed-loop error dynamics of the system under study can be rewritten as

$${\ddot{e}}_1+k_2{\dot{e}}_1+k_1{e}_1-\widehat{d}+d=0$$

(15)

and assuming disturbance d has a much smaller average rate of change than that of the error state variables, then taking the time derivative of (15) and having $\dot{d}=0$, we obtain the following:

$${\stackrel{⃛}{e}}_1+k_2{\ddot{e}}_1+(k_1+{\dot{k}}_2){\dot{e}}_1+{\dot{k}}_1{e}_1-\dot{\widehat{d}}=0.$$

(16)

Substituting the associated expressions for ${\dot{k}}_i$ and $\dot{\widehat{d}}$ in (16), we obtain

$${\stackrel{⃛}{e}}_1+k_2{\ddot{e}}_1+k_1{\dot{e}}_1+\left({\alpha }_1{c}^T{e}_1\right)e_1+\left({\alpha }_2{c}^T{\dot{e}}_1\right){\dot{e}}_1+{\alpha }_{\widehat{d}}c=0.$$

(17)

Using the expression for $c(e,k_d)$ given by Equation (11), we obtain

$${\stackrel{⃛}{e}}_1+A_1\left(\xi \right){\ddot{e}}_1+A_2\left(\xi \right){\dot{e}}_1+A_3\left(\xi \right)e_1=0$$

(18)

where $\xi ={\left[e_1^T{\dot{e}}_1^T{\ddot{e}}_1^T\right]}^T$ is the state and

$$\begin{array}{cc}& A_1=\left(k_2+{\alpha }_{\widehat{d}}\right)I_n+{\alpha }_1{e}_1{e}_1^T+{\alpha }_2{\dot{e}}_1{\dot{e}}_1^T\hfill \\ & A_2=\left(k_1+{\alpha }_{\widehat{d}}{k}_{2d}\right)I_n+k_{2d}\left({\alpha }_1{e}_1{e}_1^T+{\alpha }_2{\dot{e}}_1{\dot{e}}_1^T\right)\hfill \\ & A_3=k_{1d}\left({\alpha }_{\widehat{d}}{I}_n+{\alpha }_1{e}_1{e}_1^T+{\alpha }_2{\dot{e}}_1{\dot{e}}_1^T\right).\hfill \end{array}$$

(19)

Equation (17) is in a pseudo-linear form. A stability analysis will be performed using a Routh–Hurwitz criterion [59]. Clearly $\forall \xi$, the $A_i\left(\xi \right),i=1,2,3$ are symmetric and positive definite and the characteristic equation corresponding to $A_i\left(0\right),i=1,2,3$, is given by

$$\left[s^3+\left(k_2+{\alpha }_{\widehat{d}}\right)s^2+\left(k_1+{\alpha }_{\widehat{d}}{k}_{2d}\right)s+k_{1d}{\alpha }_{\widehat{d}}\right]I_n=0.$$

(20)

The stability of the closed-loop error system can be shown by applying the Routh–Hurwitz criterion such that the characteristic Equation (20) has all its roots with negative real parts if the following condition is satisfied:

$$\left(k_2+{\alpha }_{\widehat{d}}\right)\left(k_1+{\alpha }_{\widehat{d}}{k}_{2d}\right)>k_{1d}{\alpha }_{\widehat{d}}.$$

(21)

In what follows, $k_i,k_{id},i=1,2$, and ${\alpha }_{\widehat{d}}$ are chosen such that the stability condition in Equation (21) is satisfied.

4. Example: Simple Learning Control of a PPR Robot

As a sample nonlinear system, a planar PPR robot with three joints (two prismatic and one revolute) is considered as shown in Figure 1. The planar PPR robot is moving on a horizontal plane so that the gravitational potential can be ignored. In Figure 1, the Cartesian position of the revolute joint and the orientation of the third link are denoted by $(x,y)$ and $\theta$, respectively. The control inputs are the torque $\tau$ that is actuating the revolute joint and the forces $F_x$ and $F_y$ applied to the two prismatic joints. The physical parameters for the PPR robot are the mass of the three links $m_1$, $m_2$, $m_3$; the distance l from the revolute joint to the center of mass (CoM) of the third link; and the moment of inertia $J_3$ of the third link about its CoM.

The Lagrangian is given by

$$L=\frac{1}{2}(m_1+m_2){\dot{x}}^2+\frac{1}{2}{m}_2{\dot{y}}^2+\frac{1}{2}{m}_3{v}_3^2+\frac{1}{2}{J}_3{\dot{\theta }}^2$$

(22)

where $v_3$ is the velocity of the CoM of the third link, which is given by

$$v_3=\sqrt{{(\dot{x}-l\dot{\theta }sin\theta )}^2+{(\dot{y}+l\dot{\theta }cos\theta )}^2}.$$

(23)

The equations of motion can be obtained as below by using Lagrangian formulation.

$$\begin{array}{cc}& m_x\ddot{x}-m_{3}l\ddot{\theta }sin\theta -m_{3}l{\dot{\theta }}^{2}cos\theta =F_x\hfill \\ & m_y\ddot{y}+m_{3}l\ddot{\theta }cos\theta -m_{3}l{\dot{\theta }}^{2}sin\theta =F_y\hfill \\ & J\ddot{\theta }+m_{3}l(\ddot{y}cos\theta -\ddot{x}sin\theta )=\tau \hfill \end{array}$$

(24)

where $m_x=m_1+m_2+m_3,m_y=m_2+m_3$ and $J=J_3+m_3{l}^2$.

4.1. Computer Simulation Results

To evaluate the performance of the developed control law presented in Section 2, the PPR robot system is considered in this section. The nominal and estimated parameters of the PPR robot system are given in

Table 1. Parameters of PPR robot system.

Symbol	Parameter	Value	Unit
$m_1$	Link 1 mass	5	kg
$m_2$	Link 2 mass	10	kg
$m_3$	Link 3 mass	15	kg
$J_3$	MOI of link 3 about its CoM	1.5	kg·m${}^2$
l	Distance b/w revolute joint and CoM of link 3	$0.5$	m

and

Table 2. Estimated parameters of PPR robot system.

Symbol	Parameter	Value	Unit
$m_1$	Link 1 mass	4	kg
$m_2$	Link 2 mass	8	kg
$m_3$	Link 3 mass	12	kg
$J_3$	MOI of link 3 about its CoM	1.7	kg·m${}^2$
l	Distance b/w revolute joint and CoM of link 3	$0.8$	m

, respectively.

Three numerical simulation cases were considered for evaluating the performance of the developed simple learning-based controller.

• Case I: Tracking sinusoidal trajectories;
• Case II: Tracking constant trajectories;
• Case III: Tracking under large external disturbances.

In Case I, the developed controller generates the appropriate control actions for tracking sinusoidal trajectories. Then, in Case II, the control actions for maintaining the system behavior for properly tracking constant trajectories is studied. Finally, in Case III, the performance of the developed controller for trajectory tracking under large external disturbances is evaluated.

Simulations are carried out with the following initial conditions:

$$\begin{array}{cc}\hfill (x_0,y_0,{\theta }_0)& =(0,0,0)\hfill \\ \hfill ({\dot{x}}_0,{\dot{y}}_0,{\dot{\theta }}_0)& =(0,0,0).\hfill \end{array}$$

In all simulation cases, the following external disturbances are assumed:

$$d_x=d_y=0.5{\mathrm{m}/\mathrm{s}}^2,d_{\theta }=0.5{\mathrm{rad}/\mathrm{s}}^2$$

All the simulations using the developed control algorithm are implemented on a platform with the following specifications: MacBook Pro (macOS 13.2.1), Processor: 2.3 GHz Intel Core i5, Memory: 16.00 GB. In all cases, the total simulation time is 70 seconds and the sampling time is 0.05 s. In this work, we employed the Runge–Kutta 4th order method for numerically solving the equations of the system.

4.2. Case I: Tracking Sinusoidal Trajectories

To study the tracking of sinusoidal trajectories, the desired trajectory for PPR robot is designed as follows:

$$\begin{array}{cc}\hfill & x_d\left(t\right)=cost\left[m\right]\hfill \\ \hfill & y_d\left(t\right)=sin\frac{t}{4}\left[m\right]\hfill \\ \hfill & {\theta }_d\left(t\right)=45cos\frac{t}{2}\left[deg\right].\hfill \end{array}$$

For the simulation in Case I, the control gains below are used for the desired closed-loop error dynamics:

$$k_{1d}=1,k_{2d}=2.$$

All the learning rates and the initial values for the controller in Case I are given below:

$$\begin{array}{cc}\hfill & k_1\left(0\right)=1,k_2\left(0\right)=1\hfill \\ \hfill & {\alpha }_1=10.5,{\alpha }_2=0.2.\hfill \end{array}$$

All the learning rates and the initial value for the disturbances in Case I are as follows:

$$d_x\left(0\right)=0,d_y\left(0\right)=0,d_{\theta }\left(0\right)=0,{\alpha }_{\widehat{d}}=10.5.$$

Figure 2 illustrates that the configuration variables $x,y$, and $\theta$ closely follow the reference trajectory.

The control gains $k_1$ and $k_2$ (see Figure 3), the estimated disturbances $({\widehat{d}}_x,{\widehat{d}}_y$, and ${\widehat{d}}_{\theta })$ (see Figure 4), and the control input forces $(F_x,F_y)$ and torque $F_{\theta }$ (see Figure 5) for Case I (Tracking Sinusoidal Trajectories) are shown in Figure 3, Figure 4 and Figure 5.

4.3. Case II: Tracking Constant Trajectories

To study the tracking of constant trajectories, the desired trajectory for PPR robot is designed as follows:

$$\begin{array}{c}\hfill x_d=1\left[m\right],y_d=2\left[m\right],{\theta }_d\left(t\right)=90\left[deg\right].\end{array}$$

For the simulation in Case II, the below control gains are used for the desired closed-loop error dynamics:

$$k_{1d}=1,k_{2d}=1.2.$$

All the learning rates and the initial values for the controller in Case II are given below:

$$\begin{array}{cc}\hfill & k_1\left(0\right)=1,k_2\left(0\right)=1\hfill \\ \hfill & {\alpha }_1=1.5,{\alpha }_2=0.2.\hfill \end{array}$$

All the learning rates and the initial value for the disturbances in Case II are as follows:

$$d_x\left(0\right)=0,d_y\left(0\right)=0,d_{\theta }\left(0\right)=0,{\alpha }_{\widehat{d}}=1.5.$$

Figure 6 illustrates that the configuration variables $x,y$, and $\theta$ closely follow the reference trajectory.

The control gains $k_1$ and $k_2$ (see Figure 7), the estimated disturbances $({\widehat{d}}_x,{\widehat{d}}_y$, and ${\widehat{d}}_{\theta })$ (see Figure 8), and the control input forces $(F_x,F_y)$ and torque $F_{\theta }$ (see Figure 9) for Case II are shown in Figure 7, Figure 8 and Figure 9.

4.4. Case III: Trajectory Tracking under Large External Disturbances

To study the tracking performance of the developed control technique under large external disturbances, an extra disturbance of 2 m/s${}^2$ is additionally applied (i.e., $d_x=[0.5+2]{\mathrm{m}/\mathrm{s}}^2$) in the range of [30–31] seconds.

In this simulation, the desired trajectory for the PPR robot is designed as follows:

$$\begin{array}{c}\hfill x_d=1\left[m\right],y_d=2\left[m\right],{\theta }_d\left(t\right)=90\left[deg\right].\end{array}$$

For the simulation in Case III, the below control gains are used for the desired closed-loop error dynamics:

$$k_{1d}=1,k_{2d}=1.2.$$

All the learning rates and the initial values for the controller in Case III are given below:

$$\begin{array}{cc}\hfill & k_1\left(0\right)=1,k_2\left(0\right)=1\hfill \\ \hfill & {\alpha }_1=1.5,{\alpha }_2=0.2.\hfill \end{array}$$

All the learning rates and the initial value for the disturbances in Case III are as follows:

$$d_x\left(0\right)=0,d_y\left(0\right)=0,d_{\theta }\left(0\right)=0,{\alpha }_{\widehat{d}}=1.5.$$

Figure 10 illustrates that the configuration variables $x,y$, and $\theta$ closely follow the reference trajectory.

The control gains $k_1$ and $k_2$ (see Figure 11), the estimated disturbances $({\widehat{d}}_x,{\widehat{d}}_y$, and ${\widehat{d}}_{\theta })$ (see Figure 12), and the control input forces $(F_x,F_y)$ and torque $F_{\theta }$ (see Figure 13) for Case III are shown in Figure 11, Figure 12 and Figure 13.

5. Conclusions

A robust trajectory tracking control strategy for a class of uncertain dynamical systems is presented in this paper. The external disturbances and model uncertainties are compensated via the developed simple learning-based control technique. The developed control design minimizes the cost function associated with the error dynamics of a class of nonlinear systems. The significance of the developed robust trajectory tracking control method is demonstrated via the results of numerical computer simulations of a PPR robot.

In this paper, the results are obtained for fully actuated systems with invertible control input matrix $B\left(q\right)$. Future work includes extending these results to underactuated systems, i.e., systems with fewer inputs than their degrees of freedom, where the control input matrix $B\left(q\right)$ is not invertible [60].