Critic Learning-Based Safe Optimal Control for Nonlinear Systems with Asymmetric Input Constraints and Unmatched Disturbances

Abstract

In this paper, the safe optimal control method for continuous-time (CT) nonlinear safety-critical systems with asymmetric input constraints and unmatched disturbances based on the adaptive dynamic programming (ADP) is investigated. Initially, a new non-quadratic form function is implemented to effectively handle the asymmetric input constraints. Subsequently, the safe optimal control problem is transformed into a two-player zero-sum game (ZSG) problem to suppress the influence of unmatched disturbances, and a new Hamilton–Jacobi–Isaacs (HJI) equation is introduced by integrating the control barrier function (CBF) with the cost function to penalize unsafe behavior. Moreover, a damping factor is embedded in the CBF to balance safety and optimality. To obtain a safe optimal controller, only one critic neural network (CNN) is utilized to tackle the complex HJI equation, leading to a decreased computational load in contrast to the utilization of the conventional actor–critic network. Then, the system state and the parameters of the CNN are uniformly ultimately bounded (UUB) through the application of the Lyapunov stability method. Lastly, two examples are presented to confirm the efficacy of the presented approach.

1. Introduction

Safety-critical systems are those that, in case of accidents or failures, can result in significant consequences, including but not limited to injuries, loss of life, environmental harm, or financial losses. The emergence of safety-critical systems like unmanned aerial vehicles (UAVs) [1,2,3] and robots [4] has led to an increased focus on safety control design within the field of control systems [5,6]. Safety control designs entail control strategies that satisfy safety specifications imposed by environmental limitations or physical limitations of the system. Ignoring the detrimental impact of safety entails substantial risks to both the safety of belongings and personal security. To address the challenges of the safe controller design, researchers have provided some effective approaches [7,8,9,10,11]. The problem of safety in the presence of unmodeled dynamics or disturbances in drones has recently been addressed by designing the robust controller based on the nonlinear estimator in [9]. In ref. [10], the use of neural networks integrated with the Lyapunov theory was preliminarily treated with application in the automotive sector for critical situations, and this aspect was further addressed in an even more organic way. In ref. [11], the quadratic programming-based method was applied to develop a safe controller. Despite the fact that this method can guarantee safety at a local level for every time step, selecting a step size that is too small leads to redundant computations. In contrast, a step size that is too large causes unsafe behavior, making it challenging to ensure the safety of the system. Hence, it is crucial to identify an appropriate control design method for CT safety-critical systems that can guarantee the safety of the systems.

Recently, the CBF technique has emerged as an effective approach for ensuring the security of safety-critical systems [12,13,14]. The underlying principle of the CBF is to insure the forward invariance of the safe set. In ref. [15], the safe-based reinforcement learning approach was demonstrated, where the CBF was merged into the cost function to assure both the safety and optimality of the system. Typically, the CBF component is contained within the primitive cost function to penalize behavior that violates safety constraints. Reference [16] incorporated damping factors into the CBF and intervened selectively only in the event of safety constraint violations, aiming to reduce disruptions to the optimal controller. Reference [17] introduced the utilization of the CBF and summarized the verification approach for safety-critical control systems. Nevertheless, the methods mentioned above do not take into account the presence of external disturbances, which served as a motivation for the research conducted in this paper.

As disturbances are present in almost all industrial systems and hurt control performance, it is necessary to consider external disturbances in actual projects. Recently, several methods have been proposed to address disturbances [18,19,20,21,22,23,24]. For instance, references [18,19] used $H\infty$ control to reduce external disturbances in nonlinear systems. Reference [23] combined ADP with sliding mode control for addressing optimal control problems of CT nonlinear systems considering uncertain disturbances. Reference [24] cast external disturbances as the ZSG problem, with the control strategy aiming at minimizing the cost function and the disturbance strategy striving towards maximizing it. It is well known that for the HJI equation of the ZSG, it is difficult to find its analytical solution. Fortunately, with the evolution of optimal control [25,26,27], the ADP approach [28] was employed to approximately tackle the ZSG problem. For example, in reference [29], a new database-based adaptive critic algorithm was presented to study the infinite-scale robust control for nonlinear systems. However, the aforementioned methods fail to consider the capability limitations of the system due to asymmetric input constraints.

Although symmetric input constraints have been widely investigated in prior research and tackled with various techniques, such as the control problem that concerns the uncertain impulse system that has input constraints, which was handled in [30], and the utilization of integral reinforcement learning with the actor–critic network to address the tracking control problem under input constraints in [31,32], there has been relatively little research on the treatment of asymmetric input constraints that frequently occur in practical systems. Several optimal control methods exist for addressing CT systems with nonlinear dynamics and input constraints that are asymmetric. Among them is one that used the cost function with adjustable upper and lower limits of integration [33,34,35]. Another proposed the switching function [36] to tackle the problem, but it is only applied to linear systems. However, none of these results considered the incorporation of CBF into the CT nonlinear safety-critical systems to study the safe and optimal control problem under asymmetric input constraints with external disturbances.

This study explores the safe and optimal control issue for safety-critical nonlinear systems to reject unmatched external disturbances under the condition of asymmetric input constraints. Unlike other works, a new non-quadratic form function for handling asymmetric input constraints is proposed in this paper. To tackle the challenge posed by unmatched disturbances, a two-player ZSG is put forward to formulate the optimization problem. The ZSG is then addressed by finding the Nash equilibrium point, which is obtained by addressing the HJI equation. However, since solving the HJI equation is challenging, an ADP technique similar to that used in references [37,38,39] is exploited to estimate the solution of the HJI equation. In addition, one single CNN is used instead of a dual actor–critic neural network to diminish the computational complexity in approximating the control policy. Consequently, the optimal control policy is obtained by considering the worst disturbance.

The contributions are outlined mainly as follows:

• Asymmetric input constraints are considered in the control problem of the CT nonlinear safety-critical systems. In addition, this paper proposes a new non-quadratic form function to address the issue of asymmetric input constraints. It is important to note that when applying this approach, the optimal control policy no longer remains at 0, even when the system state reaches the equilibrium point of $x=0$ (see ${\mathrm{u}}^{*}\left(x\right)$ in later Equation (15)).
• This paper adopts the CBF to construct safety constraints and proposes designing a damping coefficient within the CBF to balance the safety and optimality of safety-critical systems based on varying safety requirements in different applications.
• The safe optimal control problem is turned into the ZSG problem to address unmatched disturbances; then, the optimal control law is gained by tackling the HJI equation using one CNN. Moreover, the use of only one CNN to approximate the HJI equation is an effective way to reduce the computational burden compared to the actor–critic network and the system state, and CNN parameters are demonstrated to be UUB.

The following structure is adopted for this article. Section 2 provides the initial formulation of the problem. Section 3 presents a safe optimal control design for the two-player ZSG problem. Then, in Section 4, an adaptive CNN method for addressing the HJI equation using an online method is proposed, and its stability is verified. Section 5 introduces two examples to demonstrate that the presented approach is effective. Lastly, Section 6 gives conclusions.

2. Problem Statement

Consider the CT nonlinear safety-critical system as

$$\dot{x}=\mathcal{F}\left(x\right)+\mathcal{G}\left(x\right)\mathrm{u}+\mathcal{P}\left(x\right)\mathrm{v},$$

(1)

where $x={[x_1,x_2,\dots ,x_n]}^T\in {\mathcal{C}}_a\subseteq {\mathcal{R}}^n$ indicates the system state vector with n-dimensional parameters, $\mathcal{F}\left(x\right)\in {\mathcal{R}}^n$ represents the internal dynamics, $\mathcal{G}\left(x\right)\in {\mathcal{R}}^{n\times m}$ and $\mathcal{P}\left(x\right)\in {\mathcal{R}}^{n\times q}$ indicate control and disturbance coefficient matrices, respectively. Additionally, $\mathrm{u}\in {\mathcal{R}}^m$ denotes an input variable with m-dimensional parameters denoted by ${\complement }_{\mathrm{u}}=\left\{\mathrm{u}|{\mathrm{u}}_{max}\ge \mathrm{u}\ge {\mathrm{u}}_{min}\right\}$, where ${\mathrm{u}}_{max}$ and ${\mathrm{u}}_{min}$ stand for the upper and lower bounds, respectively. And $\mathrm{v}\in {\mathcal{R}}^q$ is the unmatched disturbances. The paper assumes $\mathcal{F}(·)$, $\mathcal{G}(·)$, $\mathcal{P}(·)$ are Lipschitz continuous and satisfy $\mathcal{F}\left(0\right)=0$, and the safety-critical System (1) is stabilizable and controllable. Moreover, we assume there exist two constants ${\mathcal{G}}_M>0$ and ${\mathcal{P}}_M>0$. Both $\mathcal{G}\left(x\right)$ and $\mathcal{P}\left(x\right)$ have upper bounded values, i.e., ${\mathcal{G}}_M\ge ∥\mathcal{G}\left(x\right)∥,{\mathcal{P}}_M\ge ∥\mathcal{P}\left(x\right)∥$, for any $x\in {\mathcal{R}}^n$.

In addition, it is essential to emphasize that ${\mathcal{C}}_a$ represents a safe set for (1). ${\mathcal{C}}_a$ is derived from operational restrictions, such as the allowable states of the robot arm, which is mathematically determined by

$$\left\{\begin{array}{c}\begin{array}{cc}\hfill {\mathcal{C}}_a& =\left\{x\in {\mathcal{R}}^n|\mathrm{z}\left(x\right)\ge 0\right\},\hfill \\ \hfill int\left({\mathcal{C}}_a\right)& =\left\{x\in {\mathcal{R}}^n|\mathrm{z}\left(x\right)>0\right\},\hfill \\ \hfill \partial {\mathcal{C}}_a& =\left\{x\in {\mathcal{R}}^n|\mathrm{z}\left(x\right)=0\right\},\hfill \end{array}\hfill \end{array}\right.$$

(2)

where $\mathrm{z}\left(x\right)$ represents continuous concerning x. The set $int\left({\mathcal{C}}_a\right)$ denotes the interior of ${\mathcal{C}}_a$, while $\partial {\mathcal{C}}_a$ represents the boundary of ${\mathcal{C}}_a$.

Subsequently, the representation of the infinite horizon cost function from $t=0$ for System (1) is given by

$$\mathrm{V}\left(x\right)={\int }_0^{\infty }x{\left(t\right)}^T\mathcal{Q}x\left(t\right)+\mathfrak{U}\left(\mathrm{u}\right)-{{\rm Y}}^2{∥\mathrm{v}∥}^{2}dt,$$

(3)

where $\mathcal{Q}$ represents a function with positive definite properties, ${∥\mathrm{v}∥}^2$ = ${\mathrm{v}}^T\mathrm{v}$, ${\rm Y}>0$ represents a constant weight coefficient, $\mathfrak{U}\left(\mathrm{u}\right)$ is a non-quadratic form function employed for handling the asymmetric input constraints determined by

$$\begin{array}{cc}\hfill \mathfrak{U}\left(\mathrm{u}\right)& =2{\int }_{\Im }^{\mathrm{u}}\mathsf{\Psi }tan{h}^{-1}\left(\frac{t-\Im }{\mathsf{\Psi }}\right)dt\hfill \\ \hfill & =2\mathsf{\Psi }(\mathrm{u}-\Im )tan{h}^{-1}\left(\frac{\mathrm{u}-\Im }{\mathsf{\Psi }}\right)+{\mathsf{\Psi }}^2{ln}_{}(1-\frac{{(\mathrm{u}-\Im )}^2}{\mathsf{\Psi }}),\hfill \end{array}$$

(4)

with $\mathsf{\Psi }$ and ℑ defined as

$$\mathsf{\Psi }=\frac{1}{2}({\mathrm{u}}_{max}-{\mathrm{u}}_{min}),\Im =\frac{1}{2}({\mathrm{u}}_{max}+{\mathrm{u}}_{min}),$$

(5)

where $\left|{\mathrm{u}}_{max}\right|\ne \left|{\mathrm{u}}_{min}\right|$ and $tanh\left(\mathfrak{z}\right)=(e^{\mathfrak{z}}-e^{-\mathfrak{z}})/(e^{\mathfrak{z}}+e^{-\mathfrak{z}})$ with $\mathfrak{z}\in \mathcal{R}$.

Remark 1. 

Even though $tanh\left(\mathfrak{z}\right)$ is symmetric, $\mathfrak{U}\left(\mathrm{u}\right)$ in (4) generates asymmetric constraints in the control signal ${\mathrm{u}}^{*}\left(x\right)$ (see ${\mathrm{u}}^{*}\left(x\right)$ in later (15)). This is due to the fact that ℑ is not equal to 0 in (4). This feature is different from studying the symmetric input constraints.

Additionally, the ultimate objective of this paper is to devise the safe and optimal control input policy for (1), which involves the utilization of the CBF concept. In the upcoming section, this paper presents the concept of the CBF and proposes an ADP-based approach to design the safe and optimal controller.

3. Safe Optimal Control Design

This section presents a detailed explanation of the concept of the CBF. Then, the safe and optimal control problem is converted to the two-player ZSG to overcome the unmatched disturbances, and the CBF is integrated with the cost function without an intermediary to punish unsafe behavior.

3.1. Control Barrier Function

The utilization of the CBF provides a solution to address the safety constraint problem in safety-critical systems. The CBF is a function that is non-negative within the set ${\mathcal{C}}_a$ and exhibits divergence to infinity at the edge of ${\mathcal{C}}_a$. As the state x is about to reach the boundary of ${\mathcal{C}}_a$, the condition of negative derivative can bring the system state x back within ${\mathcal{C}}_a$, ensuring that the system state is always confined within ${\mathcal{C}}_a$. To better illustrate the properties of the CBF, the following assumption is given.

Assumption 1. 

The CBF candidate ${\mathcal{B}}_r\left(x\right)$ meets the subsequent three characteristics [40,41]:

(1) 

${\mathcal{B}}_r\left(x\right)\ge 0,\forall x\in int\left({\mathcal{C}}_a\right)$,

(2) 

${\mathcal{B}}_r\left(x\right)\to \infty ,\forall x\in \partial {\mathcal{C}}_a$,

(3) 

${\mathcal{B}}_r\left(x\right)$ is monotonically decreasing $\forall x\in {\mathcal{C}}_a$.

Moreover, for all $x\in {\mathcal{C}}_a$, the CBF ${\mathcal{B}}_r\left(x\right)$ has the following properties:

$$\begin{array}{c}\hfill \frac{1}{{\gamma }_1\left(\mathrm{z}\left(x\right)\right)}\le {\mathcal{B}}_r\left(x\right)\le \frac{1}{{\gamma }_2\left(\mathrm{z}\left(x\right)\right)},\dot{{\mathcal{B}}_r}\left(x\right)\le {\gamma }_3\left(\mathrm{z}\left(x\right)\right),\end{array}$$

(6)

where ${\gamma }_1(·),{\gamma }_2(·)$, and ${\gamma }_3(·)$ are class $\mathcal{K}$ functions.

Under the premise that Assumption 1 and Equation (6) both hold, a suitable choice for ${\mathcal{B}}_r\left(x\right)$ is $\rho \mathrm{y}\left(x\right)/\mathrm{z}\left(x\right)$, where $\mathrm{y}\left(x\right)$ represents a special scheduling function determined by the user to allow for flexibility in selecting ${\mathcal{B}}_r\left(x\right)$. Specifically, $\mathrm{y}\left(x\right)$ ensures that the CBF operates only when the system is close to the unsafe set. $\rho >0$ is the damping factor used to balance safety and optimality.

Remark 2. 

In contrast to the previous CBF [16], the ρ chosen here shows a positive correlation with the value of ${\mathcal{B}}_r\left(x\right)$. The larger the value of ρ, the faster the system state moves away from the unsafe set, and the smaller the value of ρ, the slower the state x moves away from the unsafe set. A smaller value of ρ emphasizes optimality and a larger value of ρ enforces safety.

3.2. Safe and Optimal Control Approach

By augmenting the selected CBF ${\mathcal{B}}_r\left(x\right)$ to the cost function (3), a new refined cost function is obtained, that is,

$$\begin{array}{c}\hfill \mathrm{V}\left(x\right)={\int }_0^{\infty }x{\left(t\right)}^T\mathcal{Q}x\left(t\right)+\mathfrak{U}\left(\mathrm{u}\right)-{{\rm Y}}^2{∥\mathrm{v}∥}^2+{\mathcal{B}}_r\left(x\right)dt.\end{array}$$

(7)

Remark 3. 

To ensure the safety of the system, it is assumed that the original system state x is confined within the set ${\mathcal{C}}_a$. This is because the rapid increase in ${\mathcal{B}}_r\left(x\right)$ as the state x nears the boundary of ${\mathcal{C}}_a$ is the reason behind the penalization of state convergence behavior when the initial state is beyond ${\mathcal{C}}_a$. This prevents the system state from converging.

The conventional control problems can be transformed into two-player ZSG problems. The Nash equilibrium point, i.e., the saddle point (${\mathrm{u}}^{*}$,${\mathrm{v}}^{*}$) can be obtained by addressing the special HJI equation. Then, the optimal cost function is defined by

$$\begin{array}{c}\hfill {\mathrm{V}}^{*}\left(x\right)=\underset{\mathrm{u}}{min}\underset{\mathrm{v}}{max}{\int }_0^{\infty }x{\left(t\right)}^T\mathcal{Q}x\left(t\right)+\mathfrak{U}\left(\mathrm{u}\right)-{\mathrm{Y}}^2{∥\mathrm{v}∥}^2+{\mathcal{B}}_r\left(x\right)dt.\end{array}$$

(8)

The purpose of the two-player ZSG problem is to identify a saddle point so that the following inequality can hold:

$$\begin{array}{c}\hfill {\mathrm{V}}^{*}(x,{\mathrm{u}}^{*},\mathrm{v})\le {\mathrm{V}}^{*}(x,{\mathrm{u}}^{*},{\mathrm{v}}^{*})\le {\mathrm{V}}^{*}(x,\mathrm{u},{\mathrm{v}}^{*}).\end{array}$$

(9)

Therefore, for the two-player ZSG problem, ${\mathrm{u}}^{*}$ is the optimal control input policy minimizing the cost function, and ${\mathrm{v}}^{*}$ represents the worst disturbance input policy maximizing the cost function.

Definition 1. 

Input policy $\mathrm{u}$ is considered admissible in relation to (7) on $\mho \in {\mathcal{R}}^n$, denoted by $\mathrm{u}\in \aleph (\mho )$, $\mathrm{u}$ stabilizes (1) on ℧ if $\mathrm{u}$ is continuous on ℧, and (7) is limited for any $x\in \mho$.

For the admissible input policy $\mathrm{u}\in \aleph (\mho )$, if Equation (7) is continuously differentiable, computing the gradient of $\mathrm{V}\left(x\right)$ with respect to t on both sides of Equation (7) yields the nonlinear Lyapunov equation as

$$\begin{array}{c}\hfill 0=\nabla \mathrm{V}{\left(x\right)}^T(\mathcal{F}\left(x\right)+\mathcal{G}\left(x\right)\mathrm{u}+\mathcal{P}\left(x\right)\mathrm{v})+x{\left(t\right)}^T\mathcal{Q}x\left(t\right)+\mathfrak{U}\left(\mathrm{u}\right)-{\mathrm{Y}}^2{∥\mathrm{v}∥}^2+{\mathcal{B}}_r\left(x\right),\end{array}$$

(10)

where $\nabla \mathrm{V}\left(x\right)$ is the gradient of $\mathrm{V}\left(x\right)$, $\mathrm{V}\left(0\right)=0.$

Based on the optimal control approach, the HJI equation for the two-player ZSG problem possesses an exclusive solution if there exists a saddle point, that is, if the following conditions hold:

$$\begin{array}{c}\hfill 0=\underset{\mathrm{u}}{min}\underset{\mathrm{v}}{max}\mathcal{H}(x,\mathrm{u},\mathrm{v},\nabla {\mathrm{V}}^{*}\left(x\right))=\underset{\mathrm{v}}{max}\underset{\mathrm{u}}{min}\mathcal{H}(x,\mathrm{u},\mathrm{v},\nabla {\mathrm{V}}^{*}\left(x\right)),\end{array}$$

(11)

where $\mathcal{H}(x,\mathrm{u},\mathrm{v},\nabla {\mathrm{V}}^{*}\left(x\right))$ refers to the Hamiltonian function of the safety-critical system (1), that is,

$$\begin{array}{cc}\hfill \mathcal{H}(x,\mathrm{u},\mathrm{v},\nabla {\mathrm{V}}^{*}\left(x\right))=\nabla & {\mathrm{V}}^{*}{\left(x\right)}^T(\mathcal{F}\left(x\right)+\mathcal{G}\left(x\right)\mathrm{u}+\mathcal{P}\left(x\right)\mathrm{v})\hfill \\ \hfill & +x{\left(t\right)}^T\mathcal{Q}x\left(t\right)+\mathfrak{U}\left(\mathrm{u}\right)-{\mathrm{Y}}^2{∥\mathrm{v}∥}^2+{\mathcal{B}}_r\left(x\right).\hfill \end{array}$$

(12)

By using Equations (11) and (12), the saddle point can be found by addressing two equations as

$${\mathrm{u}}^{*}\left(x\right)=arg\underset{\mathrm{u}}{min}\mathcal{H}(x,\mathrm{u},\mathrm{v},\nabla {\mathrm{V}}^{*}\left(x\right)),$$

(13)

and

$${\mathrm{v}}^{*}\left(x\right)=arg\underset{\mathrm{v}}{max}\mathcal{H}(x,\mathrm{u},\mathrm{v},\nabla {\mathrm{V}}^{*}\left(x\right)).$$

(14)

Thus, the saddle point $({\mathrm{u}}^{*},{\mathrm{v}}^{*})$ can be gained as

$${\mathrm{u}}^{*}\left(x\right)=-\mathsf{\Psi }tanh(\frac{1}{2\mathsf{\Psi }}\mathcal{G}{\left(x\right)}^T\nabla {\mathrm{V}}^{*}\left(x\right))+{\psi }_{\Im },$$

(15)

and

$${\mathrm{v}}^{*}\left(x\right)=\frac{1}{2{\mathrm{Y}}^2}\mathcal{P}{\left(x\right)}^T\nabla {\mathrm{V}}^{*}\left(x\right),$$

(16)

where ${\psi }_{\Im }={[\Im ,\Im ,\dots ,\Im ]}^T\in {\mathcal{R}}^m$ with ℑ given by Equation (5).

Remark 4. 

Given that $\Im \ne 0$ from (5), it can be concluded that $\mathrm{u}\left(0\right)=\Im \ne 0$. Therefore, in order to establish the equilibrium point of (1) at $x=0$, the assumption of $\mathcal{G}\left(0\right)=0$ is necessary.

Substituting Equations (15) and (16) into Equation (11), the HJI equation can be redefined as

$$\begin{array}{cc}\hfill 0=& \nabla {\mathrm{V}}^{*}{\left(x\right)}^T(\mathcal{F}\left(x\right)+\mathcal{P}\left(x\right){\mathrm{v}}^{*})+x{\left(t\right)}^T\mathcal{Q}x\left(t\right)+\mathfrak{U}(-\mathsf{\Psi }tanh\left(T\left(x\right)\right)+{\psi }_{\Im })\hfill \\ \hfill & -{\mathrm{Y}}^2{∥{\mathrm{v}}^{*}∥}^2-{(\mathsf{\Psi }\nabla {\mathrm{V}}^{*}\left(x\right))}^T\mathcal{G}\left(x\right)tanh\left(T\left(x\right)\right)+\nabla {\mathrm{V}}^{*}{\left(x\right)}^T\mathcal{G}\left(x\right){\psi }_{\Im }+{\mathcal{B}}_r\left(x\right),\hfill \end{array}$$

(17)

where $T\left(x\right)=1/\left(2\mathsf{\Psi }\right)\mathcal{G}{\left(x\right)}^T\nabla {\mathrm{V}}^{*}\left(x\right)$ and ${\mathrm{V}}^{*}\left(0\right)=0$.

For the optimal safe control problem of the ZSG with unmatched external disturbances and asymmetric input constraints, it is necessary to obtain the value corresponding to the optimal cost Function (8) for achieving the optimal control input Policy (15) and the worst disturbance input Policy (16). Therefore, the solution of Equation (17) needs to be obtained. Nevertheless, since Equation (17) represents a nonlinear partial differential equation, it is challenging to find its analytical solution using conventional mathematical approaches. Hence, the solution of this equation is estimated by using the CNN in the next section.

4. Adaptive CNN Design

4.1. Solving the HJI Equation via the CNN

This section designs a CNN to estimate cost function ${\mathrm{V}}^{*}\left(x\right)$ as

$${\mathrm{V}}^{*}\left(x\right)={\mathrm{W}}_c^T\delta \left(x\right)+\xi \left(x\right),$$

(18)

where $\xi \left(x\right)$ represents the estimation error about the CNN with $\xi \left(0\right)=0$, ${\mathrm{W}}_c\in {\mathcal{R}}^{\mathfrak{r}}$ represents the ideal weight vector of the CNN, $\delta \left(x\right)=[{\delta }_1\left(x\right);{\delta }_2\left(x\right);\dots ;{\delta }_{\mathfrak{r}}\left(x\right)]$ represents activation function with ${\delta }_j\left(0\right)=0,j=1,2,\dots ,\mathfrak{r},\mathfrak{r}$ is the number of neurons in the CNN.

The gradient of the approximate optimal cost function is

$$\nabla {\mathrm{V}}^{*}\left(x\right)=\nabla \delta {\left(x\right)}^T{\mathrm{W}}_c+\nabla \xi \left(x\right).$$

(19)

Substituting Equation (19) into Equation (15), ${\mathrm{u}}^{*}\left(x\right)$ can be represented as

$${\mathrm{u}}^{*}\left(x\right)=-\mathsf{\Psi }tanh\left(\overline{A}\left(x\right)\right)+{\xi }_{{\mathrm{u}}^{*}}\left(x\right)+{\psi }_{\Im },$$

(20)

where

$$\overline{A}\left(x\right)=\frac{1}{2\mathsf{\Psi }}\mathcal{G}{\left(x\right)}^T\nabla \delta {\left(x\right)}^T{\mathrm{W}}_c,$$

(21)

and

$${\xi }_{{\mathrm{u}}^{*}}\left(x\right)=-\frac{1}{2}(I_m-\mathsf{\Phi }\left(\mathfrak{A}\left(x\right)\right))\mathcal{G}{\left(x\right)}^T\nabla \xi \left(x\right),$$

(22)

with $\mathsf{\Phi }\left(\mathfrak{A}\left(x\right)\right)=diag\left\{tan{h}^2\left({\mathfrak{A}}_l\left(x\right)\right)\right\}(\mathfrak{l}=1,2,\dots ,m)$ with ${\mathfrak{A}}_l\left(x\right)=[{\mathfrak{A}}_1\left(x\right);{\mathfrak{A}}_2\left(x\right);\dots ;$${\mathfrak{A}}_m\left(x\right)]\in {\mathcal{R}}^m$ being selected between $\overline{A}\left(x\right)$ and $T\left(x\right)$. Then, considering Equation (19), ${\mathrm{v}}^{*}\left(x\right)$ in Equation (16) can be redefined as

$${\mathrm{v}}^{*}\left(x\right)=\frac{1}{2{\mathrm{Y}}^2}\mathcal{P}{\left(x\right)}^T\nabla \delta {\left(x\right)}^T{\mathrm{W}}_c+{\xi }_{{\mathrm{v}}^{*}}\left(x\right),$$

(23)

where ${\xi }_{{\mathrm{v}}^{*}}\left(x\right)=\frac{1}{2{\mathrm{{\rm Y}}}^2}\mathcal{P}{\left(x\right)}^T\nabla \xi \left(x\right).$

Similarly, substituting Equation (19) into Equation (17), the HJI equation can be rewritten as

$$\begin{array}{cc}\hfill 0=& {\mathrm{W}}_c^T\nabla \delta \left(x\right)(\mathcal{F}\left(x\right)+\mathcal{P}\left(x\right){\mathrm{v}}^{*})+x^T\mathcal{Q}x+\mathfrak{U}(-\mathsf{\Psi }tanh(\overline{A}\left(x\right)+K\left(x\right)+{\psi }_{\Im }))+{\mathcal{B}}_r\left(x\right)\hfill \\ \hfill & -\mathsf{\Psi }{\mathrm{W}}_c^T\nabla \delta \left(x\right)\mathcal{G}\left(x\right)tanh(\overline{A}\left(x\right)+K\left(x\right))-\mathsf{\Psi }\nabla \xi {\left(x\right)}^T\mathcal{G}\left(x\right)tanh(\overline{A}\left(x\right)+K\left(x\right))\hfill \\ \hfill & +\nabla \xi {\left(x\right)}^T(\mathcal{F}\left(x\right)+\mathcal{P}\left(x\right){\mathrm{v}}^{*})-{\mathrm{Y}}^2{∥{\mathrm{v}}^{*}∥}^2+({\mathrm{W}}_c^T\nabla \delta \left(x\right)+\nabla \xi {\left(x\right)}^T)\mathcal{G}\left(x\right){\psi }_{\Im },\hfill \end{array}$$

(24)

where $K\left(x\right)=1/\left(2\mathsf{\Psi }\right)\mathcal{G}{\left(x\right)}^T\nabla \xi \left(x\right)$.

However, since the ideal CNN weight ${\mathrm{W}}_c$ in Equation (18) is unknown, it can not be used in the control procedure. Hence, the CNN is used to estimate the cost function and its gradient as

$$\widehat{\mathrm{V}}\left(x\right)={\widehat{\mathrm{W}}}_c^T\delta \left(x\right),$$

(25)

$$\nabla \widehat{\mathrm{V}}\left(x\right)=\nabla \delta {\left(x\right)}^T{\widehat{\mathrm{W}}}_c,$$

(26)

where ${\widehat{\mathrm{W}}}_c$ represents the estimation of ${\mathrm{W}}_c$.

Therefore, the approximate optimal input and the approximate worst disturbance input become

$${\widehat{\mathrm{u}}}^{*}\left(x\right)=-\mathsf{\Psi }tanh(\frac{1}{2\mathsf{\Psi }}\mathcal{G}{\left(x\right)}^T\nabla \delta {\left(x\right)}^T{\widehat{\mathrm{W}}}_c)+{\psi }_{\Im },$$

(27)

and

$${\widehat{\mathrm{v}}}^{*}\left(x\right)=\frac{1}{2{\mathrm{Y}}^2}\mathcal{G}{\left(x\right)}^T\nabla \delta {\left(x\right)}^T{\widehat{\mathrm{W}}}_c.$$

(28)

Subsequently, the approximated Hamilton function can be formulated by

$$\begin{array}{cc}\hfill \widehat{\mathcal{H}}(x,{\widehat{\mathrm{W}}}_c,{\widehat{\mathrm{v}}}^{*})=& {\widehat{\mathrm{W}}}_c^Tð+{\widehat{\mathrm{W}}}_c^T\nabla \delta \left(x\right)\mathcal{G}\left(x\right){\psi }_{\Im }+\mathfrak{U}(-\mathsf{\Psi }tanh(\Gamma \left(x\right))+{\psi }_{\Im })\hfill \\ \hfill & +x^T\mathcal{Q}x-{\mathrm{Y}}^2{∥{\widehat{\mathrm{W}}}_c∥}^2-\mathsf{\Psi }{\widehat{\mathrm{W}}}_c^T\nabla \delta \left(x\right)\mathcal{G}\left(x\right)tanh(\Gamma \left(x\right))+{\mathcal{B}}_r\left(x\right),\hfill \end{array}$$

(29)

where

$$ð=\nabla \delta \left(x\right)(\mathcal{F}\left(x\right)+\mathcal{P}\left(x\right){\widehat{\mathrm{v}}}^{*})$$

(30)

and

$$\Gamma \left(x\right)=\frac{1}{2\mathsf{\Psi }}\mathcal{G}{\left(x\right)}^T\nabla \delta {\left(x\right)}^T{\widehat{\mathrm{W}}}_c.$$

(31)

The CNN weight estimation error is denoted by

$${\tilde{\mathrm{W}}}_c={\mathrm{W}}_c-{\widehat{\mathrm{W}}}_c,$$

(32)

and the approximation error ${\varrho }_{\mathfrak{c}}$ of the Hamiltonian function is derived as

$$\begin{array}{cc}\hfill {\varrho }_{\mathfrak{c}}& =\widehat{\mathcal{H}}(x,{\widehat{\mathrm{W}}}_c,{\widehat{\mathrm{v}}}^{*})-\mathcal{H}(x,{\mathrm{u}}^{*},{\mathrm{v}}^{*},\nabla {\mathrm{V}}^{*}\left(x\right))\hfill \\ \hfill & =\widehat{\mathcal{H}}(x,{\widehat{\mathrm{W}}}_c,{\widehat{\mathrm{v}}}^{*}).\hfill \end{array}$$

(33)

To achieve ${\widehat{\mathrm{W}}}_c\to {\mathrm{W}}_c$, it is necessary to ensure that ${\varrho }_{\mathfrak{c}}\to 0$. Therefore, the chosen target function is denoted by $\mathrm{E}=\frac{1}{2}{\varrho }_{\mathfrak{c}}^T{\varrho }_{\mathfrak{c}}(1/{(1+{ð}^Tð)}^2)$, where $\mathcal{O}=1+{ð}^Tð$. Consequently, based on a normalized gradient descent algorithm, the weight vector ${\widehat{\mathrm{W}}}_c$ is defined by

$${\dot{\widehat{\mathrm{W}}}}_c=-\frac{\alpha }{{\mathcal{O}}^2}\frac{\partial \mathrm{E}}{\partial {\widehat{\mathrm{W}}}_c}=-\frac{\alpha }{{\mathcal{O}}^2}{\varrho }_{\mathfrak{c}},$$

(34)

with $\alpha >0$ being the adjustable parameter and ${\varrho }_{\mathfrak{c}}$ defined as Equation (33).

Using Equations (32) and (34), the weight approximation error ${\dot{\tilde{\mathrm{W}}}}_c$ can be expressed as

$${\dot{\tilde{\mathrm{W}}}}_c=\frac{\alpha \zeta }{\mathcal{O}}{\xi }_c-\alpha \zeta {\zeta }^T{\tilde{\mathrm{W}}}_c,$$

(35)

where ${\xi }_c=-\nabla \xi {\left(x\right)}^T(\mathcal{F}\left(x\right)+\mathcal{P}\left(x\right){\widehat{\mathrm{v}}}^{*})$ is the residual error and $\zeta =\frac{ð}{\mathcal{O}}$.

4.2. Stability Analysis

The UUB of both the state x and the CNN parameters in the closed-loop system is demonstrated by utilizing the Lyapunov stability analysis principle in this subsection. First, two assumptions that were also used in [28,42] are required, as

Assumption 2. 

The ideal optimal CNN weight vector ${\mathrm{W}}_c$ is upper bounded, i.e., $∥{\mathrm{W}}_c∥\le b_{{\mathrm{W}}_c}$, where $b_{{\mathrm{W}}_c}>0$ is a constant. Moreover, for any $x\in \mho$, this paper assumes that there are two known constants $b_{\nabla \delta }>0$, $b_{\delta }>0$ so that $∥\nabla \delta \left(x\right)∥\le b_{\nabla \delta }, ∥\delta \left(x\right)∥\le b_{\delta }$. Meanwhile, there exist $b_{\nabla \xi }>0$ and $b_{\xi }>0$ so that $∥\nabla \xi \left(x\right)∥\le b_{\nabla \xi },∥\xi \left(x\right)∥\le b_{\xi }$ for any $x\in \mho$.

Assumption 3. 

We make $b_{{\xi }_{{\mathrm{u}}^{*}}},b_{{\xi }_{{\mathrm{v}}^{*}}},b_{{\xi }_c}$ be positive constants.

(1) 

$b_{{\xi }_{{\mathrm{u}}^{*}}}\ge ∥{\xi }_{{\mathrm{u}}^{*}}\left(x\right)∥$ for any $x\in \mho$.

(2) 

$b_{{\xi }_{{\mathrm{v}}^{*}}}\ge ∥{\xi }_{{\mathrm{v}}^{*}}\left(x\right)∥$ for any $x\in \mho$.

(3) 

$b_{{\xi }_c}\ge ∥{\xi }_c∥$ for any $x\in \mho$.

Theorem 1. 

Assuming Assumptions 1–3 are met, we consider System (1) with the associated Control (27) and the update rule of CNN (34), ensuring all signals in the nonlinear system are UUB if the following condition holds:

$$\alpha {\mathbb{k}}_{min}\left(\zeta {\zeta }^T\right)-(1/{\mathrm{Y}}^2){\Im }_{\nabla \delta }^2{\mathcal{P}}_M^2>0.$$

(36)

Proof. 

We let the Lyapunov candidate function as the following (note: for convenience, ${\mathrm{V}}^{*}\left(x\right)$ and $(1/2){\tilde{\mathrm{W}}}_c^T{\tilde{\mathrm{W}}}_c$ are abbreviated as $L_1$ and $L_2$ below):

$$L\left(t\right)=\underset{L_1}{\underbrace{{\mathrm{V}}^{*}\left(x\right)}}+\underset{L_2}{\underbrace{(1/2){\tilde{\mathrm{W}}}_c^T{\tilde{\mathrm{W}}}_c}}.$$

(37)

Taking the derivation of $L_1$ in Equation (37) and using System (1), the derivation of $L_1$ can be expressed as

$$\begin{array}{cc}\hfill {\dot{L}}_1& =\frac{\mathrm{d}{\mathrm{V}}^{*}\left(x\right)}{\mathrm{d}t}\hfill \\ \hfill & =\nabla {\mathrm{V}}^{*}{\left(x\right)}^T(\mathcal{F}\left(x\right)+\mathcal{G}\left(x\right){\widehat{\mathrm{u}}}^{*}+\mathcal{P}\left(x\right){\widehat{\mathrm{v}}}^{*})\hfill \\ \hfill & =\nabla {\mathrm{V}}^{*}{\left(x\right)}^T(\mathcal{F}\left(x\right)+\mathcal{G}\left(x\right){\mathrm{u}}^{*}+\mathcal{P}\left(x\right){\mathrm{v}}^{*})\hfill \\ \hfill & +\nabla {\mathrm{V}}^{*}{\left(x\right)}^T\mathcal{P}\left(x\right)({\widehat{\mathrm{v}}}^{*}-{\mathrm{v}}^{*})+\nabla {\mathrm{V}}^{*}{\left(x\right)}^T\mathcal{G}\left(x\right)({\widehat{\mathrm{u}}}^{*}-{\mathrm{u}}^{*}).\hfill \end{array}$$

(38)

Then, using Equations (12) and (11), it can be derived as

$$\begin{array}{c}\hfill \nabla {\mathrm{V}}^{*}{\left(x\right)}^T(\mathcal{F}\left(x\right)+\mathcal{G}\left(x\right){\mathrm{u}}^{*}+\mathcal{P}\left(x\right){\mathrm{v}}^{*})=-x^T\mathcal{Q}x-\mathfrak{U}\left({\mathrm{u}}^{*}\right)+{\mathrm{Y}}^2{∥{\mathrm{v}}^{*}∥}^2-{\mathcal{B}}_r\left(x\right).\end{array}$$

(39)

Similarly, taking into account Equations (27) and (28), the derived results are

$$\nabla {\mathrm{V}}^{*}{\left(x\right)}^T\mathcal{G}\left(x\right)=2\mathsf{\Psi }{\left(tan{h}^{-1}(({\psi }_{\Im }-{\mathrm{u}}^{*})/\left(\mathsf{\Psi }\right))\right)}^T,$$

(40)

and

$$\nabla {\mathrm{V}}^{*}{\left(x\right)}^T\mathcal{P}\left(x\right)=2{\mathrm{Y}}^2{\mathrm{v}}^{*T}.$$

(41)

According to Equations (38)–(41), Equation (38) can be rewritten as follows (note: for convenience, $\overline{\omega }-\mathfrak{U}\left({\mathrm{u}}^{*}\right)$ and $2{\mathrm{Y}}^2{\mathrm{v}}^{*T}{\widehat{\mathrm{v}}}^{*}-{\mathrm{Y}}^2{∥{\mathrm{v}}^{*}∥}^2-{\mathcal{B}}_r\left(x\right)$ are abbreviated as ${\Lambda }_1$ and ${\Lambda }_2$ below):

$${\dot{L}}_1=-x^T\mathcal{Q}x+\underset{{\Lambda }_1}{\underbrace{\overline{\omega }-\mathfrak{U}\left({\mathrm{u}}^{*}\right)}}+\underset{{\Lambda }_2}{\underbrace{2{\mathrm{Y}}^2{\mathrm{v}}^{*T}{\widehat{\mathrm{v}}}^{*}-{\mathrm{Y}}^2{∥{\mathrm{v}}^{*}∥}^2-{\mathcal{B}}_r\left(x\right)}},$$

(42)

where

$$\begin{array}{c}\hfill \overline{\omega }=2\mathsf{\Psi }\left(tan{h}^{-1}(({\psi }_{\Im }-{\mathrm{u}}^{*})/\mathsf{\Psi })\right)({\widehat{\mathrm{u}}}^{*}-{\mathrm{u}}^{*}).\end{array}$$

(43)

We apply Young’s inequality to Equation (43). Additionally, considering Equations (19), (20), (27), (40) and (41), $\overline{\omega }$ can be formulated as

$$\begin{array}{cc}\hfill \overline{\omega }& \le {∥\mathsf{\Psi }\left(tan{h}^{-1}(({\psi }_{\Im }-{\mathrm{u}}^{*})/\mathsf{\Psi })\right)∥}^2+{∥{\widehat{\mathrm{u}}}^{*}-{\mathrm{u}}^{*}∥}^2\hfill \\ \hfill & =\frac{1}{4}{∥\mathcal{G}{\left(x\right)}^T\nabla {\mathrm{V}}^{*}\left(x\right)∥}^2+{∥{\widehat{\mathrm{u}}}^{*}-{\mathrm{u}}^{*}∥}^2\hfill \\ \hfill & =\frac{1}{4}{∥\mathcal{G}{\left(x\right)}^T(\nabla \delta {\left(x\right)}^T{\mathrm{W}}_c+\nabla \xi \left(x\right))∥}^2\hfill \\ \hfill & +{\parallel -\mathsf{\Psi }tanh(\Gamma \left(x\right))+\mathsf{\Psi }tanh\left(\overline{A}\left(x\right)\right)-{\xi }_{{\mathrm{u}}^{*}}\left(x\right)\parallel }^2.\hfill \end{array}$$

(44)

Furthermore, utilizing Young’s inequality, $\overline{\omega }$ in Equation (44) further yields

$$\begin{array}{cc}\hfill \overline{\omega }\le & 2{∥-\mathsf{\Psi }tanh(\Gamma \left(x\right))+\mathsf{\Psi }tanh\left(\overline{A}\left(x\right)\right)∥}^2+2{∥{\xi }_{{\mathrm{u}}^{*}}\left(x\right)∥}^2\hfill \\ & +\frac{1}{2}\left({∥\mathcal{G}{\left(x\right)}^T\nabla \delta {\left(x\right)}^T{\mathrm{W}}_c∥}^2\right)+\frac{1}{2}\left({∥\mathcal{G}{\left(x\right)}^T\nabla \xi \left(x\right)∥}^2\right)\hfill \\ \hfill \le & 4{∥\mathsf{\Psi }tanh(\Gamma (x\left)\right)∥}^2+{∥\mathsf{\Psi }tanh\left(\overline{A}\left(x\right)\right)∥}^2+2{∥{\xi }_{{\mathrm{u}}^{*}}\left(x\right)∥}^2\hfill \\ & +\frac{1}{2}\left({∥\mathcal{G}{\left(x\right)}^T\nabla \delta {\left(x\right)}^T{\mathrm{W}}_c∥}^2\right)+\frac{1}{2}\left({∥\mathcal{G}{\left(x\right)}^T\nabla \xi \left(x\right)∥}^2\right).\hfill \end{array}$$

(45)

According to Equations (21) and (31), the following inequalities can be depicted as

$${∥tanh(\Gamma (x\left)\right)∥}^2=tan{h}^2(\Gamma \left(x\right))\le m$$

(46)

and

$${∥tanh\left(\overline{A}\left(x\right)\right)∥}^2=tan{h}^2\left(\overline{A}\left(x\right)\right)\le m.$$

(47)

Based on Equation (46) and Assumptions 2 and 3, $\overline{\omega }$ can be expressed as

$$\begin{array}{c}\hfill \overline{\omega }\le 8{\mathsf{\Psi }}^{2}m+\frac{1}{2}{\mathcal{G}}_M^2({\Im }_{\nabla \delta }^2{\Im }_{{\mathrm{W}}_c}^2+{\Im }_{\nabla \xi }^2)+2b_{{\xi }_{{\mathrm{u}}^{*}}}^2.\end{array}$$

(48)

By observing Equations (4) and (5), it can be concluded that $\mathfrak{U}\left({\mathrm{u}}^{*}\right)>0$. Using Young’s inequality and Equation (48), the expression of ${\Lambda }_1$ in Equation (42) can be rewritten as

$$\begin{array}{c}\hfill {\Lambda }_1\le 8{\mathsf{\Psi }}^{2}m+\frac{1}{2}{\mathcal{G}}_M^2({\Im }_{\nabla \delta }^2{\Im }_{\nabla {\mathrm{W}}_c}^2+{\Im }_{\nabla \xi }^2)+2b_{{\xi }_{{\mathrm{u}}^{*}}}^2.\end{array}$$

(49)

Similarly, ${\Lambda }_2$ in Equation (42) can be rewritten as follows (note: from Assumption 1, ${\mathcal{B}}_r\left(x\right)$$\ge 0$):

$$\begin{array}{cc}\hfill {\Lambda }_2& ={\mathrm{Y}}^2{∥{\widehat{\mathrm{v}}}^{*}∥}^2-{\mathcal{B}}_r\left(x\right)\hfill \\ \hfill & =-{\mathrm{Y}}^2{∥{\mathrm{v}}^{*}∥}^2+{\mathrm{Y}}^2{∥{\mathrm{v}}^{*}∥}^2+{\mathrm{Y}}^2{∥{\widehat{\mathrm{v}}}^{*}∥}^2-{\mathcal{B}}_r\left(x\right)\hfill \\ \hfill & \le -{\mathrm{Y}}^2{∥{\mathrm{v}}^{*}∥}^2+{\mathrm{Y}}^2{∥{\mathrm{v}}^{*}∥}^2+{\mathrm{Y}}^2{∥{\widehat{\mathrm{v}}}^{*}∥}^2\hfill \\ \hfill & \le -{\mathrm{Y}}^2{∥{\mathrm{v}}^{*}∥}^2+{\mathrm{Y}}^2({∥{\mathrm{v}}^{*}∥}^2+{∥{\widehat{\mathrm{v}}}^{*}∥}^2)\hfill \\ \hfill & =(1/\left(4{\mathrm{Y}}^2\right)){∥\mathcal{P}{\left(x\right)}^T\nabla \delta {\left(x\right)}^T({\mathrm{W}}_c-{\tilde{\mathrm{W}}}_c)∥}^2.\hfill \end{array}$$

(50)

Meanwhile, using Young’s inequality and Assumptions 1 and 3, ${\Lambda }_2$ in Equation (50) further yields

$$\begin{array}{cc}\hfill {\Lambda }_2& \le (1/\left(4{\mathrm{Y}}^2\right)){\mathcal{P}}_M^2{\Im }_{\nabla \delta }^2{∥{\mathrm{W}}_c-{\tilde{\mathrm{W}}}_c∥}^2\hfill \\ \hfill & \le (1/\left(2{\mathrm{Y}}^2\right)){\mathcal{P}}_M^2{\Im }_{\nabla \delta }^2({\Im }_{{\mathrm{W}}_c}^2+{∥{\tilde{\mathrm{W}}}_c∥}^2).\hfill \end{array}$$

(51)

Hence, by observing Equations (49) and (51), it can be inferred that ${\dot{L}}_1$ in Equation (42) satisfies

$$\begin{array}{cc}\hfill {\dot{L}}_1\le & -{\mathbb{k}}_{min}\left(\mathcal{Q}\right){∥x∥}^2+(1/\left(2{\mathrm{Y}}^2\right)){\mathcal{P}}_M^2{\Im }_{\nabla \delta }^2{\Im }_{{\mathrm{W}}_c}^2+8{\mathsf{\Psi }}^{2}m+2b_{{\xi }_{{\mathrm{u}}^{*}}}^2\hfill \\ \hfill & +(1/2){\mathcal{G}}_M^2({\Im }_{\nabla \delta }^2{\Im }_{{\mathrm{W}}_c}^2+{\Im }_{\nabla \xi }^2)+(1/\left(2{\mathrm{Y}}^2\right)){\mathcal{P}}_M^2{\Im }_{\nabla \delta }^2{∥{\tilde{\mathrm{W}}}_c∥}^2.\hfill \end{array}$$

(52)

Then, the derivative of $L_2$ in Equation (37) along the solution of Equation (34) is as follows (note: $\alpha {\tilde{\mathrm{W}}}_c^T(\zeta /\mathcal{O}){\xi }_c$ is abbreviated as ${\Lambda }_3$ below):

$${\dot{L}}_2={\tilde{\mathrm{W}}}_c^T{\dot{\tilde{\mathrm{W}}}}_c=\underset{{\Lambda }_3}{\underbrace{\alpha {\tilde{\mathrm{W}}}_c^T(\zeta /\mathcal{O}){\xi }_c}}-\alpha {\tilde{\mathrm{W}}}_c^T\zeta {\zeta }^T{\tilde{\mathrm{W}}}_c.$$

(53)

Immediately after, using Young’s inequality, ${\Lambda }_3$ can be depicted as

$$\begin{array}{cc}\hfill {\Lambda }_3& \le \frac{\alpha }{2\mathcal{O}}({∥{\zeta }^T{\tilde{\mathrm{W}}}_c∥}^2+{∥{\xi }_c∥}^2)\hfill \\ \hfill & \le \alpha (\frac{1}{2}{\tilde{\mathrm{W}}}_c^T\zeta {\zeta }^T{\tilde{\mathrm{W}}}_c+\frac{1}{2}{∥{\xi }_c∥}^2).\hfill \end{array}$$

(54)

Additionally, with Assumption 3 holding, it can be deduced that ${\dot{L}}_2$ in Equation (53) satisfies

$$\begin{array}{cc}\hfill {\dot{L}}_2& \le \frac{1}{2}(-\alpha {\tilde{\mathrm{W}}}_c^T\zeta {\zeta }^T{\tilde{\mathrm{W}}}_c+\alpha {∥{\xi }_c∥}^2)\hfill \\ \hfill & \le \frac{1}{2}(-\alpha {\mathbb{k}}_{min}\left(\zeta {\zeta }^T\right){∥{\tilde{\mathrm{W}}}_c∥}^2+\alpha {\Im }_{{\xi }_c}^2).\hfill \end{array}$$

(55)

Using Equations (37), (52) and (55), $\dot{L}$ can be depicted as

$$\begin{array}{cc}\hfill \dot{L}\le & -{\mathbb{k}}_{min}\left(\mathcal{Q}\right){∥x∥}^2+(1/\left(2{\mathrm{Y}}^2\right){\mathcal{P}}_M^2{\Im }_{\nabla \delta }^2{\Im }_W^2)+(1/2){\mathcal{G}}_M^2({\Im }_{\nabla \delta }^2{\Im }_W^2+{\Im }_{\nabla \xi }^2)\hfill \\ \hfill & -(1/2)(\alpha {\mathbb{k}}_{min}\zeta {\zeta }^T-(1/{\mathrm{Y}}^2){\mathcal{P}}_M^2{\Im }_{\nabla \delta }^2){∥{\tilde{\mathrm{W}}}_c∥}^2+8{\mathsf{\Psi }}^{2}m+2b_{{\xi }_{{\mathrm{u}}^{*}}}^2+(\alpha /2){\Im }_{{\xi }_c}^2.\hfill \end{array}$$

(56)

Finally, $\dot{L}<0$ is true if $x\notin \mho \left(x\right)$ or ${\tilde{\mathrm{W}}}_c\notin \mho \left({\tilde{\mathrm{W}}}_c\right)$, and based on Equation (36), $\mho \left(x\right)$ and $\mho \left({\tilde{\mathrm{W}}}_c\right)$ can be respectively formulated as

$$\begin{array}{c}\hfill \mho \left(x\right)=\left\{∥x∥\le \sqrt{\frac{\alpha {\Im }_{{\xi }_c}^2+\Xi +A_1{\mathcal{P}}_M^2}{2{\mathbb{k}}_{min}\left(\mathcal{Q}\right)}}\right\},\end{array}$$

(57)

and

$$\begin{array}{c}\hfill \mho \left({\tilde{\mathrm{W}}}_c\right)=\left\{∥{\tilde{\mathrm{W}}}_c∥\le \sqrt{\frac{\alpha {\Im }_{{\xi }_c}^2+\Xi +A_1{\mathcal{P}}_M^2}{\alpha {\mathbb{k}}_{min}\left(\zeta {\zeta }^T\right)-A_2{\mathcal{P}}_M^2}}\right\},\end{array}$$

(58)

where $\Xi ={\mathcal{G}}_M^2({\Im }_{\nabla \delta }^2{\Im }_{{\mathrm{W}}_c}^2+{\Im }_{\nabla \xi }^2)+16{\mathsf{\Psi }}^{2}m+4b_{{\xi }_{{\mathrm{u}}^{*}}}^2$, $A_1=(1/{\mathrm{Y}}^2){\Im }_{\nabla \delta }^2{\Im }_{{\mathrm{W}}_c}^2$ and $A_2=(1/{\mathrm{Y}}^2){\Im }_{\nabla \delta }^2$.

To summarize, the Lyapunov stability method has been used to demonstrate the state x of Equation (1) and ${\tilde{\mathrm{W}}}_c$ are UUB, with Equations (57) and (58) representing their respective bounds. The proof is complete. □

5. Simulation Study

Within this section, two examples are utilized to validate the efficacy of the proposed approach.

5.1. Example 1

Consider the F16 aircraft plant used in [28] as

$$\dot{x}=\mathcal{F}\left(x\right)+\mathcal{G}\left(x\right)\mathrm{u}+\mathcal{P}\left(x\right)\mathrm{v},$$

(59)

where $x\left(t\right)={[x_1,x_2,x_3]}^T\in {\mathcal{R}}^3$ with $x_0={[1,-1,1]}^T$ represents the system state vector, where $x_1, x_2$ and $x_3$ represent the attack angle, the pitch rate, and the elevator deflection angle, respectively. $\mathrm{u}$ is control input, $\mathrm{v}$ is disturbance input. The internal dynamics, control, and disturbance coefficient matrices are expressed as

$$\mathcal{F}\left(x\right)=\left[\begin{array}{c}-1.01887x_1+0.90506x_2-0.00215x_3\\ 0.82225x_1-1.07741x_2-0.17555x_3\\ -x_3\end{array}\right],\mathcal{G}\left(x\right)=\left[\begin{array}{c}0\\ 0\\ 1\end{array}\right],\mathcal{P}\left(x\right)=\left[\begin{array}{c}0\\ 0\\ -1\end{array}\right].$$

The control input $\mathrm{u}$ is constrained to be greater than −1 and less than 2. Hence, $\mathsf{\Psi }=1.5$ and $\Im =0.5$. And then, the danger region is described as a ball with a radius of $0.15$ and a center at ${[0.3,0.05,-0.05]}^T$. The $\mathrm{y}\left(x\right)$ is chosen as

$$\frac{1.5\sqrt{{(x_1-0.3)}^2+0.1{(x_2-0.05)}^2+1.2{(x_3+0.05)}^2}-0.15}{\sqrt{{(x_1-0.3)}^2+{(x_2-0.05)}^2+25{(x_3+0.05)}^2}-0.15}.$$

The $\mathrm{z}\left(x\right)$ is chosen as

$$\sqrt{{(x_1-0.3)}^2+{(x_2-0.05)}^2+{(x_3+0.05)}^2}-0.15.$$

In addition, substituting $\mathsf{\Psi }$ and ℑ into Equation (4), $\mathfrak{U}\left(\mathrm{u}\right)$ can be expressed as

$$\begin{array}{cc}\hfill \mathfrak{U}\left(\mathrm{u}\right)& =2\mathsf{\Psi }(\mathrm{u}-\Im )tan{h}^{-1}\left(\frac{\mathrm{u}-\Im }{\mathsf{\Psi }}\right)+{\mathsf{\Psi }}^2{ln}_{}(1-\frac{{(\mathrm{u}-\Im )}^2}{\mathsf{\Psi }})\hfill \\ \hfill & =3(\mathrm{u}-0.5)tan{h}^{-1}\left(\frac{\mathrm{u}-0.5}{1.5}\right)+2.25{ln}_{}(1-\frac{{(\mathrm{u}-0.5)}^2}{1.5}).\hfill \end{array}$$

(60)

Letting $\mathcal{Q}=I_3$ and $\mathrm{Y}=2$, the cost function for Equation (62) is formulated as

$$\mathrm{V}\left(x\right)={\int }_0^{\infty }x{\left(t\right)}^T\mathcal{Q}x\left(t\right)+\mathfrak{U}\left(\mathrm{u}\right)-2^2{∥\mathrm{v}∥}^2+{\mathcal{B}}_r\left(x\right)dt,$$

(61)

where ${\mathcal{B}}_r\left(x\right)=\rho \frac{\mathrm{y}\left(x\right)}{\mathrm{z}\left(x\right)}$ represents the CBF and $\rho =2$.

The activation function is given as $\delta \left(x\right)={[x_1^2,x_1{x}_2,x_1{x}_3,x_2^2,x_2{x}_3,x_3^2]}^T$ and the CNN weight vector is ${\widehat{\mathrm{W}}}_c={[{\widehat{\mathrm{W}}}_{c1},{\widehat{\mathrm{W}}}_{c2},{\widehat{\mathrm{W}}}_{c3},{\widehat{\mathrm{W}}}_{c4},{\widehat{\mathrm{W}}}_{c5},{\widehat{\mathrm{W}}}_{c6}]}^T$. In addition, the adjustable parameter $\alpha$ is 10, and the original parameters of the CNN are configured as 1. At last, the probing noise $exp(-0.1t)\left(0.001\right)(sin{\left(t\right)}^{2}cos\left(t\right)+sin{\left(2t\right)}^{2}cos\left(0.1t\right))$ is added to the control input policy for the initial 30 s in order to ensure the persistence of the excitation.

Through simulation experiments, Figure 1, Figure 2, Figure 3, Figure 4, Figure 5, Figure 6 and Figure 7 are obtained. Figure 1 displays that ${\widehat{\mathrm{W}}}_c$ is convergent after the first 10 s, and can know the ideal vector ${\mathrm{W}}_c^{*}=[16.4603,-6.5022,$${-4.3910,4.8851,3.7081,11.6158]}^T$. Figure 2 displays the convergence of the states $x_1$, $x_2$, and $x_3$. Figure 3 displays the danger region, which is represented by the ball, and the original states are in the danger area. However, the system states controlled by the safe optimal controller bypass this ball, and as the damping coefficient $\rho$ increases, the distance between the system states and the dangerous region becomes larger and larger. Figure 3 shows that as states $x_1$, $x_2$, and $x_3$ gradually approach the danger zone, the convergence of $x_3$ is accelerated due to the CBF and cost function. Figure 4 presents the control input $\mathrm{u}$ with asymmetric input constraints. The plot reveals that the value of $\mathrm{u}$ remains within the specified range, bounded by ${\mathrm{u}}_{max}=2$ and ${\mathrm{u}}_{min}=-1$, providing evidence that the asymmetric input constraints are implemented successfully. Figure 5 presents the disturbance input $\mathrm{v}$. Figure 6 presents the cost function of the system. It can be seen that when the system states confront the danger area, the cost function changes significantly and eventually converges to zero. According to the principle of optimal control, when the cost function converges to zero, the following conclusion can be drawn: The cost function imposes a higher penalty on control actions that do not comply with the asymmetric input constraints and safety constraints. Therefore, when the cost function converges to zero, the system finds the optimal control actions that satisfy all the constraints.

In order to further show the efficiency of the presented method, Equation (4) is redefined as ${\mathrm{u}}^{T}R\mathrm{u}$ (where R = $I_1$), and the simulation results are illustrated in Figure 7. Subsequently, Figure 4 illustrates the control input, which is restricted to the limits of −1 to 2. This can be observed by comparing it with Figure 7, where the input is clearly outside this range.

5.2. Example 2

We consider the nonlinear system as

$$\dot{x}=\mathcal{F}\left(x\right)+\mathcal{G}\left(x\right)\mathrm{u}+\mathcal{P}\left(x\right)\mathrm{v},$$

(62)

where $x\left(t\right)={[x_1,x_2]}^T\in {\mathcal{R}}^2$ with $x_0={[1,-1]}^T$ represents the system state vector; the internal dynamics, control, and disturbance coefficient matrices are expressed as

$$\mathcal{F}\left(x\right)=\left[\begin{array}{c}-\frac{1}{2}{x}_1+x_2\\ -2x_{2}cos\left(2x_1\right)\end{array}\right],\mathcal{G}\left(x\right)=\left[\begin{array}{c}0\\ -x_1\end{array}\right],\mathcal{P}\left(x\right)=\left[\begin{array}{c}0\\ x_1\end{array}\right].$$

Just like F16, the control input $\mathrm{u}$ is subject to an asymmetrical boundary, with a lower bound of −1 and an upper bound of 3, establishing its limits. Hence, $\mathsf{\Psi }=2$ and $\Im =1$. And then, the danger region is described as a circle with radius $=0.1$, and the center of the circle is ${[0.19,-0.12]}^T$. The $\mathrm{y}\left(x\right)$ is chosen as

$$atan\left(\frac{1}{\sqrt{{(x_1-0.19)}^2+{(x_2+0.12)}^2}-0.1}\right).$$

The $\mathrm{z}\left(x\right)$ is chosen as

$$\sqrt{{(x_1-0.19)}^2+{(x_2+0.12)}^2}-0.1.$$

In addition, substituting $\mathsf{\Psi }$ and ℑ into Equation (4), $\mathfrak{U}\left(\mathrm{u}\right)$ can be expressed as

$$\begin{array}{cc}\hfill \mathfrak{U}\left(\mathrm{u}\right)& =2\mathsf{\Psi }(\mathrm{u}-\Im )tan{h}^{-1}\left(\frac{\mathrm{u}-\Im }{\mathsf{\Psi }}\right)+{\mathsf{\Psi }}^2{ln}_{}(1-\frac{{(\mathrm{u}-\Im )}^2}{\mathsf{\Psi }})\hfill \\ \hfill & =4(\mathrm{u}-1)tan{h}^{-1}\left(\frac{\mathrm{u}-1}{2}\right)+4{ln}_{}(1-\frac{{(\mathrm{u}-1)}^2}{2}).\hfill \end{array}$$

(63)

Letting $\mathcal{Q}=I_2$ and $\mathrm{Y}=1.35$, the cost function for Equation (62) is formulated as

$$\mathrm{V}\left(x\right)={\int }_0^{\infty }x{\left(t\right)}^T\mathcal{Q}x\left(t\right)+\mathfrak{U}\left(\mathrm{u}\right)-{1.35}^2{∥\mathrm{v}∥}^2+{\mathcal{B}}_r\left(x\right)dt,$$

(64)

where ${\mathcal{B}}_r\left(x\right)=\rho \frac{\mathrm{y}\left(x\right)}{\mathrm{z}\left(x\right)}$ represents the CBF and $\rho =0.3$.

Then, the CNN presented as Equation (18) is applied to address the HJI equation for Equation (62). The activation function is given as $\delta \left(x\right)={[x_1^2,x_1{x}_2,x_2^2,x_1^4,x_1^3{x}_2,x_1^2{x}_2^2,x_1{x}_2^3,x_2^4]}^T$ and the CNN weight vector is ${\widehat{\mathrm{W}}}_c={[{\widehat{\mathrm{W}}}_{c1},{\widehat{\mathrm{W}}}_{c2},{\widehat{\mathrm{W}}}_{c3},{\widehat{\mathrm{W}}}_{c4},{\widehat{\mathrm{W}}}_{c5},{\widehat{\mathrm{W}}}_{c6},{\widehat{\mathrm{W}}}_{c7},{\widehat{\mathrm{W}}}_{c8}]}^T$. In addition, the adjustable parameter $\alpha$ is 20, the original parameters of the CNN are configured as 1. At last, the probing noise $exp(-0.001t)(-0.1(sin{\left(t\right)}^{2}cos\left(t\right)+sin{\left(t\right)}^5+sin{\left(2t\right)}^{2}cos\left(0.1t\right)+sin{(-1.2t)}^{2}cos\left(0.5t\right))$ is added to the control input policy for the initial 30 s.

Through simulation experiments, Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13 and Figure 14 are obtained. Figure 8 displays that ${\widehat{\mathrm{W}}}_c$ is convergent after the first 10 s, and can know the ideal vector ${\mathrm{W}}_c^{*}=[84.6487,-12.2017,$${9.5269,11.7425,-3.0924,3.4273,-0.5533,2.0591]}^T$. Figure 9 displays the convergence of the states $x_1$ and $x_2$. Figure 10 illustrates the relationship between the system states and the dangerous area, revealing that increasing the damping factor $\rho$ leads to a greater distance between the system states and the dangerous zone. Evidently, system states $x_1$ and $x_2$ with a safe and optimal controller take an alternate route to avoid the dangerous region, while the conventional optimal controller cannot circumvent the dangerous region. As can be seen from Figure 10, when states $x_1$ and $x_2$ gradually approach the danger zone, the convergence speed of $x_2$ is accelerated due to the influence of CBF and cost function and obtains an optimal trajectory around the danger zone again. Figure 11 shows input $\mathrm{u}$ with asymmetric input constraints. The plot reveals that the value of $\mathrm{u}$ remains within the specified range, bounded by ${\mathrm{u}}_{max}=3$ and ${\mathrm{u}}_{min}=-1$, providing evidence that the asymmetric input constraints are implemented successfully. Figure 12 presents disturbance input $\mathrm{v}$. Figure 13 presents the cost function of the system. It can be seen that the cost function eventually converges to zero. Similar to the linear system, when the cost function converges to zero, it can be concluded that the system finds the optimal control action that satisfies the asymmetric input constraints and safety constraints.

In this paper, asymmetric input constraints and unmatched disturbances are applied to nonlinear safety-critical systems for the first time, and Equation (4) is used to handle the asymmetric input constraints. To further demonstrate the efficacy of the presented algorithm, as in articles [14,16,28], (4) is redefined as ${\mathrm{u}}^{T}R\mathrm{u}$ (where R = $I_1$) and the simulation results are shown in Figure 14. Subsequently, the control input in Figure 11 is constrained to fall within the limits of −1 to 3, as can be observed by comparing it with Figure 14, while the input in Figure 14 is clearly outside this range.

6. Conclusions

The safe and optimal control problem of the nonlinear CT safety-critical systems with asymmetric input constraints and unmatched disturbances was addressed. Firstly, the new non-quadratic form function was considered for addressing the issue of asymmetric input constraints. Then, the control design was transformed into the two-player ZSG problem to handle unmatched disturbances. In order to obtain the optimal controller for safety, the combination of the CBF and cost function was directly used to penalize unsafe behavior. Moreover, the CNN was applied to reduce the computational complexity of dual actor–critic network. The effectiveness of the proposed method was validated by the simulation results.