Neural Network Iterative Learning for SISO Non-Affine Control Systems

Abstract

This work introduces an identification scheme capable of obtaining the unknown dynamics of a nonlinear plant process. The proposed method employs an iterative algorithm that prevents confinement to a sole trajectory by fitting a neural network over a series of trajectories that span the desired subset of the state space. At the core of our contributions lie the applicability of our method to open-loop unstable systems and a novel way of generating the system’s reference trajectories, which aim at sufficiently stimulating the underlying dynamics. Following this, the prescribed performance control (PPC) technique is utilized to ensure accurate tracking of the aforementioned trajectories. The effectiveness of our approach is showcased through successful identification of the dynamics of a two-degree of freedom (DOF) robotic manipulator in both a simulation study and a real-life experiment.

1. Introduction

Nonlinear dynamic models play a vital role in capturing the complex behaviors exhibited by the majority of real-world physical systems. Beyond the inherent nonlinearity, challenges emerge due to the lack of precise model knowledge, impacting various domains such as model-based control and the design of optimal and fault-tolerant control laws. In this spirit, developing a nonlinear system identification scheme is essential for enhancing our understanding of these systems’ behaviors and for crafting robust and efficient control strategies tailored to their characteristics.

Over the last decades several approaches that tackle nonlinear system identification have emerged. Volterra series [1] methods involve estimating the Volterra kernels that describe the nonlinear behavior of a system. However, a typical issue in the estimation of Volterra series models is the exponential growth of the regressor terms. Various works have been proposed to address this problem through the regularization [2,3] and incorporation of prior structural knowledge in block-oriented models [4]. Block-oriented nonlinear models such as the Hammerstein and Wiener models [5] consist of a cascade combination of a static nonlinear element and a linear dynamic model and address the computational issue of multidimensional Volterra kernels. For the Hammerstein–Wiener model, where a nonlinear block both precedes and follows a linear dynamic system [6], the authors in [7] suggest an adaptive scheme for system identification in which only quantized output observations are available due to sensor limitations. Nevertheless, these methods are generally limited to a highly specific model form in each case, and typically, knowledge of this form is required prior to the identification process.

On the other hand, the NARMAX [8] framework provides a flexible and powerful approach for modeling and understanding the dynamics of nonlinear systems in which the system is modeled in terms of a nonlinear functional expansion of lagged inputs, outputs and prediction errors. In [9], a multilayer perceptron neural network is used to build a NARMAX model in order to identify the dynamics of a DC motor. A significant challenge in system identification using NARMAX models lies in the trade-off between selection of the model parameters and an adequate representation of the system dynamics. Finally, artificial intelligence tools such as Fuzzy Logic (FL) and Artificial Neural Networks (ANNs) have proved to be successful computational tools in any attempt to model nonlinear systems. More specifically, fuzzy system identification techniques have found extensive application in a variety of nonlinear systems [10,11,12].

2. State of the Art and Contributions

ANNs have been widely employed for nonlinear system identification, due to their ability to serve as universal approximators [13]. NNs can be categorized into two main structures, namely Feedforward Neural Networks (FNNs) and Recurrent Neural Networks (RNNs). The former does not incorporate any feedback loops in its structure (i.e., it solely features feedforward connections), while the latter includes both feedforward and feedback connections. Regarding RNNs, they have been successfully utilized to approximate dynamical systems [14]. More specifically, in [15], a continuous time RNN is employed to be used as a predictive controller, while in [16], the authors suggest a modified Elman–Jordan NN for the identification and online control of a nonlinear single-input single-output (SISO) process model. Furthermore, ref. [17] proposes a fixed time control scheme for strict-feedback nonlinear systems where an RNN is used to estimate the uncertain dynamics. Additionally, in [18], a novel finite memory estimation-based learning algorithm for RNNs is introduced; however, it shows an inability for real-time implementation. The aforementioned issue is alleviated in [19] with a moving window iterative identification algorithm for unknown nonlinear systems in discrete time. Finally, Ref. [20] proposes an improved version of the classical Elman NN, where it is used to identify the unknown dynamics of time-delayed nonlinear plants.

Concerning FNNs, they have been employed in various works to handle uncertainties in adaptive neural network control [21,22]. In addition, several studies have explored adaptive identification of nonlinear systems using FNNs [23,24,25,26,27,28], utilizing Radial Basis Functions (RBFs) to approximate the unknown dynamics. While these works address a wide range of systems, they typically only consider a partial Persistency of Excitation (PE) [29] condition over periodic trajectories. The PE condition is crucial in adaptive control of nonlinear systems as it influences the convergence properties of the identification algorithm; it is, however, very difficult to be verified a priori. This issue is tackled in [30], where the full PE condition is addressed, specifically for systems in the Byrnes–Isidori [31] canonical form. In the aforementioned work, a reference trajectory is designed to a priori ensure satisfaction of the PE condition for RBF networks, based on [32]. Subsequently, the PPC technique [33] is applied to track this trajectory for uncertain input affine dynamical systems in canonical form.

In NN-based learning algorithms, the nonlinearities of a system are modeled using an NN with unknown, yet constant weights, reducing the identification process to the estimation of the network’s parameters. The effectiveness of the aforementioned approach relies heavily on factors such as the structure of the network, the training algorithm employed and the quality of the training data. During training, the network is provided with input–output data pairs, representing the system’s behavior across various conditions. These data are acquired by probing the plant with properly designed input signals. The success of the identification process depends significantly on the ability of the input signals to effectively stimulate the system dynamics across the entire compact region of interest. However, even in the presence of sufficiently rich reference trajectories, precise tracking remains a challenging task owing to the system’s nonlinearity as well as the identification error during initialization of the training process.

The majority of the existing options for nonlinear system identification are ineffective, particularly when open-loop stable dynamics cannot be assumed. This highlights the necessity of an online approach applicable to a wide range of systems. In this work, we propose a novel approach for designing reference trajectories along with an algorithm that iteratively improves the approximation of the unknown input–output mapping over a compact set. These trajectories sufficiently stimulate the system dynamics and are tracked using the PPC scheme. This allows accurate tracking of a reference trajectory, ensuring that the system’s response meets predefined performance criteria throughout both transient and steady-state phases. We note that the aforementioned control technique is vital to the identification process since it does not require prior knowledge of the underlying dynamics [34]. The proposed methodology is motivated by the inadequacy of conventional open-loop excitation signals that are utilized in the identification literature to effectively stimulate unknown dynamics since they may lead to instability, e.g., open-loop unstable dynamical systems.

The proposed method employs a two-phase strategy. Initially, the system is driven through a path characterized by known and desired traits, by linking together multiple points located over the desired workspace, with a trajectory that minimizes certain response criteria. Subsequently, a neural network is utilized to learn the dynamics of the unknown open-loop system, even in cases where it could be unstable. The iterative training procedure of the network entails adjusting it across a series of trajectories that encompass the specific subset of input and state space being targeted. This algorithm guarantees that the acquired knowledge remains applicable across the entire set and is not just limited to a single trajectory’s neighborhood.

Our key contributions in this work can be summarized as follows:

• A method for designing reference trajectories that are capable of adequately stimulating the dynamics of a system.
• An identification scheme capable of retrieving the unknown plant dynamics even in the case of open-loop instability.
• Full coverage of the targeted system state space instead of limitation to a sole trajectory.

Finally, the overall methodology is demonstrated through an extensive simulation study, as well as a real-life experiment on a two-degree of freedom robotic manipulator.

3. Problem Formulation and Preliminaries

Consider a class of SISO non-affine nonlinear dynamical systems described in the canonical form:

$$x^{\left(n\right)}=f(\overline{x},u)$$

(1)

where $\overline{x}={[x,\dot{x},\ddot{x},\dots ,x^{(n-1)}]}^T\in {\mathbf{R}}^n$ is the state vector and $n\ge 1$ is the system’s order, while $u\in \mathbf{R}$ denotes the control input. Subsequently, we define the nonlinear function $f_u$ that can be regarded as the control input gain of System (1) as:

$$f_u(\overline{x},u)=\frac{\partial f(\overline{x},u)}{\partial u}$$

(2)

Throughout this work, the following assumptions are asserted:

Assumption 1.

The function $f(·,·):{\mathbf{R}}^{n+1}\to R$ is Lipschitz continuous and $f_u(\overline{x},u)$ is known to be strictly positive (or negative), i.e., $f_u(\overline{x},u)\ne 0$ for all $(\overline{x},u)\in {\mathbf{R}}^n\times \mathbf{R}$. Without loss of generality, we assume that $f_u$ is strictly positive, i.e., it holds that $\frac{\partial f}{\partial u}\ge g>0$, where g is a positive constant.

Assumption 2.

The state vector $\overline{x}$ is available for measurement.

Assumption 1 essentially translates into a controllability condition, which is prevalent among a wide spectrum of nonlinear dynamical systems. Furthermore, regarding Assumption 2, we stress that the necessity for full-state feedback is a common feature in identification processes. The rapid advancement of sensing technologies and the wide availability of data, combined with the fact that identification processes typically take place in specific environments where access to the system’s states is available, renders this assumption less restrictive.

In this work, the objective is to establish an iterative learning framework that allows the extraction of the unknown non-affine mapping $f(\overline{x},u)$ in a compact region of interest $\Omega \subset {\Re }^{n+1}$. We have to stress that the adopted scheme can be readily extended to encompass multi-input multi-output (MIMO) square systems (i.e., with an equal number of inputs and outputs) by implementing the identification process that will be described in the sequel, separately along each output. In such a case, Assumption 1 pertains to the positive definiteness of the input Jacobian of the nonlinear dynamics.

4. Methodology

In this section, we initially present the structure that will be utilized for the approximation of the unknown system dynamics. Additionally, we design a set of trajectories that sufficiently excite the system within a targeted compact set $\Omega \subset {\mathbf{R}}^{n+1}$. Furthermore, a comprehensive presentation of the controller employed to trace the generated reference trajectories is provided. Ultimately, the entire iterative identification algorithm, along with its individual steps, will be outlined.

4.1. Approximation Structure

In order to approximate the unknown nonlinear system dynamics, the following neural network will be used

$$f(\overline{x},u)=NN(\overline{x},u;w^{*})+ϵ(\overline{x},u)$$

(3)

where $w^{*}$ represents the vector that contains the optimal synaptic weights that minimize the modeling error term $ϵ(\overline{x},u)$, i.e., $|ϵ(\overline{x},u)|<\overline{ϵ}$ within a compact set $\Omega$ for the smallest positive constant $\overline{ϵ}$. By approximating the nonlinear function f, we obtain the estimate

$$\widehat{f}(\overline{x},u)=NN(\overline{x},u;\widehat{w})$$

(4)

based on which the approximation error can be written as

$$\tilde{f}(\overline{x},u)=f(\overline{x},u)-\widehat{f}(\overline{x},u)=NN(\overline{x},u;w^{*})-NN(\overline{x},u;\widehat{w})+ϵ$$

(5)

Hence, our objective reduces to the formulation of an iterative learning scheme capable of estimating the unknown weights $w^{*}$, in order to navigate the approximation error to an arbitrarily small neighborhood of the origin.

4.2. Reference Trajectory Design

Guiding the system along a predefined path with specified characteristics produces crucial data, which will be used for the successful estimation of the unknown dynamics. To create a reference trajectory that sufficiently excites the system dynamics, multiple points scattered all over the compact set $\Omega ={\Omega }_x\times {\Omega }_{\dot{x}}\times \cdots \times {\Omega }_{x^{\left(n\right)}}\subset {\mathbf{R}}^{n+1}$ are smoothly linked together to form the trajectory path. First, a set of M points $X=[X_1,\dots ,X_M]\in {\Re }^{n+1\times M}$ is selected such that they cover $\Omega$; then, a closed path is obtained by traversing these points. Towards this direction, we connect any pair of points $X_i\to X_j$ with a trajectory that minimizes the derivative $x^{\left(n\right)}$. Thus, to obtain the trajectory that connects the aforementioned points, a minimum-energy optimization problem with a fixed final state [35] is solved for each transition.

In order to successfully acquire the desired trajectory $z\left(t\right)=x_{d_{i,j}}\left(t\right)$ between two points $X_i$ and $X_j$, consider the following dynamic system:

$$\dot{z}=Az+B\tilde{u}$$

(6)

where $\tilde{u}$ denotes the system’s input. In addition, A and B are given by:

$$\begin{array}{c}A=\left[\begin{array}{cc}0& {\mathbf{I}}_{n-1}\\ {\mathbf{0}}_{(n-1)\times 1}& {\mathbf{0}}_{1\times (n-1)}\end{array}\right]B=\left[\begin{array}{c}{\mathbf{0}}_{(n-1)\times 1}\\ 1\end{array}\right]\end{array}$$

where $\mathbf{0}$ and $\mathbf{I}$ are the zero and the identity matrix, respectively, and the subscripts denote their dimensions. Our aim is to drive the system from the initial state $z\left(t_0\right)=X_i$ to the desired final state that is fixed to $z\left(T_{ij}\right)=X_j$, where $T_{ij}$ denotes the transition time between the two points. In addition, the following positive definite quadratic form is subjected to minimization:

$$J\left(t_0\right)=\frac{1}{2}{\int }_{t_0}^{T_{ij}}{\tilde{u}}^{T}R\tilde{u}dt$$

(7)

The solution to the aforementioned optimal control problem is an open-loop control signal that can be obtained in closed form (i.e., it depends only on the initial and final states):

$${\tilde{u}}^{*}\left(t\right)=B^T{e}^{A^T(T_{ij}-t)}{G}^{-1}(t_0,T_{ij})[z\left(T_{ij}\right)-e^{A(T_{ij}-t_0)}z\left(t_0\right)]$$

(8)

where $G(t_0,T_{ij})$ denotes the reachability Grammian:

$$G(t_0,T_{ij})={\int }_{t_0}^T{e}^{A(T_{ij}-\tau )}B{B}^T{e}^{A^T(T_{ij}-\tau )}d\tau$$

(9)

Since the pair $(A,B)$ is reachable, the Grammian matrix $G(t_0,T_{ij})$ is invertible, and therefore, a minimum-energy control ${\tilde{u}}^{*}\left(t\right)$ exists for any pair $X_i$, $X_j$. Therefore, the solution to System (6), and thus the reference trajectory between $X_i$ and $X_j$, is obtained by:

$$\begin{array}{c}\hfill x_{d_{i,j}}\left(t\right)=z\left(t\right)=-{\int }_{t_0}^t{e}^{A(T_{ij}-\tau )}B{B}^T{e}^{A^T(T_{ij}-\tau )}{G}^{-1}(t_0,T_{ij})[z\left(T_{ij}\right)-e^{A(T_{ij}-t_0)}z\left(t_0\right)]d\tau \end{array}$$

(10)

where the transition time $T_{ij}$ is yet to be determined. Towards this direction, notice that since the control law is available in closed form, we can readily determine the value of the optimal quadratic cost when applying it:

$$J^{*}(t_0,T_{ij}))=\frac{1}{2}{d}^T(t_0,T_{ij})G^{-1}(t_0,T_{ij})d(t_0,T_{ij})$$

(11)

where $d(t_0,T_{ij})$ represents the final state difference:

$$d(t_0,T_{ij})=z\left(T_{ij}\right)-e^{A(T_{ij}-t_0)}z\left(t_0\right).$$

(12)

The transition time $T_{ij}$ for which the system state will traverse $X_{ij}$ is obtained by solving the following optimization problem:

$$\underset{T_{ij}}{min}\left\{\alpha J^{*}(t_0,T_{ij}))+\beta C(t_0,T_{ij}))+\gamma T_{ij}\right\}s.t.T_{min}\le T_{ij}\le T_{max}$$

(13)

whose computation can be efficiently tackled [36]. The positive weighting parameters $\alpha$, $\beta$ and $\gamma$ specify a trade-off between the quadratic cost $J(t_0,T_{ij})$, the minimization of the curve length of the transition $C(t_0,T_{ij})$ and the transition time $T_{ij}$, while $T_{min}$ and $T_{max}$ denote the lower and upper bound of the latter, respectively. More specifically, the curve length of the transition is given by:

$$C(t_0,T_{ij})={\int }_{t_0}^{T_{ij}}\sqrt{1+{\dot{z}}^2}dt$$

(14)

As our goal is to approximate the underlying dynamics within a confined workspace, specific constraints need to be imposed on the transition curve. This justifies the incorporation of the length of the curve in the performance index (13). The time $T_{ij}^{*}$ that results from minimizing (13) yields a minimum-energy open-loop control that guides the system’s state from the initial to the final state while the length of the curve is taken into account. An alternative approach would be to treat the curve length $C(t_0,T_{ij})$ as the sole performance index. However, this strategy proves to be insufficient as a trajectory comprised of a sole straight line (which is what minimizing only the curve length would achieve) fails to adequately stimulate the dynamics of the system. It is important to emphasize that the collection of transition points and their corresponding times is an offline procedure. Following this process, all the parameters are stored in a matrix H and can be accessed to generate online solutions using (10).

$$H=\left[\begin{array}{ccc}{X}_1& \dots & X_{N-1}\\ X_2& \dots & X_N\\ T_{12}& \dots & T_{N-1N}\end{array}\right]$$

(15)

Finally, the complete algorithm for generating a sole reference trajectory is summarized in Algorithm 1. Given a set of M points that cover the compact region of interest $\Omega$, the aforementioned algorithm produces a reference trajectory that spans a significant part of the targeted state space $\Omega$.

Algorithm 1 Reference Trajectory Design

1: Select a random sequence of the points $M^{*}$ (i.e., a permutation of M) 2:     Set the first point of $M^{*}$ as the initial point $X_i$ and the second one as $X_f$ 3: Repeat 4:     Construct the Performance Index of the transition (13) 5:      Minimize the Performance Index with respect to the time span T and store the transition characteristics $[X_i,X_f,T]$ in matrix H 6:      Set the previous final point as the new initial and the next point of the sequence as the new final point 7: until the final point is the starting point of the sequence (i.e., the path has closed)

4.3. Controller Design

The aforementioned reference trajectories will be tracked by employing a controller that utilizes the PPC technique. Employing this scheme proves essential when working with unknown systems since it does not incorporate any prior knowledge of the system dynamics. To achieve prescribed performance, a scalar error $\sigma \left(t\right)$ must remain bounded within a predefined region, formed by decaying functions of time, as illustrated in Figure 1:

$$-\rho \left(t\right)<\sigma \left(t\right)<\rho \left(t\right),\forall t⩾0$$

(16)

The function $\rho \left(t\right)$ is a smooth, bounded, strictly positive and decreasing function of time called performance function and is chosen as follows:

$$\rho \left(t\right)=({\rho }_0-{\rho }_{\infty })e^{-lt}+{\rho }_{\infty }$$

with ${\rho }_0,{\rho }_{\infty }$ and l being positive gains that are chosen to satisfy the designer’s specifications. Specifically,

• ${\rho }_{\infty }={lim}_{t\to \infty }\rho \left(t\right)$ is selected according to the maximum allowable tracking error at the steady state;
• l determines a lower bound on the speed of convergence;
• ${\rho }_0$ affects the maximum overshoot and is selected such that ${\rho }_0>\left|\sigma \left(0\right)\right|$.

For the scalar error $\sigma$, we define the linear filtered error:

$$\sigma \left(t\right)={(\frac{d}{dt}+\lambda )}^{n}e\left(t\right)=\sum _{j=0}^n\left(\begin{array}{c}n-1\\ j\end{array}\right){\lambda }^j{e}^{(n-j)}\left(t\right)$$

(17)

where $\lambda$ is a positive constant and $e\left(t\right)=x\left(t\right)-x_d\left(t\right)$ denotes the tracking error. In addition, consider the normalized error $\xi =\frac{\sigma \left(e\right(t\left)\right)}{\rho \left(t\right)}$ and the transformation $T\left(\xi \right(t\left)\right)$ that maps the constrained behavior as defined in (16), into an unconstrained one. More specifically, $T:(-1,1)\to (-\infty ,\infty )$ is a strictly increasing, symmetric and bijective mapping, e.g., $T\left(\xi \right)=\frac{1}{2}\ln \left(\frac{1+\xi }{1-\xi }\right)$. Finally, prescribed performance is achieved by employing the following control law [34]:

$$u=-k{T}^{\prime }\left(\xi \right){\rho }^{-1}T\left(\xi \right),k>0$$

(18)

where k is a positive gain.

4.4. Iterative Learning

The identification algorithm involves an iterative process, aiming to enhance the approximation of the unknown underlying dynamics until convergence is reached (i.e., $|ϵ(\overline{x},u)|\le \widetilde{ϵ}$, where $\widetilde{ϵ}$ is an arbitrarily small positive value). More specifically, we adopt an artificial neural network with any number of hidden layers and sigmoidal activation functions, which is trained based on the collected data over a series of reference trajectories. The aforementioned training process takes place in a supervised learning environment, where the necessary input–output data for the estimation of the network’s weights at each iteration are acquired by tracing a reference trajectory designed through the process outlined in Section 4.2.

However, in order to guarantee that the final result is devoid of bias along a specific trajectory, a distinct random sequence of points is employed to construct the reference trajectory in each iteration. In addition, to ensure accurate tracking, the aforementioned trajectory will be tracked with predefined transient and steady-state performance using the PPC technique, as described in Section 4.3. During each instance of learning, the weights of the neural network are obtained by optimizing the matching between the network’s output and the target values via a backpropagation algorithm. Notice that under Assumption 2, the system’s state $[x,\dot{x},\dots ,x^{(n-1)}]$ (i.e., the input training data) is available for measurement. However, since we do not have access to $x^{\left(n\right)}$, the target output values to train the neural network $NN(\overline{x},u;w)$ are not available.

To overcome this challenge, during each iteration $m=1,2,\dots$ the following tracking differentiator [37] is employed:

$$\begin{array}{cc}\hfill \dot{z_i}& =z_{i+1},i=1,\dots ,n-1\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill \dot{z_n}& =-\sum _{j=1}^n\left[k_{z_j}{R}^{n+1-j}(z_j-x^{(j-1)})\right]+NN(\overline{x},u;{\tilde{w}}_{m-1})\hfill \end{array}$$

(20)

with positive gains $R,k_{z_j},j=1,\dots ,n$. When $R\to \infty$, then based on [37], $z_i\to x^{(i-1)},i=1,\dots ,n$ and consequently $\dot{z_n}\to x^{\left(n\right)}$, from which we may derive the input–output pair that will be employed for the training process. Notice that during the first round ($m=1$) over the closed reference trajectory, the $NN$ structure in (20) is not initialized yet, as we have not gathered the necessary data. Therefore, the network is activated after the first training stage has taken place. As a result, in each subsequent round (m), the extracted neural structure originates from the previous iteration. After each round of training, the obtained knowledge regarding the system’s dynamics is employed not only to enhance the estimation of $x^{\left(n\right)}$ but also to provide improved initial weights for the consecutive iterations. Consequently, an accurate model of the system dynamics can be acquired after sufficient iterations (e.g., when the weights do not change above a threshold). The complete algorithm for the iterative learning of the unknown dynamics is summarized in Algorithm 2.

Remark 1.

The collection of transition points and their corresponding times that are utilized in the design of the online reference trajectories, as described in Algorithm 1, is an offline procedure. Consequently, once the trajectories have been generated, the problem is reduced to a tracking control problem with the PPC scheme, which is efficiently tackled owing to the low complexity control signal. In a similar manner, once the tracking of the reference trajectories is complete, we have successfully acquired all the necessary data that will be employed in the final part of Algorithm 2, in order to train the adopted neural structure. Therefore, we stress that our approach can be employed to tackle identification of complex systems without sacrificing online performance since the computational burden mainly stems from offline procedures.

Algorithm 2 Iterative Learning

1: Select a set of M Points that cover $\Omega$ 2: Repeat 3:       Algorithm 1 4: until K reference trajectories are created 5: Repeat 6:      Track the designed reference trajectory $K_i$ using PPC and store the required training data 7: until all trajectories are cleared 8: Repeat 9:       if Initial iteration 10:        $NN\leftarrow Null$ 11:     Employ the tracking differentiator to complete the required training data 12:     Update the neural network weights via backpropagation 13: until all the training data has been utilized

5. Results

In this section, we demonstrate the effectiveness of the proposed identification scheme with the goal of obtaining the underlying system dynamics of a two-degree of freedom robotic manipulator without assuming any prior knowledge, via two scenarios: a simulated one, showcasing the full capabilities of our method, and a real-life experiment to demonstrate the applicability of our algorithm to real-world systems.

5.1. Simulation Study

Initially, we present the simulation results for the successful identification of the unknown dynamics of a two-degree of freedom robotic manipulator, as illustrated in Figure 2. All simulations were conducted in MATLAB R2023a. The robotic manipulator obeys the following dynamic model

$$M\left(q\right)\ddot{q}+C(q,\dot{q})\dot{q}+G\left(q\right)=\tau ,$$

(21)

where $q={\left[q_1{q}_2\right]}^T$ and $\dot{q}={\left[{\dot{q}}_1{\dot{q}}_2\right]}^T$ denote the joint angular positions and velocities, respectively, $M\left(q\right)$ is a positive definite inertia matrix, $C(q,\dot{q})$ is the matrix that describes the Coriolis–centrifugal phenomena, $G\left(q\right)$ is the vector describing the influence of gravity and $\tau$ is the torque that acts as the system’s input. More specifically, the inertia matrix is formulated as

$$M=\left[\begin{array}{cc}{M}_{11}& M_{12}\\ M_{21}& M_{22}\end{array}\right]$$

where

$$\begin{array}{c}{M}_{11}=I_{Z_1}+I_{Z_2}+m_1\frac{l_1^2}{4}+m_2({l_1}^2+\frac{l_2^2}{4}+l_1{l}_2{c}_2)\\ M_{12}=M_{21}=I_{Z_2}+m_2(\frac{l_2^2}{4}+\frac{1}{2}{l}_1{l}_2{c}_2)\\ M_{22}=I_{Z_2}+m_2\frac{l_2^2}{4}\end{array}$$

In addition, the vector containing the Coriolis and centrifugal torques is defined as follows:

$$C(q,\dot{q})\dot{q}=\left[\begin{array}{cc}-c\dot{q_2}+k_1& -c(\dot{q_1}+\dot{q_2})\\ c\dot{q_1}& k_2\end{array}\right]\left[\begin{array}{c}\dot{q_1}\\ \dot{q_2}\end{array}\right]$$

with c being $c=\frac{1}{2}{m}_1{l}_1{l}_2{s}_2$. Additionally, the gravity vector is given by:

$$G\left(q\right)=\left[\begin{array}{c}\frac{1}{2}{m}_{1}g{l}_1{c}_1+m_{2}g(l_1{c}_1+\frac{1}{2}{l}_2{c}_{12})\\ \frac{1}{2}{m}_2{l}_{2}g{c}_{12}\end{array}\right]$$

and the terms $c_2,s_2$ and $c_{12}$ correspond to $cos\left(q_2\right),sin\left(q_2\right)$ and $cos(q_1+q_2)$, respectively. The values adopted for simulation are provided in

Table 1. System parameter values.

${\mathit{m}}_1$	${\mathit{l}}_1$	${\mathit{I}}_{{\mathit{Z}}_1}$	${\mathit{k}}_1$	${\mathit{m}}_2$	${\mathit{l}}_2$	${\mathit{I}}_{{\mathit{Z}}_2}$	${\mathit{k}}_2$	g
3.2	0.5	0.96	1	2.0	0.4	0.81	1	9.81

, with $m_i$, $I_{zi}$ and $l_i$ denoting the mass, the moment of inertia and the length of link i, respectively, $k_i$ being the joint friction coefficient and g the acceleration of gravity. The aforementioned dynamics can be expressed as $\ddot{q}=f(q,\dot{q},\tau )$, aligning with the form described in Section 3. Our goal is to learn the unknown nonlinear mapping f over the domain set $\Omega ={\Omega }_{q_i}\times {\Omega }_{{\dot{q}}_i}\times {\Omega }_{{\ddot{q}}_i}=[-1,1]\times [-1,1]\times [-1,1]$ for $i=1,2$.

To achieve this, a reference trajectory traversing in random order 27 points scattered over the domain set $\Omega$ is designed. A closed path of 27 such points, along with the workspace are depicted in Figure 3. It is evident that the generated trajectory, which traverses all points, spans a significant part of the desired state space. In order to obtain the time of transition between the points of the sequence, the performance index (13) was minimized, with the weighting parameters set as $\alpha =0.1$, $\beta =1$ and $\gamma =0.1$. Subsequently, a PPC controller with gains $k=1$, $\lambda =1.5$ and specifications ${\rho }_0^1=3$, ${\rho }_{\infty }^1=0.01$, $l_1=4$, ${\rho }_0^2=3$, ${\rho }_{\infty }^2=0.01$ and $l_2=4$ is used to track the reference trajectory. Finally, the gains of the tracking differentiator are set to $k_{z_1}=k_{z_2}=0.2$ and $R=100$. In Figure 4 and Figure 5, the tracking of one of the reference signals by the PPC controller is depicted, along with the error and the control input, for both of the manipulator’s joints.

In addition, for the training process, a single hidden-layer neural network with 15 neurons was utilized. To elaborate further on this, 15 different reference trajectories were formed by permuting the 27 points of the workspace to sufficiently improve iteratively the estimation of the underlying dynamics. During each iteration, data are collected by tracking a different reference trajectory with the PPC controller. The gathered data are fed to the neural network, whose training is conducted by the MATLAB toolbox employing the default Levenberg–Marquardt algorithm. After the first iteration, the resulting network is employed in the tracking differentiator to improve both the acceleration estimation and the selection of the initial weights for the consecutive iterations. To avoid overfitting of the employed neural network and achieve satisfactory generalization capabilities, after tracking the reference trajectories, the acquired dataset was split into a training and a testing set, with a ratio of 85% and 15%, respectively. This was to ensure that the acquired neural structure performs well, not only on the training set, but also on the unseen data represented by the testing set.

In Figure 6, the curve fitting evolution for both components of the unknown dynamics can be observed. Furthermore, by comparing the initial estimations in Figure 7 with the results of the approximation problem in Figure 8, it becomes evident that the proposed scheme has resulted in a highly accurate estimate of the system dynamics with no dependence on any prior knowledge. We stress that to ensure that the evaluation of the identification scheme is free of bias, the testing is conducted using a reference trajectory different from the ones employed during the training stages.

5.2. Experimental Study

This section showcases the results of an experimental study which validates the effectiveness and applicability of the proposed scheme in a real-world system. Towards this direction, we consider the two-degree of freedom planar robot manipulator with DC geared motor actuation, depicted in Figure 9. Measurements regarding the system’s state (i.e., position and velocity) for each joint are obtained through a rotary encoder attached at each motor’s shaft. Furthermore, a microcontroller was employed to acquire the necessary sensor data as well as to drive the motors. This is achieved through utilization of a motor driver that allows the direction and speed of a DC motor to be adjusted by using a Pulse–Width-Modulated (PWM) signal.

In order to calculate the applied input control signal, note that the produced torque is given by:

$$\tau =I_{\alpha }\ast K_T$$

where $I_{\alpha }$ denotes the armature current and $K_T$ is the motor torque constant. Moreover, at a constant operating point, by applying Kirchoff’s Voltage Law, we obtain:

$$\begin{array}{cc}\hfill V_{supply}& =E_b+I_{\alpha }\ast R_{\alpha }\hfill \\ \hfill E_b& =K_e\ast {\omega }_m\hfill \end{array}$$

where $V_{supply}$ is the supplied voltage, $R_a$ denotes the armature’s resistance, while ${\omega }_m$ denotes the angular velocity and $E_b$ and $K_e$ are the back-EMF voltage and constant, respectively. By applying the law of conservation of energy, it can be shown that $K_T$ = $K_e$, and therefore, the applied control input signal can be approximately obtained as follows:

$$\begin{array}{cc}\hfill \tau =& \frac{K_e\ast V_{supply}-K_e^2\ast {\omega }_m}{R_{\alpha }}\hfill \\ \hfill K_e=& \frac{V_{rated}}{{\omega }_{noload}}\hfill \end{array}$$

Finally, the constant $K_e$ can be obtained by dividing the rated voltage of each motor by its no-load speed. For the motors employed in our experiment, the following parameter values were extracted: $R_1=4.2\Omega$, $R_2=5.6\Omega$, $K_{e1}=0.027\mathrm{V}/\mathrm{rpm}$ and $K_{e2}=0.020\mathrm{V}/\mathrm{rpm}$.

Similarly to the previous study, we aim at learning the unknown nonlinear mapping f over the domain set $\Omega ={\Omega }_{q_i}\times {\Omega }_{{\dot{q}}_i}\times {\Omega }_{{\ddot{q}}_i}=[-\pi /2,\pi /2]\times [-\pi /2,\pi /2]\times [-\pi /2,\pi /2]$ for $i=1,2$; therefore, a reference trajectory traversing in random order 27 points scattered over the domain set $\Omega$ is designed. The time of transition between the points of the sequence was obtained by minimizing the performance index (13), with the weighting parameters set as $\alpha =0.1$, $\beta =1$ and $\gamma =0.1$. Subsequently, a PPC controller with gains $k=2$ and $\lambda =1.5$ and specifications ${\rho }_0^1=4.8$, ${\rho }_{\infty }^1=0.1$, $l_1=1.5$, ${\rho }_0^2=4.8$, ${\rho }_{\infty }^2=0.1$, $l_2=1.5$ is used to track the reference trajectory. In Figure 10 and Figure 11, the tracking of one of the reference signals by the PPC controller is depicted, along with the control input, for both of the robot’s joints. A Savitzky–Golay filter [38] was used to eliminate the noise of the experimental data. In addition, a video of the robotic manipulator tracking a reference trajectory of 27 points using the PPC technique can be accessed through the following hyperlink: https://youtu.be/CrUUgfKE6t4 (accessed on 27 February 2024).

In the initial phase of the experiment, we track 20 different reference trajectories and store the obtained data (i.e., position, velocity and applied torque) that will be used in the next phase to iteratively improve the underlying dynamics estimation. In the learning phase, the gathered data along with the target acceleration output (which is obtained through a tracking differentiator as described in Section 4.4) are fed to a single hidden-layer neural network with 15 neurons whose training is conducted by the MATLAB toolbox employing the default Levenberg–Marquardt algorithm. Similar to the simulation study, overfitting of the employed neural network is avoided by splitting the acquired dataset into a training and a testing set, with a ratio of 85% and 15%, respectively. After the first iteration, the resulting network is utilized in the tracking differentiator, whose gains are set to $k_{z_1}=0.25$, $k_{z_2}=0.1$ and $R=25$, to improve both the acceleration estimation and the selection of the initial weights for the consecutive iterations. To evaluate the learning capabilities of the identification scheme, a reference trajectory different from the ones employed during the training stages was used to compare the results of the approximation problem. More specifically, the initial and final estimations of the unknown dynamics are provided in Figure 12 and Figure 13, where decreases in the Root Mean Square Error (RMSE) of $43\%$ and $49\%$ are witnessed for the underlying dynamics $f_1$ and $f_2$, respectively. It is clear that the suggested approach yields an adequate estimation of the system dynamics, without relying on any prior knowledge.

6. Conclusions and Future Work

In this work, we present an iterative learning process that aims at extracting the unknown nonlinear dynamics of a plant in the targeted system state space. The approach was demonstrated through both an extensive simulation study and a real-life experiment on a robotic manipulator with two degrees of freedom. Our main contributions consist of a novel reference trajectory generation scheme that is capable of sufficiently exciting the underlying dynamics and an iterative algorithm that progressively improves the unknown parameters of the employed neural network, even in the case of open-loop instability. Future formal approaches should consider tackling the issue of input and state constraints through the design of a trajectory that inherently avoids configurations associated with physical system constraints/limitations (e.g., mechanical constraints).