Auxiliary Model Based Multi-Innovation Stochastic Gradient Identification Algorithm for Periodically Non-Uniformly Sampled-Data Hammerstein Systems

Abstract

Due to the lack of powerful model description methods, the identification of Hammerstein systems based on the non-uniform input-output dataset remains a challenging problem. This paper introduces a time-varying backward shift operator to describe periodically non-uniformly sampled-data Hammerstein systems, which can simplify the structure of the lifted models using the traditional lifting technique. Furthermore, an auxiliary model-based multi-innovation stochastic gradient algorithm is presented to estimate the parameters involved in the linear and nonlinear blocks. The simulation results confirm that the proposed algorithm is effective and can achieve a high estimation performance.

1. Introduction

The dynamics of most practical systems are inherently nonlinear due to complex physical, chemical and biological mechanisms. The modeling of nonlinear systems is challenging and has become an active research area in both academia and industry [1,2]. To simplify the modeling problem, block-oriented models, which are composed of linear dynamic blocks in combination with nonlinear memoryless blocks, have been widely utilized to describe nonlinear systems. The state of the art in designing, analyzing and implementing identification algorithms for block-oriented nonlinear systems were well summarized in a recent book by Giri and Bai [3]. Depending on the location of the static nonlinear component, block-oriented models can be classified into the Hammerstein model, the Wiener model and the Hammerstein–Wiener model [4,5,6]. The Hammerstein model represents a class of input nonlinear systems, where the nonlinear block is prior to the linear one. It can flexibly approximate various input nonlinearities, such as saturation, dead zone, backlash and hysteresis, thus having been extensively employed in realistic applications [7,8,9,10,11].

For decades, the identification of Hammerstein nonlinear systems has attracted much attention, and numerous methods have been reported in the literature. For example, Pouliquen et al. studied the parameter estimation of Hammerstein systems where the linear part is described by an output error model and presented an iterative algorithm based on the optimal bounding ellipsoid criterion [12]. Ding et al. applied the auxiliary model identification principle to deal with unmeasurable noise-free outputs in Hammerstein output error systems, presented a recursive least squares (RLS) algorithm and investigated its convergence properties [13]. Filipovic derived a robust extended RLS algorithm to estimate the parameters of Hammerstein systems interfered by non-Gaussian disturbance [14]. Gao et al. proposed a blind identification algorithm for Hammerstein systems with hysteresis nonlinearity and further developed a composite control strategy to track the reference input [15].

Among various new methodologies in the identification area, the multi-innovation theory has been considered as a useful way to improve estimation precision and convergence rate. The basic idea of the multi-innovation theory is innovation expanding, which helps to update the parameter estimates at each recursion using the data over a moving and fixed-size window [16,17]. Typically, the multi-innovation theory is incorporated with the RLS algorithm, the stochastic gradient (SG) algorithm, the stochastic Newton recursive algorithm, etc., to address identification problems [18]. For instance, a multi-innovation RLS algorithm was developed for Hammerstein AutoRegressive eXogenous (ARX) systems with backlash nonlinearity [19]. Furthermore, a multi-innovation SG algorithm [20] and an auxiliary model-based multi-innovation generalized extended SG algorithm [21] were proposed to estimate the parameters of Hammerstein nonlinear ARX and Box–Jenkins systems, respectively. Compared with the multi-innovation RLS algorithm, the multi-innovation SG algorithm is more efficient in computation because it avoids performing large matrix inversion [22].

The above-mentioned Hammerstein systems all belong to single-rate systems, the inputs and outputs of which are uniformly sampled at the same rate. However, non-uniform sampling can be encountered in practice due to hardware limitations or economic considerations [23,24,25]. For example, influenced by transmission delays and packet losses, the input-output data in networked control systems might be available at non-uniformly spaced time instants [26,27]. The non-uniform sampling includes the uniform sampling as its special case, which can always preserve controllability and observability in discretization [28]. Furthermore, it can overcome the restriction of the Nyquist limit and enable a much lower average sampling frequency. Therefore, intentional non-uniform sampling has the potential to reduce the hardware cost in control applications [29]. Due to the complexity of arbitrary non-uniform sampling, most of the literature works have focused on periodically non-uniformly sampled-data systems [30,31]. For periodically non-uniformly sampled-data Hammerstein systems, Li et al. derived the lifted transfer function model by means of the lifting technique and presented a least squares-based iterative algorithm for parameter estimation [32]. The lifting technique is a benchmark tool to deal with multirate and non-uniformly sampled-data systems [33,34]. However, the corresponding lifted models are complex and involve a large number of parameters, which brings a great challenge for identification. To simplify the model structure and reduce the identification complexity, Xie et al. put forward a novel input-output representation of linear systems with non-uniform sampling by introducing a time-varying backward shift operator ${\delta }^{-1}$ [35]. On the basis of that work, this paper aims to propose a ${\delta }^{-1}$-based model to describe periodically non-uniformly sampled-data Hammerstein systems and presents an auxiliary model-based multi-innovation SG algorithm to estimate the model parameters.

The rest of this paper is organized as follows. Section 2 formulates the identification problem of periodically non-uniformly sampled-data Hammerstein systems. Identification algorithms are proposed in Section 3, and an example is provided in Section 4 to examine their estimation performance. Finally, concluding remarks are given in Section 5.

2. Problem Description

Consider a periodically non-uniformly sampled-data Hammerstein system as depicted in Figure 1, in which $H_{\tau }$ is a periodic non-uniform zero-order hold, converting a discrete-time input sequence $\left\{u(kT+t_i)\right\}$ to a continuous-time input $u\left(t\right)$, i.e.,

$$u\left(t\right)=\left\{\begin{array}{cc}u(kT+t_0),\hfill & kT+t_0\le t<kT+t_1(t_0=0),\hfill \\ u(kT+t_1),\hfill & kT+t_1\le t<kT+t_2,\hfill \\ ⋮\hfill \\ u(kT+t_{q-1}),\hfill & kT+t_{q-1}\le t<kT+t_q(t_q=T),\hfill \end{array}\right.$$

where $k=0,1,\cdots$; T is the frame period spaced by q non-uniform sampling instants $t_i$ ($i=0,1,\cdots ,q-1$).

By passing through a nonlinear static block $f(·)$, $u\left(t\right)$ is transformed into an unmeasurable inner input $\overline{u}\left(t\right)$ to a linear dynamic process P with order n, which can be expressed as:

$$\overline{u}\left(t\right)=\sum _{m=1}^{n_c}{c}_m{f}_m\left[u\left(t\right)\right],$$

(1)

where $f_m\left[u\left(t\right)\right]$ are known nonlinear basis functions and $c_m$ are unknown coefficients to be estimated.

The noise-free output $w\left(t\right)$ of the process P is corrupted by a white noise $v\left(t\right)$, generating a measurable output $y\left(t\right)$. The non-uniform sampler $S_{\tau }$ has a synchronous sampling pattern with $H_{\tau }$; thus, the discrete-time output sequence $\left\{y(kT+t_i)\right\}$ is obtained at sampling instants $t=kT+t_i$ ($i=0,1,\cdots ,q-1$).

For the notational simplicity, in the following the data $s(kT+t_i)$, at non-uniform sampling time $t=kT+t_i$, is denoted by $s_i\left(k\right)$. By using a time-varying backward shift operator ${\delta }^{-1}$ $[{\delta }^{-1}{s}_i\left(k\right)=s_{i-1}\left(k\right)]$ proposed in [35], the mapping relationship between the inner input ${\overline{u}}_i\left(k\right)$ and the noise-free output $w_i\left(k\right)$ of the periodically non-uniformly sampled-data Hammerstein system can be represented as:

$$w_i\left(k\right)=\frac{B_i\left(\delta \right)}{A_i\left(\delta \right)}{\overline{u}}_i\left(k\right),i=0,1,2,\cdots ,q-1,$$

(2)

where:

$$\begin{array}{ccc}\hfill A_i\left(\delta \right)& :=& 1+a_{i1}{\delta }^{-1}+a_{i2}{\delta }^{-2}+\cdots +a_{in}{\delta }^{-n},\hfill \\ \hfill B_i\left(\delta \right)& :=& b_{i0}+b_{i1}{\delta }^{-1}+b_{i2}{\delta }^{-2}+\cdots +b_{in}{\delta }^{-n}.\hfill \end{array}$$

From the system schematic diagram in Figure 1, we have:

$$y_i\left(k\right)=w_i\left(k\right)+v_i\left(k\right).$$

(3)

Given the non-uniformly-sampled input-output data $\{u_i\left(k\right),y_i\left(k\right),k=0,1,2,\cdots ,$ $i=0,1,2,\cdots ,q-1\}$, the objective of this paper is to estimate the parameters of the nonlinear block in (1):

$$\mathit{c}:={[c_1,c_2,\cdots ,c_{n_c}]}^{\mathrm{T}}\in {\mathbb{R}}^{n_c},$$

and the parameters of the linear block in (2):

$$\begin{array}{ccc}\hfill {\mathit{a}}_i& :=& {[a_{i1},a_{i2},\cdots ,a_{in}]}^{\mathrm{T}}\in {\mathbb{R}}^n,\hfill \end{array}$$

$$\begin{array}{ccc}\hfill {\mathit{b}}_i& :=& {[b_{i0},b_{i1},b_{i2},\cdots ,b_{in}]}^{\mathrm{T}}\in {\mathbb{R}}^{n+1}.\hfill \end{array}$$

3. Identification Algorithms

3.1. The AM-SG Algorithm

According to the over-parameterized linear regression approach [36], define the information vector ${\mathit{\phi }}_i\left(k\right)$ and the parameter vector ${\mathit{\theta }}_i$ as:

$${\mathit{\phi }}_i\left(k\right):=\left[\begin{array}{c}{\mathit{\phi }}_{iw}\left(k\right)\hfill \\ {\mathit{\phi }}_{iu}\left(k\right)\hfill \end{array}\right]\in {\mathbb{R}}^{n_0},{\mathit{\theta }}_i:=\left[\begin{array}{c}{\mathit{a}}_i\hfill \\ {\mathit{\theta }}_{iu}\hfill \end{array}\right]\in {\mathbb{R}}^{n_0},n_0=n+(n+1)n_c,$$

$${\mathit{\phi }}_{iw}\left(k\right):=\left[\begin{array}{c}-w_{i-1}\left(k\right)\hfill \\ -w_{i-2}\left(k\right)\hfill \\ \hfill ⋮\hfill \\ -w_{i-n}\left(k\right)\hfill \end{array}\right]\in {\mathbb{R}}^n,{\mathit{\varphi }}_{ij}\left(k\right):=\left[\begin{array}{c}{f}_1\left[u_{i-j}\left(k\right)\right]\hfill \\ f_2\left[u_{i-j}\left(k\right)\right]\hfill \\ \hfill ⋮\hfill \\ f_{n_c}\left[u_{i-j}\left(k\right)\right]\hfill \end{array}\right]\in {\mathbb{R}}^{n_c},j=0,1,2,\cdots ,n,$$

$${\mathit{\phi }}_{iu}\left(k\right):=\left[\begin{array}{c}{\mathit{\varphi }}_{i0}\left(k\right)\hfill \\ {\mathit{\varphi }}_{i1}\left(k\right)\hfill \\ {\mathit{\varphi }}_{i2}\left(k\right)\hfill \\ \hfill ⋮\hfill \\ {\mathit{\varphi }}_{in}\left(k\right)\hfill \end{array}\right]\in {\mathbb{R}}^{(n+1)n_c},{\mathit{\theta }}_{iu}:=\left[\begin{array}{c}{b}_{i0}\mathit{c}\hfill \\ b_{i1}\mathit{c}\hfill \\ b_{i2}\mathit{c}\hfill \\ \hfill ⋮\hfill \\ b_{in}\mathit{c}\hfill \end{array}\right]\in {\mathbb{R}}^{(n+1)n_c}.$$

Using Equation (1), Equation (2) can be written in the following vector form:

$$\begin{array}{ccc}\hfill w_i\left(k\right)& =& -\sum _{j=1}^n{a}_{ij}{w}_{i-j}\left(k\right)+\sum _{j=0}^n{b}_{ij}\left[\sum _{m=1}^{n_c}{c}_m{f}_m\left[u_{i-j}\left(k\right)\right]\right]\hfill \\ \hfill & =& {\mathit{\phi }}_i^{\mathrm{T}}\left(k\right){\mathit{\theta }}_i.\hfill \end{array}$$

(4)

Substituting Equation (4) into Equation (3), we have:

$$y_i\left(k\right)={\mathit{\phi }}_i^{\mathrm{T}}\left(k\right){\mathit{\theta }}_i+v_i\left(k\right).$$

(5)

Equation (5) is the identification model of the periodically non-uniformly sampled-data Hammerstein system, in which the parameter vector ${\mathit{\theta }}_i$ includes the products of parameters $b_{ij}$ and $c_m$. To guarantee a unique parametrization, the first coefficient $c_1$ of the nonlinear function is assumed to be one [14,37]. Furthermore, the information vector ${\mathit{\phi }}_i\left(k\right)$ contains unmeasurable noise-free outputs $w_{i-j}\left(k\right)$. A solution to this difficulty is to replace $w_{i-j}\left(k\right)$ with their estimates ${\widehat{w}}_{i-j}\left(k\right)$ based on the auxiliary model identification idea [38,39,40]. Accordingly, define the estimate of ${\mathit{\phi }}_i\left(k\right)$ as:

$${\widehat{\mathit{\phi }}}_i\left(k\right):=\left[\begin{array}{c}{\widehat{\mathit{\phi }}}_{iw}\left(k\right)\hfill \\ {\mathit{\phi }}_{iu}\left(k\right)\hfill \end{array}\right],$$

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\phi }}}_{iw}\left(k\right)& :=& {[-{\widehat{w}}_{i-1}\left(k\right),-{\widehat{w}}_{i-2}\left(k\right),\cdots ,-{\widehat{w}}_{i-n}\left(k\right)]}^{\mathrm{T}}.\hfill \end{array}$$

Using the estimates ${\widehat{\mathit{\phi }}}_i\left(k\right)$ and ${\widehat{\mathit{\theta }}}_i\left(k\right)$ to replace ${\mathit{\phi }}_i\left(k\right)$ and ${\mathit{\theta }}_i$ in (4), respectively, yields

$${\widehat{w}}_i\left(k\right)={\widehat{\mathit{\phi }}}_i^{\mathrm{T}}\left(k\right){\widehat{\mathit{\theta }}}_i\left(k\right).$$

(6)

Applying the stochastic gradient search principle to minimize the following criterion function:

$$J_1\left({\mathit{\theta }}_i\right)=\mathrm{E}\left\{{[y_i\left(k\right)-{\widehat{\mathit{\phi }}}_i^{\mathrm{T}}\left(k\right){\mathit{\theta }}_i]}^2\right\},$$

an auxiliary model-based stochastic gradient (AM-SG) algorithm can be derived for identifying the parameter vector ${\mathit{\theta }}_i$ in (5):

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}_i\left(k\right)& =& {\widehat{\mathit{\theta }}}_i(k-1)+\frac{{\widehat{\mathit{\phi }}}_i\left(k\right)}{r_i\left(k\right)}[y_i\left(k\right)-{\widehat{\mathit{\phi }}}_i^{\mathrm{T}}\left(k\right){\widehat{\mathit{\theta }}}_i(k-1)],\hfill \end{array}$$

(7)

$$\begin{array}{ccc}\hfill r_i\left(k\right)& =& r_i(k-1)+{\parallel {\widehat{\mathit{\phi }}}_i\left(k\right)\parallel }^2,r_i\left(0\right)=1.\hfill \end{array}$$

(8)

3.2. The AM-MISG Algorithm

The AM-SG algorithm only uses the current dataset to update the parameter estimates. Therefore, it has a slow convergence rate and low estimation accuracy. To improve the identification performance of the AM-SG algorithm, an innovation length p is introduced to derive an auxiliary model-based multi-innovation stochastic gradient (AM-MISG) algorithm.

Considering the most recent p sets of input-output data, define the stacked output vector ${\mathit{Y}}_i(p,k)$, the stacked noise vector ${\mathit{V}}_i(p,k)$ and the stacked information matrix ${\mathbf{\Psi }}_i(p,k)$ as:

$$\begin{array}{ccc}\hfill {\mathit{Y}}_i(p,k)& =& {[y_i\left(k\right),y_i(k-1),\cdots ,y_i(k-p+1)]}^{\mathrm{T}}\in {\mathbb{R}}^p,\hfill \end{array}$$

$$\begin{array}{ccc}\hfill {\mathit{V}}_i(p,k)& =& {[v_i\left(k\right),v_i(k-1),\cdots ,v_i(k-p+1)]}^{\mathrm{T}}\in {\mathbb{R}}^p,\hfill \end{array}$$

$$\begin{array}{ccc}\hfill {\mathbf{\Psi }}_i(p,k)& =& [{\mathit{\phi }}_i\left(k\right),{\mathit{\phi }}_i(k-1),\cdots ,{\mathit{\phi }}_i(k-p+1)]\in {\mathbb{R}}^{n_0\times p}.\hfill \end{array}$$

Equation (5) can be expanded into the following matrix form:

$${\mathit{Y}}_i(p,k)={\mathbf{\Psi }}_i^{\mathrm{T}}(p,k){\mathit{\theta }}_i+{\mathit{V}}_i(p,k).$$

(9)

However, the information vectors ${\mathit{\phi }}_i(k-l),l=0,1,\cdots ,p-1$ in ${\mathbf{\Psi }}_i(p,k)$ include unknown noise-free outputs. Let ${\widehat{\mathit{\phi }}}_i(k-l)$ be their estimates; the estimate of ${\mathbf{\Psi }}_i(p,k)$ can be defined as:

$${\widehat{\mathbf{\Psi }}}_i(p,k)=[{\widehat{\mathit{\phi }}}_i\left(k\right),{\widehat{\mathit{\phi }}}_i(k-1),\cdots ,{\widehat{\mathit{\phi }}}_i(k-p+1)]\in {\mathbb{R}}^{n_0\times p}.$$

Define the following criterion function:

$$J_2\left({\mathit{\theta }}_i\right)={\parallel {\mathit{V}}_i(p,k)\parallel }^2={∥{\mathit{Y}}_i(p,k)-{\widehat{\mathbf{\Psi }}}_i^{\mathrm{T}}(p,k){\mathit{\theta }}_i∥}^2,$$

(10)

where ${\parallel \mathit{X}\parallel }^2=\mathrm{tr}[\mathit{X}{\mathit{X}}^{\mathrm{T}}]$ represents the norm of the matrix X. The gradient of $J_2\left({\mathit{\theta }}_i\right)$ with respect to ${\mathit{\theta }}_i$ is given by:

$$\mathrm{grad}\left[J_2\left({\mathit{\theta }}_i\right)\right]=\frac{\partial J_2\left({\mathit{\theta }}_i\right)}{\partial {\mathit{\theta }}_i}=-2{\widehat{\mathbf{\Psi }}}_i(p,k)[{\mathit{Y}}_i(p,k)-{\widehat{\mathbf{\Psi }}}_i^{\mathrm{T}}(p,k){\mathit{\theta }}_i].$$

Applying the stochastic gradient search principle to minimize the criterion function in (10), we have:

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}_i\left(k\right)& =& {\widehat{\mathit{\theta }}}_i(k-1)-{\mu }_i\left(k\right)\mathrm{grad}{J}_2[{\widehat{\mathit{\theta }}}_i(k-1)]\hfill \\ \hfill & =& {\widehat{\mathit{\theta }}}_i(k-1)+2{\mu }_i\left(k\right){\widehat{\mathbf{\Psi }}}_i(p,k)[{\mathit{Y}}_i(p,k)-{\widehat{\mathbf{\Psi }}}_i^{\mathrm{T}}(p,k){\widehat{\mathit{\theta }}}_i(k-1)],\hfill \end{array}$$

(11)

where ${\mu }_i\left(k\right)>0$ is called the step size or the convergence factor. For the convenience of formula derivation, let ${\mu }_i\left(k\right)=\frac{1}{2r_i\left(k\right)}$; Equation (11) can be rewritten into:

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}_i\left(k\right)& =& {\widehat{\mathit{\theta }}}_i(k-1)+\frac{1}{r_i\left(k\right)}{\widehat{\mathbf{\Psi }}}_i(p,k)[{\mathit{Y}}_i(p,k)-{\widehat{\mathbf{\Psi }}}_i^{\mathrm{T}}(p,k){\widehat{\mathit{\theta }}}_i(k-1)]\hfill \\ & =& \left[\mathit{I}-\frac{1}{r_i\left(k\right)}{\widehat{\mathbf{\Psi }}}_i(p,k){\widehat{\mathbf{\Psi }}}_i^{\mathrm{T}}(p,k)\right]{\widehat{\mathit{\theta }}}_i(k-1)+\frac{1}{r_i\left(k\right)}{\widehat{\mathbf{\Psi }}}_i(p,k){\mathit{Y}}_i(p,k).\hfill \end{array}$$

(12)

To guarantee the convergence of this recursive algorithm, all eigenvalues of the matrix $\left[\mathit{I}-\frac{1}{r_i\left(k\right)}{\widehat{\mathbf{\Psi }}}_i(p,k){\widehat{\mathbf{\Psi }}}_i^{\mathrm{T}}(p,k)\right]$ should be located inside the unit circle. Therefore, a conservative choice of $\frac{1}{r_i\left(k\right)}$ is:

$$0<\frac{1}{r_i\left(k\right)}⩽\frac{1}{{\lambda }_{\max }[{\widehat{\mathbf{\Psi }}}_i(p,k){\widehat{\mathbf{\Psi }}}_i^{\mathrm{T}}(p,k)]}.$$

In this paper, we take a common choice:

$$r_i\left(k\right)={\lambda }_i{r}_i(k-1)+{\parallel {\widehat{\mathit{\phi }}}_i\left(k\right)\parallel }^2,r_i\left(0\right)=1,$$

(13)

where ${\lambda }_i\in (0,1]$ is the forgetting factor.

Substituting Equation (13) into Equation (12), the auxiliary model-based multi-innovation stochastic gradient (AM-MISG) algorithm can be derived:

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}_i\left(k\right)& =& {\widehat{\mathit{\theta }}}_i(k-1)+\frac{{\widehat{\mathbf{\Psi }}}_i(p,k)}{r_i\left(k\right)}[{\mathit{Y}}_i(p,k)-{\widehat{\mathbf{\Psi }}}_i^{\mathrm{T}}(p,k){\widehat{\mathit{\theta }}}_i(k-1)],\hfill \end{array}$$

(14)

$$\begin{array}{ccc}\hfill r_i\left(k\right)& =& {\lambda }_i{r}_i(k-1)+{\parallel {\widehat{\mathit{\phi }}}_i\left(k\right)\parallel }^2,r_i\left(0\right)=1,\hfill \end{array}$$

(15)

$$\begin{array}{ccc}\hfill {\mathit{Y}}_i(p,k)& =& {[y_i\left(k\right),y_i(k-1),\cdots ,y_i(k-p+1)]}^{\mathrm{T}},\hfill \end{array}$$

(16)

$$\begin{array}{ccc}\hfill {\widehat{\mathbf{\Psi }}}_i(p,k)& =& [{\widehat{\mathit{\phi }}}_i\left(k\right),{\widehat{\mathit{\phi }}}_i(k-1),\cdots ,{\widehat{\mathit{\phi }}}_i(k-p+1)],\hfill \end{array}$$

(17)

$${\widehat{\mathit{\phi }}}_i\left(k\right)=\left[\begin{array}{c}{\widehat{\mathit{\phi }}}_{iw}\left(k\right)\hfill \\ {\mathit{\phi }}_{iu}\left(k\right)\hfill \end{array}\right],$$

(18)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\phi }}}_{iw}\left(k\right)& =& {[-{\widehat{w}}_{i-1}\left(k\right),-{\widehat{w}}_{i-2}\left(k\right),\cdots ,-{\widehat{w}}_{i-n}\left(k\right)]}^{\mathrm{T}},\hfill \end{array}$$

(19)

$$\begin{array}{ccc}\hfill {\mathit{\phi }}_{iu}\left(k\right)& =& {[{\mathit{\varphi }}_{i0}^{\mathrm{T}}\left(k\right),{\mathit{\varphi }}_{i1}^{\mathrm{T}}\left(k\right),{\mathit{\varphi }}_{i2}^{\mathrm{T}}\left(k\right),\cdots ,{\mathit{\varphi }}_{i\mathrm{n}}^{\mathrm{T}}\left(k\right)]}^{\mathrm{T}},\hfill \end{array}$$

(20)

$$\begin{array}{ccc}\hfill {\mathit{\varphi }}_{ij}\left(k\right)& =& {[f_1\left[u_{i-j}\left(k\right)\right],f_2\left[u_{i-j}\left(k\right)\right],\cdots ,f_{n_c}\left[u_{i-j}\left(k\right)\right]]}^{\mathrm{T}},j=0,1,2,\cdots ,n,\hfill \end{array}$$

(21)

$$\begin{array}{ccc}\hfill {\widehat{w}}_i\left(k\right)& =& {\widehat{\mathit{\phi }}}_i^{\mathrm{T}}\left(k\right){\widehat{\mathit{\theta }}}_i\left(k\right).\hfill \end{array}$$

(22)

Since $c_1=1$, the estimates of $a_{ij}$ and $b_{ij}$ can be directly read from ${\widehat{\mathit{\theta }}}_i\left(k\right)$,

$$\begin{array}{ccc}\hfill {\widehat{a}}_{ij}\left(k\right)& =& {\widehat{\mathit{\theta }}}_{i,j}\left(k\right),j=1,2,\cdots ,n,\hfill \end{array}$$

(23)

$$\begin{array}{ccc}\hfill {\widehat{b}}_{ij}\left(k\right)& =& {\widehat{\mathit{\theta }}}_{i,n+n_{c}j+1}\left(k\right),j=0,1,2,\cdots ,n,\hfill \end{array}$$

(24)

where ${\widehat{\mathit{\theta }}}_{i,j}\left(k\right)$ represents the j-th element of ${\widehat{\mathit{\theta }}}_i\left(k\right)$. Note that $c_m$ ($m=2,3,\cdots ,n_c$) has been estimated $n+1$ times at each non-uniform sampling instant $kT+t_i$ ($i=0,1,2,\cdots ,q-1$). Therefore, we can simply take their average like in [36] as the estimate of $c_m$ over the k-th frame period, i.e.,

$${\widehat{c}}_m\left(k\right)=\frac{1}{q(n+1)}\sum _{i=0}^{q-1}\sum _{j=0}^n\frac{{\widehat{\mathit{\theta }}}_{i,n+n_{c}j+m}\left(k\right)}{{\widehat{b}}_{ij}\left(k\right)},m=2,3,\cdots ,n_c.$$

(25)

Furthermore, a more numerically-sound SVD-based approach proposed by Bai [41] can be applied to obtain the estimates of $b_{ij}$ and $c_m$.

The flowchart of the AM-MISG algorithm in (14)–(25) for computing the parameter estimates of periodically non-uniformly sampled-data Hammerstein systems can be illustrated in Figure 2, and the detailed implementation steps are summarized as follows:

• Initialization: Choose the data length L, the innovation length p and the forgetting factor ${\lambda }_i$; give the nonlinear basis functions $\{f_m(·),m=1,2,\cdots ,n_c\}$; set $u_i\left(k\right)=0$, $y_i\left(k\right)=0$, ${\widehat{w}}_i\left(k\right)=0$ for $k⩽0$ and $i=0,1,2,\cdots ,q-1$; take the initial values to be ${\widehat{\mathit{\theta }}}_i\left(0\right)={\mathbf{1}}_{n_0}/p_0$, where $p_0={10}^6$ and ${\mathbf{1}}_{n_0}$ is a column vector of ones; let $k=1$ and $i=0$.
• Collect the non-uniformly sampled input-output data $u_i\left(k\right)$ and $y_i\left(k\right)$.
• Calculate $f_m\left[u_i\left(k\right)\right]$ based on $u_i\left(k\right)$; form ${\widehat{\mathit{\phi }}}_{iw}\left(k\right)$, ${\mathit{\phi }}_{iu}\left(k\right)$, ${\mathit{\varphi }}_{ij}\left(k\right)$ by (19)–(21); and construct ${\widehat{\mathit{\phi }}}_i\left(k\right)$ by (18).
• Form the stacked output vector ${\mathit{Y}}_i(p,k)$ and the stacked information matrix ${\widehat{\mathbf{\Psi }}}_i(p,k)$ by (16) and (17), respectively.
• Compute the step size $r_i\left(k\right)$ by (15) and update the parameter estimate ${\widehat{\mathit{\theta }}}_i\left(k\right)$ by (14); calculate ${\widehat{w}}_i\left(k\right)$ by (22); obtain ${\widehat{a}}_{ij}\left(k\right)$ and ${\widehat{b}}_{ij}\left(k\right)$ based on (23) and (24), respectively.
• If $i<q-1$, then increase i by one, and go to Step 2; otherwise, compute ${\widehat{c}}_m\left(k\right),m=2,3,\cdots ,n_c$ by (25); let $i=0$ and go to the next step.
• If $k<L$, then increase k by one, and go to Step 2; otherwise, terminate the computing process.

3.3. The Main Convergence Result

The main convergence result of the proposed AM-MISG algorithm for periodically non-uniformly sampled-data Hammerstein systems is given in the following theorem.

Theorem 1.

Assume that the noise sequences $v_i\left(k\right)$ $(i=0,1,2,\cdots ,q-1)$ satisfy:

(A1) 

$\mathrm{E}\left[v_i\left(k\right)\right]=0,\mathrm{E}\left[v_i^2\left(k\right)\right]⩽{\sigma }^2<\infty ,\mathrm{E}\left[v_i\left(k\right)v_i\left(j\right)\right]=0,k\ne j,$

and there exist a positive constant α and an integer N such that the following persistent excitation condition holds:

(A2) 

${\sum }_{j=0}^N{\sum }_{l=0}^{p-1}\frac{{\widehat{\mathit{\phi }}}_i(k+j-l){\widehat{\mathit{\phi }}}_i^{\mathrm{T}}(k+j-l)}{r_i(k+j)}⩾\alpha \mathit{I}.$

Then, the parameter estimation vector ${\widehat{\mathit{\theta }}}_i\left(k\right)$ given by the AM-MISG algorithm consistently converges to the true parameter vector ${\mathit{\theta }}_i$ in the mean-square sense.

Theorem 1 can be proven in a similar way to [42]. Therefore, its detailed proof is omitted here.

4. Simulation Example

Assume that the nonlinear block in Figure 1 is described by:

$$\overline{u}\left(t\right)=u\left(t\right)+0.5u^2\left(t\right)+0.25u^3\left(t\right),$$

and the linear continuous process P is described by:

$$P\left(s\right)=\frac{1}{4s^2+2s+1}.$$

Over a frame period of $T=2\sqrt{2}+1$ s, the input-output data are non-uniformly-sampled twice (i.e., $q=2$) at $t_0=0$ s and $t_1=\sqrt{2}$ s. According to Theorem 1 in [35], the following ${\delta }^{-1}$-based transfer function model is derived:

$$\begin{array}{ccc}\hfill w_0\left(k\right)& =& \frac{1-0.61623{\delta }^{-1}+0.50931{\delta }^{-2}}{1-0.8086{\delta }^{-1}+0.25514{\delta }^{-2}}{\overline{u}}_0\left(k\right),\hfill \end{array}$$

$$\begin{array}{ccc}\hfill w_1\left(k\right)& =& \frac{1-0.49522{\delta }^{-1}+0.75553{\delta }^{-2}}{1-0.94779{\delta }^{-1}+0.57794{\delta }^{-2}}{\overline{u}}_1\left(k\right).\hfill \end{array}$$

Therefore, the parameters of this periodically non-uniformly sampled-data Hammerstein system are:

$$\begin{array}{c}{\mathit{a}}_0={[-0.8086,0.25514]}^{\mathrm{T}},{\mathit{b}}_0={[1,-0.61623,0.50931]}^{\mathrm{T}},\\ {\mathit{a}}_1={[-0.94779,0.57794]}^{\mathrm{T}},{\mathit{b}}_1={[1,-0.49522,0.75553]}^{\mathrm{T}},\\ \mathit{c}={[1,0.5,0.25]}^{\mathrm{T}}.\hfill \end{array}$$

In the simulation, take the non-uniform inputs $\left\{u_0\left(k\right)\right\}$ and $\left\{u_1\left(k\right)\right\}$ as two uncorrelated random sequences with zero mean and unit variance, $\left\{v_0\left(k\right)\right\}$ and $\left\{v_1\left(k\right)\right\}$ as two white noise sequences with zero mean and variance ${\sigma }^2={0.10}^2$. Based on 5000 non-uniform input-output dataset, applying the AM-MISG algorithm in (14)–(25) with ${\lambda }_0={\lambda }_1=0.95$ to estimate the system parameters, the results for $p=1$, $p=5$ and $p=12$ are shown in

Table 1. The auxiliary model-based multi-innovation stochastic gradient (AM-MISG) parameter estimates and errors ($p=1$).

k	100	200	500	1000	2000	3000	4000	5000	True Values
$a_{01}$	−0.23798	−0.23467	−0.24155	−0.23676	−0.32003	−0.37711	−0.42344	−0.46120	−0.80860
$a_{02}$	−0.17253	−0.15925	−0.14095	−0.10136	−0.07608	−0.04580	−0.00265	0.02211	0.25514
$b_{00}$	0.21247	0.28325	0.36820	0.43435	0.54777	0.63347	0.70653	0.76997	1.00000
$b_{01}$	0.03019	0.01933	0.01196	0.00916	0.00769	−0.01410	−0.04420	−0.08780	−0.61623
$b_{02}$	0.08793	0.10742	0.14003	0.18442	0.23272	0.28553	0.31263	0.32151	0.50931
$a_{11}$	−0.14144	−0.17751	−0.20109	−0.25406	−0.32583	−0.39976	−0.44355	−0.49299	−0.94779
$a_{12}$	−0.26469	−0.24184	−0.17359	−0.12188	−0.07576	−0.02374	0.04034	0.07391	0.57794
$b_{10}$	0.24088	0.32232	0.39463	0.45898	0.57929	0.66877	0.73108	0.77370	1.00000
$b_{11}$	0.02779	0.05257	0.07272	0.09047	0.11098	0.11615	0.10811	0.09453	−0.49522
$b_{12}$	0.04644	0.04343	0.04713	0.07480	0.12538	0.16100	0.18916	0.22188	0.75553
$c_2$	1.05391	0.96668	0.98187	0.93895	0.80087	0.72297	0.66712	0.63075	0.50000
$c_3$	1.79633	1.68065	1.44046	1.16009	0.82335	0.65456	0.52589	0.45922	0.25000
$\epsilon$$(\%)$	103.04890	97.41924	89.40996	80.57154	68.94369	61.44114	55.52206	51.00555

,

Table 2. The AM-MISG parameter estimates and errors ($p=5$).

k	100	200	500	1000	2000	3000	4000	5000	True Values
$a_{01}$	−0.17179	−0.18574	−0.31748	−0.41405	−0.61777	−0.71905	−0.76959	−0.80135	−0.80860
$a_{02}$	−0.17111	−0.11012	−0.05496	0.01077	0.12643	0.17775	0.21975	0.24397	0.25514
$b_{00}$	0.37344	0.45616	0.62647	0.76016	0.91687	0.95693	0.98483	0.99692	1.00000
$b_{01}$	0.06673	0.04317	0.02695	−0.05431	−0.24467	−0.39466	−0.50874	−0.56748	−0.61623
$b_{02}$	0.18052	0.22395	0.27192	0.33856	0.35650	0.39392	0.41745	0.43668	0.50931
$a_{11}$	−0.34108	−0.41100	−0.47313	−0.56512	−0.68070	−0.77319	−0.82088	−0.85964	−0.94779
$a_{12}$	−0.01887	−0.03044	0.07773	0.16053	0.30682	0.37436	0.44459	0.47782	0.57794
$b_{10}$	0.38009	0.49836	0.59120	0.75693	0.92872	0.97376	0.98098	0.98525	1.00000
$b_{11}$	0.02215	0.04295	0.06078	0.02968	−0.07356	−0.20574	−0.29673	−0.36568	−0.49522
$b_{12}$	0.10985	0.13310	0.18300	0.25133	0.33733	0.43426	0.52067	0.59753	0.75553
$c_2$	0.97825	0.90595	0.80559	0.63455	0.51789	0.50729	0.50178	0.50388	0.50000
$c_3$	1.35667	1.13733	0.73326	0.47388	0.31951	0.27787	0.26181	0.25438	0.25000
$\epsilon$$(\%)$	84.52995	75.91772	61.64633	48.73054	32.94076	22.45332	15.03823	10.03271

and

Table 3. The AM-MISG parameter estimates and errors ($p=12$).

k	100	200	500	1000	2000	3000	4000	5000	True Values
$a_{01}$	−0.21097	−0.29207	−0.48337	−0.62718	−0.78913	−0.82242	−0.82167	−0.81394	−0.80860
$a_{02}$	−0.09616	−0.00999	0.04310	0.14720	0.23542	0.25631	0.25755	0.25552	0.25514
$b_{00}$	0.46831	0.60206	0.82476	0.92527	0.99463	0.99562	0.99966	1.00088	1.00000
$b_{01}$	0.05982	0.04225	−0.06860	−0.27395	−0.53707	−0.61472	−0.62645	−0.62093	−0.61623
$b_{02}$	0.27987	0.32848	0.34589	0.38704	0.42283	0.47864	0.49652	0.50327	0.50931
$a_{11}$	−0.42997	−0.51235	−0.60352	−0.72537	−0.84006	−0.90571	−0.93254	−0.94364	−0.94779
$a_{12}$	−0.00011	0.04270	0.21788	0.32975	0.47520	0.53405	0.56383	0.57217	0.57794
$b_{10}$	0.44256	0.61696	0.76526	0.92836	1.00028	0.99858	0.99669	1.00004	1.00000
$b_{11}$	0.01971	0.05871	0.01162	−0.14602	−0.33453	−0.42806	−0.46968	−0.48801	−0.49522
$b_{12}$	0.11625	0.16567	0.24897	0.36557	0.55820	0.68047	0.73284	0.75125	0.75553
$c_2$	0.88043	0.76552	0.64129	0.52932	0.49418	0.49855	0.50087	0.49915	0.50000
$c_3$	1.04485	0.79819	0.45534	0.30318	0.25542	0.25328	0.25163	0.25057	0.25000
$\epsilon$$(\%)$	73.56646	62.33004	45.43354	29.28325	12.29280	4.72652	1.75023	0.55900

, respectively, where $\epsilon$ is the estimation error defined as:

$$\epsilon =\sqrt{\frac{{\parallel {\widehat{\mathit{a}}}_0-{\mathit{a}}_0\parallel }^2+{\parallel {\widehat{\mathit{b}}}_0-{\mathit{b}}_0\parallel }^2+{\parallel {\widehat{\mathit{a}}}_1-{\mathit{a}}_1\parallel }^2+{\parallel {\widehat{\mathit{b}}}_1-{\mathit{b}}_1\parallel }^2+{\parallel \widehat{\mathit{c}}-\mathit{c}\parallel }^2}{{\parallel {\mathit{a}}_0\parallel }^2+{\parallel {\mathit{b}}_0\parallel }^2+{\parallel {\mathit{a}}_1\parallel }^2+{\parallel {\mathit{b}}_1\parallel }^2+{\parallel \mathit{c}\parallel }^2}}\times 100\%.$$

Meanwhile, the estimation errors $\epsilon$ versus k are shown in Figure 3.

From Table 1, Table 2 and Table 3 and Figure 3, we can see that the parameter estimation error gradually decreases as the data length k increases, demonstrating the effectiveness of the proposed AM-MISG algorithm. Furthermore, the AM-MISG algorithm with a larger innovation length p can result in higher identification accuracy and faster convergence to the true parameters.

To study the identification performance of the proposed AM-MISG algorithm against the output noise, 50 Monte Carlo simulations for the noise variance being ${\sigma }^2={0.10}^2$ and ${\sigma }^2={0.50}^2$ have been conducted, respectively. In each simulation run, a new dataset with length $L=5000$ is generated to estimate the model parameters. For the innovation length $p=1$, $p=5$ and $p=12$, the mean values and the standard deviations of the parameter estimates are listed in

Table 4. The AM-MISG parameter estimates with standard deviations (${\sigma }^2=0.{10}^2$).

Parameters	$\mathit{p}=1$	$\mathit{p}=5$	$\mathit{p}=12$	True Values
$a_{01}$	$-0.4618\pm 0.0241$	$-0.7999\pm 0.0067$	$-0.8156\pm 0.0016$	−0.80860
$a_{02}$	$0.0190\pm 0.0145$	$0.2390\pm 0.0050$	$0.2575\pm 0.0012$	0.25514
$b_{00}$	$0.7923\pm 0.0148$	$0.9979\pm 0.0038$	$1.0001\pm 0.0010$	1.00000
$b_{01}$	$-0.0783\pm 0.0165$	$-0.5640\pm 0.0081$	$-0.6243\pm 0.0020$	−0.61623
$b_{02}$	$0.3478\pm 0.0153$	$0.4419\pm 0.0054$	$0.5030\pm 0.0019$	0.50931
$a_{11}$	$-0.5010\pm 0.0187$	$-0.8457\pm 0.0105$	$-0.9396\pm 0.0024$	−0.94779
$a_{12}$	$0.0944\pm 0.0165$	$0.4700\pm 0.0099$	$0.5707\pm 0.0021$	0.57794
$b_{10}$	$0.7879\pm 0.0164$	$0.9951\pm 0.0049$	$0.9999\pm 0.0014$	1.00000
$b_{11}$	$0.1057\pm 0.0137$	$-0.3463\pm 0.0136$	$-0.4830\pm 0.0029$	−0.49522
$b_{12}$	$0.2357\pm 0.0127$	$0.5779\pm 0.0138$	$0.7465\pm 0.0033$	0.75553
$c_2$	$0.6136\pm 0.0106$	$0.4982\pm 0.0035$	$0.4998\pm 0.0010$	0.50000
$c_3$	$0.4436\pm 0.0153$	$0.2516\pm 0.0033$	$0.2499\pm 0.0009$	0.25000

and

Table 5. The AM-MISG parameter estimates with standard deviations (${\sigma }^2={0.50}^2$).

Parameters	$\mathit{p}=1$	$\mathit{p}=5$	$\mathit{p}=12$	True Values
$a_{01}$	$-0.4613\pm 0.0261$	$-0.7870\pm 0.0217$	$-0.7921\pm 0.0249$	−0.80860
$a_{02}$	$0.0186\pm 0.0168$	$0.2275\pm 0.0173$	$0.2376\pm 0.0268$	0.25514
$b_{00}$	$0.7931\pm 0.0161$	$1.0017\pm 0.0170$	$1.0071\pm 0.0231$	1.00000
$b_{01}$	$-0.0777\pm 0.0191$	$-0.5491\pm 0.0217$	$-0.5954\pm 0.0350$	−0.61623
$b_{02}$	$0.3489\pm 0.0178$	$0.4452\pm 0.0203$	$0.5015\pm 0.0307$	0.50931
$a_{11}$	$-0.5009\pm 0.0225$	$-0.8320\pm 0.0203$	$-0.9036\pm 0.0269$	−0.94779
$a_{12}$	$0.0951\pm 0.0195$	$0.4552\pm 0.0203$	$0.5323\pm 0.0260$	0.57794
$b_{10}$	$0.7872\pm 0.0197$	$0.9938\pm 0.0203$	$1.0007\pm 0.0295$	1.00000
$b_{11}$	$0.1037\pm 0.0168$	$-0.3375\pm 0.0244$	$-0.4518\pm 0.0327$	−0.49522
$b_{12}$	$0.2375\pm 0.0171$	$0.5744\pm 0.0256$	$0.7177\pm 0.0312$	0.75553
$c_2$	$0.6130\pm 0.0121$	$0.4961\pm 0.0141$	$0.4949\pm 0.0207$	0.50000
$c_3$	$0.4446\pm 0.0185$	$0.2510\pm 0.0138$	$0.2462\pm 0.0166$	0.25000

. From the simulation results, it can be observed that the estimation accuracy of the AM-MISG algorithm is higher when the noise variance is smaller. For a larger noise variance, increasing the innovation length p can help to obtain a satisfactory identification result.

Considering the noise variance ${\sigma }^2={0.10}^2$ and ${\sigma }^2={0.50}^2$, a separate dataset with length 30 has been generated for model validation, respectively. Using the mean values of the parameter estimates listed in Table 4 and Table 5 for $p=12$ to predict the outputs of the periodically non-uniformly sampled-data Hammerstein system, the results are shown in Figure 4, where the solid line and the x-marks represent the true measured outputs and the model predicted outputs, respectively. From Figure 4, it is clear that the model predictions can catch the trend of the true outputs well, and the prediction performance becomes better if a noise with smaller variance is introduced.

5. Conclusions

Based on the non-uniform input-output dataset, an auxiliary model-based stochastic gradient (AM-SG) algorithm is developed in this paper to estimate the parameters of Hammerstein systems. To improve the identification performance of the AM-SG algorithm, an auxiliary model-based multi-innovation stochastic gradient (AM-MISG) algorithm is proposed by introducing an innovation length p. The simulation results illustrate that the AM-MISG algorithm with a larger p can provide more accurate parameter estimates and a faster convergence rate. In addition, the proposed algorithm can be extended to identify more complex nonlinear systems with non-uniform sampling.