Parameter Estimation Algorithms for Hammerstein Finite Impulse Response Moving Average Systems Using the Data Filtering Theory

Abstract

This paper considers the parameter estimation problems of Hammerstein finite impulse response moving average (FIR–MA) systems. Based on the matrix transformation and the hierarchical identification principle, the Hammerstein FIR–MA system is recast into two models, and a decomposition-based recursive least-squares algorithm is deduced for estimating the parameters of these two models. In order to further improve the accuracy of the parameter estimation, a multi-innovation hierarchical least-squares algorithm based on the data filtering theory proposed. Finally, a simulation example demonstrates the effectiveness of the proposed scheme.

1. Introduction

The idetification methods have been very mature for multivariable linear systems [1,2]. However, many systems in practical applications and industrial control are nonlinear multivariable systems [3,4]. Nonlinear multivariable systems are different from scalar systems or linear systems, and each of their outputs is usually controlled and affected by several inputs at the same time, so the parameter estimation of multivariable systems is more difficult [5,6,7]. In recent years, many scholars have studied the parameter estimation problem of nonlinear multivariable systems and proposed many identification methods. You et al. proposed an iterative algorithm for the identification of multiple-input single-output output-error systems with unknown time-delays, based on the basis pursuit denoising criterion and the auxiliary model identification idea [8]. Wang and Ding presented a novel recursive least-squares algorithm for multiple-input multiple-output (MIMO) systems with autoregressive moving average noise, employing the auxiliary model and the data filtering technique [9]. Lenka studied the asymptotic stability of equilibrium points of multivariable fractional order systems based on the fractional comparison principle and the Laplace transform [10]. Liu et al. investigated parameter estimation problems for multivariable controlled autoregressive autoregressive moving average systems, and derived a partially coupled generalized extended stochastic gradient algorithm by using the auxiliary model [11].

Least-squares and stochastic gradient algorithms are two methods for parameter estimation [12,13,14,15]. Compared with the stochastic gradient algorithm, the recursive least-squares algorithm has fast convergence rate and high computational effort [16]. The recursive least-squares algorithm needs to calculate the inverse of the covariance matrix in the identification process; when the dimension of the covariance matrix is high, the computational complexity is very large. In order to reduce the complexity of the calculation, the decomposition technique can effectively solve the problem of a large computational burden in large-scale system identification [17,18,19]. The decomposition technique is used to decompose the original system into multiple subsystems and to identify the parameters in each subsystem separately [20,21]. For example, Wang et al. presented a hierarchical least-squares iterative algorithm to solve the difficulty that the identification model contains the unmeasurable variables and noise terms in the information matrix [22]. Ji et al. derived a hierarchical least-squares algorithm for two-input Hammerstein finite impulse response systems based on the hierarchical identification principle and the multi-innovation theory [23]. Wang et al. transformed a MIMO Hammerstein system with different types of coefficients into an over-parametrisation regression identification model, and then proposed a hierarchical extended stochastic gradient algorithm [24].

In order to reduce the error of parameter estimation, the data filtering technique can be used to get rid of the outliers and weaken the influence of noises to better extract useful information in the data [25,26]. Chen et al. combined the maximum likelihood principle, the decomposition technique and the data filtering technique, to present a maximum likelihood generalized extended gradient algorithm, and a data filtering-based maximum likelihood extended gradient algorithm [27]. Mao et al. presented a data filtering multi-innovation stochastic gradient identification algorithm for Hammerstein output-error autoregressive systems by means of the multi-innovation identification theory [28]. By eliminating the state variables in the systems, Li et al. proposed a filtering-based least-squares iterative algorithm for estimating the parameters of bilinear systems with colored noises [29]. The filtering technique can also be applied to signal processing and communication [30,31,32], and neural networks [33,34].

The parameter estimation of the system models is important for control system analysis and design. The parameters of the models can be estimated by using some identification methods [35,36,37] such as the hierarchical algorithms [38,39]. This paper studies the parameter estimation problems of two-input two-output Hammerstein finite impulse response moving average (FIR–MA) systems. The Hammerstein nonlinear system includes the nonlinear block and the linear block and its identification face more difficulties such as high computational complexity and parameter identifiability. In this paper, the decomposition technique and data filtering technique are employed to solve these problems. The main contributions of this paper lie in the following.

• Based on the decomposition technique, we decomposed the Hammerstein system into two models, each of which is expressed as a regression form in the parameters of the nonlinear part or in the parameters of the linear part, and we propose a hierarchical least-squares algorithm.
• By applying the data filtering technique, the input–output data are filtered, and a filtering-based hierarchical least-squares algorithm is presented for Hammerstein finite impulse response moving average systems to improve the accuracy of parameter estimation.

Briefly, the structure of this paper is organized as follows. Section 2 describes a two-input two-output Hammerstein FIR–MA system. Section 3 derives a hierarchical least-squares algorithm for the TITO Hammerstein system. Section 4 derives a filtering-based hierarchical least-squares algorithm for the TITO Hammerstein system. Section 5 provides an illustrative example to show the effectiveness of the proposed algorithms. Finally, we offer some concluding remarks in Section 6.

2. System Description and Identification Model

Let us define some symbols. “$A=:B$” or “$B:=A$” stands for “A is defined as B”. The superscript T denotes the matrix/vector transpose; the norm of the matrix X is defined by ${\parallel \mathit{X}\parallel }^2:=\mathrm{tr}\left[\mathit{X}{\mathit{X}}^{\mathrm{T}}\right]$. The symbol ${\mathit{I}}_n$ stands for an identity matrix of size $n\times n$. ${\mathbf{1}}_n$ represents a n-dimensional column vector whose elements are 1.

Consider the following Hammerstein FIR–MA system, depicted in Figure 1,

$$\begin{array}{ccc}\hfill \mathit{y}\left(t\right)& =& \mathit{\theta }\left(z\right)\overline{\mathit{u}}\left(t\right)+\alpha \left(z\right)\mathit{v}\left(t\right),\hfill \end{array}$$

(1)

where $\mathit{u}\left(t\right):={[{\overline{u}}_1\left(t\right),{\overline{u}}_2\left(t\right)]}^{\mathrm{T}}\in {ℝ}^2$ is the system nonline input, $\mathit{y}\left(t\right)\in {ℝ}^2$ is the system output, $\mathit{v}\left(t\right)\in {ℝ}^2$ is the stochastic white noise with zero mean, $\mathit{\theta }\left(z\right):=[{\mathit{\theta }}_1\left(z\right),{\mathit{\theta }}_2\left(z\right)]$ and $\alpha \left(z\right)$ are two matrix polynomials in the unit backward shift operator $z^{-1}$$[z^{-1}y\left(t\right)=y(t-1)]$, and defined by

$$\begin{array}{ccc}\hfill {\mathit{\theta }}_i\left(z\right)& :=& {\mathit{\theta }}_{i,1}{z}^{-1}+{\mathit{\theta }}_{i,2}{z}^{-2}+\cdots +{\mathit{\theta }}_{i,n}{z}^{-n}\in {ℝ}^2,i=1,2,\hfill \end{array}$$

(2)

$$\begin{array}{ccc}\hfill \alpha \left(z\right)& :=& 1+{\alpha }_1{z}^{-1}+{\alpha }_2{z}^{-2}+\cdots +{\alpha }_n{z}^{-n}\in ℝ.\hfill \end{array}$$

(3)

The unknown nonlinear input ${\overline{u}}_i\left(t\right)$ is assumed to be a static nonlinear function of the known basis $d_{i,k}$ with ${\gamma }_{i,k}$ as its parameters:

$$\begin{array}{ccc}\hfill {\overline{u}}_i\left(t\right)& =& d_i\left[u_i\left(t\right)\right]=\sum _{k=1}^s{\gamma }_{i,k}{d}_{i,k}\left[u_i\left(t\right)\right].\hfill \end{array}$$

(4)

Assume that the orders n and s are known and $\overline{\mathit{u}}\left(t\right)=\mathbf{0}$, $\mathit{y}\left(t\right)=\mathbf{0}$ and $\mathit{v}\left(t\right)=\mathbf{0}$ for $t\le 0$.

The purpose of this paper is to estimate three parameters ${\mathit{\theta }}_{i,n}$, ${\gamma }_{i,k}$ and ${\alpha }_i$.

3. The Hierarchical Least-Squares Algorithm

From Equation (1), we have

$$\begin{array}{ccc}\hfill \mathit{y}\left(t\right)& =& {\mathbf{A}}_1\left(z\right){\overline{u}}_1\left(t\right)+{\mathbf{A}}_2\left(z\right){\overline{u}}_2\left(t\right)+\alpha \left(z\right)\mathit{v}\left(t\right).\hfill \end{array}$$

(5)

The first two terms of the right hand side can be expressed by

$$\begin{array}{ccc}\hfill {\mathbf{A}}_i\left(z\right){\overline{u}}_i\left(t\right)& =& [{\mathbf{A}}_{i,1}{z}^{-1}+{\mathbf{A}}_{i,2}{z}^{-2}+\cdots +{\mathbf{A}}_{i,n}{z}^{-n}]{\overline{u}}_i\left(t\right)\hfill \\ & =& {\mathbf{A}}_{i,1}{\overline{u}}_i(t-1)+{\mathbf{A}}_{i,2}{\overline{u}}_i(t-2)+\cdots +{\mathbf{A}}_{i,n}{\overline{u}}_i(t-n)\hfill \\ & =& [{\mathbf{A}}_{i,1},{\mathbf{A}}_{i,2},\cdots ,{\mathbf{A}}_{i,n}]\left[\begin{array}{c}{\overline{u}}_i(t-1)\\ {\overline{u}}_i(t-2)\\ ⋮\\ {\overline{u}}_i(t-n)\end{array}\right]\hfill \\ & =& [{\mathbf{A}}_{i,1},{\mathbf{A}}_{i,2},\cdots ,{\mathbf{A}}_{i,n}]\left[\begin{array}{c}\sum _{k=1}^s{\gamma }_{i,k}{d}_{i,k}\left[u_i(t-1)\right]\\ \sum _{k=1}^s{\gamma }_{i,k}{d}_{i,k}\left[u_i(t-2)\right]\\ ⋮\\ \sum _{k=1}^s{\gamma }_{i,k}{d}_{i,k}\left[u_i(t-n)\right]\end{array}\right].\hfill \end{array}$$

(6)

Define

$${\mathit{\gamma }}_i:=\left[\begin{array}{c}{\gamma }_{i,1}\\ {\gamma }_{i,2}\\ ⋮\\ {\gamma }_{i,s}\end{array}\right]\in {ℝ}^s,{\mathit{d}}_i\left(t\right):={\left[\begin{array}{c}{d}_{i,1}\left[u_i\left(t\right)\right]\\ d_{i,2}\left[u_i\left(t\right)\right]\\ ⋮\\ d_{i,s}\left[u_i\left(t\right)\right]\end{array}\right]}^{\mathrm{T}}\in {ℝ}^{1\times s},i=1,2.\hfill$$

Then Equation (5) can be rewritten as

$$\begin{array}{ccc}\hfill \mathit{y}\left(t\right)& =& [{\mathit{\theta }}_{1,1},{\mathit{\theta }}_{1,2},\dots ,{\mathit{\theta }}_{1,n}]\left[\begin{array}{c}{\mathit{d}}_1\left[u_1(t-1)\right]{\mathit{\gamma }}_1\\ {\mathit{d}}_1\left[u_1(t-2)\right]{\mathit{\gamma }}_1\\ ⋮\\ {\mathit{d}}_1\left[u_1(t-n)\right]{\mathit{\gamma }}_1\end{array}\right]\hfill \\ & +& [{\mathit{\theta }}_{2,1},{\mathit{\theta }}_{2,2},\dots ,{\mathit{\theta }}_{2,n}]\left[\begin{array}{c}{\mathit{d}}_2\left[u_2(t-1)\right]{\mathit{\gamma }}_2\\ {\mathit{d}}_2\left[u_2(t-2)\right]{\mathit{\gamma }}_2\\ ⋮\\ {\mathit{d}}_2\left[u_2(t-n)\right]{\mathit{\gamma }}_2\end{array}\right]\hfill \\ & +& \alpha \left(z\right)\mathit{v}\left(t\right).\hfill \end{array}$$

(7)

Define

$$\begin{array}{ccc}\hfill {\mathit{\theta }}_i& :=& [{\mathit{\theta }}_{i,1},{\mathit{\theta }}_{i,2},\cdots ,{\mathit{\theta }}_{i,n}]\in {ℝ}^{2\times n},\hfill \\ \hfill {\mathit{D}}_i\left(t\right)& :=& \left[\begin{array}{c}{\mathit{d}}_i\left[u_i(t-1)\right]\\ {\mathit{d}}_i\left[u_i(t-2)\right]\\ ⋮\\ {\mathit{d}}_i\left[u_i(t-n)\right]\end{array}\right]\hfill \\ & =& \left[\begin{array}{ccccccc}{d}_{i,1}\left[u_i(t-1)\right]& & d_{i,2}\left[u_i(t-1)\right]& & \cdots & & d_{i,s}\left[u_i(t-1)\right]\\ d_{i,1}\left[u_i(t-2)\right]& & d_{i,2}\left[u_i(t-2)\right]& & \cdots & & d_{i,s}\left[u_i(t-2)\right]\\ ⋮& & ⋮& & & & ⋮\\ d_{i,1}\left[u_i(t-n)\right]& & d_{i,2}\left[u_i(t-n)\right]& & \cdots & & d_{i,s}\left[u_i(t-n)\right]\end{array}\right]\in {ℝ}^{n\times s}.\hfill \end{array}$$

Then, Equation (7) can be rewritten in the following matrix form:

$$\begin{array}{ccc}\hfill \mathit{y}\left(t\right)& =& [{\mathit{\theta }}_{1,1},{\mathit{\theta }}_{1,2},\dots ,{\mathit{\theta }}_{1,n}]{\mathit{D}}_1\left(t\right){\mathit{\gamma }}_1+[{\mathit{\theta }}_{2,1},{\mathit{\theta }}_{2,2},\dots ,{\mathit{\theta }}_{2,n}]{\mathit{D}}_2\left(t\right){\mathit{\gamma }}_2+\alpha \left(z\right)\mathit{v}\left(t\right)\hfill \\ & =& {\mathit{\theta }}_1{\mathit{D}}_1\left(t\right){\mathit{\gamma }}_1+{\mathit{\theta }}_2{\mathit{D}}_2\left(t\right){\mathit{\gamma }}_2+\alpha \left(z\right)\mathit{v}\left(t\right)\hfill \\ & =& \sum _{i=1}^2{\mathit{\theta }}_i{\mathit{D}}_i\left(t\right){\mathit{\gamma }}_i+\alpha \left(z\right)\mathit{v}\left(t\right).\hfill \end{array}$$

(8)

Notice that in the identification model (8), it is difficult to simultaneously estimate the matrix ${\mathit{\theta }}_i$ and the vector ${\mathit{\gamma }}_i$. Here we use the hierarchical identification principle to transform the Hammerstein identification model into two different forms.

Define

$$\begin{array}{ccc}\hfill {\mathit{D}}_{\gamma }\left(t\right)& :=& \left[\begin{array}{c}{\mathit{D}}_1\left(t\right){\mathit{\gamma }}_1\\ {\mathit{D}}_2\left(t\right){\mathit{\gamma }}_2\end{array}\right]\in {ℝ}^{2n},\mathit{\theta }:=[{\mathit{\theta }}_1,{\mathit{\theta }}_2]\in {ℝ}^{2\times 2n},\hfill \\ \hfill {\mathit{D}}_{\theta }\left(t\right)& :=& [{\mathit{\theta }}_1{\mathit{D}}_1\left(t\right),{\mathit{\theta }}_2{\mathit{D}}_2\left(t\right)]\in {ℝ}^{2\times 2s},\mathit{\gamma }:=\left[\begin{array}{c}{\mathit{\gamma }}_1\\ {\mathit{\gamma }}_2\end{array}\right]\in {ℝ}^{2s},\hfill \\ \hfill \mathit{\alpha }& :=& {[{\alpha }_1,{\alpha }_2,\dots ,{\alpha }_n]}^{\mathrm{T}}\in {ℝ}^n,{\mathit{\varphi }}_s\left(t\right):={[\mathit{v}(t-1),\mathit{v}(t-2),\dots ,\mathit{v}(t-n)]}^{\mathrm{T}}\in {ℝ}^{n\times 2}.\hfill \end{array}$$

The identification model in (8) can be transformed into the following two forms:

$$\begin{array}{ccc}\hfill S_1:\mathit{y}\left(t\right)& =& \mathit{\theta }{\mathit{D}}_{\gamma }\left(t\right)+{\mathit{\varphi }}_s^{\mathrm{T}}\left(t\right)\mathit{\alpha }+\mathit{v}\left(t\right),\hfill \end{array}$$

(9)

$$\begin{array}{ccc}\hfill S_2:\mathit{y}\left(t\right)& =& {\mathit{D}}_{\theta }\left(t\right)\mathit{\gamma }+{\mathit{\varphi }}_s^{\mathrm{T}}\left(t\right)\mathit{\alpha }+\mathit{v}\left(t\right).\hfill \end{array}$$

(10)

The parameter matrices $\mathit{\theta }$, $\mathit{\alpha }$ and $\mathit{\gamma }$ to be identified are included in these two new models. Then, define two cost functions about the parameter matrices $\mathit{\theta }$, $\mathit{\alpha }$ and $\mathit{\gamma }$:

$$\begin{array}{ccc}\hfill J_1(\mathit{\theta },\mathit{\alpha })& :=& \sum _{l=1}^t{[\mathit{y}\left(l\right)-\mathit{\theta }{\mathit{D}}_{\gamma }\left(l\right)-{\mathit{\varphi }}_s^{\mathrm{T}}\left(l\right)\mathit{\alpha }]}^2,\hfill \\ \hfill J_2\left(\mathit{\gamma }\right)& :=& \sum _{l=1}^t{[\mathit{y}\left(l\right)-{\mathit{D}}_{\theta }\left(l\right)\mathit{\gamma }-{\mathit{\varphi }}_s^{\mathrm{T}}\left(l\right)\mathit{\alpha }]}^2.\hfill \end{array}$$

There are several difficulties in minimizing the two cost functions to obtain the parameter estimates. As the information matrix ${\mathit{D}}_{\theta }\left(t\right)$ contains the unknown parameter matrix ${\mathit{\theta }}_i$, the information vector ${\mathit{D}}_{\gamma }\left(t\right)$ contains the unknown parameter vector ${\mathit{\gamma }}_i$, the information matrix ${\mathit{\varphi }}_s\left(t\right)$ contains the unmeasurable noise term $\mathit{v}(t-i)$. The problem is solved by replacing the unknown parameter matrix ${\mathit{\theta }}_i$ with its estimate ${\widehat{\mathit{\theta }}}_i\left(t\right)$, replacing the unknown parameter vector ${\mathit{\gamma }}_i$ with its estimate ${\widehat{\mathit{\gamma }}}_i\left(t\right)$, replacing the unmeasurable $\mathit{v}(t-i)$ with its estimate $\widehat{\mathit{v}}(t-i)$.

The estimate of ${\mathit{\varphi }}_s\left(t\right)$, ${\mathit{D}}_{\gamma }\left(t\right)$ and ${\mathit{D}}_{\theta }\left(t\right)$ can be written as

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\varphi }}}_s\left(t\right)& :=& {[\widehat{\mathit{v}}(t-1),\widehat{\mathit{v}}(t-2),\dots ,\widehat{\mathit{v}}(t-n)]}^{\mathrm{T}}.\hfill \\ \hfill {\widehat{\mathit{D}}}_{\gamma }\left(t\right)& :=& \left[\begin{array}{c}{\mathit{D}}_1\left(t\right){\widehat{\mathit{\gamma }}}_1(t-1)\\ {\mathit{D}}_2\left(t\right){\widehat{\mathit{\gamma }}}_2(t-1)\end{array}\right],\hfill \\ \hfill {\widehat{\mathit{D}}}_{\theta }\left(t\right)& :=& [{\widehat{\mathit{\theta }}}_1\left(t\right){\mathit{D}}_1\left(t\right),{\widehat{\mathit{\theta }}}_2\left(t\right){\mathit{D}}_2\left(t\right)].\hfill \end{array}$$

Then, the estimate $\widehat{\mathit{v}}\left(t\right)$ can be computed by

$$\begin{array}{c}\hfill \widehat{\mathit{v}}\left(t\right)=\mathit{y}\left(t\right)-\widehat{\mathit{\theta }}\left(t\right){\widehat{\mathit{D}}}_{\gamma }\left(t\right)-{\widehat{\mathit{\varphi }}}_s^{\mathrm{T}}\left(t\right)\widehat{\mathit{\alpha }}\left(t\right).\end{array}$$

Then, we obtain the hierarchical least-squares (HLS) parameter estimation algorithm for the TITO Hammerstein FIR-MA models as follows:

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}^{\mathrm{T}}\left(t\right)& =& {\widehat{\mathit{\theta }}}^{\mathrm{T}}(t-1)+{\mathit{L}}_{\theta }\left(t\right){[\mathit{y}\left(t\right)-\widehat{\mathit{\theta }}(t-1){\widehat{\mathit{D}}}_{\gamma }\left(t\right)-{\widehat{\mathit{\varphi }}}_s^{\mathrm{T}}\left(t\right)\widehat{\mathit{\alpha }}(t-1)]}^{\mathrm{T}},\hfill \end{array}$$

(11)

$$\begin{array}{ccc}\hfill {\mathit{L}}_{\theta }\left(t\right)& =& {\mathit{P}}_{\theta }(t-1){\widehat{\mathit{D}}}_{\gamma }\left(t\right){[1+{\widehat{\mathit{D}}}_{\gamma }^{\mathrm{T}}\left(t\right){\mathit{P}}_{\theta }(t-1){\widehat{\mathit{D}}}_{\gamma }\left(t\right)]}^{-1},\hfill \end{array}$$

(12)

$$\begin{array}{ccc}\hfill {\mathit{P}}_{\theta }\left(t\right)& =& [{\mathit{I}}_n-{\mathit{L}}_{\theta }\left(t\right){\widehat{\mathit{D}}}_{\gamma }^{\mathrm{T}}\left(t\right)]{\mathit{P}}_{\theta }(t-1),{\mathit{P}}_{\theta }\left(0\right)=p_0{\mathit{I}}_n,\hfill \end{array}$$

(13)

$$\begin{array}{ccc}\hfill \widehat{\mathit{\alpha }}\left(t\right)& =& \widehat{\mathit{\alpha }}(t-1)+{\mathit{L}}_{\alpha }\left(t\right)[\mathit{y}\left(t\right)-\widehat{\mathit{\theta }}\left(t\right){\widehat{\mathit{D}}}_{\gamma }\left(t\right)-{\widehat{\mathit{\varphi }}}_s^{\mathrm{T}}\left(t\right)\widehat{\mathit{\alpha }}(t-1)],\hfill \end{array}$$

(14)

$$\begin{array}{ccc}\hfill {\mathit{L}}_{\alpha }\left(t\right)& =& {\mathit{P}}_{\alpha }(t-1){\widehat{\mathit{\varphi }}}_s^{\mathrm{T}}\left(t\right){[{\mathit{I}}_n+{\widehat{\mathit{\varphi }}}_s\left(t\right){\mathit{P}}_{\alpha }(t-1){\widehat{\mathit{\varphi }}}_s^{\mathrm{T}}\left(t\right)]}^{-1}\hfill \end{array}$$

(15)

$$\begin{array}{ccc}\hfill {\mathit{P}}_{\alpha }\left(t\right)& =& [{\mathit{I}}_n-{\mathit{L}}_{\alpha }\left(t\right){\widehat{\mathit{\varphi }}}_s^{\mathrm{T}}\left(t\right)]{\mathit{P}}_{\alpha }(t-1),{\mathit{P}}_{\alpha }\left(0\right)=p_0{\mathit{I}}_n,\hfill \end{array}$$

(16)

$$\begin{array}{ccc}\hfill \widehat{\mathit{\gamma }}\left(t\right)& =& \widehat{\mathit{\gamma }}(t-1)+{\mathit{L}}_{\gamma }\left(t\right)[\mathit{y}\left(t\right)-{\widehat{\mathit{D}}}_{\theta }\left(t\right)\widehat{\mathit{\gamma }}(t-1)-{\widehat{\mathit{\varphi }}}_s^{\mathrm{T}}\left(t\right)\widehat{\mathit{\alpha }}(t-1)],\hfill \end{array}$$

(17)

$$\begin{array}{ccc}\hfill {\mathit{L}}_{\gamma }\left(t\right)& =& {\mathit{P}}_{\gamma }(t-1){\widehat{\mathit{D}}}_{\theta }^{\mathrm{T}}\left(t\right){[{\mathit{I}}_s+{\widehat{\mathit{D}}}_{\theta }\left(t\right){\mathit{P}}_{\gamma }(t-1){\mathit{D}}_{\theta }^{\mathrm{T}}\left(t\right)]}^{-1},\hfill \end{array}$$

(18)

$$\begin{array}{ccc}\hfill {\mathit{P}}_{\gamma }\left(t\right)& =& [{\mathit{I}}_s-{\mathit{L}}_{\gamma }\left(t\right){\widehat{\mathit{D}}}_{\theta }\left(t\right)]{\mathit{P}}_{\gamma }(t-1),{\mathit{P}}_{\gamma }\left(0\right)=p_0{\mathit{I}}_s,\hfill \end{array}$$

(19)

$$\begin{array}{ccc}\hfill {\mathit{d}}_i\left(t\right)& =& [d_{i,1}\left[u_i\left(t\right)\right],d_{i,2}\left[u_i\left(t\right)\right],\dots ,d_{i,s}\left[u_i\left(t\right)\right]]\in {ℝ}^{1\times s},i=1,2,\hfill \end{array}$$

(20)

$$\begin{array}{ccc}\hfill {\mathit{D}}_i\left(t\right)& =& \left[\begin{array}{ccccccc}{d}_{i,1}\left[u_i(t-1)\right]& & d_{i,2}\left[u_i(t-1)\right]& & \cdots & & d_{i,s}\left[u_i(t-1)\right]\\ d_{i,1}\left[u_i(t-2)\right]& & d_{i,2}\left[u_i(t-2)\right]& & \cdots & & d_{i,s}\left[u_i(t-2)\right]\\ ⋮& & ⋮& & & & ⋮\\ d_{i,1}\left[u_i(t-n)\right]& & d_{i,2}\left[u_i(t-n)\right]& & \cdots & & d_{i,s}\left[u_i(t-n)\right]\end{array}\right],\hfill \end{array}$$

(21)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{D}}}_{\gamma }\left(t\right)& =& \left[\begin{array}{c}{\mathit{D}}_1\left(t\right){\widehat{\mathit{\gamma }}}_1(t-1)\\ {\mathit{D}}_2\left(t\right){\widehat{\mathit{\gamma }}}_2(t-1)\end{array}\right],\hfill \end{array}$$

(22)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{D}}}_{\theta }\left(t\right)& =& [{\widehat{\mathit{\theta }}}_1\left(t\right){\mathit{D}}_1\left(t\right),{\widehat{\mathit{\theta }}}_2\left(t\right){\mathit{D}}_2\left(t\right)],\hfill \end{array}$$

(23)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\varphi }}}_s\left(t\right)& =& {[\widehat{\mathit{v}}(t-1),\widehat{\mathit{v}}(t-2),\dots ,\widehat{\mathit{v}}(t-n)]}^{\mathrm{T}},\hfill \end{array}$$

(24)

$$\begin{array}{ccc}\hfill \widehat{\mathit{v}}\left(t\right)& =& \mathit{y}\left(t\right)-\widehat{\mathit{\theta }}\left(t\right){\widehat{\mathit{D}}}_{\gamma }\left(t\right)-{\widehat{\mathit{\varphi }}}_s^{\mathrm{T}}\left(t\right)\widehat{\mathit{\alpha }}\left(t\right),\hfill \end{array}$$

(25)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}_i\left(t\right)& =& [{\widehat{\mathit{\theta }}}_{1i}\left(t\right),{\widehat{\mathit{\theta }}}_{2i}\left(t\right),\dots ,{\widehat{\mathit{\theta }}}_{ni}\left(t\right)],\hfill \end{array}$$

(26)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\gamma }}}_i\left(t\right)& =& {[{\widehat{\gamma }}_{i,1}\left(t\right),{\widehat{\gamma }}_{i,2}\left(t\right),\dots ,{\widehat{\gamma }}_{i,s}\left(t\right)]}^{\mathrm{T}},\hfill \end{array}$$

(27)

$$\begin{array}{ccc}\hfill \widehat{\mathit{\theta }}\left(t\right)& =& [{\widehat{\mathit{\theta }}}_1\left(t\right),{\widehat{\mathit{\theta }}}_2\left(t\right)],\hfill \end{array}$$

(28)

$$\begin{array}{ccc}\hfill \widehat{\mathit{\gamma }}\left(t\right)& =& {[{\widehat{\mathit{\gamma }}}_1^{\mathrm{T}}\left(t\right),{\widehat{\mathit{\gamma }}}_2^{\mathrm{T}}\left(t\right)]}^{\mathrm{T}}.\hfill \end{array}$$

(29)

The steps of computing the parameter estimation matrices $\widehat{\mathit{\theta }}\left(t\right)$, $\widehat{\mathit{\alpha }}\left(t\right)$ and $\widehat{\mathit{\gamma }}\left(t\right)$ by the HLS algorithm in (11)–(29) are summarised as follows.

• Set the initial values: let t = 1, ${\mathit{P}}_{\theta }\left(0\right)=p_0{\mathit{I}}_n$, ${\mathit{P}}_{\alpha }\left(0\right)=p_0{\mathit{I}}_n$, ${\mathit{P}}_{\gamma }\left(0\right)=p_0{\mathit{I}}_s$, $\widehat{\mathit{\theta }}\left(0\right)={\mathbf{1}}_{2\times 2n}/p_0$, $\widehat{\mathit{\alpha }}\left(0\right)={\mathbf{1}}_n/p_0$, $\widehat{\mathit{\gamma }}\left(0\right)={\mathbf{1}}_{2s}/p_0$, $p_0={10}^6$; $\mathit{u}\left(t\right)=\mathbf{0}$ and $\mathit{y}\left(t\right)=\mathbf{0}$ for $t\le 0$ and set a small positive number $\epsilon$.
• Collect the input and output data $\mathit{u}\left(t\right)$ and $\mathit{y}\left(t\right)$, and form ${\mathit{D}}_i\left(t\right)$ by (21), ${\widehat{\mathit{\varphi }}}_s\left(t\right)$ by (24).
• Compute ${\widehat{\mathit{D}}}_{\gamma }\left(t\right)$, ${\mathit{L}}_{\theta }\left(t\right)$ and ${\mathit{P}}_{\theta }\left(t\right)$ by (22), (12) and (13), and update the parameter estimate $\widehat{\mathit{\theta }}\left(t\right)$ by (11).
• Compute ${\widehat{\mathit{D}}}_{\theta }\left(t\right)$, ${\mathit{L}}_{\gamma }\left(t\right)$ and ${\mathit{P}}_{\gamma }\left(t\right)$ by (23), (18) and (19), and update the parameter estimate $\widehat{\mathit{\gamma }}\left(t\right)$ by (17).
• Compute ${\mathit{L}}_{\alpha }\left(t\right)$ and ${\mathit{P}}_{\alpha }\left(t\right)$ by (15) and (16), and update the parameter estimate $\widehat{\mathit{\alpha }}\left(t\right)$ by (14).
• Compute the noise term $\widehat{\mathit{v}}\left(t\right)$ using (25).
• Compare $\widehat{\mathit{\theta }}\left(t\right)$ with $\widehat{\mathit{\theta }}(t-1)$ and compare $\widehat{\mathit{\gamma }}\left(t\right)$ with $\widehat{\mathit{\gamma }}(t-1)$: if $\parallel \widehat{\mathit{\theta }}\left(t\right)-\widehat{\mathit{\theta }}(t-1)\parallel <\epsilon$ and $\parallel \widehat{\mathit{\gamma }}\left(t\right)-\widehat{\mathit{\gamma }}(t-1)\parallel <\epsilon$, terminate recursive calculation procedure and obtain $\widehat{\mathit{\theta }}\left(t\right)$, $\widehat{\mathit{\alpha }}\left(t\right)$ and $\widehat{\mathit{\gamma }}\left(t\right)$; otherwise, increase t by 1 and go to step 2.

The flowchart of computing the parameter estimates $\widehat{\mathit{\theta }}\left(t\right)$, $\widehat{\mathit{\alpha }}\left(t\right)$ and $\widehat{\mathit{\gamma }}\left(t\right)$ in the HLS algorithm is shown in Figure 2.

4. The Convergence Analysis of the Hierarchical Least-Squares Algorithm

The convergence analysis of the proposed hierarchical identification principle-based least-squares algorithm for the TITO Hammerstein system is illustrated as follows. Assume the estimated information vectors ${\widehat{\mathit{D}}}_{\gamma }\left(t\right)$, ${\widehat{\mathit{D}}}_{\theta }\left(t\right)$, and ${\widehat{\mathit{\varphi }}}_s\left(t\right)$ are persistently exciting, i.e., there exist constants $M_i>0$ and an integer N such that for $t>N$, the following strong persistent excitation conditions hold:

$$\begin{array}{c}\left(A1\right){\mathbf{M}}_{\mathbf{1}}\mathbf{I}\le \frac{1}{N}{\displaystyle \sum _{j=0}^{N-1}}{\widehat{\mathit{D}}}_{\gamma }\left(t\right){\widehat{\mathit{D}}}_{\gamma }^{\mathrm{T}}\left(t\right)\le {\mathbf{M}}_{\mathbf{2}}\mathbf{I},a.s.,\\ \left(A2\right){\mathbf{M}}_{\mathbf{3}}\mathbf{I}\le \frac{1}{N}{\displaystyle \sum _{j=0}^{N-1}}{\widehat{\mathit{D}}}_{\theta }\left(t\right){\widehat{\mathit{D}}}_{\theta }^{\mathrm{T}}\left(t\right)\le {\mathbf{M}}_{\mathbf{4}}\mathbf{I},a.s.,\\ \left(A3\right){\mathbf{M}}_{\mathbf{5}}\mathbf{I}\le \frac{1}{N}{\displaystyle \sum _{j=0}^{N-1}}{\widehat{\mathit{\varphi }}}_{\mathit{s}}\left(t\right){\widehat{\mathit{\varphi }}}_{\mathit{s}}^{\mathrm{T}}\left(t\right)\le {\mathbf{M}}_{\mathbf{6}}\mathbf{I},a.s.\end{array}$$

(30)

Theorem 1.

For the the hierarchical least-squares algorithm in (11)–(29), suppose that (A1)–(A3) hold. Additionally, assume that the σ algebra sequence ${ϝ}_t=\sigma (v\left(t\right),v(t-1),v(t-2),\cdots )$, which is generated by $v\left(t\right)$, and $\{v\left(t\right),{ϝ}_t\}$ is a martingale difference sequence on a probability space $\left(\Omega ,ϝ,P\right)$ [24]. The sequence $v\left(t\right)$ satisfies

$$\begin{array}{c}\left(B1\right)E\left[v\left(t\right)\right|{ϝ}_{t-1}]=0,a.s.,\hfill \\ {\left(B2\right)E\left[\right|\left|v\left(t\right)\right||}^2\left|{ϝ}_{t-1}\right]={\sigma }^2\left(t\right)\le {\sigma }^2<\infty ,a.s.,\hfill \\ \left(B3\right)lim\underset{t\to \infty }{sup}\frac{1}{t}{\displaystyle \sum _{i=1}^t}{v}^2\left(i\right)\le {\sigma }^2<\infty ,a.s.\hfill \end{array}$$

(31)

Then, the parameter estimation vector ${\widehat{\mathit{\theta }}}_i\left(t\right)$, $\widehat{\mathit{\gamma }}\left(t\right)$, and $\widehat{\mathit{\theta }}\left(t\right)$ consistently converges to the true parameter vector ${\mathit{\theta }}_i$, γ and θ.

5. The Filtering Based Recursive Least-Squares Algorithm

This section proposes the filtering-based recursive least-squares algorithm for the two-input two-output system with MA noise by using the data filtering technique. The input–output data are filtered through a rational polynomial $\alpha \left(z\right)$. Multiplying both sides of Equation (8) by $\frac{1}{\alpha \left(z\right)}$ yields

$$\begin{array}{ccc}\hfill \frac{1}{\alpha \left(z\right)}\mathit{y}\left(t\right)& =& \frac{{\mathit{\theta }}_1{\mathit{D}}_1\left(t\right){\mathit{\gamma }}_1}{\alpha \left(z\right)}+\frac{{\mathit{\theta }}_2{\mathit{D}}_2\left(t\right){\mathit{\gamma }}_2}{\alpha \left(z\right)}+\mathit{v}\left(t\right).\hfill \end{array}$$

(32)

Define the filtered input and the filtered output as

$$\begin{array}{ccc}\hfill {\mathit{y}}_f\left(t\right)& :=& \frac{1}{\alpha \left(z\right)}\mathit{y}\left(t\right)=\mathit{y}\left(t\right)+[1-\alpha \left(z\right)]{\mathit{y}}_f\left(t\right),\hfill \end{array}$$

(33)

$$\begin{array}{ccc}\hfill {\mathit{D}}_{f,i}\left(t\right)& :=& \frac{1}{\alpha \left(z\right)}{\mathit{D}}_i\left(t\right)={\mathit{D}}_i\left(t\right)+[1-\alpha \left(z\right)]{\mathit{D}}_{f,i}\left(t\right).\hfill \end{array}$$

(34)

Then, Equation (32) can be rewritten as

$$\begin{array}{ccc}\hfill {\mathit{y}}_f\left(t\right)& =& {\mathit{\theta }}_1{\mathit{D}}_{f,1}\left(t\right){\mathit{\gamma }}_1+{\mathit{\theta }}_2{\mathit{D}}_{f,2}\left(t\right){\mathit{\gamma }}_2+\mathit{v}\left(t\right).\hfill \end{array}$$

(35)

Substituted (33) into (35), we can have

$$\begin{array}{ccc}\hfill \mathit{y}\left(t\right)& =& {\mathit{\theta }}_1{\mathit{D}}_{f,1}\left(t\right){\mathit{\gamma }}_1+{\mathit{\theta }}_2{\mathit{D}}_{f,2}\left(t\right){\mathit{\gamma }}_2+[\alpha \left(z\right)-1]{\mathit{y}}_f\left(t\right)+\mathit{v}\left(t\right).\hfill \end{array}$$

(36)

Define

$$\begin{array}{ccc}\hfill {\mathit{D}}_{f,\gamma }\left(t\right)& :=& \left[\begin{array}{c}{\mathit{D}}_{f,1}\left(t\right){\mathit{\gamma }}_1\\ {\mathit{D}}_{f,2}\left(t\right){\mathit{\gamma }}_2\end{array}\right]\in {ℝ}^{2n},{\mathit{D}}_{f,\theta }\left(t\right):=[{\mathit{\theta }}_1{\mathit{D}}_{f,1}\left(t\right),{\mathit{\theta }}_2{\mathit{D}}_{f,2}\left(t\right)]\in {ℝ}^{2\times 2s},\hfill \\ \hfill {\mathit{\varphi }}_f\left(t\right)& :=& {[{\mathit{y}}_f(t-1),{\mathit{y}}_f(t-2),\dots ,{\mathit{y}}_f(t-n)]}^{\mathrm{T}}\in {ℝ}^{n\times 2}.\hfill \end{array}$$

Then, Equation (36) can be transformed into the following two forms:

$$\begin{array}{ccc}\hfill S_{f,1}:\mathit{y}\left(t\right)& =& \mathit{\theta }{\mathit{D}}_{f,\gamma }\left(t\right)+{\mathit{\varphi }}_f^{\mathrm{T}}\left(t\right)\mathit{\alpha }+\mathit{v}\left(t\right),\hfill \end{array}$$

(37)

$$\begin{array}{ccc}\hfill S_{f,2}:\mathit{y}\left(t\right)& =& {\mathit{D}}_{f,\theta }\left(t\right)\mathit{\gamma }+{\mathit{\varphi }}_f^{\mathrm{T}}\left(t\right)\mathit{\alpha }+\mathit{v}\left(t\right).\hfill \end{array}$$

(38)

The parameter matrices $\mathit{\theta }$, $\mathit{\alpha }$ and $\mathit{\gamma }$ to be identified are included in these two new models. Then, define two cost functions about the parameter matrices $\mathit{\theta }$, $\mathit{\alpha }$ and $\mathit{\gamma }$:

$$\begin{array}{ccc}\hfill J_3(\mathit{\theta },\mathit{\alpha })& :=& \sum _{l=1}^t{[\mathit{y}\left(l\right)-\mathit{\theta }{\mathit{D}}_{f,\gamma }\left(l\right)-{\mathit{\varphi }}_f^{\mathrm{T}}\left(l\right)\mathit{\alpha }]}^2,\hfill \\ \hfill J_4\left(\mathit{\gamma }\right)& :=& \sum _{l=1}^t{[\mathit{y}\left(l\right)-{\mathit{D}}_{f,\theta }\left(l\right)\mathit{\gamma }-{\mathit{\varphi }}_f^{\mathrm{T}}\left(l\right)\mathit{\alpha }]}^2.\hfill \end{array}$$

There are several difficulties in minimizing the two cost functions to obtain the parameter estimates. Since the information matrix ${\mathit{D}}_{f,\gamma }\left(t\right)$ contains the unknown parameter matrix ${\mathit{\theta }}_i$ and ${\mathit{D}}_{f,i}\left(t\right)$, the information vector ${\mathit{D}}_{f,\theta }\left(t\right)$ contains the unknown parameter vector ${\mathit{\gamma }}_i$ and ${\mathit{D}}_{f,i}\left(t\right)$, the information matrix ${\mathit{\varphi }}_f\left(t\right)$ contains the unmeasurable noise term ${\mathit{y}}_f(t-i)$. The problem is solved by replacing the unknown parameter matrix ${\mathit{\theta }}_i$ with its estimate ${\widehat{\mathit{\theta }}}_i\left(t\right)$, replacing the unknown parameter vector ${\mathit{\gamma }}_i$ with its estimate ${\widehat{\mathit{\gamma }}}_i\left(t\right)$, replacing the unmeasurable ${\mathit{D}}_{f,i}\left(t\right)$ and ${\mathit{y}}_f(t-i)$ with their estimate ${\widehat{\mathit{D}}}_{f,i}\left(t\right)$ and ${\widehat{\mathit{y}}}_f(t-i)$.

The estimate of ${\mathit{D}}_{f,i}\left(t\right)$ can be computed by

$$\begin{array}{ccc}\hfill {\widehat{\mathit{D}}}_{f,i}\left(t\right)& :=& {\mathit{D}}_i\left(t\right)-\sum _{i=1}^2{\alpha }_i{\widehat{\mathit{D}}}_{f,i}(t-i).\hfill \end{array}$$

Then, the estimate of ${\mathit{\varphi }}_f\left(t\right)$, ${\mathit{D}}_{f,\gamma }\left(t\right)$ and ${\mathit{D}}_{f,\theta }\left(t\right)$ can be written as

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\varphi }}}_f\left(t\right)& :=& {[{\widehat{\mathit{y}}}_f(t-1),{\widehat{\mathit{y}}}_f(t-2),\dots ,{\widehat{\mathit{y}}}_f(t-n)]}^{\mathrm{T}},\hfill \\ \hfill {\widehat{\mathit{D}}}_{f,\gamma }\left(t\right)& :=& \left[\begin{array}{c}{\widehat{\mathit{D}}}_{f,1}\left(t\right){\widehat{\mathit{\gamma }}}_1(t-1)\\ {\widehat{\mathit{D}}}_{f,2}\left(t\right){\widehat{\mathit{\gamma }}}_2(t-1)\end{array}\right],\hfill \\ \hfill {\widehat{\mathit{D}}}_{f,\theta }\left(t\right)& :=& [{\widehat{\mathit{\theta }}}_1\left(t\right){\widehat{\mathit{D}}}_{f,1}\left(t\right),{\widehat{\mathit{\theta }}}_2\left(t\right){\widehat{\mathit{D}}}_{f,2}\left(t\right)].\hfill \end{array}$$

Then, the estimate ${\widehat{\mathit{y}}}_f\left(t\right)$ can be computed by

$$\begin{array}{c}\hfill {\widehat{\mathit{y}}}_f\left(t\right)=\mathit{y}\left(t\right)-{\widehat{\mathit{\varphi }}}_f^{\mathrm{T}}\left(t\right)\widehat{\mathit{\alpha }}\left(t\right).\end{array}$$

Then, we obtain the filtering-based hierarchical least-squares (F–HLS) parameter estimation algorithm for the TITO Hammerstein FIR-MA models as follows:

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}^{\mathrm{T}}\left(t\right)& =& {\widehat{\mathit{\theta }}}^{\mathrm{T}}(t-1)+{\mathit{L}}_{\theta }\left(t\right){[\mathit{y}\left(t\right)-\widehat{\mathit{\theta }}(t-1){\widehat{\mathit{D}}}_{f,\gamma }\left(t\right)-{\widehat{\mathit{\varphi }}}_f^{\mathrm{T}}\left(t\right)\widehat{\mathit{\alpha }}(t-1)]}^{\mathrm{T}},\hfill \end{array}$$

(39)

$$\begin{array}{ccc}\hfill {\mathit{L}}_{\theta }\left(t\right)& =& {\mathit{P}}_{\theta }(t-1){\widehat{\mathit{D}}}_{f,\gamma }\left(t\right){[1+{\widehat{\mathit{D}}}_{f,\gamma }^{\mathrm{T}}\left(t\right){\mathit{P}}_{\theta }(t-1){\widehat{\mathit{D}}}_{f,\gamma }\left(t\right)]}^{-1},\hfill \end{array}$$

(40)

$$\begin{array}{ccc}\hfill {\mathit{P}}_{\theta }\left(t\right)& =& [{\mathit{I}}_n-{\mathit{L}}_{\theta }\left(t\right){\widehat{\mathit{D}}}_{f,\gamma }^{\mathrm{T}}\left(t\right)]{\mathit{P}}_{\theta }(t-1),{\mathit{P}}_{\theta }\left(0\right)=p_0{\mathit{I}}_n,\hfill \end{array}$$

(41)

$$\begin{array}{ccc}\hfill \widehat{\mathit{\alpha }}\left(t\right)& =& \widehat{\mathit{\alpha }}(t-1)+{\mathit{L}}_{\alpha }\left(t\right)[\mathit{y}\left(t\right)-\widehat{\mathit{\theta }}\left(t\right){\widehat{\mathit{D}}}_{f,\gamma }\left(t\right)-{\widehat{\mathit{\varphi }}}_f^{\mathrm{T}}\left(t\right)\widehat{\mathit{\alpha }}(t-1)],\hfill \end{array}$$

(42)

$$\begin{array}{ccc}\hfill {\mathit{L}}_{\alpha }\left(t\right)& =& {\mathit{P}}_{\alpha }(t-1){\widehat{\mathit{\varphi }}}_s^{\mathrm{T}}\left(t\right){[{\mathit{I}}_n+{\widehat{\mathit{\varphi }}}_s\left(t\right){\mathit{P}}_{\alpha }(t-1){\widehat{\mathit{\varphi }}}_f^{\mathrm{T}}\left(t\right)]}^{-1}\hfill \end{array}$$

(43)

$$\begin{array}{ccc}\hfill {\mathit{P}}_{\alpha }\left(t\right)& =& [{\mathit{I}}_n-{\mathit{L}}_{\alpha }\left(t\right){\widehat{\mathit{\varphi }}}_f^{\mathrm{T}}\left(t\right)]{\mathit{P}}_{\alpha }(t-1),{\mathit{P}}_{\alpha }\left(0\right)=p_0{\mathit{I}}_n,\hfill \end{array}$$

(44)

$$\begin{array}{ccc}\hfill \widehat{\mathit{\gamma }}\left(t\right)& =& \widehat{\mathit{\gamma }}(t-1)+{\mathit{L}}_{\gamma }\left(t\right)[\mathit{y}\left(t\right)-{\widehat{\mathit{D}}}_{f,\theta }\left(t\right)\widehat{\mathit{\gamma }}(t-1)-{\widehat{\mathit{\varphi }}}_s^{\mathrm{T}}\left(t\right)\widehat{\mathit{\alpha }}(t-1)],\hfill \end{array}$$

(45)

$$\begin{array}{ccc}\hfill {\mathit{L}}_{\gamma }\left(t\right)& =& {\mathit{P}}_{\gamma }(t-1){\widehat{\mathit{D}}}_{f,\theta }^{\mathrm{T}}\left(t\right){[{\mathit{I}}_s+{\widehat{\mathit{D}}}_{f,\theta }\left(t\right){\mathit{P}}_{\gamma }(t-1){\mathit{D}}_{f,\theta }^{\mathrm{T}}\left(t\right)]}^{-1},\hfill \end{array}$$

(46)

$$\begin{array}{ccc}\hfill {\mathit{P}}_{\gamma }\left(t\right)& =& [{\mathit{I}}_s-{\mathit{L}}_{\gamma }\left(t\right){\widehat{\mathit{D}}}_{f,\theta }\left(t\right)]{\mathit{P}}_{\gamma }(t-1),{\mathit{P}}_{\gamma }\left(0\right)=p_0{\mathit{I}}_s,\hfill \end{array}$$

(47)

$$\begin{array}{ccc}\hfill {\mathit{d}}_i\left(t\right)& =& [d_{i,1}\left[u_i\left(t\right)\right],d_{i,2}\left[u_i\left(t\right)\right],\dots ,d_{i,s}\left[u_i\left(t\right)\right]]\in {ℝ}^{1\times s},i=1,2,\hfill \end{array}$$

(48)

$$\begin{array}{ccc}\hfill {\mathit{D}}_i\left(t\right)& =& \left[\begin{array}{ccccccc}{d}_{i,1}\left[u_i(t-1)\right]& & d_{i,2}\left[u_i(t-1)\right]& & \cdots & & d_{i,s}\left[u_i(t-1)\right]\\ d_{i,1}\left[u_i(t-2)\right]& & d_{i,2}\left[u_i(t-2)\right]& & \cdots & & d_{i,s}\left[u_i(t-2)\right]\\ ⋮& & ⋮& & & & ⋮\\ d_{i,1}\left[u_i(t-n)\right]& & d_{i,2}\left[u_i(t-n)\right]& & \cdots & & d_{i,s}\left[u_i(t-n)\right]\end{array}\right],\hfill \end{array}$$

(49)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{D}}}_{f,i}\left(t\right)& :=& {\mathit{D}}_i\left(t\right)-\sum _{i=1}^2{\alpha }_i{\widehat{\mathit{D}}}_{f,i}(t-i),\hfill \end{array}$$

(50)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{D}}}_{f,\gamma }\left(t\right)& :=& \left[\begin{array}{c}{\widehat{\mathit{D}}}_{f,1}\left(t\right){\widehat{\mathit{\gamma }}}_1(t-1)\\ {\widehat{\mathit{D}}}_{f,2}\left(t\right){\widehat{\mathit{\gamma }}}_2(t-1)\end{array}\right],\hfill \end{array}$$

(51)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{D}}}_{f,\theta }\left(t\right)& :=& [{\widehat{\mathit{\theta }}}_1\left(t\right){\widehat{\mathit{D}}}_{f,1}\left(t\right),{\widehat{\mathit{\theta }}}_2\left(t\right){\widehat{\mathit{D}}}_{f,2}\left(t\right)].\hfill \end{array}$$

(52)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\varphi }}}_f\left(t\right)& :=& {[{\widehat{\mathit{y}}}_f(t-1),{\widehat{\mathit{y}}}_f(t-2),\dots ,{\widehat{\mathit{y}}}_f(t-n)]}^{\mathrm{T}},\hfill \end{array}$$

(53)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{y}}}_f\left(t\right)& =& \mathit{y}\left(t\right)-{\widehat{\mathit{\varphi }}}_f^{\mathrm{T}}\left(t\right)\widehat{\mathit{\alpha }}\left(t\right),\hfill \end{array}$$

(54)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}_i\left(t\right)& =& [{\widehat{\mathit{\theta }}}_{1i}\left(t\right),{\widehat{\mathit{\theta }}}_{2i}\left(t\right),\dots ,{\widehat{\mathit{\theta }}}_{ni}\left(t\right)],\hfill \end{array}$$

(55)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\gamma }}}_i\left(t\right)& =& {[{\widehat{\gamma }}_{i,1}\left(t\right),{\widehat{\gamma }}_{i,2}\left(t\right),\dots ,{\widehat{\gamma }}_{i,s}\left(t\right)]}^{\mathrm{T}},\hfill \end{array}$$

(56)

$$\begin{array}{ccc}\hfill \widehat{\mathit{\theta }}\left(t\right)& =& [{\widehat{\mathit{\theta }}}_1\left(t\right),{\widehat{\mathit{\theta }}}_2\left(t\right)],\hfill \end{array}$$

(57)

$$\begin{array}{ccc}\hfill \widehat{\mathit{\gamma }}\left(t\right)& =& {[{\widehat{\mathit{\gamma }}}_1^{\mathrm{T}}\left(t\right),{\widehat{\mathit{\gamma }}}_2^{\mathrm{T}}\left(t\right)]}^{\mathrm{T}}.\hfill \end{array}$$

(58)

The steps of computing the parameter estimation matrices $\widehat{\mathit{\theta }}\left(t\right)$, $\widehat{\mathit{\alpha }}\left(t\right)$ and $\widehat{\mathit{\gamma }}\left(t\right)$ by the F–HLS algorithm in (39)–(58) are summarised as follows.

• Set the initial values: let t = 1, ${\mathit{P}}_{\theta }\left(0\right)=p_0{\mathit{I}}_n$, ${\mathit{P}}_{\alpha }\left(0\right)=p_0{\mathit{I}}_n$, ${\mathit{P}}_{\gamma }\left(0\right)=p_0{\mathit{I}}_s$, $\widehat{\mathit{\theta }}\left(0\right)={\mathbf{1}}_{2\times 2n}/p_0$, $\widehat{\mathit{\alpha }}\left(0\right)={\mathbf{1}}_n/p_0$, $\widehat{\mathit{\gamma }}\left(0\right)={\mathbf{1}}_{2s}/p_0$, ${\widehat{\mathit{y}}}_f(t-i)={\mathbf{1}}_n/p_0$, $p_0={10}^6$; $\mathit{u}\left(t\right)=\mathbf{0}$ and $\mathit{y}\left(t\right)=\mathbf{0}$ for $t\le 0$ and set a small positive number $\epsilon$.
• Collect the input and output data $\mathit{u}\left(t\right)$ and $\mathit{y}\left(t\right)$, form ${\mathit{D}}_i\left(t\right)$ by (49), ${\widehat{\mathit{\varphi }}}_f\left(t\right)$ by (53), and compute ${\widehat{\mathit{D}}}_{f,i}\left(t\right)$ by (50).
• Compute ${\widehat{\mathit{D}}}_{f,\gamma }\left(t\right)$, ${\mathit{L}}_{\theta }\left(t\right)$ and ${\mathit{P}}_{\theta }\left(t\right)$ by (51), (40) and (41), and update the parameter estimate $\widehat{\mathit{\theta }}\left(t\right)$ by (39).
• Compute ${\widehat{\mathit{D}}}_{f,\theta }\left(t\right)$, ${\mathit{L}}_{\gamma }\left(t\right)$ and ${\mathit{P}}_{\gamma }\left(t\right)$ by (52), (46) and (47), and update the parameter estimate $\widehat{\mathit{\gamma }}\left(t\right)$ by (45).
• Compute ${\mathit{L}}_{\alpha }\left(t\right)$ and ${\mathit{P}}_{\alpha }\left(t\right)$ by (43) and (44), and update the parameter estimate $\widehat{\mathit{\alpha }}\left(t\right)$ by (42).
• Compute the noise term $\widehat{\mathit{v}}\left(t\right)$ using (54).
• Compare $\widehat{\mathit{\theta }}\left(t\right)$ with $\widehat{\mathit{\theta }}(t-1)$ and compare $\widehat{\mathit{\gamma }}\left(t\right)$ with $\widehat{\mathit{\gamma }}(t-1)$: if $\parallel \widehat{\mathit{\theta }}\left(t\right)-\widehat{\mathit{\theta }}(t-1)\parallel <\epsilon$ and $\parallel \widehat{\mathit{\gamma }}\left(t\right)-\widehat{\mathit{\gamma }}(t-1)\parallel <\epsilon$, terminate recursive calculation procedure and obtain $\widehat{\mathit{\theta }}\left(t\right)$, $\widehat{\mathit{\alpha }}\left(t\right)$ and $\widehat{\mathit{\gamma }}\left(t\right)$; otherwise, increase t by 1 and go to step 2.

6. Examples

Consider the following TITO Hammerstein FIR–MA system:

$$\begin{array}{ccc}\hfill \mathit{y}\left(t\right)& =& \mathit{\theta }\left(z\right)\overline{\mathit{u}}\left(t\right)+\alpha \left(z\right)\mathit{v}\left(t\right),\hfill \\ \hfill \overline{\mathit{u}}\left(t\right)& =& \left[\begin{array}{c}{\overline{u}}_1\left(t\right)\\ {\overline{u}}_2\left(t\right)\end{array}\right]=\left[\begin{array}{c}-0.35u_1\left(t\right)-0.40u_1^2\left(t\right)\\ 0.14u_2\left(t\right)-0.34u_2^2\left(t\right)\end{array}\right],\hfill \\ \hfill \mathit{y}\left(t\right)& =& \left[\begin{array}{c}{y}_1\left(t\right)\\ y_2\left(t\right)\end{array}\right],\mathit{v}\left(t\right)=\left[\begin{array}{c}{v}_1\left(t\right)\\ v_2\left(t\right)\end{array}\right],\hfill \\ \hfill \alpha \left(z\right)& =& 1+1.00z^{-1}+0.11z^{-2},\hfill \\ \hfill {\mathit{\theta }}_1\left(z\right)& =& \left[\begin{array}{c}-0.28\\ 0.06\end{array}\right]z^{-1}+\left[\begin{array}{c}-0.34\\ -0.27\end{array}\right]z^{-2},\hfill \\ \hfill {\mathit{\theta }}_2\left(z\right)& =& \left[\begin{array}{c}1.10\\ 1.40\end{array}\right]z^{-1}+\left[\begin{array}{c}0.41\\ 0.54\end{array}\right]z^{-2},\hfill \\ \hfill {\mathit{\theta }}_{1,1}& =& \left[\begin{array}{c}-0.28\\ 0.06\end{array}\right],{\mathit{\theta }}_{1,2}=\left[\begin{array}{c}-0.34\\ -0.27\end{array}\right],{\mathit{\theta }}_{2,1}=\left[\begin{array}{c}1.10\\ 1.40\end{array}\right],{\mathit{\theta }}_{2,2}=\left[\begin{array}{c}0.41\\ 0.54\end{array}\right],\hfill \\ \hfill {\mathit{\gamma }}_1& =& \left[\begin{array}{c}-0.35\\ -0.40\end{array}\right],{\mathit{\gamma }}_2=\left[\begin{array}{c}0.14\\ -0.34\end{array}\right],\mathit{\gamma }=\left[\begin{array}{c}{\mathit{\gamma }}_1\\ {\mathit{\gamma }}_2\end{array}\right],\mathit{\alpha }=\left[\begin{array}{c}1.00\\ 0.11\end{array}\right].\hfill \end{array}$$

In the simulation, the input $\overline{\mathit{u}}\left(t\right)$ was taken as an uncorrelated persistent excitation signal vector sequence with zero mean and unit variance, $\mathit{v}\left(t\right)$ was taken as a white noise sequence with zero mean and variance ${\sigma }^2={1.00}^2$ and we applied the HLS algorithm and the F–HLS algorithm to identify this example system. The parameter estimates and the errors are shown in

Table 1. The HLS estimates and their errors with ${\sigma }^2={1.00}^2$.

t	100	200	500	1000	2000	3000	True Values
${\theta }_{1,11}$	−0.50914	−0.30750	−0.19701	−0.22835	−0.29013	−0.29596	−0.28000
${\theta }_{1,12}$	0.97766	0.56617	0.51217	0.32163	0.14490	0.13167	0.06000
${\theta }_{1,21}$	0.11384	0.21689	0.03764	−0.11184	−0.19503	−0.25967	−0.34000
${\theta }_{1,22}$	−0.80709	−0.63473	−0.25479	−0.17204	−0.24346	−0.25086	−0.27000
${\theta }_{2,11}$	0.74403	0.81074	0.74457	0.81258	0.92845	0.91909	1.10000
${\theta }_{2,12}$	0.44263	1.10990	0.95009	1.12074	1.14401	1.21750	1.40000
${\theta }_{2,21}$	−0.12767	0.00039	0.27178	0.21795	0.28146	0.33896	0.41000
${\theta }_{2,22}$	0.28715	0.69936	0.51335	0.40011	0.43399	0.42026	0.54000
${\gamma }_{1,1}$	−0.52470	−0.44508	−0.39763	−0.37525	−0.37048	−0.39093	−0.35000
${\gamma }_{1,2}$	−0.20855	−0.21843	−0.24514	−0.25373	−0.27027	−0.27842	−0.40000
${\gamma }_{2,1}$	0.28346	0.34036	0.27891	0.26296	0.23975	0.23523	0.14000
${\gamma }_{2,2}$	−0.29712	−0.34020	−0.32389	−0.33012	−0.33797	−0.34912	−0.34000
${\alpha }_1$	0.14994	0.26180	0.45948	0.68975	0.90551	1.01653	1.00000
${\alpha }_2$	0.50161	0.40490	0.23834	0.11798	0.06374	0.06636	0.11000
$\delta (\%)$	83.88885	57.71077	44.53619	30.11516	18.88155	15.33941

,

Table 2. The HLS estimates and their errors with ${\sigma }^2={0.50}^2$.

t	100	200	500	1000	2000	3000	True Values
${\theta }_{1,11}$	−0.44432	−0.40006	−0.38992	−0.31198	−0.26693	−0.26510	−0.28000
${\theta }_{1,12}$	0.56573	−0.09028	0.00964	−0.06114	−0.06901	0.00239	0.06000
${\theta }_{1,21}$	−0.12045	−0.04354	−0.31996	−0.30311	−0.33949	−0.33982	−0.34000
${\theta }_{1,22}$	−0.22202	−0.11342	0.01576	−0.12291	−0.19827	−0.23634	−0.27000
${\theta }_{2,11}$	1.04323	1.00861	1.21022	1.08627	1.01177	1.03408	1.10000
${\theta }_{2,12}$	0.80616	1.08325	1.03559	1.20325	1.32250	1.35832	1.40000
${\theta }_{2,21}$	0.41692	0.42105	0.36272	0.43777	0.45083	0.41805	0.41000
${\theta }_{2,22}$	1.13348	0.87373	0.73620	0.53233	0.49740	0.50385	0.54000
${\gamma }_{1,1}$	−0.37277	−0.35597	−0.40370	−0.38316	−0.40989	−0.42941	−0.35000
${\gamma }_{1,2}$	−0.41897	−0.38966	−0.42279	−0.39275	−0.41758	−0.43019	−0.40000
${\gamma }_{2,1}$	−0.05560	0.01713	0.03196	0.08860	0.11363	0.12266	0.14000
${\gamma }_{2,2}$	−0.40705	−0.38088	−0.38310	−0.35594	−0.35451	−0.36378	−0.34000
${\alpha }_1$	0.72392	0.83949	1.00022	1.05937	1.10617	1.14589	1.00000
${\alpha }_2$	−0.20897	−0.13856	−0.02692	0.09362	0.15665	0.18479	0.11000
$\delta (\%)$	48.67846	29.82225	24.42703	12.72200	10.38521	9.39920

,

Table 3. The F-HLS estimates and their errors with ${\sigma }^2={1.00}^2$.

t	100	200	500	1000	2000	3000	True Values
${\theta }_{1,11}$	−0.46534	−0.47768	−0.40413	−0.32596	−0.27511	−0.27812	−0.28000
${\theta }_{1,12}$	0.54645	0.51758	0.32302	0.22073	0.12924	0.10861	0.06000
${\theta }_{1,21}$	0.34879	0.18063	−0.05504	−0.22830	−0.36081	−0.33090	−0.34000
${\theta }_{1,22}$	−1.13416	−1.18888	−0.91111	−0.53083	−0.36501	−0.39249	−0.27000
${\theta }_{2,11}$	1.37312	1.07871	0.98702	1.02387	1.06733	1.06065	1.10000
${\theta }_{2,12}$	1.20017	1.40717	1.39712	1.26345	1.32548	1.35968	1.40000
${\theta }_{2,21}$	−0.50420	−0.17069	0.29524	0.40066	0.39513	0.47011	0.41000
${\theta }_{2,22}$	0.46208	0.26937	0.49625	0.60009	0.72538	0.68512	0.54000
${\gamma }_{1,1}$	−0.28981	−0.29722	−0.34828	−0.35355	−0.38810	−0.38610	−0.35000
${\gamma }_{1,2}$	−0.46484	−0.45777	−0.42673	−0.41237	−0.41568	−0.42091	−0.40000
${\gamma }_{2,1}$	0.19233	0.17823	0.17974	0.14425	0.14231	0.12582	0.14000
${\gamma }_{2,2}$	−0.18200	−0.18516	−0.18579	−0.19230	−0.22195	−0.23310	−0.34000
${\alpha }_1$	0.47710	0.58835	0.74277	0.88623	0.97788	1.02303	1.00000
${\alpha }_2$	−0.17863	−0.14081	−0.14153	−0.08345	0.01088	0.05732	0.11000
$\delta (\%)$	73.00597	61.84658	37.76548	19.80014	12.36094	10.78310

and

Table 4. The F-HLS estimates and their errors with ${\sigma }^2={0.50}^2$.

t	100	200	500	1000	2000	3000	True Values
${\theta }_{1,11}$	−0.37740	−0.41815	−0.37182	−0.31015	−0.25496	−0.26324	−0.28000
${\theta }_{1,12}$	0.51706	0.52888	0.34646	0.24585	0.14247	0.12024	0.06000
${\theta }_{1,21}$	0.21828	0.05468	−0.08284	−0.18580	−0.31081	−0.27801	−0.34000
${\theta }_{1,22}$	−0.88659	−0.96682	−0.73059	−0.42252	−0.26874	−0.30345	−0.27000
${\theta }_{2,11}$	1.50800	1.15388	1.02343	1.05184	1.08214	1.07516	1.10000
${\theta }_{2,12}$	1.27160	1.47267	1.42432	1.25340	1.32645	1.36304	1.40000
${\theta }_{2,21}$	−0.47274	−0.14330	0.25538	0.31426	0.32086	0.38596	0.41000
${\theta }_{2,22}$	0.34738	0.16458	0.40585	0.52964	0.61534	0.57783	0.54000
${\gamma }_{1,1}$	−0.34054	−0.33682	−0.37719	−0.37203	−0.40047	−0.39525	−0.35000
${\gamma }_{1,2}$	−0.45282	−0.44870	−0.41742	−0.40349	−0.40820	−0.41487	−0.40000
${\gamma }_{2,1}$	0.21347	0.20244	0.21104	0.17728	0.17622	0.15573	0.14000
${\gamma }_{2,2}$	−0.22228	−0.22713	−0.22743	−0.22955	−0.26159	−0.27247	−0.34000
${\alpha }_1$	0.52496	0.64401	0.80149	0.91977	1.00864	1.05000	1.00000
${\alpha }_2$	−0.13503	−0.10317	−0.11367	−0.03595	0.05006	0.09420	0.11000
$\delta (\%)$	64.49938	53.52304	31.39408	17.18469	8.81068	6.52457

. The parameter estimation errors: $\delta :=\sqrt{\frac{\parallel \widehat{\mathit{\theta }}{\left(t\right)-\mathit{\theta }\parallel }^2+{\parallel \widehat{\mathit{\gamma }}\left(t\right)-\mathit{\gamma }\parallel }^2}{{\parallel \mathit{\theta }\parallel }^2+{\parallel \mathit{\gamma }\parallel }^2}}$ versus t are shown in Figure 3 and Figure 4.

From Table 1, Table 2, Table 3 and Table 4 and Figure 3, Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8, we can draw the following conclusions.

• The estimation errors given by the HLS algorithm and the F–HLS algorithm become generally smaller and smaller as t increases.
• Compared with the HLS algorithm, the F–HLS algorithm has higher parameter estimation accuracy.
• The predictions of the residuals are close to the true residuals, and the estimated outputs are close to the true outputs.

7. Conclusions

We derived a hierarchical least-squares algorithm and a filtering-based hierarchical least-squares algorithm for the two-input two-output Hammerstein finite impulse response moving average systems based on the hierarchical identification principle and the data filtering theory. The proposed F-HLS algorithm can effectively identify the parameters of the TITO Hammerstein FIR–MA systems and has a higher parameter estimation accuracy compared with the HLS algorithm. The proposed model parameter estimation methods in the paper can combine some mathematical strategiesand other estimation algorithms to study the parameter identification problems of linear and nonlinear systems with different disturbances, and can be applied to other fields, such as signal processing and engineering application systems.