Variational Bayesian Iterative Estimation Algorithm for Linear Difference Equation Systems

Abstract

Many basic laws of physics or chemistry can be written in the form of differential equations. With the development of digital signals and computer technology, the research on discrete models has received more and more attention. The estimates of the unknown coefficients in the discretized difference equation can be obtained by optimizing certain criterion functions. In modern control theory, the state-space model transforms high-order differential equations into first-order differential equations by introducing intermediate state variables. In this paper, the parameter estimation problem for linear difference equation systems with uncertain noise is developed. By transforming system equations into state-space models and on the basis of the considered priors of the noise and parameters, a variational Bayesian iterative estimation algorithm is derived from the observation data to obtain the parameter estimates. The unknown states involved in the variational Bayesian algorithm are updated by the Kalman filter. A numerical simulation example is given to validate the effectiveness of the proposed algorithm.

1. Introduction

Differential equations arise in many instances when using the mathematical models to describe phenomena in economics, engineering, biology, etc. [1,2]. Considering the following nth-order linear differential equation with zero initial conditions

$$\begin{array}{ccc}{y}_k^{\left(n\right)}\hfill & & +{\alpha }_1{y}_k^{(n-1)}+{\alpha }_2{y}_k^{(n-2)}+\cdots +{\alpha }_{n-1}{y}_k^{\left(1\right)}+{\alpha }_n{y}_k\hfill \\ & & ={\beta }_1{u}_k^{(n-1)}+{\beta }_2{u}_k^{(n-2)}+\cdots +{\beta }_{n-1}{u}^{\left(1\right)}+{\beta }_n{u}_k,\hfill \end{array}$$

(1)

where $y_k$ is the system output, $u_k$ is the system input, k is the time variable, ${\alpha }_i$ and ${\beta }_i$ are the unknown real parameters. Each derivative symbol is defined as

$$\begin{array}{c}\hfill y_k^{\left(i\right)}=\frac{{\mathrm{d}}^i{y}_k}{{\mathrm{d}}^{i}k},y_k^{\left(2\right)}=\frac{{\mathrm{d}}^2{y}_k}{{\mathrm{d}}^{2}k}={\ddot{y}}_k,y_k^{\left(1\right)}=\frac{y_k}{k}={\dot{y}}_k,\end{array}$$

$$\begin{array}{c}\hfill u_k^{\left(i\right)}=\frac{{\mathrm{d}}^i{u}_k}{{\mathrm{d}}^{i}k},u_k^{\left(2\right)}=\frac{{\mathrm{d}}^2{u}_k}{{\mathrm{d}}^{2}k}={\ddot{u}}_k,u_k^{\left(1\right)}=\frac{u_k}{k}={\dot{u}}_k.\end{array}$$

Although the laws of the systems are continuously evolved over time, the sampled values of the systems are always discretized. In order to obtain the estimates of the unknown parameters in the system mathematical models, there are usually two approaches to further process the differential equation models. One is called the frequency-domain analysis methods, and the other is called the time-domain analysis methods [3,4,5,6].

For linear continuous systems, Laplace transform of Equation (1) under zero initial conditions is given by

$$\begin{array}{c}\hfill (s^{n-1}+{\alpha }_1{s}^{n-1}+{\alpha }_2{s}^{n-2}+\cdots +{\alpha }_{n-1}s+{\alpha }_n)Y\left(s\right)=({\beta }_1{s}^{n-1}+{\beta }_2{s}^{n-2}+\cdots +{\beta }_{n-1}s+{\beta }_n)U\left(s\right),\end{array}$$

where the variable s is the laplace operator, $Y\left(s\right)$ and $U\left(s\right)$ are the Laplace transform of $y_k$ and $u_k$: $Y\left(s\right):=\mathcal{L}\left[y_k\right]$, $U\left(s\right):=\mathcal{L}\left[u_k\right]$. Then the transfer function of the system is written as

$$\begin{array}{c}\hfill G\left(s\right):=\frac{Y\left(s\right)}{U\left(s\right)}=\frac{{\beta }_1{s}^{n-1}+{\beta }_2{s}^{n-2}+\cdots +{\beta }_{n-1}s+{\beta }_n}{s^{n-1}+{\alpha }_1{s}^{n-1}+{\alpha }_2{s}^{n-2}+\cdots +{\alpha }_{n-1}s+{\alpha }_n}.\end{array}$$

So the output equation can be further expressed as

$$\begin{array}{c}\hfill Y\left(s\right)=G\left(s\right)U\left(s\right).\end{array}$$

The system will be excited by designing different input signals to obtain different output responses. The parameters of the system are further obtained according to the analysis of the output responses [7,8,9]. By using the Laplace transform, the transfer function is one of the most important mathematical models in control theory. Under the zero initial conditions, the Laplace transform of the system output divided by the Laplace transform of the system input is called the transfer function. However, the transfer function is only suitable for describing linear constant coefficient differential equation systems.

In the time-domain analysis method, the system is represented in discrete time and the sampling interval is assumed to be one time unit. In this case, the discrete-time operator equations or difference equations are necessary [10,11,12,13]. A typical controlled autoregressive difference equation is written as

$$\begin{array}{c}\hfill y_k+a_1{y}_{k-1}+a_2{y}_{k-2}+\cdots +a_n{y}_{k-n}=b_1{u}_{k-1}+b_2{u}_{k-2}+\cdots +b_n{u}_{k-n}.\end{array}$$

Then the output $y_k$ can be further written as a form of determining the next value for given previous observations:

$$\begin{array}{ccc}{y}_k\hfill & =\hfill & -a_1{y}_{k-1}-a_2{y}_{k-2}-\cdots -a_n{y}_{k-n}+b_1{u}_{k-1}+b_2{u}_{k-2}+\cdots +b_n{u}_{k-n}\hfill \\ & =\hfill & {\mathbf{\psi }}_k^{\mathrm{T}}\mathbf{\vartheta },\hfill \end{array}$$

(2)

where

$$\begin{array}{ccc}\hfill {\mathbf{\psi }}_k& :=& {[-y_{k-1},-y_{k-2},\dots ,-y_{k-n},u_{k-1},u_{k-2},\dots ,u_{k-n}]}^{\mathrm{T}}\in {\mathbb{R}}^{2n},\hfill \end{array}$$

$$\begin{array}{ccc}\hfill \mathbf{\vartheta }& :=& {[a_1,a_2,\dots ,a_n,b_1,b_2,\dots ,b_n]}^{\mathrm{T}}{\in }^{2n}.\hfill \end{array}$$

From Equation (2), it is easy to find that the calculation of $y_k$ from past data depends on the parameters in $\mathbf{\vartheta }$. Call the calculated value ${\widehat{y}}_{k|\vartheta }$ and write

$$\begin{array}{c}\hfill {\widehat{y}}_{k|\vartheta }={\mathbf{\psi }}_k^{\mathrm{T}}\mathbf{\vartheta }.\end{array}$$

The solution of obtaining the estimates of the parameters is to optimize a criterion function. The effective estimates make the prediction error $\epsilon (k,\vartheta )=y_k-{\widehat{y}}_{k|\vartheta }$ as small as possible. Given a batch of inputs and outputs over a time interval $1\le k\le N$:

$$\begin{array}{c}\hfill Z^N=[u_1,y_1,u_2,y_2,\dots ,u_N,y_N].\end{array}$$

The quadratic criterion function is defined as [14]:

$$\begin{array}{c}\hfill V_N(\mathbf{\vartheta },Z^N)=\frac{1}{N}\sum _{k=1}^N{[y_k-{\widehat{y}}_{k|\vartheta }]}^2.\end{array}$$

The process of computing the estimate ${\widehat{\mathbf{\vartheta }}}_N$ is the process of finding the solution of the following optimization problem:

$$\begin{array}{c}\hfill {\widehat{\mathbf{\vartheta }}}_N=\arg \underset{\vartheta }{\min }{V}_N(\mathbf{\vartheta },Z^N),\end{array}$$

where ‘$\arg \underset{\vartheta }{\min }$’ represents the value of the considered parameter vector when the objective function takes the minimum value. The classical least squares-based and gradient descent-based numerical methods have been applied to search the estimates of the parameters by minimizing the criterion function [15,16,17,18]. For example, Xu et al. developed a least squares-based iterative parameter estimation algorithm for dynamic systems based on the parameter decomposition by using the hierarchical identification principle [19].

However, the gradient and least squares-based iterative algorithm only obtain the point estimates of parameters based on the known noise variance. They minimize a cost function on the parameter vector and then obtain the estimate of the parameter vector. Different from the stochastic gradient and least squares algorithm, the variational Bayesian (VB) algorithm needs the priors of parameters and then iteratively updates the posterior expression of the parameters by maximizing the lower bound marginal likelihood function [20,21]. The VB algorithm can obtain the distributions of the unknown parameters rather than the point estimates. For example, Yang and Yin applied the VB inference to estimate the parameters of finite impulse response (FIR) systems with randomly missing output data [22].

In this paper, the difference equation is further transformed to the state-space model. In modern control theory, the state-space model is widely used to describe the dynamics of systems, because it can not only reflect the internal states of systems but also reveals the relationship between the intermediate states and the external input and output variables [23,24,25,26,27,28]. A general linear state-space model

$$\begin{array}{ccc}\hfill \left[\begin{array}{c}{x}_{1,k+1}\\ x_{2,k+1}\\ ⋮\\ x_{n,k+1}\end{array}\right]& =& \left[\begin{array}{cccc}{a}_{11}& a_{12}& \cdots & a_{1n}\\ a_{21}& a_{22}& \cdots & a_{2n}\\ ⋮& ⋮& ⋮& ⋮\\ a_{n1}& a_{n2}& \cdots & a_{nn}\end{array}\right]\left[\begin{array}{c}{x}_{1,k}\\ x_{2,k}\\ ⋮\\ x_{n,k}\end{array}\right]+\left[\begin{array}{c}{b}_1\\ b_2\\ ⋮\\ b_n\end{array}\right]u_k,\hfill \end{array}$$

$$\begin{array}{ccc}\hfill y_k& =& [c_1,c_2,\cdots ,c_n]\left[\begin{array}{c}{x}_{1,k}\\ x_{2,k}\\ ⋮\\ x_{n,k}\end{array}\right]+v_k,\hfill \end{array}$$

where $y_k$ is the system output, $u_k$ is the system input, $v_k$ is the random noise, and $x_{i,k}$ is the unmeasurable internal state, which establishes the link between system internal information and external measurements is presented. However, state estimation brings difficulty to the parameter estimation [29]. For linear Gaussian systems, the Kalman filter can achieve optimal estimation performance with a small amount of computational load, which is usually the preferred method of the state estimation for state-space models [30,31,32].

Based on the previous iterative identification schemes [33,34,35,36], this paper concerns a VB-based iterative estimation algorithm for difference equation models with unknown measurement noise. Different from the work studied in [37], which used an open-loop state observer to roughly calculate the system states without considering the effect of measurement noise in the procedure of states estimation, this paper considers the problem of state estimation by using the Kalman filter. The main contributions of this paper are as follows.

• By defining internal state variables, the difference equation model is rewritten in the form of the state-space model.
• A Kalman filter based VB (VB-KF) estimation algorithm is provided to estimate both the model states and parameters interactively. Based on the estimated states, the VB algorithm calculates the system parameters and the noise variance. Then the states are updated by means of the Kalman filter with new obtained parameters and noise variance.
• Different from the existing point parameter estimation methods, the proposed algorithm can give the distributions of the system parameters and noise variance to reflect the uncertainties of their estimates.

The remainder of this paper is organized as follows. Section 2 introduces the state-space model. Section 3 derives a VB iterative algorithm to estimate the unknown parameters of the systems by using the Kalman filter. The simulation is presented in Section 4 to verify the proposed method. Section 5 gives some conclusions.

2. Transformation of the Difference Equation

Reconsidering a general nth-order difference equation in Equation (2), in order to transform the difference equation to the state-space model, n state variables are needed. The choices of the state variables are not unique; they can be measurable or unmeasurable. Usually, the state variables which can help to convert the state equation into a standard form are selected as the intermediate states. Here, we introduce n state variables as follows:

$$\begin{array}{ccc}{x}_{1,k}\hfill & =\hfill & y_k,\hfill \\ x_{2,k}\hfill & =\hfill & y_{k+1}-b_1{u}_k,\hfill \\ x_{3,k}\hfill & =\hfill & y_{k+2}-b_1{u}_{k+1}-b_2{u}_k,\hfill \\ x_{4,k}\hfill & =\hfill & y_{k+3}-b_1{u}_{k+2}-b_2{u}_{k+1}-b_3{u}_k,\hfill \\ & ⋮\hfill \\ x_{n-1,k}\hfill & =\hfill & y_{k+n-2}-b_1{u}_{k+n-3}-b_2{u}_{k+n-4}-\cdots -b_{n-2}{u}_k,\hfill \\ x_{n,k}\hfill & =\hfill & -a_n{y}_{k-1}-a_{n-1}(y_k-b_1{u}_{k-1})-a_{n-2}(y_{k+1}-b_1{u}_k-b_2{u}_{k-1})+\cdots \hfill \\ & & -a_1(y_{k+n-2}-b_1{u}_{k+n-3}-b_2{u}_{k+n-4}-\cdots -b_{n-2}{u}_k-b_{n-1}{u}_{k-1})+b_n{u}_k.\hfill \end{array}$$

(3)

Further, replacing the time variable k with $k+1$ on both sides of Equation (3) gives the following equations:

$$\begin{array}{ccc}{x}_{1,k+1}\hfill & =\hfill & y_{k+1}=x_{2,k}+b_1{u}_k,\hfill \\ x_{2,k+1}\hfill & =\hfill & y_{k+2}-b_1{u}_{k+1}=x_{3,k}+b_2{u}_k,\hfill \\ & ⋮\hfill \\ x_{j,k+1}\hfill & =\hfill & y_{k+j}-b_1{u}_{k+j-1}-\cdots -b_{j-1}{u}_{k+1}=x_{j+1,k}+b_j{u}_k,\hfill \\ & ⋮\hfill \\ x_{n-1,k+1}\hfill & =\hfill & x_{n,k}+b_{n-1}{u}_k,\hfill \\ x_{n,k+1}\hfill & =\hfill & -a_n{x}_{1,k}-a_{n-1}{x}_{2,k}-\cdots -a_1{x}_{n,k}+b_n{u}_k.\hfill \end{array}$$

(4)

Define the state vector ${\mathit{x}}_k:={[x_{1,k},x_{2,k},\cdots ,x_{n,k}]}^{\mathrm{T}}{\in }^n$. Rewrite the state equation for each state variable in Equation (4) by matrix representation; the observability canonical form state-space model of the difference Equation (2) can be described as

$$\begin{array}{ccc}\hfill \left[\begin{array}{c}{x}_{1,k+1}\\ x_{2,k+1}\\ ⋮\\ x_{n-1,k+1}\\ x_{n,k+1}\end{array}\right]& =& \left[\begin{array}{ccccc}0& 1& 0& \cdots & 0\\ 0& 0& 1& \cdots & 0\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ 0& 0& 0& \cdots & 1\\ -a_n& -a_{n-1}& -a_{n-2}& \cdots & -a_1\end{array}\right]\left[\begin{array}{c}{x}_{1,k}\\ x_{2,k}\\ ⋮\\ x_{n-1,k}\\ x_{n,k}\end{array}\right]+\left[\begin{array}{c}{b}_1\\ b_2\\ ⋮\\ b_{n-1}\\ b_n\end{array}\right]u_k,\hfill \end{array}$$

(5)

$$\begin{array}{ccc}\hfill y_k& =& [100\cdots 0]\left[\begin{array}{c}{x}_{1,k}\\ x_{2,k}\\ ⋮\\ x_{n-1,k}\\ x_{n,k}\end{array}\right].\hfill \end{array}$$

(6)

Further, we rewrite (5) and (6) as

$$\begin{array}{ccc}\hfill {\mathit{x}}_{k+1}& =& \mathit{A}{\mathit{x}}_k+\mathit{B}{u}_k,\hfill \end{array}$$

(7)

$$\begin{array}{ccc}\hfill y_k& =& \mathit{C}{\mathit{x}}_k,\hfill \end{array}$$

(8)

where

$$\begin{array}{ccc}\hfill \mathit{A}& =& \left[\begin{array}{ccccc}0& 1& 0& \cdots & 0\\ 0& 0& 1& \cdots & 0\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ 0& 0& 0& \cdots & 1\\ -a_n& -a_{n-1}& -a_{n-2}& \cdots & -a_1\end{array}\right],\mathit{B}=\left[\begin{array}{c}{b}_1\\ b_2\\ ⋮\\ b_{n-1}\\ b_n\end{array}\right],\hfill \end{array}$$

$$\begin{array}{ccc}\hfill \mathit{C}& =& [1,0,0,\cdots ,0].\hfill \end{array}$$

The matrix $\mathit{A}\in {\mathbb{R}}^{n\times n}$ is called the system matrix, the vector $\mathit{B}\in {\mathbb{R}}^{n\times 1}$ is the system input vector, and the vector $\mathit{C}\in {\mathbb{R}}^{1\times n}$ is the system output vector. Equation (7) is called the state equation, and Equation (8) is called the output equation. Specially, for the multi-input multi-output systems, the input and output vectors will be extended to matrices. It is easy to find that those unknown parameters are in the system matrix and input vector. In order to estimate the unknown parameters of the system, the output equation needs to be redescribed to include all unknown parameters. Transforming the state equation, the output equation in Equation (8) can be further written as

$$\begin{array}{ccc}{y}_k\hfill & =\hfill & x_{1,k}\hfill \\ & =\hfill & x_{2,k-1}+b_1{u}_{k-1}\hfill \\ & =\hfill & x_{3,k-2}+b_2{u}_{k-2}+b_1{u}_{k-1}\hfill \\ & =\hfill & x_{4,k-3}+b_3{u}_{k-3}+b_2{u}_{k-2}+b_1{u}_{k-1}\hfill \\ & ⋮\hfill \\ & =\hfill & -a_n{x}_{1,k-n}-a_{n-1}{x}_{2,k-n}-\cdots -a_1{x}_{n,k-n}+b_n{u}_{k-n}+\cdots +b_2{u}_{k-2}+b_1{u}_{k-1}.\hfill \end{array}$$

(9)

Define the parameter vector $\mathbf{\theta }$, the information vector ${\mathbf{\psi }}_k$ as

$$\begin{array}{ccc}\hfill \mathbf{\theta }:& =& {[a_n,a_{n-1},\cdots ,a_1,b_n,b_{n-1},\cdots ,b_1]}^{\mathrm{T}}\in {\mathbb{R}}^{2n},\hfill \\ \hfill {\mathbf{\psi }}_k:& =& {[-x_{1,k-n},-x_{2,k-n},\cdots ,-x_{n,k-n},u_{k-n},u_{k-n+1},\cdots ,u_{k-1}]}^{\mathrm{T}}\in {\mathbb{R}}^{2n}.\hfill \end{array}$$

Considering that there will be some disturbance in the actual systems, introduce a noise term $v_k$ to the output equation. Then the output in Equation (9) can be simplified into a vector form

$$\begin{array}{c}\hfill y_k={\mathbf{\psi }}_k^{\mathrm{T}}\mathbf{\theta }+v_k,\end{array}$$

(10)

where $v_k$ is the random disturbance, which can be the white noise or colored noise. For practical systems, the effects of noise should be considered when dealing with the problem of parameter estimation.

Remark 1.

By introducing internal state variables, higher-order difference equations are transformed into first-order difference equations, which simplify the forms of the output equations of the models.

This paper develops a combined parameters, intermediate states, and noise variance estimation algorithm for linear difference equation systems. On the basis of the model shown in Equation (10), the next section will drive a VB-KF iterative algorithm to get the solution for the problem of the parameter estimation.

3. The Kalman Filter-Based Variational Bayesian Iterative Estimation Algorithm

3.1. The Variational Bayesian Parameter Estimation Algorithm

The VB algorithm is formed by introducing the variational approximation theory. It is difficult to deal with the integrations involved in the general posterior probability solution. The VB algorithm introduces a cluster of free factored distribution to approximate the complicated posterior probability, which avoids the direct multiple integrations on marginal distribution [38,39]. In this paper, the parameter vector together with the corresponding estimated error covariance matrix and noise variance are inferred by using the VB approach.

The unknown parameters that need to be estimated include the system model parameter vector $\mathbf{\theta }$, the variance of parameter estimates ${\zeta }^{-1}\mathit{I}$, and the system noise variance ${\sigma }^{-1}$. Assuming their priors follow Gaussian, Gamma, and Gamma distributions, respectively,

$$\begin{array}{ccc}\hfill p\left(\mathbf{\theta }\right)& =& N(\mathbf{0},{\zeta }^{-1}\mathit{I}),\hfill \\ \hfill p\left(\zeta \right)& =& \mathrm{Gamma}(\zeta ;d_0,e_0),\hfill \\ \hfill p\left(\sigma \right)& =& \mathrm{Gamma}(\sigma ;f_0,h_0).\hfill \end{array}$$

The VB algorithm introduces a free distribution $q(·)$ as the approximate distribution of parameter to simplify the calculation of logarithmic likelihood function of the measurement data. Then the probability distribution of observation can be expressed as

$$\begin{array}{lll}\log p\left(y_{1:k}\right)& =& \log p(\mathbf{\theta },\zeta ,\sigma ,y_{1:k})-\log p(\mathbf{\theta },\zeta ,\sigma |y_{1:k})\hfill \\ & =& \int q(\mathbf{\theta },\zeta ,\sigma )\times \log \frac{p(\mathbf{\theta },\zeta ,\sigma ,y_{1:k})}{q(\mathbf{\theta },\zeta ,\sigma )}{\mathrm{d}}_{\mathbf{\theta }}{\mathrm{d}}_{\zeta }{\mathrm{d}}_{\sigma }+\int q(\mathbf{\theta },\zeta ,\sigma )\times \log \frac{q(\mathbf{\theta },\zeta ,\sigma )}{p(\mathbf{\theta },\zeta ,\sigma |y_{1:k})}{\mathrm{d}}_{\mathbf{\theta }}{\mathrm{d}}_{\zeta }{\mathrm{d}}_{\sigma }\hfill \\ & =& \mathrm{F}\left[q(\mathbf{\theta },\zeta ,\sigma )\right]+\mathrm{KL}\left[q(\mathbf{\theta },\zeta ,\sigma )\left|\right|p(\mathbf{\theta },\zeta ,\sigma ,y_{1:k})\right],\hfill \end{array}$$

where $\mathrm{F}\left[q(\mathbf{\theta },\zeta ,\sigma )\right]$ is the lower bound of $\log p\left(y_{1:k}\right)$ (referring to the Jensen inequality [38]), that is

$$\begin{array}{c}\hfill \log p\left(y_{1:k}\right)⩾\int q(\mathbf{\theta },\zeta ,\sigma )\times \log \frac{p(\mathbf{\theta },\zeta ,\sigma ,y_{1:k})}{q(\mathbf{\theta },\zeta ,\sigma )}{\mathrm{d}}_{\mathbf{\theta }}{\mathrm{d}}_{\zeta }{\mathrm{d}}_{\sigma }.\end{array}$$

(11)

$\mathrm{KL}\left[\left(q(\mathbf{\theta },\zeta ,\sigma )\right||p(\mathbf{\theta },\zeta ,\sigma |y_{1:k})\right]$ is the Kullback–Leibler divergence between the factored approximation $q(\mathbf{\theta },\zeta ,\sigma )$ and the true posterior distribution $p(\mathbf{\theta },\zeta ,\sigma |y_{1:k})$, which demonstrates the similarity between the two probability distributions.

We assume that the joint probability distribution is decomposable:

$$\begin{array}{c}\hfill q(\mathbf{\theta },\zeta ,\sigma )=q\left(\mathbf{\theta }\right)q\left(\zeta \right)q\left(\sigma \right).\end{array}$$

We can find that when $q\left(\mathbf{\theta }\right)q\left(\zeta \right)q\left(\sigma \right)$ and $p(\mathbf{\theta },\zeta ,\sigma |y_{1:k})$ are identically distributed, the KL divergence between them can reach minimum. Therefore, these independent distributions $q\left(\mathbf{\theta }\right),q\left(\zeta \right),q\left(\sigma \right)$ can be solved by minimizing

$$\begin{array}{ccc}& & \mathrm{KL}\left[q\left(\mathbf{\theta }\right)q\left(\zeta \right)q\left(\sigma \right)\left|\right|p(\mathbf{\theta },\zeta ,\sigma |y_{1:k})\right]\hfill \\ & =\hfill & \int q(\mathbf{\theta },\zeta ,\sigma )\times \log \frac{p(\mathbf{\theta },\zeta ,\sigma |y_{1:k})}{q\left(\mathbf{\theta }\right)q\left(\zeta \right)q\left(\sigma \right)}{\mathrm{d}}_{\mathbf{\theta }}{\mathrm{d}}_{\zeta }{\mathrm{d}}_{\sigma }.\hfill \end{array}$$

(12)

Using the theorems from the calculus of variation [31], the optimal solutions for the parameter distributions satisfy the following equations

$$\begin{array}{ccc}\hfill \log q\left(\mathbf{\theta }\right)& =& \int \int \log p(\mathbf{\theta },\zeta ,\sigma ,y_{1:k})q\left(\zeta \right)q\left(\sigma \right){\mathrm{d}}_{\zeta }{\mathrm{d}}_{\sigma }+c_{\mathbf{\theta }},\hfill \end{array}$$

(13)

$$\begin{array}{ccc}\hfill \log q\left(\zeta \right)& =& \int \int \log p(\mathbf{\theta },\zeta ,\sigma ,y_{1:k})q\left(\mathbf{\theta }\right)q\left(\sigma \right){\mathrm{d}}_{\mathbf{\theta }}{\mathrm{d}}_{\sigma }+c_{\zeta },\hfill \end{array}$$

(14)

$$\begin{array}{ccc}\hfill \log q\left(\sigma \right)& =& \int \int \log p(\mathbf{\theta },\zeta ,\sigma ,y_{1:k})q\left(\mathbf{\theta }\right)q\left(\zeta \right){\mathrm{d}}_{\mathbf{\theta }}{\mathrm{d}}_{\zeta }+c_{\sigma },\hfill \end{array}$$

(15)

where $c_{\mathbf{\theta }},c_{\zeta },c_{\sigma }$ denote the terms which are independent of $\mathbf{\theta },\zeta ,\sigma$, respectively.

The initial joint probability $p(\mathbf{\theta },\zeta ,\sigma ,y_{1:k})$ can be factored as

$$\begin{array}{c}\hfill p(\mathbf{\theta },\zeta ,\sigma ,y_{1:k})=N(\mathbf{0},{\zeta }^{-1}\mathit{I})\mathrm{Gamma}(\zeta ;c_0,d_0)\mathrm{Gamma}(\sigma ;e_0,f_0)\prod _{k=1}^{N}N(y_k;{\mathbf{\psi }}_k^{\mathrm{T}}\mathbf{\theta },{\sigma }^{-1}).\end{array}$$

(16)

Using Equation (16) into Equation (13) and ignoring items which are unrelated to $\mathbf{\theta }$ gives

$$\begin{array}{ccc}\log q^{(s+1)}\left(\mathbf{\theta }\right)\hfill & =\hfill & \int \int q\left(\zeta \right)q\left(\sigma \right)\left[-\frac{1}{2}{\mathbf{\theta }}^{\mathrm{T}}\zeta \mathbf{\theta }-\frac{1}{2}\sum _{k=1}^N{(y_k-{\mathbf{\psi }}_k^{\mathrm{T}}\mathbf{\theta })}^{\mathrm{T}}\sigma (y_k-{\mathbf{\psi }}_k^{\mathrm{T}}\mathbf{\theta })\right]{\mathrm{d}}_{\zeta }{\mathrm{d}}_{\sigma }+c_{\mathbf{\theta }}\hfill \\ & =\hfill & -\frac{1}{2}\left\{{\mathbf{\theta }}^{\mathrm{T}}\left[{\zeta }^{\left(s\right)}\mathit{I}+\sum _{k=1}^N{\mathbf{\psi }}_k{\sigma }^{\left(s\right)}{\mathbf{\psi }}_k^{\mathrm{T}}\right]\mathbf{\theta }+\sum _{k=1}^N{y}_k^2-\sum _{k=1}^N{\mathbf{\theta }}^{\mathrm{T}}{\mathbf{\psi }}_k{\sigma }^{\left(s\right)}{y}_k-\sum _{k=1}^N{y}_k{\sigma }^{\left(s\right)}{\mathbf{\psi }}_k^{\mathrm{T}}\mathbf{\theta }\right\}\hfill \\ & & +c_{\mathbf{\theta }},\hfill \end{array}$$

(17)

where $q^{(s+1)}(·)$ denotes the approximation of $q(·)$ at the $(s+1)$th iteration. From Equation (17) we can find that $q^{(s+1)}\left(\mathbf{\theta }\right)$ is the probability density function of a Gaussian distribution, and the system parameter $\mathbf{\theta }$ can be updated as

$$\begin{array}{ccc}\hfill {\overline{\mathbf{\theta }}}^{(s+1)}& =& \mathrm{var}\left[{\mathbf{\theta }}^{(s+1)}\right]\sum _{k=1}^N{\mathbf{\psi }}_k{\sigma }^{\left(s\right)}{y}_k,\hfill \end{array}$$

(18)

$$\begin{array}{ccc}\hfill \mathrm{var}\left[{\mathbf{\theta }}^{(s+1)}\right]& =& {\left[{\zeta }^{\left(s\right)}\mathit{I}+\sum _{k=1}^N{\mathbf{\psi }}_k{\sigma }^{\left(s\right)}{\mathbf{\psi }}_k^{\mathrm{T}}\right]}^{-1}.\hfill \end{array}$$

(19)

Furthermore, we can calculate the mathematical expectation of $\mathbf{\theta }{\mathbf{\theta }}^{\mathrm{T}}$

$$\begin{array}{c}\hfill {\mathrm{E}}^{(s+1)}\left(\mathbf{\theta }{\mathbf{\theta }}^{\mathrm{T}}\right)=\mathrm{var}{(}^{(s+1)})+{\mathbf{\theta }}^{(s+1)}{\left({\mathbf{\theta }}^{(s+1)}\right)}^{\mathrm{T}}.\end{array}$$

(20)

According to Equation (14), we have

$$\begin{array}{ccc}\hfill \log q^{(s+1)}\left(\zeta \right)& =& \int \int q\left(\mathbf{\theta }\right)q\left(\sigma \right)\left[\frac{1}{2}\log \zeta -\frac{1}{2}\left(\mathbf{\theta }\right)\zeta \mathbf{\theta }+(d_0-1)\log \zeta -e_0\zeta \right]{\mathrm{d}}_{\mathbf{\theta }}{\mathrm{d}}_{\sigma }+c_{\zeta }\hfill \\ & =& -\left[\frac{1}{2}{\mathrm{E}}^{(s+1)}\left({\mathbf{\theta }}^{\mathrm{T}}\mathbf{\theta }\right)+e_0\right]\zeta +(\frac{1}{2}+d_0-1)\log \zeta +c_{\zeta }\hfill \\ & =& -\left\{\frac{1}{2}\mathrm{tr}\left[\mathrm{var}{(}^{(s+1)})+{\mathbf{\theta }}^{(s+1)}{\left({\mathbf{\theta }}^{(s+1)}\right)}^{\mathrm{T}}\right]+e_0\right\}\zeta +(\frac{1}{2}+d_0-1)\log \zeta +c_{\zeta },\hfill \end{array}$$

which yields

$$\begin{array}{c}\hfill q^{(s+1)}\left(\zeta \right)=\mathrm{Gamma}(\zeta ;d^{(s+1)},e^{(s+1)}),\end{array}$$

where $d^{(s+1)}$ and $e^{(s+1)}$ are the two parameters of the Gamma distribution and are given by

$$\begin{array}{ccc}\hfill d^{(s+1)}& =& \frac{1}{2}+d_0,\hfill \\ \hfill e^{(s+1)}& =& e_0+\frac{1}{2}\mathrm{tr}\left[\mathrm{var}\left({\mathbf{\theta }}^{(s+1)}\right)+{\mathbf{\theta }}^{(s+1)}{\left({\mathbf{\theta }}^{(s+1)}\right)}^{\mathrm{T}}\right].\hfill \end{array}$$

Then, update the expectation of $\zeta$ at iteration $s+1$ by

$$\begin{array}{ccc}\hfill {\overline{\zeta }}^{(s+1)}& =& \frac{d^{(s+1)}}{e^{(s+1)}}.\hfill \end{array}$$

(21)

Similarly, the variational form for $q^{(s+1)}\left(\sigma \right)$ obeys

$$\begin{array}{cc}\hfill \log q^{(s+1)}\left(\sigma \right)& =\int \int q\left(\mathbf{\theta }\right)q\left(\zeta \right)\left((f_0-1)\log \sigma -h_0\sigma +\frac{N}{2}\log \sigma -\frac{1}{2}\sum _{k=1}^N{(y_k-{\mathbf{\psi }}_k^{\mathrm{T}}\mathbf{\theta })}^{\mathrm{T}}\sigma (y_k-{\mathbf{\psi }}_k^{\mathrm{T}}\mathbf{\theta })\right){\mathrm{d}}_{\mathbf{\theta }}{\mathrm{d}}_{\zeta }+c_{\sigma }\hfill \\ & =(f_0+\frac{N}{2}-1)\log \sigma -\frac{1}{2}\left[h_0+\sum _{k=1}^N(y_k^2-2y_k{\mathbf{\psi }}_k^{\mathrm{T}}{\mathbf{\theta }}^{(s+1)}+{\mathbf{\psi }}_k^{\mathrm{T}}{\mathrm{E}}^{(s+1)}\left(\mathbf{\theta }{\mathbf{\theta }}^{\mathrm{T}}\right){\mathbf{\psi }}_k)\right]\sigma +c_{\sigma },\hfill \end{array}$$

which yields

$$\begin{array}{c}\hfill q^{(s+1)}\left(\sigma \right)=\mathrm{Gamma}(\sigma ;f^{(s+1)},h^{(s+1)}),\end{array}$$

where $f^{(s+1)}$ and $h^{(s+1)}$ are given by

$$\begin{array}{ccc}\hfill f^{(s+1)}& =& \frac{N}{2}+f_0,\hfill \\ \hfill h^{(s+1)}& =& h_0+\sum _{k=1}^N\left\{y_k^2-2y_k{\mathbf{\psi }}_k^{\mathrm{T}}{\mathbf{\theta }}^{(s+1)}+{\mathbf{\psi }}_k^{\mathrm{T}}\mathrm{tr}[\mathrm{var}{(}^{(s+1)})+{\mathbf{\theta }}^{(s+1)}{\left({\mathbf{\theta }}^{(s+1)}\right)}^{\mathrm{T}}\right]{\mathbf{\psi }}_k\}.\hfill \end{array}$$

Then $\overline{\sigma }$ can be computed as

$$\begin{array}{c}\hfill {\overline{\sigma }}^{(s+1)}=\frac{f^{(s+1)}}{h^{(s+1)}}.\end{array}$$

(22)

3.2. The State Estimation Algorithm

Above all, we obtain the posterior probability distribution or optimal solution of system parameters and the noise variance. However, the state-space model contains both unknown system parameters and unavailable system states, which brings a difficulty to obtaining the parameter estimates. The Kalman filter is usually used to estimate the intermediate states for linear state-space systems with Gaussian noise. The state observer is a deterministic state estimation algorithm, which is simpler in structure but only obtains an approximate estimation of the system states. To improve the estimation accuracy, this paper adopts a modified Kalman filter to obtain state estimates under the Bayesian framework. According to the considered state-space model, the following Kalman filter algorithm [40] is designed to generate the state estimates

$$\begin{array}{ccc}\hfill {\widehat{\mathit{x}}}_{k+1}& =& \widehat{\mathit{A}}{\widehat{\mathit{x}}}_k+\widehat{\mathit{b}}{\overline{u}}_k+{\mathit{L}}_k(y_k-\mathit{c}{\widehat{\mathit{x}}}_k),{\widehat{\mathit{x}}}_1={\mathbf{1}}_n/p_0,\hfill \end{array}$$

(23)

$$\begin{array}{ccc}\hfill {\mathit{L}}_k& =& {\widehat{\mathit{A}}}_k{\mathit{P}}^k{\mathit{c}}^{\mathrm{T}}{({\widehat{\sigma }}_k+\mathit{c}{\mathit{P}}_k{\mathit{c}}^{\mathrm{T}})}^{-1},\hfill \end{array}$$

(24)

$$\begin{array}{ccc}\hfill {\mathit{P}}_{k+1}& =& \widehat{\mathit{A}}{\mathit{P}}_k{\widehat{\mathit{A}}}^{\mathrm{T}}-{\mathit{L}}_k\mathit{c}{\mathit{P}}_k{\widehat{\mathit{A}}}^{\mathrm{T}},{\mathit{P}}_1={\mathit{I}}_n.\hfill \end{array}$$

(25)

Since the system parameters are unknown, we have to utilize the estimated parameters to construct the system parameter vector and matrix

$$\begin{array}{c}\hfill \widehat{\mathit{A}}:=\left[\begin{array}{ccccc}0& 1& 0& \cdots & 0\\ 0& 0& 1& \cdots & 0\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ 0& 0& 0& \cdots & 1\\ -{\widehat{a}}_n& -{\widehat{a}}_{n-1}& -{\widehat{a}}_{n-2}& \cdots & -{\widehat{a}}_1\end{array}\right]\in {\mathbb{R}}^{n\times n},\widehat{\mathit{b}}:=\left[\begin{array}{c}{\widehat{b}}_{1,k}\\ {\widehat{b}}_{2,k}\\ ⋮\\ {\widehat{b}}_{n-1,k}\\ {\widehat{b}}_{n,k}\end{array}\right]\in {\mathbb{R}}^n.\end{array}$$

The flowchart of executing the proposed VB-KF iterative algorithm is shown in Figure 1.

The iterative steps for calculating parameter and state estimates are summarized as follows:

(1)

Initialization: set $s=1$, ${\mathit{x}}_k^{\left(0\right)}={\mathbf{1}}_n$, ${\overline{\mathbf{\theta }}}^{\left(0\right)}={\mathbf{1}}_{n_2}$, and give a group of random positive values to parameters $d_0,e_0,f_0,h_0$ and a small positive value to $\epsilon$.

(2)

Collect input-output data $(u_k,y_k:k=1,2,\cdots ,N)$, and contract information vector ${\mathbf{\psi }}_k$ with Equation (10).

(3)

Update ${\overline{\mathbf{\theta }}}^{(s+1)}$, $\mathrm{var}\left[{\mathbf{\theta }}^{(s+1)}\right]$, ${\mathrm{E}}^{(s+1)}\left(\mathbf{\theta }{\mathbf{\theta }}^{\mathrm{T}}\right)$, and ${\overline{\zeta }}^{(s+1)}$ with Equations (18)–(21), respectively.

(4)

Update noise variance ${\overline{\sigma }}^{(s+1)}$ with Equation (22).

(5)

Use the new obtained parameter estimates and noise variance to update states with Kalman filter algorithm in Equations (23)–(25).

(6)

Evaluate the relative change between ${\overline{\mathbf{\theta }}}^{(s+1)}$ and ${\overline{\mathbf{\theta }}}^{\left(s\right)}$, if $\left|\right|{\overline{\mathbf{\theta }}}^{(s+1)}-{\overline{\mathbf{\theta }}}^{\left(s\right)}\left|\right|⩽\epsilon$ ($\left|\right|·\left|\right|$ means L2-norm), for a very small positive number $\epsilon$, then stop the iteration procedure and get the parameter estimates ${\overline{\mathbf{\theta }}}^{(s+1)}$; or else, let $s=s+1$ and go to Step 3.

The proposed methods can combine other estimation algorithms [41,42,43,44,45] and the mathematical tools [46,47,48,49] to study the parameter identification problems of other linear and nonlinear systems with colored noise [50,51,52] and can be applied to other literatures [53,54,55,56].

4. Simulation Example

Consider the following 3rd-order difference equation:

$$\begin{array}{c}\hfill y_k+0.50y_{k-1}+0.32y_{k-2}+0.18y_{k-3}=1.00u_{k-1}+0.55u_{k-2}+0.97u_{k-3}+v_k.\end{array}$$

Taking into account the interference of noise, transform the difference equation into the state-space model:

$$\begin{array}{ccc}\hfill \left[\begin{array}{c}{x}_{1,k+1}\\ x_{2,k+1}\\ x_{3,k+1}\end{array}\right]& =& \left[\begin{array}{ccc}0& 1& 0\\ 0& 0& 1\\ -0.18& -0.32& -0.50\end{array}\right]\left[\begin{array}{c}{x}_{1,k}\\ x_{2,k}\\ x_{3,k}\end{array}\right]+\left[\begin{array}{c}1.00\\ 0.55\\ 0.97\end{array}\right]u_k,\hfill \\ \hfill y_k& =& \left[100\right]\left[\begin{array}{c}{x}_{1,k}\\ x_{2,k}\\ x_{3,k}\end{array}\right]+v_k.\hfill \end{array}$$

The parameter vector to be estimated is

$$\mathbf{\theta }=\left[0.18,0.32,0.50,1.00,0.55,0.97\right].$$

In this simulation example, the input is selected as a persistent excitation signal sequence with zero mean and unit variance, $v_k$ is Gaussian noise sequence with zero mean and variance ${\sigma }^2$. Of the collected data length $N=2000$, the first 1000 sets of data $\left(N_1\right)$ are used for the training set and the rest of the $\left(N_2\right)$ sets are used for the testing set. Design different noise variances to simulate the noise interferences on the output signal. Use the proposed VB-KF iterative estimation algorithm to estimate the parameter vector.

Taking the calculated mean values of the Gaussian distributions for the parameters as the estimates of the unknown parameters, the proposed VB-KF parameter estimates and estimation error $\delta =\left|\right|{\overline{\mathbf{\theta }}}^{\left(s\right)}-\mathbf{\theta }\left|\right|/\left|\right|\mathbf{\theta }\left|\right|$ with the increasing of the iteration s under different noise variances are presented in

Table 1. The parameter estimates and estimation errors versus s under the proposed VB-KF algorithm.

${\mathit{\sigma }}^2$	s	${\mathit{a}}_1$	${\mathit{a}}_2$	${\mathit{\kappa }}_1$	${\mathit{\kappa }}_2$	${\mathit{\kappa }}_3$	${\mathit{b}}_2$	$\mathit{\delta }(\%)$
${0.10}^2$	1	0.01016	−0.02289	−0.03700	1.00397	0.53194	0.97784	40.69214
	2	−0.07361	0.00728	0.89702	0.94687	0.52043	0.96392	35.08236
	5	0.17474	0.30156	0.49471	0.97522	0.55029	0.99253	2.40308
	10	0.17243	0.31521	0.49920	0.97571	0.54883	0.99432	2.19306
	20	0.17246	0.31520	0.49918	0.97571	0.54883	0.99432	2.19278
	40	0.17246	0.31520	0.49918	0.97571	0.54883	0.99432	2.19278
${0.50}^2$	1	−0.05182	−0.05809	0.04132	1.00842	0.54880	0.99711	39.39494
	2	−0.11574	0.04963	0.90089	0.93849	0.52609	0.97003	35.19921
	5	0.17688	0.31997	0.49349	0.96759	0.56015	1.00064	2.85674
	10	0.17674	0.32778	0.49767	0.96774	0.55874	1.00189	2.90030
	20	0.17676	0.32777	0.49764	0.96774	0.55874	1.00189	2.90017
	40	0.17676	0.32777	0.49764	0.96774	0.55874	1.00189	2.90017
${1.00}^2$	1	−0.01785	0.06661	−0.15694	0.98365	0.51835	1.01729	45.26212
	2	−0.01462	−0.04935	0.75665	0.89482	0.48419	0.96419	31.18481
	5	0.17291	0.28867	0.46187	0.91682	0.50836	0.98695	6.59330
	10	0.17082	0.29553	0.46557	0.91739	0.50776	0.98825	6.41325
	20	0.17083	0.29553	0.46556	0.91739	0.50776	0.98825	6.41343
	40	0.17083	0.29553	0.46556	0.91739	0.50776	0.98825	6.41343
true values		0.18000	0.32000	0.50000	1.00000	0.55000	0.97000

and Figure 2. The curves of parameter estimates against s are plotted in Figure 3. The estimated noise variances are shown in

Table 2. The estimates of noise variance versus s under the proposed VB algorithm.

${\mathit{\sigma }}^2/\mathit{s}$	1	2	3	5	10	15	40
${0.10}^2$	0.2000	0.7509	0.0356	0.0280	0.0108	0.0108	0.0108
${0.50}^2$	0.2000	0.9935	0.3010	0.2748	0.2664	0.2664	0.2664
${1.00}^2$	0.2000	1.6252	1.0241	1.0097	0.9950	0.9950	0.9950

.

When the noise variance is ${\sigma }^2={0.10}^2$, use the estimated parameters to construct the approximate difference equation:

$$\begin{array}{c}\hfill y_k+0.49918y_{k-1}+0.31520y_{k-2}+0.17246y_{k-3}=0.97571u_{k-1}+0.54883u_{k-2}+0.99432u_{k-3}+v_k.\end{array}$$

The computed prediction ${\widehat{y}}_k$ and the true output $y_k$ are compared in Figure 4. From Figure 4, we can see that the predictions are very close to the true outputs.

For comparison, we also use the open-loop state observer to generate the estimates of unknown states [37]. Here, we combine the proposed VB parameter estimation method with state observer to form the state observer-based variational Bayesian (VB-SO) iterative estimation algorithm. In [57], the KF-based least squares iterative algorithm is used to identify the state space system, where the unknown process noise variance is set to one $(\mathit{R}=1.00)$ in the Kalman filter. Here, we use this KF based-variational Bayesian algorithm (VB-KF₂) and the proposed method (VB-KF₁), where the noise variance is accurately computed, to estimate system parameters and states under different noises. Furthermore, the parameter estimation errors of the three different algorithms are compared in Figure 5 with ${\sigma }^2={0.50}^2$.

In addition, we adopt the other batch of data set for model validation. The parameter estimates with the least estimation error are chosen for the verification test. The root mean square errors (RMSE) of the three estimate models are computed as

$$\begin{array}{ccc}\hfill {\mathrm{RMSE}}_1& =& \sqrt{\frac{1}{N_2}\sum _{i=1001}^{2000}{[y_i-{\widehat{y}}_{{\mathrm{vbkf}}_1}]}^2}=0.2833,\hfill \end{array}$$

$$\begin{array}{ccc}\hfill {\mathrm{RMSE}}_2& =& \sqrt{\frac{1}{N_2}\sum _{i=1001}^{2000}[y_i-{\widehat{y}}_{\mathrm{vbso}}]}=0.3007,\hfill \end{array}$$

$$\begin{array}{ccc}\hfill {\mathrm{RMSE}}_3& =& \sqrt{\frac{1}{N_2}\sum _{i=1001}^{2000}{[y_i-{\widehat{y}}_{{\mathrm{vbkf}}_2}]}^2}=0.2930.\hfill \end{array}$$

From the calculated results, it can be seen that the proposed VB-KF algorithm has the smallest root mean square error.

5. Conclusions

In this paper, a Kalman filter-based variational Bayesian iterative algorithm is proposed to estimate the unknown parameters for linear difference equation systems. The main advantage of the presented method is that it can give the distribution of the parameter rather than a point estimate. Besides, the proposed algorithm obtains the noise statistics and state estimation. From the simulation results shown in Table 1 and Table 2 and Figure 2, Figure 3, Figure 4 and Figure 5, it is easy to find the following conclusions.

• The parameter estimation errors are generally becoming smaller with iteration s increasing and with the noise variance decreasing. The estimated parameters are very close to the real values.
• The estimated noise variances in Table 2 show that variance estimates are relatively large in the previous period because of the inaccurate parameters and states, but with the increasing of the iteration s, the noise variance estimates can converge to their true value.
• Under the same batch of date and noise variance, the estimation accuracy of the proposed VB algorithm is higher than that of VB-SO and VB-KF₂ algorithms.
• The predicted outputs fit well with true outputs, which indicates the effectiveness of the proposed algorithm.

The proposed variational Bayesian iterative estimation algorithm in this paper for state space models from the observation data can combine some mathematical strategies [58,59,60,61] to study the parameter estimation algorithms of other stochastic systems with colored noise and can be applied to other fields [62,63,64,65,66].