Improved Minimum Entropy Filtering for Continuous Nonlinear Non-Gaussian Systems Using a Generalized Density Evolution Equation

Abstract

This paper investigates the filtering problem for multivariate continuous nonlinear non-Gaussian systems based on an improved minimum error entropy (MEE) criterion. The system is described by a set of nonlinear continuous equations with non-Gaussian system noises and measurement noises. The recently developed generalized density evolution equation is utilized to formulate the joint probability density function (PDF) of the estimation errors. Combining the entropy of the estimation error with the mean squared error, a novel performance index is constructed to ensure the estimation error not only has small uncertainty but also approaches to zero. According to the conjugate gradient method, the optimal filter gain matrix is then obtained by minimizing the improved minimum error entropy criterion. In addition, the condition is proposed to guarantee that the estimation error dynamics is exponentially bounded in the mean square sense. Finally, the comparative simulation results are presented to show that the proposed MEE filter is superior to nonlinear unscented Kalman filter (UKF).

2. Problem Formulation

2.1. System Model

Consider a nonlinear stochastic continuous system described by:

$$\{\begin{array}{l}\dot{x}(t)=A(t)x(t)+D(t)\omega (t)\\ y(t)=F(x(t),\upsilon (t))\end{array}$$

(1)

where x ∈ ℜⁿ is the state vector, y ∈ ℜ^m is the measured output vector. ω ∈ ℜ^p and υ ∈ ℜ^q are system noises and measurement noises respectively. A(t) and D(t) are two known time-varying system matrices. The following assumptions are made for simplicity, which can be satisfied by many practical systems.

Assumption 1: F(·) is a known Borel measurable and smooth vector-value nonlinear function of its arguments.

Assumption 2: ω and υ are bounded, mutually independent random vectors with known joint PDFs γ_ω and γ_υ.

2.2. Filter Dynamics

For the nonlinear stochastic system (1), the filter dynamics can be formulated as follows:

$$\{\begin{array}{l}\dot{\widehat{x}}(t)=A(t)\widehat{x}(t)+L(y(t)-\widehat{y}(t))\\ \widehat{y}(t)=F(\widehat{x}(t),0)\end{array}$$

(2)

The resulting estimation error satisfies:

(3)

where is the gain matrix to be determined and can be denoted as , L_i is the ith row vector of L. Let and thus is a stretched column vector.

The purpose of filtering algorithm for nonlinear non-Gaussian systems is to ensure that the estimation errors achieve small dispersion and approach to zero. Entropy provides a measure of the average information contained in a given PDF. When entropy of estimation errors is minimized, all moments of the error PDF are constrained. Hence, the following quadratic Renyi’s entropy of estimation error is still contained in the performance index to measure the dispersion of the estimation error:

(4)

where P_e(z) is the joint PDF of estimation error and V(e) the information potential [14]. It can be seen that it is necessary to provide the joint PDF of errors firstly. In the next section, we will deduce the expression of P_e(z) according to the generalized density evolution equation.

3. Formulation for the Joint PDF of Error

In most cases, Equation (3) is a well-posed equation, and the error vector e(t) can be determined uniquely. It may be assumed to be:

(5)

At the present stage, the explicit expression of H(·) is not requisite, and the sufficient condition just needs to know its existence and uniqueness. The derivative of e(t) can be assumed to take the form:

(6)

It is observed that all the randomness involved in this error dynamics comes from noises ω and υ, thus, the augmented system (e(t), ω, υ) is probability preserved. In other words, from the random event description of the principle of preservation of probability, it leads to:

(7)

where Ω_e, Ω_ω and Ω_υ are the distribution domains of e, ω and υ, respectively; is the joint PDF of (e(t), ω, υ). It follows from Equation (7) that:

$$\frac{D}{Dt}{\displaystyle {\int }_{{\mathrm{\Omega }}_e\times {\mathrm{\Omega }}_{\omega }\times {\mathrm{\Omega }}_{\upsilon }}{p}_{e\omega \upsilon }(e,\omega ,\upsilon ,t)ded\omega d\upsilon }={\displaystyle {\int }_{{\mathrm{\Omega }}_e\times {\mathrm{\Omega }}_{\omega }\times {\mathrm{\Omega }}_{\upsilon }}\left(\frac{\partial p_{e\omega \upsilon }}{\partial t}+h\frac{\partial p_{e\omega \upsilon }}{\partial e}\right)ded\omega d\upsilon }$$

(8)

Combining Equation (7) with Equation (8) and considering the arbitrariness of , it yields:

(9)

where Equation (6) and Equation (9) are the generalized density evolution equation (GDEE) for (e(t), ω, υ). The corresponding instantaneous PDF of (e(t)) can be obtained by solving a family of partial differential equations with the following given initial condition:

(10)

where δ(·) is the Dirac-Delta function; and is deterministic initial value of (e(t)). Then, we have:

(11)

where the joint PDF is the solution of (8), which can be obtained according to the method presented in [22].

4. Improved MEE Filtering

4.1. Performance Index

The following performance index for a minimum entropy filter was presented in [6]:

(12)

where R₁ and R₂ are weighing matrices. The first term is the entropy of the estimation errors and the second term is used to penalize the elements of the gain. Although the MEE criteria can minimize both the probabilistic uncertainty and dispersion of the estimation error, it cannot guarantee the minimum error. In this work , an improved performance index J is considered:

(13)

where is the quadratic Renyi’s entropy given in Equation (4). is the weight corresponding to the mean squared error. The third term on the right side of the equation is utilized to make the estimation errors approach to zero. Since Renyi’s entropy is a monotonic increasing function of the negative information potential, minimization of Renyi’s entropy is equivalent to minimizing the inverse of the quadratic information potential , so the performance index can be rewritten as follows:

(14)

Remark 1: The entropy of estimation error is replaced by quadratic information potential of error in order to simplify the calculation. Nonparametric estimation of the PDF of estimation errors is another alternative way to estimate the PDF except the method introduced in [22]. Nonparametric entropy (or information potential) estimation of random error can be found in [16,17].

4.2. Optimal Filter Gain Matrix

After constructing the performance index J, the optimal filter gain can be carried out by minimizing the performance index Equation (14), which can be regarded as an unconstrained nonlinear programming problem. The classical conjugate gradient approach is employed here to obtain an iterative solution, which could avoid the difficulty of solving the extremum condition ( ). The elements of optimal filter gain can be solved and summarized in Theorem 1.

Theorem 1: For a given accuracy Ɛ > 0, the gain matrix of the suboptimal improved MEE filter is given by:

(15)

where , and the Hessian matrix is:

(16)

Proof: Choose an initial value l₀, and denote k := 0. Search the next value l_k+1 from the given point l_k along the conjugate gradient direction P_k. Repeat the above procedure until reach the given accuracy Ɛ. Consequently, we can obtain the optimal filtering algorithm shown in Equation (15).

Based on the conjugate gradient method, the steps to solve the optimal filter gain are summarized as follows:

Step (1) Give an accuracy Ɛ > 0 and initialize l₀, denote k = 0

Step (2) Set .

Step (3) Obtain by solving , and update the gain .

Step (4) If , stop, and is the optimal solution; Otherwise, if , turn to step (5), and if , reset , and turn to step (2).

Step (5) Choose , set , and turn to step 3), where .

Remark 2: The conjugate gradient method here could overcome the slow convergence of the steepest descent approach. It is one of the most effective algorithms to deal with large-scale nonlinear optimization problems.

4.3. Exponentially Bounded in the Mean Square

In this section, the Lyapunov method is utilized to give the condition under which the filter error dynamics is exponentially ultimately bounded in the mean square. Firstly, in order to simplify the problem, we give the following assumption:

Assumption 3: The nonlinear vector functionis assumed to satisfy [23] and:

(17)

where , are known constant matrices, , are vectors, a is known positive constant.

For estimation error dynamics (3), the exponentially mean-square boundedness is defined as follows:

Definition 1 [24]: For all initial conditions , the dynamics of the estimation error (i.e., the solution of the system (3)) is exponentially ultimately bounded in the mean square if there exist constants α > 0, β > 0 and γ > 0 such that:

(18)

Moreover, filter (2) is said to be exponential if, for every , system (3) is exponentially ultimately bounded in the mean square.

Theorem 2: Let the filter parameter L be given. If there exist positive scalars Ɛ₁, Ɛ₂ and Ɛ₃ such that the following matrix inequality:

(19)

has a positive definite solution P > 0, then system (3) is exponentially ultimately bounded in the mean square.

Proof: Fix arbitrarily and denote . The Lyapunov function candidate can be chosen as:

(20)

where P > 0 is the solution to the matrix inequality Equation (19).

For notational convenience, we give the following definitions:

(21)

and then it follows from Equation (3) that:

(22)

Hence, the derivative of V along a given trajectory Equation (22) is obtained as:

(23)

Let Ɛ₁, Ɛ₂ and Ɛ₃ be positive scalars. Then the following matrix inequalities hold:

(24)

(25)

(26)

Noticing Assumptions 3 and Equation (17), we have:

(27)

Substituting Equations (24)–(27) into Equation (23) results in:

(28)

where:

(29)

and it is known from Equation (19) that II < 0.

Based on Equation (28), it can be found that:

(30)

Let . Integrating both sides from 0 to T > 0 and taking the expectation lead to:

(31)

where , (according to Assumption 2, ω(t) and υ(t) are bounded random vectors) , and:

(32)

Notice that and let , . Since T > 0 is arbitrary, the definition of exponential ultimate boundedness in (18) is then satisfied, and this completes the proof of Theorem 2.

5. Simulation Results

In order to show the applicability of the proposed filtering algorithm, we consider the following nonlinear system represented as :

(33)

where and . The random disturbances ω and υ obey non-Gaussian, and their distributions are shown in Figure 1. The weights in Equation (14) are selected as R₁ = 10, R₂ = 2 and R₃ = 10, respectively. The simulation results based on the MEE filter are shown in Figure 2, Figure 3, Figure 4, Figure 5, Figure 6, Figure 7a and Figure 8a). And the comparative results between MEE filter and UKF are shown in Figure 7 and Figure 8.

Figure 2 demonstrates that the performance index decreases monotonically with time. In Figure 3 and Figure 4, both the range and PDFs of errors are given, it can be seen that the shapes of PDFs of the tracking errors become narrower and sharper along with the increasing time, which illustrates the dispersions of estimation errors can be reduced. In order to clarify the improvements, the initial and final PDFs are shown in Figure 5 and Figure 6. It can be shown from Figure 3, Figure 4, Figure 5 and Figure 6 that the proposed improved MEE filter can decrease the uncertainties of the estimation errors and drive the estimation errors approaching to zero. Figure 7 shows that both the MEE filter and UKF make the estimation errors approach to zero. But the estimation errors in a are more closer to zero with smaller randomness. The variances of the filter gains in two cases are presented in Figure 8. It can be seen that the gains of UKF are not convergent.

From the above analyses, it is obvious that the proposed strategy has better performance than UKF. Therefore, the proposed MEE filter is more suitable for nonlinear stochastic systems with non-Gaussian noises.

Figure 1. Distribution of random noises ω and υ

Figure 1. Distribution of random noises ω and υ

Figure 2. Performance index.

Figure 2. Performance index.

Figure 3. PDF of estimation error e₁

Figure 3. PDF of estimation error e₁

Figure 4. PDF of estimation error e₂.

Figure 4. PDF of estimation error e₂.

Figure 5. Initial and final PDFs of e₁

Figure 5. Initial and final PDFs of e₁

Figure 6. Initial and final PDFs of e₂

Figure 6. Initial and final PDFs of e₂

Figure 7. Estimation errors under the filter: (a) MEE filter (b) UKF.

Figure 7. Estimation errors under the filter: (a) MEE filter (b) UKF.

Figure 8. Filter gain: (a) MEE filter (b) UKF.

Figure 8. Filter gain: (a) MEE filter (b) UKF.

6. Conclusions

In this paper, based on the improved MEE criterion, optimal filter design problem is studied for multivariate nonlinear continuous stochastic systems with non-Gaussian noises. A novel performance index, including Renyi’s entropy of the estimation error, mean value of the squared error and constrains on filter gain, is formulated. By using the generalized density evolution equation, the relationship among the PDFs of estimation error, random disturbances and filter gain matrix is obtained. A recursive optimal filtering algorithm is obtained by minimizing the improved MEE criterion. The exponentially mean-square boundedness condition of the estimation error systems is established. In this work, the proposed MEE filter and UKF are both applied to a continuous nonlinear and non-Gaussian stochastic system. Comparative simulation results demonstrate the superiority of the presented control algorithm.