A Two-Filter Approach for State Estimation Utilizing Quantized Output Data

Abstract

Filtering and smoothing algorithms are key tools to develop decision-making strategies and parameter identification techniques in different areas of research, such as economics, financial data analysis, communications, and control systems. These algorithms are used to obtain an estimation of the system state based on the sequentially available noisy measurements of the system output. In a real-world system, the noisy measurements can suffer a significant loss of information due to (among others): (i) a reduced resolution of cost-effective sensors typically used in practice or (ii) a digitalization process for storing or transmitting the measurements through a communication channel using a minimum amount of resources. Thus, obtaining suitable state estimates in this context is essential. In this paper, Gaussian sum filtering and smoothing algorithms are developed in order to deal with noisy measurements that are also subject to quantization. In this approach, the probability mass function of the quantized output given the state is characterized by an integral equation. This integral was approximated by using a Gauss–Legendre quadrature; hence, a model with a Gaussian mixture structure was obtained. This model was used to develop filtering and smoothing algorithms. The benefits of this proposal, in terms of accuracy of the estimation and computational cost, are illustrated via numerical simulations.

1. Introduction

It is well known that discrete-time dynamical systems can be described as first-order difference equations relating internal variables called states [1]. State estimation is a scientific discipline that studies methodologies and algorithms for estimating the state of dynamical systems from input–output measurements [2,3]. There are a variety of applications that use state estimation, such as control [4,5,6,7], parameter identification [8,9,10], power systems [11,12], fault detection [13,14,15,16,17], prognosis [18,19], cyber–physical systems [20], hydrologic and geophysical data assimilation [21,22], maritime tracking [23], consensus-based state estimation using wireless sensor networks [24,25,26], navigation systems [27], and transportation and highway traffic management [28,29,30], to mention a few. Depending on the measurements that are used, two algorithms of state estimation can be distinguished: filtering and smoothing. Filtering algorithms estimate the current state using measurements up to the current instant, and smoothing algorithms estimate the state at some time in the past using measurement up to the current instant [23,31].

In general, the experimental noisy data can suffer a significant loss of information introduced by low-resolution and cost-effective sensors [32] in the digitalization process [33]. Typically, the digitalization process encompasses a process known as quantization. Quantization is a nonlinear map that partitions the whole signal space and represents all of the values in each subspace by a single one [32]. In spite of the loss of information, the benefits of quantization have led to a number of applications in which quantized measurements arise. This occurs due to fundamental limitations on measuring equipment and bandwidth resources [34], digital and analog converters [35], and experimental designs where it is necessary to quantize the data in order to store it or minimize communication resource utilization [36]. In particular, estimation problems utilizing quantized measurements arise in networked control over limited-/finite-capacity communication channels, where usually, encoder–decoder state estimation schemes are used [37,38,39,40]. In addition, in order to deal with uncertain dynamic systems, robust estimation algorithms have also been developed; see, e.g., [37,38,41].

Currently, state estimation from quantized data has gained significant attention in a growing number of applications such as fault detection [42,43], networked control [42,44,45], and system identification [35,46,47,48]. For instance, in [49,50], Kalman-based state estimation algorithms were developed using multiple sensors for distributed systems. In [24], a Kalman smoothing algorithm was developed for any-time minimum-mean-squared error optimal-consensus-based state estimation using wireless sensor networks. In [27], an online smoothing algorithm was developed to estimate the positional and orientation parameters of integrated navigation systems utilizing a low-cost microelectromechanical system inertial sensor in near-real time.

In Figure 1, a usual representation of a process is shown, which is defined by the interconnection of three blocks: (1) an actuator, (2) a process, and (3) a sensor. This representation includes a link that can be a communication channel and a base station. The actuator input usually comes from a control system, and the output of the process is measured with noise by a sensor. The sensor introduces quantization to the noisy measurements. This representation has been used for state estimation and control in a microgrid with multiple distributed energy resources, where a dequantizer is used to reconstruct the received signal and then perform the standard Kalman filter [51]. In [42], a fault isolation filter for a discrete-time networked control system with multiple faults was developed using the Kalman filter, where the sensor measurements were transmitted only when the output signal was greater than a threshold. The authors in [52] dealt with a similar structure to the one in Figure 1 to estimate the vehicle sideslip angle of an in-vehicle networked system with sensor failure, dynamic quantization, and data dropouts. For more examples, see, e.g., [15,53].

For linear systems and Gaussian noises (without quantization), the optimal estimator of the system state is the celebrated Kalman filter [1] and smoother [22]. However, in the most general case, i.e., nonlinear systems and non-Gaussian noises, it is not possible to obtain an optimal estimator because the computation of some integrals in the filtering and smoothing equations is difficult or the integrals are just intractable. As mentioned above, the quantization process is a nonlinear map that results in a significant loss of information on the system dynamics, which produces a biased state estimation and incorrect characterization of the filtering and smoothing probability density functions (PDFs). In this context, several suboptimal filtering and smoothing algorithms for state-space systems with quantized data have been developed, for instance, standard- [49,54], unscented- [55], and extended- [36] Kalman filters for quantized data, in which some structural elements of the state-space models and the quantizer are exploited. Sequential Monte Carlo methods have been also used for filtering and smoothing with quantized data, where complex integrals are approximated by a set of weighted samples called particles [31], which define (approximately) a desired PDF. However, the most common difficulty in these methods is dealing with the quantizer model. Some approaches have been proposed for this purpose such as gradient-based approximation of the quantizer [56] or modeling the quantizer as uniformly distributed additive noise [32,57]. In [58], an approximation of the integral in the non-Gaussian probability mass function (PMF) of the quantized output given the state was proposed by using Gaussian quadrature rules in order to deal with binary data. This approximation naturally yields an explicit Gaussian mixture model (GMM) form for the PMF of the quantized output. In this paper, the approximation of this integral was extended by considering a more general (finite- and infinite-level) quantizer, and it was used to develop Gaussian sum filtering and smoothing algorithms.

Main Contributions

The main contributions of this work are:

• Developing an explicit model (of the GMM form) for the PMF of the quantized output considering a finite- and infinite-level quantizer, to solve in closed-form filtering and smoothing recursions in a Bayesian framework;
• Designing Gaussian sum filtering and smoothing algorithms to deal with quantized data, providing closed expressions for the state estimates and for the filtering and smoothing PDFs.

The filtering algorithm for quantized data presented in this paper includes, as a particular case, the filtering algorithm presented in [58], where only the case of binary data was considered. Additionally, the smoothing algorithm presented here, based on the approximation of the PMF of the quantized output given the state, is completely novel. The remainder of the paper is as follows: In Section 2, the problem of interest is defined. In Section 3, the characterization and approximation of the PMF of the quantized data given the state are presented, and using this explicit model, the Gaussian sum filtering and smoothing algorithms are developed. In Section 4, some examples are presented to show the benefits of this proposal in terms of accuracy and computational cost. Finally, in Section 5, conclusions are presented.

2. Statement of the Problem

2.1. System Model

Consider the state-space model for a discrete-time linear-time-invariant system with quantized output (see Figure 2):

$$\begin{array}{}\mathrm{(1)}& \hfill x_{t+1}& =A{x}_t+B{u}_t+w_t,\hfill \mathrm{(2)}& \hfill z_t& =C{x}_t+D{u}_t+v_t,\hfill \mathrm{(3)}& \hfill y_t& =\mathfrak{q}\left\{z_t\right\},\hfill \end{array}$$

where $x_t\in {\mathbb{R}}^n$ is the state vector, $z_t\in \mathbb{R}$ is the nonquantized output, $y_t\in \mathbb{R}$ is the quantized output, and $u_t\in {\mathbb{R}}^m$ is the input of the system. The matrices are $A\in {\mathbb{R}}^{n\times n}$, $B\in {\mathbb{R}}^{n\times m}$, $C\in {\mathbb{R}}^{1\times n}$, and $D\in {\mathbb{R}}^{1\times m}$. The nonlinear map $\mathfrak{q}\left\{·\right\}$ is the quantizer. The state noise $w_t\in {\mathbb{R}}^n$ and the output noise $v_t\in \mathbb{R}$ are zero-mean white Gaussian noises with covariance matrix Q and R, respectively. The system in (1)–(2) can be described using the state transition and conditional nonquantized output probability distributions:

$$\begin{array}{}\mathrm{(4)}& \hfill p\left(x_1\right)& ={\mathcal{N}}_{x_1}({\mu }_1,P_1),\hfill \mathrm{(5)}& \hfill p\left(x_{t+1}\right|x_t)& ={\mathcal{N}}_{x_{t+1}}(A{x}_t+B{u}_t,Q),\hfill \mathrm{(6)}& \hfill p\left(z_t\right|x_t)& ={\mathcal{N}}_{z_t}(C{x}_t+D{u}_t,R),\hfill \end{array}$$

where ${\mathcal{N}}_x(\mu ,P)$ represents a PDF corresponding to a Gaussian distribution with mean $\mu$ and the covariance matrix P of the variable x. The initial condition $x_1$, the model noise $w_t$, and the measurement noise $v_t$ are statistically independent random variables.

2.2. Quantizer Model

Let the quantizer $\mathfrak{q}\left\{·\right\}:\mathbb{R}\to \mathcal{V}$ be a map from the real line defined by the set of intervals $\left\{{\mathcal{J}}_i\subset \mathbb{R}:i\in \mathcal{I}\right\}$ to the finite or countable infinite set $\mathcal{V}=\left\{{\beta }_i\in \mathbb{R}:i\in \mathcal{I}\right\}$ [59], where $\mathcal{I}$ is a set of indices defined by the quantizer type. In this work, two quantizers were considered. The first an infinite-level quantizer, in which the output of the quantizer has infinite (countable) levels of quantization corresponding to the indices’ set:

$$\begin{array}{c}\hfill \mathcal{I}=\left\{\dots ,1,2,\dots ,L,\dots \right\}.\end{array}$$

(7)

The definition of the infinite-level quantizer is as follows (see Figure 3 (left)):

$$\mathfrak{q}\left\{z_t\right\}=\begin{array}{ccccc}{\beta }_i& & \mathrm{if}& & z_t\in {\mathcal{J}}_i,\end{array}i\in \mathcal{I},$$

(8)

where the sets ${\mathcal{J}}_i=\left\{z_t:q_{i-1}\le z_t<q_i\right\}$ are disjoint intervals and each ${\beta }_i$ is the value that the quantizer takes in the region ${\mathcal{J}}_i$. The second is a finite-level quantizer, in which the output of the quantizer is limited to a minimum and maximum values (saturated quantizer) corresponding to the indices’ set:

$$\begin{array}{c}\hfill \mathcal{I}=\left\{1,2,\dots ,L-1,L\right\}.\end{array}$$

(9)

The definition of the finite-level quantizer (see Figure 3 (right)) is similar to (8), where $\mathcal{I}$ is defined in (9), ${\mathcal{J}}_1=\left\{z_t:z_t<q_1\right\}$, ${\mathcal{J}}_L=\left\{z_t:q_{L-1}\le z_t\right\}$, and ${\mathcal{J}}_i=\left\{z_t:q_{i-1}\le z_t<q_i\right\}$ with $i=2\dots ,L-1$.

2.3. Problem Definition

The problem of interest can be defined as follows: Given the available data $u_{1:N}=\left\{u_1,u_2,\dots ,u_N\right\}$ and $y_{1:N}=\left\{y_1,y_2,\dots ,y_N\right\}$, where N is the data length, obtain the filtering and smoothing PDFs of the state given the quantized measurements, $p\left(x_t\right|y_{1:t})$ and $p\left(x_t\right|y_{1:N})$, respectively the state estimators:

$$\begin{array}{}\mathrm{(10)}& \hfill {\widehat{x}}_{t|t}& =\mathbb{E}\left\{x_t|y_{1:t}\right\},\hfill \mathrm{(11)}& \hfill {\widehat{x}}_{t|N}& =\mathbb{E}\left\{x_t|y_{1:N}\right\},\hfill \end{array}$$

and the corresponding covariance matrices of the estimation error:

$$\begin{array}{}\mathrm{(12)}& \hfill {\Sigma }_{t|t}& =\mathbb{E}\left\{(x_t-{\widehat{x}}_{t|t}){(x_t-{\widehat{x}}_{t|t})}^T|y_{1:t}\right\},\hfill \mathrm{(13)}& \hfill {\Sigma }_{t|N}& =\mathbb{E}\left\{(x_t-{\widehat{x}}_{t|N}){(x_t-{\widehat{x}}_{t|N})}^T|y_{1:N}\right\},\hfill \end{array}$$

where $t\le N$ and $\mathbb{E}\left\{x|y\right\}$ denotes the conditional expectation of x given y.

3. Gaussian Sum Filtering and Smoothing for Quantized Data

Here, the Gaussian sum filtering and smoothing algorithms for quantized output data are explained in detail.

3.1. Gaussian Mixture Models

Gaussian mixture models refer to a convex combination of Gaussian densities corresponding to a random variable $\zeta \in {\mathbb{R}}^n$. Then, the PDF can be written as [60]: $p\left(\zeta \right)={\sum }_{i=1}^{\mathcal{K}}{\phi }_i{\mathcal{N}}_{\zeta }\left({\upsilon }_i,{\Gamma }_i\right)$ subject to ${\phi }_i>0$ and ${\sum }_{i=1}^{\mathcal{K}}{\phi }_i=1$, where ${\phi }_i$ is the ith mixing weight, ${\upsilon }_i\in {\mathbb{R}}^n$ is the ith mean, and ${\Gamma }_i\in {\mathbb{R}}^{n\times n}$ is the ith covariance matrix. GMMs are used in a variety of applications to approximate non-Gaussian densities [61,62,63] and filtering and smoothing [64,65], to mention a few.

3.2. General Bayesian Framework

The well-known equations for Bayesian filtering (see, e.g., [31]) are given by:

$$\begin{array}{}\mathrm{(14)}& \hfill p\left(x_t\right|y_{1:t})& ={\displaystyle \frac{p\left(y_t\right|x_t)p\left(x_t\right|y_{1:t-1})}{p\left(y_t\right|y_{1:t-1})}},\hfill \mathrm{(15)}& \hfill p\left(x_{t+1}\right|y_{1:t})& =\int p\left(x_{t+1}\right|x_t)p\left(x_t\right|y_{1:t})d{x}_t,\hfill \end{array}$$

where $p\left(x_t\right|y_{1:t})$ and $p\left(x_{t+1}\right|y_{1:t})$ are the measurement and time-update equations, respectively. The PDF $p\left(x_{t+1}\right|x_t)$ is obtained from the model in (5), and $p\left(y_t\right|y_{1:t-1})$ is a normalization constant. On the other hand, the well-known formula for Bayesian smoothing (see, e.g., [31]) is defined by:

$$p\left(x_t\right|y_{1:N})=p\left(x_t\right|y_{1:t})\int \frac{p\left(x_{t+1}\right|y_{1:N})p\left(x_{t+1}\right|x_t)}{p\left(x_{t+1}\right|y_{1:t})}d{x}_{t+1}.$$

(16)

However, the PDF in (16) is difficult to compute because of the division by a non-Gaussian density. This difficulty was overcome in [64] where the smoothing equations were separated in a two-fold formula filter: (i) the first formula, called the backward filter, defined by the following recursion:

$$\begin{array}{}\mathrm{(17)}& \hfill p\left(y_{t+1:N}\right|x_t)& =\int p\left(y_{t+1:N}\right|x_{t+1})p\left(x_{t+1}\right|x_t)d{x}_{t+1},\hfill \mathrm{(18)}& \hfill p\left(y_{t:N}\right|x_t)& =p\left(y_t\right|x_t)p\left(y_{t+1:N}\right|x_t),\hfill \end{array}$$

where $p\left(y_{t+1:N}\right|x_t)$ and $p\left(y_{t:N}\right|x_t)$ are the backward prediction and the backward- measurement-update equations, respectively, and (ii) the second formula, given by:

$$p\left(x_t\right|y_{1:N})=\frac{p\left(x_t\right|y_{1:t-1})p\left(y_{t:N}\right|x_t)}{p\left(y_{t:N}\right|y_{1:t-1})},$$

(19)

where $p\left(x_t\right|y_{1:t-1})$ is the time-update equation from the filtering step, $p\left(y_{t:N}\right|y_{1:t-1})$ is a normalization constant, and $p\left(y_{t:N}\right|x_t)$ is obtained using the backward recursion given in (17) and (18).

Due to the non-Gaussianity of $p\left(y_t\right|x_t)$, the integrals in both the filtering and backward-filtering algorithms in (15) and (17), respectively, are difficult to compute or intractable. Widely used methods to deal with these integrals are Monte-Carlo-based algorithms, such as particle filtering and smoothing; see, e.g., [31,66]. These methods represent the posterior distributions $p\left(x_t\right|y_{1:t})$ and $p\left(x_t\right|y_{1:N})$ by a set of weighted random samples. However, in general, they exhibit a growing computational complexity as the model dimension increases. Here, an alternative method to compute the PMF $p\left(y_t\right|x_t)$ using Gauss–Legendre quadrature is proposed. This procedure results in a GMM form. Hence, the integrals in both the filtering and backward filtering in (15) and (17), respectively, can be computed in closed form.

Remark 1.

Notice that since $y_t$ is a discrete random variable, the measurement-update equation in (14) and the backward-measurement-update equation in (18) comprise PDFs and a PMF. Hence, generalized probability density functions are used here; see, e.g., [67].

3.3. Computing an Explicit Model of $p\left(y_t\right|x_t)$

From (8), it is observed that the output $z_t\in {\mathcal{J}}_i$ is mapped to a single value ${\beta }_i$. Then, the probability that $y_t$ takes the value ${\beta }_i$ is the same as the probability of $z_t$ belonging to set ${\mathcal{J}}_i$, as shown in Figure 4. Notice that the quantizer regions ${\mathcal{J}}_i$ include finite and semi-infinite intervals.

In the following theorem, the characterization of $p\left(y_t\right|x_t)$ is formalized via the computation of the probability $\mathbb{P}\left\{z_t\in {\mathcal{J}}_i\right\}$ shown in Figure 4 through the integral definition of probabilities [68]. Therefore, the approximation of this integral by using the Gauss–Legendre quadrature rule is presented (see, e.g., [69]).

Theorem 1.

Consider the system (1)–(3), the infinite- and finite-level quantizers defined in (8). Then, the PMF of the discrete random variable $y_t$ given the state $x_t$ is given by:

$$p\left(y_t\right|x_t)={\int }_{a_t}^{b_t}{\mathcal{N}}_{v_t}\left(0,R\right)d{v}_t,$$

(20)

where $a_t$ and $b_t$ are functions of the boundary values of each region of the quantizers and are defined in

Table 1. Integral limits and parameters of Theorem 1. ILQ and FLQ mean infinite- and finite-level quantizers, respectively.

	FLQ: ${\mathit{y}}_{\mathit{t}}={\mathit{\beta }}_1$	ILQ: ${\mathit{y}}_{\mathit{t}}={\mathit{\beta }}_{\mathit{i}}$ with $\mathit{i}\mathit{\in }\mathcal{I}$ in (7) FLQ: ${\mathit{y}}_{\mathit{t}}={\mathit{\beta }}_{\mathit{i}}$ with $\mathit{i}=2,\dots ,\mathit{L}-\mathit{1}$	FLQ: ${\mathit{y}}_{\mathit{t}}={\mathit{\beta }}_{\mathit{L}}$
$a_t$	$-\infty$	$q_{i-1}-C{x}_t-D{u}_t$	$q_{L-1}-C{x}_t-D{u}_t$
$b_t$	$q_1-C{x}_t-D{u}_t$	$q_i-C{x}_t-D{u}_t$	∞
${\varsigma }_t^{\tau }$	$2{\omega }_{\tau }/{(1+{\psi }_{\tau })}^2$	${\omega }_{\tau }(q_i-q_{i-1})/2$	$2{\omega }_{\tau }/{(1+{\psi }_{\tau })}^2$
${\eta }_t^{\tau }$	$-(1-{\psi }_{\tau })/(1+{\psi }_{\tau })$	${\psi }_{\tau }(q_i-q_{i-1})/2$	$(1-{\psi }_{\tau })/(1+{\psi }_{\tau })$
${\mu }_t^{\tau }$	$-q_1$	$-(q_i+q_{i-1})/2$	$-q_{L-1}$

. In addition, the integral in (20) can be approximated using the Gauss–Legendre quadrature rule, yielding:

$$p\left(y_t\right|x_t)\approx \sum _{\tau =1}^K{\varsigma }_t^{\tau }{\mathcal{N}}_{{\eta }_t^{\tau }}\left(C{x}_t+D{u}_t+{\mu }_t^{\tau },R\right),$$

(21)

where K is the number of points from the Gauss–Legendre quadrature rule (defined by the user), ${\varsigma }_t^{\tau }$, ${\eta }_t^{\tau }$, and ${\mu }_t^{\tau }$ are defined in Table 1, and ${\omega }_{\tau }$ and ${\psi }_{\tau }$ are weights and points defined by the quadrature rule, given in, e.g., [69].

Proof. 

From the infinite- and finite-level quantizers, it is observed that the random variable $y_t$ can only take the values ${\beta }_i$ with i in the indices’ sets given in (7) and (9). Then, the probability of $y_t=\dots ,{\beta }_1,{\beta }_2,\dots ,{\beta }_L,\dots$ or $y_t={\beta }_1,{\beta }_2,\dots ,{\beta }_{L-1},{\beta }_L$ is the same as the probability that the random variable $z_t$ belongs to the sets ${\mathcal{J}}_i$. This probability can be obtained from the distribution function as follows:

$$\mathbb{P}\left\{y_t={\beta }_i|x_t\right\}=\mathbb{P}\left\{z_t\in {\mathcal{J}}_i|x_t\right\},$$

(22)

where $\mathbb{P}\left\{·\right\}$ denotes probability. Considering the infinite-level quantizer and the output equation in (2), the following expressions are obtained for $y_t={\beta }_i$ with $i=\dots ,1,2\dots ,L,\dots$:

$$\begin{array}{cc}\hfill \mathbb{P}\left\{y_t={\beta }_i|x_t\right\}& =\mathbb{P}\left\{q_{i-1}\le z_t<q_i|x_t\right\},\hfill \\ \hfill & =\mathbb{P}\left\{a_t\le v_t<b_t|x_t\right\},\hfill \end{array}$$

(23)

where $a_t=q_{i-1}-C{x}_t-D{u}_t$ and $b_t=q_i-C{x}_t-D{u}_t$. Additionally, for the finite-level quantizer, (23) holds for $y_t={\beta }_i$ with $i=2,\dots ,L-1$, and for $y_t={\beta }_1$ and $y_t={\beta }_L$, the following holds:

$$\begin{array}{}\mathrm{(24)}& \hfill \mathbb{P}\left\{v_t<b_t|x_t\right\}& \mathrm{if}{y}_t={\beta }_1,\hfill \mathrm{(25)}& \hfill \mathbb{P}\left\{v_t\ge a_t|x_t\right\}& \mathrm{if}{y}_t={\beta }_L,\hfill \end{array}$$

where $b_t=q_1-C{x}_t-D{u}_t$ and $a_t=q_{L-1}-C{x}_t-D{u}_t$. Then, using the fact that $p\left(v_t\right)={\mathcal{N}}_{v_t}(0,R)$ and using (23)–(25), the integral in (20) can be obtained, where the integral limits are given in Table 1.

On the other hand, the Gauss–Legendre quadrature for approximating a definite integral over the interval $\left[-1,1\right]$ is given by:

$${\int }_{-1}^{1}f\left(r\right)dr\approx \sum _{\tau =1}^K{\omega }_{\tau }f\left({\psi }_{\tau }\right),$$

(26)

where ${\omega }_{\tau }$ and ${\psi }_{\tau }$ are the quadrature weights and the roots of the order K Legendre polynomial, respectively; see, e.g., [69]. Notice that the integral over $\left[a_t,b_t\right]$ can be mapped onto the interval $\left[-1,1\right]$ using:

$${\int }_{a_t}^{b_t}f\left(r\right)dr=\frac{b_t-a_t}{2}{\int }_{-1}^{1}f\left(\frac{b_t-a_t}{2}r+\frac{b_t+a_t}{2}\right)dr,$$

(27)

and using the definition in (26), this integral is approximated by:

$${\int }_{a_t}^{b_t}f\left(r\right)dr\approx \frac{b_t-a_t}{2}\sum _{\tau =1}^K{\omega }_{\tau }f\left(\frac{b_t-a_t}{2}{\psi }_{\tau }+\frac{b_t+a_t}{2}\right).$$

(28)

Defining $a_t=q_{i-1}-C{x}_t-D{u}_t$, $b_t=q_i-C{x}_t-D{u}_t$, and $f\left(v_t\right)={\mathcal{N}}_{v_t}\left(0,R\right)$ with $i=\dots ,1,2\dots ,L,\dots$, the approximation of $p\left(y_t\right|x_t)$ given in (21) for the infinite-level quantizer is derived.

For the finite-level quantizer, (28) holds for $y_t={\beta }_i$ with $i=2,\dots ,L-1$. Then, the approximation of $p\left(y_t\right|x_t)$ for $y_t={\beta }_1$ and $y_t={\beta }_L$ is defined. First, it is observed that the integral over the semi-infinite interval $[a_t,\infty )$ can be mapped onto the interval $(0,1]$ using $r=a_t+(1-\xi )/\xi$, so that:

$${\int }_{a_t}^{\infty }f\left(r\right)dr={\int }_0^{1}f\left(a_t+\frac{1-\xi }{\xi }\right)\frac{d\xi }{{\xi }^2}.$$

(29)

Then, using an appropriate change of variable, it can be mapped onto the interval $(-1,1]$, yielding:

$${\int }_{a_t}^{\infty }f\left(r\right)dr={\int }_{-1}^{1}f\left(a_t+\frac{1-s}{1+s}\right)\frac{2ds}{{(1+s)}^2}.$$

(30)

Using the Gauss–Legendre approximation given in (26), the approximation of $p\left(y_t\right|x_t)$ for $y_t={\beta }_L$ is obtained as follows:

$${\int }_{a_t}^{\infty }f\left(r\right)dr\approx \sum _{\tau =1}^K{\omega }_{\tau }f\left(a_t+\frac{1-{\psi }_{\tau }}{1+{\psi }_{\tau }}\right)\frac{2}{{(1+{\psi }_{\tau })}^2}.$$

(31)

Defining $a_t=q_{L-1}-C{x}_t-D{u}_t$, $b=\infty$, and $f\left(v_t\right)={\mathcal{N}}_{v_t}\left(0,R\right)$, (21) is obtained. A similar procedure for the integral over the semi-infinite interval $(-\infty ,b_t]$ can be applied using $r=b_t-(1-\xi )/\xi$ to find the approximation of $p\left(y_t\right|x_t)$ for $y_t={\beta }_1$, as follows:

$${\int }_{-\infty }^{b_t}f\left(r\right)dr\approx \sum _{\tau =1}^K{\omega }_{\tau }f\left(b_t-\frac{1-{\psi }_{\tau }}{1+{\psi }_{\tau }}\right)\frac{2}{{(1+{\psi }_{\tau })}^2}.$$

(32)

Defining $a_t=-\infty$, $b_t=q_1-C{x}_t-D{u}_t$, and $f\left(v_t\right)={\mathcal{N}}_{v_t}\left(0,R\right)$, (21) is obtained. This completes the proof.   □

Remark 2.

Notice that any quadrature rule, such as Newton–Cotes, Gauss–Laguerre, and Gauss–Hermite (see, e.g., [69,70]), used to approximate the integral in (20), yields a weighted sum of Gaussian distributions evaluated as a linear function of the values $q_i$ and the state $x_t$. Furthermore, it is possible to interpret $p\left(y_t\right|x_t)$ as a weighted sum of Gaussian distributions in the random variable $x_t$. This weighted sum is denoted as the GMM structure. Thus, this structure is considered for developing the Gaussian sum filtering and smoothing algorithms that deal with quantized data.

Remark 3.

Notice that in [70], a suboptimal filtering algorithm called the quadrature Kalman filter, where a linear regression is used to linearize the nonlinear process and measurement functions using the Gauss–Hermite quadrature rule, was developed. This approach is not directly applicable to the problem of interest in this paper, so that the quantizer is a nondifferentiable nonlinearity.

On the other hand, the cubature Kalman filter [70] and smoother [71] are approaches that use the spherical–radial cubature rule to approximate the n-dimensional integrals when computing the expected values in (10)–(13) in a nonlinear state-space model. These integral approximations are obtained under the assumption that $p\left(x_t\right|y_{1:t})$ and $p\left(x_t\right|y_{1:N})$ are Gaussian distributed. The difference between these approaches and the proposed method in this paper is that the Gauss quadrature rule was used to approximate the integral in the probabilistic model of $p\left(y_t\right|x_t)$. It is clear that in this paper, the Gaussian assumption in the filtering and smoothing densities was not used. In fact, it is shown that $p\left(x_t\right|y_{1:t})$ and $p\left(x_t\right|y_{1:N})$ are non-Gaussian PDFs.

3.4. Understanding the Gaussian Sum Filtering and Smoothing Algorithms

The general Bayesian framework for filtering leads to the product $p\left(y_t\right|x_t)$ and $p\left(x_t\right|y_{1:t-1})$, as shown in (14). From the definition of $p\left(y_t\right|x_t)$ in (21), it is observed that $p\left(x_t\right|y_{1:t})$ results in the product of two GMMs. This, in turn, results in a GMM with more components than the individual factors $p\left(y_t\right|x_t)$ and $p\left(x_t\right|y_{1:t-1})$. This implies that the time-update equation $p\left(x_{t+1}\right|y_{1:t})$ in (15) is also a GMM distribution. A similar situation occurs in the backward-measurement-update equation $p\left(y_{t:N}\right|x_t)$ in (18), which is the product of two GMM structures $p\left(y_t\right|x_t)$ and $p\left(y_{t+1:N}\right|x_t)$. This yields another GMM structure with more components than the individual factors, which implies that the backward-prediction equation in (17) is also a GMM structure. For the clarity of the presentation, reducing the product of two GMMs (structures) into one is necessary.

In order to understand the transformation of the product of two summations into another summation, the following sums are considered: $g={\sum }_{\tau =1}^K{g}_{\tau }$ and $f={\sum }_{\ell =1}^M{f}_{\ell }$. Then, for each two-tuple $(\tau ,\ell )$ where $\tau =1,\dots ,K$ and $\ell =1,\dots ,M$, the product $h=fg$ is another summation and has $KM$ terms indexed by $k=(\ell -1)K+\tau$. Then, reordering these terms, the following sum is obtained: $h={\sum }_{k=1}^{KM}{h}_k$, where $h_k=f_{\ell }{g}_{\tau }$.

3.5. Gaussian Sum Filtering for Quantized Data

Using the approximation of the PMF $p\left(y_t\right|x_t)$ defined in Theorem 1, the Gaussian sum filtering algorithm can be derived using (14) and (15) as follows:

Theorem 2 (Gaussian sum filter).

Consider the system in (4)–(6) and the approximation of $p\left(y_t\right|x_t)$ in (21). The filtering equations for state-space systems with quantized output data are defined as follows:

Initialization: For $t=1$, the predicted distribution is $p\left(x_1\right)={\mathcal{N}}_{x_1}({\mu }_1,P_1)$, where $p\left(x_1\right)$ is the prior distribution of the initial state. Then, for $t=1,\dots ,N$, the measurement-update and the time-update equations for the Gaussian sum filtering are defined as follows:

Measurement update: The PDF of the current state $x_t$ given the current and previous measurements, that is $y_1,\dots ,y_t$, is the GMM given by:

$$p\left(x_t\right|y_{1:t})=\sum _{k=1}^{M_{t|t}}{\gamma }_{t|t}^k{\mathcal{N}}_{x_t}({\widehat{x}}_{t|t}^k,{\Sigma }_{t|t}^k),$$

(33)

where ${\gamma }_{t|t}^k$, ${\widehat{x}}_{t|t}^k$, and ${\Sigma }_{t|t}^k$ are given in Appendix B, $M_{t|t}=K{M}_{t|t-1}$, and $M_{t|t-1}$ is the number of Gaussian components in the time update step. The initial values satisfy that $M_{1|0}=1$, ${\gamma }_{1|0}=1$, ${\widehat{x}}_{1|0}={\mu }_1$, and ${\Sigma }_{1|0}=P_1$.

Time update: The PDF of the future state $x_{t+1}$, one-step-ahead prediction given the measurements $y_1,\dots ,y_t$, is the GMM given by:

$$p\left(x_{t+1}\right|y_{1:t})=\sum _{k=1}^{M_{t+1|t}}{\gamma }_{t+1|t}^k{\mathcal{N}}_{x_{t+1}}({\widehat{x}}_{t+1|t}^k,{\Sigma }_{t+1|t}^k),$$

(34)

where $M_{t+1|t}=M_{t|t}$, ${\gamma }_{t+1|t}^k$, ${\widehat{x}}_{t+1|t}^k$, and ${\Sigma }_{t+1|t}^k$ are given in Appendix B.

Proof. 

Consider the recursion in (14) and (15). The required PDF $p\left(x_{t+1}\right|x_t)$ and PMF $p\left(y_t\right|x_t)$ are obtained from (5) and (21), respectively. Then, using the measurement-update equations in (14) and Lemma 2 in [65] (p. 86), the following expression is obtained:

$$p\left(x_t\right|y_{1:t})\propto \sum _{\tau =1}^K\sum _{\ell =1}^{M_{t|t-1}}{\varsigma }_t^{\tau }{\gamma }_{t|t-1}^{\ell }{\mathcal{N}}_{{\eta }_t^{\tau }}\left({\kappa }_t^{\ell \tau },V_t^{\ell }\right){\mathcal{N}}_{x_t}({\widehat{x}}_{t|t}^k,{\Sigma }_{t|t}^k)$$

(35)

where ${\kappa }_t^{\ell \tau }$, $V_t^{\ell }$, ${\widehat{x}}_{t|t}^k$, and ${\Sigma }_{t|t}^k$ are defined in (A11), (A12), (A7), and (A8), respectively. Notice that ${\mathcal{N}}_{{\eta }_t^{\tau }}({\kappa }_t^{\ell \tau },V_t^{\ell })$ is a coefficient since the measurement $y_t$ is available. Then, rewriting the double summation in (35) as a single one with a new index $k=(\ell -1)K+\tau$, the measurement-update equation in (33) is derived defining $M_{t|t}=K{M}_{t|t-1}$, the weights ${\overline{\gamma }}_{t|t}^k$ as in (A9), and computing the corresponding normalization constant. On the other hand, using the time-update equation in (15) and Lemma 2 in [65] (p. 86), the following equation is obtained:

$$p\left(x_{t+1}\right|y_{1:t})=\sum _{k=1}^{M_{t|t}}{\gamma }_{t|t}^k\int {\mathcal{N}}_{x_t}\left(m_t^k,S_t^k\right){\mathcal{N}}_{x_{t+1}}\left({\widehat{x}}_{t+1|t}^k,{\Sigma }_{t+1|t}^k\right)d{x}_t,$$

(36)

where ${\widehat{x}}_{t+1|t}^k$ and ${\Sigma }_{t+1|t}^k$ are defined in (A14), and (A15), respectively. Then, solving this integral, the time-update equation in (34) can be derived defining $M_{t+1|t}=M_{t|t}$, the weights ${\gamma }_{t+1|t}^k$ as in (A13), and computing the corresponding normalization constant. This completes the proof.   □

3.6. Computing the State Estimator ${\widehat{x}}_{t|t}$ from a GMM

Provided $p\left(x_t\right|y_{1:t})$ in (33) as a GMM, the state estimator given in (10) and the covariance matrix of the estimation error in (12) can be computed as follows:

$$\begin{array}{}\mathrm{(37)}& \hfill {\widehat{x}}_{t|t}& =\sum _{k=1}^{M_{t|t}}{\gamma }_{t|t}^k{\widehat{x}}_{t|t}^k,\hfill \mathrm{(38)}& \hfill {\Sigma }_{t|t}& =\sum _{k=1}^{M_{t|t}}{\gamma }_{t|t}^k\left[{\Sigma }_{t|t}^k+({\widehat{x}}_{t|t}^k-{\widehat{x}}_{t|t}){({\widehat{x}}_{t|t}^k-{\widehat{x}}_{t|t})}^T\right].\hfill \end{array}$$

3.7. Backward Filtering for Quantized Data

Using the approximation of the PMF $p\left(y_t\right|x_t)$ defined in Theorem 1, the backward filter recursion can be derived as follows:

Theorem 3 (Backward filtering).

Consider the system in (4)–(6) and the approximation of $p\left(y_t\right|x_t)$ in (21). Then, the backward filter for state-space systems with quantized output data is:

Initialization: For $t=N$, the backward measurement update is given by:

$$\begin{array}{cc}\hfill & p\left(y_N\right|x_N)=\sum _{k=1}^{S_{N|N}}{ϵ}_{N|N}^k{\lambda }_{N|N}^{k}exp\left\{-\frac{1}{2}\left(x_N^T{F}_{N|N}^k{x}_N-2G_{N|N}^{kT}{x}_N+H_{N|N}^k\right)\right\},\hfill \end{array}$$

(39)

where $S_{N|N}=K$, and: 

$$\begin{array}{cc}\hfill {ϵ}_{N|N}^k& ={\varsigma }_N^k,\hfill \\ \hfill F_{N|N}^k& =C^T{R}^{-1}C,\hfill \end{array}\begin{array}{cc}\hfill {\lambda }_{N|N}^k& ={\left(det\left\{2\pi R\right\}\right)}^{-1/2}\hfill \\ \hfill G_{N|N}^{kT}& ={\theta }_N^{kT}{R}^{-1}C,\hfill \end{array}\begin{array}{cc}\hfill {\theta }_N^k& ={\eta }_N^k-D{u}_N-{\mu }_N^k,\hfill \\ \hfill H_{N|N}^k& ={\theta }_N^{kT}{R}^{-1}{\theta }_N^k,\hfill \end{array}$$

(40)

where ${\varsigma }_N^k$, ${\eta }_N^k$, and ${\mu }_N^k$ are defined in Table 1. Then, for $t=N-1,\dots ,1$, the backward prediction and the backward-measurement-update equations are defined as follows:

Backward predictions: The backward-prediction equation is given by:

$$\begin{array}{cc}\hfill & p\left(y_{t+1:N}\right|x_t)=\sum _{k=1}^{S_{t|t+1}}{ϵ}_{t|t+1}^k{\lambda }_{t|t+1}^{k}exp\left\{-\frac{1}{2}\left(x_t^T{F}_{t|t+1}^k{x}_t-2G_{t|t+1}^{kT}{x}_t+H_{t|t+1}^k\right)\right\},\hfill \end{array}$$

(41)

where $S_{t|t+1}=S_{t+1|t+1}$, $S_{t+1|t+1}$ is the number of components in the backward-measurement-update step, and ${ϵ}_{t|t+1}^k$, ${\lambda }_{t|t+1}^k$, $F_{t|t+1}^k$, $G_{t|t+1}^{kT}$, $H_{t|t+1}^k$ are given in Appendix C.

Backward-measurement update: The distribution of $y_{t:N}|x_t$ evaluated at $y_t,\dots ,y_N$ is:

$$\begin{array}{cc}\hfill & p\left(y_{t:N}\right|x_t)=\sum _{k=1}^{S_{t|t}}{ϵ}_{t|t}^k{\lambda }_{t|t}^{k}exp\left\{-{\displaystyle \frac{1}{2}}\left(x_t^T{F}_{t|t}^k{x}_t-2G_{t|t}^{kT}{x}_t+H_{t|t}^k\right)\right\},\hfill \end{array}$$

(42)

where $S_{t|t}=K{S}_{t|t+1}$, $S_{t|t+1}$ is the number of components in the backward-prediction step, and ${ϵ}_{t|t}^k$, ${\lambda }_{t|t}^k$, $F_{t|t}^k$, $G_{t|t}^{kT}$, $H_{t|t}^k$ are given in Appendix C.

Proof. 

Consider the recursion in (17) and (18). The required PDF $p\left(x_{t+1}\right|x_t)$ and PMF $p\left(y_t\right|x_t)$ are given by Equations (5) and (21), respectively. The proof is carried out by induction in reverse time. First, it is verified that it holds for $t=N-1$, then it is assumed to be true for $t=s+1$, and finally, it is verified that it holds for $t=s$. Notice that the recursion starts in $t=N$ with $p\left(y_{N:N}\right|x_N)=p\left(y_N\right|x_N)$, which is defined in (39). From (17), at time $t=N-1$, the backward-prediction step is defined as:

$$p\left(y_{N:N}\right|x_{N-1})=\int p\left(y_{N:N}\right|x_N)p\left(x_N\right|x_{N-1})d{x}_N,$$

(43)

where $p\left(y_{N:N}\right|x_N)$ is given by (39) and $p\left(x_N\right|x_{N-1})$ is defined by the system model in (5). From the definition given in (43), the equation below is obtained:

$$\begin{array}{c}\hfill p\left(y_{N:N}\right|x_{N-1})=\sum _{k=1}^{S_{N|N}}\frac{{ϵ}_{N|N}^k{\lambda }_{N|N}^k}{\sqrt{det\left\{2\pi Q\right\}}}\int exp\left\{-\frac{1}{2}\left[x_N^T{\overline{P}}_N^k{x}_N-2{\overline{V}}_N^{kT}{x}_N+{\overline{S}}_N^k\right]\right\}d{x}_N,\end{array}$$

(44)

where ${\overline{P}}_N^k=\left(F_{N|N}^k+Q^{-1}\right)$, ${\overline{V}}_N^k=\left(G_{N|N}^k+J_{N-1}\right)$, ${\overline{S}}_N^k=\left(H_{N|N}^k+L_{N-1}\right)$, $J_{N-1}^T={(A{x}_{N-1}+B{u}_{N-1})}^T{Q}^{-1}$, and $L_{N-1}={(A{x}_{N-1}+B{u}_{N-1})}^T{Q}^{-1}(A{x}_{N-1}+B{u}_{N-1})$. Then, completing the square and solving the integral in (44), it is obtained:

$$\begin{array}{cc}\hfill p\left(y_{N:N}\right|x_{N-1})& =\sum _{k=1}^{S_{N-1|N}}{ϵ}_{N-1|N}^k{\lambda }_{N-1|N}^k\hfill \\ & \times exp\left\{-{\displaystyle \frac{1}{2}}\left(x_{N-1}^T{F}_{N-1|N}^k{x}_{N-1}-2G_{N-1|N}^{kT}{x}_{N-1}+H_{N-1|N}^k\right)\right\},\hfill \end{array}$$

(45)

where $S_{N-1|N}=S_{N|N}$ and the remaining terms in (45) are defined in (A22)–(A26), but evaluated at $t=N-1$. The backward-measurement-update step in (18) is as follows:

$$p\left(y_{N-1:N}\right|x_{N-1})=p\left(y_{N-1}\right|x_{N-1})p\left(y_{N:N}\right|x_{N-1}).$$

(46)

Thus, using (45) in the definition given in (46), it is obtained:

$$\begin{array}{cc}\hfill p\left(y_{N-1:N}\right|x_{N-1})& =\sum _{\tau =1}^K\sum _{\ell =1}^{S_{N-1|N}}\frac{{\varsigma }_{N-1}^{\tau }{ϵ}_{N-1|N}^{\ell }{\lambda }_{N-1|N}^{\ell }}{\sqrt{det\left\{2\pi R\right\}}}\hfill \\ \hfill & \times exp\left\{-\frac{1}{2}\left(x_{N-1}^T{\tilde{F}}_{N-1}{x}_{N-1}-2{\tilde{G}}_{N-1}^{\tau T}{x}_{N-1}+{\tilde{H}}_{N-1}^{\tau }\right)\right\}\hfill \\ \hfill & \times exp\left\{-{\displaystyle \frac{1}{2}}\left(x_{N-1}^T{F}_{N-1|N}^{\ell }{x}_{N-1}-2G_{N-1|N}^{\ell T}{x}_{N-1}+H_{N-1|N}^{\ell }\right)\right\},\hfill \end{array}$$

(47)

where ${\tilde{F}}_{N-1}=C^T{R}^{-1}C$, ${\tilde{G}}_{N-1}^{\tau T}={\theta }_{N-1}^{\tau T}{R}^{-1}C$, and ${\tilde{H}}_{N-1}^{\tau }={\theta }_{N-1}^{\tau T}{R}^{-1}{\theta }_{N-1}^{\tau }$ with ${\theta }_{N-1}^{\tau }={\eta }_{N-1}^{\tau }-D{u}_{N-1}-{\mu }_{N-1}^{\tau }$. Finally, rewriting the double summation in the last equation into a single one with a new index $k=(\ell -1)K+\tau$ results in:

$$\begin{array}{cc}\hfill & p\left(y_{N-1:N}\right|x_{N-1})=\sum _{k=1}^{S_{N-1|N-1}}{ϵ}_{N-1|N-1}^k{\lambda }_{N-1|N-1}^k\hfill \\ \hfill & \times exp\left\{-\frac{1}{2}\left[x_{N-1}^T{F}_{N-1|N-1}^k{x}_{N-1}-2G_{N-1|N-1}^{kT}{x}_{N-1}+H_{N-1|N-1}^k\right]\right\},\hfill \end{array}$$

(48)

where $S_{N-1|N-1}=K{S}_{N-1|N}$ and the remaining terms in (48) are the same as the ones shown in (A16)–(A21), but evaluated at $t=N-1$. Thus, it is concluded that it holds for $t=N-1$. A similar procedure was applied to find that, for both the backward-prediction and backward-measurement-update steps in the backward filter, it yields the same expressions in (A22)–(A26) and (A16)–(A21), but evaluated at $t=s$. Thus, it is concluded that Theorem 3 holds for all t.    

From Theorems 2 and 3, it is clear that the number of elements in the mixture grows exponentially with every iteration, making the algorithm computationally intractable after a few iterations. In addition, it would be necessary to save and manage a large amount of information in every iteration of the Gaussian sum filtering and in the backward recursion. Therefore, an algorithm that reduces the number of GMM components should be implemented in every iteration of these two algorithms in order to keep the number of components bounded. Different methods have been proposed to perform this kind of procedure, termed Gaussian sum reduction, such us pruning, joining, and integral-squared-error-based methods; see [70]. In this work, the Kullback–Leibler approach for Gaussian sum reduction proposed in [64,72] was used. The idea behind the Gaussian sum reduction is to transform the GMM ${\left\{{\phi }_i,{\upsilon }_i,{\Gamma }_i\right\}}_{i=1}^J$ into a GMM ${\left\{{\phi }_i,{\upsilon }_i,{\Gamma }_i\right\}}_{i=1}^S$, where $1\le S\le J$. In [64], it was suggested to use a measure of dissimilarity between two components and pooling the pair of components that minimize this measure. Then, based on this idea, in [72], the Kullback–Leibler information number was used as the measure of dissimilarity. The author in [72] provided an algorithm to merge two components so that the merged component preserves the first- and second-order moments of the original two components, which is given by:

$$\left({\phi }_{ij},{\upsilon }_{ij},{\Gamma }_{ij}\right)=\mathcal{M}\left\{\left({\phi }_i,{\upsilon }_i,{\Gamma }_i\right),\left({\phi }_j,{\upsilon }_j,{\Gamma }_j\right)\right\},$$

(49)

where $\mathcal{M}\left\{·,·\right\}$ is a merging function such that:

$$\begin{array}{}\mathrm{(50)}& \hfill {\phi }_{ij}& ={\phi }_i+{\phi }_j,\hfill \mathrm{(51)}& \hfill {\upsilon }_{ij}& ={\overline{\phi }}_i{\upsilon }_i+{\overline{\phi }}_j{\upsilon }_j,\hfill \mathrm{(52)}& \hfill {\Gamma }_{ij}& ={\overline{\phi }}_i{\Gamma }_i+{\overline{\phi }}_j{\Gamma }_j+{\overline{\phi }}_i{\overline{\phi }}_j\left({\upsilon }_i-{\upsilon }_j\right){\left({\upsilon }_i-{\upsilon }_j\right)}^T,\hfill \end{array}$$

where ${\overline{\phi }}_i={\phi }_i/({\phi }_i+{\phi }_j)$ and ${\overline{\phi }}_j={\phi }_j/({\phi }_i+{\phi }_j)$. The merging function applied to two components i and j ($i\ne j$) minimizes the dissimilarity measure $\mathcal{D}(i,j)$, defined as:

$$\begin{array}{c}\hfill \mathcal{D}(i,j)={\displaystyle \frac{1}{2}}\left[{\phi }_{ij}logdet\left\{{\Gamma }_{ij}\right\}-{\phi }_{i}logdet\left\{{\Gamma }_i\right\}-{\phi }_{j}logdet\left\{{\Gamma }_j\right\}\right],\end{array}$$

(53)

The function $\mathcal{D}(i,j)$ satisfies $\mathcal{D}(i,j)=\mathcal{D}(j,i)$ and $\mathcal{D}(i,i)=0$. This implies that the total number of combinations to merge is reduced to $0.5J(J-1)$. The authors in [73] used Runnalls’ algorithm in a Bayesian filtering environment, and they modified it to include a user-defined threshold for the number of components after reduction and a user-defined threshold $\epsilon$ that satisfies $\mathcal{D}(i,j)<\epsilon$. In Algorithm 1, the steps for implementing the Gaussian sum filtering are summarized.

Algorithm 1 Gaussian sum filter algorithm for quantized output data

1 Input: The PDF of the initial state $p\left(x_1\right)$, e.g., $M_{1|0}=1$, ${\gamma }_{1|0}=1$, ${\widehat{x}}_{1|0}={\mu }_1$,

${\Sigma }_{1|0}=P_1$. The points of the Gauss–Legendre quadrature ${\left\{{\omega }_{\tau },{\psi }_{\tau }\right\}}_{\tau =1}^K$.

20 Output: The state estimation in (10), the covariance matrix of the estimation error

in (12), the filtering PDFs $p\left(x_t\right|y_{1:t})$, the predictive PDFs $p\left(x_{t+1}\right|y_{1:t})$, and the set

$\left\{{\varsigma }_t^{\tau },{\eta }_t^{\tau },{\mu }_t^{\tau }\right\}$, for $t=1,\dots ,N$.

The backward recursion in Theorem 3 is used to obtain the smoothing PDF in (19). For this purpose, $p\left(y_{t:N}\right|x_t)$ is converted into a GMM structure of the random variable $x_t$. Then, the Gaussian sum reduction algorithm is applied to the GMM structure of $p\left(y_{t:N}\right|x_t)$ to obtain:

$$p\left(y_{t:N}\right|x_t)=\sum _{k=1}^{S_{\mathrm{red}}}{\delta }_{t|t}^k{\mathcal{N}}_{x_t}\left(z_{t|t}^k,U_{t|t}^k\right),$$

(54)

where $S_{\mathrm{red}}$ is the number of Gaussian components kept after the Gaussian reduction procedure and ${\delta }_{t|t}^k$, $z_{t|t}^k$, and $U_{t|t}^k$ are the corresponding weight, mean, and covariance matrix. This reduced GMM structure is used to obtain the smoothing PDFs. However, for the next recursion in the backward filter, reconverting the reduced GMM structure into the backward filter form to obtain the reduced version of the backward-measurement-update step in (42) is required. This conversion process between the backward filter and GMM structure is summarized in Lemma A3 in Appendix A. In Algorithm 2, the steps for implementing the backward filter are summarized.

Algorithm 2 Backward-filtering algorithm for quantized output data

1  Input: The initial backward measurement $p\left(y_N\right|x_N)$ given in (39), e.g., $S_{N|N}$, ${ϵ}_{N|N}^k$,

${\lambda }_{N|N}^k$, $F_{N|N}^k$, $G_{N|N}^{kT}$, and $H_{N|N}^k$. The set $\left\{{\varsigma }_t^{\tau },{\eta }_t^{\tau },{\mu }_t^{\tau }\right\}$ for $t=1,\dots ,N$ computed in Algorithm 1.

19  Output: The backward prediction $p\left(y_{t+1:N}\right|x_t)$ and the backward measurement

update $p\left(y_{t:N}\right|x_t)$ for $t=N,\dots ,1$.

3.8. Smoothing Algorithm with Quantized Data

In order to obtain the smoothing PDF in (19), the GMM structure $p\left(y_{t:N}\right|x_t)$ in the random variable $x_t$ given in (54) is used. This GMM structure is multiplied by the time-update equation $p\left(x_t\right|y_{1:t-1})$ of the filtering algorithm given in (34). Then, the smoothing PDF is obtained from the following:

Theorem 4 (Gaussian sum smoothing).

Consider the system in (4)–(6). Given $p\left(x_t\right|y_{1:t-1})$, $p\left(x_N\right|y_{1:N})$ and $p\left(y_{t:N}\right|x_t)$, then the smoothing PDF at time $t=N$ is given by $p\left(x_N\right|y_{1:N})$, and for $t=N-1,\dots ,1$, the PDF $p\left(x_t\right|y_{1:N})$ is a GMM given by:

$$p\left(x_t\right|y_{1:N})=\sum _{k=1}^{S_{t|N}}{ϵ}_{t|N}^k{\mathcal{N}}_{x_t}({\widehat{x}}_{t|N}^k,{\Sigma }_{t|N}^k),$$

(55)

where $S_{t|N}=M_{t|t-1}{S}_{red}$, $M_{t|t-1}$ and $S_{red}$ are given in (34) and (54), respectively, and ${ϵ}_{t|N}^k$, ${\widehat{x}}_{t|N}^k$, and ${\Sigma }_{t|N}^k$ are given in Appendix D.

Proof. 

Consider the definition of Bayesian smoothing given in (19), the time-update equation $p\left(x_t\right|y_{1:t-1})$ obtained from (34), and the reduced version of the measurement-update step in the backward filter $p\left(y_{t:N}\right|x_t)$ defined in (54). For $t=N-1,N-2,\dots ,1$, it is obtained:

$$p\left(x_t\right|y_{1:N})\propto \sum _{\tau =1}^{M_{t|t-1}}\sum _{\ell =1}^{S_{\mathrm{red}}}{\gamma }_{t|t-1}^{\tau }{\delta }_{t|t}^{\ell }{\mathcal{N}}_{x_t}\left(z_{t|t}^{\ell },U_{t|t}^{\ell }\right){\mathcal{N}}_{x_t}({\widehat{x}}_{t|t-1}^{\tau },{\Sigma }_{t|t-1}^{\tau }).$$

(56)

Defining $g\left(x_t\right)={\mathcal{N}}_{x_t}(z_{t|t}^{\ell },U_{t|t}^{\ell }){\mathcal{N}}_{x_t}({\widehat{x}}_{t|t-1}^{\tau },{\Sigma }_{t|t-1}^{\tau })$, the following equation is derived:

$$\begin{array}{c}\hfill g\left(x_t\right)=\frac{exp\left\{-{\displaystyle \frac{1}{2}}\left[{\varphi }_{1t}^{\ell }+{\varphi }_{2t}^{\tau }-{\varphi }_{3t}^{\ell \tau }\right]\right\}{\mathcal{N}}_{x_t}\left({\left(L_t^{\ell \tau }\right)}^{-1}{\rho }_t^{\ell \tau },{\left(L_t^{\ell \tau }\right)}^{-1}\right)}{{\left(2\pi \right)}^{\frac{n}{2}}\sqrt{det\left\{L_t^{\ell \tau }\right\}det\left\{U_{t|t}^{\ell }\right\}det\left\{{\Sigma }_{t|t-1}^{\tau }\right\}}},\end{array}$$

(57)

where $L_t^{\ell \tau }$, ${\rho }_t^{\ell \tau }$, ${\varphi }_{1t}^{\ell }$, ${\varphi }_{2t}^{\tau }$, and ${\varphi }_{3t}^{\ell \tau }$ are defined in (A32), (A31), (A34), (A35), and (A33), respectively. Next, expressing the double summation in (56) as a single one with a new index $k=(\ell -1)M_{t|t-1}+\tau$, it is obtained:

$$\begin{array}{c}\hfill p\left(x_t\right|y_{1:N})\propto \sum _{k=1}^{S_{t|N}}{\overline{ϵ}}_{t|N}^k{\mathcal{N}}_{x_t}({\widehat{x}}_{t|N}^k,{\Sigma }_{t|N}^k),\end{array}$$

(58)

where $S_{t|N}=M_{t|t-1}{S}_{\mathrm{red}}$ and ${\overline{ϵ}}_{t|N}^k$, ${\Sigma }_{t|N}^k$, and ${\widehat{x}}_{t|N}^k$ are given in (A30), (A28), and (A29), respectively. Finally, the smoothing PDF in (55) is obtained by computing the normalized wights as (A27), and this completes the proof.   □

Provided $p\left(x_t\right|y_{1:N})$ in (55) as a GMM, the state estimator given in (11) and the covariance matrix of the estimation error in (13) can be computed as follows:

$$\begin{array}{}\mathrm{(59)}& \hfill {\widehat{x}}_{t|N}& =\sum _{k=1}^{S_{t|N}}{ϵ}_{t|N}^k{\widehat{x}}_{t|N}^k,\hfill \mathrm{(60)}& \hfill {\Sigma }_{t|N}& =\sum _{k=1}^{S_{t|N}}{ϵ}_{t|N}^k\left[{\Sigma }_{t|N}^k+({\widehat{x}}_{t|N}^k-{\widehat{x}}_{t|N}){({\widehat{x}}_{t|N}^k-{\widehat{x}}_{t|N})}^T\right].\hfill \end{array}$$

In Algorithm 3, the steps to implement the Gaussian sum smoothing are summarized.

Algorithm 3 Gaussian sum smoothing algorithm for quantized output data

1  Input: The PDFs $p\left(x_t\right|y_{1:t-1})$ and $p\left(x_N\right|y_{1:N})$ obtained from Algorithm 1 and

$p\left(y_{t:N}\right|x_t)$ obtained from Algorithm 2.

2  Save the PDF $p\left(x_N\right|y_{1:N})$

13  Output: The state estimation in (11), the covariance matrix of the estimation error

in (13), and the smoothing PDFs $p\left(x_t\right|y_{1:N})$, for $t=1,\dots ,N$.

3.9. Computing the Smoothing Joint PDF $p(x_{t+1},x_t|y_{1:N})$

In Theorems 2–4, the filtering and smoothing problems for quantized data are solved. However, many strategies for the identification of state-state space systems [8,9] require the joint PDF $p(x_{t+1},x_t|y_{1:N})$, which for the quantized output data case is given by the following:

Theorem 5.

Consider the system in (4)–(6), the PDF $p\left(x_t\right|y_{1:t})$ and the backward prediction equation $p\left(y_{t+1:N}\right|x_{t+1})$ given in Theorems 2 and 3, respectively, and the PDF $p\left(x_{t+1}\right|x_t)$ given in Equation (5). Then, for $t=N-1,\dots ,1$, the joint PDF $p(x_{t+1},x_t|y_{1:N})$ is the GMM given by:

$$p(x_{t+1},x_t|y_{1:N})=\sum _{k=1}^{S_{t,t+1}}{\alpha }_k{\mathcal{N}}_{{\chi }_t}\left({\widehat{\chi }}_{t|N}^k,E_{t|N}^k\right),$$

(61)

where $S_{t,t+1}=M_{t|t}{S}_{t+1|t+1}$, $M_{t|t}$ and $S_{t+1|t+1}$ are given in (33) and (42), respectively, ${\alpha }_k$, ${\widehat{\chi }}_{t|N}^k$, and $E_{t|N}^k$ are given in Appendix E, and ${\chi }_t$ is the extended vector given by:

$${\chi }_t^T={\left[\begin{array}{cc}{x}_{t+1}^T& x_t^T\end{array}\right]}^T.$$

(62)

At time $t=N$, the joint PDF $p(x_{N+1},x_N|y_{1:N})$ is given by (61) with:

$$\begin{array}{cc}\hfill & S_{N+1|N+1}=1,{ϵ}_{N+1|N+1}^{\ell }=1,{\lambda }_{N+1|N+1}^{\ell }=1,\hfill \\ \hfill & F_{N+1|N+1}^{\ell }=0,G_{N+1|N+1}^{\ell T}=0,H_{N+1|N+1}^{\ell }=0.\hfill \end{array}$$

(63)

Proof. 

Using Bayes’ theorem, the joint PDF $p(x_{t+1},x_t|y_{1:N})$ can be obtained as:

$$p(x_{t+1},x_t|y_{1:N})=\frac{p\left(y_{t+1:N}\right|x_{t+1})p\left(x_{t+1}\right|x_t)p\left(x_t\right|y_{1:t})}{p\left(y_{t+1:N}\right|y_{1:t})},$$

(64)

where $p\left(y_{t+1:N}\right|y_{1:t})$ is a normalization constant. The required PDFs $p\left(x_t\right|y_{1:t})$ and $p\left(x_{t+1}\right|x_t)$ and backward-prediction equation $p\left(y_{t+1:N}\right|x_{t+1})$ are given in (33), (5) and (42), respectively. Using Lemma A2, $p\left(x_t\right|y_{1:t})$ in (33) can be rewritten as:

$$\begin{array}{c}\hfill p\left(x_t\right|y_{1:t})=\sum _{\tau =1}^{M_{t|t}}\frac{{\gamma }_{t|t}^{\tau }}{\sqrt{det\left\{2\pi {\Sigma }_{t|t}^{\tau }\right\}}}exp\left\{-\frac{1}{2}\left(x_t^T{\left({\Sigma }_{t|t}^{\tau }\right)}^{-1}{x}_t-2J_t^{\tau T}{x}_t+L_t^{\tau }\right)\right\},\end{array}$$

(65)

where $J_t^{\tau T}$ and $L_t^{\tau }$ are defined in (A44) and (A45), respectively. Considering $p\left(y_{t+1:N}\right|x_{t+1})$, $p\left(x_t\right|y_{1:t})$, $p\left(x_{t+1}\right|x_t)$, and the extended vector of the state given in (62), the argument of these three functions can be written as follows:

$$A_1={\chi }_t^T\left[\begin{array}{cc}0& 0\\ 0& {\left({\Sigma }_{t|t}^{\tau }\right)}^{-1}\end{array}\right]{\chi }_t-2\left[\begin{array}{cc}0& J_t^{\tau T}\end{array}\right]{\chi }_t+L_t^{\tau },$$

(66)

$$A_2={\chi }_t^T\left[\begin{array}{cc}{F}_{t+1|t+1}^{\ell }& 0\\ 0& 0\end{array}\right]{\chi }_t-2\left[\begin{array}{cc}{G}_{t+1|t+1}^{\ell T}& 0\end{array}\right]{\chi }_t+H_{t+1|t+1}^{\ell },$$

(67)

$$\begin{array}{cc}\hfill A_3& ={\chi }_t^T\left[\begin{array}{cc}{Q}^{-1}& -Q^{-1}A\\ -A^T{Q}^{-1}& A^T{Q}^{-1}A\end{array}\right]{\chi }_t\hfill \\ & -2\left[\begin{array}{cc}{u}_t^T{B}^T{Q}^{-1}& -u_t^T{B}^T{Q}^{-1}A\end{array}\right]{\chi }_t+u_t^T{B}^T{Q}^{-1}B{u}_t,\hfill \end{array}$$

(68)

Then, adding these expressions, the following equation is derived:

$$\begin{array}{cc}\hfill p(x_{t+1},x_t|y_{1:N})& \propto \sum _{\tau =1}^{M_{t|t}}\sum _{\ell =1}^{S_{t+1|t+1}}\frac{{\gamma }_{t|t}^{\tau }{ϵ}_{t+1|t+1}^{\ell }{\lambda }_{t+1|t+1}^{\ell }}{{\left(2\pi \right)}^n\sqrt{det\left\{Q\right\}det\left\{{\Sigma }_{t|t}^{\tau }\right\}}}\hfill \\ \hfill & \times exp\left\{-{\displaystyle \frac{1}{2}}\left({\chi }_t^T{\mathcal{F}}_t^{\ell \tau }{\chi }_t-2{\mathcal{G}}_t^{\ell \tau T}{\chi }_t+{\mathcal{H}}_t^{\ell \tau }\right)\right\},\hfill \end{array}$$

(69)

where ${\mathcal{F}}_t^{\ell \tau }$, ${\mathcal{G}}_t^{\ell \tau T}$, and ${\mathcal{H}}_t^{\ell \tau }$ are defined in (A41), (A42), and (A43), respectively. Completing the square and expressing the double summation as a single one with a new index $k=(\ell -1)M_{t|t}+\tau$, it is obtained:

$$\begin{array}{c}\hfill p(x_{t+1},x_t|y_{1:N})\propto \sum _{k=1}^{S_{t,t+1}}{\overline{\alpha }}_k{\mathcal{N}}_{{\chi }_t}\left({\widehat{\chi }}_{t|N}^k,E_{t|N}^k\right),\end{array}$$

(70)

where $S_{t,t+1}=M_{t|t}{S}_{t+1|t+1}$, ${\overline{\alpha }}_k$ is defined in (A39), and ${\widehat{\chi }}_{t|N}^k$ and $E_{t|N}^k$ are defined in (A37) and (A38), respectively. Finally, the smoothing joint distribution in (64) for $t=N-1,\dots ,1$ is derived by computing the normalized wights as in (A36). Notice that, for $t=N$, the smoothing joint PDF $p(x_{N+1},x_N|y_{1:N})=p\left(x_{N+1}\right|x_N)p\left(x_N\right|y_{1:N})$ is obtained as (61) considering the expressions given in (63). Then, the proof is finished.   □

In Algorithm 4, the steps to implement the Gaussian sum smoothing for computing the joint PDF $p(x_{t+1},x_t|y_{1:N})$ are summarized.

Algorithm 4 Gaussian sum smoother to compute $p(x_{t+1},x_t|y_{1:N})$ for quantized output data

1  Input: The PDF $p\left(x_t\right|y_{1:t})$ obtained in Algorithm 1, $p\left(y_{t+1:N}\right|x_{t+1})$ computed in

Algorithm 2, and the PDF $p\left(x_{t+1}\right|x_t)$ given in Equation (5).

11  Compute and store $p(x_{N+1},x_N|y_{1:N})$ according to Theorem 5 with $S_{N+1|N+1}$,

${ϵ}_{N+1|N+1}^{\ell }$, ${\lambda }_{N+1|N+1}^{\ell }$, $F_{N+1|N+1}^{\ell }$, $G_{N+1|N+1}^{\ell T}$, and $H_{N+1|N+1}^{\ell }$ given in (63).

12  Output: The smoothing PDFs $p(x_{t+1},x_t|y_{1:N})$, for $t=1,\dots ,N$.

4. Numerical Example

In this section, a numerical example to illustrate the benefits of this paper proposal is presented. Furthermore, a practical simulation example is studied: the problem of estimating the tank liquid level in a two-tank system is addressed. Typically, a numerical simulation approach is used for testing new algorithms and designs in applications of state estimation [74], control [75], and system identification [63], among others. This approach is used to evaluate the performance of the estimation algorithms, in order to avoid safety problems that can occur in a real-world processes.

To evaluate the performance of the proposed filtering and smoothing methods for quantized data, a comparison with three classical techniques is presented: standard Kalman filtering and smoothing [76], quantized Kalman filtering and smoothing [49,77], and particle filtering and smoothing [31]. Notice that it is also possible to implement a version of particle filtering and smoothing algorithms using the approximation of $p\left(y_t\right|x_t)$ given in (21). However, in the numerical examples run for filtering, the computation time was high. In order to validate the approximation in Theorem 1, the standard particle filtering and smoothing algorithms with a large number of particles were used, where $p\left(y_t\right|x_t)$ was computed using the integral in (20) from the MATLAB function mvncdf, which computes the multivariate normal cumulative distribution function. The true filtering and smoothing PDFs were considered to be provided by the particle filter and smoother with 20,000 particles, which was defined as the ground truth.

In the following examples, the discrete-time system in state-space form given in (1)–(3) is used with:

$$y_t={\Delta }_q\mathrm{round}\left\{z_t/{\Delta }_q\right\},$$

(71)

where the quantizer is defined in terms of the round function in MATLAB and the quantization step ${\Delta }_q$. The infimum and supremum values of the sets ${\mathcal{J}}_i$ defined in (8), $q_{i-1}$ and $q_i$, can be calculated for the infinite-level quantizer as $q_{i-1}=y_t-0.5{\Delta }_q$ and $q_i=y_t+0.5{\Delta }_q$ for $i=\dots ,1,2\dots ,L,\dots$.

The experiments were carried out on a computer with the following specifications: processor: Intel(R) Core(TM) i5-8300H CPU @ 2.30GHz, RAM memory: 8.00 GB, operating system: Windows 10 with MATLAB 2020b.

4.1. Example 1: First-Order System

In this example, the following state-space system was considered:

$$\begin{array}{}\mathrm{(72)}& \hfill x_{t+1}& =0.9x_t+1.0u_t+w_t,\hfill \mathrm{(73)}& \hfill z_t& =2.0x_t+0.5u_t+v_t,\hfill \end{array}$$

where $w_t\sim {\mathcal{N}}_{w_t}\left(0,1\right)$ and $v_t\sim {\mathcal{N}}_{v_t}\left(0,0.5\right)$. Furthermore, the input was considered to be drawn from ${\mathcal{N}}_{u_t}(0,2)$ and $x_1\sim {\mathcal{N}}_{x_1}(1,1)$. In Figure 5, the filtering PDFs for some time instants are shown. The quantization step used in this example and the number of Gaussian components to approximate $p\left(y_t\right|x_t)$ were considered to be ${\Delta }_q=7$ and $K=10$, respectively. Furthermore, $S_{\mathrm{red}}=K$.

Figure 5 shows that the filtering PDFs obtained with our proposal, the Gaussian sum filter, were that best fit to the ground truth. In contrast, the PDFs obtained with the Kalman filter, quantized Kalman filter, and particle filter were different from the ground truth. Notice that the results obtained using the particle filter with 100 particles were good at $t=3,9,40$, whilst the resulting PDFs for $t=25,32,59,72,82,99$ differed slightly more from the ground truth. However, the performance of particle filtering can be improved by increasing the number of particles, which produces an increment in the execution time.

To compare the accuracy of the state estimation, 100 Monte Carlo trials were run, and the mean-squared error between the true state and the estimation obtained by the Kalman filter, quantized Kalman filter, Gaussian sum filter, and particle filter is computed as follows:

$$\mathrm{MSE}=\frac{1}{\mathcal{R}}\sum _{k=1}^{\mathcal{R}}\Vert x_t-{\widehat{x}}_{t|t}{\Vert }_2^2,$$

(74)

where $x_t$ is the true state (which in a real-word system is not available, but in simulations, it can be used to analyze the performance of the estimation techniques), ${\widehat{x}}_{t|t}$ is the estimation of the state system, and $\Vert ·{\Vert }_2^2$ denotes the squared Euclidean norm. In Figure 6, the box plots corresponding to 100 Monte Carlo runs for ${\Delta }_q=\left\{3,5,7\right\}$ are shown.

Figure 6 shows that the MSE between the true state and the estimation obtained with the Kalman filter and quantized Kalman filter increased fast as ${\Delta }_q$ increased. However, the MSE increased slowly for the state estimation obtained with the Gaussian sum filter and particle filter.

In Figure 7, the smoothing PDFs at time $t=100$ are shown. The quantization step was considered to be chosen from ${\Delta }_q=\left\{3,5,7\right\}$, and the number of Gaussian components to approximate $p\left(y_t\right|x_t)$ was chosen from $K=\left\{6,8,10\right\}$. Furthermore, the number of Gaussian components kept after the reduction procedure was considered as $S_{\mathrm{red}}=K$. In order to obtain the adequate number of particles to compare the particle smoother with the Gaussian sum smoother, 100 Monte Carlo simulations were carried out to obtain the number of particles that yielded smoothing PDFs that were as close to the true PDFs as the Gaussian sum smoother using $K=\left\{6,8,10\right\}$. For comparison purposes, the particle smoother execution time corresponding to the time required to implement the particle smoother algorithm using the number of particles that produces a similar result to the Gaussian sum smoother with K components is defined as Par(K). The $L_2$-norm of the difference between the true and the estimated PDFs as the measure of similarity was used:

$$\Vert q-\widehat{q}{\Vert }_2={\left[\sum _{k=1}^M{|q-\widehat{q}|}^2\right]}^{1/2},$$

(75)

where q represents the true PDF and $\widehat{q}$ represents the estimated PDF, which was chosen so that $\Vert q-\widehat{q}{\Vert }_2<1\times {10}^{-6}$. The approximated number of particles (labeled in each PDF in Figure 7) was used to compare the execution time of both algorithms, the Gaussian sum smoother and particle smoother, and the results are shown in

Table 2. Time in seconds required to perform the smoothing algorithm for the scalar system. Par(K) represents the number of particles (labeled in Figure 7) that produce a similar result to the Gaussian sum smoother with K components. KS, QKS, GSS, and PS stand for the Kalman smoother, quantized Kalman smoother, Gaussian sum smoother, and particle smoother, respectively.

	KS	QKS		GSS				PS
${\Delta }_q$	-	-		$K=6$	$K=8$	$K=10$		Par(6)	Par(8)	Par(10)
7	0.0379	0.1494		0.4158	0.4221	0.4735		2.0420	3.3538	4.0462
5	0.0048	0.0044		0.2695	0.3706	0.4588		3.0074	3.5665	4.0192
3	0.0033	0.0043		0.2609	0.3491	0.4618		4.2471	5.5501	6.3408

. Figure 7 shows that the smoothing PDFs obtained with our proposal using a small number of Gaussian components and the PDFs obtained using the particle smoother with a large number of particles fit the ground truth. In contrast, the PDFs obtained with the Kalman smoother and quantized Kalman smoother were different from the ground truth. Furthermore, the execution time in Table 2 shows that the required time to perform the Gaussian sum smoother was less compared to the time to perform the particle smoother, which needs a large number of particles to produce a result similar to the Gaussian sum smoother.

From the results shown in Figure 5, Figure 6 and Figure 7 and Table 2, it can be concluded that:

• The filtering and smoothing PDFs are non-Gaussian, although the process and output noises in (1) and (2) are Gaussian distributed;
• The accuracy of the standard and quantized Kalman filtering and smoothing decreased as the quantization step increased;
• The state estimates obtained with particle filter and smoother were similar to the results obtained using the Gaussian sum filter and smoother. However, the characterization of the filtering and smoothing PDFs using the Gaussian sum filter and smoother were better than the PDF obtained by the particle filter and smoother. Notice that a correct characterization of a PDF is important when high-order moments need to be computed, especially in system identification tasks;
• In order to implement the Gaussian sum filter and smoother, the parameters K (the number that defines the quality of the $p\left(y_t\right|x_t)$ approximation) and $S_{\mathrm{red}}$ (the Gaussian components kept after the Gaussian sum reduction algorithm) need to be chosen by the user. These parameters can be found in a simulation study by the trial and error approach and should be set by a trade-off between the time complexity and the accuracy of the estimation. A large value of K produces an accurate estimate, but a high computational load;
• The larger the quantization step is, the larger the number of Gaussian components, K, needed to approximate $p\left(y_t\right|x_t)$ in order to obtain an accurate approximation. However, for a large quantization step, the number K needed to obtain a good approximation of the filtering and smoothing PDFs is relatively small compared to the number of particles required to obtain similar results using the particle filter and smoother;
• The maximum number of Gaussian components kept after the Gaussian reduction procedure is important for the accuracy of the approximation. In the simulations, $S_{\mathrm{red}}=K$ was used. Furthermore, it was noticed that once an adequate $S_{\mathrm{red}}$ was defined, incrementing this value did not produce a significant improvement in the estimation. However, this increment in $S_{\mathrm{red}}$ was really critical for the resulting numerical complexity of the algorithm (and hence, the execution time), which increased since the Gaussian sum reduction procedure (e.g., Kullback–Leibler reduction) utilized more time to reduce a large amount of Gaussian components;
• The Gaussian sum smoother execution time for all values of ${\Delta }_q$ was small. This occurred because in each case, a relatively small number of Gaussian components to approximate $p\left(y_t\right|x_t)$ were used. However, the particle smoother execution time is variable for different values of ${\Delta }_q$. As ${\Delta }_q$ decreased, the $L_2$-norm between the Gaussian sum smoother and the ground truth decreased, and a larger number of particles to obtain a comparable $L_2$-norm between the particle smoother and the ground truth were required.

4.2. Real-World Application: Tank Liquid Level

In this example, the problem of estimating the tank liquid level in a two-tank system by using the measurements taken by a low-cost sensor based on a variable resistor that is attached to an arm with a floater is considered; see Figure 8.

A linearized model of this system can be found in [78]. Here, it was assumed that $h_2$ can be measured and $h_1$ cannot. The discrete-time version of the model in [78] with sample time $0.1$ s was considered:

$$\begin{array}{}\mathrm{(76)}& \hfill x_{t+1}& = \left[\begin{array}{cc}0.9959& 0.0041\\ 0.0041& 0.9959\end{array}\right]x_t+\left[\begin{array}{cc}0.0998& -0.0002\\ 0.0002& -0.0998\end{array}\right]u_t+w_t,\hfill \mathrm{(77)}& \hfill z_t& = \left[\begin{array}{cc}0& 1.0\end{array}\right]x_t+v_t,\hfill \end{array}$$

where $x_t={\left[h_1{h}_2\right]}^T$ and $u_t={[f_1-k{f}_2+k]}^T$ with $k=0.4111$. The sensor measures $h_2$ vary its resistance in discrete steps, with minimum and maximum values ${\beta }_1=2$ and ${\beta }_L=10$. To simulate this system, it was considered that: $w_t\sim {\mathcal{N}}_{w_t}\left(0,0.001I_2\right)$, $v_t\sim {\mathcal{N}}_{v_t}\left(0,0.0001\right)$; the input ${\left[f_1{f}_2\right]}^T$ was drawn from $\mathcal{N}\left({\left[102\right]}^T,10I_2\right)$; the initial condition $x_1\sim {\mathcal{N}}_{x_1}({\left[105\right]}^T,0.01I_2)$. For this example, 100 Monte Carlo runs were simulated. In Figure 9 (left), the output $z_t$ that corresponds to the values of $h_2$ that are nonquantized and the output $y_t$ that corresponds to the measurements given by the sensor (for one of the Monte Carlo runs) are shown. In this figure, the level of quantization in the measurements can be observed. In Figure 9 (right), the MSE between the true and estimated state is shown. It was observed that the proposal presented in this paper, the Gaussian sum filter, yielded the most accurate estimation of $h_1$, followed by the particle filter, Kalman filter, and quantized Kalman filter. In this example, $K=50$ and 1000 particles were used to implement the Gaussian sum filter and particle filter, respectively.

In this example, a relatively high number of Gaussian components $(K=50)$ were required to obtain a good estimation of the filtering and smoothing distributions, and hence of the state. This produced an increment in the execution time since the Gaussian sum reduction algorithm needed more time to deal with a high number of Gaussian components in every iteration. This resulted in similar execution times for our proposed algorithm and the traditional particle filter. However, the execution time of the Gaussian sum filter and smoother was smaller than the execution time of the ground truth (particle filter and smoother with 20,000 particles).

5. Conclusions

In this paper, Gaussian sum filtering and smoothing algorithms for linear-time-invariant state-space systems with quantized output data were developed. An approximation of the integral equation that defines the probability mass function of the quantized data given the current state, $p\left(y_t\right|x_t)$, using Gaussian quadrature was considered. This approximation naturally yielded an explicit mathematical model with a GMM structure for this probability function. Using the approximation of $p\left(y_t\right|x_t)$ summarized in Theorem 1, it was possible to solve in closed form the general equations of filtering and smoothing to deal with quantized data. This fact allowed for a closed-form expression of the system state estimators given the quantized data ${\widehat{x}}_{t|t}$ and ${\widehat{x}}_{t|N}$. Via numerical simulations, it was shown that approximating $p\left(y_t\right|x_t)$ with a small number of Gaussian components was adequate, yielding an approximation comparable to the true filtering and smoothing PDFs given by the particle approach (using a large number of particles, namely 20,000 particles). This reduced number of Gaussian components allowed for a low computational load, especially when the system order increased. In addition, our results showed overall less computational load for our proposed techniques since the number of Gaussian components was considerably less than the number of particles used in particle filtering and smoothing.

The proposed Gaussian sum filtering and smoothing algorithms can be utilized, in principle, to develop system identification algorithms and control strategies having quantized output measurements.