Kalman Filter Using a Third-Order Tensorial Decomposition of the Impulse Response

Abstract

For system identification problems associated with long-length impulse responses, the recently developed decomposition-based technique that relies on a third-order tensor (TOT) framework represents a reliable choice. It is based on a combination of three shorter filters, which merge their estimates in tandem with the Kronecker product. In this way, the global impulse response is modeled in a more efficient manner, with a significantly reduced parameter space (i.e., fewer coefficients). In this paper, we further develop a Kalman filter based on the TOT decomposition method. As compared to the recently designed recursive least-squares (RLS) counterpart, the proposed Kalman filter achieves superior performance in terms of the main criteria (e.g., tracking and accuracy). In addition, it significantly outperforms the conventional Kalman filter, while also having a lower computational complexity. Simulation results obtained in the context of echo cancellation support the theoretical framework and the related advantages.

1. Introduction

The Kalman filter is recognized as a popular signal processing tool in the framework of many real-world applications [1,2,3]. Among them, it has successfully been used in echo cancellation scenarios [4,5,6,7,8,9], aiming to model an unknown impulse response corresponding to the echo path. Nevertheless, the finite impulse response filter associated with this impulse response is usually characterized by a very large number of coefficients, which reaches the order of hundreds or even thousands. Such a large parameter space significantly impacts the overall performance of the Kalman filter, especially in terms of the convergence behavior, solution accuracy, and computational complexity.

In system identification problems like echo cancellation it is always helpful to exploit the particular characteristics of the impulse responses to be estimated. For example, echo paths are sparse in nature, i.e., only a small number of coefficients (as compared to the full length of the impulse response) have a significant magnitude, while the rest of them are close to zero or even null. The adaptive algorithms that take advantage of this feature are designed based on a “proportionate” approach [10], in which the significant coefficients are emphasized during the updating process. Due to this technique, the resulting proportionate-type adaptive filters achieve higher converge rates as compared to their conventional counterparts. Following the proportionate normalized least-mean-squares (PNLMS) algorithm from [10], different related versions were developed, like the mu-law PNLMS [11], the sparseness-controlled PNLMS [12], the PNLMS with individual activation factors [13], or the block-sparse PNLMS [14]. Also, the sparseness features were exploited in the framework of other categories of adaptive filtering algorithms, like the sign affine projection algorithm [15], recursive least-squares (RLS) algorithms with penalties [16,17], nonlinear filtering architectures [18], and subband adaptive filters [19,20]. In addition to echo cancellation, these sparsity-aware filters have been applied in the context of a wide range of applications, including underwater acoustics [21,22], channel estimation [23], and system identification [24]. A connection between the Kalman filter and several proportionate-type algorithms was established in [7]. Nevertheless, all these proportionate adaptive filters still involve the full length of the impulse response, so they face the related difficulties associated with a large number of coefficients.

In this context, a useful technique to reduce the parameter space takes advantage of the low-rank characteristic of the impulse responses, which fits very well for the estimation of echo path models [25]. In other words, the corresponding matrices associated with the reshaped impulse responses of the echo paths are never full-rank. This can be further exploited based on the nearest Kronecker product (NKP) decomposition, so that the adaptive filter estimate results based on a combination of shorter impulse responses, thus involving a smaller number of coefficients. In this decomposition framework, the NKP-based technique was applied to the RLS algorithm [26], the Kalman filter [27], the normalized least-mean-squares (NLMS) algorithm [28], correntropy-based methods [29], and nonlinear adaptive filters [30,31,32]. Related applications include acoustic feedback cancellation [33], adaptive beamforming [34], and acoustic echo cancellation [35], among others.

Following the NKP-based approach, the decomposition technique has been recently applied in conjunction with higher-order tensors [36,37], thus leading to higher-dimensionality reduction. It is known that tensor-based signal processing [38,39,40] represents an important asset in the framework of many applications, like big data analysis [41], biomedical engineering [42], blind source separation [43], and machine learning [44]. As a result, in long-length system identification scenarios, the RLS adaptive filter using a third-order tensor (TOT) decomposition from [37] outperforms its previously developed NKP-based counterpart designed in [26]. There is a well-established relation between the RLS algorithm and the Kalman filter, as supported in [2]. While there is a (misleading) resemblance between the two algorithms, the Kalman filter allows for better control in time-varying environments [7]. Nevertheless, due to the specific features of the TOT decomposition, there is not a straightforward extension from the RLS-TOT [37] towards the corresponding Kalman filter.

In this paper, motivated by the previous aspects and expected performance gain, we design a Kalman filter based on TOT decomposition. The main goals of our proposal target three basic directions. First, we aim to show that the developed Kalman filter based on TOT decomposition outperforms the conventional Kalman filter, in terms of both performance and complexity. Second, with the proposed TOT-based solution, we target a better performance as compared to the previously developed NKP-based version from [27], which exploits a second-order decomposition level. Third, the goal is to outperform the recently introduced RLS-TOT algorithm from [37], which applies the TOT decomposition in the framework of the RLS filter.

In order to develop our proposal, the rest of the manuscript is organized as follows. Section 2 presents the necessary background concerning the system identification scenario, the conventional Kalman filter, and the TOT decomposition framework. Then, the proposed Kalman filter based on TOT is developed in Section 3. Simulation results are provided in Section 4, using echo cancellation as the experimental setup. Finally, conclusions and future works are summarized in Section 5.

2. System Identification and TOT Decomposition

This section reviews the required background for the upcoming developments. First, the system identification scenario is presented, followed by the solution based on the conventional Kalman filter. Finally, the TOT decomposition of the impulse response is summarized.

2.1. System Identification Framework

The system identification framework addressed in this paper considers a linear single-input single-output (SISO) scenario with zero-mean real-valued signals, where the main goal is to identify an unknown system characterized by the impulse response $\mathbf{h}\left(n\right)={\left[\begin{array}{cccc}{h}_0\left(n\right)& h_1\left(n\right)& \cdots & h_{L-1}\left(n\right)\end{array}\right]}^T$ of length L, where n is the discrete-time index and superscript ${\hspace{0.25em}}^T$ denotes transposition. Also, let us consider that this time-varying impulse response evolves based on a simplified first-order Markov model, i.e.,

$$\begin{array}{cc}\hfill \mathbf{h}\left(n\right)& =\mathbf{h}(n-1)+\mathbf{w}\left(n\right),\hfill \end{array}$$

(1)

where $\mathbf{w}\left(n\right)$ is a vector of length L, which contains the samples of a zero-mean white Gaussian noise. In this context, $\mathbf{h}(n-1)$ and $\mathbf{w}\left(n\right)$ are uncorrelated. This standard model has been extensively involved in many analyses of time-varying systems in the framework of adaptive filtering algorithms [45,46,47]. In addition, it fits very well in echo cancellation scenarios [48]. Since $\mathbf{w}\left(n\right)$ is white and Gaussian, its covariance matrix is a diagonal one, which can be expressed as ${\mathbf{R}}_{\mathbf{w}}\left(n\right)={\sigma }_w^2\left(n\right){\mathbf{I}}_L$, where ${\sigma }_w^2\left(n\right)$ is the noise variance and ${\mathbf{I}}_L$ denotes the identity matrix of size $L\times L$. It can be noticed that the uncertainties in $\mathbf{h}\left(n\right)$ are captured by ${\sigma }_w^2\left(n\right)$.

2.2. Conventional Kalman Filter

In the context of Kalman filtering, the model from (1) represents the so-called state equation. In this framework, the observation equation is defined as the output of the system, related to the reference (or desired) signal, $d\left(n\right)$. It contains the convolution product between the samples of the input signal, $x\left(n\right)$, and the coefficients of the impulse response, $\mathbf{h}\left(n\right)$, and is corrupted by an additive noise, $v\left(n\right)$. It is considered that this noise is white and Gaussian, with variance ${\sigma }_v^2=E\left[v^2\left(n\right)\right]$, where $E[·]$ represents mathematical expectation. Consequently,

$$\begin{array}{cc}\hfill d\left(n\right)& ={\mathbf{h}}^T\left(n\right)\mathbf{x}\left(n\right)+v\left(n\right),\hfill \end{array}$$

(2)

where the vector $\mathbf{x}\left(n\right)={\left[\begin{array}{cccc}x\left(n\right)& x(n-1)& \cdots & x(n-L+1)\end{array}\right]}^T$ contains the L most recent time samples of the input signal. Next, considering an estimate of the impulse response at time index n, denoted by $\widehat{\mathbf{h}}\left(n\right)={\left[\begin{array}{cccc}{\widehat{h}}_0\left(n\right)& {\widehat{h}}_1\left(n\right)& \cdots & {\widehat{h}}_{L-1}\left(n\right)\end{array}\right]}^T$, the a priori estimation error results in

$$\begin{array}{cc}\hfill e\left(n\right)& =d\left(n\right)-{\widehat{\mathbf{h}}}^T(n-1)\mathbf{x}\left(n\right),\hfill \end{array}$$

(3)

while the a posteriori error is defined as

$$\begin{array}{cc}\hfill \epsilon \left(n\right)& =d\left(n\right)-{\widehat{\mathbf{h}}}^T\left(n\right)\mathbf{x}\left(n\right).\hfill \end{array}$$

(4)

Related to these estimates, we can also define the a priori and a posteriori misalignments, i.e.,

$$\begin{array}{cc}\hfill \mathbf{m}\left(n\right)& =\mathbf{h}\left(n\right)-\widehat{\mathbf{h}}(n-1),\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill \tilde{\mathbf{m}}\left(n\right)& =\mathbf{h}\left(n\right)-\widehat{\mathbf{h}}\left(n\right),\hfill \end{array}$$

(6)

which are also known as the system mismatches [48] or state estimation errors. Based on (1), it can be easily checked that $\mathbf{m}\left(n\right)=\tilde{\mathbf{m}}(n-1)+\mathbf{w}\left(n\right)$. Also, using (3) and (4), we have $\epsilon \left(n\right)={\tilde{\mathbf{m}}}^T\left(n\right)\mathbf{x}\left(n\right)+v\left(n\right)$.

At this point, the Kalman filter can be obtained based on the linear sequential Bayesian approach, which states that the optimal estimate of the state vector is [49]

$$\begin{array}{cc}\hfill \widehat{\mathbf{h}}\left(n\right)& =\widehat{\mathbf{h}}(n-1)+\mathbf{k}\left(n\right)e\left(n\right),\hfill \end{array}$$

(7)

where $\mathbf{k}\left(n\right)$ is the Kalman gain vector [1]. This results by minimizing the cost function

$$\begin{array}{cc}\hfill J\left[\mathbf{k}\left(n\right)\right]& =\frac{1}{L}\mathrm{tr}\left[{\mathbf{R}}_{\tilde{\mathbf{m}}}\left(n\right)\right],\hfill \end{array}$$

(8)

where $\mathrm{tr}(·)$ represents the trace of a square matrix and ${\mathbf{R}}_{\tilde{\mathbf{m}}}\left(n\right)=E\left[\tilde{\mathbf{m}}\left(n\right){\tilde{\mathbf{m}}}^T\left(n\right)\right]$ is the covariance matrix of the a posteriori misalignment from (6). From this optimization criterion, as shown in [49], two important results are obtained, as follows:

$$\begin{array}{cc}\hfill \mathbf{k}\left(n\right)& =\frac{{\mathbf{R}}_{\mathbf{m}}\left(n\right)\mathbf{x}\left(n\right)}{{\mathbf{x}}^T\left(n\right){\mathbf{R}}_{\mathbf{m}}\left(n\right)\mathbf{x}\left(n\right)+{\sigma }_v^2},\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill {\mathbf{R}}_{\tilde{\mathbf{m}}}\left(n\right)& =\left[{\mathbf{I}}_L-\mathbf{k}\left(n\right){\mathbf{x}}^T\left(n\right)\right]{\mathbf{R}}_{\mathbf{m}}\left(n\right),\hfill \end{array}$$

(10)

where ${\mathbf{R}}_{\mathbf{m}}\left(n\right)=E\left[\mathbf{m}\left(n\right){\mathbf{m}}^T\left(n\right)\right]$ is the covariance matrix of the a priori misalignment from (5), with

$$\begin{array}{cc}\hfill {\mathbf{R}}_{\mathbf{m}}\left(n\right)& ={\mathbf{R}}_{\tilde{\mathbf{m}}}(n-1)+{\sigma }_w^2\left(n\right){\mathbf{I}}_L.\hfill \end{array}$$

(11)

The initialization of the Kalman filter is $\widehat{\mathbf{h}}\left(n\right)={\mathbf{0}}_L$ and ${\mathbf{R}}_{\tilde{\mathbf{m}}}\left(0\right)=ϵ{\mathbf{I}}_L$, where ${\mathbf{0}}_L$ denotes an all-zeros vector of length L and $ϵ$ is a positive constant. Then, in each iteration, the algorithm starts by computing the a priori error $e\left(n\right)$ from (3) and evaluating the matrix ${\mathbf{R}}_{\mathbf{m}}\left(n\right)$ from (11). Next, the Kalman gain vector $\mathbf{k}\left(n\right)$ and the matrix ${\mathbf{R}}_{\tilde{\mathbf{m}}}\left(n\right)$ are computed using (9) and (10), respectively. Finally, the filter is updated based on (7). The specific parameters ${\sigma }_w^2\left(n\right)$ and ${\sigma }_v^2$ can be a priori provided or estimated within the algorithm, as indicated in [7].

2.3. TOT Decomposition

The computational complexity of the conventional Kalman filter is proportional to the square of the filter length, i.e., $\mathcal{O}\left(L^2\right)$. This represents a significant challenge when dealing with the identification of long-length impulse responses, which is usually the case in echo cancellation scenarios. In addition, other important performance criteria are also influenced by the length of the filter, like the convergence rate and tracking capability, together with the accuracy of the filter estimate (in terms of misalignment).

In this context, the TOT decomposition of the impulse response from [36] represents a useful approach, which could lead to important performance gains. The main idea is to take advantage of the low-rank features of the impulse response, in conjunction with the NKP decomposition, so that $\mathbf{h}\left(n\right)$ of length $L=L_{11}{L}_{12}{L}_2$ (with $L_{11}{L}_{12}\gg L_2$) can be decomposed as [36]

$$\begin{array}{cc}\hfill \mathbf{h}\left(n\right)& =\sum _{l=1}^{L_2}\sum _{p=1}^P{\mathbf{h}}_2^l\left(n\right)\otimes {\mathbf{h}}_{12}^{lp}\left(n\right)\otimes {\mathbf{h}}_{11}^{lp}\left(n\right),\hfill \end{array}$$

(12)

where $P<L_{12}$ and the shorter impulse responses ${\mathbf{h}}_2^l\left(n\right)$, ${\mathbf{h}}_{12}^{lp}\left(n\right)$, and ${\mathbf{h}}_{11}^{lp}\left(n\right)$ have the lengths $L_2$, $L_{12}$, and $L_{11}$, respectively, while ⊗ denotes the Kronecker product [50]. Therefore, in a system identification framework, the problem of estimating large parameter spaces, i.e., L coefficients of $\mathbf{h}\left(n\right)$, is reformulated based on a combination (via the Kronecker product) of three shorter estimates, which result in a total of $L_2^2+P{L}_{12}{L}_2+P{L}_{11}{L}_2$ coefficients. As a supporting example, let us consider that $L=512$, which can be decomposed using $L_{11}=L_{12}=16$ and $L_2=2$. Usually, $P\ll L_{12}$, as shown in [36] and also indicated by the simulation results provided in this paper (in the upcoming Section 4). For example, let us consider that $P=1$ or $P=2$. In these cases, instead of estimating 512 coefficients of $\mathbf{h}\left(n\right)$, the TOT decomposition requires only $4+32+32=68$ (for $P=1$) or $4+64+64=132$ (for $P=2$) coefficients to be estimated. Consequently, there is a significant reduction in the parameter space, which could lead to important advantages in terms of both performance and complexity.

For the purpose of the upcoming developments, the three sets of coefficients from the right-hand side of (12) are grouped into three impulse responses of lengths $L_2^2$, $P{L}_{12}{L}_2$, and $P{L}_{11}{L}_2$, respectively, as follows:

$$\begin{array}{cc}\hfill {\underline{\mathbf{h}}}_2\left(n\right)& ={\left[\begin{array}{cccc}{\left({\mathbf{h}}_2^1\right)}^T\left(n\right)& {\left({\mathbf{h}}_2^2\right)}^T\left(n\right)& \cdots & {\left({\mathbf{h}}_2^{L_2}\right)}^T\left(n\right)\end{array}\right]}^T,\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill {\underline{\overline{\mathbf{h}}}}_{12}\left(n\right)& ={\left[\begin{array}{cccc}{\left({\overline{\mathbf{h}}}_{12}^1\right)}^T\left(n\right)& {\left({\overline{\mathbf{h}}}_{12}^2\right)}^T\left(n\right)& \cdots & {\left({\overline{\mathbf{h}}}_{12}^{L_2}\right)}^T\left(n\right)\end{array}\right]}^T,\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill {\underline{\overline{\mathbf{h}}}}_{11}\left(n\right)& ={\left[\begin{array}{cccc}{\left({\overline{\mathbf{h}}}_{11}^1\right)}^T\left(n\right)& {\left({\overline{\mathbf{h}}}_{11}^2\right)}^T\left(n\right)& \cdots & {\left({\overline{\mathbf{h}}}_{11}^{L_2}\right)}^T\left(n\right)\end{array}\right]}^T,\hfill \end{array}$$

(15)

where

$$\begin{array}{cc}\hfill {\overline{\mathbf{h}}}_{12}^l\left(n\right)& ={\left[\begin{array}{cccc}{\left({\mathbf{h}}_{12}^{l1}\right)}^T\left(n\right)& {\left({\mathbf{h}}_{12}^{l2}\right)}^T\left(n\right)& \cdots & {\left({\mathbf{h}}_{12}^{lP}\right)}^T\left(n\right)\end{array}\right]}^T,\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill {\overline{\mathbf{h}}}_{11}^l\left(n\right)& ={\left[\begin{array}{cccc}{\left({\mathbf{h}}_{11}^{l1}\right)}^T\left(n\right)& {\left({\mathbf{h}}_{11}^{l2}\right)}^T\left(n\right)& \cdots & {\left({\mathbf{h}}_{11}^{lP}\right)}^T\left(n\right)\end{array}\right]}^T,\hfill \end{array}$$

(17)

with $l=1,2,\dots ,L_2$. The next goal is to reformulate the Kalman filter in terms of the component impulse responses from (13)–(15). In this manner, their estimates can be further combined similar to (12), in order to model the global impulse response of the unknown system.

3. Proposed Kalman Filter Based on TOT Decomposition

The proposed Kalman filter based on TOT decomposition, which will be further developed in this section, considers simplified first-order Markov models for the impulse responses from (13)–(15), such that

$$\begin{array}{cc}\hfill {\underline{\mathbf{h}}}_2\left(n\right)& ={\underline{\mathbf{h}}}_2(n-1)+{\mathbf{w}}_2\left(n\right),\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill {\underline{\overline{\mathbf{h}}}}_{12}\left(n\right)& ={\underline{\overline{\mathbf{h}}}}_{12}(n-1)+{\mathbf{w}}_{12}\left(n\right),\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill {\underline{\overline{\mathbf{h}}}}_{11}\left(n\right)& ={\underline{\overline{\mathbf{h}}}}_{11}(n-1)+{\mathbf{w}}_{11}\left(n\right),\hfill \end{array}$$

(20)

where ${\mathbf{w}}_2\left(n\right)$, ${\mathbf{w}}_{12}\left(n\right)$, and ${\mathbf{w}}_{11}\left(n\right)$ are three vectors of lengths $L_2^2$, $P{L}_{12}{L}_2$, and $P{L}_{11}{L}_2$, respectively, which contain samples of zero-mean white Gaussian noises. Their associated covariance matrices are ${\mathbf{R}}_{{\mathbf{w}}_2}\left(n\right)={\sigma }_{w_2}^2\left(n\right){\mathbf{I}}_{L_2^2}$, ${\mathbf{R}}_{{\mathbf{w}}_{12}}\left(n\right)={\sigma }_{w_{12}}^2\left(n\right){\mathbf{I}}_{P{L}_{12}{L}_2}$, and ${\mathbf{R}}_{{\mathbf{w}}_{11}}\left(n\right)={\sigma }_{w_{11}}^2\left(n\right){\mathbf{I}}_{P{L}_{11}{L}_2}$, where ${\sigma }_{w_2}^2\left(n\right)$, ${\sigma }_{w_{12}}^2\left(n\right)$, and ${\sigma }_{w_{11}}^2\left(n\right)$ are the corresponding variances (i.e., level of uncertainties), while ${\mathbf{I}}_{★}$ generally denotes the identity matrix of size indicated in subscript ($★\times ★$). As outlined at the end of Section 2.3, the goal is to find the estimates of the component impulse responses from (13)–(15), which are denoted by ${\widehat{\underline{\mathbf{h}}}}_2\left(n\right)$, ${\widehat{\underline{\overline{\mathbf{h}}}}}_{12}\left(n\right)$, and ${\widehat{\underline{\overline{\mathbf{h}}}}}_{11}\left(n\right)$, respectively. Their components can be obtained similar to (13)–(17), such that the estimate of the global impulse response is similar to (12), i.e.,

$$\begin{array}{cc}\hfill \widehat{\mathbf{h}}\left(n\right)& =\sum _{l=1}^{L_2}\sum _{p=1}^P{\widehat{\mathbf{h}}}_2^l\left(n\right)\otimes {\widehat{\mathbf{h}}}_{12}^{lp}\left(n\right)\otimes {\widehat{\mathbf{h}}}_{11}^{lp}\left(n\right),\hfill \end{array}$$

(21)

where ${\widehat{\mathbf{h}}}_2^l\left(n\right)$, ${\widehat{\mathbf{h}}}_{12}^{lp}\left(n\right)$, and ${\widehat{\mathbf{h}}}_{11}^{lp}\left(n\right)$ denote the estimates of ${\mathbf{h}}_2^l\left(n\right)$, ${\mathbf{h}}_{12}^{lp}\left(n\right)$, and ${\mathbf{h}}_{11}^{lp}\left(n\right)$, respectively (with $l=1,2,\dots ,L_2$ and $p=1,2,\dots ,P$).

At this point, it is important to express the a priori error from (3) in such a way as to separate (or extract) the component impulse responses. To this purpose, we take advantage of the properties of the Kronecker product [50], which allow us to equivalently express the term (of the sum) from the right-hand side of (21) in three different ways, i.e.,

$$\begin{array}{cc}\hfill {\widehat{\mathbf{h}}}_2^l\left(n\right)\otimes {\widehat{\mathbf{h}}}_{12}^{lp}\left(n\right)\otimes {\widehat{\mathbf{h}}}_{11}^{lp}\left(n\right)& =\left[{\mathbf{I}}_{L_2}\otimes {\widehat{\mathbf{h}}}_{12}^{lp}\left(n\right)\otimes {\widehat{\mathbf{h}}}_{11}^{lp}\left(n\right)\right]{\widehat{\mathbf{h}}}_2^l\left(n\right)={\widehat{\mathbf{H}}}_{12,11}^{lp}\left(n\right){\widehat{\mathbf{h}}}_2^l\left(n\right)\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill & =\left[{\widehat{\mathbf{h}}}_2^l\left(n\right)\otimes {\mathbf{I}}_{L_{12}}\otimes {\widehat{\mathbf{h}}}_{11}^{lp}\left(n\right)\right]{\widehat{\mathbf{h}}}_{12}^{lp}\left(n\right)={\widehat{\mathbf{H}}}_{2,11}^{lp}\left(n\right){\widehat{\mathbf{h}}}_{12}^{lp}\left(n\right)\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill & =\left[{\widehat{\mathbf{h}}}_2^l\left(n\right)\otimes {\widehat{\mathbf{h}}}_{12}^{lp}\left(n\right)\otimes {\mathbf{I}}_{L_{11}}\right]{\widehat{\mathbf{h}}}_{11}^{lp}\left(n\right)={\widehat{\mathbf{H}}}_{2,12}^{lp}\left(n\right){\widehat{\mathbf{h}}}_{11}^{lp}\left(n\right),\hfill \end{array}$$

(24)

where ${\widehat{\mathbf{H}}}_{12,11}^{lp}\left(n\right)$, ${\widehat{\mathbf{H}}}_{2,11}^{lp}\left(n\right)$, and ${\widehat{\mathbf{H}}}_{2,12}^{lp}\left(n\right)$ denote the terms (i.e., matrices) that multiply the “extracted” components from (22)–(24), respectively.

As a result, grouping the components ${\widehat{\mathbf{h}}}_2^l\left(n\right)$ into ${\widehat{\underline{\mathbf{h}}}}_2\left(n\right)$ [similar to (13)], while using (21) and (22) in (3), the a priori error equivalently results in

$$\begin{array}{cc}\hfill e\left(n\right)& =d\left(n\right)-{\widehat{\underline{\mathbf{h}}}}_2^T(n-1){\mathbf{x}}_{12,11}\left(n\right),\hfill \end{array}$$

(25)

where ${\mathbf{x}}_{12,11}\left(n\right)={\widehat{\underline{\overline{\mathbf{H}}}}}_{12,11}^T(n-1)\mathbf{x}\left(n\right)$, with

$$\begin{array}{c}\hfill {\widehat{\underline{\overline{\mathbf{H}}}}}_{12,11}(n-1)=\left[\begin{array}{cccc}{\widehat{\overline{\mathbf{H}}}}_{12,11}^1(n-1)& {\widehat{\overline{\mathbf{H}}}}_{12,11}^2(n-1)& \cdots & {\widehat{\overline{\mathbf{H}}}}_{12,11}^{L_2}(n-1)\end{array}\right]\end{array}$$

and ${\widehat{\overline{\mathbf{H}}}}_{12,11}^l(n-1)={\sum }_{p=1}^P{\widehat{\mathbf{H}}}_{12,11}^{lp}(n-1)$, for $l=1,2,\dots ,L_2$. Similarly, grouping the components ${\widehat{\mathbf{h}}}_{12}^{lp}\left(n\right)$ into ${\widehat{\underline{\overline{\mathbf{h}}}}}_{12}\left(n\right)$ [as shown in (14) and (16)], while using (21) and (23) in (3), we obtain a second equivalent expression for the a priori error, i.e.,

$$\begin{array}{cc}\hfill e\left(n\right)& =d\left(n\right)-{\widehat{\underline{\overline{\mathbf{h}}}}}_{12}^T(n-1){\mathbf{x}}_{2,11}\left(n\right),\hfill \end{array}$$

(26)

where ${\mathbf{x}}_{2,11}\left(n\right)={\widehat{\underline{\overline{\mathbf{H}}}}}_{2,11}^T(n-1)\mathbf{x}\left(n\right)$, with

$$\begin{array}{c}\hfill {\widehat{\underline{\overline{\mathbf{H}}}}}_{2,11}(n-1)=\left[\begin{array}{cccc}{\widehat{\overline{\mathbf{H}}}}_{2,11}^1(n-1)& {\widehat{\overline{\mathbf{H}}}}_{2,11}^2(n-1)& \cdots & {\widehat{\overline{\mathbf{H}}}}_{2,11}^{L_2}(n-1)\end{array}\right]\end{array}$$

and ${\widehat{\overline{\mathbf{H}}}}_{2,11}^l(n-1)=\left[\begin{array}{cccc}{\widehat{\mathbf{H}}}_{2,11}^{l1}(n-1)& {\widehat{\mathbf{H}}}_{2,11}^{l2}(n-1)& \cdots & {\widehat{\mathbf{H}}}_{2,11}^{lP}(n-1)\end{array}\right]$, for $l=1,2,\dots ,L_2$. Finally, grouping the components ${\widehat{\mathbf{h}}}_{11}^{lp}\left(n\right)$ into ${\widehat{\underline{\overline{\mathbf{h}}}}}_{11}\left(n\right)$ [as shown in (15) and (17)], while using (21) and (24) in (3), leads to the third and final way to equivalently express the a priori error, which becomes

$$\begin{array}{cc}\hfill e\left(n\right)& =d\left(n\right)-{\widehat{\underline{\overline{\mathbf{h}}}}}_{11}^T(n-1){\mathbf{x}}_{2,12}\left(n\right),\hfill \end{array}$$

(27)

where ${\mathbf{x}}_{2,12}\left(n\right)={\widehat{\underline{\overline{\mathbf{H}}}}}_{2,12}^T(n-1)\mathbf{x}\left(n\right)$, with

$$\begin{array}{c}\hfill {\widehat{\underline{\overline{\mathbf{H}}}}}_{2,12}(n-1)=\left[\begin{array}{cccc}{\widehat{\overline{\mathbf{H}}}}_{2,12}^1(n-1)& {\widehat{\overline{\mathbf{H}}}}_{2,12}^2(n-1)& \cdots & {\widehat{\overline{\mathbf{H}}}}_{2,12}^{L_2}(n-1)\end{array}\right]\end{array}$$

and ${\widehat{\overline{\mathbf{H}}}}_{2,12}^l(n-1)=\left[\begin{array}{cccc}{\widehat{\mathbf{H}}}_{2,12}^{l1}(n-1)& {\widehat{\mathbf{H}}}_{2,12}^{l2}(n-1)& \cdots & {\widehat{\mathbf{H}}}_{2,12}^{lP}(n-1)\end{array}\right]$, for $l=1,2,\dots ,L_2$.

Similar to (7), in the framework of Kalman filtering and the linear sequential Bayesian approach [49], the updates of the three components result from using the associated expressions of the error signal from (25)–(27), respectively, i.e.,

$$\begin{array}{cc}\hfill {\widehat{\underline{\mathbf{h}}}}_2\left(n\right)& ={\widehat{\underline{\mathbf{h}}}}_2(n-1)+{\mathbf{k}}_2\left(n\right)e\left(n\right),\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill {\widehat{\underline{\overline{\mathbf{h}}}}}_{12}\left(n\right)& ={\widehat{\underline{\overline{\mathbf{h}}}}}_{12}(n-1)+{\mathbf{k}}_{12}\left(n\right)e\left(n\right),\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill {\widehat{\underline{\overline{\mathbf{h}}}}}_{11}\left(n\right)& ={\widehat{\underline{\overline{\mathbf{h}}}}}_{11}(n-1)+{\mathbf{k}}_{11}\left(n\right)e\left(n\right),\hfill \end{array}$$

(30)

where ${\mathbf{k}}_2\left(n\right)$, ${\mathbf{k}}_{12}\left(n\right)$, and ${\mathbf{k}}_{11}\left(n\right)$ are the corresponding Kalman vectors. At this point, we can define the a posteriori misalignments related to the component filters:

$$\begin{array}{cc}\hfill {\tilde{\mathbf{m}}}_2\left(n\right)& ={\underline{\mathbf{h}}}_2\left(n\right)-{\widehat{\underline{\mathbf{h}}}}_2\left(n\right),\hfill \end{array}$$

(31)

$$\begin{array}{cc}\hfill {\tilde{\mathbf{m}}}_{12}\left(n\right)& ={\underline{\overline{\mathbf{h}}}}_{12}\left(n\right)-{\widehat{\underline{\overline{\mathbf{h}}}}}_{12}\left(n\right),\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill {\tilde{\mathbf{m}}}_{11}\left(n\right)& ={\underline{\overline{\mathbf{h}}}}_{11}\left(n\right)-{\widehat{\underline{\overline{\mathbf{h}}}}}_{11}\left(n\right),\hfill \end{array}$$

(33)

with the covariance matrices ${\mathbf{R}}_{{\tilde{\mathbf{m}}}_2}\left(n\right)=E\left[{\tilde{\mathbf{m}}}_2\left(n\right){\tilde{\mathbf{m}}}_2^T\left(n\right)\right]$, ${\mathbf{R}}_{{\tilde{\mathbf{m}}}_{12}}\left(n\right)=E\left[{\tilde{\mathbf{m}}}_{12}\left(n\right){\tilde{\mathbf{m}}}_{12}^T\left(n\right)\right]$, and ${\mathbf{R}}_{{\tilde{\mathbf{m}}}_{11}}\left(n\right)=E\left[{\tilde{\mathbf{m}}}_{11}\left(n\right){\tilde{\mathbf{m}}}_{11}^T\left(n\right)\right]$, respectively. Also, using the estimates from time index $n-1$, the a priori misalignments can be evaluated as

$$\begin{array}{cc}\hfill {\mathbf{m}}_2\left(n\right)& ={\underline{\mathbf{h}}}_2\left(n\right)-{\widehat{\underline{\mathbf{h}}}}_2(n-1),\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill {\mathbf{m}}_{12}\left(n\right)& ={\underline{\overline{\mathbf{h}}}}_{12}\left(n\right)-{\widehat{\underline{\overline{\mathbf{h}}}}}_{12}(n-1),\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill {\mathbf{m}}_{11}\left(n\right)& ={\underline{\overline{\mathbf{h}}}}_{11}\left(n\right)-{\widehat{\underline{\overline{\mathbf{h}}}}}_{11}(n-1),\hfill \end{array}$$

(36)

which are associated with the covariance matrices ${\mathbf{R}}_{{\mathbf{m}}_2}\left(n\right)=E\left[{\mathbf{m}}_2\left(n\right){\mathbf{m}}_2^T\left(n\right)\right]$, ${\mathbf{R}}_{{\mathbf{m}}_{12}}\left(n\right)=E\left[{\mathbf{m}}_{12}\left(n\right){\mathbf{m}}_{12}^T\left(n\right)\right]$, and ${\mathbf{R}}_{{\mathbf{m}}_{11}}\left(n\right)=E\left[{\mathbf{m}}_{11}\left(n\right){\mathbf{m}}_{11}^T\left(n\right)\right]$, respectively.

The Kalman gain vectors required within the updates (28)–(30) result from minimizing the cost functions

$$\begin{array}{cc}\hfill J\left[{\mathbf{k}}_2\left(n\right)\right]& =\frac{1}{L_2^2}\mathrm{tr}\left[{\mathbf{R}}_{{\tilde{\mathbf{m}}}_2}\left(n\right)\right],\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill J\left[{\mathbf{k}}_{12}\left(n\right)\right]& =\frac{1}{P{L}_{12}{L}_2}\mathrm{tr}\left[{\mathbf{R}}_{{\tilde{\mathbf{m}}}_{12}}\left(n\right)\right],\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill J\left[{\mathbf{k}}_{11}\left(n\right)\right]& =\frac{1}{P{L}_{11}{L}_2}\mathrm{tr}\left[{\mathbf{R}}_{{\tilde{\mathbf{m}}}_{11}}\left(n\right)\right].\hfill \end{array}$$

(39)

In each case, the optimization criterion is applied based on a multilinear strategy [51,52,53] by considering that two of the component filters are fixed within the optimization criterion of the third one. Consequently, following (37) and similar to (8) [49], we obtain

$$\begin{array}{cc}\hfill {\mathbf{k}}_2\left(n\right)& =\frac{{\mathbf{R}}_{{\mathbf{m}}_2}\left(n\right){\mathbf{x}}_{12,11}\left(n\right)}{{\mathbf{x}}_{12,11}^T\left(n\right){\mathbf{R}}_{{\mathbf{m}}_2}\left(n\right){\mathbf{x}}_{12,11}\left(n\right)+{\sigma }_v^2},\hfill \end{array}$$

(40)

$$\begin{array}{cc}\hfill {\mathbf{R}}_{{\tilde{\mathbf{m}}}_2}\left(n\right)& =\left[{\mathbf{I}}_{L_2^2}-{\mathbf{k}}_2\left(n\right){\mathbf{x}}_{12,11}^T\left(n\right)\right]{\mathbf{R}}_{{\mathbf{m}}_2}\left(n\right).\hfill \end{array}$$

(41)

Similarly, following (38) and (39), the corresponding Kalman vectors result in

$$\begin{array}{cc}\hfill {\mathbf{k}}_{12}\left(n\right)& =\frac{{\mathbf{R}}_{{\mathbf{m}}_{12}}\left(n\right){\mathbf{x}}_{2,11}\left(n\right)}{{\mathbf{x}}_{2,11}^T\left(n\right){\mathbf{R}}_{{\mathbf{m}}_{12}}\left(n\right){\mathbf{x}}_{2,11}\left(n\right)+{\sigma }_v^2},\hfill \end{array}$$

(42)

$$\begin{array}{cc}\hfill {\mathbf{k}}_{11}\left(n\right)& =\frac{{\mathbf{R}}_{{\mathbf{m}}_{11}}\left(n\right){\mathbf{x}}_{2,12}\left(n\right)}{{\mathbf{x}}_{2,12}^T\left(n\right){\mathbf{R}}_{{\mathbf{m}}_{11}}\left(n\right){\mathbf{x}}_{2,12}\left(n\right)+{\sigma }_v^2},\hfill \end{array}$$

(43)

respectively. Also,

$$\begin{array}{cc}\hfill {\mathbf{R}}_{{\tilde{\mathbf{m}}}_{12}}\left(n\right)& =\left[{\mathbf{I}}_{P{L}_{12}{L}_2}-{\mathbf{k}}_{12}\left(n\right){\mathbf{x}}_{2,11}^T\left(n\right)\right]{\mathbf{R}}_{{\mathbf{m}}_{12}}\left(n\right),\hfill \end{array}$$

(44)

$$\begin{array}{cc}\hfill {\mathbf{R}}_{{\tilde{\mathbf{m}}}_{11}}\left(n\right)& =\left[{\mathbf{I}}_{P{L}_{11}{L}_2}-{\mathbf{k}}_{11}\left(n\right){\mathbf{x}}_{2,12}^T\left(n\right)\right]{\mathbf{R}}_{{\mathbf{m}}_{11}}\left(n\right).\hfill \end{array}$$

(45)

Finally, based on (31)–(36), we obtain the following relations between the covariance matrices:

$$\begin{array}{cc}\hfill {\mathbf{R}}_{{\mathbf{m}}_2}\left(n\right)& ={\mathbf{R}}_{{\tilde{\mathbf{m}}}_2}(n-1)+{\sigma }_{w_2}^2\left(n\right){\mathbf{I}}_{L_2^2},\hfill \end{array}$$

(46)

$$\begin{array}{cc}\hfill {\mathbf{R}}_{{\mathbf{m}}_{12}}\left(n\right)& ={\mathbf{R}}_{{\tilde{\mathbf{m}}}_{12}}(n-1)+{\sigma }_{w_{12}}^2\left(n\right){\mathbf{I}}_{P{L}_{12}{L}_2},\hfill \end{array}$$

(47)

$$\begin{array}{cc}\hfill {\mathbf{R}}_{{\mathbf{m}}_{11}}\left(n\right)& ={\mathbf{R}}_{{\tilde{\mathbf{m}}}_{11}}(n-1)+{\sigma }_{w_{11}}^2\left(n\right){\mathbf{I}}_{P{L}_{11}{L}_2}.\hfill \end{array}$$

(48)

The resulting Kalman filter based on TOT decomposition, namely, KF-TOT, is defined by the relations (28)–(30) and (40)–(48). Within the algorithm, we also need to evaluate the auxiliary matrices ${\widehat{\mathbf{H}}}_{12,11}^{lp}\left(n\right)$, ${\widehat{\mathbf{H}}}_{2,11}^{lp}\left(n\right)$, and ${\widehat{\mathbf{H}}}_{2,12}^{lp}\left(n\right)$, according to (22)–(24), respectively. These auxiliary matrices are used to evaluate the “inputs” ${\mathbf{x}}_{12,11}\left(n\right)$, ${\mathbf{x}}_{2,11}\left(n\right)$, and ${\mathbf{x}}_{2,12}\left(n\right)$, as defined after (25)–(27), respectively.

The initialization setup of the KF-TOT is

$$\begin{array}{cc}\hfill {\widehat{\mathbf{h}}}_2^l\left(0\right)& ={\left[\begin{array}{cc}\xi & {\mathbf{0}}_{L_2-1}^T\end{array}\right]}^T,\hfill \end{array}$$

(49)

$$\begin{array}{cc}\hfill {\widehat{\mathbf{h}}}_{12}^{lp}\left(0\right)& ={\left[\begin{array}{cc}\xi & {\mathbf{0}}_{L_{12}-1}^T\end{array}\right]}^T,\hfill \end{array}$$

(50)

$$\begin{array}{cc}\hfill {\widehat{\mathbf{h}}}_{11}^{lp}\left(0\right)& ={\left[\begin{array}{cc}\xi & {\mathbf{0}}_{L_{11}-1}^T\end{array}\right]}^T,\hfill \end{array}$$

(51)

with $l=1,2,\dots ,L_2$, $p=1,2,\dots ,P$, and $0<\xi \le 1$; here, ${\mathbf{0}}_{•}$ generally denotes an all-zeros vector with the length indicated in subscript. In addition, we need to initialize ${\mathbf{R}}_{{\tilde{\mathbf{m}}}_2}\left(0\right)=\zeta {\mathbf{I}}_{L_2^2}$, ${\mathbf{R}}_{{\tilde{\mathbf{m}}}_{12}}\left(0\right)=\zeta {\mathbf{I}}_{P{L}_{12}{L}_2}$, and ${\mathbf{R}}_{{\tilde{\mathbf{m}}}_{11}}\left(0\right)=\zeta {\mathbf{I}}_{P{L}_{11}{L}_2}$, where $\zeta$ is a positive constant.

The additive noise power ${\sigma }_v^2$ is required within the evaluation of the Kalman gain vectors from (40), (42) and (43). This parameter could be a priori available or estimated within the algorithm, e.g., during the silence periods of a talker (in echo cancellation or noise reduction applications). Other methods for estimating ${\sigma }_v^2$ can be found in [7,54].

The other important parameters of the KF-TOT are the uncertainties ${\sigma }_{w_2}^2\left(n\right)$, ${\sigma }_{w_{12}}^2\left(n\right)$, and ${\sigma }_{w_{11}}^2\left(n\right)$, which are required in (46)–(48). Similar to the conventional Kalman filter and the influence of its uncertainties [7], small values of these parameters lead to a good accuracy of the solution (i.e., low misalignment), but a slow tracking reaction in case of an abrupt change in the unknown system. On the other hand, large uncertainties keep the adaptive system alert (i.e., good tracking behavior), but sacrifice in terms of accuracy. Using constant values for these parameters (e.g., a priori set) leads to an inherent compromise between the main performance criteria. In order to improve this aspect, the uncertainties can be evaluated within the algorithm, in a time-dependent manner, by using a certain measure that captures the variation in the system from one iteration to another. Both approaches will be analyzed in the next section, in the context of the experimental results.

The computational complexity order of the conventional Kalman filter (KF) is ${\mathcal{O}}_{\mathrm{KF}}=L^2$, since it is proportional to the square of the filter (full) length. On the other hand, the proposed KF-TOT combines the solutions of three shorter Kalman filters, of lengths $L_2^2$, $P{L}_{12}{L}_2$, and $P{L}_{11}{L}_2$, with $L=L_{11}{L}_{12}{L}_2$. Thus, its complexity order is ${\mathcal{O}}_{\mathrm{KF}-\mathrm{TOT}}=L_2^4+{\left(P{L}_{12}{L}_2\right)}^2+{\left(P{L}_{11}{L}_2\right)}^2$. These complexity orders are provided in

Table 1. Complexity orders ($\times {10}^3$) of the conventional KF ($L=512$) and KF-TOT ($L_{11}=L_{12}=16$ and $L_2=2$).

${\mathcal{O}}_{\mathbf{KF}}$	${\mathcal{O}}_{\mathbf{KF}-\mathbf{TOT}}$ ($\mathit{P}=1$)	${\mathcal{O}}_{\mathbf{KF}-\mathbf{TOT}}$ ($\mathit{P}=2$)	${\mathcal{O}}_{\mathbf{KF}-\mathbf{TOT}}$ ($\mathit{P}=3$)
262.144	2.064	8.208	18.448

, related to the example from Section 2.3, where $L=512$, so that the TOT factorization uses $L_{11}=L_{12}=16$ and $L_2=2$. For the decomposition parameter of the KF-TOT, we consider $P=1,2,$ and 3 (as also supported in the next section). As we can notice in Table 1, the complexity order of the KF-TOT is much lower as compared to its conventional counterpart.

4. Simulation Results

The experimental setup considers an echo cancellation scenario, which basically represents a system identification problem, where the main goal is to model (or estimate) the impulse response of an echo path [45,47,48]. In our experimental framework, this impulse response is obtained based on the first cluster from G.168 ITU-T recommendation [55], which is padded with zeros up to the length $L=512$; the sampling rate is 8 kHz. As indicated by the example provided in Section 2.3, this length can be factorized using $L_{11}=L_{12}=16$ and $L_2=2$, which represents the TOT decomposition setup. In order to assess the tracking behavior of the algorithms, an echo path change scenario is considered in some of the experiments, by changing the sign of the coefficients (of the impulse response) at a certain moment of time. The performance of the algorithms is evaluated in terms of the normalized misalignment (in dB), which is defined as $20{\log }_{10}{∥\mathbf{h}\left(n\right)-\widehat{\mathbf{h}}\left(n\right)∥}_2/{∥\mathbf{h}\left(n\right)∥}_2$, where the notation ${∥·∥}_2$ stands for the Euclidean norm. When the echo path $\mathbf{h}\left(n\right)$ abruptly changes, the reference/desired signal also changes according to (2), which results in a sudden increase in the error signal and, consequently, in the normalized misalignment (due to the bias of the filter estimate). Therefore, it is desirable that the algorithm should recover (as fast as possible) after this change, which depends on its tracking capability.

The input signal is either a stationary sequence obtained by filtering a white Gaussian noise through a first-order autoregressive (AR) model with the pole at 0.8, thus resulting in an AR(1) process, or a nonstationary sequence obtained from a recorded speech signal of a female voice. The AR(1) process has a high correlation degree, since the pole is set closer to 1; similar results can be obtain using higher-order AR processes. Also, using speech as input is a challenging case (specific to echo cancellation), since this is a nonstationary and highly correlated signal. The output of the echo path is corrupted by a white Gaussian noise, $v\left(n\right)$, with different levels of the signal-to-noise ratio (SNR). This is defined as ${\sigma }_y^2/{\sigma }_v^2$, where ${\sigma }_y^2=E\left[y^2\left(n\right)\right]$ stands for the variance in the output signal, $y\left(n\right)={\mathbf{h}}^T\left(n\right)\mathbf{x}\left(n\right)$. In echo cancellation, $v\left(n\right)$ is the background noise, which biases the adaptive filter estimate [48]. Three scenarios are considered in the experiments, using $\mathrm{SNR}=20$ dB, 10 dB, and 0 dB, which correspond to mild, moderate, and heavy noise conditions, respectively. An SNR level of 20 dB or higher corresponds to favorable conditions, with low background noise, so that a good accuracy of the adaptive filter (i.e., echo canceler) is expected. When the SNR is decreased to 10 dB, the background noise already has a noticeable influence, which is reflected in higher misalignment. It should be outlined that in echo cancellation applications the background noise could be significant, thus influencing the accuracy of the estimates, while the algorithms should be robust in different noisy environments. This is the reason for also including in the experiments a heavy noise scenario with $\mathrm{SNR}=0$ dB, in order to test the robustness of the algorithms under this challenging condition.

As explained in Section 3, the KF-TOT requires ${\sigma }_v^2$ within the computation of its Kalman gain vectors. This is also valid for the conventional Kalman filter, according to (9). While there are different approaches for estimating this parameter in practice [7,54], the influence of these methods on the overall performance of the KF-TOT is beyond the scope of this paper. Consequently, we consider that ${\sigma }_v^2$ is available in all the experiments reported in this section. It is also important to note that the so-called regularization parameter is usually related to the value of SNR, i.e., the lower the SNR, the larger the regularization term should be [48]. In case of the KF-TOT, such a parameter (i.e., $\zeta$) is used in the initialization step, for the matrices ${\mathbf{R}}_{{\tilde{\mathbf{m}}}_2}\left(0\right)$, ${\mathbf{R}}_{{\tilde{\mathbf{m}}}_{12}}\left(0\right)$, and ${\mathbf{R}}_{{\tilde{\mathbf{m}}}_{11}}\left(0\right)$. Hence, a simple and practical way to relate its value to the SNR level is $\zeta =1/\mathrm{SNR}$.

4.1. Parameters of KF-TOT

In the first set of experiments, we focus on the main parameters of KF-TOT in order to outline their influence on the overall performance of the algorithm. In these simulations, the input signal is an AR(1) process and $\mathrm{SNR}=20$ dB. First, we assess the influence of the decomposition parameter P. As shown in [36], its values are usually much smaller than $L_{12}$, while for network impulse responses (like the one involved in the experiments), the value of P is in the vicinity of $L_{12}/8$. In Figure 1, three values of P are considered within the KF-TOT, while using null uncertainty parameters, i.e., ${\sigma }_{w_2}^2\left(n\right)={\sigma }_{w_{12}}^2\left(n\right)={\sigma }_{w_{11}}^2\left(n\right)=0$. As explained in Section 3, this setting targets the best accuracy of the algorithm, i.e., the lowest misalignment, while sacrificing in terms of tracking (as shown in the next experiment). First, we can notice in Figure 1 that the KF-TOT using $P=1$ leads to a reasonable attenuation of the misalignment, with the smallest parameter space, i.e., using only 68 coefficients, according to the example provided in Section 2.3, related to (12). Increasing the value of P improves the performance of the KF-TOT, but up to a certain limit, which is related to the low-rank features of the impulse response to be identified. As we can notice in Figure 1, using $P=2$ and 3 leads to similar performances. Consequently, increasing the value of P beyond this limit will not bring additional gains. Moreover, two of the three component filters of KF-TOT have $P{L}_{11}{L}_2$ and $P{L}_{12}{L}_2$ coefficients, respectively. Hence, increasing the value of P also leads to a larger parameter space, which is not justified in terms of better performance. In our scenario, a value of $P=2$ represents a reasonable choice, taking into account the compromise between performance and complexity. In this case, the parameter space of KF-TOT comprises 132 coefficients (as also indicated in Section 2.3), still representing a significant reduction as compared to the length $L=512$ of the full impulse response.

Next, in Figure 2, we analyze the tracking capabilities of the KF-TOT (with $P=2$) when using constant values of the uncertainties parameters. For the sake of simplicity, we generally denote these parameters by ${\sigma }_{\ast }^2\left(n\right)$, where $\ast \in \{2,12,11\}$. The echo path changes after 15 s from the start of the experiment. As outlined in the previous simulation (related to Figure 1), the values ${\sigma }_{\ast }^2\left(n\right)=0$ lead to good accuracy, but poor tracking, as we can also notice in Figure 2. Reducing the values of ${\sigma }_{\ast }^2\left(n\right)$ improves the tracking but increases the misalignment (i.e., reduces the accuracy of the estimate). Consequently, a compromise should be considered between these main performance criteria.

For this purpose, it would be useful to evaluate the uncertainties parameters within the algorithm, using a certain measure that captures the variation from $\widehat{\mathbf{h}}(n-1)$ to $\widehat{\mathbf{h}}\left(n\right)$. A simple and practical way to assess this issue was proposed in [7], by computing in each iteration the quantity ${∥\widehat{\mathbf{h}}\left(n\right)-\widehat{\mathbf{h}}(n-1)∥}_2^2/L$. Setting the values of ${\sigma }_{\ast }^2\left(n\right)$ in this manner allows a better compromise between convergence/tracking and misalignment, as outlined in the rest of the experiments from this section. In other words, a small difference between $\widehat{\mathbf{h}}\left(n\right)$ and $\widehat{\mathbf{h}}(n-1)$ indicates that the algorithm is in steady-state, where small values of ${\sigma }_{\ast }^2\left(n\right)$ should be used. On the other hand, a large difference between the estimates from two consecutive time indices requires a fast reaction of the algorithm, by increasing the values of ${\sigma }_{\ast }^2\left(n\right)$.

4.2. Conventional Kalman Filter versus KF-TOT

In the second set of experiments, we compare the performance of the conventional KF with the proposed KF-TOT (using $P=2$). For a fair comparison, both algorithms evaluate the uncertainty parameters in the same manner, as explained before. The input signal is an AR(1) process, the echo path changes after 3.75 s from the beginning of simulation, and different SNRs are used. First, in Figure 3, we set $\mathrm{SNR}=20$ dB. As we can notice, the KF-TOT outperforms the conventional KF in terms of both performance criteria, i.e., convergence/tracking and misalignment. This is due to the fact that the conventional KF involves a single (long-length) filter, with $L=512$, while the proposed KF-TOT combines the estimates provided by three (much) shorter filters, with $P{L}_{12}{L}_2=64$, $P{L}_{11}{L}_2=64$, and $L_2^2=4$ coefficients, respectively.

These performance gains are also valid in low SNR environments, as we can notice in Figure 4a,b, where the SNR is set to 10 dB and 0 dB, respectively. Even if there is a slightly slower tracking reaction of the KF-TOT in heavy noise conditions, like in Figure 4b, it still outperforms by far its conventional KF counterpart.

4.3. KF-NKP versus KF-TOT

In the third set of experiments, the NKP-based version of the Kalman filter from [27] is involved in comparisons, namely, KF-NKP. This is also a decomposition-based algorithm, but exploits a second-order level, combining the solutions of two shorter filters of lengths $P^{*}{L}_1^{*}$ and $P{L}_2^{*}$, where the length of the global filter is factorized as $L=L_1^{*}{L}_2^{*}$, while $P^{*}<L_2^{*}$. In our scenario, since $L=512$, we use the decomposition setup $L_1^{*}=32$ and $L_2^{*}=16$. In Figure 5, the input signal is a speech sequence, $\mathrm{SNR}=20$ dB, and different values of $P^{*}$ and P are used for the KF-NKP and KF-TOT, respectively. As we can notice, the KF-TOT outperforms the KF-NKP for all the corresponding values of the decomposition parameters. In the case of the KF-NKP, the value of $P^{*}$ is related to the rank of the corresponding matrix (of size $L_1^{*}\times L_2^{*}$) associated with the reshaped impulse response [27]. In the case of the impulse response used in the experiments, this rank is equal to 3, so that increasing the value of $P^{*}$ beyond this value does not lead to any performance improvement.

These decomposition-based Kalman filters are further analyzed in Figure 6, using lower SNRs and considering a change in the echo path (after 3.75 s). In both scenarios (i.e., $\mathrm{SNR}=10$ dB and 0 dB), the proposed KF-TOT achieves better performance than the KF-NKP, in terms of both the tracking reaction and the misalignment level. Moreover, as supported in Figure 6b, the KF-TOT can operate with $P<P^{*}$, while still outperforming its KF-NKP counterpart.

4.4. RLS-TOT versus KF-TOT

Finally, in the fourth set of experiments, the recently developed RLS-TOT [37] is involved in comparisons. It is known that there are inherent similarities between RLS-based algorithms and the Kalman filter [2,7]. However, due to its specific parameters, the Kalman filter allows a better control, especially in nonstationary conditions. In the case of the RLS-TOT, its overall performance is controlled in terms of the forgetting factors, denoted by ${\lambda }_{*}$, where $*\in \{2,12,11\}$. These are positive constants, which are smaller or equal to one. Larger values of ${\lambda }_{*}$ (i.e., close or equal to one) lead to good accuracy of the filter estimate, thus reaching lower misalignment levels. On the other hand, the tracking reaction of the algorithm is reduced. This can be improved by decreasing the values of ${\lambda }_{*}$, while sacrificing in terms of accuracy (i.e., increasing the misalignment). In this context, a reliable compromise between these main performance criteria was proposed in [37], by using ${\lambda }_2=1$ (which corresponds to the shortest filter of length $L_2^2$), while the other two forgetting factors are set according to ${\lambda }_{12}=1-1/\left(KP{L}_{12}{L}_2\right)$ and ${\lambda }_{11}=1-1/\left(KP{L}_{11}{L}_2\right)$, with $K>1$. In the scenario analyzed in Figure 7, different values of K are used for the RLS-TOT. The input signal is speech, $\mathrm{SNR}=20$ dB, the echo path changes after 3.75 s, and both TOT-based algorithms use the same value of the decomposition parameter, which is set to $P=2$. First, we can notice the compromise between the main performance criteria of the RLS-TOT (i.e., tracking versus misalignment) when using different values of K, i.e., different forgetting factors. On the other hand, the proposed KF-TOT achieves a much better tracking behavior, while still reaching a lower misalignment level.

The accuracy of the solution provided by the KF-TOT can also be achieved in lower SNR environments, as supported in Figure 8a,b, where $\mathrm{SNR}=10$ dB and 0 dB, respectively. The other conditions are basically the same as in the previous experiment, but without changing the echo path, in order to focus only on the accuracy of the estimate. As expected, increasing the values of the forgetting factors of RLS-TOT (i.e., increasing the values of K) leads to lower misalignment levels. Nevertheless, the KF-TOT provides a smooth convergence behavior toward a low steady-state misalignment level. To reach this accuracy, the forgetting factors of the RLS-TOT should be further increased, but the price to pay will be a poor tracking reaction in nonstationary environments (as previously outlined in Figure 7). Concluding, taking into account its overall performance, the KF-TOT represents a more reliable choice.

5. Conclusions and Future Work

In the current paper, following the recently proposed TOT-based decomposition of the impulse response [36], we have developed a Kalman filter that exploits this technique, in the framework of system identification. The proposed KF-TOT combines (with the Kronecker product) the estimates provided by three filters, which can be much shorter as compared to the full length of the global impulse response. Since the length of an adaptive filter is related to its main performance criteria, this decomposition-based solution leads to important gains. As a result, following the announced goals from Section 1, the KF-TOT first outperforms the conventional Kalman filter in terms of both convergence rate and tracking, but also in terms of accuracy, reaching a significantly lower misalignment level. In addition, as supported in Table 1, these performance gains are achieved owning a lower computational complexity. Second, the KF-TOT leads to improved performances as compared to its KF-NKP counterpart [27], which exploits a second-order decomposition level by combining the estimates provided by two filters. Third, based on the evaluation of the KF-TOT as compared to the recently developed RLS-TOT [37], we reach the conclusion that the proposed algorithm is more reliable in nonstationary conditions, owning to a better tracking behavior and an improved accuracy of the solution.

Due to its performance features, the proposed KF-TOT fits very well for applications that involve the identification of long-length low-rank impulse responses, especially for time-varying systems. In this context, echo cancellation would represent an excellent application framework, since the echo paths (for both network and acoustic scenarios) are characterized by impulse responses that could easily reach hundreds/thousands of coefficients, while also being highly variable in time. Since this is a real-time application, a fast identification (and tracking) of the echo path impulse response, with low computational demands, would be a very appealing asset. In addition, different long-length low-rank systems can be involved in various applications, which include digital TV transmission [56], satellite-linked communications [57], underwater communications [58], and adaptive beamformers [59]. Hence, the developed KF-TOT could also represent a practical solution for such real-world applications.

Future work will focus on analyzing a higher-order decomposition level in conjunction with the Kalman filter. In this context, the main challenge is related to the way we handle the rank of a higher-order tensor that results from the development. Usually, approximation techniques are used to find such a tensor rank, which could represent a practical limitation. In the TOT framework, i.e., third-order decomposition level [36], the corresponding tensor associated with the reshaped impulse response results as a sum of P third-order tensors of rank $L_2$, so this parameter is controlled and limited to small values. Since there is not a straightforward extension from one decomposition level to another, applying the decomposition technique to a higher order while still controlling the tensor rank represents a challenging task. Several preliminary ideas can be found in [60]. Also, we aim to involve the KF-TOT and the related decomposition-based algorithms in the framework of other applications that require the identification of long-length impulse responses, especially related to the acoustic domain.