Wiener Filter Using the Conjugate Gradient Method and a Third-Order Tensor Decomposition

Abstract

In linear system identification problems, the Wiener filter represents a popular tool and stands as an important benchmark. Nevertheless, it faces significant challenges when identifying long-length impulse responses. In order to address the related shortcomings, the solution presented in this paper is based on a third-order tensor decomposition technique, while the resulting sets of Wiener–Hopf equations are solved with the conjugate gradient (CG) method. Due to the decomposition-based approach, the number of coefficients (i.e., the parameter space of the filter) is greatly reduced, which results in operating with smaller data structures within the algorithm. As a result, improved robustness and accuracy can be achieved, especially in harsh scenarios (e.g., limited/incomplete sets of data and/or noisy conditions). Besides, the CG-based solution avoids matrix inversion operations, together with the related numerical and complexity issues. The simulation results are obtained in a network echo cancellation scenario and support the performance gain. In this context, the proposed iterative Wiener filter outperforms the conventional benchmark and also some previously developed counterparts that use matrix inversion or second-order tensor decompositions.

1. Introduction

Linear system identification problems have to be worked out in the context of various applications [1,2], including echo cancellation, active noise control, interference reduction, and channel modeling among others. A benchmark technique used to address such problems is the well-known Wiener filter, which basically relies on solving a linear system (namely the Wiener–Hopf equations) using a set of statistics. The Wiener–Hopf equations involve a set of estimates for the covariance matrix of the input signal and the cross-correlation vector between the input and reference sequences. The problem is formulated following an optimization criterion in terms of minimizing the mean-squared error (MSE), while the error is defined as the difference between the reference sequence and the output signal. The resulting optimal filter also represents an important basis for the development of other related tools for system identification problems, such as adaptive filtering algorithms [3,4].

There are some inherent limitations associated with the conventional Wiener filter solution, which is obtained by directly solving (using matrix inversion) the Wiener–Hopf equations. First, the accuracy of the solution is highly influenced by the accuracy of the statistics’ estimates. On the other hand, obtaining a reliable set of these estimates requires a large amount of data, i.e., much larger than the filter length. This could represent a significant shortcoming when dealing with a limited (or incomplete) amount of data and/or a long-length filter. Second, the external noise (that is part of the reference signal) biases the Wiener filter solution, which becomes less accurate when the signal-to-noise ratio (SNR) decreases. This could be the case in noisy environments, where different types of perturbations are likely to emerge. Third, the conventional solution involves the covariance matrix inversion, which is a very challenging operation in terms of both computational complexity and numerical accuracy [5,6]. The difficulty could increase significantly when operating with long-length filters, which further entail large dimension matrices.

Most of the previously discussed limitations are connected to the length of the filter, which could be very large in many scenarios. For example, in applications like echo cancellation and noise reduction [7], the acoustic impulse responses to be identified have hundreds/thousands of coefficients when using the common sampling rate of 8 or 16 kHz. Therefore, dealing with such long-length filters could lead to significant limitations in terms of both the accuracy and complexity of the solution. In order to reformulate such high-dimension system identification problems (with a large parameter space) more efficiently, a recently developed decomposition-based technique has been involved [8]. The main idea behind this technique is to exploit the low-rank feature of the system impulse response, in conjunction with its nearest Kronecker product (NKP) decomposition. As a result, a system identification problem featuring a large parameter space is designed as a combination of two shorter filters, with a significantly reduced number of coefficients. This further implies the operation with smaller matrices/vectors and, consequently, leads to improved robustness in terms of the accuracy of the final solution, even for the challenging cases mentioned above (e.g., a limited amount of data and/or low SNRs). Due to these important gains, the NKP-based approach has been involved in a wide range of applications, among which can be mentioned echo cancellation, adaptive beamforming, linear prediction, speech dereverberation, and microphone arrays, e.g., see [9,10,11,12,13,14,15,16,17] and the references therein.

Recently, the NKP technique has been applied in conjunction with a third-order tensor (TOT) decomposition of the impulse response [18], leading to a higher efficiency in terms of reducing the dimensionality of the system identification problem. This was not a straightforward extension of the low-rank approach presented in [8], since handling the rank of a tensor is a sensitive issue that usually involves different approximation techniques [19,20,21,22,23,24,25,26,27,28]. On the other hand, the solution proposed in [18] avoids such an approximation, by controlling and limiting the tensor rank to very small values. However, the resulting Wiener filter based on TOT decomposition solves the involved sets of Wiener–Hopf equations using the conventional approach, which relies on matrix inversion. Alternatively, different iterative techniques could be used to avoid such an operation [29,30,31], like the conjugate gradient (CG) method [32]. In [33], the CG algorithm has been applied in conjunction with the NKP-based technique from [8], showing improved performance. However, applying the CG method together with the TOT decomposition is a more challenging task, due to the particular connection between the three (shorter) component filters and the need for auxiliary variables within the algorithm.

Motivated by these aspects, in the current paper, we design an improved iterative version of the Wiener filter. The proposed algorithm involves the TOT-based decomposition, together with the CG method to solve three sets of Wiener–Hopf equations. As a result, it outperforms the counterpart version from [18], which uses the direct matrix inversion for solving the Wiener–Hopf equations, and also the CG-based solution from [33], which exploits the second-order NKP decomposition. Following this introduction, in Section 2 we provide some background on the conventional Wiener filter, the CG method (to avoid matrix inversion), and the TOT-based decomposition. Next, in Section 3, the proposed algorithm is developed. Simulation results provided in Section 4 support its performance and advantages compared to the existing solutions. The paper is summarized in Section 5, outlining the main conclusions and several perspectives for future works.

2. Conventional Wiener Filter, Conjugate Gradient Method, and Impulse Response Decomposition Based on a Third-Order Tensor

In this section, several backgrounds related to the upcoming developments are provided. First, we present the conventional Wiener filter for solving linear system identification problems. Next, the CG method is introduced as an efficient (iterative) alternative to avoid the matrix inversion required by the direct solution of Wiener–Hopf equations. Finally, the TOT-based decomposition of the impulse response is presented, outlining the main idea recently introduced in [18].

The main framework considered in this paper is related to a single-input single-output (SISO) linear system identification scenario, where all the involved signals are zero-mean and real-valued. In this context, the available signals are the input $x\left(t\right)$ and the reference $d\left(t\right)$, while t represents the discrete-time index. Under this scenario, there is a correlation between these two sequences, since the reference signal is obtained at the output of an unknown system driven by the input signal, while the output is corrupted by an additive noise, as shown in Figure 1. Thus,

$$\begin{array}{cc}\hfill d\left(t\right)& ={\mathbf{h}}^T\mathbf{x}\left(t\right)+v\left(t\right)\hfill \\ \hfill & =y\left(t\right)+v\left(t\right),\hfill \end{array}$$

(1)

where the vector h contains the L coefficients of the unknown impulse response (with superscript ${ }^T$ denoting transposition), $\mathbf{x}\left(t\right)={\left[\begin{array}{cccc}x\left(t\right)& x(t-1)& \cdots & x(t-L+1)\end{array}\right]}^T$ is a vector that contains the L most recent time samples of the input signal $x\left(t\right)$, and $v\left(t\right)$ is an additive noise, which is uncorrelated with $x\left(t\right)$. In the second line of (1), $y\left(t\right)={\mathbf{h}}^T\mathbf{x}\left(t\right)$ represents the output signal.

Based on the correlation between the reference sequence and the input signal, and following the MSE optimization criterion, an estimate of $\mathbf{h}$ can be obtained by solving the Wiener–Hopf equations [2], i.e.,

$$\begin{array}{c}\hfill {\mathbf{R}}_{\mathbf{x}}{\mathbf{h}}_{\mathrm{W}}={\mathbf{r}}_{\mathbf{x}d},\end{array}$$

(2)

where

$$\begin{array}{cc}\hfill {\mathbf{R}}_{\mathbf{x}}& =E\left[\mathbf{x}\left(t\right){\mathbf{x}}^T\left(t\right)\right],\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill {\mathbf{r}}_{\mathbf{x}d}& =E\left[\mathbf{x}\left(t\right)d\left(t\right)\right]\hfill \end{array}$$

(4)

represent the covariance matrix of the input signal and the cross-correlation vector between the input and reference sequences, respectively, ${\mathbf{h}}_{\mathrm{W}}$ contains the coefficients of the Wiener filter (i.e., L parameters), while $E(·)$ denotes mathematical expectation. Thus, the conventional Wiener filter results by using the matrix inversion operation, so that

$$\begin{array}{c}\hfill {\mathbf{h}}_{\mathrm{W}}={\mathbf{R}}_{\mathbf{x}}^{-1}{\mathbf{r}}_{\mathbf{x}d}.\end{array}$$

(5)

In order to avoid matrix inversion, several alternative methods for solving (5) can be applied. The basic idea is to obtain the final solution in an iterative manner. In this context, the CG method [32] represents a popular choice, being included in the category of exact line search methods [34,35]. Hence, considering the initialization ${\mathbf{h}}_{\mathrm{W}\left(0\right)}$, an initial residual ${\mathbf{z}}_{\left(0\right)}={\mathbf{r}}_{\mathbf{x}d}-{\mathbf{R}}_{\mathbf{x}}{\mathbf{h}}_{\mathrm{W}\left(0\right)}$ can be computed. Also, in the initial step, it requires a conjugate vector ${\mathbf{c}}_{\left(0\right)}={\mathbf{z}}_{\left(0\right)}$ and an auxiliary scalar ${\gamma }_{\left(0\right)}={\mathbf{z}}_{\left(0\right)}^T{\mathbf{z}}_{\left(0\right)}$. Using this initialization, the CG algorithm runs for k steps, each one involving the relations:

$$\begin{array}{cc}\hfill {\mathbf{q}}_{\left(k\right)}& ={\mathbf{R}}_{\mathbf{x}}{\mathbf{c}}_{(k-1)},\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill {\alpha }_{\left(k\right)}& =\frac{{\gamma }_{(k-1)}}{{\mathbf{c}}_{(k-1)}^T{\mathbf{q}}_{\left(k\right)}},\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill {\mathbf{h}}_{\mathrm{W}\left(k\right)}& ={\mathbf{h}}_{\mathrm{W}(k-1)}+{\alpha }_{\left(k\right)}{\mathbf{c}}_{(k-1)},\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill {\mathbf{z}}_{\left(k\right)}& ={\mathbf{z}}_{(k-1)}-{\alpha }_{\left(k\right)}{\mathbf{q}}_{\left(k\right)},\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill {\gamma }_{\left(k\right)}& ={\mathbf{z}}_{\left(k\right)}^T{\mathbf{z}}_{\left(k\right)},\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill {\beta }_{\left(k\right)}& =\frac{{\gamma }_{\left(k\right)}}{{\gamma }_{(k-1)}},\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill {\mathbf{c}}_{\left(k\right)}& ={\mathbf{z}}_{\left(k\right)}+{\beta }_{\left(k\right)}{\mathbf{c}}_{(k-1)}.\hfill \end{array}$$

(12)

The stopping criterion can be related to a maximum number of updates (i.e., for $k=1,2,\dots ,K$) or a predefined threshold for the residual.

The convergence of the CG algorithm is influenced by the condition number of ${\mathbf{R}}_{\mathbf{x}}$ [5], i.e., the larger this number, the slower the convergence is. In order to improve the convergence rate, a preconditioning procedure could be applied to this matrix. There are different methods for choosing the so-called preconditioner (i.e., a matrix that multiplies ${\mathbf{R}}_{\mathbf{x}}$), like Jacobi, Gauss–Seidel, etc. Basically, the algorithm from (6)–(12) is reformulated, while incorporating the preconditioning directly into the iteration. On the other hand, this procedure involves an additional computational amount. Nevertheless, the purpose of this paper is not to analyze the influence of different preconditioners on the overall performance of the CG algorithm. Our primary goal is to develop the decomposition-based approach in conjunction with the CG method for solving the Wiener–Hopf equations. In this context, the main challenges are related to the connection between the component filters and the specific initialization (as will be shown in the next section), and not to the performance of the CG algorithm itself. Consequently, in the following, the basic CG algorithm from (6)–(12) is considered without preconditioning. However, in order to keep the positive-definite character of the covariance matrix and to avoid any potential numerical/stability problems [6], it is recommended to add a very small positive constant to the elements of the main diagonal.

The maximum number of updates required by the Wiener filter using the CG method (namely WF-CG) to reach the solution of the conventional Wiener filter (WF) is generally much smaller compared to the filter length. This is supported in Figure 2 and Figure 3, where the performances of the conventional WF and WF-CG are analyzed in two different scenarios for the identification of a network echo path of length $L=512$ (using a sampling rate of 8 kHz). This impulse response results from the first cluster of ITU-T G168 Recommendation [36] concerning digital network echo cancellers. It contains 64 coefficients padded with zeros up to the full-length L. The required statistics (${\mathbf{R}}_{\mathbf{x}}$ and ${\mathbf{r}}_{\mathbf{x}d}$) are estimated by averaging across $N=ML$ data samples of $x\left(t\right)$ and $d\left(t\right)$, with $M>1$. The reference signal is obtained according to (1), using a first-order autoregressive [AR(1)] process as input, which results from filtering white Gaussian noise through an AR(1) model; the pole of this model is set to 0.8. The additive noise is white and Gaussian, with $\mathrm{SNR}={\sigma }_y^2/{\sigma }_v^2$, where ${\sigma }_y^2$ and ${\sigma }_v^2$ stand for the variances of $y\left(t\right)$ and $v\left(t\right)$, respectively. The results are shown using a common performance measure involved in system identification scenarios, which is the normalized misalignment (in dB). It is defined as $20{\log }_{10}{∥\mathbf{h}-{\mathbf{h}}_{\mathrm{W}}∥}_2/{∥\mathbf{h}∥}_2$ (where ${∥·∥}_2$ denotes the Euclidean norm) and basically shows the “difference” between the true impulse response and the estimated one. The lower this quantity, the better the accuracy of the estimate.

In the first scenario considered in Figure 2, we evaluate the impact of using different amounts of data ($N=ML$) to estimate the statistics by varying the value of M. It can be noticed that a low amount of data significantly influences the accuracy of the Wiener solution. Nevertheless, the WF-CG converges toward the conventional WF after a small number of CG iterations (as compared to the filter length). Second, in Figure 3, the SNR influence is outlined. As expected, a lower SNR reduces the accuracy of the Wiener estimate. Similarly to the previous experiment, the WF-CG reaches the conventional WF for $K\ll L$. Both analyzed scenarios support the influence of the main factors that affect the behavior of the Wiener filter, i.e., the amount of available data for estimating the statistics and the SNR level. Thus, it is of great importance to improve the overall performance and robustness related to these aspects.

In terms of computational complexity, the conventional Wiener solution based on matrix inversion requires $\mathcal{O}\left(L^3\right)$ operations, while the iterative version that uses the CG method needs an $\mathcal{O}\left(K{L}^2\right)$ amount, with $K\ll L$. Nevertheless, when identifying a long-length impulse response, a large value of K could be required for the CG iterations. This also represents a motivation for the dimensionality reduction of the problem, by reformulating a system identification scenario with a large parameter space (i.e., a large number of coefficients) as a combination of the estimates provided by shorter filters.

In this regard, the recent solution from [18] is based on a third-order tensor decomposition of the impulse response, namely TOT decomposition. As a result, the final estimate results as a combination (via the Kronecker product) of the coefficients associated with three filters, which are significantly shorter (as compared to the original impulse response). This idea is briefly explained in the following. First, let us consider that the length of the filter can be factorized as $L=L_1{L}_2$, with $L_1\gg L_2$, so that the impulse response of the system results in [8]

$$\begin{array}{c}\hfill \mathbf{h}=\sum _{i=1}^{L_2}{\mathbf{h}}_2^i\otimes {\mathbf{h}}_1^i.\end{array}$$

(13)

Here, the shorter impulse responses ${\mathbf{h}}_1^i$ and ${\mathbf{h}}_2^i$ have the lengths $L_1$ and $L_2$, respectively, while ⊗ denotes the Kronecker product [37]. At this point, let us assume that ${\mathbf{h}}_1^i$ is low rank [8]. Moreover, its length can be factorized as $L_1=L_{11}{L}_{12}$ (with $L_{11}\ge L_{12}$). Consequently, the global impulse response results in

$$\begin{array}{c}\hfill \mathbf{h}=\sum _{i=1}^{L_2}\sum _{j=1}^P{\mathbf{h}}_2^i\otimes {\mathbf{h}}_{12}^{ij}\otimes {\mathbf{h}}_{11}^{ij},\end{array}$$

(14)

where the two impulse responses ${\mathbf{h}}_{11}^{ij}$ and ${\mathbf{h}}_{12}^{ij}$ have the lengths $L_{11}$ and $L_{12}$, respectively, while $P<L_{12}$. It can be noticed that the coefficients of $\mathbf{h}$ can be “rearranged” in the form of a third-order tensor, i.e.,

$$\begin{array}{c}\hfill \mathcal{H}=\sum _{j=1}^P\left(\sum _{i=1}^{L_2}{\mathbf{h}}_{11}^{ij}\circ {\mathbf{h}}_{12}^{ij}\circ {\mathbf{h}}_2^i\right),\end{array}$$

(15)

where ∘ stands as the notation for the outer product. Furthermore, $\mathcal{H}$ is in fact a sum of P third-order tensors, each one of rank $L_2$ [19]. As indicated in [18], the recommended values for $L_2$ are small (e.g., 2 or 3).

Summarizing, the identification of the global impulse response $\mathbf{h}$ of length L (i.e., with $L_{11}{L}_{12}{L}_2$ coefficients) is transformed into a combination of three (shorter) sets of impulse responses, i.e., ${\mathbf{h}}_{11}^{ij}$, ${\mathbf{h}}_{12}^{ij}$, and ${\mathbf{h}}_2^i$ (for $i=1,2,\dots ,L_2$ and $j=1,2,\dots ,P$). As a result, the new parameter space of the filter involves only $P{L}_{11}{L}_2$, $P{L}_{12}{L}_2$, and $L_2^2$ coefficients, respectively. Since usually $P\ll L_{12}$ [18], the TOT-based decomposition approach leads to a significant dimensionality reduction, which is achieved especially when dealing with long-length filters (i.e., large values of L). While the conventional Wiener filter using matrix inversion involves a computational complexity proportional to $\mathcal{O}\left(L^3\right)=\mathcal{O}\left[\left(L_{11}^3{L}_{12}^3{L}_2^3\right)\right]$, the decomposition-based solution using the CG method combines the estimates of three shorter filters, which results in a computational complexity proportional to $\mathcal{O}\left[{\left(P{L}_{11}{L}_2\right)}^2+{\left(P{L}_{12}{L}_2\right)}^2+L_2^4\right]$, with $P\ll L_{12}$.

3. Iterative Wiener Filter Based on TOT and CG

The current section is dedicated to the development of the proposed solution, which results in the form of an iterative Wiener filter based on TOT decomposition and using the CG method for solving the associated sets of Wiener–Hopf equations. For this purpose and a better readability of the upcoming developments, several preliminary elements from [18] and the specific notation are presented at the beginning of this section. These preliminaries are related to the TOT-based decomposition framework and the associated Wiener–Hopf equations. Further, the proposed solution is developed. The differences between the version from [18] and the current proposal based on the CG method are mainly related to (i) the specific initialization that involves auxiliary matrices and (ii) the connection between the component filters from one CG cycle to another within the main iterations of the proposed CG-based Wiener filter. Moreover, since the impulse responses from (14) have different lengths, the CG cycles corresponding to the component filters use different numbers of updates.

As shown in [18], the estimates of the component impulse responses from (14) can be obtained based on a multilinear optimization approach [38,39]. In other words, two of the component impulse responses are considered fixed, while optimizing the third (remaining) one. This approach leads to three sets of Wiener–Hopf equations, i.e.,

$$\begin{array}{cc}\hfill {\left({\underline{\overline{\mathbf{G}}}}_{12,11}\right)}^T{\mathbf{R}}_{\mathbf{x}}{\underline{\overline{\mathbf{G}}}}_{12,11}{\underline{\mathbf{g}}}_{2,\mathrm{W}}& ={\left({\underline{\overline{\mathbf{G}}}}_{12,11}\right)}^T{\mathbf{r}}_{\mathbf{x}d},\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill {\left({\underline{\overline{\mathbf{G}}}}_{2,11}\right)}^T{\mathbf{R}}_{\mathbf{x}}{\underline{\overline{\mathbf{G}}}}_{2,11}{\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}}& ={\left({\underline{\overline{\mathbf{G}}}}_{2,11}\right)}^T{\mathbf{r}}_{\mathbf{x}d},\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill {\left({\underline{\overline{\mathbf{G}}}}_{2,12}\right)}^T{\mathbf{R}}_{\mathbf{x}}{\underline{\overline{\mathbf{G}}}}_{2,12}{\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}}& ={\left({\underline{\overline{\mathbf{G}}}}_{2,12}\right)}^T{\mathbf{r}}_{\mathbf{x}d}.\hfill \end{array}$$

(18)

The corresponding data structures and the associated notation are shown in

Table 1. Specific Data Structures and Notation Related to Wiener–Hopf equations with TOT Decomposition.

$$\begin{array}{c}\underline{\mathrm{Data}\mathrm{structures}\mathrm{and}\mathrm{notation}\mathrm{from} \left(16\right)}:\hfill \\ {\underline{\overline{\mathbf{G}}}}_{12,11}=\left[\begin{array}{cccc}{\overline{\mathbf{G}}}_{12,11}^1& {\overline{\mathbf{G}}}_{12,11}^2& \cdots & {\overline{\mathbf{G}}}_{12,11}^{L_2}\end{array}\right],\hfill \\ \mathrm{where}{\overline{\mathbf{G}}}_{12,11}^i={\sum }_{j=1}^P{\mathbf{G}}_{12,11}^{ij},i=1,2,\dots ,L_2,\hfill \\ \mathrm{with}{\mathbf{G}}_{12,11}^{ij}={\mathbf{I}}_{L_2}\otimes {\mathbf{g}}_{12,\mathrm{W}}^{ij}\otimes {\mathbf{g}}_{11}^{ij,\mathrm{W}},i=1,2,\dots ,L_2,j=1,2,\dots ,P\hfill \\ {\underline{\mathbf{g}}}_{2,\mathrm{W}}={\left[\begin{array}{cccc}{\left({\mathbf{g}}_{2,\mathrm{W}}^1\right)}^T& {\left({\mathbf{g}}_{2,\mathrm{W}}^2\right)}^T& \cdots & {\left({\mathbf{g}}_{2,\mathrm{W}}^{L_2}\right)}^T\end{array}\right]}^T\hfill \\ \underline{\mathrm{Data}\mathrm{structures}\mathrm{and}\mathrm{notation}\mathrm{from} \left(17\right)}:\hfill \\ {\underline{\overline{\mathbf{G}}}}_{2,11}=\left[\begin{array}{cccc}{\overline{\mathbf{G}}}_{2,11}^1& {\overline{\mathbf{G}}}_{2,11}^2& \cdots & {\overline{\mathbf{G}}}_{2,11}^{L_2}\end{array}\right],\hfill \\ \mathrm{where}{\overline{\mathbf{G}}}_{2,11}^i=\left[\begin{array}{cccc}{\mathbf{G}}_{2,11}^{i1}& {\mathbf{G}}_{2,11}^{i2}& \cdots & {\mathbf{G}}_{2,11}^{iP}\end{array}\right],i=1,2,\dots ,L_2\hfill \\ \mathrm{with}{\mathbf{G}}_{2,11}^{ij}={\mathbf{g}}_{2,\mathrm{W}}^i\otimes {\mathbf{I}}_{L_{12}}\otimes {\mathbf{g}}_{11,\mathrm{W}}^{ij},i=1,2,\dots ,L_2,j=1,2,\dots ,P\hfill \\ {\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}}={\left[\begin{array}{cccc}{\left({\overline{\mathbf{g}}}_{12,\mathrm{W}}^1\right)}^T& {\left({\overline{\mathbf{g}}}_{12,\mathrm{W}}^2\right)}^T& \cdots & {\left({\overline{\mathbf{g}}}_{12,\mathrm{W}}^{L_2}\right)}^T\end{array}\right]}^T\hfill \\ \mathrm{where}{\overline{\mathbf{g}}}_{12,\mathrm{W}}^i={\left[\begin{array}{cccc}{\left({\mathbf{g}}_{12,\mathrm{W}}^{i1}\right)}^T& {\left({\mathbf{g}}_{12,\mathrm{W}}^{i2}\right)}^T& \cdots & {\left({\mathbf{g}}_{12,\mathrm{W}}^{iP}\right)}^T\end{array}\right]}^T,i=1,2,\dots ,L_2\hfill \\ \underline{\mathrm{Data}\mathrm{structures}\mathrm{and}\mathrm{notation}\mathrm{from} \left(18\right)}:\hfill \\ {\underline{\overline{\mathbf{G}}}}_{2,12}=\left[\begin{array}{cccc}{\overline{\mathbf{G}}}_{2,12}^1& {\overline{\mathbf{G}}}_{2,12}^2& \cdots & {\overline{\mathbf{G}}}_{2,12}^{L_2}\end{array}\right],\hfill \\ \mathrm{where}{\overline{\mathbf{G}}}_{2,12}^i=\left[\begin{array}{cccc}{\mathbf{G}}_{2,12}^{i1}& {\mathbf{G}}_{2,12}^{i2}& \cdots & {\mathbf{G}}_{2,12}^{iP}\end{array}\right],i=1,2,\dots ,L_2,\hfill \\ \mathrm{with}{\mathbf{G}}_{2,12}^{ij}={\mathbf{g}}_{2,\mathrm{W}}^i\otimes {\mathbf{g}}_{12,\mathrm{W}}^{ij}\otimes {\mathbf{I}}_{L_{11}},i=1,2,\dots ,L_2,j=1,2,\dots ,P\hfill \\ {\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}}={\left[\begin{array}{cccc}{\left({\overline{\mathbf{g}}}_{11,\mathrm{W}}^1\right)}^T& {\left({\overline{\mathbf{g}}}_{11,\mathrm{W}}^2\right)}^T& \cdots & {\left({\overline{\mathbf{g}}}_{11,\mathrm{W}}^{L_2}\right)}^T\end{array}\right]}^T,\hfill \\ \mathrm{where}{\overline{\mathbf{g}}}_{11,\mathrm{W}}^i={\left[\begin{array}{cccc}{\left({\mathbf{g}}_{11,\mathrm{W}}^{i1}\right)}^T& {\left({\mathbf{g}}_{11,\mathrm{W}}^{i2}\right)}^T& \cdots & {\left({\mathbf{g}}_{11,\mathrm{W}}^{iP}\right)}^T\end{array}\right]}^T,i=1,2,\dots ,L_2\hfill \end{array}$$

, where ${\mathbf{g}}_{★,\mathrm{W}}$ generally denotes the estimate of ${\mathbf{h}}_{★}$ from (14), while ${\mathbf{I}}_{L_{•}}$ is the identity matrix of size $L_{•}\times L_{•}$.

At this point, (16)–(18) are going to be solved with the CG method. The resulting solutions will then be sequentially iterated and combined (via the Kronecker product). Finally, the Wiener filter ${\mathbf{g}}_{\mathrm{W}}$, which represents an estimate of $\mathbf{h}$, will be obtained as

$$\begin{array}{cc}\hfill {\mathbf{g}}_{\mathrm{W}}& =\sum _{i=1}^{L_2}\sum _{j=1}^P{\mathbf{g}}_{2,\mathrm{W}}^i\otimes {\mathbf{g}}_{12,\mathrm{W}}^{ij}\otimes {\mathbf{g}}_{11,\mathrm{W}}^{ij},\hfill \end{array}$$

(19)

where ${\mathbf{g}}_{2,\mathrm{W}}^i$, ${\mathbf{g}}_{12,\mathrm{W}}^{ij}$, and ${\mathbf{g}}_{11,\mathrm{W}}^{ij}$ are obtained from ${\underline{\mathbf{g}}}_{2,\mathrm{W}}$,${\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}}$, and ${\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}}$, respectively. All these steps of the designed algorithm are detailed in the following.

As mentioned before, the developed iterative Wiener filter is based on the TOT decomposition of the global impulse response, while the CG updates are used to efficiently solve (16), (17), and (18), respectively. To this purpose, the main iterations of the Wiener filter are denoted as superscripts ${ }^{\left(n\right)}$, while the CG updates appear in subscripts ${ }_{\left(k\right)}$. The initialization of the algorithm concerns the three component filters, which are initially defined as

$$\begin{array}{cc}\hfill {\underline{\mathbf{g}}}_{2,\mathrm{W}\left(K_2\right)}^{\left(0\right)}& ={\left[\begin{array}{cc}ϵ& {\mathbf{0}}_{L_2^2-1}^T\end{array}\right]}^T,\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill {\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}\left(K_1\right)}^{\left(0\right)}& ={\left[\begin{array}{cc}ϵ& {\mathbf{0}}_{P{L}_{12}{L}_2-1}^T\end{array}\right]}^T,\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill {\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(K_1\right)}^{\left(0\right)}& ={\left[\begin{array}{cc}ϵ& {\mathbf{0}}_{P{L}_{11}{L}_2-1}^T\end{array}\right]}^T,\hfill \end{array}$$

(22)

where $K_1$ and $K_2$ represent the maximum number of CG updates (for the component filters), $ϵ$ is a very small positive number, while ${\mathbf{0}}_{•}$ denotes an all-zeros vector with the length indicated in subscript. The reason for using $K_1\ne K_2$ is that we are dealing with different lengths for the component filters. Among them, the length of ${\underline{\mathbf{g}}}_{2,\mathrm{W}}$ (which has $L_2^2$ coefficients) could be significantly smaller, taking into account that ${\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}}$ and ${\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}}$ have $P{L}_{12}{L}_2$ and $P{L}_{11}{L}_2$ coefficients, respectively, while $L_2\ll L_{11}{L}_{12}$.

At this point, we need to introduce the auxiliary matrices (for $i=1,2,\dots ,L_2$ and $j=1,2,\dots ,P$):

$$\begin{array}{cc}\hfill {\mathbf{M}}_{12,11}^{ij\left(0\right)}& ={\mathbf{I}}_{L_2}\otimes {\left[\begin{array}{cc}ϵ& {\mathbf{0}}_{L_{12}-1}^T\end{array}\right]}^T\otimes {\left[\begin{array}{cc}ϵ& {\mathbf{0}}_{L_{11}-1}^T\end{array}\right]}^T,\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill {\mathbf{M}}_{11}^{ij\left(0\right)}& ={\mathbf{I}}_{L_{12}}\otimes {\left[\begin{array}{cc}ϵ& {\mathbf{0}}_{L_{11}-1}^T\end{array}\right]}^T,\hfill \end{array}$$

(24)

which will further facilitate the definition of matrices ${\underline{\overline{\mathbf{G}}}}_{12,11}$ and ${\underline{\overline{\mathbf{G}}}}_{2,11}$. Hence, in each main iteration ${ }^{\left(n\right)}$ of the algorithm, we first construct using (23):

$$\begin{array}{cc}\hfill {\mathbf{G}}_{12,11}^{ij\left(n\right)}& ={\mathbf{M}}_{12,11}^{ij(n-1)},i=1,2,\dots ,L_2,j=1,2,\dots ,P,\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill {\overline{\mathbf{G}}}_{12,11}^{i\left(n\right)}& =\sum _{j=1}^P{\mathbf{G}}_{12,11}^{ij\left(n\right)},i=1,2,\dots ,L_2,\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill {\underline{\overline{\mathbf{G}}}}_{12,11}^{\left(n\right)}& =\left[\begin{array}{cccc}{\overline{\mathbf{G}}}_{12,11}^{1\left(n\right)}& {\overline{\mathbf{G}}}_{12,11}^{2\left(n\right)}& \cdots & {\overline{\mathbf{G}}}_{12,11}^{L_2\left(n\right)}\end{array}\right].\hfill \end{array}$$

(27)

These allow us to compute

$$\begin{array}{cc}\hfill {\mathbf{R}}_2^{\left(n\right)}& ={\left({\underline{\overline{\mathbf{G}}}}_{12,11}^{\left(n\right)}\right)}^T{\mathbf{R}}_{\mathbf{x}}{\underline{\overline{\mathbf{G}}}}_{12,11}^{\left(n\right)},\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill {\mathbf{r}}_2^{\left(n\right)}& ={\left({\underline{\overline{\mathbf{G}}}}_{12,11}^{\left(n\right)}\right)}^T{\mathbf{r}}_{\mathbf{x}d}.\hfill \end{array}$$

(29)

The structures from (28) and (29) are used to process (16) with the CG method. Consequently, using (20), the initial settings are

$$\begin{array}{cc}\hfill {\underline{\mathbf{g}}}_{2,\mathrm{W}\left(0\right)}^{\left(n\right)}& ={\underline{\mathbf{g}}}_{2,\mathrm{W}\left(K_2\right)}^{(n-1)},\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill {\mathbf{z}}_{2\left(0\right)}^{\left(n\right)}& ={\mathbf{r}}_2^{\left(n\right)}-{\mathbf{R}}_2^{\left(n\right)}{\underline{\mathbf{g}}}_{2,\mathrm{W}\left(0\right)}^{\left(n\right)},\hfill \end{array}$$

(31)

$$\begin{array}{cc}\hfill {\mathbf{c}}_{2\left(0\right)}^{\left(n\right)}& ={\mathbf{z}}_{2\left(0\right)}^{\left(n\right)},\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill {\gamma }_{2\left(0\right)}^{\left(n\right)}& ={\left({\mathbf{z}}_{2\left(0\right)}^{\left(n\right)}\right)}^T{\mathbf{z}}_{2\left(0\right)}^{\left(n\right)}.\hfill \end{array}$$

(33)

Next, for $k_2=1,2,\dots ,K_2$, we perform similar to (6)–(12):

$$\begin{array}{cc}\hfill {\mathbf{q}}_{2\left(k_2\right)}^{\left(n\right)}& ={\mathbf{R}}_2^{\left(n\right)}{\mathbf{c}}_{2(k_2-1)}^{\left(n\right)},\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill {\alpha }_{2\left(k_2\right)}^{\left(n\right)}& =\frac{{\gamma }_{2(k_2-1)}^{\left(n\right)}}{{\left({\mathbf{c}}_{2(k_2-1)}^{\left(n\right)}\right)}^T{\mathbf{q}}_{2\left(k_2\right)}^{\left(n\right)}},\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill {\underline{\mathbf{g}}}_{2,\mathrm{W}\left(k_2\right)}^{\left(n\right)}& ={\underline{\mathbf{g}}}_{2,\mathrm{W}(k_2-1)}^{\left(n\right)}+{\alpha }_{2\left(k_2\right)}^{\left(n\right)}{\mathbf{c}}_{2(k_2-1)}^{\left(n\right)},\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill {\mathbf{z}}_{2\left(k_2\right)}^{\left(n\right)}& ={\mathbf{z}}_{2(k_2-1)}^{\left(n\right)}-{\alpha }_{2\left(k_2\right)}^{\left(n\right)}{\mathbf{q}}_{2\left(k_2\right)}^{\left(n\right)},\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill {\gamma }_{2\left(k_2\right)}^{\left(n\right)}& ={\left({\mathbf{z}}_{2\left(k_2\right)}^{\left(n\right)}\right)}^T{\mathbf{z}}_{2\left(k_2\right)}^{\left(n\right)},\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill {\beta }_{2\left(k_2\right)}^{\left(n\right)}& =\frac{{\gamma }_{2\left(k_2\right)}^{\left(n\right)}}{{\gamma }_{2(k_2-1)}^{\left(n\right)}},\hfill \end{array}$$

(39)

$$\begin{array}{cc}\hfill {\mathbf{c}}_{2\left(k_2\right)}^{\left(n\right)}& ={\mathbf{z}}_{2\left(k_2\right)}^{\left(n\right)}+{\beta }_{2\left(k_2\right)}^{\left(n\right)}{\mathbf{c}}_{2(k_2-1)}^{\left(n\right)}.\hfill \end{array}$$

(40)

The final solution ${\underline{\mathbf{g}}}_{2,\mathrm{W}\left(K_2\right)}^{\left(n\right)}$ will represent the initialization for the CG cycle associated with this filter [similar to (30)] in the next main iteration of the algorithm. Also, it is decomposed as

$$\begin{array}{cc}\hfill {\underline{\mathbf{g}}}_{2,\mathrm{W}\left(K_2\right)}^{\left(n\right)}& ={\left[\begin{array}{cccc}{\left({\mathbf{g}}_{2,\mathrm{W}\left(K_2\right)}^{1\left(n\right)}\right)}^T& {\left({\mathbf{g}}_{2,\mathrm{W}\left(K_2\right)}^{2\left(n\right)}\right)}^T& \cdots & {\left({\mathbf{g}}_{2,\mathrm{W}\left(K_2\right)}^{L_2\left(n\right)}\right)}^T\end{array}\right]}^T,\hfill \end{array}$$

(41)

which further allows the evaluation of

$$\begin{array}{cc}\hfill {\mathbf{G}}_{2,11}^{ij\left(n\right)}& ={\mathbf{g}}_{2,\mathrm{W}\left(K_2\right)}^{i\left(n\right)}\otimes {\mathbf{M}}_{11}^{ij(n-1)},i=1,2,\dots ,L_2,j=1,2,\dots ,P,\hfill \end{array}$$

(42)

$$\begin{array}{cc}\hfill {\overline{\mathbf{G}}}_{2,11}^{i\left(n\right)}& =\left[\begin{array}{cccc}{\mathbf{G}}_{2,11}^{i1\left(n\right)}& {\mathbf{G}}_{2,11}^{i2\left(n\right)}& \cdots & {\mathbf{G}}_{2,11}^{iP\left(n\right)}\end{array}\right],i=1,2,\dots ,L_2,\hfill \end{array}$$

(43)

$$\begin{array}{cc}\hfill {\underline{\overline{\mathbf{G}}}}_{2,11}^{\left(n\right)}& =\left[\begin{array}{cccc}{\overline{\mathbf{G}}}_{2,11}^{1\left(n\right)}& {\overline{\mathbf{G}}}_{2,11}^{2\left(n\right)}& \cdots & {\overline{\mathbf{G}}}_{2,11}^{L_2\left(n\right)}\end{array}\right],\hfill \end{array}$$

(44)

so that we can compute

$$\begin{array}{cc}\hfill {\mathbf{R}}_{12}^{\left(n\right)}& ={\left({\underline{\overline{\mathbf{G}}}}_{2,11}^{\left(n\right)}\right)}^T{\mathbf{R}}_{\mathbf{x}}{\underline{\overline{\mathbf{G}}}}_{2,11}^{\left(n\right)},\hfill \end{array}$$

(45)

$$\begin{array}{cc}\hfill {\mathbf{r}}_{12}^{\left(n\right)}& ={\left({\underline{\overline{\mathbf{G}}}}_{2,11}^{\left(n\right)}\right)}^T{\mathbf{r}}_{\mathbf{x}d}.\hfill \end{array}$$

(46)

The notation from (45) and (46) is used to process (17) with the CG updates. Hence, in this step, we follow the initial settings from (21), so that

$$\begin{array}{cc}\hfill {\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}\left(0\right)}^{\left(n\right)}& ={\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}\left(K_1\right)}^{(n-1)},\hfill \end{array}$$

(47)

$$\begin{array}{cc}\hfill {\mathbf{z}}_{12\left(0\right)}^{\left(n\right)}& ={\mathbf{r}}_{12}^{\left(n\right)}-{\mathbf{R}}_{12}^{\left(n\right)}{\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}\left(0\right)}^{\left(n\right)},\hfill \end{array}$$

(48)

$$\begin{array}{cc}\hfill {\mathbf{c}}_{12\left(0\right)}^{\left(n\right)}& ={\mathbf{z}}_{12\left(0\right)}^{\left(n\right)},\hfill \end{array}$$

(49)

$$\begin{array}{cc}\hfill {\gamma }_{12\left(0\right)}^{\left(n\right)}& ={\left({\mathbf{z}}_{12\left(0\right)}^{\left(n\right)}\right)}^T{\mathbf{z}}_{12\left(0\right)}^{\left(n\right)}.\hfill \end{array}$$

(50)

Consequently, the CG cycle for the second filter is defined by the relations:

$$\begin{array}{cc}\hfill {\mathbf{q}}_{12\left(k_1\right)}^{\left(n\right)}& ={\mathbf{R}}_{12}^{\left(n\right)}{\mathbf{c}}_{12(k_1-1)}^{\left(n\right)},\hfill \end{array}$$

(51)

$$\begin{array}{cc}\hfill {\alpha }_{12\left(k_1\right)}^{\left(n\right)}& =\frac{{\gamma }_{12(k_1-1)}^{\left(n\right)}}{{\left({\mathbf{c}}_{12(k_1-1)}^{\left(n\right)}\right)}^T{\mathbf{q}}_{12\left(k_1\right)}^{\left(n\right)}},\hfill \end{array}$$

(52)

$$\begin{array}{cc}\hfill {\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}\left(k_1\right)}^{\left(n\right)}& ={\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}(k_1-1)}^{\left(n\right)}+{\alpha }_{12\left(k_1\right)}^{\left(n\right)}{\mathbf{c}}_{12(k_1-1)}^{\left(n\right)},\hfill \end{array}$$

(53)

$$\begin{array}{cc}\hfill {\mathbf{z}}_{12\left(k_1\right)}^{\left(n\right)}& ={\mathbf{z}}_{12(k_1-1)}^{\left(n\right)}-{\alpha }_{12\left(k_1\right)}^{\left(n\right)}{\mathbf{q}}_{12\left(k_1\right)}^{\left(n\right)},\hfill \end{array}$$

(54)

$$\begin{array}{cc}\hfill {\gamma }_{12\left(k_1\right)}^{\left(n\right)}& ={\left({\mathbf{z}}_{12\left(k_1\right)}^{\left(n\right)}\right)}^T{\mathbf{z}}_{12\left(k_1\right)}^{\left(n\right)},\hfill \end{array}$$

(55)

$$\begin{array}{cc}\hfill {\beta }_{12\left(k_1\right)}^{\left(n\right)}& =\frac{{\gamma }_{12\left(k_1\right)}^{\left(n\right)}}{{\gamma }_{12(k_1-1)}^{\left(n\right)}},\hfill \end{array}$$

(56)

$$\begin{array}{cc}\hfill {\mathbf{c}}_{12\left(k_1\right)}^{\left(n\right)}& ={\mathbf{z}}_{12\left(k_1\right)}^{\left(n\right)}+{\beta }_{12\left(k_1\right)}^{\left(n\right)}{\mathbf{c}}_{12(k_1-1)}^{\left(n\right)},\hfill \end{array}$$

(57)

for $k_1=1,2,\dots ,K_1$. The final solution ${\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}\left(K_1\right)}^{\left(n\right)}$ will represent the initial setting in the next main iteration of the algorithm [similar to (47)]. The decomposition of this impulse response is performed in two steps, i.e.,

$$\begin{array}{cc}\hfill {\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}\left(K_1\right)}^{\left(n\right)}& ={\left[\begin{array}{cccc}{\left({\overline{\mathbf{g}}}_{12,\mathrm{W}\left(K_1\right)}^{1\left(n\right)}\right)}^T& {\left({\overline{\mathbf{g}}}_{12,\mathrm{W}\left(K_1\right)}^{2\left(n\right)}\right)}^T& \cdots & {\left({\overline{\mathbf{g}}}_{12,\mathrm{W}\left(K_1\right)}^{L_2\left(n\right)}\right)}^T\end{array}\right]}^T,\hfill \end{array}$$

(58)

$$\begin{array}{cc}\hfill {\overline{\mathbf{g}}}_{12,\mathrm{W}\left(K_1\right)}^{i\left(n\right)}& ={\left[\begin{array}{cccc}{\left({\mathbf{g}}_{12,\mathrm{W}\left(K_1\right)}^{i1\left(n\right)}\right)}^T& {\left({\mathbf{g}}_{12,\mathrm{W}\left(K_1\right)}^{i2\left(n\right)}\right)}^T& \cdots & {\left({\mathbf{g}}_{12,\mathrm{W}\left(K_1\right)}^{iP\left(n\right)}\right)}^T\end{array}\right]}^T,i=1,2,\dots ,L_2.\hfill \end{array}$$

(59)

At this point, having the components from (41) and (59), we continue with the development associated with the last component filter, starting with the evaluation of

$$\begin{array}{cc}\hfill {\mathbf{G}}_{2,12}^{ij\left(n\right)}& ={\mathbf{g}}_{2,\mathrm{W}\left(K_2\right)}^{i\left(n\right)}\otimes {\mathbf{g}}_{12,\mathrm{W}\left(K_1\right)}^{ij\left(n\right)}\otimes {\mathbf{I}}_{L_{11}},i=1,2,\dots ,L_2,j=1,2,\dots ,P,\hfill \end{array}$$

(60)

$$\begin{array}{cc}\hfill {\overline{\mathbf{G}}}_{2,12}^{i\left(n\right)}& =\left[\begin{array}{cccc}{\mathbf{G}}_{2,12}^{i1\left(n\right)}& {\mathbf{G}}_{2,12}^{i2\left(n\right)}& \cdots & {\mathbf{G}}_{2,12}^{iP\left(n\right)}\end{array}\right],i=1,2,\dots ,L_2,\hfill \end{array}$$

(61)

$$\begin{array}{cc}\hfill {\underline{\overline{\mathbf{G}}}}_{2,12}^{\left(n\right)}& =\left[\begin{array}{cccc}{\overline{\mathbf{G}}}_{2,12}^{1\left(n\right)}& {\overline{\mathbf{G}}}_{2,12}^{2\left(n\right)}& \cdots & {\overline{\mathbf{G}}}_{2,12}^{L_2\left(n\right)}\end{array}\right].\hfill \end{array}$$

(62)

Therefore, introducing the notation:

$$\begin{array}{cc}\hfill {\mathbf{R}}_{11}^{\left(n\right)}& ={\left({\underline{\overline{\mathbf{G}}}}_{2,12}^{\left(n\right)}\right)}^T{\mathbf{R}}_{\mathbf{x}}{\underline{\overline{\mathbf{G}}}}_{2,12}^{\left(n\right)},\hfill \end{array}$$

(63)

$$\begin{array}{cc}\hfill {\mathbf{r}}_{11}^{\left(n\right)}& ={\left({\underline{\overline{\mathbf{G}}}}_{2,12}^{\left(n\right)}\right)}^T{\mathbf{r}}_{\mathbf{x}d},\hfill \end{array}$$

(64)

we can further process (18) with the CG method. To this purpose, the initialization relies on (22), so that the settings are

$$\begin{array}{cc}\hfill {\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(0\right)}^{\left(n\right)}& ={\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(K_1\right)}^{(n-1)},\hfill \end{array}$$

(65)

$$\begin{array}{cc}\hfill {\mathbf{z}}_{11\left(0\right)}^{\left(n\right)}& ={\mathbf{r}}_{11}^{\left(n\right)}-{\mathbf{R}}_{11}^{\left(n\right)}{\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(0\right)}^{\left(n\right)},\hfill \end{array}$$

(66)

$$\begin{array}{cc}\hfill {\mathbf{c}}_{11\left(0\right)}^{\left(n\right)}& ={\mathbf{z}}_{11\left(0\right)}^{\left(n\right)},\hfill \end{array}$$

(67)

$$\begin{array}{cc}\hfill {\gamma }_{11\left(0\right)}^{\left(n\right)}& ={\left({\mathbf{z}}_{11\left(0\right)}^{\left(n\right)}\right)}^T{\mathbf{z}}_{11\left(0\right)}^{\left(n\right)}.\hfill \end{array}$$

(68)

Thus, for $k_1=1,2,\dots ,K_1$, the CG cycle for the third filter consists of the relations:

$$\begin{array}{cc}\hfill {\mathbf{q}}_{11\left(k_1\right)}^{\left(n\right)}& ={\mathbf{R}}_{11}^{\left(n\right)}{\mathbf{c}}_{11(k_1-1)}^{\left(n\right)},\hfill \end{array}$$

(69)

$$\begin{array}{cc}\hfill {\alpha }_{11\left(k_1\right)}^{\left(n\right)}& =\frac{{\gamma }_{11(k_1-1)}^{\left(n\right)}}{{\left({\mathbf{c}}_{11(k_1-1)}^{\left(n\right)}\right)}^T{\mathbf{q}}_{11\left(k_1\right)}^{\left(n\right)}},\hfill \end{array}$$

(70)

$$\begin{array}{cc}\hfill {\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(k_1\right)}^{\left(n\right)}& ={\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}(k_1-1)}^{\left(n\right)}+{\alpha }_{11\left(k_1\right)}^{\left(n\right)}{\mathbf{c}}_{11(k_1-1)}^{\left(n\right)},\hfill \end{array}$$

(71)

$$\begin{array}{cc}\hfill {\mathbf{z}}_{11\left(k_1\right)}^{\left(n\right)}& ={\mathbf{z}}_{11(k_1-1)}^{\left(n\right)}-{\alpha }_{11\left(k_1\right)}^{\left(n\right)}{\mathbf{q}}_{11\left(k_1\right)}^{\left(n\right)},\hfill \end{array}$$

(72)

$$\begin{array}{cc}\hfill {\gamma }_{11\left(k_1\right)}^{\left(n\right)}& ={\left({\mathbf{z}}_{11\left(k_1\right)}^{\left(n\right)}\right)}^T{\mathbf{z}}_{11\left(k_1\right)}^{\left(n\right)},\hfill \end{array}$$

(73)

$$\begin{array}{cc}\hfill {\beta }_{11\left(k_1\right)}^{\left(n\right)}& =\frac{{\gamma }_{11\left(k_1\right)}^{\left(n\right)}}{{\gamma }_{11(k_1-1)}^{\left(n\right)}},\hfill \end{array}$$

(74)

$$\begin{array}{cc}\hfill {\mathbf{c}}_{11\left(k_1\right)}^{\left(n\right)}& ={\mathbf{z}}_{11\left(k_1\right)}^{\left(n\right)}+{\beta }_{11\left(k_1\right)}^{\left(n\right)}{\mathbf{c}}_{11(k_1-1)}^{\left(n\right)}.\hfill \end{array}$$

(75)

The decomposition of the final solution ${\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(K_1\right)}^{\left(n\right)}$ results in

$$\begin{array}{cc}\hfill {\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(K_1\right)}^{\left(n\right)}& ={\left[\begin{array}{cccc}{\left({\overline{\mathbf{g}}}_{11,\mathrm{W}\left(K_1\right)}^{1\left(n\right)}\right)}^T& {\left({\overline{\mathbf{g}}}_{11,\mathrm{W}\left(K_1\right)}^{2\left(n\right)}\right)}^T& \cdots & {\left({\overline{\mathbf{g}}}_{11,\mathrm{W}\left(K_1\right)}^{L_2\left(n\right)}\right)}^T\end{array}\right]}^T,\hfill \end{array}$$

(76)

$$\begin{array}{cc}\hfill {\overline{\mathbf{g}}}_{11,\mathrm{W}\left(K_1\right)}^{i\left(n\right)}& ={\left[\begin{array}{cccc}{\left({\mathbf{g}}_{11,\mathrm{W}\left(K_1\right)}^{i1\left(n\right)}\right)}^T& {\left({\mathbf{g}}_{11,\mathrm{W}\left(K_1\right)}^{i2\left(n\right)}\right)}^T& \cdots & {\left({\mathbf{g}}_{11,\mathrm{W}\left(K_1\right)}^{iP\left(n\right)}\right)}^T\end{array}\right]}^T,i=1,2,\dots ,L_2,\hfill \end{array}$$

(77)

and provides the final elements for evaluating the estimated impulse response based on (19). Also, ${\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(K_1\right)}^{\left(n\right)}$ represents the initialization for the next main iteration of the algorithm, according to (65).

Summarizing, using (41), (59), and (77), we obtain

$$\begin{array}{c}\hfill {\mathbf{g}}_{\mathrm{W}}^{\left(n\right)}=\sum _{i=1}^{L_2}\sum _{j=1}^P{\mathbf{g}}_{2,\mathrm{W}\left(K_2\right)}^{i\left(n\right)}\otimes {\mathbf{g}}_{12,\mathrm{W}\left(K_1\right)}^{ij\left(n\right)}\otimes {\mathbf{g}}_{11,\mathrm{W}\left(K_1\right)}^{ij\left(n\right)}.\end{array}$$

(78)

Finally, using the same components, we evaluate the auxiliary matrices (for $i=1,2,\dots ,L_2$ and $j=1,2,\dots ,P$):

$$\begin{array}{cc}\hfill {\mathbf{M}}_{12,11}^{ij\left(n\right)}& ={\mathbf{I}}_{L_2}\otimes {\mathbf{g}}_{12,\mathrm{W}\left(K_1\right)}^{ij\left(n\right)}\otimes {\mathbf{g}}_{11,\mathrm{W}\left(K_1\right)}^{ij\left(n\right)},\hfill \end{array}$$

(79)

$$\begin{array}{cc}\hfill {\mathbf{M}}_{11}^{ij\left(n\right)}& ={\mathbf{I}}_{L_{12}}\otimes {\mathbf{g}}_{11,\mathrm{W}\left(K_1\right)}^{ij\left(n\right)}.\hfill \end{array}$$

(80)

These will be used in the next main iteration of the algorithm, in order to compute the structures from (25) and (42), respectively.

The resulting iterative Wiener filter (IWF) is based on TOT decomposition and uses the CG method, which will be referred to as IWF-TOT-CG. Its main steps are summarized in

Table 2. Iterative Wiener Filter Based on a Third-Order Tensor Decomposition and Using the CG Method (IWF-TOT-CG).

$$\begin{array}{c}\underline{\mathrm{Data}:}{\mathbf{R}}_{\mathbf{x}},{\mathbf{r}}_{\mathbf{x}d}\left(\mathrm{estimated}\mathrm{statistics}\mathrm{based}\mathrm{on}N\mathrm{data}\mathrm{samples}\right)\hfill \\ L=L_{11}{L}_{12}{L}_2,L_{12}\le L_{11},L_2\ll L_{11}{L}_{12},P<L_{12}\hfill \\ \underline{\mathrm{Initialization}:}\hfill \\ {\underline{\mathbf{g}}}_{2,\mathrm{W}\left(K_2\right)}^{\left(0\right)}={\left[\begin{array}{cc}ϵ& {\mathbf{0}}_{L_2^2-1}^T\end{array}\right]}^T,0<ϵ\le 1\hfill \\ {\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}\left(K_1\right)}^{\left(0\right)}={\left[\begin{array}{cc}ϵ& {\mathbf{0}}_{P{L}_{12}{L}_2-1}^T\end{array}\right]}^T\hfill \\ {\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(K_1\right)}^{\left(0\right)}={\left[\begin{array}{cc}ϵ& {\mathbf{0}}_{P{L}_{11}{L}_2-1}^T\end{array}\right]}^T\hfill \\ \mathrm{for}i=1,2,\dots ,L_2,j=1,2,\dots ,P:\hfill \\ {\mathbf{M}}_{12,11}^{ij\left(0\right)}={\mathbf{I}}_{L_2}\otimes {\left[\begin{array}{cc}ϵ& {\mathbf{0}}_{L_{12}-1}^T\end{array}\right]}^T\otimes {\left[\begin{array}{cc}ϵ& {\mathbf{0}}_{L_{11}-1}^T\end{array}\right]}^T\hfill \\ {\mathbf{M}}_{11}^{ij\left(0\right)}={\mathbf{I}}_{L_{12}}\otimes {\left[\begin{array}{cc}ϵ& {\mathbf{0}}_{L_{11}-1}^T\end{array}\right]}^T\hfill \\ \underline{\mathrm{For}}n=1,2,\dots :\hfill \\ {\mathbf{G}}_{12,11}^{ij\left(n\right)}={\mathbf{M}}_{12,11}^{ij(n-1)},i=1,2,\dots ,L_2,j=1,2,\dots ,P\hfill \\ {\overline{\mathbf{G}}}_{12,11}^{i\left(n\right)}={\sum }_{j=1}^P{\mathbf{G}}_{12,11}^{ij\left(n\right)},i=1,2,\dots ,L_2\hfill \\ {\underline{\overline{\mathbf{G}}}}_{12,11}^{\left(n\right)}=\left[\begin{array}{cccc}{\overline{\mathbf{G}}}_{12,11}^{1\left(n\right)}& {\overline{\mathbf{G}}}_{12,11}^{2\left(n\right)}& \cdots & {\overline{\mathbf{G}}}_{12,11}^{L_2\left(n\right)}\end{array}\right]\hfill \\ {\underline{\mathbf{g}}}_{2,\mathrm{W}\left(0\right)}^{\left(n\right)}={\underline{\mathbf{g}}}_{2,\mathrm{W}\left(K_2\right)}^{(n-1)}\hfill \\ \mathrm{Solve} \left(16\right)\underset{(\mathrm{see}Table\; 3)}{\overset{\mathrm{CG}\left(K_2\right)}{\to }}{\underline{\mathbf{g}}}_{2,\mathrm{W}\left(K_2\right)}^{\left(n\right)}\stackrel{\left(41\right)}{\to }{\mathbf{g}}_{2,\mathrm{W}\left(K_2\right)}^{i\left(n\right)},i=1,2,\dots ,L_2\hfill \\ {\mathbf{G}}_{2,11}^{ij\left(n\right)}={\mathbf{g}}_{2,\mathrm{W}\left(K_2\right)}^{i\left(n\right)}\otimes {\mathbf{M}}_{11}^{ij(n-1)},i=1,2,\dots ,L_2,j=1,2,\dots ,P\hfill \\ {\overline{\mathbf{G}}}_{2,11}^{i\left(n\right)}=\left[\begin{array}{cccc}{\mathbf{G}}_{2,11}^{i1\left(n\right)}& {\mathbf{G}}_{2,11}^{i2\left(n\right)}& \cdots & {\mathbf{G}}_{2,11}^{iP\left(n\right)}\end{array}\right],i=1,2,\dots ,L_2\hfill \\ {\underline{\overline{\mathbf{G}}}}_{2,11}^{\left(n\right)}=\left[\begin{array}{cccc}{\overline{\mathbf{G}}}_{2,11}^{1\left(n\right)}& {\overline{\mathbf{G}}}_{2,11}^{2\left(n\right)}& \cdots & {\overline{\mathbf{G}}}_{2,11}^{L_2\left(n\right)}\end{array}\right]\hfill \\ {\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}\left(0\right)}^{\left(n\right)}={\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}\left(K_1\right)}^{(n-1)}\hfill \\ \mathrm{Solve} \left(17\right)\underset{(\mathrm{see}Table\; 3)}{\overset{\mathrm{CG}\left(K_1\right)}{\to }}{\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}\left(K_1\right)}^{\left(n\right)}\stackrel{\left(58\right),\left(59\right)}{\to }{\mathbf{g}}_{12,\mathrm{W}\left(K_1\right)}^{ij\left(n\right)},i=1,2,\dots ,L_2,j=1,2,\dots ,P\hfill \\ {\mathbf{G}}_{2,12}^{ij\left(n\right)}={\mathbf{g}}_{2,\mathrm{W}\left(K_2\right)}^{i\left(n\right)}\otimes {\mathbf{g}}_{12,\mathrm{W}\left(K_1\right)}^{ij\left(n\right)}\otimes {\mathbf{I}}_{L_{11}},i=1,2,\dots ,L_2,j=1,2,\dots ,P\hfill \\ {\overline{\mathbf{G}}}_{2,12}^{i\left(n\right)}=\left[\begin{array}{cccc}{\mathbf{G}}_{2,12}^{i1\left(n\right)}& {\mathbf{G}}_{2,12}^{i2\left(n\right)}& \cdots & {\mathbf{G}}_{2,12}^{iP\left(n\right)}\end{array}\right],i=1,2,\dots ,L_2\hfill \\ {\underline{\overline{\mathbf{G}}}}_{2,12}^{\left(n\right)}=\left[\begin{array}{cccc}{\overline{\mathbf{G}}}_{2,12}^{1\left(n\right)}& {\overline{\mathbf{G}}}_{2,12}^{2\left(1\right)}& \cdots & {\overline{\mathbf{G}}}_{2,12}^{L_2\left(n\right)}\end{array}\right]\hfill \\ {\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(0\right)}^{\left(n\right)}={\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(K_1\right)}^{(n-1)}\hfill \\ \mathrm{Solve} \left(18\right)\underset{(\mathrm{see}Table\; 3)}{\overset{\mathrm{CG}\left(K_1\right)}{\to }}{\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(K_1\right)}^{\left(n\right)}\stackrel{\left(76\right),\left(77\right)}{\to }{\mathbf{g}}_{11,\mathrm{W}\left(K_1\right)}^{ij\left(n\right)},i=1,2,\dots ,L_2,j=1,2,\dots ,P\hfill \\ {\mathbf{M}}_{12,11}^{ij\left(n\right)}={\mathbf{I}}_{L_2}\otimes {\mathbf{g}}_{12,\mathrm{W}\left(K_1\right)}^{ij\left(n\right)}\otimes {\mathbf{g}}_{11,\mathrm{W}\left(K_1\right)}^{ij\left(n\right)},i=1,2,\dots ,L_2,j=1,2,\dots ,P\hfill \\ {\mathbf{M}}_{11}^{ij\left(n\right)}={\mathbf{I}}_{L_{12}}\otimes {\mathbf{g}}_{11,\mathrm{W}\left(K_1\right)}^{ij\left(n\right)},i=1,2,\dots ,L_2,j=1,2,\dots ,P\hfill \\ {\mathbf{g}}_{\mathrm{W}}^{\left(n\right)}={\sum }_{i=1}^{L_2}{\sum }_{j=1}^P{\mathbf{g}}_{2,\mathrm{W}\left(K_2\right)}^{i\left(n\right)}\otimes {\mathbf{g}}_{12,\mathrm{W}\left(K_1\right)}^{ij\left(n\right)}\otimes {\mathbf{g}}_{11,\mathrm{W}\left(K_1\right)}^{ij\left(n\right)}\hfill \end{array}$$

, while the CG cycles for solving (16)–(18) are detailed in

Table 3. CG Solutions of the Wiener–Hopf Equations within IWF-TOT-CG.

$$\begin{array}{c}\underline{\mathrm{Solution}\mathrm{of} \left(16\right)}:\hfill \\ {\mathbf{R}}_2^{\left(n\right)}={\left({\underline{\overline{\mathbf{G}}}}_{12,11}^{\left(n\right)}\right)}^T{\mathbf{R}}_{\mathbf{x}}{\underline{\overline{\mathbf{G}}}}_{12,11}^{\left(n\right)},{\mathbf{r}}_2^{\left(n\right)}={\left({\underline{\overline{\mathbf{G}}}}_{12,11}^{\left(n\right)}\right)}^T{\mathbf{r}}_{\mathbf{x}d}\hfill \\ {\underline{\mathbf{g}}}_{2,\mathrm{W}\left(0\right)}^{\left(n\right)}={\underline{\mathbf{g}}}_{2,\mathrm{W}\left(K_2\right)}^{(n-1)},{\mathbf{z}}_{2\left(0\right)}^{\left(n\right)}={\mathbf{r}}_2^{\left(n\right)}-{\mathbf{R}}_2^{\left(n\right)}{\underline{\mathbf{g}}}_{2,\mathrm{W}\left(0\right)}^{\left(n\right)},{\mathbf{c}}_{2\left(0\right)}^{\left(n\right)}={\mathbf{z}}_{2\left(0\right)}^{\left(n\right)},{\gamma }_{2\left(0\right)}^{\left(n\right)}={\left({\mathbf{z}}_{2\left(0\right)}^{\left(n\right)}\right)}^T{\mathbf{z}}_{2\left(0\right)}^{\left(n\right)}\hfill \\ \mathrm{For}{k}_2=1,2,\dots ,K_2:\hfill \\ {\mathbf{q}}_{2\left(k_2\right)}^{\left(n\right)}={\mathbf{R}}_2^{\left(n\right)}{\mathbf{c}}_{2(k_2-1)}^{\left(n\right)},{\alpha }_{2\left(k_2\right)}^{\left(n\right)}=\frac{{\gamma }_{2(k_2-1)}^{\left(n\right)}}{{\left({\mathbf{c}}_{2(k_2-1)}^{\left(n\right)}\right)}^T{\mathbf{q}}_{2\left(k_2\right)}^{\left(n\right)}}\hfill \\ {\underline{\mathbf{g}}}_{2,\mathrm{W}\left(k_2\right)}^{\left(n\right)}={\underline{\mathbf{g}}}_{2,\mathrm{W}(k_2-1)}^{\left(n\right)}+{\alpha }_{2\left(k_2\right)}^{\left(n\right)}{\mathbf{c}}_{2(k_2-1)}^{\left(n\right)}\hfill \\ {\mathbf{z}}_{2\left(k_2\right)}^{\left(n\right)}={\mathbf{z}}_{2(k_2-1)}^{\left(n\right)}-{\alpha }_{2\left(k_2\right)}^{\left(n\right)}{\mathbf{q}}_{2\left(k_2\right)}^{\left(n\right)}\hfill \\ {\gamma }_{2\left(k_2\right)}^{\left(n\right)}={\left({\mathbf{z}}_{2\left(k_2\right)}^{\left(n\right)}\right)}^T{\mathbf{z}}_{2\left(k_2\right)}^{\left(n\right)},{\beta }_{2\left(k_2\right)}^{\left(n\right)}=\frac{{\gamma }_{2\left(k_2\right)}^{\left(n\right)}}{{\gamma }_{2(k_2-1)}^{\left(n\right)}}\hfill \\ {\mathbf{c}}_{2\left(k_2\right)}^{\left(n\right)}={\mathbf{z}}_{2\left(k_2\right)}^{\left(n\right)}+{\beta }_{2\left(k_2\right)}^{\left(n\right)}{\mathbf{c}}_{2(k_2-1)}^{\left(n\right)}\hfill \\ \underline{\mathrm{Solution}\mathrm{of} \left(17\right)}:\hfill \\ {\mathbf{R}}_{12}^{\left(n\right)}={\left({\underline{\overline{\mathbf{G}}}}_{2,11}^{\left(n\right)}\right)}^T{\mathbf{R}}_{\mathbf{x}}{\underline{\overline{\mathbf{G}}}}_{2,11}^{\left(n\right)},{\mathbf{r}}_{12}^{\left(n\right)}={\left({\underline{\overline{\mathbf{G}}}}_{2,11}^{\left(n\right)}\right)}^T{\mathbf{r}}_{\mathbf{x}d}\hfill \\ {\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}\left(0\right)}^{\left(n\right)}={\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}\left(K_1\right)}^{(n-1)},{\mathbf{z}}_{12\left(0\right)}^{\left(n\right)}={\mathbf{r}}_{12}^{\left(n\right)}-{\mathbf{R}}_{12}^{\left(n\right)}{\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}\left(0\right)}^{\left(n\right)},{\mathbf{c}}_{12\left(0\right)}^{\left(n\right)}={\mathbf{z}}_{12\left(0\right)}^{\left(n\right)},{\gamma }_{12\left(0\right)}^{\left(n\right)}={\left({\mathbf{z}}_{12\left(0\right)}^{\left(n\right)}\right)}^T{\mathbf{z}}_{12\left(0\right)}^{\left(n\right)}\hfill \\ \mathrm{For}{k}_1=1,2,\dots ,K_1:\hfill \\ {\mathbf{q}}_{12\left(k_1\right)}^{\left(n\right)}={\mathbf{R}}_{12}^{\left(n\right)}{\mathbf{c}}_{12(k_1-1)}^{\left(n\right)},{\alpha }_{12\left(k_1\right)}^{\left(n\right)}=\frac{{\gamma }_{12(k_1-1)}^{\left(n\right)}}{{\left({\mathbf{c}}_{12(k_1-1)}^{\left(n\right)}\right)}^T{\mathbf{q}}_{12\left(k_1\right)}^{\left(n\right)}}\hfill \\ {\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}\left(k_1\right)}^{\left(n\right)}={\underline{\overline{\mathbf{g}}}}_{12,\mathrm{W}(k_1-1)}^{\left(n\right)}+{\alpha }_{12\left(k_1\right)}^{\left(n\right)}{\mathbf{c}}_{12(k_1-1)}^{\left(n\right)}\hfill \\ {\mathbf{z}}_{12\left(k_1\right)}^{\left(n\right)}={\mathbf{z}}_{12(k_1-1)}^{\left(n\right)}-{\alpha }_{12\left(k_1\right)}^{\left(n\right)}{\mathbf{q}}_{12\left(k_1\right)}^{\left(n\right)}\hfill \\ {\gamma }_{12\left(k_1\right)}^{\left(n\right)}={\left({\mathbf{z}}_{12\left(k_1\right)}^{\left(n\right)}\right)}^T{\mathbf{z}}_{12\left(k_1\right)}^{\left(n\right)},{\beta }_{12\left(k_1\right)}^{\left(n\right)}=\frac{{\gamma }_{12\left(k_1\right)}^{\left(n\right)}}{{\gamma }_{12(k_1-1)}^{\left(n\right)}}\hfill \\ {\mathbf{c}}_{12\left(k_1\right)}^{\left(n\right)}={\mathbf{z}}_{12\left(k_1\right)}^{\left(n\right)}+{\beta }_{12\left(k_1\right)}^{\left(n\right)}{\mathbf{c}}_{12(k_1-1)}^{\left(n\right)}\hfill \\ \underline{\mathrm{Solution}\mathrm{of} \left(18\right)}:\hfill \\ {\mathbf{R}}_{11}^{\left(n\right)}={\left({\underline{\overline{\mathbf{G}}}}_{2,12}^{\left(n\right)}\right)}^T{\mathbf{R}}_{\mathbf{x}}{\underline{\overline{\mathbf{G}}}}_{2,12}^{\left(n\right)},{\mathbf{r}}_{11}^{\left(n\right)}={\left({\underline{\overline{\mathbf{G}}}}_{2,12}^{\left(n\right)}\right)}^T{\mathbf{r}}_{\mathbf{x}d}\hfill \\ {\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(0\right)}^{\left(n\right)}={\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(K_1\right)}^{(n-1)},{\mathbf{z}}_{11\left(0\right)}^{\left(n\right)}={\mathbf{r}}_{11}^{\left(n\right)}-{\mathbf{R}}_{11}^{\left(n\right)}{\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(0\right)}^{\left(n\right)},{\mathbf{c}}_{11\left(0\right)}^{\left(n\right)}={\mathbf{z}}_{11\left(0\right)}^{\left(n\right)},{\gamma }_{11\left(0\right)}^{\left(n\right)}={\left({\mathbf{z}}_{11\left(0\right)}^{\left(n\right)}\right)}^T{\mathbf{z}}_{11\left(0\right)}^{\left(n\right)}\hfill \\ \mathrm{For}{k}_1=1,2,\dots ,K_1:\hfill \\ {\mathbf{q}}_{11\left(k_1\right)}^{\left(n\right)}={\mathbf{R}}_{11}^{\left(n\right)}{\mathbf{c}}_{11(k_1-1)}^{\left(n\right)},{\alpha }_{11\left(k_1\right)}^{\left(n\right)}=\frac{{\gamma }_{11(k_1-1)}^{\left(n\right)}}{{\left({\mathbf{c}}_{11(k_1-1)}^{\left(n\right)}\right)}^T{\mathbf{q}}_{11\left(k_1\right)}^{\left(n\right)}}\hfill \\ {\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}\left(k_1\right)}^{\left(n\right)}={\underline{\overline{\mathbf{g}}}}_{11,\mathrm{W}(k_1-1)}^{\left(n\right)}+{\alpha }_{11\left(k_1\right)}^{\left(n\right)}{\mathbf{c}}_{11(k_1-1)}^{\left(n\right)}\hfill \\ {\mathbf{z}}_{11\left(k_1\right)}^{\left(n\right)}={\mathbf{z}}_{11(k_1-1)}^{\left(n\right)}-{\alpha }_{11\left(k_1\right)}^{\left(n\right)}{\mathbf{q}}_{11\left(k_1\right)}^{\left(n\right)}\hfill \\ {\gamma }_{11\left(k_1\right)}^{\left(n\right)}={\left({\mathbf{z}}_{11\left(k_1\right)}^{\left(n\right)}\right)}^T{\mathbf{z}}_{11\left(k_1\right)}^{\left(n\right)},{\beta }_{11\left(k_1\right)}^{\left(n\right)}=\frac{{\gamma }_{11\left(k_1\right)}^{\left(n\right)}}{{\gamma }_{11(k_1-1)}^{\left(n\right)}},\hfill \\ {\mathbf{c}}_{11\left(k_1\right)}^{\left(n\right)}={\mathbf{z}}_{11\left(k_1\right)}^{\left(n\right)}+{\beta }_{11\left(k_1\right)}^{\left(n\right)}{\mathbf{c}}_{11(k_1-1)}^{\left(n\right)}\hfill \end{array}$$

.

4. Simulation Results

The experimental setup is based on a network echo cancellation scenario [36], as previously described in Section 2, related to the results reported in Figure 2 and Figure 3. The experiments were performed using MATLAB R2018b (for programming and graphic representations), running on a GIGABYTE AORUS 15G XC device (Windows 10 OS) sourced by GYGABYTE, Taipei, Taiwan, which has an Intel Core i7-10870H CPU with 8 Cores, 16 Logical Processors (@2.21 GHz base speed), and 32 GB of RAM. As a performance measure, the normalized misalignment (in dB) is involved in all the following experiments. Summarizing, the main goal is to identify an impulse response $\mathbf{h}$ of length $L=512$ (corresponding to a network echo path), while the input signal $x\left(t\right)$ is an AR(1) process. In this context, the reference signal $d\left(t\right)$ is obtained based on (1), using a white Gaussian additive noise $v\left(t\right)$ with different SNRs. Specifically, three SNR levels are used, i.e., 20 dB, 10 dB, and 0 dB. The first one corresponds to good SNR conditions, where the noise level is mild, thus expecting a good accuracy of the Wiener filter. Second, using $\mathrm{SNR}=10$ dB corresponds to moderate noisy conditions, while noticing the influence in reducing the accuracy of the solution (i.e., an increase in the misalignment). Finally, heavy noise conditions are considered when $\mathrm{SNR}=0$ dB, critically influencing the reliability of the Wiener filter.

4.1. IWF-TOT-CG versus WF-CG

In the first set of simulations, the performance of the proposed IWF-TOT-CG is analyzed according to its main parameters, i.e., $K_1$, $K_2$, and P. The benchmark algorithm involved in comparisons is the WF-CG, using $K=50$ CG updates. As shown in Figure 2 and Figure 3, this value of K is sufficient for the WF-CG to reach the conventional WF solution. Since $L=512$, the decomposition of the IWF-TOT-CG is performed using $L_{11}=L_{12}=16$ and $L_2=2$.

In Figure 4, the influence of $K_1$ is assessed, i.e., the maximum number of CG updates used for the two longer filters within the IWF-TOT-CG. Since the third filter (of length $L_2^2=4$) is very short and the corresponding CG parameter ($K_2$) should be much smaller than its length, it is natural to use the smallest value $K_2=1$. In this simulation, the decomposition parameter of the IWF-TOT-CG is set to $P=2$; the influence of this parameter will be analyzed in an upcoming experiment. Also, the amount of data available for estimating the statistics is $N=5L$, while $\mathrm{SNR}=20$ dB. These represent good conditions for the WF-CG (i.e., the comparing algorithm) to obtain a reliable estimate in terms of accuracy. Nevertheless, as we can notice from Figure 4, the proposed IWF-TOT-CG reaches a significantly lower misalignment level (i.e., a better accuracy) for all the values of $K_1$. A larger value of this parameter leads to a faster convergence rate, but up to a certain limit. On the other hand, a larger number of iterations also increases the computational complexity in terms of the number of operations. Consequently, a compromise should be made. As we can notice from Figure 4, increasing the value of $K_1$ to more than 12 (e.g., 14) does not lead to performance improvements. Besides, the difference between the cases $K_1=10$ and $K_1=12$ is not so apparent (as compared to the difference between $K_1=8$ and $K_1=10$), slightly improving the convergence rate, while reaching the same misalignment level (i.e., accuracy). In this context, using $K_1=10$ or 12 are reasonable choices.

A similar experiment is considered in Figure 5, but using a fixed value for the first CG parameter (i.e., $K_1=10$) and two different values for $K_2$. These represent the minimum and maximum values for this parameter, i.e., $K_2=1$ and $K_2=4$, respectively. The maximum value is associated with the length of the corresponding filter ($L_2^2$). The other conditions remain the same as in the previous simulation. Under these circumstances, it can be noticed in Figure 5 that a higher value of $K_2$ does not significantly influence the overall performance of the IWF-TOT-CG so it is natural to use $K_2=1$ in the following experiments. The experiment from Figure 5 was performed to assess the influence of $K_2$, i.e., the number of CG iterations for the shorter filter. We should note that similar conclusions can be obtained when using other values of $K_1$ (for the two longer filters).

Next, the influence of the decomposition parameter P is analyzed in Figure 6. Based on the previous experiments, the setup used for the IWF-TOT-CG is $K_1=12$ and $K_2=1$, while $N=5L$ and $\mathrm{SNR}=20$ dB. As shown in Section 2, the decomposition parameter is chosen such that $P\ll L_{12}$, relying on the low-rank feature of the impulse response. For all the values of P considered in Figure 6, the IWF-TOT-CG outperforms the WF-CG. Even the minimum value $P=1$ produces a reasonable attenuation of the normalized misalignment, showing improved accuracy as compared to the benchmark algorithm. Moreover, we can notice that increasing the value of P beyond a certain value does not improve the overall performance, which also certifies the low-rank approach.

Finally, the last experiment from this first set concerns the influence of different conditions on the performance of the proposed IWF-TOT-CG, as compared to the WF-CG. To this purpose, in Figure 7, several scenarios are considered using different amounts of data for estimating the statistics and lower SNRs. The IWF-TOT-CG uses the same CG parameters as in the previous simulation, while $P=1$. This represents a very advantageous setup as compared to the WF-CG counterpart. While the benchmark algorithm involves a single filter of length $L=512$, the proposed version combines the estimates provided by three filters of lengths $P{L}_{11}{L}_2$, $P{L}_{12}{L}_2$, and $L_2^2$, which have 32, 32, and 4 coefficients, respectively. Consequently, there is an important reduction in terms of the parameter space, using only 68 coefficients instead of 512. As a result, due to the significantly smaller data structures used within the IWF-TOT-CG, the proposed algorithm is much more robust in harsh conditions, as compared to the WF-CG. This is supported in Figure 7, where the IWF-TOT-CG outperforms the WF-CG algorithm, especially when using a low amount of data for estimating the statistics (e.g., $N=L$ in Figure 7b) and/or in noisy environments (e.g., $\mathrm{SNR}=10$ or 0 dB in Figure 7c and d, respectively).

4.2. IWF-TOT-CG versus IWF-TOT

The second set of experiments focuses on the comparison between the proposed IWF-TOT-CG and its counterpart recently developed in [18], namely IWF-TOT. This iterative Wiener filter is also based on TOT decomposition but uses direct matrix inversion to solve the associated sets of Wiener–Hopf equations. Therefore, in terms of their decomposition, both algorithms will use the same setup, i.e., $L=L_{11}{L}_{12}{L}_2$, with $L_{11}=L_{12}=16$ and $L_2=2$. Besides, the IWF-TOT-CG involves its specific CG cycles, using the settings $K_1=12$ and $K_2=1$ (as in the previous set of simulations).

In Figure 8, the two TOT-based algorithms are compared for different values of the decomposition parameter P under favorable conditions, i.e., $N=5L$ data samples (to estimate the statistics) and $\mathrm{SNR}=20$ dB. While for $P=1$ the performances are very similar, the CG-based version achieves a better accuracy for a larger value of P. This supports the advantage of the line search methods (like CG), as compared to the traditional matrix inversion approach, which is also indicated in other previous works [33,34,35]. As shown in the experiment related to Figure 6, increasing the value of P beyond a certain value does not lead to performance improvement, while increasing the computational complexity on the other hand. In fact, this trade-off relies on the low-rank approach. As indicated in [18], for network impulse responses, the rank of the corresponding matrices is much lower than $L_{12}$, e.g., usually less than $L_{12}/5$. In our scenario, this leads to a range for P between 1 and 3. This was also previously supported in Figure 6, where we can notice that the misalignment curves for $P=2$ and $P=3$ are very similar. Consequently, there is no reason for choosing P beyond these values, since there is no performance gain while paying in terms of computational complexity.

The robustness of the line search methods in noisy environments is also an important feature to be considered. The experiment provided in Figure 9 outlines this gain. As we can notice, even for $P=1$ (which led to similar results in good SNR conditions), the IWF-TOT-CG performs better as compared to the previous IWF-TOT [18] in low SNR environments.

4.3. IWF-TOT-CG versus IWF-NKP-CG

The last set of experiments aims to evaluate the Wiener filters based on the CG method in conjunction with the decomposition-based approaches. The proposed IWF-TOT-CG is compared with a recently designed version of the iterative Wiener filter [33]. This algorithm (namely IWF-NKP-CG) also uses the CG method to solve the associated Wiener–Hopf equations, but it relies on the second-order NKP decomposition, thus following the initial approach from [8]. In this case, the length of the filter is factorized as $L=L_1^{*}{L}_2^{*}$, while the low-rank approach relies on the decomposition parameter $P^{*}<L_2^{*}$. In our scenario that considers an impulse response of length $L=512$, this decomposition is performed using $L_1^{*}=32$ and $L_2^{*}=16$. The IWF-NKP-CG involves two sets of Wiener–Hopf equations, which correspond to two component Wiener filters of lengths $P^{*}{L}_1^{*}$ and $P^{*}{L}_2^{*}$. Their solutions are obtained based on the CG method, using a maximum number of updates (denoted by $K^{*}$). Since in the involved setup, we have $L_2^{*}=L_{12}$, it is reasonable to use $K^{*}=K_1$.

In Figure 10, the required statistics are obtained by averaging across $N=5L$ available data samples, while $\mathrm{SNR}=20$ dB. These represent reasonably good conditions, which favor reliable estimates. Nevertheless, the IWF-NKP-CG [33] is outperformed by the proposed IWF-TOT-CG for different values of their decomposition parameters. This performance gain results from the TOT decomposition compared to the second-order NKP-based approach, which further supports the initial findings from [18]. For this experiment, the values of P and $P^{*}$ were selected according to the low-rank approach. Concerning the values of P (for IWF-TOT-CG), these considerations were previously indicated related to Figure 6 and Figure 8. Similar aspects can be outlined when choosing the values of $P^{*}$ (for IWF-NKP-CG). As recently indicated in [33] and previously supported in [8], for network impulse responses, the rank of the corresponding matrix of size $L_1^{*}\times L_2^{*}$ (where $L_1^{*}\times L_2^{*}=L$) associated with the reshaped impulse response is much lower than $L_2^{*}$, e.g., usually less than $L_2^{*}/6$. In this context, it is reasonable to consider $P^{*}<3$ in Figure 10, in order to properly address the trade-off between performance and complexity. Otherwise, using larger values of $P^{*}$ will not justify the performance gain, while increasing the computational amount.

Finally, the IWF-NKP-CG [33] and the proposed IWF-TOT-CG (using $P^{*}=P=1$) are compared in more challenging conditions, when using smaller amounts of data for estimating the required statistics, i.e., $N=2L$ and $N=L$. As we can notice in Figure 11, the performance gain of the IWF-TOT-CG is more apparent in these cases, which shows the robustness of the TOT-based decomposition in conjunction with the CG method.

5. Conclusions and Future Works

In this paper, we have developed an iterative version of the Wiener filter using the CG method and exploiting a tensorial decomposition of the impulse response. The resulting IWF-TOT-CG combines the solutions of three sets of Wiener–Hopf equations which are solved with CG updates, thus avoiding matrix inversion operations. An important gain is related to the dimensionality reduction in a long-length system identification problem, which can be reformulated using a reduced set of coefficients, corresponding to three (much) shorter filters. This approach fits very well with the identification of low-rank impulse responses, like in echo cancellation.

In terms of its performance, the proposed IWF-TOT-CG outperforms the conventional Wiener filter, especially in some challenging scenarios. Here, we can mention the cases when dealing with a small amount of available data (to estimate the statistics) or working in low SNR environments. While the accuracy of the conventional Wiener filter is highly affected in such conditions, the proposed version (which operates with smaller data structures) is still robust and provides reliable solutions. Moreover, the IWF-TOT-CG performs better as compared to the previously developed IWF-TOT [18], which involves matrix inversion operations to solve the Wiener–Hopf equations. Also, the proposed algorithm provides improved performance as compared to its counterpart based on the second-order NKP decomposition, i.e., the IWF-NKP-CG [33].

Future works will focus on three main directions. First, we can exploit other line search methods to solve the Wiener–Hopf equations, like those based on the coordinate descent technique [34,35]. A comparison between these methods is beyond the scope of this paper; however, this could open the path toward using inexact line search methods in conjunction with decomposition-based algorithms. Among them, we can mention the dichotomous coordinate descent technique [40,41,42,43,44], which is very appealing in terms of computational efficiency. Second, another direction for future works targets the extension to higher-order tensors, which could lead to improved decomposition-based solutions and higher dimensionality reduction. Third, it would be highly useful to extend the decomposition-based approach and the tensorial framework to other potential solutions used for system identification problems, like the Kalman filter and different adaptive filtering algorithms. These developments could be further used in real-world applications, like echo cancellation, active noise control, and interference reduction.