Robust Bias Compensation Method for Sparse Normalized Quasi-Newton Least-Mean with Variable Mixing-Norm Adaptive Filtering

Abstract

Input noise causes inescapable bias to the weight vectors of the adaptive filters during the adaptation processes. Moreover, the impulse noise at the output of the unknown systems can prevent bias compensation from converging. This paper presents a robust bias compensation method for a sparse normalized quasi-Newton least-mean (BC-SNQNLM) adaptive filtering algorithm to address these issues. We have mathematically derived the biased-compensation terms in an impulse noisy environment. Inspired by the convex combination of adaptive filters’ step sizes, we propose a novel variable mixing-norm method, BC-SNQNLM-VMN, to accelerate the convergence of our BC-SNQNLM algorithm. Simulation results confirm that the proposed method significantly outperforms other comparative works regarding normalized mean-squared deviation (NMSD) in the steady state.

1. Introduction

Adaptive filtering algorithms play a pivotal role in signal processing, encompassing tasks such as system identification, channel estimation, feedback cancellation, and noise removal [1]. While literature commonly assumes a Gaussian distribution for system noise, real-world scenarios, including underwater acoustics [2,3,4,5], low-frequency atmospheric disturbances [6], and artificial interference [7,8,9], often exhibit sudden changes in signal or noise intensity [10]. These abrupt variations can disrupt algorithms, serving as external solid interference or outliers [11,12].

Recently, the sparse quasi-Newton least-mean mixed-norm (SQNLMMN) algorithm [13] has emerged as a potential solution to mitigate the impact of both Gaussian and non-Gaussian noises on the convergence behavior of adaptive algorithms [14]. This algorithm introduces a novel cost function incorporating a linear combination of $L_2$ and $L_4$ norms while promoting sparsity. Despite its promise, the SQNLMMN algorithm exhibits specific weaknesses. Firstly, it overlooks the presence of input noise at the adaptive filter inputs, leading to biased coefficient estimates [15,16]. Secondly, the fixed mixing parameter $\delta$, governing the balance between the two norms, fails to adapt dynamically. This rigidity in parameter choice trades off convergence rate and mean squared deviation (MSD) concerning the weight coefficients. Notably, the approach of employing a mixed step size for least mean fourth (LMF) and normalized LMF algorithms [17] to address such trade-offs [18] differs from the concept of a variable mixing parameter [19,20].

Based on the unbiased criteria, several methods have been reported to compensate the biases caused by the noisy inputs, such as the bias compensation least mean square algorithm (BC-LMS) [15,21], bias compensation normalized LMS algorithm (BC-NLMS) [22], bias compensation proportional normalized least mean square algorithm (BC-PNLMS) [23], bias compensation normalized least mean fourth algorithm (BC-NLMF) [24], bias compensated affine-projection-like (BC-APL) [25], and bias compensation least mean mixed norm algorithm (BC-LMMN) [26]. However, the BC-LMMN algorithm used a fixed mixing factor, which resulted in a higher misadjustment. In [27], the authors proposed using a biased-compensated generalized mixed norm algorithm and cooperating with correntropy-induced metric (CIM) as the sparse penalty constraint for sparse system identification problems. Unlike the conventional mixed norm approach, they mixed norm with $p=1.1$ and $q=1.2$ to better combat non-Gaussian noise as well as impulse noise. A modified cost function that considered the cost caused by the input noise was used to compensate for the bias [27]. Hereinafter, we refer to it as the BC-CIM-LGMN algorithm. Yet, estimating the input noise power might be adversely affected by impulse noise present at the output of the unidentified system. The same trick that adopted CIM as the sparse penalty constraint is applied in [23], which is referred to as BC-CIM-PNLMS hereinafter. On the other hand, an $L_0$ norm cost function was used to accelerate the convergence for the sparse systems. In [28], the authors combined it with an improved BC-NLMS, referred to as the BC-ZA-NLMS algorithm. Unfortunately, the BC-ZA-NLMS algorithm fails to consider the impact of the impulse noise. Researchers have proposed combining the variable step-size (VSS) method with BC-LMS to the direction of arrival (DOA) estimation problem [29].

However, few studies have comprehensively addressed all impairments, including noisy input, impulse noise in observations (measurements), and sparse unknown systems. Building upon the SQNLMMN algorithm, this paper introduces a robust bias compensation method for the sparse normalized quasi-Newton least-mean with variable mixing-norm (BC-SNQNLM-VMN) adaptive filtering algorithm. The key contributions of this research are as follows. Firstly, we introduce a normalized variant of the SQNLMN algorithm and incorporate it with the Huber function to alleviate the impact of impulse noise. Secondly, we develop a bias compensation method to counteract the influence of noisy input on the weight coefficients of the adaptive filter. Thirdly, we introduce a convex combination approach concerning the mixing parameter, enabling the utilization of the variable mixing parameter in the mixed norm approach. Consequently, our proposed method can simultaneously achieve rapid convergence and low misadjustment.

The rest of this paper is organized as follows. Section 2 describes the system model we considered in this work. Section 3 briefly reviews the SQNLMMN algorithm and outlines the proposed BC-SNQNLM-VMN adaptive filtering algorithm. Section 4 validates the effectiveness of our proposed BC-SNQNLM-VMN algorithm by using computer simulations. Conclusions and future prospects are drawn in Section 5.

2. System Models

The system with finite impulse represented by the vector $\mathbf{w}\in {\mathbb{R}}^{M\times 1}$ to be identified that considers both input noise and observation noise is depicted in Figure 1. The outputs from this system are subject to corruption by two types of noise. The observable desired signal $d\left(n\right)$ can be mathematically defined as follows:

$$\begin{array}{c}\hfill d\left(n\right)=y\left(n\right)+\left[v\left(n\right)+\Omega \left(n\right)\right]={\mathbf{u}}^T\left(n\right)\mathbf{w}\left(n\right)+\left[v\left(n\right)+\Omega \left(n\right)\right]\end{array},$$

(1)

where ${\left(·\right)}^T$ is the transpose symbol; the weight vector $\mathbf{w}\left(n\right)$, with components $w_0\left(n\right),w_1\left(n\right),\dots ,w_{M-1}\left(n\right)$, arranged in column form, represents the unknown system to be identified. $\mathbf{w}\left(n\right)={\left[w_0\left(n\right),w_1\left(n\right),\dots ,w_{M-1}\left(n\right)\right]}^T$ represents the weight vector of the unknown system; $\mathbf{u}\left(n\right)={\left[u\left(n\right),u(n-1),\dots ,u(n-M+1)\right]}^T$ denotes the input regressor vector. Note that the measurement noise is assumed to consist of two components: background additive Gaussian white noise (AGWN) denoted as $v\left(n\right)$ and impulse noise denoted as $\Omega \left(n\right)$. The AGWN noise $v\left(n\right)$ has a zero mean and variance ${\sigma }_v^2$. This variance is related to the signal-to-noise ratio (SNR) as follows:

$$\begin{array}{c}\hfill {\mathrm{SNR}}_v=10{log}_{10}({\sigma }_y^2/{\sigma }_v^2),\end{array}$$

(2)

where ${\sigma }_y^2$ represents the variance of $y\left(n\right)$. In addition, impulse noise is accounted for in the system model. Two conventional models are employed in this study. The first one is the Bernoulli Gaussian (BG) model [30], defined as follows:

$$\begin{array}{c}\hfill \Omega \left(n\right)=b\left(n\right)·v_{\Omega }\left(n\right),\end{array}$$

(3)

where, $b\left(n\right)$ takes the value of one with a probability of $Pr$ and zero with a probability of $(1-Pr)$. Additionally, $v_{\Omega }\left(n\right)$ represents a Gaussian white process characterized by a mean of zero and a variance of ${\sigma }_{\Omega }^2$. The strength of this impulse noise is $SN{R}_{\Omega }$ is used to quantify its strength as follows:

$${\mathrm{SNR}}_{\Omega }=10{log}_{10}({\sigma }_y^2/{\sigma }_{\Omega }^2).$$

(4)

Another model utilized is the alpha-stable impulse noise model [13], which can be characterized by the parameter vector $V=({\alpha }_s,{\beta }_s,{\Gamma }_s,{\Delta }_s)$. Here ${\alpha }_s\in (0,2]$ represents the characteristic factor, ${\beta }_s\in [-1,1]$ denotes the symmetry parameter, ${\Gamma }_s\ge 0$ stands for the dispersion parameter, and ${\Delta }_s$ indicates the location parameter. A reduced ${\alpha }_s$ value signifies a heightened presence of impulse noise.

Figure 1. Model of system identification with the proposed BC-SNQNLM-VMN algorithm.

Figure 1. Model of system identification with the proposed BC-SNQNLM-VMN algorithm.

In this paper, we consider the noisy input case, i.e., the input of the adaptive filter $\overline{u}\left(n\right)$ differs from that of the unknown system. We assume an AGWN input noise $\eta \left(n\right)$ with zero-mean and variance ${\sigma }_{\eta }^2$ is added to the original input $u\left(n\right)$, i.e., $\overline{u}\left(n\right)=u\left(n\right)+\eta \left(n\right)$. The strength of $\eta \left(n\right)$ is determined by the $SN{R}_{\eta }$ as follows:

$${\mathrm{SNR}}_{\eta }=10{log}_{10}({\sigma }_u^2/{\sigma }_{\eta }^2),$$

(5)

where ${\sigma }_u^2$ denotes the variance of $u\left(n\right)$. The weights of the adaptive filter, denoted by $\widehat{\mathbf{w}}\left(n\right)$, are updated iteratively through an adaptive algorithm, which computes correction terms based on $\widehat{y}\left(n\right)$. These corrections rely on the error signal, expressed as:

$$\overline{e}\left(n\right)=d\left(n\right)-\widehat{y}\left(n\right)=d\left(n\right)-{\overline{\mathbf{u}}}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right),$$

(6)

where $\overline{\mathbf{u}}={[\overline{u}\left(n\right),\overline{u}(n-1),\dots ,\overline{u}(n-M+1)]}^T$ denotes the input regressor vector linked to the adaptive filter.

3. Proposed BC-SNQNLM-VMN Adaptive Filtering Algorithm

3.1. Review of SQNLMMN Algorithm [13]

The cost function of the SQNLMMN algorithm is expressed as:

$$\begin{array}{c}\hfill J\left(\mathbf{w}\left(n\right)\right)=\frac{\delta }{2}{J}_2\mathbf{w}\left(n\right)+\frac{1-\delta }{4}{J}_4\mathbf{w}\left(n\right)+\gamma S\left(\mathbf{w}\left(n\right)\right),\end{array}$$

(7)

where $J_2\triangleq E\left[e^2\left(n\right)\right]$ and $J_4\triangleq E\left[e^4\left(n\right)\right]$ are the cost functions for least mean square (LMS) and LMF algorithms, respectively; a fixed mixing parameter $0\le \delta \le 1$ is used to control the mixture of the two cost functions; $S(·)$ denotes the sparsity-promoting term, which is regulated by a positive parameter $\gamma$. According to [13], we have the resulting updating recursion of the sparse quasi-Newton least-mean mixed-norm (SQNLMMN) algorithm as follows:

$$\widehat{\mathbf{w}}(n+1)=\widehat{\mathbf{w}}\left(n\right)+{\mu }_1\mathbf{P}\left(n\right)\mathbf{u}\left(n\right)e\left(n\right)+{\mu }_2\mathbf{P}\left(n\right)\mathbf{u}\left(n\right)e^3\left(n\right)-p\mathbf{P}\left(n\right)\mathbf{g}\left(n\right),$$

(8)

where the step size is chosen as ${\mu }_1=\delta \mu$ and ${\mu }_2=(1-\delta )\mu$ that controls the convergence rate for LMS and LMF algorithms, respectively. Note that $\mu$ is a common step size; $p\mathbf{P}\left(n\right)\mathbf{g}\left(n\right)$ denotes the sparsity penalty term, and p denotes the parameter that controls zero-attraction [13]. The matrix $\mathbf{P}\left(n\right)\in {\mathbf{R}}^{M\times M}$ that approximates the inverse of the Hessian matrix of the cost function can be expressed as follows:

$$\mathbf{P}\left(n\right)=\mathbf{B}\left(n\right)\left[\mathbf{I}-\alpha \gamma \left(\mathbf{I}-\mathbf{H}(n-1)\mathbf{B}\left(n\right)\alpha \gamma \right)\mathbf{H}(n-1)\mathbf{B}\left(n\right)\right],$$

(9)

where $\mathbf{B}\left(n\right)\in {\mathbf{R}}^{M\times M}$ is described as follows:

$$\mathbf{B}\left(n\right)=\frac{1}{1-\alpha }\left[\mathbf{P}(n-1)\frac{{\mathbf{P}}^T(n-1)\mathbf{u}\left(n\right){\mathbf{u}}^T\left(n\right)\mathbf{P}(n-1)}{\frac{1-\alpha }{{\alpha }_1+{\alpha }_2{e}^2\left(n\right)}+{\mathbf{u}}^T\left(n\right)\mathbf{P}(n-1)\mathbf{u}\left(n\right)}\right]$$

(10)

with ${\alpha }_1=\delta \alpha$ and ${\alpha }_2=3\left(1-\delta \right)\alpha$. Note that $\mathbf{H}\left(n\right)\in {\mathbf{R}}^{M\times M}$ is the Hessian matrix for $S\left(\widehat{\mathbf{w}}\left(n\right)\right)$. Let $S(·)$ be the $L_0$ norm and approximate as follows:

$$\begin{array}{c}\hfill S\left(\widehat{\mathbf{w}}\left(n\right)\right)\approx \sum _{i=0}^{M-1}\left(1-e^{-\beta |{\widehat{w}}_i\left(n\right)|}\right),\end{array}$$

(11)

where the parameter $\beta >0$ is used to determine the region of zero attraction [31].

The derivation of the gradient for this penalty term is as follows:

$$\mathbf{g}\left(n\right)={\left[t\left({\widehat{w}}_0\left(n\right)\right),t\left({\widehat{w}}_1\left(n\right)\right),\dots ,t\left({\widehat{w}}_{M-1}\left(n\right)\right)\right]}^T,$$

(12)

where

$$\begin{array}{c}\hfill t\left({\widehat{w}}_i\left(n\right)\right)=\beta \mathrm{sgn}\left({\widehat{w}}_i\left(n\right)\right)e^{-\beta |{\widehat{w}}_i\left(n\right)|},\forall 0\le i\le (M-1)\end{array},$$

(13)

and the operator $\mathrm{sgn}(·)$ denotes the sign function. In order to streamline Equation (13), we utilize the first-order Taylor approximation of the exponential function in the following manner:

$$\begin{array}{c}\hfill e^{-\beta \left|u\right|}\approx \left\{\begin{array}{c}1-\beta \left|u\right|,\left|u\right|\le {\beta }^{-1}\hfill \\ 0,\mathrm{otherwise}\hfill \end{array}\right..\end{array}$$

(14)

Therefore, we can approximate Equation (13) as follows:

$$\begin{array}{c}\hfill t\left({\widehat{w}}_i\left(n\right)\right)=\left\{\begin{array}{c}-\beta (1+\beta {\widehat{w}}_i\left(n\right)),-{\beta }^{-1}\le {\widehat{w}}_i\left(n\right)<0\hfill \\ \beta (1-\beta {\widehat{w}}_i\left(n\right)),0<{\widehat{w}}_i\left(n\right)\le {\beta }^{-1}\hfill \\ 0,\mathrm{otherwise}\hfill \end{array}\right..\end{array}$$

(15)

The Hessian of Equation (11) can be derived as:

$$\mathbf{H}\left(n\right)=\mathrm{diag}\left(t^{\prime }\left({\widehat{w}}_0\left(n\right)\right),t^{\prime }\left({\widehat{w}}_1\left(n\right)\right),\dots ,t^{\prime }\left({\widehat{w}}_{M-1}\left(n\right)\right)\right)$$

(16)

with

$$\begin{array}{c}\hfill t^{\prime }\left({\widehat{w}}_i\left(n\right)\right)=\left\{\begin{array}{cc}-{\beta }^2\hfill & ,|{\widehat{w}}_i\left(n\right)|\le {\beta }^{-1}\hfill \\ 0\hfill & ,\mathrm{otherwise}\hfill \end{array}\right..\end{array}$$

(17)

3.2. Normalized SQNLMMN

Inspired by the design of normalized LMS and normalized LMF [32], we propose a normalized version of SQNNLMMN algorithm by modifying Equation (8) that considers the noisy inputs $\overline{u}\left(n\right)$ as follows:

$$\widehat{\mathbf{w}}(n+1)=\widehat{\mathbf{w}}\left(n\right)+\frac{{\mu }_1\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right)\overline{e}\left(n\right)}{{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2}+\frac{{\mu }_2\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right){\overline{e}}^3\left(n\right)}{{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2({\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2+{\overline{e}}^2\left(n\right))}-p\overline{\mathbf{P}}\left(n\right)\mathbf{g}\left(n\right)$$

(18)

where $\overline{\mathbf{u}}\left(n\right)=\mathbf{u}\left(n\right)+\mathit{\eta }\left(n\right)$ denotes the noisy input regressor vector and $\mathit{\eta }\left(n\right)={[\eta \left(n\right),\eta (n-1),...,\eta (n-M+1)]}^T$ represents the input noise vector. The noisy error signal $\overline{e}\left(n\right)$ is calculated as follows:

$$\begin{array}{cc}\hfill \overline{e}\left(n\right)& =d\left(n\right)-{\overline{\mathbf{u}}}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)=e\left(n\right)-{\mathit{\eta }}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right).\hfill \end{array}$$

(19)

Note that the matrix $\overline{\mathbf{P}}\left(n\right)$ is a contaminated version of $\mathbf{P}\left(n\right)$ (see Equation (9)) defined as follows:

$$\begin{array}{c}\hfill \overline{\mathbf{P}}\left(n\right)=\overline{\mathbf{B}}\left(n\right)-\overline{\mathbf{B}}\left(n\right)\alpha \gamma (\mathbf{I}-\mathbf{H}(n-1)\overline{\mathbf{B}}\left(n\right)\alpha \gamma )\mathbf{H}(n-1)\overline{\mathbf{B}}\left(n\right),\end{array}$$

(20)

where $\gamma =p/\mu >0$ governs the impact of the penalty term; $0<\alpha \le 0.1$ denotes a forgetting factor [13]; the matrix $\overline{\mathbf{B}}\left(n\right)$ is a contaminated version of $\mathbf{B}\left(n\right)$ (see Equation (10)) defined as follows:

$$\begin{array}{c}\hfill \overline{\mathbf{B}}\left(n\right)=\frac{1}{1-\alpha }\left[\overline{\mathbf{P}}(n-1)\frac{{\overline{\mathbf{P}}}^T(n-1)\overline{\mathbf{u}}\left(n\right){\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{P}}(n-1)}{\frac{1-\alpha }{{\alpha }_1+{\alpha }_2{\overline{e}}^2\left(n\right)}+{\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{P}}(n-1)\overline{\mathbf{u}}\left(n\right)}\right]\end{array}.$$

(21)

Note that the difference between $\overline{e}\left(n\right)$ and $e\left(n\right)$, i.e., ${\mathit{\eta }}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)$, results in the biases during the weight updating process.

3.3. Bias Compensation Design

To compensate for the bias of the normalized SQNLMMN algorithm, we introduce a bias compensation vector $\mathbf{b}\left(n\right)\in {\mathbf{R}}^{M\times 1}$ into the weight-updating recursion and rewrite Equation (18) as follows:

$$\widehat{\mathbf{w}}(n+1)=\widehat{\mathbf{w}}\left(n\right)+\Delta \widehat{\mathbf{w}}\left(n\right)+\mathbf{b}\left(n\right),$$

(22)

with

$$\begin{array}{c}\hfill \Delta \widehat{\mathbf{w}}\left(n\right)=\frac{{\mu }_1\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right)\overline{e}\left(n\right)}{{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2}+\frac{{\mu }_2\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right){\overline{e}}^3\left(n\right)}{{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2({\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2+{\overline{e}}^2\left(n\right))}-p\overline{\mathbf{P}}\left(n\right)\mathbf{g}\left(n\right).\end{array}$$

(23)

We further define the weight estimation error vector as follows:

$$\begin{array}{c}\hfill \tilde{\mathbf{w}}\left(n\right)=\widehat{\mathbf{w}}\left(n\right)-\mathbf{w}\left(n\right).\end{array}$$

(24)

By combining Equations (22) and (24), we then have the following recursion

$$\tilde{\mathbf{w}}(n+1)=\tilde{\mathbf{w}}\left(n\right)+\Delta \widehat{\mathbf{w}}\left(n\right)+\mathbf{b}\left(n\right).$$

(25)

It has been reported that the sparsity terms in Equation (23), i.e., $p\overline{\mathbf{P}}\left(n\right)\mathbf{g}\left(n\right)$, should be ignored when deriving the bias compensation term $\mathbf{b}\left(n\right)$; otherwise the derived vector $\mathbf{b}\left(n\right)$ will compensate for the bias caused by both the input noise and this term [28]. Hence, the recursion for weight updating can be formulated as follows:

$$\tilde{\mathbf{w}}(n+1)=\tilde{\mathbf{w}}\left(n\right)+\frac{{\mu }_1\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right)\overline{e}\left(n\right)}{{\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{u}}\left(n\right)}+\frac{{\mu }_2\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right){\overline{e}}^3\left(n\right)}{{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2({\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2+{\overline{e}}^2\left(n\right))}+\mathbf{b}\left(n\right).$$

(26)

Given the noisy input vector $\overline{\mathbf{u}}\left(n\right)$, we then derive $\mathbf{b}\left(n\right)$ based on the unbiased criterion as follows:

$$\begin{array}{c}\hfill E\left[\tilde{\mathbf{w}}(n+1)|\overline{\mathbf{u}}\left(n\right)\right]=\mathbf{0}\mathrm{whenever}E\left[\tilde{\mathbf{w}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]=\mathbf{0}.\end{array}$$

(27)

Furthermore, to simplify the analysis, two commonly used assumptions have been employed [33] as follows:

Assumption 1.

The input noise $\eta \left(n\right)$ and background noise are zero-mean AGWN noises and the ratio $\rho ={\sigma }_v^2/{\sigma }_{\eta }^2$ is a prior knowledge.

Assumption 2.

The signals $\eta \left(n\right)$, $v\left(n\right)$, $\Omega \left(n\right)$, $u\left(n\right)$, and $\tilde{\mathit{w}}\left(n\right)$ are statistically independent.

By taking expectation on both sides of Equation (26) for the given $\overline{\mathbf{u}}\left(n\right)$ and assuming $E\left[\tilde{\mathbf{w}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]=\mathbf{0}$, we have

$$\begin{array}{cc}\hfill E\left[\tilde{\mathbf{w}}(n+1)|\overline{\mathbf{u}}\left(n\right)\right]& =E\left[\left.\frac{{\mu }_1\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right)\overline{e}\left(n\right)}{{\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{u}}\left(n\right)}\right|\overline{\mathbf{u}}\left(n\right)\right]\hfill \\ \hfill & +E\left[\left.\frac{{\mu }_2\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right){\overline{e}}^3\left(n\right)}{{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2({\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2+{\overline{e}}^2\left(n\right))}\right|\overline{\mathbf{u}}\left(n\right)\right]+E\left[\mathbf{b}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right].\hfill \end{array}$$

(28)

By replacing $\overline{\mathbf{u}}\left(n\right)$ and $\overline{e}\left(n\right)$ in the first term of the right-hand side (RHS) of Equation (28), we have

$$\begin{array}{cc}\hfill E\left[\left.\frac{{\mu }_1\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right)\overline{e}\left(n\right)}{{\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{u}}\left(n\right)}\right|\overline{\mathbf{u}}\left(n\right)\right]& =E\left[\left.\frac{{\mu }_1\overline{\mathbf{P}}\left(n\right)\mathbf{u}\left(n\right)e\left(n\right)}{{\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{u}}\left(n\right)}\right|\overline{\mathbf{u}}\left(n\right)\right]\hfill \\ \hfill & -E\left[\left.\frac{{\mu }_1\overline{\mathbf{P}}\left(n\right)\mathit{\eta }\left(n\right){\mathit{\eta }}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)}{{\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{u}}\left(n\right)}\right|\overline{\mathbf{u}}\left(n\right)\right],\hfill \end{array}$$

(29)

where

$$\begin{array}{cc}\hfill E& \left[\left.\frac{{\mu }_1\overline{\mathbf{P}}\left(n\right)\mathbf{u}\left(n\right)e\left(n\right)}{{\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{u}}\left(n\right)}\right|\overline{\mathbf{u}}\left(n\right)\right]=\frac{{\mu }_{1}E\left[\overline{\mathbf{P}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]}{{\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{u}}\left(n\right)}E\left[\left.\mathbf{u}\left(n\right)\left(-{\mathbf{u}}^T\left(n\right)\tilde{\mathbf{w}}\left(n\right)+v\left(n\right)\right)\right|\overline{\mathbf{u}}\left(n\right)\right]\hfill \\ \hfill =& \frac{{\mu }_{1}E\left[\overline{\mathbf{P}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]}{{\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{u}}\left(n\right)}\begin{array}{c}\hfill \left\{-E\left[\mathbf{u}\left(n\right){\mathbf{u}}^T\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]E\left[\tilde{\mathbf{w}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]+E\left[\mathbf{u}\left(n\right)v\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]\right\}\end{array}=\mathbf{0}\hfill \end{array}$$

(30)

and

$$\begin{array}{cc}\hfill E\left[\left.\frac{{\mu }_1\overline{\mathbf{P}}\left(n\right)\mathit{\eta }\left(n\right){\mathit{\eta }}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)}{{\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{u}}\left(n\right)}\right|\overline{\mathbf{u}}\left(n\right)\right]& =\frac{{\mu }_{1}E\left[\overline{\mathbf{P}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]}{{\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{u}}\left(n\right)}E\left[\mathit{\eta }\left(n\right){\mathit{\eta }}^T\left(n\right)\right]E\left[\widehat{\mathbf{w}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]\hfill \\ \hfill & =\frac{{\mu }_{1}E\left[\overline{\mathbf{P}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]{\sigma }_{\eta }^2}{{\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{u}}\left(n\right)}E\left[\widehat{\mathbf{w}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right].\hfill \end{array}$$

(31)

Note that as the condition ${\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2\gg {\overline{e}}^2\left(n\right)$ and the deviation of ${\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2$ being small hold, the the second term of the RHS of Equation (28) can be approximated as follows [33]:

$$\begin{array}{c}\hfill E\left[\left.\frac{{\mu }_2\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right){\overline{e}}^3\left(n\right)}{{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2({\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2+{\overline{e}}^2\left(n\right))}\right|\overline{\mathbf{u}}\left(n\right)\right]\approx \frac{E\left[{\mu }_2\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right){\overline{e}}^3\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]}{E\left[{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2({\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2+{\overline{e}}^2\left(n\right))|\overline{\mathbf{u}}\left(n\right)\right]}.\end{array}$$

(32)

Thus, we can rewrite the nominator of Equation (32) as follows:

$$\begin{array}{cc}\hfill E\left[{\mu }_2\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right){\overline{e}}^3\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]& =E\left[\left.{\mu }_2\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right){\left(e\left(n\right)-{\mathit{\eta }}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)\right)}^3\right|\overline{\mathbf{u}}\left(n\right)\right]\hfill \\ \hfill & =E\left[{\mu }_2\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right)\begin{array}{c}\hfill e^3\left(n\right)\end{array}|\overline{\mathbf{u}}\left(n\right)\right]\hfill \\ \hfill & +E\left[\left.{\mu }_2\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right)\begin{array}{c}\hfill 3e\left(n\right){\left({\mathit{\eta }}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)\right)}^2\end{array}\right|\overline{\mathbf{u}}\left(n\right)\right]\hfill \\ \hfill & -E\left[\left.{\mu }_2\overline{\mathbf{P}}\left(n\right)\overline{\mathbf{u}}\left(n\right){\left({\mathit{\eta }}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)\right)}^3\right|\overline{\mathbf{u}}\left(n\right)\right]\hfill \\ \hfill & ={\sigma }_{\eta }^{4}E\left[\left.{\mu }_2\overline{\mathbf{P}}\left(n\right)\widehat{\mathbf{w}}\left(n\right){\widehat{\mathbf{w}}}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)\right|\overline{\mathbf{u}}\left(n\right)\right].\hfill \end{array}$$

(33)

Furthermore, we can rewrite the denominator of the RHS of Equation (32) as follows:

$$\begin{array}{cc}\hfill & E\left[\left.{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2({\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2+{\overline{e}}^2\left(n\right))\right|\overline{\mathbf{u}}\left(n\right)\right]\hfill \\ \hfill & =E\left[{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2\left.\left({\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2+e^2\left(n\right)-2e\left(n\right){\mathit{\eta }}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)+{\left({\mathit{\eta }}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)\right)}^2\right)\right|\overline{\mathbf{u}}\left(n\right)\right]\hfill \\ \hfill & ={\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2\left({\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2+E\left[e^2\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]+E\left[\left.{\left({\mathit{\eta }}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)\right)}^2\right|\overline{\mathbf{u}}\left(n\right)\right]\right),\hfill \end{array}$$

(34)

where

$$\begin{array}{c}\hfill E\left[e^2\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]=E\left[\left.{\left({\mathbf{u}}^T\left(n\right)\tilde{\mathbf{w}}\left(n\right)\right)}^2-2{\mathbf{u}}^T\left(n\right)\tilde{\mathbf{w}}\left(n\right)v\left(n\right)+v^2\left(n\right)\right|\overline{\mathbf{u}}\left(n\right)\right]={\sigma }_v^2\end{array}$$

(35)

and

$$\begin{array}{c}\hfill E\left[\left.{\left({\mathit{\eta }}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)\right)}^2\right|\overline{\mathbf{u}}\left(n\right)\right]=E\left[{\widehat{\mathbf{w}}}^T\left(n\right)\mathit{\eta }\left(n\right){\mathit{\eta }}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]={\sigma }_{\eta }^{2}E\left[{\widehat{\mathbf{w}}}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right].\end{array}$$

(36)

Combining the results Equations (29) to (36) and substituting them into Equation (28), we obtain the following results:

$$\begin{array}{cc}\hfill E\left[\mathbf{b}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]& =\frac{{\mu }_{1}E\left[\overline{\mathbf{P}}\left(n\right)\right|\overline{\mathbf{u}}\left(n\right)]{\sigma }_{\eta }^2}{{\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{u}}\left(n\right)}E\left[\widehat{\mathbf{w}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]\hfill \\ \hfill & +{\mu }_2\frac{{\sigma }_{\eta }^{4}E\left[\overline{\mathbf{P}}\left(n\right)\widehat{\mathbf{w}}\left(n\right){\widehat{\mathbf{w}}}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]}{{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2\left({\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2+{\sigma }_{\eta }^{2}E\left[{\widehat{\mathbf{w}}}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]\right)}.\hfill \end{array}$$

(37)

By using the stochastic approximation [34], we derive the bias-compensation vector as follows:

$$\begin{array}{cc}\hfill \mathbf{b}\left(n\right)& ={\mu }_1\frac{E\left[\overline{\mathbf{P}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]{\sigma }_{\eta }^2\widehat{\mathbf{w}}\left(n\right)}{{\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{u}}\left(n\right)}+{\mu }_2\frac{E\left[\overline{\mathbf{P}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]{\sigma }_{\eta }^4\widehat{\mathbf{w}}\left(n\right){\widehat{\mathbf{w}}}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)}{{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2\left({\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2+{\sigma }_v^2+{\sigma }_{\eta }^2{\widehat{\mathbf{w}}}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)\right)}\hfill \end{array}$$

(38)

with

$$\begin{array}{cc}\hfill E\left[\overline{\mathbf{P}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]& :={\mathbf{P}}_{bias}\left(n\right)\hfill \\ \hfill & ={\mathbf{B}}_{bias}\left(n\right)\left[\mathbf{I}-\alpha \gamma \left(\mathbf{I}-\mathbf{H}(n-1){\mathbf{B}}_{bias}\left(n\right)\alpha \gamma \right)\mathbf{H}(n-1){\mathbf{B}}_{bias}\left(n\right)\right],\hfill \end{array}$$

(39)

where

$$\begin{array}{cc}\hfill {\mathbf{B}}_{bias}\left(n\right)& :=E\left[\overline{\mathbf{B}}\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]\hfill \\ \hfill & =\frac{1}{1-\alpha }\left[{\mathbf{P}}_{bias}(n-1)\frac{{\mathbf{P}}_{bias}^T(n-1)\overline{\mathbf{u}}\left(n\right){\overline{\mathbf{u}}}^T\left(n\right){\mathbf{P}}_{bias}(n-1)}{\frac{1-\alpha }{{\alpha }_1+{\alpha }_2\left({\sigma }_v^2+{\sigma }_{\eta }^2{\widehat{\mathbf{w}}}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)\right)}+{\overline{\mathbf{u}}}^T\left(n\right){\mathbf{P}}_{bias}(n-1)\overline{\mathbf{u}}\left(n\right)}\right].\hfill \end{array}$$

(40)

3.4. Variable Mixing Parameter Design

For the conventional SQNLMMN algorithm, it was suggested to use a fixed mixing parameter $\delta =0.8$ to achieve the best performance in terms of the convergence rate. However, a small mixing parameter, say $\delta =0.2$, could slowly achieve a lower misadjustment in the steady state than that with a large $\delta$. This inspires us to use a variable mixing parameter to attain a fast convergence rate and small misadjustment simultaneously.

Figure 2 depicts the block diagram of the variable mixing parameter scheme design. Two adaptive filters are combined as follows:

$$\begin{array}{c}\hfill \widehat{\mathbf{w}}(n+1)={\widehat{\mathbf{w}}}_1(n+1){\lambda }_C(n+1)+{\widehat{\mathbf{w}}}_2(n+1)\left[1-{\lambda }_C(n+1)\right],\end{array}$$

(41)

where ${\widehat{\mathbf{w}}}_1\left(n\right)$ and ${\widehat{\mathbf{w}}}_2\left(n\right)$ are the fast and slow filters, respectively, i.e., ${\delta }_1>{\delta }_2$; $\lambda (n+1)$ is the smoothed combination factor. Referring to Equation (18), the adaptation recursion for each filter can be expressed as follows:

$${\widehat{\mathbf{w}}}_i(n+1)={\widehat{\mathbf{w}}}_i\left(n\right)+\Delta {\widehat{\mathbf{w}}}_i\left(n\right)+\kappa {\mathbf{b}}_i\left(n\right),$$

(42)

with

$$\begin{array}{c}\hfill \Delta {\widehat{\mathbf{w}}}_i\left(n\right)=\frac{{\mu }_{1,i}{\overline{\mathbf{P}}}_i\left(n\right)\overline{\mathbf{u}}\left(n\right){\overline{e}}_i\left(n\right)}{{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2}+\frac{{\mu }_{2,i}{\overline{\mathbf{P}}}_i\left(n\right)\overline{\mathbf{u}}\left(n\right){\overline{e}}_i^3\left(n\right)}{{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2({\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2+{\overline{e}}_i^2\left(n\right))}-p{\overline{\mathbf{P}}}_i\left(n\right){\mathbf{g}}_i\left(n\right),\end{array}$$

(43)

where ${\mathbf{g}}_i\left(n\right)$ can be calculated by Equation (12) for ${\widehat{\mathbf{w}}}_i\left(n\right)$, and

$$\begin{array}{c}\hfill {\overline{\mathbf{P}}}_i\left(n\right)={\overline{\mathbf{B}}}_i\left(n\right)\left\{\mathbf{I}-\alpha \gamma \left[\mathbf{I}-{\mathbf{H}}_i(n-1){\overline{\mathbf{B}}}_i\left(n\right)\alpha \gamma )\right]{\mathbf{H}}_i(n-1){\overline{\mathbf{B}}}_i\left(n\right)\right\},\end{array}$$

(44)

where ${\mathbf{H}}_i(n-1)$ can be calculated by Equation (16) for ${\widehat{\mathbf{w}}}_i(n-1)$ and the scaling factor $0\ll \kappa <1$ in Equation (42) is used to mitigate the interaction between $\Delta {\widehat{\mathbf{w}}}_i\left(n\right)$ and ${\mathbf{b}}_i\left(n\right)$; ${\mu }_{1,i}=\mu {\delta }_i$ and ${\mu }_{2,i}=\mu (1-{\delta }_i)$; the matrix $\overline{\mathbf{B}}\left(n\right)$ is a contaminated version of $\mathbf{B}\left(n\right)$ (see Equation (10)) defined as follows:

$$\begin{array}{c}\hfill {\overline{\mathbf{B}}}_i\left(n\right)=\frac{1}{1-\alpha }\left[{\overline{\mathbf{P}}}_i(n-1)\frac{{\overline{\mathbf{P}}}_i^T(n-1)\overline{\mathbf{u}}\left(n\right){\overline{\mathbf{u}}}^T\left(n\right){\overline{\mathbf{P}}}_i(n-1)}{\frac{1-\alpha }{{\alpha }_{1,i}+{\alpha }_{2,i}{\overline{e}}_i^2\left(n\right)}+{\overline{\mathbf{u}}}^T\left(n\right){\overline{\mathbf{P}}}_i(n-1)\overline{\mathbf{u}}\left(n\right)}\right],\end{array}$$

(45)

where ${\alpha }_{1,i}={\delta }_i\alpha$ and ${\alpha }_{2,i}=3(1-{\delta }_i)\alpha$. The bias compensation vector ${\mathbf{b}}_i\left(n\right)$ associated with ${\widehat{\mathbf{w}}}_i\left(n\right)$ can be expressed as follows:

$$\begin{array}{cc}\hfill {\mathbf{b}}_i\left(n\right)& ={\mu }_{1,i}\frac{E\left[{\overline{\mathbf{P}}}_i\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]{\sigma }_{\eta }^2{\widehat{\mathbf{w}}}_i\left(n\right)}{{\overline{\mathbf{u}}}^T\left(n\right)\overline{\mathbf{u}}\left(n\right)}+{\mu }_{2,i}\frac{E\left[{\overline{\mathbf{P}}}_i\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]{\sigma }_{\eta }^4{\widehat{\mathbf{w}}}_i\left(n\right){\widehat{\mathbf{w}}}_i^T\left(n\right){\widehat{\mathbf{w}}}_i\left(n\right)}{{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2\left({\parallel \overline{\mathbf{u}}\left(n\right)\parallel }^2+{\sigma }_v^2+{\sigma }_{\eta }^2{\widehat{\mathbf{w}}}_i^T\left(n\right){\widehat{\mathbf{w}}}_i\left(n\right)\right)}\hfill \end{array}$$

(46)

with

$$\begin{array}{cc}\hfill E\left[{\overline{\mathbf{P}}}_i\left(n\right)|\overline{\mathbf{u}}\left(n\right)\right]& :={\mathbf{P}}_{bias,i}\left(n\right)\hfill \\ \hfill & ={\mathbf{B}}_{bias,i}\left(n\right)-{\mathbf{B}}_{bias,i}\left(n\right)\alpha \gamma \left(\mathbf{I}-{\mathbf{H}}_i(n-1){\mathbf{B}}_{bias,i}\left(n\right)\alpha \gamma \right){\mathbf{H}}_i(n-1){\mathbf{B}}_{bias,i}\left(n\right),\hfill \end{array}$$

(47)

where

$$\begin{array}{c}\hfill {\mathbf{B}}_{bias,i}\left(n\right)=\frac{1}{1-\alpha }\left[{\mathbf{P}}_{bias,i}(n-1)\frac{{\mathbf{P}}_{bias,i}^T(n-1)\overline{\mathbf{u}}\left(n\right){\overline{\mathbf{u}}}^T\left(n\right){\mathbf{P}}_{bias,i}(n-1)}{\frac{1-\alpha }{{\alpha }_{1,i}+{\alpha }_{2,i}\left({\sigma }_v^2+{\sigma }_{\eta }^2{\widehat{\mathbf{w}}}_i^T\left(n\right){\widehat{\mathbf{w}}}_i\left(n\right)\right)}+{\overline{\mathbf{u}}}^T\left(n\right){\mathbf{P}}_{bias,i}(n-1)\overline{\mathbf{u}}\left(n\right)}\right].\end{array}$$

(48)

The smoothed combination equation for ${\lambda }_C$ is given by

$${\lambda }_C(n+1)=\frac{1}{C}\sum _{k=n-C+2}^{n+1}\lambda \left(k\right),$$

(49)

where C is the length used to smooth the combination factor $\lambda$, which can be calculated as follows [35]:

$$\begin{array}{c}\hfill \lambda (n+1)=\mathrm{sgm}\left[a(n+1)\right]=\frac{1}{1+e^{-a(n+1)}}\end{array}$$

(50)

with

$$\begin{array}{cc}\hfill a(n+1)& =a\left(n\right)+{\mu }_a\mathrm{sgn}\left(\overline{e}\left(n\right)\right)\left[{\widehat{y}}_1\left(n\right)-{\widehat{y}}_2\left(n\right)\right]\lambda \left(n\right)\left[1-\lambda \left(n\right)\right],\hfill \end{array}$$

(51)

where ${\mu }_a$ is the step size for adjusting the recursion of $a\left(n\right)$; $\mathrm{sgm}\left(·\right)$ and $\mathrm{sgn}\left(·\right)$ denote the sigmoid and sign function, respectively. Note that we confine $\left|a(n+1)\right|\le a^{+}$, and we check if the condition holds every $N_0$ iterations. We force $\lambda (n+1)=0$ when $a(n+1)\le -a^{+}$ and set $\lambda (n+1)=1$ when $a(n+1)\ge a^{+}$ [36].

3.5. Robustness Consideration

To obtain the impact of impulse noise $\Omega \left(n\right)$ on the convergence of the adaptive filter ${\widehat{\mathbf{w}}}_i\left(n\right)$ in the proposed BC-SNQNLM-VMN algorithm, we propose applying the modified Huber function $\psi$(·) on ${\overline{e}}_i\left(n\right)$ as follows [37]:

$$\begin{array}{c}\hfill \psi \left({\overline{e}}_i\left(n\right)\right)=\left\{\begin{array}{cc}{\overline{e}}_i\left(n\right),\hfill & \mathrm{for}-{\xi }_i\le {\overline{e}}_i\left(n\right)\le {\xi }_i\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.,\end{array}$$

(52)

where ${\xi }_i$ is a threshold as follows:

$$\begin{array}{c}\hfill {\xi }_i=k_{\xi }{\widehat{\sigma }}_{{\overline{e}}_i\left(n\right)},\end{array}$$

(53)

with $k_{\xi }=2.576$.

$$\begin{array}{c}\hfill {\widehat{\sigma }}_{{\overline{e}}_i\left(n\right)}={\lambda }_{\sigma }{\widehat{\sigma }}_{{\overline{e}}_i(n-1)}+c_1(1-{\lambda }_{\sigma })+\mathrm{med}\left(A_{{\overline{e}}_i}\left(n\right)\right),\end{array}$$

(54)

where $\mathrm{med}\left(·\right)$ denotes the median operation; $A_{{\overline{e}}_i}\left(n\right)$ is an observation vector for ${\overline{e}}_i^2\left(n\right)$ with length $N_w$ defined as follows:

$$\begin{array}{c}\hfill A_{{\overline{e}}_i}\left(n\right)=\left[{\overline{e}}_i^2,\dots ,{\overline{e}}_i^2(n-N_w+1)\right].\end{array}$$

(55)

Note that we choose $c_1=2.13$, $N_w=9$, and ${\lambda }_{\sigma }=0.99$ in the computer simulations.

Furthermore, the estimation of ${\sigma }_{\eta }^2$ is required to calculate the bias compensation vector ${\mathbf{b}}_i\left(n\right)$. Concerning robustness, it has been reported that ${\sigma }_{\eta }^2$ can be estimated as follows [22,24]:

$$\begin{array}{c}\hfill {\widehat{\sigma }}_{\eta ,i}^2=\left\{\begin{array}{cc}{\overline{e}}_i^2\left(n\right)/({\parallel {\mathbf{w}}_i\left(n\right)\parallel }_2^2+\rho ),\hfill & \mathrm{if}{\overline{e}}_i^2\le 2{\parallel {\mathbf{w}}_i\left(n\right)\parallel }_2^2{\parallel \overline{\mathbf{u}}\left(n\right)\parallel }_2^2\hfill \\ {\widehat{\sigma }}_{\eta ,i}^2,\hfill & \mathrm{otherwise}\hfill \end{array}\right.\end{array}$$

(56)

where $\rho$, which is the ratio of ${\sigma }_v^2$ and ${\sigma }_{\eta }^2$, is assumed to be available as in [38].

3.6. Computational Cost Analysis

Table 1. Computational analysis for the proposed method.

No.	Operations	Adds	Muls
1	${\widehat{y}}_i\left(n\right)={\widehat{\mathbf{w}}}_i^T\left(n\right){\overline{\mathbf{u}}}_i\left(n\right)$	$M-1$	M
2	${\overline{e}}_i\left(n\right)=d\left(n\right)-{\widehat{y}}_i\left(n\right)$	1	-
3	${\overline{\mathbf{B}}}_i\left(n\right)$ in Equation (45)	$M^3+M^2-M+2$	$M^3+4M^2+M+7$
4	${\overline{\mathbf{P}}}_i\left(n\right)$ in Equation (44)	$2M^3$	$2M^3+4M^2$
5	$\Delta {\widehat{\mathbf{w}}}_i\left(n\right)$ in Equation (43)	$3M^2$	$3M^2+4M+7$
6	${\mathbf{P}}_{bias,i}\left(n\right)$ in Equation (47)	$2M^3-M^2$	$2M^3+4M^2+1$
7	${\mathbf{B}}_{bias,i}\left(n\right)$ in Equation (48)	$M^3+2M^2+3$	$M^3+4M^2+2M+4$
8	${\mathbf{b}}_i\left(n\right)$ in Equation (46)	$M^2+M$	$M^2+4M+6$
9	${\widehat{\mathbf{w}}}_i(n+1)$ in Equation (42)	$2M$	M
10	$\widehat{\mathbf{w}}(n+1)$ in Equation (41)	$M+1$	$2M$

lists the major computational cost for the proposed BC-SNQNLM-VMN algorithm in terms of the required number of adders (Adds) and multipliers (Muls). Moreover,

Table 2. Dominating computational costs comparisons.

Methods	Adds	Muls
BC-NLMS [22]	$2M$	$4M$
BC-NLMF [24]	$2M$	$5M$
BC-LMMN [26]	$3M$	$5M$
BC-CIM-LGMN [27]	$3M$	$M^2$
BC-CIM-PNLMS [23]	$3M^2$	$3M^2$
Proposed	$6M^3$	$6M^3$

compares the dominating computational costs with other comparative works. Note that we focus on the dominating terms of the total number of required adders and multipliers for simplicity consideration. Due to the high computational costs caused by the original SQNLMMN algorithm, the proposed method incurs much higher computational costs than comparative works.

4. Simulation Results

4.1. Setup

Computer simulations evaluated the effectiveness of the proposed algorithm. The unknown sparse system $\mathbf{w}$ comprises 32 taps (M = 32), which has $K=8$ nonzero taps, i.e., the sparsity is $0.75$. We randomly choose the positions of the nonzero taps among M taps, and their values follow a standard Gaussian distribution. A standard AGWN models the input signal. The signal-to-noise ratio (SNR) for the input signal ${\mathrm{SNR}}_{\eta }=10$ dB (see Equation (5)) and the SNR of the observed signal ${\mathrm{SNR}}_v=30$ dB (see Equation (2)). For the BG impulse noise model, we choose the SNR of the additive impulse noise ${\mathrm{SNR}}_{\Omega }=-30$ dB (see Equation (4)). Moreover, we designate the occurrence probability of BG impulse as $Pr={10}^{-3}$ for weak BG noise and $Pr=6\times {10}^{-2}$ for strong BG noise. For the alpha-stable impulse noise model, we define $V=(1.8,0,0.1,0)$ for weak alpha stable noise and $V=(1.5,0,0.1,0)$ for strong alpha stable noise [11]. Other main parameters are setting as follows: $\gamma =2\times {10}^{-6}$, $\mu =0.5$, $p={10}^{-6}$, $\kappa =0.8$, $\beta =5$, $a^{+}=8$, ${\mu }_a=5$, $N_0=2$, and $\alpha =0.01$.

The fast filter ${\widehat{\mathbf{w}}}_1\left(n\right)$ with ${\delta }_1=0.8$ and the slow filter ${\widehat{\mathbf{w}}}_2\left(n\right)$ with ${\delta }_2=0.2$ were used to combine the filter $\widehat{\mathbf{w}}\left(n\right)$ (see Equation (41)). We used a vector with length C to store the C consecutive values of instantaneous $\lambda \left(k\right)$ (see Equation (49)). The initial value of each element in this vector was 1. This makes the value of ${\lambda }_C$ lean to 1 during the adjustment phase, i.e., the combined filter behaves like the fast filter. The performance metric is the normalized mean-square deviation (NMSD), which can be calculated as follows:

$$\mathrm{NMSD}\left(n\right)=10{log}_{10}\left(\frac{\left|\right|\mathbf{w}-\widehat{\mathbf{w}}\left(n\right){\left|\right|}^2}{{\left|\right|\mathbf{w}\left|\right|}^2}\right).$$

(57)

We employed MATLAB^® R2022a installed on a Windows 10 64-bit operating system to conduct simulations on a computer equipped with an Intel^® Core™ i7-13700K CPU from Santa Clara, CA, United States, and 128GB DDR4 RAM. We plotted the NMSD learning curves and the evolution of mixing parameters in the simulation results by averaging over 100 independent Monte Carlo trials. The comparative works were BC-NLMS [22], BC-NLMF [24], BC-LMMN [26], BC-CIM-LGMN [27], and BC-CIM-PNLMS [23] algorithms.

4.2. Results

4.2.1. Baseline: No Impulse Noise

We compare the NMSD learning curves concerning various bias-compensated adaptive filtering algorithms in the absence of impulse noise. As shown in Figure 3, our proposed method can achieve the lowest NMSD during the steady state, outperforming the comparative works by 2.5–3.5 dB. Note that $C=1$ (see Equation (49)) implies no smoothing was applied. The results confirmed that combining two filters with a smoothed factor (see Equation (41)) exhibits a more smooth convergence behavior in the transient stage ($0.5\times {10}^4\le n\le 0.9\times {10}^4$) during the simulation. Note that: (1) the step sizes are chosen so that the convergence rate is at the same level for all algorithms to have a fair comparison; (2) the NMSD loss is about 12 dB without bias compensation.

4.2.2. Evaluation of Variable Mixing Parameter Method

As shown in Figure 4, we evaluate the effectiveness of the proposed variable mixing parameter scheme. In Figure 4a,b, the additive impulse noise corresponds to the weak and strong BG impulse noise, respectively. The results exhibit that our proposed combining mixing parameter $\delta$ scheme worked well in both types of additive impulse noise. Referring to Figure 4a, we can observe the variation of the mixing parameter in the weak BG impulse noise case as follows:

• First, ${\lambda }_C$ keeps at 1 in the early stage ($n\in [0,0.5\times {10}^4]$), which implies the proposed method behaves like $\delta =0.8$.
• Then, ${\lambda }_C$ decreases gradually during the transient stage ($n\in (0.5\times {10}^4,1.25\times {10}^4]$), which implies the proposed method behaves changing from $\delta =0.8$ to $\delta =0.2$.
• Finally, ${\lambda }_C$ keeps around 0 in the steady-state stage ($n\in (1.25\times {10}^4,2.5\times {10}^4]$).

Figure 4. NMSD learning curves (left) and evolution of ${\lambda }_C$ (right) for BG impulse noise mode: (a) weak ($Pr$ = 0.001) and (b) strong ($Pr$ = 0.06).

Figure 4. NMSD learning curves (left) and evolution of ${\lambda }_C$ (right) for BG impulse noise mode: (a) weak ($Pr$ = 0.001) and (b) strong ($Pr$ = 0.06).

Note that because the initial values for the vector with length C used to calculate ${\lambda }_C$ are all set to 1, we observed a slight decrease followed by an increase in ${\lambda }_C$ at the beginning of the adaptation process. Similar results were observed in the strong BG impulse noise case (see Figure 4b).

In Figure 5, we evaluated the impact of the additive alpha-stable impulse noise in both weak (Figure 5a) and strong (Figure 5b) cases. In this scenario, we observe that the strong alpha stable impulse noise makes the NMSD learning curves exhibit more fluctuations in the steady state. In addition, we observe that the evolution of $\lambda$ shows more fluctuations at the beginning of the transient stage and quickly reaches its steady state. This implies the proposed method behaves by changing from $\delta =0.8$ to $\delta =0.2$ earlier than in the case of weak impulse noise. Compared with the baseline, the smoothed combination method exhibits fewer fluctuations, especially in the strong impulse noise cases. Thus, we choose $C={10}^3$ without an explicit statement in the following simulation. The results have shown the impact of the noisy input on the resulting NMSD. Without bias compensation, the NMSD loss in the steady status is about 12 dB, the same as the baseline. Therefore, we can confirm the robustness of our proposed method.

4.2.3. Performance Comparisons in the Presence of Impulse Noise

As shown in Figure 6, we compare our proposed method with other comparative works for the BG impulse noise case. In the weak BG impulse noise case (see Figure 6a), our proposed method achieves the lowest NMSD and improves by 3 dB to 15 dB compared to comparative works. Note that the BC-CIM-PNLMS did not consider the impact of impulse noise, which resulted in the worst NMSD performance. However, BC-CIM-LGMN, BC-LMMN, and BC-NLMF algorithms diverged in the strong BG impulse noise case (see Figure 6b). In this case, only the BC-NLMS and our proposed method still function well, and our method improves the NMSD by 3 dB in the steady state.

As shown in Figure 7, we compare our proposed method with other comparative works for the alpha stable impulse noise case. In the weak alpha stable impulse noise case (see Figure 7a), our proposed method achieves the lowest NMSD and improves by 4.5 dB to 7 dB compared to comparative works. Note that the BC-CIM-PNLMS did not consider the impact of impulse noise, exhibiting some NMSD learning curve spikes. In addition, the comparative works exhibit poor performance in the strong alpha stable impulse noise case (see Figure 7b). In this case, the BC-CIM-PNLMS exhibits stronger spikes than that in the weak alpha stable impulse noise case. However, our proposed method shows the lowest NMSD loss (about 0.9 dB) compared to other comparative works in the steady state.

5. Conclusions and Future Prospects

The noisy input signals result in a significant NMSD loss even in the absence of an impulse noise scenario. In this paper, we have presented a robust bias compensation method for the SNQNLM algorithm. Furthermore, we have proposed a variable mixing-norm method to attain a high convergence rate and low misadjustment during adaptation. Simulation results have confirmed that our proposed BC-SNQNLM-VMN algorithm outperforms the comparative works by 3 to 15 dB for BG impulse noise and 4.5 to 7 dB for alpha stable impulse noise in terms of NMSD, respectively. Additionally, we illuminate potential pathways for overcoming remaining challenges and broadening the applicability of our methodologies:

• The interaction between weight-vector correction term $\Delta {\widehat{\mathbf{w}}}_i\left(n\right)$ and bias compensation term $\kappa {\mathbf{b}}_i\left(n\right)$ (see Equation (42)): We have employed a constant scaling factor, $\kappa$, to mitigate the interaction between the weight-vector correction term, $\Delta {\widehat{\mathbf{w}}}_i\left(n\right)$, and the bias compensation term, $\kappa {\mathbf{b}}_i\left(n\right)$. Nonetheless, devising a dynamic algorithm for adapting $\kappa$ would enhance the robustness of bias compensation methods to varying input noise over time or in scenarios involving time-varying unknown systems.
• Extension to general mixed-norm algorithms: While our study focused on $L_2$ and $L_4$ norms, the methodology can be extended to encompass bias compensation in adaptive filtering algorithms utilizing a mix of $L_p$ and $L_q$ norms, where p and q are positive parameters.
• Bias compensation for non-linear adaptive filtering systems: Addressing bias compensation becomes notably intricate in non-linear adaptive filtering systems. A future avenue of research involves developing techniques to estimate biases induced by noisy inputs in such non-linear contexts.