An Improved Set-Membership Proportionate Adaptive Algorithm for a Block-Sparse System

Abstract

In this paper, an improved set-membership proportionate normalized least mean square (SM-PNLMS) algorithm is proposed for block-sparse systems. The proposed algorithm, which is named the block-sparse SM-PNLMS (BS-SMPNLMS), is implemented by inserting a penalty of a mixed $l_{2,1}$ norm of weight-taps into the cost function of the SM-PNLMS. Furthermore, an improved BS-SMPNLMS algorithm (the (BS-SMIPNLMS algorithm) is also derived and analyzed. The proposed algorithms are well investigated in the framework of network echo cancellation. The results of simulations indicate that the devised BS-SMPNLMS and BS-SMIPNLMS algorithms converge faster and have smaller estimation errors compared with related algorithms.

1. Introduction

Echo cancellation is one of the most typical applications in adaptive filtering scenarios [1,2]. Among the many adaptive algorithms, the normalized least mean square (NLMS) is a classical algorithm due to its good performance and easy implementation [3,4]. The current research on echo cancellation focuses on network echo cancellation (NEC) and acoustic echo cancellation (AEC). Both of these applications have long and sparse impulse responses, which means that the filter needs many more taps and there are mostly zero or close to zero responses [5]. Additionally, the input signal of these two systems is usually a speech signal rather than white noise. Therefore, the performance of the traditional NLMS algorithm may reduce, and the convergence speed will slow down when it is used for dealing with acoustic and network echo cancellation. To further exploit the sparse characteristic of the echo path, the proportionate NLMS (PNLMS) and zero-attracting algorithms were proposed [6,7,8,9,10,11,12,13,14,15,16]. The proportionate adaptive filtering algorithms assign corresponding step sizes to each coefficient according to its magnitude, in contrast to the NLMS algorithm whose step sizes are unified. The result of the proportionate assignment is that the large coefficients obtain large step sizes while the small coefficients obtain small step sizes. Thus, proportionate adaptive filtering algorithms are more suitable for sparse systems. However, the convergent performance of the PNLMS algorithm decreases sharply following the initial fast convergence, and the PNLMS may even behave more poorly than the NLMS. We can draw the conclusion that we can benefit from the PNLMS algorithm only in the case when the system is quite sparse. To address this problem, many related improved algorithms have been proposed [17,18,19,20,21]. The improved PNLMS (IPNLMS) algorithm is the most classical of the variation algorithms [17]. The IPNLMS algorithm fully exploits the "proportionate" characteristic and has better performance than the NLMS and PNLMS when the channel response is neither sparse nor extremely dispersive.

The sparse system responses can usually be divided into three types based on their sparse features: generalized sparse response; single-clustering block-sparse response; and two-cluster or multi-cluster block-sparse response, as shown in Figure 1 [22].

In the generalized sparse system, the nonzero coefficients distribute randomly. In the clustering block-sparse systems, the nonzero coefficients distribute in clumps of 1–2 or even more. If there is more than one cluster, we generally consider the system as a multi-clustering block-sparse system. The echo response is a kind of single-clustering block-sparse system, and the satellite communication is a kind of multi-clustering block-sparse system. To take further advantage of the prior knowledge of the block-sparse feature, many corresponding algorithms were proposed in [22,23,24,25]. The algorithm proposed in [22] is named the BS-LMS, in which the main idea is to insert a mixed $l_{2,0}$ norm as a penalty to the cost function of the conventional LMS with same group partition sizes. The PB-IPNLMS algorithm proposed in [23] divides the channel response into two parts, the sparse part and the dispersive part. The proportionate algorithm is used for the sparse part while the non-proportionate algorithm is used for the dispersive part. However, the PB-IPNLMS has a notable unpleasant property in that the assumption of just one cluster exists, and its location must be known ahead of time. The IAF-PNLMS algorithm devised in [24] uses individual factors for each coefficient rather than a overall one. As a result, the IAF-PNLMS algorithm can get outstanding performance only when the impulse response exhibits high sparseness. The IIPNLMS algorithm proposed in [25] divides the echo path response into an active region and inactive region. The NLMS algorithm is used for the former, while the PNLMS is used for the latter. Some of the algorithms mentioned above cannot fully exploit the system block-sparsity and others do not deal well with multi-clustering block-sparsity. The BS-PNLMS algorithm proposed in [26] inserts a penalty of a mixed $l_{2,1}$ norm to the cost function of the PNLMS algorithm. This algorithm further exploits the block-sparsity of the estimated system as compared to the BS-LMS, and it can serve for the multi-clustering case in addition to the single-clustering case. In spite of this, the performance in terms of convergence rate, estimation error, and computational complexity still needs to be improved.

In recent decades, the set-membership (SM) filtering technique has gained extensive attention because of its good estimation performance and computational burden reduction [27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42]. The SM filtering technique sets a model space in which there are input and output vector pairs. The updating of filter taps only occurs when the error of estimation is larger than the preset upper limit [33].

In this paper, we use the SM principle and the mixed $l_{2,1}$ norm to devise the block-sparse proportionate adaptive filtering algorithm to improve the estimation performances of the previous block-sparse algorithms. We replace the elements of the step size assignment matrix with the $l_2$ norm of the blocks instead of the absolute value. Therefore, the step size distribution of the new algorithm is based on the whole block value rather than the single coefficient value, which speeds up the convergence. In addition, the introduction of the SM principle also can help to improve the estimation accuracy and computation burden reduction. The proposed block-sparse SM-PNLMS (BS-SMPNLMS) algorithm, and its variant the improved BS-SMPNLMS (BS-SMIPNLMS) algorithm are well deduced and analyzed in detail. The performances of the devised BS-SMPNLMS and BS-SMIPNLMS algorithms are discussed in terms of the estimation error and tracking. The obtained results show that the two new block-sparse algorithms converge faster and provide smaller estimation errors compared with related algorithms .

The structure of this paper can be summarized as follows. In Section 2, we review the SM principle and the PNLMS algorithm. In Section 3, the BS-SMPNLMS and BS-SMIPNLMS algorithms are proposed within the framework of the block-sparse and set-membership theories. Section 4 presents the evaluation performances of both the BS-SMPNLMS and BS-SMIPNLMS algorithms in block-sparse signal processing. Section 5 summarizes the full text.

2. Review of Corresponding Algorithms

2.1. The PNLMS Algorithm

We assume the system input signal is $\mathbf{x}\left(n\right)={\left[x\left(n\right),x(n-1),x(n-2),\cdots ,x(n-N+1)\right]}^T$, and the impulse response of estimated system is $\mathbf{w}\left(n\right)={\left[w_0\left(n\right),w_1\left(n\right),\cdots ,w_{N-1}\left(n\right)\right]}^T$, whose length is N. The observed output signal can be described as

$$d\left(n\right)={\mathbf{x}}^T\left(n\right)\mathbf{w}\left(n\right)+u\left(n\right),$$

(1)

where $u\left(n\right)$ denotes the noise signal which is assumed to be uncorrelated with $\mathbf{x}\left(n\right)$. The estimation error is

$$e\left(n\right)=d\left(n\right)-{\mathbf{x}}^T\left(n\right)\hat{\mathbf{w}}(n-1),$$

(2)

where $\hat{\mathbf{w}}(n-1)$ denotes the estimation signal. The updating equation of the PNLMS algorithm can be described as

$$\hat{\mathbf{w}}\left(n\right)=\hat{\mathbf{w}}(n-1)+\frac{\mu \mathbf{x}\left(n\right)\mathbf{Q}(n-1)e\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{Q}(n-1)\mathbf{x}\left(n\right)+{\epsilon }_{}},$$

(3)

where $\mu$ is an overall step size, and $\epsilon$ is a small regularization parameter. $\mathbf{Q}(n-1)$ is the step size assignment matrix, which is diagonal and can be described as

$$\mathbf{Q}(n-1)=\mathrm{diag}\left\{\left.q_0(n-1),q_1(n-1),\dots ,q_{N-1}(n-1)\right\}\right..$$

(4)

The elements in $\mathbf{Q}(n-1)$ are calculated by

$$q_j(n-1)=\frac{{\alpha }_j(n-1)}{{\displaystyle \sum _{i=0}^{N-1}}{\alpha }_i(n-1)} ,0\le j\le N-1,$$

(5)

where

$${\alpha }_j(n-1)=max\left\{\rho \max \left\{\left.\delta ,\left|{\hat{w}}_0(n-1)\right|,\left|{\hat{w}}_1(n-1)\right|,\cdots ,\left|{\hat{w}}_{N-1}(n-1)\right|\right\}\right.\right.,\left.\left|{\hat{w}}_j(n-1)\right|\right\}.$$

(6)

Herein, $\rho$ is a positive constant of which range is usually $\frac{1}{N}\sim \frac{5}{N}$, and its purpose is to avoid ${\hat{w}}_j(n-1)$ stalling in the case of it being much smaller than the largest element. $\delta$ is a regularization parameter, and it is used to avoid the updating stopping when all of the taps are zeros at beginning

The IPNLMS has the same update equation as the PNLMS, which is shown in Equation (3). The IPNLMS improves the elements in $\mathbf{Q}(n-1)$ by introducing an adjusting parameter c into Equation (5)

$$q_j(n-1)=\frac{1-c}{2N}+\left(1+c\right)\frac{\left|{\hat{w}}_j\right|}{2{\displaystyle \sum _{i=0}^{N-1}}\left|{\hat{w}}_i\right|},0\le j\le N-1,$$

(7)

where $-1\le c\le 1$. When $c=-1$, the IPNLMS behaves as the NLMS, while when $c=1$, the IPNLMS works like the PNLMS. We usually choose the value of c as 0 or –0.5.

2.2. Review of the SM Principle and Corresponding Algorithm

The model space in which includes input–output vector pairs is defined as $\mathsf{\Theta }$. A upper bound of estimated error is $\gamma$. The criterion of set-membership is to seek the optimization subject to

$$|e\left(n\right){|}^2\le {\gamma }^2.$$

(8)

When $\hat{\mathbf{w}}\left(n\right)$ does not belong to $\mathsf{\Theta }$, the problem of solving optimization of the SM-PNLMS can be described as

$$min{∥\hat{\mathbf{w}}\left(n\right)-\hat{\mathbf{w}}(n-1)∥}_2^2\mathrm{s}.\mathrm{t}.d\left(n\right)-{\mathbf{x}}^T\left(n\right)\hat{\mathbf{w}}\left(n\right)=\gamma .$$

(9)

The updating equation of the SM-PNLMS is

$$\hat{\mathbf{w}}\left(n\right)=\hat{\mathbf{w}}(n-1)+\frac{{\mu }_{\mathrm{SM}}\mathbf{x}\left(n\right)\mathbf{Q}(n-1)e\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{Q}(n-1)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{SM}}},$$

(10)

where

$${\mu }_{\mathrm{SM}}=\left\{\begin{array}{c}1-\frac{\gamma }{\left|e\left(n\right)\right|},\mathrm{if}\left|e\left(n\right)\right|>\gamma \hfill \\ 0,\mathrm{otherwise}\hfill \end{array}\right..$$

(11)

Herein, the matrix $\mathbf{Q}(n-1)$ is the same as Equation (4). The role of ${\epsilon }_{\mathrm{SM}}$ in Equation (10) is the same as that of $\epsilon$ in Equation (3).

3. The New BS-SMPNLMS and BS-SMIPNLMS Algorithms

Although the SM-PNLMS algorithm can both utilize the sparsity and simplify the computational complexity, its performance still needs to be improved when the response is block-sparse, which is common in real-world echo cancellation systems. Time-domain segmentation is an effective approach for block-sparse processing, and is adopted by some existing algorithms mentioned in Section 1. In addition, using a mixed norm such as the $l_{2,1}$ norm, $l_{2,0}$ norm, or $l_{q,1}$ norm is also an effective method of block-sparse system identification [43,44,45,46,47]. To further take advantage of the block-sparsity, we propose the block-sparse set-membership PNLMS (BS-SMPNLMS) algorithm to obtain better convergence behavior and a lower level of misadjustment. We introduce the $l_{2,1}$ norm into the cost function, and then the optimization problem turns to

$$min{∥\hat{\mathbf{w}}\left(n\right)-\hat{\mathbf{w}}(n-1)∥}_{2,1{\mathbf{Q}}^{-1}(n-1)}^2\mathrm{s}.\mathrm{t}.d\left(n\right)-{\mathbf{x}}^T\left(n\right)\hat{\mathbf{w}}\left(n\right)=\gamma .$$

(12)

Using the Lagrange multiplier, we can get the cost function of the devised BS-SMPNLMS algorithm

$$J\left(n\right)={\left({∥\hat{\mathbf{w}}\left(n\right)∥}_{2,1}-{∥\hat{\mathbf{w}}(n-1)∥}_{2,1}\right)}^T{\mathbf{Q}}^{-1}\left(n-1\right)\left({∥\hat{\mathbf{w}}\left(n\right)∥}_{2,1}-{∥\hat{\mathbf{w}}(n-1)∥}_{2,1}\right)+\lambda \left(d\left(n\right)-{\mathbf{x}}^T\left(n\right)\hat{\mathbf{w}}\left(n\right)-\gamma \right),$$

(13)

where

$${∥\hat{\mathbf{w}}\left(n\right)∥}_{2,1}={∥\left[\begin{array}{c}{∥{\hat{\mathbf{w}}}_{\left[1\right]}∥}_2\\ \begin{array}{c}{∥{\hat{\mathbf{w}}}_{\left[2\right]}∥}_2\\ ⋮\\ {∥{\hat{\mathbf{w}}}_{\left[B\right]}∥}_2\end{array}\end{array}\right]∥}_1 =\sum _{s=1}^B{∥{\hat{\mathbf{w}}}_{\left[s\right]}∥}_2.$$

(14)

Herein, B is the number of blocks which can be described as $B=N/L$ , and L denotes the size of each block. Take derivative of Equation (14) , we can get

$$\frac{\partial {∥\hat{\mathbf{w}}\left(n\right)∥}_{2,1}}{\partial \hat{\mathbf{w}}\left(n\right)}={\left[\frac{\partial {∥\hat{\mathbf{w}}\left(n\right)∥}_{2,1}}{\partial {\hat{w}}_1}\right.,\frac{\partial {∥\hat{\mathbf{w}}\left(n\right)∥}_{2,1}}{\partial {\hat{w}}_2},\cdots ,\left.\frac{\partial {∥\hat{\mathbf{w}}\left(n\right)∥}_{2,1}}{\partial {\hat{w}}_N}\right]}^T.$$

(15)

For the k th coefficient which belongs to the t th block, there is

$$\begin{array}{cc}\frac{\partial {∥\hat{\mathbf{w}}\left(n\right)∥}_{2,1}}{\partial {\hat{w}}_k}\hfill & =\frac{\partial {\displaystyle \sum _{s=1}^B}{∥{\hat{\mathbf{w}}}_{\left[s\right]}∥}_2}{\partial {\hat{w}}_k}=\frac{\partial {\left[{{\hat{w}}^2}_{\left(t-1\right)L+1}+{{\hat{w}}^2}_{\left(t-1\right)L+2}+\cdots +{{\hat{w}}^2}_{tL}\right]}^{\frac{1}{2}}}{\partial {\hat{w}}_k}\hfill \\ & =\frac{1}{2}\frac{2{\hat{w}}_k}{{\left[{{\hat{w}}^2}_{\left(t-1\right)L+1}+{{\hat{w}}^2}_{\left(t-1\right)L+2}+\cdots +{{\hat{w}}^2}_{tL}\right]}^{\frac{1}{2}}}=\frac{{\hat{w}}_k}{{∥{\hat{\mathbf{w}}}_{\left[t\right]}∥}_2}.\hfill \end{array}$$

(16)

Let

$$\frac{\partial J\left(n\right)}{\partial \hat{\mathbf{w}}\left(n\right)}=\mathbf{0},$$

(17)

and

$$\frac{\partial J\left(n\right)}{\partial \lambda }=\mathbf{0}.$$

(18)

Then, we will get

$${\mathbf{Q}}^{-1}\left(n-1\right)\left(\frac{\hat{\mathbf{w}}\left(n\right)}{{∥{\hat{\mathbf{w}}}_{\left[t\right]}∥}_2}-\frac{\hat{\mathbf{w}}\left(n-1\right)}{{∥{\hat{\mathbf{w}}}_{\left[t\right]}∥}_2}\right)-\lambda \mathbf{x}\left(n\right)=\mathbf{0},$$

(19)

and

$$d\left(n\right)-{\mathbf{x}}^T\left(n\right)\hat{\mathbf{w}}\left(n\right)-\gamma =0.$$

(20)

Left multiply $\mathbf{Q}(n-1)$ in Equation (19), and we can get

$$\frac{\hat{\mathbf{w}}\left(n\right)}{{∥{\hat{\mathbf{w}}}_{\left[t\right]}∥}_2}-\frac{\hat{\mathbf{w}}\left(n-1\right)}{{∥{\hat{\mathbf{w}}}_{\left[t\right]}∥}_2}-\lambda \mathbf{Q}\left(n-1\right)\mathbf{x}\left(n\right)=\mathbf{0}.$$

(21)

Then left multiply ${\mathbf{x}}^T\left(n\right)$ in the Equation (21),

$${\mathbf{x}}^T\left(n\right)\hat{\mathbf{w}}\left(n\right)-{\mathbf{x}}^T\left(n\right)\hat{\mathbf{w}}\left(n-1\right)-\lambda {∥{\hat{\mathbf{w}}}_{\left[t\right]}∥}_2{\mathbf{x}}^T\left(n\right)\mathbf{Q}\left(n-1\right)\mathbf{x}\left(n\right)=\mathbf{0}.$$

(22)

Equation (20) can also described as

$${\mathbf{x}}^T\left(n\right)\hat{\mathbf{w}}\left(n\right)=d\left(n\right)-\gamma .$$

(23)

Substituting Equation (23) into (22), we can get

$$d\left(n\right)-{\mathbf{x}}^T\left(n\right)\hat{\mathbf{w}}\left(n-1\right)=\gamma +\lambda {∥{\hat{\mathbf{w}}}_{\left[t\right]}∥}_2{\mathbf{x}}^T\left(n\right)\mathbf{Q}\left(n-1\right)\mathbf{x}\left(n\right),$$

(24)

and we combine the following equation

$$d\left(n\right)-{\mathbf{x}}^T\left(n\right)\hat{\mathbf{w}}\left(n-1\right)=e\left(n\right).$$

(25)

Therefore, we can get

$$\lambda =\frac{e\left(n\right)-\gamma }{{∥{\hat{\mathbf{w}}}_{\left[t\right]}∥}_2{\mathbf{x}}^T\left(n\right)\mathbf{Q}\left(n-1\right)\mathbf{x}\left(n\right)}.$$

(26)

Substituting (26) into (22), we can obtain

$$\hat{\mathbf{w}}\left(n\right)=\hat{\mathbf{w}}\left(n-1\right)+\frac{e\left(n\right)-\gamma }{{\mathbf{x}}^T\left(n\right)\mathbf{Q}\left(n-1\right)\mathbf{x}\left(n\right)}\mathbf{Q}\left(n-1\right)\mathbf{x}\left(n\right).$$

(27)

The updating function is

$$\hat{\mathbf{w}}\left(n\right)=\hat{\mathbf{w}}\left(n-1\right)+{\mu }_{\mathrm{SM}}\frac{\mathbf{Q}\left(n-1\right)\mathbf{x}\left(n\right)e\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{Q}\left(n-1\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{BS}}},$$

(28)

where ${\epsilon }_{\mathrm{BS}}$ is a positive constant whose value is very small, and it is used to avoid the denominator being zero. The ${\mu }_{\mathrm{SM}}$ in (28) is the same as that in (11). Herein, the step size assignment matrix $\mathbf{Q}\left(n-1\right)$ is described as

$$\mathbf{Q}\left(n-1\right)=\mathrm{diag}\left\{q_0\left(n-1\right){\mathbf{1}}_L,q_1\left(n-1\right){\mathbf{1}}_L,\cdots ,q_{B-1}\left(n-1\right){\mathbf{1}}_L\right\},$$

(29)

where

$$q_t(n-1)=\frac{{\alpha }_t(n-1)}{{\displaystyle \sum _{s=0}^{B-1}}{\alpha }_s(n-1)},0\le t\le B-1,$$

(30)

and

$${\alpha }_t(n-1)=max\left\{\rho max\left\{\delta ,{∥{\hat{\mathbf{w}}}_{\left[1\right]}∥}_2,{∥{\hat{\mathbf{w}}}_{\left[2\right]}∥}_2,\cdots ,{∥{\hat{\mathbf{w}}}_{\left[B\right]}∥}_2\right\},{∥{\hat{\mathbf{w}}}_{\left[t\right]}∥}_2\right\}.$$

(31)

The parameter ${\mathbf{1}}_L$ in Equation (29) denotes the row vector with L length and all of its elements are ones. In order to expand the BS-SMPNLMS to make it more useful for the dispersive application field, we propose an improved BS-SMPNLMS (BS-SMIPNLMS) algorithm. The relationship between the BS-SMPNLMS and the BS-SMIPNLMS is similar to that between the PNLMS and the IPNLMS. Hence, the updating equation of the BS-SMIPNLMS is the same as that of the BS-SMPNLMS. Equation (30) in the BS-SMIPNLMS is replaced by

$$q_t(n-1)=\frac{1-c}{2N}+\left(1+c\right)\frac{{∥{\hat{\mathbf{w}}}_{\left[t\right]}∥}_2}{2{\displaystyle \sum _{s=0}^{B-1}}{∥{\hat{\mathbf{w}}}_{\left[s\right]}∥}_2},0\le t\le B-1.$$

(32)

From the updating function, we can conclude the computations in the two proposed algorithms. Because of inserting the SM principle, the total numbers of additions of the BS-SMPNLMS and BS-SMIPNLMS are less than $4N-1$ and $4N+B-1$. The total numbers of multiplications are less than $6N+3$ and $6N+B+1$. The total numbers of divisions of the two algorithms are less than 2. The number of comparison of the BS-SMPNLMS is $B+1$. Besides, both of the algorithms need a B square root.

4. Simulation and Result Analysis

In this section, four experiments are constructed to verify the performances of the devised BS-SMPNLMS and BS-SMIPNLMS algorithms. All of the experiments are created with white Gaussian noise (WGN), colored noise, and a speech signal as input signals, respectively. Herein, we assume that the signal-to-noise ratio (SNR) is 30 dB. The colored noise is generated from the WGN with a pole at 0.8. The sampling frequency of the speech signal is 8 KHz. In the first and second experiments, the influences of the block size on the BS-SMPNLMS and BS-SMIPNLMS are investigated. In the third experiment, the performances of the BS-SMPNLMS and the BS-SMIPNLMS are compared with other related algorithms. In the fourth experiment, the upper bound of estimation error is investigated in detail, which is the key parameter of the BS-SMPNLMS and BS-SMIPNLMS algorithms.

Experiment 1. In this experiment, the estimation behaviors of the BS-SMPNLMS with different block sizes are studied. The response length of the block-sparse channel is assumed to be 1024. The single-clustering response as shown in Figure 1b is located at [257,288] which has 32 taps, while the two-clustering response as shown in Figure 1c is located at [257,272] and [769,800] which has 16 and 32 taps [26]. The performances of the BS-SMPNLMS for WGN, colored noise, and the speech signal with different block sizes of 4, 16, 32, and 64 are shown in Figure 2, Figure 3 and Figure 4, respectively. Each figure is divided into two parts. The first half of the 80,000 sample shows the performance of single-clustering, while the second half of the 80,000 sample shows the performance of two-clustering.

The normalized misalignment is used to evaluate the estimating performance, and each simulation is repeated 10 times to get an average value. The simulation parameters are set as follows: when the input is the speech signal, the step sizes of the NLMS, PNLMS, and BS-PNLMS are ${\mu }_{\mathrm{NLMS}}={\mu }_{\mathrm{PNLMS}}={\mu }_{\mathrm{BSPNLMS}}=0.2$. When the input is colored noise, all of the step sizes of the algorithms above are 0.1. When the input is WGN, all the step sizes of the algorithms are $0.05$. The small positive constants of the NLMS, PNLMS, BS-PNLMS, and BS-SMPNLMS are $\epsilon ={\epsilon }_{\mathrm{BS}}=0.01$. The parameters of the PNLMS, BS-PNLMS, and BS-SMPNLMS are $\rho =5/N$, $\delta =0.01$. The bound of the BS-SMPNLMS is $\gamma =0.01$.

From the simulation results we can see that independent of single-clustering or two-clustering, the BS-SMPNLMS with $L=4$ behaves best, with the smallest block size among the four cases. On the contrary, the BS-SMPNLMS with $L=64$ behaves the worst. The PNLMS and NLMS can be considered as the BS-PNLMS with $L=1$ and $L=N$. Therefore, to obtain good estimation performance, the block size should be selected appropriately. Overall, the proposed BS-SMPNLMS algorithm with single-clustering performs better than that with two-clustering. The normalized misalignment of the colored noise input signal is smaller than that of the speech signal but larger than that of WGN. The convergence speed of the colored noise is faster than that of speech signal but slower than that of WGN.

Experiment 2. The estimation behaviors of the BS-SMIPNLMS with different block sizes are studied in this experiment. The simulation environment is almost the same as Experiment 1. The adjusted parameter of the IPNLMS, BS-IPNLMS, and BS-SMIPNLMS algorithms is $c=0$. The performances of the BS-SMIPNLMS for WGN, colored noise, and the speech signal with the block sizes 4, 16, 32, and 64 are shown in Figure 5, Figure 6 and Figure 7, respectively.

It is found that among the four situations of the block sizes 4, 16, 32, 64, the performance of the BS-SMIPNLMS with a block size of 4 is the best. The larger the block size is, the worse the algorithm behaves, with a trend is consistent with that of the BS-SMPNLMS.

Experiment 3. In this experiment, the performances of the BS-SMPNLMS and BS-SMIPNLMS algorithms are studied by comparison with the NLMS, PNLMS, IPNLMS, BS-PNLMS, and BS-IPNLMS algorithms. Herein, the block sizes are set to $L=4$ and $L=16$. All the step sizes of the NLMS, PNLMS, IPNLMS, BS-PNLMS, and BS-IPNLMS algorithms are the same. The values are 0.05, 0.1, and 0.2 for WGN, colored noise, and the speech signal, respectively. The small positive constants of the NLMS, PNLMS, BS-PNLMS, and BS-SMPNLMS algorithms are $\epsilon ={\epsilon }_{\mathrm{BS}}=0.01$, the parameters of the PNLMS, BS-PNLMS, and BS-SMPNLMS algorithms are $\rho =5/N$, $\delta =0.01$, and the bound of the BS-SMPNLMS algorithm is $\gamma =0.01$. The adjusted parameter of the IPNLMS, BS-IPNLMS, and BS-SMIPNLMS is $c=0$. The results of the comparison with different input signals are shown in Figure 8, Figure 9 and Figure 10.

It is observed that the devised BS-SMPNLMS and BS-SMIPNLMS algorithms behave better than the related algorithms. For the WGN input signal, the devised algorithms have a faster convergence rate. For the colored noise and the speech signal, the devised algorithms have both faster convergence and smaller misalignment.

Experiment 4. The parameter $\gamma$ of the BS-SMPNLMS and BS-SMIPNLMS is a key parameter which can affect the convergence rate and the normalized misalignment. In this experiment, we study the performances of the BS-SMPNLMS and BS-SMIPNLMS algorithms, with $\gamma =0.5,0.1,0.05,0.01,0.001$. The effects of $\gamma$ on the BS-SMPNLMS and BS-SMIPNLMS algorithms are presented in Figure 11 and Figure 12, respectively. Herein, we assume the input is the speech signal, and the $SNR=30dB$. Other parameters are set as follows: block size $L=4$, $\rho =5/N$, $\delta =0.01$, ${\epsilon }_{\mathrm{BS}}=0.01$, $c=0$.

We find that when $\gamma$ decreases from $0.5$ to $0.01$, the normalized misalignment is reduced. However, the normalized misalignment stops decreasing when $\gamma =0.01$ because the performances of $\gamma =0.01$ and $\gamma =0.001$ are almost the same. In previous experiments, we choose $0.01$ as the value of $\gamma$.

5. Conclusions

In this paper, two improved block-sparse SM-PNLMS algorithms for a block-sparse system have been proposed. The BS-SMPNLMS and BS-SMIPNLMS algorithms have been derived and their performances investigated and analyzed in detail with various inputs such as WGN, colored noise, and a speech signal . The simulation results showed that the BS-SMPNLMS behaves better than the traditional algorithms for both the single-clustering and two-clustering block-sparse systems. Besides, the influence of the key parameter of the BS-SMPNLMS algorithm was also studied.