Switching Mechanism on the Order of Affine Projection Algorithm

Abstract

Conventional affine projection (AP) algorithm with a fixed order is subject to a tradeoff between convergence speed and steady-state misalignment. In order to address such problem, a switching mechanism on the order of AP algorithm is proposed by comparing the performance of two AP algorithms with different orders. Firstly, the mean square deviations (MSD) behavior of the AP algorithm is analyzed, and a calculation formula for computing MSD at each iteration is derived. Secondly, we design a switching mechanism to select the better order of the two AP algorithms by comparing the MSDs of them; the MSD of the chosen order is smaller than that of the other. We also give the theoretical analysis, including steady-state mean square error (MSE) and computational complexity. Finally, the experiments in system identification and echo-cancellation scenarios demonstrate that the proposed algorithm has good performance not only in a stationary environment but also in a non-stationary environment.

1. Introduction

With the rapid development of signal processing technology, filter theory plays an increasingly important role in this field [1,2,3]. As one of the typical filters, the Wiener filter needs the a priori knowledge about the statistical properties of the signal, which is frequently unavailable in practice. With regard to the adaptive filter, one of its greatest advantages is that it does not require this knowledge to accomplish the signal processing. Thus, adaptive filters are attracting considerable interest in numerous applications, such as system identification, communication, acoustic processing, industrial control, and biomedicine [4,5,6,7,8]. As for the adaptive filter, its performance is strongly influenced by the adaptive filtering algorithms. Being one of the most widely used algorithms, the least mean square (LMS) [9] algorithm is simple in structure and low in computational complexity, but slow in convergence. On the basis of the LMS algorithm, the normalized LMS (NLMS) [10] algorithm was proposed, which has further advantages in convergence speed. However, when the input signals are highly correlated, these two LMS-type algorithms will deteriorate with respect to the convergence. Refs. [11,12] presented the affine projection (AP) algorithm, which can accelerate the convergence rate by reusing the past signals. Nevertheless, there is still a conflicting problem between convergence speed and steady-state misalignment for the conventional AP algorithm.

To solve the aforementioned contradiction, some scholars put forward a few constructive strategies in the conventional AP algorithm employing time-varying parameters. By use of variable step-size (VSS) technique, a number of VSS-AP algorithms have been proposed [13,14,15]. These VSS-AP algorithms essentially use a larger step-size when the output error is large and vice versa. Furthermore, the order of the AP algorithm is another important parameter that affects the convergence rate and the misalignment. Based on this fact, an evolving-order AP (EAP) algorithm was proposed in [16], which selected the algorithm’s order by comparing the output error with a threshold. Furthermore, adaptively combining two AP algorithms with different orders is also an effective idea to address the conflict. Ref. [12] proposed a combination structure, which can perform at least as well as the best of these two components. However, there is an obvious two-stage convergence phenomenon in this algorithm. In response to such issue, a transferring approach was presented in [17], which makes the convergence process coherently at the cost of increasing the computational complexity. Later on [18], proposed a combination scheme for the NLMS and AP algorithm to achieve a faster convergence rate while reducing the cost of calculation.

In this paper, our proposal to address the conflicting problem in conventional AP algorithm is to use a mechanism that can switch the AP algorithm’s order adaptively. Through comparing the MSDs of two AP algorithms with different orders, the proposed switching mechanism is able to select the better order at each iteration so that fast convergence speed and small steady-state misalignment can be obtained simultaneously. Our proposed algorithm does not require any predetermined parameters and has relatively small computational complexity. We have conducted theoretical analysis about the MSE to prove that the proposed algorithm is reasonable and has significant advantages. Finally, the simulation results in system identification and echo-cancellation scenarios further verify the good performance of the proposed algorithm.

The components of this paper are as follows. In Section 2, the canonical AP algorithm is concisely described. Taking the MSD as the entry point, Section 3 illustrates the proposed switching mechanism for the order of AP algorithm and analyses the computational complexity. The convergence analysis of the mean square error (MSE) is also carried out in Section 4. Section 5 exhibits the simulation experiments aimed at verifying the performance of the proposed algorithm. Finally, Section 6 summarizes the conclusion.

2. Affine Projection Algorithm

In the system identification problem [13,18], the output signal of the system to be identified is defined as

$$d(i)={\mathbf{U}}_i^T{w}_o+v(i),$$

(1)

where i is the time index, whereas u_i = [u(i),u(I − 1),…, u(I − M + 1)]^T is the input vector to adaptive filter, and w_o = [w(0),w(1),…,w(M + 1)] are the filter coefficients of the system to be identified. M denotes the filter length, and v(i) is the additive background noise with variance ${\sigma }_v^2$, which is generally assumed to be Gaussian white noise. The update equation of the AP algorithm is as follows:

$$w_i=w_{i-1}+\mu {\mathbf{U}}_i^T{({\mathbf{U}}_i^T{\mathbf{U}}_i)}^{-1}{\mathbf{e}}_i,$$

(2)

where U_i = [u_i, u_i−1, …, u_i−K+1], and d_i = [d(i), d(I − 1), …, d(I − K + 1)]^T. μ is the step size, and w_i and K denote the coefficients and the order of the AP algorithm, respectively.

$$e_i={[e(i),e(i-1),\dots ,e(i-K+1)]}^T=d_i-U_i^T{w}_{i-1},$$

(3)

which is the estimation error between the outputs of the system to be identified and the AP filter. When the order K is set to 1, the AP algorithm will evolve into the NLMS algorithm with the update equation as follows:

$$w_i=w_{i-1}+\mu u_i^T{(u_i^T{u}_i)}^{-1}e(i),$$

(4)

$$e(i)=d(i)-u_i^T{w}_{i-1}.$$

(5)

The performance of the AP algorithm is mainly controlled by the order factor K. A small value for this factor results in low steady-state misalignment but slow convergence speed, while a large value will lead to fast convergence speed but increased steady-state misalignment. In conventional AP algorithm, the order factor is usually set to be a fixed value, which inevitably leads to a conflicting problem between convergence speed and steady-state misalignment.

3. Proposed Algorithm

In this section, we first study the MSD behavior of the AP algorithm and obtain its iterative calculation formula. Then, a switching mechanism on the order of AP algorithm is proposed by comparing the MSDs of two AP algorithms with different orders. The computational complexity of the proposed algorithm as well as other commonly used AP algorithms are analyzed and compared at the end of this section.

3.1. MSD Analysis of the AP Algorithm

We define the error vector between w_o and w_i as follows:

$${\tilde{w}}_i=w_o-w_i.$$

(6)

Based on the Equation (2), we obtain:

$$\begin{array}{l}{\tilde{w}}_i={\tilde{w}}_{i-1}-\mu U_i{(U_i^T{U}_i)}^{-1}{e}_i\\ ={\Phi }_i{\tilde{w}}_{i-1}-\mu U_i{(U_i^T{U}_i)}^{-1}{v}_i\end{array}$$

(7)

where

$${\Phi }_i=I-\mu U_i{(U_i^T{U}_i)}^{-1}{U}_i^T.$$

(8)

The autocorrelation matrix of the error vector ${\tilde{w}}_i$ is defined as

$$P_i=\mathrm{E}({\tilde{w}}_i{\tilde{w}}_i^T).$$

(9)

Then, the MSD of the AP algorithm at iteration i can be obtained as follows:

$${\mathrm{MSD}}_i\triangleq \mathrm{E}({\tilde{w}}_i^T{\tilde{w}}_i)\equiv \mathrm{Tr}(P_i)=p_i,$$

(10)

where E() and Tr() are the expectation and the trace operators, respectively. Assuming that the noise v(i) is iid signal and statistically independent with ${\tilde{w}}_{I-1}$, then we can obtain the following by substituting Equation (7) into (9):

$$P_i={\Phi }_i{P}_i{\Phi }_i^T+D_i$$

(11)

where

$$D_i={\sigma }_v^2{\mu }^2{(U_i^T{U}_i)}^{-1}$$

(12)

From Equations (10) and (11), we can obtain

$$p_i=\mathrm{Tr}({\Phi }_i^T{\Phi }_i{P}_{i-1})+\mathrm{Tr}(D_i)$$

(13)

The above equation is computationally simplified by dividing it into two parts, and considering the first half of Tr(${\Phi }_i^T$Φ_iP_i−1) in Equation (13), the matrix ${\Phi }_i^T$Φ_i can be expressed in the following form:

$${\Phi }_i^T{\Phi }_i=I+{\mu }^2{U}_i{(U_i^T{U}_i)}^{-1}{U}_i^T-2\mu U_i{(U_i^T{U}_i)}^{-1}{U}_i^T.$$

(14)

Thus,

$$\begin{array}{l}\mathrm{Tr}({\Phi }_i^T{\Phi }_i{P}_{i-1})=\mathrm{Tr}(P_{i-1})+{\mu }^2\mathrm{Tr}[U_i{\left(U_i^T{U}_i\right)}^{-1}{U}_i^T{P}_{i-1}]\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}-2\mu \mathrm{Tr}[U_i{\left(U_i^T{U}_i\right)}^{-1}{U}_i^T{P}_{i-1}]\end{array}$$

(15)

According to the [19,20], the following approximation can be used in the AP algorithm:

$$\mathrm{Tr}[U_i{(U_i^T{U}_i)}^{-1}{U}_i^T{P}_i]\simeq \frac{K}{M}\mathrm{Tr}(P_i)$$

(16)

Therefore, substituting (16) into (15) for simplification gives

$$\mathrm{Tr}({\Phi }_i^T{\Phi }_i{P}_{i-1})=[1+\frac{K({\mu }^2-2\mu )}{M}\left]\mathrm{Tr}\right(P_{i-1})$$

(17)

Next, considering the second term Tr(D_i) in Equation (13) shows

$$\mathrm{Tr}(D_i)={\sigma }_v^2{\mu }^2\mathrm{Tr}(U_i^T{U}_i{)}^{-1}$$

(18)

Thus, synthesizing Equations (13), (17), and (18), we can obtain the iterative calculation formula of the MSD for the AP algorithm as follows:

$$p_i=\mathrm{Tr}({\Phi }_i^T{\Phi }_i{P}_{i-1})+\mathrm{Tr}(D_i)=[1+\frac{K({\mu }^2-2\mu )}{M}]p_{i-1}+{\sigma }_v^2{\mu }^2{Q}_i$$

(19)

where

$$Q_i=\mathrm{Tr}{(U_i^T{U}_i)}^{-1}.$$

(20)

As i tends to infinity, the steady-state value of the MSD can be obtained as follows:

$$\underset{i\to \infty }{\lim } p_i=\frac{{\sigma }_v^2\mu \mathrm{Tr}{(U_i^T{U}_i)}^{-1}M}{(2-\mu )K}.$$

(21)

To verify the correctness of the above derivation results, this paper compares the current p_i and the steady-state MSD values obtained, respectively, in Equations (19) and (21) through simulation experiments. Figure 1 and Figure 2 show the results when the order K is set to 8 and 1, respectively. It can be seen from the figures that the two curves converge almost exactly to the same value as the iterations proceed. It is thus concluded that the derived p_i in this paper can predict the value of MSD very well.

3.2. Switching Mechanism

In the switching mechanism scheme, the mode with the lower MSD estimate is selected by comparing the p_i values of two AP algorithms with different orders. The update equation of the proposed algorithm can be described as follows:

$$w_i=\{\begin{array}{c}{w}_{i-1}+\mu U_{1,i}^T{(U_{1,i}^T{U}_{1,i})}^{-1}{e}_i,\mathrm{if} p_{1,i+1}\le p_{2,i+1}\\ w_{i-1}+\mu U_{2,i}^T{(U_{2,i}^T{U}_{2,i})}^{-1}{e}_i,\mathrm{if} p_{1,i+1}>p_{2,i+1}\end{array}$$

(22)

where p_1,i+1 and p_2,i+1 are the current estimates of MSDs for the AP algorithms with the order K₁ and K₂, respectively. U₁ and U₂ are the input matrices corresponding to the different orders K₁ and K₂ of the AP algorithm. Here, we set K₁ > K₂, which means that the first AP algorithm occupies fast convergence speed, while the other has low steady-state misalignment.

According to Equation (19), the current p_1,i+1 and p_2,i+1 of each AP algorithm are calculated as follows:

$$p_{1,i+1}=[1+\frac{K_1}{M}({\mu }^2-2\mu )]p_i+{\sigma }_v^2{\mu }^2\mathrm{Tr}{(U_{1,i}^T{U}_{1,i})}^{-1}$$

(23)

$$p_{2,i+1}=[1+\frac{K_2}{M}({\mu }^2-2\mu )]p_i+{\sigma }_v^2{\mu }^2\mathrm{Tr}{(U_{2,i}^T{U}_{2,i})}^{-1}$$

(24)

Based on the switching scheme in Equation (22), the mode with the lower MSD estimate is selected, and the value of p_i at the next iteration is determined as

$$p_{i+1}=\{\begin{array}{c}{p}_{1,i+1} if p_{1,i+1}\le p_{2,i+1}\\ p_{2,i+1} if p_{1,i+1}>p_{2,i+1}\end{array}.$$

(25)

The proposed switching scheme on the order for AP algorithm is summarized in

Table 1. The proposed algorithm.

Initialization: p1,0 = p2,0 = p0 = 1, w0 = 0 Computing the current pi of each AP algorithm: $p_{1,i+1}=[1+\frac{K_1}{M}({\mu }^2-2\mu )]p_i+{\sigma }_v^2{\mu }^2\mathrm{Tr}{(U_{1,i}^T{U}_{1,i})}^{-1}$ $p_{2,i+1}=[1+\frac{K_2}{M}({\mu }^2-2\mu )]p_i+{\sigma }_v^2{\mu }^2\mathrm{Tr}{(U_{2,i}^T{U}_{2,i})}^{-1}$ 2 Switching mechanism: $w_i=\{\begin{array}{c}{w}_{i-1}+\mu U_{1,i}^T{(U_{1,i}^T{U}_{1,i})}^{-1}{e}_i,\mathrm{if} p_{1,i+1}\le p_{2,i+1}\\ w_{i-1}+\mu U_{2,i}^T{(U_{2,i}^T{U}_{2,i})}^{-1}{e}_i,\mathrm{if} p_{1,i+1}>p_{2,i+1}\end{array}$ 3 Updating the pi value at the next iteration: $p_{i+1}=\{\begin{array}{c}{p}_{1,i+1} if p_{1,i+1}\le p_{2,i+1}\\ p_{2,i+1} if p_{1,i+1}>p_{2,i+1}\end{array}$ End

. It also can be seen from Equation (22) that if a bigger step size μ₁ and a smaller one μ₂ are set to the first and second modes, respectively, the proposed algorithm is able to combine the advantages of order and step size to achieve faster convergence speed and smaller steady-state error.

3.3. Computational Complexity

Table 2. Computational complexity of various algorithms.

Algorithm	Multiplication
AP	$(K^2+2K)M+K^3+K^2$
CAP	$(K_1^2+K_2^2+2K_1+2K_2+2)M+K_1^3+K_2^3+K_1+K_2+9$
CAP-WF	$(K_1^2+K_2^2+2K_1+2K_2+4)M+K_1^3+K_2^3+K_1+K_2+9$
VSS-AP	$(K^2+2K)M+K^3+K^2+2$
EAP	$(K^2+2K)M+K^3+K^2+1$
AP-NLMS	$(K^2+2K+4)M+K^3+K^2+22$
Proposed	$(K^2+2K)M+K^3+K^2+12$

lists the total number of multiplication operations required at each iteration for the conventional AP algorithm, CAP algorithm [12], the CAP-WF algorithm [17], the VSS-AP algorithm [13], EAP algorithm [16], AP-NLMS algorithm [18], and the proposed algorithm. From the results illustrated in Table 2, we can see that the computational complexity of our proposed algorithm is much smaller than that of the CAP, CAP-WF, and AP-NLMS algorithms. As the CAP and CAP-WF algorithms both need to update two filters independently at the same time, their computational complexity increases exponentially. Compared with the VSS-AP and EAP algorithms, the proposed algorithm only has 10 and 11 more multiplication operations by comparison, respectively.

4. Mean Square Error Convergence Analysis

The equation for the steady-state MSE of an AP algorithm in a stationary system, as already noted by [21], is as follows:

$$MSE={\sigma }_v^2+\frac{\mu {\sigma }_v^{2}Tr\left(E\left[{\mathrm{A}}_i\right]\right)}{(2-\mu )E\left[{\mathrm{A}}_i(1,1)\right]},$$

(26)

where A_i= (${\mathbf{U}}_i^T$U_i)⁻¹.

Since calculating the exact expected value in (26) is not practicable, here, we introduce some approximate assumptions. For one thing, the expectation of the ratio between two random variables can be similar to the ratio of their expectations, and together with Tr(E[A_i]) ≈ E[K/‖u_i‖²]), (26) can be simplified to

$$\begin{array}{l}MSE={\sigma }_v^2+\frac{\mu {\sigma }_v^{2}E\left[{\Vert u_i\Vert }^2\right]\left(E\left[\frac{K}{{\Vert u_i\Vert }^2}\right]\right)}{\left(2-\mu \right)}\\ \hspace{1em}\hspace{1em}={\sigma }_v^2+\frac{\mu {\sigma }_v^{2}E\left[{\Vert u_i\Vert }^2\right]\frac{K}{E\left[{\Vert u_i\Vert }^2\right]}}{\left(2-\mu \right)}\\ \hspace{1em}\hspace{1em}={\sigma }_v^2+\frac{\mu {\sigma }_v^{2}K}{\left(2-\mu \right)}\end{array}$$

(27)

with E[A_i(1,1)]) = E[1/‖u_i‖²]).

From the above theoretical analyses, it can be concluded that the MSE of the AP algorithm is influenced by the order K. As our proposed algorithm can select a better order of the two AP algorithms at each iteration, when the number of iterations approaches infinity, the MSE of the proposed algorithm will undoubtedly converge to the following value:

$$MSE={\sigma }_v^2+\frac{\mu {\sigma }_v^2{K}_2}{(2-\mu )}$$

(28)

5. Simulation Experiments

5.1. System Identification

To verify the performance of the proposed switching algorithm, the simulations were carried out both in stationary and non-stationary environments based on a system identification model. The MSD is used as the judging criterion in order to analyze the performance of the algorithms. A Gaussian white noise, which has no correlation with the input signal, is used as the background noise, and its variance can be easily estimated online in many practical applications. The filter coefficients of the system to be identified are generated randomly with the length M = 20, and we assume the adaptive filter has the same length. The input signal is generated by filtering a zero-mean Gaussian white sequence through a first-order AR system:

$$G_1(z)=\frac{1}{1-0.8z^{-1}}$$

(29)

The purpose for doing this is to produce a correlated input signal. All of the following results of the simulation experiments are obtained by 200 Monte Carlo trials for receiving more clearer convergence curves.

Figure 3 and Figure 4 show the convergence curves of the proposed algorithm and the conventional AP, CAP, CAP-WF, VSS-AP, and EAP algorithms in a stationary environment with SNR = 20 dB and SNR = 30 dB, respectively. The orders used in the proposed algorithm are K₁ = 16 and K₂ = 2, and the step-sizes are set as μ₁ = 0.5 and μ₂ = 0.05. To observe the performance of the algorithms more clearly, the order and the step size of CAP and CAP-WF algorithms are set to the same values as the proposed algorithm. The order and step-size of the VSS-AP algorithm are set to K = 16 and μ = 0.5, while in the EAP algorithm, they are set to K_max = 16 and μ = 0.02.

As can be seen from the Figure 3 and Figure 4, there is an inherent problem between convergence speed and steady-state misalignment for the conventional AP algorithm: a large K leads to fast convergence speed but increased steady-state misalignment and vice versa. As for the CAP algorithm, an obvious two-stage convergence phenomenon comes to light in its convergence curve. Compared with the CAP-WF, VSS-AP, and EAP algorithms, the proposed algorithm can obtain faster convergence speed and lower steady-state misalignment. Since the order values of the above algorithms are all set to greater than or equal to 2, the performance curve of the AP-NLMS algorithm is not added to the above two figures.

To demonstrate the convergence performance of the algorithms in a different system, the input signal is generated through another AR system as follows:

$$G_2(z)=\frac{1}{1-0.9z^{-1}}.$$

(30)

The length of the filter coefficients of the system to be identified is set to M = 16. Figure 5 and Figure 6 show the convergence curves of the algorithms with SNR = 20 dB and SNR = 30 dB, respectively. The orders used in the proposed algorithm are K₁ = 8 and K₂ = 1, and the step sizes are both taken as 0.1. The order and step size of the CAP and CAP-WF algorithms are set in the same ways as the proposed algorithm. The order and step size of the VSS-AP algorithm are set to K = 16 and μ = 0.5, while in the EAP algorithm, they are set to K_max = 16 and μ = 0.02. The step-size of the AP-NLMS algorithm is set to 0.1. We can see from Figure 5 and Figure 6 that the proposed algorithm clearly outperforms all of the other algorithms.

Additionally, as for the adaptive algorithm, the tracking capability is another crucial issue. Therefore, Figure 7 and Figure 8 plot the convergence curves of the algorithms in anon-stationary environment with different SNRs. We make the unknown system change abruptly at iteration 2000 and 2500 by regenerating the coefficients of the system to be identified, and all the parameters of the algorithms are consistent with Figure 3 and Figure 4. In Figure 9 and Figure 10, the unknown system changes abruptly at iteration 5000 and 6000, respectively, and all the parameters of the algorithms are consistent with Figure 5 and Figure 6. It can be seen from Figure 7, Figure 8, Figure 9 and Figure 10 that the proposed algorithm is able to track the changed unknown system quickly and provides the best performance in all different SNRs and environments, neither losing the convergence speed nor increasing the misalignment.

5.2. Echo Cancellation

The adaptive filter plays an important role in the field of echo cancellation. In this subsection, the performance of the proposed algorithm is tested and compared with the conventional AP, CAP, CAP-WF, VSS-AP, EAP, and AP-NLMS algorithms again for echo cancellation. Figure 11 shows the basic principle diagram of echo-cancellation technology, in which the echo signal r(i) is generated by transmitting the input vector u(i) through an echo path in room B. Together with the noise signal v(i) in room B, it constitutes the desired signal d(i). At the same time, the estimated signal y(i) is obtained by filtering u(i) with the coefficients of adaptive filter. If the coefficients of adaptive filter and echo path impulse response achieve the same, y(i) will be equal to the actual echo signal r(i), and then, any echo signal cannot be included in signal e(i). In this experiment, the length of impulse response for the echo path is set to 128 and shown in Figure 12. The input is a real speech signal with a sampling rate of 8 kHz, and the order and step sizes of the proposed algorithm and the CAP, CAP-WF, and AP-NLMS algorithms are set to K₁ = 8, K₂ = 1 and μ₁ = 0.1, μ₂ = 0.05. In the VSS-AP and EAP algorithms, the parameters of K and μ are both set to 8 and 0.1. Figure 13 reveals the convergence curves of the proposed algorithm together with the others. From the simulation plot, it can be seen that the proposed algorithm has the fastest convergence rate and lowest steady-state error, which again confirms the good performance of our proposed switching mechanism in the echo-cancellation scenario.

6. Conclusions

In this paper, we proposed a novel method for switching the order of the AP algorithm aimed at improving the performance of the algorithm in the convergence speed and steady-state error. By analyzing and comparing the MSDs of two AP filters with different orders, the switching mechanism can select the favorable order at each iteration so that the advantages of a large order and a small one can be combined at the same time. Moreover, our proposed algorithm does not need to introduce excessive parameters and rely on some a priori knowledge. Through extensive mathematical analysis, we proved that the algorithm is reasonable and has significant advantage in the calculation. At last, the simulations also demonstrated that our proposed algorithm has a competitive convergence speed and produces a smaller steady-state misalignment in the cases of system identification and echo cancellation in comparison to other existing algorithms.