Sparse Bayesian Learning for DOA Estimation with Mutual Coupling

Abstract

Sparse Bayesian learning (SBL) has given renewed interest to the problem of direction-of-arrival (DOA) estimation. It is generally assumed that the measurement matrix in SBL is precisely known. Unfortunately, this assumption may be invalid in practice due to the imperfect manifold caused by unknown or misspecified mutual coupling. This paper describes a modified SBL method for joint estimation of DOAs and mutual coupling coefficients with uniform linear arrays (ULAs). Unlike the existing method that only uses stationary priors, our new approach utilizes a hierarchical form of the Student t prior to enforce the sparsity of the unknown signal more heavily. We also provide a distinct Bayesian inference for the expectation-maximization (EM) algorithm, which can update the mutual coupling coefficients more efficiently. Another difference is that our method uses an additional singular value decomposition (SVD) to reduce the computational complexity of the signal reconstruction process and the sensitivity to the measurement noise.

1. Introduction

The problem of estimating the direction-of-arrival (DOA) of multiple narrow-band sources has received considerable attention in many fields, e.g., radar, sonar, radio astronomy and mobile communications [1]. Many high-resolution DOA estimation algorithms have been proposed in the last few decades. Their excellent performance relies crucially on perfect knowledge of the array manifold. In practice, however, the array manifold usually suffers from imperfections, such as unknown mutual coupling between antenna elements, imperfectly-known sensor positions and orientations and gain-phase imbalances. Without array manifold calibration, the performance of DOA estimation may degrade substantially. Hence, it is necessary to calibrate imperfections prior to carrying out DOA estimation.

A large number of calibration methods has been proposed to deal with the imperfect manifold caused by unknown or misspecified mutual coupling [2,3,4,5,6,7,8,9,10,11]. For example, some iterative mutual coupling auto-calibration methods for uniform linear arrays (ULAs) and uniform circular arrays (UCAs) were proposed in [2,3]. However, the high computational complexity required by iterations may be time consuming, and the convergence is not theoretical guaranteed, thus resulting in algorithmic instability. Recently, by taking advantage of the special structure of the mutual coupling matrix for ULAs, Ye and coauthors [4,5] demonstrated that DOAs can be accurately estimated without compensating for mutual coupling if a group of auxiliary sensors are added into ULAs. Dai and coauthors further extended the result to the spatial smoothing method [6], real-valued method [7] and $l_1$-norm regularized method [12]. These methods are referred to as the auxiliary methods. Neither a calibration source nor iteration is required in the auxiliary methods. However, because of only applying a middle subarray, they reduce the working array aperture, which limits the applicability of these methods to cases with either few DOAs or many array elements. Some alternatives that can calibrate the imperfect manifold with the whole array in the presence of mutual coupling are proposed in [8,9,10,11].

Recently, the emerging technique of compressive sensing (CS) has given renewed interest to the problem of DOA estimation [12,13,14,15,16,17,18]. These methods exhibit many advantages, e.g., improved robustness to noise, limited number of snapshots and correlation of signals. Sparse Bayesian learning (SBL) is a popular and important technique for the sparse signal recovery in CS [19,20,21], which formulates the signal recovery problem from a Bayesian perspective, while the sparsity information is exploited by assuming a sparse prior for the signal of interest. It is worth noting that $l_1$-norm regularized optimization is deemed to be a special example of SBL, if a maximum a posteriori (MAP) optimal estimate is adopted with a Laplace signal prior. Theoretical and empirical results show that SBL methods can achieve enhanced performance over $l_1$-norm regularized optimization [20,21]. It is generally assumed that the measurement matrix in SBL is precisely known. Unfortunately, this assumption is invalid if perturbations on the measurement matrix (e.g., when the array manifold suffers from the aforementioned imperfections) are considered. Additive perturbations [22,23] and multiplicative perturbations [16] have been addressed in the recent literature.

In this paper, we will propose a modified Bayesian method for joint estimation of DOAs and the mutual coupling coefficients with ULAs, where the optimization problem is formulated in a SBL framework with a two-stage hierarchical prior, and then, we adopt an expectation-maximization (EM) algorithm that treats the signals of interest and the mutual coupling coefficients as hidden variables and parameters, respectively. To the best of our knowledge, there are very limited SBL methods that can resolve the DOAs with mutual coupling. This problem was first addressed in [24], where a unified SBL framework was proposed to address the DOA estimation problem with typical perturbations of mutual coupling, gain-phase uncertainty and sensor location error. The main differences between our method and the method in [24] are:

• Our SBL model constitutes a two-stage hierarchical form, which results in the Student t prior in a hierarchical manner; while the model in [24] has only stationary priors. The advantage of the Student t prior is that it enforces the sparsity constraint more heavily [21,25].
• Our method provides a distinct Bayesian inference for the EM algorithm, which can update the mutual coupling coefficients more efficiently.
• Our method uses an additional singular value decomposition (SVD) to reduce the computational complexity of the signal reconstruction process and the sensitivity to the measurement noise.

These differences guarantee that our method can give a better joint estimation performance of DOAs and the mutual coupling coefficients.

2. Data Model from the Bayesian Viewpoint

2.1. DOA Estimation Model

Consider K narrow-band far-field sources impinging on an M-element ULA, where the distance between adjacent sensors is d. The K signals, $s_1\left(t\right),s_2\left(t\right),\dots ,s_K\left(t\right)$, arrive at the array from distinct directions, ${\theta }_1,{\theta }_2,\dots ,{\theta }_K$, with respect to the normal line of the array. The $M\times 1$ array output vector $\mathbf{y}\left(t\right)$ is then given by:

$$\begin{array}{c}\hfill \mathbf{y}\left(t\right)=\mathbf{C}\mathbf{A}\mathbf{s}\left(t\right)+\mathbf{n}\left(t\right),t\in \{t_1,t_2,\dots ,t_T\}\end{array}$$

(1)

where $\mathbf{y}\left(t\right)={[y_1\left(t\right),y_2\left(t\right),\dots ,y_M\left(t\right)]}^T$, $\mathbf{s}\left(t\right)={[s_1\left(t\right),s_2\left(t\right),\dots ,s_K\left(t\right)]}^T$, $\mathbf{A}=[\mathbf{\alpha }\left({\theta }_1\right),\mathbf{\alpha }\left({\theta }_2\right),\dots ,\mathbf{\alpha }\left({\theta }_K\right)]$, $\mathbf{\alpha }\left({\theta }_k\right)={[1,e^{j\varphi \left({\theta }_k\right)},\dots ,e^{j(M-1)\varphi \left({\theta }_k\right)}]}^T$, $\varphi \left({\theta }_k\right)=(-2\pi d/\lambda )sin\left({\theta }_k\right)$ and $\mathbf{n}\left(t\right)={\left[n_1\left(t\right),n_2\left(t\right),\dots ,n_M\left(t\right)\right]}^T$ is an unknown noise vector. The matrix $\mathbf{C}\in {\mathbb{C}}^{M\times M}$ is the mutual coupling matrix (MCM) for ULAs. Many theoretical or experimental studies [26,27] have demonstrated that a banded symmetric Toeplitz MCM with m ($\le \frac{M-1}{2}$) coefficients can provide a good approximation of real-world situations, i.e., $\mathbf{C}=\mathrm{toeplitz}\left(\mathbf{c}\right)$, where $\mathbf{c}=[c_0,c_1,c_2,\dots ,c_m]$ with $0<|c_1|,|c_2|,\dots ,|c_m|<c_0=1$ and $\mathrm{toeplitz}\left(\mathbf{c}\right)$ denotes a symmetric Toeplitz matrix constructed by the vector $\mathbf{c}$.

Denoting $\mathbf{Y}=[\mathbf{y}\left(t_1\right),\mathbf{y}\left(t_2\right),\dots ,\mathbf{y}\left(t_T\right)]$, $\mathbf{N}=[\mathbf{n}\left(t_1\right),\mathbf{n}\left(t_2\right),\dots ,\mathbf{n}\left(t_T\right)]$ and $\mathbf{S}=[\mathbf{s}\left(t_1\right),\mathbf{s}\left(t_2\right),\dots ,\mathbf{s}\left(t_T\right)]$, we have:

$$\begin{array}{c}\hfill \mathbf{Y}=\mathbf{C}\mathbf{A}\mathbf{S}+\mathbf{N}.\end{array}$$

(2)

Following the convention in [14], we use the singular value decomposition (SVD) to reduce the computational complex of the signal reconstruction process and the sensitivity to the measurement noise. Let the SVD of $\mathbf{Y}$ be written in the form of:

$$\begin{array}{c}\hfill \mathbf{Y}={\mathbf{U}}_s{\mathbf{\Lambda }}_s{\mathbf{V}}_s^H+{\mathbf{U}}_n{\mathbf{\Lambda }}_n{\mathbf{V}}_n^H\end{array}$$

(3)

where ${\mathbf{U}}_s\in {\mathbb{C}}^{M\times \widehat{K}}$ and ${\mathbf{V}}_s\in {\mathbb{C}}^{T\times \widehat{K}}$ are unitary matrices whose columns are the singular vectors corresponding to the $\widehat{K}$ largest singular values, while the columns of ${\mathbf{U}}_n\in {\mathbb{C}}^{M\times (M-\widehat{K})}$ and ${\mathbf{V}}_n\in {\mathbb{C}}^{T\times (M-\widehat{K})}$ are the singular vectors corresponding to the rest $M-\widehat{K}$ singular values. Note that the value of $\widehat{K}$ is generally determined by the singular values; especially if the number of sources K is exactly known, $\widehat{K}$ is set to K.

Using Equation (2) and defining $\widehat{\mathbf{Y}}=\mathbf{Y}{\mathbf{V}}_s$, we obtain:

$$\begin{array}{c}\hfill \widehat{\mathbf{Y}}=\mathbf{C}\mathbf{A}\widehat{\mathbf{S}}+\widehat{\mathbf{N}}\end{array}$$

(4)

where $\widehat{\mathbf{S}}=\mathbf{S}{\mathbf{V}}_s$ and $\widehat{\mathbf{N}}=\mathbf{N}{\mathbf{V}}_s$. In order to cast the problem of DOA estimation with unknown mutual coupling as a sparse representation problem, we let Ω denote the set of possible locations and let $\widehat{\theta }$ be a generic location parameter. Furthermore, let ${\left\{{\widehat{\theta }}_i\right\}}_{i=1}^{K_{\widehat{\theta }}}$ denote a grid that covers Ω. If the grid is fine enough, such that the true DOAs lie on (or, practically, close to) the grid, we can use the following model for $\widehat{\mathbf{Y}}$:

$$\begin{array}{c}\hfill \widehat{\mathbf{Y}}=\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}{\widehat{\mathbf{S}}}_{\widehat{\theta }}+\widehat{\mathbf{N}}\end{array}$$

(5)

where ${\mathbf{A}}_{\widehat{\theta }}\triangleq [\mathbf{\alpha }\left({\widehat{\theta }}_1\right),\mathbf{\alpha }\left({\widehat{\theta }}_2\right),\dots ,\mathbf{\alpha }\left({\widehat{\theta }}_{K_{\widehat{\theta }}}\right)]$ and ${\widehat{\mathbf{S}}}_{\widehat{\theta }}$ is a $K_{\widehat{\theta }}\times \widehat{K}$ complex matrix whose i-th row corresponds to the signal impinging on the array from a possible source at ${\widehat{\theta }}_i$. It is easy to verify that the i-th row is nonzero and equals the k-th row of $\widehat{\mathbf{S}}$ if signal k comes from ${\widehat{\theta }}_i$ for some k and zero otherwise. As a result, the aim of this paper is to find a row-sparse ${\widehat{\mathbf{S}}}_{\widehat{\theta }}$ (in other words, with a few nonzero rows) and a Toeplitz matrix $\mathbf{C}$ that minimize the following objective function:

$$\begin{array}{c}\hfill \parallel \widehat{\mathbf{Y}}-\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}{\widehat{\mathbf{S}}}_{\widehat{\theta }}{\parallel }_2\end{array}$$

(6)

where ${\parallel ·\parallel }_2$ stands for the Frobenius norm. Finding the sparse solution to the above problem through the $l_1$-norm regularized optimization is intractable, as it is a non-convex optimization problem with unknown $\mathbf{C}$, which cannot be solved in polynomial time. In the next section, we will propose an SBL method for the DOA estimation in the presence of unknown coupling. To this end, we have to preliminarily model the noisy and sparse signals as in [19].

2.2. Noise Model

Firstly, we address the noise model that is commonly used in SBL. Assume elements in the noise vector $\mathbf{n}$ defined in Equation (1) are independent and each has a complex Gaussian distribution with zero mean and a common variance ${\sigma }^2$. Since the orthogonal invariance property of the Gaussian random matrix makes the distribution impervious to multiplication by orthogonal matrices, each element in $\widehat{\mathbf{N}}$ is approximately i.i.d. complex Gaussian with the same mean and variance [15,28], i.e., $p\left({\widehat{N}}_{i,j}\right)=\mathcal{CN}\left({\widehat{N}}_{i,j}\right|0,{\beta }^{-1})$, where $\beta ={\sigma }^{-2}$ denotes the noise precision and ${\widehat{N}}_{i,j}$ is the $(i,j)$-th element in $\widehat{\mathbf{N}}$. Then, we have:

$$\begin{array}{c}\hfill p\left(\widehat{\mathbf{Y}}\right|{\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ;\mathbf{C})=\mathbf{\prod }_{k=1}^{\widehat{K}}\mathcal{CN}\left({\widehat{\mathbf{y}}}_k\right|\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}{\widehat{\mathbf{s}}}_k,{\beta }^{-1}\mathbf{I})\end{array}$$

(7)

where ${\widehat{\mathbf{y}}}_k$ and ${\widehat{\mathbf{s}}}_k$ denote the k-th columns of $\widehat{\mathbf{Y}}$ and ${\widehat{\mathbf{S}}}_{\widehat{\theta }}$, respectively. Usually, the noise variance ${\sigma }^2$ is unknown, so is the noise precision β. Hence, we model β as a Gamma hyperprior:

$$\begin{array}{c}\hfill p\left(\beta \right)=\Gamma (\beta ;a,b)\end{array}$$

(8)

where we set $a,b\to 0$ as in [19,21] so as to obtain a broad hyperprior. The reason why we choose the Gamma hyperprior is that it is a conjugate prior (in Bayesian probability theory, a prior $p\left(\theta \right)$ is said to be conjugate to $p\left(x\right|\theta )$ if the posterior distribution $p\left(\theta \right|x)$ has the same functional form as the prior) of the Gaussian distribution.

2.3. Sparse Signal Model

A widely-used sparseness prior for ${\widehat{\mathbf{S}}}_{\widehat{\theta }}$ is the Laplace distribution; however, it is not readily accomplished with such a prior in SBL, because the Laplace prior is not conjugate to the Gaussian likelihood, and hence, it is unlikely to perform the associated Bayesian inference in closed form [21]. A typical SBL treatment of ${\widehat{\mathbf{S}}}_{\widehat{\theta }}$, proposed in RVM [19], begins by assigning a non-stationary Gaussian prior distribution with a distinct inverse variance ${\delta }_i$ for each row of ${\widehat{\mathbf{S}}}_{\widehat{\theta }}$. Letting $\mathit{\delta }={[{\delta }_1,{\delta }_2,\dots ,{\delta }_{K_{\widehat{\theta }}}]}^T$ and $\mathbf{\Delta }=\mathrm{diag}\left(\mathit{\delta }\right)$, we have:

$$\begin{array}{c}\hfill p\left({\widehat{\mathbf{S}}}_{\widehat{\theta }}\right|\mathit{\delta })=\mathbf{\prod }_{k=1}^{\widehat{K}}\mathcal{CN}\left({\widehat{\mathbf{s}}}_k\right|\mathbf{0},{\mathbf{\Delta }}^{-1})\end{array}$$

(9)

In order to make the Bayesian inference convenient and to obtain a two-stage hierarchical prior that favors most rows of ${\widehat{\mathbf{S}}}_{\widehat{\theta }}$ being zeros, the hyper-parameter ${\delta }_i$’s are further modeled as independent Gamma distributions [19,21], i.e.,

$$\begin{array}{c}\hfill p\left(\mathit{\delta }\right)=\mathbf{\prod }_{i=1}^{K_{\widehat{\theta }}}\Gamma ({\delta }_i;c,d)\end{array}$$

(10)

where $c\to 1$ and $d\to 0$. In this case, the integral ${\int }_0^{\infty }\mathcal{CN}\left({\widehat{\mathbf{s}}}_k\right|\mathbf{0},{\mathbf{\Delta }}^{-1})·p(\mathit{\delta };c,d)d\delta$ corresponds to the Student t distribution [19], which can be evaluated analytically. Alternatively, a two-stage hierarchical method that results in Laplace priors was addressed in [21].

3. The Proposed Sparse Bayesian Learning Method

The associated learning problem becomes the search for the hyperparameters β and δ, as well as the parameter $\mathbf{C}$. As $p({\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ,\mathit{\delta }|\widehat{\mathbf{Y}};\mathbf{C})$ cannot be explicitly calculated, a Type-II ML [19] (or evidence maximization) procedure is exploited to perform the Bayesian inference. In other words, the hyperparameters (β and δ) and parameter ($\mathbf{C}$) are estimated by maximizing $p\left(\widehat{\mathbf{Y}}\right|\beta ,\mathit{\delta };\mathbf{C})$ or its logarithm $\mathcal{L}(\beta ,\mathit{\delta };\mathbf{C})\triangleq lnp\left(\widehat{\mathbf{Y}}\right|\beta ,\mathit{\delta };\mathbf{C})$. However, explicitly finding the values of β, δ and $\mathbf{C}$ that maximize $\mathcal{L}(\beta ,\mathit{\delta };\mathbf{C})$ is intractable, and here, we adopt an EM algorithm that treats ${\widehat{\mathbf{S}}}_{\widehat{\theta }}$ as a hidden variable. The principle behind the EM algorithm is to instead repeatedly construct a lower bound on $\mathcal{L}(\beta ,\mathit{\delta };\mathbf{C})$ (E-step) and then to optimize that lower-bound (M-step). Specifically,

• E-step: Compute:

  $$p\left({\widehat{\mathbf{S}}}_{\widehat{\theta }}\right|\widehat{\mathbf{Y}},\beta ,\mathit{\delta };\mathbf{C})$$

  (11)

  and evaluate:

  $${〈lnp(\widehat{\mathbf{Y}},{\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ,\mathit{\delta };\mathbf{C})〉}_{E^{\left(t\right)}}\triangleq E_{p\left({\widehat{\mathbf{S}}}_{\widehat{\theta }}\right|\widehat{\mathbf{Y}},{\beta }^{\left(t\right)},{\mathit{\delta }}^{\left(t\right)};{\mathbf{C}}^{\left(t\right)})}\left[lnp(\widehat{\mathbf{Y}},{\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ,\mathit{\delta };\mathbf{C})\right]$$

  (12)

  where ${(·)}^{\left(t\right)}$ denotes the estimated value in the t-th iteration.
• M-step: Find the hyperparameter updates and the parameter updates:

  $$\begin{array}{c}\hfill \{{\beta }^{(t+1)},{\mathit{\delta }}^{(t+1)},{\mathbf{C}}^{(t+1)}\}=\arg \underset{\beta ,\mathit{\delta },\mathbf{C}}{max}{〈lnp(\widehat{\mathbf{Y}},{\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ,\mathit{\delta };\mathbf{C})〉}_{E^{\left(t\right)}}.\end{array}$$

  (13)

In the rest of this section, we will discuss the two steps in detail.

3.1. E-Step

Using the Bayes rule, we can verify that the posterior distribution of ${\widehat{\mathbf{S}}}_{\widehat{\theta }}$ in Equation (11) is also a complex Gaussian [19]:

$$p({\widehat{\mathbf{S}}}_{\widehat{\theta }}|\widehat{\mathbf{Y}},\beta ,\mathit{\delta };\mathbf{C})=\frac{p\left(\widehat{\mathbf{Y}}\right|{\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ;\mathbf{C})·p\left({\widehat{\mathbf{S}}}_{\widehat{\theta }}\right|\mathit{\delta })}{p\left(\widehat{\mathbf{Y}}\right|\beta ,\mathit{\delta };\mathbf{C})}=\mathbf{\prod }_{k=1}^{\widehat{K}}\mathcal{CN}\left({\widehat{\mathbf{s}}}_k\right|{\mathit{\mu }}_k,\mathbf{\Sigma })$$

(14)

where:

$${\mathit{\mu }}_k=\beta \mathbf{\Sigma }{\mathbf{A}}_{\widehat{\theta }}^H{\mathbf{C}}^H{\widehat{\mathbf{y}}}_k,k=1,2,\dots ,\widehat{K}$$

(15)

$$\mathbf{\Sigma }={(\beta {\mathbf{A}}_{\widehat{\theta }}^H{\mathbf{C}}^H\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}+\mathbf{\Delta })}^{-1}={\mathbf{\Delta }}^{-1}-{\mathbf{\Delta }}^{-1}{\left(\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}\right)}^H{({\beta }^{-1}\mathbf{I}+\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}{\mathbf{\Delta }}^{-1}{\left(\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}\right)}^H)}^{-1}\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}{\mathbf{\Delta }}^{-1}.$$

(16)

On the other hand, by combining the stages of the hierarchical Bayesian model, $p(\widehat{\mathbf{Y}},{\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ,\mathit{\delta };\mathbf{C})$ can be rewritten as:

$$p(\widehat{\mathbf{Y}},{\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ,\mathit{\delta };\mathbf{C})=p(\widehat{\mathbf{Y}}|{\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ;\mathbf{C})p({\widehat{\mathbf{S}}}_{\widehat{\theta }}|\mathit{\delta })p\left(\beta \right)p\left(\mathit{\delta }\right).$$

(17)

Therefore, Equation (12) leads to:

$${〈lnp(\widehat{\mathbf{Y}},{\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ,\mathit{\delta };\mathbf{C})〉}_{E^{\left(t\right)}}={〈lnp(\widehat{\mathbf{Y}}|{\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ;\mathbf{C})p\left({\widehat{\mathbf{S}}}_{\widehat{\theta }}\right|\mathit{\delta })p\left(\beta \right)p\left(\mathit{\delta }\right)〉}_{E^{\left(t\right)}}.$$

(18)

3.2. M-Step

In this subsection, we will address the derivation of hyperparameter updates (${\beta }^{(t+1)}$ and ${\mathit{\delta }}^{(t+1)}$), as well as the parameter update (${\mathbf{C}}^{(t+1)}$).

• For β, ignoring terms in the logarithm independent thereof, we just have to maximize:

  $$\begin{array}{cc}& {〈lnp(\widehat{\mathbf{Y}}|{\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ;\mathbf{C})p\left(\beta \right)〉}_{E^{\left(t\right)}}\hfill \end{array}$$

  $$\begin{array}{cc}\hfill =& {〈lnp(\widehat{\mathbf{Y}}|{\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ;\mathbf{C})〉}_{E^{\left(t\right)}}+lnp\left(\beta \right)\hfill \end{array}$$

  $$\begin{array}{cc}\hfill =& \widehat{K}Mln\beta -\beta \sum _{k=1}^{\widehat{K}}{〈\parallel {\widehat{\mathbf{y}}}_k-\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}{\widehat{\mathbf{s}}}_k{\parallel }_2^2〉}_{E^{\left(t\right)}}+(a-1)ln\beta -b\beta +\mathrm{const}\hfill \end{array}$$

  $$\begin{array}{cc}\hfill =& \widehat{K}Mln\beta -\beta \sum _{k=1}^{\widehat{K}}{\parallel {\widehat{\mathbf{y}}}_k-{\mathbf{C}}^{\left(t\right)}{\mathbf{A}}_{\widehat{\theta }}{\mathit{\mu }}_k^{\left(t\right)}\parallel }_2^2\hfill \end{array}$$

  $$\begin{array}{cc}& -\beta \widehat{K}\mathrm{tr}\left({\mathbf{C}}^{\left(t\right)}{\mathbf{A}}_{\widehat{\theta }}{\mathbf{\Sigma }}^{\left(t\right)}{\left({\mathbf{C}}^{\left(t\right)}{\mathbf{A}}_{\widehat{\theta }}\right)}^H\right)+(a-1)ln\beta -b\beta +\mathrm{const}\hfill \end{array}$$

  (19)

  which, through differentiation, gives the update for ${\beta }^{(t+1)}$:

  $${\beta }^{(t+1)}=\frac{\widehat{K}M+(a-1)}{\left(\begin{array}{c}b+{\sum }_{k=1}^{\widehat{K}}{\parallel {\widehat{\mathbf{y}}}_k-{\mathbf{C}}^{\left(t\right)}{\mathbf{A}}_{\widehat{\theta }}{\mathit{\mu }}_k^{\left(t\right)}\parallel }_2^2\hfill \\ +\widehat{K}\mathrm{tr}\left({\mathbf{C}}^{\left(t\right)}{\mathbf{A}}_{\widehat{\theta }}{\mathbf{\Sigma }}^{\left(t\right)}{\left({\mathbf{C}}^{\left(t\right)}{\mathbf{A}}_{\widehat{\theta }}\right)}^H\right)\hfill \end{array}\right)}.$$

  (20)
• For δ, also ignoring terms in the logarithm independent thereof, we have:

  $$\begin{array}{cc}& {〈lnp({\widehat{\mathbf{S}}}_{\widehat{\theta }}|\mathit{\delta })p\left(\mathit{\delta }\right)〉}_{E^{\left(t\right)}}\hfill \end{array}$$

  $$\begin{array}{cc}\hfill =& {〈lnp({\widehat{\mathbf{S}}}_{\widehat{\theta }}|\mathit{\delta })〉}_{E^{\left(t\right)}}+lnp\left(\mathit{\delta }\right)\hfill \end{array}$$

  $$\begin{array}{cc}\hfill =& \widehat{K}ln|\mathbf{\Delta }|-\sum _{k=1}^{\widehat{K}}{〈{\widehat{\mathbf{s}}}_k^H\mathbf{\Delta }{\widehat{\mathbf{s}}}_k〉}_{E^{\left(t\right)}}-d\sum _{i=1}^{K_{\widehat{\theta }}}{\delta }_i+(c-1)\sum _{i=1}^{K_{\widehat{\theta }}}ln{\delta }_i+\mathrm{const}\hfill \end{array}$$

  $$\begin{array}{cc}\hfill =& (\widehat{K}+c-1)\sum _{i=1}^{K_{\widehat{\theta }}}ln{\delta }_i-d\sum _i^{K_{\widehat{\theta }}}{\delta }_i-\sum _{k=1}^{\widehat{K}}\mathrm{tr}\left(\left({\mathit{\mu }}_k^{\left(t\right)}{\left({\mathit{\mu }}_k^{\left(t\right)}\right)}^H+{\mathbf{\Sigma }}^{\left(t\right)}\right)\mathbf{\Delta }\right)+\mathrm{const}.\hfill \end{array}$$

  (21)

  Setting the derivative of Equation (21), with respect to δ, to zero and solving for each ${\delta }_i$ gives the updates:

  $$\begin{array}{c}\hfill {\delta }_i^{(t+1)}=\frac{\widehat{K}+c-1}{d+{\sum }_{k=1}^{\widehat{K}}{\left[{\mathbf{\Xi }}_k^{\left(t\right)}\right]}_{ii}},i=1,2,\dots ,K_{\widehat{\theta }}\end{array}$$

  (22)

  where ${\mathbf{\Xi }}_k^{\left(t\right)}\triangleq {\mathit{\mu }}_k^{\left(t\right)}{({\mathit{\mu }}_k^{\left(t\right)})}^H+{\mathbf{\Sigma }}^{\left(t\right)}$.
• For $\mathbf{C}$, its estimate should maximize the following expected value:

  $$\begin{array}{cc}& {〈lnp(\widehat{\mathbf{Y}}|{\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ;\mathbf{C})〉}_{E^{\left(t\right)}}=-\beta \sum _{k=1}^{\widehat{K}}{\parallel {\widehat{\mathbf{y}}}_k-\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}{\mathit{\mu }}_k^{\left(t\right)}\parallel }_2^2-\beta \widehat{K}\mathrm{tr}\left(\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}{\mathbf{\Sigma }}^{\left(t\right)}{\left(\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}\right)}^H\right)+\mathrm{const}.\hfill \end{array}$$

  (23)

  In order to calculate the derivative of Equation (23), we need the following lemma.

Lemma 1 (see [2]). Let $\mathbf{C}$ and $\mathbf{c}$ be defined as earlier, then for any vector $\mathbf{x}$, we have:

$$\begin{array}{c}\hfill \mathbf{C}\mathbf{x}=\mathbf{T}\left(\mathbf{x}\right)\mathbf{c}\end{array}$$

(24)

where the matrix $\mathbf{T}\left(\mathbf{x}\right)\in {\mathbb{C}}^{M\times (m+1)}$ is the sum of the two $M\times (m+1)$ matrices:

$$\begin{array}{c}{\left[{\mathbf{T}}_1\left(\mathbf{x}\right)\right]}_{p,q}=\left\{\begin{array}{cc}{\left[\mathbf{x}\right]}_{p+q-1}\hfill & p+q\le M+1\hfill \\ 0\hfill & otherwise\hfill \end{array}\right.\hfill \end{array}$$

(25)

$$\begin{array}{c}{\left[{\mathbf{T}}_2\left(\mathbf{x}\right)\right]}_{p,q}=\left\{\begin{array}{cc}{\left[\mathbf{x}\right]}_{p-q+1}\hfill & p\ge q\ge 2\hfill \\ 0\hfill & otherwise\hfill \end{array}\right.\hfill \end{array}$$

(26)

With the assistance of Lemma 1, we are capable of rewriting the terms $\parallel {\widehat{\mathbf{y}}}_k-\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}{\mathit{\mu }}_k^{\left(t\right)}{\parallel }_2^2$ and $\mathrm{tr}\left(\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}{\mathbf{\Sigma }}^{\left(t\right)}{(\mathbf{C}{\mathbf{A}}_{\widehat{\theta }})}^H\right)$ in Equation (23) as:

$$\begin{array}{c}\hfill \parallel {\widehat{\mathbf{y}}}_k-\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}{\mathit{\mu }}_k^{\left(t\right)}{\parallel }_2^2={\parallel {\widehat{\mathbf{y}}}_k-\mathbf{T}\left({\mathbf{A}}_{\widehat{\theta }}{\mathit{\mu }}_k^{\left(t\right)}\right)\mathbf{c}\parallel }_2^2\end{array}$$

(27)

and:

$$\begin{array}{c}\hfill \mathrm{tr}\left(\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}{\mathbf{\Sigma }}^{\left(t\right)}{\left(\mathbf{C}{\mathbf{A}}_{\widehat{\theta }}\right)}^H\right)=\parallel \mathbf{C}{\mathbf{D}}^{\left(t\right)}{\parallel }_2^2=\sum _{i=1}^M\parallel \mathbf{C}{\mathbf{d}}_i^{\left(t\right)}{\parallel }_2^2=\sum _{i=1}^M{\parallel \mathbf{T}\left({\mathbf{d}}_i^{\left(t\right)}\right)\mathbf{c}\parallel }_2^2\end{array}$$

(28)

respectively, where ${\mathbf{D}}^{\left(t\right)}{({\mathbf{D}}^{\left(t\right)})}^H$ stands for a decomposition of ${\mathbf{A}}_{\widehat{\theta }}{\mathbf{\Sigma }}^{\left(t\right)}{\mathbf{A}}_{\widehat{\theta }}^H$ and ${\mathbf{d}}_i^{\left(t\right)}$ denotes the i-th column of ${\mathbf{D}}^{\left(t\right)}$. Using Equations (27) and (28), we can calculate the derivative of ${〈lnp(\widehat{\mathbf{Y}}|{\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ;\mathbf{C})〉}_{p^{\left(t\right)}}$, with respect to $\mathbf{c}$, as:

$$\begin{array}{cc}& \frac{\partial }{\partial \mathbf{c}}{〈lnp\left(\widehat{\mathbf{Y}}\right|{\widehat{\mathbf{S}}}_{\widehat{\theta }},\beta ;\mathbf{C})〉}_{p^{\left(t\right)}}\hfill \end{array}$$

$$\begin{array}{cc}\hfill =& \beta \sum _{k=1}^{\widehat{K}}\mathbf{T}{\left({\mathbf{A}}_{\widehat{\theta }}{\mathit{\mu }}_k^{\left(t\right)}\right)}^H\left({\widehat{\mathbf{y}}}_k-\mathbf{T}\left({\mathbf{A}}_{\widehat{\theta }}{\mathit{\mu }}_k^{\left(t\right)}\right)\mathbf{c}\right)-\beta \widehat{K}\sum _{i=1}^M\mathbf{T}{\left({\mathbf{d}}_i^{\left(t\right)}\right)}^H\mathbf{T}\left({\mathbf{d}}_i^{\left(t\right)}\right)\mathbf{c}.\hfill \end{array}$$

(29)

Setting the derivative to zero gives:

$$\begin{array}{cc}\hfill \mathbf{c}=& \left(\sum _{k=1}^{\widehat{K}}\mathbf{T}{\left({\mathbf{A}}_{\widehat{\theta }}{\mathit{\mu }}_k^{\left(t\right)}\right)}^H\mathbf{T}\left({\mathbf{A}}_{\widehat{\theta }}{\mathit{\mu }}_k^{\left(t\right)}\right)\right.{\left.+\widehat{K}\sum _{i=1}^M\mathbf{T}{\left({\mathbf{d}}_i^{\left(t\right)}\right)}^H\mathbf{T}\left({\mathbf{d}}_i^{\left(t\right)}\right)\right)}^{-1}·\sum _{k=1}^{\widehat{K}}\mathbf{T}{\left({\mathbf{A}}_{\widehat{\theta }}{\mathit{\mu }}_k^{\left(t\right)}\right)}^H{\widehat{\mathbf{y}}}_k.\hfill \end{array}$$

(30)

Remark 1. Note that [9,10,11] provided an alternative parameterization for the steering vector:

$$\begin{array}{c}\hfill \mathbf{C}\mathbf{\alpha }\left({\theta }_k\right)=g({\theta }_k,\mathbf{c})\overline{\mathbf{T}}\left({\theta }_k\right)\mathit{\tau }({\theta }_k,\mathbf{c})\end{array}$$

(31)

where $g({\theta }_k,\mathbf{c})\triangleq {\sum }_{m^{\prime }=-m}^m{c}_{|m^{\prime }|}{e}^{j{m}^{\prime }\varphi \left({\theta }_k\right)}$, $\overline{\mathbf{T}}\left({\theta }_k\right)\triangleq \mathrm{blkdiag}\{{\overline{\mathbf{T}}}_1\left({\theta }_k\right),{\overline{\mathbf{T}}}_2\left({\theta }_k\right),{\overline{\mathbf{T}}}_3\left({\theta }_k\right)\}$ is a block diagonal matrix with ${\overline{\mathbf{T}}}_1\left({\theta }_k\right)=\mathrm{diag}\{1,e^{j\varphi \left({\theta }_k\right)},\dots ,e^{j(m-1)\varphi \left({\theta }_k\right)}\}$, ${\overline{\mathbf{T}}}_2\left({\theta }_k\right)={[e^{jm\varphi \left({\theta }_k\right)},\dots ,e^{j(M-m-1)\varphi \left({\theta }_k\right)}]}^T$ and ${\overline{\mathbf{T}}}_3\left({\theta }_k\right)=\mathrm{diag}\{e^{j(M-m)\varphi \left({\theta }_k\right)},\dots ,e^{j(M-1)\varphi \left({\theta }_k\right)}\}$ and $\mathit{\tau }({\theta }_k,\mathbf{c})\triangleq [{\nu }_1,\dots ,{\nu }_m,1,{\eta }_1,\dots ,{\eta }_m]$ with ${\nu }_k=\frac{1}{g({\theta }_k,\mathbf{c})}\left({\sum }_{m^{\prime }=1-k}^m{c}_{|m^{\prime }|}{e}^{j{m}^{\prime }\varphi \left({\theta }_k\right)}\right)$ and ${\eta }_k=\frac{1}{g({\theta }_k,\mathbf{c})}\left({\sum }_{m^{\prime }=-m}^{m-k}{c}_{|m^{\prime }|}{e}^{j{m}^{\prime }\varphi \left({\theta }_k\right)}\right)$. However, it is unlikely that Equation (31) can be applied to the M-step for $\mathbf{C}$. The most difficult aspect for applying the alternative parameterization in Equation (31) is that it is dependent on the geometric progression in $\mathbf{\alpha }\left({\theta }_k\right)$; while the terms ${\mathbf{A}}_{\widehat{\theta }}{\mathit{\mu }}_k^{\left(t\right)}$ and ${\mathbf{d}}_i^{\left(t\right)}$ in Equations (27) and (28) do not have the property of geometric progression.

The EM algorithm proceeds by repeated application of Equations (20), (22) and (30), until a prescribed accuracy is achieved. Once the algorithm is convergent, we are able to obtain the calibrated $\mathbf{C}$. As far as the estimated ${\delta }_i$’s are concerned, we observed that many of ${\delta }_i$ tend to infinity in the EM process. Clearly, the rows corresponding to these infinity ${\delta }_i$’s are zero; while the rows corresponding to small ${\delta }_i$’s imply the true DOAs.

4. Simulation Results

In this section, we will present several simulation results to illustrate the performance of our proposed method. We will compare the proposed method to the original SBL method in [24], the iterative method in [3] and the auxiliary methods in [5,12].

Simulation 1 addresses the performance comparisons of the joint estimation of DOAs and mutual coupling coefficients. Consider a scenario where a ULA composed of $M=10$ sensors is used to receive $K=2$ uncorrelated signals coming from ${\theta }_1$ = –19.7 °C and ${\theta }_2$ = 10.1 °C. Note that the effect of mutual coupling is negligible between two sensors that are far enough away from each other, because the mutual coupling coefficient is inversely proportional to their distance. Hence, it is reasonable to approximate the mutual coupling effect with just a few nonzero coefficients. In the simulation, we assume that the number of mutual coupling coefficients is $m=2$ with $c_1=0.5-0.4i$ and $c_2=0.3+0.1i$. Figure 1 shows the root mean square error (RMSE) of DOA estimation versus input SNR computed via 200 Monte Carlo runs, where the number of snapshots is 100. For the ease of comparison, we also include the curve for CRB. As can be seen from the figure, our method outperforms the state-of-the-art methods. This is because: (1) compared to the method in [24], our method utilizes a hierarchical form of the Student t prior to enforce the sparsity of unknown signal more heavily; moreover, our method uses the SVD to reduce the sensitivity to the noise; (2) compared to the methods in [5,12], our method uses the whole array, rather than a subarray, to estimate DOAs and to compensate for the mutual coupling effect.

Figure 2 shows the estimation bias and variance for each DOA. Compared to the $l_1$-norm regularized method [12], our method can significantly reduce the DOA estimation bias, as well as the variance. To assess the performance of mutual coupling calibration, Figure 3 shows the relative RMSE of the estimation of the mutual coupling coefficients. This relative RMSE is defined as:

$$\begin{array}{c}\hfill \sqrt{\frac{1}{200}\sum _{n=1}^{200}\sum _{i=1}^m\frac{|c_{ni}-c_i{|}^2}{|c_i{|}^2}}\times 100\%\end{array}$$

(32)

where $c_{ni}$ is the estimate of $c_i$ at the n-th Monte Carlo run. As shown from the Figure 3, the mutual coupling coefficients can be estimated more accurately in our method, especially for low SNR signals. Hence, if all of the methods embed the mutual coupling coefficients within the same classical eigendecomposition algorithm, such as MUSIC and ESPRIT, to estimate the DOA, our method can achieve the best performance.

Figure 1. RMSE of the DOA estimate against SNR.

Figure 1. RMSE of the DOA estimate against SNR.

Figure 2. Estimation bias and variance against SNR. (a) Bias; (b) Variance.

Figure 2. Estimation bias and variance against SNR. (a) Bias; (b) Variance.

Figure 3. RMSE of mutual coupling coefficients against SNR.

Figure 3. RMSE of mutual coupling coefficients against SNR.

Simulation 2 is used to investigate the capability of resolving closely-spaced sources with limited snapshots ($T=50$) for several methods. The simulation considers a scenario where a ULA composed of $M=13$ sensors is used to receive $K=2$ uncorrelated signals coming from ${\theta }_1$ = –2.5 °C and ${\theta }_2$ = 3.5 °C, and the number of mutual coupling coefficients is $m=2$ with $c_1=0.5-0.4i$ and $c_2=0.3+0.1i$. We say that the two signals are exactly resolved in a given run, if ${max}_{k=1,2}\left\{\right|{\tilde{\theta }}_k-{\theta }_k\left|\right\}$ is smaller than $|{\theta }_1-{\theta }_2|/2$, where ${\tilde{\theta }}_k$ stands for the estimated DOA for the k-th signal. As can be seen from Figure 4, the resolution performance of our method outperforms others. The superior resolution performance of sparse representation is natural, which is consistent with the simulation results in [12].

Figure 4. Resolution probability against SNR for closely-spaced sources.

Figure 4. Resolution probability against SNR for closely-spaced sources.

Simulation 3 addresses the “blind angle” phenomenon, which may result in the substantially degraded performance of DOA estimation [4,5]. We consider a scenario where a ULA composed of $M=12$ sensors is used to receive $K=4$ uncorrelated signals coming from ${\theta }_1$ = 10 °C, ${\theta }_2$ = 20 °C, ${\theta }_3$ = 30 °C and ${\theta }_4$ = 40 °C, and the number of mutual coupling coefficients is $m=2$ with $c_1=0.6+0.5i$ and $c_2=0.3844-0.3476i$. It is easy to verify that the blind angle occurs at ${\theta }_4$ = 40 °C. The number of snapshots is 30, and the SNR is 10 dB. Figure 5 illustrates that: (1) our method can estimate all of the true DOAs accurately, including the blind DOA ${\theta }_4$ = 40 °C; (2) the $l_1$-norm regularized method [12] fails in estimating the blind DOA from ${\theta }_4$ = 40 °C, but succeeds in obtaining the DOAs from other directions; (3) the iterative method [3] misses two DOAs ${\theta }_3$ = 30 °C and ${\theta }_4$ = 40 °C. Obviously, our method has the best performance of “blind angle” suppression.

Figure 5. DOA estimation in the case of a “blind angle”.

Figure 5. DOA estimation in the case of a “blind angle”.

5. Conclusions

We have proposed a modified SBL method that approaches the problem of DOA estimation with unknown mutual coupling. Unlike the original SBL method in [24] that only uses stationary priors, our new method utilized a hierarchical form of the Student t prior to enforce the sparsity of unknown signal more heavily. To efficiently perform the Bayesian inference, we adopted a refined EM algorithm that treats ${\widehat{\mathbf{S}}}_{\widehat{\theta }}$ and $\mathbf{C}$ as a hidden variable and a parameter, respectively. It can update the mutual coupling coefficients more efficiently. Another difference is that our method used an additional SVD to reduce the computational complexity of the signal reconstruction process and the sensitivity to the measurement noise. Hence, the proposed method is expected to give a better estimation performance. Simulation results have verified its efficiency.