Sparse Method for Direction of Arrival Estimation Using Denoised Fourth-Order Cumulants Vector

Abstract

Fourth-order cumulants (FOCs) vector-based direction of arrival (DOA) estimation methods of non-Gaussian sources may suffer from poor performance for limited snapshots or difficulty in setting parameters. In this paper, a novel FOCs vector-based sparse DOA estimation method is proposed. Firstly, by utilizing the concept of a fourth-order difference co-array (FODCA), an advanced FOCs vector denoising or dimension reduction procedure is presented for arbitrary array geometries. Then, a novel single measurement vector (SMV) model is established by the denoised FOCs vector, and efficiently solved by an off-grid sparse Bayesian inference (OGSBI) method. The estimation errors of FOCs are integrated in the SMV model, and are approximately estimated in a simple way. A necessary condition regarding the number of identifiable sources of our method is presented that, in order to uniquely identify all sources, the number of sources K must fulfill $K\le (M^4-2M^3+7M^2-6M)/8$. The proposed method suits any geometry, does not need prior knowledge of the number of sources, is insensitive to associated parameters, and has maximum identifiability $O(M^4)$, where M is the number of sensors in the array. Numerical simulations illustrate the superior performance of the proposed method.

1. Introduction

Direction of arrival (DOA) estimation is a topic that has received a great deal of attention in the last several decades. It arises in many practical scenarios, such as radar, sonar, and communications [1,2,3,4]. For example, in radar and sonar systems, objects including airplanes, birds, missiles, submarines, and fish torpedoes can be tracked by estimating their DOAs. In communication systems, the DOAs of the sources can be used to improve the communication quality.

Higher-order cumulants (HOCs)-based DOA estimation methods have been developed for non-Gaussian sources [1,5,6,7,8], mainly to overcome the limitations of second-order statistics-based DOA estimation methods such as MUltiple SIgnal Classification(MUSIC) [9] and Estimation of Signal Parameters via Rotational Invariance Techniques (ESPRIT) [10]. Their limitations include identifiable number of sources, colored Gaussian noise, and modeling errors. The famous 4-MUSIC method [5] uses the fourth-order cumulants (FOCs) to form a MUSIC-like method. Then, the $2q$-MUSIC [6] and rectangular $2q$-MUSIC [7] methods extend 4-MUSIC by utilizing $2q$th-order cumulants. The rectangular $2q$-MUSIC method achieves a trade-off between performance and maximal identifiable number of sources compared with the 2q-MUSIC method. To further increase the degrees of freedom (DOFs) of the FOCs-based methods, a kind of sparse linear array named multiple level nested array and the corresponding spatial smoothing MUSIC (SS-MUSIC) method were developed, which make the DOFs increase from $O(M^q)$ to $O(M^{2q})$ [8], where M is the number of sensors in the array. However, HOCs-based methods always suffer from performance degradation when snapshots are limited, mainly because accurate estimations of HOCs require a large number of snapshots.

Recently, a series of covariance vector-based DOA methods were proposed [11,12,13] which built a single measurement vector (SMV) model by covariance vector and were solved by sparse methods such as L1-norm-based methods [14] and sparse Bayesian learning (SBL) [15]. Consequently, these methods always perform better than other covariance-based DOA methods such as MUSIC when snapshots are limited, mainly because estimation errors of covariance are utilized to build their sparse models. Moreover, the DOFs of these methods increase to $O(M^2)$, which is more than MUSIC. Analogously, the FOCs vector-based L1-norm method (4-L1) is applied in [16,17,18]. Compared with FOCs vector-based SS-MUSIC (4-SS-MUSIC) in [8], the 4-L1 method can utilize all the virtual sensors in the fourth-order difference co-array (FODCA) [8], which may give a better performance than 4-SS-MUSIC. Moreover, the 4-L1 method can be applied to any array, while 4-SS-MUSIC only suits for linear arrays. Nonetheless, the 4-L1 method is inconvenient to use in practice, since the parameter of allowable error bound is difficult to choose, and the solution of 4-L1 is sensitive to it. The allowable error bound usually needs to be chosen manually through trial-and-error for each scenario, as in [16,17,18]. Furthermore, it has been demonstrated both theoretically and empirically that the SBL technique induces less structural error (biased global minimum) and convergence error (failure in achieving the global minimum) than the L1 methods [11,19,20,21]. On the other hand, an advanced denoising procedure of the FOCs vector may further promote the performance of FOCs vector-based DOA estimation methods. An analogous procedure can be found in covariance vector-based DOA methods in [22].

In this paper, we establish a novel denoised SMV model by FOCs vector, and solve it by an off-grid sparse Bayesian inference (OGSBI) method [21]. By utilizing the concept of the fourth-order difference co-array, the advanced denoising or dimension reduction procedure of FOCs vector is presented for any geometry. The estimation errors of FOCs are integrated in the proposed SMV model, and are approximately estimated in a simple way. A necessary condition of our method regarding the number of identifiable sources is presented such that in order to uniquely identify all sources, the number of sources K must fulfill $K\le (M^4-2M^3+7M^2-6M)/8$. The proposed method suits for any geometry, does not require prior knowledge of the number of sources, has maximum identifiability $O(M^4)$, and is insensitive to associated parameters. The off-grid parameter estimation in our method promotes superior performance in overdetermined cases when the number of snapshots is large and signal–noise-ratio (SNR) is high, and also ensures good performance when choosing a relatively coarse grid. Numerical simulations illustrate the superior performance of the proposed method.

2. Data Model

One-dimensional DOA estimation is discussed in this paper. Consider an array with any geometry which consists of M identical, omnidirectional, unpolarized and isotropic sensors. The sensor locations ${\mathbf{n}}_1,{\mathbf{n}}_2,\dots ,{\mathbf{n}}_M$ are measured by $d=\lambda /2$, and collected by the vector set $\mathbb{S}\triangleq \{{\mathbf{n}}_1,{\mathbf{n}}_2,\dots ,{\mathbf{n}}_M\}\subset {\mathbb{R}}^3$, where ${\mathbf{n}}_m={[n_{m,x},n_{m,y},n_{m,z}]}^{\mathrm{T}}$ is expressed in three dimensions for $m=1,2,\dots ,M$, ${(·)}^{\mathrm{T}}$ is the transpose operator, $\lambda$ is the wavelength, and $\mathbb{R}$ denotes the set of real numbers. The cardinality of $\mathbb{S}$ is denoted by $\left|\mathbb{S}\right|$, and $\left|\mathbb{S}\right|=M$. Each sensor receives the contribution of K zero-mean statistically independent stationary narrowband non-Gaussian sources. We denote by $\mathbf{x}(t)={[x_1(t),x_2(t),\dots ,x_M(t)]}^{\mathrm{T}}$, $\mathbf{s}(t)={[s_1(t),s_2(t),\dots ,s_K(t)]}^{\mathrm{T}}$ and $\mathbf{v}(t)={[v_1(t),v_2(t),\dots ,v_M(t)]}^{\mathrm{T}}$ the observed signals of the sensors, the source signals, and the noises of the observed signals at the tth snapshot, respectively, where $t=1,2,\dots ,+\infty$. The observed signals are modeled as:

$$\begin{array}{c}\hfill \mathbf{x}(t)=\mathbf{As}(t)+\mathbf{v}(t),\end{array}$$

(1)

where $\mathbf{A}=[\mathbf{a}({\theta }_1),\mathbf{a}({\theta }_2),\dots ,\mathbf{a}({\theta }_K)]\in {\mathbb{C}}^{M\times K}$ denotes the array manifold matrix, $\mathbf{a}({\theta }_k)={[{\mathrm{e}}^{\mathrm{j}2\pi {\mathbf{u}}_k^{\mathrm{T}}{\mathbf{n}}_1},{\mathrm{e}}^{\mathrm{j}2\pi {\mathbf{u}}_k^{\mathrm{T}}{\mathbf{n}}_2},\dots ,{\mathrm{e}}^{\mathrm{j}2\pi {\mathbf{u}}_k^{\mathrm{T}}{\mathbf{n}}_M}]}^{\mathrm{T}}$ is the steering vector for ${\theta }_k$, $\mathrm{j}$ is an imaginary number, ${\mathbf{u}}_k=d/\lambda ·{[\cos ({\theta }_k),\sin ({\theta }_k),0]}^{\mathrm{T}}$, ${\theta }_k$ denotes the azimuth angle of the kth source, and $\mathbb{C}$ is the set of complex numbers. The noises are assumed to be statistically independent zero-mean white Gaussian and independent from each source.

Build the FOCs vector ${\mathbf{c}}_x\in {\mathbb{C}}^{M^4}$, where the ith entry is ${[{\mathbf{c}}_x]}_i=\mathrm{cum}\{x_{i_1}(t),x_{i_2}(t),x_{i_3}^{*}(t),x_{i_4}^{*}(t)\}$ with $i=(i_1-1)M^3+(i_2-1)M^2+(i_3-1)M+i_4$, $\mathrm{cum}\{·\}$ denotes the cumulant operator, ${(·)}^{*}$ denotes complex conjugate, and the indexes $i_{\gamma }$ and $\gamma$ are integers satisfying $1\le i_{\gamma }\le M$ and $1\le \gamma \le 4$, respectively. Due to model (1), the above assumptions of the sources and noises, and the properties of the cumulants [23,24], we can obtain

$$\begin{array}{c}\hfill {[{\mathbf{c}}_x]}_i=\sum _{k=1}^K{c}_{s_k}{a}_{i_1}({\theta }_k)a_{i_2}({\theta }_k)a_{i_3}^{*}({\theta }_k)a_{i_4}^{*}({\theta }_k),\end{array}$$

(2)

where $c_{s_k}=\mathrm{cum}\{s_{k_1}(t),s_{k_2}(t),s_{k_3}^{*}(t),s_{k_4}^{*}(t)\}$ with $k_{\gamma }=k$, $a_{i_{\gamma }}({\theta }_k)$ is the $i_{\gamma }$th element of ${\mathbf{a}}_{\gamma }({\theta }_k)$, and ${\mathbf{a}}_{\gamma }({\theta }_k)=\mathbf{a}({\theta }_k)$. Then, we have

$$\begin{array}{c}\hfill {\mathbf{c}}_x={\mathbf{Bc}}_s\in {\mathbb{C}}^{M^4},\end{array}$$

(3)

where ${\mathbf{c}}_s={[c_{s_1},c_{s_2},\dots ,c_{s_K}]}^{\mathrm{T}}$, $\mathbf{B}=[\mathbf{b}({\theta }_1),\mathbf{b}({\theta }_2),\dots ,\mathbf{b}({\theta }_K)]$, $\mathbf{b}({\theta }_k)={\mathbf{a}}_1({\theta }_k)\otimes {\mathbf{a}}_2({\theta }_k)\otimes {\mathbf{a}}_3^{*}({\theta }_k)\otimes {\mathbf{a}}_4^{*}({\theta }_k)$, and ⊗ denotes the Kronecker product.

3. The Proposed Method

In this section, we present an FOCs vector-based method to perform DOA estimation. Firstly, we provide a dimension reduction method to build a novel data model by employing the concept of a fourth-order difference co-array [8]. This method can also be treated as a denoising procedure to reduce the estimation errors of FOCs. Next, according to the newly built data model, our sparse model is efficiently established and solved by the OGSBI method. Some information about the proposed method (e.g., identifiability, complexity, and comparison with the state-of-the-art methods) is also discussed.

3.1. Dimension Reduction or Denoising Procedure

In this subsection, the dimension of model (3) is reduced by utilizing a fourth-order difference co-array. Here we supplement several definitions and properties to facilitate dimension reduction.

Definition 1.

Fourth-order difference co-array (FODCA): Given an array with arbitrary geometry $\mathbb{S}$, the fourth-order difference co-array $\mathbb{D}$ is defined as the set of distinct vector elements from the set

$$\begin{array}{c}\hfill \mathbb{D}=\{{\mathbf{n}}_{i_1}+{\mathbf{n}}_{i_2}-{\mathbf{n}}_{i_3}-{\mathbf{n}}_{i_4}:\forall {\mathbf{n}}_{i_1},{\mathbf{n}}_{i_2},{\mathbf{n}}_{i_3},{\mathbf{n}}_{i_4}\in \mathbb{S}\},\end{array}$$

(4)

where $i_1,i_2,i_3,i_4=1,2,\dots ,M$, as described in Section 2.

Let ${\mathbf{m}}_1,{\mathbf{m}}_2,\dots ,{\mathbf{m}}_{\left|\mathbb{D}\right|}$ denote the distinct elements in $\mathbb{D}$, where $\left|\mathbb{D}\right|$ is the cardinality of $\mathbb{D}$. The following definition gives the concept of a transformation matrix, which describes the relation between $\mathbf{b}(\theta )$ and the steering vector on $\mathbb{D}$ (see Property 2 in this subsection).

Definition 2.

The transformation matrix $\mathbf{J}$: Given array $\mathbb{S}$, the associated FODCA $\mathbb{D}$ can be obtained by Definition 1. The transformation matrix $\mathbf{J}$ is an $M^4\times \left|\mathbb{D}\right|$ matrix, where the ξth column corresponds to the vector ${\mathbf{m}}_{\xi }\in \mathbb{D}$, and the $(i,\xi )$th entry of $\mathbf{J}$ satisfies

$$\begin{array}{c}\hfill {\mathbf{J}}_{i,\xi }=\{\begin{array}{c}1,\mathrm{if}{\mathbf{n}}_{i_1}+{\mathbf{n}}_{i_2}-{\mathbf{n}}_{i_3}-{\mathbf{n}}_{i_4}={\mathbf{m}}_{\xi },\hfill \\ 0,\mathrm{otherwise},\hfill \end{array}\end{array}$$

(5)

where the index $\xi =1,2,\dots ,\left|\mathbb{D}\right|$, and recall that $i=(i_1-1)M^3+(i_2-1)M^2+(i_3-1)M+i_4$ in Section 2.

Property 1.

The transformation matrix $\mathbf{J}$ has full column rank.

Property 2.

Let ${\mathbf{a}}_{\mathbb{D}}(\theta )$ be the steering vector of $\mathbb{D}$ (i.e., ${\mathbf{a}}_{\mathbb{D}}(\theta )={[{\mathrm{e}}^{\mathrm{j}2\pi {\mathbf{u}}^{\mathrm{T}}{\mathbf{m}}_1},{\mathrm{e}}^{\mathrm{j}2\pi {\mathbf{u}}_k^{\mathrm{T}}{\mathbf{m}}_2},\dots ,{\mathrm{e}}^{\mathrm{j}2\pi {\mathbf{u}}_k^{\mathrm{T}}{\mathbf{m}}_{\left|\mathbb{D}\right|}}]}^{\mathrm{T}}$), we have

$$\begin{array}{c}\hfill \mathbf{b}(\theta )={\mathbf{Ja}}_{\mathbb{D}}(\theta ),\end{array}$$

(6)

where $\mathbf{b}(\theta )={\mathbf{a}}_1(\theta )\otimes {\mathbf{a}}_2(\theta )\otimes {\mathbf{a}}_3^{*}(\theta )\otimes {\mathbf{a}}_4^{*}(\theta )$, ${\mathbf{a}}_{\gamma }(\theta )$ is the steering vector for θ, and $\gamma =1,2,3,4$.

Property 3.

Define $\mathbb{H}=\left\{\left|\mathbb{D}\right|:\mathbb{D}=f(\mathbb{S}),\mathbb{S}=\{{\mathbf{n}}_1,\dots ,{\mathbf{n}}_M\},\forall {\mathbf{n}}_1,\dots ,{\mathbf{n}}_M\in {\mathbb{R}}^3\right\}$, where $f(·)$ is the function that $\mathbb{S}$ maps to $\mathbb{D}$ by Definition 1. Then the supremum of $\mathbb{H}$ is

$$\begin{array}{c}\hfill sup\{\mathbb{H}\}=(M^4-2M^3+7M^2-6M+4)/4.\end{array}$$

(7)

Properties 1, 2, and 3 are proved in Appendix A, Appendix B, and Appendix C, respectively.

From Property 2, we have $\mathbf{B}={\mathbf{JA}}_{\mathbb{D}}$, where ${\mathbf{A}}_{\mathbb{D}}=[{\mathbf{a}}_{\mathbb{D}}({\theta }_1),{\mathbf{a}}_{\mathbb{D}}({\theta }_2),\dots ,{\mathbf{a}}_{\mathbb{D}}({\theta }_K)]$. Substituting it to (3), we obtain ${\mathbf{c}}_x={\mathbf{JA}}_{\mathbb{D}}{\mathbf{c}}_s$. Due to Property 1, we have

$$\begin{array}{c}\hfill {\mathbf{J}}^{+}{\mathbf{c}}_x={\mathbf{A}}_{\mathbb{D}}{\mathbf{c}}_s\in {\mathbb{C}}^{\left|\mathbb{D}\right|},\end{array}$$

(8)

where ${\mathbf{J}}^{+}={({\mathbf{J}}^{\mathrm{H}}\mathbf{J})}^{-1}{\mathbf{J}}^{\mathrm{H}}$ is the Moore–Penrose inverse of $\mathbf{J}$. Comparing the model (8) with (3), it is clear that the row dimension decreases from $M^4$ to $\left|\mathbb{D}\right|$.

Remark 1: An analogous procedure can be found in covariance vector-based DOA estimation methods [22]. Among HOCs-based DOA estimation methods, rectangular $2q$-MUSIC provides two strategies to reduce the dimension of a rectangle FOCs matrix (i.e., selection strategy and averaging strategy), by eliminating redundancies of associated left and right virtual steering vectors [7]. From this perspective, the proposed dimension reduction procedure is the essential realization of the averaging strategy applying to FODCA, with an explicit formulation (8). It is clear that the averaging strategy is better than the selection strategy due to the averaging of the noise in the associated observation components [7], if the computation burden is not taken into consideration. Thus, the averaging strategy can also be treated as a denoising procedure. In addition, it can be seen that FODCA has the same geometry as a fourth-order virtual array with parameters $(q,l)=(4,2)$ [25], which is deduced from an eighth-order cumulants-based array processing problem. In TABLE IX from [25], the max number of identical virtual sensors in the fourth-order virtual array, with parameters $(q,l)=(4,2)$ and space diversity (without angular and polarization diversity), is illustrated, and it is equal to (7) after simplification due to the same geometry. The detailed derivation of (7) is provided in Appendix C, since it is not provided in [25].

Remark 2: It is clear that when $2\le K<M$, the limiting root mean square error (RMSE) of covariance vector-based methods is not necessarily zero [22,26], while the RMSE of the traditional MUSIC method converges to zero as signal–noise-ratio (SNR) approaches infinity. As the FODCA can be regarded as two levels of second-order difference co-array [8,18], applied to construct covariance vector-based methods, it can be deduced that the FOCs vector-based DOA estimation methods, including [8,16,17,18], will be outperformed by traditional MUSIC in high-SNR regions when $2\le K<M$. From this point, we only suggest to use these methods (including the proposed method) in underdetermined cases ($K\ge M$).

3.2. Sparse Model

In this subsection we develop the sparse model from (8). Let $\tilde{\mathbf{\theta }}=[{\tilde{\theta }}_1,\dots ,{\tilde{\theta }}_I]$ be a fixed sampling grid in the DOA range, where $I\gg \left|\mathbb{D}\right|$. Let $\tilde{\mathbf{\theta }}$ be a uniform grid with a grid resolution $r={\tilde{\theta }}_2-{\tilde{\theta }}_1\propto I^{-1}$. Suppose ${\tilde{\theta }}_{w_k}$ is the nearest grid point to ${\theta }_k$, where $w_k\in \{1,\dots ,I\}$. Then, the steering vector ${\mathbf{a}}_{\mathbb{D}}({\theta }_k)$ can be approximated by its first-order Taylor series

$$\begin{array}{c}\hfill {\mathbf{a}}_{\mathbb{D}}({\theta }_k)\approx {\mathbf{a}}_{\mathbb{D}}({\tilde{\theta }}_{w_k})+{\mathbf{a}}_{\mathbb{D}}^{\prime }({\tilde{\theta }}_{w_k})({\theta }_k-{\tilde{\theta }}_{w_k}).\end{array}$$

(9)

Denote

$$\begin{array}{c}\hfill {\tilde{\mathbf{A}}}_{\mathbb{D}}=[{\mathbf{a}}_{\mathbb{D}}({\tilde{\theta }}_1),\dots ,{\mathbf{a}}_{\mathbb{D}}({\tilde{\theta }}_I)],\end{array}$$

(10)

$$\begin{array}{c}\hfill {\tilde{\mathbf{A}}}_{\mathbb{D}}^{\prime }=[{\mathbf{a}}_{\mathbb{D}}^{\prime }({\tilde{\theta }}_1),\dots ,{\mathbf{a}}_{\mathbb{D}}^{\prime }({\tilde{\theta }}_I)].\end{array}$$

(11)

Denote $\mathbf{\beta }={[{\beta }_1,\dots ,{\beta }_I]}^{\mathrm{T}}\in {[-r/2,r/2]}^I$ and $\mathbf{\Phi }={\tilde{\mathbf{A}}}_{\mathbb{D}}+{\tilde{\mathbf{A}}}_{\mathbb{D}}^{\prime }\mathrm{diag}\{\mathbf{\beta }\}$, where for $w=1,\dots ,I$, ${\beta }_w={\theta }_k-{\tilde{\theta }}_{w_k}$ if $w=w_k$ for any $k=1,\dots ,K$ and ${\beta }_w=0$ otherwise, and $\mathrm{diag}\{·\}$ denotes the operator to form a diagonal matrix by a vector or to form a vector by diagonal entries of a matrix. Neglecting approximation errors of (9), model (8) can be expressed as:

$$\begin{array}{c}\hfill {\mathbf{J}}^{+}{\mathbf{c}}_x=\mathbf{\Phi }\mathbf{z},\end{array}$$

(12)

where $\mathbf{z}={[z_1,z_2,\dots ,z_I]}^{\mathrm{T}}$ is the zero-padded extension of ${\mathbf{c}}_s$ from $[{\theta }_1,\dots ,{\theta }_K]$ to $[{\tilde{\theta }}_1,\dots ,{\tilde{\theta }}_I]$.

Consider the case of existing estimation errors of FOCs. Let ${\widehat{\mathbf{c}}}_x={\mathbf{c}}_x+\mathbf{e}$, where ${\widehat{\mathbf{c}}}_x$ is the estimate of ${\mathbf{c}}_x$, and $\mathbf{e}$ is the estimation errors vector. Then, model (12) becomes

$$\begin{array}{c}\hfill {\mathbf{J}}^{+}{\widehat{\mathbf{c}}}_x=\mathbf{\Phi }\mathbf{z}+{\mathbf{J}}^{+}\mathbf{e}.\end{array}$$

(13)

Assume $\mathbf{e}\sim \mathrm{AsCN}({\mathbf{0}}_{M^4},\mathbf{Q})$ [1], where $\mathrm{AsCN}$ denotes the asymptotic normal distribution, ${\mathbf{0}}_{M^4}$ denotes zero vector with $M^4$ elements, and $\mathbf{Q}\in {\mathbb{C}}^{M^4\times M^4}$ is a positive definite (Hermitian) matrix. The estimation of FOCs and the matrix $\mathbf{Q}$ are discussed in Section 3.3. Following this assumption, we have ${\mathbf{J}}^{+}\mathbf{e}\sim \mathrm{AsCN}({\mathbf{0}}_{\left|\mathbb{D}\right|},\tilde{\mathbf{Q}})$, where $\tilde{\mathbf{Q}}={\mathbf{J}}^{+}\mathbf{Q}{({\mathbf{J}}^{+})}^{\mathrm{T}}$. Furthermore, we can obtain ${\tilde{\mathbf{Q}}}^{-\frac{1}{2}}{\mathbf{J}}^{+}\mathbf{e}\sim \mathrm{AsCN}({\mathbf{0}}_{\left|\mathbb{D}\right|},{\mathbf{I}}_{\left|\mathbb{D}\right|\times \left|\mathbb{D}\right|})$, where ${\mathbf{I}}_{\left|\mathbb{D}\right|\times \left|\mathbb{D}\right|}$ is $\left|\mathbb{D}\right|$ dimensional identity matrix. We denote by

$$\begin{array}{c}{\tilde{\mathbf{c}}}_x={\tilde{\mathbf{Q}}}^{-\frac{1}{2}}{\mathbf{J}}^{+}{\widehat{\mathbf{c}}}_x\in {\mathbb{C}}^{\left|\mathbb{D}\right|},\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill \tilde{\mathbf{\Phi }}={\tilde{\mathbf{Q}}}^{-\frac{1}{2}}\mathbf{\Phi }& ={\tilde{\mathbf{Q}}}^{-\frac{1}{2}}({\tilde{\mathbf{A}}}_{\mathbb{D}}+{\tilde{\mathbf{A}}}_{\mathbb{D}}^{\prime }\mathrm{diag}\{\mathbf{\beta }\})\in {\mathbb{C}}^{\left|\mathbb{D}\right|\times I},\hfill \end{array}$$

(15)

$$\begin{array}{c}\tilde{\mathbf{e}}={\tilde{\mathbf{Q}}}^{-\frac{1}{2}}{\mathbf{J}}^{+}\mathbf{e}\in {\mathbb{C}}^{\left|\mathbb{D}\right|}.\hfill \end{array}$$

(16)

Then, we can obtain the following SMV off-grid sparse model

$$\begin{array}{c}\hfill {\tilde{\mathbf{c}}}_x=\tilde{\mathbf{\Phi }}\mathbf{z}+\tilde{\mathbf{e}},\end{array}$$

(17)

with $\tilde{\mathbf{e}}\sim \mathrm{AsCN}({\mathbf{0}}_{\left|\mathbb{D}\right|},{\mathbf{I}}_{\left|\mathbb{D}\right|\times \left|\mathbb{D}\right|})$. Note that if setting ${\beta }_w=0$ for $w=1,\dots ,I$, the model (17) is simplified to an on-grid DOA model.

3.3. Estimation of FOCs and the Matrix $\mathbf{Q}$

To facilitate the implementation of the proposed method, in this subsection we present the estimation methods of FOCs and the matrix $\mathbf{Q}$.

Let us introduce the following general notations: ${\mathfrak{m}}_{4,x}(i_1,i_2,i_3*,i_4*)=\mathrm{E}\{x_{i_1}(t)x_{i_2}(t)x_{i_3}^{*}(t)x_{i_4}^{*}(t)\}$, ${\mathfrak{m}}_{2,x}(i_1,i_2*)=\mathrm{E}\{x_{i_1}(t)x_{i_2}^{*}(t)\}$, and ${\mathfrak{m}}_{2,x}(i_1,i_2)=\mathrm{E}\{x_{i_1}(t)x_{i_2}(t)\}$. Their empirical estimators are ${\widehat{\mathfrak{m}}}_{4,x}(i_1,i_2,i_3*,i_4*)=\frac{1}{L}{\sum }_{t=1}^L\{x_{i_1}(t)x_{i_2}(t)x_{i_3}^{*}(t)x_{i_4}^{*}(t)\}$, ${\widehat{\mathfrak{m}}}_{2,x}(i_1,i_2*)=\frac{1}{L}{\sum }_{t=1}^L\{x_{i_1}(t)x_{i_2}^{*}(t)\}$, and ${\widehat{\mathfrak{m}}}_{2,x}(i_1,i_2)=\frac{1}{L}{\sum }_{t=1}^L\{x_{i_1}(t)x_{i_2}(t)\}$, respectively, where L is the number of snapshots. Due to the assumptions of sources and noises, ${[{\mathbf{c}}_x]}_i$ can be expressed as [25]:

$$\begin{array}{c}{[{\mathbf{c}}_x]}_i=\mathrm{cum}\{x_{i_1}(t),x_{i_2}(t),x_{i_3}^{*}(t),x_{i_4}^{*}(t)\}\hfill \\ ={\mathfrak{m}}_{4,x}(i_1,i_2,i_3*,i_4*)-{\mathfrak{m}}_{2,x}(i_1,i_2){\mathfrak{m}}_{2,x}^{*}(i_3,i_4)\hfill \\ -{\mathfrak{m}}_{2,x}(i_1,i_3*){\mathfrak{m}}_{2,x}(i_2,i_4*)-{\mathfrak{m}}_{2,x}(i_1,i_4*){\mathfrak{m}}_{2,x}(i_2,i_3*).\hfill \end{array}$$

(18)

Utilizing the corresponding empirical estimators of terms in (18), ${[{\mathbf{c}}_x]}_i$ can be estimated by

$$\begin{array}{c}{[{\widehat{\mathbf{c}}}_x]}_i={\widehat{\mathfrak{m}}}_{4,x}(i_1,i_2,i_3*,i_4*)-{\widehat{\mathfrak{m}}}_{2,x}(i_1,i_2){\mathfrak{m}}_{2,x}^{*}(i_3,i_4)\hfill \\ -{\widehat{\mathfrak{m}}}_{2,x}(i_1,i_3*){\mathfrak{m}}_{2,x}(i_2,i_4*)-{\widehat{\mathfrak{m}}}_{2,x}(i_1,i_4*){\mathfrak{m}}_{2,x}(i_2,i_3*).\hfill \end{array}$$

(19)

Consider complex variables $X(t)=x_{i_1}(t)x_{i_2}(t)x_{i_3}^{*}(t)x_{i_4}^{*}(t)-x_{i_1}(t)x_{i_2}(t){\sum }_{t_0=1}^L{x}_{i_3}^{*}(t_0)x_{i_4}^{*}(t_0)/L-x_{i_1}(t)x_{i_3}^{*}(t){\sum }_{t_1=1}^L{x}_{i_2}(t_1)x_{i_4}^{*}(t_1)/L-x_{i_1}(t)x_{i_4}^{*}(t){\sum }_{t_2=1}^L{x}_{i_2}(t_2)x_{i_3}^{*}(t_2)/L$ for $t=1,2,\dots ,L$. It is obvious that ${lim}_{L\to +\infty }\mathrm{E}\{X(t)\}={[{\mathbf{c}}_x]}_i$. According to central limit theorem, as $L\to +\infty$, the sample average $\overline{X}={\sum }_{t=1}^{L}X(t)={[{\widehat{\mathbf{c}}}_x]}_i$ converges in distribution to a complex normal. This solution theoretically supports the assumption that $\mathbf{e}={\widehat{\mathbf{c}}}_x-{\mathbf{c}}_x\sim \mathrm{AsCN}({\mathbf{0}}_{M^4},\mathbf{Q})$ in Section 3.2, and we aim for a robust estimation of $\mathbf{Q}$ in the following derivation.

Consider signals given by

$$\begin{array}{c}\hfill \mathbf{y}(t)=\mathbf{x}(t)\otimes \mathbf{x}(t)-\mathrm{E}\{\mathbf{x}(t)\otimes \mathbf{x}(t)\}.\end{array}$$

(20)

It is obvious that $\mathrm{E}\{\mathbf{y}(t)\}={\mathbf{0}}_{M^2}$. Let $y_g(t)$ denote the gth element of $\mathbf{y}(t)$, $g_1=(i_1-1)M+i_2$, and $g_2=(i_3-1)M+i_4$, then we have

$$\begin{array}{ccc}\hfill {\mathfrak{m}}_{2,y}(g_1,g_2*)=& {\mathfrak{m}}_{4,x}(i_1,i_2,i_3*,i_4*)-\hfill & \hfill {\mathfrak{m}}_{2,x}(i_1,i_2){\mathfrak{m}}_{2,x}^{*}(i_3,i_4).\end{array}$$

(21)

Rewrite (18) as:

$$\begin{array}{c}{[{\mathbf{c}}_x]}_i={\mathfrak{m}}_{2,y}(g_1,g_2*)-{\mathfrak{m}}_{2,x}(i_1,i_3*){\mathfrak{m}}_{2,x}(i_2,i_4*)\hfill \\ -{\mathfrak{m}}_{2,x}(i_1,i_4*){\mathfrak{m}}_{2,x}(i_2,i_3*),\hfill \end{array}$$

(22)

where $\mathbf{y}(t)$ is given by (20).

Lemma 1.

Given two joint distributed complex-valued random variables $Y_t$ and $Z_t$ with snapshots index $t=1,2,\dots ,L$, $Y_t$, and $Z_t$ are independent from snapshot to snapshot, respectively. As L approaches infinity, it holds true that

$$\begin{array}{c}\hfill \mathrm{var}(\overline{Y_t{Z}_t})/\mathrm{var}({\overline{Y}}_t{\overline{Z}}_t)=O(L),\end{array}$$

(23)

where $\mathrm{var}(·)$ is variance, and $\overline{Y_t{Z}_t}$, ${\overline{Y}}_t$, and ${\overline{Z}}_t$ are sample averages of $Y_t{Z}_t$, $Y_t$, and $Z_t$ with L snapshots, respectively.

Proof. See Appendix D.

Due to Lemma 1, from (19) and (22), it is easy to find that when the snapshots number L is large, the term ${\widehat{\mathfrak{m}}}_{2,y}(g_1,g_2*)$ has much larger variance than the other two terms in (22). This implies that the estimation error of ${[{\widehat{\mathbf{c}}}_x]}_i$ is mainly caused by the term ${\widehat{\mathfrak{m}}}_{2,y}(g_1,g_2*)$, giving rise to the following approximate expression of $e_i$:

$$\begin{array}{c}\hfill e_i={[{\widehat{\mathbf{c}}}_x]}_i-{[{\mathbf{c}}_x]}_i\approx {\widehat{\mathfrak{m}}}_{2,y}(g_1,g_2*)-{\mathfrak{m}}_{2,y}(g_1,g_2*).\end{array}$$

(24)

Since the signals are independent from snapshot to snapshot, and $\mathrm{E}\{\mathbf{y}(t)\}={\mathbf{0}}_{M^2}$, we can obtain

$$\begin{array}{cc}\hfill \mathrm{E}\{e_i{e}_j^{*}\}& \approx \frac{1}{L^2}\sum _{t_1=1}^L\sum _{t_2=1}^L\mathrm{E}\{[y_{g_1}(t_1)y_{g_2}^{*}(t_1)]{[y_{h_1}(t_2)y_{h_2}^{*}(t_2)]}^{*}\}\hfill \\ & -{\mathfrak{m}}_{2,y}(g_1,g_2*){\mathfrak{m}}_{2,y}^{*}(h_1,h_2*)\hfill \\ & =\mathrm{cum}\{y_{g_1}(t),y_{g_2}^{*}(t),y_{h_1}^{*}(t),y_{h_2}(t)\}/L\hfill \\ & +{\mathfrak{m}}_{2,y}(g_1,h_1*){\mathfrak{m}}_{2,y}(g_2*,h_2)/L\hfill \\ & +{\mathfrak{m}}_{2,y}(g_1,h_2){\mathfrak{m}}_{2,y}(g_2*,h_1*)/L,\hfill \end{array}$$

(25)

where $h_1=(j_1-1)M+j_2$, $h_2=(j_3-1)M+j_4$, $j=(h_1-1)M^2+h_2$, and $j_1,j_2,j_3,j_4=1,2,\dots ,M$.

In (25), the term $\mathrm{cum}\{y_{g_1}(t),y_{g_2}^{*}(t),y_{h_1}^{*}(t),y_{h_1}^{*}(t)\}/L$ contains the eighth-order statistics of $\mathbf{x}(t)$, which significantly aggravates the inaccuracy of estimation if the number of snapshots is not large enough. In addition, as the number of snapshots number approaches infinity, we get $\mathrm{E}\{e_i{e}_j^{*}\}\approx 0$. Therefore, we neglect this term to obtain a robust estimation of $\mathbf{Q}$, given by

$$\begin{array}{c}{\widehat{\mathbf{Q}}}_{i,j}={\widehat{\mathfrak{m}}}_{2,y}(g_1,h_1*){\widehat{\mathfrak{m}}}_{2,y}(g_2*,h_2)/L\hfill \\ +{\widehat{\mathfrak{m}}}_{2,y}(g_1,h_2){\widehat{\mathfrak{m}}}_{2,y}(g_2*,h_1*)/L.\hfill \end{array}$$

(26)

Rewrite (26) in matrix form

$$\begin{array}{c}\hfill \widehat{\mathbf{Q}}={\widehat{\mathbf{R}}}_y\otimes {\widehat{\mathbf{R}}}_y^{\mathrm{T}}/L+[{\widehat{\mathbf{T}}}_y\otimes {\widehat{\mathbf{T}}}_{y,1}^{*},{\widehat{\mathbf{T}}}_y\otimes {\widehat{\mathbf{T}}}_{y,2}^{*},\dots ,{\widehat{\mathbf{T}}}_y\otimes {\widehat{\mathbf{T}}}_{y,M^2}^{*}]/L,\end{array}$$

(27)

where ${\widehat{\mathbf{R}}}_y={\sum }_{t=1}^L\mathbf{y}(t){\mathbf{y}}^{\mathrm{H}}(t)/L$, ${\widehat{\mathbf{T}}}_y={\sum }_{t=1}^L\mathbf{y}(t){\mathbf{y}}^{\mathrm{T}}(t)/L$, and ${\widehat{\mathbf{T}}}_{y,h_1}$ is the $h_1$th column of ${\widehat{\mathbf{T}}}_y$.

Note that the estimator $\widehat{\mathbf{Q}}$ in (30) need to be revised in certain cases. It is clear that, for the noise-free case, ${\widehat{\mathbf{R}}}_y=(\mathbf{A}\otimes \mathbf{A}){\sum }_{t=1}^L\{[\mathbf{s}(t){\mathbf{s}}^{\mathrm{H}}(t)]\otimes [\mathbf{s}(t){\mathbf{s}}^{\mathrm{H}}(t)]\}{(\mathbf{A}\otimes \mathbf{A})}^{\mathrm{H}}/L$, and ${\widehat{\mathbf{T}}}_y=(\mathbf{A}\otimes \mathbf{A}){\sum }_{t=1}^L\{[\mathbf{s}(t){\mathbf{s}}^{\mathrm{T}}(t)]\otimes [\mathbf{s}(t){\mathbf{s}}^{\mathrm{T}}(t)]\}{(\mathbf{A}\otimes \mathbf{A})}^{\mathrm{T}}/L$. When $K<M$, the ranks of both matrices ${\widehat{\mathbf{R}}}_y$ and ${\widehat{\mathbf{T}}}_y$ are $K^2$. As a result, the estimator (27) may be singular, which may lead to a singular matrix $\tilde{\mathbf{Q}}$. This is unallowable for model (17). Therefore, when $K<M$, and SNR is high, to avoid above problems, $\mathbf{Q}$ is estimated by

$$\begin{array}{c}\hfill \widehat{\mathbf{Q}}={\widehat{\mathbf{R}}}_y\otimes {\widehat{\mathbf{R}}}_y^{\mathrm{T}}/L+[{\widehat{\mathbf{T}}}_y\otimes {\widehat{\mathbf{T}}}_{y,1}^{*},{\widehat{\mathbf{T}}}_y\otimes {\widehat{\mathbf{T}}}_{y,2}^{*},\dots ,{\widehat{\mathbf{T}}}_y\otimes {\widehat{\mathbf{T}}}_{y,M^2}^{*}]/L+{ϵ}_1{\mathbf{I}}_{\left|\mathbb{D}\right|\times \left|\mathbb{D}\right|},\end{array}$$

(28)

where ${ϵ}_1$ is a small positive number.

Specially, if the observed signals are circular, we have ${\mathfrak{m}}_{2,x}(l_1,l_2)=0$ and ${\mathfrak{m}}_{4,x}(l_1,l_2,l_3,l_4)=0$ for $\forall l_1,l_2,l_3,l_4=1,\dots ,M$. Then, we obtain $\mathbf{y}(t)=\mathbf{x}(t)\otimes \mathbf{x}(t)$, and ${\mathbf{c}}_i$ can be estimated by

$$\begin{array}{c}{[{\widehat{\mathbf{c}}}_x]}_i={\widehat{\mathfrak{m}}}_{4,x}(i_1,i_2,i_3*,i_4*)-{\widehat{\mathfrak{m}}}_{2,x}(i_1,i_3*){\widehat{\mathfrak{m}}}_{2,x}(i_2,i_4*)\hfill \\ -{\widehat{\mathfrak{m}}}_{2,x}(i_1,i_4*){\widehat{\mathfrak{m}}}_{2,x}(i_2,i_3*).\hfill \end{array}$$

(29)

Furthermore, it is obvious that ${\mathfrak{m}}_{2,y}(g_1,h_2)$ and ${\mathfrak{m}}_{2,y}(g_2*,h_1*)$ become zero in (25). Then, the estimator of $\mathbf{Q}$ in (27) becomes:

$$\begin{array}{c}\hfill \widehat{\mathbf{Q}}={\widehat{\mathbf{R}}}_y\otimes {\widehat{\mathbf{R}}}_y^{\mathrm{T}}/L.\end{array}$$

(30)

Accordingly, when $K<M$ and SNR is high, for circular sources, $\mathbf{Q}$ is estimated by

$$\begin{array}{c}\hfill \widehat{\mathbf{Q}}={\widehat{\mathbf{R}}}_y\otimes {\widehat{\mathbf{R}}}_y^{\mathrm{T}}/L+{ϵ}_2{\mathbf{I}}_{\left|\mathbb{D}\right|\times \left|\mathbb{D}\right|},\end{array}$$

(31)

where ${ϵ}_2$ is a small positive number.

Remark 3: Approximate expressions for the covariances of FOCs which are more precise than (25) are derived in [5]. However, it is not easy to use in our method in practice, because several higher-order (greater than four) moments need to be estimated, which may cause the estimation of $\mathbf{Q}$ to be not as robust as FOCs. For example, if the number of snapshots is large enough to generate precise estimates of FOCs, but not large enough to generate ones of higher-order (greater than four) moments, the DOA estimation results may suffer from a mass of outliers in independent experiments. In comparison, (26) contains at most fourth-order statistics, which means that it is more robust and has lower computational cost. We will show that our estimators work well for the proposed method in numerical simulations.

3.4. DOA Estimation by OGSBI Method

In this subsection, the sparse model (17) is solved to obtain the DOAs by the off-grid sparse Bayesian inference (OGSBI) method. Recall the sparse model (17):

$$\begin{array}{c}\hfill {\tilde{\mathbf{c}}}_x=\tilde{\mathbf{\Phi }}\mathbf{z}+\tilde{\mathbf{e}}.\end{array}$$

(32)

In Section 3.3, we showed that the estimation errors of FOCs obey $\mathbf{e}={\widehat{\mathbf{c}}}_x-{\mathbf{c}}_x\sim \mathrm{AsCN}({\mathbf{0}}_{M^4},\mathbf{Q})$, and the robust estimators of $\mathbf{Q}$ are also given. As $\tilde{\mathbf{e}}={\tilde{\mathbf{Q}}}^{-\frac{1}{2}}{\mathbf{J}}^{+}\mathbf{e}\in {\mathbb{C}}^{\left|\mathbb{D}\right|}$, it is obvious that $\tilde{\mathbf{e}}\sim \mathrm{AsCN}({\mathbf{0}}_{\left|\mathbb{D}\right|},{\mathbf{I}}_{\left|\mathbb{D}\right|\times \left|\mathbb{D}\right|})$. Following the sparse Bayesian formulation in [21], further assume $\tilde{\mathbf{e}}\sim \mathrm{CN}({\mathbf{0}}_{\left|\mathbb{D}\right|},{\mathbf{I}}_{\left|\mathbb{D}\right|\times \left|\mathbb{D}\right|})$ for simplicity. Then, we have

$$\begin{array}{c}\hfill p({\tilde{\mathbf{c}}}_x|\mathbf{z},\mathbf{\beta })=\mathrm{CN}(\tilde{\mathbf{\Phi }}\mathbf{z},{\mathbf{I}}_{\left|\mathbb{D}\right|\times \left|\mathbb{D}\right|}).\end{array}$$

(33)

Adopt the two-stage hierarchical prior for $\mathbf{z}$ that $p(\mathbf{z};\rho )=\int p(\mathbf{z}|\mathbf{\alpha })p(\mathbf{\alpha };\rho )\mathrm{d}\mathbf{\alpha }$, in which

$$\begin{array}{c}p(\mathbf{z}|\mathbf{\alpha })=\mathrm{CN}({\mathbf{0}}_I,\Lambda ),\hfill \end{array}$$

(34)

$$\begin{array}{c}p(\mathbf{\alpha };\rho )=\prod _{w=1}^I\Gamma ({\alpha }_w;1,\rho ),\hfill \end{array}$$

(35)

where $\rho >0$, $\mathbf{\alpha }\in {\mathbb{R}}^I$, $\Lambda =\mathrm{diag}\{\mathbf{\alpha }\}$, $\Gamma ({\alpha }_w;1,\rho )=\rho {\mathrm{e}}^{-\rho {\alpha }_w}$ is the probability density function (PDF) of the Gamma distribution with parameters ($1,\rho$), and ${\alpha }_w$ is the wth entry of $\mathbf{\alpha }$. Here the Gamma hyperprior is assumed for ${\alpha }_w$, since it is a conjugate prior of the Gaussian distribution, and it is widely used in SBL techniques and demonstrated with good performance and robustness [13,21,27,28].

Assume a uniform prior for $\mathbf{\beta }$: $\mathbf{\beta }\sim \mathrm{U}({[-r/2,r/2]}^I)$. By combining the stages of the hierarchical Bayesian model, the joint PDF is

$$\begin{array}{c}\hfill p(\mathbf{z},{\tilde{\mathbf{c}}}_x,\mathbf{\alpha },\mathbf{\beta })=p({\tilde{\mathbf{c}}}_x|\mathbf{z},\mathbf{\beta })p(\mathbf{z}|\mathbf{\alpha })p(\mathbf{\alpha })p(\mathbf{\beta }).\end{array}$$

(36)

The posterior distribution of $\mathbf{z}$ can be obtained

$$\begin{array}{c}\hfill p(\mathbf{z}|{\tilde{\mathbf{c}}}_x,\Lambda ,\mathbf{\beta })=\mathrm{CN}(\mathbf{\mu },\Sigma ),\end{array}$$

(37)

with

$$\begin{array}{c}\mathbf{\mu }=\Sigma {\tilde{\mathbf{\Phi }}}^{\mathrm{H}}{\tilde{\mathbf{c}}}_x\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill \Sigma ={({\tilde{\mathbf{\Phi }}}^{\mathrm{H}}\tilde{\mathbf{\Phi }}+{\Lambda }^{-1})}^{-1}& =\Lambda -{\Lambda \tilde{\mathbf{\Phi }}}^{\mathrm{H}}{(\mathbf{I}+{\tilde{\mathbf{\Phi }}\Lambda \tilde{\mathbf{\Phi }}}^{\mathrm{H}})}^{-1}\tilde{\mathbf{\Phi }}\Lambda .\hfill \end{array}$$

(39)

As a single snapshot case of the OGSBI method, we can obtain the following updates of $\mathbf{\alpha }$ from the deduction in [21]:

$$\begin{array}{c}\hfill {\alpha }_w^{\mathrm{new}}=(\sqrt{1+4\rho \mathrm{E}\{|z_w{|}^2\}}-1)/(2\rho ),\end{array}$$

(40)

where $\mathrm{E}\{|z_w{|}^2\}=|{\mu }_w{|}^2+{\Sigma }_{ww}$, ${\mu }_w$ is the wth entry of $\mathbf{\mu }$, and ${\Sigma }_{ww}$ is $(w,w)$th entry of $\Sigma$. For $\mathbf{\beta }$, by maximizing $\mathrm{E}\{\log p({\tilde{\mathbf{c}}}_x|\mathbf{z},\mathbf{\beta })p(\mathbf{\beta })\}$, we can obtain the updates

$$\begin{array}{c}\hfill {\mathbf{\beta }}^{\mathrm{new}}=\underset{\mathbf{\beta }\in {[-r/2,r/2]}^I}{argmin}\{{\mathbf{\beta }}^{\mathrm{T}}\mathbf{P}\mathbf{\beta }-2{\mathbf{q}}^{\mathrm{T}}\mathbf{\beta }\},\end{array}$$

(41)

where $\mathbf{P}=\mathfrak{R}\{{[{({\tilde{\mathbf{A}}}_{\mathbb{D}}^{\prime })}^{\mathrm{H}}{\tilde{\mathbf{A}}}_{\mathbb{D}}^{\prime }]}^{*}\odot ({\mathbf{\mu }\mathbf{\mu }}^{\mathrm{H}}+\Sigma )\}$, $\mathbf{q}=\mathfrak{R}\{\mathrm{diag}\{{\mathbf{\mu }}^{*}\}{({\tilde{\mathbf{A}}}_{\mathbb{D}}^{\prime })}^{\mathrm{H}}({\tilde{\mathbf{c}}}_x-{\tilde{\mathbf{A}}}_{\mathbb{D}}\mathbf{\mu })\}-\mathfrak{R}\{\mathrm{diag}\{{({\tilde{\mathbf{A}}}_{\mathbb{D}}^{\prime })}^{\mathrm{H}}{\tilde{\mathbf{A}}}_{\mathbb{D}}\Sigma \}\}$, $\mathfrak{R}(·)$ takes the real part of a complex variable and ⊙ is the Hadamard product. An easier way to estimate $\mathbf{\beta }$ is provided in [21]. We use $\mathbf{\beta }$, $\mathbf{P}$, and $\mathbf{q}$ hereafter to denote their truncated versions for simplicity. By (41) and $\frac{\partial }{\partial \mathbf{\beta }}\{{\mathbf{\beta }}^{\mathrm{T}}\mathbf{P}\mathbf{\beta }-2{\mathbf{q}}^{\mathrm{T}}\mathbf{\beta }\}=2(\mathbf{P}\mathbf{\beta }-\mathbf{q})$, we have ${\mathbf{\beta }}^{\mathrm{new}}=\check{\mathbf{\beta }}$ if $\mathbf{P}$ is invertible and $\check{\mathbf{\beta }}={\mathbf{P}}^{-1}\mathbf{q}\in {[-r/2,r/2]}^K$. Otherwise, we update $\mathbf{\beta }$ elementwise. That is, at each step we update one ${\beta }_w$ by fixing up the other entries of $\mathbf{\beta }$. For $w=1,\dots ,K$, we first let $\check{\mathbf{\beta }}=\frac{q_k-{({\mathbf{P}}_k)}_{-k}^{\mathrm{T}}{\mathbf{\beta }}_{-k}}{P_{kk}}$, where ${\mathbf{\beta }}_{-k}$ denotes $\mathbf{\beta }$ without the kth entry. Then, by constraining ${\beta }_k\in [-r/2,r/2]$, we have

$${\beta }_k^{\mathrm{new}}=\{\begin{array}{c}{\check{\beta }}_k,\mathrm{if}{\check{\beta }}_k\in [-r/2,r/2];\hfill \\ -r/2,\mathrm{if}{\check{\beta }}_k<-r/2;\hfill \\ r/2,\mathrm{otherwise}.\hfill \end{array}$$

(42)

The OGSBI is terminated if $\parallel {\mathbf{\alpha }}^{\kappa +1}-{\mathbf{\alpha }}^{\kappa }{\parallel }_2/{\parallel {\mathbf{\alpha }}^{\kappa }\parallel }_2<\tau$ or the maximum number of iterations is reached, where $\kappa$ is the iteration and $\tau$ is the predefined tolerance parameter. Then, $\mathbf{\alpha }$ can be treated as the pseudo-spectrum

$$\begin{array}{c}\hfill {\mathcal{P}}_w={\alpha }_w.\end{array}$$

(43)

By finding the grid indices of highest K peaks of $\mathcal{P}$, denoted by ${\widehat{p}}_k,k=1,\dots ,K$, we can obtain K estimated DOAs by

$$\begin{array}{c}\hfill {\widehat{\theta }}_k={\widehat{\theta }}_{w_k}+{\widehat{\beta }}_{w_k}.\end{array}$$

(44)

Note that the parameter $\rho$ in the method should be given in advance. According to associated information in [21] and our testing results, the OGSBI method is insensitive to $\rho$ if $\rho$ is not too large. Furthermore, we have observed that by setting $\rho =0.01$, the sparse Bayesian learning methods in [13,21,28] always achieve a good performance in a wide range of scenarios. By contrast, the 4-L1 method [16,17,18] is sensitive to the parameter of allowable error bound, which is not easy to estimate (it was chosen to give the best results through trial-and-error for each scenario in the corresponding simulations in [16,17,18]). Therefore, compared with the 4-L1 method, the OGSBI method has the advantage of convenience in setting parameters.

Finally, we summarize our method in Algorithm 1.

Algorithm 1   The proposed sparse method using denoised FOCs vector
1)	Input: $\mathbf{x}$, $\mathbb{S}$, and d.
2)	Initialization: r, $\mathbf{\alpha }$, $\rho$, $ϵ$, and $\tau$.
3)	According to the associated scenarios, calculate ${\widehat{\mathbf{c}}}_x$ by (19) or (29).
4)	According to the associated scenarios, calculate $\widehat{\mathbf{Q}}$ by (27), (28), (30), or (31).
5)	Calculate $\mathbf{J}$, ${\tilde{\mathbf{A}}}_{\mathbb{D}}$, ${\tilde{\mathbf{A}}}_{\mathbb{D}}^{\prime }$, and ${\tilde{\mathbf{c}}}_x$ by Definition 2, (10), (11), and (14), respectively.
6)	Repeat the following:
	a)  Calculate $\mathbf{\mu }$ and $\Sigma$ by (38) and (39), respectively.
	b)  Update $\mathbf{\alpha }$ by (40).
	c)  Update $\mathbf{\beta }$ by (42).
	d)  If $\parallel {\mathbf{\alpha }}^{\kappa +1}-{\mathbf{\alpha }}^{\kappa }{\parallel }_2/{\parallel {\mathbf{\alpha }}^{\kappa }\parallel }_2<\tau$ or the maximum number of iterations is reached, iteration terminates.
7)	Find the grid indices of the highest peaks of (43), and output the DOAs by (44).

3.5. Identifiability of the Number of Sources

According to the theorems about parameter identifiability in [29,30,31] and the model (8), we have the following proposition.

Proposition 1.

Any K sources can be uniquely identified from model (17) if and only if

$$\begin{array}{c}\hfill K<\mathrm{spark}\{{\mathbb{B}}_{\theta }\}/2,\end{array}$$

(45)

where $\mathrm{spark}\{{\mathbb{B}}_{\theta }\}$ is defined as the smallest number of elements in ${\mathbb{B}}_{\theta }$ that are linearly dependent [32], ${\mathbb{B}}_{\theta }=\{\mathbf{b}(\theta ):\theta \in \Theta \}\subset {\mathbb{C}}^{\left|\mathbb{D}\right|}$, and Θ is $[-\pi /2,\pi /2)$ for linear array and $[-\pi ,\pi )$ for planar array.

It is generally difficult to compute $\mathrm{spark}\{{\mathbb{B}}_{\theta }\}$, except for when $\mathbb{D}$ is a uniform linear array (ULA). In this circumstance, $\mathbb{D}$ is hole-free, $\mathrm{spark}\{{\mathbb{B}}_{\theta }\}=\left|\mathbb{D}\right|+1$, and then $K<(|\mathbb{D}|+1)/2$ sources can be identified. However, hole-free FODCA with $O(M^4)$ DOFs has not been studied so far to the best of our knowledge. Recently, [16,17,18] provided several methods to construct FODCA with larger ULA segments, with which $K<(|{\mathbb{D}}_{\mathrm{ULA}}|+1)/2$ sources can be identified, where ${\mathbb{D}}_{\mathrm{ULA}}$ is defined as the largest ULA segment in $\mathbb{D}$.

For general arrays, because ${\mathbb{B}}_{\theta }\subset {\mathbb{C}}^{\left|\mathbb{D}\right|}$, it is obvious that

$$\begin{array}{c}\hfill \mathrm{spark}\{{\mathbb{B}}_{\theta }\}\le \left|\mathbb{D}\right|+1.\end{array}$$

(46)

According to Property 3, (46), and Proposition 1, we can obtain a necessary condition regarding the identifiability of the number of sources.

Theorem 1.

Consider model (17), a necessary condition to uniquely identify all sources is that the number of sources K fulfills

$$\begin{array}{c}\hfill K\le (M^4-2M^3+7M^2-6M)/8.\end{array}$$

(47)

Proof. See Appendix E.

In other words, any more than $(M^4-2M^3+7M^2-6M)/8$ sources cannot be uniquely identified by model (17).

In addition, there are $O(M^4)$ distinct elements in ${\mathbb{D}}_{\mathrm{ULA}}$ for certain sparse linear arrays (SLAs), such as four-level nested arrays [8] and arrays constructed from a expanding and shift scheme which consists of coprime arrays or nested arrays [18]. Therefore, our proposed method can identify $O(M^4)$ sources at most, if these arrays (not limited to) are used.

3.6. Complexity

Consider the worst case of complexity. For non-circular sources, the estimation of ${\mathbf{c}}_x$ using (19) needs $O(L{M}^4)$ multiplications. The estimation of $\mathbf{Q}$ using (27) or (28) needs $O(L{M}^8)$ multiplications. Note that all the symmetries of ${\mathbf{c}}_x$ and $\mathbf{Q}$ are not taken into account here, while it does not affect the order of complexity. Each iteration of (38) and (39) needs $O(I^2|\mathbb{D}|)$ multiplications, which are much more than each iteration of (40) and (42). Let T denote the iteration number, and consider the worst case that $O(|\mathbb{D}|)=O(M^4)$. Combing all of the above processes, we can obtain that the complexity of the proposed method is $O(L{M}^8+I^2{M}^{4}T)$.

3.7. Comparison with State-of-the-Art Methods

In this subsection, we compare the proposed method with state-of-the-art FOCs-based ones, in terms of required geometry, identifiability, prior knowledge of the number of sources, ability to handle correlated sources, numerical complexity, and sensitivity to parameters. Associated methods are listed as follows:

• 4-MUSIC [5,6]: non-redundant version with averaging strategy [7], denoted by A-4-MUSIC;
• 4-SS-MUSIC [8]: with averaging strategy, denoted by A-4-SS-MUSIC;
• 4-L1 [16,17,18].

Associated comparison results are illustrated in

Table 1. Comparison with state-of-the-art methods. 4-L1: fourth-order cumulants (FOCs) vector-based L1-norm method; A-4-MUSIC: non-redundant version of 4-MUSIC with averaging strategy; A-4-SS-MUSIC: FOCs vector-based spatial smoothing (SS)-MUSIC with averaging strategy; FODCA: fourth-order difference co-array; ULA: uniform linear array.

	A-4-MUSIC	A-4-SS-MUSIC	4-L1	Proposed
Required geometry	Any geometry	LA, with FODCA- containing ULA segment	Any geometry	Any geometry
Maximum identifiability	$O(M^2)$	$O(M^4)$	$O(M^4)$	$O(M^4)$
Prior knowledge of the number of sources	Need	Need	No need	No need
Handles correlated sources	Yes	No	No	No
Parameters and sensitivity	With no parameters	With no parameters	With sensitive parameters	With insensitive parameters
Complexity	$O(L{M}^4+I{M}^4+M^6)$	$O(L{M}^4+I{M}^8+M^{12})$	$O(L{M}^4+I^3{T}_2)$	$O(L{M}^8+I^2{M}^4{T}_1)$

.

In Table 1, $T_1$ and $T_2$ denote the iteration number of the proposed method and 4-L1, respectively. All comparison results are obvious or were discussed in the previous subsections, except for the comparison of complexity. Recall that M is the number of sensors, L is the number of snapshots, and I is the number of grid points. It should be noted that the complexities of four methods are derived under the assumption that $O(|\mathbb{D}|)=O(M^4)$, which gives the best identifiability and highest complexities for all methods. Due to the condition $I\gg \left|\mathbb{D}\right|$ required to perform the sparsity in the proposed method and 4-L1, it is clear that $O(I)\ge O(M^4)$, which results in $O(I^3{T}_2)>O(I{M}^8)$ and $O(I^2{M}^4{T}_1)>O(I{M}^8)$. Consequently, the proposed method and 4-L1 always have higher complexity than A-4-MUSIC and A-4-SS-MUSIC.

Next, we make a comparison of complexity between the proposed method and 4-L1. Firstly, if the number of snapshots L is large enough that the terms $O(I^3{T}_2)$ and $O(I^2{M}^4{T}_1)$ can be negligible, the proposed method has higher complexity than 4-L1 due to the term $O(L{M}^8)$ which is caused by the estimation of $\mathbf{Q}$. Secondly, if L is small, we limit the comparison between $O(I^3{T}_2)$ and $O(I^2{M}^4{T}_1)$. It is clear that $O(I^3)\ge O(I^2{M}^4)$, implying that the proposed method has lower complexity than 4-L1 in one iteration. However, it is a fact that 4-L1 always has less iterations than the proposed method (i.e., $T_1>T_2$). Thus, it is not easy to draw a conclusion. In fact, to achieve the super resolution, 4-L1 needs a dense grid, while the proposed method only needs a coarser grid due to the off-grid parameter estimation, which gives our method an advantage for complexity comparison. It should be noted that off-grid parameter estimation technique has been successfully applied to L1-norm minimization methods in [33,34]. However, to the best of our knowledge, the off-grid version of the 4-L1 method has not been published, and it is beyond the scope of this paper.

4. Numerical Simulations

Numerical simulations were carried out to demonstrate the superior performance of the proposed method. In the following simulations, the sampled analytic signals of statistically independent quaternary phase-shift keying (QPSK) signals were used as circular sources. The RMSE was chosen as the performance criterion, and is computed by

$$\begin{array}{c}\hfill RMSE=\sqrt{\frac{1}{FK}\sum _{f=1}^F\sum _{k=1}^K{|{\widehat{\theta }}_{k,f}-{\theta }_k|}^2},\end{array}$$

(48)

where F is the number of independent experiments, and ${\widehat{\theta }}_{k,f}$ is the estimate of ${\theta }_k$ in the fth independent experiment. The RMSE performances of the four methods in Table 1 with respect to number of snapshots, grid, adjacent sources, SNR, modelling errors, geometry, and circularity of sources were further investigated. In the proposed method, we set that $\rho =0.01$ and $\tau ={10}^{-4}$ for all scenarios, and set ${ϵ}_1={10}^{-5}$ or ${ϵ}_2={10}^{-5}$ for the case that $K<M$ and $SNR\ge 8$ dB. 4-L1 was solved by the SeDuMi toolbox [35], and the associated allowable error bound was chosen to give the best results through trial-and-error for each scenario as done in [16,17,18]. Each simulation executed $F=200$ independent experiments for all methods.

4.1. Identifiability

Consider an SLA with three sensors located at $\mathbb{S}=\{1,2,6\}\times d$. It can be deduced that the associated FODCA has 19 distinct elements. That is, $\left|\mathbb{D}\right|=19$, which is equal to $sup\{\mathbb{H}\}$ when $M=3$. According to Theorem 1, any greater than nine sources cannot be uniquely identified for the proposed method. Let us consider nine sources with DOAs $\{-{60}^{\circ },-{45}^{\circ },-{30}^{\circ },-{15}^{\circ },0^{\circ },{15}^{\circ },{30}^{\circ },{45}^{\circ },{60}^{\circ }\}$. The number of snapshots and SNR were set as $L={10}^4$ and $SNR=20$ dB, respectively. Grid resolution was set to $r={0.5}^{\circ }$. Figure 1a,b shows the normalized spectra of the proposed method and 4-L1, respectively. As can be seen, all DOAs of the sources were identified successfully. Note that this was unachievable for A-4-MUSIC and A-4-SS-MUSIC. The virtual array of A-4-MUSIC contained seven distinct virtual sensors, which could identify six sources at most. The largest ULA segment of the FOCDA contained 13 distinct elements, giving rise to a maximum identifiable number of six for A-4-SS-MUSIC.

4.2. Impact of the Grid Resolution

Consider an SLA $\{1,2,4,11,25\}\times d$, which is constructed from an expanding and shift scheme that consists of two nested arrays $\{1,2,4\}\times d$ and $\{1,2,4\}\times d$ [18]. It can be obtained that the number of distinct elements of the corresponding FODCA and associated largest ULA segment is $\left|\mathbb{D}\right|=85$ and ${\left|\mathbb{D}\right|}_{\mathrm{ULA}}=55$, respectively. Consider two independent sources with DOAs $\{0^{\circ },{20}^{\circ }\}+{\zeta }_1$, where ${\zeta }_1$ is a random variable chosen uniformly within $[-1^{\circ },1^{\circ }]$. SNR was set at $SNR=3$ dB.

Figure 2a,b shows the RMSE performance of the four methods with respect to number of snapshots, with grid resolution $r={0.5}^{\circ }$ and $r={0.1}^{\circ }$, respectively. In Figure 2a, when snapshots number was large, the RMSEs of A-4-MUSIC, A-4-SS-MUSIC, and 4-L1 did not reduce as snapshots increased, mainly because of the mismatch errors between grid and true DOAs. Due to the off-grid parameters estimation, the proposed method outperformed the other three methods for most of simulated snapshot numbers. In Figure 2b, the grid resolution is $r={0.1}^{\circ }$, and it is clear that the RMSEs of A-4-MUSIC, A-4-SS-MUSIC, and 4-L1 increased to the same level of the proposed method when snapshots were greater than 50 and less than 2000. Nonetheless, the proposed method still outperformed the other three methods when snapshots were more than or equal to 2000. It can also be observed that the proposed method had smaller RMSE than A-4-MUSIC and 4-L1 when snapshots were less than 100. The RMSEs of A-4-SS-MUSIC were slightly larger than the other three methods when snapshots were greater than 50 and less than 5000, which may be caused by the fact that 4-SS-MUSIC does not utilize all the virtual sensors in associated FODCA. In addition, the dashed line in Figure 2b gives the RMSEs of the proposed method for $r={0.5}^{\circ }$, and it is clear that the grid resolution seldom affected the RMSEs of the proposed method when snapshots were less than 2000, which implies that when snapshots are not too large, the proposed method can reduce the computational complexity by selecting a relatively coarser grid without heavy loss of RMSE performance.

4.3. Identifiability for Adjacent Sources

Next, we investigated the identifiability of adjacent sources using the proposed method. The array and parameters were set as in Section 4.2, except the grid resolution was set to $r={0.1}^{\circ }$, and the DOAs of sources were set to $\{{\zeta }_2,{\zeta }_2+▵\theta \}$, where ${\zeta }_2$ is a random variable chosen uniformly within $[-1^{\circ },1^{\circ }]$, and $▵\theta ={0.5}^{\circ },1^{\circ },\dots ,5^{\circ }$ are the different DOA intervals. Figure 3a,b shows the RMSE performance of four methods with different DOA intervals, when the number of snapshots is 300 and SNR is 3 dB, and when the number of snapshots is 1000 and SNR is 10 dB, respectively. It is clear that the proposed method had smallest RMSEs for most simulated DOA intervals in both simulations, demonstrating the superior identifiability of adjacent sources with the proposed method.

4.4. Impact of SNR

The array and parameters were set as in Section 4.2, except the grid resolution was set to $r={0.1}^{\circ }$. Figure 4 plots the RMSEs of four methods as a function of SNR, where the number of snapshots was set to $L=500$. It is clear that the four methods had almost equal RMSEs when SNR was smaller than 4 dB. In the high SNR region, a relatively better performance of the proposed method appeared, benefiting from the off-grid parameter estimation as discussed in the large snapshot number region in Section 4.2.

4.5. Impact of Modelling Errors

We also studied the effect of modelling errors on the proposed method. We used the same array geometry and parameters, and the same way of generating $K=2$ DOAs as in Section 4.4. We perturbed the steering vector by adding a modelling error vector ${\mathbf{e}}_{\mathbf{a}({\theta }_k)}$ to get the perturbed steering vector as $\tilde{\mathbf{a}}({\theta }_k)=\mathbf{a}({\theta }_k)+{\mathbf{e}}_{\mathbf{a}({\theta }_k)}$, where $k=1,\dots ,K$, and ${\mathbf{e}}_{\mathbf{a}({\theta }_k)}$ are assumed to be zero mean statistically independent Gaussian distributed with $\mathrm{E}\{{\mathbf{e}}_{\mathbf{a}({\theta }_k)}{\mathbf{e}}_{\mathbf{a}({\theta }_k)}^{\mathrm{H}}={\sigma }_{\mathbf{a}({\theta }_k)}^2\mathbf{I}$, where ${\sigma }_{\mathbf{a}({\theta }_k)}^2=0.0025$. Figure 5 shows the RMSE performance of four methods with such modelling errors. In order to facilitate comparison, the RMSEs of the proposed methods with no modelling errors are also plotted in a dashed line. It can be observed that the RMSE performance of the four methods did not degrade heavily. However, when snapshots were more than 1000, the superior performance of the proposed method disappeared, suggesting that modelling errors did have a large impact on the off-grid parameters estimation of the proposed method.

4.6. Underdetermined Case

We investigated the performance of the proposed method in underdetermined cases. The same array $\{1,2,4,11,25\}\times d$ was used, and the grid resolution $r={0.1}^{\circ }$, while six independent sources with DOAs $\{-{45}^{\circ },-{30}^{\circ },-{15}^{\circ },0^{\circ },{25}^{\circ },{40}^{\circ }\}$ were set. Figure 6 shows the RMSE performance of the four methods. In Figure 6a, the number of snapshots varying from 100 to 10,000 and the SNR was fixed at 20 dB. It is clear that the proposed method had the smallest RMSEs for most of the simulated snapshot numbers. In Figure 6b, SNR varied from −4 dB to 28 dB and the snapshots number was fixed at 1000. It can be observed that the proposed method had smaller RMSEs when $SNR>4$ dB, but had slightly larger values when $SNR\le 4$ dB, when compared with the other three methods. The RMSE performance of A-4-SS-MUSIC was obviously outperformed by the other three methods in the large snapshot number and high SNR regions, because 4-SS-MUSIC does not utilize all the virtual sensors in the associated FODCA. Moreover, the RMSEs of each method seemed to converge to a positive constant as SNR increased. A similar phenomenon can be observed and explained in covariance vector-based methods in the underdetermined case [22,26].

4.7. Non-Linear Array Case

We now give an example to show the effectiveness of the proposed method in the non-linear array case. We randomly generated a planar array as shown in Figure 7a, where the minimum distance between sensors was half of the wavelength. The corresponding FODCA geometry is also illustrated, which had $\left|\mathbb{D}\right|=131$ distinct elements. Note that the A-4-SS-MUSIC cannot be applied for the non-linear array case. We set three sources with DOAs $\{-{60}^{\circ },0^{\circ },{40}^{\circ }\}+{\zeta }_3$, where ${\zeta }_3$ is a random variable chosen uniformly within $[-1^{\circ },1^{\circ }]$. SNR was set at $SNR=6$ dB, and the RMSEs of A-4-MUSIC, 4-L1, and the proposed method were plotted as a function of the number of snapshots (Figure 7b). It can also be seen that the proposed method had better RMSE performance when snapshots were greater than 200.

4.8. Effectiveness for Non-Circular Sources

Finally, we consider the case of non-circular sources, using the same array $\{1,2,4,11,25\}\times d$, and six independent sources with DOAs $\{-{45}^{\circ },-{30}^{\circ },-{15}^{\circ },0^{\circ },{25}^{\circ },{40}^{\circ }\}$ as in Section 4.6. In contrast, we replaced the imaginary parts of sources by their real parts. That is, ${\mathbf{s}}_{\mathrm{NC}}(t)=\mathfrak{R}\{\mathbf{s}(t)\}+\mathrm{j}\mathfrak{R}\{\mathbf{s}(t)\}$, where ${\mathbf{s}}_{\mathrm{NC}}(t)$ is the non-circular sources to be simulated. We set $SNR=20$ dB and $r={0.1}^{\circ }$. We estimated the FOCs by (19) for all the methods, and estimated $\widehat{\mathbf{Q}}$ by (27) for the proposed method. Figure 8 shows the RMSEs of the four methods versus the number of snapshots. Similar to Figure 6a, it is clear that the proposed method had the smallest RMSEs for all simulated snapshot numbers. Beyond doubt, all the methods were effective for non-circular sources with the estimation of FOCs by (19).

5. Conclusion

In this paper, a novel SMV sparse model for non-Gaussian sources using denoised FOCs vector is established to perform DOA estimation, and solved efficiently by the OGSBI method. An advanced denoising and dimension reduction procedure of FOCs vector is provided. The estimation errors of FOCs are integrated in the proposed SMV model, and are approximately estimated in a simple way. The proposed method suits any geometry, does not need prior knowledge of the number of sources, has maximum identifiability $O(M^4)$, and is insensitive to associated parameters. The off-grid parameter estimation in our method promotes superior performance when the number of snapshots is large and SNR is high, and also ensures good performance when choosing a relatively coarse grid. Numerical simulations illustrate that the proposed method has superior identifiability or RMSE performance in different scenarios, namely grid resolution, adjacent sources, SNR, modelling errors, geometry, and non-circular sources, when compared with other state-of-the-art methods.