Non-Circular Signal DOA Estimation with Nested Array via Off-Grid Sparse Bayesian Learning

Abstract

For the traditional uniform linear array (ULA) direction of arrival (DOA) estimation method with a limited array aperture, a non-circular signal off-grid sparse Bayesian DOA estimation method based on nested arrays is proposed. Firstly, the extended matrix of the received data is constructed by taking advantage of the fact that the statistical properties of non-circular signals are not rotationally invariant. Secondly, we use the difference and sum co-arrays for the nested array technique, thus increasing the array aperture and improving the estimation accuracy. Finally, we take the noise as part of the interest signal and iteratively update the grid points using the sparse Bayesian learning (SBL) method to eliminate the modeling errors caused by off-grid gaps. The simulation results show that the proposed algorithm can improve the accuracy of DOA estimation compared with the existing algorithms.

1. Introduction

Direction of arrival (DOA) estimation is a problem in the field of array signal processing that has been widely used in radar, sonar, indoor positioning, and mobile communications [1], as well as in some near-field localization scenarios [2,3]. The earliest subspace-based super-resolution DOA estimation methods were developed, including multiple signal classification (MUSIC) [4], estimation of signal parameters via rotational invariance techniques (ESPRIT) [5], weighted subspace fitting (WSF) [6], etc. Usually, these algorithms use the uniform linear array (ULA) to receive the signal, where the M-element ULA can only distinguish $M-1$ sources at most. Many non-uniform arrays [7], such as minimum redundant array (MRA) [8], co-prime array (CA) [9,10,11], and nested array (NA) [12,13,14,15], can increase the degrees of freedom (DOFs) and improve the accuracy of DOA estimation by employing the virtualization technology. However, the virtualization technique turns the received signal into a single snapshot vector, resulting in rank loss of the covariance matrix. Scholars propose many effective methods for decoherence, such as spatial smoothing (SS)-based MUSIC (SS-MUSIC [12]) and Toeplitz-based algorithms [16]. Nevertheless, SS-MUSIC and Toeplitz-based algorithms seriously destroy the DOFs, so they require a large number of snapshots to obtain the signal or noise subspace accurately.

In recent years, algorithms based on sparse representation (SR) [17], compressive sensing (CS) [18,19], and sparse Bayesian learning (SBL) [20] have resolved coherent sources with limited snapshots and have gradually become hotspots in DOA estimation research. The $l_1$-SVD [21] algorithm assumes that the unknown source is on the grid, which can be a sparse signal representation (SSR) problem when the source of interest constitutes a sparse signal condition. An off-grid DOA estimation model based on CS is proposed in [22], where the estimated DOA no longer depends on a fixed grid. Yang Z et al. propose the off-grid sparse Bayesian inference (OGSBI) method [23] and developed an iterative algorithm by dividing the search range uniformly into grids, maintaining high estimation accuracy, even with very coarse sampling grids. Subsequently, Zhang proposed an improved off-grid SBL method to reduce the effect of noise variance [24], but a significant problem with this method is that the performance depends on the trade-off between accuracy and computational effort. In [25], the author proposes a computationally efficient SBL method for root off-grid DOA estimation that takes the sampling locations within the coarse grid as adjustable parameters and refines the coarse grid using the expectation–maximization (EM) algorithm [26], significantly reducing computational complexity and eliminating model errors. Although the above algorithms perform well in the off-grid field, the estimation accuracy of the algorithms can be further improved if combined with sparse arrays.

In [27], the author proposes a received model for a nested array that treats the noise variance as part of the unknown signal of interest and iterates the grid points by SBL, eliminating the model errors caused by off-grid gaps. In [28], the second-order Taylor expansion is used to replace the first-order Taylor expansion to alleviate the gap error from the grid, but it increases the amount of calculation. Since the nested array technique [27,28] constitutes a virtual differential co-array that is uniformly distributed and has no holes, the DOFs can be fully utilized.

However, the methods mentioned above only pay attention to the difference co-array of sparse arrays while ignoring the contribution of sum co-arrays. Non-circular (NC) signals [29,30] (such as binary phase shift keying (BPSK), pulse amplitude modulation (PAM), and amplitude shift keying (ASK)) can expand the array aperture and increase the DOFs, which are widely used in modern digital modulation schemes such as telecommunication or satellite systems [31,32,33]. The author in [34] converts the received model with NC signals into a real-valued sparse model and proposes a real-valued signal processing method in impulsive noise based on SBL. In [35], the author proposes a new SBL method to solve the off-grid DOA estimation problem, which can automatically identify sources from a grid of candidate angles and match the direction information of interest from covariance and pseudo-covariance vectors when circular and NC signals coexist. In [36], a new ESPRIT-like method is proposed, which derives a generalized covariance and is suitable for strictly NC signals. Nonetheless, the above-mentioned methods [34,35,36] are based on ULA, so the application of NC signals with sparse arrays may need to be discussed.

Through the above analysis, we propose an NC signal DOA estimation with the nested array method based on off-grid sparse Bayesian learning. The main contributions are as follows:

• We combine virtual difference co-arrays and sum co-arrays by exploiting the property of NC signals, which extends the array aperture and improves estimation accuracy.
• We take the noise variance as part of the NC signals of interest and then iterate over the internal parameters by the OGSBI method to maintain the standard SBL form after computing the selection matrix and removing redundant information in the nested array.

Other parts of this paper are organized as follows. Some backgrounds are presented in Section 2, including the data model and the concepts of sum and difference co-arrays. The mathematical analysis and discussion of the DOA estimation of the proposed algorithm is provided in Section 3. Numerical simulation and the conclusion are presented in Section 4 and Section 5, respectively.

Statement 1:

uppercase (lowercase) bold characters are used to represent matrices (vectors). The ${\left(·\right)}^T$, ${\left(·\right)}^H$, and ${\left(·\right)}^{*}$ operators correspond to the transpose, conjugate transpose, and conjugate operations, respectively. The $diag\left(·\right)$ notation is employed for diagonal matrices, while vec$\left(·\right)$ represents the vectorization operation. $〈a,b〉$ indicates the set $\left\{x\in \mathbb{Z}\left|a⩽x⩽b\right.\right\}$, the symbols ⊗, ⊙ and ⊕ stand for the Kronecker, Khatri-Rao, and Hadamard products, respectively. Matrix ${\mathbf{I}}_N$ denotes an identity matrix of size $N\times N$, and $\mathbb{E}$ signifies the expectation operator. Finally, $\left|·\right|$ is used to represent the magnitude of a complex value, and $Re\left(·\right)$ denotes the real part of the complex variable.

Statement 2:

We collect all the acronyms employed throughout the manuscript in

Table 1. Acronyms and their full names.

Acronyms	Full Name
DOA	direction of arrival
ULA	uniform linear array
SBL	sparse Bayesian learning
MUSIC	multiple signal classification
ESPRIT	estimation of signal parameters via rotational invariance techniques
WSF	weighted subspace fitting
CA	co-prime array
NA	nested array
MRA	minimum redundant array
DOFs	degrees of freedom
SS-MUSIC	spatial smoothing MUSIC
CS	compressive sensing
SR	sparse representation
SSR	sparse signal representation
OGSBI	off-grid sparse Bayesian inference
EM	expectation–maximization
NC	non-circular
PAM	pulse amplitude modulation
BPSK	binary phase shift keying
ASK	amplitude shift keying
CRB	Cramér–Rao bound
RMSE	root mean square error

for the readers’ convenience.

2. Background

2.1. The Data Model

Consider K narrowband far-field sources ${\theta }_k,k=1,2,\cdots ,K$ impinging on a nested array that consists of two concatenated ULAs (shown in Figure 1), where the inner and outer ULAs have M and N sensors with array spacing d and $\left(M+1\right)d$, respectively. $d=\frac{\lambda }{2}$ and $\lambda$ is the signal carrier wavelength. The positions of the sensors can be expressed as $\mathbb{L}=Ld$, where

$$\begin{array}{c}L=\left\{l_i,i=1,2,\cdots ,P\right\}\hfill \\ =\left\{0,1,\cdots ,M-1,M,2(M+1)-1,\cdots ,N(M+1)-1\right\},\hfill \end{array}$$

(1)

is a set containing the location information of all sensors, and $l_1<l_2<\cdots <l_i<\cdots <l_P$, $P=M+N$ denotes the number of sensors.

We consider K far-field narrowband independent sources ${\mathbf{s}}_k\left(t\right),k=1,\cdots ,K$, $t=1,2,\cdots ,T$ impinging into the linear sparse array, where T denotes the number of snapshots. Then, the received data at time t can be indicated as

$$\mathbf{y}\left(t\right)=\sum _{k=1}^K\mathbf{a}\left({\theta }_k\right){\mathbf{s}}_k\left(t\right)+\mathbf{n}\left(t\right)=\mathbf{As}\left(t\right)+\mathbf{n}\left(t\right),$$

(2)

where $\mathbf{s}\left(t\right)={\left[{\mathbf{s}}_1\left(t\right),{\mathbf{s}}_2\left(t\right),\cdots ,{\mathbf{s}}_K\left(t\right)\right]}^T\in {\mathbb{C}}^{K\times 1}$ is the signal vector and the noise $\mathbf{n}\left(t\right)$ obeys a Gaussian distribution, i.e., $\mathbf{n}\left(t\right)\sim \mathcal{C}\mathcal{N}\left(·\left|0,{\sigma }_n^2\right.\right)$, ${\sigma }_n^2$ denotes the noise variance and it is uncorrelated with the signal. $\mathbf{A}=\left[\mathbf{a}\left({\theta }_1\right),\cdots ,\mathbf{a}\left({\theta }_k\right),\cdots ,\mathbf{a}\left({\theta }_K\right)\right]\in {\mathbb{C}}^{P\times K}$ is the direction matrix with

$$\mathbf{a}\left({\theta }_k\right)={\left[1,e^{-j\frac{2\pi }{\lambda }{l}_{2}d sin{\theta }_k},\cdots ,e^{-j\frac{2\pi }{\lambda }{l}_{P}d sin{\theta }_k}\right]}^T.$$

(3)

Considering the strict NC signal model [37], the received data Equation (2) can be rewritten as

$$\mathbf{y}\left(t\right)=\mathbf{A}\mathbf{\Phi }{\mathbf{s}}_R\left(t\right)+\mathbf{n}\left(t\right).$$

(4)

where $\mathbf{\Phi }=\mathrm{diag}\left\{e^{-j{\phi }_1},\cdots ,e^{-j{\phi }_k},\cdots ,e^{-j{\phi }_K}\right\}$ with ${\phi }_k$ the NC phase of the k-th circular signal ${\mathbf{s}}_R\left(t\right)$, and ${\mathbf{s}}_R\left(t\right)\in {\mathbb{R}}^{K\times 1}$. By exploiting the NC characteristics of the signal, the received signal can be extended as

$$\begin{array}{c}{\mathbf{y}}_0\left(t\right)=\left[\begin{array}{c}\mathbf{y}\left(t\right)\\ {\mathbf{y}}^{*}\left(t\right)\end{array}\right]=\left[\begin{array}{c}\mathbf{A}\mathbf{\Phi }\\ {\mathbf{A}}^{*}{\mathbf{\Phi }}^{*}\end{array}\right]{\mathbf{s}}_R\left(t\right)+\left[\begin{array}{c}\mathbf{n}\left(t\right)\\ {\mathbf{n}}^{*}\left(t\right)\end{array}\right]\hfill \\ =\mathbf{E}{\mathbf{s}}_R\left(t\right)+{\mathbf{n}}_0\left(t\right),\end{array}$$

(5)

where $\mathbf{E}=\left[\mathbf{e}\left({\theta }_1,{\phi }_1\right),\cdots ,\mathbf{e}\left({\theta }_K,{\phi }_K\right)\right]\in {\mathbb{C}}^{2P\times K}$ is the extended direction matrix with $\mathbf{e}\left({\theta }_k,{\phi }_k\right)=\left[\begin{array}{c}\mathbf{a}\left({\theta }_k\right)e^{-j{\phi }_k}\\ {\mathbf{a}}^{*}\left({\theta }_k\right)e^{j{\phi }_k}\end{array}\right]$ the k-th extended steering vector and ${\mathbf{n}}_0\left(t\right)$ represents the extended noise. Then, the covariance matrix of the output matrix ${\mathbf{y}}_0\left(t\right)$ can be

$${\mathbf{R}}_y=\mathbb{E}\left\{{\mathbf{y}}_0\left(t\right){\mathbf{y}}_0^H\left(t\right)\right\}=\mathbf{E}{\mathbf{R}}_S{\mathbf{E}}^H+{\sigma }_n^2{\mathbf{I}}_{2\left(M+N\right)},$$

(6)

where ${\mathbf{R}}_S=\mathbb{E}\left\{{\mathbf{s}}_R\left(t\right){{\mathbf{s}}_R}^H\left(t\right)\right\}=diag\left\{{\delta }_1^2,{\delta }_2^2,\cdots ,{\delta }_K^2\right\}$ is the signal covariance matrix, where $\mathbb{E}$ denotes the expectation operator, and ${\mathbf{I}}_{2\left(M+N\right)}$ is the identity matrix. By ‘vectorizing’ ${\mathbf{R}}_y$, we obtain

$$\mathbf{y}\triangleq vec\left({\mathbf{R}}_y\right)=\left({\mathbf{E}}^{*}\odot \mathbf{E}\right)\mathbf{p}+{\sigma }_n^2{\mathbf{1}}_n,$$

(7)

where ${\mathbf{E}}^{*}\odot \mathbf{E}=\left[\dot{\mathbf{e}}\left({\theta }_1,{\phi }_1\right),\cdots ,\dot{\mathbf{e}}\left({\theta }_k,{\phi }_k\right),\cdots ,\dot{\mathbf{e}}\left({\theta }_K,{\phi }_K\right)\right]$ with $\dot{\mathbf{e}}\left({\theta }_k,{\phi }_k\right)={\mathbf{e}}^{*}\left({\theta }_k,{\phi }_k\right)\otimes \mathbf{e}\left({\theta }_k,{\phi }_k\right)$ is the k-th virtual array steering vector, $\mathbf{p}={\left[{\delta }_1^2,{\delta }_2^2,\cdots ,{\delta }_K^2\right]}^T$ is a single snapshot signal vector, ${\mathbf{1}}_n={\left[{\mathbf{i}}_1^T,\cdots ,{\mathbf{i}}_l^T,\cdots ,{\mathbf{i}}_{2\left(M+N\right)}^T\right]}^T$ with ${\mathbf{i}}_l^T\in {\mathbb{R}}^{2P\times 1},l=1,2,\cdots ,2P$ being a column vector of all zeros except 1 at the l-th position. The covariance matrix ${\mathbf{R}}_y$ is usually estimated from finite snapshots, i.e.,

$${\widehat{\mathbf{R}}}_y\approx \frac{1}{T}\sum _{t=1}^T{\mathbf{y}}_0\left(t\right){{\mathbf{y}}_0}^H\left(t\right).$$

(8)

According to Equations (6) and (8), there exists an approximation error [38,39]

$$\mathbf{\epsilon }\triangleq vec\left({\widehat{\mathbf{R}}}_y-{\mathbf{R}}_y\right)\sim \mathcal{C}\mathcal{N}\left(\mathbf{0},\frac{1}{T}{\mathbf{R}}_y^T\otimes {\mathbf{R}}_y\right).$$

(9)

Then, Equation (7) can be approximated as

$$\widehat{\mathbf{y}}\triangleq vec\left({\widehat{\mathbf{R}}}_y\right)=\left({\mathbf{E}}^{*}\odot \mathbf{E}\right)\mathbf{p}+{\sigma }_n^2{\mathbf{1}}_n+\mathbf{\epsilon }.$$

(10)

2.2. Difference and Sum Co-Arrays

The vectorization process corresponds to a sum-and-difference operation on the position information in the array steering vectors. The difference co-array is given by

$$\mathbb{D}=\left\{l_i-l_j,l_i,l_j\in L\right\},$$

(11)

and the positive and negative sum co-arrays are given as follows: [40]

$$\begin{array}{c}\hfill {\mathbb{S}}_{+}=\left\{l_i+l_j,l_i,l_j\in L\right\},\\ \hfill {\mathbb{S}}_{-}=\left\{-l_i-l_j,l_i,l_j\in L\right\}.\end{array}$$

(12)

According to [12], the virtual difference co-array formed by the nested array is a uniform linear array with a range of $〈-\left(MN+N-1\right),MN+N-1〉$. For the positive sum co-array, the range of the ULA is $〈0,MN+M+N-1〉$, while the negative sum co-array with ULA ranges from $-\left(MN+M+N-1\right)$ to 0. By introducing a virtual array of sum co-arrays and difference co-arrays, the continuous ULA can be denoted as

$$\begin{array}{c}〈-\left(MN+M+N-1\right),0〉\cup \\ 〈-\left(MN+N-1\right),MN+N-1〉\cup 〈0,MN+M+N-1〉.\end{array}$$

(13)

For example, when $M=3, N=3$. Figure 2 shows that the difference co-array is $〈-11,11〉d$, the continuous negative sum co-array is $〈-14,0〉d$, the continuous positive sum co-array is $〈0,14〉d$, and the ULA can be expressed as

$$〈-14,0〉d\cup 〈-11,11〉d\cup 〈0,14〉d.$$

(14)

3. The Proposed Method

3.1. Data Extension

From Equation (10), we have

$${\mathbf{E}}_0={\mathbf{E}}^{*}\odot \mathbf{E}=\left[\begin{array}{c}{\mathbf{A}}^{*}{\mathbf{\Phi }}^{*}\\ \mathbf{A}\mathbf{\Phi }\end{array}\right]\odot \left[\begin{array}{c}\mathbf{A}\mathbf{\Phi }\\ {\mathbf{A}}^{*}{\mathbf{\Phi }}^{*}\end{array}\right],$$

(15)

where ${\mathbf{E}}_0=\left[\dot{\mathbf{e}}\left({\theta }_1,{\phi }_1\right),\cdots ,\dot{\mathbf{e}}\left({\theta }_k,{\phi }_k\right),\cdots ,\dot{\mathbf{e}}\left({\theta }_K,{\phi }_K\right)\right]$, and

$$\dot{\mathbf{e}}\left({\theta }_k,{\phi }_k\right)=\left[\begin{array}{c}{\mathbf{a}}^{*}\left({\theta }_k\right)e^{j{\phi }_k}\\ \mathbf{a}\left({\theta }_k\right)e^{-j{\phi }_k}\end{array}\right]\otimes \left[\begin{array}{c}\mathbf{a}\left({\theta }_k\right)e^{-j{\phi }_k}\\ {\mathbf{a}}^{*}\left({\theta }_k\right)e^{j{\phi }_k}\end{array}\right].$$

(16)

For ease of computation, we introduce a row exchange matrix $\mathbf{J}\in {\mathbb{R}}^{4P^2\times 4P^2}$ [40], i.e.,

$$\mathbf{J}=\left[\begin{array}{cc}\left[\begin{array}{c}{\mathbf{J}}_1\\ {\mathbf{J}}_2\end{array}\right]& {\mathbf{0}}_{2P^2}\\ {\mathbf{0}}_{2P^2}& \left[\begin{array}{c}{\mathbf{J}}_1\\ {\mathbf{J}}_2\end{array}\right]\end{array}\right],$$

(17)

where ${\mathbf{J}}_1={\mathbf{I}}_P\otimes \left[\begin{array}{cc}{\mathbf{I}}_P& {\mathbf{0}}_P\end{array}\right]\in {\mathbb{R}}^{P^2\times 2P^2}$, and ${\mathbf{J}}_2={\mathbf{I}}_P\otimes \left[\begin{array}{cc}{\mathbf{0}}_P& {\mathbf{I}}_P\end{array}\right]\in {\mathbb{R}}^{P^2\times 2P^2}$. Through the row exchange matrix, the extended direction vector becomes

$$\begin{array}{c}\mathbf{c}\left({\theta }_k,{\phi }_k\right)=\mathbf{J}\dot{\mathbf{e}}\left({\theta }_k,{\phi }_k\right)=\hfill \\ \left[\begin{array}{c}{\mathbf{a}}^{*}\left({\theta }_k\right)\otimes \mathbf{a}\left({\theta }_k\right)\\ {\mathbf{a}}^{*}\left({\theta }_k\right)\otimes {\mathbf{a}}^{*}\left({\theta }_k\right)e^{2j{\phi }_k}\\ \mathbf{a}\left({\theta }_k\right)\otimes \mathbf{a}\left({\theta }_k\right)e^{-2j{\phi }_k}\\ \mathbf{a}\left({\theta }_k\right)\otimes {\mathbf{a}}^{*}\left({\theta }_k\right)\end{array}\right]=\left[\begin{array}{c}{\mathbf{c}}_1\\ {\mathbf{c}}_2\\ {\mathbf{c}}_3\\ {\mathbf{c}}_4\end{array}\right],\end{array}$$

(18)

where ${\mathbf{c}}_1={\mathbf{c}}_4^{*}$ is the steering vector of the difference co-array, ${\mathbf{c}}_2$ denotes the steering vector of the negative sum co-array, and ${\mathbf{c}}_3$ is the steering vector of the positive sum co-array. Then, $\mathbf{J}\widehat{\mathbf{y}}$ can be expressed as

$$\begin{array}{c}\mathbf{z}=\mathbf{J}\widehat{\mathbf{y}}=\mathbf{J}{\mathbf{E}}_0\mathbf{p}+\mathbf{J}{\sigma }_n^2{\mathbf{1}}_n+\mathbf{J}\mathbf{\epsilon }\hfill \\ =\mathbf{Cp}+{\sigma }_n^2{{\mathbf{1}}^{\prime }}_n+\tilde{\mathbf{\epsilon }},\hfill \end{array}$$

(19)

where $\mathbf{C}=\mathbf{J}{\mathbf{E}}_0=\left[\mathbf{c}\left({\theta }_1,{\phi }_1\right),\cdots ,\mathbf{c}\left({\theta }_k,{\phi }_k\right),\cdots ,\mathbf{c}\left({\theta }_K,{\phi }_K\right)\right]$ means a direction matrix of virtual difference co-arrays and sum co-arrays, $\tilde{\mathbf{\epsilon }}=\mathbf{J}\mathbf{\epsilon }$, and ${{\mathbf{1}}^{\prime }}_n=\mathbf{J}{\mathbf{1}}_n$.

3.2. Sparse Bayesian Inference for DOA Estimation

The received data in Equation (2) can be solved by a sparse approximation method [21]. Let $\Theta =\left[{\widehat{\theta }}_1,{\widehat{\theta }}_2,\cdots ,{\widehat{\theta }}_I\right]$, where I represents the number of sampling grids, which satisfies $I\gg K$ and $r={\widehat{\theta }}_I-{\widehat{\theta }}_{I-1}$ is the grid resolution. As [27], the over-complete basis can be written as

$$\widehat{\mathbf{z}}={\overline{\mathbf{C}}}_0\overline{\mathbf{p}}+{\sigma }_n^2{{\mathbf{1}}^{\prime }}_n+\overline{\mathbf{\epsilon }}=\left[\begin{array}{cc}{\overline{\mathbf{C}}}_0& {{\mathbf{1}}^{\prime }}_n\end{array}\right]\left[\begin{array}{c}\overline{\mathbf{p}}\\ {\sigma }_n^2\end{array}\right]+\overline{\mathbf{\epsilon }},$$

(20)

where ${\overline{\mathbf{C}}}_0=\left[\mathbf{c}\left({\theta }_1,{\phi }_1\right),\cdots ,\mathbf{c}\left({\theta }_{I_k},{\phi }_{I_k}\right),\cdots ,\mathbf{c}\left({\theta }_I,{\phi }_I\right)\right]$ with $\mathbf{c}\left({\theta }_{I_k},{\phi }_{I_k}\right)=[{\mathbf{a}}^{*}\left({\theta }_{I_k}\right)\otimes \mathbf{a}\left({\theta }_{I_k}\right);$${\mathbf{a}}^{*}\left({\theta }_{I_k}\right)\otimes {\mathbf{a}}^{*}\left({\theta }_{I_k}\right)e^{2j{\phi }_{I_k}};\mathbf{a}\left({\theta }_{I_k}\right)\otimes \mathbf{a}\left({\theta }_{I_k}\right)e^{-2j{\phi }_{I_k}};\mathbf{a}\left({\theta }_{I_k}\right)\otimes {\mathbf{a}}^{*}\left({\theta }_{I_k}\right)]$, $\overline{\mathbf{p}}$ is a zero-padded extension matrix of $\mathbf{p}$ whose non-zero elements correspond to the true DOA at $\left\{{\theta }_k,k=1,\cdots ,K\right\}$, and $\overline{\mathbf{\epsilon }}\sim \mathcal{C}\mathcal{N}\left(0,\mathbf{W}\right),\mathbf{W}=\frac{\mathbf{J}\left({\mathbf{R}}_Y^T\otimes {\mathbf{R}}_Y\right){\mathbf{J}}^T}{T}$.

Ideally, DOA is assumed to be on the grid. However, in practical scenarios, this phenomenon often only exists sometimes. To deal with the problem of the grid-gap, a method based on the linear approximation of the first-order Taylor is proposed in [23]. Assuming that ${\widehat{\theta }}_{I_k},I_k\in \left\{1,2,\cdots ,I\right\}$ is the nearest grid point to a DOA ${\theta }_k\notin \Theta$, the steering vector can be linearized as

$$\mathbf{c}\left({\theta }_k,{\phi }_k\right)\approx \mathbf{c}\left({\widehat{\theta }}_{I_k},{\phi }_{I_k}\right)+\mathbf{b}\left({\widehat{\theta }}_{I_k},{\phi }_{I_k}\right)\left({\theta }_k-{\widehat{\theta }}_{I_k}\right),$$

(21)

where $\mathbf{b}\left({\widehat{\theta }}_{I_k},{\phi }_{I_k}\right)={\mathbf{c}}^{\prime }\left({\widehat{\theta }}_{I_k},{\phi }_{I_k}\right)$ is the first-order derivative of $\mathbf{c}\left({\widehat{\theta }}_{I_k},{\phi }_k\right)$ to ${\widehat{\theta }}_{I_k}$. Equation (20) can be rewritten as

$$\widehat{\mathbf{z}}=\left[\begin{array}{cc}{\overline{\mathbf{C}}}_0\left(\mathit{\beta }\right)& {{\mathbf{1}}^{\prime }}_n\end{array}\right]\left[\begin{array}{c}\overline{\mathbf{p}}\\ {\sigma }_n^2\end{array}\right]+\overline{\mathbf{\epsilon }},$$

(22)

where $\mathbf{B}=\left[\mathbf{b}\left({\widehat{\theta }}_1,{\phi }_1\right),\cdots ,\mathbf{b}\left({\widehat{\theta }}_{I_k},{\phi }_{I_k}\right),\cdots ,\mathbf{b}\left({\widehat{\theta }}_I,{\phi }_I\right)\right]$ and ${\overline{\mathbf{C}}}_0\left(\mathit{\beta }\right)={\overline{\mathbf{C}}}_0+\mathbf{B}diag\left\{\mathit{\beta }\right\}$, $\mathit{\beta }={\left[{\beta }_1,{\beta }_2,\cdots ,{\beta }_I\right]}^T$ is a zero vector except that the $I_k$-th element ${\left[\beta \right]}_{I_k}={\theta }_k-{\widehat{\theta }}_{I_k},$ $k=1,2,\cdots ,K$.

According to Equation (18), $\widehat{\mathbf{z}}$ can be divided into four parts, i.e., $\widehat{\mathbf{z}}=\left[\begin{array}{cccc}{\mathbf{z}}_1;& {\mathbf{z}}_2;& {\mathbf{z}}_3;& {\mathbf{z}}_4\end{array}\right]$. Since there are holes in sum co-arrays, we only select continuous virtual array elements, i.e., ${\mathbb{S}}_{-}^{\prime }=〈-\left(MN+M+N-1\right),0〉$ and ${\mathbb{S}}_{+}^{\prime }=〈0,MN+M+N-1〉$. We remove the redundancy items on ${\mathbf{z}}_1,{\mathbf{z}}_2,{\mathbf{z}}_3,{\mathbf{z}}_4$, which depend on select matrices ${\mathbf{F}}_1$, ${\mathbf{F}}_2$, ${\mathbf{F}}_3$, and ${\mathbf{F}}_4$. We define the weight functions for the virtual array locations $\mathbb{D}$, ${\mathbb{S}}_{-}^{\prime }$ and ${\mathbb{S}}_{+}^{\prime }$ as

$$\begin{array}{c}{\omega }_1=〈\left\{\mathrm{card}\left({\mu }_1\right)|{\mu }_1=l_i-l_j,l_i,l_j\in L,{\mu }_1\in \mathbb{D}\right\}〉\hfill \\ {\omega }_2=〈\left\{\mathrm{card}\left({\mu }_2\right)|{\mu }_2=-l_i-l_j,l_i,l_j\in L,{\mu }_2\in {{\mathbb{S}}^{\prime }}_{-}\right\}〉\hfill \\ {\omega }_3=〈\left\{\mathrm{card}\left({\mu }_3\right)|{\mu }_3=l_i+l_j,l_i,l_j\in L,{\mu }_3\in {{\mathbb{S}}^{\prime }}_{+}\right\}〉\hfill \\ {\omega }_4=〈\left\{\mathrm{card}\left({\mu }_4\right)|{\mu }_4=l_i-l_j,l_i,l_j\in L,{\mu }_4\in \mathbb{D}\right\}〉\hfill \end{array}$$

(23)

where $\mathrm{card}\left(\mu \right)$ is the number of elements $\mu$. Let $D_v=MN+N-1,S_v=MN+M+N-1$, $i,j=1,2,\cdots ,P$, then the corresponding selection matrix can be expressed as follows:

$$\begin{array}{c}{\left[{\mathbf{F}}_1\right]}_{{\mu }_1+D_v+1,i+P\left(j-1\right)}=\left\{\begin{array}{c}\frac{1}{{\omega }_1\left({\mu }_1\right)},{\mu }_1=l_i-l_j\\ 0,otherwise\end{array}\right.\hfill \\ {\left[{\mathbf{F}}_2\right]}_{{\mu }_2+S_v+1,i+P\left(j-1\right)}=\left\{\begin{array}{c}\frac{1}{{\omega }_2\left({\mu }_2\right)},{\mu }_2=-l_i-l_j\\ 0,otherwise\end{array}\right.\hfill \\ {\left[{\mathbf{F}}_3\right]}_{{\mu }_3+1,i+P\left(j-1\right)}=\left\{\begin{array}{c}\frac{1}{{\omega }_3\left({\mu }_3\right)},{\mu }_3=l_i+l_j\\ 0,otherwise\end{array}\right.\hfill \\ {\left[{\mathbf{F}}_4\right]}_{{\mu }_4+D_v+1,i+P\left(j-1\right)}=\left\{\begin{array}{c}\frac{1}{{\omega }_4\left({\mu }_4\right)},{\mu }_4=l_j-l_i\\ 0,otherwise\end{array}\right..\end{array}$$

(24)

By removing the redundancy items of $\widehat{\mathbf{z}}$, we can obtain $\tilde{\mathbf{z}}$, where ${\tilde{\mathbf{z}}}_1={\tilde{\mathbf{z}}}_4^{*}\in {\mathbb{C}}^{(2D_v+1)\times 1}$ represents the difference co-array data. ${\tilde{\mathbf{z}}}_2={\tilde{\mathbf{z}}}_3^{*}\in {\mathbb{C}}^{\left(S_v+1\right)\times 1}$, ${\tilde{\mathbf{z}}}_2$ denotes the negative sum co-array data while ${\tilde{\mathbf{z}}}_3$ represents the positive sum co-array data. Hence, all valid received data can be expressed as $\tilde{\mathbf{z}}=\left[{\tilde{\mathbf{z}}}_1;{\tilde{\mathbf{z}}}_2;{\tilde{\mathbf{z}}}_3;{\tilde{\mathbf{z}}}_4\right]\in {\mathbb{C}}^{(4D_v+2S_v+4)\times 1}$, i.e.,

$$\begin{array}{c}\tilde{\mathbf{z}}=\mathbf{F}\widehat{\mathbf{z}}\hfill \\ =\left[\begin{array}{cccc}{\mathbf{F}}_1& & & \\ & {\mathbf{F}}_2& & \\ & & {\mathbf{F}}_3& \\ & & & {\mathbf{F}}_4\end{array}\right]\left[\begin{array}{c}{\mathbf{z}}_1\\ {\mathbf{z}}_2\\ {\mathbf{z}}_3\\ {\mathbf{z}}_4\end{array}\right]=\left[\begin{array}{c}{\tilde{\mathbf{z}}}_1\\ {\tilde{\mathbf{z}}}_2\\ {\tilde{\mathbf{z}}}_3\\ {\tilde{\mathbf{z}}}_4\end{array}\right].\end{array}$$

(25)

According to Equations (22) and (25), we have

$$\tilde{\mathbf{z}}=\mathbf{F}\underset{\mathbf{\Psi }\left(\mathit{\beta }\right)}{\underbrace{\left[\begin{array}{cc}{\overline{\mathbf{C}}}_0\left(\mathit{\beta }\right)& {{\mathbf{1}}^{\prime }}_n\end{array}\right]}}\underset{\mathbf{d}}{\underbrace{\left[\begin{array}{c}\overline{\mathbf{p}}\\ {\sigma }_n^2\end{array}\right]}}+\mathbf{F}\tilde{\mathbf{\epsilon }}=\mathbf{F}\mathbf{\Psi }\left(\mathit{\beta }\right)\mathbf{d}+\overrightarrow{\mathbf{\epsilon }},$$

(26)

where $\mathbf{d}$ represents the sparse signal, and $\overrightarrow{\mathbf{\epsilon }}$, after removing the redundancy, satisfies a complex Gaussian distribution, i.e.,

$$\overrightarrow{\mathbf{\epsilon }}=\mathbf{F}·\left[{\mathbf{W}}^{-\frac{1}{2}}\left[\widehat{\mathbf{z}}-\mathbf{\Psi }\left(\mathit{\beta }\right)\mathbf{d}\right]\right]\sim \mathcal{C}\mathcal{N}\left(0,{\mathbf{I}}_{(4D_v+2S_v+4)}\right).$$

(27)

A typical SBL treatment of $\mathbf{d}$ involves assigning a non-stationary Gaussian prior distribution with variance ${\delta }_i$ to each $\mathbf{d}$ element. Suppose that hyperparameter $\Lambda =diag\left(\mathit{\delta }\right)$, and $p\left(\mathbf{d}|\mathit{\delta }\right)=\mathcal{C}\mathcal{N}\left(\mathbf{d}|\mathbf{0},\Lambda \right)$ with $\mathit{\delta }={\left[{\delta }_1,\cdots ,{\delta }_i,\cdots ,{\delta }_{I+1}\right]}^T$, where $\mathit{\delta }$ can be modeled as a Gamma distribution [23], i.e.,

$$\Gamma \left(\mathit{\delta }\right)=\prod _{i=1}^{I+1}\Gamma \left({\delta }_i;1,\upsilon \right),$$

(28)

where $\upsilon$ is a small positive constraint (e.g., $\upsilon$ = 0.01 [23,41]). From Equation (26), we have

$$p\left(\tilde{\mathbf{z}}|\mathbf{d},\mathit{\beta }\right)=\mathcal{C}\mathcal{N}\left(\tilde{\mathbf{z}}|\mathbf{F}\mathbf{\Psi }\left(\mathit{\beta }\right)\mathbf{d},\mathbf{Q}\right),$$

(29)

where $\mathbf{Q}={\mathbf{FWF}}^T$. Assuming that $\mathbf{d}$ is a hidden variable, the posterior probability density [23,26] is

$$p\left(\mathbf{d}|\tilde{\mathbf{z}},\mathit{\delta },\mathit{\beta }\right)=\mathcal{C}\mathcal{N}\left(\mathbf{d}|\mathit{\mu },\mathbf{\Sigma }\right),$$

(30)

where

$$\mathit{\mu }=\mathbf{\Sigma }\left({\mathbf{\Psi }}^H\left(\mathit{\beta }\right){\mathbf{F}}^T\right){\mathbf{Q}}^{-1}\tilde{\mathbf{z}},$$

(31)

$$\mathbf{\Sigma }={\left({\mathbf{\Psi }}^H\left(\mathit{\beta }\right){\mathbf{F}}^T{\mathbf{Q}}^{-1}\mathbf{F}\mathbf{\Psi }\left(\mathit{\beta }\right)+{\Lambda }^{-1}\right)}^{-1}.$$

(32)

Then, the EM algorithm [23] can be applied for recursive calculating until it reaches a prescribed accuracy. In the E-step, we need to have the lower bound of $lnp\left(\tilde{\mathbf{z}},\mathit{\delta },\mathit{\beta }\right)$, which is

$$\begin{array}{c}\mathcal{L}\left(\mathit{\delta };\mathit{\beta }\right)=\mathbb{E}{\left\{lnp\left(\mathbf{d},\tilde{\mathbf{z}},\mathit{\delta },\mathit{\beta }\right)\right\}}_{p\left(\mathbf{d}|\tilde{\mathbf{z}},\mathit{\delta },\mathit{\beta }\right)}\hfill \\ =\mathbb{E}{\left\{lnp\left(\tilde{\mathbf{z}}|\mathbf{d},\mathit{\beta }\right)p\left(\mathbf{d}|\mathit{\delta }\right)p\left(\mathit{\delta }\right)\right\}}_{p\left(\mathbf{d}|\tilde{\mathbf{z}},\mathit{\delta },\mathit{\beta }\right)},\end{array}$$

(33)

where $\mathbb{E}{\left\{·\right\}}_{p\left(x\right)}$ represents the expected value of $p\left(x\right)$. In the M-step, the hyperparameter updates that maximize the lower bound function

$$\left({\mathit{\delta }}^{new},{\mathit{\beta }}^{new}\right)=arg\underset{\mathit{\delta },\mathit{\beta }}{max}\mathcal{L}\left(\mathit{\delta };\mathit{\beta }\right).$$

(34)

According to [27], the hyperparameter updates for ${\mathit{\delta }}_i$ can be simplified as

$${\mathit{\delta }}_i^{new}=\frac{-1+\sqrt{1+4\upsilon {\left[\mathit{\mu }{\mathit{\mu }}^H+diag\left(\mathbf{\Sigma }\right)\right]}_{ii}}}{2\upsilon },i=1,2,\cdots ,I+1,$$

(35)

where ${\left[·\right]}_{ii}$ denotes the $\left(i,i\right)$-th element of the matrix. However, since the structure of the construction matrix $\mathbf{F}\mathbf{\Psi }\left(\mathit{\beta }\right)$ is different in [23], we need to redefine the update of ${\mathit{\beta }}^{new}$.

3.3. Grid Refining

Ignoring the independent terms in Equation (33), we obtain

$$\begin{array}{c}\mathbb{E}{\left\{lnp\left(\tilde{\mathbf{z}}|\mathbf{d},\mathit{\beta }\right)\right\}}_{p\left(\mathbf{d}|\tilde{\mathbf{z}},\mathit{\delta },\mathit{\beta }\right)}\hfill \\ =-\mathbb{E}{\left\{{\left(\tilde{\mathbf{z}}-\mathbf{F}\mathbf{\Psi }\left(\mathit{\beta }\right)\mathbf{d}\right)}^H{\mathbf{Q}}^{-1}\left(\tilde{\mathbf{z}}-\mathbf{F}\mathbf{\Psi }\left(\mathit{\beta }\right)\mathbf{d}\right)\right\}}_{p\left(\mathbf{d}|\tilde{\mathbf{z}},\mathit{\delta },\mathit{\beta }\right)}\hfill \\ =-\mathbb{E}{\left\{{∥{\mathbf{Q}}^{-1/2}·\left(\tilde{\mathbf{z}}-\mathbf{F}\mathbf{\Psi }\left(\mathit{\beta }\right)\mathbf{d}\right)∥}_2^2\right\}}_{p\left(\mathbf{d}|\tilde{\mathbf{z}},\mathit{\delta },\mathit{\beta }\right)}\hfill \\ =-\underset{a}{\underbrace{{∥{\mathbf{Q}}^{-1/2}\tilde{\mathbf{z}}-{\mathbf{Q}}^{-1/2}\mathbf{F}\mathbf{\Psi }\left(\mathit{\beta }\right)\mathit{\mu }∥}_2^2}}-\underset{b}{\underbrace{tr\left({\mathbf{Q}}^{-1}\mathbf{F}\mathbf{\Psi }\left(\mathit{\beta }\right)\mathbf{\Sigma }{\mathbf{\Psi }}^H\left(\mathit{\beta }\right){\mathbf{F}}^T\right)}}+const.\hfill \end{array}$$

(36)

The term a in the above formula can be simplified to Equation (37),

$$\begin{array}{c}{∥{\mathbf{Q}}^{-1/2}\tilde{\mathbf{z}}-{\mathbf{Q}}^{-1/2}\mathbf{F}\mathbf{\Psi }\left(\mathit{\beta }\right)\mathit{\mu }∥}_2^2\hfill \\ ={∥\underset{{\mathbf{Z}}_{\mathbf{w}}}{\underbrace{{\mathbf{Q}}^{-1/2}\tilde{\mathbf{z}}}}-\left[{\mathbf{Q}}^{-1/2}\mathbf{F}{\overline{\mathbf{C}}}_{\mathbf{0}}\left(\mathit{\beta }\right)\right]{\mathit{\mu }}_I-{\mu }_0\underset{{\mathbf{I}}_{\mathbf{w}}}{\underbrace{{\mathbf{Q}}^{-1/2}\mathbf{F}{{\mathbf{1}}^{\prime }}_n}}∥}_2^2\hfill \\ ={∥\underset{{\mathbf{Z}}_{\mathbf{w}}}{\underbrace{{\mathbf{Q}}^{-1/2}\tilde{\mathbf{z}}}}-\left[\underset{{\mathbf{C}}_{\mathbf{w}}}{\underbrace{{\mathbf{Q}}^{-1/2}\mathbf{F}{\overline{\mathbf{C}}}_0}}+\underset{{\mathbf{B}}_{\mathbf{w}}}{\underbrace{{\mathbf{Q}}^{-1/2}\mathbf{FB}}}\mathrm{diag}\left\{\mathit{\beta }\right\}\right]{\mathit{\mu }}_I-{\mu }_0\underset{{{\mathbf{1}}^{\prime }}_{\mathbf{w}}}{\underbrace{{\mathbf{Q}}^{-1/2}\mathbf{F}{{\mathbf{1}}^{\prime }}_n}}∥}_2^2\hfill \\ ={∥{\mathbf{Z}}_{\mathbf{w}}-{\mathbf{C}}_{\mathbf{w}}{\mathit{\mu }}_I-{\mathbf{B}}_{\mathbf{w}}\mathrm{diag}\left\{\mathit{\beta }\right\}{\mathit{\mu }}_I-{\mu }_0{{\mathbf{1}}^{\prime }}_{\mathbf{w}}∥}_2^2\hfill \\ ={\mathit{\beta }}^T\left({\mathbf{B}}_{\mathbf{w}}^T{\mathbf{B}}_{\mathbf{w}}^{*}\oplus {\mathit{\mu }}_I{\mathit{\mu }}_I^H\right)\mathit{\beta }-2Re{\left\{\mathrm{diag}\left\{{\mathit{\mu }}_I^{*}\right\}{\mathbf{B}}_{\mathbf{w}}^H\left({\mathbf{Z}}_{\mathbf{w}}-{\mathbf{C}}_{\mathbf{w}}{\mathit{\mu }}_I-{\mu }_0{\mathbf{I}}_{\mathbf{w}}\right)\right\}}^T\mathit{\beta }+const\hfill \end{array}$$

(37)

where ${\mathit{\mu }}_I$ represents the sub-vector of $\mathit{\mu }$, whose elements are indexed from 1 to I, and ${\mu }_0$ represents the end element of $\mathit{\mu }$, i.e., $\mathit{\mu }={\left[\begin{array}{cccc}{\mu }_1& \cdots & {\mu }_I& {\mu }_0\end{array}\right]}^T={\left[\begin{array}{cc}{\mathit{\mu }}_I& {\mu }_0\end{array}\right]}^T$. Let $\mathbf{\Sigma }=\left[\begin{array}{cc}{\rm Y}& \mathbf{\gamma }\\ {\mathbf{\gamma }}^H& {\gamma }_0\end{array}\right]$.

Similarly, term b can be rewritten as Equation (38).

$$\begin{array}{c}tr\left({\mathbf{Q}}^{-1}\mathbf{F}\mathbf{\Psi }\left(\mathit{\beta }\right)\mathbf{\Sigma }{\mathbf{\Psi }}^H\left(\mathit{\beta }\right){\mathbf{F}}^T\right)\hfill \\ =tr\left({\left(\mathbf{F}\mathbf{\Psi }\left(\mathit{\beta }\right)\right)}^H·{\left({\mathbf{Q}}^{-1/2}\right)}^H{\mathbf{Q}}^{-1/2}\mathbf{F}\mathbf{\Psi }\left(\mathit{\beta }\right)\mathbf{\Sigma }\right)\hfill \\ =tr\left({\left[{\mathbf{Q}}^{-1/2}\left(\left[\mathbf{F}{\overline{\mathbf{C}}}_0+\mathbf{FB}\mathrm{diag}\left\{\mathit{\beta }\right\},\mathbf{F}{{\mathbf{1}}^{\prime }}_n\right]\right)\right]}^H\left[{\mathbf{Q}}^{-1/2}\left[\mathbf{F}{\overline{\mathbf{C}}}_0+\mathbf{FB}\mathrm{diag}\left\{\mathit{\beta }\right\},\mathbf{F}{{\mathbf{1}}^{\prime }}_n\right]\right]\mathbf{\Sigma }\right)\hfill \\ =tr\left({\left[{\mathbf{C}}_{\mathbf{w}}+{\mathbf{B}}_{\mathbf{w}}\mathrm{diag}\left\{\mathit{\beta }\right\},{{\mathbf{1}}^{\prime }}_{\mathbf{w}}\right]}^H\left[{\mathbf{C}}_{\mathbf{w}}+{\mathbf{B}}_{\mathbf{w}}\mathrm{diag}\left\{\mathit{\beta }\right\},{{\mathbf{1}}^{\prime }}_{\mathbf{w}}\right]\mathbf{\Sigma }\right)\hfill \\ =tr\left({\left[{\mathbf{C}}_{\mathbf{w}}+{\mathbf{B}}_{\mathbf{w}}\mathrm{diag}\left\{\mathit{\beta }\right\},{{\mathbf{1}}^{\prime }}_{\mathbf{w}}\right]}^H\left[{\mathbf{C}}_{\mathbf{w}}{\rm Y}+{\mathbf{B}}_{\mathbf{w}}\mathrm{diag}\left\{\mathit{\beta }\right\}{\rm Y}+{{\mathbf{1}}^{\prime }}_{\mathbf{w}}{\mathbf{\gamma }}^H,{\mathbf{C}}_{\mathbf{w}}\mathbf{\gamma }+{\mathbf{B}}_{\mathbf{w}}\mathrm{diag}\left\{\mathit{\beta }\right\}\mathbf{\gamma }+{{\mathbf{1}}^{\prime }}_{\mathbf{w}}{\gamma }_0\right]\right)\hfill \\ =tr\left({\left({\mathbf{C}}_{\mathbf{w}}+{\mathbf{B}}_{\mathbf{w}}\mathrm{diag}\left\{\mathit{\beta }\right\}\right)}^H\left({\mathbf{C}}_{\mathbf{w}}{\rm Y}+{\mathbf{B}}_{\mathbf{w}}\mathrm{diag}\left\{\mathit{\beta }\right\}{\rm Y}+{{\mathbf{1}}^{\prime }}_{\mathbf{w}}{\mathbf{\gamma }}^H\right)\right)+{\left({{\mathbf{1}}^{\prime }}_{\mathbf{w}}\right)}^H{\mathbf{B}}_{\mathbf{w}}\mathrm{diag}\left\{\mathit{\beta }\right\}\mathbf{\gamma }+const\hfill \\ =2Re\left\{tr\left({\mathbf{B}}_{\mathbf{w}}^H{\mathbf{C}}_{\mathbf{w}}{\rm Y}\mathrm{diag}\left\{\mathit{\beta }\right\}\right)\right\}+tr\left(\mathrm{diag}\left\{\mathit{\beta }\right\}{\rm Y}\mathrm{diag}\left\{\mathit{\beta }\right\}{\mathbf{B}}_{\mathbf{w}}^H{\mathbf{B}}_{\mathbf{w}}\right)\hfill \\ +tr\left(\mathrm{diag}\left\{\mathit{\beta }\right\}{\mathbf{B}}_{\mathbf{w}}^H{{\mathbf{1}}^{\prime }}_{\mathbf{w}}{\mathbf{\gamma }}^H\right)+{\left({{\mathbf{1}}^{\prime }}_{\mathbf{w}}\right)}^H{\mathbf{B}}_{\mathbf{w}}\mathrm{diag}\left\{\mathit{\beta }\right\}\mathbf{\gamma }+const\hfill \\ =2Re\left\{{\left(\mathrm{diag}\left\{{\mathbf{B}}_{\mathbf{w}}^H{\mathbf{C}}_{\mathbf{w}}{\rm Y}\right\}\right)}^T+{\left({{\mathbf{1}}^{\prime }}_{\mathbf{w}}\right)}^H{\mathbf{B}}_{\mathbf{w}}\mathrm{diag}\left\{\mathbf{\gamma }\right\}\right\}\mathit{\beta }+{\mathit{\beta }}^T\left({\rm Y}\oplus \left({\mathbf{B}}_{\mathbf{w}}^T{\mathbf{B}}_{\mathbf{w}}^{*}\right)\right)\mathit{\beta }+const\hfill \end{array}$$

(38)

Substituting Equations (37) and (38) into Equation (36), we have

$$\begin{array}{c}\mathbb{E}{\left\{lnp\left(\tilde{\mathbf{z}}|\mathbf{d},\mathit{\beta }\right)\right\}}_{p\left(\mathbf{d}|\overline{\mathbf{Z}},\mathit{\delta },\mathit{\beta }\right)}\hfill \\ =-{\mathit{\beta }}^T\mathit{P}\mathit{\beta }+2{\mathbf{v}}^T\mathit{\beta }+const\hfill \end{array}$$

(39)

with

$$\mathbf{P}=Re\left\{{\mathbf{B}}_{\mathbf{w}}^T{\mathbf{B}}_{\mathbf{w}}^{*}\oplus \left({\mathit{\mu }}_I{\mathit{\mu }}_I^H+{\rm Y}\right)\right\},$$

(40)

$$\begin{array}{c}\mathbf{v}=Re\left\{\mathrm{diag}\left({\mathit{\mu }}_I^{*}\right){\mathbf{B}}_{\mathbf{w}}^H\left({\mathbf{Z}}_{\mathbf{w}}-{\mathbf{C}}_{\mathbf{w}}{\mathit{\mu }}_I-{\mu }_0{{\mathbf{1}}^{\prime }}_{\mathbf{w}}\right)\right\}\hfill \\ -Re\left\{\mathrm{diag}\left({\mathbf{B}}_{\mathbf{w}}^H{\mathbf{C}}_{\mathbf{w}}{\rm Y}\right)\right\}+\mathrm{diag}\left(\mathbf{\gamma }\right){\mathbf{B}}_{\mathbf{w}}^T{\left({{\mathbf{1}}^{\prime }}_{\mathbf{w}}\right)}^{*}.\hfill \end{array}$$

(41)

Find the partial derivative to $\mathit{\beta }$ in Equation (39), then

$$\mathit{\beta }={\mathbf{P}}^{-1}\mathbf{v}.$$

(42)

If $\mathbf{P}$ is invertible, Equation (42) holds. Otherwise, ${\beta }_i=\frac{{\mathbf{v}}_i}{{\mathbf{P}}_{ii}},\forall i$. Reference [27] proposes a grid update method to reduce the interval error from the grid. Similarly, we define the grid to update as follows:

$${\widehat{\theta }}_{I_k}^{new}=\left\{\begin{array}{c}{\widehat{\theta }}_{I_k},{\widehat{\theta }}_{I_k}+{\beta }_{I_k}\notin \left[{\widehat{\theta }}_{I_k-1},{\widehat{\theta }}_{I_k+1}\right]\\ {\widehat{\theta }}_{I_k}+{\beta }_{I_k},{\widehat{\theta }}_{I_k}+{\beta }_{I_k}\in \left[{\widehat{\theta }}_{I_k-1},{\widehat{\theta }}_{I_k+1}\right]\end{array}\right..$$

(43)

Then, the $\mathbf{\Psi }\left(\mathit{\beta }\right)$ matrix can provide a better approximation of the actual steering matrix; we update the $\mathbf{\Psi }\left(\mathit{\beta }\right)$ matrix with the new grid ${\widehat{\theta }}_{I_k}^{new}$ so that the next iteration is closer to the true value, i.e., $\mathbf{\Psi }\left(\mathit{\beta }\right)=\left[\begin{array}{cc}{\overline{\mathbf{C}}}_0+\mathbf{B}\mathrm{diag}\left({\mathit{\beta }}^{new}\right)& {{\mathbf{1}}^{\prime }}_n\end{array}\right]$.

Table 2. The pseudocode of the proposed algorithm.

Input: ${\mathit{y}}_0\left(t\right),t=1,2,\cdots ,t$,${\overline{\mathbf{C}}}_0\left(\mathit{\beta }\right)$ and ${1^{\prime }}_n$.
Output: Parameter estimator: ${\mathit{\delta }}^{new}$, ${\mathit{\beta }}^{new}$ and ${\widehat{\theta }}_{I_k}^{new}$.
1: Initialization: Set $\mathbf{\Psi }\left(\mathit{\beta }\right)=\left[\begin{array}{cc}{\overline{\mathbf{C}}}_{\mathbf{0}}\left(\mathit{\beta }\right)& {{\mathbf{1}}^{\prime }}_n\end{array}\right]$, ${\mathit{\beta }}_0=1$ and ${\mathit{\delta }}_0={\mathbf{0}}_{I+1}$.
2: Calculate the covariance matrix: ${\widehat{\mathbf{R}}}_Y=\frac{1}{T}\sum _{t=1}^T{\mathbf{y}}_0\left(t\right){{\mathbf{y}}_0}^H\left(t\right)$.
3: Vectorize ${\widehat{\mathbf{R}}}_Y$, obtain $\widehat{\mathbf{y}}$    according to Equation (10). Then, multiply the row
exchange matrix $\mathbf{J}$ to obtain Equation (19).
4: Construct the over-complete information $\widehat{\mathbf{z}}$ according to Equation (22).
5: Remove the redundancy items according to Equation (25), we can obtain $\tilde{\mathbf{z}}$.
6: Calculate the weight matrix $\mathbf{W}$, use it to normalize the vectorized covariance matrix, and then go through the remove redundancy matrix $\mathbf{F}$ to obtain Equation (27).
7: Build $d$ and $\mathbf{\Psi }\left(\mathit{\beta }\right)$ based on the current values of $\mathit{\delta }$ and $\mathit{\beta }$ separately.
8: While$\frac{{∥{\mathit{\delta }}_t-{\mathit{\delta }}_{t-1}∥}_2}{{∥{\mathit{\delta }}_t∥}_2}⩽{10}^{-3}$, do
Calculate the mean $\mathit{\mu }$ and covariance $\mathbf{\Sigma }$ according to Equation (31) and Equation (32), respectively.
Update the $\mathit{\delta }$ according to Equation (35), respectively.
Calculate the $\mathit{P}$ and $\mathit{v}$.
Update the $\mathit{\beta }$ according to Equation (42).
Update the grid ${\widehat{\theta }}_{I_k}^{new}$ according to Equation (43).
Update $\mathbf{\Psi }\left(\mathit{\beta }\right)$.
9: end

illustrates the pseudocode of the proposed DOA estimation algorithm.

4. Numerical Simulation

To evaluate the performance of the proposed algorithm, we compare the proposed algorithm with the SS-MUSIC method [12], the $l_1$-SVD method [17], the OGSBI-ULA method [23], and the OGSBI-NA method [27]. The Cramér–Rao bound (CRB) for circular and NC signals [42] is considered for comparison. To compare the performance of the proposed algorithm with those reported recently, we assume that the NC phases of all signals are consistent ${\phi }_1={\phi }_2=\cdots ={\phi }_K=5^{\circ }$. ${\delta }_k^2$ is the power of the k-th signal, and ${\sigma }_n^2$ is the noise power, and the signal-to-noise ratio $\mathrm{SNR}=10{log}_{10}\left({\delta }_k^2/{\sigma }_n^2\right)$. We employ the root mean square error (RMSE) to evaluate the performance of DOA estimation, which is defined as

$$\mathrm{RMSE}=\sqrt{\frac{1}{\mathrm{MC}K}\left(\sum _{i=1}^{\mathrm{MC}}\sum _{k=1}^K{\left({\widehat{\theta }}_{k,i}-{\theta }_k\right)}^2\right)},$$

(44)

where ${\widehat{\theta }}_{k,i}$ is the estimation value of the k-th signal in the i-th Monte Carlo (MC), and ${\theta }_k$ represents the true DOA of the k-th signal.

4.1. Computational Complexity

In this section, we use the number of multiplications of real (or complex) numbers as a criterion for the complexity. The pseudocode of the proposed algorithm is given in Table 2, and the computational complexity of the proposed algorithm consists of many components, such as the nested array non-circular technique, the OGSBI algorithm, and the grid refining operation. In order to facilitate the reader’s understanding, we provide the main computational complexities of the proposed algorithms (computational expressions and the corresponding computational complexity) in

Table 3. Computational expressions and computational complexity.

Computational Expressions	Computational Complexity
$\mathbf{Q}=\mathbf{FW}{\mathbf{F}}^T$	$O\left\{4P^{2}G(4P^2+G)\right\}$, $G=4D_v+2S_v+4$
${\mathbf{Q}}^{-1}$	$O\left\{G^3\right\}$
$\mathit{\mu }=\mathbf{\Sigma }\left({\mathbf{\Psi }}^H\left(\mathit{\beta }\right){\mathbf{F}}^T\right){\mathbf{Q}}^{-1}\tilde{\mathbf{z}}$, Equation (31)	$O\left\{4P^2(I_1^2+I_{1}G)+I_{1}G+I_1{G}^2\right\}$,$I_1=I+1$
${\Lambda }^{-1}$	$O\left\{I_1^3\right\}$
$\mathbf{\Sigma }={\left({\mathbf{\Psi }}^H\left(\mathit{\beta }\right){\mathbf{F}}^T{\mathbf{Q}}^{-1}\mathbf{F}\mathbf{\Psi }\left(\mathit{\beta }\right)+{\Lambda }^{-1}\right)}^{-1}$, Equation (32)	$O\left\{4P^2(2I_{1}G+I_1{G}^2+I_1^2)+2I_1^3\right\}$
$\mathit{\delta }$, Equation (35)	$O\left\{I_1^2\right\}$
${\mathbf{B}}_{\mathbf{w}}={\mathbf{Q}}^{-1/2}\mathbf{FB}$	$O\left\{4P^2(G^2+IG)+G^3\right\}$
${\mathbf{Q}}^{-1/2}\mathbf{F}{\overline{\mathbf{C}}}_0$	$O\left\{4P^2(G^2+IG)+G^3\right\}$
${\mathbf{Q}}^{-1/2}\mathbf{F}{{\mathbf{1}}^{\prime }}_n$	$O\left\{4P^2(G^2+G)+G^3\right\}$
$\mathbf{P}=Re\left\{{\mathbf{B}}_{\mathbf{w}}^T{\mathbf{B}}_{\mathbf{w}}^{*}\oplus \left({\mathit{\mu }}_I{\mathit{\mu }}_I^H+{\rm Y}\right)\right\}$, Equation (40)	$O\left\{I^{2}G+2I^2\right\}$
$\mathbf{v}$, Equation (41)	$O\left\{3I^{2}G+3IG+I^3\right\}$

. In addition, the computational complexity of the proposed algorithm is mainly increased by the non-circular extension of the signal compared to the OGSBI-NA algorithm, but the metric level is still $O\left\{P^2\right\}$.

4.2. The Spatial Spectrum with Different DOAs

In the first simulation, the spatial spectrum of the proposed method and OGSBI-NA [27] are estimated with different values of DOA. The following different DOAs are considered:

(a) ${\theta }_k\in \left[\begin{array}{cc}-{50}^{\circ }+\eta ,& {30}^{\circ }+\eta \end{array}\right]$ with $\eta ={10}^{\circ }\times \mathrm{randn}\left(-1,1\right)$, K = 2;

(b) ${\theta }_k\in \left[-{50}^{\circ }+\eta ,-{30}^{\circ }+\eta ,{30}^{\circ }+\eta ,{50}^{\circ }+\eta \right]$ with $\eta ={10}^{\circ }\times \mathrm{randn}\left(-1,1\right)$, K = 4;

(c) ${\theta }_k\in \left[-{50}^{\circ }:{10}^{\circ }:{50}^{\circ }\right]+\eta$ with $\eta ={10}^{\circ }\times \mathrm{randn}\left(-1,1\right)$, K = 11;

(d) ${\theta }_k\in \left[-{70}^{\circ }:{10}^{\circ }:{70}^{\circ }\right]+\eta$ with $\eta ={10}^{\circ }\times \mathrm{randn}\left(-1,1\right)$, K = 15.

Figure 3 illustrates the spatial spectrum of the proposed algorithm and OGSBI-NA when SNR = 0 dB, the number of snapshots T = 200, and the grid resolution $2^{\circ }$. In Figure 3a,b, there are fewer sources (K = 2 or K = 4) than the number of array elements ($P=10$). Both the OGSBI-NA and the proposed method successfully find all sources, but the OGSBI-NA signal peaks have lower magnitudes than the proposed algorithm. In Figure 3c–d, there are more sources (K = 11 or K = 15) than the number of array sensors ($P=10$). Obviously, both the OGSBI-NA method and the proposed algorithm in this paper successfully locate all 11 or 15 sources. However, the amplitudes of some signal peaks of the OGSBI-NA method are lower than those of the proposed algorithm, which indicates that our method is superior to the existing algorithms.

4.3. The RMSE of Overdetermined DOA Estimation

We consider 10 physical array elements, i.e., M = 5, N = 5. There are two narrowband NC signals $\left[\begin{array}{cc}-{50}^{\circ }+\eta ,& {30}^{\circ }+\eta \end{array}\right]$ with NC phases $\phi =5^{\circ }$ impinging on the nested array, where $\eta ={10}^{\circ }*\mathrm{randn}\left(-1,1\right)$ to eliminate the prior information that may be contained in the predefined DOA set.

Figure 4 compares the RMSE performance and SNR with T = 200. As shown in Figure 4, since the grid-gap is relatively large (the grid resolution $2^{\circ }$), the $l_1$-SVD method and the SS-MUSIC method have similar accuracy. Based on a nested array, the proposed method has higher performance than the OGSBI-NA method because of the use of conjugate information formed by NC signals.

The RMSE results vs. the number of snapshots with SNR = 0 dB are depicted in Figure 5. As the number of snapshots increases, the DOA estimation accuracies of $l_1$-SVD and SS-MUSIC improve slightly. However, these two methods perform worse than the proposed algorithm, mainly due to the relatively large grid-gap. In addition, the accuracy of the OGSBI-ULA and OGSBI-NA methods also increases with the increase in the number of snapshots. When the number of snapshots reaches 500, the RMSE of the OGSBI-NA algorithm drops to 0.05, indicating that using nested arrays can improve the estimation accuracy. With the increase in the number of snapshots, the RMSE of the proposed algorithm is the lowest. At the same time, it is proved that the conjugate property of NC signals could improve the estimation accuracy.

4.4. The RMSE of Underdetermined DOA Estimation

To demonstrate that the proposed algorithm can estimate more sources than the number of physical sensors, we investigate the performance of the proposed algorithm in underdetermined DOA estimation scenarios. For this purpose, we assume 11 narrowband NC signals with NC phases $\phi =5^{\circ }$ from directions $\left[-{50}^{\circ }:10:{50}^{\circ }\right]+\eta$ incident to a nested array of M = N = 5, $\eta ={10}^{\circ }*\mathrm{randn}\left(-1,1\right)$ and the grid resolution is $2^{\circ }$. When the number of sources is greater than the number of array elements, the $l_1$-SVD method and the OGSBI-ULA method do not work, so we compare the proposed algorithm with the SS-MUSIC method [12] and the OGSBI-NA method [27].

We then choose the SNR of each signal to vary from −4 dB to 10 dB, T = 200, MC = 200. The RMSEs of three methods are given, including SS-MUSIC, OGSBI-NA (the grid resolution is $2^{\circ }$), and the proposed algorithm. As shown in Figure 6, SS-MUSIC still has a large error when the number of estimated sources exceeds the number of array elements. The performances of the three algorithms gradually improve as the SNR increases. Compared to the OGSBI-NA method, the RMSE performance of the algorithm proposed in this problem is superior, thanks to the use of the conjugate information created by the NC signal, which extends the virtual array aperture.

Then, we kept the SNR at 0 dB, and increased the snapshot number from 50 to 1000, and MC = 200. The RMSEs of the three methods vs. the number of snapshots are shown in Figure 7. The accuracy of DOA estimation by SS-MUSIC slightly improves as the number of snapshots increases, but the error is still relatively large due to the low signal-to-noise ratio. When the snapshots $T>300$, the RMSE of the proposed algorithm is close to that of OGSBI-NA. As the number of snapshots increases, the RMSE of the algorithm gradually stabilizes. It is demonstrated that the conjugate property of NC signals can improve the accuracy of estimation.

5. Conclusions

In this paper, a DOA estimation algorithm based on off-grid sparse Bayesian inference using nested arrays with NC signals is proposed. Since the statistical properties of NC signals do not have rotational invariance, the received data can be expanded. Based on this advantage, we construct difference co-arrays and sum co-arrays, which increase the array aperture and improve the estimation accuracy of DOA. Then, we consider the noise as part of the signal of interest and use it for the recursion of the OGSBI method after computing the selection matrix and removing redundant information. After iterations of sparse Bayesian inference, we can update the grid and find the optimal value. The simulation results show that the proposed algorithm significantly enhances the accuracy of DOA estimation. Future research could focus on developing advanced techniques for processing NC signals. It may involve exploring new signal processing methods or statistical models to better capture the unique properties of NC signals and avoid the limitation of considering only the same NC phases.