Robust Sparse Bayesian Two-Dimensional Direction-of-Arrival Estimation with Gain-Phase Errors

Abstract

To reduce the influence of gain-phase errors and improve the performance of direction-of-arrival (DOA) estimation, a robust sparse Bayesian two-dimensional (2D) DOA estimation method with gain-phase errors is proposed for L-shaped sensor arrays. The proposed method introduces an auxiliary angle to transform the 2D DOA estimation problem into two 1D angle estimation problems. A sparse representation model with gain-phase errors is constructed using the diagonal element vector of the cross-correlation covariance matrix of two submatrices of the L-shaped sensor array. The expectation maximization algorithm derives unknown parameter expression, which is used for iterative operations to obtain off-grid and signal precision. Using these parameters, a new spatial spectral function is constructed to estimate the auxiliary angle. The obtained auxiliary angle is substituted into a sparse representation model with gain and phase errors, and then the sparse Bayesian learning method is used to estimate the elevation angle of the incident signal. Finally, according to the relationship of the three angles, the azimuth angle can be estimated. The simulation results show that the proposed method can effectively realize the automatic matching of the azimuth and elevation angles of the incident signal, and improves the accuracy of DOA estimation and angular resolution.

1. Introduction

The estimation of the direction-of-arrival of sensor arrays is an important research direction in the fields of radar, sonar and mobile communications, which has attracted the interest of many scholars [1,2,3]. At present, the research on uniform linear arrays (ULAs) is relatively mature, and many high-resolution DOA estimation methods have been proposed to improve the accuracy and resolution, such as the subspace method [4,5], maximum likelihood methods [6,7] and compressed sensing methods. Compared with the first two types of methods, the DOA estimation method based on compressed sensing can effectively estimate the incident angle in the case of a few snapshots and low signal-to-noise ratio (SNR). The reconstruction methods of compressed sensing technology are mainly divided into three categories: convex optimization methods [8,9], greedy iteration methods [10,11] and sparse Bayesian learning (SBL) methods.

Among these methods, by exploring the spatial sparsity of incident signals, sparse Bayesian learning methods have better adaptive performance than the other traditional methods under the conditions of a limited number of snapshots, low SNR and spatially adjacent signals. The SBL method [12], compared with convex optimization methods, not only has the same global convergence, but also has relatively higher computational efficiency. Although the SBL estimation method improves the resolution, its estimation performance is limited by the degree of spatial dispersion. Increasing the estimation accuracy by adding a preset grid will bring a huge computational burden; however, the estimation accuracy will be reduced without increasing the preset grid, that is, there is a mesh mismatch problem. Therefore, Yang et al. [13] considered introducing an off-grid model and proposed an off-grid Sparse Bayesian Inference (OGSBI) method, which further reduced the influence of grid mismatch and improved the DOA estimation accuracy. Dai et al. [14] proposed a root-off grid sparse Bayesian learning (RootSBL) DOA estimation method, which improves the operation efficiency by ensuring high direction estimation accuracy. However, because the off-grid models in the above two methods use first-order Taylor expansion, their estimation performance under a coarse grid is not satisfactory. To solve this problem, the authors of [15] introduced second-order Taylor approximation into off-grid DOA estimation. Although the higher-order Taylor approximation reduces the approximate error and improves the accuracy of DOA estimation, it also increases the computational complexity. In order to reduce the amount of computation, Refs. [16,17] introduced different forms of real-value transformation, using compressed sensing technology to transform the data in the complex number field into the real number field and reduce the operation time of each iteration. Then, for different application backgrounds such as color noise and broadband signals, many researchers used the DOA estimation methods based on variational sparse Bayes [18,19,20], and the DOA estimation performances of such methods were further improved using SBL. Nevertheless, the above methods based on sparse Bayesian learning are all based on ULAs, which can only provide one-dimensional angle information. In practical applications, it is more useful to obtain the 2D direction of the source, that is, the azimuth angle and the elevation angle. In recent years, the research on 2D DOA estimation has gradually increased, and scholars have proposed many array geometries for 2D estimation, such as circular arrays, rectangular arrays and cross arrays. Compared with these 2D arrays, L-shaped arrays are simpler in structure, easier to implement, and have better estimation performance, which has attracted much attention [21].

Although the above types of estimation methods have good DOA estimation performances, they depend on the ideal array manifold, while in practical application scenarios, array manifolds are often affected by unknown gain-phase errors. Without array manifold calibration, the performance of DOA estimation may be greatly degraded [22]. Therefore, the study of DOA estimation methods with array gain-phase errors has theoretical significance and practical value. Of the existing methods to solve the gain-phase error problem, there are two main categories: pre-calibration and self-calibration methods. Pre-calibration methods generally require placing an additional pre-calibration source, and the pre-calibration source needs to have accurate bearing information. The placement of the pre-calibration source is used to estimate the gain phase error and correct the DOA [23,24]. However, in practical engineering applications, it is not easy to ensure the existence of calibration sources, so it is not suitable for practical engineering applications. However, self-calibration methods can directly estimate the gain-phase error during array operation without placing the pre-calibration source [25,26,27]. Although these methods usually adopt iterative methods and require a large amount of computation, compared with pre-calibration methods, it is more likely to be implemented in actual engineering.

In order to solve the problem of low accuracy and poor angular resolution caused by unknown gain-phase errors, a robust sparse Bayesian two-dimensional DOA estimation method with gain-phase errors is proposed by combining L-shaped matrices. The main contributions of this method are as follows:

• A new auxiliary angle is introduced to transform the 2D DOA estimation problem into two 1D DOA estimation problems to reduce the computational complexity.
• For the sparse reconstruction problem with gain-phase errors, the expected maximum algorithm is used to derive the estimation expressions of all unknown parameters to obtain new spatial spectral functions and the angle information to overcome the gain-phase errors.
• By solving two sparse reconstruction problems with gain-phase errors, the auxiliary angle and its corresponding elevation angle are estimated and then the azimuth angle estimation and automatic angle matching are completed according to the relationship between the three angles.

Notation: The superscripts ${(\cdot )}^{\mathrm{T}}$, ${(\cdot )}^{\mathrm{H}}$, and ${(\cdot )}^{\ast }$ denote the transpose, the conjugate transpose, and the conjugate, respectively. ${(\cdot )}^{-1}$ denotes matrix inversion. $\mathrm{diag}(\cdot )$ returns a diagonal matrix whose main diagonal is given in the bracket and $\mathrm{blkdiag}(\cdot )$ represents the block diagonal matrix. $\Re \{\cdot \}$ denotes the real part of a complex value. $Tr\{\cdot \}$ represents the trace operation of a matrix, and ${\Vert \cdot \Vert }_2$ denotes the $l_2$ norm. The mathematical symbols employed throughout the manuscript are listed in

Table 1. Mathematical symbols and their meanings.

Symbol	Representative Meaning
${(\cdot )}^{\mathrm{T}}$	Transpose
${(\cdot )}^{\mathrm{H}}$	Conjugate transpose
${(\cdot )}^{\ast }$	Conjugate
${(\cdot )}^{-1}$	Matrix inversion
$\mathrm{diag}(\cdot )$	Diagonal matrix
$\mathrm{blkdiag}(\cdot )$	Block diagonal matrix
$\Re \{\cdot \}$	Real part of the complex value
$Tr\{\cdot \}$	Trace operation of a matrix
${\Vert \cdot \Vert }_2$	$l_2$ norm

.

2. Signal Model

In practical applications, 2D DOA estimation is closer to the directional characteristics of real electromagnetic propagation signals, and is usually based on uniform rectangular arrays (URAs) and uniform circular arrays (UCAs). However, compared with URAs and UCAs, L-shaped arrays covers a larger area and have a wider application range, so this study used uniform L-shaped arrays, as shown in Figure 1. The subarray on the $y$ axis consists of a uniform linear array with $M$ elements; the element spacing of the ULA is $d=\lambda /2$, where $\lambda$ is the wavelength of the signal; and the subarray on the z-axis has the same structure as the subarray on the y-axis.

Suppose that $K$ far-field narrowband sources $s_k\left(t\right)$ are incident on a ULA equipped with $2M-1$ array elements. The incoming wave direction of the incident signal is $\left({\theta }_k,{\varphi }_k\right)$,$k=1,2,\cdots ,K$, ${\theta }_k$ is the elevation angle, ${\varphi }_k$ is the azimuth angle and the number of snapshots is $T$; then, the model of the received signal of the subarray on the z-axis and y-axis can be expressed as

$$\mathit{z}\left(t\right)=\mathit{A}\left(\mathit{\theta }\right)\mathit{s}\left(t\right)+{\mathit{n}}_z\left(t\right),$$

(1)

$$\mathit{y}\left(t\right)=\mathit{A}\left(\mathit{\varphi }\right)\mathit{s}\left(t\right)+{\mathit{n}}_y\left(t\right),$$

(2)

where $\mathit{z}\left(t\right)={[z_1\left(t\right),z_2\left(t\right),\cdots ,z_M\left(t\right)]}^{\mathrm{T}}$ and $\mathit{y}\left(t\right)={[y_1\left(t\right),y_2\left(t\right),\cdots ,y_M\left(t\right)]}^{\mathrm{T}}$ represents the array receiving signals on the z and y axes, respectively; $\mathit{s}\left(t\right)={[s_1\left(t\right),s_2\left(t\right),\cdots ,s_K\left(t\right)]}^{\mathrm{T}}$ represents the incident signal, where “$\mathrm{T}$” denotes the transpose operator; both ${\mathit{n}}_z\left(t\right)$ and ${\mathit{n}}_x\left(t\right)$ stand for the complex additive Gaussian white noise vector with zero means that are independent from the incident signal. $\mathit{A}\left(\mathit{\theta }\right)=[\mathit{a}\left({\theta }_1\right),\mathit{a}\left({\theta }_2\right),\cdots ,\mathit{a}\left({\theta }_K\right)]$ and $\mathit{A}\left(\mathit{\varphi }\right)=[\mathit{a}\left({\varphi }_1\right),\mathit{a}\left({\varphi }_2\right),\cdots ,\mathit{a}\left({\varphi }_K\right)]$ indicate the ideal array manifold matrix, where $\mathit{a}\left({\theta }_k\right)$ and $\mathit{a}\left({\varphi }_k\right)$ satisfy the following:

$$\mathit{a}\left({\theta }_k\right)={[1,e^{\frac{-\mathit{j}2\mathit{\pi }d\cos {\theta }_k}{\lambda }},\cdots ,e^{\frac{-\mathit{j}2\mathit{\pi }\left(M-1\right)d\cos {\theta }_k}{\lambda }}]}^{\mathrm{T}},$$

(3)

$$\mathit{a}\left({\varphi }_k\right)={[1,e^{\frac{-\mathit{j}2\mathit{\pi }d\cos {\varphi }_k}{\lambda }},\cdots ,e^{\frac{-\mathit{j}2\mathit{\pi }\left(M-1\right)d\cos {\varphi }_k}{\lambda }}]}^{\mathrm{T}}.$$

(4)

In the case of multiple snapshots, the array signal models can be expressed in matrix form as follows:

$$\mathit{Z}={\mathit{A}}_z\mathit{S}+{\mathit{N}}_z,$$

(5)

$$\mathit{Y}={\mathit{A}}_y\mathit{S}+{\mathit{N}}_y.$$

(6)

At present, most signal processing methods are based on the ideal condition of the array manifold, but gain-phase errors may exist in practical applications. Therefore, considering the condition of gain-phase errors, the array signal models are expressed as

$$\mathit{Z}={\mathit{\Gamma }}_z{\mathit{A}}_z\mathit{S}+{\mathit{N}}_z,$$

(7)

$$\mathit{Y}={\mathit{\Gamma }}_y{\mathit{A}}_y\mathit{S}+{\mathit{N}}_y,$$

(8)

where ${\mathit{\Gamma }}_z=\mathrm{diag}\left({\rho }_{z1}{e}^{j{\phi }_{z1}},\cdots ,{\rho }_{zm}{e}^{j{\phi }_{zm}},\cdots ,{\rho }_{zM}{e}^{j{\phi }_{zM}}\right)$ and ${\mathit{\Gamma }}_y=\mathrm{diag}\left({\rho }_{y1}{e}^{j{\phi }_{y1}},\cdots ,{\rho }_{ym}{e}^{j{\phi }_{ym}},\cdots ,{\rho }_{yM}{e}^{j{\phi }_{yM}}\right)$ are the two matrices of the gain-phase error coefficients between subarray elements, and ${\rho }_m$ and ${\phi }_m$ denote the gain error and the phase error of the $m$th array, respectively.

From the above statement, it is clear that the elevation angles and azimuth angles are estimated completely separately. However, if the elevation angles and azimuth angles are estimated separately, and then additional angle matching is performed, the overall performance of the method will be poor at low SNRs and with small snapshots. Therefore, we consider an auxiliary angle ${\eta }_k$ which satisfies $\cos {\eta }_k=\cos {\varphi }_k-\cos {\theta }_k$, and then the received data can be expressed as

$$\mathit{X}=\left[\begin{array}{c}\mathit{Z}\\ \mathit{Y}\end{array}\right]=\left[\begin{array}{c}{\mathit{\Gamma }}_z{\mathit{A}}_z\\ {\mathit{\Gamma }}_y{\mathit{A}}_y\end{array}\right]\mathit{S}+\left[\begin{array}{c}{\mathit{N}}_z\\ {\mathit{N}}_y\end{array}\right]=\mathit{\Gamma }\mathit{A}\left(\theta ,\varphi \right)\mathit{S}+\mathit{N},$$

(9)

where $\mathit{\Gamma }=\mathrm{blkdiag}\left({\mathit{\Gamma }}_z,{\mathit{\Gamma }}_x\right)$, $\mathit{A}\left(\mathit{\theta },\varphi \right)=\left[\mathit{a}\left({\theta }_1,{\varphi }_1\right),\cdots ,\mathit{a}\left({\theta }_K,{\varphi }_K\right)\right]$ and $\mathit{a}\left({\theta }_k,{\varphi }_k\right)={\left[{\mathit{a}}^{\mathrm{T}}\left({\theta }_k\right),{\mathit{a}}^{\mathrm{T}}\left({\varphi }_k\right)\right]}^{\mathrm{T}}$; based on the relationship between the three angles, the new direction vector can be expressed as $\mathit{a}\left({\theta }_k,{\varphi }_k\right)=\mathit{\Theta }\left({\eta }_k\right)\mathit{a}\left({\theta }_k\right)$, $\mathit{\Theta }\left({\eta }_k\right)={\left[{\mathit{I}}_M,\mathrm{diag}\left(\mathit{e}{\left({\eta }_k\right)}^{\ast }\right)\right]}^{\mathrm{T}}$ and $\mathit{e}\left({\eta }_k\right)=\left[1,e^{-j2\pi d\cos {\eta }_k},\cdots ,e^{-j2\pi d\left(M-1\right)\cos {\eta }_k}\right]$.

3. The Proposed Method

3.1. Off-Grid Model

Let $\overline{\mathit{\theta }}=\{{\overline{\theta }}_1,{\overline{\theta }}_2,\cdots ,{\overline{\theta }}_N\}$ be a fixed sampling grid in the DOA range $[-90^{°},90^{°}]$, where $N$ denotes the grid number and typically satisfies $N>>M>K$. Assuming that the source is incident to the preset grid point without bias, that is ${\theta }_k\in \overline{\mathit{\theta }}$, the general array signal model with gain-phase errors can be expressed as

$$\mathit{X}=\mathit{\Gamma }\mathit{A}\left(\overline{\mathit{\theta }}\right)\overline{\mathit{S}}+\mathit{N},$$

(10)

where $\mathit{A}\left(\overline{\mathit{\theta }}\right)=[\mathit{a}\left({\overline{\theta }}_1\right),\mathit{a}\left({\overline{\theta }}_2\right),\cdots ,\mathit{a}\left({\overline{\theta }}_N\right)]$ is the over-complete dictionary set, and $\overline{\mathit{S}}$ is the row sparse signal matrix obtained after extension.

Since, in actual situations, the real incident signal position is not likely to be located exactly on the preset discrete spatial grid, we introduce an off-grid model. Let ${\overline{\theta }}_k$ represent the uniform grid with a grid interval $r={\overline{\theta }}_2-{\overline{\theta }}_1$ that we set up in advance. ${\theta }_k\notin \left\{{\overline{\theta }}_1,{\overline{\theta }}_2,\cdots ,{\overline{\theta }}_N\right\}$ denotes the real signal that does not belong to the set of preset discrete angles, and the first-order Taylor expansion of its steering vector can be approximated by

$$\begin{array}{l}\mathit{a}\left({\theta }_k\right)=\mathit{a}\left({\overline{\theta }}_{nk}\right)+\mathit{b}\left({\overline{\theta }}_{nk}\right)\left({\theta }_k-{\overline{\theta }}_{nk}\right)\\ =\mathit{a}\left({\overline{\theta }}_{nk}\right)+\mathit{b}\left({\overline{\theta }}_{nk}\right){\beta }_{nk}\end{array},$$

(11)

where ${\overline{\theta }}_{nk}$ represents the preset grid point closest to ${\theta }_k$, ${\beta }_{nk}={\theta }_k-{\overline{\theta }}_{nk}$, and $\mathit{b}\left({\overline{\theta }}_{nk}\right)=\mathit{a}\prime \left({\overline{\theta }}_{nk}\right)$ represents the first derivative of $\mathit{a}\left({\overline{\theta }}_{nk}\right)$. Expressed in matrix form, $\mathit{A}\left(\overline{\mathit{\theta }}\right)=[\mathit{a}\left({\overline{\theta }}_1\right),\mathit{a}\left({\overline{\theta }}_2\right),\cdots ,\mathit{a}\left({\overline{\theta }}_N\right)]$, $\mathit{B}\left(\overline{\mathit{\theta }}\right)=[\mathit{b}\left({\overline{\theta }}_1\right),\mathit{b}\left({\overline{\theta }}_2\right),\cdots ,\mathit{b}\left({\overline{\theta }}_N\right)]$ and $\mathit{\beta }=[{\beta }_1,\cdots ,{\beta }_N]$, $\mathit{A}\left(\overline{\mathit{\theta }}\right)$ in Equation (10) can be expressed as

$$\mathit{\Phi }\left(\mathit{\beta }\right)=\mathit{A}\left(\overline{\mathit{\theta }}\right)+\mathit{B}\left(\overline{\mathit{\theta }}\right)\mathrm{diag}\left(\mathit{\beta }\right).$$

(12)

Therefore, Equation (10) can be further updated as

$$\mathit{X}=\mathit{\Gamma }\mathit{\Phi }\left(\mathit{\beta }\right)\overline{\mathit{S}}+\mathit{N}=\mathit{\Gamma }\left[\mathit{A}\left(\overline{\mathit{\theta }}\right)+\mathit{B}\left(\overline{\mathit{\theta }}\right)\mathrm{diag}\left(\mathit{\beta }\right)\right]\overline{\mathit{S}}+\mathit{N}$$

(13)

Lemma 1.

For any $N\times 1$ dimensional complex vector $\mathit{a}$ and any $N\times N$ dimensional complex diagonal matrix $\mathit{\Gamma }$, $\mathit{\Gamma }\mathit{a}=diag\left(\mathit{a}\right)\mathit{\gamma }$ can be obtained, where $\mathit{\gamma }$ is the element on the main diagonal of the complex diagonal matrix $\mathit{\Gamma }$, i.e., ${\gamma }_i={\mathit{\Gamma }}_{i,i}$.

According to Lemma 1, we obtain

$$\mathit{\Gamma }\mathit{A}\left(\overline{\mathit{\theta }}\right)=\mathit{\Gamma }[\mathit{a}\left({\overline{\theta }}_1\right),\mathit{a}\left({\overline{\theta }}_2\right),\cdots ,\mathit{a}\left({\overline{\theta }}_N\right)]={\mathit{A}}_{\Lambda }\left({\mathit{I}}_N\otimes \mathit{\gamma }\right),$$

(14)

$$\mathit{\Gamma }\mathit{B}\left(\overline{\mathit{\theta }}\right)\mathrm{diag}\left(\mathit{\beta }\right)=\mathit{\Gamma }[\mathit{b}\left({\overline{\theta }}_1\right),\mathit{b}\left({\overline{\theta }}_2\right),\cdots ,\mathit{b}\left({\overline{\theta }}_N\right)]\mathrm{diag}(\mathit{\beta })={\mathit{B}}_{\Lambda }\left({\mathit{I}}_N\otimes \mathit{\gamma }\right)\mathrm{diag}\left(\mathit{\beta }\right),$$

(15)

where ${\mathit{A}}_{\Lambda }=\left[\mathrm{diag}\left(\mathit{a}\left({\overline{\theta }}_1\right)\right),\mathrm{diag}\left(\mathit{a}\left({\overline{\theta }}_2\right)\right),\cdots ,\mathrm{diag}\left(\mathit{a}\left({\overline{\theta }}_N\right)\right)\right]$, ${\mathit{I}}_N$ represents the $N\times N$ dimensional identity matrix, and $\otimes$ denotes the Kronecker product operation: ${\mathit{B}}_{\Lambda }=[{\mathit{B}}_{\Lambda 1},{\mathit{B}}_{\Lambda 2},\cdots ,{\mathit{B}}_{\Lambda N}]=\left[\mathrm{diag}\left(\mathit{b}\left({\overline{\theta }}_1\right)\right),\mathrm{diag}\left(\mathit{b}\left({\overline{\theta }}_2\right)\right),\cdots ,\mathrm{diag}\left(\mathit{b}\left({\overline{\theta }}_N\right)\right)\right]$, which can be obtained by summing Equations (14) and (15):

$$\mathit{\Gamma }\left[\mathit{A}\left(\overline{\mathit{\theta }}\right)+\mathit{B}\left(\overline{\mathit{\theta }}\right)\mathrm{diag}\left(\mathit{\beta }\right)\right]\overline{\mathit{S}}=\left({\mathit{A}}_{\Lambda }+{\mathit{B}}_{\Lambda }\left(\mathrm{diag}\left(\mathit{\beta }\right)\otimes {\mathit{I}}_M\right)\right)\left(\overline{\mathit{S}}\otimes \mathit{\gamma }\right).$$

(16)

By setting $\mathit{\Psi }\left(\mathit{\beta }\right)=\left({\mathit{A}}_{\Lambda }+{\mathit{B}}_{\Lambda }\left(\mathrm{diag}\left(\mathit{\beta }\right)\otimes {\mathit{I}}_M\right)\right)$, the sparse representation model of the array signal with gain-phase errors can be rewritten as

$$\mathit{X}=\mathit{\Psi }\left(\mathit{\beta }\right)\left(\overline{\mathit{S}}\otimes \mathit{\gamma }\right)+\mathit{N}.$$

(17)

To further simplify the above equation, let $\mathit{\Omega }\left(\mathit{\beta },\mathit{\gamma }\right)=\mathit{\Psi }\left(\mathit{\beta }\right)\left(\overline{\mathit{S}}\otimes \mathit{\gamma }\right)$.

3.2. Hierarchical Sparse Bayesian Framework

First, it is assumed that the signal received by the array is independent between different snapshots, and the noise is circular symmetric complex Gaussian white noise, which follows a Gaussian distribution. Thus, we obtain

$$p\left(\mathit{N}|{\alpha }_0\right)={\displaystyle \prod _{t=1}^{T}CN(\mathit{n}(t)|\mathsf{0},{\alpha }_0^{-1}\mathit{I})},$$

(18)

where ${\alpha }_0^{-1}\triangleq {\sigma }_n^2$ is the variance of noise. The complex Gaussian distributed random variable with mean $\mathit{\mu }$ and covariance $\mathit{\Sigma }$ obeys the probability density function of $CN\left(\mathit{\mu },\mathit{\Sigma }\right)$ as

$$CN\left(\mathit{x}|\mathit{\mu },\mathit{\Sigma }\right)=\frac{1}{{\mathit{\pi }}^N\left|\mathit{\Sigma }\right|}\exp \left\{-{\left(\mathit{x}-\mathit{\mu }\right)}^{\mathrm{H}}{\mathit{\Sigma }}^{-1}\left(\mathit{x}-\mathit{\mu }\right)\right\}.$$

(19)

where “$|\cdot |$”means taking the determinant.

In this paper, the variance of noise is unknown, so it should be assumed. Since the conjugate of the inverse of the variance of the Gaussian distribution is a Gamma distribution [28], a Gamma distribution is specified for the hyperparameter ${\alpha }_0$, which can be described as

$$p\left({\alpha }_0\right)=\Gamma \left({\alpha }_0;a,b\right)={\left[\Gamma \left(a\right)\right]}^{-1}{b}^a{\alpha }_0^{a-1}\exp \left\{-b{\alpha }_0\right\},$$

(20)

where $a$ and $b$ are the hyperparameters of ${\alpha }_0$, $\Gamma \left(a\right)={\displaystyle \int {}_0^{\infty }}{x}^{a-1}{e}^{-x}dx$.

For received signals that are independent of each other, the sparse matrix $\overline{\mathit{S}}$ is assumed to follow a Gaussian distribution with a zero mean:

$$p\left(\overline{\mathit{S}}|{\mathit{\alpha }}_s\right)={\displaystyle \prod _{t=1}^{T}CN\left({\overline{\mathit{s}}}_t|0,\mathit{\Lambda }\right)},\begin{array}{cc}& \mathit{\Lambda }=\mathrm{diag}\left({\mathit{\alpha }}_s\right)\end{array}.$$

(21)

Let the precision vector ${\mathit{\alpha }}_s={\left[{\alpha }_{s1},{\alpha }_{s2},\cdots ,{\alpha }_{sN}\right]}^{\mathrm{T}}$ be a hyperparameter, which controls the estimation accuracy of the parameter $\overline{\mathit{S}}$ to be estimated. Then, ${\mathit{\alpha }}_s$ can be expressed by the Gamma prior with hyperparameters $c$ and $d$:

$$p({\mathit{\alpha }}_s)={\displaystyle \prod _{n=1}^N\Gamma \left({\alpha }_{sn}|c,d\right)}.$$

(22)

For the case where the gain-phase error is unknown, and assuming that the gain phase error vector coefficients between arrays are independent of each other, the error vector distribution can also be expressed as a Gaussian distribution:

$$p\left(\mathit{\gamma }|{\mathit{\alpha }}_{\gamma }\right)={\displaystyle \prod _{m=1}^{M}CN\left({\gamma }_m|\mathsf{0},{\mathit{\Lambda }}_{\gamma }\right)},\begin{array}{cc}& {\mathit{\Lambda }}_{\gamma }=\mathrm{diag}\left({\mathit{\alpha }}_{\gamma }\right)\end{array}.$$

(23)

where ${\mathit{\alpha }}_{\gamma }={[{\alpha }_{\gamma 1},{\alpha }_{\gamma 2},\cdots ,{\alpha }_{\gamma M}]}^{\mathrm{T}}$ represents the gain-phase error precision, and its distribution is described by Gamma distributions with hyperparameters $e$ and $f$:

$$p\left({\mathit{\alpha }}_{\gamma };e,f\right)={\displaystyle \prod _{m=1}^M\Gamma \left({\alpha }_{\gamma m};e,f\right)}.$$

(24)

Then, the off-grid vector is assumed; if it follows a uniform distribution, then it can be expressed as

$$p\left(\mathit{\beta }\right)=U\left(\left[-\frac{1}{2}r,\frac{1}{2}r\right]\right),$$

(25)

where

$$U\left([a,b]\right)=\{\begin{array}{c}\frac{1}{b-a},\begin{array}{cc}& a\le x\le b\end{array}\\ 0,\begin{array}{cc}& otherwise\end{array}\end{array}.$$

(26)

3.3. Sparse Bayesian Inference

In this subsection, a self-correcting DOA estimation method for the unknown gain-phase error is proposed that uses sparse Bayesian inference under the assumption of the unknown parameter’s distribution in the previous section.

First, we give the following equation to maximize the posterior probability:

$$\left\{\overline{\mathit{S}},\mathit{\beta },\mathit{\gamma },{\alpha }_0,{\mathit{\alpha }}_s,{\mathit{\alpha }}_{\gamma }\right\}=\underset{\left\{\overline{\mathit{S}},\mathit{\beta },\mathit{\gamma },{\alpha }_0,{\mathit{\alpha }}_s,{\mathit{\alpha }}_{\gamma }\right\}}{\arg \max }p\left(\overline{\mathit{S}},\mathit{\beta },\mathit{\gamma },{\alpha }_0,{\mathit{\alpha }}_s,{\mathit{\alpha }}_{\gamma }|\mathit{X}\right).$$

(27)

However, since this posterior probability problem is too complicated to be solved directly, the expectation maximization (EM) algorithm is considered here to obtain a solution.

$$p\left(\mathit{X},\overline{\mathit{S}},\mathit{\beta },\mathit{\gamma },{\alpha }_0,{\mathit{\alpha }}_s,{\mathit{\alpha }}_{\gamma }\right)=p\left(\mathit{X}|\overline{\mathit{S}},\mathit{\beta },\mathit{\gamma },{\alpha }_0\right)p\left(\overline{\mathit{S}}|{\mathit{\alpha }}_s\right)p\left(\mathit{\gamma }|{\mathit{\alpha }}_{\gamma }\right)p\left({\alpha }_0\right)p\left({\mathit{\alpha }}_s\right)p\left({\mathit{\alpha }}_{\gamma }\right)p\left(\mathit{\beta }\right).$$

(28)

According to the received data $\mathit{X}$ and the parameters of the prior distribution, the posterior distribution of $\overline{\mathit{S}}$ can be represented as

$$\begin{array}{l}p\left(\overline{\mathit{S}}|\mathit{X},\mathit{\beta },\mathit{\gamma },{\alpha }_0,{\mathit{\alpha }}_s,{\mathit{\alpha }}_{\gamma }\right)=\frac{p\left(\mathit{X}|\overline{\mathit{S}},\mathit{\beta },\mathit{\gamma },{\alpha }_0,{\mathit{\alpha }}_s,{\mathit{\alpha }}_{\gamma }\right)p\left(\overline{\mathit{S}}|{\mathit{\alpha }}_s\right)}{p\left(\mathit{X}|\mathit{\beta },\mathit{\gamma },{\alpha }_0,{\mathit{\alpha }}_s,{\mathit{\alpha }}_{\gamma }\right)}\\ \begin{array}{cccc}\begin{array}{cc}\begin{array}{ccc}& & \end{array}& \end{array}& & & \end{array}\propto p\left(\mathit{X}|\overline{\mathit{S}},\mathit{\beta },\mathit{\gamma },{\alpha }_0,{\mathit{\alpha }}_s,{\mathit{\alpha }}_{\gamma }\right)p\left(\overline{\mathit{S}}|{\mathit{\alpha }}_s\right)\end{array},$$

(29)

where

$$\begin{array}{cc}\hfill p\left(\mathit{X}|\overline{\mathit{S}},\mathit{\beta },\mathit{\gamma },{\alpha }_0,{\mathit{\alpha }}_s,{\mathit{\alpha }}_{\gamma }\right)& ={\displaystyle \prod _{t=1}^{T}CN\left({\mathit{y}}_t|\mathit{\Psi }\left(\mathit{\beta }\right)\left({\overline{\mathit{s}}}_t\otimes \mathit{\gamma }\right),{\alpha }_0^{-1}\mathit{I}\right)}\hfill \\ & ={\displaystyle \prod _{t=1}^T\frac{{\alpha }_0^M}{{\mathit{\pi }}^M}}\exp \left\{-{\alpha }_0{\Vert {\mathit{y}}_t-\mathit{\Psi }\left(\mathit{\beta }\right)\left({\overline{\mathit{s}}}_t\otimes \mathit{\gamma }\right)\Vert }_2^2\right\}\hfill \end{array},$$

(30)

$$\begin{array}{cc}\hfill p\left(\overline{\mathit{S}}|{\mathit{\alpha }}_s\right)& ={\displaystyle \prod _{t=1}^{T}CN\left({\overline{\mathit{s}}}_t|\mathsf{0},\mathrm{diag}\left({\mathit{\alpha }}_s\right)\right)}\hfill \\ & ={\displaystyle \prod _{t=1}^T\left({\displaystyle \prod _{n=1}^N{\alpha }_{sn}}\right)}\frac{1}{{\mathit{\pi }}^N}\exp \left\{-{\overline{\mathit{s}}}_t^{\mathrm{H}}\mathrm{diag}\left({\mathit{\alpha }}_s\right){\overline{\mathit{s}}}_t\right\}\hfill \end{array}.$$

(31)

Both $p\left(\overline{\mathit{S}}|{\mathit{\alpha }}_s\right)$ and $p\left(\mathit{X}|\overline{\mathit{S}},\mathit{\beta },\mathit{\gamma },{\alpha }_0,{\mathit{\alpha }}_s,{\mathit{\alpha }}_{\gamma }\right)$ are Gaussian functions; therefore, the posterior function of $p\left(\overline{\mathit{S}}|\mathit{X},\mathit{\beta },\mathit{\gamma },{\alpha }_0,{\mathit{\alpha }}_s,{\mathit{\alpha }}_{\gamma }\right)$ can also be expressed as a Gaussian function:

$$p\left(\overline{\mathit{S}}|\mathit{X},\mathit{\beta },\mathit{\gamma },{\alpha }_0,{\mathit{\alpha }}_s,{\mathit{\alpha }}_{\gamma }\right)={\displaystyle \prod _{t=1}^{T}CN\left({\overline{\mathit{s}}}_t|{\mathit{\mu }}_t,{\mathit{\Sigma }}_s\right)},$$

(32)

where

$${\mathit{\mu }}_t={\alpha }_0^{\mathrm{H}}{\mathit{\Sigma }}_s^{\mathrm{H}}{\mathit{\Omega }}^{\mathrm{H}}\left(\mathit{\beta },\mathit{\gamma }\right){\mathit{x}}_t,$$

(33)

$${\mathit{\Sigma }}_s={\left[{\alpha }_0{\mathit{\Omega }}^{\mathrm{H}}\left(\mathit{\beta },\mathit{\gamma }\right)\mathit{\Omega }\left(\mathit{\beta },\mathit{\gamma }\right)+\mathrm{diag}\left({\mathit{\alpha }}_s\right)\right]}^{-1},$$

(34)

The next step is hyperparameter learning, which is equivalent to maximizing the posterior probability distribution of all the parameters to be estimated. In order to use Equation (28) to estimate other unknown parameters, the following likelihood can be used:

$$\begin{array}{c}L\left(\mathit{\beta },\mathit{\gamma },{\alpha }_0,{\mathit{\alpha }}_s,{\mathit{\alpha }}_{\gamma }\right)\\ =\epsilon \left\{\ln p\left(\mathit{X}|\overline{\mathit{S}},\mathit{\beta },\mathit{\gamma },{\alpha }_0,{\mathit{\alpha }}_s,{\mathit{\alpha }}_{\gamma }\right)p\left(\overline{\mathit{S}}|{\mathit{\alpha }}_s\right)p\left(\mathit{\gamma }|{\mathit{\alpha }}_{\gamma }\right)p\left({\alpha }_0\right)p\left({\mathit{\alpha }}_s\right)p\left({\mathit{\alpha }}_{\gamma }\right)p\left(\mathit{\beta }\right)\right\}\end{array},$$

(35)

where $\epsilon \{\cdot \}$ represents ${\epsilon }_{\overline{\mathit{S}}|\mathit{X},\mathit{\beta },\mathit{\gamma },{\alpha }_n,{\mathit{\alpha }}_s,{\mathit{\alpha }}_{\gamma }}\{\cdot \}$, which stands for the likelihood function under the expectation with respect to the posterior of $\overline{\mathit{S}}$.

(1) For the noise precision ${\alpha }_0$. Ignoring the unrelated terms, the likelihood function of ${\alpha }_0$ can be expressed as follows:

$$\begin{array}{cc}\hfill L\left({\alpha }_0\right)& =\epsilon \left\{\ln p\left(\mathit{X}|\overline{\mathit{S}},\mathit{\beta },\mathit{\gamma },{\alpha }_0\right)p\left({\alpha }_0\right)\right\}\hfill \\ & =\epsilon \left\{\ln {\displaystyle \prod _{t=1}^{T}CN\left({\mathit{x}}_t|\mathit{\Psi }\left(\mathit{\beta }\right)\left({\overline{\mathit{s}}}_t\otimes \mathit{\gamma }\right),{\alpha }_0^{-1}\mathit{I}\right)}\right\}+\ln \Gamma \left({\alpha }_0;a,b\right)\hfill \end{array}.$$

(36)

Setting the derivative $\frac{\partial L\left({\alpha }_0\right)}{\partial {\alpha }_0}$ to zero, solving for ${\alpha }_0$ with the given updates:

$${\alpha }_0=\frac{TM+a-1}{T\times Tr\left\{{\mathit{\Omega }}^{\mathrm{H}}\left(\mathit{\beta },\mathit{\gamma }\right)\mathit{\Omega }\left(\mathit{\beta },\mathit{\gamma }\right){\mathit{\Sigma }}_s\right\}+{{\displaystyle {\sum }_{t=1}^T\Vert {\mathit{x}}_t-\mathit{\Psi }\left(\mathit{\beta }\right)\left({\mathit{\mu }}_t\otimes \mathit{\gamma }\right)\Vert }}_2^2+b}.$$

(37)

(2) For the signal precision ${\mathit{\alpha }}_s$. Ignoring the unrelated terms, the likelihood function of ${\mathit{\alpha }}_s$ can be expressed as follows:

$$\begin{array}{cc}\hfill L\left({\mathit{\alpha }}_s\right)& =\epsilon \left\{\ln p\left(\overline{\mathit{S}}|{\mathit{\alpha }}_s\right)p\left({\mathit{\alpha }}_s\right)\right\}\hfill \\ & =\epsilon \left\{\ln {\displaystyle \prod _{t=1}^{T}CN\left({\overline{\mathit{s}}}_t|0,{\mathit{\Lambda }}_s\right)}\right\}+\ln {\displaystyle \prod _{n=1}^N\Gamma \left({\alpha }_{sn};c,d\right)}\hfill \end{array}.$$

(38)

Setting the derivative $\frac{\partial L\left({\mathit{\alpha }}_s\right)}{\partial {\mathit{\alpha }}_s}$ to zero, the $n$th update value of ${\mathit{\alpha }}_s$ can be solved as

$${\alpha }_{sn}=\frac{T+c-1}{d+T{\mathit{\Sigma }}_{s,n,n}+{{\displaystyle {\sum }_{t=1}^T\left|{\mu }_{n,t}\right|}}^2},$$

(39)

where ${\mathit{\Sigma }}_{s,n,n}$ refers to the $n$th row and $n$th column elements of ${\mathit{\Sigma }}_s$, and ${\mu }_{n,t}$ refers to the $n$th value of ${\mathit{\mu }}_t$.

(3) For the gain-phase error vector $\mathit{\gamma }$. Ignoring the unrelated terms, the likelihood function of $\mathit{\gamma }$ can be expressed as follows:

$$\begin{array}{l}L\left(\mathit{\gamma }\right)=\epsilon \left\{\ln p\left(\mathit{X}|\overline{\mathit{S}},\mathit{\beta },\mathit{\gamma },{\alpha }_0\right)p\left(\mathit{\gamma }|{\mathit{\alpha }}_{\gamma }\right)\right\}\\ \begin{array}{cc}& \end{array}=\epsilon \left\{\ln {\displaystyle \prod _{t=1}^{T}CN\left({\mathit{x}}_t|\mathit{\Psi }\left(\mathit{\beta }\right)\left({\overline{\mathit{s}}}_t\otimes \mathit{\gamma }\right),{\alpha }_0^{-1}\mathit{I}\right)}\right\}+\ln {\displaystyle \prod _{m=1}^{M}CN\left({\gamma }_m|0,{\alpha }_{\gamma m}\right)}\\ \begin{array}{cc}& \end{array}=TM\ln \frac{{\alpha }_0}{\mathit{\pi }}-{\alpha }_{0}T\cdot Tr\left\{{\mathit{\Omega }}^{\mathrm{H}}\left(\mathit{\beta },\mathit{\gamma }\right)\mathit{\Omega }\left(\mathit{\beta },\mathit{\gamma }\right){\mathit{\Sigma }}_s\right\}\\ \begin{array}{cc}& \end{array}-{\alpha }_0{\displaystyle {\sum }_{t=1}^T{\Vert {\mathit{x}}_t-\mathit{\Psi }\left(\mathit{\beta }\right)\left({\mathit{\mu }}_t\otimes \mathit{\gamma }\right)\Vert }_2^2}-{\displaystyle {\sum }_{m=1}^M{\alpha }_{\gamma m}{\left|{\gamma }_m\right|}^2}\\ \begin{array}{cc}& \end{array}\propto -{\alpha }_{0}T\cdot Tr\left\{{\mathit{\Omega }}^{\mathrm{H}}\left(\mathit{\beta },\mathit{\gamma }\right)\mathit{\Omega }\left(\mathit{\beta },\mathit{\gamma }\right){\mathit{\Sigma }}_s\right\}\\ \begin{array}{cc}& \end{array}-{\alpha }_0{\displaystyle {\sum }_{t=1}^T{\Vert {\mathit{x}}_t-\mathit{\Psi }\left(\mathit{\beta }\right)\left({\mathit{\mu }}_t\otimes \mathit{\gamma }\right)\Vert }_2^2}-{\displaystyle {\sum }_{m=1}^M{\alpha }_{\gamma m}{\left|{\gamma }_m\right|}^2}\end{array}.$$

(40)

To simplify this complicated equation, let us set

$$Q_1\left(\mathit{\beta },\mathit{\gamma }\right)=Tr\left\{{\mathit{\Omega }}^{\mathrm{H}}\left(\mathit{\beta },\mathit{\gamma }\right)\mathit{\Omega }\left(\mathit{\beta },\mathit{\gamma }\right){\mathit{\Sigma }}_s\right\},$$

(41)

$$Q_{2,t}\left(\mathit{\beta },\mathit{\gamma }\right)={\Vert {\mathit{x}}_t-\mathit{\Psi }\left(\mathit{\beta }\right)\left({\mathit{\mu }}_t\otimes \mathit{\gamma }\right)\Vert }_2^2,$$

(42)

$$Q_3\left(\mathit{\gamma }\right)={\displaystyle {\sum }_{m=1}^M{\alpha }_{\gamma m}{\left|{\gamma }_m\right|}^2}$$

(43)

Calculating $\frac{\partial L\left(\mathit{\gamma }\right)}{\partial \mathit{\gamma }}$ according to the above equation, the following simplification can be obtained:

$$\frac{\partial L\left(\mathit{\gamma }\right)}{\partial \mathit{\gamma }}=-{\alpha }_{0}T\frac{\partial Q_1\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\gamma }}-{\displaystyle \sum _{t=1}^T{\alpha }_0}\frac{\partial Q_{2,t}\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\gamma }}-\frac{\partial Q_3\left(\mathit{\gamma }\right)}{\partial \mathit{\gamma }}.$$

(44)

For Equation (44), we calculate $\frac{\partial Q_1\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\gamma }}$, $\frac{\partial Q_{2,t}\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\gamma }}$ and $\frac{\partial Q_3\left(\mathit{\gamma }\right)}{\partial \mathit{\gamma }}$ separately. For $\frac{\partial Q_1\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\gamma }}$, first compute the $m$th value of $\frac{\partial Q_1\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\gamma }}$:

$$\begin{array}{l}{\left[\frac{\partial Q_1\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\gamma }}\right]}_m=Tr\left\{\frac{\partial {\mathit{\Omega }}^{\mathrm{H}}\left(\mathit{\beta },\mathit{\gamma }\right)\mathit{\Omega }\left(\mathit{\beta },\mathit{\gamma }\right){\mathit{\Sigma }}_s}{\partial {\gamma }_m}\right\}\\ \begin{array}{ccc}\begin{array}{ccc}& & \end{array}& & \end{array}=Tr\left\{\frac{\partial {\left({\mathit{I}}_N\otimes \mathit{\gamma }\right)}^{\mathrm{H}}}{\partial {\gamma }_m}{\mathit{\Psi }}^{\mathrm{H}}\left(\mathit{\beta }\right)\mathit{\Omega }\left(\mathit{\beta },\mathit{\gamma }\right){\mathit{\Sigma }}_s+{\mathit{\Omega }}^{\mathrm{H}}\left(\mathit{\beta },\mathit{\gamma }\right)\mathit{\Psi }\left(\mathit{\beta }\right)\frac{\partial \left({\mathit{I}}_N\otimes \mathit{\gamma }\right)}{\partial {\gamma }_m}{\mathit{\Sigma }}_s\right\}\\ \begin{array}{ccc}\begin{array}{ccc}& & \end{array}& & \end{array}=Tr\left\{{\mathit{\Omega }}^{\mathrm{H}}\left(\mathit{\beta },\mathit{\gamma }\right)\mathit{\Omega }\left(\mathit{\beta },{\mathit{e}}_m^M\right){\mathit{\Sigma }}_s\right\}\\ \begin{array}{ccc}\begin{array}{ccc}& & \end{array}& & \end{array}={\mathit{\gamma }}^{\mathrm{H}}\left({\displaystyle \sum _{p=1}^N{\displaystyle \sum _{k=1}^N{\mathit{\Psi }}_p^{\mathrm{H}}\left(\mathit{\beta }\right){\mathit{\Psi }}_k\left(\mathit{\beta }\right)}}{\mathit{\Sigma }}_{s,k,p}\right){\mathit{e}}_m^M\end{array},$$

(45)

where ${\mathit{e}}_m^M$ is the $M\times 1$ dimensional vector with the $m$th entry being 1 and the other entries being 0. Therefore, we can obtain

$$\frac{\partial Q_1\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\gamma }}={\mathit{\gamma }}^{\mathrm{H}}\left({\displaystyle \sum _{p=1}^N{\displaystyle \sum _{k=1}^N{\mathit{\Psi }}_p^{\mathrm{H}}\left(\mathit{\beta }\right){\mathit{\Psi }}_k\left(\mathit{\beta }\right){\mathit{\Sigma }}_{s,k,p}}}\right).$$

(46)

For $\frac{\partial Q_{2,t}\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\gamma }}$, we can simplify it to

$$\begin{array}{cc}\hfill \frac{\partial Q_{2,t}\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\gamma }}& =-{\left[{\mathit{x}}_t-\mathit{\Psi }\left(\mathit{\beta }\right)\left({\mathit{\mu }}_t\otimes \mathit{\gamma }\right)\right]}^{\mathrm{H}}\mathit{\Psi }\left(\mathit{\beta }\right)\frac{\partial {\mathit{\mu }}_t\otimes \mathit{\gamma }}{\partial \mathit{\gamma }}\hfill \\ & =-{\left[{\mathit{x}}_t-\mathit{\Psi }\left(\mathit{\beta }\right)\left({\mathit{\mu }}_t\otimes \mathit{\gamma }\right)\right]}^{\mathrm{H}}\mathit{\Psi }\left(\mathit{\beta }\right)\left({\mathit{\mu }}_t\otimes {\mathit{I}}_M\right)\hfill \end{array}.$$

(47)

For $\frac{\partial Q_3\left(\mathit{\gamma }\right)}{\partial \mathit{\gamma }}$, we can simplify it to

$$\frac{\partial Q_3\left(\mathit{\gamma }\right)}{\partial \mathit{\gamma }}={\mathit{\gamma }}^{\mathrm{H}}\mathrm{diag}\left({\mathit{\alpha }}_{\gamma }\right).$$

(48)

By setting $\frac{\partial L\left(\mathit{\gamma }\right)}{\partial \mathit{\gamma }}=0$, we can obtain

$$\mathit{\gamma }={\mathit{H}}^{-1}\mathit{d},$$

(49)

where

$$\mathit{H}={\displaystyle \sum _{t=1}^T{\alpha }_0{\mathit{\Xi }}^{\mathrm{H}}\left(\mathit{\beta },{\mathit{\mu }}_t\right)\mathit{\Xi }\left(\mathit{\beta },{\mathit{\mu }}_t\right)}+{\alpha }_{0}T{\left({\displaystyle \sum _{p=0}^N{\displaystyle \sum _{k=0}^N{\mathit{\Psi }}_p^{\mathrm{H}}\left(\mathit{\beta }\right){\mathit{\Psi }}_k\left(\mathit{\beta }\right){\mathit{\Sigma }}_{s,k,p}}}\right)}^{\mathrm{H}}+\mathrm{diag}\left({\mathit{\alpha }}_{\gamma }\right),$$

(50)

$$\mathit{d}={\displaystyle \sum _{t=1}^T{\alpha }_0{\mathit{\Xi }}^{\mathrm{H}}\left(\mathit{\beta },{\mathit{\mu }}_t\right){\mathit{x}}_t},$$

(51)

where $\mathit{\Xi }\left(\mathit{\beta },{\mathit{\mu }}_t\right)\triangleq \mathit{\Psi }\left(\mathit{\beta }\right)\left({\mathit{\mu }}_t\otimes {\mathit{I}}_M\right)$.

(4) For the gain-phase error precision ${\mathit{\alpha }}_{\gamma }$. Ignoring the independent terms, we can obtain the following likelihood function of ${\mathit{\alpha }}_{\gamma }$:

$$\begin{array}{cc}\hfill L\left({\mathit{\alpha }}_{\gamma }\right)& =\epsilon \left\{\ln p\left(\mathit{\gamma }|{\mathit{\alpha }}_{\gamma }\right)p\left({\mathit{\alpha }}_{\gamma }\right)\right\}\hfill \\ & =\epsilon \left\{\ln {\displaystyle \prod _{m=0}^{M}CN\left({\gamma }_m|0,{\alpha }_{\gamma m}\right)}\right\}+\ln {\displaystyle \prod _{m=0}^M\Gamma \left({\alpha }_{\gamma m};e,f\right)}\hfill \end{array}.$$

(52)

Let $\frac{\partial L\left({\mathit{\alpha }}_{\gamma }\right)}{\partial {\mathit{\alpha }}_{\gamma }}=0$; then, we can obtain a newer version of the $m$th value of ${\mathit{\alpha }}_{\gamma }$:

$${\alpha }_{\gamma m}\approx \frac{1}{f+{\gamma }_n^{\mathrm{H}}{\gamma }_n}.$$

(53)

(5) For the off-grid vector $\mathit{\beta }$. Ignoring the independent terms, we can obtain the following likelihood function of $\mathit{\beta }$:

$$\begin{array}{l}L\left(\mathit{\beta }\right)=\epsilon \left\{\ln p\left(\mathit{X}|\overline{\mathit{S}},\mathit{\beta },\mathit{\gamma },{\alpha }_0\right)p\left(\mathit{\beta }\right)\right\}\\ \begin{array}{cc}& \begin{array}{cc}& \propto -T{Q}_1\left(\mathit{\beta },\mathit{\gamma }\right)\end{array}\end{array}-{\displaystyle {\sum }_{t=1}^T{Q}_{2,t}\left(\mathit{\beta },\mathit{\gamma }\right)}\end{array}.$$

(54)

Calculate $\frac{\partial L\left(\mathit{\beta }\right)}{\partial \mathit{\beta }}$ according to the above equation, and similarly calculate $\frac{\partial Q_1\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\beta }}$ and $\frac{\partial Q_{2,t}\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\beta }}$. First, compute the $n$th value of $\frac{\partial Q_1\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\beta }}$:

$$\begin{array}{cc}\hfill {\left[\frac{\partial Q_1\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\beta }}\right]}_n& =Tr\left\{\frac{\partial {\mathit{\Omega }}^{\mathrm{H}}\left(\mathit{\beta },\mathit{\gamma }\right)\mathit{\Omega }\left(\mathit{\beta },\mathit{\gamma }\right){\mathit{\Sigma }}_s}{\partial {\beta }_m}\right\}\hfill \\ & =Tr\left\{\left[0,{\mathit{\Omega }}^{\mathrm{H}}\left(\mathit{\beta },\mathit{\gamma }\right){\mathit{B}}_{\Lambda n}\mathit{\gamma },0\right]{\mathit{\Sigma }}_s\right\}+Tr\left\{{\left[0,{\mathit{\Omega }}^{\mathrm{H}}\left(\mathit{\beta },\mathit{\gamma }\right){\mathit{B}}_{\Lambda n}\mathit{\gamma },0\right]}^{\mathrm{H}}{\mathit{\Sigma }}_s\right\}\hfill \\ & =2\Re \left\{{\left[{\mathit{\Omega }}^{\mathrm{H}}\left(\mathit{\beta },\mathit{\gamma }\right){\mathit{B}}_{\Lambda n}\mathit{\gamma }\right]}^{\mathrm{H}}{\mathit{\Sigma }}_{s,:,n}\right\}\hfill \end{array}.$$

(55)

Then, $\frac{\partial Q_1\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\beta }}$ can be simplified as

$$\frac{\partial Q_1\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\beta }}=2\Re \left\{\mathrm{diag}\left({\mathit{\Sigma }}_s{\mathit{\Omega }}^{\mathrm{H}}\left(\mathit{\beta },\mathit{\gamma }\right){\mathit{B}}_{\Lambda }\left({\mathit{I}}_N\otimes \mathit{\gamma }\right)\right)\right\}.$$

(56)

For $\frac{\partial Q_{2,t}\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\beta }}$, we can simplify it to

$$\begin{array}{cc}\hfill \frac{\partial Q_{2,t}\left(\mathit{\beta },\mathit{\gamma }\right)}{\partial \mathit{\beta }}& =-2\Re \left\{{\left[{\mathit{x}}_t-\mathit{\Psi }\left(\mathit{\beta }\right)\left({\mathit{\mu }}_t\otimes \mathit{\gamma }\right)\right]}^{\mathrm{H}}\frac{\partial \mathit{\Psi }\left(\mathit{\beta }\right)\left({\mathit{\mu }}_t\otimes \mathit{\gamma }\right)}{\partial \mathit{\beta }}\right\}\hfill \\ & =-2\Re \left\{{\left[{\mathit{x}}_t-\mathit{\Psi }\left(\mathit{\beta }\right)\left({\mathit{\mu }}_t\otimes \mathit{\gamma }\right)\right]}^{\mathrm{H}}{\mathit{B}}_{\Lambda }\left(\mathrm{diag}\left({\mathit{\mu }}_t\right)\otimes \mathit{\gamma }\right)\right\}\hfill \end{array}.$$

(57)

By setting $\frac{\partial L\left(\mathit{\beta }\right)}{\partial \mathit{\beta }}=0$, we can obtain

$$\mathit{\beta }={\mathit{P}}^{-1}\mathit{v},$$

(58)

where

$${\mathit{P}}_{n,:}=\Re \left\{T{\mathit{\gamma }}^{\mathrm{H}}{\mathit{B}}_{\Lambda n}^{\mathrm{H}}{\mathit{B}}_{\Lambda }\left(\mathrm{diag}\left({\mathit{\Sigma }}_{s,:,n}\right)\otimes \mathit{\gamma }\right)\right\}+{\displaystyle \sum _{t=1}^T\Re \left\{{\mu }_{t,n}{\mathit{\gamma }}^{\mathrm{T}}{\mathit{B}}_{\Lambda n}^{\mathrm{H}}{\mathit{B}}_{\Lambda }^{\ast }\left({\mathit{I}}_N\otimes {\mathit{\gamma }}^{\ast }\right){\mathrm{diag}}^{\ast }\left({\mathit{\mu }}_t\right)\right\}},$$

(59)

$${\mathit{v}}_n={\displaystyle \sum _{t=1}^T\Re \left\{{\left[{\mathit{x}}_t-{\mathit{A}}_{\Lambda }\left({\mathit{\mu }}_t\otimes \mathit{\gamma }\right)\right]}^{\mathrm{H}}{\mathit{B}}_{\Lambda n}{\mu }_{t,n}\mathit{\gamma }\right\}}-T\Re \left\{{\mathit{\gamma }}^{\mathrm{H}}{\mathit{B}}_{\Lambda n}^{\mathrm{H}}{\mathit{A}}_{\Lambda }\left({\mathit{I}}_N\otimes \mathit{\gamma }\right){\mathit{\Sigma }}_{s,:,n}\right\}.$$

(60)

4. Matching of Elevation Angle and Azimuth Angle

In most existing compressed sensing DOA estimation methods based on L-shaped arrays, one-dimensional DOA is used to estimate the elevation angle and azimuth angles, and then the elevation and azimuth angles are matched. The process for this kind of method is complicated and the estimation accuracy is not high. Therefore, in order to solve this problem, an auxiliary angle is proposed to realize the automatic matching of elevation and azimuth angles.

According to Equations (7) and (8), the cross-correlation covariance matrix of the two subarrays can be expressed as

$${\mathit{R}}_{zy}={\rm E}\left[\mathit{z}\left(t\right){\mathit{y}}^{\mathrm{H}}\left(t\right)\right]= \mathit{\Gamma } \prime \mathit{A}\left(\mathit{\theta }\right){\mathit{R}}_s{\mathit{A}}^{\mathrm{H}}\left(\varphi \right),$$

(61)

where ${\mathit{R}}_s={\rm E}\left[\mathit{s}\left(t\right){\mathit{s}}^{\mathrm{H}}\left(t\right)\right]$; we then take the diagonal elements of its cross-correlation covariance matrix:

$$\begin{array}{cc}\hfill {\mathit{r}}_{zy}& =\mathrm{diag}\left({\mathit{R}}_{zy}\right)\hfill \\ & = \mathit{\Gamma } \prime \left[{\displaystyle \sum _{k=1}^K{\delta }_k},{\displaystyle \sum _{k=1}^K{\delta }_k{e}^{\frac{-j2\pi d\left(\cos {\varphi }_k-\cos {\theta }_k\right)}{\lambda }},\cdots ,{\displaystyle \sum _{k=1}^K{\delta }_k{e}^{\frac{-j2\pi d\left(M-1\right)\left(\cos {\varphi }_k-\cos {\theta }_k\right)}{\lambda }}}}\right]\hfill \\ & = \mathit{\Gamma } \prime \left[{\displaystyle \sum _{k=1}^K{\delta }_k},{\displaystyle \sum _{k=1}^K{\delta }_k{e}^{\frac{-j2\pi d\cos {\eta }_k}{\lambda }},\cdots ,{\displaystyle \sum _{k=1}^K{\delta }_k{e}^{\frac{-j2\pi d\left(M-1\right)\cos {\eta }_k}{\lambda }}}}\right]\hfill \\ & = \mathit{\Gamma } \prime \mathit{A}\left(\mathit{\eta }\right)\mathit{\delta }\hfill \end{array},$$

(62)

where $\mathit{\Gamma } \prime ={\mathit{\Gamma }}_z{\mathit{\Gamma }}_y$, $\mathit{\delta }=\left[{\delta }_1,{\delta }_2,\cdots ,{\delta }_K\right]$ and ${\delta }_k$ represents the power of the $k$th incident signal $s_k\left(t\right)$, which is the $k$th diagonal element of ${\mathit{R}}_s$. Equation (62) can be regarded as the received signal with angle parameter ${\eta }_k$ incident on a one-dimensional array of $M$ sensor elements, which satisfies the array signal model of Equation (10). Therefore, the auxiliary angle ${\eta }_k$ can be solved using the one-dimensional sparse Bayesian self-calibration DOA estimation method described in Section 2.

Then, according to Equation (9), the ${\eta }_k$ obtained by the solution is substituted into the expression of the signal received by the array:

$$\begin{array}{cc}\hfill \mathit{X}& =\mathit{\Gamma }\mathit{A}\left(\mathit{\theta },\varphi \right)\mathit{S}+\mathit{N}=\mathit{\Gamma }\mathit{\Theta }\left(\mathit{\eta }\right)\left({\mathit{I}}_K\otimes \mathit{A}\left(\mathit{\theta }\right)\right)\mathit{S}+\mathit{N}\hfill \\ & =\mathit{\Gamma }\tilde{\mathit{A}}\left(\mathit{\theta }\right)\mathit{S}+\mathit{N}\hfill \end{array},$$

(63)

where $\mathit{\Theta }\left(\mathit{\eta }\right)=\left[\mathit{\Theta }\left({\widehat{\eta }}_1\right),\mathit{\Theta }\left({\widehat{\eta }}_2\right),\cdots ,\mathit{\Theta }\left({\widehat{\eta }}_K\right)\right]$ and ${\widehat{\eta }}_k$ represents the estimated value of the solved auxiliary angle ${\eta }_k$.

Equation (63) also satisfies the array signal model of Equation (10), and can also be solved using the one-dimensional sparse Bayes self-correcting DOA estimation method in Section 2; then, the corresponding block sparse estimation results are obtained. For each “block”, the sparse signal value is recovered by searching the spectral peak, and the elevation angle ${\theta }_k$ corresponding to the auxiliary angle ${\eta }_k$ is estimated. Finally, according to the relationship between the three angles, the corresponding azimuth angle estimate can be obtained as

$${\widehat{\varphi }}_k=\mathrm{argcos}\left(\cos {\widehat{\theta }}_k+{\widehat{\eta }}_k\right).$$

(64)

5. Simulation Results

Simulation experiments were conducted to compare the proposed method with the 2D-MUSIC method, OMP method and OGSBI method, so as to verify the estimation performance of the proposed method. With reference to the relevant literature, the gain-phase error of the array was assumed to conform to the following model:

$$\{\begin{array}{c}{\rho }_m=1+\sqrt{12}{\sigma }_{\rho }{\kappa }_m\\ {\phi }_m=\sqrt{12}{\sigma }_{\phi }{\zeta }_m\end{array},$$

(65)

where ${\sigma }_{\rho }$ and ${\sigma }_{\phi }$ stand for the standard deviations of gain errors and phase errors, respectively, and ${\kappa }_m$ and ${\zeta }_m$ are independent random variables distributed uniformly over $\left[-0.5,0.5\right]$.

Consider a uniform L-shaped array with $M=8$ array elements on the subarray, with an array element spacing of half the wavelength; the wavelength was set to 0.06, and the three DOAs of the far-field narrow band signal impacting the array were $\left({\theta }_1,{\varphi }_1\right)=\left(15.2°,80.3°\right)$, $\left({\theta }_2,{\varphi }_2\right)=\left(60.5°,20.6°\right)$ and $\left({\theta }_3,{\varphi }_3\right)=\left(75.8°,40.7°\right)$. ${\sigma }_{\rho }={\sigma }_{\phi }=0.5$, the number of snapshots was 100, and the grid interval was 1°. The noise was additive white Gaussian noise. The estimation results of 150 Monte Carlo experiments under the condition of SNR = 0 dB and SNR = 15 dB are shown in Figure 2. Figure 3 shows the spatial estimation results obtained by the proposed method, 2D-MUSIC method, OMP method and OGSBI method when the SNR was 10 dB.

As can be seen from Figure 2, when the SNR was 15 dB, the DOA estimation result of the proposed method was close to the true direction of the incident source. When the SNR was 0 dB, although the estimation accuracy was reduced, the approximate location of the source was still effectively estimated. Compared with the other three methods, the proposed method reduced the influence of the gain-phase errors, and its estimate was the closest to the true source position (Figure 3).

The root mean square error (RMSE) was used to evaluate the estimation errors of the proposed method and the 2D-MUSIC, OMP and OGSBI methods. The number of Monte Carlo simulation experiments was set to $I=300$, and the root-mean-square error of the elevation angle and the azimuth angle estimations of $K$ incident signals are defined as

$$RMSE1=\sqrt{\frac{1}{2IK}{\displaystyle \sum _{i=1}^I{\displaystyle \sum _{k=1}^K{\left({\widehat{\theta }}_k^i-{\theta }_k\right)}^2}}}$$

(66)

$$RMSE2=\sqrt{\frac{1}{2IK}{\displaystyle \sum _{i=1}^I{\displaystyle \sum _{k=1}^K{\left({\widehat{\varphi }}_k^i-{\varphi }_k\right)}^2}}}$$

(67)

where ${\widehat{\theta }}_k^i$ represents the estimated azimuth angle of the $k$th signal in the $i$th Monte Carlo simulation experiment and ${\widehat{\varphi }}_k^i$ represents the estimated elevation angle of the $k$th signal in the $i$th Monte Carlo simulation experiment.

Figure 4 shows the RMSEs of the OGSBI method, OMP method, 2D-MUSIC method and the proposed method when the SNR was changed without affecting any of the other settings. It can be seen that the RMSE curves of the four methods all decreased gradually with increasing SNR. The RMSE curves of the 2D-MUSIC and OMP methods were higher and the error was large. The proposed method was superior to the other methods in this SNR interval with the lowest RMSE curve and the smallest error. When the SNR was 15 dB, not only the RMSE of the elevation angle, but also the RMSE of the azimuth angle were at the lowest values, only 0.2838 and 0.5537. This shows that when the SNR is high, the advantages of the proposed method are more obvious.

Figure 5 shows the RMSEs of the four methods when the SNR was set to 10 dB and the other experimental conditions were unchanged. The RMSEs of the four methods showed a decreasing trend with increasing number of snapshots. The RMSE curve of the proposed method was lower than that of the other three methods, and the RMSEs of the elevation and azimuth angles were only 1.5596 and 1.7965 when the number of snapshots was 20. This shows that the proposed method has the best estimation performance under different snapshot numbers compared to the other methods.

Figure 6 shows the RMSE of the four methods when, with the other experimental conditions unchanged, the SNR was set to 10 dB, the number of snapshots was 100, and the magnitude of the gain-phase error (i.e., the standard deviation coefficient of the gain-phase error) was changed. The RMSEs of the four methods increased with the increase in gain-phase errors, but the proposed method was less affected by the magnitude of the gain-phase errors, and the RMSE curve remained the lowest. The 2D-MUSIC method was significantly affected by gain-phase errors, and its RMSE fluctuated the most. This shows that, compared to other methods, the proposed method can effectively reduce the influence of gain-phase errors and has the best estimation performance.

The angular resolution ability is another important index that can be used to judge the performance of DOA estimation, and the resolution ability of different methods is different. In this study, the angular resolution ability of the proposed method was evaluated from the perspective of SNR and the standard deviation coefficient of the gain-phase error. The method can identify the source position correctly, when the following conditions are met:

$$\max \left\{\left|{\overline{\theta }}_k-{\theta }_k\right|,\left|{\overline{\varphi }}_k-{\varphi }_k\right|\right\}<{\epsilon }_e,$$

(68)

where ${\overline{\theta }}_k$ is the estimate of the true azimuth angle ${\theta }_k$, ${\overline{\varphi }}_k$ is the estimate of the true elevation angle ${\varphi }_k$, and ${\epsilon }_e$ represents the set angle error threshold, which is set to 1 here.

Keeping the other experimental conditions unchanged, the number of snapshots was set to 100, ${\sigma }_{\rho }={\sigma }_{\phi }=0.5$, and the SNR was varied. The results of the probability of successful resolution (PSR) of the OGSBI method, OMP method, 2D-MUSIC method and the proposed method are shown in Figure 7.

It can be seen from Figure 7 that the PSR of the proposed method was higher than that of the other methods in both low and high SNR scenarios. The PSR of the proposed method reached 100% when the SNR was 9 dB, while the other three methods did not reached 100% even when the SNR was 15 dB, and their success probability of resolution was low, and their resolution ability was poor. This shows that the proposed method has better angular resolution and robustness compared to the other methods under different SNR conditions.

In Figure 8, the PSRs of the four methods are compared when, keeping the other experimental conditions unchanged, the number of snapshots was set to 100, the SNR was set to 15 dB, and the standard deviation coefficients of the gain-phase error ${\sigma }_{\rho }$ and ${\sigma }_{\rho }$ were changed. The figure shows that, in the case of large gain-phase errors, the PSRs of the other methods were low, and the OMP and 2D-MUSIC methods could not successfully predict the angles, while the PSR of the proposed method reached 66.7%. In the case of small gain-phase errors, the PSR of the proposed method reached 100%, which is obviously better than the other methods.

6. Conclusions

In order to solve the problems of low accuracy and poor angular resolution caused by unknown gain-phase errors, a robust sparse Bayesian two-dimensional DOA estimation method for scenarios with gain-phase errors was proposed in this paper. To avoid the additional angle matching problem in the traditional method, the proposed method introduces an auxiliary angle to match the elevation angle and the azimuth angle. A new sparse representation model with gain-phase errors is constructed by using the diagonal elements of the cross-correlation matrix received by two submatrices, and auxiliary angles are obtained by sparse Bayes inference. By substituting in the obtained auxiliary angles, a new sparse representation model is constructed again according to the relationship between the three angles, and the corresponding block sparse estimation results are obtained. For each “block”, the corresponding DOA is obtained by searching the spectral peak. The simulation results show that the DOA estimation accuracy and angle resolution of the proposed method are superior to the OGSBL, OMP and 2D-MUSIC methods in cases where gain-phase errors exist.