On the Complementarity of Sparse L0 and CEL0 Regularized Loss Landscapes for DOA Estimation

Abstract

L0 sparse methods are not widespread in Direction-Of-Arrival (DOA) estimation yet, despite their potential superiority over classical methods in difficult scenarios. This comes from the difficulties encountered for global optimization on hill-climbing error surfaces. In this paper, we explore the loss landscapes of L0 and Continuous Exact L0 (CEL0) regularized problems in order to design a new optimization scheme. As expected, we observe that the recently introduced CEL0 penalty leads to an error surface with less local minima than the L0 one. This property explains the good behavior of the CEL0-regularized sparse DOA estimation problem for well-separated sources. Unfortunately, CEL0-regularized landscape enlarges L0-basins in the middle of close sources, and CEL0 methods are thus unable to resolve two close sources. Consequently, we propose to alternate between both error surfaces to increase the probability of reaching the global solution. Experiments show that the proposed approach offers better performance than existing ones, and particularly an enhanced resolution limit.

1. Introduction

The study of Direction-Of-Arrival (DOA) estimation has a long history in signal processing. Conventional methods [1] such as beamforming or Capon’s method are still the subject of numerous works, e.g., [2]. However, they present degraded performance in the presence of multiple close sources. Subspace-based methods such as MUltiple SIgnal Classification (MUSIC) have been introduced to improve the resolution limit for multiple sources. Unfortunately, these methods fail in presence of correlated sources [3]. They also often require a priori knowledge of the number of sources and need a sufficient number of snapshots. Sparse DOA estimation has received much attention in the last decade due to its potential performance in such scenarios [4,5,6,7,8,9,10].

Sparsity naturally arises in DOA estimation when considering a discretization of the field-of-view in numerous candidate angles of arrival on a grid. The aim is to estimate a vector $\mathit{\gamma }$ whose dimension covers the whole grid and whose only the very few entries corresponding to sources DOA are non-zero.

The purpose of sparse estimation, under the Single Measurement Vector (SMV) framework, is to retrieve the sparse vector $\mathit{\gamma }\in {\mathbb{C}}^G$ from the noisy measurement $\mathbf{y}=\mathbf{B}\mathit{\gamma }+\mathbf{w}$, with $\mathbf{y}\in {\mathbb{C}}^{N^2}$, $N^2\ll G$, knowing a dictionary $\mathbf{B}$. The dictionary $\mathbf{B}$ depends on the array’s responses for the different angles of arrival candidates. It can be formulated as the following regularized problem:

$$\underset{\mathit{\gamma }}{min}\left\{J_{{\ell }_0}(\lambda ,\mathit{\gamma })=\frac{1}{2}{\parallel \mathbf{B}\mathit{\gamma }-\mathbf{y}\parallel }_2^2+\lambda {\parallel \mathit{\gamma }\parallel }_0\right\},$$

(1)

where the so-called ${\ell }_0$-norm is defined as ${\parallel \mathit{\gamma }\parallel }_0=\mathrm{Card}\left\{g\in \{1,\dots ,G\}:{\mathit{\gamma }}_g\ne 0\right\}$, ${\gamma }_g$ being the the g-th component of vector $\mathit{\gamma }$. The ${\ell }_0$-norm is the natural measure of sparsity: it counts the number of non zero components of the vector. The regularization parameter $\lambda$ aims to balance the relative importance between the data fidelity term ${1/2\parallel \mathbf{B}\mathit{\gamma }-\mathbf{y}\parallel }_2^2$ and the ${\ell }_0$-norm enforcing sparsity of the solution. The sparse estimation problem can also be formulated as a constraint problem. The relationship between the two ${\ell }_0$-problems has been studied in [11]. Based on this study, recent theoretical results [12,13] have been provided for an off-line selection of $\lambda$ so that the ${\ell }_0$-problems are equivalent. The regularization parameter is here chosen in accordance with those results.

The ${\ell }_0$-minimization problem is known to be NP-hard: its resolution usually requires an exhaustive search. The use of the very recently proposed global optimization method [14] is limited to small size problems and is thus unadapted here because of its huge computational cost. So far, many suboptimal methods have therefore been proposed, as the well-known Iterative Hard Thresholding (IHT) algorithm. IHT is a proximal gradient descent algorithm: it iteratively produces estimates ${\widehat{\mathit{\gamma }}}^{(i)}$ so that the cost function $J_{{\ell }_0}$ decreases, starting from an initial point ${\widehat{\mathit{\gamma }}}^{(0)}$. However, the ${\ell }_0$-regularized error surface $J_{{\ell }_0}$ exhibits numerous local minima, and convergence is only proved to a stationary point. Convex relaxation of (1) by the ${\ell }_1$-norm is also a popular alternative. However, conditions [15] under which the sparse vector can be reliably recovered are usually too restrictive for practical applications as in DOA estimation.

More recently, minimization of a regularized criterion using nonsmooth nonconvex but continuous penalties has drawn considerable attention [16], and it has been shown in many applications that it can yield significantly better performance than with using the ${\ell }_1$-norm [17]. Such penalties include ${\ell }_q$-norms ($0<q<1$), Smoothly Clipped Absolute Deviation (SCAD), and Minimax Concave Penalty (MCP) [18]. The Continuous Exact ${\ell }_0$ (CEL0) penalty [19] corresponds to the limit case of MCP for the SMV framework. CEL0 is shown to suppress some local minima of $J_{{\ell }_0}$ while preserving the global one. The CEL0-regularized cost function is:

$$J_{\mathrm{CEL}0}(\lambda ,\mathit{\gamma })=\frac{1}{2}{\parallel \mathbf{B}\mathit{\gamma }-\mathbf{y}\parallel }_2^2+{\Phi }_{\mathrm{CEL}0}(\lambda ,\mathit{\gamma }),$$

(2)

with

$$\begin{array}{cc}\hfill & {\Phi }_{\mathrm{CEL}0}(\lambda ,\mathit{\gamma })=\sum _{i\in {\mathbb{I}}_G}\varphi (\lambda ,{\alpha }_i,{\gamma }_i)\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill & \varphi (\lambda ,{\alpha }_i,{\gamma }_i)=\lambda -\frac{{\left(|{\gamma }_i|-\sqrt{2\lambda {\alpha }_i}\right)}^2}{2{\alpha }_i}{\mathbb{1}}_{\left\{|{\gamma }_i|\le \sqrt{2\lambda {\alpha }_i}\right\}}\hfill \end{array}$$

(4)

and ${\alpha }_i=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\parallel {\mathbf{B}}_{·,i}{\parallel }_2^2$}\right.$, ${\mathbf{B}}_{·,i}$ being the i-th column of matrix $\mathbf{B}$. $𝟙$ is the indicator function whose value is one if the given condition is respected and zero otherwise. Despite its promising interest, we have shown in [12] that traditional suboptimal optimization schemes of CEL0-regularized functional as Iterative Reweighted ${\ell }_1$ (IRL1) or Forward Backward (FB) are unable to resolve close sources.

The aim of this paper is to investigate the properties of $J_{{\ell }_0}$ and $J_{CEL0}$ loss surfaces in order to propose a sparse optimization strategy to resolve close sources. The goal is to improve the resolution limit of both MUSIC method, limited for low Signal-to-Noise Ratios, and existing sparse methods. The proposed approach follows an iterative scheme that requires little computational cost. It shows good performance for close sources and does not require any particular initialization.

Outline of the paper: we first explain the model and the sparse DOA estimation problem in Section 2. In Section 3, we compare $J_{{\ell }_0}$ and $J_{\mathrm{CEL}0}$ loss surfaces in the context of multi-source DOA estimation: an in-deep analysis of the minimizers is provided for two close sources. Based on this analysis, Section 4 presents the proposed optimization scheme, whose originality is to alternate between both loss surfaces. Numerical simulations of Section 5 finally show the validity and advantages of our approach.

Notations: Upper-case and lower-case boldface letters denote matrices and vectors, respectively. ${·}^{\ast }$ denotes the conjugate, ${·}^T$ the transpose and ${·}^H$ the conjugate transpose of a vector or matrix. $x_i$ is the $i^{th}$ component of vector $\mathbf{x}$, and ${\omega }_i$ the $i^{th}$ component of the set $\omega$. Given a matrix $\mathbf{X}$, the $i^{th}$ column is denoted ${\mathbf{X}}_{·,i}$. Considering a matrix $\mathbf{X}$ of dimension $N\times G$, ${\mathbf{X}}_{\omega }$ is the submatrix of $\mathbf{X}$ containing the columns indicated by the set $\omega \subseteq {\mathbb{I}}_G$, where ${\mathbb{I}}_G=\left\{1,\dots ,G\right\}$ is the ordered index set. Similarly, ${\mathbf{x}}_{\omega }$ is the subvector of $\mathbf{x}$ defined as: ${\mathbf{x}}_{\omega }={\left(x_{{\omega }_1},\dots ,x_{{\omega }_{♯\omega }}\right)}^T$, with $♯\omega$ the number of elements in $\omega$.

2. Sparse DOA Estimation Problem

2.1. On-Grid Array Signal Modeling

Consider M far field narrow band sources impinging an array of N antennas from angles ${\tilde{\theta }}_m$, $m=1\dots M$. For a single snapshot at time t, the output array signal $\mathbf{x}(t)\in {\mathbb{C}}^N$ is expressed as $\mathbf{x}(t)={\left[x_1(t)\dots x_N(t)\right]}^T={\sum }_{m=1}^M\mathbf{a}({\tilde{\theta }}_m){\tilde{s}}_m(t)+\mathbf{n}(t)$, where $\mathbf{a}({\tilde{\theta }}_m)$ is the steering vector (or array response) for the direction ${\tilde{\theta }}_m$, ${\tilde{s}}_m(t)$ the complex envelope of the signal of the mth source, and $\mathbf{n}(t)\in {\mathbb{C}}^N$ a white gaussian noise vector of covariance $E\left[\mathbf{n}(t){\mathbf{n}}^H(t)\right]={\sigma }_n^2{\mathbf{I}}_N$, where ${\mathbf{I}}_N$ is the $N\times N$ identity matrix. Let us suppose the directions of the sources are part of a predefined set $\mathbf{\Theta }=\left\{{\theta }_1,\dots ,{\theta }_G\right\}$ resulting from the discretization of the field-of-view, with $G\gg N$: for all arrival angles ${\tilde{\theta }}_m,m\in [1,\dots ,M]$, there exists $g\in [1,\dots ,G]$ such that ${\tilde{\theta }}_m={\theta }_g$. This assumption is often considered in operational systems to measure the calibration table $\mathbf{A}=\left[\mathbf{a}({\theta }_1),\dots ,\mathbf{a}({\theta }_G)\right]$ containing the array responses for the angles in $\mathbf{\Theta }$. Considering this calibration table, the measurement $\mathbf{x}(t)$ can be expressed as:

$$\mathbf{x}(t)=\mathbf{A}\mathbf{s}(t)+\mathbf{n}(t),$$

(5)

where $\mathbf{s}(t)\in {\mathbb{C}}^G$ is sparse with only M non-zero entries corresponding to the sources signals ${\tilde{s}}_m(t)$, with $M\ll G$.

Under the assumption that the sources are uncorrelated, it is interesting to use the vectorized covariance matrix in algorithms: it allows us to consider the contribution of multiple snapshots thus increasing the accuracy, without increasing the computational cost by much. It also has the advantage of being a SMV model, thus all associated methods can be used, and it additionally gives the possibility of estimating more sources than the number of sensors.

2.2. Vectorized Covariance Matrix Model

Considering uncorrelated sources, the covariance matrix ${\mathbf{R}}_{xx}\widehat{=}E[\mathbf{x}(t){\mathbf{x}}^H(t)]$ is given by:

$${\mathbf{R}}_{xx}=\sum _{m=1}^M\mathbf{a}({\tilde{\theta }}_m){\mathbf{a}}^H({\tilde{\theta }}_m){\tilde{\gamma }}_m+{\sigma }_n^2{\mathbf{I}}_N,$$

(6)

with ${\tilde{\gamma }}_m$ the power of the mth source and ${\mathbf{I}}_N$ the square identity matrix of dimension N. The vectorized covariance matrix, noted as $\mathbf{r}=\mathrm{vec}({\mathbf{R}}_{xx})$, is the vector obtained from the concatenation of the columns of ${\mathbf{R}}_{xx}$. It can be expressed as

$$\mathbf{r}=\sum _{m=1}^M\mathbf{b}({\tilde{\theta }}_m){\tilde{\gamma }}_m+{\sigma }_n^2\mathrm{vec}({\mathbf{I}}_N)$$

(7)

with $\mathbf{b}={\mathbf{a}}^{\ast }\otimes \mathbf{a}$, ${\mathbf{a}}^{\ast }$ is the conjugate of $\mathbf{a}$, and ⊗ is the Kronecker product. Considering a dictionary $\mathbf{B}=\left[\mathbf{b}({\theta }_1),\dots ,\mathbf{b}({\theta }_G)\right]$ computed from the calibration matrix $\mathbf{A}$, we have $\mathbf{r}=\mathbf{B}\mathit{\gamma }+{\sigma }_n^2\mathrm{vec}({\mathbf{I}}_N)$. Considering K finite samples, the covariance matrix is estimated by ${\widehat{\mathbf{R}}}_{xx}=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$K$}\right.{\sum }_{k=1}^K\mathbf{x}(t_k){\mathbf{x}}^H(t_k)$, and we denote $\widehat{\mathbf{r}}$ the associated estimated vectorized covariance matrix. Let us suppose that the power of the noise is known. We consider the noisy observation vector $\mathbf{y}\in {\mathbb{C}}^{N^2}$ computed as:

$$\mathbf{y}=\widehat{\mathbf{r}}-{\sigma }_n^2\mathrm{vec}({\mathbf{I}}_N)=\mathbf{B}\mathit{\gamma }+\mathbf{w},$$

(8)

with $\mathbf{B}\in {\mathbb{C}}^{N^2\times G}$. The noise vector $\mathbf{w}$ results from the estimation of ${\mathbf{R}}_{xx}$ with a finite number of snapshots. The power vector $\mathit{\gamma }\in {\mathbb{C}}^G$ is sparse and the indices of non-zero components indicate the directions of the sources. The aim of sparse DOA estimation is to retrieve the indices of non-zero components of vector $\mathit{\gamma }$ in Equation (8), through the resolution of the problem given by Equation (1).

3. Description and Numerical Investigations of the Minimizers of ${\mathit{J}}_{{\mathbf{\ell }}_{\mathbf{0}}}$ and ${\mathit{J}}_{\mathbf{CEL}\mathbf{0}}$

It is known that there are numerous local minima in $J_{{\ell }_0}$, which complicates the minimization of the criterion. Initialization of iterative descent algorithms is particularly delicate, as also highlighted in the DOA estimation literature. In [12], we have successfully used CEL0 penalty for DOA estimation of well-separated sources. Those good results do not seem to be transposed to the case of close sources. It is important now to analyze more deeply the minimizers of ${\ell }_0$ and CEL0 penalized problems in order to propose an optimization scheme able to resolve closer sources.

3.1. Simulation Setup

Although our approach is array and scenario independent, we illustrate it in this paper with the following setup. We consider an Uniform Circular Array (UCA) with $N=7$ antennas and radius $d=\raisebox{1ex}{${\lambda }_0$}\!\left/ \!\raisebox{-1ex}{$2$}\right.$, where ${\lambda }_0$ is the wavelength. This array could allow for two-dimensions direction-of-arrival estimation, but we limit ourselves to azimuth estimation. The −3 dB beamwidth of this array is 40°. UCAs are well known for their $\theta$ invariant performance. We study the case of $M=2$ incoming sources located at ${\tilde{\theta }}_1$ = 32° and a varying ${\tilde{\theta }}_2$. The number of snapshots is fixed to $K=50$. In this part, the received signal is noiseless. The field-of-view is the range [0, 360]° with a grid spacing of 0.5° ($G=720$). The mutual coherence, corresponding to the maximum absolute correlation between two columns of the dictionary, is in this case close to 1: ${\ell }_1$ methods are thus ineffective.

3.2. Minimizers of $J_{{\ell }_0}$

Let us define ${\mathbb{I}}_G=\left\{1,\dots ,G\right\}$ the ordered index set. For a given observation $\mathbf{y}\in {\mathbb{C}}^G$ and a set $\omega \subseteq {\mathbb{I}}_G$, we define the constrained problem $({\mathcal{C}}_{\omega })$ as follows:

$$({\mathcal{C}}_{\omega }):\underset{\mathit{\gamma }}{min}{\parallel \mathbf{B}\mathit{\gamma }-\mathbf{y}\parallel }_2^2,\mathrm{s}.\mathrm{t}.{\gamma }_i=0,\forall i\in {\mathbb{I}}_G\setminus \omega$$

(9)

where ${\gamma }_i$ is the ith component of vector $\mathit{\gamma }$. Let us note ${\widehat{\mathit{\gamma }}}_{\omega }$ the subvector of $\widehat{\mathit{\gamma }}$ composed only with the terms indicated by $\omega$: ${\widehat{\mathit{\gamma }}}_{\omega }={\left({\widehat{\gamma }}_{{\omega }_1},\dots ,{\widehat{\gamma }}_{{\omega }_{♯\omega }}\right)}^T$, with $♯\omega$ the number of elements in $\omega$. The constraint ensures the solutions $\widehat{\mathit{\gamma }}$ are sparse vectors whose indices of all non zero entries are in $\omega$, i.e., all components whose indices are not in $\omega$ are null. We can then write $\widehat{\mathit{\gamma }}=Z_p({\widehat{\mathit{\gamma }}}_{\omega })$, where $Z_p$ is a zero padding operator in ${\mathbb{I}}_G$:

$${\widehat{\gamma }}_i=\left\{\begin{array}{cc}\hfill 0& \mathrm{if}i\notin \omega \hfill \\ \hfill {\widehat{\gamma }}_{{\omega }_k}& \mathrm{for}\mathrm{the}\mathrm{unique}k\mathrm{such}\mathrm{that}{\omega }_k=i.\hfill \end{array}\right.$$

(10)

For any $\omega \subseteq {\mathbb{I}}_G$, $\widehat{\mathit{\gamma }}\in {\mathbb{C}}^G$ solves $({\mathcal{C}}_{\omega })$ if and only if ${\widehat{\mathit{\gamma }}}_{\omega }\in {\mathbb{C}}^{♯\omega }$ solves ${\mathbf{B}}_{\omega }^H{\mathbf{B}}_{\omega }\mathbf{x}={\mathbf{B}}_{\omega }^H\mathbf{y}$ and $\widehat{\mathit{\gamma }}=Z_p({\widehat{\mathit{\gamma }}}_{\omega })$. ${\mathbf{B}}_{\omega }$ is the submatrix of $\mathbf{B}$ composed only with the columns whose indices are in $\omega$.

There are strong connections between the minimizers of the constrained problem $({\mathcal{C}}_{\omega })$ and the minimizers of the regularized criterion $J_{{\ell }_0}(\lambda ,·)$ that we want to analyze. For $\mathbf{y}\in {\mathbb{C}}^N$, given a set $\omega \subseteq {\mathbb{I}}_G$, let $\widehat{\mathit{\gamma }}$ solve problem $({\mathcal{C}}_{\omega })$. Then for any $\lambda$, $J_{{\ell }_0}(\lambda ,·)$ reaches a (local) minimum at $\widehat{\mathit{\gamma }}$ (Proposition 2.3 [20]). Conversely, for $\mathbf{y}\in {\mathbb{C}}^N$ and $\lambda >0$, let $J_{{\ell }_0}(\lambda ,·)$ have a (local) minimum at $\widehat{\mathit{\gamma }}$. Then $\widehat{\mathit{\gamma }}$ solves $({\mathcal{C}}_{\widehat{\omega }})$ for $\widehat{\omega }=\mathrm{supp}(\widehat{\mathit{\gamma }})$ (Lemma 2.4 [20]). Moreover, the (local) minimum that $J_{{\ell }_0}(\lambda ,·)$ has at $\widehat{\mathit{\gamma }}$ is strict iff $\mathrm{rank}({\mathbf{B}}_{\widehat{\omega }})=♯\widehat{\omega }$ (Theorem 3.2 [20]). There is thus a large number of local minima: the number of supports $\omega \subseteq {\mathbb{I}}_G$ such that $\mathrm{rank}({\mathbf{B}}_{\omega })=♯\omega$ that lead to strict local minima is upper bounded by ${\sum }_{k=0}^{N^2}\left(\genfrac{}{}{0pt}{}{G}{k}\right)$.

In [20], it is shown that under mild conditions, $J_{{\ell }_0}(\lambda ,·)$ have a unique strict global minimizer. It is common knowledge that the optimal solution of the regularized problem given by Equation (1) depends on the regularization parameter $\lambda$, which balances the relative importance between data fidelity and sparsity. In most papers, this parameter is empirically tuned. In previous works [12,13], we proposed a theoretical analysis for an off-line selection of $\lambda$. In the sequel of this paper, $\lambda$ will be selected in an appropriate interval I as defined in [12,13].

Figure 1a–d represents projections of $J_{{\ell }_0}$ for the scenario described above, for ${\tilde{\theta }}_1$ = 32° and ${\tilde{\theta }}_2$ = 62° in the noiseless case. Iso-levels of $J_{{\ell }_0}$ are reported for a vector $\mathit{\gamma }$ having at most two non-zero components: $\mathit{\gamma }=Z_p({\mathit{\gamma }}_{\omega })$ with $♯\omega =2$. Those components correspond to fixed directions ${\theta }_{{\omega }_1}={\tilde{\theta }}_1$ = 32° and ${\theta }_{{\omega }_2}$ which changes on the different figures. On (a), ${\theta }_{{\omega }_2}$ = 47°, which corresponds to $\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.\left({\tilde{\theta }}_1+{\tilde{\theta }}_2\right)$; on (d), ${\theta }_{{\omega }_2}={\tilde{\theta }}_2$ = 62°. In between, we set: ${\theta }_{{\omega }_2}$ = 52° and 57°. The values of the two components ${\mathit{\gamma }}_{{\omega }_1}$ and ${\mathit{\gamma }}_{{\omega }_2}$, which are the only components allowed to be non-zero, are varying along the two axis. $\lambda$ is fixed to 9.5, which belongs to the interval I. In each figure, we see four (local) minima: the local minimum at $\mathbf{0}$, local minima along the axis (i.e., one non-zero component), and those corresponding to strictly two non-zero components. The global minimum (black filled circle) is located on Figure 1d for ${\gamma }_{{\omega }_1}={\gamma }_{{\omega }_2}=7$ and its value is $2\lambda =19$.

3.3. Minimizers of $J_{\mathrm{CEL}0}$

The global minimizer of $J_{{\ell }_0}$ is preserved in $J_{\mathrm{CEL}0}$, but the number of local minima of $J_{\mathrm{CEL}0}$ may be inferior to the number of local minima of $J_{{\ell }_0}$. Particularly, a local minimum $\mathit{\gamma }$ of $J_{\mathrm{CEL}0}$ verifies $|{\gamma }_i|\in \left\{0\right\}\cup [\sqrt{2\lambda },+\infty )$; hence local minimizers of $J_{{\ell }_0}$ having at least one component $|{\gamma }_i|\in (0,\sqrt{2\lambda })$ are not local minimizers of $J_{\mathrm{CEL}0}$ [21]. Figure 1e–h represents the loss surfaces of $J_{\mathrm{CEL}0}$. We observe the suppression of local minima on $J_{\mathrm{CEL}0}$ for ${\theta }_{{\omega }_2}$ close to $\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.\left({\tilde{\theta }}_1+{\tilde{\theta }}_2\right)$ = 47°, for which $0<{\gamma }_{{\omega }_2}<\sqrt{2\lambda }$ = 4.36. Some local minima of $J_{{\ell }_0}$ are also only critical points of $J_{\mathrm{CEL}0}$. Moreover, the local minima that $J_{{\ell }_0}$ has at $\mathbf{0}$ is no longer one in $J_{\mathrm{CEL}0}$, which is particularly interesting for the initialization of iterative optimization algorithms.

However, those good properties also have a disadvantage: it leads to “flat” minima, i.e., large connected regions where the error remains approximately constant. This is illustrated on Figure 2, comparing the minimum of $J_{{\ell }_0}$ and $J_{\mathrm{CEL}0}$ as a function of ${\theta }_{{\omega }_1}$ and ${\theta }_{{\omega }_2}$ for two (or one) non-zero components. Numerous points of the CEL0 surface are approximately at the level of the local minima corresponding to ${\theta }_{{\omega }_1}={\theta }_{{\omega }_2}=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.({\tilde{\theta }}_1+{\tilde{\theta }}_2)$. This behavior appears for close sources when this point is also close to the global minimum.

4. Alternating between Loss Surfaces

For well separated sources, we have shown in [12] than IRL1 algorithm used to minimize $J_{\mathrm{CEL}0}$ (IRL1-CEL0) gives better statistical results than IHT and at a lower computational cost. Indeed, IRL1-CEL0 benefits from the suppression of local minima in this case. Unfortunately, IRL1-CEL0 fails for close sources. This behavior is illustrated on Figure 3a, for true sources at 32° and 48°. Let us note that MUSIC fails for such close sources. The IRL1-CEL0 algorithm is rapidly attracted by a local bad basin corresponding to a few non-zero components for directions in the middle of the true directions (lack of resolution). In this example, denoting $\widehat{\mathit{\gamma }}$ the final estimated vector and $\widehat{\omega }$ the set indicating the non-zero components, we verify that ${\mathbf{B}}_{\widehat{\omega }}^H{\mathbf{B}}_{\widehat{\omega }}{\widehat{\mathit{\gamma }}}_{\widehat{\omega }}={\mathbf{B}}_{\widehat{\omega }}^H\mathbf{y}$ and $\mathrm{rank}({\mathbf{B}}_{\widehat{\omega }})=♯\widehat{\omega }=5$: it is a strict local minimum of $J_{{\ell }_0}$. The IHT algorithm also remains stuck in a local minimum with this time numerous non-zero components (Figure 3b), forming two clusters around the true directions. In order to avoid being attracted by a bad basin and take advantage of both regularizations, we propose to alternate the minimization between them. Based on this heuristic, we propose the optimization scheme ALICE-L0 (Alternated Landscapes Iterations for Complementary Enhancement for ${\ell }_0$) detailed on Algorithm 1.

Algorithm 1. Optimization Scheme ALICE-L0 (Alternated Landscapes Iterations for Complementary Enhancement for ${\ell }_0$)

Input: dictionary $\mathbf{B}$, observation $\mathbf{y}$, $\beta =\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$L$}\right.$, L Lipschitz-constant of $\mathbf{B}$, ${\tau }_1$, ${\tau }_2$, stopping criteria

Initialization:${\widehat{\mathit{\gamma }}}^{(0)}=\mathbf{0}$, $i=0$, $i_{outer}=0$

• ${ϵ}^{(i)}={\parallel \mathbf{B}{\widehat{\mathit{\gamma }}}^{(i)}-\mathbf{y}\parallel }_2^2$

while $\frac{|{ϵ}^{(i-1)}-{ϵ}^{(i-2)}|}{{ϵ}^{(i-1)}}>{ϵ}_{lim}$ and $i_{outer}<n_{lim}$ do

• $i_{outer}=i_{outer}+1$

• Compute $\mathbf{w}$ weighting vector by:     $w_g=(\sqrt{2\lambda }-|{\gamma }_g^{(i)}|){𝟙}_{\left\{|{\gamma }_g^{(i)}|\le \sqrt{2\lambda }\right\}}$, $g=1..G$

• $j=1$, $T^{(1)}=1$, ${\mathbf{z}}^{(1)}={\widehat{\mathit{\gamma }}}^{(i)}$

while $\frac{|{ϵ}^{(i-1)}-{ϵ}^{(i-2)}|}{{ϵ}^{(i-1)}}>{ϵ}_{lim,1}$ and $j<n_{lim,1}$ do             (weighted FISTA iterations)

• $j=j+1$, $i=i+1$

• ${\widehat{\mathit{\gamma }}}^{(i)}={prox}_{{\parallel ·\parallel }_1,\lambda \beta \mathbf{w}}\left({\mathbf{z}}^{(j-1)}-\beta {\mathbf{B}}^H(\mathbf{B}{\mathbf{z}}^{(j-1)}-\mathbf{y})\right)$

• $T^{(j)}=\frac{1+\sqrt{1+{(2T^{(j-1)})}^2}}{2}$

• ${\mathbf{z}}^{(j)}={\widehat{\mathit{\gamma }}}^{(i)}+\frac{T^{(j-1)}-1}{T^{(j)}}\left({\widehat{\mathit{\gamma }}}^{(i)}-{\widehat{\mathit{\gamma }}}^{(i-1)}\right)$

end while

• $k=0$

while $\frac{|{ϵ}^{(i-1)}-{ϵ}^{(i-2)}|}{{ϵ}^{(i-1)}}>{ϵ}_{lim,2}$ and $k<n_{lim,2}$ do             (IHT iterations)

• $k=k+1$, $i=i+1$

• ${\widehat{\mathit{\gamma }}}^{(i)}={prox}_{{\parallel ·\parallel }_0,\lambda \beta }\left({\widehat{\mathit{\gamma }}}^{(i-1)}-\beta {\mathbf{B}}^H(\mathbf{B}{\widehat{\mathit{\gamma }}}^{(i-1)}-\mathbf{y})\right)$

end while

• ${ϵ}_{lim,1}={\tau }_1{ϵ}_{lim,1}$, ${ϵ}_{lim,2}={\tau }_2{ϵ}_{lim,2}$

end while

return ${\widehat{\mathit{\gamma }}}^{(i_{end})}$

We start the minimization considering the CEL0-regularized functional, and using ${\widehat{\mathit{\gamma }}}^{(0)}=\mathbf{0}$ as initialization. Indeed, we previously saw that this local minimum in $J_{{\ell }_0}$ is suppressed in $J_{\mathrm{CEL}0}$. Iterations of the weighted Fast Iterative Soft Thresholding Algorithm (weighted FISTA) aim to minimize the convex majorizer of the nonconvex CEL0 functional. For a weighting vector $\mathbf{w}$, iterations use the proximal of the weighted ${\ell }_1$ function, defined component by component as:

$${\left[{prox}_{{\parallel ·\parallel }_1,\lambda \beta \mathbf{w}}\right]}_g(\mathbf{x})=max\left\{0,1-\frac{\lambda \beta \mathbf{w}}{|x_g|}\right\}{x}_g{\mathbb{1}}_{\left\{|x_g|\ne 0\right\}}.$$

(11)

After some iterations, the estimated vector is used as initialization for IHT, which performs minimization steps over the ${\ell }_0$-regularized cost function. The hard threshold corresponds to the proximal of the ${\ell }_0$-norm, defined by:

$${\left[{prox}_{{\parallel ·\parallel }_0,\lambda \beta }\right]}_g(\mathbf{x})=x_g{\mathbb{1}}_{\left\{|x_g|\ge \sqrt{\lambda \beta }\right\}}.$$

(12)

We then loop back to alternate between the loss surfaces. The behavior of our algorithm is represented on Figure 3c for close sources (in this article, we set ${ϵ}_{lim}=1\times {10}^{-6}$, ${ϵ}_{lim,1}=1\times {10}^{-2}$, ${ϵ}_{lim,2}=1\times {10}^{-6}$, $n_{lim}=2000$, $n_{lim,1}=200$, $n_{lim,2}=200$, ${\tau }_1=0.9$, ${\tau }_2=1$). In this noiseless case, this is the only algorithm attaining the global minimum of $J_{{\ell }_0}$.

5. Statistical Performance

The purpose of this section is to numerically quantify the algorithms’ performance as a function of the sources’ separation. For that, two criteria will be used: the first one is the percentage of outliers (only one estimated direction or located at more than half a beamwidth of the true directions), and the second one the Root-Mean-Square-Error (RMSE) between estimated and true directions, calculated without outliers. The simulation setup is the one described in Section 3.1, for a Signal-to-Noise Ratio per source equal to 0 dB. Results are presented in Figure 4: we observe that the proposed scheme ALICE-L0 outperforms other methods in terms of statistical accuracy and resolution limit. Let us note that the behavior of IHT is unreliable with noise, and thus statistical results are not presented.

6. Conclusions

We linked the operating limits of IHT and IRL1-CEL0 to the properties of corresponding loss landscapes in DOA estimation. To avoid the weaknesses of both criteria, an optimization scheme is proposed using alternatively $J_{{\ell }_0}$ and $J_{\mathrm{CEL}0}$, i.e., alternating between the two regularizations. A particular implementation using $\lambda$ obtained by [12,13] has been successfully tested, improving for example the resolution limit. Ongoing work concerns algorithm parameters (when to change of regularization) which are here left to the user.