On an Extension of a Spare Regularization Model

Abstract

In this paper, we would first like to promote an interesting idea for identifying the local minimizer of a non-convex optimization problem with the global minimizer of a convex optimization one. Secondly, to give an extension of their sparse regularization model for inverting incomplete Fourier transforms introduced. Thirdly, following the same lines, to develop convergence guaranteed efficient iteration algorithm for solving the resulting nonsmooth and nonconvex optimization problem but here using applied nonlinear analysis tools. These both lead to a simplification of the proofs and to make a connection with classical works in this filed through a startling comment.

1. Introduction

Compressed sensing (see, for example, [1,2,3,4,5,6,7]), was used to invert incomplete Fourier transforms in the context of sparse signal/image processing, and the $l_1$-norm was applied as a regularization for reconstructing an object from randomly selected incomplete frequency samples. Both the sparse regularization method and the compressed sensing method use the $l_1$-norm as a regularization to impose sparsity for the reconstructed signal under certain transforms. Because the models based on the $l_1$-norm are convex, they can be solved efficiently by available algorithms. Recently, the application of non-convex metrics as alternative approaches to $l_1$ norm has been favored, see for example, [8,9,10,11]. The main goal of this paper is to suggest the employ of the Moreau envelope associated with the $l_0$-norm as a regularization. Note that the sparsity of a vector is originally measured by the $l_0$-norm of the vector, i.e., the number of its nonzero components. However, the $l_0$-norm is discontinuous at the origin, which is not appropriate from a computational point of view. The envelope of the $l_0$-norm is a Lipschitz surrogate of the $l_0$-norm, which is nonconvex. Through [7], a local minimizer of a function that is the sum of a convex function and the $l_0$-norm can be identified with a global minimizer of a convex function which permits algorithmic development of convex optimization problems. For inverting incomplete Fourier transforms, the use of the $l_0$-norm allows to formulate a sparsity regularization model that can reduce artifacts and outliers in the reconstructed signal. It also allow us to design an efficient algorithm for the resulting nonconvex and nonsmooth optimization problem by means of a fixed-point formulation. Moreover, the link of this minimization problem with the related convex minimization problem will permit to prove convergence of our proposed algorithm. Furthermore, a connection with proximal/projection gradient methods is also provided by appealing to two key formulas.

2. A Sparse Regularization Model

In order to obtain to the essential information to share, we took the same paper outline as in [6] and we assume the reader has some basic knowledge of monotone operator theory and convex analysis as can be found, for example, in [12,13,14,15].

In what follows, we propose an extension of a sparse regularization model based on the Moreau envelope of the $l_0$-norm for inverting incomplete Fourier transforms considered in [6]. Likewise, relying on properties of the Moreau envelope of the $l_0$-norm, we obtain an equivalent formulation favorable for algorithmic development.

Given two Euclidean spaces of dimensions N and d, a nonempty, closed and convex subset $Q\subset I{R}^d$ and a matrix $K:I{R}^N\to I{R}^d$, we are interested in this work regarding the following problem:

$$\mathrm{Find}y\in I{R}^N\mathrm{such} \mathrm{that}Ky\in Q,$$

(1)

This formalism is also at the heart of the modeling of many inverse problems posed by phase recovery problems and other real-world problems, see [16] and references therein.

Our job is to describe the sparse regularization model for Equation (1) in order to obtain a sparse vector y. The $l_0$-norm, which counts the number of nonzero components of a vector $x\in I{R}^N$, is naturally used to measure its sparsity and is defined by

$${\parallel x\parallel }_0=\sum _{i=1}^N{\left|x_i\right|}_0,$$

with $|x_i{|}_0=1$ if $x_i\ne 0$ and $|x_i{|}_0=0$ if $x_i=0$.

Now, let $P_Q$ be the projection from $I{R}^N$ onto the set Q. Since the constraint is equivalent to the fact that $Ky-P_Q\left(Ky\right)=0$, we derive the following equivalent Lagrangian formulation

$$\underset{y\in I{R}^N}{min}\frac{1}{2}\parallel (I-P_Q){Ky\parallel }^2+\gamma {\parallel y\parallel }_0,$$

(2)

with $\gamma >0$ a Lagrangian multiplier.

Both non-convexity and discontinuity of the $l_0$-norm at the origin lead to computational difficulties. To overcome these problems, we use a Lipschitz regularization of the $l_0$-norm by its Moreau envelope. According to [14,17], for a positive number $\lambda$, the Moreau envelope of ${\parallel ·\parallel }_0$ with index $\lambda$ at $x\in I{R}^N$ is defined by

$$en{v}_{{\lambda \parallel ·\parallel }_0}\left(x\right)=\underset{z\in I{R}^N}{min}\left({\parallel z\parallel }_0+\frac{1}{2\lambda }{\parallel x-z\parallel }^2\right).$$

(3)

$en{v}_{{\lambda \parallel ·\parallel }_0}$ is continuous and locally convex near the origin. Moreover, as ${lim}_{\lambda \to 0}en{v}_{{\lambda \parallel ·\parallel }_0}={\parallel ·\parallel }_0$, $en{v}_{{\lambda \parallel ·\parallel }_0}$ is a good approximation of ${\parallel ·\parallel }_0$ when $\lambda$ is small enough. Therefore, with an appropriate choice of the parameter $\lambda$, $en{v}_{{\lambda \parallel ·\parallel }_0}$ can be used as a measure of sparsity and allows to avoid drawbacks ${\parallel ·\parallel }_0$. For a fixed $Q\subset I{R}^d$ and for $y\in I{R}^N$, we let

$$H\left(y\right)=\frac{1}{2}{\parallel (I-P_Q)Ky\parallel }^2+\gamma en{v}_{{\lambda \parallel ·\parallel }_0}\left(y\right),$$

(4)

where $\gamma$ is a positive parameter.

To recover a sparse vector y from (1), we now propose the sparse regularization model based on the Moreau envelope of the $l_0$-norm

$$\overline{y}=argmi{n}_{y\in I{R}^N}H\left(y\right).$$

(5)

Since $en{v}_{{\lambda \parallel ·\parallel }_0}$ is an approximation of ${\parallel ·\parallel }_0$, we expect that the proposed model enjoys nice properties and can be solved by efficient iteration algorithms.

As was pointed out earlier, $en{v}_{{\lambda \parallel ·\parallel }_0}$ is an excellent sparsity promoting function. Therfore, we adopt $\mathit{v}=en{v}_{{\lambda \parallel ·\parallel }_0}\left(y\right)$ in this paper.

We reformulate problem (5) to obtain a problem that is well suited and favorable for computation. Relying on definition (3) of $en{v}_{{\lambda \parallel ·\parallel }_0}$ in problem (5) and r being fixed, we introduce the following function

$$F(x,y)=\frac{1}{2}\parallel (I-P_Q){Ky\parallel }^2+\frac{\gamma }{2\lambda }{\parallel x-y\parallel }^2+\gamma {\parallel x\parallel }_0.$$

(6)

The non-convex function $F(x,y)$ is a special case of those considered in [7]. We then consider the problem

$$(\overline{x},\overline{y})=argmi{n}_{(x,y)\in I{R}^N\times I{R}^N}F(x,y).$$

(7)

Next, we prove that problems (5) and (7) are essentially equivalent. A global minimizer of any of these problems will also be called a solution of the problem. We first present a relation between $H\left(y\right)$ and $F(x,y)$. Remember that for $\lambda >0$, the proximity operator of ${\parallel ·\parallel }_0$ at $z\in I{R}^N$ is defined by

$$pro{x}_{{\lambda \parallel ·\parallel }_0}\left(z\right)=argmi{n}_{x\in I{R}^N}{\{\parallel x\parallel }_0+\frac{1}{2\lambda }{\parallel x-z\parallel }^2\}.$$

(8)

Clearly, if $x\in pro{x}_{{\lambda \parallel ·\parallel }_0}\left(z\right)$, then we have that

$$en{v}_{{\lambda \parallel ·\parallel }_0}\left(z\right)={\parallel x\parallel }_0+\frac{1}{2\lambda }{\parallel x-z\parallel }^2.$$

(9)

By relation (9), we obtain

$$H\left(y\right)=F(x,y),\forall x\in pro{x}_{{\lambda \parallel ·\parallel }_0}\left(y\right)\mathrm{and}\forall y\in I{R}^N.$$

(10)

We now give a direct proof of [6], Proposition 1.

Proposition 1.

Let $\lambda >0$ and $\gamma >0$. A pair $(\overline{x},\overline{y})$ solves problem (7) if, and only if, $\overline{y}$ solves problem (5) with $\overline{x}$, verifying the following relation

$$\overline{x}\in pro{x}_{{\lambda \parallel ·\parallel }_0}\left(\overline{y}\right).$$

Proof. 

This follows directly from the following successive equalities.

$$\begin{array}{ccc}\hfill \underset{(x,y)\in I{R}^N\times I{R}^N}{inf}F(x,y)& =& \underset{(x,y)\in I{R}^N\times I{R}^N}{inf}\left(\frac{1}{2}\parallel (I-P_Q){Ky\parallel }^2+\frac{\gamma }{2\lambda }{\parallel x-y\parallel }^2+\gamma {\parallel x\parallel }_0\right)\hfill \\ & =& \underset{y\in I{R}^N}{inf}\left(\frac{1}{2}{\parallel (I-P_Q)Ky\parallel }^2+\gamma \underset{x\in I{R}^N}{inf}\left(\frac{1}{2\lambda }{\parallel x-y\parallel }^2+{\parallel x\parallel }_0\right)\right).\hfill \\ & =& \underset{y\in I{R}^N}{inf}\left(\frac{1}{2}{\parallel (I-P_Q)Ky\parallel }^2+\gamma en{v}_{{\lambda \parallel ·\parallel }_0}\left(y\right)\right).\hfill \end{array}$$

   □

Based on the fact that problems (5) and (7) are essentially equivalent, it suffices to establish that a local minimizer of the nonconvex problem (7) is a minimizer of a convex problem on a subdomain. To that end, we first present a convex optimization problem on a proper subdomain of $I{R}^N\times I{R}^N$ related to problem (7) and recall the notion of the support of a vector $x\in I{R}^N$, denoted by $N\left(x\right)$, namely the index set on which the components of x is nonzero, that is $N\left(x\right)=\{i:x_i\ne 0\}$. Note that when the support of x in problem (7) is specified, the non-convex problem (7) reduces to a convex one. Based on this observation, we introduce a convex function by

$$G(x,y)=\frac{1}{2}\parallel (I-P_Q){Ky\parallel }^2+\frac{\gamma }{2\lambda }{\parallel x-y\parallel }^2,(x,y)\in I{R}^N\times I{R}^N.$$

(11)

Clearly, $F(x,y)=G(x,y)+{\gamma \parallel x\parallel }_0$ and $G(x,y)$ is convex and differentiable on $I{R}^N\times I{R}^N$.

We define now, for a given index set $\mathit{N}$, a subspace of $I{R}^N$ by setting

$${\mathit{B}}_{\mathit{N}}=\{x\in I{R}^N,N\left(x\right)\subset \mathit{N}\}.$$

(12)

${\mathit{B}}_{\mathit{N}}$ is convex and closed (see [6]), and we consider then the minimization problem on ${\mathit{B}}_{\mathit{N}}\times I{R}^N$ defined by

$$argmin\{G(x,y),(x,y)\in (x,y)\in {\mathit{B}}_{\mathit{N}}\times I{R}^N\}.$$

(13)

Problem (13) is convex, thanks to the convexity of both the function G and the set ${\mathit{B}}_{\mathit{N}}\times I{R}^N$. Next, we will show the equivalence between the non-convex problem (7) and the convex problem (13) with an appropriate choose of the index set $\mathit{N}$. To this end, we investigate properties of the support set of certain sequences in $I{R}^N$ and for a given index set $\mathit{N}$, we define an operator $P_{{\mathit{B}}_{\mathit{N}}}:I{R}^N\to {\mathit{B}}_{\mathit{N}}$ by

$$P_{{\mathit{B}}_{\mathit{N}}}\left(y\right)=y_{i}ifi\in \mathit{N}\mathrm{and}0\mathrm{otherwise}.$$

(14)

This operator is indeed the orthogonal projection from $I{R}^N$ onto $\mathit{N}$, (see [6] Lemma 3). A convenient identification of the proximity operator of the $l_0$-norm with the projection $P_{B_{\mathit{N}}}$ and some properties of the sequence generated by $pro{x}_{{\lambda \parallel ·\parallel }_0}$, with respect to the existence of an integer which will denote $\overline{\kappa }$, were developed in (Lemmas 4–7 together with Proposition 2 [6]), and which are still valid in our context.

Recall also the closed form formula of the proximity of $l_0$. For all $z\in I{R}^N$,

$$pro{x}_{{\lambda |·|}_0}\left(z\right)=\left\{\begin{array}{cc}\left\{z_i\right\}\hfill & \mathrm{if}|z_i|>\sqrt{2\lambda };\hfill \\ \{z_i,0\}\hfill & \mathrm{if}|z_i|=\sqrt{2\lambda };\hfill \\ \left\{0\right\}\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(15)

A connection between problems (7) and (13) is given by the following result.

Theorem 1.

$\lambda ,\gamma >0$, and $(\overline{x},\overline{y})\in I{R}^N\times I{R}^N$ be given. The pair $(\overline{x},\overline{y})$ is a local minimizer of the non-convex problem (7) if, and only if, $(\overline{x},\overline{y})$ is a minimizer of the convex problem (13) with $\mathit{N}:=N\left(\overline{x}\right)$.

Proof. 

Follows directly by using ([6] Corollary 4.9), with $\varphi \left(y\right):=\frac{1}{2}{\parallel (I-P_Q)Ky\parallel }^2,\mu :=\frac{\gamma }{2\lambda }$ and $D:=I$.   □

Following the same lines of ([6] Propositions 1 and 3), we can identify and connect global and local minimizers of (7) with those of (5).

3. A Fixed Point Approach

We will propose an iterative method for finding a local minimizer of (7) relying on a fixed-point formulation. For all the facts we will use, we refer to [14].

Let us begin with a characterization of the convex problem (13).

Proposition 2.

Suppose $\lambda ,\gamma >0$. If $\mathit{C}\subset \{1,2,···,N\}$, then the problem (13) with $\mathit{N}:=\mathit{C}$ has a solution and a pair $(\overline{x},\overline{y})\in I{R}^N\times I{R}^N$ solves (13) with $\mathit{N}:=\mathit{C}$ if, and only if,

$$\overline{x}=P_{\mathit{N}\left(\overline{\mathit{x}}\right)}\left(\overline{y}\right)\mathrm{and}\overline{y}=\overline{x}-\frac{\lambda }{\gamma }{K}^{*}(I-P_Q)K\overline{y}.$$

(16)

Proof. 

The existence of solutions follows by the fact that ${\mathit{B}}_{\mathit{N}}$ is compact together with the coercivity of G with respect to the second variable. On the other hand, the optimality condition of the minimization problem

$$\underset{(x,y)\in {\mathit{B}}_{\mathit{N}}\times I{R}^N}{min}\left(\frac{1}{2}\parallel (I-P_Q){Ky\parallel }^2+\frac{\gamma }{2\lambda }{\parallel x-y\parallel }^2\right),$$

reads as

$$(0,0)\in \left(K^{*}(I-P_Q)K\overline{y}-\frac{\gamma }{\lambda }(\overline{x}-\overline{y}),\frac{\gamma }{\lambda }(\overline{x}-\overline{y})+N_{{\mathit{B}}_{\mathit{N}}}\left(\overline{x}\right)\right),$$

or, equivalently,

$$\overline{y}=\overline{x}-\frac{\lambda }{\gamma }{K}^{*}(I-P_Q)K\overline{y}\mathrm{and}\overline{y}\in \overline{x}+\frac{\lambda }{\gamma }{N}_{{\mathit{B}}_{\mathit{N}}}\left(\overline{x}\right)\iff \overline{x}={(I+\frac{\lambda }{\gamma }{N}_{{\mathit{B}}_{\mathit{N}}})}^{-1}\left(\overline{y}\right)=P_{{\mathit{B}}_{\mathit{N}}}\left(\overline{y}\right).$$

   □

Application of both Theorem 1 and Proposition 2 leads to the following characterization of a local minimizer of the problem (7).

Theorem 2.

Let $\lambda ,\gamma >0$ be fixed. A pair $(\overline{x},\overline{y})\in I{R}^N\times I{R}^N$ is a local minimizer of (7) if, and only if, $(\overline{x},\overline{y})$ verifies (16).

Let us now give a characterization of a global minimizer of (16).

Theorem 3.

Let $\lambda ,\gamma >0$ be fixed. If a pair $(\overline{x},\overline{y})\in I{R}^N\times I{R}^N$ is a local minimizer of (7), then $(\overline{x},\overline{y})$ satisfies the relations

$$\overline{x}=pro{x}_{{\lambda \parallel ·\parallel }_0}\left(\overline{y}\right)\mathrm{and}\overline{y}=\overline{x}-\frac{\lambda }{\gamma }{K}^{*}(I-P_Q)K\overline{y}.$$

(17)

Conversely, if a pair $(\overline{x},\overline{y})$ verifies (17), then If a pair $(\overline{x},\overline{y})$ is a local minimizer of (7).

Proof. 

The optimality condition of the minimization problem

$$\underset{(x,y)\in I{R}^N\times I{R}^N}{min}\left(\frac{1}{2}\parallel (I-P_Q){Ky\parallel }^2+\frac{\gamma }{2\lambda }{\parallel x-y\parallel }^2+\gamma {\parallel x\parallel }_0\right),$$

reads as

$$(0,0)\in \left(K^{*}(I-P_Q)K\overline{y}-\frac{\gamma }{\lambda }(\overline{x}-\overline{y}),\frac{\gamma }{\lambda }(\overline{x}-\overline{y})+\gamma \partial {\parallel ·\parallel }_0\left(\overline{x}\right)\right),$$

or, equivalently,

$$\overline{y}=\overline{x}-\frac{\lambda }{\gamma }{K}^{*}(I-P_Q)K\overline{y}\mathrm{and}\overline{y}\in \overline{x}{+\lambda \partial \parallel ·\parallel }_0\left(\overline{x}\right)\iff \overline{x}{\in (I+\lambda \partial \parallel ·\parallel }_0{)}^{-1}\left(\overline{y}\right)=pro{x}_{{\lambda \parallel ·\parallel }_0}\left(\overline{y}\right).$$

   □

In view of Theorem 3, we propose the following explicit–implicit Algorithm for solving problem (7)

$$x_{k+1}\in pro{x}_{{\lambda \parallel ·\parallel }_0}\left(y_k\right)\mathrm{and}{y}_{k+1}=x_{k+1}-\frac{\lambda }{\gamma }{K}^{*}(I-P_Q)K{y}_{k+1}$$

(18)

When the projection can be computed efficiently, updates of both variables x and y in Algorithm (18) at each iteration can be efficiently implemented.

Proposition 3.

If $\lambda ,\gamma >0$, then the operator $I+\frac{\lambda }{\gamma }{K}^{*}(I-P_Q)K$ is invertible. Hence, the second part of (18) reads as

$$y_{k+1}:=J_{\frac{\lambda }{\gamma }}^{K^{*}(I-P_Q)K}\left(x_{k+1}\right)={(I+\frac{\lambda }{\gamma }{K}^{*}(I-P_Q)K)}^{-1}\left(x_{k+1}\right).$$

(19)

Proof. 

It is well known that $K^{*}(I-P_Q)K$ is a maximal monotone operator (more precisely, it is an inverse strongly monotone operator, see the beginning of the proof of Theorem 4), hence $I+\frac{\lambda }{\gamma }{K}^{*}(I-P_Q)K$ is invertible by Minty Theorem and its inverse, which is the so-called resolvent operator, is single valued and firmly nonexpansive.   □

4. Convergence Analysis

In this section, we investigate the convergence behavior of Algorithm (18). As in [6], after a finite number of iterations, the support of the sparse variable $x_k$ defined by Algorithm (18) will remain unchanged, and hence solving the non-convex optimization problem (7) by algorithm (18) reduces to solving a convex optimization problem on the support.

First, we consider a function E, which is closely related to both functions F and G, and we define

$$E:I{R}^N\to IR\mathrm{at}y\in I{R}^N\mathrm{by}E\left(y\right):=\frac{L}{2}{\parallel (I-P_Q)Ky\parallel }^2,$$

(20)

where $L:=1+\frac{\lambda }{\gamma }$; $E\left(y\right)$ will be denoted for short by $E\left(y\right)$.

Now, we prove a convergence result of Algorithm (18).

Theorem 4.

Let ${(x_k,y_k)}_{k\in IN}$ be a sequence generated by Algorithm (18) with an initial $(x_0,y_0)\in I{R}^N\times I{R}^N$ for problem (7). If $\lambda ,\gamma >0$ are positive numbers, then we have the following properties

1. 

$F(x_{k+1},y_{k+1})\le F(x_k,y_k)$ for all $k\ge 0$ and the sequence ${\left(F(x_k,y_k)\right)}_{k\in IN}$ is convergent;

2. 

The sequence ${(x_k,y_k)}_{k\in IN}$ has a finite length, namely

$$\sum _{k=0}^{+\infty }\parallel x_{k+1}-x_k{\parallel }^2<+\infty ,and\sum _{k=0}^{+\infty }{\parallel y_{k+1}-y_k\parallel }^2<+\infty ,$$

(21)

3. 

The sequence ${(x_k,y_k)}_{k\in IN}$ is asymptotically regular, that is

$$\underset{k\to +\infty }{lim}\parallel x_{k+1}-x_k\parallel =0and\underset{k\to +\infty }{lim}{\parallel y_{k+1}-y_k\parallel }^2=0.$$

Proof. 

The function E is differentiable with a 1-Lipschitz continuous gradient. Indeed,

$$\begin{array}{ccc}\hfill \langle K^{*}(I-P_Q)K\left(x\right)-K^{*}(I-P_Q)K\left(y\right),x-y\rangle & =& \langle (I-P_Q)K\left(x\right)-(I-P_Q)K\left(y\right),Kx-Ky\rangle \hfill \\ & \ge & \parallel (I-P_Q)K\left(x\right)-(I-P_Q)K\left(y\right){\parallel }^2\hfill \\ & \ge & \parallel K^{*}(I-P_Q)K\left(x\right)-K^{*}(I-P_Q)K\left(y\right){\parallel }^2.\hfill \end{array}$$

This ensures that $\nabla E$ is 1-Lipschitz continuous.

On the other hand, since $y_{k+1}=x_{k+1}-\frac{\lambda }{\gamma }{K}^{*}(I-P_Q)K{y}_{k+1}$, we can write

$$\frac{\gamma }{2\lambda }\parallel x_{k+1}-y_{k+1}{\parallel }^2=\frac{\gamma }{2\lambda }{\parallel \frac{\lambda }{\gamma }{K}^{*}(I-P_Q)K{y}_{k+1}\parallel }^2.$$

Taking into account definition of the $\parallel ·\parallel$ and the fact that $K{K}^{*}=I$, we obtain that

$$\frac{\gamma }{2\lambda }\parallel x_{k+1}-y_{k+1}{\parallel }^2=\frac{\lambda }{2\gamma }{\parallel (I-P_Q)K{y}_{k+1}\parallel }^2.$$

Hence,

$$F(x_{k+1},y_{k+1})=\frac{1}{2}(1+\frac{\lambda }{\gamma })\parallel (I-P_Q)K{y}_{k+1}{\parallel }^2+\gamma \parallel x_{k+1}{\parallel }_0=E\left(y_{k+1}\right)+\gamma {\parallel x_{k+1}\parallel }_0.$$

Using the celebrate descent Lemma, see for example [12], we can write

$$F(x_{k+1},y_{k+1})\le E\left(y_k\right)+\langle \nabla E\left(y_k\right),y_{k+1}-y_k\rangle +\frac{L}{2}\parallel y_{k+1}-y_k{\parallel }^2+\gamma {\parallel x_{k+1}\parallel }_0.$$

(22)

Now, we have

$$\begin{array}{ccc}\hfill \frac{\gamma }{\lambda +\gamma }\langle \nabla E\left(y_k\right),x_{k+1}-x_k\rangle & =& \langle K^{*}(I-P_Q)K{y}_k,y_{k+1}-y_k+\frac{\lambda }{\gamma }(K^{*}(I-P_Q)K{y}_{k+1}-K^{*}(I-P_Q)K{y}_k)\rangle \hfill \\ & =& \langle K^{*}(I-P_Q)K{y}_k,y_{k+1}-y_k\rangle +\frac{\lambda }{\gamma }\langle K^{*}(I-P_Q)K{y}_k,y_{k+1}-y_k\rangle \hfill \\ & -& \frac{\lambda }{\gamma }\langle (I-P_Q)K{y}_k,P_{Q}K{y}_{k+1}-P_{Q}K{y}_k\rangle \hfill \\ & \ge & (1+\frac{\lambda }{\gamma })\langle K^{*}(I-P_Q)K{y}_k,y_{k+1}-y_k\rangle =\langle \nabla E\left(y_k\right),y_{k+1}-y_k\rangle .\hfill \end{array}$$

The Characterization of the orthogonal projection, namely

$$\langle (y-P_{Q}y,z-P_{Q}y\rangle \le 0\forall z\in Q$$

assures that

$$\langle (I-P_Q)K{y}_k,P_{Q}K{y}_{k+1}-P_{Q}K{y}_k\rangle \le 0,$$

and thus

$$\langle \nabla E\left(y_k\right),y_{k+1}-y_k\rangle \le \frac{\gamma }{\lambda +\gamma }\langle \nabla E\left(y_k\right),x_{k+1}-x_k\rangle .$$

(23)

Now, by using the second equation of (18) and by taking into account the fact that $I-P_Q$ is firmly nonexpansive and that $K{K}^{*}=I$, we can write

$$\begin{array}{ccc}\hfill \parallel x_{k+1}-x_k{\parallel }^2& =& \parallel y_{k+1}-y_k+\frac{\lambda }{\gamma }(K^{*}(I-P_Q)K{y}_{k+1}-K^{*}(I-P_Q)K{y}_k){\parallel }^2\hfill \\ & =& \parallel y_{k+1}-y_k{\parallel }^2+\frac{2\lambda }{\gamma }\langle K^{*}(I-P_Q)K{y}_{k+1}-K^{*}(I-P_Q)K{y}_k,y_{k+1}-y_k\rangle \hfill \\ & +& \frac{{\lambda }^2}{{\gamma }^2}{\parallel K^{*}(I-P_Q)K{y}_{k+1}-K^{*}(I-P_Q)K{y}_k\parallel }^2\hfill \\ & \ge & \parallel y_{k+1}-y_k{\parallel }^2+\frac{2\lambda }{\gamma }{\parallel (I-P_Q)K{y}_{k+1}-(I-P_Q)K{y}_k\parallel }^2\hfill \\ & +& \frac{{\lambda }^2}{{\gamma }^2}{\parallel (I-P_Q)K{y}_{k+1}-(I-P_Q)K{y}_k\parallel }^2.\hfill \end{array}$$

This yields

$$\parallel x_{k+1}-x_k{\parallel }^2\le \parallel y_{k+1}-y_k{\parallel }^2+\frac{\lambda }{\gamma }(2+\frac{\lambda }{\gamma }){\parallel (I-P_Q)K{y}_{k+1}-(I-P_Q)K{y}_k\parallel }^2,$$

(24)

hence

$$\parallel y_{k+1}-y_k{\parallel }^2\le {\parallel x_{k+1}-x_k\parallel }^2.$$

Taking into account the fact that $0<\frac{\lambda }{\gamma }<\frac{-1+\sqrt{5}}{2}$, we have $L=1+\frac{\lambda }{\gamma }<\frac{\gamma }{\lambda }$ which, combined with the last inequality, ensures that

$$L\parallel y_{k+1}-y_k{\parallel }^2\le \frac{\gamma }{\lambda }{\parallel x_{k+1}-x_k\parallel }^2.$$

(25)

Combining (22), (23) and (25) yields

$$F(x_{k+1},y_{k+1})\le E\left(y_k\right)+\frac{\gamma }{\gamma +\lambda }\langle \nabla E\left(y_k\right),x_{k+1}-x_k\rangle +\frac{\gamma }{2\lambda }\parallel x_{k+1}-x_k{\parallel }^2+\gamma {\parallel x_{k+1}\parallel }_0.$$

(26)

To prove the no-increasing of the sequence ${\left(F(x_k,y_k)\right)}_{k\in IN}$, we first notice that the second part of (18) can be reads as $y_k=x_k-\frac{\lambda }{\lambda +\gamma }\nabla E\left(y_k\right)$. Now, by applying definition of the proximal operator of ${\lambda \parallel ·\parallel }_0$ at $y_k$, we have

$$x_{k+1}\in arg\underset{x\in I{R}^N}{min}\left({\parallel x\parallel }_0+\frac{1}{2\lambda }{\parallel x-x_k+\frac{\lambda }{\lambda +\gamma }\nabla E\left(y_k\right)\parallel }^2\right)$$

or, equivalently,

$$x_{k+1}\in arg\underset{x\in I{R}^N}{min}\left({\parallel x\parallel }_0+\frac{1}{2\lambda }{\parallel x-x_k\parallel }^2+\frac{\lambda }{\lambda +\gamma }\langle \nabla E\left(y_k\right),x-x_k\rangle \right),$$

which ensures that

$$\parallel x_{k+1}{\parallel }_0+\frac{1}{2\lambda }\parallel x_{k+1}-x_k{\parallel }^2+\frac{1}{\lambda +\gamma }\langle \nabla E\left(y_k\right),x_{k+1}-x_k\rangle \le {\parallel x_k\parallel }_0.$$

(27)

Finally, from (26) and (28), we deduce that

$$F(x_{k+1},y_{k+1})\le E\left(y_k\right)+\gamma {\parallel x_k\parallel }_0=F(x_k,y_k).$$

It follows from (27) that

$$\gamma \parallel x_{k+1}{\parallel }_0+\frac{\gamma }{2\lambda }{\parallel x_{k+1}-x_k\parallel }^2+\frac{\gamma }{\lambda +\gamma }\langle \nabla E\left(y_k\right),x_{k+1}-x_k\rangle \le F(x_k,y_k).$$

In addition form (22), we obtain that

$$-E\left(y_k\right)-\langle \nabla E\left(y_k\right),y_{k+1}-y_k\rangle +\frac{L}{2}\parallel y_{k+1}-y_k{\parallel }^2-\gamma {\parallel x_{k+1}\parallel }_0\le -F(x_{k+1},y_{k+1}).$$

Summing the above two inequalities and using (23), we obtain

$$\frac{\gamma }{2\lambda }\parallel x_{k+1}-x_k{\parallel }^2-\frac{L}{2}{\parallel y_{k+1}-y_k\parallel }^2\le F(x_k,y_k)-F(x_{k+1},y_{k+1}).$$

This, combined with (24), yields

$$\frac{1}{2}(\frac{\gamma }{2\lambda }-L)\parallel y_{k+1}-y_k{\parallel }^2+\frac{\lambda }{\gamma }(2+\frac{\lambda }{\gamma }){\parallel (I-P_Q)y_{k+1}-(I-P_Q)y_k\parallel }^2\le F(x_k,y_k)-F(x_{k+1},y_{k+1}).$$

(28)

By summing the last inequality and by taking into account the fact the convergence of the sequence ${\left(F(x_k,x_y)\right)}_{k\in IN}$ together with the fact that $\frac{\gamma }{2\lambda }-L>0$, we deduce first that

$$\sum _{k=0}^{\infty }\parallel y_{k+1}-y_k{\parallel }^2<+\infty \mathrm{and}\sum _{k=0}^{\infty }{\parallel (I-P_Q)y_{k+1}-(I-P_Q)y_k\parallel }^2<+\infty .$$

The property ${\sum }_{k=0}^{\infty }{\parallel x_{k+1}-x_k\parallel }^2<+\infty$ follows then from relation (24). The latter properties ensures clearly the asymptotic regularity of the sequence ${(x_k,y_k)}_{k\in IN}$.   □

As in ([6] Lemma 12), in our setting, we also have that the invariant support set of the sequence defined by Algorithm (18) exists for the nonconvex problem (7). Now, let us prove, more directly than in [6], the convergence of the sequence ${(x_k,y_k)}_{k\in IN}$ generated by (18) relying on averaged operators and Krasnoselskii–Mann Theorem. Averaged mappings are convenient in studying the convergence sequences generated by iterative algorithms for fixed-point problems thanks to the following celebrate theorem, see for example [12,13].

Theorem 5

(Krasnoselskii–Mann Theorem). Let $M:I{R}^N\to I{R}^N$ be averaged and assume $FixM\ne \varnothing$. Then, for any starting point $x_0$, the sequence $\left\{M^k{x}_0\right\}$ converges weakly to a fixed-point of M.

Recall also the definitions of nonexpansive and averaged operators, which appear naturally when using iterative algorithms for solving fixed-point problems and which are commonly encountered in the literature; see, for instance, [13]. A mapping $T:I{R}^N\to I{R}^N$ is said to be nonexpansive if, for all $x,y\in I{R}^N,\parallel Tx-Ty\parallel \le \parallel x-y\parallel$, firmly nonexpansive if $2T-I$ is nonexpansive, or equivalently $\langle Tx-Ty,x-y\rangle \ge {\parallel Tx-Ty\parallel }^2$, for all $x,y\in I{R}^N$. It is well known that T is firmly nonexpansive if and only if T can be written as $T=\frac{1}{2}(I+S)$, where $S:I{R}^N\to I{R}^N$ is nonexpansive. Recall also that mapping $T:I{R}^N\to I{R}^N$ is said to be averaged if it can be expressed as $T=(1-\alpha )I+\alpha S$ with $S:I{R}^N\to I{R}^N$ a nonexpansive mapping and $\alpha \in$ [0, 1]. Thus, firmly nonexpansive mappings (e.g., projections on convex convex and nonempty subsets and resolvent of maximal monotone operators) are averaged mappings.

Mimicking the analysis in [6], $(\overline{x},\overline{y})$ is a solution of (13) if, and if $(\overline{x},\overline{y})$ satisfies (16) and thus $\overline{x}$ is verified as

$$\overline{x}=P_{B_{\mathit{N}}}\circ J_{\frac{\lambda }{\gamma }}^{K^{*}(I-P_Q)K}\left(\overline{x}\right).$$

(29)

Similarly, using the same arguments, we drive that $(x_k,y_k)$ generated by (18) leads to

$$x_{k+1}=P_{B_N}\left(y_k\right)\forall k\ge \overline{\kappa }$$

and thus for all $k\ge \overline{\kappa }$, it satisfies

$$x_{k+1}=P_{B_{\mathit{N}}}\left(y_k\right)\mathrm{and}{y}_{k+1}=x_{k+1}-\frac{\lambda }{\gamma }{K}^{*}(I-P_Q)K{y}_{k+1}.$$

(30)

Consequently,

$$\overline{x}=P_{B_{\mathit{N}}}\circ J_{\frac{\lambda }{\gamma }}^{K^{*}(I-P_Q)K}\left(\overline{x}\right),$$

(31)

and

$$x_{k+1}=P_{B_{\mathit{N}}}\circ J_{\frac{\lambda }{\gamma }}^{K^{*}(I-P_Q)K}\left(x_k\right).$$

(32)

It is well-known that firmly nonexpansive mappings (including orthogonal projections on closed convex nonempty subsets and resolvent mappings of maximal monotone operators) are averaged operators. In view of the fact that the composite of finitely many averaged mappings is averaged, see for instance [12], and by applying Krasnoselskii–Mann Theorem, we deduce the convergence of the subsequence ${\left(x_k\right)}_{k\ge \overline{\kappa }}$ to a solution $\overline{x}$ of (29). $\overline{x}\in {\mathit{B}}_{\mathit{N}}$, since ${\left(x_k\right)}_{k\ge \overline{\kappa }}\subset {\mathit{B}}_{\mathit{N}}$, which is closed, and we also have $\mathit{N}=N\left(x_{\overline{\kappa }}\right)=N\left(\overline{x}\right)$ in view of ([6] Lemma 7). In addition, since the resolvent of a maximal monotone operator is nonexpansive, for all $m>n>\overline{\kappa }$, we can write that

$$\parallel y_m-y_n\parallel =\parallel J_{\frac{\lambda }{\gamma }}^{K^{*}(I-P_Q)K}\left(x_m\right)-J_{\frac{\lambda }{\gamma }}^{K^{*}(I-P_Q)K}\left(x_n\right)\parallel \le \parallel x_m-x_n\parallel .$$

${\left(x_k\right)}_{k\ge \overline{\kappa }}$ being convergent, it is a Cauchy sequence and thus it is also the case of the subsequence ${\left(y_k\right)}_{k\ge \overline{\kappa }}$ which, in turn, converges to some limit $\overline{y}$. Now, by passing to the limit in $y_{k+1}=J_{\frac{\lambda }{\gamma }}^{K^{*}(I-P_Q)K}\left(x_k\right)$ and taking into account the continuity of the resolvent, we obtain $\overline{y}=J_{\frac{\lambda }{\gamma }}^{K^{*}(I-P_Q)K}\left(\overline{x}\right)$, where $\overline{x}$ is a solution of (16) which ensures that $(\overline{x},\overline{y})$ is a solution of (13) with $\mathit{N}=N\left(\overline{x}\right)$.

Based on the above, these lead to the following Theorem.

Theorem 6.

Let $(x_k,y_k)\in I{R}^N\times I{R}^N$ a sequence generated by (18) form an initial point $(x_0,y_0)$. If $\lambda ,\gamma >0$ are chosen such that $0<\frac{\lambda }{\gamma }<\frac{1+\sqrt{5}}{2}$, then $(x_k,y_k)$ converges to a local minimizer $(\overline{x},\overline{y})$ of (7). Moreover, ${\left(F(x_k,y_k)\right)}_{k\in IN}$ is a convergent sequence and if, in addition $|{\overline{y}}_j|\ne \sqrt{2\beta }$ for all $j\in N\left(\overline{y}\right)$, then $\overline{y}$ is a local minimizer of (5).

Finally, let us point out that, since $K{K}^{*}=I$, the fixed point iteration in (18) turns into

$$y_{k+1}=x_{k+1}-\frac{\lambda }{2\gamma }{K}^{*}(I-P_Q)\left(K{x}_{k+1}\right).$$

(33)

In particular, when $Q=\left\{r\right\}$, this reduces to

$$y_{k+1}=x_{k+1}-\frac{\lambda }{2\gamma }{K}^{*}(K{x}_{k+1}-r).$$

Indeed, as $K{K}^{*}=I$,

$$\begin{array}{ccc}\hfill J_{\frac{\lambda }{\gamma }}^{K^{*}(I-P_Q)K}\left(x_{k+1}\right)& =& x_{k+1}-\frac{\lambda }{\gamma }{K}^{*}{(I-P_Q)}_1\left(K{x}_{k+1}\right)=x_{k+1}-\frac{\lambda }{\gamma }{K}^{*}{\left({(\partial i_Q)}_1\right)}_1\left(K{x}_{k+1}\right)\hfill \\ & =& x_{k+1}-\frac{\lambda }{\gamma }{K}^{*}{(\partial i_Q)}_2\left(K{x}_{k+1}\right)=x_{k+1}-\frac{\lambda }{2\gamma }{K}^{*}(I-P_Q)\left(K{x}_{k+1}\right).\hfill \end{array}$$

We used both the fact that for any maximal monotone operator A and $\nu >0$, we have

$$J_{\nu }^{K^{*}AK}\left(x\right)=x-\nu K^{*}{A}_1\left(Kx\right),$$

$A_1=I-J_1^A$ being the Yosida operator of A with parameter 1, and that for all $\nu ,\mu >0$, we have

$${\left(A_{\nu }\right)}_{\mu }=A_{\nu +\mu }.$$

We used also the fact that the Yosida operator of the subdifferential of the indicator function of Q with parameter 2 (the latter is nothing but the the normal cone to Q) is exactly $\frac{I-P_Q}{2}$.

Therefore, the proposed algorithms are nothing else than a Proximal Gradient and a Projection Gradient Algorithms. Nevertheless, the crucial idea (namely, the support of the sparse main variable generated by the Algorithm remains unchanged after a finite number of iterations) that permits to locate a local minimizer of the nonconvex optimization problem with a global minimizer of a convex optimization one deserves a great interest.

Clearly, the analysis developed here can be extended to split feasibility problems, namely

$$\mathrm{Find}y\in C\mathrm{such} \mathrm{that}Ky\in Q,$$

(34)

with $C\subset I{R}^N$, $Q\subset I{R}^d$ being two closed, convex subsets and $K:I{R}^N\to I{R}^d$ a given matrix. Since the sum of a maximal monotone operator (the normal cone to C) and the monotone Lipschitz one (the operator $\frac{\lambda }{2\gamma }{K}^{*}(I-P_Q)K$) is still maximal monotone [14], this can be naturally extended to the following general minimization problem by means of its regularized version, i.e.,

$$\underset{y\in I{R}^N}{min}\left(f\left(y\right)+g\left(Ky\right)+{\gamma \parallel y\parallel }_0\right)through\underset{y\in I{R}^N}{min}\left(f\left(y\right)+g_{\nu }\left(Ky\right)+{\gamma (\parallel y\parallel }_0{)}_{\lambda }\right),$$

with $f,g$ being two proper, convex, lower semicontinuous functions defined on $I{R}^N,I{R}^d$, respectively, and $\nu ,\lambda >0$. The proximity map of g and the subdifferential of f will act as the projection on the set Q and the normal cone to C, respectively, since they share both the same properties.

5. Conclusions

Based on an interesting idea developed in [6], which leads to identifying a local minimizer of a nonconvex minimization problem with a global optimizer of a convex optimization one, we provide an extension of the sparse regularization model for inverting incomplete Fourier transforms. Next, we propose an efficient convergence guaranteed iteration algorithm for solving the resulting non-convex and non-smooth optimization problem. The fixed-point approach is preferred, as it enables us to develop efficient algorithms with guaranteed convergence. Combined with applied nonlinear analysis tools, this leads both to a simplification of the proofs and to make a connection with classical works as split convex feasibility problems. With this generalization, the proposed approach may be applicable to other real world applications such as inverse problem of intensity-modulated radiation therapy (IMRT) treatment planning [18]. It can be applied equally to cooperative wireless sensor network positioning [19] or adaptive image denoising [20]. Thus, the proposed method is expected to work efficiently for problems that can be reformulated as sparse optimization and convex feasibility problems. We will consider this as a future project for numerical applications as well as other potential extensions, for example, in a non-convex framework. These in turn will pave the way for other applications in the real world, which is the case, for example, in [21].