The Capped Separable Difference of Two Norms for Signal Recovery

Abstract

This paper introduces a novel capped separable difference of two norms (CSDTN) method for sparse signal recovery, which generalizes the well-known minimax concave penalty (MCP) method. The CSDTN method incorporates two shape parameters and one scale parameter, with their appropriate selection being crucial for ensuring robustness and achieving superior reconstruction performance. We provide a detailed theoretical analysis of the method and propose an efficient iteratively reweighted ${\ell }_1$ (IRL1)-based algorithm for solving the corresponding optimization problem. Extensive numerical experiments, including electrocardiogram (ECG) and synthetic signal recovery tasks, demonstrate the effectiveness of the proposed CSDTN method. Our method outperforms existing methods in terms of recovery accuracy and robustness. These results highlight the potential of CSDTN in various signal-processing applications.

1. Introduction

Regularization is a powerful technique in machine learning that helps to improve model generalization by introducing constraints on the model’s complexity. Convex regularization such as ${\ell }_1$ and ${\ell }_2$ has been widely used due to its simplicity and desirable properties [1]. In particular, the ${\ell }_1$ minimization technique has been remarkably effective for sparse signal recovery and has spurred extensive research in the field of compressed sensing (CS); see the monograph [2] for a comprehensive view. CS leverages the sparsity of signals to recover them from far fewer measurements. However, convex regularization may not always be suitable for handling complex and high-dimensional data, where nonconvex regularization can provide better solutions. Nonconvex regularization methods have gained increasing attention in recent years due to their ability to capture complex structures in the data and improve prediction accuracy; see the survey [3] and the references therein. These methods impose nonconvex penalties on the objective function of the model optimization problem, taking various forms such as ${\ell }_p(0<p<1)$ [4], smoothly clipped absolute deviation (SCAD) [5], minimax concave penalty (MCP) [6], ${\ell }_1-{\ell }_2$ [7], and weighted ${\ell }_r-{\ell }_1$ minimization [8,9], among others. The resulting optimization problems are often challenging to solve due to the nonconvexity of the penalty functions, making it difficult to find the global minimum of the objective function. Despite these challenges, nonconvex regularization has shown great potential for improving the performance of machine learning models in different fields, including computer vision, natural language processing, signal processing, and many more.

Sparse signal recovery serves as a cornerstone of modern signal processing, machine learning, and statistics (refer to the matrix factorization techniques reviewed in [10] and the references cited therein). It aims to retrieve an unknown sparse signal $\mathbf{x}\in {\mathbb{R}}^N$ from few noisy linear measurements $\mathbf{y}=A\mathbf{x}+\mathit{\epsilon }\in {\mathbb{R}}^m$, where $A\in {\mathbb{R}}^{m\times N}$ is the measurement matrix with $m\ll N$, and $\mathit{\epsilon }$ is the noise vector. If the signal $\mathbf{x}$ is sparse and the measurement matrix A is suitably designed, $\mathbf{x}$ can be effectively reconstructed from $\mathbf{y}$ using algorithms such as basis pursuit (BP) [11], least absolute shrinkage and selection operator (Lasso) [12], orthogonal matching pursuit (OMP) [13], compressive sampling matching pursuit (CoSaMP) [14], or iterative hard thresholding (IHT) [15], among others. Furthermore, nonconvex regularization techniques, including ${\ell }_p(0<p<1)$, SCAD, and MCP, also play a significant role in sparse signal recovery. Generally, a nonconvex regularization method for sparse signal recovery has a form of

$$\begin{array}{c}\hfill \underset{\mathbf{x}\in {\mathbb{R}}^N}{min}{\displaystyle \frac{1}{2}}{\parallel \mathbf{y}-A\mathbf{x}\parallel }_2^2+\lambda R_{\theta }\left(\mathbf{x}\right),\end{array}$$

(1)

where $R_{\theta }(·):{\mathbb{R}}^N\to {\mathbb{R}}_{+}$ represents a nonconvex regularization function that relies on a parameter or vector of parameters $\theta$ and $\lambda >0$ is a tuning parameter. In particular, a separable regularization function can be expressed as ${\sum }_{j=1}^N{F}_{\theta }\left(\right|x_j\left|\right)$ with $F_{\theta }(·):{\mathbb{R}}_{+}\to {\mathbb{R}}_{+}$ being an univariate function.

In this paper, we propose to use the capped separable difference of two norms (CSDTN) for sparse signal recovery, which takes the form of

$$R\left(\mathbf{x}\right)=\sum _{j=1}^{N}F\left(\right|x_j\left|\right),$$

where

$$\begin{array}{c}\hfill F\left(x\right)=\left\{\begin{array}{cc}{\displaystyle \frac{2}{2-p}}{(x/t)}^p-{\displaystyle \frac{p}{2-p}}{(x/t)}^q\hfill & x\in [0,{\left({\displaystyle \frac{2}{q}}\right)}^{{\displaystyle \frac{1}{q-p}}}t],\hfill \\ {\displaystyle \frac{2}{2-p}}{\left({\displaystyle \frac{2}{q}}\right)}^{{\displaystyle \frac{p}{q-p}}}(1-{\displaystyle \frac{p}{q}})\hfill & x>{\left({\displaystyle \frac{2}{q}}\right)}^{{\displaystyle \frac{1}{q-p}}}t.\hfill \end{array}\right.\end{array}$$

(2)

Here, $0<p\le 1$, $1\le q<+\infty$ and $t>0$. Throughout the paper, we assume that $q\ne 1$ when $p=1$. It is worth noting that $t>0$ is a scale parameter, while both $0<p\le 1$ and $1\le q<+\infty$ are shape parameters. To simplify the notation, we hide the dependence on these parameters in $R(·)$ and $F(·)$.

1.1. Related Work

• The minimax concave penalty (MCP) method was proposed in [6]. For any $\gamma >0$ and $\lambda >0$, its regularization term $R\left(\mathbf{x}\right)={\sum }_{j=1}^{N}F\left(\right|x_j\left|\right)$ with

  $$F\left(\right|x_j\left|\right)={\int }_0^{|x_j|}\lambda {\left(1-{\displaystyle \frac{t}{\gamma \lambda }}\right)}_{+}dt=\left\{\begin{array}{c}\lambda |x_j|-{\displaystyle \frac{x_j^2}{2\gamma }},\left|x_j\right|\le \gamma \lambda \hfill \\ {\displaystyle \frac{\gamma }{2}}{\lambda }^2,\left|x_j\right|>\gamma \lambda \hfill \end{array}\right..$$
  Therefore, if the effect of constants is excluded, its equivalent regularization term goes to

  $$F\left(\right|x_j\left|\right)=\left\{\begin{array}{c}{\displaystyle \frac{2|x_j|}{\gamma \lambda }}-{\displaystyle \frac{x_j^2}{{\gamma }^2{\lambda }^2}},\left|x_j\right|\le \gamma \lambda \hfill \\ 1,|x_j|>\gamma \lambda \hfill \end{array}\right.,$$

  so that the MCP method corresponds to the special case of the CSDTN method with $p=1$, $q=2$, and $t=\gamma \lambda$. Additionally, according to [16], this CSDTN regularization method corresponds to a continuous probability distribution function $H(·)$ on ${\mathbb{R}}_{+}$; that is,

  $$\begin{array}{c}\hfill H\left(x\right)=\left\{\begin{array}{cc}{\left({\displaystyle \frac{q}{2}}\right)}^{{\displaystyle \frac{p}{q-p}}}{\displaystyle \frac{q}{q-p}}[{\left({\displaystyle \frac{x}{t}}\right)}^p-{\displaystyle \frac{p}{2}}{\left({\displaystyle \frac{x}{t}}\right)}^q]\hfill & x\in [0,{\left({\displaystyle \frac{2}{q}}\right)}^{{\displaystyle \frac{1}{q-p}}}t],\hfill \\ 1\hfill & x>{\left({\displaystyle \frac{2}{q}}\right)}^{{\displaystyle \frac{1}{q-p}}}t.\hfill \end{array}\right.\end{array}$$
  As a result, many theoretical results and algorithms from the existing literature, such as [16,17,18], can be applied to it.
• The proposed regularization function $R\left(\mathbf{x}\right)$ exhibits a form of the separable difference of two norms ${\displaystyle \frac{2}{2-p}}{\parallel {\displaystyle \frac{\mathbf{x}}{t}}\parallel }_p^p-{\displaystyle \frac{p}{2-p}}{\parallel {\displaystyle \frac{\mathbf{x}}{t}}\parallel }_q^q$ when ${\parallel \mathbf{x}\parallel }_{\infty }\le {\left({\displaystyle \frac{2}{q}}\right)}^{{\displaystyle \frac{1}{q-p}}}t$. Therefore, if the scale parameter $t>0$ is appropriately chosen to satisfy the constraint condition ${\parallel \mathbf{x}\parallel }_{\infty }\le {\left({\displaystyle \frac{2}{q}}\right)}^{{\displaystyle \frac{1}{q-p}}}t$, it becomes the hybrid $L_2$-$L_p$ regularization method proposed in [19] when $q=2$. In the special case of $p=1$ and $q=2$, it reduces to the piecewise quadratic approximation (PQA) approach studied in [20,21]. The springback model in [22] uses ${\parallel \mathbf{x}\parallel }_1-\alpha {\parallel \mathbf{x}\parallel }_2^2$ as the regularization function, which performs similarly to the PQA when the weight $\alpha$ is well-selected.

1.2. Contribution

This paper presents a novel approach for sparse signal recovery by introducing the capped separable difference of two norms (CSDTN) as a regularization technique. The main contributions of this work can be summarized as follows:

(i)

We propose the CSDTN model, which integrates the difference of two norms framework with a capped function to enhance sparse signal recovery. This approach effectively mitigates the bias commonly introduced by convex regularizers, such as the ${\ell }_1$ minimization, thereby improving the accuracy of signal reconstruction.

(ii)

We provide a detailed theoretical analysis of the CSDTN, establishing the condition for exact recovery under this model in terms of a generalized null space property.

(iii)

To solve the CSDTN-regularized problem efficiently, we develop an algorithm based on the iteratively reweighted ${\ell }_1$ (IRL1).

(iv)

We conduct comprehensive experiments on electrocardiogram (ECG) and synthetic data, illustrating the advantages of the CSDTN model in sparse recovery scenarios. Our method outperforms traditional ${\ell }_1$-based approaches and other nonconvex regularizers in terms of accuracy and robustness.

1.3. Organization

The present paper is organized as follows. In Section 2, we study generic properties of the proposed CSDTN regularization function. In Section 3, we present the theoretical analysis result based on the null space property. In Section 4, we provide its solving algorithm. Numerical experiments are carried out in Section 5. Conclusions are included in Section 6.

2. The CSDTN Regularization Function

To begin with, we consider the sub-additive property for the proposed CSDTN regularization function $R(·)$.

Proposition 1. 

The function $R(·)$ is sub-additive, i.e., $R(\mathbf{x}+\mathbf{y})\le R\left(\mathbf{x}\right)+R\left(\mathbf{y}\right)$ holds for any $\mathbf{x},\mathbf{y}\in {\mathbb{R}}^N$.

Proof. 

Since $R\left(\mathbf{x}\right)={\sum }_{j=1}^{N}F\left(\right|x_j\left|\right)$, it suffices to show that for any $j\in \left[N\right]:=\{1,2,\cdots ,N\}$,

$$\begin{array}{c}\hfill F\left(\right|x_j+y_j\left|\right)\le F\left(\right|x_j\left|\right)+F\left(\right|y_j\left|\right).\end{array}$$

(3)

This holds trivially when $|x_j|=0$ or $|y_j|=0$, since $F\left(0\right)=0$. For the case of $|x_j|>0$ and $|y_j|>0$, the inequality (3) follows from the fact that

$$\begin{array}{cc}\hfill F\left(\right|x_j+y_j\left|\right)& \le F\left(\right|x_j|+|y_j\left|\right)\hfill \\ \hfill & (\mathrm{Non}\text{-}\mathrm{increasing}\mathrm{property}\mathrm{of}F\left(x\right)\mathrm{for}x>0)\hfill \\ \hfill & =|x_j|{\displaystyle \frac{F\left(\right|x_j|+|y_j\left|\right)}{|x_j|+|y_j|}}+\left|y_j\right|{\displaystyle \frac{F\left(\right|x_j|+|y_j\left|\right)}{|x_j|+|y_j|}}\hfill \\ \hfill & \le |x_j|{\displaystyle \frac{F\left(\right|x_j\left|\right)}{|x_j|}}+\left|y_j\right|{\displaystyle \frac{F\left(\right|y_j\left|\right)}{|y_j|}}\hfill \\ \hfill & (\mathrm{Non}\text{-}\mathrm{increasing}\mathrm{property}\mathrm{of}F\left(x\right)/x\mathrm{for}x>0)\hfill \\ \hfill & =F\left(\right|x_j\left|\right)+F\left(\right|y_j\left|\right).\hfill \end{array}$$

The proof is completed.    □

The additive property follows trivially from its separability when the items in the summation have disjoint supports.

Proposition 2. 

Let $S=S_1\cup S_2$ with $S_1\cap S_2=\varnothing$; then, for any $\mathbf{x}\in {\mathbb{R}}^N$, it holds that

$$\begin{array}{c}\hfill R\left({\mathbf{x}}_S\right)=R\left({\mathbf{x}}_{S_1}\right)+R\left({\mathbf{x}}_{S_2}\right).\end{array}$$

(4)

In what follows, the asymptotic behaviors of the regularization function $R(·)$ is presented.

Proposition 3. 

For any nonzero $\mathbf{x}\in {\mathbb{R}}^N$, $t>0$, $0<p\le 1$, and $1\le q<\infty$, we have

(i) 

$R\left(\mathbf{x}\right)\to C_{p,q}{\parallel \mathbf{x}\parallel }_0$ with $C_{p,q}={\displaystyle \frac{2}{2-p}}{\left({\displaystyle \frac{2}{q}}\right)}^{{\displaystyle \frac{p}{q-p}}}(1-{\displaystyle \frac{p}{q}})$, as $t\to 0^{+}$.

(ii) 

$t^{p}R\left(\mathbf{x}\right)\to {\displaystyle \frac{2}{2-p}}{\parallel \mathbf{x}\parallel }_p^p$, as $t\to +\infty$.

(iii) 

$R\left(\mathbf{x}\right)\to {\displaystyle \frac{1}{t}}{\parallel \mathbf{x}\parallel }_1$ when $p=1$ and $q\to 1^{+}$ (or when $q=1$ and $p\to 1^{-}$).

Proof. 

If we let the standard function $G\left(x\right)=\left\{\begin{array}{cc}{\displaystyle \frac{2}{2-p}}{x}^p-{\displaystyle \frac{p}{2-p}}{x}^q\hfill & x\in [0,{\left({\displaystyle \frac{2}{q}}\right)}^{{\displaystyle \frac{1}{q-p}}}]\hfill \\ {\displaystyle \frac{2}{2-p}}{\left({\displaystyle \frac{2}{q}}\right)}^{{\displaystyle \frac{q}{q-p}}}(1-{\displaystyle \frac{p}{q}})\hfill & x>{\left({\displaystyle \frac{2}{q}}\right)}^{{\displaystyle \frac{1}{q-p}}}\hfill \end{array}\right.$, then $F\left(x\right)=G(x/t)$.

As for (i), when $x>0$, it holds that

$$\begin{array}{cc}\hfill \underset{t\to 0^{+}}{lim}F\left(x\right)& =\underset{t\to 0^{+}}{lim}G(x/t)\hfill \\ \hfill & \stackrel{\sigma =x/t}{=}\underset{\sigma \to +\infty }{lim}G\left(\sigma \right)={\displaystyle \frac{2}{2-p}}{\left({\displaystyle \frac{2}{q}}\right)}^{{\displaystyle \frac{p}{q-p}}}(1-{\displaystyle \frac{p}{q}})=C_{p,q}.\hfill \end{array}$$

When $x=0$, it is obvious that $F\left(x\right)=F\left(0\right)=0=x$.

To prove (ii), we have ${lim}_{t\to +\infty }{t}^{p}F\left(x\right)={lim}_{t\to +\infty }{t}^{p}F\left(0\right)=0$ when $x=0$ and for any $x>0$,

$$\begin{array}{cc}\hfill \underset{t\to +\infty }{lim}{\displaystyle \frac{t^{p}F\left(x\right)}{x^p}}& =\underset{t\to +\infty }{lim}{\displaystyle \frac{G(x/t)}{{(x/t)}^p}}\stackrel{\sigma =x/t}{=}\underset{\sigma \to 0^{+}}{lim}G\left(\sigma \right)/{\sigma }^p\hfill \\ \hfill & =\underset{\sigma \to 0^{+}}{lim}\left({\displaystyle \frac{2}{2-p}}-{\displaystyle \frac{p}{2-p}}{\sigma }^{q-p}\right)={\displaystyle \frac{2}{2-p}}.\hfill \end{array}$$

With regard to (iii), it holds trivially, by noticing the facts that ${\displaystyle \frac{2}{2-p}}{(x/t)}^p-{\displaystyle \frac{p}{2-p}}{(x/t)}^q\to {\displaystyle \frac{x}{t}}$ and ${\left({\displaystyle \frac{2}{q}}\right)}^{{\displaystyle \frac{1}{q-p}}}t\to +\infty$ when $p=1$ and $q\to 1^{+}$, or when $q=1$ and $p\to 1^{-}$.    □

To illustrate the proposition, we present in Figure 1 the plots of the regularization function $R\left(\mathbf{x}\right)$ when $\mathbf{x}\in \mathbb{R}$. The functions are all concave on ${\mathbb{R}}_{+}$ and provide good approximations to the ${\ell }_0$ function for a small value of t, while approaching the ${\ell }_p$ function as t becomes larger. In addition, for any fixed t, all the regularization functions approach the ${\ell }_1$ function when either $p=1$ and $q=1.1$, or $p=0.9$ and $q=1$. These findings are largely consistent with the statements made in Proposition 3. By selecting an appropriate parameter set $(p,q,t)$, we can effectively control the smoothness and concavity of the regularization function to achieve the best recovery performance.

3. Theoretical Result

In this section, we present the recovery analysis results for the following constrained noiseless CSDTN problem:

$$\begin{array}{c}\hfill \underset{\mathbf{z}\in {\mathbb{R}}^N}{min}R\left(\mathbf{z}\right)\mathrm{subject}\mathrm{to}A\mathbf{z}=A\mathbf{x}.\end{array}$$

(5)

We start with the definition of a generalized version of the null space property (NSP) [23].

Definition 1. 

We say a matrix $A\in {\mathbb{R}}^{m\times N}$ satisfies a generalized null space property (gNSP) relative to $R(·)$ and $S\subset \left[N\right]$ if

$$\begin{array}{c}\hfill R\left({\mathbf{v}}_S\right)<R\left({\mathbf{v}}_{S^c}\right)\mathit{for}\mathit{all}\mathbf{v}\in \mathrm{Ker}\left(A\right)\setminus \left\{\mathbf{0}\right\}.\end{array}$$

(6)

Furthermore, we say it satisfies the gNSP of order s relative to $R(·)$ if it satisfies the gNSP relative to $R(·)$ and any $S\subset \left[N\right]$ with $\left|S\right|\le s$.

By applying the sub-additivity of $R(·)$, we can easily derive the following sufficient and necessary condition for an exact uniform sparse recovery via the minimization problem (5). This result is also presented in Lemma 1 of [18] or Theorem 1 of [16]. However, for completeness, we provide a detailed proof of this result in this paper.

Theorem 1. 

For any given measurement matrix $A\in {\mathbb{R}}^{m\times N}$, every s-sparse vector $\mathbf{x}\in {\mathbb{R}}^N$ is the unique solution of the problem (5) if and only if A satisfies the gNSP of order s relative to $R(·)$.

Proof. 

It suffices to verify that every vector $\mathbf{x}\in {\mathbb{R}}^N$ supported on a set $S\subset \left[N\right]$ is the unique solution of the problem (5) if and only if A satisfies the gNSP relative to $R(·)$ and S. Then, this theorem holds by letting the set S vary.

Given any fixed index set S, let us first assume that every vector $\mathbf{x}\in {\mathbb{R}}^N$ supported on S is the unique minimizer of the problem (5). Thus, for any $\mathbf{v}\in \mathrm{Ker}\left(A\right)\setminus \left\{\mathbf{0}\right\}$, the vector ${\mathbf{v}}_{\mathbf{S}}$ is the unique minimizer of $R\left(\mathbf{z}\right)$ subject to $A\mathbf{z}=A{\mathbf{v}}_{\mathbf{S}}$. However, we have $A(-{\mathbf{v}}_{S^c})=A{\mathbf{v}}_S$ and $-{\mathbf{v}}_{S^c}\ne {\mathbf{v}}_S$ because $A({\mathbf{v}}_S+{\mathbf{v}}_{S^c})=A\mathbf{v}=\mathbf{0}$ and $\mathbf{v}\ne \mathbf{0}$. We conclude that $R\left({\mathbf{v}}_S\right)<R(-{\mathbf{v}}_{S^c})=R\left({\mathbf{v}}_{S^c}\right)$. This establishes the generalized null space property relative to $R(·)$ and S.

Conversely, let us assume that the generalized null space property relative to $R(·)$ and S holds. Then, given a vector $\mathbf{x}\in {\mathbb{R}}^N$ supported on S and a vector $\mathbf{z}\in {\mathbb{R}}^N$, $\mathbf{z}\ne \mathbf{x}$, satisfying $A\mathbf{z}=A\mathbf{x}$, we consider the vector $\mathbf{v}:=\mathbf{x}-\mathbf{z}\in \mathrm{Ker}\left(A\right)\setminus \left\{\mathbf{0}\right\}$. According to the generalized null space property, we obtain

$$\begin{array}{cc}\hfill R\left(\mathbf{x}\right)& \le R(\mathbf{x}-{\mathbf{z}}_S)+R\left({\mathbf{z}}_S\right)(\mathrm{By}\mathrm{using}\mathrm{Proposition}1)\hfill \\ \hfill & =R\left({\mathbf{v}}_S\right)+R\left({\mathbf{z}}_S\right)\hfill \\ \hfill & <R\left({\mathbf{v}}_{S^c}\right)+R\left({\mathbf{z}}_S\right)\hfill \\ \hfill & =R(-{\mathbf{z}}_{S^c})+R\left({\mathbf{z}}_S\right)\hfill \\ \hfill & =R\left({\mathbf{z}}_{S^c}\right)+R\left({\mathbf{z}}_S\right)\hfill \\ \hfill & =R\left(\mathbf{z}\right),(\mathrm{By}\mathrm{adopting}\mathrm{Proposition}2)\hfill \end{array}$$

which establishes the required minimality of $R\left(\mathbf{x}\right)$ and completes the proof.    □

• Remark: Since $R(·)$ is both separable and symmetric on ${\mathbb{R}}^N$, as well as concave on ${\mathbb{R}}_{+}^N$, Proposition 4.6 of [24] readily establishes that the gNSP relative to $R(·)$ is less stringent than the NSP for ${\ell }_1$-minimization. Moreover, the extension of the gNSP to its stable and robust versions to address the problems of sparsity defect and measurement error could be carried out as in Chapter 4 of [2].

4. Algorithm

In this section, we provide the solving algorithm for the proposed CSDTN method via the iteratively reweighted ${\ell }_1$ (IRL1) algorithm [25,26,27]. We focus on the following unconstrained problem:

$$\begin{array}{c}\hfill \underset{\mathbf{x}\in {\mathbb{R}}^N}{min}{\displaystyle \frac{1}{2}}{\parallel \mathbf{y}-A\mathbf{x}\parallel }_2^2+\lambda R\left(\mathbf{x}\right),\end{array}$$

(7)

where $R\left(\mathbf{x}\right)={\sum }_{j=1}^{N}F\left(\right|x_j\left|\right)$ with $F(·)$ defined in (2) and $\lambda >0$ is the tuning parameter. By a simple calculation, we can obtain the following derivative function of $F(·)$, i.e.,

$$\begin{array}{cc}\hfill f\left(x\right)& =\left\{\begin{array}{cc}{\displaystyle \frac{2p}{(2-p)t^p}}{x}^{p-1}-{\displaystyle \frac{pq}{(2-p)t^q}}{x}^{q-1}\hfill & x\in (0,{\left({\displaystyle \frac{2}{q}}\right)}^{{\displaystyle \frac{1}{q-p}}}t],\hfill \\ 0\hfill & x>{\left({\displaystyle \frac{2}{q}}\right)}^{{\displaystyle \frac{1}{q-p}}}t.\hfill \end{array}\right.\hfill \\ \hfill & =max\left({\displaystyle \frac{2p}{(2-p)t^p}}{x}^{p-1}-{\displaystyle \frac{pq}{(2-p)t^q}}{x}^{q-1},0\right),x>0.\hfill \end{array}$$

(8)

Then, the procedure of IRL1 for solving the problem (7) is summarized in Algorithm 1.

It is worth noting that in practice, we use the weights

$${\lambda }_j^{\left(k\right)}=\lambda max\left({\displaystyle \frac{2p}{(2-p)t^p}}\left(\right|x_j^{\left(k\right)}|+c^{\left(k\right)}{)}^{p-1}-{\displaystyle \frac{pq}{(2-p)t^q}}{\left|x_j^{\left(k\right)}\right|}^{q-1},0\right),$$

where $c^{\left(k\right)}$ is a sequence of positive real numbers that approach zero to avoid the problem of division by zeros when $p\in (0,1)$. In our numerical experiments, we set the initial value as $c^{\left(0\right)}=1$ and update it in each iteration as $c^{(k+1)}=c^{\left(k\right)}/10$. Also, we adopt an alternating direction method of multipliers (ADMM) algorithm [28] to solve the weighted Lasso subproblem given by (9).

Algorithm 1 IRL1 for CSDTN

1: Initialization: Set ${\lambda }_j^{\left(0\right)}=\lambda$ for $j\in \left[N\right]$, and $k=0$. 2: Iteration: Repeat until the stopping rule is met, $$\begin{array}{c}\hfill {\mathbf{x}}^{(k+1)}=\underset{\mathbf{x}\in {\mathbb{R}}^N}{\mathrm{argmin}}{\displaystyle \frac{1}{2}}{\parallel \mathbf{y}-A\mathbf{x}\parallel }_2^2+\sum _{j=1}^N{\lambda }_j^{\left(k\right)}\left|x_j\right|,\end{array}$$ (9) where ${\lambda }_j^{\left(k\right)}=\lambda f\left(\right|x_j^{\left(k\right)}\left|\right)$ for $j\in \left[N\right]$. 3: Update iteration: $k=k+1$.

5. Numerical Experiments

In this section, we conduct various numerical experiments to demonstrate the superior performance of the proposed CSDTN method in sparse signal recovery.

5.1. ECG Signal Recovery

In this subsection, we apply the proposed CSDTN method to reconstruct electrocardiogram (ECG) signals, following the discussion in [9]. The ECG signal, as shown in Figure 4 of [9], has a length of 512. While not inherently sparse, it becomes approximately sparse when represented in the discrete cosine transform domain. Using 200 noiseless Gaussian random measurements, we aim to accurately recover the original ECG signal. We compare the reconstruction performance of the ECG signal reconstruction via the CSDTN method by varying the parameter set $(p,q,t)$. The signal-to-noise ratio (SNR) is used to assess the reconstruction performance, which is defined as $\mathrm{SNR}=20{log}_{10}\left({\displaystyle \frac{{\parallel \mathbf{x}\parallel }_2}{{\parallel \mathbf{x}-\widehat{\mathbf{x}}\parallel }_2}}\right)$.

For the case where $q=2$ and $t=1$, we consider $p\in \{0.05,0.1,\cdots ,1\}$. Similarly, when $p=1$ and $t=1$, we set $q\in \{1.1,1.2,\cdots ,3\}$, while we set $t\in \{0.1,0.2,\cdots ,2\}$ for $p=1$ and $q=2$. From the results shown in Figure 2, we can observe that $p=0.8$ yields the best reconstruction performance when $q=2$ and $t=1$, while $q=1.7$ is the optimal value when $p=1$ and $t=1$. Moreover, $t=0.5$ is the best when $p=1$ and $q=2$.

We illustrate the influence of parameters $(p,q,t)$ on sparse signal recovery through this simple experiment. In practice, however, selecting the optimal parameters remains challenging. Choosing tuning parameters in nonconvex regularization methods requires a careful balance between enforcing sparsity and achieving accurate model estimation. These parameters control the regularization strength, impacting the trade-off between data fidelity and sparsity promotion. Common techniques for parameter selection include cross-validation and information criteria, such as AIC or BIC, which can help identify suitable values based on model performance.

5.2. Synthetic Signal Recovery

In the synthetic signal reconstruction experiment, we compare the proposed CSDTN method with other sparse algorithms, including the weighted ${\ell }_r-{\ell }_1$ minimization ($r=0.5$ and $\alpha =0.1$) based on DCA [9], the ${\mathrm{IRLS}}_{{\ell }_{0.5}}$ [4], the ADMM-Lasso [28], and the ${\ell }_1-{\ell }_2$ minimization [7]. We use the CSDTN method under three settings: $p=0.5$, $q=2$, and $t=1$, denoted as $\mathrm{CSDTN}(0.5,2,1)$; $p=1$, $q=2$, and $t=1$, denoted as $\mathrm{CSDTN}(1,2,1)$; and $p=0.5$, $q=1$, and $t=1$, denoted as $\mathrm{CSDTN}(0.5,1,1)$.

We start by considering the noiseless case with $\lambda ={10}^{-5}$. The tuning parameters used for other tested algorithms are the same as those in [8,9]. The original signal $\mathbf{x}$ has a length of $N=256$ and is set to be 5-sparse, with its nonzero entries generated from the standard normal distribution. In this set of experiments, we vary the number of measurements m from 20 to 80 with increments of 5. Additionally, we consider two scenarios for measurement matrices. The first scenario is the Gaussian random matrix with its entries drawn from i.i.d. standard normal distribution. For the second scenario, we use oversampled discrete cosine transform (DCT) matrices with $F\in \{2,5,10\}$, i.e., $A=[{\mathbf{a}}_1,{\mathbf{a}}_2,\cdots ,{\mathbf{a}}_{256}]\in {\mathbb{R}}^{m\times 256}$ with ${\mathbf{a}}_j={\displaystyle \frac{1}{\sqrt{m}}}cos\left({\displaystyle \frac{2\pi \mathbf{w}(j-1)}{F}}\right)$, for $j=1,2,\cdots ,256$, where $\mathbf{w}$ is a random vector uniformly distributed in ${[0,1]}^m$. It is important to note that increasing the value of F results in a more coherent oversampled DCT matrix. All the experiments are replicated 100 times, and we consider it a successful recovery if the relative reconstruction error ${\parallel \widehat{\mathbf{x}}-\mathbf{x}\parallel }_2/{\parallel \mathbf{x}\parallel }_2$ is less than ${10}^{-3}$.

Figure 3 shows the success rates of various reconstruction algorithms with both Gaussian and oversampled DCT measurements while varying the number of measurements m. The results indicate that the CSDTN methods and the ${\mathrm{IRLS}}_{{\ell }_{0.5}}$ are among the top-performing methods when the measurement matrix is Gaussian or oversampled DCT with $F=2$. However, the ${\mathrm{IRLS}}_{{\ell }_{0.5}}$ exhibits poor performance when the measurement matrix is oversampled DCT with $F=5$ and $F=10$, whereas the proposed CSDTN methods still yield satisfactory results. Interestingly, for the oversampled DCT matrix with $F=5$ and $F=10$, the $\mathrm{CSDTN}(1,2,1)$ method achieves the best reconstruction performance.

Next, we conducted a simulation study for the noisy case. We aimed to recover a 130-sparse signal $\mathbf{x}$ of length $N=512$ from noisy Gaussian random measurements with Gaussian noise (zero mean and standard deviation $\sigma =0.1$) in this experiment. We fixed $\lambda ={10}^{-1}$ for all trials in this set of experiments. The measurement matrix was a size of $m\times 512$ with m ranging from 240 to 400. We evaluated the reconstruction performance using the mean-square-error (MSE) metric over 100 replications. Moreover, we computed the benchmark MSE of an oracle solution as ${\sigma }^2\mathrm{tr}{\left(A_S^T{A}_S\right)}^{-1}$, where S was the support of the ground-truth $\mathbf{x}$. The oracle solution is the solution obtained when the support of the sparse signal is known in advance. We compared our proposed CSDTN methods with the ${\ell }_1$ and the oracle solutions. Figure 4 illustrates that all of our proposed CSDTN methods were very close to the oracle solution for large values of m, while the ${\ell }_1$ solution showed obvious bias.

Furthermore, we examined the effects of parameters $(p,q,t)$ on the performance in this noisy setting when m was fixed at 260. Our results indicated that when $q=2$ and $t=1$, $p=0.35$ achieved the lowest MSE. When $p=1$ and $t=1$, $q=1.4$ was the best choice, while $t=2$ was optimal when $p=1$ and $q=2$.

6. Conclusions

In this paper, we introduced the CSDTN as an innovative concave regularization approach for sparse signal recovery. The CSDTN method leverages the flexibility of nonconvex regularization to enhance performance in recovering sparse signals. The theoretical analysis demonstrated that the CSDTN provides strong guarantees for its robustness and stability. The experimental results show that the CSDTN method outperforms traditional methods in both reconstruction accuracy and robustness.

The CSDTN method offers flexibility in tuning parameters to achieve an optimal balance between sparsity and reconstruction fidelity. Future research could focus on adaptive strategies for parameter tuning, which would make the CSDTN approach even more robust and user-friendly. Additionally, exploring its practical applications in other areas of machine learning and signal processing holds promise, and these avenues are left open for future investigation.