An Inertial Relaxed CQ Algorithm with Two Adaptive Step Sizes and Its Application for Signal Recovery

Abstract

This article presents a novel inertial relaxed CQ algorithm for solving split feasibility problems. Note that the algorithm incorporates two adaptive step sizes here. A strong convergence theorem is established for the problem under some standard conditions. Additionally, we explore the utility of the algorithm in solving signal recovery problems. Its performance is evaluated against existing techniques from the literature.

1. Introduction

Throughout this article, all sets are assumed to be nonempty. Let $\mathbb{N},\mathbb{R}$, and ${\mathbb{R}}^N$ denote the sets of positive integers, of real numbers, and of ordered N-tuples of real numbers, where $N\in \mathbb{N}$, respectively. Let $X:=\left(X,⟨·,·⟩\right)$ and $Y:=\left(Y,⟨·,·⟩\right)$ be two inner product spaces with the induced norm $\parallel ·\parallel$. Also, let C and Q be closed and convex subsets of X and Y, respectively. We denote a useful subset of X as follows:

$$\begin{array}{c}\hfill \Omega :=\left\{x\in C|Ax\in Q\mathrm{where}A:X\to Y\mathrm{is}\mathrm{a}\mathrm{bounded}\mathrm{linear}\mathrm{operator}\right\}.\end{array}$$

(1)

The problem of finding $x\in \Omega$ has been well known as the split feasibility problem (SFP). Introduced by Censor and Elfving [1] in 1994, the SFP has garnered significant attention. Its applications span diverse fields, including image and signal processing, inverse problems, and especially machine learning, which is very attractive nowadays (for example, see [2,3,4,5]). Initially, various algorithms were proposed to solve the SFP in both finite- and infinite-dimensional spaces, all requiring the existence of the inverse of A. However, the CQ algorithm, introduced by Byrne [6], stands out as one of the most notable. This algorithm is computationally efficient when the metric projections onto C and Q can be conveniently computed. The algorithm generates the sequence $\left\{x_n\right\}$ by

$$x_{n+1}=P_C\left[x_n-\gamma A^{*}(I-P_Q)A{x}_n\right],\mathrm{for}\mathrm{all}n\ge 1,$$

where $P_C$ and $P_Q$ are the (nearest point) metric projections on C and Q, respectively; $A^{*}$ is the adjoint operator of A; and $\gamma$ (called the step size) is in $\left(0,{\displaystyle \frac{2}{{\parallel A\parallel }^2}}\right).$ Nevertheless, it may be impractical or computationally intensive to determine the metric projections exactly. More importantly, determining the step size, which depends on the operator norm, can be computationally challenging or difficult to estimate accurately.

In real-world applications, the sets C and Q are typically defined as the level sets of convex functions given by

$$\begin{array}{c}\hfill C=\left\{x\in X:c\left(x\right)\le 0\right\}\mathrm{and}Q=\left\{y\in Y:q\left(y\right)\le 0\right\}\end{array}$$

where $c:X\to \mathbb{R}$ and $q:Y\to \mathbb{R}$ are lower semi-continuous convex functions, and $\partial c$ and $\partial q$ are assumed to be bounded operators.

However, when C and Q are complicated, the nearest point projections $P_C$ and $P_Q$ have no closed forms, and thus, the computation is expensive. In 2004, Yang [7] presented a modification of the CQ algorithm, called a relaxed CQ algorithm, by considering C and Q as subsets of the half-spaces $C_n$ and $Q_n$ given by

$$\begin{array}{cc}\hfill C_n& =\left\{x\in X:c\left(x_n\right)\le ⟨{\lambda }_n,x_n-x⟩\right\}\mathrm{where}{\lambda }_n\in \partial c\left(x_n\right);\hfill \\ \hfill Q_n& =\left\{y\in Y:q\left(A{x}_n\right)\le ⟨{\eta }_n,A{x}_n-y⟩\right\}\mathrm{where}{\eta }_n\in \partial q\left(A{x}_n\right).\hfill \end{array}$$

(2)

Note that $P_{C_n}$ and $P_{Q_n}$ now have closed forms. Since these projections are easily calculated, this method appears to be very practical.

In the following, we define, for $n\in \mathbb{N}$,

$$\begin{array}{c}\hfill f_n(·)={\displaystyle \frac{1}{2}}{∥(I-P_{Q_n})A(·)∥}^2.\end{array}$$

(3)

Thus, for $n\in \mathbb{N}$,

$$\begin{array}{c}\hfill \nabla f_n=A^{*}(I-P_{Q_n})A.\end{array}$$

(4)

Next, in 2005, Yang [8] solved the SFP using a variable step size defined by

$${\gamma }_n={\displaystyle \frac{{\rho }_n}{\parallel \nabla f_n\left(x_n\right)\parallel }}\mathrm{for}\mathrm{all}n\in \mathbb{N},$$

where $\left\{{\rho }_n\right\}\subset (0,\infty )$ satisfies ${\displaystyle \sum _{n=1}^{\infty }{\rho }_n=\infty }$, and ${\displaystyle \sum _{n=1}^{\infty }{\rho }_n^2<\infty }$. Note that using this step size still requires two strong conditions, which are the boundedness of Q and the full column rank of $A.$ Later, López et al. [9] removed these two conditions by altering the step size as

$${\gamma }_n={\displaystyle \frac{{\rho }_n{f}_n\left(x_n\right)}{\parallel \nabla f_n\left(x_n\right){\parallel }^2}}\mathrm{for}\mathrm{all}n\in \mathbb{N}$$

where the sequence $\left\{{\rho }_n\right\}\subseteq (0,4)$ satisfies that ${\displaystyle \underset{n}{\inf }{\rho }_n(4-{\rho }_n)>0.}$ This step size is effective because its convergence result can be proven without relying on the matrix norm or any specific properties of Q and $A.$ Recent developments have introduced several algorithms that eliminate the need for the matrix norm in solving the SFP and other related problems, for example, see [10,11,12].

Subsequently, numerous algorithms and novel step sizes were proposed with the aim of enhancing the convergence rate of optimization algorithms. Among these, the inertial technique has gained significant attention. Introduced by Polyak [13] in 1964, it serves as an acceleration method for solving convex minimization problems with improved convergence speed. Its distinctive feature lies in its utilization of two preceding steps to compute the subsequent one. Alvarez et al. [14] extended the concept of the heavy ball method to the proximal point algorithm, leading to the development of the inertial proximal point algorithm. This algorithm is formulated as follows:

$$\begin{array}{cc}\hfill {\omega }_n& =x_n+{\chi }_n(x_n-x_{n-1}),\hfill \\ \hfill x_{n+1}& ={(I+{\gamma }_{n}F)}^{-1}\left({\omega }_n\right),\forall n\ge 1,\hfill \end{array}$$

(5)

where F is a maximal monotone operator. The convergence of the zero point of F is proven under the assumptions that $\left\{{\gamma }_n\right\}$ is non-decreasing and ${\chi }_n\in [0,1)$ satisfies ${\displaystyle \sum _{n=1}^{\infty }{\chi }_n{\parallel x_n-x_{n-1}\parallel }^2<\infty }$.

In [15], the authors proposed a modified inertial relaxed CQ algorithm (IRCQVA) by incorporating a viscosity approximation method and a novel step size strategy to solve the SFP as follows: for any $x_0,x_1\in X$ and all $n\in \mathbb{N}$,

$$\begin{array}{cc}\hfill {\omega }_n& =x_n+{\chi }_n(x_n-x_{n-1}),\hfill \\ \hfill x_{n+1}& =P_{C_n}\left[{\alpha }_{n}f\left({\omega }_n\right)+(1-{\alpha }_n)({\omega }_n-{\phi }_n\nabla f_n\left({\omega }_n\right))\right],\hfill \end{array}$$

(6)

where f is a contraction on $X,\left\{{\alpha }_n\right\}\subset (0,1),\left\{{\chi }_n\right\}\subset [0,1),\left\{{\rho }_n\right\}\subset (0,4)$, and

$$\begin{array}{c}\hfill {\phi }_n={\displaystyle \frac{{\rho }_n{f}_n\left({\omega }_n\right)}{\parallel \nabla f_n\left({\omega }_n\right){\parallel }^2+{\parallel (I-P_{C_n})\left({\omega }_n\right)\parallel }^2}}.\end{array}$$

(7)

It was shown that the sequence $\left\{x_n\right\}$ generated by (6) converges strongly to a point in $\Omega$ under the conditions ${\displaystyle \underset{n\to \infty }{\lim }{\alpha }_n=\underset{n\to \infty }{\lim }\frac{{\chi }_n}{{\alpha }_n}\parallel x_n-x_{n-1}\parallel =0,\sum _{n=1}^{\infty }{\alpha }_n=\infty }$, and ${\displaystyle \underset{n}{\inf }{\rho }_n(4-{\rho }_n)>0}$. Consequently, in 2019, Gibali et al. [16] obtained a new inertial relaxed CQ algorithm (M-IRCQA) to solve SFP (1) given as follows: for any $x_0,x_1\in X$ and all $n\in \mathbb{N}$, select ${\chi }_n$ such that $0\le {\chi }_n\le {\overline{\chi }}_n$, where ${\epsilon }_n=o\left({\alpha }_n\right)$ and

$${\overline{\chi }}_n=\left\{\begin{array}{lll}& \min \left\{{\displaystyle \frac{{\epsilon }_n}{\parallel x_n-x_{n-1}\parallel }},\chi \right\}& \mathrm{if}{x}_n\ne x_{n-1};\\ & \chi & \mathrm{otherwise}\end{array}\right.$$

for some $\chi \in [0,1)$. Compute

$$\begin{array}{cc}\hfill {\omega }_n& =x_n+{\chi }_n(x_n-x_{n-1}),\hfill \\ \hfill x_{n+1}& =(1-{\alpha }_n-{\gamma }_n){\omega }_n+{\gamma }_n{P}_{C_n}\left[{\omega }_n-{\phi }_n\nabla f_n\left({\omega }_n\right)\right],\hfill \end{array}$$

(8)

where $\left\{{\alpha }_n\right\},\left\{{\gamma }_n\right\}\subset (0,1),\left\{{\rho }_n\right\}\subset (0,4)$, and ${\phi }_n$ are defined as in (7). It also was shown that the sequence $\left\{x_n\right\}$ generated by (8) converges strongly to the minimum-norm element of $\Omega$ under the conditions that ${\displaystyle \underset{n\to \infty }{\lim }{\alpha }_n=0,\sum _{n=1}^{\infty }{\alpha }_n=\infty ,\underset{n}{\inf }{\gamma }_n(1-{\alpha }_n-{\gamma }_n)>0}$, and ${\displaystyle \underset{n}{\inf }{\rho }_n(4-{\rho }_n)>0}$.

In a recent study published in [4], the authors presented the inertial gradient CQ algorithm (IGCQHA). This algorithm employs Halpern iteration with a new step size to address the problem of solving the SFP. The algorithm is given as follows: for any $u,x_0,x_1\in X$ and all $n\in \mathbb{N}$,

$$\begin{array}{cc}\hfill {\omega }_n& =x_n+{\chi }_n(x_n-x_{n-1}),\hfill \\ \hfill y_n& ={\omega }_n-{\gamma }_n\nabla f_n\left({\omega }_n\right),\hfill \\ \hfill x_{n+1}& ={\alpha }_{n}u+(1-{\alpha }_n)P_{C_n}(y_n-{\phi }_n^{*}\nabla f_n\left(y_n\right)),\hfill \end{array}$$

(9)

where ${\gamma }_n={\displaystyle \frac{{\rho }_n{f}_n\left(x_n\right)}{\parallel \nabla f_n\left(x_n\right){\parallel }^2+{\theta }_n}},{\phi }_n^{*}={\displaystyle \frac{{\rho }_n{f}_n\left(y_n\right)}{\parallel \nabla f_n\left(y_n\right){\parallel }^2+{\theta }_n}},\left\{{\alpha }_n\right\},\left\{{\theta }_n\right\}\subset (0,1)$, $\left\{{\chi }_n\right\}\subset [0,1)$, and $\left\{{\rho }_n\right\}\subset (0,4).$ As a result, the sequence $\left\{x_n\right\}$ generated by (9) converges strongly to $P_{\Omega }u$ under the conditions that ${\displaystyle \underset{n\to \infty }{\lim }{\alpha }_n=\underset{n\to \infty }{\lim }{\theta }_n=\underset{n\to \infty }{\lim }\frac{{\chi }_n}{{\alpha }_n}\parallel x_n-x_{n-1}\parallel =0,\sum _{n=1}^{\infty }{\alpha }_n=\infty }$, and ${\displaystyle \underset{n}{\inf }{\rho }_n(4-{\rho }_n)>0}$.

Additionally, Ma and Liu [5] recently proposed the inertial Halpern-type CQ algorithm (IHTCQA) for solving the SFP with a step size that is bounded away from 0. Their algorithm scheme reads as follows: for any $u,x_0,x_1\in X$ and all $n\in \mathbb{N}$, select ${\chi }_n\subset [0,\chi )\subset [0,1)$ such that

$${\chi }_n=\left\{\begin{array}{ll}\min \left\{{\displaystyle \frac{{\epsilon }_n}{\parallel x_n-x_{n-1}\parallel }},\chi \right\}& \mathrm{if}{x}_n\ne x_{n-1};\\ \chi & \mathrm{otherwise},\end{array}\right.$$

where ${\epsilon }_n=o\left({\alpha }_n\right)$. Compute

$$\begin{array}{cc}\hfill {\omega }_n& =x_n+{\chi }_n(x_n-x_{n-1}),\hfill \\ \hfill x_{n+1}& ={\alpha }_{n}u+(1-{\alpha }_n)P_{C_n}\left[{\omega }_n-{\tau }_n\nabla f_n\left({\omega }_n\right)\right],\hfill \end{array}$$

(10)

using the step size scheme as follows: for any ${\tau }_1>0$ and $\delta \in (0,1]$,

$$\begin{array}{c}{\tau }_{n+1}=\left\{\begin{array}{lll}& \min \left\{{\displaystyle \frac{2\delta f_n\left({\omega }_n\right)}{{∥\nabla f_n\left({\omega }_n\right)∥}^2}},{\varphi }_n{\tau }_n+{\psi }_n\right\}& \mathrm{if}\nabla f_n\left({\omega }_n\right)\ne 0;\\ & {\varphi }_n{\tau }_n+{\psi }_n& \mathrm{otherwise},\end{array}\right.\end{array}$$

where $\left\{{\alpha }_n\right\}\subset (0,1)$, $\left\{{\varphi }_n\right\}\subset [1,\infty )$, and $\left\{{\psi }_n\right\}\subset [0,\infty ).$ It follows that the sequence $\left\{x_n\right\}$ defined by (10) converges strongly to $P_{\Omega }u$ under the conditions that ${\displaystyle \underset{n\to \infty }{\lim }{\alpha }_n=0,\sum _{n=1}^{\infty }{\alpha }_n=\infty }$, ${\displaystyle \sum _{n=1}^{\infty }({\varphi }_n-1)<\infty }$ and ${\displaystyle \sum _{n=1}^{\infty }{\psi }_n<\infty }$.

In this article, we propose a novel inertial relaxed CQ algorithm with two adaptive step sizes for solving the SFP. The algorithm is based on a combination of inertial acceleration and adaptive step size control. Note that the proposed algorithm has several advantages over existing methods. First, it incorporates an inertial term, which helps to accelerate the convergence speed. Second, it uses two adaptive step sizes, which allows the algorithm to automatically adjust its step size during the iteration process. In Section 3, we provide a comprehensive convergence analysis that establishes the strong convergence of the proposed algorithm. In Section 4, numerical experiments demonstrate the efficiency and reliability of the algorithm. It outperforms existing algorithms in terms of accuracy and speed. These advantages make the proposed algorithm a promising tool for solving a wide range of optimization problems.

2. Preliminaries

In this section, we present some notations, definitions, and results that will be used in the next section.

We denote ⇀ and → as weak and strong convergence, respectively.

Proposition 1.

For $u,v,w\in X$ and $a,b,c\in [0,1]$ such that $a+b+c=1$,

$$\begin{array}{cc}\hfill {\parallel u+v\parallel }^2& \le {\parallel u\parallel }^2+2⟨v,u+v⟩,\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill {\parallel au+(1-a)v\parallel }^2& ={a\parallel u\parallel }^2+{(1-a)\parallel v\parallel }^2-a(1-a){\parallel u-v\parallel }^2,\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill {\parallel au+bv+cw\parallel }^2& ={a\parallel u\parallel }^2+{b\parallel v\parallel }^2+{c\parallel w\parallel }^2{-ab\parallel u-v\parallel }^2{-ac\parallel u-w\parallel }^2-bc{\parallel v-w\parallel }^2.\hfill \end{array}$$

(13)

Definition 1.

A mapping $S:X\to X$ is said to be L-Lipschitz if there exists an $L\ge 0$ satisfying, for all $u,v\in X,$

$$\parallel Su-Sv\parallel \le L\parallel u-v\parallel .$$

The mapping S is called an L-contraction when $L\in [0,1),$ and if $L=1$, then S is called nonexpansive.

For $u\in X$, define $P_C:X\to C$ by

$$P_C\left(u\right):=\arg \underset{v\in C}{\min }\parallel u-v\parallel .$$

It is well known that $P_C$ is the metric projection such that for any $u\in X$ and $v\in C$,

$$\begin{array}{c}\hfill ⟨u-P_C\left(u\right),v-P_C\left(u\right)⟩\le 0.\end{array}$$

(14)

It is clear that $P_C$ is nonexpansive.

Definition 2.

Let $c:X\to \mathbb{R}$ be a function. Then, we have the following:

(i) 

A function c is said to be weakly lower semi-continuous (w-lsc) at z if for $z_n\in X$ such that $z_n\rightharpoonup z$ implies that

$$c\left(z\right)\le \underset{n\to \infty }{\lim \inf } c\left(z_n\right).$$

(ii) 

A function $c:X\to \mathbb{R}$ is convex if for all $u,v\in X$ and all $t\in (0,1),$

$$c(tu+(1-t\left)v\right)\le tc\left(u\right)+(1-t)c\left(v\right).$$

Proposition 2.

A differentiable function $c:X\to \mathbb{R}$ is convex if and only if for all $x\in X,$

$$⟨\nabla c\left(z\right),x-z⟩+c\left(z\right)\le c\left(x\right).$$

Definition 3.

An element $\overline{b}\in X$ is said to be a subgradient of $c:X\to \mathbb{R}$ at b if for all $x\in X$,

$$⟨\overline{b},x-b⟩+c\left(b\right)\le c\left(x\right).$$

This relation is called the subdifferentiable inequality.

Definition 4.

A function $c:X\to \mathbb{R}$ is subdifferentiable at z if it has at least one subgradient at z. A function c is called subdifferentiable if it is subdifferentiable at all $z\in X$.

The set of subgradients of c at the point z is called the subdifferential of c at z, and it is denoted by $\partial c\left(z\right)$. For a differentiable and convex function, its gradient and subgradient coincide.

We next collect some necessary lemmas for proving our main result.

Lemma 1

([17,18]). Let $\left\{{\lambda }_n\right\}$ and $\left\{b_n\right\}$ be sequences of $[0,\infty ),$ and let $\left\{c_n\right\}$ be a sequence of $\mathbb{R}$ such that ${\displaystyle \sum _{n=1}^{\infty }{c}_n<\infty }$. For a sequence $d_n\subseteq (0,1)$, suppose that there is $n_0\in \mathbb{N}$ such that

$$\begin{array}{c}\hfill {\lambda }_{n+1}\le (1-d_n){\lambda }_n+b_n+c_n\mathit{for}\mathit{all}n\ge n_0.\end{array}$$

Then, we have the following statements:

(i) 

If there exists some $K>0$ such that $b_n\le d_{n}K$, then $\left\{{\lambda }_n\right\}$ is bounded.

(ii) 

${\displaystyle \underset{n\to \infty }{\lim }{\lambda }_n=0}$ when ${\displaystyle \underset{n\to \infty }{\lim \sup }\frac{b_n}{d_n}\le 0}$ and ${\displaystyle \sum _{n=1}^{\infty }{d}_n=\infty .}$

Lemma 2

([19]). For a sequence $\left\{{\lambda }_n\right\}\subseteq \mathbb{R},$ if there exists a subsequence $\left\{{\lambda }_{n_j}\right\}$ such that ${\lambda }_{n_j}<{\lambda }_{n_j+1}$ for all $j,$ then we have the following:

(i) 

${\displaystyle \underset{n\to \infty }{\lim }\psi \left(n\right)=\infty }$ where $\psi \left(n\right):=\max \{k\le n:{\lambda }_k<{\lambda }_{k+1}\};$

(ii) 

${\left\{\psi \left(n\right)\right\}}_{n\ge n^{*}}$ is nondecreasing such that for all $n\ge n^{*}$,

$${\lambda }_{\psi \left(n\right)}\le {\lambda }_{\psi \left(n\right)+1}and{\lambda }_n\le {\lambda }_{\psi \left(n\right)+1}$$

3. Main Result

In this section, we first list all requirements for our algorithm below:

(R1)

$A:X\to Y$ is a bounded linear operator with its adjoint $A^{*}.$

(R2)

The sets $C_n$, $Q_n,$ and $f_n$ and its gradient $\nabla f_n$ are defined as in (2), (3), and (4), respectively, and $\partial c$ and $\partial q$ are bounded operators.

(R3)

$h:X\to \mathbb{R}$ is a function such that $\nabla h$ is $\varrho$-contractive (note that this implies that h is differentiable).

(R4)

$\left\{{\chi }_n\right\}\subseteq [0,\infty )$ is a sequence for which ${\displaystyle \underset{n\to \infty }{\lim }\frac{{\chi }_n}{{\alpha }_n}\parallel x_n-x_{n-1}\parallel =0}$, where $\left\{{\alpha }_n\right\}\subseteq (0,1)$ satisfies ${\displaystyle \underset{n\to \infty }{\lim }{\alpha }_n=0}$ and ${\displaystyle \sum _{n=1}^{\infty }{\alpha }_n=\infty }$ (see an example of $\left\{{\chi }_n\right\}$ in Section 4).

(R5)

$\left\{{\beta }_n\right\}$ and $\left\{{\gamma }_n\right\}$ are sequences in $(0,1)$ such that ${\alpha }_n+{\beta }_n+{\gamma }_n=1$ and ${\gamma }_n\in (\gamma ,{\gamma }^{*})\subset (0,1-{\alpha }_n)$ for some $\gamma ,{\gamma }^{*}\in (0,1).$

(R6)

$\left\{{\varphi }_n\right\},\left\{{\varphi }_n^{*}\right\}\subseteq [1,\infty )$ are sequences such that ${\displaystyle \sum _{n=1}^{\infty }({\varphi }_n-1)<\infty }$, ${\displaystyle \sum _{n=1}^{\infty }({\varphi }_n^{*}-1)<\infty }$, and $\left\{{\psi }_n\right\}$ and $\left\{{\psi }_n^{*}\right\}$ are sequences in $[0,\infty )$ such that ${\displaystyle \sum _{n=1}^{\infty }{\psi }_n<\infty }$ and ${\displaystyle \sum _{n=1}^{\infty }{\psi }_n^{*}<\infty }$.

We are now ready to present our algorithm (IVTCQA) (Algorithm 1).

Algorithm 1 Inertial viscosity-type CQ algorithm (IVTCQA)

Initialization: Take $x_0,x_1\in X$, ${\tau }_1,{\nu }_1>0,\delta ,\mu \in (0,2)$, and set $n:=1.$ Iterative Steps: Construct $\left\{x_n\right\}$ using the following steps: Step 1. Set ${\omega }_n=x_n+{\chi }_n\left(x_n-x_{n-1}\right)$ and compute $$\begin{array}{c}\hfill {\zeta }_n={\omega }_n-{\tau }_n\nabla f_n\left({\omega }_n\right).\end{array}$$ Step 2. Define $$\begin{array}{c}\hfill {\vartheta }_n=P_{C_n}\left[{\zeta }_n-{\nu }_n\nabla f_n\left({\zeta }_n\right)\right].\end{array}$$ If ${\vartheta }_n={\zeta }_n$, then stop, and ${\vartheta }_n\in \Omega$. Otherwise, continue. Step 3. Calculate $$\begin{array}{c}\hfill x_{n+1}={\alpha }_n\nabla h\left(x_n\right)+{\beta }_n{x}_n+{\gamma }_n{\vartheta }_n.\end{array}$$ Step 4. Update $$\begin{array}{c}\hfill {\tau }_{n+1}=\left\{\begin{array}{lll}& \min \left\{{\displaystyle \frac{2\delta f_n\left({\omega }_n\right)}{{∥\nabla f_n\left({\omega }_n\right)∥}^2}},{\varphi }_n{\tau }_n+{\psi }_n\right\}& \mathrm{if}\nabla f_n\left({\omega }_n\right)\ne 0;\\ & {\varphi }_n{\tau }_n+{\psi }_n& \mathrm{otherwise}\end{array}\right.\end{array}$$ and $$\begin{array}{c}\hfill {\nu }_{n+1}=\left\{\begin{array}{lll}& \min \left\{{\displaystyle \frac{2\mu f_n\left({\zeta }_n\right)}{{∥\nabla f_n\left({\zeta }_n\right)∥}^2}},{\varphi }_n^{*}{\nu }_n+{\psi }_n^{*}\right\}& \mathrm{if}\nabla f_n\left({\zeta }_n\right)\ne 0;\\ & {\varphi }_n^{*}{\nu }_n+{\psi }_n^{*}& \mathrm{otherwise}.\end{array}\right.\end{array}$$ Replace n with $n+1$ and then repeat Step 1.

Prior to presenting our main theorem, we establish the following necessary results.

Proposition 3.

Let $\left\{{\vartheta }_n\right\}$ and $\left\{{\zeta }_n\right\}$ be the sequences generated by the IVTCQA. If ${\vartheta }_n={\zeta }_n$, then ${\vartheta }_n\in \Omega$.

Proof. 

The proof is similar to that of Proposition 3.2 (1) in [20]. □

Lemma 3.

For $\left\{{\tau }_n\right\}$ and $\left\{{\nu }_n\right\}$ generated by the IVTCQA, ${\displaystyle \underset{n\to \infty }{\lim }{\tau }_n=\tau }$ and ${\displaystyle \underset{n\to \infty }{\lim }{\nu }_n=\nu }$, where $\tau >\min \left\{{\displaystyle \frac{\delta }{\parallel A^{*}A\parallel }},{\tau }_1\right\}>0$ and $\nu >\min \left\{{\displaystyle \frac{\mu }{\parallel A^{*}A\parallel }},{\nu }_1\right\}>0$.

Proof. 

This proof is analogous to the proof of Lemma 3.1 [5]. □

We now proceed to present our main result.

Theorem 1.

Let $\left\{x_n\right\}$ be a sequence generated by the IVTCQA. Then, it converges strongly to

$$s=P_{\Omega }\left(\nabla h\left(s\right)\right).$$

Proof. 

Let $r\in \Omega$. Since $C\subset C_n$ and $Q\subset Q_n$, then $r=P_C\left(r\right)=P_{C_n}\left(r\right)$ and $Ar=P_Q\left(Ar\right)=P_{Q_n}\left(Ar\right)$. It follows that $\nabla f_n\left(r\right)=0$, and thus,

$$\begin{array}{cc}\hfill ⟨{\zeta }_n-r,\nabla f_n\left({\zeta }_n\right)⟩& =⟨{\zeta }_n-r,\nabla f_n\left({\zeta }_n\right)-\nabla f_n\left(r\right)⟩\hfill \\ \hfill & =⟨A{\zeta }_n-Ar,(I-P_{Q_n})A{\zeta }_n-(I-P_{Q_n})Ar⟩\hfill \\ \hfill & \ge {∥(I-P_{Q_n})A{\zeta }_n∥}^2\hfill \\ \hfill & =2f_n\left({\zeta }_n\right).\hfill \end{array}$$

(15)

Also, we have

$$\begin{array}{c}\hfill ⟨{\omega }_n-r,\nabla f_n\left({\omega }_n\right)⟩\ge 2f_n\left({\omega }_n\right).\end{array}$$

(16)

Set ${\delta }_n=2-{\displaystyle \frac{\delta {\tau }_n}{{\tau }_{n+1}}}$ and ${\mu }_n=2-{\displaystyle \frac{\mu {\nu }_n}{{\nu }_{n+1}}}$.

By Lemma 3, ${\displaystyle \underset{n\to \infty }{\lim }{\delta }_n=2-\delta >0}$ and ${\displaystyle \underset{n\to \infty }{\lim }{\mu }_n=2-\mu >0.}$ Then, there exists $n_0\in \mathbb{N}$ such that ${\delta }_n>0$ and ${\mu }_n>0$ for all $n\ge n_0$. If follows from the properties of ${\tau }_{n+1}$ and ${\nu }_{n+1}$ that

$$\begin{array}{c}\hfill {\tau }_{n+1}{∥\nabla f_n\left({\omega }_n\right)∥}^2\le 2\delta f_n\left({\omega }_n\right)\mathrm{and}{\nu }_{n+1}{∥\nabla f_n\left({\zeta }_n\right)∥}^2\le 2\mu f_n\left({\zeta }_n\right).\end{array}$$

(17)

By the nonexpansivity of $P_{C_n}$, and the definition of ${\zeta }_n$, (15)–(17), we obtain that

$$\begin{array}{cc}\hfill {∥{\vartheta }_n-r∥}^2& ={∥P_{C_n}({\zeta }_n-{\nu }_n\nabla f_n\left({\zeta }_n\right))-P_{C_n}\left(r\right)∥}^2\hfill \\ \hfill & \le {∥{\zeta }_n-r-{\nu }_n\nabla f_n\left({\zeta }_n\right)∥}^2\hfill \\ \hfill & ={∥{\zeta }_n-r∥}^2+{\nu }_n^2{∥\nabla f_n\left({\zeta }_n\right)∥}^2-2{\nu }_n⟨{\zeta }_n-r,\nabla f_n\left({\zeta }_n\right)⟩\hfill \\ \hfill & ={∥{\omega }_n-r∥}^2+{∥{\omega }_n-{\zeta }_n∥}^2+{\nu }_n^2{∥\nabla f_n\left({\zeta }_n\right)∥}^2+2⟨{\omega }_n-r,{\zeta }_n-{\omega }_n⟩\hfill \\ \hfill & -2{\nu }_n⟨{\zeta }_n-r,\nabla f_n\left({\zeta }_n\right)⟩\hfill \\ \hfill & ={∥{\omega }_n-r∥}^2+{\tau }_n^2{∥\nabla f_n\left({\omega }_n\right)∥}^2+{\nu }_n^2{∥\nabla f_n\left({\zeta }_n\right)∥}^2-2{\tau }_n⟨{\omega }_n-r,\nabla f_n\left({\omega }_n\right)⟩\hfill \\ \hfill & -2{\nu }_n⟨{\zeta }_n-r,\nabla f_n\left({\zeta }_n\right)⟩\hfill \\ \hfill & \le {∥{\omega }_n-r∥}^2+{\displaystyle \frac{2\delta {\tau }_n^2{f}_n\left({\omega }_n\right)}{{\tau }_{n+1}}}+{\displaystyle \frac{2\mu {\nu }_n^2{f}_n\left({\zeta }_n\right)}{{\nu }_{n+1}}}-4{\tau }_n{f}_n\left({\omega }_n\right)-4{\nu }_n{f}_n\left({\zeta }_n\right)\hfill \\ \hfill & ={∥{\omega }_n-r∥}^2-2{\delta }_n{\tau }_n{f}_n\left({\omega }_n\right)-2{\mu }_n{\nu }_n{f}_n\left({\zeta }_n\right).\hfill \end{array}$$

(18)

Then, from (18), we have that ${∥{\vartheta }_n-r∥}^2\le {∥{\omega }_n-r∥}^2$, and thus,

$$\begin{array}{c}\hfill ∥{\vartheta }_n-r∥\le ∥{\omega }_n-r∥\le ∥x_n-r∥+{\chi }_n∥x_n-x_{n-1}∥\mathrm{for}\mathrm{all}n\ge n_0.\end{array}$$

(19)

From the $\varrho$-contractivity of $\nabla h$ and the property of $x_{n+1}$,

$$\begin{array}{cc}\hfill ∥x_{n+1}-r∥& =∥{\alpha }_n\left(\nabla h\left(x_n\right)-r\right)+{\beta }_n\left(x_n-r\right)+{\gamma }_n\left({\vartheta }_n-r\right)∥\hfill \\ \hfill & \le {\alpha }_n∥\nabla h\left(x_n\right)-\nabla h\left(r\right)∥+{\alpha }_n∥\nabla h\left(r\right)-r∥+{\beta }_n∥x_n-r∥+{\gamma }_n∥{\vartheta }_n-r∥\hfill \\ \hfill & \le {\alpha }_n∥\nabla h\left(r\right)-r∥+\left(\varrho {\alpha }_n+{\beta }_n\right)∥x_n-r∥+{\gamma }_n∥{\vartheta }_n-r∥.\hfill \end{array}$$

Combining the above inequality with (19), we obtain that, for all $n\ge n_0$,

$$\begin{array}{cc}\hfill ∥x_{n+1}-r∥& \le \left(1-d_n\right)∥x_n-r∥+{\gamma }_n{\chi }_n∥x_n-x_{n-1}∥+{\alpha }_n∥\nabla h\left(r\right)-r∥\hfill \\ \hfill & \le \left(1-d_n\right)∥x_n-r∥+d_n{\kappa }_n\hfill \end{array}$$

(20)

where $d_n={\alpha }_n\left(1-\varrho \right)$ and ${\kappa }_n={\displaystyle \frac{1}{1-\varrho }}\left({\displaystyle \frac{{\chi }_n}{{\alpha }_n}}∥x_n-x_{n-1}∥+∥\nabla h\left(r\right)-r∥\right)$.

Next, we set $M_n=2M{\chi }_n∥x_n-x_{n-1}∥+{\chi }_n^2{∥x_n-x_{n-1}∥}^2,$ where ${\displaystyle M:=\underset{n\in \mathbb{N}}{\sup }∥x_n-s∥}$. Since ${\displaystyle \underset{n\to \infty }{\lim }\frac{{\chi }_n}{{\alpha }_n}∥x_n-x_{n-1}∥=0}$, $\left\{{\kappa }_n\right\}$ is bounded,

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }{M}_n=\underset{n\to \infty }{\lim }{\displaystyle \frac{M_n}{d_n}}=\underset{n\to \infty }{\lim }\parallel {\omega }_n-x_n\parallel =0.\end{array}$$

(21)

From (20) and the boundedness of $\left\{{\kappa }_n\right\}$, by Lemma 1 $\left(i\right)$, $\left\{∥x_n-r∥\right\}$ is bounded for any $r\in \Omega .$ Subsequently, $\left\{x_n\right\}$ is bounded and so is $\{\nabla h\left(x_n\right)\}.$ Since $\Omega$ is a nonempty, closed, and convex set, we have that $P_{\Omega }\circ \nabla h$ is $\rho$-contractive. It follows from the Banach fixed-point theorem that there exists a unique $s\in \Omega$ such that $s=P_{\Omega }\circ \nabla h\left(s\right).$

Next, let ${\lambda }_n={∥x_n-s∥}^2$. By (13) and (18), we have that

$$\begin{array}{cc}\hfill {\lambda }_{n+1}& ={\alpha }_n{∥\nabla h\left(x_n\right)-s∥}^2+{\beta }_n{\lambda }_n+{\gamma }_n{∥{\vartheta }_n-s∥}^2-{\alpha }_n{\beta }_n{∥\nabla h\left(x_n\right)-x_n∥}^2-{\beta }_n{\gamma }_n{∥x_n-{\vartheta }_n∥}^2\hfill \\ \hfill & -{\alpha }_n{\gamma }_n{∥\nabla h\left(x_n\right)-{\vartheta }_n∥}^2\hfill \\ \hfill & \le {\alpha }_n{∥\nabla h\left(x_n\right)-s∥}^2+{\beta }_n{\lambda }_n+{\gamma }_n{∥{\omega }_n-s∥}^2-2{\gamma }_n\left({\delta }_n{\tau }_n{f}_n\left({\omega }_n\right)+{\mu }_n{\nu }_n{f}_n\left({\zeta }_n\right)\right)\hfill \\ \hfill & -{\beta }_n{\gamma }_n{∥x_n-{\vartheta }_n∥}^2\hfill \\ \hfill & ={\alpha }_n{∥\nabla h\left(x_n\right)-s∥}^2+{\beta }_n{\lambda }_n-{\beta }_n{\gamma }_n{∥x_n-{\vartheta }_n∥}^2-2{\gamma }_n\left({\delta }_n{\tau }_n{f}_n\left({\omega }_n\right)+{\mu }_n{\nu }_n{f}_n\left({\zeta }_n\right)\right)\hfill \\ \hfill & +{\gamma }_n\left({\lambda }_n+2{\chi }_n⟨x_n-s,x_n-x_{n-1}⟩+{\chi }_n^2{\parallel x_n-x_{n-1}\parallel }^2\right)\hfill \\ \hfill & \le {\lambda }_n+{\alpha }_n{∥\nabla h\left(x_n\right)-s∥}^2+M_n-{\beta }_n{\gamma }_n{∥x_n-{\vartheta }_n∥}^2-2{\gamma }_n\left({\delta }_n{\tau }_n{f}_n\left({\omega }_n\right)+{\mu }_n{\nu }_n{f}_n\left({\zeta }_n\right)\right).\hfill \end{array}$$

Rearranging the above inequality, we have that

$$\begin{array}{c}\hfill {\beta }_n{\gamma }_n{∥x_n-{\vartheta }_n∥}^2+2{\gamma }_n\left({\delta }_n{\tau }_n{f}_n\left({\omega }_n\right)+{\mu }_n{\nu }_n{f}_n\left({\zeta }_n\right)\right)\le {\lambda }_n-{\lambda }_{n+1}+{\alpha }_n{∥\nabla h\left(x_n\right)-s∥}^2+M_n.\end{array}$$

(22)

Next, we consider the following two cases.

Case 1. Assume that there is $N\in \mathbb{N}$ such that for all $n\ge N$, ${\lambda }_{n+1}\le {\lambda }_n$ holds. Then, ${\lambda }_n$ is convergent. Also, from requirements (R4) and (R5), when taking $n\to \infty$, the right-handed side of (22) tends to zero, and thus,

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\parallel {\vartheta }_n-x_n\parallel =\underset{n\to \infty }{\lim }∥(I-P_{Q_n})A{\omega }_n∥=\underset{n\to \infty }{\lim }∥(I-P_{Q_n})A{\zeta }_n∥=0.\end{array}$$

(23)

Also, from ${\alpha }_n+{\beta }_n+{\gamma }_n=1,$

$$\begin{array}{cc}\hfill ∥x_{n+1}-x_n∥& \le ∥x_{n+1}-{\vartheta }_n∥+∥{\vartheta }_n-x_n∥\hfill \\ \hfill & =∥{\alpha }_n\nabla h\left(x_n\right)+{\beta }_n{x}_n+{\gamma }_n{\vartheta }_n-{\vartheta }_n∥+∥{\vartheta }_n-x_n∥\hfill \\ \hfill & \le {\alpha }_n∥\nabla h\left(x_n\right)-x_n∥+\left(1-{\gamma }_n\right)∥{\vartheta }_n-x_n∥+∥{\vartheta }_n-x_n∥\hfill \\ \hfill & ={\alpha }_n∥\nabla h\left(x_n\right)-x_n∥+(2-{\gamma }_n)∥{\vartheta }_n-x_n∥\to 0\mathrm{as}n\to \infty .\hfill \end{array}$$

(24)

From $P_{Q_n}\left(A{\omega }_n\right)\in Q_n$, we have that

$$\begin{array}{c}\hfill q\left(A{\omega }_n\right)\le ⟨{\eta }_n,\left(I-P_{Q_n}\right)A{\omega }_n⟩\end{array}$$

(25)

The boundedness of $\partial q$ implies that $\left\{{\eta }_n\right\}$ is bounded. By (23) and (25), we have

$$\begin{array}{c}\hfill q\left(A{\omega }_n\right)\le ∥{\eta }_n∥∥\left(I-P_{Q_n}\right)A{\omega }_n∥\to 0\mathrm{as}n\to \infty .\end{array}$$

(26)

We know that $\left\{x_n\right\}$ is bounded, so there exists a subsequence $\left\{x_{n_k}\right\}$ of $\left\{x_n\right\}$, which converges weakly to $x^{*}\in X$. Thus, from (21), we have ${\omega }_{n_k}\rightharpoonup x^{*}$ as $k\to \infty$, and so $A{\omega }_{n_k}\rightharpoonup A{x}^{*}$ as $k\to \infty$.

Since q is w-lsc and (26),

$${\displaystyle q\left(A{x}^{*}\right)\le \underset{k\to \infty }{\lim \inf } q\left(A{\omega }_{n_k}\right)\le 0.}$$

This implies that $A{x}^{*}\in Q$. From ${\vartheta }_n\in C_n$,

$$\begin{array}{c}\hfill c\left({\omega }_n\right)\le ⟨{\lambda }_n,{\omega }_n-{\vartheta }_n⟩.\end{array}$$

(27)

Now, the boundedness of $\partial c$ implies the boundedness of $\left\{{\lambda }_n\right\}.$ Combining this with (21), (23), and (27), we obtain that

$$c\left({\omega }_n\right)\le ∥{\lambda }_n∥\left(∥{\omega }_n-x_n∥+∥x_n-{\vartheta }_n∥\right)\to 0\mathrm{as}n\to \infty .$$

Similarly, one can show that $c\left(x^{*}\right)\le 0$, i.e., $x^{*}\in C$. We now have that $x^{*}\in \Omega$.

Next, let $h_n=⟨\nabla h\left(s\right)-s,x_n-s⟩$ and $s_n=x_n+{\gamma }_n\left({\vartheta }_n-x_n\right)$ for all $n\in \mathbb{N}$. Then,

$$\begin{array}{c}\hfill ∥s_n-x_n∥={\gamma }_n∥{\vartheta }_n-x_n∥.\end{array}$$

(28)

By (19), for all $n\ge n_0$,

$$\begin{array}{cc}\hfill ∥s_n-s∥& =∥(1-{\gamma }_n)x_n+{\gamma }_n{\vartheta }_n-s∥\hfill \\ \hfill & \le (1-{\gamma }_n)∥x_n-s∥+{\gamma }_n∥{\vartheta }_n-s∥\hfill \\ \hfill & \le (1-{\gamma }_n)∥x_n-s∥+{\gamma }_n∥x_n-s∥+{\gamma }_n{\chi }_n∥x_n-x_{n-1}∥\hfill \\ \hfill & =∥x_n-s∥+{\gamma }_n{\chi }_n∥x_n-x_{n-1}∥\hfill \end{array}$$

(29)

From (14) and (24),

$$\begin{array}{cc}\hfill \underset{n\to \infty }{\lim \sup } h_{n+1}& \le \underset{n\to \infty }{\lim \sup } ⟨\nabla h\left(s\right)-s,x_{n+1}-x_n⟩+\underset{n\to \infty }{\lim \sup } h_n\hfill \\ \hfill & =\underset{k\to \infty }{\lim }{h}_{n_k}=⟨\nabla h\left(s\right)-s,x^{*}-s⟩\le 0.\hfill \end{array}$$

(30)

Consequently, from (11) and (12),

$$\begin{array}{cc}\hfill {\lambda }_{n+1}& ={∥(1-{\alpha }_n)(s_n-s)+{\alpha }_n(\nabla h\left(x_n\right)-\nabla h\left(s\right))-{\alpha }_n(x_n-s_n)-{\alpha }_n(s-\nabla h\left(s\right))∥}^2\hfill \\ \hfill & \le {∥(1-{\alpha }_n)(s_n-s)+{\alpha }_n(\nabla h\left(x_n\right)-\nabla h\left(s\right))∥}^2-2{\alpha }_n⟨x_n-s_n+s-\nabla h\left(s\right),x_{n+1}-s⟩\hfill \\ \hfill & \le (1-{\alpha }_n){∥s_n-s∥}^2+{\alpha }_n{∥\nabla h\left(x_n\right)-\nabla h\left(s\right)∥}^2+2{\alpha }_n⟨s_n-x_n,x_{n+1}-s⟩+2{\alpha }_n{h}_{n+1}.\hfill \end{array}$$

The above inequality together with (28) and (29) implies that for all $n\ge n_0$,

$$\begin{array}{cc}\hfill {\lambda }_{n+1}& \le (1-{\alpha }_n){\left(∥x_n-s∥+{\gamma }_n{\chi }_n∥x_n-x_{n-1}∥\right)}^2+{\alpha }_n\varrho {\lambda }_n+2{\alpha }_n∥s_n-x_n∥∥x_{n+1}-s∥+2{\alpha }_n{h}_{n+1}\hfill \\ \hfill & \le \left(1-d_n\right){\lambda }_n+M_n+2M{\alpha }_n{\gamma }_n∥{\vartheta }_n-x_n∥+2{\alpha }_n{h}_{n+1}\hfill \\ \hfill & \le \left(1-d_n\right){\lambda }_n+b_n,\hfill \end{array}$$

(31)

where $b_n=d_n\left({\displaystyle \frac{M_n}{d_n}}+{\displaystyle \frac{2}{1-\varrho }}\left(M∥{\vartheta }_n-x_n∥+h_{n+1}\right)\right)$. Finally, from ${\displaystyle \sum _{n=1}^{\infty }{\alpha }_n=\infty }$, (21), (23), (30), (31), and Lemma 1 $\left(ii\right)$, we can conclude that ${\displaystyle \underset{n\to \infty }{\lim }{\lambda }_n=0.}$

Case 2. Suppose that for all $j\in \mathbb{N}$, there is $n_j\ge j$ such that ${\lambda }_{n_j}<{\lambda }_{n_j+1}$.

From Lemma 2, there is $n^{*}\in \mathbb{N}$ such that for all $n\ge n^{*},{\lambda }_{\psi \left(n\right)}\le {\lambda }_{\psi \left(n\right)+1}$ and ${\lambda }_n\le {\lambda }_{\psi \left(n\right)+1}$, where $\psi \left(n\right):=\max \left\{k\le n:{\lambda }_{\psi \left(k\right)}<{\lambda }_{\psi (k+1)}\right\}.$ Now, it follows from (22) that for all $n\ge n^{*}$,

$$\begin{array}{cc}\hfill {\beta }_{\psi \left(n\right)}{\gamma }_{\psi \left(n\right)}{∥x_{\psi \left(n\right)}-{\vartheta }_{\psi \left(n\right)}∥}^2& +2{\gamma }_{\psi \left(n\right)}\left({\delta }_{\psi \left(n\right)}{\tau }_{\psi \left(n\right)}{f}_{\psi \left(n\right)}\left({\omega }_{\psi \left(n\right)}\right)+{\mu }_{\psi \left(n\right)}{\nu }_{\psi \left(n\right)}{f}_{\psi \left(n\right)}\left({\zeta }_{\psi \left(n\right)}\right)\right)\hfill \\ \hfill & \le {\alpha }_{\psi \left(n\right)}{∥\nabla h\left(x_{\psi \left(n\right)}\right)-s∥}^2+M_{\psi \left(n\right)}.\hfill \end{array}$$

Since ${\displaystyle \underset{n\to \infty }{\lim }\psi \left(n\right)=\infty }$, then ${\displaystyle \underset{n\to \infty }{\lim }{\alpha }_{\psi \left(n\right)}=\underset{n\to \infty }{\lim }{M}_{\psi \left(n\right)}=0}$, which implies that

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\parallel {\vartheta }_{\psi \left(n\right)}-x_{\psi \left(n\right)}\parallel =\underset{n\to \infty }{\lim }∥(I-P_{Q_{\psi \left(n\right)}})A{\omega }_{\psi \left(n\right)}∥=\underset{n\to \infty }{\lim }∥(I-P_{Q_{\psi \left(n\right)}})A{\zeta }_{\psi \left(n\right)}∥=0.\end{array}$$

Now, an analogous argument to that employed in the preceding case shows that

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim \sup } h_{\psi \left(n\right)+1}\le 0.\end{array}$$

By (31), for a large enough n, we have that

$$\begin{array}{cc}\hfill {\lambda }_{\psi \left(n\right)+1}& \le \left(1-d_{\psi \left(n\right)}\right){\lambda }_{\psi \left(n\right)}+b_{\psi \left(n\right)}\hfill \\ \hfill & \le \left(1-d_{\psi \left(n\right)}\right){\lambda }_{\psi \left(n\right)+1}+b_{\psi \left(n\right)}.\hfill \end{array}$$

It follows that

$$\begin{array}{cc}\hfill {\lambda }_{\psi \left(n\right)+1}& \le {\displaystyle \frac{M_{\psi \left(n\right)}}{d_{\psi \left(n\right)}}}+{\displaystyle \frac{2}{1-\varrho }}\left(M∥{\vartheta }_{\psi \left(n\right)}-x_{\psi \left(n\right)}∥+h_{\psi \left(n\right)+1}\right).\hfill \end{array}$$

Then, ${\displaystyle \underset{n\to \infty }{\lim \sup } {\lambda }_{\psi \left(n\right)+1}\le 0}$. Therefore,

$${\displaystyle \underset{n\to \infty }{\lim }{\parallel x_{\psi \left(n\right)+1}-s\parallel }^2=\underset{n\to \infty }{\lim }{\lambda }_{\psi \left(n\right)+1}=0.}$$

From Lemma 2, ${\displaystyle \underset{n\to \infty }{\lim }{\lambda }_n\le \underset{n\to \infty }{\lim }{\lambda }_{\psi \left(n\right)+1}=0.}$ Finally, we can conclude that $\left\{x_n\right\}$ converges strongly to s. The proof is now complete. □

4. Numerical Exemplifications

In this section, we investigate a signal recovery problem within the framework of compressed sensing. In mathematical terms, a signal recovery problem can be expressed as a system of linear equations with more unknowns than equations:

$$\begin{array}{c}\hfill b=Ax+\epsilon ,\end{array}$$

(32)

where $x\in {\mathbb{R}}^N$ is the original signal, $b\in {\mathbb{R}}^M$ is the observed signal with noise $\epsilon$, and A is the $M\times N$ filter matrix with $M<N.$ It is well known that problem (32) can be solved through the least absolute shrinkage and selection operator (LASSO) problem:

$$\begin{array}{c}\hfill {\displaystyle \min \left\{\frac{1}{2}{\parallel A\omega -b\parallel }_2^2:\omega \in {\mathbb{R}}^N,{\parallel \omega \parallel }_1\le \varsigma \right\},}\end{array}$$

(33)

where $\varsigma >0$ is a given constant.

Under specific conditions imposed on A, the solution to minimization problem (33) is equivalent to the ${\ell }_0$-norm solution of the linear system. For the considered SFP, we define $f\left(\omega \right)={\displaystyle \frac{1}{2}}{\parallel (I-P_Q)A\omega \parallel }_2^2$, $C=\{\omega :\parallel \omega {\parallel }_1\le \varsigma \}$, and $Q=\left\{b\right\}$. Since the metric projection onto the closed convex set C does not have a closed-form solution, we utilize the subgradient projection. We define a convex function $c\left(\omega \right)={\parallel \omega \parallel }_1-\varsigma$ and, for a sequence $\left\{{\omega }_n\right\}\subset {\mathbb{R}}^N$, denote the level set $C_n$ by $\left\{\omega :c\left({\omega }_n\right)\le ⟨{\lambda }_n,{\omega }_n-\omega ⟩\right\}$, where ${\lambda }_n\in \partial c\left({\omega }_n\right)$. Then, the orthogonal projection onto $C_n$ can be calculated using the following formula:

$$P_{C_n}\left(\omega \right)=\left\{\begin{array}{c}\omega +{\displaystyle \frac{⟨{\lambda }_n,{\omega }_n-\omega ⟩-c\left({\omega }_n\right)}{\parallel {\lambda }_n{\parallel }_2^2}}{\lambda }_n\mathrm{if}c\left({\omega }_n\right)>⟨{\lambda }_n,{\omega }_n-\omega ⟩,\hfill \\ \omega \mathrm{otherwise}.\hfill \end{array}\right.$$

We note that the subdifferential $\partial c$ at ${\omega }_n$ is

$$\partial c\left({\omega }_n\right)=\left\{\begin{array}{cc}& 1\mathrm{if}{\omega }_n^j>0,\hfill \\ & [-1,1]\mathrm{if}{\omega }_n^j=0,\hfill \\ & -1\mathrm{if}{\omega }_n^j<0,\hfill \end{array}\right.$$

where ${\omega }_n^j$ is the jth component of the vector ${\omega }_n$.

For the experiments, we evaluate the performances of five algorithms in solving problem (33): (1) IGCQHA, (2) M-IRCQA, (3) IRCQVA, (4) IHTCQA, and our purposed algorithm (5) IVTCQA. For algorithms (1–4), the parameters were selected based on experiments from [4,5,15,16], respectively. For algorithm (5), we conducted multiple experiments using various sets of parameters and selected the set that performed the best.

The experimental setup is listed below:

-

The original signal x is generated uniformly from the interval $[-2,2]$ with k nonzero elements.

-

The Gaussian matrix A is generated using MATLAB’s $randn(M,N)$ function.

-

The observation b is generated by adding white Gaussian noise with a signal-to-noise ratio (SNR) of 40 and $\varsigma =k$.

-

The vectors $x_0$ and $x_1$ are randomly generated.

-

For all $n\in \mathbb{N}$, we set ${\alpha }_n={\displaystyle \frac{1}{n+1}}$ and

$${\chi }_n=\left\{\begin{array}{lll}& \min \left\{{\displaystyle \frac{1}{{(n+1)}^3{\parallel x_n-x_{n-1}\parallel }_2}},0.3\right\}& \mathrm{if}{x}_n\ne x_{n-1};\\ & 0.3& \mathrm{otherwise}.\end{array}\right.$$

-

For (1) IGCQHA, (2) M-IRCQA, and (3) IRCQVA, let ${\rho }_n=0.3$ for all $n\in \mathbb{N}.$

-

The vector u is generated randomly for (1) IGCQHA and (4) IHTCQA.

-

For (2) M-IRCQA and (5) IVTCQA, set ${\gamma }_n={\displaystyle \frac{4n}{5n+5}}$ for all $n\in \mathbb{N}.$

-

For (4) IHTCQA and (5) IVTCQA, let ${\tau }_1=0.01$, $\delta =0.3$, ${\psi }_n=0$, and ${\varphi }_n={\displaystyle \frac{22}{{(n+1)}^{1.1}}}+1$ for all $n\in \mathbb{N}.$

-

For (1) IGCQHA, let ${\theta }_n={\displaystyle \frac{1}{{(n+1)}^3}}$ for any $n\in \mathbb{N}$, and set $f(·)=0.9(·)$ for (3) IRCQVA.

-

For (5) IVTCQA, let ${\nu }_1=0.01$, $\mu =0.3$, ${\displaystyle h\left(\omega \right)=\frac{9}{20}\sum _{j=1}^N{\omega }_j^2}$, ${\psi }_n^{*}={\displaystyle \frac{1}{{(n+1)}^3}}$, and ${\varphi }_n^{*}={\displaystyle \frac{44}{{(n+1)}^{1.1}}}+1$ for all $n\in \mathbb{N}.$ We can verify that these meet our algorithm’s requirements.

Now, we consider three cases as follows:

• Case 1: $M=2^9,N=2^{10}$ and $k=2^6;$
• Case 2: $M=2^{10},N=2^{11}$ and $k=2^7;$
• Case 3: $M=2^{11},N=2^{12}$ and $k=2^8.$

We evaluate the accuracy of the recovered signals using the mean squared error (MSE), given as

$${\mathrm{MSE}}_n={\displaystyle \frac{1}{N}}{\parallel x_n-x\parallel }_2^2<5\times {10}^{-5}.$$

The computations were performed using MATLAB R2021a on an iMac equipped with an Apple M1 chip and 16 GB of RAM. The results obtained are presented below Table 1 and Figure 1, Figure 2, Figure 3, Figure 4, Figure 5 and Figure 6.

Table 1. Numerical comparison of five algorithms.

Algorithms	Case 1		Case 2		Case 3
	Iter	CPU Time	Iter	CPU Time	Iter	CPU Time
(1) IGCQHA	2140	1.4103	2878	4.2726	4266	33.3395
(2) M-IRCQA	3919	1.3405	4338	3.3317	6906	28.0780
(3) IRCQVA	505	0.2114	542	0.4056	732	3.0214
(4) IHTCQA	2183	0.7261	2818	2.1335	4343	18.2522
(5) IVTCQA	203	0.1579	234	0.3503	323	2.6396

Figure 1. From top to bottom: the original signal, the measurement, and the recovery signals from the five algorithms for Case 1.

Figure 2. From top to bottom: the original signal, the measurement, and the recovery signals from the five algorithms for Case 2.

Figure 3. From top to bottom: the original signal, the measurement, and the recovery signals from the five algorithms for Case 3.

Figure 4. Plots of MSE_n over Iter for Case 1.

Figure 5. Plots of MSE_n over Iter for Case 2.

Figure 6. Plots of MSE_n over Iter for Case 3.

After conducting multiple iterations of the experiment, we determined that $\chi =0.3$ consistently met the specified criteria. This parameter value yielded the most optimal results for the given example. Based on the aforementioned results, our proposed algorithm, (5) IVTCQA, stands out for its exceptional computational efficiency, outperforming the other four algorithms in terms of both execution time and the number of iterations required. This enhanced performance demonstrates the effectiveness of our algorithm in solving the considered problem.

5. Conclusions

We proposed a novel inertial relaxed CQ algorithm with two adaptive step sizes to solve the SFP. Our main theorem establishes the strong convergence of the proposed algorithm under certain conditions. We also applied the algorithm to the problem of signal recovery in compressed sensing. Numerical experiments demonstrated the efficiency and reliability of the proposed algorithm, and it outperforms existing algorithms in terms of accuracy and speed. The proposed algorithm has several advantages over existing methods. It incorporates an inertial term, which helps to accelerate the convergence speed. Also, it uses two adaptive step sizes, which allows the algorithm to automatically adjust its step size during the iteration process.