Two Bregman Projection Methods for Solving Variational Inequality Problems in Hilbert Spaces with Applications to Signal Processing

Abstract

Studying Bregman distance iterative methods for solving optimization problems has become an important and very interesting topic because of the numerous applications of the Bregman distance techniques. These applications are based on the type of convex functions associated with the Bregman distance. In this paper, two different extragraident methods were proposed for studying pseudomonotone variational inequality problems using Bregman distance in real Hilbert spaces. The first algorithm uses a fixed stepsize which depends on a prior estimate of the Lipschitz constant of the cost operator. The second algorithm uses a self-adaptive stepsize which does not require prior estimate of the Lipschitz constant of the cost operator. Some convergence results were proved for approximating the solutions of pseudomonotone variational inequality problem under standard assumptions. Moreso, some numerical experiments were also given to illustrate the performance of the proposed algorithms using different convex functions such as the Shannon entropy and the Burg entropy. In addition, an application of the result to a signal processing problem is also presented.

1. Introduction

Let H be a real Hilbert space with norm $\parallel ·\parallel$ and inner product $\langle ·,·\rangle .$ We study the Variational Inequality Problem (shortly, VIP) of the form:

$$\mathrm{Find}\overline{v}\in C\mathrm{such}\mathrm{that}\langle A\overline{v},u-\overline{v}\rangle \ge 0,\forall u\in C,$$

(1)

where $A:H\to H$ is a given operator and C is a nonempty closed convex subset of $H.$ We shall denote the set of solutions of the VIP (1) by ${\mathsf{\Omega }}_{VI}.$ The VIP has drawn much attention from researchers because of its importance as a core model for studying many mathematical problems which include convex programming, equilibrium problem, inclusion problem, split feasibility problem, complimentarity problem, minimization problem, etc, see e.g., [1,2]. Recently, VIP has also been used for studying optimization problems arising in machine learning, signal processing and linear inverse problem; see, for instance [3,4,5].

Due to its importance, many researchers have developed several iterative methods for solving the VIP (1) in Hilbert and Banach spaces, see for example [6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22] and references therein. One of the simplest methods known for solving the VIP is the Extragradient Method (EM) introduced by Korpelevich [23] for a case where A is monotone and Lipschitz operator in a finite-dimensional space as follows: Given $a_0\in C\subseteq {\mathbb{R}}^n,$ compute

$$\left\{\begin{array}{c}{b}_n=P_C(a_n-\gamma A{a}_n),\hfill \\ a_{n+1}=P_C(a_n-\gamma A{b}_n),n\ge 1,\hfill \end{array}\right.$$

(2)

where $\gamma \in (0,1/L),$ $A:H\to H$ is monotone and L-Lipschitz continuous which has been extended to infinite-dimensional spaces by many authors, e.g., [8,9,13,19,24,25]. However, the EM requires computing the projection onto C twice per each iteration which is not cost effective in practice. As a result of this drawback, there has been an increasing effort in finding appropriate method for solving the VIP. Censor  et al. [13] introduced a Subgradient Extragradient Method (SEM) by replacing the secong projection in SEM with projection onto a half-space as follows:

Algorithm SEM: Given $a_0\in C,$ $\gamma \in (0,\frac{1}{L})$, compute

$$\left\{\begin{array}{c}{a}_0\in C,\hfill \\ b_n=P_C(a_n-\gamma A{a}_n),\hfill \\ a_{n+1}=P_{T_n}(a_n-\gamma A{b}_n),\hfill \\ \mathrm{where}{T}_n:=\{x\in H:\langle a_n-\gamma A{a}_n-b_n,x-b_n\ge 0\}n\ge 0.\hfill \end{array}\right.$$

(3)

Note that the set $T_n$ is a half-space whose projection can be explicitly calculated (see for instance [26]). Under some mild assumptions, Censor et al. [13] obtained a weak convergence result for solving VIP using (3), and by modifying the SEM with Halpern iterative scheme (see [27]), they proved the strong convergences of the SEM under certain mild conditions (see for instance [14,15]).

It is worthy to mention that both the EM and SEM require two evaluations of A per each iteration. Popov [28] later proposed a nice modification of the EM that required the evaluation of A at only one point in each iteration. Malitsky and Semenov [29] modified the Popov’s extragradient method by replacing the second projection onto C with a projection onto a half-space. Further extension of Popov’s extragradient method was introduced by Nomirovskii et al. [30] by using Bregman distance function [31] in a finite-dimensional setting. Recently, Gibali [32] extended the methods of Malitsky and Semenov [29] and Nemirovskii et al. [30] to infinite-dimensional Hilbert space using a Bregman distance function. A weak convergence result is proved for the sequence generated by these methods provided the stepsize satisfies the condition $\lambda \in \left(0,\frac{\sqrt{2}-1}{L}\right),$ where A is monotone and L-Lipschitz continuous.

An obvious disadvantage in the above-mentioned results is that the stepsize of the algorithms requires a prior estimate of the Lipschitz constant L. It is known that the Lipschitz constant are too difficult to estimate in general. Even when it is possible, the estimate is often too small which affects the convergence of the algorithm. An effort in solving this problem for the case of monotone VIP was recently introduced by Hieu and Cholamjiak [33]. They introduced an extragradient method with Bregman distance which does not require a prior estimate of the Lipschitz constant. However, their algorithm can only be applicable to monotone VIP but not pseudomonotone. When the cost operator is not monotone, the result of [33] can not be applied to such VIP, for instance, see Example 3.12 and 4.2 of [19].

Motivated by these results, in the present paper, we first introduce two Bregman extragradient methods of Popov’s type for solving pseudomonotone variational inequalities in real Hilbert spaces. In the first algorithm, the stepsize requires a prior estimate of the Lipschitz constant. This extends the results of [30,32] from monotone VIP to pseudomonotone VIP in real Hilbert spaces. In the second algorithm, the stepsize is selected self-adaptively and does not require a prior estimate of the Lipschitz constant. This improves Algorithm 1 of the paper and also the results of  [30,32]. It also improves the method of [33,34] from the extragradient type method with Bregman distance to the subgradient extragradient method with Bregman distance. More so, it extends the result of [33,34] from monotone VIP to pseudomonotone VIP. We further provide some numerical examples using different types of convex functions and a numerical application of our result to the Least Absolute Shrinkage and Selection Operator (LASSO) problem in compressing sensing. We denote the strong and weak convergence of a sequence $\left\{a_n\right\}\subseteq H$ to a point $x\in H$ by $a_n\to x$ and $a_n\rightharpoonup x$ respectively.

The paper is organized as follows: We first recall some basic definitions and results in Section 2. We present our algorithm and discuss its convergence in Section 3. We give some numerical experiments in Section 4. We conclude with a final remark in Section 5.

2. Preliminaries

In this section, we recall some basic definitions and concepts that are needed for establishing our results.

Definition 1.

[20] Let$A:C\to H$be an operator. Then A is called

(i) monotone if $\langle Au-Az,u-z\rangle \ge 0$ for all $u,z\in C;$

(ii) α-strongly pseudomonotone if

$$\langle Au,z-u\rangle \ge 0\Rightarrow \langle Az,z-u\rangle \ge {\alpha \parallel u-z\parallel }^2\forall u,z\in C,\alpha >0;$$

(iii) pseudomonotone if for all $u,z\in C,$

$$\langle Au,z-u\rangle \ge 0\Rightarrow \langle Az,z-u\rangle \ge 0;$$

(iv) Lipschitz continuous if there exists a constant $L>0$ such that

$$\parallel Au-Az\parallel \le L\parallel u-z\parallel \forall u,z\in C;$$

(v) weakly sequentially continuous if for each sequence $\left\{a_n\right\}$, we have:

$$a_n\rightharpoonup p\Rightarrow A{a}_n\rightharpoonup Ap.$$

Definition 2.

A function $f:H\to \mathbb{R}\cup \{+\infty \}$ is said to be proper if its effective domain, i.e., $domf=\{v\in H:f(v)<+\infty \}$ is nonempty. The Fenchel conjugate function of f is the convex function $f^{*}:H\to \mathbb{R}$ defined by $f^{*}\left(b\right)=sup\{\langle b,x\rangle -f\left(x\right):x\in H\}.$ f is said to be subdifferentiable at a point $v\in E$ if the set $\partial f\left(v\right)=\{e\in H:f(y)-f(v)\ge \langle e,y-v\rangle ,\forall y\in H\}$ is nonempty. An element $e\in \partial f\left(x\right)$ is called subgradient of $f.$ If f is continuously differentiable, then $\partial f\left(x\right)=\left\{\mathsf{\nabla }f\right(x\left)\right\}$, where $\mathsf{\nabla }f\left(x\right)$ denotes the gradient of f at $x.$; f is call Legendre if

(i) int(domf) $\ne \mathsf{\varnothing }$ and $\partial f$ is single-valued on its domain

(ii) int(dom$f^{*}$) $\ne \mathsf{\varnothing }$ and $\partial f^{*}$ is single-valued on its domain.

The function f is said to be strongly convex with strong convexity constant $\sigma >0,$ if

$$f\left(u\right)\ge f\left(v\right)-\langle \mathsf{\nabla }f\left(v\right),u-v\rangle +\frac{\sigma }{2}{\parallel u-v\parallel }^2,\forall u\in \mathrm{dom}f\mathrm{and}v\in \mathrm{int}\left(\mathrm{dom}f\right).$$

(4)

A function $f:H\to \mathbb{R}$ is Gâteaux differentiable at $u\in int\left(domf\right)$ if the limit

$$f^o(u,v)=\underset{\lambda \to 0^{+}}{lim}{\displaystyle \frac{f(u+\lambda v)-f\left(u\right)}{\lambda }}\forall v\in H,$$

(5)

exists for any $v\in H.$ The gradient of f at u is a linear function $\mathsf{\nabla }f\left(u\right)$ defined by $\langle \mathsf{\nabla }f\left(u\right),y\rangle =f^o(u,v)$$\forall v\in H.$ When the limit in (5) holds for all $u\in int\left(domf\right),$ we say that f is Gâteaux differentiable. More so, when (5) is attained uniformly for any $v\in H$ with $\parallel v\parallel =1,$ we say that f is Fréchet differentiable. It is well-known that f is Fréchet differentiable if and only if the gradient $\mathsf{\nabla }f$ is norm-to-norm continuous at u (see [35]).

Given a Gâteaux differentiable function $f:H\to \mathbb{R}\cup \{+\infty \}$. The Bregman distance corresponding to the function f is defined by (see [31])

$${\mathsf{\Delta }}_f(v,u)=f\left(v\right)-f\left(u\right)-\langle \mathsf{\nabla }f\left(u\right),v-u\rangle \forall v\in \mathrm{dom}f\mathrm{and}u\in \mathrm{int}\left(\mathrm{dom}f\right).$$

(6)

Remark 1.

The Bregman distance was introduced by Bregman [31] as a nice and effective method for designing and analyzing feasibility and optimization algorithms. It is easy to see that for any $v\in domf,u,w\in int\left(domf\right)$, we have

$${\mathsf{\Delta }}_f(v,w)={\mathsf{\Delta }}_f(v,u)+{\mathsf{\Delta }}_f(u,w)+\langle \mathsf{\nabla }f\left(u\right)-\mathsf{\nabla }f\left(w\right),v-u\rangle .$$

(7)

Moreover, when f is strongly convex with strong convexity modulus $\sigma >0$, then the Bregman distance satisfies the following identity:

$${\mathsf{\Delta }}_f(v,u)\ge \frac{\sigma }{2}{\parallel v-u\parallel }^2.$$

(8)

It is important we emphasize that various types of function f gives different Bregman distance. We give the following important examples of some practical important types of function f and their corresponding Bregman distance (see, e.g., [33,36]):

Example 1.

(i) If $f\left(v\right)=\frac{1}{2}{\parallel v\parallel }^2,$ then

$${\mathsf{\Delta }}_f(v,u)=\frac{1}{2}{\parallel v-u\parallel }^2,$$

which is the squared Euclidean distance (SE).

(ii) If $f\left(v\right)={\displaystyle \sum _{i=1}^n}{a}_{i}log\left(v_i\right)$ called Shannon entropy, then its corresponding Bregman distance is given as

$${\mathsf{\Delta }}_f(v,u)=\sum _{i=1}^n(v_{i}log(\frac{v_i}{u_i})+u_i-v_i).$$

This distance is called Kullback–Leibler distance (KL) and as become a very important tool in several areas of applied mathematics such as machine learning.

(iii) If $f\left(v\right)=-{\displaystyle \sum _{i=1}^n}log\left(v_i\right)$ called Burg entropy, then its corresponding Bregman distance is given as

$${\mathsf{\Delta }}_f(v,u)={\displaystyle \sum _{i=1}^n}(log(\frac{v_i}{u_i})+\frac{v_i}{u_i}-1).$$

This is called Itakura–Saito distance (IS), which is very important in information theory.

(iv) If $f\left(v\right)=\frac{1}{2}{v}^{T}Dv$ where $v^T$ is stands for the transpose of $v\in {\mathbb{R}}^n,$ and $D=diag(1,2,\dots ,n)\in {\mathbb{R}}^n$ is a positive semidefinite matrix, then

$${\mathsf{\Delta }}_f(v,u)=\frac{1}{2}(v-u)D^T(v-u),$$

which is called the Mahalanobis distance (MD) used for cluster analysis.

Definition 3.

[37] The Bregman projection with respect to f of $v\in$ int(domf) onto a nonempty closed convex set $C\subset$ int(domf) is the unique vector ${\mathsf{\Pi }}_C\left(v\right)\in C$ satisfying

$${\mathsf{\Pi }}_C\left(v\right):=inf\{{\mathsf{\Delta }}_f(v,u):u\in C\}.$$

Similar with the metric projection, the Bregman projection is characterized by the following identities (see [37]):

$$z={\mathsf{\Pi }}_C\left(v\right)\iff \langle \mathsf{\nabla }f\left(v\right)-\mathsf{\nabla }f\left(z\right),u-z\rangle \le 0,\forall u\in C,$$

(9)

and

$${\mathsf{\Delta }}_f(u,{\mathsf{\Pi }}_C\left(v\right))+{\mathsf{\Delta }}_f({\mathsf{\Pi }}_C\left(v\right),v)\le {\mathsf{\Delta }}_f(u,v)\forall u\in C,v\in \mathrm{int}\left(\mathrm{dom}f\right).$$

(10)

Lemma 1.

[38] Let E be a Banach space and let $f:E\to (-\infty ,\infty ]$ be a proper strictly convex function so that it is G$\widehat{a}$teaux differentiable and $\left\{a_n\right\}$ is a sequence in E such that $a_n\rightharpoonup u$ for some $u\in E.$ Then

$$\underset{n\to \infty }{lim\; sup}{\mathsf{\Delta }}_f(u,a_n)<\underset{n\to \infty }{lim\; sup}{\mathsf{\Delta }}_f(v,a_n),$$

for all v in the interior of domf with $u\ne v$.

The following result is well-known; see, e.g., [30,32].

Lemma 2.

Let $\left\{a_n\right\}$ and $\left\{b_n\right\}$ be two non-negative real sequences such that

$$a_{n+1}\le a_n-b_n.$$

Then $\left\{a_n\right\}$ is bounded and ${\sum }_{n=0}^{\infty }{b}_n<\infty .$

Definition 4.

 [39,40] The Minty Variational Inequalities (MVI) are defined as finding a point $\overline{x}\in C$ such that

$$\begin{array}{c}\hfill \langle Ay,y-\overline{x}\rangle \ge 0,\forall y\in C.\end{array}$$

(11)

We denote by $M(C,A)$, the set of solution of (11). Some existence results for the MVIP have been presented in [39]. The assumption that $M(C,A)\ne \mathsf{\varnothing }$ has already been used for solving $VI(C,A)$ in finite dimensional spaces (see e.g., [41]).

Lemma 3.

 [40] Consider the VI (1). If the mapping $h:[0,1]\to E^{*}$ defined as $h\left(t\right)=A(tx+(1-t\left)y\right)$ is continuous for all $x,y\in C$ (i.e., h is hemicontinuous), then $M(C,A)\subset VI(C,A)$. Moreover, if A is pseudomonotone, then $VI(C,A)$ is closed, convex and $VI(C,A)=M(C,A).$

3. Main Results

In this section, we present our algorithms and their convergence analysis. In the sequel, we assume that the following assumptions hold.

Assumption 1.

(A1) The feasible set $C\subseteq H$ is a nonempty, closed and convex set.

(A2) The mapping $A:C\to H$ is pseudomonotone, L-Lipschitz continuous and weakly sequentially continuous on H.

(A3) The solution set $Sol:={\mathsf{\Omega }}_{VI}$ is nonempty.

(A4) The function $f:H\to \mathbb{R}\cup \{+\infty \}$ uniformly Fréchet differentiable, Legendre, strongly coercive and strongly convex with a strong convexity modulus $\sigma >0.$

3.1. Bregman Projection Method with Fixed Stepsize

We present our first algorithm as follows:

Algorithm BPMFS

Initialization: Pick $a_0,b_0\in H$ arbitrarily and let $\lambda >0.$ Set $n=0.$

Step 1: Compute

$$\begin{array}{ccc}\hfill a_1& =& {\mathsf{\Pi }}_C(▽f^{*}(\mathsf{\nabla }f\left(a_0\right)-\lambda A{b}_0)),\hfill \\ \hfill b_1& =& {\mathsf{\Pi }}_C\left(\mathsf{\nabla }{f}^{*}(\mathsf{\nabla }f\left(a_1\right)-\lambda A{b}_0)\right).\hfill \end{array}$$

Step 2: Given the current iterates $a_n,b_n$ and $b_{n-1},$ calculate $a_{n+1}$ and $b_{n+1}$ as follows:

$$\begin{array}{ccc}\hfill a_{n+1}& =& {\mathsf{\Pi }}_{T_n}\left(\mathsf{\nabla }{f}^{*}(\mathsf{\nabla }f\left(a_n\right)-\lambda A{b}_n)\right),\hfill \end{array}$$

(12)

$$\begin{array}{ccc}\hfill b_{n+1}& =& {\mathsf{\Pi }}_C\left(\mathsf{\nabla }{f}^{*}(\mathsf{\nabla }f\left(a_{n+1}\right)-\lambda A{b}_n)\right),\hfill \end{array}$$

(13)

where

$$T_n:=\{z\in H:\langle \mathsf{\nabla }f\left(a_n\right)-\lambda A{b}_{n-1}-\mathsf{\nabla }f\left(b_n\right),z-b_n\rangle \le 0\}.$$

(14)

If $a_{n+1}=a_n$ and $b_n=b_{n-1}$ or $A{b}_n=0:$ STOP. Otherwise, set $n\leftarrow n+1$ and repeat Step 2.

Remark 2.

The projection for $a_{n+1}$ in Step 2 can be calculated explicitly using the projection formula in [26]. Algorithm BPMFS is constructed based on the concepts of [28,29] using Bregman projections. As in [29], the main task of Algorithm BPMFS is to compute one projection for $b_{n+1}$ onto C and one evaluation of A at the current approximation $b_n.$ More so, it is easy to see from (12) that $C\subset T_n.$ Indeed from (12), we have that

$$b_n={\mathsf{\Pi }}_C\left(\mathsf{\nabla }{f}^{*}(\mathsf{\nabla }f\left(a_n\right)-\lambda A{b}_{n-1})\right),$$

hence, using (9), we obtain

$$\langle \mathsf{\nabla }f\left(a_n\right)-\lambda A{b}_{n-1}-\mathsf{\nabla }f\left(b_n\right),\omega -b_n\rangle \le 0\forall \omega \in C.$$

It follows from (14) that $C\subset T_n.$

Remark 3.

Moreover, if for some $n\ge 0,$$a_{n+1}=a_n$ and $b_n=b_{n-1}$ in Algorithm BPMFS. Then $b_n$ is a solution of the VI (1).

Proof. 

From $a_{n+1}=a_n,$ this implies that

$$\langle \mathsf{\nabla }f\left(a_{n+1}\right)-\mathsf{\nabla }f\left(a_n\right)+\lambda A{b}_n,z-a_{n+1}\rangle \ge 0\forall z\in T_n.$$

Hence

$$\lambda \langle A{b}_n,z-a_n\rangle \ge 0\forall z\in T_n.$$

Equivalently

$$\langle A{b}_n,z-b_n\rangle \ge \langle A{b}_n,a_n-b_n\rangle \forall z\in T_n.$$

Since $C\subset T_n,$ we get

$$\langle A{b}_n,z-b_n\rangle \ge \langle A{b}_n,a_n-b_n\rangle \forall z\in C.$$

(15)

On the other hand, it follows from the definition of $T_n$ that

$$\langle \mathsf{\nabla }f\left(a_n\right)-\lambda A{b}_{n-1}-\mathsf{\nabla }f\left(b_n\right),z-b_n\rangle \le 0\forall z\in T_n.$$

Substituting $b_{n-1}=b_n$ and $z=a_n$ into the above inequality, we get

$$\langle \mathsf{\nabla }f\left(a_n\right)-\lambda A{b}_n-\mathsf{\nabla }f\left(b_n\right),a_n-b_n\rangle \le 0.$$

Then we have

$$\lambda \langle A{b}_n,a_n-b_n\rangle \ge 0.$$

(16)

Combining (15) and (16), we get

$$\langle A{b}_n,z-b_n\rangle \ge 0\forall z\in C.$$

Therefore $b_n$ is a solution of the VI (1). □

Lemma 4.

Let $\left\{a_n\right\}$ and $\left\{b_n\right\}$ be the sequences generated from Algorithm BPMFS. Then the following inequality holds:

$$\begin{array}{ccc}\hfill {\mathsf{\Delta }}_f(z,a_{n+1})& \le & {\mathsf{\Delta }}_f(z,a_n)-\left(1-\frac{\lambda L\sqrt{2}}{\sigma }\right){\mathsf{\Delta }}_f(a_{n+1},b_n)-\left(1-\frac{\lambda L}{\sigma }(1+\sqrt{2})\right){\mathsf{\Delta }}_f(b_n,a_n)\hfill \\ & & +\frac{\lambda L}{\sigma }{\mathsf{\Delta }}_f(a_n,b_{n-1}),\hfill \end{array}$$

(17)

for any $z\in Sol.$

Proof. 

Since $z\in Sol\subset C\subset T_n$ and $a_{n+1}={\mathsf{\Pi }}_{T_n}\left(\mathsf{\nabla }{f}^{*}(\mathsf{\nabla }f\left(a_n\right)-\lambda A{b}_n)\right),$ we have from (9)

$$\langle \mathsf{\nabla }f\left(a_{n+1}\right)-(\mathsf{\nabla }f\left(a_n\right)-\lambda A{b}_n),z-a_{n+1}\rangle \ge 0.$$

This implies that

$$\langle \mathsf{\nabla }f\left(a_{n+1}\right)-\mathsf{\nabla }f\left(a_n\right)+\lambda A{b}_n,z-a_{n+1}\rangle \ge 0,$$

thus we get

$$\lambda \langle A{b}_n,z-a_{n+1}\rangle \ge \langle \mathsf{\nabla }f\left(a_{n+1}\right)-\mathsf{\nabla }f\left(a_n\right),a_{n+1}-z\rangle .$$

(18)

By the three point identity (7), we have

$$\langle \mathsf{\nabla }f\left(a_{n+1}\right)-\mathsf{\nabla }f\left(a_n\right),a_{n+1}-z\rangle ={\mathsf{\Delta }}_f(z,a_n)-{\mathsf{\Delta }}_f(z,a_{n+1})-{\mathsf{\Delta }}_f(a_{n+1},a_n).$$

This implies that

$$\lambda \langle A{b}_n,z-a_{n+1}\rangle \ge {\mathsf{\Delta }}_f(z,a_n)-{\mathsf{\Delta }}_f(z,a_{n+1})-{\mathsf{\Delta }}_f(a_{n+1},a_n).$$

Thus

$${\mathsf{\Delta }}_f(z,a_{n+1})\le {\mathsf{\Delta }}_f(z,a_n)-{\mathsf{\Delta }}_f(a_{n+1},a_n)+\lambda \langle A{b}_n,z-a_{n+1}\rangle .$$

(19)

Since A is pseudomonotone and $b_n\in C,$ it follows that $\langle A{b}_n,z-b_n\rangle \le 0.$ Then this implies that

$$\begin{array}{ccc}\hfill \langle A{b}_n,z-a_{n+1}\rangle & =& \langle A{b}_n,b_n-a_{n+1}\rangle +\langle A{b}_n,z-b_n\rangle \hfill \\ & \le & \langle A{b}_n,b_n-a_{n+1}\rangle .\hfill \end{array}$$

(20)

Combining (19) and (20), we get

$$\begin{array}{ccc}\hfill {\mathsf{\Delta }}_f(z,a_{n+1})& \le & {\mathsf{\Delta }}_f(z,a_n)-{\mathsf{\Delta }}_f(a_{n+1},a_n)+\lambda \langle A{b}_n,b_n-a_{n+1}\rangle +\lambda \langle A{b}_n,z-b_n\rangle \hfill \\ & \le & {\mathsf{\Delta }}_f(z,a_n)-{\mathsf{\Delta }}_f(a_{n+1},a_n)+\lambda \langle A{b}_n,b_n-a_{n+1}\rangle \hfill \\ & =& {\mathsf{\Delta }}_f(z,a_n)-{\mathsf{\Delta }}_f(a_{n+1},a_n)+\lambda \langle A{b}_{n-1},b_n-a_{n+1}\rangle \hfill \\ & & +\lambda \langle A{b}_n-A{b}_{n-1},b_n-a_{n+1}\rangle .\hfill \end{array}$$

(21)

However, we have

$$\begin{array}{ccc}\hfill \lambda \langle A{b}_{n-1},b_n-a_{n+1}\rangle & =& \langle \mathsf{\nabla }f\left(a_n\right)-\lambda A{b}_{n-1}-\mathsf{\nabla }f\left(b_n\right),a_{n+1}-b_n\rangle \hfill \\ & & +\langle \mathsf{\nabla }f\left(b_n\right)-\mathsf{\nabla }f\left(a_n\right),a_{n+1}-b_n\rangle .\hfill \end{array}$$

Since $a_{n+1}\in T_n$ and by the definition of $T_n,$ we get

$$\lambda \langle A{b}_{n-1},b_n-a_{n+1}\rangle \le \langle \mathsf{\nabla }f\left(b_n\right)-\mathsf{\nabla }f\left(a_n\right),a_{n+1}-b_n\rangle .$$

(22)

Using the three points identity (7) in (22), we obtain

$$\lambda \langle A{b}_{n-1},b_n-a_{n+1}\rangle \le {\mathsf{\Delta }}_f(a_{n+1},a_n)-{\mathsf{\Delta }}_f(a_{n+1},b_n)-{\mathsf{\Delta }}_f(b_n,a_n).$$

(23)

Combining (21) and (23), we get

$${\mathsf{\Delta }}_f(z,a_{n+1})\le {\mathsf{\Delta }}_f(z,a_n)-{\mathsf{\Delta }}_f(a_{n+1},b_n)-{\mathsf{\Delta }}_f(b_n,a_n)+\lambda \langle A{b}_n-A{b}_{n-1},b_n-a_{n+1}\rangle .$$

(24)

Note that

$$\begin{array}{ccc}\hfill \lambda \langle A{b}_{n-1}-A{b}_n,a_{n+1}-b_n\rangle & \le & \lambda \parallel A{b}_{n-1}-A{b}_n\parallel \parallel a_{n+1}-b_n\parallel \hfill \\ & \le & \lambda L\parallel b_{n-1}-b_n\parallel \parallel a_{n+1}-b_n\parallel \hfill \\ & \le & \lambda L\left\{\frac{1}{2\sqrt{2}}\parallel b_{n-1}-b_n{\parallel }^2+\frac{1}{\sqrt{2}}{\parallel a_{n+1}-b_n\parallel }^2\right\}\hfill \\ & \le & \frac{\lambda L}{2\sqrt{2}}((2+\sqrt{2})\parallel a_n-b_n{\parallel }^2+\sqrt{2}{\parallel a_n-b_{n-1}\parallel }^2)+\frac{\lambda L}{\sqrt{2}}{\parallel a_{n+1}-b_n\parallel }^2\hfill \\ & \le & \frac{\lambda L}{\sigma }(1+\sqrt{2}){\mathsf{\Delta }}_f(b_n,a_n)+\frac{\lambda L}{\sigma }{\mathsf{\Delta }}_f(a_n,b_{n-1})+\frac{\lambda L\sqrt{2}}{\sigma }{\mathsf{\Delta }}_f(a_{n+1},b_n),\hfill \end{array}$$

(25)

where in (25), we have used the following basic inequalities with (8):

$$ab\le \frac{{ϵ}^2}{2}{a}^2+\frac{1}{2{ϵ}^2}{b}^2\mathrm{and}{(a+b)}^2\le \sqrt{2}{a}^2+(2+\sqrt{2})b^2.$$

It follows from (24) and (25) that

$$\begin{array}{ccc}\hfill {\mathsf{\Delta }}_f(z,a_{n+1})& \le & {\mathsf{\Delta }}_f(z,a_n)-(1-\frac{\lambda L\sqrt{2}}{\sigma }){\mathsf{\Delta }}_f(a_{n+1},b_n)-(1-\frac{\lambda L}{\sigma }(1+\sqrt{2})){\mathsf{\Delta }}_f(b_n,a_n)\hfill \\ & & +\frac{\lambda L}{\sigma }{\mathsf{\Delta }}_f(a_n,b_{n-1}).\hfill \end{array}$$

Hence, we obtained the desired result. □

Theorem 1.

Suppose that Assumption A1 holds, and let $\lambda \in (0,\frac{(\sqrt{2}-1)\sigma }{L}).$ Then, the sequences $\left\{a_n\right\}$ and $\left\{b_n\right\}$ generated by Algorithm BPMFS converges weakly to a solution of the VI (1).

Proof. 

Let $z\in Sol.$ First, we show that the sequence $\left\{a_n\right\}$ is bounded. From Lemma 4, we obtain

$${\mathsf{\Delta }}_f(z,a_{n+1})\le {\mathsf{\Delta }}_f(z,a_n)+\frac{\lambda L}{\sigma }{\mathsf{\Delta }}_f(a_n,b_{n-1}).$$

Now let ${\mathsf{\Gamma }}_n={\mathsf{\Delta }}_f(z,a_n)+\frac{\lambda L}{\sigma }{\mathsf{\Delta }}_f(a_n,b_{n-1})$ and ${\mathsf{\Lambda }}_n=(1-\frac{\lambda L(1+\sqrt{2})}{\sigma })({\mathsf{\Delta }}_f(b_n,a_n)+{\mathsf{\Delta }}_f(a_{n+1},b_n)).$ Note that (17) can be re-written as

$$a_{n+1}\le {\mathsf{\Gamma }}_n-{\mathsf{\Lambda }}_n.$$

Hence, it follows from Lemma 2 that $\left\{a_n\right\}$ is bounded and ${lim}_{n\to \infty }{\mathsf{\Delta }}_f(b_n,a_n)=0.$ Consequently, $\left\{a_n\right\}$ is bounded and $\parallel a_n-b_n\parallel \to 0,$$\parallel a_{n+1}-b_n\parallel \to 0$ as $n\to \infty .$ This implies that $\parallel a_{n+1}-a_n\parallel \to 0$ as $n\to \infty .$ Furthermore

$$\parallel b_{n+1}-b_n\parallel \le \parallel b_{n+1}-a_{n+1}\parallel +\parallel a_{n+1}-a_n\parallel +\parallel a_n-b_n\parallel \to 0,\mathrm{as}n\to \infty .$$

(26)

Consequent

$$\underset{n\to \infty }{lim}\parallel a_{n+1}-b_{n+1}\parallel \le \underset{n\to \infty }{lim}(\parallel a_{n+1}-b_n\parallel +\parallel b_{n+1}-b_n\parallel )=0.$$

Since f is uniformly Fréchet differentiable, the $\mathsf{\nabla }f$ is norm-to-norm uniformly continuous on bounded subsets of E. Thus, we have

$$\underset{n\to \infty }{lim}\parallel \mathsf{\nabla }f\left(a_{n+1}\right)-\mathsf{\nabla }f\left(b_{n+1}\right)\parallel =0.$$

(27)

Since $\left\{a_n\right\}$ is bounded, then we can choose a subsequence $\left\{a_{n_k}\right\}$ of $\left\{a_n\right\}$ such that $a_{n_k}\rightharpoonup p.$ Since $\parallel a_n-b_n\parallel \to 0,$ then $b_{n_k}\rightharpoonup p.$ Since $\left\{b_n\right\}\subset C,$ thus $p\in C.$ We now show that $p\in Sol.$ Since $b_{n+1}={\mathsf{\Pi }}_C\left(\mathsf{\nabla }{f}^{*}(\mathsf{\nabla }f\left(a_{n+1}\right)-\lambda A{b}_n)\right),$ we have

$$\langle \mathsf{\nabla }f\left(a_{n+1}\right)-\lambda A{b}_n-\mathsf{\nabla }f\left(b_{n+1}\right),v-b_{n+1}\rangle \le 0,\forall v\in C.$$

Equivalently

$$\langle \mathsf{\nabla }f\left(a_{n+1}\right)-\mathsf{\nabla }f\left(b_{n+1}\right),v-b_{n+1}\rangle \le \langle \lambda A{b}_n,v-b_{n+1}\rangle \forall v\in C.$$

This implies that

$$\begin{array}{c}\hfill \langle \frac{\mathsf{\nabla }f\left(a_{n+1}\right)-\mathsf{\nabla }f\left(b_{n+1}\right)}{\lambda },v-b_{n+1}\rangle +\langle A{b}_n,b_{n+1}-b_n\rangle \le \langle A{b}_n,v-b_n\rangle \forall v\in C.\end{array}$$

(28)

Hence, we have

$$\begin{array}{c}\hfill \langle \frac{\mathsf{\nabla }f\left(a_{n_k+1}\right)-\mathsf{\nabla }f\left(b_{n_k+1}\right)}{\lambda },v-b_{n_k+1}\rangle +\langle A{b}_{n_k},b_{n_k+1}-b_{n_k}\rangle \le \langle A{b}_{n_k},v-b_{n_k}\rangle \forall v\in C.\end{array}$$

Taking limits of the above inequality as $k\to \infty ,$ we get

$$\underset{k\to \infty }{lim\; inf}\langle A{b}_{n_k},v-b_{n_k}\rangle \ge 0\forall v\in C.$$

(29)

We choose a sequence $\left\{{ϵ}_k\right\}$ of positive numbers such that $\left\{{ϵ}_k\right\}$ is decreasing and convergence to $0.$ For each $k\ge 1,$ we denote by $N_k$ the smallest positive integer such that

$$\langle A{b}_{n_k},v-b_{n_k}\rangle +{ϵ}_k\ge 0\forall k\ge N_k.$$

Since $\left\{{ϵ}_k\right\}$ is decreasing, it is easy to see that the sequence $\left\{N_k\right\}$ is non-decreasing. Thus, for each $k\ge 1,$ since $\left\{b_n\right\}\subset C,$ we have $A{b}_{N_k}\ne 0$ and by setting

$${\vartheta }_{N_k}=\frac{A{b}_{N_k}}{\parallel A{b}_{N_k}{\parallel }^2},$$

we have $\langle A{b}_{N_k},{\vartheta }_{N_k}\rangle =1$ for each $k\ge 1.$ Now, it follows from (29) that for each $k\ge 1,$

$$\langle A{b}_{N_k},x+{ϵ}_k{\vartheta }_{N_k}-b_{N_k}\rangle \ge 0.$$

Since A is pseudomonotone, we get

$$\langle A(v+{ϵ}_k{\vartheta }_{N_k}),v+{ϵ}_k{\vartheta }_{N_k}-b_{N_k}\rangle \ge 0.$$

This implies that

$$\langle Ax,v-b_{N_k}\rangle \ge \langle Av-A(x+{ϵ}_k{\vartheta }_{N_k}),v+{ϵ}_k{\vartheta }_{N_k}-b_{N_k}\rangle -{ϵ}_k\langle Ax,{\vartheta }_{N_k}\rangle .$$

Now we show that ${lim}_{k\to \infty }{ϵ}_k{\vartheta }_{N_k}=0.$ Indeed, since $a_{n_k}\rightharpoonup p$ and ${lim}_{k\to \infty }\parallel a_{n_k}-b_{n_k}\parallel =0,$ we obtain that $b_{N_k}\rightharpoonup p$ as $k\to \infty .$ Since $\left\{b_n\right\}\subset C$ and A is weakly sequentially continuous on H, it follows that $\left\{A{b}_{N_k}\right\}$ converges weakly to $Ap.$ We can suppose that $Ap\ne 0$ (otherwise, p is a solution). Since the norm is sequentially weakly lower semicontinuous, we have

$$0>\parallel Ap\parallel \le \underset{k\to \infty }{lim\; inf}\parallel A{b}_{n_k}\parallel .$$

Since $\left\{b_{N_k}\right\}\subset \left\{b_{n_k}\right\}$ and ${ϵ}_k\to 0$ as $k\to \infty ,$ we get

$$0\le \underset{k\to \infty }{lim\; sup}\parallel {ϵ}_k{\vartheta }_{N_k}\parallel =\underset{k\to \infty }{lim\; sup}\left(\frac{{ϵ}_k}{\parallel A{b}_{n_k}\parallel }\right)\le \frac{{lim\; sup}_{k\to \infty }{ϵ}_k}{{lim\; inf}_{k\to \infty }\parallel A{b}_{n_k}\parallel }=0,$$

which implies that ${lim}_{k\to \infty }{ϵ}_k{\vartheta }_{N_k}=0.$ Thus, we get

$$\underset{k\to \infty }{lim\; inf}\langle Ax,v-b_{N_k}\rangle \ge 0.$$

Hence for all $v\in C,$ we get

$$\langle Ax,v-p\rangle =\underset{k\to \infty }{lim}\langle Ax,x-b_{N_k}\rangle =\underset{k\to \infty }{lim\; inf}\langle Ax,v-b_{N_k}\rangle \ge 0.$$

Therefore by Lemma 3, $p\in Sol={\mathsf{\Omega }}_{VI}.$

Finally, we show that p is unique. Assume the contrary, i.e., there exists a subsequence $\left\{a_{n_j}\right\}$ of $\left\{a_n\right\}$ such that $a_{n_j}\rightharpoonup q$ with $q\ne p.$ Following similar argument has above, we get $q\in {\mathsf{\Omega }}_{VI}.$ It follows from the Bregman opial-like property of H (more precisely, Lemma 1) that

$$\begin{array}{ccc}\hfill \underset{n\to \infty }{lim}{\mathsf{\Delta }}_f(p,a_n)& =& \underset{k\to \infty }{lim}{\mathsf{\Delta }}_f(p,a_{n_k})>\underset{k\to \infty }{lim}{\mathsf{\Delta }}_f(q,a_{n_k})\hfill \\ & =& \underset{n\to \infty }{lim}{\mathsf{\Delta }}_f(q,a_n)=\underset{j\to \infty }{lim}{\mathsf{\Delta }}_f(q,a_{n_j})\hfill \\ & >& \underset{j\to \infty }{lim}{\mathsf{\Delta }}_f(p,a_{n_j})=\underset{n\to \infty }{lim}{\mathsf{\Delta }}_f(p,a_n),\hfill \end{array}$$

which is a contradiction. Thus, we have $p=q$ and the desired result follows. This completes the proof. □

Remark 4.

We note that Algorithm BPMFS converges strongly to a solution of the VI (1) if the operator $A:E\to E^{*}$ is γ-strongly pseudomonotone and L-Lipschitz continuous.

Proof. 

Indeed, let $p\in Sol.$ Following similar argument in the proof of Lemma 4 till (19), we have

$${\mathsf{\Delta }}_f(p,a_{n+1})\le {\mathsf{\Delta }}_f(p,a_n)-{\mathsf{\Delta }}_f(a_{n+1},a_n)+\lambda \langle A{b}_n,p-a_{n+1}\rangle .$$

(30)

Since p is a solution of the VI (1), we have $\langle Ap,v-p\rangle \ge 0$ for all $v\in C.$ Using the strong pseudomonotonicity of A, we obtain $\langle Ax,v-p\rangle \ge {\gamma \parallel x-p\parallel }^2$ for all $x\in C.$ Taking $v=b_n\in C,$ we get

$$\langle A{b}_n,p-b_n\rangle \le -\gamma {\parallel b_n-p\parallel }^2.$$

Hence, we have

$$\begin{array}{ccc}\hfill \langle A{b}_n,p-a_{n+1}\rangle & =& \langle A{b}_n,p-b_n\rangle +\langle A{b}_n,b_n-a_{n+1}\rangle \hfill \\ & \le & -\gamma \parallel b_n{-p\parallel }^2+\langle A{b}_n,b_n-a_{n+1}\rangle .\hfill \end{array}$$

(31)

Therefore (29) and (30), we get

$$\begin{array}{ccc}\hfill {\mathsf{\Delta }}_f(p,a_{n+1})& \le & {\mathsf{\Delta }}_f(p,a_n)-{\mathsf{\Delta }}_f(a_{n+1},a_n)+\lambda \langle A{b}_n,b_n-a_{n+1}\rangle -\gamma \lambda {\parallel b_n-p\parallel }^2.\hfill \end{array}$$

Using (7), we have

$$\begin{array}{ccc}\hfill {\mathsf{\Delta }}_f(p,a_{n+1})& \le & {\mathsf{\Delta }}_f(p,a_n)-{\mathsf{\Delta }}_f(a_{n+1},b_n)-{\mathsf{\Delta }}_f(b_n,a_n)-\langle \mathsf{\nabla }f\left(b_n\right)-\mathsf{\nabla }f\left(a_n\right),a_{n+1}-b_n\rangle \hfill \\ & & +\lambda \langle A{b}_n,b_n-a_{n+1}\rangle -\gamma \lambda {\parallel b_n-p\parallel }^2\hfill \\ & =& {\mathsf{\Delta }}_f(p,a_n)-{\mathsf{\Delta }}_f(a_{n+1},b_n)-{\mathsf{\Delta }}_f(b_n,a_n)+\langle \mathsf{\nabla }f\left(a_n\right)-\lambda A{b}_n-\mathsf{\nabla }f\left(b_n\right),a_{n+1}-b_n\rangle \hfill \\ & & -\gamma \lambda \parallel b_n{-p\parallel }^2.\hfill \end{array}$$

(32)

Note that

$$\begin{array}{ccc}\hfill \langle \mathsf{\nabla }f\left(a_n\right)-\lambda A{b}_n-\mathsf{\nabla }f\left(b_n\right),a_{n+1}-b_n\rangle & =& \langle \mathsf{\nabla }f\left(a_n\right)-\lambda A{b}_{n-1}-\mathsf{\nabla }f\left(b_n\right),a_{n+1}-b_n\rangle \hfill \\ & & +\lambda \langle A{b}_{n-1}-A{b}_n,a_{n+1}-b_n\rangle .\hfill \end{array}$$

Since $a_{n+1}\in T_n,$ then we have

$$\langle \mathsf{\nabla }f\left(a_n\right)-\lambda A{b}_{n-1}-\mathsf{\nabla }f\left(b_n\right),a_{n+1}-b_n\rangle \le 0.$$

Hence, we have

$$\begin{array}{ccc}\hfill \langle \mathsf{\nabla }f\left(a_n\right)-\lambda A{b}_n-\mathsf{\nabla }f\left(b_n\right),a_{n+1}-b_n\rangle & \le & \lambda \langle A{b}_{n-1}-A{b}_n,a_{n+1}-b_n\rangle \hfill \\ & \le & \lambda L\parallel b_{n-1}-b_n\parallel \parallel a_{n+1}-b_n\parallel \hfill \\ & \le & \lambda L\{\frac{1}{2\sqrt{2}}\parallel b_{n-1}-b_n{\parallel }^2+\frac{1}{\sqrt{2}}{\parallel a_{n+1}-b_n\parallel }^2\}\hfill \\ & \le & \frac{\lambda L}{2\sqrt{2}}((2+\sqrt{2})\parallel a_n-b_n{\parallel }^2+\sqrt{2}{\parallel a_n-b_{n-1}\parallel }^2)\hfill \\ & & +\frac{\lambda L}{\sqrt{2}}{\parallel a_{n+1}-b_n\parallel }^2\hfill \\ & \le & \frac{\lambda L}{\sigma }(1+\sqrt{2}){\mathsf{\Delta }}_f(b_n,a_n)+\frac{\lambda L}{\sigma }{\mathsf{\Delta }}_f(a_n,b_{n-1})\hfill \\ & & +\frac{\lambda L\sqrt{2}}{\sigma }{\mathsf{\Delta }}_f(a_{n+1},b_n).\hfill \end{array}$$

(33)

Combining (31) and (32), we get

$$\begin{array}{ccc}\hfill {\mathsf{\Delta }}_f(p,a_{n+1})& \le & {\mathsf{\Delta }}_f(p,a_n)-\left(1-\frac{\lambda L\sqrt{2}}{\sigma }\right){\mathsf{\Delta }}_f(a_{n+1},b_n)-\left(1-\frac{\lambda L}{\sigma }(1+\sqrt{2})\right){\mathsf{\Delta }}_f(b_n,a_n)\hfill \\ & & +\frac{\lambda L}{\sigma }{\mathsf{\Delta }}_f(a_n,b_{n-1})-\gamma \lambda {\parallel b_n-p\parallel }^2.\hfill \end{array}$$

(34)

Following similar argument as in the Proof of Theorem 1, we have that the sequence $\left\{a_n\right\}$ is bounded and $\parallel b_n-a_n\parallel \to 0,$$\parallel a_{n+1}-b_n\parallel \to 0$ as $n\to \infty .$$\parallel a_{n+1}-a_n\parallel \to 0$ as $n\to \infty .$ Since f is continuous on bounded sets and $\mathsf{\nabla }f$ is weakly–weakly continuous, we get $\parallel f\left(a_{n+1}\right)-f\left(a_n\right)\parallel \to 0$ and $\parallel \mathsf{\nabla }f\left(a_{n+1}\right)-\mathsf{\nabla }f\left(a_n\right)\parallel \to 0$ as $n\to \infty .$ Therefore

$$\begin{array}{ccc}\hfill {\mathsf{\Delta }}_f(p,a_n)-{\mathsf{\Delta }}_f(p,a_{n+1})& =& f\left(p\right)-f\left(a_n\right)-\langle \mathsf{\nabla }f\left(a_n\right),p-a_n\rangle -f\left(p\right)+f\left(a_{n+1}\right)+\langle \mathsf{\nabla }f\left(a_{n+1}\right),p-a_{n+1}\rangle \hfill \\ & =& f\left(a_{n+1}\right)-f\left(a_n\right)+\langle \mathsf{\nabla }f\left(a_{n+1}\right),p-a_{n+1}\rangle -\langle \mathsf{\nabla }f\left(a_n\right),p-a_n\rangle \hfill \\ & =& f\left(a_{n+1}\right)-f\left(a_n\right)+\langle \mathsf{\nabla }f\left(a_{n+1}\right)-\mathsf{\nabla }f\left(a_n\right),p-a_n\rangle +\langle \mathsf{\nabla }f\left(a_{n+1}\right),a_n-a_{n+1}\rangle .\hfill \end{array}$$

Thus, we have that

$$\underset{n\to \infty }{lim}({\mathsf{\Delta }}_f(p,a_n)-{\mathsf{\Delta }}_f(p,a_{n+1}))=0.$$

Note that

$${\mathsf{\Delta }}_f(a_n,b_{n-1})=f\left(a_n\right)-f\left(b_{n-1}\right)-\langle \mathsf{\nabla }f\left(b_{n-1}\right),a_n-b_{n-1}\rangle \to 0.$$

It follows from (34) that

$$\gamma \lambda \parallel b_n{-p\parallel }^2\le {\mathsf{\Delta }}_f(p,a_n)-{\mathsf{\Delta }}_f(p,a_{n+1})+\frac{\lambda L}{\sigma }{\mathsf{\Delta }}_f(a_n,b_{n-1})\to 0.$$

Therefore, we get

$$\underset{n\to \infty }{lim}\parallel b_n-p\parallel =0.$$

Since $\parallel a_n-b_n\parallel \to 0,$ then $\parallel a_n-p\parallel \to 0$ as $n\to \infty .$ This completes the proof. □

3.2. Bregman Projection Method with Self-Adaptive Stepsize

Next, we propose a Bregman subgradient extragradient method whose convergence does not require a prior estimate of the Lipschitz constant of the cost operator A. The importance of the second algorithm is to cater for the situations when estimating the Lipschitz constant of A is very difficult. So, the algorithm uses a self-adaptive process for selecting its stepsize and does not require a prior estimate of the Lipschitz constant of A.

Algorithm BPMSAS

Initialization: Pick $a_0,b_0\in H,{\lambda }_0>0$ arbitrarily and $\alpha \in (0,1).$

Step 1: Compute

$$\begin{array}{ccc}\hfill a_1& =& {\mathsf{\Pi }}_C\left(\mathsf{\nabla }{f}^{*}(\mathsf{\nabla }f\left(a_0\right)-{\lambda }_{n}A{b}_0)\right),\hfill \\ \hfill b_1& =& {\mathsf{\Pi }}_C\left(\mathsf{\nabla }{f}^{*}(\mathsf{\nabla }f\left(a_1\right)-{\lambda }_{n}A{b}_0)\right).\hfill \end{array}$$

Step 2: Given the current iterates $a_n,b_n$ and $b_{n-1},$ and ${\lambda }_n$, calculate $a_{n+1}$ and $b_{n+1}$ as follows:

$$\begin{array}{ccc}\hfill a_{n+1}& =& {\mathsf{\Pi }}_{T_n}\left(\mathsf{\nabla }{f}^{*}(\mathsf{\nabla }f\left(a_n\right)-{\lambda }_{n}A{b}_n)\right),\hfill \\ \hfill b_{n+1}& =& {\mathsf{\Pi }}_C\left(\mathsf{\nabla }{f}^{*}(\mathsf{\nabla }f\left(a_{n+1}\right)-{\lambda }_{n+1}A{b}_n)\right),\hfill \end{array}$$

(35)

where

$${\lambda }_{n+1}=\{\begin{array}{cc}min\{{\lambda }_n,\frac{\alpha \parallel b_n-b_{n-1}\parallel }{\parallel A{b}_n-A{b}_{n-1}\parallel }\},\hfill & A{b}_n=A{b}_{n-1},\hfill \\ {\lambda }_n,\hfill & \mathrm{otherwise},\hfill \end{array}$$

(36)

and

$$T_n:=\{z\in H:\langle \mathsf{\nabla }f\left(a_n\right)-{\lambda }_{n}A{b}_{n-1}-\mathsf{\nabla }f\left(b_n\right),z-b_n\rangle \le 0\}.$$

If $a_{n+1}=a_n$ and $b_n=b_{n-1}$ or $A{b}_n=0:$ STOP. Otherwise, set $n\leftarrow n+1$ and repeat Step 2.

Remark 5.

Note that in Algorithm BPMSAS, the stepsize ${\lambda }_n$ is chosen by a self-adaptive process. This means that the stepsize is improved along with the iterations. Moreso, the sequence $\left\{{\lambda }_n\right\}$ is monotonically non-increasing and bounded below by $min\{{\lambda }_0,\frac{\alpha }{L}\}.$ Then, we can say that the limit ${lim}_{n\to \infty }{\lambda }_n$ exist, which we denote by λ, i.e., ${lim}_{n\to \infty }{\lambda }_n=\lambda >0.$

We now give the convergence of Algorithm BPMSAS.

Theorem 2.

Suppose that Assumption A1 holds, and let $\alpha \in \left(0,(\sqrt{2}-1)\sigma \right).$ Then, the sequences $\left\{a_n\right\}$ and $\left\{b_n\right\}$ generated by Algorithm BPMSAS converges weakly to a solution of the VI (1).

Proof. 

Let $z\in VI(C,A).$ Following similar argument as in the proof of (24), we obtain

$$\begin{array}{c}\hfill {\mathsf{\Delta }}_f(z,a_{n+1})\le {\mathsf{\Delta }}_f(z,a_n)-{\mathsf{\Delta }}_f(a_{n+1},b_n)-{\mathsf{\Delta }}_f(b_n,a_n)+{\lambda }_n\langle A{b}_n-A{b}_{n-1},b_n-a_{n+1}\rangle .\end{array}$$

Now, using Cauchy-Schwartz inequality and the defintion of ${\lambda }_{n+1}$ in (36), we get

$$\begin{array}{ccc}\hfill {\mathsf{\Delta }}_f(z,a_{n+1})& \le & {\mathsf{\Delta }}_f(z,a_n)-{\mathsf{\Delta }}_f(a_{n+1},b_n)-{\mathsf{\Delta }}_f(b_n,a_n)\hfill \\ & & +\frac{{\lambda }_n}{{\lambda }_{n+1}}{\lambda }_{n+1}\parallel A{b}_n-A{b}_{n-1}\parallel \parallel b_n-a_{n+1}\parallel \hfill \\ & \le & {\mathsf{\Delta }}_f(z,a_n)-{\mathsf{\Delta }}_f(a_{n+1},b_n)-{\mathsf{\Delta }}_f(b_n,a_n)\hfill \end{array}$$

$$\begin{array}{ccc}& & +\frac{{\lambda }_n}{2{\lambda }_{n+1}}\alpha \left(\frac{1}{\sqrt{2}}\parallel b_{n-1}-b_n{\parallel }^2+\sqrt{2}{\parallel b_n-a_{n+1}\parallel }^2\right)\hfill \\ & & \le {\mathsf{\Delta }}_f(z,a_n)-{\mathsf{\Delta }}_f(a_{n+1},b_n)-{\mathsf{\Delta }}_f(b_n,a_n)\hfill \end{array}$$

(37)

$$\begin{array}{ccc}& & +\frac{{\lambda }_n\alpha }{2\sqrt{2}{\lambda }_{n+1}}\left(\sqrt{2}\parallel b_{n-1}-a_n{\parallel }^2+(2+\sqrt{2}\parallel a_n-b_n{\parallel }^2)\right)+\frac{{\lambda }_n\alpha }{2{\lambda }_{n+1}}\sqrt{2}{\parallel b_n-a_{n+1}\parallel }^2\hfill \\ & & ={\mathsf{\Delta }}_f(z,a_n)-{\mathsf{\Delta }}_f(a_{n+1},b_n)-{\mathsf{\Delta }}_f(b_n,a_n)\hfill \\ & & +\frac{{\lambda }_n\alpha }{2{\lambda }_{n+1}}\parallel b_{n-1}-a_n{\parallel }^2+\frac{(\sqrt{2}+1){\lambda }_n\alpha }{2{\lambda }_{n+1}}\parallel a_n-b_n{\parallel }^2+\frac{{\lambda }_n\alpha \sqrt{2}}{2{\lambda }_{n+1}}{\parallel b_n-a_{n+1}\parallel }^2,\hfill \end{array}$$

(38)

where in (37) and (38), we have used the identities: $2ab\le \frac{1}{\sqrt{2}}{a}^2+\sqrt{2}{b}^2$ and ${(a+b)}^2\le \sqrt{2}{a}^2+(2+\sqrt{2})b^2$ respectively. Since f is strongly convex and $\sigma >0$, it follows from (8) that

$$\begin{array}{ccc}\hfill {\mathsf{\Delta }}_f(z,a_{n+1})& \le & {\mathsf{\Delta }}_f(z,a_n)-{\mathsf{\Delta }}_f(a_{n+1},b_n)-{\mathsf{\Delta }}_f(b_n,a_n)\hfill \\ & & +\frac{{\lambda }_n\alpha }{\sigma {\lambda }_{n+1}}{\mathsf{\Delta }}_f(a_n,b_{n-1})+\frac{(\sqrt{2}+1){\lambda }_n\alpha }{\sigma {\lambda }_{n+1}}{\mathsf{\Delta }}_f(b_n,a_n)+\frac{{\lambda }_n\alpha \sqrt{2}}{\sigma {\lambda }_{n+1}}{\mathsf{\Delta }}_f(a_{n+1},b_n)\hfill \\ & =& {\mathsf{\Delta }}_f(z,a_n)-\left(1-\frac{{\lambda }_n\alpha \sqrt{2}}{\sigma {\lambda }_{n+1}}\right){\mathsf{\Delta }}_f(a_{n+1},b_n)-\left(1-\frac{(\sqrt{2}+1){\lambda }_n\alpha }{\sigma {\lambda }_{n+1}}\right){\mathsf{\Delta }}_f(b_n,a_n)\hfill \\ & & +\frac{{\lambda }_n\alpha }{\sigma {\lambda }_{n+1}}{\mathsf{\Delta }}_f(a_n,b_{n-1}).\hfill \end{array}$$

(39)

Now put

$${\mathsf{\Gamma }}_n={\mathsf{\Delta }}_f(z,a_n)+\frac{\alpha {\lambda }_n}{\sigma {\lambda }_{n+1}}{\mathsf{\Delta }}_f(a_n,b_{n-1}),$$

and

$${\mathsf{\Lambda }}_n=\left(1-(\sqrt{2}+1)\frac{\alpha {\lambda }_n}{\sigma {\lambda }_{n+1}}\right)({\mathsf{\Delta }}_f(b_n,a_n)+{\mathsf{\Delta }}_f(a_{n+1},b_n)).$$

Then (39) can be rewritten as

$$a_{n+1}\le {\mathsf{\Gamma }}_n-{\mathsf{\Lambda }}_n-{\mathsf{\Omega }}_n,$$

where ${\mathsf{\Omega }}_n=\frac{\alpha }{\sigma }\left(\frac{{\lambda }_n}{{\lambda }_{n+1}}+\frac{{\lambda }_{n+1}}{{\lambda }_{n+2}}\right){\mathsf{\Delta }}_f(a_{n+1},b_n).$ It follows from Lemma 2 that $\left\{a_n\right\}$ is bounded. Note that since ${lim}_{n\to \infty }{\lambda }_n=\lambda >0$ and $\alpha \in (0,(\sqrt{2}-1)\sigma ),$ there exists a number $n_0\ge 0$ such that

$$\underset{n\to \infty }{lim}\left(1-\frac{{\lambda }_n\alpha \sqrt{2}}{\sigma {\lambda }_{n+1}}\right)=1-\frac{\alpha \sqrt{2}}{\sigma }>0,$$

$$\underset{n\to \infty }{lim}\left(1-\frac{{\lambda }_n\alpha (\sqrt{2}+1)}{\sigma {\lambda }_{n+1}}\right)=1-\frac{(\sqrt{2}+1)\alpha }{\sigma }>0.$$

This implies that there exists $ϵ>0$ such that

$$(1-\frac{{\lambda }_n\alpha (\sqrt{2}+1)}{\sigma {\lambda }_{n+1}})\ge ϵ>0,\forall n\ge n_0.$$

Hence from Lemma 2, we obtain

$$\sum _{n=0}^{\infty }ϵ({\mathsf{\Delta }}_f(b_n,a_n)+{\mathsf{\Delta }}_f(a_{n+1},b_n))<\infty ,$$

which implies that

$$\underset{n\to \infty }{lim}{\mathsf{\Delta }}_f(b_n,a_n)=\underset{n\to \infty }{lim}{\mathsf{\Delta }}_f(a_{n+1},b_n)=0.$$

Consequently

$$\underset{n\to \infty }{lim}\parallel a_n-b_n\parallel =\underset{n\to \infty }{lim}\parallel a_{n+1}-b_n\parallel =0.$$

Hence

$$\underset{n\to \infty }{lim}\parallel a_{n+1}-a_n\parallel =\underset{n\to \infty }{lim}\parallel b_{n+1}-b_n\parallel =\underset{n\to \infty }{lim}\parallel a_{n+1}-b_{n+1}\parallel =0.$$

Since f is uniformly Fréchet differentiable, then $\mathsf{\nabla }f$ is norm-to-norm continuous and therefore

$$\underset{n\to \infty }{lim}\parallel \mathsf{\nabla }f\left(a_{n+1}\right)-\mathsf{\nabla }f\left(b_{n+1}\right)\parallel =0.$$

Since $\left\{a_n\right\}$ is bounded, then there exists a subsequence $\left\{a_{n_k}\right\}$ of $\left\{a_n\right\}$ such that $a_{n_k}\rightharpoonup \overline{x}.$ Following similar argument from (27) of Theorem 1, we get that $\overline{x}\in VI(C,A).$ This completes the proof. □

4. Numerical Examples

In this section, we test the performance of Algorithm BPMFS and Algorithm BPMSAS for solving some VI available in the literature. All numerical computation are carried out using MATLAB program on a PC (with Intel(R) Core (TM) i7-5600U CPU @ 3.40GHz, RAM 8.00GB) in MATLAB 9.7 2019(a).

Example 2.

First, we consider the variational inequality problem for a finite dimensional space ${\mathbb{R}}^m.$ The cost operator $A:{\mathbb{R}}^m\to {\mathbb{R}}^m$ is defined by $Ax=Gx+q$, such that

$$G=S{S}^T+Q+R,$$

where S is a $m\times m$ matrix, Q is a $m\times m$ skew-symmetric matrix, R is a $m\times m$ diagonal matrix, whose diagonal entries are positive (so, G is positive definite) and q is a vector in ${\mathbb{R}}^m.$ The feasible set C is the closed convex polyhedron which is defined by $C=\{x\in {\mathbb{R}}^m:Bx\le b\}$, where B is a $l\times m$ ($l=10$) matrix and b is a non-negative vector in ${\mathbb{R}}^m.$ It is easy to see that A is monotone (thus, pseudomonotone) and L-Lipschitz continuous with $L=\parallel G\parallel .$ For experimental purpose, all the entries of $S,Q,R,B$ and b are generated randomly such that their properties are satisfied and the starting points $a_0,b_0$ are generated randomly in ${\mathbb{R}}^m$ ($m=50,100,150,200$). The Bregman projection onto the feasible set C is easily computed using thefminconsolver in MATLAB Optimization toolbox. We set $q=0\in {\mathbb{R}}^m,$ in this case, we know that the unique solution of the VI is $\left\{0\right\}.$ We choose $f\left(x\right)=\frac{1}{2}{\parallel x\parallel }^2$ which is strongly convex with modulus 1. We test Algorithm BPMFS (named, Alg. 1), Algorithm BPMSAS (named, Alg. 2) and theSEM(Algorithm (3)) using the following parameters: for Algorithm BPMFS andSEM, we choose $\lambda =0.3L$ while for Algorithm BPMSAS, we choose ${\lambda }_0=0.3.$ The computation is stopped when $D_n=\parallel a_n-0\parallel <{10}^{-7}$ is satisfied. We plot the graphs of $D_n$ against the number of iteration in each case for each iteration. The numerical results are shown in

Table 1. Computation result for Example 2.

		Alg. 1	Alg. 2	SEM
$m=50$	Iter.	29	24	41
	CPU time (sec)	0.0050	0.0022	0.0083
$m=100$	Iter.	32	24	44
	CPU time (sec)	0.0165	0.0101	0.0203
$m=150$	Iter	35	24	44
	CPU time (sec)	0.0161	0.0075	0.0211
$m=200$	Iter.	37	24	45
	CPU time (sec)	0.0321	0.0146	0.0385

and Figure 1. The numerical results shows that Algorithm BPMSAS has computational advantage than Algorithm BPMFS and the SEM in terms of number of iteration and CPU time taken for computation. This is due to the fact that the stepsize is improve in each iteration compare to the Algorithm BPMFS and SEM which used fixed stepsize. More so, Algorithm BPMFS performs better than theSEM.

Example 3.

Here, we compare the performance of Algorithm BPMFS and Algorithm BPMSAS for different convex functions. Let C be defined by

$$C=\left\{x={(a_1,\dots ,a_m)}^T:\parallel x\parallel \le 1\mathrm{and}{a}_i>0,i=1,\dots ,m\right\}.$$

Note that the corresponding values of $\mathsf{\nabla }f\left(x\right)$ and $\mathsf{\nabla }{f}^{*}\left(x\right)=\mathsf{\nabla }{f}^{-1}\left(x\right)$ for the convex function given in Example 1 are respectively:

(i) 

$\mathsf{\nabla }f\left(x\right)=x$ and $\mathsf{\nabla }{f}^{*}\left(x\right)=x$ for SE,

(ii) 

$\mathsf{\nabla }f\left(x\right)={\left(1+log\left(a_1\right),\dots ,1+log\left(a_m\right)\right)}^T$ and $\mathsf{\nabla }{f}^{*}\left(x\right)={\left(exp(a_1-1),\dots ,exp(a_m-1)\right)}^T$ for KL,

(iii) 

$\mathsf{\nabla }f\left(x\right)=-{\left(\frac{1}{x},\dots ,\frac{1}{a_m}\right)}^T$ and $\mathsf{\nabla }{f}^{*}\left(x\right)=-{\left(\frac{1}{x},\dots ,\frac{1}{a_m}\right)}^T$ for IS,

(iv) 

$\mathsf{\nabla }f\left(x\right)=Qx$ and $\mathsf{\nabla }{f}^{*}\left(x\right)=Q^{-1}\left(x\right)$ for MD.

We test the algorithms using similar parameters as in Example 1 with $\parallel a_n-0\parallel >{10}^{-4}$ as stopping criterion. The numerical results are shown in

Table 2. Example 3 Numerical result showing number of iteration and CPU time (sec) for various types of function f using Algorithm BPMFS and Algorithm BPMSAS.

		$\mathit{m}=50$		$\mathit{m}=100$		$\mathit{M}=150$		$\mathit{m}=200$
		Iter.	CPU	Iter.	CPU	Iter.	CPU	Iter.	CPU
SE	Alg. 1	23	0.0049	26	0.0099	18	0.0153	18	0.0163
	Alg. 2	15	0.0021	16	0.0039	16	0.0109	16	0.0138
KL	Alg. 1	15	0.0044	17	0.0081	18	0.0150	18	0.00160
	Alg 2.	15	0.0040	15	0.0058	15	0.00109	15	0.0143
IS	Alg. 1	14	0.0048	14	0.0104	14	0.0133	12	0.0183
	Alg. 2	7	0.0028	5	0.0025	4	0.0041	3	0.0057
MD	Alg. 1	24	0.0104	27	0.0419	28	0.0960	30	0.1910
	Alg. 2	12	0.0094	12	0.0254	12	0.0423	15	0.0698

. The computation result shows that the IS has the best performance while the MD has the poorest performance for both Algorithm BPMFS and Algorithm BPMSAS.

Example 4.

Next, we apply our algorithms to solve the Least Absolute Selection and Shrinkage Operator (LASSO) problem [42]. Let z be a vector in ${\mathbb{R}}^M$, B be a matrix of order $M\times N$ ($M<<N)$ and $\lambda >0.$ The LASSO problem is given by

$$\begin{array}{c}\hfill \underset{x\in {\mathbb{R}}^N}{min}\left\{\frac{1}{2}{\parallel Bx-z\parallel }_2^2+\lambda {\parallel x\parallel }_1\right\},\end{array}$$

(40)

where ${\parallel x\parallel }_2$ is the Euclidean norm of x and ${\parallel x\parallel }_1={\displaystyle \sum _{i=1}^N}\left|a_i\right|$ is the $l_1$-norm of x. This problem has been considered as a great tool in several branches of science and engineering. Clearly, (40) is a convex unconstrained minimization problem which appears in compress sensing and image reconstruction, where the original signal (or image) is sparse in some orthogonal basis by the process

$$z=Bx+\xi ,$$

where x is the original signal (or image), B is the blurring operator, ξ is a noise and z is the degraded or blurred data which needs to be recovered. Several iterative methods for solving (40) have been introduced with the earliest being the projection method by Figureido et al. [43]. Equivalently, the LASSO problem (40) can be expressed as a VI by setting $A=B^T(Bx-z)$ (see, [44]). We apply our algorithms to solve the signal problem with $N=4096$ which is generated by the uniform distribution on the interval $[-2,2]$ with 160 non-zero elements. The matrix B is generated by the normal distribution with mean zero and variance one while the observation z with $M=512$ is generated by Gaussian noise with SNR = 40. The initial points $a_0,y_0$ are picked randomly and in Algorithm 1, we choose $\lambda =0.35,$ in Algorithm 2, ${\lambda }_0=0.02$ and in SEM, $\lambda =0.35.$ We compare the performance of each algorithm with respect to the numer of iterations, CPU time and MSE values, see Figure 2 and Figure 3.

5. Conclusions

In this paper, we introduce two Bregman subgradient extragradient method for solving variational inequalities with pseudomonotone and Lipschitz continuous operator in a real Hilbert space. We established some weak convergence results for the sequence generated by the algorithms. We also gives some numerical experiments to illustrate the performance of the proposed algorithms. The numerical results show that the proposed methods perform better than the usual subgradient extragradient method and their efficiency depends on the type of convex function used in the algorithms.