Relaxed Variable Metric Primal-Dual Fixed-Point Algorithm with Applications

Abstract

In this paper, a relaxed variable metric primal-dual fixed-point algorithm is proposed for solving the convex optimization problem involving the sum of two convex functions where one is differentiable with the Lipschitz continuous gradient while the other is composed of a linear operator. Based on the preconditioned forward–backward splitting algorithm, the convergence of the proposed algorithm is proved. At the same time, we show that some existing algorithms are special cases of the proposed algorithm. Furthermore, the ergodic convergence and linear convergence rates of the proposed algorithm are established under relaxed parameters. Numerical experiments on the image deblurring problems demonstrate that the proposed algorithm outperforms some existing algorithms in terms of the number of iterations.

1. Introduction

In this paper, we focus on the following convex optimization problem:

$$\underset{x\in H}{min}f\left(x\right)+h\left(Lx\right),$$

(1)

where $f:H\to R$ is convex differentiable and its gradient $\nabla f$ is $\frac{1}{\beta }-$ Lipschitz-continuous for some $\beta >0$, $h:G\to (-\infty ,+\infty ]$ is a proper lower semi-continuous convex function, $L:H\to G$ is a bounded linear operator, and H and G are real Hilbert spaces. This problem is widely used in signal and image processing [1,2], compressed sensing [3], and machine learning [4]. For instance, a classical model in image restoration and medical image reconstruction is:

$$\underset{x\in R^n}{min}\frac{1}{2}{\parallel Ax-b\parallel }^2+\mu {\parallel x\parallel }_{TV},$$

(2)

where $A:R^n\to R^m$ is a blurring operator, $b\in R^m$ is the observed image, $\mu >0$ is the regularization parameter, and ${\parallel x\parallel }_{TV}$ is the total variation, which can be represented by a composition of a convex function with a discrete gradient operator.

The corresponding dual problem of (1) is

$$\underset{v\in G}{max}-f^{*}(-L^{*}v)-h^{*}\left(v\right),$$

(3)

and the associated saddle point problem of (1) and (3) is

$$\underset{x\in H}{min}\underset{v\in G}{max}\{K(v,x)=f\left(x\right)+⟨Lx,v⟩-h^{*}\left(v\right)\}.$$

(4)

We say that $(x^{*},v^{*})$ is a saddle point of (4) if and only if $x^{*}$ is a solution of (1) and $v^{*}$ is a solution of (3), respectively. The optimal solution set of problem (4) is denoted by $\mathsf{\Omega }$. In this paper, we always assume that $\mathsf{\Omega }$ is nonempty.

Many efficient algorithms have been proposed for solving problem (1) in the last decades. Most of them are based on the alternating direction method of multipliers (ADMM) [5,6,7] and the forward–backward splitting (FBS) algorithm [8]. In [9,10], the authors showed that ADMM is equivalent to the Douglas–Rachford splitting algorithm [8]. The proximal gradient algorithm (PGA, also known as FBS) [11] is an efficient algorithm to solve (1) if $L=I$, and some accelerated versions of PGA had been studied [12,13,14]. The primal dual hybrid gradient algorithm [15,16] was proposed to solve (1) without the smoothness of f. Combining the FBS algorithm [11] with the fixed-point algorithm based on the proximity operator (FP${}^2$O) [17], Argyriou et al. [18] proposed a FBS_FP${}^2$O to solve (1). Note that the FBS_FP${}^2$O algorithm needs to solve a subproblem. Thus, it involves inner and outer iterations. To avoid choosing the number of inner iterations, Chen et al. [19] first a proposed the primal-dual fixed-point algorithm based on the proximity operator (PDFP${}^2$O) to solve (1). Compared with the FBS_FP${}^2$O, PDFP${}^2$O only performs one inner iteration and reduces to the generalized iterative soft-thresholding algorithm [20] when $f={\parallel Ax-b\parallel }^2$. The PDFP${}^2$O provided desirable performances to solve MRI reconstruction and TV-L1 wavelet inpainting [21,22]. In contrast, Combettes et al. [23] proposed a variable metric forward–backward splitting (VMFBS) algorithm to solve the saddle-point problem (4). By choosing a special variable metric, the PDFP${}^2$O could be recovered by the VMFBS algorithm. Moreover, the proximal alternating predictor-corrector (PAPC) algorithm [24] was proposed to solve the equivalent minimization problem of (1) and was proved to converge linearly [25]. To speed up the PDFP${}^2$O, Chen et al. [26] proposed an adapted metric version of PDFP${}^2$O, which is termed as PDFP${}^2$O_AM. The key feature of the PDFP${}^2$O_AM is that it uses a symmetric positive matrix to replace the stepsize of the PDFP${}^2$O. In contrast, Wen et al. [27] generalized the stepsize in the PDFP${}^2$O to the dynamic stepsize. Later, a larger stepsize of PDFP${}^2$O was proved [28]. Recently, Zhu and Zhang [29] introduced an inertial PDFP${}^2$O (IPDFP${}^2$O).

In Table 1, we summarize some variants of PDFP${}^2$O.

Table 1. Listing of existing primal-dual fixed point type algorithms.

Algorithm	${\mathit{\rho }}_{\mathit{k}}$	$\mathit{\lambda }$	$\mathit{\gamma }$	Variable Metric	Ergodic Rate	Linear Rate
PDFP${}^2$O [19]	$(0,1]$	$(0,\frac{1}{{\lambda }_{max}\left(L{L}^{*}\right)}]$	$(0,2\beta )$			√
VMPD [23]	$(0,1]$	$(0,\frac{1}{{\lambda }_{max}\left(L{Q}_k^{-1}{L}^{*}\right)})$	$(0,2\beta )$	√
PAPC [24]	1	$(0,\frac{1}{{\lambda }_{max}\left(L{L}^{*}\right)}]$	$(0,\beta ]$		√	√
PDFP${}^2$O_AM [26]	$(0,1]$	$(0,\frac{1}{{\lambda }_{max}\left(L{Q}^{-1}{L}^{*}\right)})$	$(0,2\beta )$
PDFP${}^2$O_DS [27]	$(0,1)$	$(0,\frac{1}{{\lambda }_{max}\left(L{L}^{*}\right)})$	$(0,2\beta )$	√		√
IPDFP${}^2$O [29]	1	$(0,\frac{1}{{\lambda }_{max}\left(L{L}^{*}\right)})$	$(0,2\beta )$

From Table 1, we note that the relaxation parameter of these algorithms belongs to $(0,1]$. It is well known that the convergence speed of the iterative algorithm can be accelerated when the relaxation parameter is greater than 1. This allows us to accelerate the PDFP${}^2$O_AM with larger relaxed parameters. We reformulate the PDFP${}^2$O_AM as the FBS algorithm and propose a primal dual fixed-point algorithm based on the proximity operator with relaxed parameters and variable metrics (Rv_PDFP${}^2$O). Based on the fixed point theory, we prove the convergence of the proposed algorithm. At the same time, we point out that PDFP${}^2$O_AM [26], PDFP${}^2$O_DS [27], and PDFP${}^2$O [19] are particular cases of Rv_PDFP${}^2$O. Further, the convergence rates are established under the larger relaxed parameters, including ergodic and linear convergence. To verify the effectiveness and superiority of Rv_PDFP${}^2$O, we apply it for solving the image-restoration problem and compare it with other algorithms.

The rest of the paper is organized as follows. In Section 2, we recall some preliminaries and related work. In Section 3, we deduce the Rv_PDFP${}^2$O from the preconditioned FBS algorithm and provide some convergence results. In Section 4, we show the numerical results of Rv_PDFP${}^2$O on solving image deblurring problem. Finally, we provide the conclusions.

2. Preliminaries and Related Work

In this section, we first provide some notations and definitions. Then, we briefly review some existing algorithms for solving (1).

Throughout this paper, H denotes a real Hilbert space endowed with scalar product $⟨·,·⟩$, and the associated norm is $\parallel ·\parallel$. Let $S_{++}^H$ ($S_{+}^H$) denote the set of the symmetric positive definite (semi-definite) operator in H. For $U\in S_{++}^H$, the U-weighted inner product is ${⟨x,y⟩}_U=⟨x,Uy⟩$ and the corresponding U-weighted norm is defined by ${\parallel x\parallel }_U=\sqrt{{⟨x,x⟩}_U}$. $H_1$ and $H_2$ are the real Hilbert space endowed with the scalar product. Let $U_1\in S_{++}^{H_1}$ and $U_2\in S_{++}^{H_2}$, the $U_1,U_2$-weighted norm in $H_1\times H_2$ is ${\parallel x\parallel }_{U_1,U_2}=\sqrt{\parallel x_1{\parallel }_{U_1}^2+{\parallel x_2\parallel }_{U_2}^2}$ for $x=(x_1,x_2)\in H_1\times H_2$. We denote by ${\mathsf{\Gamma }}_0\left(H\right)$ the class of all proper, lower semi-continuous, convex functions from H to $(-\infty ,+\infty ]$. Most of these definitions can be found in [8].

Let $A:H\to 2^H$ be a set-valued operator. The domain, the graph, the zeros, and the inverse of A are represented by $domA=\{x\in H:Ax\ne \varnothing \}$, $graA=\left\{\right(x,u)\in H\times H:u\in Ax\}$ and $zerA=\{x\in H:0\in Ax\}$, $A^{-1}:H\mapsto 2^H:u\mapsto \{x\in H:u\in Ax\}$, and the resolvent of A is

$$J_A={(I+A)}^{-1}.$$

The operator $A:H\to 2^H$ is monotone, if $⟨u-v,x-y⟩\ge 0$ for all $(x,u)\in graA,(y,v)\in graA$. The monotone operator A is maximally monotone if there is no monotone operator B such that $graA\subseteq graB\ne graA$. Further, A is $\delta$- monotone if $⟨u-v,x-y⟩\ge {\delta \parallel x-y\parallel }^2$ for $\delta >0$, $\forall (x,u),(y,v)\in graA$. An operator $B:H\to H$ is $\beta$-cocoercive, for some $\beta >0$, if $⟨x-y,Bx-By⟩\ge {\beta \parallel Bx-By\parallel }^2$, $\forall x,y\in H$.

Let $D\subseteq H$ be nonempty and let $T:D\to H$. The fixed point set of T is denoted by $FixT$, i.e., $FixT=\{x\in D:Tx=x\}$. T is $\alpha$-averaged, for some $\alpha \in (0,1)$, if

$${\parallel Tx-Ty\parallel }^2+\frac{1-\alpha }{\alpha }{\parallel (I-T)x-(I-T)y\parallel }^2\le {\parallel x-y\parallel }^2,\forall x,y\in D.$$

Let $f\in {\mathsf{\Gamma }}_0\left(H\right)$, the Fenchel conjugate of f is

$$f^{*}\left(u\right)=\underset{x\in H}{sup}\{⟨x,u⟩-f\left(x\right)\},$$

and the subdifferential of f is the maximally monotone operator

$$\partial f:x\mapsto \{u\in H:⟨y-x,u⟩+f(x)\le f(y),\forall y\in H\}.$$

Further, $\partial f\left(x\right)=\{\nabla f(x\left)\right\}$ when f is differentiable.

Let $f\in {\mathsf{\Gamma }}_0\left(H\right)$ and $U\in S_{++}^H$, the scale proximity operator of f with respect to the metric U is

$$pro{x}_f^U\left(x\right)=arg\underset{u\in H}{min}\left\{\frac{1}{2}{\parallel u-x\parallel }_U^2+f\left(u\right)\right\}.$$

(5)

The scale proximity operator is the standard proximity operator when $U=I$.

Related Work

To solve (1), Argyriou et al. [18] considered the following forward–backward splitting algorithm:

$$x^{k+1}=pro{x}_{\gamma (h\circ L)}(x^k-\gamma \nabla f\left(x^k\right)),$$

(6)

where $\gamma \in (0,2\beta )$. By the definition of proximity operator, (6) is equivalent to

$$x^{k+1}=arg\underset{x}{min}\left\{\frac{1}{2}{\parallel x-(x^k-\gamma \nabla f\left(x^k\right))\parallel }^2+\gamma h\left(Lx\right)\right\}.$$

(7)

Then, Argyriou et al. [18] employed the FBS_FP${}^2$O algorithm to solve (7):

$$\left\{\begin{array}{cc}\hfill v^{k+1}& =arg\underset{v}{min}\left\{\frac{1}{2}{\parallel L^{*}v-\frac{1}{\gamma }(x^k-\gamma \nabla f\left(x^k\right))\parallel }^2+\frac{1}{\gamma }{h}^{*}\left(v\right)\right\},\hfill \\ \hfill x^{k+1}& =x^k-\gamma \nabla f\left(x^k\right)-\gamma L^{*}{v}^{k+1}.\hfill \end{array}\right.$$

(8)

In (8), one needs to solve the subproblem of v to obtain the update of $\left\{v^{k+1}\right\}$. More precisely, we obtain the following inner–outer iterative algorithm:

$$\left\{\begin{array}{cc}\hfill v^{k+1,j}& =pro{x}_{\frac{\lambda }{\gamma }{h}^{*}}(v^{k,j}-\lambda L(L^{*}{v}^{k,j}-\frac{1}{\gamma }(x^k-\gamma \nabla f(x^k)))),\hfill \\ \hfill x^{k+1}& =x^k-\gamma \nabla f\left(x^k\right)-\gamma L^{*}{v}^{k+1,J},\hfill \end{array}\right.$$

(9)

where $\lambda \in (0,\frac{2}{{\lambda }_{max}(L{L}^{*})})$, j denotes the inner iteration, and J represents the maximum number of inner iteration. Here, ${\lambda }_{max}\left(L\right)$ denotes the largest eigenvalue of L when L is a matrix.

Chen et al. [19] proposed the PDFP${}^2$O as follows:

$$\left\{\begin{array}{cc}\hfill {\tilde{v}}^{k+1}& =(I-pro{x}_{\frac{\gamma }{\lambda }h})(L(x^k-\gamma \nabla f\left(x^k\right))+(I-\lambda L{L}^{*})v^k),\hfill \\ \hfill {\tilde{x}}^{k+1}& =x^k-\gamma \nabla f\left(x^k\right)-\lambda L^{*}{\tilde{v}}^{k+1},\hfill \\ \hfill (v^{k+1}& ,x^{k+1})=(1-{\rho }_k)(v^k,x^k)+{\rho }_k({\tilde{v}}^{k+1},{\tilde{x}}^{k+1}).\hfill \end{array}\right.$$

(10)

where $\lambda \in (0,\frac{1}{{\parallel L\parallel }^2}]$, $\gamma \in (0,2\beta )$, and ${\rho }_k\in (0,1]$. Let ${\rho }_k=1$, and with the help of the Moreau decomposition, we obtain from (10) that

$$\left\{\begin{array}{cc}\hfill v^{k+1}& =\frac{\gamma }{\lambda }pro{x}_{\frac{\lambda }{\gamma }{h}^{*}}(\frac{\lambda }{\gamma }L(x^k-\gamma \nabla f\left(x^k\right))+\frac{\lambda }{\gamma }(I-\lambda L{L}^{*})v^k),\hfill \\ \hfill x^{k+1}& =x^k-\gamma \nabla f\left(x^k\right)-\lambda L^{*}{v}^{k+1}.\hfill \end{array}\right.$$

(11)

Let ${\overline{v}}^k=\frac{\lambda }{\gamma }{v}^k$, we have

$$\left\{\begin{array}{cc}\hfill {\overline{v}}^{k+1}& =pro{x}_{\frac{\lambda }{\gamma }{h}^{*}}(\frac{\lambda }{\gamma }L(x^k-\gamma \nabla f\left(x^k\right))+(I-\lambda L{L}^{*})v^k),\hfill \\ \hfill x^{k+1}& =x^k-\gamma \nabla f\left(x^k\right)-\gamma L^{*}{\overline{v}}^{k+1}.\hfill \end{array}\right.$$

(12)

Compared with FBS_FP${}^2$O (9), PDFP${}^2$O (12) performs only one inner iteration to calculate $v^{k+1}\left({\overline{v}}^{k+1}\right)$. On the other hand, if we add a proximal term $\frac{1}{2}{\parallel v-v^k\parallel }_{\frac{1}{\lambda }I-L{L}^{*}}^2$ to the subproblem of v in (8), i.e.,

$$v^{k+1}=arg\underset{v}{min}\left\{\frac{1}{2}\parallel L^{*}v-\frac{1}{\gamma }(x^k-\gamma \nabla f\left(x^k\right)){\parallel }^2+\frac{1}{\gamma }{h}^{*}\left(v\right)+\frac{1}{2}{\parallel v-v^k\parallel }_{\frac{1}{\lambda }I-L{L}^{*}}^2\right\},$$

after simple calculation, we could also recover the PDFP${}^2$O (12).

3. Relaxed Variable Metric Primal-Dual Fixed-Point Algorithm Based on Proximity Operator

In this section, we propose Rv_PDFP${}^2$O for solving the minimization problem (1). The Rv_PDFP${}^2$O is

$$\left\{\begin{array}{cc}\hfill {\tilde{v}}^{k+1}& =pro{x}_{h^{*}}^{P_k}((I-P_k^{-1}L{Q}_k^{-1}{L}^{*})v^k+P_k^{-1}L(x^k-Q_k^{-1}\nabla f\left(x^k\right))),\hfill \\ \hfill {\tilde{x}}^{k+1}& =x^k-Q_k^{-1}\nabla f\left(x^k\right)-Q_k^{-1}{L}^{*}{\tilde{v}}^{k+1},\hfill \\ \hfill (v^{k+1}& ,x^{k+1})=(1-{\rho }_k)(v^k,x^k)+{\rho }_k({\tilde{v}}^{k+1},{\tilde{x}}^{k+1}).\hfill \end{array}\right.$$

(13)

3.1. Convergence Analysis

First, let us introduce the product space $K=G\times H$ and define the operators:

$$A:K\to 2^K(v,x)\mapsto (\partial h^{*}\left(v\right)-Lx)\times \left(L^{*}v\right),$$

and

$$B:K\to K(v,x)\mapsto (0,\nabla f(x\left)\right).$$

Notice that $u^{*}=(v^{*},x^{*})\in \mathsf{\Omega }$ if and only if $0\in A{u}^{*}+B{u}^{*}$. Although A is maximally monotone and B is cocoercive in K, the forward–backward splitting algorithm could not be applicable since ${(I+\tau A)}^{-1}$, $\tau >0$ does not have a closed-form solution. To overcome this difficulty, we consider a preconditioned forward–backward splitting algorithm as follows:

$$u^{k+1}=(1-{\rho }_k)u^k+{\rho }_k{J}_{U_{k}A}(u^k-U_{k}B{u}^k),$$

(14)

where $P_k\in S_{++}^G$, $Q_k\in S_{++}^H$, ${\rho }_k>0$ and

$$U_k^{-1}=\left(\begin{array}{cc}{P}_k-L{Q}_k^{-1}{L}^{*}& 0\\ 0& Q_k\end{array}\right).$$

(15)

After simple calculation, we recover (13) from (14). In order to analyze the theoretical convergence of Rv_PDFP${}^2$O, we make the following assumptions:

(A1): 

$\parallel P_k^{-1}{\parallel }_2\in (0,\frac{1}{{\lambda }_{max}\left(L{Q}_k^{-1}{L}^{*}\right)})$, $\parallel Q_k^{-1}{\parallel }_2\in (0,2\beta )$;

(A2): 

${\rho }_k\in [\rho ,\frac{4\beta -\parallel Q_k^{-1}{\parallel }_2}{2\beta }-{\xi }_k]$, for $\rho ,{\xi }_k>0$;

(A3): 

$U_{k+1}-U_k\in S_{++}^K,\forall k\in \mathbb{N}$;

(A4): 

$\overline{\vartheta }={sup}_{k\in \mathbb{N}}{∥U_k∥}_2<+\infty$.

Under the assumption (A1), we have $U_k^{-1}\in S_{++}^K$. Denote $J_{U_{k}A}(I-U_{k}B)$ by ${\widehat{T}}_{\left(k\right)}$.

Lemma 1.

Suppose that (A1) holds. Then the following statements hold:

(1)

$I-U_{k}B$ is $\frac{\parallel Q_k^{-1}{\parallel }_2}{2\beta }$-averaged under ${\parallel ·\parallel }_{U_k^{-1}}$;

(2)

${\widehat{T}}_{\left(k\right)}$ is $\frac{2\beta }{4\beta -\parallel Q_k^{-1}{\parallel }_2}$-averaged under ${\parallel ·\parallel }_{U_k^{-1}}$.

Proof. 

(1) Let $u_1=(v_1,x_1),u_2=(v_2,x_2)\in K$; we have

$$\begin{array}{cc}\hfill & {⟨U_{k}B{u}_1-U_{k}B{u}_2,u_1-u_2⟩}_{U_k^{-1}}\hfill \\ \hfill =& ⟨\nabla f\left(x_1\right)-\nabla f\left(x_2\right),x_1-x_2⟩\hfill \\ \hfill \ge & \beta \parallel \nabla f\left(x_1\right)-\nabla f\left(x_2\right){\parallel }^2\hfill \\ \hfill \ge & \frac{\beta }{\parallel Q_k^{-1}{\parallel }_2}{\parallel Q_k^{-1}\nabla f\left(x_1\right)-Q_k^{-1}\nabla f\left(x_2\right)\parallel }_{Q_k}^2\hfill \\ \hfill =& \frac{\beta }{\parallel Q_k^{-1}{\parallel }_2}{\parallel U_{k}B{u}_1-U_{k}B{u}_2\parallel }_{U_k^{-1}}^2,\hfill \end{array}$$

which means that $U_{k}B$ is $\frac{\beta }{\parallel Q_k^{-1}{\parallel }_2}$-cocoercive. Hence, $I-U_{k}B$ is $\frac{\parallel Q_k^{-1}{\parallel }_2}{2\beta }$-averaged.

(2) Since $U_k\in S_{++}^K$, $U_{k}A$ is maximally monotone and $J_{U_{k}A}$ is $\frac{1}{2}$-averaged. Hence, $J_{U_{k}A}(I-U_{k}B)$ is $\frac{2\beta }{4\beta -\parallel Q_k^{-1}{\parallel }_2}$-averaged. □

Now, we are ready to present the main convergence theorem of Rv_PDFP${}^2$O (13).

Theorem 1.

Suppose that (A1)–(A4) hold. Let $\{u^k=(v^k,x^k)\}$ be generated by (13). Then, we have the following:

(1)

For any $u^{*}\in \mathsf{\Omega }$, $\{\parallel u^k-u^{*}{\parallel }_{U_k^{-1}}\}$ is monotonically decreasing and ${lim}_{k\to +\infty }{\parallel u^k-u^{*}\parallel }_{U_k^{-1}}$ exists;

(2)

${lim}_{k\to +\infty }\parallel u^k-{\widehat{T}}_{\left(k\right)}\left(u^k\right)\parallel =0$;

(3)

$\left\{u^k\right\}$ converges weakly to a point in Ω.

Proof. 

(1) Let ${\alpha }_k=\frac{2\beta }{4\beta -\parallel Q_k^{-1}{\parallel }_2}$. Notice that ${\parallel u\parallel }_{U_{k+1}^{-1}}\le {\parallel u\parallel }_{U_k^{-1}}$, $\forall u\in G\times H$. Then, we obtain

$$\begin{array}{cc}\hfill & \parallel u^{k+1}-u^{*}{\parallel }_{U_{k+1}^{-1}}^2\hfill \\ \hfill \le & \parallel u^{k+1}-u^{*}{\parallel }_{U_k^{-1}}^2\hfill \\ \hfill =& (1-{\rho }_k)\parallel u^k-u^{*}{\parallel }_{U_k^{-1}}^2+{\rho }_k\parallel {\widehat{T}}_{\left(k\right)}\left(u^k\right)-u^{*}{\parallel }_{U_k^{-1}}^2-{\rho }_k(1-{\rho }_k){\parallel u^k-{\widehat{T}}_{\left(k\right)}\left(u^k\right)\parallel }_{U_k^{-1}}^2\hfill \\ \hfill \le & \parallel u^k-u^{*}{\parallel }_{U_k^{-1}}^2-{\rho }_k(\frac{1}{{\alpha }_k}-{\rho }_k){\parallel u^k-{\widehat{T}}_{\left(k\right)}\left(u^k\right)\parallel }_{U_k^{-1}}^2,\hfill \end{array}$$

(16)

which implies that $\parallel u^k-u^{*}{\parallel }_{U_k^{-1}}$ is decreasing and ${lim}_{k\to +\infty }{\parallel u^k-u^{*}\parallel }_{U_k^{-1}}$ exists.

(2) Summing (16) from $k=0$ to $N-1$, we obtain

$$\sum _{k=0}^{N-1}{\rho }_k(\frac{1}{{\alpha }_k}-{\rho }_k)\parallel u^k-{\widehat{T}}_{\left(k\right)}\left(u^k\right){\parallel }_{U_k^{-1}}^2\le {\parallel u^0-u^{*}\parallel }_{U_0^{-1}}.$$

(17)

It follows from (17) that

$$\underset{k\to +\infty }{lim}\parallel u^k-{\widehat{T}}_{\left(k\right)}\left(u^k\right)\parallel =0.$$

(18)

(3) Let $\left\{u^{k_j}\right\}\subset \left\{u^k\right\}$ such that $u^{k_j}\rightharpoonup {\widehat{u}}^{*}$. It follows from Lemma 2.3 in [30] that there is $U^{-1}\in S_{++}^K$ such that $U_k^{-1}\to U^{-1}$. Define $T=J_{UA}(I-UB)$, we have

$$\begin{array}{cc}\hfill & \parallel u^{k_j}-T\left(u^{(k_j}\right))\parallel \hfill \\ \hfill \le & \parallel u^{k_j}-{\widehat{T}}_{\left(k_j\right)}\left(u^{k_j}\right)\parallel +\parallel {\widehat{T}}_{\left(k_j\right)}\left(u^{k_j}\right)-T\left(u^{k_j}\right)\parallel \hfill \\ \hfill \le & \parallel u^{k_j}-{\widehat{T}}_{\left(k_j\right)}\left(u^{k_j}\right)\parallel +\frac{1}{{\lambda }_{min}\left(U_{k_j}^{-1}\right)}\parallel (U_{k_j}^{-1}-U^{-1})(u^{k_j}-T\left(u^{k_j}\right)\parallel \hfill \\ \hfill \le & \parallel u^{k_j}-{\widehat{T}}_{\left(k_j\right)}\left(u^{k_j}\right)\parallel +\overline{\vartheta }\parallel (U_{k_j}^{-1}-U^{-1})(u^{k_j}-T\left(u^{k_j}\right)\parallel ,\hfill \end{array}$$

(19)

which implies that ${lim}_{j\to \infty }\parallel u^{k_j}-T\left(u^{k_j}\right)\parallel =0$. The second inequality in (19) holds by Lemma 3.4 of [31]. It follows from the demiclosedness of T that ${\widehat{u}}^{*}\in \mathsf{\Omega }$. By Opial’s lemma, we conclude that $u^k\rightharpoonup {\widehat{u}}^{*}$. This completes the proof. □

3.2. Connections to Existing Algorithms

In this subsection, we present a series of special cases of the proposed algorithm and point out connections to other existing algorithms.

(i) Let $P_k=\frac{1}{\lambda }I$, $Q_k=Q$; then, (13) reduces to the PDFP${}^2$O_AM [26].

$$\left\{\begin{array}{cc}\hfill {\tilde{v}}^{k+1}& =pro{x}_{\lambda h^{*}}(\frac{1}{\lambda }L(x^k-Q^{-1}\nabla f\left(x^k\right))+(I-\lambda L{Q}^{-1}{L}^{*})v^k),\hfill \\ \hfill {\tilde{x}}^{k+1}& =x^k-Q^{-1}\nabla f\left(x^k\right)-Q^{-1}{L}^{*}{\tilde{v}}^{k+1},\hfill \\ \hfill (v^{k+1}& ,x^{k+1})=(1-{\rho }_k)(v^k,x^k)+{\rho }_k({\tilde{v}}^{k+1},{\tilde{x}}^{k+1}).\hfill \end{array}\right.$$

(20)

(ii) Let $P_k=\frac{{\gamma }_k}{{\lambda }_k}I$ and $Q_k=\frac{1}{{\gamma }_k}I$; then, we obtain from (13) that

$$\left\{\begin{array}{cc}\hfill {\tilde{v}}^{k+1}& =pro{x}_{\frac{{\lambda }_k}{{\gamma }_k}{h}^{*}}(\frac{{\lambda }_k}{{\gamma }_k}L(x^k-{\gamma }_k\nabla f\left(x^k\right))+(I-{\lambda }_{k}L{L}^{*})v^k),\hfill \\ \hfill {\tilde{x}}^{k+1}& =x^k-{\gamma }_k\nabla f\left(x^k\right)-{\gamma }_k{L}^{*}{\tilde{v}}^{k+1},\hfill \\ \hfill (v^{k+1}& ,x^{k+1})=(1-{\rho }_k)(v^k,x^k)+{\rho }_k({\tilde{v}}^{k+1},{\tilde{x}}^{k+1}),\hfill \end{array}\right.$$

(21)

which recovers the PDFP${}^2$O_DS [27].

(iii) Let $P_k=\frac{\gamma }{\lambda }I$ and $Q_k=\frac{1}{\gamma }I$; then, (13) becomes

$$\left\{\begin{array}{cc}\hfill {\tilde{v}}^{k+1}& =pro{x}_{\frac{\lambda }{\gamma }{h}^{*}}(\frac{\lambda }{\gamma }L(x^k-\gamma \nabla f\left(x^k\right))+(I-\lambda L{L}^{*})v^k),\hfill \\ \hfill {\tilde{x}}^{k+1}& =x^k-\gamma \nabla f\left(x^k\right)-\gamma L^{*}{\tilde{v}}^{k+1},\hfill \\ \hfill (v^{k+1}& ,x^{k+1})=(1-{\rho }_k)(v^k,x^k)+{\rho }_k({\tilde{v}}^{k+1},{\tilde{x}}^{k+1}),\hfill \end{array}\right.$$

(22)

which is the original PDFP${}^2$O (12).

Rv_PDFP${}^2$O (13) reduces to the above three algorithms (20)–(22) for different $P_k$ and $Q_k$. It is eailsy confirmed that Rv_PDFP${}^{2}O$ (13) generalizes these algorithms.

3.3. Convergence Rates

In this subsection, we discuss convergence rates of (13).

3.3.1. $O\left(\frac{1}{k}\right)-$Ergodic Convergence Rate

First, we establish the ergodic convergence rate.

Lemma 2.

Suppose that (A1) holds. Let $\{{\tilde{u}}^{k+1}=({\tilde{v}}^{k+1},{\tilde{x}}^{k+1})\}$ be generated by (13). Then, for any $u=(v,x)\in K$, it holds that

$$\begin{array}{cc}\hfill & K(v,{\tilde{x}}^{k+1})-K({\tilde{v}}^{k+1},x)\hfill \\ \hfill \le & \frac{1}{2}(\parallel u^k{-u\parallel }_{U_k^{-1}}^2-\parallel {\tilde{u}}^{k+1}{-u\parallel }_{U_k^{-1}}^2-\parallel {\tilde{u}}^{k+1}-u^k{\parallel }_{U_k^{-1}}^2)+\frac{1}{2\beta }{\parallel {\tilde{x}}^{k+1}-x^k\parallel }^2.\hfill \end{array}$$

(23)

Proof. 

It follows from the property of proximity operator that

$$\begin{array}{cc}\hfill h^{*}\left(v\right)\ge & h^{*}({\tilde{v}}^{k+1})+⟨L(x^k-Q_k^{-1}{L}^{*}{v}^k-Q_k^{-1}\nabla f\left(x^k\right)),v-{\tilde{v}}^{k+1}⟩\hfill \\ \hfill +& \frac{1}{2}(\parallel {\tilde{v}}^{k+1}-v^k{\parallel }_{P_k}^2+\parallel {\tilde{v}}^{k+1}{-v\parallel }_{P_k}^2-\parallel v^k-v{\parallel }_{P_k}^2).\hfill \end{array}$$

(24)

By the differentiability of f, we have

$$\begin{array}{cc}\hfill f\left(x\right)\ge & f\left({\tilde{x}}^{k+1}\right)+⟨\nabla f\left(x^k\right),x-{\tilde{x}}^{k+1}⟩-\frac{1}{2\beta }{\parallel {\tilde{x}}^{k+1}-x^k\parallel }^2\hfill \\ \hfill \ge & f\left({\tilde{x}}^{k+1}\right)+\frac{1}{2}\parallel {\tilde{x}}^{k+1}-x^k{\parallel }_{Q_k}^2+\frac{1}{2}\parallel x-{\tilde{x}}^{k+1}{\parallel }_{Q_k}^2-\frac{1}{2}{\parallel x-x^k\parallel }_{Q_k}^2\hfill \\ \hfill +& ⟨{\tilde{v}}^{k+1},L({\tilde{x}}^{k+1}-x)⟩-\frac{1}{2\beta }{\parallel {\tilde{x}}^{k+1}-x^k\parallel }^2.\hfill \end{array}$$

(25)

Adding (24) and (25) and rearranging to arrive at (23). □

Theorem 2.

Suppose that (A1)–(A3) hold with ${\xi }_k=\frac{\parallel Q_k^{-1}{\parallel }_2}{2\beta }$. Let $\{{\tilde{u}}^{k+1}=({\tilde{v}}^{k+1},{\tilde{x}}^{k+1})\}$ be generated by (13). Then, for $u^{*}=(v^{*},x^{*})\in \mathsf{\Omega }$, it holds that

$$K(v^{*},{\overline{X}}^N)-K({\overline{V}}^{N+1},x^{*})\le \frac{1}{2\rho N}{\parallel u^0-u^{*}\parallel }_{U_0^{-1}}^2,$$

(26)

where ${\overline{X}}^N=\frac{1}{N}{\sum }_{k=0}^{N-1}{\tilde{x}}^{k+1},{\overline{V}}^N=\frac{1}{N}{\sum }_{k=0}^{N-1}{\tilde{v}}^{k+1}$.

Proof. 

Note that ${\tilde{u}}^{k+1}=u^k+\frac{1}{{\rho }_k}(u^{k+1}-u^k)$. Substituting it back into (23), we have

$$\begin{array}{cc}\hfill & K(v^{*},{\tilde{x}}^{k+1})-K({\tilde{v}}^{k+1},x^{*})\hfill \\ \hfill \le & \frac{1}{2{\rho }_k}(\parallel u^k-u^{*}{\parallel }_{U_k^{-1}}^2-\parallel u^{k+1}-u^{*}{\parallel }_{U_{k+1}^{-1}}^2)-\frac{2-{\rho }_k}{2{\rho }_k^2}{\parallel x^{k+1}-x^k\parallel }_{Q_k-\frac{I}{\beta (2-{\rho }_k)}}^2\hfill \\ \hfill \le & \frac{1}{2\rho }(\parallel u^k-u^{*}{\parallel }_{U_k^{-1}}^2-\parallel u^{k+1}-u^{*}{\parallel }_{U_{k+1}^{-1}}^2).\hfill \end{array}$$

(27)

Summing (27) from $k=0,\dots ,N-1$, we obtain

$$\sum _{k=0}^{N-1}(K(v^{*},{\tilde{x}}^{k+1})-K({\tilde{v}}^{k+1},x^{*}))\le \frac{1}{2\rho }{\parallel u^0-u^{*}\parallel }_{U_0^{-1}}^2.$$

The final estimation (26) follows directly from the Jensen inequality. □

3.3.2. Linear Convergence Rate

Next, we establish a linear convergence rate of (13) with $P_k=P$ and $Q_k=Q$. Therefore, ${\widehat{T}}_{\left(k\right)}=T$. For convenience, we give an equivalent formulation of T as follows:

$$T_1(v,x)=pro{x}_{h^{*}}^P((I-P^{-1}L{Q}^{-1}{L}^{*})v+P^{-1}L(x-Q^{-1}\nabla f\left(x\right))),$$

(28)

$$T_2(v,x)=x-Q^{-1}\nabla f\left(x\right)-Q^{-1}{L}^{*}\circ T_1(v,x),$$

(29)

$$T(v,x)=(T_1(v,x),T_2(v,x)).$$

(30)

In addition, we make some additional assumptions. More precisely,

(A5): 

$\partial h^{*}$ is ${\tau }_h-$strongly monotone under ${\parallel ·\parallel }_{I-P^{-1}L{Q}^{-1}{L}^{*}}$, i.e.,

$$\forall (x_1,v_1),(x_2,v_2)\in gra\partial h^{*},⟨x_1-x_2,v_1-v_2⟩\ge {\tau }_h{\parallel x_1-x_2\parallel }_{I-P^{-1}L{Q}^{-1}{L}^{*}}^2.$$

(A6): 

$\nabla f$ is ${\tau }_f-$strongly monotone under the norm ${\parallel ·\parallel }_Q$, i.e.,

$$\forall x_1,x_2\in H,⟨x_1-x_2,\nabla f\left(x_1\right)-\nabla f\left(x_2\right)⟩\ge {\tau }_f{\parallel x_1-x_2\parallel }_Q^2.$$

(A7): 

There is ${\theta }_1,{\theta }_2\in (0,1)$ such that $\parallel I-P^{-1}L{Q}^{-1}{L}^{*}{\parallel }_2\le {\theta }_1$ and ${\parallel x_1-Q^{-1}\nabla f\left(x_1\right)-x_2-Q^{-1}\nabla f\left(x_2\right)\parallel }_Q\le \sqrt{{\theta }_2}{\parallel x_1-x_2\parallel }_Q$ for all $x_1,x_2\in H$.

Lemma 3.

Suppose that $\left(A1\right)$, $\left(A5\right)$, and $\left(A6\right)$ hold. Then,

$$\begin{array}{cc}\hfill & \parallel T\left(u_1\right)-T\left(u_2\right){\parallel }_{(P+2{\tau }_{h}I)(I-P^{-1}L{Q}^{-1}{L}^{*}),Q}^2\hfill \\ \hfill \le & \theta \parallel u_1-u_2{\parallel }_{(P+2{\tau }_{h}I)(I-P^{-1}L{Q}^{-1}{L}^{*}),Q}^2,\hfill \end{array}$$

(31)

for $u_1=(v_1,x_1),u_2=(v_2,x_2)\in K$, where $\theta \in (0,1)$.

Proof. 

Let $u_{h_1}=M{v}_1+L(x_1-Q^{-1}\nabla f\left(x_1\right))-P{T}_1\left(u_1\right)\in \partial h^{*}\left(T_1\left(u_1\right)\right)$ and $u_{h_2}=M{v}_2+L(x_2-Q^{-1}\nabla f\left(x_2\right))-P{T}_1\left(u_2\right)\in \partial h^{*}\left(T_1\left(u_2\right)\right)$. Then, we have

$$\begin{array}{cc}\hfill & \parallel T\left(u_1\right)-T\left(u_2\right){\parallel }_{P-L{Q}^{-1}{L}^{*},Q}^2\hfill \\ \hfill =& {\parallel u_1-u_2\parallel }_{P-L{Q}^{-1}{L}^{*},Q}^2-2⟨\nabla f\left(x_1\right)-\nabla f\left(x_2\right),T_2\left(u_1\right)-T_2\left(u_2\right)-(x_1-x_2)⟩\hfill \\ \hfill -& \parallel T\left(u_1\right)-T\left(u_2\right)-(u_1-u_2){\parallel }_{P-L{Q}^{-1}{L}^{*},Q}^2-2⟨\nabla f\left(x_1\right)-\nabla f\left(x_2\right),x_1-x_2⟩\hfill \\ \hfill -& 2⟨T_1\left(u_1\right)-T_1\left(u_2\right),u_{h_1}-u_{h_2}⟩\hfill \\ \hfill \le & \parallel u_1-u_2{\parallel }_{P-L{Q}^{-1}{L}^{*},Q}^2-(2-\frac{\parallel Q^{-1}{\parallel }_2}{\beta })⟨\nabla f\left(x_1\right)-\nabla f\left(x_2\right),x_1-x_2⟩\hfill \\ \hfill -& 2⟨T_1\left(u_1\right)-T_1\left(u_2\right),u_{h_1}-u_{h_2}⟩\hfill \\ \hfill \le & \parallel u_1-u_2{\parallel }_{P-L{Q}^{-1}{L}^{*},(1-(2-\frac{\parallel Q^{-1}{\parallel }_2}{\beta }){\tau }_f)Q}^2-2{\tau }_h{\parallel T_1\left(u_1\right)-T_1\left(u_2\right)\parallel }_{I-P^{-1}L{Q}^{-1}{L}^{*}}^2,\hfill \end{array}$$

(32)

which concludes the proof with $\theta =max\{1-(2-\frac{\parallel Q^{-1}{\parallel }_2}{\beta }){\tau }_f,\frac{1}{1+2{\tau }_h{\lambda }_{min}\left(P^{-1}\right)}\}$. □

Lemma 4.

Suppose that $\left(A1\right)$ and $\left(A7\right)$ hold. Then, for $u_1=(v_1,x_1),u_2=(v_2,x_2)\in K$,

$$\parallel T\left(u_1\right)-T\left(u_2\right){\parallel }_{P,Q}^2\le \theta {\parallel u_1-u_2\parallel }_{P,Q}^2,$$

where $\theta \in (0,1)$.

Proof. 

Define $M=P-L{Q}^{-1}{L}^{*}$. It follows from the fact that $pro{x}_{h^{*}}^P$ is firmly nonexpansive that

$$\begin{array}{cc}\hfill & \parallel T\left(u_1\right)-T\left(u_2\right){\parallel }_{P,Q}^2\hfill \\ \hfill \le & {\parallel x_1-Q^{-1}\nabla f\left(x_1\right)-x_2-Q^{-1}\nabla f\left(x_2\right)\parallel }_Q^2-{\parallel T_1\left(u_1\right)-T_1\left(u_2\right)\parallel }_M^2\hfill \\ \hfill +& 2⟨T_1\left(u_1\right)-T_1\left(u_2\right),M(v_1-v_2)⟩\hfill \\ \hfill =& {\parallel x_1-Q^{-1}\nabla f\left(x_1\right)-x_2-Q^{-1}\nabla f\left(x_2\right)\parallel }_Q^2+{\parallel v_1-v_2\parallel }_M^2\hfill \\ \hfill -& \parallel T_1\left(u_1\right)-T_1\left(u_2\right)-(v_1-v_2){\parallel }_M^2\hfill \\ \hfill \le & {\theta }_2\parallel x_1-x_2{\parallel }_Q^2+{\theta }_1{\parallel v_1-v_2\parallel }_P^2\hfill \\ \hfill \le & \theta \parallel u_1-u_2{\parallel }_{P,Q}^2,\hfill \end{array}$$

(33)

where $\theta =max\{{\theta }_1,{\theta }_2\}\in (0,1)$. □

Theorem 3.

Suppose that $\left(A1\right)$ holds. Suppose that $\left(A5\right)–\left(A6\right)$ hold or $\left(A7\right)$ holds. Let $\{u^{k+1}=(v^{k+1},x^{k+1})\}$ be generated by (13). Let ${\rho }_k\in (0,\overline{\rho })$ for $\overline{\rho }=min\{\frac{2}{1+\sqrt{\theta }},\frac{4\beta -\parallel Q^{-1}{\parallel }_2}{2\beta }\}$. Then, $\left\{u^{k+1}\right\}$ converges linearly to the unique point $u^{*}\in \mathsf{\Omega }$, i.e.,

$$\parallel u^{k+1}-u^{*}\parallel \le c{\eta }^{k+1},$$

where $c>0$, $\eta \in (0,1)$.

Proof. 

Define $T_{\left({\rho }_k\right)}=(1-{\rho }_k)I+{\rho }_{k}T$. Note that $u^{k+1}=T_{\left({\rho }_k\right)}\left(u^k\right)$ and $Fix{T}_{\left({\rho }_k\right)}=FixT=\mathsf{\Omega }$. It is clear that $T_{\left({\rho }_k\right)}$ is ${\eta }_k$-contractive for ${\eta }_k=|1-{\rho }_k|+{\rho }_k\sqrt{\theta }$. Therefore, $\left\{u^{k+1}\right\}$ converges linearly to the unique fixed point of $T_{\left({\rho }_k\right)}$. □

4. Numerical Experiments

In this section, we apply the proposed Rv_PDFP${}^2$O (13) to solve the $L_2+TV$ deblurring problem (2) and compare it with those of the ADMM [5], PDS [32], PDFP${}^2$O [19], and PDFP${}^2$O_AM [26]. All of the experiments are performed under Windows 7 and MATLAB $7.2$ (R2014a) running on a laptop with an Intel Core 2 Quad CPU $2.3$ GHz with 4 GB of memory.

The test images are the standard “Text” image with a size of $256\times 256$, and “Barbara” and “Goldhill” with a size $512\times 512$, which are shown in Figure 1. We report numerical results on the image restoration for blurred images, corrupted by the Gaussian noise and the average kernel; a is the size of average kernel, and $\eta$ is the standard variance of the Gaussian noise. To evaluate the ability of the algorithm to remove different noises, we set four kinds of $a,\eta$: (1) $a=3,\eta =0.01$; (2) $a=3,\eta =0.05$; (3) $a=7,\eta =0.01$; and (4) $a=7,\eta =0.05$.

Figure 1. These are the test images: (a) Text, (b) Barbara, and (c) Goldhill.

For the two common parameters $\gamma$ and $\lambda$ in PDS, and PDFP${}^2$O, we set $\gamma =1.9$ and $\lambda =0.125$. Similarly to the literature [26], we choose $Q=A^{T}A+\zeta L^{T}L$, where $\zeta =0.1$. In particular, $Q^{-1}$ can be easily computed by FFT with periodic boundary conditions. We tune the regularization parameter $\mu$ to achieve the maximum SNR, which is listed in Table 2.

Table 2. The best selection of $\mu$ in the current noise level.

Images	$\mathit{a}=3$		$\mathit{a}=7$
	$\mathbf{\eta }=\mathbf{0}.\mathbf{01}$	$\mathbf{\eta }=\mathbf{0}.\mathbf{05}$	$\mathbf{\eta }=\mathbf{0}.\mathbf{01}$	$\mathbf{\eta }=\mathbf{0}.\mathbf{05}$
“Text”	$0.0013$	$0.0078$	$0.0003$	$0.0027$
“Barbara”	$0.0004$	$0.0101$	$0.0004$	$0.009$
“Goldhill”	$0.0011$	$0.0148$	$0.0006$	$0.0085$

The relative error of the iterative sequences is defined as the stopping criteria:

$$\frac{\parallel x^{k+1}-x^k{\parallel }_2}{\parallel x^k{\parallel }_2}<\epsilon ,$$

where $\epsilon >0$ is a prescribed tolerance value. In the experiment, we choose $\epsilon ={10}^{-4},{10}^{-6},$${10}^{-8}$. The quality of the restored images is evaluated by signal-to-noise (SNR), which is defined by

$$SNR=10log\frac{{\parallel x\parallel }^2}{\parallel x^r{-x\parallel }^2},$$

where x and $x^r$ denote the original and the recovered images. The obtained numerical results are listed in Table 3 and Table 4.

Table 3. The performance of $a=3$ of the compared algorithms in terms of SNR (dB) and the number of iterations k for given tolerance values $\epsilon$.

$\mathit{\eta }$	Image	Method	$\mathit{\epsilon }={10}^{-4}$		$\mathit{\epsilon }={10}^{-6}$		$\mathit{\epsilon }={10}^{-8}$
			$\mathit{SNR}\left(\mathit{dB}\right)$	$\mathit{k}$	$\mathit{SNR}\left(\mathit{dB}\right)$	$\mathit{k}$	$\mathit{SNR}\left(\mathit{dB}\right)$	$\mathit{k}$
$0.01$	Text	ADMM	$26.6288$	289	$27.6605$	813	$27.6686$	1537
		PDS	$26.4334$	296	$27.6402$	933	$27.6500$	1693
		PDFP${}^2$O	$27.0279$	182	$27.6455$	521	$27.6500$	915
		PDFP${}^2$O_AM	$27.1659$	178	$27.6650$	456	$27.6686$	831
		Rv_PDFP${}^2$O	$27.1838$	173	$27.6615$	441	$27.6686$	803
	Barbara	ADMM	$21.8641$	138	$21.6752$	1423	$21.6704$	5332
		PDS	$21.8186$	152	$21.6783$	1499	$21.6733$	5338
		PDFP${}^2$O	$21.7928$	118	$21.6757$	958	$21.6733$	3022
		PDFP${}^2$O_AM	$21.8115$	109	$21.6727$	911	$21.6704$	3058
		Rv_PDFP${}^2$O	$21.8075$	108	$21.6726$	887	$21.6704$	2962
	Goldhill	ADMM	$26.6924$	68	$26.5822$	513	$26.5807$	1554
		PDS	$26.6686$	79	$26.5769$	510	$26.5754$	1471
		PDFP${}^2$O	$26.5980$	95	$26.5761$	306	$26.5754$	829
		PDFP${}^2$O_AM	$26.6467$	51	$26.5814$	314	$26.5807$	878
		Rv_PDFP${}^2$O	$26.6448$	50	$26.5814$	305	$26.5807$	849
$0.05$	Text	ADMM	$14.7157$	150	$14.7797$	511	$14.7801$	1048
		PDS	$14.6905$	161	$14.7623$	533	$14.7626$	1904
		PDFP${}^2$O	$14.7341$	103	$14.7627$	384	$14.7626$	1892
		PDFP${}^2$O_AM	$14.7510$	96	$14.7800$	308	$14.7801$	1185
		Rv_PDFP${}^2$O	$14.7520$	93	$14.7800$	296	$14.7801$	1184
	Barbara	ADMM	$18.3425$	47	$18.3337$	231	$18.3336$	883
		PDS	$18,3496$	74	$18.3454$	349	$18.3454$	1683
		PDFP${}^2$O	$18.3459$	95	$18.3453$	342	$18.3454$	1681
		PDFP${}^2$O_AM	$18.3362$	42	$18.3336$	226	$18.3336$	1154
		Rv_PDFP${}^2$O	$18.3359$	42	$18.3336$	226	$18.3336$	1154
	Goldhill	ADMM	$22.7421$	47	$22.7367$	260	$22.7367$	1134
		PDS	$22.7033$	84	$22.7015$	524	$22.7014$	2529
		PDFP${}^2$O	$22.7027$	97	$22.7015$	508	$22.7014$	2527
		PDFP${}^2$O_AM	$22.7383$	53	$22.7367$	328	$22.7367$	1571
		Rv_PDFP${}^2$O	$22.7382$	53	$22.7367$	328	$22.7367$	1568

Table 4. The performance of $a=7$ of the compared algorithms in terms of SNR (dB) and the number of iterations k for given tolerance values $\epsilon$.

$\mathit{\eta }$	Image	Method	$\mathit{\epsilon }={10}^{-4}$		$\mathit{\epsilon }={10}^{-6}$		$\mathit{\epsilon }={10}^{-8}$
			$\mathit{SNR}\left(\mathit{dB}\right)$	$\mathit{k}$	$\mathit{SNR}\left(\mathit{dB}\right)$	$\mathit{k}$	$\mathit{SNR}\left(\mathit{dB}\right)$	$\mathit{k}$
$0.01$	Text	ADMM	$12.8474$	832	$14.1337$	6220	$14.1548$	19,540
		PDS	$12.4021$	1114	$14.1117$	7138	$14.1382$	19,512
		PDFP${}^2$O	$13.1403$	835	$14.1252$	4277	$14.1383$	10,939
		PDFP${}^2$O_AM	$13.4212$	646	$14.1446$	3828	$14.1549$	11,248
		Rv_PDFP${}^2$O	$13.4436$	634	$14.1450$	3718	$14.1549$	10,899
	Barbara	ADMM	$18.5769$	145	$18.4559$	2153	$18.4523$	9172
		PDS	$18.5597$	192	$18.4592$	2549	$18.4541$	9851
		PDFP${}^2$O	$18.5490$	161	$18.4567$	1647	$18.4541$	5623
		PDFP${}^2$O_AM	$18.5445$	124	$18.4541$	1401	$18.4523$	5288
		Rv_PDFP${}^2$O	$18.5429$	122	$18.4541$	1365	$18.4523$	5124
	Goldhill	ADMM	$23.2362$	116	$23.0522$	1475	$23.0480$	8688
		PDS	$23.1623$	167	$23.0451$	1791	$23.0395$	8417
		PDFP${}^2$O	$23.1506$	129	$23.0424$	1146	$23.0395$	5284
		PDFP${}^2$O_AM	$23.1775$	92	$23.0500$	966	$23.0480$	5608
		Rv_PDFP${}^2$O	$23.1739$	91	$23.0499$	943	$23.0480$	5463
$0.05$	Text	ADMM	$7.0043$	374	$6.9970$	2466	$6.9972$	6803
		PDS	$6.9372$	535	$6.9771$	3074	$6.9773$	7579
		PDFP${}^2$O	$6.9601$	373	$6.9772$	1859	$6.9773$	4154
		PDFP${}^2$O_AM	$6.9997$	276	$6.9971$	1528	$6.9972$	3737
		Rv_PDFP${}^2$O	$6.9995$	270	$6.9971$	1488	$6.9972$	3611
	Barbara	ADMM	$17.1962$	95	$17.1656$	764	$17.1653$	2939
		PDS	$17.2239$	161	$17.1937$	900	$17.1932$	3370
		PDFP${}^2$O	$17.2073$	121	$17.1933$	648	$17.1932$	3091
		FP${}^2$O_AM	$17.1783$	80	$17.1654$	506	$17.1653$	1955
		Rv_PDFP${}^2$O	$17.1776$	79	$17.1654$	501	$17.1653$	1947
	Goldhill	ADMM	$20.4785$	83	$20.4481$	612	$20.4477$	2441
		PDS	$20.4285$	153	$20.4010$	762	$20.4005$	2684
		PDFP${}^2$O	$20.4114$	116	$20.4006$	566	$20.4005$	2323
		PDFP${}^2$O_AM	$20.4595$	72	$20.4478$	430	$20.4477$	1529
		Rv_PDFP${}^2$O	$20.4586$	72	$20.4478$	426	$20.4477$	1505

It can be seen from Table 3 and Table 4 that the proposed Rv_PDFP${}^2$O converges faster than other algorithms in terms of the number of iterations. In addition, Figure 2, Figure 3 and Figure 4 show the recovered images with $\epsilon ={10}^{-8}$. Figure 2, Figure 3 and Figure 4 show that the visual qualities of these images obtained by the proposed algorithm are slightly better than the compared algorithms.

Figure 2. These are the “Text” images: Row 1: the blurry and noisy images, Row 2: the images restored by ADMM, Row 3: the images restored by PDS, Row 4: the images restored by PDFP${}^2$O, Row 5: the images restored by PDFP${}^2$O_AM, and Row 6: the images restored by Rv_PDFP${}^2$O.

Figure 3. These are the “Goldhill” images: Row 1: the blurry and noisy images, Row 2: the images restored by ADMM, Row 3: the images restored by PDS, Row 4: the images restored by PDFP${}^2$O, Row 5: the images restored by PDFP${}^2$O_AM, and Row 6: the images restored by Rv_PDFP${}^2$O.

Figure 4. These are the “Goldhill” images: Row 1: the blurry and noisy images, Row 2: the images restored by ADMM, Row 3: the images restored by PDS, Row 4: the images restored by PDFP${}^2$O, Row 5: the images restored by PDFP${}^2$O_AM, and Row 6: the images restored by Rv_PDFP${}^2$O.

5. Conclusions

In this article, we proposed a Rv_PDFP${}^2$O to solve the convex optimization problem (1). The proposed algorithm combined the over-relaxed parameters and the variable metric. Under a proper preconditioned operator, we derived the Rv_PDFP${}^2$O and established the convergence. By defining different stepsizes, we showed that the Rv_PDFP${}^2$O recovers some existing algorithms, including PDFP${}^2$O, PDFP${}^2$O_AM, and PDFP${}^2$O_DS, and we provide larger relaxed parameters for these algorithms. Furthermore, we studied the $O\left(\frac{1}{k}\right)$ ergodic convergence rate in the partial primal-dual gap. Under some strong conditions on the objective functions and the stepsizes, we proved that the iterative sequences converge linearly. We applied the Rv_PDFP${}^2$O to solve the TV image-restoration problem (2). The numerical results show that the Rv_PDFP${}^2$O performs better than some existing algorithms. As we all know, the self-adaptive stepsize and the inertial variant could improve the algorithm. However, these two accelerated strategies are not introduced to the Rv_PDFP${}^2$O algorithm. We would like to derive a self-adaptive Rv_PDFP${}^2$O and an inertial Rv_PDFP${}^2$O in the future.