Generalized Inexact Newton-Landweber Iteration for Possibly Non-Smooth Inverse Problems in Banach Spaces

Abstract

In this paper, we consider a generalized inexact Newton-Landweber iteration to solve nonlinear ill-posed inverse problems in Banach spaces, where the forward operator might not be Gâteaux differentiable. The method is designed with non-smooth convex penalty terms, including $L^1$-like and total variation-like penalty functionals, to capture special features of solutions such as sparsity and piecewise constancy. Furthermore, the inaccurate inner solver is incorporated into the minimization problem in each iteration step. Under some assumptions, based on $\epsilon$-subdifferential, we establish the convergence analysis of the proposed method. Finally, some numerical simulations are provided to illustrate the effectiveness of the method for solving both smooth and non-smooth nonlinear inverse problems.

1. Introduction

We are interested in solving the following ill-posed inverse problem

$$F\left(x\right)=y,$$

(1)

where $F:\mathcal{D}\left(F\right)\subset \mathcal{X}\to \mathcal{Y}$ is a possibly non-smooth nonlinear operator from Banach space $\mathcal{X}$ to Banach space $\mathcal{Y}$. In general, (1) may have many solutions. To pick the one with desired feature, we choose a convex function $\Theta :\mathcal{X}\to \left(-\infty ,\infty \right]$ and determine a solution $x^{†}$ such that

$$D_{{\xi }_0}\Theta \left(x^{†},x_0\right):=\underset{x\in \mathcal{D}\left(\Theta \right)\cap \mathcal{D}\left(F\right)}{min}\left\{D_{{\xi }_0}\Theta \left(x,x_0\right):F\left(x\right)=y\right\},$$

(2)

where $x_0\in \mathcal{D}\left(\partial \Theta \right)$ and ${\xi }_0\in \partial \Theta \left(x_0\right)$ denote the initial guesses, and $D_{{\xi }_0}\Theta \left(x,x_0\right)$ denotes the Bregman distance induced by $\Theta$. Due to the error of measurement, instead of y, only perturbed data $y^{\delta }$ is available, satisfying

$$\parallel y^{\delta }{-y\parallel }_{\mathcal{Y}}\le \delta$$

(3)

with known noise level $\delta >0$. A typical property of such equations is their ill-posedness, i.e., small perturbations of data may lead to huge deviations in solutions of (1). Therefore, regularization techniques are needed to obtain a stable solution $x^{†}$ from $y^{\delta }$; see [1,2,3] and references therein.

In [4,5,6,7,8], inexact Newton regularization methods were developed for solving nonlinear inverse problems in Banach spaces, where the forward mapping F is assumed to be smooth, i.e., continuously Fréchet differentiable. This type of method updates the current iteration $x_n^{\delta }$ by applying an iterative regularization scheme to solve approximately the local linearization of (1) at $x_n^{\delta }$, i.e.,

$$F^{\prime }\left(x_n^{\delta }\right)\left(x-x_n^{\delta }\right)=y^{\delta }-F\left(x_n^{\delta }\right),$$

(4)

where $F^{\prime }\left(x_n^{\delta }\right)$ is the Fréchet derivative of F at $x_n^{\delta }$. Assuming the nth iterates $\left({\xi }_n^{\delta },x_n^{\delta }\right)$ are available, by employing the Landweber iteration in [9] to (4), the inexact Newton-Landweber iteration in [6] generates the inner iterates $\left\{({\xi }_{n,k}^{\delta },x_{n,k}^{\delta })\right\}$ by

$$\begin{array}{c}{\xi }_{n,k+1}^{\delta }={\xi }_{n,k}^{\delta }+{\mu }_{n,k}^{\delta }{F}^{\prime }{\left(x_n^{\delta }\right)}^{*}{J}_r^{\mathcal{Y}}\left(y^{\delta }-F\left(x_n^{\delta }\right)-F^{\prime }\left(x_n^{\delta }\right)\left(x_{n,k}^{\delta }-x_n^{\delta }\right)\right),\hfill \\ x_{n,k+1}^{\delta }=arg\underset{x\in \mathcal{X}}{min}\left\{{\Theta \left(x\right)-〈{\xi }_{n,k+1}^{\delta },x〉}_{{\mathcal{X}}^{*},\mathcal{X}}\right\}\hfill \end{array}$$

(5)

with ${\xi }_{n,0}^{\delta }={\xi }_n^{\delta }$ and $x_{n,0}^{\delta }=x_n^{\delta }$ and suitable step length ${\mu }_{n,k}^{\delta }$, and $J_r^{\mathcal{Y}}\left(y\right):=\partial \left(\frac{1}{r}{∥y∥}_{\mathcal{Y}}^r\right)(1<r<\infty )$ denotes the duality mapping from $\mathcal{Y}$ to its dual ${\mathcal{Y}}^{*}$. Let $k_n^{\delta }$ be the smallest integer such that

$${∥y^{\delta }-F\left(x_n^{\delta }\right)-F^{\prime }\left(x_n^{\delta }\right)\left(x_{n,k_n^{\delta }}^{\delta }-x_n^{\delta }\right)∥}_{\mathcal{Y}}<\gamma {∥y^{\delta }-F\left(x_n^{\delta }\right)∥}_{\mathcal{Y}}$$

(6)

for some $0<\gamma <1$, then the next outer iterates are defined as

$${\xi }_{n+1}^{\delta }={\xi }_{n,k_n^{\delta }}^{\delta }\mathrm{and}{x}_{n+1}^{\delta }=x_{n,k_n^{\delta }}^{\delta }.$$

(7)

It has been shown in [6] that this renders a regularization method if the outer iteration (7) is terminated by the discrepancy principle. Recently, in [7,8] the authors considered an inexact Newton regularization method employing a so-called two-point gradient method [10] as inner scheme and derived the convergence result under the discrepancy principle. When $\mathcal{X}$ and $\mathcal{Y}$ are both Hilbert spaces and $\Theta \left(x\right)={∥x∥}^2/\phantom{{∥x∥}^{2}2}2$, one may refer to [11,12,13,14,15,16] for some convergence and convergence rates.

Except for the tangential cone condition on the forward operator, the convergence analysis for inexact Newton regularization methods in [4,5,6,7,8] requires the continuity of the derivative $F^{\prime }\left(x\right)$ and the exact resolution of the minimization problem in (5). However, there are some cases where the forward operator F is not Gâteaux differentiable [17], which leads to the existing inexact Newton method impractical. Moreover, the exact solution of (5) can only be found for some special $\Theta$; this minimization problem in general can only be solved numerically inaccurately. Therefore, it is necessary to generalize the method (5)–(7) to cover the non-smooth forward operator case and to incorporate an inner inexact solver for the minimization problem in (5).

It has been shown in [18] that the Fréchet derivative $F^{\prime }\left(x\right)$ can be substituted by another bounded linear operator sufficiently close to $F^{\prime }\left(x\right)$. In [17,19], by introducing the Bouligand subderivative, the authors considered the Bouligand–Landweber iteration and the Bouligand–Levenberg–Marquardt method in Hilbert spaces. By employing a bounded operator $A_F$ (depending only on a certain point) as the replacement of $F^{\prime }\left(x\right)$, an extension of the Gauss–Newton method was proposed in [20,21,22]. Inspired by the previous work, in this work, we propose a generalized inexact Newton-Landweber iteration for solving inverse problems with possibly non-smooth nonlinear operators, given by

$$\begin{array}{c}{\xi }_{n,k+1}^{\delta }={\xi }_{n,k}^{\delta }+{\mu }_{n,k}^{\delta }{A}_F^{*}{J}_r^{\mathcal{Y}}\left(y^{\delta }-F\left(x_n^{\delta }\right)-A_F\left(x_{n,k}^{\delta }-x_n^{\delta }\right)\right),\hfill \\ \Theta \left(x_{n,k+1}^{\delta }\right)-{〈{\xi }_{n,k+1}^{\delta },x_{n,k+1}^{\delta }〉}_{{\mathcal{X}}^{*},\mathcal{X}}\le \underset{x\in \mathcal{X}}{min}\left\{\Theta \left(x\right)-{〈{\xi }_{n,k+1}^{\delta },x〉}_{{\mathcal{X}}^{*},\mathcal{X}}\right\}+{\epsilon }_{n,k+1},\hfill \end{array}$$

(8)

where $A_F$ is a bounded operator satisfying certain conditions (see (17)), ${\epsilon }_{n,k+1}>0$. The inner stopping index is chosen as $k_n^{\delta }:=min\left\{{\tilde{k}}_n^{\delta },k_{max}\right\}$, where ${\tilde{k}}_n^{\delta }$ is the first integer such that

$${∥y^{\delta }-F\left(x_n^{\delta }\right)-A_F\left(x_{n,k}^{\delta }-x_n^{\delta }\right)∥}_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k}<{\left(\gamma {∥y^{\delta }-F\left(x_n^{\delta }\right)∥}_{\mathcal{Y}}\right)}^p$$

for $0<\gamma <1$, $\sigma >0$ and given integer $k_{max}\ge 1$. The next iterates are then constructed by ${\xi }_{n+1}^{\delta }={\xi }_{n,k_n^{\delta }}^{\delta }\mathrm{and}{x}_{n+1}^{\delta }=x_{n,k_n^{\delta }}^{\delta }$. Please note that when ${\epsilon }_{n,k+1}=0$, $k_{max}=\infty$, F is continuously Fréchet differentiable and $A_F:=F^{\prime }\left(x_n^{\delta }\right)$, our method (8) reduces to the method (5) in [6]. In contrast to existing methods [4,5,6,7,8], our proposed method (8) does not require the Fréchet differentiablility of F and the exact solver of the minimization problem; therefore, our method is effective for solving not only smooth but also non-smooth and nonlinear inverse problems. Under certain conditions on F and $A_F$, based on $\epsilon$-subdifferential, we develop a detailed convergence analysis of the method in Section 3. The numerical results in Section 4 demonstrate the effectiveness of our method.

The rest of this paper is built up as follows. In Section 2, we give some preliminaries on Banach spaces and convex analysis. In Section 3, we formulate our proposed method, and further elaborate well-posedness and the regularization property of the proposed method. Finally, in Section 4, we provide some numerical results to indicate the effectiveness of the method in dealing with smooth as well as non-smooth inverse problems.

2. Preliminaries

In this section, we review some basic concepts on convex analysis and Banach spaces. More details are available in [23,24,25].

Let $\mathcal{X}$ be a Banach space with norm ${\parallel ·\parallel }_{\mathcal{X}}$, and ${\mathcal{X}}^{*}$ is named its dual space. If $x\in \mathcal{X}$ and $x^{*}\in {\mathcal{X}}^{*}$, we denote by ${\langle x^{*},x\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}=x^{*}\left(x\right)$ the duality pair. Given another Banach space $\mathcal{Y}$ and a bounded linear operator A from $\mathcal{X}$ to $\mathcal{Y}$, we write $A^{*}:{\mathcal{Y}}^{*}\to {\mathcal{X}}^{*}$ as its adjoint, i.e., ${\langle A^{*}{y}^{*},x\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}={\langle y^{*},Ax\rangle }_{{\mathcal{Y}}^{*},\mathcal{Y}}$ for any $x\in \mathcal{X}$ and $y^{*}\in {\mathcal{Y}}^{*}$. Set $\mathcal{N}\left(A\right)=\{x\in \mathcal{X}:Ax=0\}$ as the null space of A and

$$\mathcal{N}{\left(A\right)}^{\perp }:=\{\xi \in X^{*}:\langle \xi ,x\rangle =0\mathrm{for}\mathrm{all}x\in \mathcal{N}\left(A\right)\}$$

as the annihilator of $\mathcal{N}\left(A\right)$. When $\mathcal{X}$ is reflexive, there holds $\mathcal{N}{\left(A\right)}^{\perp }:=\overline{\mathcal{R}\left(A^{*}\right)}$, where $\overline{\mathcal{R}\left(A^{*}\right)}$ is the closure of $\mathcal{R}\left(A^{*}\right)$. On a Banach space $\mathcal{X}$, for any $r\in (1,\infty )$, the subdifferential of convex function $x\to {\parallel x\parallel }_{\mathcal{X}}^r/r$ at x is given by

$$J_r^{\mathcal{X}}\left(x\right):=\{\xi \in {\mathcal{X}}^{*}{:\parallel \xi \parallel }_{{\mathcal{X}}^{*}}={\parallel x\parallel }_{\mathcal{X}}^{r-1}\mathrm{and}{\langle \xi ,x\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}={\parallel x\parallel }_{\mathcal{X}}^r\},$$

which is called the duality mapping $J_r^{\mathcal{X}}:\mathcal{X}\to 2^{{\mathcal{X}}^{*}}$ of $\mathcal{X}$ with gauge function $t\to t^{r-1}$.

In addition, the duality mapping $J_r^{\mathcal{X}}$, for each $1<r<\infty$, is single-valued and uniformly continuous on bounded sets when $\mathcal{X}$ is uniformly smooth, in the sense that its modulus of smoothness

$${\rho }_{\mathcal{X}}\left(s\right):=\frac{1}{2}sup\{\parallel \overline{x}{+x\parallel }_{\mathcal{X}}+\parallel \overline{x}{-x\parallel }_{\mathcal{X}}-2:\parallel \overline{x}{\parallel }_{\mathcal{X}}={1,\parallel x\parallel }_{\mathcal{X}}\le s\}$$

satisfies ${lim}_{s\to 0}\frac{{\rho }_{\mathcal{X}}\left(s\right)}{s}=0$.

For a given convex function $\Theta :\mathcal{X}\to (-\infty ,\infty ]$ with effective domain $\mathcal{D}(\Theta ):=\{x\in \mathcal{X}:\Theta (x)<\infty \}$, we call $\Theta$ proper if $\mathcal{D}(\Theta )\ne \varnothing$. For $\epsilon \ge 0$, we define the $\epsilon$-subdifferential of the function $\Theta$ at x by

$${\partial }_{\epsilon }\Theta \left(x\right):=\{\xi \in {\mathcal{X}}^{*}:\Theta \left(\overline{x}\right)\ge \Theta \left(x\right)+{\langle \xi ,\overline{x}-x\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}-\epsilon \mathrm{for}\mathrm{all}\overline{x}\in \mathcal{X}\}.$$

Any element $\xi \in {\partial }_{\epsilon }\Theta \left(x\right)$ is a $\epsilon$-subgradient of $\Theta$ at x [24]. When $\epsilon =0$, then the $\epsilon$-subdifferential of $\Theta$ corresponds to the subdifferential $\partial \Theta$. One can see that ${\partial }_{\epsilon }\Theta \left(x\right)\ne \varnothing$ implies $x\in \mathcal{D}(\Theta )$. If $\Theta$ is lower semicontinuous, then for any $x\in \mathcal{D}(\Theta )$, $\epsilon$-subdifferential ${\partial }_{\epsilon }\Theta \left(x\right)$ is always non-empty for any $\epsilon >0$, see [23] (Theorem 2.4.4).

For $\xi \in {\partial }_{\epsilon }\Theta \left(x\right)$ with $\epsilon \ge 0$,

$$D_{\xi }\Theta (\overline{x},x)=\Theta \left(\overline{x}\right)-\Theta \left(x\right)-{\langle \xi ,\overline{x}-x\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}+\epsilon ,\forall \overline{x}\in \mathcal{X},$$

(9)

which is called the $\epsilon$-Bregman distance, induced by $\Theta$ at x in the direction $\xi$. Obviously, $D_{\xi }\Theta (\overline{x},x)\ge 0$. When $\epsilon =0$, the $\epsilon$-Bregman distance becomes the Bregman distance [24]. A proper convex function $\Theta :\mathcal{X}\to (-\infty ,\infty ]$ is uniformly convex if there exists a strictly increasing continuous function $\phi :[0,\infty )\to [0,\infty )$ with $\phi \left(0\right)=0$ such that

$$\Theta (\lambda \overline{x}+(1-\lambda )x)+c_0\lambda (1-\lambda )\phi (\parallel \overline{x}-x{\parallel }_{\mathcal{X}})\le \lambda \Theta \left(\overline{x}\right)+(1-\lambda )\Theta \left(x\right)$$

for any $\overline{x},x\in \mathcal{X}$ and $\lambda \in [0,1]$. $\Theta$ is p-convex if $\phi \left(t\right)=c_0{t}^p$ for some $c_0>0$ and $p\ge 2$; see [23] (Theorem 3.5.10).

Given a proper, lower semicontinuous, convex function $\Theta :\mathcal{X}\to (-\infty ,\infty ]$, its Legendre-Fenchel conjugate is defined by

$${\Theta }^{*}\left(\xi \right):=\underset{x\in \mathcal{X}}{sup}\{{\langle \xi ,x\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}-\Theta \left(x\right)\},\xi \in {\mathcal{X}}^{*}.$$

It is easily seen that ${\Theta }^{*}$ is also proper, lower semicontinuous, and convex. If, in addition, $\mathcal{X}$ is reflexive, then [23] (Theorem 2.4.2)

$$\xi \in {\partial }_{\epsilon }\Theta \left(x\right)⟺x\in {\partial }_{\epsilon }{\Theta }^{*}\left(\xi \right)⟺\Theta \left(x\right)+{\Theta }^{*}\left(\xi \right)\le {\langle \xi ,x\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}+\epsilon .$$

(10)

The following lemma gives further results of p-convex functionals; refer to [23] (Corollary 3.5.11) and [26] (Lemma 2.1 and Lemma 2.3) for more details.

Lemma 1.

Let $\mathcal{X}$ be a reflexive Banach space and $\Theta :\mathcal{X}\to (-\infty ,\infty ]$ be proper, lower semicontinuous and p-convex with $\phi \left(t\right)=c_0{t}^p$ for some $c_0>0$, $p\ge 2$, $\frac{1}{p}+\frac{1}{p^{*}}=1$. Then,

(i)

If $x\in \mathcal{D}(\Theta )$ and $\xi \in {\partial }_{\epsilon }\Theta \left(x\right)$ for some $\epsilon \ge 0$, we have

$$c_0{\parallel \overline{x}-x\parallel }_{\mathcal{X}}^p\le 2D_{\xi }\Theta (\overline{x},x)+2\epsilon ,\forall \overline{x}\in \mathcal{X}.$$

(11)

(ii)

If $x\in \mathcal{D}(\Theta )$ and $\xi \in {\partial }_{\epsilon }\Theta \left(x\right)$ for some $\epsilon \ge 0$, for $\eta \in {\mathcal{X}}^{*}$, there holds

$${\langle \eta ,x-\nabla {\Theta }^{*}\left(\xi \right)\rangle }_{\mathcal{X},{\mathcal{X}}^{*}}\le \frac{1}{p^{*}{\left(2c_0\right)}^{p^{*}-1}}{\parallel \eta \parallel }_{{\mathcal{X}}^{*}}^{p^{*}}+\epsilon$$

(12)

and therefore

$$\parallel x-\nabla {\Theta }^{*}{\left(\xi \right)\parallel }_{\mathcal{X}}^p\le \frac{p}{2c_0}\epsilon .$$

(13)

(iii)

For $x\in \mathcal{D}(\Theta )$, $\xi \in {\partial }_{\epsilon }\Theta \left(x\right)$ and $\eta \in {\mathcal{X}}^{*}$, there holds

$${\Theta }^{*}\left(\eta \right)-{\Theta }^{*}\left(\xi \right)-{\langle x,\eta -\xi \rangle }_{\mathcal{X},{\mathcal{X}}^{*}}\le \frac{1}{p^{*}{\left(2c_0\right)}^{p^{*}-1}}{\parallel \eta -\xi \parallel }_{{\mathcal{X}}^{*}}^{p^{*}}.$$

(14)

(iv)

$D\left({\Theta }^{*}\right)={\mathcal{X}}^{*}$. ${\Theta }^{*}$ is Fréchet differentiable on ${\mathcal{X}}^{*}$ and its gradient $\nabla {\Theta }^{*}:{\mathcal{X}}^{*}\to \mathcal{X}$ satisfies

$$\parallel \nabla {\Theta }^{*}\left(\eta \right)-\nabla {\Theta }^{*}{\left(\xi \right)\parallel }_{\mathcal{X}}\le {\left(\frac{{\parallel \eta -\xi \parallel }_{{\mathcal{X}}^{*}}}{2c_0}\right)}^{p^{*}-1}$$

(15)

for all $\eta ,\xi \in {\mathcal{X}}^{*}$.

3. The Method

We consider (1), where $F:\mathcal{D}\left(F\right)\subset \mathcal{X}\to \mathcal{Y}$ is a possibly non-smooth nonlinear operator between Banach spaces $\mathcal{X}$ and $\mathcal{Y}$. For carrying out convergence analysis, we pose the following assumptions on the convex function $\Theta$ and the operator F and $A_F$.

Assumption 1.

Θ is a proper, weakly lower semicontinuous and uniformly convex function with $p\ge 2$ satisfying (11) for some $c_0>0$.

Assumption 2.

$\left(a\right)$ There is $\rho >0$ such that $B_{2\rho }\left(x_0\right)\subset \mathcal{D}\left(F\right)$, where $B_{\rho }\left(x_0\right):=\{x\in \mathcal{X}:\parallel x-x_0{\parallel }_{\mathcal{X}}\le \rho \}$. (1) has a solution $x^{*}$ in $\mathcal{D}(\Theta )$ satisfying

$$D_{{\xi }_0}\Theta (x^{*},x_0)\le \frac{1}{4}{c}_0{\rho }^p.$$

(16)

$\left(b\right)$ F is weakly closed on $\mathcal{D}\left(F\right)$, i.e., for any sequence $\left\{x_n\right\}\subset \mathcal{D}\left(F\right)$ with $x_n\to x\in \mathcal{X}$ and $F\left(x_n\right)\rightharpoonup v\in \mathcal{Y}$, there hold $x\in \mathcal{D}\left(F\right)$ and $F\left(x\right)=v$.

$\left(c\right)$ There is a constant $0\le \eta <1$ such that

$$\parallel F\left(\overline{x}\right)-F\left(x\right)-A_F(\overline{x}-x){\parallel }_{\mathcal{Y}}\le \eta {\parallel F\left(\overline{x}\right)-F\left(x\right)\parallel }_{\mathcal{Y}}$$

(17)

for all $\overline{x},x\in B_{2\rho }\left(x_0\right)\cap \mathcal{D}\left(F\right)$. Moreover, there is a constant $\widehat{B}>0$ such that $\parallel A_F\parallel \le \widehat{B}$.

In Assumption 2, the condition (17) can be viewed as a transformation of the tangential cone condition widely used in nonlinear regularization methods [1,25,27]. When $\mathcal{X}$ is reflexive, by the weak closedness of F and the lower semi-continuity and p-convexity of $\Theta$, one can show that $x^{†}$ in (2) exists. Moreover, it has been shown in [28] (Lemma 3.2) that $x^{†}$ is uniquely defined. We note that our generalized inexact Newton-Landweber iteration method (8) in each iteration step involves an inaccurate inner solver of the minimization problem

$$x:=arg\underset{z\in \mathcal{X}}{min}\left\{\Theta \left(z\right)-{\langle \xi ,z\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}\right\}$$

(18)

for any $\xi \in {\mathcal{X}}^{*}$. Concerning the inexact resolution of (8), we make the following assumption.

Assumption 3.

For any given $\epsilon \ge 0$, there is a procedure $S_{\epsilon }:{\mathcal{X}}^{*}\to \mathcal{X}$ for solving (18) such that for any $\xi \in {\mathcal{X}}^{*}$, the element $x:=S_{\epsilon }\left(\xi \right)$ satisfies

$$\Theta \left(x\right)-{\langle \xi ,x\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}\le \underset{z\in \mathcal{X}}{min}\left\{\Theta \left(z\right)-{\langle \xi ,z\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}\right\}+\epsilon .$$

(19)

Moreover, for each $\epsilon \ge 0$, the mapping $S_{\epsilon }:{\mathcal{X}}^{*}\to \mathcal{X}$ is continuous.

We summarize generalized the inexact Newton-Landweber iteration in Algorithm 1.

Algorithm 1 Generalized inexact Newton-Landweber iteration for noisy data

Input: Parameters $\eta <\gamma <1$, ${\mu }_0,{\mu }_1>0$, $k_{max}\ge 1$, $\sigma >0$ and $\tau >1$; a sequence of positive numbers ${\left\{{\epsilon }_{n,k}\right\}}_{k\ge 0,n\ge 0}$ satisfying ${\sum }_{n=0}^{\infty }{\sum }_{k=0}^{\infty }{\epsilon }_{n,k}<\infty$; the operator $A_F$.

Initial guess: $x_0\in \mathcal{X}$ and ${\xi }_0\in \partial \Theta \left(x_0\right)$;

Set $n=0$.

while ${∥F\left(x_n^{\delta }\right)-y^{\delta }∥}_{\mathcal{Y}}>\tau \delta$ do     (i) Set ${\xi }_{n,0}^{\delta }:={\xi }_n^{\delta }$, $x_{n,0}^{\delta }:=x_n^{\delta }$; the inner iterates $\left\{({\xi }_{n,k}^{\delta },x_{n,k}^{\delta })\right\}$ are constructed by $$\begin{gathered} {\xi }_{n,k+1}^{\delta }={\xi }_{n,k}^{\delta }+{\mu }_{n,k}^{\delta }{A}_F^{*}{J}_r^{\mathcal{Y}}\left(s_{n,k}^{\delta }\right), \\ {x}_{n,k+1}^{\delta }=S_{{\epsilon }_{n,k+1}}\left({\xi }_{n,k+1}^{\delta }\right), \end{gathered}$$ (20) where $s_{n,k}^{\delta }=y^{\delta }-F\left(x_n^{\delta }\right)-A_F(x_{n,k}^{\delta }-x_n^{\delta })$ and $${\mu }_{n,k}^{\delta }={\tilde{\mu }}_{n,k}^{\delta }(\parallel s_{n,k}^{\delta }{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{1-\frac{r}{p}}$$ (21) with $${\tilde{\mu }}_{n,k}^{\delta }=min\left\{\frac{{\mu }_0{\parallel s_{n,k}^{\delta }\parallel }_{\mathcal{Y}}^{p(r-1)}}{\parallel A_F^{*}{J}_r^{\mathcal{Y}}\left(s_{n,k}^{\delta }\right){\parallel }_{{\mathcal{X}}^{*}}^p},{\mu }_1\right\}.$$     (ii) Take $k_n^{\delta }=min\{{\tilde{k}}_n^{\delta },k_{max}\}$, where $k_{max}\ge 1$ is a given integer and ${\tilde{k}}_n^{\delta }\ge 1$ is the first integer satisfying $$\parallel s_{n,k}^{\delta }{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k}<(\gamma \parallel y^{\delta }-F\left(x_n^{\delta }\right){{\parallel }_{\mathcal{Y}})}^p.$$ (22) Update the outer iterates by $${\xi }_{n+1}^{\delta }={\xi }_{n,k_n^{\delta }}^{\delta }\mathrm{and}{x}_{n+1}^{\delta }=x_{n,k_n^{\delta }}^{\delta }.$$ Set $n:=n+1$.

end while

We denote by $n_{\delta }=n(\delta ,y^{\delta })$ the outer stopping index such that the discrepancy principle is fulfilled, i.e., $$\parallel F\left(x_{n_{\delta }}^{\delta }\right)-y^{\delta }{\parallel }_{\mathcal{Y}}\le \tau \delta <{\parallel F\left(x_n^{\delta }\right)-y^{\delta }\parallel }_{\mathcal{Y}},0\le n<n_{\delta }.$$ (23)

Output: An approximate solution $x_{n_{\delta }}^{\delta }$ of (1).

By definition of $x_{n,k}^{\delta }$ in (20), we have

$$\Theta \left(x_{n,k}^{\delta }\right)-{〈{\xi }_{n,k}^{\delta },x_{n,k}^{\delta }〉}_{{\mathcal{X}}^{*},\mathcal{X}}\le \Theta \left(x\right)-{〈{\xi }_{n,k}^{\delta },x〉}_{{\mathcal{X}}^{*},\mathcal{X}}+{\epsilon }_{n,k},\forall x\in \mathcal{X},$$

which implies that ${\xi }_{n,k}^{\delta }\in {\partial }_{{\epsilon }_{n,k}}\Theta \left(x_{n,k}^{\delta }\right)$. This fact will be used in the forthcoming theoretical analysis. The following lemma shows that Algorithm 1 is well-defined.

Lemma 2.

Let $\mathcal{X}$ be reflexive and $\mathcal{Y}$ be uniformly smooth. Assume that Assumption 1, 2 and 3 hold. Let ${\left\{{\epsilon }_{n,k}\right\}}_{n\ge 0,k\ge 0}$ be a sequence of positive numbers satisfying

$$\sum _{n=0}^{\infty }\sum _{k=0}^{\infty }{\epsilon }_{n,k}<\frac{1}{16}{c}_0{\rho }^p.$$

(24)

Let $\beta >1$, $\eta <\gamma <1$, ${\mu }_0>0$, and $\tau >1$ be chosen such that

$$c_1:=\frac{1}{\beta }-\frac{\eta }{\gamma }-\frac{1+\eta }{\tau \gamma }-\frac{p-1}{p}{\left(\frac{{\mu }_0}{2c_0}\right)}^{\frac{1}{p-1}}>0,$$

(25)

then, there holds:

(i)

for each $0\le n<n_{\delta }$, ${\tilde{k}}_n^{\delta }<\infty$ and $x_{n,k}^{\delta }\in B_{2\rho }\left(x_0\right)$ for all $0\le k\le k_n^{\delta }$;

(ii)

Algorithm 1 terminates after $n_{\delta }<\infty$ iteration steps;

(iii)

for any solution $\widehat{x}$ of (1) in $B_{2\rho }\left(x_0\right)\cap \mathcal{D}(\Theta )$, we have

$$D_{{\xi }_{n+1}^{\delta }}\Theta (\widehat{x},x_{n+1}^{\delta })-D_{{\xi }_n^{\delta }}\Theta (\widehat{x},x_n^{\delta })\le 3\sum _{k=0}^{k_n^{\delta }-1}{\epsilon }_{n,k}$$

for $0\le n<n_{\delta }$. Here we may take ${\epsilon }_0=0$ since ${\xi }_0\in \partial \Theta \left(x_0\right)$ and ${\epsilon }_n={\epsilon }_{n,0}$.

Proof. 

We first show that if $x_n^{\delta }\in B_{2\rho }\left(x_0\right)$ for some $0\le n<n_{\delta }$, then

$$\begin{array}{cc}\hfill D_{{\xi }_{n,k+1}^{\delta }}\Theta (\widehat{x},x_{n,k+1}^{\delta })& -D_{{\xi }_{n,k}^{\delta }}\Theta (\widehat{x},x_{n,k}^{\delta })\hfill \\ \hfill & \le -c_1{\mu }_{n,k}^{\delta }(\parallel s_{n,k}^{\delta }{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{r}{p}}+2{\epsilon }_{n,k}+{\epsilon }_{n,k+1}\hfill \end{array}$$

(26)

for $0\le k<k_n^{\delta }$. Using the definition of $\epsilon$-Bregman distance (9), we can arrive at, for $0\le k<k_n^{\delta }$,

$$\begin{array}{cc}\hfill D_{{\xi }_{n,k+1}^{\delta }}\Theta (\widehat{x},x_{n,k+1}^{\delta })-D_{{\xi }_{n,k}^{\delta }}\Theta (\widehat{x},x_{n,k}^{\delta })& =[\Theta \left(x_{n,k}^{\delta }\right)-{\langle {\xi }_{n,k}^{\delta },x_{n,k}^{\delta }\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}-{\epsilon }_{n,k}]\hfill \\ \hfill & +[{\langle {\xi }_{n,k+1}^{\delta },x_{n,k+1}^{\delta }\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}-\Theta \left(x_{n,k+1}^{\delta }\right)]\hfill \\ \hfill & -{\langle {\xi }_{n,k+1}^{\delta }-{\xi }_{n,k}^{\delta },\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}+{\epsilon }_{n,k+1}.\hfill \end{array}$$

Using (10) and the definition of ${\Theta }^{*}$, we further have

$$\begin{array}{c}{D}_{{\xi }_{n,k+1}^{\delta }}\Theta (\widehat{x},x_{n,k+1}^{\delta })-D_{{\xi }_{n,k}^{\delta }}\Theta (\widehat{x},x_{n,k}^{\delta })\hfill \\ \le {\Theta }^{*}\left({\xi }_{n,k+1}^{\delta }\right)-{\Theta }^{*}\left({\xi }_{n,k}^{\delta }\right)-{\langle {\xi }_{n,k+1}^{\delta }-{\xi }_{n,k}^{\delta },\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}+{\epsilon }_{n,k+1}\hfill \\ ={\Theta }^{*}\left({\xi }_{n,k+1}^{\delta }\right)-{\Theta }^{*}\left({\xi }_{n,k}^{\delta }\right)-{\langle {\xi }_{n,k+1}^{\delta }-{\xi }_{n,k}^{\delta },\nabla {\Theta }^{*}\left({\xi }_{n,k}^{\delta }\right)\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}\hfill \\ +{\langle {\xi }_{n,k+1}^{\delta }-{\xi }_{n,k}^{\delta },\nabla {\Theta }^{*}\left({\xi }_{n,k}^{\delta }\right)-x_{n,k}^{\delta }\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}\hfill \\ +{\langle {\xi }_{n,k+1}^{\delta }-{\xi }_{n,k}^{\delta },x_{n,k}^{\delta }-\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}+{\epsilon }_{n,k+1}.\hfill \end{array}$$

Since $\Theta$ is p-convex, we may use (12), (14) and the definition of ${\xi }_{n,k+1}^{\delta }$ to derive that

$$\begin{array}{c}{D}_{{\xi }_{n,k+1}^{\delta }}\Theta (\widehat{x},x_{n,k+1}^{\delta })-D_{{\xi }_{n,k}^{\delta }}\Theta (\widehat{x},x_{n,k}^{\delta })\le \frac{2}{p^{*}{\left(2c_0\right)}^{p^{*}-1}}{\left({\mu }_{n,k}^{\delta }\right)}^{p^{*}}{\parallel A_F^{*}{J}_r^{\mathcal{Y}}\left(s_{n,k}^{\delta }\right)\parallel }_{{\mathcal{X}}^{*}}^{p^{*}}\hfill \\ +{\mu }_{n,k}^{\delta }{\langle J_r^{\mathcal{Y}}\left(s_{n,k}^{\delta }\right),A_F(x_{n,k}^{\delta }-\widehat{x})\rangle }_{{\mathcal{Y}}^{*},\mathcal{Y}}+{\epsilon }_{n,k}+{\epsilon }_{n,k+1}.\hfill \end{array}$$

Please note that $A_F(x_{n,k}^{\delta }-\widehat{x})=-s_{n,k}^{\delta }+[y^{\delta }-F\left(x_n^{\delta }\right)-A_F(\widehat{x}-x_n^{\delta })],$ in view of (3), Assumption 2(c) and the property of $J_r^{\mathcal{Y}}$, we further have

$$\begin{array}{c}{D}_{{\xi }_{n,k+1}^{\delta }}\Theta (\widehat{x},x_{n,k+1}^{\delta })-D_{{\xi }_{n,k}^{\delta }}\Theta (\widehat{x},x_{n,k}^{\delta })\hfill \\ \le \frac{2}{p^{*}{\left(2c_0\right)}^{p^{*}-1}}{\left({\mu }_{n,k}^{\delta }\right)}^{p^{*}}\parallel A_F^{*}{J}_r^{\mathcal{Y}}\left(s_{n,k}^{\delta }\right){\parallel }_{{\mathcal{X}}^{*}}^{p^{*}}-{\mu }_{n,k}^{\delta }{\parallel s_{n,k}^{\delta }\parallel }_{\mathcal{Y}}^r\hfill \\ +{\mu }_{n,k}^{\delta }\parallel s_{n,k}^{\delta }{\parallel }_{\mathcal{Y}}^{r-1}((1+\eta )\delta +\eta \parallel y^{\delta }-F\left(x_n^{\delta }\right){\parallel }_{\mathcal{Y}})+{\epsilon }_{n,k}+{\epsilon }_{n,k+1}.\hfill \end{array}$$

(27)

By the definition of ${\mu }_{n,k}^{\delta }$, it follows that

$$\begin{array}{c}{\left({\mu }_{n,k}^{\delta }\right)}^{p^{*}}\parallel A_F^{*}{J}_r^{\mathcal{Y}}\left(s_{n,k}^{\delta }\right){\parallel }_{{\mathcal{X}}^{*}}^{p^{*}}={\mu }_{n,k}^{\delta }{\left({\mu }_{n,k}^{\delta }\right)}^{p^{*}-1}{\parallel A_F^{*}{J}_r^{\mathcal{Y}}\left(s_{n,k}^{\delta }\right)\parallel }_{{\mathcal{X}}^{*}}^{p^{*}}\hfill \\ \le {\left({\mu }_0\right)}^{p^{*}-1}{\mu }_{n,k}^{\delta }\parallel s_{n,k}^{\delta }{\parallel }_{\mathcal{Y}}^{p^{*}(r-1)}(\parallel s_{n,k}^{\delta }{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{(p-r)(p^{*}-1)}{p}}\hfill \\ \le {\left({\mu }_0\right)}^{p^{*}-1}{\mu }_{n,k}^{\delta }(\parallel s_{n,k}^{\delta }{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{r}{p}}.\hfill \end{array}$$

(28)

Using again the definition of ${\mu }_{n,k}^{\delta }$, together with (22) and (23), we derive that

$$\begin{array}{c}{\mu }_{n,k}^{\delta }\parallel s_{n,k}^{\delta }{\parallel }_{\mathcal{Y}}^{r-1}((1+\eta )\delta +\eta \parallel y^{\delta }-F\left(x_n^{\delta }\right){\parallel }_{\mathcal{Y}})\hfill \\ \le \frac{1+\eta }{\tau }{\mu }_{n,k}^{\delta }\parallel s_{n,k}^{\delta }{\parallel }_{\mathcal{Y}}^{r-1}\parallel y^{\delta }-F\left(x_n^{\delta }\right){\parallel }_{\mathcal{Y}}+\eta {\mu }_{n,k}^{\delta }\parallel s_{n,k}^{\delta }{\parallel }_{\mathcal{Y}}^{r-1}{\parallel y^{\delta }-F\left(x_n^{\delta }\right)\parallel }_{\mathcal{Y}}\hfill \\ \le \frac{1+\eta }{\gamma \tau }{\mu }_{n,k}^{\delta }\parallel s_{n,k}^{\delta }{\parallel }_{\mathcal{Y}}^{r-1}(\parallel s_{n,k}^{\delta }{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k}{)}^{\frac{1}{p}}+\frac{\eta }{\gamma }{\mu }_{n,k}^{\delta }\parallel s_{n,k}^{\delta }{\parallel }_{\mathcal{Y}}^{r-1}(\parallel s_{n,k}^{\delta }{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{1}{p}}\hfill \\ \le (\frac{1+\eta }{\gamma \tau }+\frac{\eta }{\gamma }){\mu }_{n,k}^{\delta }(\parallel s_{n,k}^{\delta }{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{r}{p}}.\hfill \end{array}$$

(29)

Analogous to the proof of [26] (Lemma 3.1), we can show that

$${\mu }_{n,k}^{\delta }\parallel s_{n,k}^{\delta }{\parallel }_{\mathcal{Y}}^r\ge \frac{1}{\beta }{\mu }_{n,k}^{\delta }(\parallel s_{n,k}^{\delta }{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{r}{p}}-\kappa {\tilde{\mu }}_{n,k}^{\delta }\sigma {\epsilon }_{n,k},$$

where $\sigma >0$ and $\kappa {\mu }_1\sigma \le 1$ with

$$\kappa =\left\{\begin{array}{c}1,ifp\ge r,\hfill \\ {({\beta }^{\frac{p}{r-p}}-1)}^{\frac{p-r}{p}},ifp<r.\hfill \end{array}\right.$$

By inserting the above inequality, (28), (29) into (27) and using ${\tilde{\mu }}_{n,k}^{\delta }\le {\mu }_1$, we obtain the estimate

$$\begin{array}{cc}\hfill & D_{{\xi }_{n,k+1}^{\delta }}\Theta (\widehat{x},x_{n,k+1}^{\delta })-D_{{\xi }_{n,k}^{\delta }}\Theta (\widehat{x},x_{n,k}^{\delta })\hfill \\ \hfill & \le -c_1{\mu }_{n,k}^{\delta }(\parallel s_{n,k}^{\delta }{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{r}{p}}+\kappa {\tilde{\mu }}_{n,k}^{\delta }\sigma {\epsilon }_{n,k}+{\epsilon }_{n,k}+{\epsilon }_{n,k+1}\hfill \\ \hfill & \le -c_1{\mu }_{n,k}^{\delta }(\parallel s_{n,k}^{\delta }{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{r}{p}}+2{\epsilon }_{n,k}+{\epsilon }_{n,k+1},\hfill \end{array}$$

which yields the assertion (26). By summing (26) over k from $k=0$ to $k=l$ for any $l<{\tilde{k}}_n^{\delta }$, we have

$$c_1\sum _{k=0}^l{\mu }_{n,k}^{\delta }(\parallel s_{n,k}^{\delta }{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{r}{p}}\le D_{{\xi }_n^{\delta }}\Theta (\widehat{x},x_n^{\delta })-D_{{\xi }_{n,l+1}^{\delta }}\Theta (\widehat{x},x_{n,l+1}^{\delta })+3\sum _{k=0}^l{\epsilon }_{n,k}.$$

(30)

In view of Assumption 2$\left(c\right)$ and the definition of ${\mu }_{n,k}^{\delta }$, we have

$${\mu }_{n,k}^{\delta }\ge c_2(\parallel s_{n,k}^{\delta }{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{1-\frac{r}{p}}\mathrm{with}{c}_2:=min\{{\mu }_0{\widehat{B}}^{-p},{\mu }_1\},$$

which together with (22) and (23) gives

$${\mu }_{n,k}^{\delta }(\parallel s_{n,k}^{\delta }{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k}{)}^{\frac{r}{p}}\ge c_2(\parallel s_{n,k}^{\delta }{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})\ge c_2{\gamma }^p{\parallel y^{\delta }-F\left(x_n^{\delta }\right)\parallel }_{\mathcal{Y}}^p\ge c_2{\gamma }^p{\tau }^p{\delta }^p.$$

(31)

By inserting (31) into (30), there holds

$$c_1{c}_2{\gamma }^p{\tau }^p{\delta }^p(l+1)\le D_{{\xi }_n^{\delta }}\Theta (\widehat{x},x_n^{\delta })+3\sum _{k=0}^l{\epsilon }_{n,k}<\infty ,\forall 0\le l<{\tilde{k}}_n^{\delta },$$

which implies that ${\tilde{k}}_n^{\delta }<\infty$. By taking $l=k_n^{\delta }-1(k_n^{\delta }=min\{{\tilde{k}}_n^{\delta },k_{max}\})$ in (30), we can obtain

$$c_1\sum _{k=0}^{k_n^{\delta }-1}{\mu }_{n,k}^{\delta }(\parallel s_{n,k}^{\delta }{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{r}{p}}\le D_{{\xi }_n^{\delta }}\Theta (\widehat{x},x_n^{\delta })-D_{{\xi }_{n+1}^{\delta }}\Theta (\widehat{x},x_{n+1}^{\delta })+3\sum _{k=0}^{k_n^{\delta }-1}{\epsilon }_{n,k},$$

(32)

which yields assertion (iii).

In view of (16) and (11), we have $\parallel x_0-x^{†}{\parallel }_{\mathcal{X}}\le \rho$, i.e., $x^{†}$ is a solution of (1) in $B_{\rho }\left(x_0\right)$. Then, by taking (26) with $\widehat{x}=x^{†}$, we can inductively deduce that

$$D_{{\xi }_{n,k}^{\delta }}\Theta (x^{†},x_{n,k}^{\delta })\le D_{{\xi }_0}\Theta (x^{†},x_0)+3\sum _{n=0}^{\infty }\sum _{k=0}^{k_n^{\delta }-1}{\epsilon }_{n,k}.$$

Using (11) and (24), together with (16), we can obtain

$$\begin{array}{cc}\hfill c_0{\parallel x_{n,k}^{\delta }-x^{†}\parallel }_{\mathcal{X}}^p& \le 2D_{{\xi }_{n,k}^{\delta }}\Theta (x^{†},x_{n,k}^{\delta })+2{\epsilon }_{n,k}\le 2D_{{\xi }_0}\Theta (x^{†},x_0)+8\sum _{n=0}^{\infty }\sum _{k=0}^{k_n^{\delta }-1}{\epsilon }_{n,k}\hfill \\ \hfill & \le \frac{1}{2}{c}_0{\rho }^p+\frac{1}{2}{c}_0{\rho }^p=c_0{\rho }^p,\hfill \end{array}$$

which gives $\parallel x_{n,k}^{\delta }-x^{†}{\parallel }_{\mathcal{X}}\le \rho$, and thus $\parallel x_{n,k}^{\delta }-x_0{\parallel }_{\mathcal{X}}\le 2\rho$, i.e., $x_{n,k}^{\delta }\in B_{2\rho }\left(x_0\right)$ for all $0\le n<n_{\delta }$ and $0\le k\le k_n^{\delta }$.

Finally, we prove that $n_{\delta }<\infty$. By summing (32) over n from $n=0$ to $n=m$ for any $m<n_{\delta }$, and using (31), we can further obtain

$$\begin{array}{cc}\hfill c_1{c}_2{\gamma }^p{\tau }^p{\delta }^p(m+1)& \le c_1{c}_2\sum _{n=0}^m\sum _{k=0}^{k_n^{\delta }-1}{\mu }_{n,k}^{\delta }(\parallel s_{n,k}^{\delta }{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{r}{p}}\hfill \\ \hfill & \le D_{{\xi }_0}\Theta (\widehat{x},x_0)+3\sum _{n=0}^{\infty }\sum _{k=0}^{k_n^{\delta }-1}{\epsilon }_{n,k}<\infty ,\hfill \end{array}$$

which yields $n_{\delta }<\infty$.  □

3.1. Convergence Analysis

To carry out the convergence analysis of Algorithm 1, it is necessary to consider its counterpart with exact data. The algorithm for the noise-free case is reformulated as follows. Noting that the inner iteration number $k_n$ might not be unique (see Lemma 3 below), using different integer $k_n$ to update the outer iteration may lead to different iterative sequences $\left\{\left({\xi }_n,x_n\right)\right\}$. Next, let ${\Gamma }_{\gamma ,{\mu }_0,{\mu }_1}\left({\xi }_0,x_0\right)$ denote the set of all possible sequences $\left\{\left({\xi }_n,x_n\right)\right\}$ generated by Algorithm 2 from $\left({\xi }_0,x_0\right)$ with $k_n$ chosen as in (35).

Algorithm 2 Generalized inexact Newton-Landweber iteration for exact data

Input: Parameters $\eta <\gamma <1$, ${\mu }_0,{\mu }_1>0$, $k_{max}\ge 1$ and $\sigma >0$; a sequence of positive numbers ${\left\{{\epsilon }_{n,k}\right\}}_{k\ge 0,n\ge 0}$ satisfying ${\sum }_{n=0}^{\infty }{\sum }_{k=0}^{\infty }{\epsilon }_{n,k}<\infty$; the operator $A_F$.

Initial guess:$x_0\in \mathcal{X}$ and ${\xi }_0\in \partial \Theta \left(x_0\right)$;

Set $n=0.$

Repeat:     (i) Assuming that $({\xi }_n,x_n)$ is constructed, we define the inner iterates $\left\{({\xi }_{n,k},x_{n,k})\right\}$ by setting ${\xi }_{n,0}={\xi }_n$, $x_{n,0}=x_n$ and $$\begin{gathered} {\xi }_{n,k+1}={\xi }_{n,k}+{\mu }_{n,k}{A}_F^{*}{J}_r^{\mathcal{Y}}\left(s_{n,k}\right), \\ {x}_{n,k+1}=S_{{\epsilon }_{n,k+1}}\left({\xi }_{n,k+1}\right), \end{gathered}$$ (33) where $s_{n,k}=y-F\left(x_n\right)-A_F(x_{n,k}-x_n)$ and $$\begin{gathered} {\mu }_{n,k}=\left\{\begin{array}{c}{\tilde{\mu }}_{n,k}(\parallel s_{n,k}{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{1-\frac{r}{p}},ifF\left(x_n\right)\ne y, \\ 0,otherwise\hfill \end{array}\right. \end{gathered}$$ (34) with $${\tilde{\mu }}_{n,k}=min\left\{\frac{{\mu }_0{\parallel s_{n,k}\parallel }_{\mathcal{Y}}^{p(r-1)}}{\parallel A_F^{*}{J}_r^{\mathcal{Y}}\left(s_{n,k}\right){\parallel }_{{\mathcal{X}}^{*}}^p},{\mu }_1\right\}.$$     (ii) Determine an integer $1\le k_n\le k_{max}$ satisfying $$\parallel s_{n,k}{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k}\ge (\gamma \parallel y-F\left(x_n\right){{\parallel }_{\mathcal{Y}})}^p,\forall 0\le k<k_n$$ (35) with given $k_{max}\ge 1$ and define $${\xi }_{n+1}={\xi }_{n,k_n}\mathrm{and}{x}_{n+1}=x_{n,k_n}.$$ Set $n:=n+1.$

Until stopping criterion is satisfied.

The following lemma shows the well-definedness of $\left\{\left({\xi }_n,x_n\right)\right\}\in {\Gamma }_{\gamma ,{\mu }_0,{\mu }_1}\left({\xi }_0,x_0\right)$.

Lemma 3.

Let all the conditions in Lemma 2 hold. Then, for any sequence $\left\{\left({\xi }_n,x_n\right)\right\}\in {\Gamma }_{\gamma ,{\mu }_0,{\mu }_1}\left({\xi }_0,x_0\right)$, $x_{n,k}\in B_{2\rho }\left(x_0\right)$ for all $n\ge 0$ and $k_n$ is well-defined. Moreover, for any solution $\widehat{x}$ of (1) in $B_{2\rho }\left(x_0\right)\cap \mathcal{D}\left(\Theta \right)$, there hold

$$c_3\sum _{k=0}^{k_n-1}{\mu }_{n,k}(\parallel s_{n,k}{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{r}{p}}\le D_{{\xi }_n}\Theta \left(\widehat{x},x_n\right)-D_{{\xi }_{n+1}}\Theta \left(\widehat{x},x_{n+1}\right)+3\sum _{k=0}^{k_n-1}{\epsilon }_{n,k}$$

(36)

and

$$\sum _{n=0}^{\infty }\sum _{k=0}^{k_n-1}(\parallel s_{n,k}{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})<\infty$$

(37)

with $c_3:=\frac{1}{\beta }-\frac{\eta }{\gamma }-\frac{p-1}{p}{\left(\frac{{\mu }_0}{2c_0}\right)}^{\frac{1}{p-1}}>0$ for all $n\ge 0$.

Proof. 

Proceeding as in the proof of Lemma 3.1 in [6], we can immediately obtain (36). Summing (36) over n from $n=0$ to $n=\infty$, we establish that

$$\begin{array}{c}\hfill c_3\sum _{n=0}^{\infty }\sum _{k=0}^{k_n-1}{\mu }_{n,k}(\parallel s_{n,k}{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{r}{p}}\le D_{{\xi }_0}\Theta (\widehat{x},x_0)+3\sum _{n=0}^{\infty }\sum _{k=0}^{k_n-1}{\epsilon }_{n,k}<\infty .\end{array}$$

(38)

By the definition of ${\mu }_{n,k}$, we have ${\mu }_{n,k}(\parallel s_{n,k}{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k}{)}^{\frac{r}{p}}\ge c_2(\parallel s_{n,k}{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k}),$ from which we can obtain (37).

Next, we show that there exists $1\le k_n\le k_{max}$ such that (35) holds. If $F\left(x_n\right)=y$ for some n, then (34) yields ${\mu }_{n,k}=0$ for $k\ge 0$. Then it follows from the definition of ${\xi }_{n,k+1}$ in (33) that ${\xi }_{n,k+1}={\xi }_{n,k}={\xi }_n$ for all $k\ge 0$. Since $S_{\epsilon }$ is continuous, we have

$$x_{n,k+1}=S_{{\epsilon }_{n,k+1}}\left({\xi }_{n,k+1}\right)=S_{{\epsilon }_{n,k}}\left({\xi }_{n,k}\right)=x_{n,k}=x_n,k\ge 0.$$

(39)

Therefore, (35) holds for any $1\le k_n\le k_{max}$. Now, suppose that $F\left(x_n\right)\ne y$, in view of the definition of ${\mu }_{n,k}$ and (35), there holds, for $0\le k<k_n$,

$${\mu }_{n,k}(\parallel s_{n,k}{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{r}{p}}\ge c_2{\gamma }^p{∥y-F\left(x_n\right)∥}_{\mathcal{Y}}^p.$$

(40)

Combining with (36), we can obtain

$$\begin{array}{c}{c}_3{c}_2{\gamma }^p{k}_n{∥y-F\left(x_n\right)∥}_{\mathcal{Y}}^p\le c_3{\displaystyle \sum _{k=0}^{k_n-1}}{\mu }_{n,k}(\parallel s_{n,k}{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{r}{p}}\hfill \\ \le D_{{\xi }_n}\Theta \left(\widehat{x},x_n\right)-D_{{\xi }_{n+1}}\Theta \left(\widehat{x},x_{n+1}\right)+3\sum _{k=0}^{k_n-1}{\epsilon }_{n,k}\hfill \\ \le D_{{\xi }_n}\Theta \left(\widehat{x},x_n\right)+3\sum _{k=0}^{k_n-1}{\epsilon }_{n,k}<\infty ,\hfill \end{array}$$

which suggests that $k_n$ has an upper bound, denoted by ${\tilde{k}}_n$. Therefore, (35) holds for any $k_n$ satisfying $1\le k_n\le min\left\{{\tilde{k}}_n,k_{max}\right\}$.  □

We next show the convergence of the sequence $\left\{({\xi }_n,x_n)\right\}$ in ${\Gamma }_{\gamma ,{\mu }_0,{\mu }_1}({\xi }_0,x_0)$. The next proposition will be useful for the forthcoming convergence analysis.

Proposition 1.

Let all the conditions in Lemma 2 hold. Then, for any solution $\widehat{x}$ of (1) in $B_{2\rho }\left(x_0\right)\cap \mathcal{D}\left(\theta \right)$, for any $\left\{({\xi }_n,x_n)\right\}\in {\Gamma }_{\gamma ,{\mu }_0,{\mu }_1}({\xi }_0,x_0)$, the sequence $\{D_{{\xi }_n}\Theta \left(\widehat{x},x_n\right)\}$ is convergent.

Proof. 

Please refer to [26] (Lemma 3.3) for detailed proof.  □

Theorem 1.

Let $\mathcal{X}$ be reflexive and $\mathcal{Y}$ be uniformly smooth. Assume that Assumption 1, 2 and 3 hold. Then, for any sequence $\left\{({\xi }_n,x_n)\right\}\in {\Gamma }_{\gamma ,{\mu }_0,{\mu }_1}({\xi }_0,x_0)$, there is a solution $x^{*}$ of (1.1) in $B_{2\rho }\left(x_0\right)\cap \mathcal{D}(\Theta )$ such that

$$\underset{n\to \infty }{lim}{\parallel x_n-x^{*}\parallel }_{\mathcal{X}}=0and\underset{n\to \infty }{lim}{D}_{{\xi }_n}\Theta (x^{*},x_n)=0.$$

Moreover, there holds $x^{*}=x^{†}$.

Proof. 

We first prove that $\left\{x_n\right\}$ has a convergent subsequence. By using (38), (40) and the fact $k_n\ge 1$, we have

$$c_3{c}_2{\gamma }^p\sum _{n=0}^{\infty }{∥y-F\left(x_n\right)∥}_{\mathcal{Y}}^p\le D_{{\xi }_0}\Theta \left(\widehat{x},x_0\right)+3\sum _{n=0}^{\infty }\sum _{k=0}^{k_n-1}{\epsilon }_{n,k}<\infty .$$

Consequently, ${lim}_{n\to \infty }{\parallel F\left(x_n\right)-y\parallel }_{\mathcal{Y}}=0$. If $F\left(x_n\right)=y$ for some n, by using the argument in deriving (39) repeatedly, we can show that $F\left(x_m\right)=y$ for all $m\ge n$. Therefore, we can find a strictly increasing subsequence $\left\{n_l\right\}$ such that $n_0=0$, and $n_l$ with $l\ge 0$ being the first integer satisfying

$$n_l\ge n_{l-1}+1\mathrm{and}\parallel F\left(x_{n_l}\right){-y\parallel }_{\mathcal{Y}}\le {\parallel F\left(x_{n_{l-1}}\right)-y\parallel }_{\mathcal{Y}}.$$

For such a sequence $\left\{n_l\right\}$, it is easy to see that

$$\parallel F\left(x_{n_l}\right){-y\parallel }_{\mathcal{Y}}\le {\parallel F\left(x_n\right)-y\parallel }_{\mathcal{Y}},0\le n\le n_l.$$

(41)

Next, we show that $\left\{x_{n_l}\right\}$ is a Cauchy sequence. For any solution $\widehat{x}$ of (1) in $B_{2\rho }\left(x_0\right)\cap \mathcal{D}\left(\theta \right)$, by the definition of $\epsilon$-Bregman distance, we have, for $0\le j<l<\infty$

$$D_{{\xi }_{n_j}}\Theta (x_{n_l},x_{n_j})=D_{{\xi }_{n_j}}\Theta (\widehat{x},x_{n_j})-D_{{\xi }_{n_l}}\Theta (\widehat{x},x_{n_l})+{\langle {\xi }_{n_l}-{\xi }_{n_j},x_{n_l}-\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}+{\epsilon }_{n_l}.$$

(42)

From Proposition 1, one can see that $D_{{\xi }_{n_j}}\Theta (\widehat{x},x_{n_j})-D_{{\xi }_{n_l}}\Theta (\widehat{x},x_{n_l})$ tends to zero as $l,j\to \infty$. We next estimate the term ${\langle {\xi }_{n_l}-{\xi }_{n_j},x_{n_l}-\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}$ for $0\le j<l<\infty$. By using the fact that ${\xi }_n={\xi }_{n,0}$ and ${\xi }_{n+1}={\xi }_{n,k_n}$, we have

$$\begin{array}{cc}\hfill {\langle {\xi }_{n_l}-{\xi }_{n_j},x_{n_l}-\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}& =\sum _{n=n_j}^{n_l-1}{\langle {\xi }_{n+1}-{\xi }_n,x_{n_l}-\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}\hfill \\ \hfill & =\sum _{n=n_j}^{n_l-1}{\langle {\xi }_{n,k_n}-{\xi }_{n,0},x_{n_l}-\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}.\hfill \end{array}$$

(43)

By the definition of ${\xi }_{n,k}$, ${\mu }_{n,k}$ and the property of $J_r^{\mathcal{Y}}$, we further have

$$\begin{array}{cc}\hfill \left|{\langle {\xi }_{n,k_n}-{\xi }_{n,0},x_{n_l}-\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}\right|& =\left|\sum _{k=0}^{k_n-1}{\langle {\xi }_{n,k+1}-{\xi }_{n,k},x_{n_l}-\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}\right|\hfill \\ \hfill & =\left|\sum _{k=0}^{k_n-1}{\mu }_{n,k}{\langle J_r^{\mathcal{Y}}\left(s_{n,k}\right),A_F(x_{n_l}-\widehat{x})\rangle }_{{\mathcal{Y}}^{*},\mathcal{Y}}\right|\hfill \\ \hfill & \le \sum _{k=0}^{k_n-1}{\mu }_{n,k}\parallel s_{n,k}{\parallel }_{\mathcal{Y}}^{r-1}{\parallel A_F(x_{n_l}-\widehat{x})\parallel }_{\mathcal{Y}}\hfill \\ \hfill & \le \sum _{k=0}^{k_n-1}{\mu }_1(\parallel s_{n,k}{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k}{)}^{1-\frac{1}{p}}{\parallel A_F(x_{n_l}-\widehat{x})\parallel }_{\mathcal{Y}}.\hfill \end{array}$$

(44)

Using (41), (35) and Assumption 2(c), we have

$$\begin{array}{cc}\hfill \parallel A_F(x_{n_l}-\widehat{x}){\parallel }_{\mathcal{Y}}& \le \parallel A_F(x_{n_l}-x_n){\parallel }_{\mathcal{Y}}+{\parallel A_F(x_n-\widehat{x})\parallel }_{\mathcal{Y}}\hfill \\ \hfill & \le (1+\eta )(\parallel F\left(x_{n_l}\right)-F\left(x_n\right){\parallel }_{\mathcal{Y}}+\parallel F\left(x_n\right)-y{\parallel }_{\mathcal{Y}})\hfill \\ \hfill & \le (1+\eta )(\parallel F\left(x_{n_l}\right)-y{\parallel }_{\mathcal{Y}}+2\parallel F\left(x_n\right)-y{\parallel }_{\mathcal{Y}})\hfill \\ \hfill & \le 3(1+\eta )\parallel F\left(x_n\right){-y\parallel }_{\mathcal{Y}}\hfill \\ \hfill & \le \frac{3(1+\eta )}{\gamma }(\parallel s_{n,k}{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{\frac{1}{p}}\hfill \end{array}$$

for $0\le n\le n_l$ and $0\le k<k_n$. By inserting the above inequality into (44), there holds

$$\begin{array}{cc}\hfill \left|{\langle {\xi }_{n,k_n}-{\xi }_{n,0},x_{n_l}-\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}\right|& \le \frac{3(1+\eta ){\mu }_1}{\gamma }\sum _{k=0}^{k_n-1}(\parallel s_{n,k}{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k}).\hfill \end{array}$$

Combining with (43), we can deduce that

$$|{\langle {\xi }_{n_l}-{\xi }_{n_j},x_{n_l}-\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}|\le \frac{3(1+\eta ){\mu }_1}{\gamma }\sum _{n=n_j}^{n_l-1}\sum _{k=0}^{k_n-1}(\parallel s_{n,k}{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k}).$$

Furthermore, together with (37), it follows that

$$\underset{j\to \infty }{lim}\underset{l\ge j}{sup}\left|{\langle {\xi }_{n_l}-{\xi }_{n_j},x_{n_l}-\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}\right|=0.$$

(45)

Since ${lim}_{n\to \infty }{\epsilon }_n=0$, we can conclude from (42) that

$$D_{{\xi }_{n_j}}\Theta (x_{n_l},x_{n_j})\to 0\mathrm{as}j,l\to \infty .$$

By the p-convexity of $\Theta$, there also holds $\parallel x_{n_l}-x_{n_j}{\parallel }_{\mathcal{X}}\to 0\mathrm{as}j,l\to \infty .$ Therefore, $\left\{x_{n_l}\right\}$ is a Cauchy sequence in $\mathcal{X}$. There exists some $x^{*}\in \mathcal{X}$ such that $x_{n_l}\to x^{*}$ as $l\to \infty$. Please note that ${lim}_{n\to \infty }{\parallel F\left(x_n\right)-y\parallel }_{\mathcal{Y}}=0$, it follows from the continuity of F that $F\left(x^{*}\right)=y$.

We next prove that $x^{*}\in B_{2\rho }\left(x_0\right)\cap \mathcal{D}\left(\Theta \right)$. Due to $\left\{x_{n_l}\right\}\subset B_{2\rho }\left(x_0\right)$, we must have $x^{*}\in B_{2\rho }\left(x_0\right)$. By ${\xi }_{n_l}\in {\partial }_{{\epsilon }_{n_l}}\Theta \left(x_{n_l}\right)$, there holds

$$\Theta \left(x_{n_l}\right)\le \Theta \left(\widehat{x}\right)+{\langle {\xi }_{n_l},x_{n_l}-\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}+{\epsilon }_{n_l}.$$

(46)

Using (45) and $x_{n_l}\to x^{*}$, we can find a constant $C_0$ such that

$$|{\langle {\xi }_{n_l}-{\xi }_{n_0},x_{n_l}-\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}|\le C_0\mathrm{and}\left|{\langle {\xi }_{n_0},x_{n_l}-\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}\right|\le C_0,\forall l.$$

Therefore, $|{\langle {\xi }_{n_l},x_{n_l}-\widehat{x}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}|\le 2C_0$ for all l. Since $\Theta$ is lower semicontinuous and ${lim}_{n\to \infty }{\epsilon }_n$ = 0, we can derive from (46) that

$$\Theta \left(x^{*}\right)\le \underset{l\to \infty }{lim\; inf}\Theta \left(x_{n_l}\right)\le \Theta \left(\widehat{x}\right)+2C_0<\infty ,$$

which implies that $x^{*}\in \mathcal{D}\left(\Theta \right)$. Thus, $x^{*}\in B_{2\rho }\left(x_0\right)\cap \mathcal{D}\left(\Theta \right)$ is a solution of (1).

Finally, we prove the convergence of the whole sequence $\left\{x_n\right\}$ to $x^{*}$. Let

$${\eta }_0:=\underset{n\to \infty }{lim}{D}_{{\xi }_n}\Theta (x^{*},x_n),$$

whose existence is guaranteed by Lemma 1. By the non-negativity of $\epsilon$-Bregman distance, we have ${\eta }_0\ge 0$. Using (42) with $\widehat{x}=x^{*}$ and taking $l\to \infty$, we have

$$D_{{\xi }_{n_j}}\Theta (x^{*},x_{n_j})\le D_{{\xi }_{n_j}}\Theta (x^{*},x_{n_j})-{\eta }_0+\underset{l\ge j}{sup}{\left|\langle {\xi }_{n_l}-{\xi }_{n_j},x_{n_l}-x^{*}\rangle \right|}_{{\mathcal{X}}^{*},\mathcal{X}},$$

which suggests that

$${\eta }_0\le \underset{l\ge j}{sup}{\left|\langle {\xi }_{n_l}-{\xi }_{n_j},x_{n_l}-x^{*}\rangle \right|}_{{\mathcal{X}}^{*},\mathcal{X}}$$

for all j. By virtue of (45) and taking $j\to \infty$, we obtain ${\eta }_0\le 0$. Therefore, ${\eta }_0=0$, i.e., ${lim}_{n\to \infty }{D}_{{\xi }_n}\Theta (x^{*},x_n)=0$. Using the p-convexity of $\Theta$ and ${lim}_{n\to \infty }{\epsilon }_n=0$, we can further conclude that ${lim}_{n\to \infty }{\parallel x_n-x^{*}\parallel }_{\mathcal{X}}=0$.

It remains to be shown $\widehat{x}=x^{†}$. From the definition of ${\xi }_n$, we observe that

$${\xi }_{n+1}-{\xi }_n\in \mathcal{R}\left({A_F}^{*}\right)\subset \mathcal{N}{\left(A_F\right)}^{\perp }=\overline{\mathcal{R}\left(A_F\right)}.$$

Therefore, we can make use of the second part of [28] (Proposition 3.6) to complete the proof.  □

3.2. Regularization Property

In this subsection, we will prove the regularization property of Algorithm 1. Before proceeding further, the stability results of Algorithm 1 are presented in the following two lemmas: Lemma 4 is the stability property of the inner scheme, and Lemma 5 concerns the stability of the whole algorithm.

Lemma 4.

Let all the conditions in Lemma 2 hold. The sequence of noisy data $\left\{y^{{\delta }_l}\right\}$ satisfies $\parallel y^{{\delta }_l}{-y\parallel }_{\mathcal{Y}}\le {\delta }_l\to 0$ as $l\to \infty$. The sequence $\left\{({\xi }_n^{{\delta }_l},x_n^{{\delta }_l})\right\}$ is produced by Algorithm 1. For any integer $n\ge 0$, if

$${\xi }_n^{{\delta }_l}\to {\xi }_{n}and{x}_n^{{\delta }_l}\to x_{n}asl\to \infty$$

for some $({\xi }_n,x_n)\in {\mathcal{X}}^{*}\times \mathcal{X}$, then there holds for each $k=0,1,\dots ,$

$${\xi }_{n,k}^{{\delta }_l}\to {\xi }_{n,k}and{x}_{n,k}^{{\delta }_l}\to x_{n,k}asl\to \infty .$$

with $\left\{({\xi }_{n,k},x_{n,k})\right\}$ defined by (33).

Proof. 

We use an induction argument on k. When $k=0$, the result automatically holds since ${\xi }_{n,0}^{{\delta }_l}={\xi }_{n,0}={\xi }_n$ and $x_{n,0}^{{\delta }_l}=x_{n,0}=x_n$. Assume that the result holds for some $k\ge 1$, we will show that it also holds for $k+1$. The following two cases will be considered:

(i) $s_{n,k}=0$. By the definition of ${\xi }_{n,k+1}$, we have ${\xi }_{n,k+1}={\xi }_{n,k}$. Consequently,

$${\xi }_{n,k+1}^{{\delta }_l}-{\xi }_{n,k+1}={\xi }_{n,k}^{{\delta }_l}-{\xi }_{n,k}+{\mu }_{n,k}^{{\delta }_l}{A}_F^{*}{J}_r^{\mathcal{Y}}\left(s_{n,k}^{{\delta }_l}\right).$$

Together with ${\mu }_{n,k}^{\delta }\le {\mu }_1(\parallel s_{n,k}^{\delta }{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{1-\frac{r}{p}}$ and the property of $J_r^{\mathcal{Y}}$, we can deduce that

$$\parallel {\xi }_{n,k+1}^{{\delta }_l}-{\xi }_{n,k+1}{\parallel }_{{\mathcal{X}}^{*}}\le \parallel {\xi }_{n,k}^{{\delta }_l}-{\xi }_{n,k}{\parallel }_{{\mathcal{X}}^{*}}+{\mu }_1\widehat{B}(\parallel s_{n,k}^{{\delta }_l}{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k}{)}^{1-\frac{r}{p}}{\parallel s_{n,k}^{{\delta }_l}\parallel }_{\mathcal{Y}}^{r-1}.$$

By the induction hypothesis and the continuity of F, there holds ${\xi }_{n,k+1}^{{\delta }_l}\to {\xi }_{n,k+1}$ as $l\to \infty$. The definition of $x_{n,k+1}^{{\delta }_l}$($x_{n,k+1}^{{\delta }_l}=S_{{\epsilon }_{n,k+1}}\left({\xi }_{n,k+1}^{{\delta }_l}\right)$) and the continuity of $S_{\epsilon }$ yield that $x_{n,k+1}^{{\delta }_l}\to x_{n,k+1}$ as $l\to \infty$.

(ii) $s_{n,k}\ne 0$. We first show that ${\mu }_{n,k}^{{\delta }_l}\to {\mu }_{n,k}$ as $l\to \infty$. Recall that

$${\mu }_{n,k}=min\left\{\frac{{\mu }_0{\parallel s_{n,k}\parallel }_{\mathcal{Y}}^{p(r-1)}}{\parallel A_F^{*}{J}_r^{\mathcal{Y}}\left(s_{n,k}\right){\parallel }_{{\mathcal{X}}^{*}}^p},{\mu }_1\right\}(\parallel s_{n,k}{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{1-\frac{r}{p}}$$

and

$${\mu }_{n,k}^{{\delta }_l}=min\left\{\frac{{\mu }_0{\parallel s_{n,k}^{{\delta }_l}\parallel }_{\mathcal{Y}}^{p(r-1)}}{\parallel A_F^{*}{J}_r^{\mathcal{Y}}\left(s_{n,k}^{{\delta }_l}\right){\parallel }_{{\mathcal{X}}^{*}}^p},{\mu }_1\right\}(\parallel s_{n,k}^{{\delta }_l}{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{1-\frac{r}{p}}.$$

If $A_F^{*}{J}_r^{\mathcal{Y}}\left(s_{n,k}\right)=0$, we have ${\mu }_{n,k}={\mu }_1(\parallel s_{n,k}{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{1-\frac{r}{p}}$ and ${\mu }_{n,k}^{{\delta }_l}={\mu }_1(\parallel s_{n,k}^{{\delta }_l}{{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k})}^{1-\frac{r}{p}}$ for sufficiently large l, which gives ${\mu }_{n,k}^{{\delta }_l}\to {\mu }_{n,k}$ as $l\to \infty$. If $A_F^{*}{J}_r^{\mathcal{Y}}\left(s_{n,k}\right)\ne 0$, it follows from the induction hypothesis that ${\mu }_{n,k}^{{\delta }_l}\to {\mu }_{n,k}$ as $l\to \infty$. Then, using the induction hypotheses and the continuity of F, $J_r^{\mathcal{Y}}$ and $S_{\epsilon }$, we can conclude that ${\xi }_{n,k+1}^{{\delta }_l}\to {\xi }_{n,k+1}$ and $x_{n,k+1}^{{\delta }_l}\to x_{n,k+1}$ as $l\to \infty$.  □

Lemma 5.

Let all the conditions in Lemma 2 hold. The sequence of noisy data $\left\{y^{{\delta }_l}\right\}$ satisfies $\parallel y^{{\delta }_l}{-y\parallel }_{\mathcal{Y}}\le {\delta }_l\to 0$ as $l\to \infty$. The sequence $\left\{({\xi }_n^{{\delta }_l},x_n^{{\delta }_l})\right\}$ is generated by Algorithm 1. Then, for any integer $n\ge 0$, by taking a subsequence of $\left\{y^{{\delta }_l}\right\}$ if necessary, there is a sequence $\left\{({\xi }_n,x_n)\right\}\in {\Gamma }_{\gamma ,{\mu }_0,{\mu }_1}({\xi }_0,x_0)$ such that

$${\xi }_m^{{\delta }_l}\to {\xi }_{m}and{x}_m^{{\delta }_l}\to x_{m}asl\to \infty$$

for all $0\le m\le n$.

Proof. 

We use an induction to complete the proof. For $n=0$, the result is trivial again. Assume, for some $n>0$, the result is true for some sequence $\left\{({\xi }_n,x_n)\right\}\in {\Gamma }_{\gamma ,{\mu }_0,{\mu }_1}({\xi }_0,x_0)$. We next show that it is also valid for $n+1$. To this end, we can obtain a sequence from ${\Gamma }_{\gamma ,{\mu }_0,{\mu }_1}({\xi }_0,x_0)$ by redefining ${\xi }_{n+1}$ and $x_{n+1}$ in the sequence $\left\{({\xi }_n,x_n)\right\}$ and applying Algorithm 2 to generate the remaining terms. We may follow Lemma 4 to derive that

$${\xi }_{n,k}^{{\delta }_l}\to {\xi }_{n,k},x_{n,k}^{{\delta }_l}\to x_{n,k}\mathrm{as}l\to \infty$$

(47)

for $k=0,1,\dots$, where $\left\{({\xi }_{n,k},x_{n,k})\right\}$ are defined by (33).

Let $k_n^{{\delta }_l}$ be the integer used to define ${\xi }_{n+1}^{{\delta }_l}$ and $x_{n+1}^{{\delta }_l}$. By the definition of $k_n^{{\delta }_l}$ ($k_n^{{\delta }_l}=min\{{\tilde{k}}_n^{{\delta }_l},k_{max}\}$), we know that $1\le k_n^{{\delta }_l}\le k_{max}$. By taking a subsequence of $\left\{y^{{\delta }_l}\right\}$ if necessary, we may assume that $k_n^{{\delta }_l}$ takes the same integer value $k_n$. Then, $1\le k_n\le k_{max}$ and

$$\parallel s_{n,k}^{{\delta }_l}{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k}\ge (\gamma \parallel y^{{\delta }_l}-F\left(x_n^{{\delta }_l}\right){{\parallel }_{\mathcal{Y}})}^p,0\le k<k_n.$$

Then, by taking $l\to \infty$ and using Lemma 4, we have

$$\parallel s_{n,k}{\parallel }_{\mathcal{Y}}^p+\sigma {\epsilon }_{n,k}\ge (\gamma \parallel y-F\left(x_n\right){{\parallel }_{\mathcal{Y}})}^p,0\le k<k_n.$$

With this choice of $k_n$, we can redefine ${\xi }_{n+1}$ and $x_{n+1}$ in the sequence $\left\{({\xi }_n,x_n)\right\}$ by ${\xi }_{n+1}:={\xi }_{n,k_n}$ and $x_{n+1}:=x_{n,k_n}$. The application of Lemma 4 yields ${\xi }_{n+1}^{{\delta }_l}\to {\xi }_{n+1}$ and $x_{n+1}^{{\delta }_l}\to x_{n+1}$ as $l\to \infty$. The proof is thus complete.  □

We are now in a position to show the regularization property of Algorithm 1.

Theorem 2.

Let $\mathcal{X}$ be reflexive and let $\mathcal{Y}$ be uniformly smooth. Let Assumption 1, 2 and 3 hold. Let $\beta >1$, $\eta <\gamma <1$, ${\mu }_0>0$, and $\tau >1$ be chosen such that (25) holds. Assume further that $\left\{y^{\delta }\right\}$ is a family of noisy data satisfying ${∥y^{\delta }-y∥}_{\mathcal{Y}}\le \delta \to 0$ and let $n_{\delta }$ be determined by (23) for each $y^{\delta }$. Then,

$${∥x_{n_{\delta }}^{\delta }-x^{†}∥}_{\mathcal{X}}\to 0\mathrm{and}{D}_{{\xi }_{n_{\delta }}^{\delta }}\Theta \left(x^{†},x_{n_{\delta }}^{\delta }\right)\to 0$$

as $\delta \to 0$.

Proof. 

Due to the p-convexity of $\Theta$, it is sufficient to show ${lim}_{\delta \to 0}{D}_{{\xi }_{n_{\delta }}^{\delta }}\Theta (x^{†},x_{n_{\delta }}^{\delta })=0$. Since ${\epsilon }_{n,k}>0$ for all $n,k$, from (22) and (23), we must have $n_{\delta }\to \infty$ as $\delta \to 0$. Then, for any arbitrary but fixed integer $\widehat{n}>0$, we have $n_{\delta }>\widehat{n}$ for small $\delta$. From Lemma 2, we have

$$D_{{\xi }_{n_{\delta }}^{\delta }}\Theta (x^{†},x_{n_{\delta }}^{\delta })\le D_{{\xi }_{\widehat{n}}^{\delta }}\Theta (x^{†},x_{\widehat{n}}^{\delta })+3\sum _{n=\widehat{n}}^{n_{\delta }-1}\sum _{k=0}^{k_n^{\delta }-1}{\epsilon }_{n,k}.$$

Together with the lower semi-continuity of $\Theta$, there holds

$$\begin{array}{cc}\hfill & \underset{\delta \to 0}{limsup}{D}_{{\xi }_{n_{\delta }}^{\delta }}\Theta (x^{†},x_{n_{\delta }}^{\delta })\hfill \\ \hfill & \le \Theta \left(x^{†}\right)-\underset{\delta \to 0}{liminf}\Theta \left(x_{\widehat{n}}^{\delta }\right)-\underset{\delta \to 0}{lim}{\langle {\xi }_{\widehat{n}}^{\delta },x^{†}-x_{\widehat{n}}^{\delta }\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}+3\sum _{n=\widehat{n}}^{\infty }\sum _{k=0}^{k_n^{\delta }-1}{\epsilon }_{n,k}\hfill \\ \hfill & =\Theta \left(x^{†}\right)-\Theta \left(x_{\widehat{n}}\right)-{\langle {\xi }_{\widehat{n}},x^{†}-x_{\widehat{n}}\rangle }_{{\mathcal{X}}^{*},\mathcal{X}}+3\sum _{n=\widehat{n}}^{\infty }\sum _{k=0}^{k_n^{\delta }-1}{\epsilon }_{n,k}\hfill \\ \hfill & =D_{{\xi }_{\widehat{n}}}\Theta (x^{†},x_{\widehat{n}})+3\sum _{n=\widehat{n}}^{\infty }\sum _{k=0}^{k_n^{\delta }-1}{\epsilon }_{n,k}.\hfill \end{array}$$

Since $\widehat{n}>0$ is arbitrary, by taking $\widehat{n}\to \infty$, we may use (24) and Theorem 1 to deduce that ${lim}_{\delta \to 0}{D}_{{\xi }_{n_{\delta }}^{\delta }}\Theta (x^{†},x_{n_{\delta }}^{\delta })=0$. By the p-convexity of $\Theta$, we can conclude that $\parallel x_{n_{\delta }}^{\delta }-x^{†}{\parallel }_{\mathcal{X}}\to 0$ as $\delta \to 0$.  □

4. Numerical Experiments

In this section, we provide two numerical experiments. The aim of the first one is to test the effectiveness of our Algorithm 1 in identifying non-smooth solutions of parametric identification problems. The second one is to validate the efficiency of Algorithm 1 for solving non-smooth source-term problems.

4.1. Elliptic Parameter Identification

We first consider the reconstruction of parameter c in the boundary value problem [3]

$$\left\{\begin{array}{cc}\hfill -u^{″}+cu=f& \mathrm{in}\Omega ,\hfill \\ \hfill u=g& \mathrm{on}\partial \Omega .\hfill \end{array}\right.$$

(48)

from $L^2(\Omega )$—measurements of the state u, where $f\in H^{-1}(\Omega )$ and $g\in H^{\frac{1}{2}}(\Omega ).$ Let

$$\mathcal{D}:=\{c\in L^2(\Omega ):\parallel c-\widehat{c}{\parallel }_{L^2(\Omega )}\le {\gamma }_0\mathrm{for}\mathrm{all}\widehat{c}\ge 0\mathrm{a}.\mathrm{e}.\}$$

for some ${\gamma }_0>0.$ If $c\in \mathcal{D}$ is given, then (48) has a unique solution $u:=u\left(c\right)$. Therefore, (48) reduces to solving $F\left(c\right)=u$ if we define the nonlinear operator $F:L^2(\Omega )\to L^2(\Omega )$ by $F\left(c\right):=u\left(c\right).$ From [3], it is known that F is Fréchet differentiable; the Fréchet derivative of F and its adjoint are given by

$$F^{\prime }\left(c\right)h=-A{\left(c\right)}^{-1}\left(hF\left(c\right)\right)\mathrm{and}{F}^{\prime }{\left(c\right)}^{*}\omega =-u\left(c\right)A{\left(c\right)}^{-1}\omega$$

(49)

for $h,\omega \in L^2(\Omega )$ and $A\left(c\right):H^2(\Omega )\bigcap H_0^1(\Omega )\to L^2(\Omega )$ is defined by $A\left(c\right)u=-u^{″}+cu.$ According to [21], if we choose $A_F=F^{\prime }\left(c_f\right)$ with given $c_f\in \mathcal{D}$, condition (c) in Assumption 2 holds.

In this experiment, we pick $\Omega =[0,1]$, $u\left(0\right)=1$ and $u\left(1\right)=6$. The sought parameter $c^{†}\left(t\right)$ is taken to be

$$c^{†}\left(t\right)=\left\{\begin{array}{cc}0,\hfill & 0\le t\le 0.1563,\hfill \\ 1.5,\hfill & 0.1563<t\le 0.3125,\hfill \\ 2.5,\hfill & 0.3125<t\le 0.5469,\hfill \\ 1.3,\hfill & 0.5469<t\le 0.7813,\hfill \\ 0.5,\hfill & 0.7813<t\le 1.\hfill \end{array}\right.$$

In addition, we take $u\left(c^{†}\right)=1+5t$ and the source term $f\left(t\right)=\left(1+5t\right)c^{†}\left(t\right)$. Our aim is to reconstruct $c^{†}$ from noisy data $u^{\delta }$ with noise level $\delta ={∥u^{\delta }-u\left(c^{†}\right)∥}_{L^2}=0.0001$. When implementing Algorithm 1, we use the initial guess $c_0={\xi }_0=0$ and

$$\Theta \left(x\right)=\frac{1}{2\beta }{\int }_{\Omega }{\left|x\left(w\right)\right|}^{2}dw+TV\left(x\right)$$

with $\beta =1$ to identify the non-smooth feature of $c^{†}$. We fix $\eta =0.01$, $\sigma =0.001$, $k_{max}=500$, $\gamma =0.98$ in (22), and $\tau =1.5$ in (23). To meet the condition (25), we require ${\mu }_0<2\left(1-\frac{\eta }{\gamma }-\frac{1+\eta }{\gamma \tau }\right)/\phantom{\left(1-\frac{\eta }{\gamma }-\frac{1+\eta }{\gamma \tau }\right)\beta }\beta$. Thus, we take ${\mu }_0=\left(1-1/\phantom{1\tau }\tau \right)/\phantom{1.8\left(1-1/\phantom{1\tau }\tau \right)\beta }\beta$ and ${\mu }_1=10,000$. By dividing the interval $[0,1]$ into 128 subintervals of equal length, the involved differential equations are solved by the finite difference method. The minimization problems concerning $\Theta$ are solved by the primal dual hybrid gradient (PDHG) method [29] which is terminated as long as the relative duality gap is $\le {\left(n+1\right)}^{-1.5}{\left(k+1\right)}^{-1.5}$.

Figure 1a,b show the reconstruction results of Algorithm 1 with $A_F=F^{\prime }\left(c_f\right)$ at two different choices of $c_f$: $c_f=0$ and $c_f=1$. As comparisons, we also consider the inexact Newton-Landweber iteration in [6], i.e., Algorithm 1 with $A_F$ being the Fréchet derivative of F at each iteration, i.e., $A_F=F^{\prime }\left(c_n^{\delta }\right)$; the corresponding reconstructed solution is plotted in Figure 1c. As can be seen, the reconstructions of our Algorithm 1 with different values of $c_f$ have quality comparable to the ones reconstructed by [6], while Algorithm 1 does not require the information of the Fréchet derivative at each iteration. In Figure 1d, we present the evolution of the relative errors ${∥c_n^{\delta }-c^{†}∥}_{L^2}/\phantom{{∥c_n^{\delta }-c^{†}∥}_{L^2}{∥c∥}_{L^2}}{∥c^{†}∥}_{L^2}$ with respect to inner iteration number n. Since our Algorithm 1 avoids the calculation of the Fréchet derivative at each iteration, the computational work is considerably reduced; inevitably, Algorithm 1 may require more iterations to execute, which, however, can be offset by the numerous advantages.

4.2. A Non-Smooth Ill-Posed Problem

Let $\Omega \subset {\mathbb{R}}^2$ be a bounded domain with a Lipschitz boundary $\partial \Omega .$ We consider the inverse problem of the estimation of the source term f in the non-smooth semi-linear elliptic equation

$$-\Delta u+u^{+}=f\mathrm{in}\Omega ,u=0\mathrm{on}\partial \Omega$$

(50)

from an $L^2(\Omega )$ measurement $\tilde{u}$ of the state $u,$ where $u^{+}=max\{u\left(x\right),0\}$ for all $x\in \Omega .$ It is easy to see that, for each $f\in L^2(\Omega ),$ (50) has a unique solution $u:=u\left(f\right)\in H_0^1(\Omega )\bigcap C\left(\overline{\Omega }\right)\subset L^2(\Omega );$ see [30] (Theorem 4.7). If we know the sought solution $f^{†}$ and define $F\left(f\right)=u\left(f\right),$ then the problem (50) reduces to the inverse problem of (1). It has been shown that F is weakly closed and is not Gâteaux differentiable at f if the measurement of the set $\left\{u\right(f)=0\}$ is positive; see [17] and [30] (Proposition 3.4). In this case, the Bouligand subderivative of F at f exists, which is defined by a limit of the Fréchet derivatives of F in differentiable points. Christof introduced in [31] (Proposition) a specific Bouligand subderivative of F which states that, for $f\in L^2(\Omega ),$ the bounded linear operator $G\left(f\right):L^2(\Omega )\to L^2(\Omega )$ maps $h\in H_0^1(\Omega )$ to the unique solution $v:=G\left(f\right)h\in H_0^1(\Omega )\hookrightarrow L^2(\Omega )$ of

$$-\Delta v+{\chi }_{\left\{u\right(f)>0\}}v=h\mathrm{in}\Omega ,v=0\mathrm{on}\partial \Omega .$$

(51)

It has been shown in [21] that if we take $A_F=G\left(M_f\right)$ with given $M_f\in L^2(\Omega )$, the condition (17) holds for sufficiently small $\rho >0.$

In the computation, we pick $\Omega :=(0,1)\times (0,1)\subset {\mathbb{R}}^2$ and assume that the sought solution is

$$\begin{array}{cc}\hfill f^{†}(x_1,x_2):=& max(u^{†}(x_1,x_2),0)+[4{\pi }^2{u}^{†}(x_1,x_2)\hfill \\ \hfill & -2{(2x_1-1)}^2+2(x_1-1+\beta )(x_1-\beta )sin\left(2\pi x_2\right)]{\chi }_{[\beta ,1-\beta ]}\left(x_1\right)\hfill \end{array}$$

with $\beta =0.1,$ where

$$u^{†}(x_1,x_2):=[{(x_1-\beta )}^2{(x_1-1+\beta )}^{2}sin\left(2\pi x_2\right)]{\chi }_{[\beta ,1-\beta ]}\left(x_1\right)$$

is the corresponding exact state. Obviously, $u^{†}\in H_0^1(\Omega )\bigcap H^2(\Omega )$, together the right-hand side $f^{†}$ satisfies (50). Since $u^{†}$ vanishes on a set of measures $2\beta ,$ the forward operator F is not Gâteaux differential at $f^{†}.$ To carry out the computation, we employ a uniform triangular Friedrichs–Keller triangulation with $128\times 128.$ The differential Equations (50) and (51) will be discretized using a finite-element method; see [17]. The corresponding discrete system is then solved by a semi-smooth Newton iteration [32]. We generate noisy data $u^{\delta }$ by adding Gaussian noise to u with the noise level $\delta ={∥u^{\delta }-u^{†}∥}_{L^2}$. In the following, we pick $\delta =0.001$ and the initial guess

$$f_0=f^{†}-20\times sin\left(\pi x_1\right)sin\left(2\pi x_2\right).$$

When executing Algorithm 1, we use $\Theta \left(x\right)=\frac{1}{2}{\int }_{\Omega }{\left|x\left(w\right)\right|}^{2}dw$ and $\gamma =0.9,{\mu }_0=0.8,{\mu }_1=1000,k_{max}=300$, $\tau =1.4$.

In Figure 2, we report the reconstruction of Algorithm 1 corresponding to $A_F=G\left(M_f\right)$ with two different choices of $M_f$: $M_f=f_0$ and $M_f=0$. Figure 2a displays the exact solution $f^{†}$. The reconstruction results of Algorithm 1 are presented in Figure 2c,d. Observe that our algorithm can provide satisfactory results for non-smooth inverse problems.

In summary, we can see from the above experiments that our Algorithm 1 could efficiently deal with smooth as well as non-smooth inverse problems by choosing the operator $A_F$ appropriately; when the forward mapping is not Gâteaux differentiable, the operator $A_F$, as a replacement of the nonexisting Gâteaux derivative, can be taken as the Bouligand subderivative, which extends the feasibility of inexact Newton methods [4,5,6,7] for non-smooth inverse problems.

5. Conclusions

In this paper, by employing a bounded linear operator $A_F$ and inexact inner solvers, we propose a generalized inexact Newton-Landweber iteration method; this method does not require the Fréchet derivative of the forward mapping, which makes the method feasible for not only smooth but also non-smooth nonlinear inverse problems in Banach spaces. Under certain conditions on $A_F$, the convergence analysis is carefully established. The numerical simulations on smooth parameter identification problems and non-smooth inverse source-term problems indicate that our method could effectively solve inverse problems with smooth as well as non-smooth forward operators by choosing the appropriate operator $A_F$.

There are several possible lines of future research. First, the bounded linear operator $A_F$ may be replaced by a family of bounded linear operators $A_F\left(x\right)$ and the corresponding convergence theory can be developed without relying on the continuity of $x\to A_F\left(x\right)$ [17]. Second, application of the method to other non-smooth inverse problems is another interesting research direction.