The Proximal Gradient Method for Composite Optimization Problems on Riemannian Manifolds

Abstract

In this paper, the composite optimization problem is studied on Riemannian manifolds. To tackle this problem, the proximal gradient method to solve composite optimization problems is proposed on Riemannian manifolds. Under some reasonable conditions, the convergence of the proximal gradient method with the backtracking procedure in the nonconvex case is presented. Furthermore, a sublinear convergence rate and the complexity result of the proximal gradient method for convex case are also established on Riemannian manifolds.

1. Introduction

Optimization is a fundamental concept in various fields, ranging from engineering and economics to machine learning and logistics. In the case of composite optimization, the objective function or the constraints are composed of multiple individual functions or constraints, making the problem more complex. Composite optimization problems [1,2,3,4] arise in diverse applications, such as signal processing, image reconstruction, and data analysis. The challenge in composite optimization lies in effectively optimizing these composite functions, potentially involving iterative algorithms or gradient-based methods. Researchers in optimization continually develop new techniques and algorithms to efficiently solve composite optimization problems and improve decision-making processes in various domains.

Solving composite optimization problems efficiently requires specialized algorithms that can exploit the problem’s composite nature to achieve optimal solutions. Many popular methods, including the proximal gradient method, the alternating direction method of multipliers (ADMM), the block coordinate descent, and the primal–dual method can be applied to solve the composite optimization problem [5,6,7,8,9,10,11]. The proximal gradient method is a widely used optimization algorithm for solving composite optimization problems. This method leverages both gradient descent and proximal operators to handle composite objective functions, where the objective is the sum of a smooth and a nonsmooth component. By iteratively updating variables using gradient steps and proximal operators, proximal gradient methods can efficiently solve composite optimization problems. For example, Sahu et al. [8] proposed first-order methods such as proximal gradient, which use forward–backward splitting techniques. In addition, they derived convergence rates for the proposed formulations and showed that the speed DJM of convergence of these algorithms is significantly better than the traditional forward–backward algorithm. In [9], Neal et al. discussed many different interpretations of proximal operators and algorithms; they also described their connections to many other topics in optimization and applied mathematics.

On the other hand, extending some theory results and methods of optimization problems from Euclidean spaces to Riemannian manifolds has attracted significant attention in recent years; see, e.g., [12,13,14,15,16,17,18,19,20,21]. For instance, Riemannian proximal gradient methods have been developed in [22,23], which utilize the Riemannian metric and curvature information to iteratively minimize the objective function. In [22], Bento et al. presented the proximal point method for finding minima of a special class of nonconvex function on Hadamard manifolds. The well definedness of the sequence generated by the proximal point method was established, and its convergence for a minima was obtained. Based on the results of [22], Feng et al. proposed a monotone proximal gradient algorithm with a fixed step size on Hadamard manifolds in [23]. They also established the convergence theorem of the proposed method under the reasonable definition of proximal gradient mapping on manifolds. There are many advantages of transforming algorithms from Euclidean space to algorithms on Riemannian manifolds: First, Riemannian manifolds capture the underlying geometry of the data space, allowing algorithms to leverage this structure for more accurate and efficient computations; second, Riemannian manifolds can model nonlinear and curved data more effectively than Euclidean spaces, enabling algorithms to better represent and process complex data distributions; third, algorithms designed for Riemannian manifolds are often more suitable on curved surfaces or non-Euclidean spaces, leading to improved performance and results.

Composite optimization problems on Riemannian manifolds pose a unique computational challenge due to the non-Euclidean geometry of the manifold. In such problems, the objective function is composed of several terms defined on the Riemannian manifold, which requires specialized optimization techniques that respect the manifold structure. In this paper, we propose the proximal gradient method for composite optimization problems on Riemannian manifolds. The proximal gradient method on Riemannian manifolds performs a gradient descent step and a proximal operator step. For the gradient descent step on Riemannian manifolds, the algorithm computes the gradient of the smooth function with respect to the Riemannian metric at the current iteration. This gradient is then used to update the iteration in the direction that minimizes the smooth component of the objective function on Riemannian manifolds. After the gradient descent step, the algorithm applies a proximal operator to the current iteration on Riemannian manifolds. The proximal operator is a mapping that projects the updated iteration onto the set of feasible points on the manifold, taking into account the nonsmooth component of the objective function. By iteratively alternating between these two steps, the proximal gradient method aims to efficiently minimize the composite objective function on the Riemannian manifold while respecting the manifold’s geometry and constraints. This approach is particularly useful for optimization problems in machine learning, computer vision, and other fields where data lie on complex geometric structures.

The proximal gradient method is a powerful optimization algorithm commonly used to solve composite optimization problems on Euclidean spaces. However, extending this method to Riemannian manifolds involves some challenges due to the non-Euclidean geometry of these spaces. One approach to address this is the Riemannian proximal gradient method, which combines ideas from optimization theory with Riemannian geometry to efficiently solve composite optimization problems on Riemannian manifolds. Since a Riemannian manifold, in general, does not have a linear structure, usual techniques in the Euclidean space cannot be applied, and new techniques have to be proposed. Our results contribute as follows. First, the proximal gradient method for composite optimization problems is introduced on Riemannian manifolds, and its convergence results are established, which generalizes some algorithm results in [8,9] from ${\mathbb{R}}^n$ to Riemannian manifolds. Second, some global convergence results of the proximal gradient method for composite optimization problems are proved using the backtracking procedure, which makes the algorithm more efficient. Furthermore, a sublinear convergence rate of the generalized sequence of function values to the optimal value is established on Riemannian manifolds, and the complexity result of the proximal gradient for convex case is also obtained. Third, since the computation of exponential mappings and parallel transports can be quite expensive in manifolds, and many convergence results show that the nice properties of some algorithms hold for all suitably defined retractions and general vector transports on Riemannian manifolds [22,23], the geodesic and parallel transports are replaced by retractions and general vector transports, respectively.

This work is organized as follows. In Section 2, some necessary definitions and concepts are provided for Riemannian manifolds. In Section 3, the proximal gradient method for composite optimization problems is presented for Riemannian manifolds. In Section 4, under some reasonable conditions, some convergence results of the proximal gradient method for composite optimization problems are provided for Riemannian manifolds.

2. Preliminaries

In this section, some standard definitions and results from Riemannian manifolds are recalled, which can be found in some introductory books on Riemannian geometry; see, for example, [24,25].

Let M be a finite-dimensional differentiable manifold and $x\in M$. The tangent space of M at x is denoted by $T_{x}M$ and the tangent bundle of M by $TM={\bigcup }_{x\in M}{T}_{x}M$. ${\langle ,\rangle }_x$ is denoted by the inner product on $T_{x}M$ with the associated norm ${\parallel ·\parallel }_x$. If there is no confusion, then the subscript x is omitted. If M is endowed with a Riemannian metric g, then M is a Riemannian manifold. Given a piecewise smooth curve $\gamma :[t_0,t_1]\to M$ joining x to y, that is, $\gamma \left(t_0\right)=x$ and $\gamma \left(t_1\right)=y$, the length of $\gamma$ by $l\left(\gamma \right)={\int }_a^b\parallel {\gamma }^{\prime }\left(t\right)\parallel dt$ can be defined. Minimizing this length functional over the set of all curves, a Riemannian distance $d(x,y)$, which induces the original topology on M, is obtained.

A Riemannian manifold is complete if, for any $x\in M$, all geodesics emanating from x are defined for all $t\in \mathbb{R}$. By the Hopf–Rinow theorem [13], any pair of points $x,y\in M$ can be joined by a minimal geodesic. The exponential mapping ${exp}_x:T_{x}M\to M$ is defined by ${exp}_{x}v={\gamma }_v(1,x)$ for each $v\in T_{x}M$, where $\gamma (·)={\gamma }_v(·,x)$ is the geodesic starting x with velocity v, that is, $\gamma \left(0\right)=x$ and ${\gamma }^{\prime }\left(0\right)=v$. It is easy to see that ${exp}_{x}tv={\gamma }_v(t,x)$ for each real number t.

The exponential mapping ${exp}_x$ provides a local parametrization of M via $T_{x}M$. However, the systematic use of the exponential mapping may not be desirable in all cases. Some local mappings to $T_{x}M$ may reduce the computational cost while preserving the useful convergence properties of the considered method.

Definition 1 

([21]). Given $x\in M$, a retraction is a smooth mapping $R_x:T_{x}M\to M$, such that

(i) 

$R_x\left(0_x\right)=x$ for all $x\in M$, where $0_x$ denotes the zero element of $T_{x}M$;

(ii) 

$D{R}_x\left(0_x\right)={\mathrm{id}}_{T_{x}M}$, where $D{R}_x$ denotes the derivative of $R_x$, and id denotes the identity mapping.

It is well-known that the exponential mapping is a special retraction, and some retractions are approximations of the exponential mapping.

The parallel transport is often too expensive to compute in a practical method, so a more general vector transport can be considered, see, for example, [14,15], which is built upon the retraction $R_x$. A vector transport $\mathcal{T}:TM⨁TM\to TM$, $({\eta }_x,{\xi }_x)\mapsto {\mathcal{T}}_{{\eta }_x}{\xi }_x$ with the associated retraction $R_x$ is a smooth mapping, such that, for all ${\eta }_x$ in the domain of $R_x$ and all ${\xi }_x,{\zeta }_x\in T_{x}M$, (i) ${\mathcal{T}}_{{\eta }_x}{\xi }_x\in T_{R_x\left({\eta }_x\right)}M$; (ii) ${\mathcal{T}}_{0_x}{\xi }_x={\xi }_x$; and (iii) ${\mathcal{T}}_{{\eta }_x}$ is a linear mapping. Let ${\mathcal{T}}_S$ denote the isometric vector transport (see, for example, [15,20]) with $R_x$ as the associated retraction. Then, it satisfies (i), (ii), (iii), and

$$\begin{array}{c}\hfill \left(\mathrm{iv}\right)g({\mathcal{T}}_{S\left({\eta }_x\right)}{\xi }_x,{\mathcal{T}}_{S\left({\eta }_x\right)}{\zeta }_x)=g({\xi }_x,{\zeta }_x).\end{array}$$

In most practical cases, ${\mathcal{T}}_{S\left({\eta }_x\right)}$ exists for all ${\eta }_x\in T_{x}M$, and this assumption is made throughout the paper. Furthermore, let ${\mathcal{T}}_{{\eta }_x}$ denote the derivative of the retraction, i.e.,

$$\begin{array}{c}\hfill {\mathcal{T}}_{{\eta }_x}{\xi }_x=D{R}_x\left({\eta }_x\right)\left[{\xi }_x\right]={\displaystyle \frac{\mathrm{d}}{\mathrm{d}t}}{R}_x({\eta }_x+t{\xi }_x){|}_{t=0}.\end{array}$$

Let $\mathfrak{L}(TM,TM)$ denote a fiber bundle with base space $M\times M$, such that the fiber over $(x,y)\in M\times M$ is $\mathfrak{L}(T_{x}M,T_{y}M)$, the set of all linear mappings from $T_{x}M$ to $T_{y}M$. From [21], it follows that a transporter $\mathfrak{L}$ on M is a smooth section of the bundle $\mathfrak{L}(TM,TM)$. Furthermore, ${\mathfrak{L}}^{-1}(x,y)=\mathfrak{L}(y,x)$, and $\mathfrak{L}(x,z)=\mathfrak{L}(y,z)\mathfrak{L}(x,y)$. Given a retraction $R_x$, for any ${\eta }_x,{\xi }_x\in T_{x}M$, the isometric vector transport ${\mathcal{T}}_S$ can be defined by

$$\begin{array}{c}\hfill {\mathcal{T}}_{S\left({\eta }_x\right)}{\xi }_x=\mathfrak{L}(x,R_x\left({\eta }_x\right))\left({\xi }_x\right).\end{array}$$

In this paper, from the locking condition proposed by Huang [20],

$$\begin{array}{c}\hfill {\mathcal{T}}_{{\eta }_x}{\xi }_x={\mathcal{T}}_{S\left({\eta }_x\right)}{\xi }_x\end{array}$$

is required. In some manifolds, there exist retractions, such that the above equality holds, e.g., the Stiefel manifold and the Grassman manifold [20]. Furthermore, from the above results, it follows that

$$\begin{array}{c}\hfill \parallel {\xi }_x\parallel =\parallel {\mathcal{T}}_{S\left({\eta }_x\right)}{\xi }_x\parallel =\parallel \mathfrak{L}(x,R_x\left({\eta }_x\right))\left({\xi }_x\right)\parallel =\parallel {\mathcal{T}}_{{\eta }_x}{\xi }_x\parallel =\parallel D{R}_x\left({\eta }_x\right)\left[{\xi }_x\right]\parallel .\end{array}$$

3. The Proximal Gradient Method

In this paper, the following composite optimization problems are studied on Riemannian manifolds:

$$\begin{array}{c}\hfill \underset{x\in M}{min}F\left(x\right)=f\left(x\right)+g\left(x\right).\end{array}$$

(1)

Suppose the following assumption holds.

Assumption 1. 

(i) 

$g:M\to (-\infty ,+\infty ]$ is proper, closed, and convex;

(ii) 

$f:M\to (-\infty ,+\infty ]$ is proper and closed, $\mathrm{dom}\left(f\right)$ is convex, $\mathrm{dom}\left(g\right)\subseteq \mathrm{int}\left(\mathrm{dom}\right(f\left)\right)$, and f is $L_f$—smooth over $\mathrm{int}\left(\mathrm{dom}\right(f\left)\right)$, that is, for any $x\in M,\parallel \mathrm{grad}f\left(x\right)\parallel \le L_f$ for some $L_f>0$;

(iii) 

The optimal set of (1) is nonempty and denoted by $X^{*}$, and the optimal value of problem (1) is denoted by $F^{*}$.

Remark 1. 

There are some special cases of problem (1).

(i) 

If $g=0$ and $\mathrm{dom}\left(f\right)=M$, then (1) reduces to the unconstrained smooth minimization problem on Riemannian manifolds

$$\begin{array}{c}\hfill \underset{x\in M}{min}f\left(x\right),\end{array}$$

where $f:M\to \mathbb{R}$ is an $L_f$-smooth function on Riemannian manifolds.

(ii) 

If $g={\delta }_C$, where C is a nonempty closed and convex set on M, then (1) reduces to the problem of minimizing a different function over a nonempty closed and convex set on Riemannian manifolds.

Let $\widehat{f}:f\circ R$ and $\widehat{g}:g\circ R$ denote the pullback of f and g through R, respectively. For any $x\in M$, let

$$\begin{array}{c}\hfill {\widehat{f}}_x=f\circ R,{\widehat{g}}_x=g\circ R\end{array}$$

(2)

denote the restriction of $\widehat{f}$ and $\widehat{g}$ to $T_{x}M$. In [21], it follows that

$$\begin{array}{c}\hfill \mathrm{grad}{\widehat{f}}_x\left(0_x\right)=\mathrm{grad}f\left(x\right).\end{array}$$

(3)

For problem (1), it is natural to define the following iteration:

$$\begin{array}{c}\hfill {\eta }^k={\mathrm{argmin}}_{\eta \in T_{x^k}M}\{{\widehat{f}}_{x^k}\left(0_{x^k}\right)+\langle \mathrm{grad}{\widehat{f}}_{x^k}\left(0_{x^k}\right),\eta \rangle +{\widehat{g}}_{x^k}\left(\eta \right)+{\displaystyle \frac{1}{2t_k}}{\parallel \eta \parallel }^2\}.\end{array}$$

(4)

After some simple manipulation, (4) can be rewritten as

$$\begin{array}{c}\hfill {\eta }^k={\mathrm{argmin}}_{\eta \in T_{x^k}M}\{t_k{\widehat{g}}_{x^k}\left(\eta \right)+{\displaystyle \frac{1}{2}}\parallel \eta +t_k\mathrm{grad}{\widehat{f}}_{x^k}\left(0_{x^k}\right){\parallel }^2\},\end{array}$$

(5)

which, by the definition of the proximal operator, is the same as

$$\begin{array}{c}\hfill {\eta }^k={\mathrm{prox}}_{t_k{\widehat{g}}_{x^k}}(-t_k\mathrm{grad}{\widehat{f}}_{x^k}\left(0_{x^k}\right)).\end{array}$$

Now, the proximal gradient method for composite optimization problems is introduced for Riemannian manifolds.

Let $T_L^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right):={\mathrm{prox}}_{{\displaystyle \frac{1}{L}}{\widehat{g}}_x}(\eta -{\displaystyle \frac{1}{L}}\mathrm{grad}{\widehat{f}}_x\left(\eta \right)),\forall \eta \in T_{x}M\bigcap \mathrm{int}\left(\mathrm{dom}\left({\widehat{f}}_x\right)\right)$. Then, the general update step of the proximal gradient method can be written as

$$\begin{array}{c}\hfill x^{k+1}=R_{x^k}{T}_{L_k}^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right).\end{array}$$

Lemma 1 

([11]). Let $f:\mathbb{E}\to (-\infty ,+\infty ]$ be an L-smooth function $(L\ge 0)$ over a given convex set D. Then, for any $x,y\in D$,

$$\begin{array}{c}\hfill f\left(y\right)\le f\left(x\right)+\langle \mathrm{grad}f\left(x\right),y-x\rangle +{\displaystyle \frac{L}{2}}{\parallel x-y\parallel }^2.\end{array}$$

Lemma 2 

([11]). Let $f:\mathbb{E}\to (-\infty ,+\infty ]$ be a proper, closed, and convex function. Then, for any $x,u\in \mathrm{E}$, the following three claims are equivalent:

(i) 

$u={\mathrm{prox}}_f\left(x\right)$;

(ii) 

$x-u\in \partial f\left(u\right)$;

(iii) 

$\langle x-u,y-u\rangle \le f\left(y\right)-f\left(u\right),\forall y\in \mathbb{E}$.

Lemma 3. 

Suppose that f and g satisfy Assumption 1. Let ${\widehat{F}}_x={\widehat{f}}_x+{\widehat{g}}_x$. Then, for any $\eta \in T_{x}M\bigcap \mathrm{int}\left(\mathrm{dom}\left({\widehat{f}}_x\right)\right)$ and $L\in ({\displaystyle \frac{L_{{\widehat{f}}_x}}{2}},+\infty )$, the following inequality holds:

$$\begin{array}{c}\hfill {\widehat{F}}_x\left(\eta \right)-{\widehat{F}}_x\left(T_L^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\right)\ge {\displaystyle \frac{L-\frac{L_{{\widehat{f}}_x}}{2}}{L^2}}{\parallel G_L^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\parallel }^2,\end{array}$$

where $G_L^{{\widehat{f}}_x,{\widehat{g}}_x}:T_{x}M\bigcap \mathrm{int}\left(\mathrm{dom}\left({\widehat{f}}_x\right)\right)\to T_{x}M$ is the operator defined by

$$\begin{array}{c}\hfill G_L^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right):=L(\eta -T_L^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)).\end{array}$$

Proof. 

Using the notation ${\eta }^{+}=T_L^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)$. By Lemma 1, it follows that

$$\begin{array}{c}\hfill {\widehat{f}}_x\left({\eta }^{+}\right)\le {\widehat{f}}_x\left(\eta \right)+\langle \mathrm{grad}{\widehat{f}}_x\left(y\right),{\eta }^{+}-\eta \rangle +{\displaystyle \frac{L_{{\widehat{f}}_x}}{2}}{\parallel {\eta }^{+}-\eta \parallel }^2.\end{array}$$

(6)

By Lemma 2, since ${\eta }^{+}={\mathrm{prox}}_{{\displaystyle \frac{1}{L}}{\widehat{g}}_x}(\eta -{\displaystyle \frac{1}{L}}\mathrm{grad}{\widehat{f}}_x)$, it follows that

$$\begin{array}{c}\hfill \langle \eta -{\displaystyle \frac{1}{L}}\mathrm{grad}{\widehat{f}}_x\left(\eta \right)-{\eta }^{+},\eta -{\eta }^{+}\rangle \le {\displaystyle \frac{1}{L}}{\widehat{g}}_x\left(\eta \right)-{\displaystyle \frac{1}{L}}{\widehat{g}}_x\left({\eta }^{+}\right),\end{array}$$

which implies that

$$\begin{array}{c}\hfill \langle \mathrm{grad}{\widehat{f}}_x\left(\eta \right),{\eta }^{+}-\eta \rangle \le -L{\parallel {\eta }^{+}-\eta \parallel }^2+{\widehat{g}}_x\left(\eta \right)-{\widehat{g}}_x\left({\eta }^{+}\right),\end{array}$$

which, together with (6), implies that

$$\begin{array}{c}\hfill {\widehat{f}}_x\left({\eta }^{+}\right)+{\widehat{g}}_x\left({\eta }^{+}\right)\le {\widehat{f}}_x\left(\eta \right)+{\widehat{g}}_x\left(\eta \right)+(-L+{\displaystyle \frac{L_{{\widehat{f}}_x}}{2}}){\parallel {\eta }^{+}-\eta \parallel }^2.\end{array}$$

Therefore,

$$\begin{array}{c}\hfill {\widehat{F}}_x\left(\eta \right)-{\widehat{F}}_x\left(T_L^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\right)\ge {\displaystyle \frac{L-\frac{L_{{\widehat{f}}_x}}{2}}{L^2}}{\parallel G_L^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\parallel }^2.\end{array}$$

   □

Definition 2. 

Suppose that f and g satisfy Assumption 1. Then, for any $x\in M$, the gradient mapping is the operator $G_L^{{\widehat{f}}_x,{\widehat{g}}_x}:T_{x}M\bigcap \mathrm{int}\left(\mathrm{dom}\left({\widehat{f}}_x\right)\right)\to T_{x}M$ defined by

$$\begin{array}{c}\hfill G_L^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right):=L(\eta -T_L^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)).\end{array}$$

The update step of the proximal gradient method can be rewritten as

$$\begin{array}{c}\hfill {\eta }^k=-{\displaystyle \frac{1}{L_k}}{G}_{L_k}^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right),\end{array}$$

and

$$\begin{array}{c}\hfill x^{k+1}=R_{x^k}{\eta }^k=R_{x^k}(-{\displaystyle \frac{1}{L_k}}{G}_{L_k}^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right)).\end{array}$$

Theorem 1. 

Let f and g satisfy Assumption 1, and let $L>0$. Then,

(i) 

$G_L^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)=\mathrm{grad}{\widehat{f}}_x\left(\eta \right)$ for all $\eta \in T_{x}M\bigcap \mathrm{int}\left(\mathrm{dom}\left({\widehat{f}}_x\right)\right)$, where $g=0$;

(ii) 

For $0_{x^{*}}\in T_{x^{*}}M\bigcap \mathrm{int}\left(\mathrm{dom}\left({\widehat{f}}_{x^{*}}\right)\right)$, it holds that $G_L^{{\widehat{f}}_{x^{*}},{\widehat{g}}_{x^{*}}}\left(0_{x^{*}}\right)=0$ if and only if $x^{*}$ is a stationary point of problem (1).

Proof. 

(i) For $g=0$, since

$$\begin{array}{ccc}\hfill {\mathrm{prox}}_{{\displaystyle \frac{1}{L}}{\widehat{g}}_x}\left(\eta \right)& =& {\mathrm{argmin}}_{\xi \in T_{x}M}\{{\displaystyle \frac{1}{L}}{\widehat{g}}_x\left(\eta \right)+{\displaystyle \frac{1}{2}}{\parallel \xi -(0+\eta )\parallel }^2\}\hfill \\ & =& {\mathrm{argmin}}_{\xi \in T_{x}M}\{{\displaystyle \frac{1}{2}}{\parallel \xi -(0+\eta )\parallel }^2\}=\eta .\hfill \end{array}$$

It follows that

$$\begin{array}{ccc}\hfill G_L^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)& =& L(\eta -T_L^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right))=L(\eta -{\mathrm{prox}}_{{\displaystyle \frac{1}{L}}{\widehat{g}}_x}(\eta -{\displaystyle \frac{1}{L}}\mathrm{grad}{\widehat{f}}_x\left(\eta \right))\hfill \\ & =& L(\eta -(\eta -{\displaystyle \frac{1}{L}}\mathrm{grad}{\widehat{f}}_x\left(\eta \right)))=\mathrm{grad}{\widehat{f}}_x\left(\eta \right),\forall \eta \in T_{x}M\bigcap \mathrm{int}\left(\mathrm{dom}\left({\widehat{f}}_x\right)\right).\hfill \end{array}$$

(ii) $G_L^{{\widehat{f}}_{x^{*}},{\widehat{g}}_{x^{*}}}\left(0_{x^{*}}\right)=0$ if and only if $0_{x^{*}}={\mathrm{prox}}_{{\displaystyle \frac{1}{L}}{\widehat{g}}_{x^{*}}}(0_{x^{*}}-{\displaystyle \frac{1}{L}}\mathrm{grad}{\widehat{f}}_{x^{*}}\left(0_{x^{*}}\right))$; from Lemma 2, the latter relation holds if and only if

$$\begin{array}{c}\hfill 0_{x^{*}}-{\displaystyle \frac{1}{L}}\mathrm{grad}{\widehat{f}}_{x^{*}}\left(0_{x^{*}}\right)-0_{x^{*}}\in {\displaystyle \frac{1}{L}}\partial {\widehat{g}}_{x^{*}}\left(0_{x^{*}}\right),\end{array}$$

that is,

$$\begin{array}{c}\hfill 0_{x^{*}}\in \mathrm{grad}{\widehat{f}}_{x^{*}}\left(0_{x^{*}}\right)+\partial {\widehat{g}}_{x^{*}}\left(0_{x^{*}}\right).\end{array}$$

This implies that $0_{x^{*}}$ is a stationary point of ${\widehat{f}}_{x^{*}}+{\widehat{g}}_{x^{*}}$. Then, $x^{*}$ is a stationary point of problem (1).    □

Next, the monotonicity property $\parallel G_L\left(x\right)\parallel$ with the respect to the parameter L is obtained for Riemannian manifolds.

Theorem 2. 

Suppose that f and g satisfy Assumption 1, and $L_1\ge L_2>0$. Then, for any $\eta \in T_{x}M\bigcap \mathrm{int}\left(\mathrm{dom}\left({\widehat{f}}_x\right)\right)$, it holds that

$$\begin{array}{c}\hfill \parallel G_{L_1}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\parallel \ge \parallel G_{L_2}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\parallel .\end{array}$$

(7)

Proof. 

For any $x\in M,{\xi }_1,{\xi }_2\in T_{x}M\bigcap \mathrm{int}\left(\mathrm{dom}\left({\widehat{f}}_x\right)\right)$ and $L>0$, from Lemma 2, the following inequality holds:

$$\begin{array}{c}\hfill \langle {\xi }_1-{\mathrm{prox}}_{{\displaystyle \frac{1}{L}}{\widehat{g}}_x}\left({\xi }_1\right),{\mathrm{prox}}_{{\displaystyle \frac{1}{L}}{\widehat{g}}_x}\left({\xi }_1\right)-{\xi }_2\rangle \ge {\displaystyle \frac{1}{L}}{\widehat{g}}_x\left({\mathrm{prox}}_{{\displaystyle \frac{1}{L}}{\widehat{g}}_x}\left({\xi }_1\right)\right)-{\displaystyle \frac{1}{L}}{\widehat{g}}_x\left({\xi }_2\right).\end{array}$$

Plugging $L=L_1,{\xi }_1=\eta -{\displaystyle \frac{1}{L_1}}\mathrm{grad}{\widehat{f}}_x\left(\eta \right)$, and ${\xi }_2={\mathrm{prox}}_{{\displaystyle \frac{1}{L_2}}{\widehat{g}}_x}(\eta -{\displaystyle \frac{1}{L_2}}\mathrm{grad}{\widehat{f}}_x\left(\eta \right))=T_{L_2}\left(\eta \right)$ into the last inequality, it follows that

$$\begin{array}{c}\hfill \langle \eta -{\displaystyle \frac{1}{L_1}}\mathrm{grad}{\widehat{f}}_x\left(\eta \right)-T_{L_1}\left(\eta \right),T_{L_1}\left(\eta \right)-T_{L_2}\left(\eta \right)\rangle \ge {\displaystyle \frac{1}{L_1}}{\widehat{g}}_x\left(T_{L_1}\left(\eta \right)\right)-{\displaystyle \frac{1}{L_1}}{\widehat{g}}_x\left(T_{L_2}\left(\eta \right)\right),\end{array}$$

or

$$\begin{array}{c}\hfill \langle {\displaystyle \frac{1}{L_1}}{G}_{L_1}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)-{\displaystyle \frac{1}{L_1}}\mathrm{grad}{\widehat{f}}_x\left(\eta \right),{\displaystyle \frac{1}{L_2}}{G}_{L_2}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)-{\displaystyle \frac{1}{L_1}}{G}_{L_1}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\rangle \ge {\displaystyle \frac{1}{L_1}}{\widehat{g}}_x\left(T_{L_1}\left(\eta \right)\right)-{\displaystyle \frac{1}{L_1}}{\widehat{g}}_x\left(T_{L_2}\left(\eta \right)\right).\end{array}$$

Exchanging the roles of $L_1$ and $L_2$ yields the following inequality:

$$\begin{array}{c}\hfill \langle {\displaystyle \frac{1}{L_2}}{G}_{L_2}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)-{\displaystyle \frac{1}{L_2}}\mathrm{grad}{\widehat{f}}_x\left(\eta \right),{\displaystyle \frac{1}{L_1}}{G}_{L_1}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)-{\displaystyle \frac{1}{L_2}}{G}_{L_2}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\rangle \ge {\displaystyle \frac{1}{L_2}}{\widehat{g}}_x\left(T_{L_2}\left(\eta \right)\right)-{\displaystyle \frac{1}{L_2}}{\widehat{g}}_x\left(T_{L_1}\left(\eta \right)\right).\end{array}$$

Multiplying the first inequality by $L_1$ and the second by $L_2$ and adding them, it follows that

$$\begin{array}{c}\hfill \langle G_{L_1}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)-G_{L_2}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right),{\displaystyle \frac{1}{L_2}}{G}_{L_2}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)-{\displaystyle \frac{1}{L_1}}{G}_{L_1}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\rangle \ge 0.\end{array}$$

That is,

$$\begin{array}{ccc}\hfill {\displaystyle \frac{1}{L_1}}\parallel G_{L_1}^{{\widehat{f}}_x,{\widehat{g}}_x}{\left(\eta \right)\parallel }^2+{\displaystyle \frac{1}{L_2}}{\parallel G_{L_2}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\parallel }^2& \le & ({\displaystyle \frac{1}{L_1}}+{\displaystyle \frac{1}{L_2}})\langle G_{L_1}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right),G_{L_2}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\rangle \hfill \\ & \le & ({\displaystyle \frac{1}{L_1}}+{\displaystyle \frac{1}{L_2}})\parallel G_{L_1}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\parallel \parallel G_{L_2}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\parallel .\hfill \end{array}$$

(8)

Note that if $G_{L_2}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)=0$, then, by (8), $G_{L_1}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)=0$. Assume that $G_{L_2}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\ne 0$, and define $t={\displaystyle \frac{G_{L_1}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)}{G_{L_2}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)}}$. Then, by (8), it follows that

$$\begin{array}{c}\hfill {\displaystyle \frac{1}{L_1}}{t}^2-({\displaystyle \frac{1}{L_1}}+{\displaystyle \frac{1}{L_2}})t+{\displaystyle \frac{1}{L_2}}\le 0.\end{array}$$

This implies that

$$\begin{array}{c}\hfill 1\le t\le {\displaystyle \frac{L_1}{L_2}}.\end{array}$$

Therefore, $\parallel G_{L_1}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\parallel \ge \parallel G_{L_2}^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\parallel .$    □

4. The Convergence Result

4.1. The Non-Convex Case

In this section, the convergence of the proximal gradient method is analyzed for Riemannian manifolds. Now, the backtracking procedure B1 is considered as follows.

The procedure requires three parameters $(s,r,q)$, where $s>0,r\in (0,1)$, and $q>1$. The choice of $L_k$ is performed as follows.

First, $L_k$ is the set to be equal to the initial s. Then, while

$$\begin{array}{c}\hfill {\widehat{F}}_{x^k}\left(0_{x^k}\right)-{\widehat{F}}_{x^k}\left(T_{L_k}\left(0_{x^k}\right)\right)<{\displaystyle \frac{r}{L_k}}{\parallel G_{L_k}^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right)\parallel }^2,\end{array}$$

we set $L_k:=q{L}_k$. In other words, $L_k$ is chosen as $L_k:=s{q}^{i_k}$, where $i_k$ is the smallest non-negative integer for which the condition

$$\begin{array}{c}\hfill {\widehat{F}}_{x^k}\left(0_{x^k}\right)-{\widehat{F}}_{x^k}\left(T_{s{q}^{i_k}}\left(0_{x^k}\right)\right)\ge {\displaystyle \frac{r}{s{q}^{i_k}}}{\parallel G_{s{q}^{i_k}}^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right)\parallel }^2\end{array}$$

(9)

is satisfied.

Lemma 4. 

Suppose that Assumption 1 holds. Let $\left\{x^k\right\}$ be the sequence generated by Algorithm 1 with a step size chosen by the backtracking procedure B1. Then, for any $k\ge 0$,

$$\begin{array}{c}\hfill F\left(x^k\right)-F\left(x^{k+1}\right)\ge m{\parallel G_s^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right)\parallel }^2,\end{array}$$

(10)

where $m={\displaystyle \frac{r}{max\left(s,\frac{q{L}_{{\widehat{f}}_{x^k}}}{2(1-r)}\right)}}$.

Algorithm 1The proximal gradient method for Riemannian manifolds.

Initialization: pick $x^0\in \mathrm{int}\left(\mathrm{dom}\left(f\right)\right)$.

General step: for any $k=0,1,\cdots ,$ execute the following step: $$\begin{array}{c}\hfill {\eta }_k:={\mathrm{prox}}_{{\displaystyle \frac{1}{L_k}}{\widehat{g}}_{x^k}}(-{\displaystyle \frac{1}{L_k}}\mathrm{grad}{\widehat{f}}_{x^k}\left(0_{x^k}\right)),\end{array}$$ (11)

and set $x^{k+1}=R_{x^k}{\eta }_k,$ where $L_k>0$.

Stopping criteria:${\eta }_k=0$.

Proof. 

It follows from Lemma 3 that

$$\begin{array}{c}\hfill {\widehat{F}}_{x^k}\left(0_{x^k}\right)-{\widehat{F}}_{x^k}\left(T_L^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right)\right)\ge {\displaystyle \frac{L-\frac{L_{{\widehat{f}}_{x^k}}}{2}}{L^2}}{\parallel G_L^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right)\parallel }^2.\end{array}$$

(12)

If $L\ge {\displaystyle \frac{L_{{\widehat{f}}_{x^k}}}{2(1-r)}}$, then ${\displaystyle \frac{L-\frac{L_{{\widehat{f}}_{x^k}}}{2}}{L}}\ge r$; hence, by (12), it follows that

$$\begin{array}{c}\hfill {\widehat{F}}_{x^k}\left(0_{x^k}\right)-{\widehat{F}}_{x^k}(T_L^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right)\ge {\displaystyle \frac{r}{L}}{\parallel G_L^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right)\parallel }^2\end{array}$$

(13)

holds. This implies that the backtracking procedure B1 must end when $L_k\ge {\displaystyle \frac{L_{{\widehat{f}}_{x^k}}}{2(1-r)}}$. An upper bound on $L_k$ can be computed: either $L_k$ is equal to s, or the backtracking procedure B1 in invoked, meaning that ${\displaystyle \frac{L_k}{q}}$ did not satisfy the backtracking condition, which implies that ${\displaystyle \frac{L_k}{q}}<{\displaystyle \frac{L_{{\widehat{f}}_{x^k}}}{2(1-r)}}$; so, $L_k<{\displaystyle \frac{q{L}_{{\widehat{f}}_{x^k}}}{2(1-r)}}$. That is,

$$\begin{array}{c}\hfill L_k\le max\{s,{\displaystyle \frac{q{L}_{{\widehat{f}}_{x^k}}}{2(1-r)}}\}.\end{array}$$

This, together with (13), implies that

$$\begin{array}{ccc}\hfill {\widehat{F}}_{x^k}\left(0_{x_k}\right)-{\widehat{F}}_{x^k}(T_{L_k}^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right)& \ge & {\displaystyle \frac{r}{L_k}}{\parallel G_{L_k}^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right)\parallel }^2\hfill \\ & \ge & {\displaystyle \frac{r}{max\{s,\frac{q{L}_{{\widehat{f}}_{x^k}}}{2(1-r)}\}}}{\parallel G_{L_k}^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x_k}\right)\parallel }^2.\hfill \end{array}$$

(14)

By Theorem 2, it follows that

$$\begin{array}{c}\hfill \parallel G_{L_k}^{{\widehat{f}}_{x_k},{\widehat{g}}_{x_k}}\left(0_{x_k}\right)\parallel \ge \parallel G_s^{{\widehat{f}}_{x_k},{\widehat{g}}_{x_k}}\left(0_{x_k}\right)\parallel .\end{array}$$

This, together with (13), (14) and Theorem 2, implies that

$$\begin{array}{c}\hfill F\left(x_k\right)-F\left(x_{k+1}\right)={\widehat{F}}_{x^k}\left(0_{x_k}\right)-{\widehat{F}}_{x^k}\left(T_{L_k}\left(0_{x_k}\right)\right)\ge m{\parallel G_s^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x_k}\right)\parallel }^2,\end{array}$$

where $m={\displaystyle \frac{r}{max\{s,\frac{q{L}_{{\widehat{f}}_{x^k}}}{2(1-r)}\}}}$.

 □

Theorem 3. 

Suppose that Assumption 1 holds, and let $\left\{x^k\right\}$ be the sequence generated by Algorithm 1 with a step size chosen by the backtracking procedure B1. Then,

(i) 

The sequence $\left\{F\left(x^k\right)\right\}$ is non-increasing;

(ii) 

${min}_{n=0,1,\cdots ,k}\parallel G_s^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right)\parallel \le {\displaystyle \frac{\sqrt{F\left(x^0\right)-F^{*}}}{\sqrt{m(k+1)}}}$;

(iii) 

All limit points of $\left\{x^k\right\}$ are stationary points of (1).

Proof. 

(i) By Lemma 4, it follows that

$$\begin{array}{c}\hfill F\left(x^k\right)-F\left(x^{k+1}\right)\ge m{\parallel G_s^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right)\parallel }^2,\end{array}$$

(15)

where $m={\displaystyle \frac{r}{max\{s,\frac{q{L}_{{\widehat{f}}_{x^k}}}{2(1-r)}\}}}$. From the above equality, it follows that $F\left(x^k\right)\ge F\left(x^{k+1}\right)$.

(ii) Since the sequence $\left\{F\left(x^k\right)\right\}$ is non-increasing and bounded below, it converges. Thus,

$$\begin{array}{c}\hfill F\left(x^k\right)-F\left(x^{k+1}\right)\to 0,\end{array}$$

which implies that

$$\begin{array}{c}\hfill \parallel G_s^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right)\parallel \to 0.\end{array}$$

Summing the inequality

$$\begin{array}{c}\hfill F\left(x^k\right)-F\left(x^{k+1}\right)\ge m{\parallel G_s^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right)\parallel }^2\end{array}$$

over $n=0,1,\cdots ,k$ implies that

$$\begin{array}{c}\hfill F\left(x^0\right)-F\left(x^{k+1}\right)\ge \sum _{n=0}^{k}m\parallel G_s^{{\widehat{f}}_{x^n},{\widehat{g}}_{x^n}}\left(0_{x^n}\right){\parallel }^2 \ge m(k+1)\underset{n=0,1,\cdots ,k}{min}{\parallel G_s^{{\widehat{f}}_{x^n},{\widehat{g}}_{x^n}}\left(0_{x^n}\right)\parallel }^2.\end{array}$$

Since $F\left(x^{k+1}\right)\ge F^{*}$, it follows that (ii) holds.

(iii) Let $\overline{x}$ be a limit point $\left\{x^k\right\}$. Then, there exists a subsequence $\left\{x^{k_j}\right\}$ converging to $\overline{x}$. From (15), it follows that $\parallel G_s^{{\widehat{f}}_{x^k},{\widehat{g}}_{x^k}}\left(0_{x^k}\right) \parallel \to 0$. It is easy to check that when $x^{k_j}\to \overline{x}$,

$$\begin{array}{c}\hfill G_s^{{\widehat{f}}_{x^{k_j}},{\widehat{g}}_{x^{k_j}}}\left(0_{x^{k_j}}\right)\to G_s^{{\widehat{f}}_{\overline{x}},{\widehat{g}}_{\overline{x}}}\left(0_{\overline{x}}\right).\end{array}$$

Since

$$\begin{array}{c}\hfill \parallel G_s^{{\widehat{f}}_{\overline{x}},{\widehat{g}}_{\overline{x}}}\left(0_{\overline{x}}\right)\parallel \le \parallel G_s^{{\widehat{f}}_{x^{k_j}},{\widehat{g}}_{x^{k_j}}}\left(0_{x^{k_j}}\right)-G_s^{{\widehat{f}}_{\overline{x}},{\widehat{g}}_{\overline{x}}}\left(0_{\overline{x}}\right)\parallel + \parallel G_s^{{\widehat{f}}_{x^{k_j}},{\widehat{g}}_{x^{k_j}}}\left(0_{x^{k_j}}\right)\parallel ,\end{array}$$

(16)

and the right hand side of (16) goes to 0 as $j\to +\infty$, this implies that $G_s^{{\widehat{f}}_{\overline{x}},{\widehat{g}}_{\overline{x}}}\left(0_{\overline{x}}\right)=0$. Therefore, from Theorem 1 (ii), it follows that $\overline{x}$ is a stationary point of (1). □

4.2. The Convex Case

In this section, suppose that f is convex on M. Under some conditions, some convergence results of the proximal gradient method for composite optimization problems are obtained for Riemannian manifolds.

Definition 3 

([11]). A function $f:\mathbb{E}\to (-\infty ,+\infty ]$ is called σ-strongly convex for a given $\sigma >0$ if $\mathrm{dom}\left(f\right)$ is convex, and the following inequality holds for any $x,y\in \mathrm{dom}\left(f\right)$ and $\lambda \in [0,1]$:

$$\begin{array}{c}\hfill f(\lambda x+(1-\lambda )y)\le \lambda f\left(x\right)+(1-\lambda )f\left(y\right)-{\displaystyle \frac{\sigma }{2}}\lambda (1-\lambda ){\parallel x-y\parallel }^2.\end{array}$$

(17)

Lemma 5 

([11]). Let $f:\mathbb{E}\to (-\infty ,+\infty ]$ be a proper closed and σ-strongly convex function ($\sigma >0$). Then,

$$\begin{array}{c}\hfill f\left(x\right)-f\left(x^{*}\right)\ge {\displaystyle \frac{\sigma }{2}}{\parallel x-x^{*}\parallel }^2,\forall x\in \mathrm{dom}\left(f\right),\end{array}$$

where $x^{*}$ is the unique minimizer of f.

Theorem 4. 

Suppose that f and g satisfy Assumption 1. For any $x\in M,\xi ,\eta \in T_{x}M\bigcap \mathrm{int}\left(\mathrm{dom}\left({\widehat{f}}_x\right)\right)$ and $L>0$ satisfying

$$\begin{array}{c}\hfill {\widehat{f}}_x\left(T_L\left(\eta \right)\right)\le {\widehat{f}}_x\left(\eta \right)+\langle \mathrm{grad}{\widehat{f}}_x\left(\eta \right),T_L\left(\eta \right)-\eta \rangle +{\displaystyle \frac{L}{2}}{\parallel T_L\left(\eta \right)-\eta \parallel }^2,\end{array}$$

(18)

it holds that

$$\begin{array}{c}\hfill {\widehat{F}}_x\left(\xi \right)-{\widehat{F}}_x\left(T_L\left(\eta \right)\right)\ge {\displaystyle \frac{L}{2}}\parallel \xi -T_L{\left(\eta \right)\parallel }^2-{\displaystyle \frac{L}{2}}{\parallel \eta -\xi \parallel }^2+l_{{\widehat{f}}_x}(\xi ,\eta ),\end{array}$$

(19)

where

$$\begin{array}{c}\hfill l_{{\widehat{f}}_x}(\xi ,\eta )={\widehat{f}}_x\left(\xi \right)-{\widehat{f}}_x\left(\eta \right)-\langle \mathrm{grad}{\widehat{f}}_x\left(\eta \right),\xi -\eta \rangle ,\end{array}$$

Proof. 

Consider the function

$$\begin{array}{c}\hfill {\varphi }_x\left(u\right)={\widehat{f}}_x\left(\eta \right)+\langle \mathrm{grad}{\widehat{f}}_x\left(\eta \right),u-\eta \rangle +{\widehat{g}}_x\left(u\right)+{\displaystyle \frac{L}{2}}{\parallel u-\eta \parallel }^2.\end{array}$$

It is easy to check that $\varphi$ is $L-$strongly convex, and

$$\begin{array}{c}\hfill T_L\left(\eta \right)={\mathrm{argmin}}_{u\in T_{x}M}{\varphi }_x\left(u\right).\end{array}$$

From Lemma 5, it follows that

$$\begin{array}{c}\hfill {\varphi }_x\left(\xi \right)-{\varphi }_x\left(T_L\left(\eta \right)\right)\ge {\parallel \xi -T_L\left(\eta \right)\parallel }^2.\end{array}$$

(20)

By (18), it is easy to check that

$$\begin{array}{ccc}\hfill {\varphi }_x\left(T_L\left(\eta \right)\right)& =& {\widehat{f}}_x\left(\eta \right)+\langle \mathrm{grad}{\widehat{f}}_x\left(\eta \right),T_L\left(\eta \right)-\eta \rangle +{\widehat{g}}_x\left(T_L\left(\eta \right)\right)+{\displaystyle \frac{L}{2}}{\parallel T_L\left(\eta \right)-\eta \parallel }^2\hfill \\ & \ge & {\widehat{f}}_x\left(T_L\left(\eta \right)\right)+{\widehat{g}}_x\left(T_L\left(\eta \right)\right)={\widehat{F}}_x\left(T_L\left(\eta \right)\right).\hfill \end{array}$$

This, together with (20), implies that

$$\begin{array}{c}\hfill {\varphi }_x\left(\xi \right)-{\widehat{F}}_x\left(T_L\left(\eta \right)\right)\ge {\displaystyle \frac{L}{2}}{\parallel \xi -T_L\left(\eta \right)\parallel }^2.\end{array}$$

By the definition of ${\varphi }_x\left(\xi \right)$, it follows that

$$\begin{array}{c}\hfill {\widehat{f}}_x\left(\eta \right)+\langle \mathrm{grad}{\widehat{f}}_x\left(\eta \right),\xi -\eta \rangle +{\widehat{g}}_x\left(\xi \right)+{\displaystyle \frac{L}{2}}{\parallel \xi -\eta \parallel }^2-{\widehat{F}}_x\left(T_L\left(\eta \right)\right)\ge {\displaystyle \frac{L}{2}}{\parallel \xi -T_L\left(\eta \right)\parallel }^2,\end{array}$$

which is equivalent to

$$\begin{array}{c}\hfill {\widehat{F}}_x\left(\xi \right)-{\widehat{F}}_x\left(T_{L_{{\widehat{f}}_x}}\left(\eta \right)\right)\ge {\displaystyle \frac{L}{2}}\parallel \xi -T_L{\left(\eta \right)\parallel }^2-{\displaystyle \frac{L}{2}}{\parallel \xi -\eta \parallel }^2+{\widehat{f}}_x\left(\xi \right)-{\widehat{f}}_x\left(\eta \right)-\langle \mathrm{grad}{\widehat{f}}_x\left(\eta \right),\xi -\eta \rangle .\end{array}$$

□

The following result is a direct result of Theorem 4.

Corollary 1. 

Suppose that f and g satisfy Assumption 1. For any $x\in M,\xi ,\eta \in T_{x}M\bigcap \mathrm{int}\left(\mathrm{dom}\left({\widehat{f}}_x\right)\right)$, for which

$$\begin{array}{c}\hfill {\widehat{f}}_x\left(T_L\left(\eta \right)\right)\le {\widehat{f}}_x\left(\eta \right)+\langle \mathrm{grad}{\widehat{f}}_x\left(\eta \right),T_L\left(\eta \right)-\eta \rangle +{\displaystyle \frac{L}{2}}{\parallel T_L\left(\eta \right)-\eta \parallel }^2,\end{array}$$

(21)

it holds that

$$\begin{array}{c}\hfill {\widehat{F}}_x\left(\eta \right)-{\widehat{F}}_x\left(T_L\left(\eta \right)\right)\ge {\displaystyle \frac{1}{2L}}{\parallel G_L^{{\widehat{f}}_x,{\widehat{g}}_x}\left(\eta \right)\parallel }^2.\end{array}$$

(22)

Next, the backtracking procedure B2 for the case f is convex is considered. The procedure requires two parameters $(s,q)$, where $s>0$ and $q>1$. Define $L_{-1}=s$. The choice $L_k$ is obtained as follows. First, $L_k$ is set to be $L_{k-1}$. Then, while

$$\begin{array}{c}\hfill {\widehat{f}}_{x^k}\left(T_{L_k}\left(0_{x^k}\right)\right)\ge {\widehat{f}}_{x^k}\left(0_{x^k}\right)+\langle \mathrm{grad}{\widehat{f}}_{x^k}\left(0_{x^k}\right),T_{L_k}\left(0_{x^k}\right)\rangle +{\displaystyle \frac{L_k}{2}}{\parallel T_{L_k}\left(0_{x^k}\right)\parallel }^2,\end{array}$$

(23)

we set $L_k:=q{L}_k$. That is, $L_k$ is chosen as $L_k=L_{k-1}{q}^{i_k}$, where $i_k$ is the smallest non-negative integer for which the condition

$$\begin{array}{ccc}\hfill {\widehat{f}}_{x^k}\left(T_{L_{k-1}{q}^{i_k}}\left(0_{x^k}\right)\right)& \le & {\widehat{f}}_{x^k}\left(0_{x^k}\right)+\langle \mathrm{grad}{\widehat{f}}_{x^k}\left(0_{x^k}\right),T_{L_{k-1}{q}^{i_k}}\left(0_{x^k}\right)\rangle \hfill \\ & +& {\displaystyle \frac{L_{k-1}{q}^{i_k}}{2}}{\parallel T_{L_{k-1}{q}^{i_k}}\left(0_{x^k}\right)\parallel }^2,\hfill \end{array}$$

(24)

is satisfied. Under Assumption 1 and Lemma 1, it follows that

$$\begin{array}{c}\hfill s\le L_k\le max\{q{L}_f,s\}.\end{array}$$

(25)

$s\le L_k$ is obvious. For the inequality $L_k\le max\{q{L}_f,s\}$, if $L_k>s$, the inequality (24) is not satisfied with ${\displaystyle \frac{L_k}{q}}$ replacing $L_k$. By Lemma 1, it follows that ${\displaystyle \frac{L_k}{q}}<L_f$; so, $L_k\le max\{q{L}_f,s\}$.

Next, an $O(1/k)$ rate of convergence of the generated sequence of function values to the optimal value is established for Riemannian manifolds. This rate of convergence is called a sublinear rate.

Theorem 5. 

Suppose that Assumption 1 holds, and f is convex on M. Let $\left\{x^k\right\}$ be the sequence generated by Algorithm 1 with the backtracking procedure B2. Then, for any $x^{*}\in X^{*}$ and $k>0$, there exists $\alpha >0$, such that

$$\begin{array}{c}\hfill F\left(x^k\right)-F^{*}\le {\displaystyle \frac{\alpha L_f{d}^2(x^0,x^{*})}{2k}}.\end{array}$$

(26)

Proof. 

For any $n\ge 0$, it follows from (19) that

$$\begin{array}{ccc}\hfill {\displaystyle \frac{2}{L_n}}(F\left(x^{*}\right)-F\left(x^{n+1}\right))& =& {\displaystyle \frac{2}{L_n}}({\widehat{F}}_{x^n}\left(R_{x^n}^{-1}\left(x^{*}\right)\right)-{\widehat{F}}_{x^n}\left(T_{L_n}\left(0_{x^n}\right)\right)\hfill \\ & \ge & \parallel R_{x^n}^{-1}\left(x^{*}\right)-T_{L_n}\left(0_{x^n}\right){\parallel }^2-{\parallel 0_{x^n}-R_{x^n}^{-1}\left(x^{*}\right)\parallel }^2\hfill \\ & +& {\displaystyle \frac{2}{L_n}}{l}_{f_{x^n}}(R_{x^n}^{-1}\left(x^{*}\right),0_{x^n})\hfill \\ & \ge & \parallel R_{x^n}^{-1}\left(x^{*}\right)-T_{L_n}\left(0_{x^n}\right){\parallel }^2-{\parallel 0_{x^n}-R_{x^n}^{-1}\left(x^{*}\right)\parallel }^2,\hfill \end{array}$$

(27)

where the last inequality is obtained by the convexity of f. From [21], it is easy to check that there exists $m>0$, such that

$$\begin{array}{ccc}\hfill \parallel R_{x^n}^{-1}\left(x^{*}\right)-T_{L_n}\left(0_{x^n}\right){\parallel }^2-{\parallel 0_{x^n}-R_{x^n}^{-1}\left(x^{*}\right)\parallel }^2& =& \parallel R_{x^n}^{-1}\left(x^{*}\right)-R_{x^n}^{-1}\left(x^{n+1}\right){\parallel }^2-{\parallel R_{x^n}^{-1}\left(x^{*}\right)\parallel }^2\hfill \\ & \ge & m[d^2(x^{n+1},x^{*})-d^2(x^n,x^{*})].\hfill \end{array}$$

(28)

Summing (27) over $n=0,1,\cdots ,k-1$ and using $L_n\le p{L}_f$, where $p=max\{q,{\displaystyle \frac{s}{L_f}}\}$, together with (28), implies that

$$\begin{array}{c}\hfill {\displaystyle \frac{2}{p{L}_f}}\sum _{n=0}^{k-1}(F\left(x^{*}\right)-F\left(x^k\right))\ge m[d^2(x^{n+1},x^{*})-d^2(x^0,x^{*})].\end{array}$$

Thus,

$$\begin{array}{c}\hfill \sum _{n=0}^{k-1}(F\left(x^{n+1}\right)-F\left(x^{*}\right))\le {\displaystyle \frac{mp{L}_f}{2}}{d}^2(x^0,x^{*})-{\displaystyle \frac{m\alpha L_f}{2}}{d}^2(x^k,x^{*})\le {\displaystyle \frac{mp{L}_f}{2}}{d}^2(x^0,x^{*}).\end{array}$$

From (22), it follows that $F\left(x^{n+1}\right)\le F\left(x^n\right)$ for all $n\ge 0$; so,

$$\begin{array}{c}\hfill k(F\left(x^k\right)-F\left(x^{*}\right))\le \sum _{n=0}^{k-1}(F\left(x^{n+1}\right)-F\left(x^{*}\right))\le {\displaystyle \frac{mp{L}_f}{2}}{d}^2(x^0,x^{*}).\end{array}$$

Therefore,

$$\begin{array}{c}\hfill F\left(x^k\right)-F\left(x^{*}\right)\le {\displaystyle \frac{mp{L}_f}{2k}}{d}^2(x^0,x^{*}).\end{array}$$

Let $\alpha =mp$. Then,

$$\begin{array}{c}\hfill F\left(x^k\right)-F^{*}\le {\displaystyle \frac{\alpha L_f{d}^2(x^0,x^{*})}{2k}}.\end{array}$$

□

To derive the complexity result for the proximal gradient method for Riemannian manifolds, let us assume that $d(x^0,x^{*})\le R$ for some $x^{*}\in X^{*}$ and some constant $R>0$. For example, if $\mathrm{dom}\left(g\right)$ is bounded, the R might be taken as its diameter. In order to obtain an $ϵ$-optimal solution of (1), by (26), it is enough to require that ${\displaystyle \frac{\alpha L_f{R}^2}{2k}}\le ϵ$. The following complexity of the proximal gradient method is a direct result of Theorem 5.

Theorem 6. 

Suppose that Assumption 1 holds, and f is convex on M. Let $\left\{x^k\right\}$ be the sequence generated by Algorithm 1 with the backtracking procedure B2. For k satisfying

$$\begin{array}{c}\hfill k\ge \left[{\displaystyle \frac{\alpha L_f{R}^2}{2ϵ}}\right],\end{array}$$

it holds that $F\left(x^k\right)-F^{*}\le ϵ$, where R is an upper bound on $d(x^0,x^{*})$ for some $x^{*}\in X^{*}$.

Example 1. 

On the unit sphere $S^{n-1}$ considered as a Riemannian submanifold of ${\mathbb{R}}^n$, the inner product inherited from the standard inner product on ${\mathbb{R}}^n$ is given by

$$\begin{array}{c}\hfill \langle {\xi }_x,{\eta }_x\rangle ={\xi }_x^T{\eta }_x,\forall {\xi }_x,{\eta }_x\in T_x{S}^{n-1}.\end{array}$$

The normal space is

$$\begin{array}{c}\hfill {\left(T_x{S}^{n-1}\right)}^{\perp }=\{x\alpha :\alpha \in \mathbb{R}\},\end{array}$$

and the projections are given by

$$\begin{array}{c}\hfill P_x{\xi }_x=(I-x{x}^T){\xi }_x,P_x^{\perp }{\xi }_x=x{x}^T\xi ,\forall x\in S^{n-1}.\end{array}$$

From Section 4 in [21], it follows that $R_x\left({\eta }_x\right)={\displaystyle \frac{x+{\eta }_x}{\parallel x+{\eta }_x\parallel }},\forall {\eta }_x\in T_x{S}^{n-1}$. The tangent space to $S^{n-1}$, viewed as a subspace of $T_x{\mathbb{R}}^n\simeq {\mathbb{R}}^n$, is

$$\begin{array}{c}\hfill T_x{S}^{n-1}=\{\xi \in {\mathbb{R}}^n:x^T\xi =0\}.\end{array}$$

The function is considered as follows:

$$\begin{array}{c}\hfill f:S^{n-1}\to \mathbb{R}:x\mapsto {\displaystyle \frac{1}{2}}{\parallel Ax-b\parallel }^2,\end{array}$$

(29)

and

$$\begin{array}{c}\hfill g:S^{n-1}\to \mathbb{R}:x\mapsto \mu {\parallel x\parallel }_1,\end{array}$$

(30)

on the unit sphere $S^{n-1}$, viewed as a Riemannian submanifold of the Euclidean space ${\mathbb{R}}^n$. Furthermore,

$$\begin{array}{c}\hfill \overline{f}:{\mathbb{R}}^n\to \mathbb{R}:x\mapsto {\displaystyle \frac{1}{2}}{\parallel Ax-b\parallel }^2,\end{array}$$

(31)

and

$$\begin{array}{c}\hfill \overline{g}:{\mathbb{R}}^n\to \mathbb{R}:x\mapsto \mu {\parallel x\parallel }_1,\end{array}$$

(32)

whose restriction to $S^{n-1}$ is f. From Section 4 in [21], it follows that

$$\begin{array}{c}\hfill \mathrm{grad}f\left(x\right)=P_x\mathrm{grad}\overline{f}\left(x\right)=P_x\left[A^T(Ax-b)\right]=(I-x{x}^T)A^T(Ax-b).\end{array}$$

(33)

It is easy to obtain

$$\begin{array}{c}\hfill {\mathrm{prox}}_{t\overline{g}}\left(x\right)=\mathrm{sign}\left(x\right)max\left\{\right|x|-t\mu ,0\}.\end{array}$$

(34)

From Algorithm 1, pick $L_k>0$, which is defined by (9),

$$\begin{array}{c}\hfill {\xi }_k:=-{\displaystyle \frac{1}{L_k}}\mathrm{grad}f\left(x^k\right)=-{\displaystyle \frac{1}{L_k}}(I-x^k{\left(x^k\right)}^T)A^T(A{x}^k-b),\end{array}$$

(35)

$$\begin{array}{c}\hfill {\eta }_k:=P_x\left({\mathrm{prox}}_{{\displaystyle \frac{1}{L_k}}{\overline{g}}_{x^k}}\left({\xi }_k\right)\right)=(I-x^k{\left(x^k\right)}^T)\mathrm{sign}\left({\xi }_k\right)max\left\{\right|{\xi }_k|-{\displaystyle \frac{1}{L_k}}\mu ,0\},\end{array}$$

(36)

and $x^{k+1}=R_{x^k}{\eta }_k={\displaystyle \frac{x^k+{\eta }_k}{\parallel x^k+{\eta }_k\parallel }}$. Set $\mu ={10}^{-3}$, and $L_k={\lambda }_{\max }\left(A^{T}A\right)$. Let $\left\{x^k\right\}$ be the sequence generated by Algorithm 1. It is easy to check that all assumptions of Theorem 4 are satisfied; so, the sequence $\left\{x^k\right\}$ converges to $x^{*}$ sublinearly.

5. Conclusions

The proximal gradient method is a popular optimization algorithm for solving composite optimization problems for Riemannian manifolds. In this context, the algorithm involves minimizing a sum of a smooth function and a nonsmooth function, where the function is defined for the Riemannian manifold. In this paper, the global convergence result of the proximal gradient method for the composite optimization problem is established, and the sublinear convergence rate and the complexity result of the proximal gradient method for convex case are also obtained for Riemannian manifolds. Future research directions involve establishing explicit asymptotic and non-asymptotic convergence rates and a numerically competitive proximal gradient method for the composite optimization problem for Riemannian manifolds. Furthermore, a version of the block proximal gradient method in which, at each iteration, a prox-grad step is performed at a randomly chosen block will be considered on Riemannian manifolds.