A Quantum-Behaved Particle Swarm Optimization Algorithm on Riemannian Manifolds

Abstract

The Riemannian manifold optimization algorithms have been widely used in machine learning, computer vision, data mining, and other technical fields. Most of these algorithms are based on the geodesic or the retracement operator and use the classical methods (i.e., the steepest descent method, the conjugate gradient method, the Newton method, etc.) to solve engineering optimization problems. However, they lack the ability to solve non-differentiable mathematical models and ensure global convergence for non-convex manifolds. Considering this issue, this paper proposes a quantum-behaved particle swarm optimization (QPSO) algorithm on Riemannian manifolds named RQPSO. In this algorithm, the quantum-behaved particles are randomly distributed on the manifold surface and iteratively updated during the whole search process. Then, the vector transfer operator is used to translate the guiding vectors, which are not in the same Euclidean space, to the tangent space of the particles. Through the searching of these guiding vectors, we can achieve the retracement and update of points and finally obtain the optimized result. The proposed RQPSO algorithm does not depend on the expression form of a problem and could deal with various engineering technical problems, including both differentiable and non-differentiable ones. To verify the performance of RQPSO experimentally, we compare it with some traditional algorithms on three common matrix manifold optimization problems. The experimental results show that RQPSO has better performance than its competitors in terms of calculation speed and optimization efficiency.

1. Introduction

Optimization algorithms are indispensable mathematical methods for solving various scientific and engineering optimization problems [1,2], such as the Hermitian eigenvalue problems [3], adaptive regularization [4], camera pose estimation [5], low-rank matrix approximation [6], and so on. In some of these practical engineering problems, their objective functions are often not in the complete Euclidean space but are restricted to some curves or surfaces [3,4,5,6]. In other words, the problem itself has no linear structure, and its solution space has a complex topology, which inevitably needs to be optimized on the manifold. For example, in the process of target tracking based on computer vision, digital images need to undergo a continuous set transformation, where the set of the geometric transformations of such images constitutes an SL Lie group [7]. The feature subspaces of videos or image sequences constitute the Grassmann manifold structure [8]. In some specific fields, such as image processing and data analysis, the data matrix has a semi-positive symmetric structure, so many problems can be modeled by the semi-positive symmetric matrix SDP manifold [9].

Matrix manifold optimization [10] is a minimization problem of a real-valued function of the Riemannian manifold, which can be stated as follows:

$$\underset{X\in \mathcal{M}}{\min }f\left(X\right),$$

(1)

where $X$ is the matrix variable.

In general, the Riemannian manifold is a geometric structure defined in the Riemann space. The Riemann space is different from the Euclidean space, so the operators defined in the Euclidean space are no longer applicable to the Riemann space. Optimization theory based on Euclidean space cannot be directly applied to the Riemannian manifold. However, the Riemannian manifold optimization theory can well overcome the limitations of the optimization algorithms on Euclidean space, which forms the unique advantages of the Riemannian manifold optimization algorithm:

• The iteration operation on the Riemannian manifold can reduce the modeling error and improve the expression ability of nonlinear problems;
• It can make full use of the intrinsic geometric structure of the objective function and embed the constraints into the search space to solve the unconstrained optimization problems based on the constrained search space, such as the low-rank matrix constraint problems and the positive semidefinite programming problems [11];
• Solving the target directly on the Riemannian manifold can reduce the damage to the original structural information caused by the vectorization of high-dimensional data.

With the development of Riemannian manifold optimization theory and related technologies, the Riemannian manifold optimization methods have been used in a variety of fields, such as big data, computer vision, target tracking, data mining, and so on [3,4,12,13,14,15].

At present, the commonly used Riemannian manifold optimization algorithms are based on the classical optimization methods, such as the steepest gradient descent method, the conjugate gradient method [16], and the Newton method [17], in the framework of the geodesic or retracement operator. These algorithms have good efficiency and local convergence for gradient-based optimization problems, but they still have shortcomings. Specifically, it is difficult to use the classical gradient-based methods directly because many problems, in reality, are non-smooth or non-differentiable. On the other hand, since most Riemannian manifold optimization problems are non-convex, the algorithms based on gradient descent would fall into the local optimum easily and thus cannot find the global optimum solution.

In order to overcome these algorithm defects, more and more researchers have focused on the improvement and development of Riemannian optimization algorithms. The Swarm Intelligence Algorithm (SIA) is a typical optimization algorithm [18]. It neither depends on the structural properties of the problem to be solved nor focuses on a specific problem. It has low requirements on the mathematical properties of the target problem and has strong global exploration ability. However, the original SIA designed in Euclidean space cannot be directly used for Riemannian manifold optimization. Therefore, in recent years, some researchers have begun to use the improved SIAs to solve black-box optimization problems on manifolds. For example, Borckmans et al. [19] introduced the Riemann gradient in the particle swarm optimization (PSO) algorithm and applied it to solve the low-rank tensor approximation problems. During the search process, the velocity vector of the particle is updated in the tangent space and then retracted to the manifold through the retracement operator. Colutto et al. [20] extended the Covariance Matrix Adaptation Evolutionary Strategies (CMA-ES) to the Riemannian manifold. It first synthesizes the functional on the minimized two-dimensional spherical surface and then reconstructs the shape of the three-dimensional spherical data. Compared with the manifold-based gradient method and the differential evolution algorithm (Differential Evolution, DE), the convergence rate of this algorithm is greatly improved. Arnold [21] combined the standard evolutionary strategy (ES) with the mutation strength adaptation mechanism and analyzed the convergence ability of the algorithm on spherical manifolds. Arnold [22] proposed an adaptive standard ES, which adaptively controls the complete covariance matrix of its offspring distribution, generating the mutation vector in the tangent space of the current population centroid. The points in the tangent space are mapped to the manifold by exponential mapping. Nevertheless, these SIAs still encounter problems of poor versatility, poor robustness, and low computational efficiency.

In 2004, Sun et al. [23] proposed the quantum-behaved particle swarm optimization (QPSO), inspired by the quantum mechanics and the trajectory analysis of PSO [24]. Because of its strong optimization performance, few control parameters, easy realization, and versatility, the QPSO algorithm has aroused the extensive interest of many researchers in solving real-world optimization problems. For example, it has been used to tackle the problems of constrained optimization [25], multi-objective optimization [26], neural network training [27], electromagnetic design [28], semiconductor design [29], image processing [30], and so on.

In this paper, we extend the QPSO algorithm from the Euclidean space to Riemannian manifolds and then propose a QPSO algorithm on Riemannian manifolds named RQPSO. It is tested on three kinds of Riemannian manifold optimization problems, including the positive semidefinite programming (SDP) problem [31], the secant-based dimension reduction (SDR) problem [32], and the robust principal component analysis (Robust PCA) problem [33].

The rest of the paper is organized as follows. Section 2 provides a brief introduction of the Riemannian manifold and the QPSO algorithm. Section 3 describes the details of our proposed RQPSO algorithm. The experimental results and analysis are presented in Section 4. Section 5 concludes the paper.

2. Preliminaries

2.1. The Riemannian Manifold

The Riemannian topological manifold, referred to as the Riemannian manifold, is a non-Euclidean geometric space that is locally isomorphic to European space and has a differential structure and Riemannian metric [10].

Stiefel Manifold: A Stiefel manifold is a set of matrices with any two columns orthogonal to each other, which can be defined as follows:

$$St\left(n,p\right)=\{X\in R^{n\times p}|X^{T}X=I_p\},$$

(2)

where $X$ represents the matrix variable, and $X^T$ is the transposed matrix of $X$. $n, p$ are the sizes of the rows and columns of the matrix, respectively, and $n>p$. ${ℝ}^{n\times p}$ is the $n\times p$-order real number space, and $I_p$ is the $p$-order identity matrix. The set of an $p$-order orthogonal matrix can be defined as follows:

$$\mathcal{O}\left(p\right)=\{X\in R^{p\times p}|X^{T}X=X{X}^T=I_p\}.$$

(3)

Grassmann manifold: Based on the above Stiefel manifold and $p$-order orthogonal matrix set, the quotient manifold definition of Grassmann manifold can be stated as follows:

$$Gr\left(n,p\right)=St\left(n,p\right)/\mathcal{O}\left(p\right).$$

(4)

For any two points $X_1,X_2\in St\left(n,p\right)$ belongs to an equivalent class, if and only if there is an orthogonal matrix $O\in \mathcal{O}\left(p\right)$, s.t. $X_1=X_{2}O$. We can use $\left[X\right]$ to represent an equivalent class in the Grassmann manifold; that is, $\left[X\right]$ is a point on the Grassmann manifold, and then $X\in St\left(n,p\right)$ is a representative element of the point $\left[X\right]$ on the Grassmann manifold. In other words, the Grassmann manifold is a set of $p$-dimensional subspaces in $R^n$, so another definition of the Grassmann manifold can be presented as follows:

$$Gr\left(n,p\right)=\left\{span\left(X\right)\right|X\in {ℝ}^{n\times p},X^{T}X=I_p\},$$

(5)

where $\mathit{span}\left(X\right)$ represents the subspace formed by the expansion of the $X$ columns.

Oblique manifold: When the column vector of the constraint matrix is normalized and does not constrain the angle between the columns, the Oblique manifold is formed, which is denoted as follows:

$$Ob\left(n,p\right)=\{X\in {ℝ}^{n\times p}|diag\left(X^{T}X\right)=I_p\},$$

(6)

where $diag$ is the diagonal element of the matrix.

2.2. Basic Operators on Manifold

The tangent space can translate one point on the manifold to the tangent space where another point is located; then both points are located in the same Euclidean space, making it possible to perform a Euclidean space operation.

Suppose that $\mathsf{\gamma }\left(\mathrm{t}\right)$ is a smooth curve defined on the manifold and has a smooth map:

$$\mathsf{\gamma }: \mathrm{R} \to \mathcal{M}, \mathrm{t}\to \mathsf{\gamma }\left(\mathrm{t}\right),$$

(7)

where $\mathsf{\gamma }$ is a smooth curve on the manifold $\mathcal{M}$ and $\mathsf{\gamma }\left(0\right)=\mathrm{X}$, then we can obtain

$${\mathrm{v}}_{\mathrm{X}}: {\Im }_{\mathrm{X}}\to \mathrm{R} , \mathrm{f}\to {\mathrm{v}}_{\mathrm{X}}\mathrm{f}=\frac{\mathrm{df}\left(\mathsf{\gamma }\left(\mathrm{t}\right)\right)}{\mathrm{dt}}{|}_{\mathrm{t}=0},$$

(8)

where ${\mathrm{v}}_{\mathrm{X}}$ is a map, represented as the tangent vector based on the curve $\mathsf{\gamma }\left(\mathrm{t}=0\right)$. $\mathrm{f}: \mathcal{M}\to \mathrm{R}$ is a function defined on the manifold, and ${\Im }_{\mathrm{X}}$ represents the functions of whole defined on the manifold.

The space composed of all tangent vectors at any point $\mathrm{X}$ on manifold $\mathcal{M}$ is called the tangent space about $\mathrm{X}$, denoted as ${\mathrm{T}}_{\mathrm{X}}\mathcal{M}$. It is a linear space which is tangent to point $\mathrm{X}$ on manifold $\mathcal{M}$ and regard $\mathrm{X}$ as the origin point.

When performing the operations (i.e., adding and subtracting) to the tangent vectors in different tangent spaces, it is necessary to translate one tangent vector into the tangent space where another tangent vector is located, but the properties of the tangent vectors cannot be changed during the translation process. This requires vector transfer, which is an operator that maintains the properties of the vectors on the manifolds.

The vector transfer on the manifold $\mathcal{M}$ is a smooth map,

$${\mathrm{Transp}}_{\mathrm{Y}\leftarrow \mathrm{X}}\left(\mathsf{\xi }\right): {\mathrm{T}}_{\mathrm{X}}\mathcal{M}\to {\mathrm{T}}_{\mathrm{Y}}\mathcal{M}\}.$$

(9)

It transfers the vector $\mathsf{\xi }$ from a point $\mathrm{X}$ to a point $\mathrm{Y}$ and has $\mathsf{\xi }\prime ={\mathrm{Transp}}_{\mathrm{Y}\leftarrow \mathrm{X}}\left(\mathsf{\xi }\right)$, where $\mathsf{\xi }\in {\mathrm{T}}_{\mathrm{X}}\mathcal{M}$, $\mathsf{\xi }\prime \in {\mathrm{T}}_{\mathrm{Y}}\mathcal{M}$. Taking $St\left(3,1\right)$ as an example, the vector transfer process is shown in Figure 1.

The retracement operator on the manifold $\mathcal{M}$ is a smooth mapping from the tangent space ${\mathrm{T}}_{\mathrm{X}}\mathcal{M}$ to $\mathcal{M}$, and

$${\mathrm{R}}_{\mathrm{X}}: {\mathrm{T}}_{\mathrm{X}}\mathcal{M}\to \mathcal{M}.$$

(10)

As shown in Figure 2, ${\mathrm{R}}_{\mathrm{X}}\left(\mathrm{Y}\right)$ is a point on the manifold, indicating that the point ${\mathrm{T}}_{\mathrm{X}}\mathcal{M}$ on the tangent space ${\mathrm{T}}_{\mathrm{X}}\mathcal{M}$ retracts to the corresponding point on the manifold, that is

$${\mathrm{R}}_{\mathrm{X}}\left(\mathrm{Y}\right)={\mathrm{R}}_{\mathrm{X}}\left(\mathrm{t}\mathsf{\zeta }\right),$$

(11)

where $\mathrm{Y}=\mathrm{X}+\mathrm{t}\mathsf{\zeta }$ is a random point on the tangent space ${\mathrm{T}}_{\mathrm{X}}\mathcal{M}$, $\mathsf{\zeta }$ is the tangent line on the tangent space ${\mathrm{T}}_{\mathrm{X}}\mathcal{M}$, and $\mathrm{t}$ is the retracement parameter.

Suppose that $\mathcal{M}$ is a smooth manifold with connections $\nabla$, and that $X\in \mathcal{M}$. For each $\xi \in T_X\mathcal{M}$, there is an open interval $H\ni 0$, and a unique geodesic $\gamma \left(t;X,\xi \right): H\to \mathcal{M}$, $\gamma \left(0\right)=X,\stackrel{·}{\gamma }\left(0\right)=\xi$. Meanwhile, the geodesic has the same centroid, i.e., $\gamma \left(t;X,a\xi \right)$= $\gamma \left(at;X,\xi \right)$, $a\in R$. The mapping is defined as follows:

$${\mathrm{Exp}}_X: T_X\mathcal{M}\to \mathcal{M}: \xi \to {\mathrm{Exp}}_X\left(\xi \right)=\gamma \left(1;X,\xi \right),$$

(12)

which is known as the exponential mapping at $X$.

Logarithmic mapping is the inverse operator of exponential mapping. It only exists when exponential mapping is reversible, which is defined as follows:

$$\begin{gathered} Lo{g}_X: \mathcal{M}\to T_X\mathcal{M}: Y\to Lo{g}_X\left(Y\right)=\xi \\ \mathrm{s}.\mathrm{t}. {\mathrm{Exp}}_X\left(\xi \right)=Y, {\Vert \xi \Vert }_X=\mathrm{dist}\left(X,Y\right) \end{gathered}$$

(13)

where ${\Vert \xi \Vert }_X$ is the norm of the tangent vector $\xi \in T_X\mathcal{M}$ at $X$. ${\Vert \xi \Vert }_X=\sqrt{\langle \xi ,{\xi \rangle }_X}$, $\langle \xi ,{\xi \rangle }_X$ is the inner product in the tangent space determined by $X$, and $\mathrm{dist}\left(X,Y\right)$ is the Riemannian distance on the manifold.

When given a starting point $X$ and a target point $Y$, the logarithmic mapping will return a tangent vector from $X$ to $Y$, and $\Vert Lo{g}_X\left(Y\right)=\mathrm{dist}\left(X,Y\right)\Vert$ (see Figure 2).

2.3. Test Problems

2.3.1. Semidefinite Programming Problem

Semidefinite programming (SDP) [34] is one of four standard convex optimization problems. In recent years, SDP has gradually become a new research hotspot. Many engineering problems can be modeled as SDP problems or approximated in the form of SDP.

SDP problems can be defined as follows:

$$\underset{X\in Ob\left(n,p\right)}{\min }f\left(X\right)=\frac{1}{2}\sum \left(\sum \left(AX\right)·X\right),$$

(14)

where $X\in R^{n\times p}$ is a point on $Ob\left(n,p\right)$, the symbol “$·$” represents point multiplication, and $A$ is a real symmetric matrix of $n\times n$.

Normally, for experiments, an upper triangular matrix $A$ with a value of 1 can be generated with the probability of 0.1, and then the final $A$ can be obtained in the form of $A=\frac{1}{2n}\left(A+A^T\right)$. Then, 30 test examples are selected from $n\in \left\{25,50,100,150,200,250\right\}$ and $p\in \left\{3,5,7,9,11\right\}$ for the test.

2.3.2. Secant-Based Data Dimensionality Reduction Problem

Secant-based Dimension Reduction (SDR) is a kind of optimization problem that finds a mapping from high-dimensional data, that is, a single map [35]. In this paper, it is implemented on the basis of the study in the references [32,36].

Assuming that $X\in R^{n\times N}$ is a data matrix, $\mathcal{S}$ is a subset of $R^n$ and $\Sigma =\left\{\frac{X_{:,i}-X_{:,j}}{{\Vert X}_{:,i}-X_{:,j}\Vert }|X_{:,i},X_{:,j}\in \mathcal{S},X_{:,i}\ne X_{:,j}\right\}$ is the set of unit secants of X, the SDR problem can be defined as minimizing the following function:

$$\underset{U\in Gr\left(n,p\right)}{\min }f\left(U\right)=-\underset{sϵ\Sigma }{\min }{\left|U{U}^{T}s\right|}_2$$

(15)

where $U$ is a point on the Grassmann manifold, $s$ is an element in the unit secant set. The data matrix is generated in the following way. $\mathrm{X}=\mathrm{Q}\Lambda \mathrm{Z},\mathrm{Q}\in {\mathrm{R}}^{\mathrm{n}\times \mathrm{r}}$ is a randomly generated orthogonal matrix, $\Lambda \in {\mathrm{R}}^{\mathrm{r}\times \mathrm{r}}$ is a diagonal matrix, and its diagonal elements are denoted as ${\Lambda }_{\mathrm{i},\mathrm{i}}={\mathsf{\beta }}^{1-\mathrm{i}}$, $\mathrm{Z}\in {\mathrm{R}}^{\mathrm{r}\times \mathrm{N}}$ is a random matrix, and each element is subject to $\mathrm{N}\left(0,1\right)$ distribution. $\mathsf{\beta }$ is the condition number of the control data matrix.

For the experiment, we set $\beta =1.05$, $n\in \left\{50,100,150,200,250,300\right\}$, $p\in \left\{3,5,7,9,11\right\}$, and 30 different test instances are generated on this benchmark.

2.3.3. Robust Principal Component Analysis

Principal Component Analysis (PCA) is a data dimensionality reduction algorithm that can remove redundant information in the data and retain the primary information. Similar to classical PCA, Robust PCA [33] is also a data dimensionality reduction in nature. However, when the data are huge or contain a certain amount of noise, PCA cannot achieve the expected effect, while Robust PCA is competent for such data.

Assuming that $A$ is the data matrix and $N$ is the sample size of the data, the problem can be defined as follows:

$$\underset{X\in Gr\left(n,p\right)}{\min }f\left(X\right)={{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}{{\displaystyle \sum }}_{\mathrm{j}=1}^p\left|A^{T}A-A^{T}X{X}^{T}A\right|,$$

(16)

where $\mathrm{X}$ is a point on the Grassmann manifold.

For the experiment, in order to generate the data matrix $\mathrm{A}$, we first need to generate a noise-free data matrix $\mathrm{B}\in {\mathrm{R}}^{\mathrm{n}\times \mathrm{N}}$, which satisfies $\mathrm{B}=\mathrm{Q}\Lambda \mathrm{Z}$. $\mathrm{Q}\in {\mathrm{R}}^{\mathrm{n}\times \mathrm{r}}$ is a randomly generated orthogonal matrix. $\Lambda \in {\mathrm{R}}^{\mathrm{r}\times \mathrm{r}}$ is the diagonal matrix, its diagonal elements are denoted as ${\Lambda }_{\mathrm{i},\mathrm{i}}={\mathsf{\beta }}^{1-\mathrm{i}}$, $\mathsf{\beta }$ is the attenuation index. $\mathrm{Z}\in {\mathrm{R}}^{\mathrm{r}\times \mathrm{N}}$ is the random matrix, and each element obeys the $\mathrm{N}\left(0,1\right)$ distribution. $\mathrm{r}\in \left[\mathrm{p}+1,\mathrm{n}\right]$ is the rank of matrix X. $\mathsf{\beta }>1$ determines the decay of the eigenvalues of the sample covariance matrix $\frac{{\mathrm{BB}}^{\mathrm{T}}}{\mathrm{N}}$. After obtaining the noise-free data matrix $\mathrm{B}$, we can insert the non-Gaussian noise (−1 or 1) into the data matrix $\mathrm{B}$ with a probability of $\frac{\mathsf{\theta }}{2}$, and then obtain the data matrix $\mathrm{A}$ containing noise. In addition, we set $N=200,\beta =1.5,r=p+10,\theta \in \left\{0.1,0.01,0.001\right\}$,$n\in \left\{50,100,150,200\right\}$, $p\in \left\{3,5,7,9,11\right\}$, and 36 different test instances are generated on this benchmark.

2.4. The Quantum-Behaved Particle Swarm Optimization Algorithm

Sun [23,37,38] et al. introduced the concept of quantum behavior into the standard PSO algorithm and proposed the QPSO algorithm. In the QPSO with $M$ particles, the position of each particle is updated as follows during the $t$th iteration.

$$p_i\left(t\right)=\phi \left(t\right)·P_i\left(t\right)+\left[1-\phi \left(t\right)\right]·G\left(t\right),$$

(17)

$$X_i\left(t+1\right)=p_i\left(t\right)\pm \alpha \left|C\left(t\right)-X_i\left(t\right)\right|·\ln \left[1/u_i\left(t\right)\right],$$

(18)

where $p_i\left(t\right)$ is the local focus of particle $i$, $P_i\left(t\right)$ is the personal best position of particle $i$, $G\left(t\right)$ is the global best position of the swarm, $X_i\left(t\right)$ is the position of particle $i$, $i=1,2,\cdots ,M.$ $\phi \left(t\right)$ and $u_i\left(t\right)$ are both random numbers randomly distributed on $\left(0,1\right)$, denoted as $\phi \left(t\right)~U\left(0,1\right)$,$u_i\left(t\right)~U\left(0,1\right)$. $\alpha$ is the contraction and expansion coefficient, and $C\left(t\right)$ is the mean best position, defined as

$$C\left(t\right)=\frac{1}{M}{\displaystyle \sum }_{i=1}^M{P}_i\left(t\right).$$

(19)

In the QPSO algorithm, there are two methods of iteration. One is to calculate the mean best position by Formula (19); the other is to calculate the mean best position by randomly selecting from the population. In this paper, we chose the latter one as the method of producing the mean best position, for it needs less calculation [39].

3. The Proposed RQPSO Algorithm

3.1. The QPSO Algorithm on Riemannian Manifold

As represented in Section 2.4, the QPSO algorithm can only work in Euclidean space. Specifically, the update of particles’ positions relies on the linear operators in Euclidean space. If the particles are distributed on the manifold, those positional update operators cannot be directly applied to the manifold. Therefore, in this paper, we extend the QPSO to manifolds and propose the RQPSO algorithm in order to solve the Riemannian manifold optimization problems.

The proposed RQPSO algorithm uses the retracement operator to project the to-be-update particles into the tangent space after initialization. Because the Riemann manifolds are local isomorphism in Euclidean space, the update operations of particles can be carried out in the tangent space. Then, the retracement operator retracts the updated particles back to the manifold. The details of the RQPSO algorithm are described as follows.

As for Equation (17), a point is randomly selected on the connecting line between the current best position $P_i\left(t\right)$ and the global best positions $G\left(t\right)$ in Euclidean space:

$$p_i\left(t\right)=R_{P_i\left(t\right)}\left(rand·Lo{g}_{P_i\left(t\right)}\left(G\left(t\right)\right)\right),$$

(20)

where $rand$ is a random number uniformly distributed on $\left(0,1\right)$, $Lo{g}_{P_i\left(t\right)}\left(G\left(t\right)\right)$ is the logarithmic mapping, which is expressed as the tangent vector from $P_i\left(t\right)$ to $G\left(t\right)$. $R_X\left(\lambda \xi \right)$ means that a tangent vector $\lambda \xi$ in the tangent space corresponding to $X$ is retracted to the manifold. and the random point $\mathrm{Z}$ between $\mathrm{X}$ and $\mathrm{Y}$ on the manifold $\mathcal{M}$ falls on the geodesic between the two points (see Figure 3).

For Equation (18), according to the $\delta$ potential well model, the position of a particle near can be defined as follows:

$$X=p\pm \frac{L}{2}\ln \left(\frac{1}{u}\right),$$

(21)

where $L$ is the characteristic length, and $u$ is a random number uniformly distributed on $\left(0,1\right)$, i.e., $u~U\left(0,1\right)$. In other words, this equation expresses the probability of particles in a quantum state appearing near point $p$, so, this equation can be rewritten as

$$L\left(t\right)=P_{p_i\left(t\right)}\left(Lo{g}_{X_i\left(t\right)}\left(C\left(t\right)\right)\right),$$

(22)

$$X_i\left(t+1\right)=R_{p_i\left(t\right)}\left(\alpha ·L\left(t\right)·\ln \left[1/u_i\left(t\right)\right]\right),u_i\left(t\right)~U\left(0,1\right),$$

(23)

where $L\left(t\right)$ represents the characteristic length and $Lo{g}_{X_i\left(t\right)}\left(C\left(t\right)\right)$ represents the tangent from $X_i\left(t\right)$ to $C\left(t\right)$.

Through the vector transfer, the tangent from $X_i\left(t\right)$ to $C\left(t\right)$ is translated into the tangent space corresponding to $p_i\left(t\right)$, and then the eigenvectors in the tangent space are retracted to the manifold using Equation (23).

During the $t$th iteration, the RQPSO algorithm is executed as follows:

1.

Randomly select a point from the population $P=\left\{P_1,P_2,\dots P_m\right\}$ as the mean best position $C\left(t\right)$;

2.

For particle $i, i=1,2,\cdots ,M$, randomly select a point along the geodesic line between $P_i$ and $G\left(t\right)$. Generate a tangent $\zeta =Lo{g}_{P_i}\left(G\left(t\right)\right)$ from $P_i$ to $G\left(t\right)$, and then randomly select a point on the tangent and retract it to $p_i\left(t\right)=R_{P_i}\left(rand\cdot \zeta \right)$ on the manifold;

3.

Generate a tangent $Q=P_{p_i\left(t\right)}\left(Lo{g}_{X_i^{\left(t\right)}}\left(C\left(t\right)\right)\right)$ in the same direction as that from $X_i^{\left(t\right)}$ to $C\left(t\right)$ at the point produced in (2). Then, retract $Q$ to $R_X\left(Q\right)$ on the manifold at point $X$ and obtain a new population.

The text continues here.

Algorithm 1 provides the pseudo-code of the proposed RQPSO algorithm.

Algorithm 1 Quantum Particle Swarm Optimization on Riemannian Manifold (RQPSO)

Input:  $\mathcal{M}$: matrix manifold; $f:R^{n\times p}\to R$: objective function; $R_X:T_X\mathcal{M}\to \mathcal{M}$: retracement operator; $Lo{g}_X:\mathcal{M}\to T_X\mathcal{M}:y\to Lo{g}_x\left(y\right)=\xi$: logarithmic map; $T_{y\leftarrow x}\left(\xi \right):T_x\mathcal{M}\to T_y\mathcal{M}$: vector transfer; Output:  The optimal solution  1:  $t\leftarrow 0$  2:  $X_i^{\left(t\right)}\leftarrow$ Randomly distributed matrix in $\mathcal{M}$, $i\in 1,2,\cdots ,m$  3:  $P_i=X_i^{\left(t\right)}\left(i=1,2,\dots ,m\right)$ the current personal best position of the particle  4:  $F_i^{\left(t\right)}=f(X_i^{\left(t\right)})\left(i=1,2,\dots ,m\right)$ the particles’ fitness value the current population  5:  $Fbes{t}_i=F_i^{\left(t\right)}\left(i=1,2,\dots ,m\right)$ the particles’ best fitness value the current population  6:  $Y^t=\min (F_i^{\left(t\right)})\left(i=1,2,\dots ,m\right)$ optimal fitness (optimal objective function value)  7:  $G\left(t\right)=P_g\left(t\right)$, where, $g=\arg \underset{1\le i\le m}{\min }f\left[P_i\left(t\right)\right]$  8:   while $(t<maxiter$ or $|Y^t-Y^{t-1}|\le {10}^{-6}$) do  9:    $C\left(t\right)\leftarrow$ Select one at random from $P_i$  10:     for $(i\leftarrow 1:m)$ do  11:      ${\zeta }_1=Lo{g}_{P_i}\left(G\left(t\right)\right)$  12:      $p=R_{P_i}\left({\zeta }_1\right)$  13:      ${\zeta }_2=Lo{g}_{X_i^{\left(t\right)}}\left(C\left(t\right)\right)$  14:      ${\zeta }_3=T_p\left({\zeta }_2\right)$  15:      if (rand () < 0.5)  16:       $X_i^{\left(t\right)}=R_p\left(log\left(1/rand\left(n,p\right)\right)\cdot {\zeta }_3\right)$  17:      else  18:       $X_i^{\left(t\right)}=R_p\left(-log\left(1/rand\left(n,p\right)\right)\cdot {\zeta }_3\right)$  19:      end if  20:      end for  21:      for $(i\leftarrow 1:m)$ do  22:       $F_i^{\left(t\right)}=f(X_i^{\left(t\right)})$  23:       if $(F_i^{\left(t\right)}<Fbes{t}_i)$  24:        $Fbes{t}_i=F_i^{\left(t\right)}$  25:        $P_i=X_i^{\left(t\right)}$  26:       end if  27:      end for  28:      if ($\min (F^{\left(t\right)})$ $<Y^t$)  29:       $Y^t=\min \left(F^{\left(t\right)}\right)$  30:       $u=index\left(\min \left(F^{\left(t\right)}\right)\right):$ $G\left(t\right)=X_u^{\left(t\right)}$  31:    end if  32:  end while  33:  return$Y^t$, $G\left(t\right)$

3.2. Quantum Behavior of the RQPSO Algorithm

As we know, the Riemannian manifold is locally isomorphic to Euclidean space. Generally, the dimensions of the problems are finite. Therefore, it can be assumed that the manifold where the problem is solved is locally isomorphic to an N-dimensional Euclidean space instead of being isomorphic to the infinite space. Since finite-dimensional Euclidean space is a special case of Hilbert space, the neighborhood of a point on the Riemannian manifold has the properties of Hilbert space. On the basis of this assumption, we can study the quantum behavior of particles on Riemannian manifolds.

In the quantum time-space framework, the quantum state of a particle is described by the wave function $\mathsf{\Psi }\left(X,t\right)$ [40]. In a three-dimensional space, the wave function $\mathsf{\Psi }\left(X,t\right)$ of a particle satisfies the formula as below.

$${\left|\mathsf{\Psi }\right|}^{2}dxdydz=Qdxdydz,$$

(24)

where $Qdxdydz$ is the probability that the particle will appear in the infinitesimal element about the point $\left(\mathrm{x},\mathrm{y},\mathrm{z}\right)$.

We assume that each single particle in QPSO is treated as a spin-less particle moving in an N-dimensional Hilbert space with a given energy, and thus its state is characterized by a wave function which only depends on the position of the particle. In this paper, we denote $P_i$ as $p$. With point $p$ being the center of the potential well, the potential energy of the particle in the one-dimensional $\mathsf{\delta }$ potential well is represented as follows:

$$V\left(X\right)=-\gamma \delta \left(X-p\right)=-\gamma \delta \left(Y\right),$$

(25)

where $Y=X-p$ and $\gamma$ is the intensity of the potential well.

According to Theorem 2 in reference [38], it can be known that if a particle moves in the bound state in the one-dimensional $\delta$ potential well, as described by Equation (25), its position can be determined by using the stochastic Equation (21). Since there is no addition or subtraction operation on the manifold, the addition and subtraction operation in Equation (21) is modified to the retraction operation of the feature vector in the tangent space where the particle is located. Therefore, the update of particles on the Riemannian manifold is characterized by quantum behavior, as in Euclidean space.

4. Experimental Studies

4.1. Parameters Analysis

As stated in Section 2.4 and Section 3, the RQPSO algorithm has two main parameters, i.e., $\phi$ and $\alpha$, where $\phi$ affects the search ability of quantum particles on the manifold, and $\alpha$ is the influence factor of the characteristic length $L$ in the vicinity of the local attractor $p$. The values of $\phi$ and $\alpha$ will affect the algorithm’s optimization performance on the manifold to a certain extent. Therefore, in this section, we experimentally investigate the effect of these two factors on the algorithmic performance of RQPSO.

4.1.1. The Effect of $\phi$ on the Performance of RQPSO

$\phi$ represents the distance from the current personal best position of a particle to the current global best position along the geodesic line in each iteration. As shown in Figure 4, if $\phi$ is too small, the particle deviates from the current global best position. The particle is mainly concentrated in the vicinity of its own personal best position, resulting in a decrease in the search ability of the particle. If $\phi$ is too large, the particle moves towards the current global best position, which may cause the particle to converge too fast and thus fall into the local best area. Therefore, it is important to select an appropriate value for $\phi$.

For the three test problems, we chose $n=50,p=3$ instances to test the parameter performance. We selected $\phi \left(t\right)\in U\left(0,0.1\right), \phi \left(t\right)\in U\left(0,0.5\right)$, and $\phi \left(t\right)\in U\left(0,1\right)$ for testing on the SDP problem. On the SDR and Robust PCA problems, we chose $\phi \left(t\right)\in U\left(0,0.01\right), \phi \left(t\right)\in U\left(0,0.1\right), \phi \left(t\right)\in U\left(0,0.5\right)$ and $\phi \left(t\right)\in U\left(0,1\right)$, fixed another parameter $\alpha =0.1$, and ran each test case 20 times to find the average value. The obtained test results are shown in Figure 5a–c.

According to the experimental results, we can find that on the Robust PCA problem and the SDP problem, when $\phi \left(t\right)\in U\left(0,1\right)$, the algorithm can converge faster. When $\phi \left(t\right)\in U\left(0,0.1\right)$, the algorithm can achieve better results in the SDR problem. The reason is that the PCA problem and the SDP problem have better properties than the SDR problem, which is a non-differentiable objective function. Therefore, when the particle is updated, it has a broader update range and can easily find the optimal solution. The SDR problem is a more complicated objective function, so that $\phi \left(t\right)$ cannot be set large, otherwise the swarm would hardly find the global optima. However, when the value of $\phi \left(t\right)$ is too small, the search speed is too slow, as shown in Figure 5b.

4.1.2. The Effect of $\alpha$ on the Performance of RQPSO

Based on the retracement operator framework, parameter $\alpha$ can be regarded as the step size in the gradient descent algorithm. As shown in Figure 6, if $\alpha$ is too large, the step size of the algorithm would be too large to meet the condition of local isomorphism. If $\alpha$ is too small, the moving distance of the algorithm would be too small, and the convergence speed would be too slow.

Therefore, we selected $\alpha$ parameters of different orders of magnitude in each test problem. Under each set parameter, the algorithm ran 20 times, and then the average values of the results during these 20 runs were calculated. The parameter with the minimized value and the fastest decreasing rate of the objective function was selected. The test cases were the same as those in Section 4.1.1, and we chose $\alpha =0.01, \alpha =0.1$, and $\alpha =0.5$. The experimental results are shown in Figure 7a–c.

4.1.3. The Effect of Parameters on the Performance of RQPSO

As mentioned above, $\alpha$ and $\phi$ are two parameters of the RQPSO algorithm, and the performance of the proposed algorithm depends on the choice of parameters $\alpha$ and $\phi$. In addition to studying the effect of the algorithm in different test problems in Section 4.1.1, in this paper, we will use the PCA problem as the result measurement function to study the effect of different parameter selection on the performance of the algorithm.

In this paper, the selection range of $\alpha$ and $\phi$ is $\left[0.01,0.05,0.1,0.3,0.5,0.7,0.9,1\right]$. According to the Taguchi method [41], there are two factors under investigation, and each factor has eight selection levels. On this basis, generating an orthogonal table, and then the experiment is carried out with the combination of different factors in the orthogonal table. Each factor combination is run 20 times to obtain the average value, and the obtained results are shown in

Table 1. The Effect of Parameters on the Performance of RQPSO in PCA problem.

	$\mathsf{\alpha }$	0.01	0.05	0.1	0.3	0.5	0.7	0.9	1	Average
$\mathsf{\phi }$
0.01		8.19 × 102	7.94 × 102	7.73 × 102	8.89 × 102	8.90 × 102	8.14 × 102	7.59 × 102	8.60 × 102	8.25 × 102
0.05		7.90 × 102	8.16 × 102	8.54 × 102	8.50 × 102	8.74 × 102	9.46 × 102	8.38 × 102	8.38 × 102	8.51 × 102
0.1		7.86 × 102	8.87 × 102	8.59 × 102	8.77 × 102	8.41 × 102	8.06 × 102	8.33 × 102	7.55 × 102	8.30 × 102
0.3		8.94 × 102	8.32 × 102	8.12 × 102	8.04 × 102	8.16 × 102	7.89 × 102	8.48 × 102	8.85 × 102	8.35 × 102
0.5		8.32 × 102	9.04 × 102	8.16 × 102	7.42 × 102	7.99 × 102	7.98 × 102	8.37 × 102	7.67 × 102	8.12 × 102
0.7		9.15 × 102	8.22 × 102	8.46 × 102	8.38 × 102	8.56 × 102	7.47 × 102	8.74 × 102	8.82 × 102	8.48 × 102
0.9		9.33 × 102	9.30 × 102	8.65 × 102	8.22 × 102	9.00 × 102	8.46 × 102	8.96 × 102	8.35 × 102	8.79 × 102
1		9.78 × 102	9.26 × 102	8.13 × 102	8.44 × 102	8.30 × 102	9.07 × 102	9.75 × 102	8.98 × 102	8.97 × 102
Average		8.68 × 102	8.64 × 102	8.30 × 102	8.33 × 102	8.51 × 102	8.32 × 102	8.58 × 102	8.40 × 102

.

It can be seen from Table 1 that when $\alpha =0.1$, the measurement result obtains the best average value, and when $\phi =0.5$, the measurement result obtains the best average value. According to the rest results in Section 4.1.1 and this section, for different problems, the algorithm can achieve good results when $\alpha =0.1$. However, the parameter of $\phi$ needs to be selected in combination with different Riemannian manifolds. In this paper, $\phi \in U\left(0,0.1\right)$ is selected for the experiment on the Oblique manifold, and $\phi \in U\left(0,1\right)$ is selected for the test on the Grassmann manifold.

4.2. Comparison Experiments

In this section, we chose the manifold-based Steepest Descent (SD), Particle Swarm Optimization (PSO), and Differential Evolution (DE) [15] as the comparison algorithms, and all of them, as well as the RQPSO, were tested on three sets of test problems mentioned before.

As for the RQPSO, we set $\alpha =0.1$ and $\phi \left(t\right)\in U\left(0,0.1\right)$ on the SDR problem, and $\alpha =0.1$ and $\phi \left(t\right)\in U\left(0,1\right)$ on both the SDP problem and robust PCA problem.

For SD, the steepest descent method on Riemannian manifolds is a generalization of the classical steepest descent method [10] on Riemannian manifolds. It is also a manifold optimization algorithm based on retracement. Since the search direction of the algorithm is in the steepest descent direction, we used the Armijo condition [42] to calculate the search step, and then retracted the forward-moving point to the manifold, and thus completed a descent process. The implementation of SD was conducted in the Manopt toolbox [43].

For PSO [19], it was also implemented in the Manopt toolbox with the default parameters. For DE, the mutation operation and crossover operation were implemented with a certain probability in the tangent space by using exponential mapping and logarithmic mapping. After that, the points were retracted to the manifold. According to the literature [44], the mutation rate was set to $P_F=0.4$ and the crossover rate was $1-P_F$. In addition, the scaling factor was set to $F=0.s7$ and the cross coefficient was $k=0.85$.

The experimental results are shown in tables from

Table 2. Average and quartile results of optimal values of four algorithms running 20 times in SDPproblem.

$\mathit{n}$	$\mathit{p}$	RQPSO	DE	PSO	SD
50	3	−6.1752 × 10−2(4.9 × 10−3)	−2.6914 × 10−2(3.2 × 10−3)	−2.6504 × 10−2(3.0 × 10−3)	−6.1607 × 10−2(4.7 × 10−3)
50	5	−1.0293 × 10−1(5.9 × 10−3)	−4.0609 × 10−2(5.1 × 10−3)	−3.5358 × 10−2(6.4 × 10−3)	−1.0289 × 10−1(5.8 × 10−3)
50	7	−1.4504 × 10−1(9.9 × 10−3)	−5.0532 × 10−2(8.0 × 10−3)	−4.3870 × 10−2(5.4 × 10−3)	−1.4231 × 10−1(1.2 × 10−2)
50	9	−1.9149 × 10−1(1.1 × 10−2)	−5.8637 × 10−2(1.4 × 10−2)	−5.2653 × 10−2(1.0 × 10−2)	−1.9150 × 10−1(1.1 × 10−2)
100	3	−4.5067 × 10−2(1.3 × 10−3)	−1.5092 × 10−2(2.3 × 10−3)	−1.2780 × 10−2(1.4 × 10−3)	−4.4553 × 10−2(1.0 × 10−3)
100	5	−7.2857 × 10−2(3.0 × 10−3)	−1.9977 × 10−2(3.8 × 10−3)	−1.8332 × 10−2(2.0 × 10−3)	−7.2205 × 10−2(3.2 × 10−3)
100	7	−1.0462 × 10−1(3.9 × 10−3)	−2.9372 × 10−2(3.2 × 10−3)	−2.5277 × 10−2(8.0 × 10−4)	−1.0509 × 10−1(3.9 × 10−3)
100	9	−1.3619 × 10−1(6.0 × 10−4)	−3.1572 × 10−2(4.5 × 10−3)	−2.8639 × 10−2(9.0 × 10−4)	−1.3589 × 10−1(1.1 × 10−3)
150	3	−3.6335 × 10−2(1.3 × 10−3)	−8.6850 × 10−3(1.0 × 10−3)	−8.8120 × 10−3(9.0 × 10−4)	−3.5789 × 10−2(1.3 × 10−3)
150	5	−6.0190 × 10−2(2.6 × 10−3)	−1.2079 × 10−2(1.6 × 10−3)	−1.2042 × 10−2(1.4 × 10−3)	−5.9307 × 10−2(2.8 × 10−3)
150	7	−8.6039 × 10−2(7.0 × 10−4)	−1.6084 × 10−2(5.3 × 10−3)	−1.6833 × 10−2(2.2 × 10−3)	−8.4907 × 10−2(2.0 × 10−3)
150	9	−1.1066 × 10−1(2.8 × 10−3)	−2.1269 × 10−2(3.2 × 10−3)	−1.9045 × 10−2(3.0 × 10−4)	−1.0970 × 10−1(2.6 × 10−3)
200	3	−3.1323 × 10−2(1.0 × 10−3)	−6.9490 × 10−3(1.7 × 10−3)	−7.1720 × 10−3(3.0 × 10−4)	−3.0667 × 10−2(9.0 × 10−4)
200	5	−5.3004 × 10−2(2.0 × 10−3)	−9.8660 × 10−3(2.2 × 10−3)	−9.9820 × 10−3(1.4 × 10−3)	−5.1632 × 10−2(3.4 × 10−3)
200	7	−7.2677 × 10−2(1.9 × 10−3)	−1.1306 × 10−2(2.7 × 10−3)	−1.2830 × 10−2(1.1 × 10−3)	−7.2031 × 10−2(1.9 × 10−3)
200	9	−9.3857 × 10−2(3.2 × 10−3)	−1.2133 × 10−2(3.7 × 10−3)	−1.5398 × 10−2(1.4 × 10−3)	−9.2505 × 10−2(4.1 × 10−3)
250	3	−2.8527 × 10−2(5.0 × 10−4)	−5.6330 × 10−3(1.6 × 10−3)	−5.8580 × 10−3(9.0 × 10−4)	−2.7597 × 10−2(1.1 × 10−3)
250	5	−4.7342 × 10−2(1.0 × 10−3)	−7.0910 × 10−3(1.5 × 10−3)	−8.5570 × 10−3(1.1 × 10−3)	−4.6141 × 10−2(1.2 × 10−3)
250	7	−6.5982 × 10−2(4.0 × 10−3)	−1.0577 × 10−2(9.0 × 10−4)	−1.0990 × 10−2(2.0 × 10−4)	−6.5406 × 10−2(4.3 × 10−3)
250	9	−8.4324 × 10−2(4.0 × 10−4)	−1.1266 × 10−2(3.4 × 10−3)	−1.3213 × 10−2(1.6 × 10−3)	−8.5312 × 10−2(4.0 × 10−4)

,

Table 3. Average and quartile results of optimal values of four algorithms running 20 times in SDR problem.

$\mathit{n}$	$\mathit{p}$	RQPSO	DE	PSO	SD
50	3	−5.1412 × 10−1(3.6 × 10−2)	−3.6914 × 10−1(3.9 × 10−2)	−2.2311 × 10−1(3.7 × 10−2)	−1.5535 × 10−1(4.0 × 10−2)
50	5	−7.2470 × 10−1(4.3 × 10−2)	−5.4939 × 10−1(4.4 × 10−2)	−3.4967 × 10−1(3.9 × 10−2)	−2.9375 × 10−1(1.3 × 10−1)
50	7	−8.4877 × 10−1(2.4 × 10−2)	−6.4097 × 10−1(4.4 × 10−2)	−4.2784 × 10−1(3.7 × 10−2)	−3.7782 × 10−1(1.4 × 10−1)
50	9	−9.5784 × 10−1(2.5 × 10−2)	−6.4802 × 10−1(2.5 × 10−2)	−4.7087 × 10−1(4.2 × 10−2)	−4.5747 × 10−1(2.7 × 10−2)
50	11	−9.7983 × 10−1(4.8 × 10−3)	−7.6750 × 10−1(3.8 × 10−2)	−5.2336 × 10−1(2.5 × 10−2)	−5.4464 × 10−1(5.9 × 10−2)
100	3	−4.5088 × 10−1(6.3 × 10−2)	−2.7348 × 10−1(2.8 × 10−2)	−1.6399 × 10−1(2.9 × 10−2)	−1.2822 × 10−1(6.4 × 10−2)
100	5	−7.0184 × 10−1(5.8 × 10−2)	−3.9794 × 10−1(3.5 × 10−2)	−2.3570 × 10−1(1.6 × 10−2)	−2.1470 × 10−1(5.6 × 10−2)
100	7	−8.4863 × 10−1(3.8 × 10−2)	−4.8878 × 10−1(5.2 × 10−2)	−2.8928 × 10−1(4.1 × 10−2)	−3.2030 × 10−1(1.2 × 10−1)
100	9	−9.3729 × 10−1(9.1 × 10−3)	−5.6107 × 10−1(5.5 × 10−2)	−3.3634 × 10−1(2.7 × 10−2)	−4.0881 × 10−1(6.0 × 10−2)
100	11	−9.6894 × 10−1(2.1 × 10−2)	−5.7446 × 10−1(2.5 × 10−2)	−3.6837 × 10−1(3.9 × 10−2)	−3.9402 × 10−1(3.8 × 10−2)
150	3	−4.5232 × 10−1(4.0 × 10−2)	−2.1901 × 10−1(1.5 × 10−2)	−1.3250 × 10−1(3.0 × 10−2)	−1.0771 × 10−1(4.3 × 10−2)
150	5	−7.1913 × 10−1(3.6 × 10−2)	−3.5151 × 10−1(2.1 × 10−2)	−1.8971 × 10−1(2.0 × 10−2)	−2.2270 × 10−1(4.9 × 10−2)
150	7	−8.4141 × 10−1(5.1 × 10−2)	−4.4002 × 10−1(1.7 × 10−2)	−2.4282 × 10−1(3.1 × 10−2)	−2.6939 × 10−1(1.5 × 10−1)
150	9	−9.3148 × 10−1(2.7 × 10−2)	−4.6331 × 10−1(2.5 × 10−2)	−2.6749 × 10−1(1.9 × 10−2)	−3.507 × 10−1(6.2 × 10−2)
150	11	−9.5823 × 10−1(1.9 × 10−2)	−5.4493 × 10−1(2.9 × 10−2)	−3.0970 × 10−1(2.1 × 10−2)	−3.9021 × 10−1(1.0 × 10−1)
200	3	−4.5178 × 10−1(4.9 × 10−2)	−1.8594 × 10−1(1.7 × 10−2)	−1.1556 × 10−1(1.9 × 10−2)	- 9.438 × 10−2(3.1 × 10−2)
200	5	−7.0589 × 10−1(4.6 × 10−2)	−3.0637 × 10−1(1.3 × 10−2)	−1.6907 × 10−1(1.2 × 10−2)	−1.9180 × 10−1(6.6 × 10−2)
200	7	−8.4805 × 10−1(4.5 × 10−2)	−3.8202 × 10−1(2.7 × 10−2)	−2.1149 × 10−1(1.3 × 10−2)	−2.5795 × 10−1(1.1 × 10−1)
200	9	−9.1661 × 10−1(4.3 × 10−2)	−4.2616 × 10−1(2.2 × 10−2)	−2.3520 × 10−1(2.9 × 10−2)	−3.1237 × 10−1(8.2 × 10−2)
200	11	−9.5415 × 10−1(2.1 × 10−2)	−4.3796 × 10−1(2.0 × 10−2)	−2.6385 × 10−1(2.2 × 10−2)	−3.1408 × 10−1(7.2 × 10−2)
250	3	−4.4076 × 10−1(5.2 × 10−2)	−1.6892 × 10−1(1.2 × 10−2)	−1.0321 × 10−1(2.2 × 10−2)	−1.1178 × 10−1(5.2 × 10−2)
250	5	−7.1519 × 10−1(6.4 × 10−2)	−2.6108 × 10−1(2.1 × 10−2)	−1.4771 × 10−1(1.7 × 10−2)	−1.6973 × 10−1(8.9 × 10−2)
250	7	−8.2571 × 10−1(4.9 × 10−2)	−3.4541 × 10−1(3.7 × 10−2)	−1.8112 × 10−1(1.2 × 10−2)	−2.3421 × 10−1(9.1 × 10−2)
250	9	−9.1166 × 10−1(5.3 × 10−2)	−3.7130 × 10−1(2.6 × 10−2)	−2.1360 × 10−1(2.2 × 10−2)	−2.8784 × 10−1(7.0 × 10−2)
250	11	−9.4470 × 10−1(3.3 × 10−2)	−3.9195 × 10−1(1.6 × 10−2)	−2.3519 × 10−1(2.0 × 10−2)	−3.1495 × 10−1(5.8 × 10−2)
300	3	−4.3421 × 10−1(4.8 × 10−2)	−1.5870 × 10−1(1.1 × 10−2)	−8.8721 × 10−2(1.9 × 10−2)	−1.0580 × 10−1(6.7 × 10−2)
300	5	−7.1296 × 10−1(3.0 × 10−2)	−2.5536 × 10−1(1.0 × 10−2)	−1.3469 × 10−1(2.6 × 10−2)	−2.0357 × 10−1(1.1 × 10−1)
300	7	−8.4278 × 10−1(4.1 × 10−2)	−3.1615 × 10−1(1.5 × 10−2)	−1.7673 × 10−1(2.4 × 10−2)	−2.4069 × 10−1(1.2 × 10−1)
300	9	−9.1008 × 10−1(2.5 × 10−2)	−3.6700 × 10−1(2.1 × 10−2)	−1.8787 × 10−1(2.5 × 10−2)	−2.2254 × 10−1(6.7 × 10−2)
300	11	−9.2532 × 10−1(1.6 × 10−2)	−3.6645 × 10−1(1.4 × 10−2)	−2.1657 × 10−1(1.6 × 10−2)	−2.9021 × 10−1(7.4 × 10−2)

and

Table 4. Average and quartile results of optimal values of four algorithms running 20 times in PCA problem.

$\mathit{\theta }$	$\mathit{n}$	$\mathit{p}$	RQPSO	DE	PSO	SD
0.1	50	3	7.5654 × 102(5.4 × 101)	8.7453 × 102(9.1 × 101)	9.6618 × 102(1.0 × 102)	7.6972 × 102(5.1 × 101)
0.1	50	5	5.8195 × 102(5.3 × 101)	7.1959 × 102(6.7 × 101)	9.2457 × 102(5.4 × 101)	5.9085 × 102(3.4 × 101)
0.1	50	7	5.2952 × 102(6.0 × 101)	6.8135 × 102(8.3 × 101)	9.5001 × 102(9.1 × 101)	5.0008 × 102(4.6 × 101)
0.1	50	9	4.1113 × 102(2.1 × 101)	5.4901 × 102(6.1 × 101)	8.2810 × 102(5.6 × 101)	3.8311 × 102(3.6 × 101)
0.1	50	11	3.3373 × 102(3.1 × 101)	4.9010 × 102(7.4 × 101)	8.0673 × 102(7.6 × 101)	3.2929 × 102(8.3 × 101)
0.1	100	3	1.3832 × 103(1.3 × 102)	1.56508 × 103(1.3 × 102)	1.6717 × 103(1.2 × 102)	1.3888 × 102(9.8 × 101)
0.1	100	5	1.1628 × 103(8.3 × 101)	1.4193 × 103(1.2 × 102)	1.7113 × 103(1.4 × 102)	1.1765 × 103(5.6 × 101)
0.1	100	7	9.1449 × 102(1.5 × 102)	1.2066 × 103(1.6 × 102)	1.5274 × 103(1.9 × 102)	9.1183 × 102(1.4 × 102)
0.1	100	9	8.3650 × 102(1.8 × 102)	1.1813 × 103(1.2 × 102)	1.5791 × 103(1.4 × 102)	8.3439 × 102(1.1 × 102)
0.1	100	11	7.2025 × 102(6.1 × 101)	1.0461 × 103(6.2 × 10¹)	1.5885 × 103(1.3 × 102)	6.9774 × 102(3.5 × 101)
0.1	150	3	1.9575 × 103(9.6 × 101)	2.2522 × 103(8.3 × 101)	2.3563 × 103(6.9 × 101)	1.9948 × 103(4.8 × 101)
0.1	150	5	1.6554 × 103(2.1 × 102)	2.0202 × 103(1.8 × 102)	2.2774 × 103(1.7 × 102)	1.6435 × 102(2.3 × 102)
0.1	150	7	1.4248 × 103(9.2 × 101)	1.8835 × 103(2.0 × 102)	2.2789 × 103(2.3 × 102)	1.4109 × 103(1.4 × 102)
0.1	150	9	1.1981 × 103(1.1 × 102)	1.7073 × 103(1.3 × 102)	2.1912 × 103(2.0 × 102)	1.2012 × 103(8.1 × 101)
0.1	150	11	1.0890 × 102(1.4 × 102)	1.6137 × 103(1.5 × 102)	2.1808 × 103(1.9 × 102)	1.1446 × 103(1.5 × 102)
0.1	200	3	2.4460 × 103(3.2 × 102)	2.7945 × 103(4.1 × 102)	2.9424 × 103(3.7 × 102)	2.4734 × 103(3.3 × 102)
0.1	200	5	2.1402 × 103(6.3 × 101)	2.6536 × 103(1.8 × 102)	2.9612 × 103(1.4 × 102)	2.1360 × 103(6.9 × 102)
0.1	200	7	1.8318 × 103(9.5 × 101)	2.4882 × 103(1.2 × 102)	2.8832 × 103(2.5 × 102)	1.8842 × 103(1.3 × 102)
0.1	200	9	1.6339 × 103(1.8 × 102)	2.3872 × 103(1.8 × 102)	2.9148 × 103(1.8 × 102)	1.6754 × 103(9.6 × 101)
0.1	200	11	1.4853 × 103(1.3 × 102)	2.3016 × 103(4.9 × 101)	2.8639 × 103(5.4 × 101)	1.6038 × 103(1.5 × 102)
0.01	50	3	7.1973 × 101(2.6 × 101)	1.5195 × 102(1.1 × 101)	2.0471 × 102(2.8 × 101)	7.4976 × 101(3.1 × 101)
0.01	50	5	5.1583 × 101(3.2 × 101)	1.0843 × 102(2.5 × 101)	1.7699 × 102(1.5 × 101)	6.2757 × 101(3.0 × 101)
0.01	50	7	4.7165 × 101(2.6 × 101)	8.3367 × 101(2.4 × 101)	1.7559 × 102(1.7 × 101)	5.9280 × 101(2.8 × 101)
0.01	50	9	2.9900 × 101(1.2 × 101)	5.5275 × 101(1.1 × 101)	1.3470 × 102(4.2 × 101)	3.7299 × 101(1.3 × 101)
0.01	50	11	3.2033 × 101(1.3 × 101)	4.9114 × 101(1.5 × 101)	1.0878 × 102(1.7 × 101)	4.0297 × 101(1.4 × 101)
0.01	100	3	1.2717 × 102(2.6 × 101)	2.6042 × 102(3.9 × 101)	2.9370 × 102(2.3 × 101)	1.3677 × 101(1.6 × 101)
0.01	100	5	9.7236 × 101(3.5 × 101)	2.0219 × 102(7.3 × 101)	2.7024 × 102(7.7 × 101)	1.1542 × 102(2.5 × 101)
0.01	100	7	7.3738 × 101(2.4 × 101)	1.5125 × 102(1.9 × 101)	2.3166 × 102(6.2 × 101)	9.2338 × 101(1.2 × 101)
0.01	100	9	6.9656 × 101(2.7 × 101)	1.5284 × 102(5.0 × 101)	2.8383 × 102(7.4 × 101)	8.5461 × 101(2.1 × 101)
0.01	100	11	5.6931 × 101(2.5 × 101)	1.1947 × 102(2.1 × 101)	2.1254 × 102(4.6 × 101)	9.1455 × 101(3.0 × 101)
0.01	150	3	1.7892 × 102(6.9 × 101)	3.5798 × 102(8.2 × 101)	4.0475 × 102(1.1 × 102)	1.9433 × 102(8.0 × 101)
0.01	150	5	1.4577 × 102(2.6 × 101)	2.8052 × 102(5.1 × 101)	3.4794 × 102(5,0 × 101)	1.6277 × 102(2.9 × 101)
0.01	150	7	1.5671 × 102(4.2 × 101)	2.8247 × 102(3.7 × 101)	3.6967 × 102(4.2 × 101)	1.7908 × 102(4.9 × 101)
0.01	150	9	1.1167 × 102(1.9 × 101)	2.2275 × 102(3.4 × 101)	3.1155 × 102(4.2 × 101)	1.4133 × 102(1.4 × 101)
0.01	150	11	1.0527 × 102(1.4 × 101)	1.9597 × 102(2.3 × 101)	2.9205 × 102(5.1 × 101)	1.4177 × 102(2.1 × 101)
0.01	200	3	2.0611 × 102(3.8 × 101)	4.2597 × 102(1.7 × 102)	4.4865 × 102(1.4 × 102)	2.1420 × 102(4.6 × 101)
0.01	200	5	1.5747 × 102(6.1 × 101)	3.2010 × 102(2.7 × 101)	3.5707 × 102(2.4 × 101)	1.7202 × 102(6.8 × 101)
0.01	200	7	1.5383 × 102(6.8 × 101)	3.0628 × 102(8.2 × 101)	3.9345 × 102(1.3 × 102)	1.7395 × 102(6.3 × 101)
0.01	200	9	1.6872 × 102(3.1 × 101)	3.4866 × 102(8.1 × 101)	4.3993 × 102(5.9 × 101)	2.1596 × 102(3.5 × 101)
0.01	200	11	1.6727 × 102(3.2 × 101)	2.8346 × 102(4.4 × 101)	3.5677 × 102(1.7 × 101)	2.0155 × 102(6.7 × 101)
0.001	50	3	2.5855 × 101(9.3 × 100)	1.2590 × 102(3.6 × 101)	1.5201 × 102(3.4 × 101)	3.4484 × 101(1.7 × 101)
0.001	50	5	8.3620 × 100(7.0 × 100)	6.1819 × 101(2.1 × 101)	1.2659 × 102(2.2 × 101)	9.3607 × 100(1.1 × 101)
0.001	50	7	6.7883 × 100(5.5 × 100)	3.8484 × 101(6.1 × 100)	1.0937 × 102(1.1 × 101)	6.4283 × 100(1.1 × 101)
0.001	50	9	6.4813 × 100(4.4 × 100)	2.5437 × 101(8.9 × 100)	9.5061 × 101(3.0 × 101)	6.5565 × 100(1.0 × 101)
0.001	50	11	3.6024 × 100(1.4 × 100)	1.1593 × 101(2.4 × 100)	7.9012 × 101(4.3 × 101)	2.1094 × 100(7.7 × 10−1)
0.001	100	3	2.8134 × 101(1.3 × 101)	1.9378 × 102(6.1 × 101)	2.3001 × 102(4.5 × 101)	2.9517 × 101(1.4 × 101)
0.001	100	5	1.5818 × 101(5.3 × 100)	1.8256 × 102(4.2 × 101)	2.4844 × 102(5.2 × 101)	2.1447 × 101(8.4 × 100)
0.001	100	7	1.1198 × 101(5.6 × 100)	1.1245 × 102(3.5 × 101)	1.7682 × 102(2.7 × 101)	1.2736 × 101(9.5 × 100)
0.001	100	9	1.3117 × 10¹(5.5 × 100)	9.3403 × 101(4.7 × 101)	1.8227 × 102(8.1 × 101)	1.8515 × 101(1.4 × 101)
0.001	100	11	7.6223 × 100(4.7 × 100)	6.2281 × 101(1.4 × 101)	1.4409 × 102(3.1 × 101)	1.0546 × 101(1.0 × 101)
0.001	150	3	3.7829 × 101(1.0 × 101)	2.2759 × 102(1.1 × 102)	2.5950 × 102(1.1 × 102)	4.2692 × 101(1.4 × 101)
0.001	150	5	2.7834 × 101(1.5 × 101)	2.1556 × 102(6.3 × 101)	2.7175 × 102(6.7 × 101)	4.0423 × 101(2.5 × 101)
0.001	150	7	1.4826 × 101(1.2 × 101)	1.3634 × 102(3.1 × 101)	1.9849 × 102(5.9 × 101)	1.9988 × 101(1.5 × 101)
0.001	150	9	1.6358 × 101(6.0 × 100)	1.5928 × 102(3.5 × 101)	2.4219 × 102(8.5 × 101)	3.1013 × 101(1.6 × 101)
0.001	150	11	1.8158 × 101(5.1 × 100)	1.2162 × 102(2.6 × 101)	1.9174 × 102(3.0 × 101)	2.4575 × 101(8.0 × 100)
0.001	200	3	3.2492 × 101(8.1 × 100)	2.6359 × 102(1.0 × 102)	2.8572 × 102(9.5 × 101)	3.8216 × 101(1.6 × 101)
0.001	200	5	2.6752 × 101(9.8 × 100)	2.1531 × 102(9.5 × 101)	2.5783 × 102(1.1 × 102)	3.6128 × 101(1.7 × 101)
0.001	200	7	2.6361 × 101(1.9 × 101)	1.8300 × 102(1.4 × 102)	2.2845 × 102(1.6 × 102)	3.4054 × 101(2.6 × 101)
0.001	200	9	1.8149 × 101(8.3 × 100)	1.6877 × 102(6.9 × 101)	2.1808 × 102(1.0 × 102)	2.7946 × 101(1.1 × 101)
0.001	200	11	2.2633 × 101(5.2 × 100)	1.5670 × 102(3.5 × 101)	2.0801 × 102(4.1 × 101)	3.0098 × 101(3.9 × 100)

and figures from Figure 8, Figure 9 and Figure 10.

It can be seen from Table 2 and Figure 8 that the effect of the RQPSO algorithm is significantly better than other comparison algorithms. This is due to the global search ability of RQPSO, which can efficiently find the global optimum. PSO is not a globally convergent algorithm [24], and it is easy to fall into the local optimum solution. The DE algorithm adopts the evolutionary algorithm framework, but it achieved the worst results because of its monotonous updating method and insufficient convergence ability. Compared with the PSO algorithm, the DE algorithm achieved poor results when the matrix dimension was high, for it is difficult to achieve good results through simple crossover and mutation. SD used the Riemannian gradient to search according to the gradient direction, achieving good results in solving the SDP problem. It can be seen from Table 2 that the SD is slightly better than the RQPSO algorithm in some examples, but its convergence speed is slower than RQPSO. In the RQPSO algorithm, particles randomly distributed on the manifold tend to move closer to the global optimal position $G\left(t\right)$. This helps particles along geodesics to find an optimal solution.

From Table 3 and Figure 9, it can be seen that the RQPSO algorithm has achieved significant results in all examples. Because RQPSO has no requirements for the differentiability of the objective function, it only needs to be updated with the fitness value of the objective function. In addition to the global convergence capability of the RQPSO algorithm, each instance of RQPSO on the SDR problem can achieve relatively good results. Although the PSO algorithm has no special requirements for the objective function, it cannot achieve good results in solving the problem because PSO lacks the ability of global convergence. As shown in Figure 4, Figure 5 and Figure 6a, the PSO algorithm achieves better results than SD in the case of $n=50$ and $p=3$, but in the case of $n=300$ and $p=11$ in Figure 9b, the experimental results of PSO and DE are poor, which is due to a disadvantage of PSO and DE in dealing with problems with high dimensions, especially the ones in manifold optimization [45,46]. RQPSO performed stably on both tests, which shows that RQPSO has relatively stable performance on both low-dimensional and high-dimensional optimization problems. DE algorithm achieved better results than PSO, to some extent, especially when the objective function is non-differentiable. Due to the non-differentiability of the SDR problem, it is difficult for SD to find the correct decline direction on this problem, so the performance is very different from that of the SDP problem and robust PCA problem. In Figure 9b, given the sufficient number of operations of each algorithm, SD may break through the problem of gradient disappearance when the gradient direction is bad so that the algorithm could find the direction to decline the objective function again. However, it is impossible to judge whether the SD algorithm falls into local minimum or gradient ill conditions in practice. Generally, it will be treated as the end of the search, so the SD algorithm cannot achieve better results than DE in simulation experiments, but it can achieve better results than PSO in some cases. In addition, it can be seen that the optimization performance of the PSO algorithm on the manifold is inefficient. RQPSO can break through the essential defects of the PSO algorithm and achieve good results in manifold optimization.

According to Table 4, we can see that RQPSO can achieve good results in most instances, and the performance of the SD algorithm is slightly better than the RQPSO algorithm in a small number of instances. In general, the RQPSO algorithm has a good optimization ability in dealing with PCA problems on manifolds. It can be seen from Table 4 that when $\mathsf{\alpha }=0.1$, the SD algorithm achieved slightly better results than RQPSO. When $\mathsf{\alpha }=0.01$ and $\mathsf{\alpha }=0.001$, RQPSO obtained better results than the SD in most instances. It shows that noise affects the performance of the RQPSO algorithm to some extent. The reason is that the noise with a certain probability affects the value of the objective function and thus affects the algorithm that only depends on the value of the objective function. From Figure 10a,b, it can be seen that the RQPSO algorithm achieved the best result. The convergence speed is comparable to the SD algorithm with the steepest descent method, which shows that RQPSO is a fast and efficient Riemannian manifold optimization algorithm. SD algorithm performed better on the PCA problem than on the first two problems, mainly due to the differentiability of the PCA problem [33]. DE achieved better results than PSO on this problem due to its evolutionary strategy. In addition, the PSO algorithm achieved the worst effect. The reason is that PSO has insufficient optimization ability to solve problems on high-dimensional matrix manifolds. However, the RQPSO algorithm can overcome its shortcoming to a great extent and is well-qualified for the optimization problem on the Riemannian manifold.

5. Conclusions

Given the current shortcomings of the optimization algorithms in solving Riemannian manifold optimization problems, this paper extended the QPSO algorithm from Euclidean space to Riemannian manifolds. A QPSO algorithm on the Riemannian manifold is proposed and named RQPSO. In this algorithm, the particle’s position is updated in the tangent space by using a guiding vector and vector transfer. The retracement operator is also employed to the updated position to the manifold for fitness evaluation. The operation process does not depend on the form of the optimization problem. In the process of optimization, the RQPSO algorithm only needs to use the fitness value of the objective function, which is independent of the form of the objective function. In addition, the historical and global best points of the particles distributed on the manifold guide the particles to move forward to the best position. The update of the particles does not depend on the local topology of the manifold, which can well avoid the particles falling into the local optimal area. The proposed algorithm was tested on three classical Riemannian manifold optimization problems, including the SDP problem, the SDR problem, and the Robust PCA problem. The experimental results showed that the RQPSO has better performance than its competitors. In the future, we will try to modify the updated strategies of RQPSO and extend them to more complex Riemannian manifold optimization problems.

The proposed algorithm uses particles as the unit in the update process. The particles search and update along the geodesic line, so the search range is reduced compared with the QPSO algorithm. In future research, the probability density distribution of particles will be added to the particle update process, and the generation and update of particles are guided by the probability density distribution function to enhance the search range of the particles.