An Inexact Feasible Quantum Interior Point Method for Linearly Constrained Quadratic Optimization

Abstract

Quantum linear system algorithms (QLSAs) have the potential to speed up algorithms that rely on solving linear systems. Interior point methods (IPMs) yield a fundamental family of polynomial-time algorithms for solving optimization problems. IPMs solve a Newton linear system at each iteration to compute the search direction; thus, QLSAs can potentially speed up IPMs. Due to the noise in contemporary quantum computers, quantum-assisted IPMs (QIPMs) only admit an inexact solution to the Newton linear system. Typically, an inexact search direction leads to an infeasible solution, so, to overcome this, we propose an inexact-feasible QIPM (IF-QIPM) for solving linearly constrained quadratic optimization problems. We also apply the algorithm to ${\ell }_1$-norm soft margin support vector machine (SVM) problems, and demonstrate that our algorithm enjoys a speedup in the dimension over existing approaches. This complexity bound is better than any existing classical or quantum algorithm that produces a classical solution.

1. Introduction

Linearly constrained quadratic optimization (LCQO) is defined as optimizing a convex quadratic objective function over a set of linear constraints. Linear optimization is a special case of LCQO that corresponds to the case where the objective function is linear. LCQO has rich theory, algorithms, and applications. Many problems in machine learning can be formulated as LCQO problems, including variants of least square problems and variants of support vector machine training [1,2]. Some important optimization algorithms also have LCQO subproblems, e.g., sequential quadratic programming [1].

The modern age of IPMs was launched by Karmarkar’s projective method for linear optimization (LO). Since then, many variants of IPMs have also been applied to nonlinear optimization problems, including LCQO problems [3,4]. Contemporary IPMs progress towards the set of optimal solutions by moving within a neighbourhood of an analytic curve known as the central path. IPMs can be categorized according to whether or not the the sequence of iterates produced by the algorithm satisfies feasibility. Feasible IPMs are initialized with a strictly feasible solution and maintain feasibility in each iteration, whereas infeasible IPMs start from an infeasible interior solution and do not require feasibility to be exactly satisfied at any point of the algorithm. For LCQO problems with n variables, feasible IPMs can produce an $ϵ$-approximate solution using $\mathcal{O}(\sqrt{n}log(1/ϵ))$ iterations, whereas infeasible IPMs require $\mathcal{O}(n^{2}log(1/ϵ))$ IPM iterations to converge to an $ϵ$-approximate solution [5,6].

At each IPM iteration, a linear system needs to be solved to obtain the search direction, called the Newton direction. This so-called Newton linear system is traditionally in the form of the augmented system or the normal equation system. Classically, these linear systems can be solved exactly using Bunch–Parlett factorization if the matrices in the systems are symmetric indefinite [7], or Cholesky factorization if the matrices are symmetric positive definite. Solving the Newton linear systems using direct factorization approaches requires the use of $\mathcal{O}\left(n^3\right)$ arithmetic operations, which suggests that feasible IPMs based on factoring methods cannot exhibit complexity better than $\mathcal{O}(n^{3.5}log(1/ϵ))$, whereas, with the partial update, they achieve $\mathcal{O}(n^{3}log(1/ϵ))$ arithmetic operation complexity. The linear systems can also be solved inexactly using some inexact methods, e.g., Krylov subspace methods, which may require fewer iterations if the desired accuracy of the solutions to the linear systems is not high. However, inaccurately solving the Newton linear systems (i.e., the inaccuracy of the search directions) may result in the infeasibility of the sequence of solutions generated by IPMs; therefore, they have only been used in infeasible IPMs.

The advent of quantum technology has led to the development of many quantum-assisted algorithms for optimization and machine learning applications, such as linear regression [8] and the support vector machine training problem [9]. Following the seminal work on quantum algorithms for solving linear systems of equations [10], researchers have been studying whether QLSAs could yield quantum speedups in classical optimization algorithms. In particular, quantum IPMs (QIPMs) that utilize QLSAs to solve the Newton linear system arising in each iteration have been proposed for LO problems [11,12] and semidefinite optimization problems [13]. To maintain the feasibility of the iterates using quantum subroutines, the authors of [13,14] introduce the so-called orthogonal subspace system (OSS) for SDO and LO problems, and, in particular, demonstrate that a feasible solution to the original Newton system can be recovered from an inexact solution to the OSS. However, linearly constrained quadratic optimization problems, which are fundamental to both optimization and machine learning, have yet to be formally studied in the quantum literature.

In this work, we generalize the OSS for LO problems in [14] to LCQO problems and provide an efficient method for constructing the OSS using a quantum computer. Using the OSS, we can obtain an inexact feasible IPM, solving for the search directions inexactly but maintaining the feasibility of the iterates throughout the process of our IPM. The feasibility of the iterates gives better IPM iteration complexity and the bottleneck becomes solving the linear system, OSS. In particular, we show that a quantum implementation of our algorithm with access to quantum RAM (QRAM) obtains an $ϵ$-approximate solution to a given LCQO problem with worst-case complexity

$${\tilde{\mathcal{O}}}_{n,\overline{\omega },\frac{1}{ϵ}}\left(\sqrt{n}\left(n\left(\frac{{\overline{\omega }}^2}{ϵ}+{\sigma }_{max}\left(Q\right)\right){\kappa }_{VAQ}+n^2\right)\right),$$

where $\overline{\omega }={max}_k{\omega }^k$, ${\sigma }_{max}\left(Q\right)$ is the maximum singular value of the Hessian of the objective function and ${\kappa }_{VAQ}$ is the condition number of a matrix determined by initial data; see Lemma 3. We also consider the application of ${\ell }_1$-norm soft margin SVM problems, in which case, an $ϵ$-approximate solution is obtained with complexity

$${\tilde{\mathcal{O}}}_{m,n,\overline{\omega },\frac{1}{ϵ}}\left({(m+n)}^{1.5}\left(\frac{{\overline{\omega }}^2}{ϵ}+{\sigma }_{max}\left(Q\right)\right){\kappa }_{VAQ}+{(m+n)}^{2.5}\right).$$

Here, m is the number of features and n is the number of data points. $\overline{\omega }$, Q, and ${\kappa }_{VAQ}$ are defined similarly from the LCQO formulation of the SVM problem; see Section 4. The dependence on dimension is better than any existing quantum or classical algorithm.

The rest of this paper is organized as follows: in Section 2, we introduce IPMs for LCQO and the OSS system; in Section 3, we discuss how to use quantum algorithms to find the Newton directions and analyze the complexity of our IF-QIPM; in Section 4, we apply our IF-QIPM to the support vector machine problem. Discussions are provided in Section 5, and some technical proofs are moved to the Appendix A and Appendix B.

2. Preliminaries

In this section, we introduce notations before reviewing the theory of IPMs applied to LCQO, and derive the OSS system for the class of problems.

2.1. Notation

Vectors are typically represented by lower-case letters. We write $0_n$ when referring to the n-dimensional all-zeros vector, and the n-dimensional all-ones vector is denoted by $e_n$. When the dimension is obvious from the context, we may write 0 or e, respectively. Matrices are typically represented with upper-case letters. The identity of dimension n is denoted by $I_{n\times n}$, and $0_{n\times m}$ represents the $n\times m$-dimensional all-zero matrix, again, dropping these subscripts when the dimension is obvious from the context. For a general $n\times m$-dimensional matrix H, we write $H_{i·}$ to refer to its ith row, and, similarly, denote the jth column by $H_{·j}$. For the $(i,j)$th element of H, we write $H_{ij}$ or $H_{i,j}$.

For real-valued functions $f_1$, $f_2$, and $f_3$, we write

$$f_1=\mathcal{O}\left(f_2\right)$$

if there exists a positive number $k_4$ such that $f_1\le k_4{f}_2$. We write

$$f_1={\tilde{\mathcal{O}}}_{f_3}\left(f_2\right)$$

if there exists a positive number $k_5$ such that $f_1\le k_5{f}_2\times \mathrm{poly}log\left(f_3\right)$.

2.2. IPMs for LCQO

In this work, LCQO is defined as follows.

 Definition 1

(LCQO Problem). For vectors $b\in {\mathbb{R}}^m$, $c\in {\mathbb{R}}^n$, and matrices $A\in {\mathbb{R}}^{m\times n}$ and $Q\in {\mathbb{R}}^{n\times n}$ with $rank\left(A\right)=m\le n$ and Q symmetric positive semidefinite, we define the primal and dual LCQO problems as:

$$\left(\mathrm{P}\right)\begin{array}{cc}\hfill min{c}^T& x+\frac{1}{2}{x}^{T}Qx,\hfill \\ \hfill \mathrm{s}.\mathrm{t}.Ax& =b,\hfill \\ \hfill x& \ge 0,\hfill \end{array}\left(\mathrm{D}\right)\begin{array}{cc}\hfill max{b}^{T}y-\frac{1}{2}{x}^{T}Q& x,\hfill \\ \hfill \mathrm{s}.\mathrm{t}.A^{T}y+s-Qx& =c,\hfill \\ \hfill s& \ge 0,\hfill \end{array}$$

(1)

where $x\in {\mathbb{R}}^n$ is the vector of primal variables, and $y\in {\mathbb{R}}^m$, $s\in {\mathbb{R}}^n$ are vectors of the dual variables. Problem $\left(P\right)$ is called the primal problem and $\left(D\right)$ is called the dual problem.

Since A is of full row-rank, A does not contain any null rows, and we further make the following assumption on matrix A.

Assumption 1. 

Matrix A has no all-zero columns.

Remark 1. 

Suppose that A has zero columns. Without a loss of generality, assume that the nth column is all-zero. Introducing a new variable $x_{n+1}$, we can rewrite the problem as

$$\begin{array}{cc}\hfill min{\left[\begin{array}{c}c\\ 0\end{array}\right]}^T\left[\begin{array}{c}x\\ x_{n+1}\end{array}\right]+\frac{1}{2}{\left[\begin{array}{c}x\\ x_{n+1}\end{array}\right]}^T\left[\begin{array}{cc}Q& 0_{n\times 1}\\ 0_{1\times n}& 0\end{array}\right]& \hfill \left[\begin{array}{c}x\\ x_{n+1}\end{array}\right],\\ \hfill \mathrm{s}.\mathrm{t}.\left[\begin{array}{ccccc}{A}_{·1}& \cdots & A_{·(n-1)}& 0_{m\times 1}& 0_{m\times 1}\\ 0& \cdots & 0& 1& -1\end{array}\right]\left[\begin{array}{c}x\\ x_{n+1}\end{array}\right]& \hfill =\left[\begin{array}{c}b\\ 0\end{array}\right],\\ \hfill x\ge 0,x_{n+1}& \hfill \ge 0.\end{array}$$

The new LCQO problem is equivalent to the original one, and contains fewer all-zero columns. Iterating this procedure to eliminate each of the all-zero columns, we obtain a new LCQO problem satisfying Assumption 1 with no more than $2n-m$ variables and n constraints in the worst case.

Assumption 2. 

There exists a solution $(x,y,s)\in {\mathbb{R}}^n\times {\mathbb{R}}^m\times {\mathbb{R}}^n$ such that

$$Ax=b,x>0,A^{T}y+s-Qx=c,\mathrm{and}s>0.$$

The set of primal–dual feasible solutions is defined as

$$\mathcal{PD}:=\left\{(x,y,s)\in {\mathbb{R}}^n\times {\mathbb{R}}^m\times {\mathbb{R}}^n:Ax=b,A^{T}y+s-Qx=c,(x,s)\ge 0\right\}$$

and, similarly, the set of interior feasible primal–dual solutions is given by

$${\mathcal{PD}}^0:=\left\{(x,y,s)\in {\mathbb{R}}^n\times {\mathbb{R}}^m\times {\mathbb{R}}^n:Ax=b,A^{T}y+s-Qx=c,(x,s)>0\right\}.$$

By strong duality, the set of optimal solutions can be characterized as

$${\mathcal{PD}}^{*}:=\left\{(x,y,s)\in \mathcal{PD}:xs=0\right\},$$

where $xs$ denotes the Hadamard, i.e., component-wise product of x and s. Let $ϵ>0$; then, the set of $ϵ$-approximate solutions to Problem (1) can be defined as

$${\mathcal{PD}}_{ϵ}:=\left\{(x,y,s)\in \mathcal{PD}:x^{T}s\le nϵ\right\}.$$

(2)

Let X and S be diagonal matrices of x and s, respectively. Under Assumption 2, for all $\mu >0$, the perturbed system of optimality conditions

$$\begin{array}{cc}\hfill Ax& =b,\hfill \\ \hfill A^{T}y+s-Qx& =c,\hfill \\ \hfill XSe& =\mu e,\hfill \\ \hfill (x,s)& \ge 0\hfill \end{array}$$

(3)

has a unique solution $\left(x\right(\mu ),y(\mu ),s(\mu \left)\right)$, and this set of solutions gives rise to the primal–dual central path

$$\mathcal{CP}:=\left\{(x,y,s)\in {\mathcal{PD}}^0|x_i{s}_i=\mu \mathrm{for}i\in \{1,\cdots ,n\};\mathrm{for}\mu >0\right\}.$$

IPMs apply Newton’s method to solve system (3). At each iteration of infeasible IPMs, a candidate solution to the primal–dual LCQO pair in (1) is updated by solving the following linear system to find the Newton direction:

$$\left[\begin{array}{ccc}A& 0& 0\\ -Q& A^T& I\\ S& 0& X\end{array}\right]\left[\begin{array}{c}\mathsf{\Delta }x\\ \mathsf{\Delta }y\\ \mathsf{\Delta }s\end{array}\right]=\left[\begin{array}{c}{r}_p\\ r_d\\ r_c\end{array}\right],$$

(4)

where

$$\begin{array}{cc}\hfill r_p& =b-Ax\hfill \\ \hfill r_d& =c-A^{T}y-s\hfill \\ \hfill r_c& =\sigma \mu e-XSe,\hfill \end{array}$$

are residuals, and $\sigma \in (0,1)$ is the barrier reduction parameter. If $r_p=0$ and $r_d=0$, then the solution $(x,y,s)$ exactly satisfies primal–dual feasibility. We can also define residuals in different ways as we will show later. Once the Newton direction is found, one can move along the direction but has to stay in a neighbourhood of the central path, which is defined as

$${\mathcal{N}}_2\left(\theta \right):=\left\{(x,y,s)\in {\mathcal{PD}}^0{|\parallel XSe-\mu e\parallel }_2\le \theta \mu \right\},$$

(5)

where $\theta \in (0,1)$.

Until relatively recently, inexact solution approaches to solve the Newton linear system (4) had only been utilized in inexact infeasible IPMs (II-IPMs). For LCQO problems, ref. [6] proposes an II-IPM using an iterative method to solve the Newton systems and obtains a worst-case iteration complexity $\mathcal{O}(n^{2}log\left(\frac{1}{ϵ}\right))$. On the other hand, feasible IPMs for LCQO problems enjoy $\mathcal{O}(\sqrt{n}log\left(\frac{1}{ϵ}\right))$ iteration complexity [15,16,17]. In [5], the author provides a general inexact feasible IPM for LCQO problems but does not discuss how the sequence of iterates could be guaranteed to maintain primal–dual feasibility exactly when using inexact linear system solvers. This is a vital consideration, as the feasible neighborhood of the central path as outlined in (5) is a subset of the primal–dual feasible set; if primal and dual feasibility are not satisfied exactly at any point in the algorithm, the iterates leave this neighborhood and the method fails. Our work fills this gap by using a method inspired by the QIPMs of [13,14].

2.3. Orthogonal Subspaces System

Assume that $(x,y,s)\in {\mathcal{PD}}^0$. To maintain the feasibility of the primal and dual variables, the first two linear equations in system (4) need to be solved with $r_p=0$ and $r_d=0$ exactly, which can be guaranteed if $\mathsf{\Delta }x$ lies in the null space of A, denoted as $Null\left(A\right)$, and $\mathsf{\Delta }s=Q\mathsf{\Delta }x-A^T\mathsf{\Delta }y$. Accordingly, we can rewrite system (4) by representing $\mathsf{\Delta }x$ as a linear combination of basis elements of $Null\left(A\right)$. To achieve this, we partition A as $A=\left[\begin{array}{cc}{A}_B& A_N\end{array}\right]$, where $A_B$ is a basis of A. Then, we construct the following matrix:

$$V=\left[\begin{array}{c}{A}_B^{-1}{A}_N\\ -I\end{array}\right].$$

Matrix V has a full column rank and satisfies $AV=0$, i.e., the columns of V span the null space of A. Let $\mathsf{\Delta }x=V\lambda$, where $\lambda \in {\mathbb{R}}^{n-m}$ is the unknown coefficient vector used to determine $\mathsf{\Delta }x$. Subsequently, we can rewrite system (4) by substituting $\mathsf{\Delta }x$ and $\mathsf{\Delta }s$ in the third equation as

$$\begin{array}{c}\hfill SV\lambda +X\left(QV\lambda -A^T\mathsf{\Delta }y\right)=r_c\iff \left[\begin{array}{cc}SV+XQV& -X{A}^T\end{array}\right]·\left[\begin{array}{c}\lambda \\ \mathsf{\Delta }y\end{array}\right]=r_c.\end{array}$$

(6)

A similar system was proposed and called “Orthogonal Subspaces System” (OSS) in [13,14], and we use the same name in this work. The matrix in the OSS system (6) is of size $n\times n$, and it is nonsingular. Even if the OSS system is solved inexactly, primal and dual feasibility are preserved by computing $\mathsf{\Delta }x=V\lambda$ and $\mathsf{\Delta }s=QV\lambda -A^T\mathsf{\Delta }y$. Thus, we can conclude that any inexactness will only impact the third equation of (4), i.e., $r_p=0$ and $r_d=0$. This property of the OSS system is very convenient when analyzing the proposed inexact IPM, and allows us to obtain the best known iteration complexity for IPMs.

3. Inexact Feasible IPM with QLSAs

In this section, we propose our IF-QIPM for LCQO problems. We begin with the IF-IPM structure introduced by [5] and describe how to quantize it into an IF-QIPM. Then, we analyze the construction of the OSS system and conclude by analyzing the overall complexity of our IF-QIPM.

3.1. IF-IPM for LCQO

In [5], the author studies a general conceptual form IF-IPM for QCLO problems by assuming the feasibility of the primal and dual iterates, which induces the following system:

$$\left[\begin{array}{ccc}A& 0& 0\\ -Q& A^T& I\\ S& 0& X\end{array}\right]\left[\begin{array}{c}\mathsf{\Delta }x\\ \mathsf{\Delta }y\\ \mathsf{\Delta }s\end{array}\right]=\left[\begin{array}{c}0\\ 0\\ r_c\end{array}\right],$$

(7)

where $r_c=\sigma \mu e-XSe$, with $\sigma \in (0,1)$ being the reduction factor of the central path parameter $\mu$, i.e., ${\mu }^{new}=\sigma \mu$. When system (7) is solved with $r_c=\sigma \mu e-XSe$ inexactly yielding an error r, if ${\parallel r\parallel }_2\le \delta {\parallel r_c\parallel }_2$ for some $\delta \in (0,1)$, the inexact IPM converges to an $ϵ$-approximate solution to Problem (1) in at most $\mathcal{O}(\sqrt{n}log(1/ϵ))$ iterations. As we mentioned earlier, it is not specified in [5] how to preserve primal and dual feasibility when system (7) is solved inexactly. Thus, it is presently not clear whether one could recover the convergence conditions described in [5] using inexact approaches, which are reliant on the assumption of primal–dual feasibility (see, e.g., system (7)).

Now, we present a general procedure of how to solve system (7) inexactly, while the inexactness error occurs only in the third equation of system (7). Let $(\lambda ,\mathsf{\Delta }y)$ be an inexact solution for system (6) and r be the error at this solution, i.e.,

$$\begin{array}{c}\hfill \left[\begin{array}{cc}SV+XQV& -X{A}^T\end{array}\right]·\left[\begin{array}{c}\lambda \\ \mathsf{\Delta }y\end{array}\right]=r_c+r.\end{array}$$

The corresponding Newton step

$$\begin{array}{cc}\hfill \mathsf{\Delta }x& =V\lambda \hfill \\ \hfill \mathsf{\Delta }s& =Q\mathsf{\Delta }x-A^T\mathsf{\Delta }y\hfill \end{array}$$

satisfies

$$\left[\begin{array}{ccc}A& 0& 0\\ -Q& A^T& I\\ S& 0& X\end{array}\right]·\left[\begin{array}{c}\mathsf{\Delta }x\\ \mathsf{\Delta }y\\ \mathsf{\Delta }s\end{array}\right]=\left[\begin{array}{c}0\\ 0\\ r_c+r\end{array}\right].$$

Recall that once $(\lambda ,\mathsf{\Delta }y)$ is determined, then $(\mathsf{\Delta }x,\mathsf{\Delta }s)$ is also (uniquely) determined. An interesting property is that, if $(\lambda ,\mathsf{\Delta }y)$ and $(\mathsf{\Delta }x,\mathsf{\Delta }y,\mathsf{\Delta }s)$ can be deduced from each other, then the OSS system and system (7) yield the same error term r. Hence, the convergence conditions built upon system (7) can be directly examined using the residual $r_c$ and error r of the OSS system. Let ${ϵ}_{OSS}$ be the target accuracy of the OSS system (6), i.e.,

$$\parallel \left(\lambda -{\lambda }^{*},\mathsf{\Delta }y-\mathsf{\Delta }{y}^{*}\right){\parallel }_2\le {ϵ}_{OSS},$$

where $({\lambda }^{*},\mathsf{\Delta }{y}^{*})$ is the accurate solution. According to [5], in order to guarantee that the IF-IPM converges, we must have

$$\begin{array}{cc}\hfill {\parallel r\parallel }_2& ={∥\left[\begin{array}{cc}SV+XQV& -X{A}^T\end{array}\right]·\left[\begin{array}{c}\lambda \\ \mathsf{\Delta }y\end{array}\right]-r_c∥}_2\hfill \\ \hfill & \le {∥\left[\begin{array}{cc}SV+XQV& -X{A}^T\end{array}\right]∥}_2{ϵ}_{OSS}\hfill \\ \hfill & \le \delta \parallel r_c{\parallel }_2,\hfill \end{array}$$

where $\delta \in (0,1)$ is a constant parameter. Therefore, to ensure the convergence of the IF-IPM, it suffices to set

$$\begin{array}{c}\hfill {ϵ}_{OSS}\le \delta \frac{\parallel r_c{\parallel }_2}{{∥\left[\begin{array}{cc}SV+XQV& -X{A}^T\end{array}\right]∥}_2}.\end{array}$$

The IF-IPM is presented in full detail in Algorithm 1. In each iteration, we build and solve system (6) classically. We solve system (6) to the accuracy just introduced above and then compute the feasible Newton step from the inexact solution and take a full Newton step.

Algorithm 1: Short-step IF-IPM

1: Choose $ϵ>0$, $\delta \in (0,1)$, $\theta \in (0,1)$, $\beta \in (0,1)$ and $\sigma =(1-\frac{\beta }{\sqrt{n}})$. 2: $k\leftarrow 0$ 3: Choose initial feasible interior solution $(x^0,y^0,s^0)\in \mathcal{N}\left(\theta \right)$ 4: while$(x^k,y^k,s^k)\notin {\mathcal{PD}}_{ϵ}$do 5:    ${\mu }^k\leftarrow \frac{{\left(x^k\right)}^T{s}^k}{n}$ 6:    ${ϵ}_{OSS}^k\leftarrow \delta {\parallel r_c^k\parallel }_2/{∥\left[\begin{array}{cc}{S}^{k}V+X^{k}Q{V}^k& -X^k{A}^T\end{array}\right]∥}_2$ 7:    $({\lambda }^k,\mathsf{\Delta }{y}^k)\leftarrow$solve system (6) with accuracy ${ϵ}_{OSS}^k$ 8:    $\mathsf{\Delta }{x}^k=V{\lambda }^k$ and $\mathsf{\Delta }{s}^k=-A^T\mathsf{\Delta }{y}^k$ 9:    $(x^{k+1},y^{k+1},s^{k+1})\leftarrow (x^k,y^k,s^k)+(\mathsf{\Delta }{x}^k,\mathsf{\Delta }{y}^k,\mathsf{\Delta }{s}^k)$ 10:    $k\leftarrow k+1$ 11: end while 12: return$(x^k,y^k,s^k)$

In the quantum-assisted IF-IPM, or IF-QIPM, we propose accelerating Step 7 using quantum subroutines. In the next sections, we investigate how to use quantum algorithms to build and solve the OSS system and obtain the Newton direction.

3.2. IF-QIPM for LCQO

The pseudocode of our IF-QIPM is presented in Algorithm 2. At each iteration of the IF-QIPM, we construct and solve system (6) and compute the Newton direction using quantum algorithms. To obtain an ${ϵ}_{OSS}$-approximate solution of system (6), we first block encode system (8); see Appendix A. Then, we use quantum algorithms to solve for an ${ϵ}_{QLSA}$-approximate solution of system (8). This solution is normalized but we can rescale it to obtain an ${ϵ}_{OSS}$-approximate solution of system (6). Details are discussed later in this section.

Algorithm 2: Short-step IF-QIPM

1: Choose $ϵ>0$, $\delta \in (0,1)$, $\theta \in (0,{\theta }_0)$, $\beta \in (0,1)$ and $\sigma =(1-\frac{\beta }{\sqrt{n}})$. 2: $k\leftarrow 0$ 3: Choose initial feasible interior solution $(x^0,y^0,s^0)\in \mathcal{N}\left(\theta \right)$ 4: while$(x^k,y^k,s^k)\notin {\mathcal{PD}}_{ϵ}$do 5:    ${\mu }^k\leftarrow \frac{{\left(x^k\right)}^T{s}^k}{n}$ 6:    ${ϵ}_{OSS}^k\leftarrow \delta {\parallel r_c^k\parallel }_2/\left(\sqrt{2}{∥\left[\begin{array}{cc}{S}^{k}V+X^{k}Q{V}^k& -X^k{A}^T\end{array}\right]∥}_2\right)$ 7:    $({\lambda }^k,\mathsf{\Delta }{y}^k)\leftarrow$solve system (6) with accuracy ${ϵ}_{OSS}^k$ quantumly 8:    $\mathsf{\Delta }{x}^k=V{\lambda }^k$ and $\mathsf{\Delta }{s}^k=-A^T\mathsf{\Delta }{y}^k$ 9:    $(x^{k+1},y^{k+1},s^{k+1})\leftarrow (x^k,y^k,s^k)+(\mathsf{\Delta }{x}^k,\mathsf{\Delta }{y}^k,\mathsf{\Delta }{s}^k)$ 10:    $k\leftarrow k+1$ 11: end while 12: return$(x^k,y^k,s^k)$

Here, ${\theta }_0<1$ and its value will be discussed later. First, we introduce some notations to simplify the OSS system. In the kth iteration of Algorithm 2, let

$$\begin{array}{c}\hfill M^k=\left[\begin{array}{cc}{S}^{k}V+X^{k}QV& -X^k{A}^T\end{array}\right],z^k=\left[\begin{array}{c}{\lambda }^k\\ \mathsf{\Delta }{y}^k\end{array}\right].\end{array}$$

Then, the OSS system can be rewritten as

$$M^k{z}^k=r_c^k.$$

As discussed in [14], to solve the OSS system (6) using quantum algorithms, we can first rewrite it as the normalized Hermitian OSS system

$$\begin{array}{c}\hfill \frac{1}{\sqrt{2}{∥M^k∥}_F}\left[\begin{array}{cc}0& M^k\\ {\left(M^k\right)}^T& 0\end{array}\right]·\left[\begin{array}{c}0\\ z^k\end{array}\right]=\frac{1}{\sqrt{2}{∥M^k∥}_F}.\end{array}\left[\begin{array}{c}{r}_c^k\\ 0\end{array}\right].$$

(8)

To use the QLSAs mentioned earlier, we need to turn the linear system (8) into a quantum linear system using the block encoding introduced in [18]. To this end, we first decompose the coefficient matrix in linear system (8) as

$$\begin{array}{cc}\hfill \frac{1}{\sqrt{2}{∥M^k∥}_F}\left[\begin{array}{cc}0& M^k\\ {\left(M^k\right)}^T& 0\end{array}\right]& =\frac{1}{\sqrt{2}{∥M^k∥}_F}\left[\begin{array}{cc}0& 0\\ {\left(M^k\right)}^T& 0\end{array}\right]+\frac{1}{\sqrt{2}{∥M^k∥}_F}\left[\begin{array}{cc}0& M^k\\ 0& 0\end{array}\right],\hfill \end{array}$$

(9)

where

$$\begin{array}{cc}\hfill \left[\begin{array}{cc}0& 0\\ {\left(M^k\right)}^T& 0\end{array}\right]=& \left[\begin{array}{ccc}{0}_{n\times n}& 0_{n\times n}& 0_{n\times n}\\ 0_{(n-m)\times n}& V^T& 0_{(n-m)\times n}\\ 0_{m\times n}& 0_{m\times n}& -A\end{array}\right]\times \hfill \\ \hfill & \left(\left[\begin{array}{cc}{0}_{n\times n}& 0_{n\times n}\\ S^k& 0_{n\times n}\\ 0_{n\times n}& 0_{n\times n}\end{array}\right]+\left[\begin{array}{ccc}{0}_{n\times n}& 0_{n\times n}& 0_{n\times n}\\ 0_{n\times n}& Q& 0_{n\times n}\\ 0_{n\times n}& 0_{n\times n}& I_{n\times n}\end{array}\right]\left[\begin{array}{cc}{0}_{n\times n}& 0_{n\times n}\\ X^k& 0_{n\times n}\\ X^k& 0_{n\times n}\end{array}\right]\right).\hfill \end{array}$$

(10)

To compute matrix V, we need to find a basis matrix $A_B$ of matrix A and we need to compute the inverse matrix $A_B^{-1}$. Both steps are nontrivial and can be expensive. However, we can reformulate the LCQO problem as follows:

$$\begin{array}{cc}\hfill min{c}^{T}x& +\frac{1}{2}{x}^{T}Qx\hfill \\ \hfill \mathrm{s}.\mathrm{t}.\left[\begin{array}{ccc}I& 0& A\\ 0& I& -A\end{array}\right]& \left[\begin{array}{c}{x}^{\prime }\\ x^{″}\\ x\end{array}\right]=\left[\begin{array}{c}b\\ -b\end{array}\right]\hfill \\ \hfill x\ge 0,x^{\prime }& \ge 0,x^{″}\ge 0.\hfill \end{array}$$

In this case, we have an obvious basis

$$A_B=\left[\begin{array}{cc}I& 0\\ 0& I\end{array}\right]$$

and matrix V can be constructed efficiently

$$V=\left[\begin{array}{c}{A}_B^{-1}{A}_N\\ -I\end{array}\right]=\left[\begin{array}{c}\left[\begin{array}{cc}I& 0\\ 0& I\end{array}\right]\left[\begin{array}{c}A\\ -A\end{array}\right]\\ -I\end{array}\right]=\left[\begin{array}{c}A\\ -A\\ -I\end{array}\right].$$

Since matrix A has no all-zero rows, matrix V has no all-zero rows either. This property of the reformulation is useful in the analysis of the proposed IF-QIPM but we do not want to build the complexity analysis on the reformulated problem. Thus, without a loss of generality we may make the following assumption.

Assumption 3. 

Matrix A is of the form$A=\left[\begin{array}{cc}I& A_N\end{array}\right]$.

To simplify the analysis, we further assume that the input data are integers.

Assumption 4. 

The input data of Problem (1) are integers.

Based on the two assumptions above, we have the following lemma.

Lemma 1. 

Matrix V equals

$$V=\left[\begin{array}{c}{A}_N\\ -I\end{array}\right]$$

and

$$\underset{i=1,\cdots ,n}{min}\{\parallel V_{i·}{\parallel }_2^2\}=min\{1,\underset{i=1,\dots ,m}{min}\parallel {\left(A_N\right)}_{i·}{\parallel }_2^2\}=1,$$

where $V_{i·}$ and ${\left(A_N\right)}_{i·}$ are the ith row of V and $A_N$, respectively.

Now, we are ready to give ${\theta }_0$ in our definition of the central path neighborhood; see (5). We set

$${\theta }_0=min\left\{\frac{1}{3\sqrt{n}},\frac{1}{4{∥QV{V}^T∥}_F+1}\right\}.$$

(11)

We also define ${\omega }^k$ as the maximum of the values of primal variables and dual slack variables in the kth iteration.

Definition 2. 

Let $(x^k,y^k,s^k)$ be a candidate solution for Problem (1); then,

$${\omega }^k=\underset{i\in \{1,\cdots ,n\}}{max}\{x_i^k,s_i^k\}.$$

As is standard in the literature on quantum algorithms, in this work, we assume access to quantum random access memory (QRAM). Then, Step 7 of Algorithm 2 consists of three parts: (1). use block encoding to build system (8); (2). use QLSAs to solve system (8); (3). use quantum tomography algorithms (QTAs) to extract the classical solution. We use the block-encoding methods introduced in [18] to block-encode linear system (8).

Proposition 1. 

In the kth iteration of Algorithm 2, using the block-encoding methods introduced in [18] and the decomposition described in Equations (9) and (10), a

$$\begin{array}{c}\left(\frac{\sqrt{{\parallel V\parallel }_F^2+{\parallel A\parallel }_F^2}\sqrt{2}{\omega }^k}{\parallel M^k{\parallel }_F}(\sqrt{2}{\parallel Q\parallel }_F+\sqrt{2}+1),\mathcal{O}(polylog\left(n\right)),\frac{{ϵ}_{QLSA}}{{\kappa }_{M^k}^3}\right)\end{array}$$

-block-encoding of the matrix in system (8) can be implemented efficiently and the complexity will be dominated by the complexity of the QLSA step. Here,${ϵ}_{QLSA}$is the accuracy required for the QLSA step and${\kappa }_{M^k}$is the condition number of matrix$M^k$.

Proof. 

See Appendix A for proof. □

Provided access to QRAM, the complexity associated with block encoding the OSS system coefficient matrix and preparing a quantum state encoding the right hand side amounts to polylogarithmic overhead. The cost of these steps is therefore negligible when compared with the complexity contributed by QLSAs and QTAs, so we ignore it here. To bound the total complexity contributed by QLSAs and QTAs, we first need to analyze the accuracy of QLSA characterized by ${ϵ}_{QLSA}$, the accuracy of QTA characterized by ${ϵ}_{QTA}$, and their relationship.

In each iteration, we use a QLSA to solve the block-encoded version of system (8) and obtain an ${ϵ}_{QLSA}$-approximate solution. Then, we use a QTA to extract an ${ϵ}_{QTA}$-approximate solution from the quantum machine. In the context of QLSAs and QTAs, if $\tilde{z}$ is an $ϵ$-approximate solution of z, then $\tilde{z}$ satisfies

$${∥\frac{\tilde{z}}{\parallel \tilde{z}{\parallel }_2}-\frac{z}{{\parallel z\parallel }_2}∥}_2\le ϵ$$

Observe that this definition of accuracy differs from the concept of $ϵ$-approximate solutions defined in (2).

Similar to [12,13], the QLSA we use is proposed by [19] and the QTA we use is proposed by [20]. Following the argument in Section 2 in [12], we can establish the relationship among ${ϵ}_{QLSA}$, ${ϵ}_{QTA}$, and ${ϵ}_{OSS}^k$ as

$$\begin{array}{c}\hfill {ϵ}_{QLSA}={ϵ}_{QTA}=\frac{1}{2}·\frac{\sqrt{2}{\parallel M^k\parallel }_F}{\parallel r_c^k{\parallel }_2}{ϵ}_{OSS}^k,\end{array}$$

(12)

where ${ϵ}_{OSS}^k$ is defined as the ${\ell }_2$ norm of the residual when solving system (8) inexactly in the kth iteration. This coefficient is also used to rescale the solution. According to [12], we rescale the normalized solution obtained from QLSA and QTA by

$$\frac{\parallel r_c^k{\parallel }_2}{\sqrt{2}{\parallel M^k\parallel }_F}$$

to obtain the ${ϵ}_{OSS}^k$-approximate solution for system (6). Here, we did not add superscript to ${ϵ}_{QLSA}$ and ${ϵ}_{QTA}$, and the reason shall be revealed later. Let

$$\left[\begin{array}{c}{\tilde{0}}^k\\ {\tilde{z}}^k\end{array}\right]$$

be an inexact solution for system (8) in the kth iteration. Then, the norm of residual of system (8), which is ${ϵ}_{OSS}^k$, and the norm of residual of system (6), which is $\parallel M^k{\tilde{z}}^k-r_c^k{\parallel }_2$, satisfies

$$\begin{array}{cc}\hfill {ϵ}_{OSS}^k& ={∥\frac{1}{\sqrt{2}{\parallel M^k\parallel }_F}\left[\begin{array}{cc}0& M^k\\ {\left(M^k\right)}^T& 0\end{array}\right]\left[\begin{array}{c}{\tilde{0}}^k\\ {\tilde{z}}^k\end{array}\right]-\frac{1}{\sqrt{2}{\parallel M^k\parallel }_F}\left[\begin{array}{c}{r}_c^k\\ 0\end{array}\right]∥}_2\hfill \\ \hfill & ={∥\frac{1}{\sqrt{2}{\parallel M^k\parallel }_F}\left[\begin{array}{c}{M}^k{\tilde{z}}^k\\ {\left(M^k\right)}^T{\tilde{0}}^k\end{array}\right]-\frac{1}{\sqrt{2}{\parallel M^k\parallel }_F}\left[\begin{array}{c}{r}_c^k\\ 0\end{array}\right]∥}_2\hfill \\ \hfill & \ge {∥\frac{1}{\sqrt{2}{\parallel M^k\parallel }_F}{M}^k{\tilde{z}}^k-\frac{1}{\sqrt{2}{\parallel M^k\parallel }_F}{r}_c^k∥}_2\hfill \\ \hfill & \ge \frac{1}{\sqrt{2}{\parallel M^k\parallel }_F}{\parallel M^k{\tilde{z}}^k-r_c^k\parallel }_2.\hfill \end{array}$$

Recall that the error arising from the OSS system (6) is the same as the error in the full Newton system (7); then, we can directly use the convergence condition in [5], i.e.,

$$\parallel M^k{\tilde{z}}^k-r_c^k{\parallel }_2\le \delta {\parallel r_c^k\parallel }_2.$$

We can require

$$\parallel M^k{\tilde{z}}^k-r_c^k{\parallel }_2\le \sqrt{2}\parallel M^k{\parallel }_F{ϵ}_{OSS}^k\le \delta {\parallel r_c^k\parallel }_2$$

and it follows that

$${ϵ}_{OSS}^k\le \frac{\delta \parallel r_c^k{\parallel }_2}{\sqrt{2}{∥M^k∥}_F}.$$

Then, choosing

$${ϵ}_{OSS}^k=\frac{\delta \parallel r_c^k{\parallel }_2}{\sqrt{2}{∥M^k∥}_F}\mathrm{and}{ϵ}_{QLSA}={ϵ}_{QTA}=\frac{\parallel M^k{\parallel }_F{ϵ}_{OSS}^k}{\sqrt{2}{\parallel r_c^k\parallel }_2}=\frac{\delta }{2}$$

ensures the convergence of the IF-QIPM. The complexities for each step are also available now. Using the QLSA from [19] and QTA from [20], we have the complexity for QLSA and QTA:

$$\begin{array}{cc}\hfill T_{QLSA}& ={\tilde{\mathcal{O}}}_{n,\overline{\omega },\frac{1}{ϵ}}\left({\kappa }_{M^k}\frac{{\omega }^k}{\parallel M^k{\parallel }_F}\right),\hfill \\ \hfill T_{QTA}& ={\tilde{\mathcal{O}}}_n\left(\frac{n}{{ϵ}_{QTA}}{T}_{QLSA}\right)={\tilde{\mathcal{O}}}_{n,\overline{\omega },\frac{1}{ϵ}}\left(n{\kappa }_{M^k}\frac{{\omega }^k}{\parallel M^k{\parallel }_F}\right).\hfill \end{array}$$

Since we have ${ϵ}_{QTA}={\displaystyle \frac{\delta }{2}}$ and $\delta \in (0,1)$ is a constant parameter, we omit ${ϵ}_{QTA}$ in the Big-O notation. Note that the complexity of the block-encoding procedure is dominated by that of QLSA and QTA and thus we ignore the complexity contributed by block encoding. In Step 8, the complexity contributed by computing Newton step from OSS solution is $\mathcal{O}\left(n^2\right)$. The total complexity for the kth iteration of IF-QIPM will be

$$\mathcal{O}\left(T_{QTA}+n^2\right)={\tilde{\mathcal{O}}}_{n,\overline{\omega },\frac{1}{ϵ}}\left(\frac{n{\omega }^k{\kappa }_{M^k}}{\parallel M^k{\parallel }_F}+n^2\right).$$

(13)

3.2.1. Bound for ${\omega }^k/{\parallel M^k\parallel }_F$

In this section, all of the quantities that we consider are from the kth iteration. For simplicity, we omit the superscript k in this section unless we need it. Using the property of trace, we have

$$\begin{array}{cc}\hfill {\parallel M\parallel }_F^2& =tr\left(M^{T}M\right)\hfill \\ \hfill & =tr\left((SV+XQV){(SV+XQV)}^T+X{A}^{T}AX\right)\hfill \\ \hfill & =tr\left((SV+XQV){(SV+XQV)}^T\right)+tr\left(X{A}^{T}AX\right)\hfill \\ \hfill & =tr\left(SV{V}^{T}S\right)+tr\left(XQV{V}^{T}S\right)+tr\left(SV{V}^{T}QX\right)+tr\left(XQV{V}^{T}QX\right)+tr\left(X{A}^{T}AX\right).\hfill \end{array}$$

For the non-symmetric term, due to the cyclic invariant property of trace, we have

$$tr\left(XQV{V}^{T}S\right)=tr\left(SXQV{V}^T\right).$$

Recalling the central path neighborhood that we defined in (5), we define a matrix E such that

$$E=\frac{1}{\mu \theta }(XS-\mu I).$$

(14)

It is obvious that E is a diagonal matrix and satisfies

$${\parallel Ee\parallel }_2<1,$$

which leads to

$$|tr\left(E\right)|\le {\parallel Ee\parallel }_1\le \sqrt{n}{\parallel E\parallel }_F=\sqrt{n}{\parallel Ee\parallel }_2<\sqrt{n}\mathrm{and}I-E\succ 0\mathrm{and}I+E\succ 0.$$

With this, we can have

$$\begin{array}{cc}\hfill tr\left(XQV{V}^{T}S\right)& =tr\left(SXQV{V}^T\right)\hfill \\ \hfill & =tr\left((\theta \mu E+\mu I)QV{V}^T\right)\hfill \\ \hfill & =tr\left(\theta \mu EQV{V}^T\right)+tr\left(\mu QV{V}^T\right).\hfill \end{array}$$

For the second term, we know that Q and $V^{T}QV$ are both positive semidefinite. Thus, we can have

$$tr\left(QV{V}^T\right)=tr\left(V^{T}QV\right)\ge 0$$

because of the cyclic invariant property of trace. According to the Cauchy–Schwarz inequality, we have

$$tr{\left(EQV{V}^T\right)}^2\le {\parallel E\parallel }_F^2{\parallel QV{V}^T\parallel }_F^2.$$

Thus, we have

$$tr\left(EQV{V}^T\right)\ge -{\parallel QV{V}^T\parallel }_F.$$

Thus, we have

$$\begin{array}{cc}\hfill tr\left(XQV{V}^{T}S\right)& =tr\left(\theta \mu EQV{V}^T\right)+tr\left(\mu QV{V}^T\right)\hfill \\ \hfill & \ge \mu \left(tr\left(QV{V}^T\right)-\theta {\parallel QV{V}^T\parallel }_F\right)\hfill \\ \hfill & \ge -\theta \mu \parallel QV{V}^T{\parallel }_F\hfill \\ \hfill & \ge -\frac{\mu }{4},\hfill \end{array}$$

where the last inequality holds due to condition (11). Thus, we can bound ${\parallel M\parallel }_F$ by

$$\begin{array}{cc}\hfill {\parallel M\parallel }_F^2=& tr\left(SV{V}^{T}S\right)+tr\left(XQV{V}^{T}S\right)+tr\left(SV{V}^{T}QX\right)\hfill \\ \hfill & +tr\left(XQV{V}^{T}QX\right)+tr\left(X{A}^{T}AX\right)\hfill \\ \hfill \ge & tr\left(SV{V}^{T}S\right)+tr\left(XQV{V}^{T}QX\right)+tr\left(X{A}^{T}AX\right)-\frac{\mu }{2}.\hfill \end{array}$$

Since $XQV{V}^{T}QX⪰0$, we have

$$\begin{array}{cc}\hfill {\parallel M\parallel }_F^2& \ge tr\left(SV{V}^{T}S\right)+tr\left(X{A}^{T}AX\right)-\frac{\mu }{2}.\hfill \end{array}$$

Since X and S are both positive diagonal matrices, we have

$$\begin{array}{cc}\hfill {\parallel M\parallel }_F^2& \ge tr\left(SV{V}^{T}S\right)+tr\left(X{A}^{T}AX\right)-\frac{\mu }{2}\hfill \\ \hfill & =\sum _i{s}_i^2{\left(V{V}^T\right)}_{ii}+\sum _i{x}_i^2{\left(A^{T}A\right)}_{ii}-\frac{\mu }{2}\hfill \\ \hfill & \ge {\omega }^2-\frac{\mu }{2}.\hfill \end{array}$$

As we mentioned in the very beginning of this section, at each iteration, $\omega$ is indeed ${\omega }^k$, but the superscript is ignored here. Now, we aim to find a bound for $\mu$ so we can further bound ${\parallel M\parallel }_F^2$. Since $\omega$ is the upper bound for the magnitude of the primal and dual slack variables, we have

$$\begin{array}{cc}\hfill {\omega }^2& \ge x_i{s}_i.\hfill \end{array}$$

Recall the definition of matrix E; see (14). Thus, we have

$$\begin{array}{cc}\hfill {\omega }^2& \ge x_i{s}_i=\mu +\theta \mu E_{ii}\ge \mu -\theta \mu =(1-\theta )\mu .\hfill \end{array}$$

Thus,

$$\begin{array}{cc}\hfill {\parallel M\parallel }_F^2& \ge {\omega }^2-\frac{\mu }{2}\ge {\omega }^2-\frac{1}{2}\frac{{\omega }^2}{1-\theta }\ge {\omega }^2-\frac{1}{2}\frac{{\omega }^2}{1-1/3}=\frac{{\omega }^2}{4},\hfill \end{array}$$

where the last inequality follows from the bound for $\theta$; see (11). Thus, we have

$$\begin{array}{c}\hfill \frac{\omega }{{\parallel M\parallel }_F}\le 2=\mathcal{O}\left(1\right).\end{array}$$

3.2.2. Bound for ${\kappa }_{M^k}$

Similar to the previous section, we ignore the superscript k unless we need it. We will start with a general result and then work on the matrix $M^k$. The following lemma is a well-known result regarding condition numbers of matrices and can be proven using Courant–Fischer–Weyl min-max principle [21].

Lemma 2. 

For any full row rank matrix $P\in {\mathbb{R}}^{m\times n}$ and symmetric positive definite matrix $D\in {\mathbb{R}}^{n\times n}$, their condition number satisfies

$$\kappa \left(PD{P}^T\right)\le \kappa \left(D\right)\kappa \left(P{P}^T\right).$$

Next, we analyze the matrix in the OSS system (8). Specifically, we focus on $M^{T}M$ since we are interested in the spectral property of the OSS system (8). Using the matrix E defined in (14), we have the following decomposition:

$$\begin{array}{cc}\hfill M^{T}M& =\left[\begin{array}{cc}{V}^T{(S+XQ)}^T(S+XQ)V& -V^T{(S+XQ)}^{T}X{A}^T\\ -AX(S+XQ)V& A{X}^2{A}^T\end{array}\right]\hfill \\ \hfill & =\left[\begin{array}{cc}{V}^T{(S+XQ)}^T(S+XQ)V& -V^T\mu \left(\theta E\right)A^T-V^T{Q}^T{X}^2{A}^T\\ -A\mu \left(\theta E\right)V-A{X}^{2}QV& A{X}^2{A}^T\end{array}\right]\hfill \\ \hfill & =\left[\begin{array}{cc}{V}^T& 0\\ 0& A\end{array}\right]\left[\begin{array}{cc}{(S+XQ)}^T(S+XQ)& -\mu \theta E-Q{X}^2\\ -\mu \theta E-X^{2}Q& X^2\end{array}\right]{\left[\begin{array}{cc}{V}^T& 0\\ 0& A\end{array}\right]}^T.\hfill \end{array}$$

The second equality holds because

$$\begin{array}{cc}\hfill -V^{T}SX{A}^T-V^T{Q}^T{X}^2{A}^T& =-V^T\mu \left(I+\theta E\right)A^T-V^T{Q}^T{X}^2{A}^T\hfill \\ \hfill & =-V^T\mu \left(\theta E\right)A^T-V^{T}Q{X}^2{A}^T,\hfill \end{array}$$

as $AV=0$ and Q is symmetric. Then, plugging (14) into the first diagonal block of the decomposition we obtained earlier, we have

$$\begin{array}{cc}\hfill M^{T}M=& \left[\begin{array}{cc}{V}^T& 0\\ 0& A\end{array}\right]\left(\left[\begin{array}{cc}{S}^2+2\mu Q+\mu \theta (EQ+QE)+Q{X}^{2}Q& -\mu \theta E-Q{X}^2\\ -\mu \theta E-X^{2}Q& X^2\end{array}\right]\right){\left[\begin{array}{cc}{V}^T& 0\\ 0& A\end{array}\right]}^T\hfill \\ \hfill =& \left[\begin{array}{cc}{V}^T& 0\\ 0& A\end{array}\right]\left(\left[\begin{array}{cc}{S}^2+2\mu Q+\mu \theta (EQ+QE)& -\mu \theta E\\ -\mu \theta E& 0\end{array}\right]+\left[\begin{array}{cc}Q{X}^{2}Q& -Q{X}^2\\ -X^{2}Q& X^2\end{array}\right]\right){\left[\begin{array}{cc}{V}^T& 0\\ 0& A\end{array}\right]}^T\hfill \\ \hfill =& \left[\begin{array}{cc}{V}^T& 0\\ 0& A\end{array}\right]\left(\left[\begin{array}{cc}I& -Q\\ 0& I\end{array}\right]\left[\begin{array}{cc}{S}^2+2\mu Q& -\mu \theta E\\ -\mu \theta E& 0\end{array}\right]\left[\begin{array}{cc}I& 0\\ -Q& I\end{array}\right]\right){\left[\begin{array}{cc}{V}^T& 0\\ 0& A\end{array}\right]}^T\hfill \\ \hfill & +\left[\begin{array}{cc}{V}^T& 0\\ 0& A\end{array}\right]\left(\left[\begin{array}{cc}I& -Q\\ 0& I\end{array}\right]\left[\begin{array}{cc}0& 0\\ 0& X^2\end{array}\right]\left[\begin{array}{cc}I& 0\\ -Q& I\end{array}\right]\right){\left[\begin{array}{cc}{V}^T& 0\\ 0& A\end{array}\right]}^T\hfill \\ \hfill =& \left[\begin{array}{cc}{V}^T& 0\\ 0& A\end{array}\right]\left[\begin{array}{cc}I& -Q\\ 0& I\end{array}\right]\left[\begin{array}{cc}{S}^2+2\mu Q& -\mu \theta E\\ -\mu \theta E& X^2\end{array}\right]\left[\begin{array}{cc}I& 0\\ -Q& I\end{array}\right]{\left[\begin{array}{cc}{V}^T& 0\\ 0& A\end{array}\right]}^T.\hfill \end{array}$$

The first two matrices are nonsingular, so we can apply the Lemma 2, and thus we only need to study the middle matrix. Denote the middle matrix by $\mathrm{\Psi }$. Observe that $\mathrm{\Psi }$ is almost the same as its counterpart in [14]. Subsequently, we have the following result regarding the spectral property of $M^k$.

Lemma 3. 

When $(x,y,s)\in \mathcal{N}\left(\theta \right)$ and $\theta \in \left(0,min\left\{\frac{1}{3\sqrt{n}},\frac{1}{4\parallel QV{V}^T{\parallel }_F+1}\right\}\right)$, the condition number of matrix $M^k$ satisfies

$${\kappa }_{M^k}=\mathcal{O}\left(\frac{{\left({\omega }^k\right)}^2+{\mu }^k{\sigma }_{max}\left(Q\right)}{{\mu }^k}{\kappa }_{VAQ}\right),$$

where${\kappa }_{VAQ}$is the condition number of the matrix$\left[\begin{array}{cc}{V}^T& 0\\ 0& A\end{array}\right]\left[\begin{array}{cc}I& -Q\\ 0& I\end{array}\right]$.

Proof. 

The proof is in Appendix B. □

Putting all of these together, we have the complexity for our IF-QIPM for LCQO problems.

Theorem 1. 

The IF-QIPM for LCQO problems stops with the final duality gap less than ϵ in at most$\mathcal{O}\left(\sqrt{n}log(1/ϵ)\right)$IPM iterations and, in each IPM iteration, the Newton direction can be obtained with complexity${\tilde{\mathcal{O}}}_{n,\overline{\omega },\frac{1}{ϵ}}\left(n\left(\frac{{\overline{\omega }}^2}{ϵ}+{\sigma }_{max}\left(Q\right)\right){\kappa }_{VAQ}+n^2\right)$, where$\overline{\omega }={max}_k{\omega }^k$.

Proof. 

The complexity bound for the IPM iterations comes from the result in [5]. According to (13), the complexity for obtaining the Newton direction is

$$\begin{array}{c}\hfill {\tilde{\mathcal{O}}}_{n,\overline{\omega },\frac{1}{ϵ}}\left(\frac{n{\omega }^k{\kappa }_{M^k}}{\parallel M^k{\parallel }_F}+n^2\right).\end{array}$$

Combining this with the result in Section 3.2.1, the bound in Lemma 3, and ${\mu }^k\ge ϵ$, we have

$$\begin{array}{c}\hfill {\tilde{\mathcal{O}}}_{n,\overline{\omega },\frac{1}{ϵ}}\left(\frac{n{\omega }^k{\kappa }_{M^k}}{\parallel M^k{\parallel }_F}+n^2\right)={\tilde{\mathcal{O}}}_{n,\overline{\omega },\frac{1}{ϵ}}\left(n\left(\frac{{\overline{\omega }}^2}{ϵ}+{\sigma }_{max}\left(Q\right)\right){\kappa }_{VAQ}+n^2\right).\end{array}$$

□

4. Application in Support Vector Machine Problems

In this section, we discuss how to use our IF-QIPM to solve SVM problems. We show that our algorithm can solve ${\ell }_1$-norm soft margin SVM problems faster than any existing classical or quantum algorithms with respect to dimension.

The ordinary SVM problem works on a linearly separable dataset, in which the data points have binary labels. The ordinary SVM aims to find a hyperplane correctly separating the data points with a maximum margin. However, in practice, the data points are not necessarily linearly separable. To allow for mislabelling, the concept of a soft margin SVM was introduced in [22]. Let $\{({\varphi }_i,{\zeta }_i)\in {\mathbb{R}}^m\times \{-1,+1\}|i=1,\dots ,n\}$ be the set of data points, $\mathrm{\Phi }$ be a matrix with the ith column being ${\varphi }_i$, and Z be a diagonal matrix with the ith diagonal element being ${\zeta }_i$. The SVM problem with an $l_1$-norm soft margin can be formulated as below.

$$\begin{array}{cc}\hfill \underset{(\xi ,w,t)\in {\mathbb{R}}^n\times {\mathbb{R}}^m\times \mathbb{R}}{min}& \frac{1}{2}{\parallel w\parallel }_2^2+C{\parallel \xi \parallel }_1\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\zeta }_i\left(〈w,{\varphi }_i〉+t\right)\ge 1-{\xi }_i,i=1,\dots ,n\hfill \\ \hfill & {\xi }_i\ge 0,i=1,\dots ,n.\hfill \end{array}$$

(15)

Here, $(w,t)$ determines a hyperplane and C is a penalty parameter. In [9], the authors rewrote the SVM problem as a second-order conic optimization (SOCO) problem and used the quantum algorithm that they proposed to solve the resulting SOCO problem. They claim the complexity of their algorithm has $\mathcal{O}\left(n^2\right)$ dependence on the dimension, which is better than any classical algorithm. However, the algorithm in [9] is invalid. Their algorithm is an inexact infeasible-QIPM (II-QIPM), while they used the IPM complexity for the feasible-QIPM, which ignores at least $\mathcal{O}\left(n^{1.5}\right)$ dependence on n. They also missed the symmetrization of the Newton step, which is necessary for SOCO problems and makes their Newton step invalid.

Aside from [9], some pure quantum algorithms for SVM problems are also proposed. In [23], the authors propose a pure quantum algorithm for SVM problems. They claim the complexity is $\mathcal{O}({\kappa }_{\mathrm{eff}}^3{ϵ}^{-3}log\left(mn\right))$, where ${\kappa }_{\mathrm{eff}}$ is the condition number of a matrix involving the kernel matrix and $ϵ$ is the accuracy. In the worst case, ${\kappa }_{\mathrm{eff}}=\mathcal{O}\left(m\right)$. Their complexity is worse than ours regarding the dependence of dimension and accuracy. In addition, their algorithm does not provide classical solutions. Namely, the solution is in the quantum machine and we cannot read or use it in a classical computer. However, our algorithm produces a classical solution.

To convert the problem into standard-form LCQO, we introduce $(w_{+},w_{-})\in {\mathbb{R}}_{+}^m\times {\mathbb{R}}_{+}^m$, $(t_{+},t_{-})\in {\mathbb{R}}_{+}\times {\mathbb{R}}_{+}$, and a slack variable $\rho \in {\mathbb{R}}_{+}^n$. Then, we can obtain the following formulation:

$$\begin{array}{cc}\hfill \underset{w_{+},w_{-},t_{+},t_{-},\xi ,\rho }{min}& \frac{1}{2}\parallel w_{+}-w_{-}{\parallel }_2^2+C{\parallel \xi \parallel }_1\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\zeta }_i\left(〈w_{+}-w_{-},{\varphi }_i〉+t_{+}-t_{-}\right)+{\xi }_i-{\rho }_i=1,i=1,\dots ,n\hfill \\ \hfill & (\xi ,w_{+},w_{-},t_{+},t_{-},\rho )\ge 0.\hfill \end{array}$$

This is a standard-form LCQO problem with non-negative variables $(w_{+},w_{-},t_{+},t_{-},\xi ,$$\rho )\in {\mathbb{R}}^m\times {\mathbb{R}}^m\times \mathbb{R}\times \mathbb{R}\times {\mathbb{R}}^n\times {\mathbb{R}}^n$ and parameters

$$\begin{array}{cc}\hfill c& =\left[\begin{array}{c}{0}_{2m+2}\\ C{e}_n\\ 0_n\end{array}\right]\hfill \\ \hfill Q& =\left[\begin{array}{ccc}{I}_{m\times m}& -I_{m\times m}& 0_{m\times (2+2n)}\\ -I_{m\times m}& I_{m\times m}& 0_{m\times (2+2n)}\\ 0_{(2+2n)\times m}& 0_{(2+2n)\times m}& 0_{(2+2n)\times (2+2n)}\end{array}\right]\hfill \\ \hfill A& =\left[\begin{array}{cccccc}Z{\Phi }^T& -Z{\Phi }^T& Z& -Z& I_{n\times n}& -I_{n\times n}\end{array}\right]\hfill \\ \hfill b& =e.\hfill \end{array}$$

Thus, we can use the proposed IF-QIPM for LCQO problems to solve the ${\ell }_1$-norm soft margin SVM problems and obtain an $ϵ$-approximate solution with complexity

$${\tilde{\mathcal{O}}}_{m,n,\overline{\omega },\frac{1}{ϵ}}\left({(m+n)}^{1.5}\left(\frac{{\overline{\omega }}^2}{ϵ}+{\sigma }_{max}\left(Q\right)\right){\kappa }_{VAQ}+{(m+n)}^{2.5}\right).$$

This dependence on dimension is better than any existing quantum or classical algorithm.

5. Discussion

In this work, we present an IF-QIPM for LCQO problems by combining the IF-IPM framework proposed in [5] and the OSS system introduced in [14]. Our algorithm has $n^{1.5}$ dependence on n, which is better than any existing algorithms for LCQO problems. The dependence on the accuracy is polynomial, which is worse than classic IPMs. Iterative refinement techniques might help to improve the dependence on the accuracy but they are beyond the discussion of this work.