On the Finite Complexity of Solutions in a Degenerate System of Quadratic Equations: Exact Formula

Abstract

The paper describes an application of the p-regularity theory to Quadratic Programming (QP) and nonlinear equations with quadratic mappings. In the first part of the paper, a special structure of the nonlinear equation and a construction of the 2-factor operator are used to obtain an exact formula for a solution to the nonlinear equation. In the second part of the paper, the QP problem is reduced to a system of linear equations using the 2-factor operator. The solution to this system represents a local minimizer of the QP problem along with its corresponding Lagrange multiplier. An explicit formula for the solution of the linear system is provided. Additionally, the paper outlines a procedure for identifying active constraints, which plays a crucial role in constructing the linear system.

1. Introduction

Consider the nonlinear equation $F\left(x\right)=0$ with the mapping F defined by:

$$F\left(x\right)=B{\left[x\right]}^2+Mx+N,$$

(1)

where $M\in {\mathbb{R}}^{n\times n}$ is a matrix, $N\in {\mathbb{R}}^n$ is a vector, and $B:{\mathbb{R}}^n\to {\mathbb{R}}^n$ is the map defined for $x\in {\mathbb{R}}^n$ by:

$$B{\left[x\right]}^2=B(x,x)={\left(x^T{B}_{1}x,\dots ,x^T{B}_{n}x\right)}^T,$$

(2)

where $B_i$ is an $n\times n$ symmetric matrix for $i=1,\dots ,n$.

We also consider the quadratic programming (QP) problem with inequality constraints:

$$\begin{array}{cc}\underset{x}{\mathrm{minimize}}\hfill & f\left(x\right)=\frac{1}{2}{x}^{T}Qx+c^{T}x\hfill \\ \mathrm{subject}\mathrm{to}\hfill & Ax\le b\hfill \end{array}\left(\mathrm{QP}\right),$$

(3)

where Q is an $n\times n$ symmetric matrix, A is an $m\times n$ matrix, $c,x\in {\mathbb{R}}^n$, and $b\in {\mathbb{R}}^m$.

The paper describes an application of the p-regularity theory to nonlinear equations with the mapping F introduced in (1) and to the quadratic programming problem (3).

In recent years, there has been growing interest in nonlinear problems, including quadratic and polynomial equations, as well as nonlinear optimization problems, attracting specialists from various disciplines (see, for example, refs. [1,2,3,4] and references therein). Furthermore, it was observed that nonlinear problems are closely related to singular problems, as demonstrated in [5]. In fact, it has been discovered that essentially nonlinear problems and singular problems are locally equivalent. In this work, we aim to provide a theoretical foundation for this claim by introducing several auxiliary concepts as proposed in [5].

Definition 1. 

Let V be a neighborhood of $x^{*}$ in ${\mathbb{R}}^n$, and let $U\subset {\mathbb{R}}^n$ be a neighborhood of 0. A mapping $F:V\to {\mathbb{R}}^n$, where $F\in {\mathcal{C}}^2\left(V\right)$, is considered essentially nonlinear at $x^{*}$ if there exists a perturbation of the form:

$$\tilde{F}(x^{*}+x)=F(x^{*}+x)+\omega \left(x\right),\mathit{where}\parallel \omega \left(x\right)\parallel =o(\parallel x\parallel ),$$

such that no nondegenerate transformation of coordinates $\phi \left(x\right):U\to V$, where $\phi \in {\mathcal{C}}^1\left(U\right)$, satisfies $\phi \left(0\right)=x^{*}$, ${\phi }^{\prime }\left(0\right)=I_n$, where $I_n$ is the identity map in ${\mathbb{R}}^n$, and:

$$\tilde{F}\left(\phi \left(x\right)\right)=\tilde{F}\left(x^{*}\right)+{\tilde{F}}^{\prime }\left(x^{*}\right)x\mathit{for}\mathit{all}x\in U.$$

Definition 2. 

We say that the mapping F is singular (or degenerate) at $x^{*}$ if it fails to be regular, meaning its derivative is not onto:

$$Im{F}^{\prime }\left(x^{*}\right)\ne {\mathbb{R}}^n.$$

The relationship between the notions of essential nonlinearity and singularity is established in Theorem 1, which was derived in [5].

Theorem 1. 

Let V be a neighborhood of $x^{*}$ in ${\mathbb{R}}^n$. Suppose $F:V\to {\mathbb{R}}^n$ is $C^2$ and that $x^{*}$ is a solution of $F\left(x\right)=0$. Then F is essentially nonlinear at the point $x^{*}$ if and only if F is singular at the point $x^{*}$.

The work presented in [5] primarily focuses on the construction of p-regularity and its applications in various areas of mathematics. However, it does not specifically cover quadratic nonlinear equations and quadratic programming problems. The current paper builds upon the foundation of the p-regularity theory established in [5] but introduces novel results. The main objective of this paper is to explore the key aspects of nonlinear problems, with a particular emphasis on systems of quadratic equations and quadratic programming problems that may involve singular solutions.

Specifically, we begin by considering the nonlinear equation $F\left(x\right)=0$. One of the main goals of the paper is to derive the exact formula for a solution $x^{*}$ of the nonlinear equation $F\left(x\right)=0$ using the special form of the quadratic mapping F defined in (1). We demonstrate how to use a construction of a special 2-factor-operator to transform the original problem into a system of linear equations. The construction of the 2-factor-operator combines the mapping F with its first derivative $F^{\prime }\left(x\right)$.

In the second part of the paper, we apply a similar approach to the quadratic programming problem (3) in order to derive explicit formulas for the solution $(x^{*},{\lambda }^{*})$, where $x^{*}$ represents a local minimizer of the QP problem and ${\lambda }^{*}$ is the corresponding Lagrange multiplier. Namely, using the special form of the QP problem and the 2-factor-operator, we reduce the system of optimality conditions for the QP problem to a system of linear equations, with the point $(x^{*},{\lambda }^{*})$ as its solution. The paper also describes a procedure for identifying the active constraints, which plays a vital role in constructing the linear system.

Although there is literature on solutions of degenerate systems of quadratic equations, the approach presented in this paper is novel and distinct from the methods proposed by other authors. This approach can be applied to various problems and areas of mathematics where the problem involves solving a degenerate equation $F\left(x\right)=0$ with a quadratic mapping F. Such nonlinear problems can arise in the numerical solutions and analysis of ordinary differential equations, partial differential equations, optimal control problems, algebraic geometry, and other fields. In the second part of the paper, we specifically focus on using the methods developed in the first sections to solve the QP problem (3). The quadratic programming problems have attracted the attention of many researchers and scientists, so there is an extensive body of literature on the topic. Some publications in this area include [6,7,8,9,10,11,12].

The outline of the paper. The main contribution and novelty of the paper are in the exact formulas for a solution of a nonlinear equation and of the quadratic programming problems, presented in Section 3 and Section 5, respectively.

In Section 2, we recall the main definitions of the p-regularity theory, as presented in [5], including the special case of $p=2$. Additionally, we introduce the p-factor method for solving singular nonlinear equations of the form $F\left(x\right)=0$ and describe various versions of the 2-factor method.

Section 3 presents some of the key results of the paper, focusing on the application of a modified 2-factor method to solve the nonlinear equation $F\left(x\right)=0$ with the mapping F defined as $F\left(x\right)=B{\left[x\right]}^2+Mx+N,$ where $M\in {\mathbb{R}}^{n\times n}$ is a matrix, $N\in {\mathbb{R}}^n$ is a vector, and $B:{\mathbb{R}}^n\to {\mathbb{R}}^n$ is defined by (2). In this section, we introduce multiple approaches to obtain exact formulas for a solution to the nonlinear equation $F\left(x\right)=0$, demonstrating that the proposed methods converge to a solution $x^{*}$ of the nonlinear equation in just one iteration.

Section 4 focuses on an auxiliary result used in other parts of the paper. We present a theorem that describes the properties of a special mapping $\mu \left(x\right)$, which enables us to propose a procedure for determining r linearly independent vectors $f_{i_k}^{\prime }\left(x^{*}\right)$, $k=1,\dots ,r$, at the solution $x^{*}$ of $F\left(x\right)=0$, without needing to know the exact value of $x^{*}$. This procedure relies on information about the system of vectors $\{f_1^{\prime }\left(x\right),\dots ,f_m^{\prime }\left(x\right)\}$ at some point x within a small neighborhood of $x^{*}$.

Section 5 presents other novel results, focusing on deriving exact formulas for a solution of quadratic programming problems. The section is divided into three parts. In Section 5.1, we consider regular quadratic programming problems and propose three approaches to solving the QP problem and obtaining a formula for its solution. These approaches are based on the construction of the 2-factor-operator. Section 5.2 addresses the issue of identifying the active constraints and proposes strategies for numerically determining the set of active constraints $I\left(x^{*}\right)$. These techniques are then applied in the final part, Section 5.3, to address degenerate QPs. The paper concludes with some closing remarks in Section 6.

Notation. Let $a_i$ denote the rows of the $m\times n$ matrix A in problem (3), and let $b={(b_1,\dots ,b_m)}^T$, so that $a_i^T\in {\mathbb{R}}^n$ and $b_i\in \mathbb{R}$ for $i=1,\dots ,m$.

The active set $I\left(x^{*}\right)$ at any feasible point $x^{*}$ of problem (3) is the set of indices of the active constraints at $x^{*}$, i.e., $I\left(x^{*}\right)=\{i=1,\dots ,m\mid a_i{x}^{*}-b_i=0\}$.

Furthermore, $KerS=\{x\in {\mathbb{R}}^n\mid Sx=0\}$ denotes the null-space (kernel) of a given linear operator $S:{\mathbb{R}}^n\to {\mathbb{R}}^m$, and $ImS=\{y\in {\mathbb{R}}^m\mid y=Sx \mathrm{for} \mathrm{some} x\in {\mathbb{R}}^n\}$ is its image space.

Let $B:{\mathbb{R}}^n\times {\mathbb{R}}^n\to {\mathbb{R}}^n$ be a bilinear symmetric mapping. The 2-form associated with B is the map $B{[·]}^2:{\mathbb{R}}^n\to {\mathbb{R}}^n$ defined by $B{\left[x\right]}^2=B(x,x)$ for $x\in {\mathbb{R}}^n$. We also use the following notation: ${Im}^{2}B=\{y\in {\mathbb{R}}^n\mid B{\left[x\right]}^2=y \mathrm{for} \mathrm{some} x\in {\mathbb{R}}^n\}$ and ${Ker}^{2}B=\{x\in {\mathbb{R}}^n\mid B{\left[x\right]}^2=0\}$. We denote by $N\left(x^{*}\right)$ and $N_{\epsilon }\left(x^{*}\right)$neighborhoods of a point $x^{*}$, where $N_{\epsilon }\left(x^{*}\right)$ is an $\epsilon$-neighborhood of $x^{*}$, i.e., an open ball of radius $\epsilon$ centered at $x^{*}$.

The notation for the scalar (dot) product of vectors x and y in ${\mathbb{R}}^n$, used in the paper, is $x·y=x^{T}y$.

We denote by $span(a^1,\dots ,a^m)$ the linear span of the given vectors $a^1,\dots ,a^m$. We also denote by $d(x,S)$ the distance between a point x and a set S.

2. Elements of the $\mathit{p}$-Regularity Theory

We begin this section with the main definitions of the p-regularity theory, which are given in [5]. The primary focus is on the sufficiently smooth mapping F from ${\mathbb{R}}^n$ to ${\mathbb{R}}^n$, defined as:

$$F\left(x\right)={(f_1\left(x\right),\dots ,f_n\left(x\right))}^T,$$

(4)

where $f_i\left(x\right):{\mathbb{R}}^n\to \mathbb{R}$ for $i=1,\dots ,n$. After presenting the general case, we focus on the specific case of $p=2$. We introduce the definitions of the 2-regular mapping and the 2-factor-operator, which play a central role in the subsequent sections.

2.1. The Main Definitions and Constructions of the p-Regularity Theory

Throughout this section, we consider the nonlinear equation:

$$F\left(x\right)=0,$$

(5)

where F is defined in Equation (4). Let $x^{*}\in {\mathbb{R}}^n$ represent a solution to the nonlinear Equation (5).

The mapping F is called regular at $x^{*}$ if:

$$\mathrm{Im}F{}^{\prime }\left(x^{*}\right)={\mathbb{R}}^n,$$

(6)

or in other words, if:

$$\mathrm{rank}F{}^{\prime }\left(x^{*}\right)=n,$$

where $F^{\prime }\left(x^{*}\right)$ is the Jacobian matrix of the mapping F at $x^{*}$. Conversely, the mapping F is called nonregular (irregular, degenerate) if the regularity condition (6) is not satisfied.

Let the space ${\mathbb{R}}^n$ be decomposed into the direct sum:

$${\mathbb{R}}^n=Y_1\oplus \dots \oplus Y_p,$$

(7)

where $Y_1=Im{F}^{\prime }\left(x^{*}\right),$ is defined as the closure of the image of the first derivative of F evaluated at $x^{*}$, and p is chosen as the minimum number for which Equation (7) holds.

The remaining spaces are defined as follows. Let $Z_1={\mathbb{R}}^n$, and let $Z_2$ be a closed complementary subspace to $Y_1$. Let $P_{Z_2}:{\mathbb{R}}^n\to Z_2$ be the projection operator onto $Z_2$ along $Y_1$. Define $Y_2$ as the closed linear span of the image of the quadratic map $P_{Z_2}{F}^{\left(2\right)}\left(x^{*}\right){[·]}^2$. More generally, we define $Y_i$ inductively for $i=2,\cdots ,p-1$ as:

$$Y_i=\mathrm{span}Im{P}_{Z_i}{F}^{\left(i\right)}\left(x^{*}\right){[·]}^i\subseteq Z_i,$$

where $Z_i$ is a choice of a complementary subspace for $(Y_1\oplus \dots \oplus Y_{i-1})$ with respect to Y, $i=2,\cdots ,p$, and $P_{Z_i}:Y\to Z_i$ is the projection operator onto $Z_i$ along $(Y_1\oplus \dots \oplus Y_{i-1})$ with respect to Y, $i=2,\cdots ,p$. Finally, we let $Y_p=Z_p.$

Define the following mappings:

$$F_i\left(x\right):{\mathbb{R}}^n\to Y_i,F_i\left(x\right)=P_{Y_i}F\left(x\right),i=1,\cdots ,p,$$

(8)

where $P_{Y_i}:{\mathbb{R}}^n\to Y_i$ is the projection operator onto $Y_i$ along $(Y_1\oplus \dots \oplus Y_{i-1}\oplus Y_{i+1}\oplus \dots \oplus Y_p)$ with respect to ${\mathbb{R}}^n$ for $i=1,\cdots ,p$. Then F can be represented as $F\left(x\right)=F_1\left(x\right)+\dots +F_p\left(x\right)$ or equivalently as $F\left(x\right)=(F_1\left(x\right),\dots ,F_p\left(x\right)).$

Definition 3. 

The linear operator ${\mathsf{\Psi }}_p\left(h\right)\in \mathcal{L}({\mathbb{R}}^n,Y_1\oplus \dots \oplus Y_p)$, where $h\in {\mathbb{R}}^n$, $h\ne 0$, is defined by:

$${\mathsf{\Psi }}_p\left(h\right)=F_1^{\prime }\left(x^{*}\right)+F_2^{″}\left(x^{*}\right)\left[h\right]+\dots +F_p^{\left(p\right)}\left(x^{*}\right){\left[h\right]}^{p-1},$$

and is called the p-factor operator.

Consider the nonlinear operator ${\mathsf{\Psi }}_p{[·]}^p$ defined by:

$${\mathsf{\Psi }}_p{\left[x\right]}^p=F_1^{\prime }\left(x^{*}\right)\left[x\right]+F_2^{″}\left(x^{*}\right){\left[x\right]}^2+\dots +F_p^{\left(p\right)}\left(x^{*}\right){\left[x\right]}^p.$$

Notice that ${\mathsf{\Psi }}_p{\left[x\right]}^p={\mathsf{\Psi }}_p\left(x\right)\left[x\right]$.

Definition 4. 

The p-kernel of the operator ${\mathsf{\Psi }}_p$ at the point $x^{*}$ is the set $H_p\left(x^{*}\right)$ defined by:

$$H_p\left(x^{*}\right)={Ker}^p{\mathsf{\Psi }}_p,$$

where:

$${Ker}^p{\mathsf{\Psi }}_p=\{h\in {\mathbb{R}}^n\mid F_1^{\prime }\left(x^{*}\right)\left[h\right]+F_2^{″}\left(x^{*}\right){\left[h\right]}^2+\dots +F_p^{\left(p\right)}\left(x^{*}\right){\left[h\right]}^p=0\}.$$

Please note that ${Ker}^p{\mathsf{\Psi }}_p={\displaystyle \bigcap _{k=1}^p}{Ker}^k{F}_k^{\left(k\right)}\left(x^{*}\right)$, where:

$${Ker}^k{F}_k^{\left(k\right)}\left(x^{*}\right)=\{\xi \in {\mathbb{R}}^n\mid F_k^{\left(k\right)}\left(x^{*}\right){\left[\xi \right]}^k=0\}$$

is the k-kernel of $F^{\left(k\right)}(·){[·]}^k$.

Definition 5. 

A mapping F is called p-regular at $x^{*}$ along h if $Im{\mathsf{\Psi }}_p\left(h\right)={\mathbb{R}}^n$.

Definition 6. 

A mapping F is called p-regular at $x^{*}$ if it is p-regular along all $h\in H_p\left(x^{*}\right)\setminus \left\{0\right\}$ or $H_p\left(x^{*}\right)=\left\{0\right\}$.

Now, we will focus on the special case of $p=2$, which we are using in the paper. We denote the image of the Jacobian matrix $F{}^{\prime }\left(x^{*}\right)$ by $R_1$: $R_1=\mathrm{Im}F{}^{\prime }\left(x^{*}\right),$ and the orthogonal complementary subspace of $R_1$ in ${\mathbb{R}}^n$ by $R_2$. Then:

$${\mathbb{R}}^n=R_1\oplus R_2.$$

We also denote an $n\times n$ matrix of the orthogonal projection onto $R_i$ in ${\mathbb{R}}^n$ by $P_{R_i}$, $i=1,2.$

Similarly to Equation (8), we introduce the mappings:

$$F_i\left(x\right)=P_{R_i}F\left(x\right),i=1,2.$$

The p-factor operator plays the central role in the p-regularity theory. We give the following definition of the p-factor-operator for $p=2$.

Definition 7. 

We define a 2–factor-operator of the mapping F at $x^{*}$ with respect to some vector $h\in {\mathbb{R}}^n$, $h\ne 0$, as a linear operator from ${\mathbb{R}}^n$ to ${\mathbb{R}}^n$, defined by one of the following equations:

$${\mathsf{\Psi }}_2\left(h\right)=F_1{}^{\prime }\left(x^{*}\right)+F_2{}^{″}\left(x^{*}\right)\left[h\right],$$

(9)

$${\overline{\mathsf{\Psi }}}_2\left(h\right)=F{}^{\prime }\left(x^{*}\right)+P_{R_2}F{}^{″}\left(x^{*}\right)\left[h\right],$$

(10)

$${\tilde{\mathsf{\Psi }}}_2\left(h\right)=F{}^{\prime }\left(x^{*}\right)+F{}^{″}\left(x^{*}\right)\left[h\right].$$

(11)

Now we are ready to introduce another very important definition of the 2-regularity theory.

Definition 8. 

The mapping F is called 2-regular at the point $x^{*}$ with respect to the element h if the image of a 2–factor-operator, defined by one of the Equations (9)–(11) is equal to ${\mathbb{R}}^n$.

Definition 9. 

The mapping F is called 2-regular at $x^{*}$ if it is 2-regular at $x^{*}$ with respect to all the elements h from a set that is defined as:

1. 

$H_2\left(x^{*}\right)=\mathrm{Ker}{F}_1^{\prime }\left(x^{*}\right)\bigcap {\mathrm{Ker}}^2{F}_2^{″}\left(x^{*}\right)$ for ${\mathsf{\Psi }}_2\left(h\right)$ defined by (9);

2. 

${\overline{H}}_2\left(x^{*}\right)=\mathrm{Ker}F{}^{\prime }\left(x^{*}\right)\bigcap {\mathrm{Ker}}^2{P}_{R_2}F{}^{″}\left(x^{*}\right)$ for ${\overline{\mathsf{\Psi }}}_2\left(h\right)$ defined by (10);

3. 

${\tilde{H}}_2\left(x^{*}\right)=\mathrm{Ker}F{}^{\prime }\left(x^{*}\right)\bigcap {\mathrm{Ker}}^{2}F{}^{″}\left(x^{*}\right)$ for ${\tilde{\mathsf{\Psi }}}_2\left(h\right)$ defined by (11).

2.2. The p-Factor-Method for Solving Singular Nonlinear Equations

In this section, we introduce the p-factor-method for solving the singular nonlinear equation $F\left(x\right)=0$. Then we consider the special case of $p=2$ and describe several versions of the 2-factor-method.

Consider Equation (5) in the case when mapping $F^{\prime }\left(x\right)$ is singular at $x^{*}$. In this case, the p-factor method is an iterative procedure defined by:

$$\begin{array}{ccc}\hfill x^{k+1}& =& x^k-{\left(F^{\prime }\left(x^k\right)+P_2{F}^{\prime \prime }\left(x^k\right)\left[h\right]+\dots +P_p{F}^{\left(p\right)}\left(x^k\right){\left[h\right]}^{p-1}\right)}^{-1}\hfill \\ & & ·\left(F\left(x^k\right)+P_2{F}^{\prime }\left(x^k\right)\left[h\right]+\dots +P_p{F}^{(p-1)}\left(x^k\right){\left[h\right]}^{p-1}\right),\hfill \end{array}$$

(12)

where $k=0,1,\dots$, $P_i=P_{Y_i}$ for $i=1,2,\dots ,p$, and vector h, $\parallel h\parallel =1$, is chosen in such a way that the p-factor operator ${\mathsf{\Psi }}_p$ is nonsingular, which implies that the mapping F is p-regular at $x^{*}$ along h. The following theorem is valid for the p-factor-method (12).

Theorem 2. 

Assume that mapping $F\in {\mathcal{C}}^{p+1}\left({\mathbb{R}}^n\right)$ and there exists vector h, $\parallel h\parallel =1$, such that the p-factor operator ${\mathsf{\Psi }}_p$ is nonsingular. Given a point $x^0\in N_{\epsilon }\left(x^{*}\right)$, where $\epsilon >0$ is sufficiently small and $N_{\epsilon }\left(x^{*}\right)$ is a neighborhood of $x^{*}$, the sequence $\left\{x^k\right\}$ defined by Equation (12) converges quadratically to the solution $x^{*}$ of (5):

$$\parallel x^{k+1}-x^{*}\parallel \le C\parallel x^k-x^{*}{\parallel }^2,k=0,1,\dots ,$$

(13)

where $C>0$ is an independent constant.

Now, we are ready to describe several versions of the 2-factor-method.

For solving singular nonlinear Equation (5), the following iterative method, called the 2-factor-method, was proposed in [13]:

$$x^{k+1}=x^k-{\left(F{}^{\prime }\left(x^k\right)+P_{R_2}F{}^{″}\left(x^k\right)\left[h\right]\right)}^{-1}\left(F\left(x^k\right)+P_{R_2}F{}^{\prime }\left(x^k\right)\left[h\right]\right),\hspace{1em}k=0,1,2,\dots ,$$

(14)

where the vector h, $\parallel h\parallel =1$, is chosen in such a way that matrix $\left(F{}^{\prime }\left(x^{*}\right)+P_{R_2}F{}^{″}\left(x^{*}\right)\left[h\right]\right)$ is invertible.

The following theorem states the convergence properties of the 2-factor-method (14).

Theorem 3. 

Given a mapping $F\in C^3\left({\mathbb{R}}^n\right)$, let $x^{*}$ be a solution of Equation (5). Assume that there exists a vector $h\in {\mathbb{R}}^n$ such that $\parallel h\parallel =1$ and F is 2-regular at the point $x^{*}$ with respect to the vector h with the 2-factor-operator ${\overline{\mathsf{\Psi }}}_2\left(h\right)$ defined by (10).

Then there is a neighborhood $N\left(x^{*}\right)$ of $x^{*}$ in ${\mathbb{R}}^n$ such that for any $x^0\in N\left(x^{*}\right)$, the sequence $\left\{x^k\right\}$ generated by the 2-factor-method (14) converges to $x^{*}$ and:

$$\parallel x^{k+1}-x^{*}\parallel \le C\parallel x^k-x^{*}{\parallel }^2,$$

(15)

where $C>0$ is some constant.

Proof. 

Since $P_{R_2}$ is the orthoprojector onto subspace $R_2=R_1^{\perp }$, then for the mapping:

$$\mathsf{\Phi }\left(x\right)=F\left(x\right)+P_{R_2}F{}^{\prime }\left(x\right)\left[h\right],$$

we have $\mathsf{\Phi }\left(x^{*}\right)=0$. Moreover, because ${\mathsf{\Phi }}^{\prime }\left(x^{*}\right)={\overline{\mathsf{\Psi }}}_2\left(h\right)$ and the mapping F is 2-regular with respect to the vector h, by Definition 8 with ${\overline{\mathsf{\Psi }}}_2\left(h\right)$ defined by (10), we obtain that $\mathrm{Im}{\overline{\mathsf{\Psi }}}_2\left(h\right)={\mathbb{R}}^n$. Hence, the matrix ${\mathsf{\Phi }}^{\prime }\left(x^{*}\right)$ is invertible.

Therefore, the 2-factor-method given in (14) is an application of Newton’s method to system $\mathsf{\Phi }\left(x\right)=0$ in a sufficiently small neighborhood of $x^{*}$. Then the statement of the theorem follows from the properties of Newton’s method [14] (Proposition 1.4.1). □

Now, we will introduce a modified version of the 2-factor-method (14). Assume that there exists a vector $h\ne 0$ such that $F{}^{\prime }\left(x^{*}\right)h=0$ and the matrix $(F{}^{\prime }\left(x^{*}\right)$$+F{}^{″}\left(x^{*}\right)\left[h\right])$ is invertible. Then for solving Equation (5), we can use the following modified 2-factor-method:

$$x^{k+1}=x^k-{\left(F{}^{\prime }\left(x^k\right)+F{}^{″}\left(x^k\right)\left[h\right]\right)}^{-1}\left(F\left(x^k\right)+F{}^{\prime }\left(x^k\right)\left[h\right]\right),k=0,1,2,\dots .$$

(16)

The following theorem states the convergence properties of method (16).

Theorem 4. 

Given a mapping $F\in C^3\left({\mathbb{R}}^n\right)$, let $x^{*}$ be a solution of Equation (5). Assume that there exists a vector $h\in {\mathbb{R}}^n$ such that $\parallel h\parallel =1$, $F{}^{\prime }\left(x^{*}\right)h=0$, and F is 2-regular at the point $x^{*}$ with respect to the vector h, where the 2-factor-operator ${\tilde{\mathsf{\Psi }}}_2\left(h\right)$ is defined by (11).

Then there is a neighborhood $N\left(x^{*}\right)$ of $x^{*}$ in ${\mathbb{R}}^n$ such that for any $x^0\in N\left(x^{*}\right)$, the sequence $\left\{x^k\right\}$ generated by the 2-factor-method (16) converges to $x^{*}$, and relation (15) holds with some $C>0$.

The proof is similar to one of Theorem 3.

Now we introduce another version of the 2-factor-method.

Assume that the following conditions hold:

$$\begin{array}{c}1. \mathrm{There} \mathrm{exists} a \mathrm{vector} h\ne 0 \mathrm{such} \mathrm{that} F{}^{\prime }\left(x^{*}\right)h=0;\hfill \\ 2. \mathrm{Matrix} \left(F{}^{″}\left(x^{*}\right)\left[h\right]\right) \mathrm{is} \mathrm{invertible}.\hfill \end{array}$$

(17)

Note that if conditions (17) are satisfied, then to solve Equation (5), we can use the following modified 2-factor-method:

$$x^{k+1}=x^k-{\left(F{}^{″}\left(x^k\right)\left[h\right]\right)}^{-1}\left(F{}^{\prime }\left(x^k\right)\left[h\right]\right),k=0,1,2,\dots .$$

(18)

For numerical realization of the 2-factor-method in the form (18), we only have to construct vector h satisfying conditions (17). Specifics of some problems allow us to choose vector h without any knowledge of the solution $x^{*}$. We discuss the choice of the vector h in the following sections of the paper.

The following theorem states the convergence properties of method (18).

Theorem 5. 

Given a mapping $F\in C^3\left({\mathbb{R}}^n\right)$, let $x^{*}$ be a solution of Equation (5). Assume that there exists a vector $h\in {\mathbb{R}}^n$ such that $\parallel h\parallel =1$ and conditions (17) are satisfied.

Then, there is a neighborhood $N\left(x^{*}\right)$ of $x^{*}$ in ${\mathbb{R}}^n$ such that for any $x^0\in N\left(x^{*}\right)$, the sequence $\left\{x^k\right\}$ generated by the 2-factor-method (18) converges to $x^{*}$ and relation (15) holds with some $C>0$.

The proof is similar to one of Theorem 3.

3. Nonlinear Equations with Quadratic Mappings: the Exact Solution Formula

In this section, we consider the mapping F defined by Equation (1) as follows:

$$F\left(x\right)=B{\left[x\right]}^2+Mx+N,$$

where $M\in {\mathbb{R}}^{n\times n}$ is a matrix, $N\in {\mathbb{R}}^n$ is a vector, and $B:{\mathbb{R}}^n\to {\mathbb{R}}^n$ is the map defined by (2). The mapping B is twice continuously differentiable [15], and its derivatives are given by ${\left(B{\left[x\right]}^2\right)}^{\prime }=2B\left[x\right]$ and ${\left(B{\left[x\right]}^2\right)}^{\prime \prime }=2B$ for some arbitrary $x\in {\mathbb{R}}^n$. Let $x^{*}$ denote a solution of the equation $F\left(x\right)=0$.

We will now illustrate the application of the 2-factor method (18) for solving the nonlinear equation $F\left(x\right)=0$ with the mapping F defined by (1). We will present multiple approaches to obtain an exact formula for $x^{*}$, with the first approach being a specific case of the second approach. Additionally, we will show that for the mapping F, the method (18) converges to $x^{*}$ in just one iteration.

• First approach to obtain an exact formula for the solution $x^{*}$.

For the mapping F defined by (1), the assumptions (17) of Theorem 5 can be simplified to the existence of a vector h that satisfies the following conditions:

$$\begin{array}{c}\left(1\right)(2B\left[x^{*}\right]+M)h=0;\hfill \\ \left(2\right)\mathrm{matrix} \left(2B\right[h\left]\right)\mathrm{is} \mathrm{invertible}.\hfill \end{array}$$

(19)

Under these assumptions (19), for the mapping F defined by (1) and a given point $x^0$, the first iteration of the 2-factor-method (18) can be written as:

$$x^1=x^0-{\left(2B\left[h\right]\right)}^{-1}\left((2B\left[x^0\right]+M)\left[h\right]\right),$$

which is equivalent to:

$$\left(2B\left[h\right]\right)(x^1-x^0)=-\left((2B\left[x^0\right]+M)\left[h\right]\right).$$

Using the property $2B(h,x^0)=2B(x^0,h)$, the last equation implies a one-step method for calculating $x^1$ and, consequently, finding the solution $x^{*}$:

$$x^{*}=-\frac{1}{2}{\left(B\left[h\right]\right)}^{-1}\left(Mh\right),$$

(20)

where the vector h satisfies conditions (19).

The numerical determination of the vector h depends on the specific characteristics of the problem. Alternatively, it can be obtained using the same method as described in the third approach below, which involves transforming the initial system into a system that is completely degenerate at the point $x^{*}$.

• Second approach to obtain an exact formula for the solution $x^{*}$.

Now we present an alternative approach for obtaining a formula for the solution $x^{*}$ of the equation $F\left(x\right)=0$ using the same mapping F defined by (1). This second approach is applicable to a broader variety of problems compared to the first approach.

Let $P_1$ denote the projector onto $Y_1=\mathrm{span}\left({\mathrm{Im}}^{2}B\right)$, and let $P_2$ denote the projector onto $Y_1^{\perp }$, which is the orthogonal complementary subspace of $Y_1$ in ${\mathbb{R}}^n$. We note that $P_2\left(B{\left[x^{*}\right]}^2\right)=0$ and:

$$4P_{2}B(x^{*},x)=P_2(B{[x+x^{*}]}^2-B{[x-x^{*}]}^2)=0.$$

Then for the mapping F defined in (1),

$$P_{1}F{}^{\prime }\left(x^{*}\right)=P_1(2B\left[x^{*}\right]+M) \mathrm{and} P_{2}F{}^{\prime }\left(x^{*}\right)=P_{2}M.$$

Assume that there exists a vector $h\in {\mathbb{R}}^n$ satisfying the conditions:

$$\begin{array}{c}\left(1\right)P_1(2B\left[x^{*}\right]+M)h=0;\hfill \\ \left(2\right)\mathrm{matrix} (P_{2}M+2P_{1}B\left[h\right]) \mathrm{is} \mathrm{invertible}.\hfill \end{array}$$

(21)

Given the definition of $P_2$, it follows that $P_{2}B{\left[x^{*}\right]}^2=0$. Substituting this into (1), we obtain $P_2(M{x}^{*}+N)=0$. Hence, the point $x^{*}$ satisfies the following identities:

$$\begin{array}{c}{P}_2(M{x}^{*}+N)=0,\hfill \\ P_1(2B\left[x^{*}\right]+M)h=0.\hfill \end{array}$$

By adding these equations and assuming (21), we obtain the exact formula for the solution $x^{*}$:

$$x^{*}=-{(P_{2}M+2P_{1}B\left[h\right])}^{-1}(P_{2}N+P_{1}Mh).$$

(22)

Remark 1. 

In the case when $P_1=I_n$ and, hence, $P_2=O_{n\times n}$, assumptions (21) become (19), and Equation (22) reduces to (20).

Example 1. 

Consider mapping $F:{\mathbb{R}}^2\to {\mathbb{R}}^2$ given by:

$$F\left(x\right)=\left(\begin{array}{c}{x}_1^2-x_2^2-2x_1+1\\ x_1{x}_2-x_2\end{array}\right).$$

(23)

We can represent the mapping F in the form (1) with:

$$B=\left(\begin{array}{c}\left(\begin{array}{cc}1& 0\\ 0& -1\end{array}\right)\\ \left(\begin{array}{cc}0& 1/2\\ 1/2& 0\end{array}\right)\end{array}\right),M=\left(\begin{array}{cc}-2& 0\\ 0& -1\end{array}\right),andN=\left(\begin{array}{c}1\\ 0\end{array}\right).$$

The equation $F\left(x\right)=0$ has a locally unique solution $x^{*}={(1,0)}^T$. In this example, $P_1=I_2$ and $P_2=O_{2\times 2}$. Hence, by Remark 1, we apply Equation (20) with $h={(1,0)}^T$ to obtain:

$$x^{*}=-\frac{1}{2}{\left(Bh\right)}^{-1}\left(Mh\right)=-\frac{1}{2}\left(\begin{array}{cc}1& 0\\ 0& 2\end{array}\right)\left(\begin{array}{c}-2\\ 0\end{array}\right)=\left(\begin{array}{c}1\\ 0\end{array}\right),$$

as claimed.

In a numerical implementation, an additional procedure is required to construct the vector h. Since the exact point $x^{*}$ is not known in advance, we only assume that a sufficiently small neighborhood of $x^{*}$ is provided to apply the procedure.

• Third approach to obtain an exact formula for the solution $x^{*}$.

While the first two approaches rely on knowledge of the element h, which is determined by $x^{*}$, the third approach does not require such knowledge. Instead, all we need is for the starting point to belong to a sufficiently small neighborhood $N_{\epsilon }\left(x^{*}\right)$ of $x^{*}$. Specifically, we have $x^0\in N_{\epsilon }\left(x^{*}\right)$, where $\epsilon >0$ is sufficiently small.

Suppose that at the point $x^{*}$, the first r vectors $\{f_1^{\prime }\left(x^{*}\right),\dots ,f_r^{\prime }\left(x^{*}\right)\}$ are linearly independent, where $f_i$ is defined in (4) for $i=1,\dots ,r$. Assume also that the other vectors $\{f_{r+1}^{\prime }\left(x^{*}\right),\dots ,f_n^{\prime }\left(x^{*}\right)\}$ are linear combinations of the first r vectors. Therefore, there exist coefficients ${\alpha }_i^j$ such that:

$$f_j^{\prime }\left(x^{*}\right)=\sum _{i=1}^r{\alpha }_i^j{f}_i^{\prime }\left(x^{*}\right),j=r+1,\dots ,n.$$

Let us introduce the subspace $L\left(x\right)$ defined by:

$$L\left(x\right)=\mathrm{span}(f_1^{\prime }\left(x\right),\dots ,f_r^{\prime }\left(x\right)).$$

We denote the orthogonal projection on the subspace $L\left(x\right)$ as $P_{L\left(x\right)}$. Then, there exist coefficients ${\alpha }_i^j\left(x\right)$ such that:

$$P_{L\left(x\right)}{f}_j^{\prime }\left(x\right)=\sum _{i=1}^r{\alpha }_i^j\left(x\right)f_i^{\prime }\left(x\right),j=r+1,\dots ,n.$$

In addition, introduce the notation:

$${\widehat{f}}_j^{\prime }\left(x\right)=f_j^{\prime }\left(x\right)-P_{L\left(x\right)}{f}_j^{\prime }\left(x\right),j=r+1,\dots ,n.$$

Then:

$${\widehat{f}}_j\left(x\right)=f_j\left(x\right)-\sum _{i=1}^r{\alpha }_i^j\left(x\right)f_i\left(x\right),j=r+1,\dots ,n.$$

(24)

Notice that $x^{*}$ is also a solution of the equation $\widehat{F}\left(x\right)=0$, where $\widehat{F}\left(x\right)$ is defined as:

$$\widehat{F}\left(x\right)={(f_1\left(x\right),\dots ,f_r\left(x\right),{\widehat{f}}_{r+1}\left(x\right),\dots ,{\widehat{f}}_n\left(x\right))}^T.$$

The definition of $\widehat{F}\left(x\right)$ implies that $\widehat{F}$ is 2-regular at the point $x^{*}$. In the case that some of the vectors $f_{k^{\prime }}^{\prime }\left(x^{*}\right)$, $k^{\prime }\in \{r+1,\dots ,n\},$ are not zero vectors, transformation (24) can be used to reduce those vectors to zero vectors. This ensures that ${\widehat{f}}_{k^{\prime }}^{\prime }\left(x^{*}\right)=0$ for all $k^{\prime }\in \{r+1,\dots ,n\}$. Therefore, without loss of generality, we can assume that the mapping $F\left(x\right)$ satisfies $f_j^{\prime }\left(x^{*}\right)=0$ for $j=r+1,\dots ,n$. An example of a mapping that satisfies these conditions is:

$$F\left(x\right)=\left(\begin{array}{c}{x}_1+x_2\\ x_1{x}_2\end{array}\right),$$

where $n=2$, $r=1$, $j=2$, $f_1\left(x\right)=x_1+x_2$, $f_2\left(x\right)=x_1{x}_2$, and $x^{*}={(0,0)}^T.$

Suppose there exist vectors ${\xi }_1\ne 0,\dots$, ${\xi }_r\ne 0$, and $h\ne 0$, and indices $k_i\in \{r+1,\dots ,n\}$, $i=1,\dots ,r,$ such that the system:

$$\left\{f_{k_1}^{{}^{″}}\left(x^{*}\right){\xi }_1,\dots ,f_{k_r}^{{}^{″}}\left(x^{*}\right){\xi }_r,f_{r+1}^{{}^{″}}\left(x^{*}\right)h,\dots ,f_n^{{}^{″}}\left(x^{*}\right)h\right\}$$

is linearly independent.

Then the mapping $\overline{F}\left(x\right)$ defined by:

$$\overline{F}\left(x\right)={\left(f_{k_1}^{\prime }\left(x\right)·{\xi }_1,\dots ,f_{k_r}^{\prime }\left(x\right)·{\xi }_r,f_{r+1}^{\prime }\left(x\right)·h,\dots ,f_n^{\prime }\left(x\right)·h\right)}^T$$

(25)

has $x^{*}$ as its zero, that is $\overline{F}\left(x^{*}\right)=0$. At the same time, compared to the Jacobian matrix of $F\left(x\right)$, the matrix:

$${\overline{F}}^{\prime }\left(x^{*}\right)=\left[\begin{array}{c}{\left(f_{k_1}^{\prime \prime }\left(x^{*}\right){\xi }_1\right)}^T\\ \dots \\ {\left(f_{k_r}^{\prime \prime }\left(x^{*}\right){\xi }_r\right)}^T\\ {\left(f_{r+1}^{\prime \prime }\left(x^{*}\right)h\right)}^T\\ \dots \\ {\left(f_n^{\prime \prime }\left(x^{*}\right)h\right)}^T\end{array}\right]$$

is nonsingular. We can, therefore, consider the method:

$$x^{k+1}=x^k-{\left({\overline{F}}^{\prime }\left(x^k\right)\right)}^{-1}\overline{F}\left(x^k\right),k=1,2,\dots .$$

(26)

Theorem 6. 

Given a mapping $F\in C^2\left({\mathbb{R}}^n\right)$, let $x^{*}$ be a solution of Equation (5). Assume that there exist vectors ${\xi }_1\ne 0,\dots$, ${\xi }_r\ne 0$, and $h\ne 0$, such that mapping $\overline{F}\left(x\right)$ defined in (25) is nonsingular at $x^{*}$. Let $x^0\in N_{\epsilon }\left(x^{*}\right)$, where $N_{\epsilon }\left(x^{*}\right)$ is a neighborhood of $x^{*}$ and $\epsilon >0$ is sufficiently small.

Then the sequence $\left\{x^k\right\}$, $k=1,2,\dots$ defined by (26) is convergent to the point $x^{*}$ with the quadratic rate of convergence, that is:

$$\parallel x^{k+1}-x^{*}\parallel \le C\parallel x^k-x^{*}{\parallel }^2,$$

where $C>0$ is an independent constant.

Using definition of mapping F given by Equation (1), mappings $f_i$ introduced in (4) will have the following form:

$$f_i\left(x\right)=x^T{B}_{i}x+M_i^{T}x+N_i,i=1,2,\dots ,n,$$

where $B_i$ is an $n\times n$ symmetric matrix, $M_i\in {\mathbb{R}}^n$, and $N_i\in \mathbb{R}$, $i=1,2,\dots ,n.$

Given an initial point $x^0$, we use the iterative method (26) to obtain:

$$x^1=x^0-{\left[\begin{array}{c}{\left(2B_{k_1}{\xi }_1\right)}^T\\ ⋮\\ {\left(2B_{k_r}{\xi }_r\right)}^T\\ {\left(2B_{r+1}h\right)}^T\\ ⋮\\ {\left(2B_{n}h\right)}^T\end{array}\right]}^{-1}\left(\begin{array}{c}{(2B_{k_1}{x}^0+M_{k_1})}^T{\xi }_1\\ ⋮\\ {(2B_{k_r}{x}^0+M_{k_r})}^T{\xi }_r\\ {(2B_{r+1}{x}^0+M_{r+1})}^{T}h\\ ⋮\\ {(2B_n{x}^0+M_n)}^{T}h\end{array}\right)=$$

$$={\left[\begin{array}{c}{\left(2B_{k_1}{\xi }_1\right)}^T\\ ⋮\\ {\left(2B_{k_r}{\xi }_r\right)}^T\\ {\left(2B_{r+1}h\right)}^T\\ ⋮\\ {\left(2B_{n}h\right)}^T\end{array}\right]}^{-1}\left(\left(\begin{array}{c}{\left(2B_{k_1}{\xi }_1\right)}^T{x}^0\\ ⋮\\ {\left(2B_{k_r}{\xi }_r\right)}^T{x}^0\\ {\left(2B_{r+1}h\right)}^T{x}^0\\ ⋮\\ {\left(2B_{n}h\right)}^T{x}^0\end{array}\right)-\left(\begin{array}{c}{\left(2B_{k_1}{x}^0\right)}^T{\xi }_1\\ ⋮\\ {\left(2B_{k_r}{x}^0\right)}^T{\xi }_r\\ {\left(2B_{r+1}{x}^0\right)}^{T}h\\ ⋮\\ {\left(2B_n{x}^0\right)}^{T}h\end{array}\right)-\left(\begin{array}{c}{M}_{k_1}^T{\xi }_1\\ ⋮\\ M_{k_r}^T{\xi }_r\\ M_{r+1}^{T}h\\ ⋮\\ M_n^{T}h\end{array}\right)\right).$$

Because matrix $B_i$ is symmetric for any index i, then for any index j, we have:

$${\left(2B_i{\xi }_j\right)}^T{x}^0=\left(2B_i{\xi }_j\right)·x^0={\xi }_j·\left(2B_i^T{x}^0\right)={\xi }_j·\left(2B_i{x}^0\right)={\left(2B_i{x}^0\right)}^T{\xi }_j.$$

Therefore,

$$x^1=-\frac{1}{2}{\left[\begin{array}{c}{\left(B_{k_1}{\xi }_1\right)}^T\\ ⋮\\ {\left(B_{k_r}{\xi }_r\right)}^T\\ {\left(B_{r+1}h\right)}^T\\ ⋮\\ {\left(B_{n}h\right)}^T\end{array}\right]}^{-1}\left(\begin{array}{c}{M}_{k_1}·{\xi }_1\\ ⋮\\ M_{k_r}·{\xi }_r\\ M_{r+1}·h\\ ⋮\\ M_n·h\end{array}\right)=x^{*}$$

(27)

Example 1 

(Continuation). Consider mapping $F:{\mathbb{R}}^2\to {\mathbb{R}}^2$ defined in (23):

$$F\left(x\right)=\left(\begin{array}{c}{x}_1^2-x_2^2-2x_1+1\\ x_1{x}_2-x_2\end{array}\right),$$

where

$$B_1=\left[\begin{array}{cc}1& 0\\ 0& -1\end{array}\right],B_2=\left[\begin{array}{cc}0& 1/2\\ 1/2& 0\end{array}\right],$$

$$M_1={(-2,0)}^T,M_2={(0,-1)}^T,N_1=1,N_2=0.$$

In this example, $x^{*}={(1,0)}^T$ is a solution of $F\left(x\right)=0$ and:

$$f_1^{\prime }\left(x^{*}\right)={(2x_1^{*}-2,2x_2^{*})}^T={(0,0)}^T,f_2^{\prime }\left(x^{*}\right)={(x_2^{*},x_1^{*}-1)}^T={(0,0)}^T.$$

Therefore, mapping $\overline{F}$ defined in (25) takes the form:

$$\overline{F}\left(x\right)=\left(\begin{array}{c}{f}_1^{\prime }\left(x\right)·h\\ f_2^{\prime }\left(x\right)·h\end{array}\right),$$

where h is chosen in such a way that the matrix ${\overline{F}}^{\prime }\left(x^{*}\right)$ is nonsingular, and vectors ${\xi }_i$ are not used. For example, we can take $h={(1,0)}^T$. Then Equation (27) has the form:

$$\begin{array}{ccc}\hfill x^1& =& -{\displaystyle \frac{1}{2}}{\left[\begin{array}{c}{\left(B_{1}h\right)}^T\\ {\left(B_{2}h\right)}^T\end{array}\right]}^{-1}\left(\begin{array}{c}{M}_1·h\\ M_2·h\end{array}\right)=-{\displaystyle \frac{1}{2}}{\left[\begin{array}{cc}1& 0\\ 0& 1/2\end{array}\right]}^{-1}\left(\begin{array}{c}-2\\ 0\end{array}\right)\hfill \\ \hfill & =& -\left[\begin{array}{cc}1/2& 0\\ 0& 1\end{array}\right]\left(\begin{array}{c}-2\\ 0\end{array}\right)=\left(\begin{array}{c}1\\ 0\end{array}\right),\hfill \end{array}$$

(28)

which is a solution of $F\left(x\right)=0$ in this example.

The approaches described above can be modified to derive other methods for solving the equation $F\left(x\right)=0$. For example, using the equation $F^{\prime }{\left(x\right)}^{T}h=0$, where $h\in \mathrm{Ker}{F}^{\prime }{\left(x^{*}\right)}^T$, we obtain the following method:

$$x^{k+1}=x^k-{\left({\left(F^{\prime }{\left(x^k\right)}^T\right)}^{\prime }h\right)}^{-1}\left({\left(F^{\prime }\left(x^k\right)\right)}^{T}h\right),k=0,1,\dots .$$

The sequence $\left\{x_k\right\}$ converges to $x^{*}$ under the assumption that ${\left(F^{\prime }{\left(x^{*}\right)}^T\right)}^{\prime }h$ is nonsingular. In this modification, unlike the second approach, we can construct an element h without the knowledge of the point $x^{*}$, based on the information at an initial point $x^0$.

Applying the modified method to Example 1, we obtain the same formulas and results as shown in Equation (28) above. To implement this approach, it is necessary to determine the vectors $f_i^{\prime }\left(x\right)$, $i=1,2,\dots ,n$, which correspond to linearly independent vectors $f_i^{\prime }\left(x^{*}\right),$$i\in \{1,2,\dots ,n\}.$ This can be achieved using information at a point $x^0\in N_{\epsilon }\left(x^{*}\right)$, where $\epsilon >0$ is sufficiently small. If the assumption of p-regularity is satisfied, the identification of linearly independent vectors is performed using the method described in the next section.

4. Procedure for Identifying Zero Elements

The procedure for identifying zero elements could be used to implement the methods described in the previous sections numerically. Let $F\left(x\right):{\mathbb{R}}^n\to {\mathbb{R}}^m$ be defined as:

$$F\left(x\right)={(f_1\left(x\right),\dots ,f_m\left(x\right))}^T.$$

(29)

In this section, we present a theorem that describes the properties of a special mapping $\mu \left(x\right)$, which allows us to propose the method for determining r linear independent vectors $f_{i_k}^{\prime }\left(x^{*}\right)$, $k=1,\dots ,r,$ at the solution $x^{*}$ of $F\left(x\right)=0$. This procedure is based on the information about the system of vectors $\{f_1^{\prime }\left(x\right),\dots ,f_m^{\prime }\left(x\right)\}$ at some point x in a small neighborhood of $x^{*}$. As a result, we can define the mapping $\widehat{F}\left(x\right)$ with the first r components $f_{i_k}\left(x\right)$, $k=1,\dots ,r,$ corresponding to the linearly independent vectors $f_{i_k}^{\prime }\left(x^{*}\right)$, $k=1,\dots ,r.$

Let $F\in C^3\left({\mathbb{R}}^n\right)$ be 2-regular at the point $x^{*}$. For some $x\in N_{\epsilon }\left(x^{*}\right)$, where $\epsilon$ is sufficiently small, we define the following mappings:

$$\rho \left(x\right)=\underset{i=1,\dots ,m}{min}d\left(f_i^{\prime }\left(x\right),\mathrm{span}(f_1^{\prime }\left(x\right),\dots ,f_{i-1}^{\prime }\left(x\right),f_{i+1}^{\prime }\left(x\right),\dots ,f_m^{\prime }\left(x\right))\right)$$

and:

$$\mu \left(x\right)=max\left\{{\parallel F\left(x\right)\parallel }^{\frac{1}{2}},\rho \left(x\right)\right\},$$

(30)

where $d(x,S)$ denotes the distance between an element x and the set S. Note that if $F\left(x\right)=f_1\left(x\right)$, then $\rho \left(x\right)=\parallel f_1^{\prime }\left(x\right)\parallel$.

The mapping $\mu \left(x\right)$ is used to determine the maximum number r of linearly independent vectors in the system $\{f_1^{\prime }\left(x^{*}\right),\dots ,f_m^{\prime }\left(x^{*}\right)\}$ using a special procedure that relies on the information about the mapping $F\left(x\right)$ at the point $x\in N_{\epsilon }\left(x^{*}\right)$. The properties of the mapping $\mu \left(x\right)$ are stated in the following theorem, and the proof can be found in [16].

Theorem 7 

(Minorant theorem). Let $F\in C^3\left({\mathbb{R}}^n\right)$ be 2-regular at the point $x^{*}$, and $F\left(x^{*}\right)=0$. Then there exist constants $\epsilon >0,$ $C^{\prime }>0$, and $C^{″}>0$ such that the following inequality holds for any $x\in N_{\epsilon }\left(x^{*}\right)$:

$$C^{\prime }\parallel x-x^{*}\parallel \le \mu \left(x\right)\le C^{″}{\parallel x-x^{*}\parallel }^{\frac{1}{2}},$$

where function $\mu \left(x\right)$ is defined in (30).

In addition to the properties of the mapping $\mu \left(x\right)$ given in Theorem 7, we also need the following lemma (for the proof, see [16]).

Lemma 1. 

For the non-negative mappings $g\left(x\right)$ and $\mu \left(x\right)$, let the following inequalities hold:

$$|g\left(x_1\right)-g\left(x_2\right)|\le L\parallel x_1-x_2\parallel \mathit{for}\mathit{all}{x}_1,x_2\in N_{\sigma }\left(x^{*}\right),$$

$$C^{\prime }\parallel x-x^{*}\parallel \le \mu \left(x\right)\le C^{″}{\parallel x-x^{*}\parallel }^{\frac{1}{p}}\mathit{for}\mathit{all}x\in N_{\sigma }\left(x^{*}\right),$$

where $L,$$C^{\prime }$, $C^{″}$, and σ are positive constants, with $C^{″}\ge C^{\prime }$ and $p\ge 2$.

Then, there exists a sufficiently small $\epsilon >0$ such that one of the following conditions holds:

1. 

If $g\left(x\right)\le \sqrt{\mu \left(x\right)}$ for all $x\in N_{\epsilon }\left(x^{*}\right)$, then $g\left(x^{*}\right)=0.$

2. 

If $g\left(x\right)>\sqrt{\mu \left(x\right)}$ for all $x\in N_{\epsilon }\left(x^{*}\right)$, then $g\left(x^{*}\right)\ne 0.$

Remark 2. 

Based on the assumptions of Lemma 1, there exists a sufficiently small $\epsilon >0$ such that if the inequality $g\left(\overline{x}\right)\le \sqrt{\mu \left(\overline{x}\right)}$ is satisfied for any $\overline{x}\in N_{\epsilon }\left(x^{*}\right)$, then the inequality $g\left(x\right)\le \sqrt{\mu \left(x\right)}$ is satisfied for all $x\in N_{\epsilon }\left(x^{*}\right)$, and hence $g\left(x^{*}\right)=0$.

Similarly, if the inequality $g\left(\overline{x}\right)>\sqrt{\mu \left(\overline{x}\right)}$ is satisfied for any $\overline{x}\in N_{\epsilon }\left(x^{*}\right)$, then the inequality $g\left(x\right)>\sqrt{\mu \left(x\right)}$ is satisfied for all $x\in N_{\epsilon }\left(x^{*}\right)$, and hence $g\left(x^{*}\right)\ne 0$.

Now we are ready to introduce an iterative method that determines indices $1,\dots ,r$ corresponding to the linearly independent vectors $f_i^{\prime }\left(x^{*}\right)$, $i=1,\dots ,r.$

• Method for determining linearly independent gradients at $x^{*}$ (identifying zero elements).

Using Lemma 1 and Remark 2, for a sufficiently small $\epsilon >0$, $x\in N_{\epsilon }\left(x^{*}\right)$, and $i=1,\dots ,m$, consider two possible cases:

Case 1. $g\left(x\right)=\parallel f_i^{\prime }\left(x\right)\parallel \le \sqrt{\mu \left(x\right)}$.

Case 2. $g\left(x\right)=\parallel f_i^{\prime }\left(x\right)\parallel >\sqrt{\mu \left(x\right)}$.

In Case 1, according to Remark 2, it follows that $f_i^{\prime }\left(x^{*}\right)=0$, whereas in Case 2, we have $f_i^{\prime }\left(x^{*}\right)\ne 0$.

• In addition to the properties of the mapping

Let $F\left(x\right):{\mathbb{R}}^n\to {\mathbb{R}}^m$ be defined by (29) and $x^{*}$ be a solution of $F\left(x\right)=0$. Let x be in $N_{\epsilon }\left(x^{*}\right)$, where $\epsilon$ is sufficiently small. Define function $\mu \left(x\right)$ using Equation (30).

• Step 1. Identify the smallest index $i_1$ in the set $S_1=\{1,\dots ,m\}$ such that $\parallel f_{i_1}^{\prime }\left(x\right)\parallel >\sqrt{\mu \left(x\right)}$. According to Case 2 above, this implies that $f_{i_1}^{\prime }\left(x^{*}\right)\ne 0$.
• Step 2. Use Step 1 to identify if the set $S_2=\{1,\dots ,m\}\setminus \left\{i_1\right\}$ has at least one index j such that $f_j^{\prime }\left(x^{*}\right)\ne 0$. If it does not, the method is finished. Otherwise, identify the next smallest index $i_2$ in the set $S_1$ such that the following condition is satisfied:

  $$d\left(f_{i_2}^{\prime }\left(x\right),\mathrm{span}\left(f_{i_1}^{\prime }\left(x\right)\right)\right)>\sqrt{\mu \left(x\right)}.$$

According to Case 2 above, it means that the vectors $f_{i_1}^{\prime }\left(x^{*}\right)$ and $f_{i_2}^{\prime }\left(x^{*}\right)$ are linearly independent.

• Step k. By this step, we have identified $k-1$ linearly independent vectors $f_{i_1}^{\prime }\left(x^{*}\right)$, …, $f_{i_{k-1}}^{\prime }\left(x^{*}\right)$, where $k=3,4,\dots ,m$. Use Step 1 to identify if the set $S_k=\{1,\dots ,m\}\setminus \{i_1,\dots ,i_{k-1}\}$ has at least one index j such that $f_j^{\prime }\left(x^{*}\right)\ne 0$. If it does not, the method is finished. Otherwise, identify the next smallest index $i_k\in \{1,\dots ,m\}\setminus \{i_1,\dots ,i_{k-1}\}$ such that the following condition is satisfied:

$$d\left(f_{i_k}^{\prime }\left(x\right),\mathrm{span}\left(f_{i_1}^{\prime }\left(x\right),\dots ,f_{i_{k-1}}^{\prime }\left(x\right)\right)\right)>\sqrt{\mu \left(x\right)}.$$

The inequality implies that the vectors $f_{i_1}^{\prime }\left(x^{*}\right)$, $\dots ,$$f_{i_k}^{\prime }\left(x^{*}\right)$ are linearly independent.

Repeat Step k until the method is finished.

Without loss of generality, assume that the first r vectors $\{f_1^{\prime }\left(x^{*}\right),\dots ,f_r^{\prime }\left(x^{*}\right)\}$ are linearly independent and define mapping $\widehat{F}\left(x\right)$ as:

$$\widehat{F}\left(x\right)=\left(\begin{array}{c}{f}_1\left(x\right)\\ ⋮\\ f_r\left(x\right)\\ {\widehat{f}}_{r+1}\left(x\right)\\ ⋮\\ {\widehat{f}}_m\left(x\right)\end{array}\right),$$

(31)

where vectors ${\widehat{f}}_k\left(x\right)$ are defined in such a way that:

$${\widehat{f}}_k^{\prime }\left(x^{*}\right)=0,k=r+1,\dots ,m.$$

Namely, let:

$$f_k^{\prime }\left(x^{*}\right)=\sum _{i=1}^r{\alpha }_i^k\left(x^{*}\right)f_i^{\prime }\left(x^{*}\right),k=r+1,\dots ,m,$$

be a linear combination of the vectors $f_1^{\prime }\left(x^{*}\right)$, …, $f_r^{\prime }\left(x^{*}\right).$ Coefficients ${\alpha }^k\left(x\right)$ are determined by solving the following system of equations:

$$\left(f_k^{\prime }\left(x\right)-\sum _{i=1}^r{\alpha }_i^k\left(x\right)f_i^{\prime }\left(x\right)\right)·f_j^{\prime }\left(x\right)=0,j=1,2,\dots ,r,k=r+1,\dots ,m.$$

In addition, define $B\left(x\right)$ to be a nonsingular matrix of the form:

$$B\left(x\right)=\left(\begin{array}{cccccc}1& \cdots & 0& 0& \cdots & 0\\ ⋮& \ddots & ⋮& ⋮& \ddots & ⋮\\ 0& \cdots & 1& 0& \cdots & 0\\ -{\alpha }_1^{r+1}\left(x\right)& \cdots & -{\alpha }_r^{r+1}\left(x\right)& 1& \cdots & 0\\ ⋮& \ddots & ⋮& ⋮& \ddots & ⋮\\ -{\alpha }_1^m\left(x\right)& \cdots & -{\alpha }_r^m& 0& \cdots & 1\end{array}\right).$$

Let:

$${\alpha }^k\left(x\right)=({\alpha }_1^k\left(x\right),\dots ,{\alpha }_r^k\left(x\right)),k=r+1,\dots ,m.$$

Define the following vectors:

$${\widehat{f}}_k\left(x\right)=f_k\left(x\right),k=1,\dots ,r,$$

$${\widehat{f}}_k\left(x\right)=f_k\left(x\right)-\sum _{i=1}^r{\alpha }_i^k\left(x\right)f_i\left(x\right),k=r+1,\dots ,m.$$

These vectors allow us to transform the mapping $F\left(x\right)$ to $\widehat{F}\left(x\right)=B\left(x\right)·F\left(x\right)$, where ${\widehat{f}}_1^{\prime }\left(x^{*}\right)\ne 0,$$\dots ,$${\widehat{f}}_r^{\prime }\left(x^{*}\right)\ne 0,$ and ${\widehat{f}}_{r+1}^{\prime }\left(x^{*}\right)=0,$$\dots ,$${\widehat{f}}_m^{\prime }\left(x^{*}\right)=0.$ The purpose of this transformation is to simplify the structure of the projection operators.

We present a simple example to illustrate an application of the proposed method.

Example 2. 

Let $F:{\mathbb{R}}^2\to {\mathbb{R}}^2$, $F\left(x\right)={(f_1\left(x\right),f_2\left(x\right))}^T$, where:

$$f_1\left(x\right)=x_1+x_2,f_2\left(x\right)=x_1{x}_2.$$

Then $x^{*}={(0,0)}^T$ is a solution of $F\left(x\right)=0$. Take $\epsilon =\frac{1}{\sqrt{2}}$ and consider $\overline{x}={\left(\frac{1}{2},\frac{1}{2}\right)}^T\in N_{\epsilon }\left(x^{*}\right)$. The Jacobian matrix of F is:

$$F^{\prime }\left(x\right)=\left(\begin{array}{cc}1& 1\\ x_2& x_1\end{array}\right),$$

$$f_1^{\prime }\left(x^{*}\right)=\left(\begin{array}{c}1\\ 1\end{array}\right),and{f}_2^{\prime }\left(x^{*}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right).$$

It is easy to see that vectors $f_1^{\prime }\left(x^{*}\right)$ and $f_2^{\prime }\left(x^{*}\right)$ are linearly dependent. We can check this by applying the method introduced above.

By using Equation (30), we define function $\mu \left(x\right)=max\left\{{\parallel F\left(x\right)\parallel }^{\frac{1}{2}},\rho \left(x\right)\right\}$, where:

$$\rho \left(x\right)=sin\alpha \parallel f_2^{\prime }\left(x\right)\parallel ,$$

and α is the angle between vectors $f_1^{\prime }\left(x\right)$ and $f_2^{\prime }\left(x\right)$. Note that:

$$cos\alpha =\frac{f_1^{\prime }\left(x\right)·f_2^{\prime }\left(x\right)}{\parallel f_1^{\prime }\left(x\right)\parallel \parallel f_2^{\prime }\left(x\right)\parallel },$$

and hence:

$$\rho \left(x\right)=\sqrt{\frac{\parallel f_1^{\prime }{\left(x\right)\parallel }^2{\parallel f_2^{\prime }\left(x\right)\parallel }^2-{\left(f_1^{\prime }\left(x\right)·f_2^{\prime }\left(x\right)\right)}^2}{\parallel f_1^{\prime }{\left(x\right)\parallel }^2{\parallel f_2^{\prime }\left(x\right)\parallel }^2}}\parallel f_2^{\prime }\left(x\right)\parallel =\frac{|x_1-x_2|}{\sqrt{2}}.$$

Using $\overline{x}={\left(\frac{1}{2},\frac{1}{2}\right)}^T$, we obtain:

$$\rho \left(\overline{x}\right)=0,{\parallel F\left(\overline{x}\right)\parallel }^{\frac{1}{2}}=\sqrt[4]{1^2+{\left(\frac{1}{4}\right)}^2}=\sqrt[4]{\frac{17}{16}},\mathrm{and}\mathrm{so}\mu \left(\overline{x}\right)=\sqrt[4]{\frac{17}{16}}.$$

We are ready to apply the method described above.

In Step 1, we obtain $\parallel f_1^{\prime }\left(\overline{x}\right)\parallel >\sqrt{\mu \left(\overline{x}\right)}$ because:

$$\sqrt{2}>\sqrt[8]{\frac{17}{16}}.$$

Hence, vector $f_1^{\prime }\left(x^{*}\right)\ne 0$ and $i_1=1$.

Then in Step 1 of the method with vector $f_2^{\prime }\left(x\right)$, we also verify whether the following inequality holds:

$$\parallel f_2^{\prime }\left(\overline{x}\right)\parallel >\mu {\left(\overline{x}\right)}^{\frac{1}{2}}or\sqrt{{\overline{x}}_2^2+{\overline{x}}_1^2}>\mu {\left(\overline{x}\right)}^{\frac{1}{2}}.$$

Using point $\overline{x}={\left(\frac{1}{2},\frac{1}{2}\right)}^T$, we obtain:

$$\sqrt{\frac{1}{2}}<\sqrt[8]{\frac{17}{16}}.$$

Therefore, we conclude that $f_2^{\prime }\left(x^{*}\right)=0$.

Thus, in this example, the mapping $\widehat{F}\left(x\right)$ defined in (31) has the form $\widehat{F}\left(x\right)={(f_1,{\widehat{f}}_2)}^T$, where $f_1\left(x\right)=x_1+x_2$ and ${\widehat{f}}_2=x_1{x}_2$.

5. Quadratic Programming Problems

In this section, we consider the quadratic programming (QP) problem (3):

$$\begin{array}{cc}\underset{x}{\mathrm{minimize}}\hfill & f\left(x\right)=\frac{1}{2}{x}^{T}Qx+c^{T}x\hfill \\ \mathrm{subject}\mathrm{to}\hfill & Ax\le b\hfill \end{array}\left(\mathrm{QP}\right),$$

where Q is an $n\times n$ symmetric matrix, A is an $m\times n$ matrix, $c,x\in {\mathbb{R}}^n$, and $b\in {\mathbb{R}}^m$. The Lagrangian for problem (3) is defined by:

$$\mathcal{L}(x,\lambda )=\frac{1}{2}{x}^{T}Qx+c^{T}x+\sum _{i=1}^m{\lambda }_i(a_{i}x-b_i),$$

(32)

where $\lambda =({\lambda }_1,\dots ,{\lambda }_m)$ is the vector of Lagrange multipliers and $a_i$ is the ith row of the matrix A. The Karush-Kuhn-Tucker (KKT) conditions [17] are satisfied at $x^{*}$ with some ${\lambda }^{*}\in {\mathbb{R}}^m$ if:

$$\begin{array}{cc}\hfill Q{x}^{*}+c+{\displaystyle \sum _{i=1}^m}{\lambda }_i^{*}{a}_i^T=0,& \\ \hfill {\lambda }_i^{*}(a_i{x}^{*}-b_i)=0,{\lambda }_i^{*}\ge 0,a_i{x}^{*}\le b_i,& \mathrm{for} \mathrm{all} i=1,\dots ,m.\hfill \end{array}$$

(33)

The point $x^{*}$ at which relations (33) are satisfied is called a stationary point or a KKT point. Observe that $(x^{*},{\lambda }^{*})$ is a solution of the following system:

$$\mathsf{\Phi }(x,\lambda )=\left(\begin{array}{c}Qx+c+A^T\lambda \\ \mathsf{\Lambda }(Ax-b)\end{array}\right)=0,\mathsf{\Lambda }=\mathrm{diag}{\left({\lambda }_i\right)}_{i=1,\dots ,m},{\lambda }_i^{*}\ge 0,A{x}^{*}\le b.$$

(34)

We denote by $I\left(x^{*}\right)$ the set of indices of the active constraints at $x^{*}$:

$$I\left(x^{*}\right)=\{i=1,\dots ,m\mid a_i{x}^{*}=b_i\}.$$

The following constraint qualification is used in the paper.

Definition 10 

(Linear independence constraint qualification). The linear independence constraint qualification (LICQ) holds at a feasible point $x^{*}$ if the row-vectors $a_j$, $j\in I\left(x^{*}\right)$, corresponding to the active at $x^{*}$ constraints, are linearly independent.

The modified second-order sufficient conditions (MSOSC) state that there exist a Lagrange multiplier vector ${\lambda }^{*}$ and a scalar $\nu >0$ such that:

$${\omega }^T{\nabla }^2{\mathcal{L}}_{xx}(x^{*},{\lambda }^{*})\omega \ge \nu {\parallel \omega \parallel }^2$$

(35)

for all $\omega$ satisfying:

$$a_i\omega =0\forall i\in I\left(x^{*}\right).$$

We divide the presentation in this section into three parts. We start by considering regular QP problems in Section 5.1. Then, in Section 5.2, we discuss the issue of identifying the active constraints and propose numerical strategies for determining the set $I\left(x^{*}\right)$. We apply these techniques to degenerate QP problems in Section 5.3.

5.1. Regular Quadratic Programming

In this section, we consider regular quadratic programming (QP) problem (3). In other words, we assume that the Linear Independence Constraint Qualification (LICQ) and the Mangasarian-Fromovitz Constraint Qualification (MFCQ) conditions (35) hold. Recall that A is an $m\times n$ matrix of coefficients representing the constraints $Ax\le b$ in problem (3). Without loss of generality, assume that the first p constraints are active at $x^{*}$, so that:

$$I\left(x^{*}\right)=\{1,\dots ,p\}.$$

Then we can rewrite the matrix A in the following form:

$$A=\left[\begin{array}{c}{A}_A\\ A_N\end{array}\right],$$

(36)

where $A_A$ is a $p\times n$ matrix of coefficients corresponding to the active constraints at $x^{*}$, and $A_N$ is an $(m-p)\times n$ matrix of coefficients corresponding to the nonactive constraints at $x^{*}$. It is important to note that we do not have prior knowledge of the set $I\left(x^{*}\right)$. We will discuss possible numerical realizations to approximate the set of active constraints in Section 5.2. Additionally, we introduce the following notation associated with the active constraints at the point $x^{*}$:

$$b_A={(b_1,\dots ,b_p)}^T,{\lambda }_A={({\lambda }_1,\dots ,{\lambda }_p)}^T.$$

Similarly,

$$b_N={(b_{p+1},\dots ,b_m)}^T,{\lambda }_N={({\lambda }_{p+1},\dots ,{\lambda }_m)}^T.$$

In the following subsections, we will introduce three approaches to solving the QP problem (3) and provide formulas for the solution.

5.1.1. First Approach to Solving the QP Problem

In this subsection, we present an approach to solving the QP problem and obtaining a formula for its solution. This approach is based on the construction of the 2-factor-operator. For our consideration below, we need the following lemma.

Lemma 2. 

Let V be an $n\times n$ matrix, G be a $p\times n$ matrix, such that the columns of $G^T$ are linearly independent, L be an $n\times l$ matrix, $G_N=diag{\left(g_i\right)}_{i=1}^l$ be a diagonal full rank matrix, and:

$$(Vx,x)>0 for all x\in \mathrm{Ker}G\setminus \left\{0\right\}.$$

(37)

Then matrix Γ defined by:

$$\mathsf{\Gamma }=\left(\begin{array}{ccc}V& G^T& L\\ G& O_{p\times p}& O_{p\times l}\\ O_{l\times n}& O_{l\times p}& G_N\end{array}\right)$$

(38)

is nonsingular.

Proof. 

To prove the lemma, we must prove that the matrix $\mathsf{\Gamma }$ defined by (38) has zero nullspace. Consider the following system that defines the nullspace of $\mathsf{\Gamma }$ in the form of a vector $v=(x,y,z)$, where $x\in {\mathbb{R}}^n$, $y\in {\mathbb{R}}^p$, and $z\in {\mathbb{R}}^l$:

$$\begin{array}{c}\hfill Vx+G^{T}y+Lz=0\\ \hfill Gx=0\\ \hfill G_{N}z=0.\end{array}$$

(39)

Since $G_N$ is a full-rank diagonal matrix, the third equation in the system (39) implies that $z=0$. Then, using the first equation, we obtain:

$$\begin{array}{cc}\hfill 0& =\left(Vx\right)·x+\left(G^{T}y\right)·x\hfill \\ \hfill & =\left(Vx\right)·x+y·\left(Gx\right)\hfill \\ \hfill & =\left(Vx\right)·x.\hfill \end{array}$$

Consequently, $x=0$; otherwise, $\left(Vx\right)·x=0$, which contradicts the assumption (37) of the lemma. Therefore, the first equation in (39) reduces to $G^{T}y=0$, and since the columns of $G^T$ are linearly independent, we obtain $y=0$. Thus, the matrix $\mathsf{\Gamma }$ (38) has a zero nullspace, $(x,y,z)=(0,0,0)$, and therefore, $\mathsf{\Gamma }$ is nonsingular. This concludes the proof of the lemma. □

Let $x\in {\mathbb{R}}^n$, $\lambda \in {\mathbb{R}}^m,$ and mapping $\mathsf{\Phi }$ be defined in (34), so that $\mathsf{\Phi }(x,\lambda )=\left(\begin{array}{c}Qx+c+A^T\lambda \\ \mathsf{\Lambda }(Ax-b)\end{array}\right)$. Introduce mappings $P_1$ and $P_2$ as:

$$P_1=\left[\begin{array}{cc}{I}_{n\times n}& O_{n\times m}\\ O_{m\times n}& O_{m\times m}\end{array}\right],P_2=\left[\begin{array}{cc}{O}_{n\times n}& O_{n\times m}\\ O_{m\times n}& I_{m\times m}\end{array}\right].$$

Recall that matrix $A_A$ is defined in (36), and introduce vector $\overline{h}\in {\mathbb{R}}^{n+m}$ such that:

$$\overline{h}={(h^1,h^2)}^T,h^1\in {\mathbb{R}}^n,h^2\in {\mathbb{R}}^m,$$

where $A_A{h}^1=0$, $h^1\ne 0$, $h^2=(\underset{p}{\underbrace{\phantom{(}1\cdots 1}},\underset{m-p}{\underbrace{\phantom{(}0\cdots 0}}).$

Define mapping $\mathsf{\Psi }$ as:

$$\mathsf{\Psi }(x,\lambda )=P_1\mathsf{\Phi }(x,\lambda )+P_2{\mathsf{\Phi }}^{\prime }(x,\lambda )\overline{h}.$$

(40)

Recall that $a_i$ is the ith row of the matrix A and $b={(b_1,\dots ,b_m)}^T$. Then:

$${\mathsf{\Phi }}^{\prime }(x,\lambda )=\left(\begin{array}{cc}Q& A^T\\ \mathsf{\Lambda }A& S\end{array}\right),S=\left(\begin{array}{ccc}{a}_{1}x-b_1& 0& 0\\ & \ddots & \\ 0& 0& a_{m}x-b_m\end{array}\right),$$

and mapping $\mathsf{\Psi }$ defined in (40) can be rewritten as:

$$\mathsf{\Psi }(x,\lambda )=\left(\begin{array}{c}Qx+c+A^T\lambda \\ \mathsf{\Lambda }A{h}^1+S{h}^2\end{array}\right).$$

Introduce matrix ${\mathsf{\Lambda }}_N=\mathrm{diag}{\left({\lambda }_i\right)}_{i=p+1,\dots ,m}$. Then, taking into account the definition of $h^1$ and $h^2$, we obtain:

$$\mathsf{\Psi }(x,\lambda )=\left(\begin{array}{c}Qx+c+A^T\lambda \\ A_{A}x-b_A\\ {\mathsf{\Lambda }}_N{A}_N{h}^1\end{array}\right).$$

Observe that if $(x^{*},{\lambda }^{*})$ is a solution of (34), it is also a solution of $\mathsf{\Psi }(x,\lambda )=0,$ or, equivalently,

$$\mathsf{\Psi }(x,\lambda )=\left(\begin{array}{c}Qx+c+A^T\lambda \\ A_{A}x-b_A\\ {\mathsf{\Lambda }}_N{A}_N{h}^1\end{array}\right)=0.$$

(41)

To obtain the formula for the solution $(x^{*},{\lambda }^{*})$, we rewrite the system (41) as:

$$\left(\begin{array}{ccc}Q& A_A^T& A_N^T\\ A_A& O& O\\ O_{(m-p)\times n}& O& K\end{array}\right)\left(\begin{array}{c}x\\ {\lambda }_A\\ {\lambda }_N\end{array}\right)=\left(\begin{array}{c}-c\\ b_A\\ O_{m-p}\end{array}\right),K=\left(\begin{array}{ccc}{a}_{p+1}{h}^1& 0& 0\\ & \ddots & \\ 0& 0& a_m{h}^1\end{array}\right).$$

Assuming that LICQ and MSOSC hold and apply Lemma 2, we obtain that the matrix:

$$\left(\begin{array}{ccc}Q& A_A^T& A_N^T\\ A_A& O& O\\ O_{(m-p)\times n}& O& K\end{array}\right)$$

is invertible and obtain the formula for $(x^{*},{\lambda }^{*})$:

$$\left(\begin{array}{c}{x}^{*}\\ {\lambda }_A^{*}\\ {\lambda }_N^{*}\end{array}\right)={\left(\begin{array}{ccc}Q& A_A^T& A_N^T\\ A_A& O& O\\ O_{(m-p)\times n}& O& K\end{array}\right)}^{-1}\left(\begin{array}{c}-c\\ b_A\\ O_{m-p}\end{array}\right).$$

(42)

5.1.2. Second Approach to Solving the QP Problem

Assume that we can estimate the set $I\left(x^{*}\right)$, which is in our notation $I\left(x^{*}\right)=\{1,2,\dots ,p\}$. Taking into account that ${\lambda }_{p+1}^{*}=0,\dots ,{\lambda }_m^{*}=0$ and that $A_A{x}^{*}=b_A$, system (34) can be reduced to the following one:

$$\mathsf{\Phi }(x,\lambda )=\left(\begin{array}{c}Qx+c+A_A^T{\lambda }_A\\ A_{A}x-b_A\end{array}\right)=0,$$

(43)

which can be written as:

$$\left(\begin{array}{cc}Q& A_A^T\\ A_A& O_{p\times p}\end{array}\right)\left(\begin{array}{c}x\\ {\lambda }_A\end{array}\right)=\left(\begin{array}{c}-c\\ b_A\end{array}\right).$$

(44)

Under the assumptions LICQ and MSOSC, the following matrix is invertible,

$$\left(\begin{array}{cc}Q& A_A^T\\ A_A& O_{p\times p}\end{array}\right)$$

and system (44) yields the formula for the solution $(x^{*},{\lambda }^{*})$:

$$\left(\begin{array}{c}{x}^{*}\\ {\lambda }_A^{*}\end{array}\right)={\left[\begin{array}{cc}Q& A_A^T\\ A_A& O_{p\times p}\end{array}\right]}^{-1}\left(\begin{array}{c}-c\\ b_A\end{array}\right).$$

(45)

Remark 3. 

System (41) reduces to system (43) by removing equations ${\mathsf{\Lambda }}_N{A}_N{h}^1$, corresponding to the nonactive constraints. Similarly, Equation (42) reduces to (45).

Remark 4. 

We note that solutions of QP problems have the following specific property: if $x^{*}$ is a solution of the QP problem and $h^T{\mathcal{L}}_{xx}^{″}(x,\lambda )h=0$ for the vector $h\in \mathrm{Ker}\mathrm{A}$, then the points $x=x^{*}+th$ are also solutions of the QP problem.

5.1.3. Examples

In this section, we illustrate the two described approaches with examples. Namely, we consider the construction of system (41) required for the first approach. Then we illustrate using the exact formula (45) derived in the second approach.

Example 2. 

Consider the problem:

$$\begin{array}{cc}\underset{x}{\mathrm{minimize}}\hfill & \frac{1}{2}{x}_2^2-x_1\hfill \\ \mathrm{subject}\mathrm{to}\hfill & x_1\le 0,\hfill \\ \hfill & x_2\le 1.\hfill \end{array}$$

(46)

The matrix A in this example is $A=\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)$ and $b=\left(\begin{array}{c}0\\ 1\end{array}\right).$ The solution to this problem is the point $(x_1^{*},x_2^{*})=(0,0)$ with ${\lambda }_1^{*}=1$ and ${\lambda }_2^{*}=0$. Hence, $I\left(x^{*}\right)=\left\{1\right\}$, $A_A=(1,0)$, and $b_A=0$. Moreover,

$$Q=\left(\begin{array}{cc}0& 0\\ 0& 1\end{array}\right)andc=\left(\begin{array}{c}-1\\ 0\end{array}\right).$$

By choosing $h^1={(0,1)}^T$ and $h^2={(1,0)}^T$, the system (41) reduces to the linear system:

$$\mathsf{\Psi }(x,\lambda )=\left(\begin{array}{c}Qx+c+A^T\lambda \\ A_{A}x-b_A\\ {\mathsf{\Lambda }}_N{A}_N{h}^1\end{array}\right)=\left(\begin{array}{c}{\lambda }_1-1\\ x_2+{\lambda }_2\\ x_1\\ {\lambda }_2\end{array}\right).$$

Solving the system $\mathsf{\Psi }(x,\lambda )=0$ yields $(x_1^{*},x_2^{*},{\lambda }_1^{*},{\lambda }_2^{*})=(0,0,1,0)$, as claimed.

Now, let us illustrate the second approach. Specifically, using the formula (45) for the solution of problem (46) with ${\lambda }_A={\lambda }_1$, we obtain:

$$\left(\begin{array}{c}{x}_1^{*}\\ x_2^{*}\\ {\lambda }_1^{*}\end{array}\right)={\left[\begin{array}{ccc}0& 0& 1\\ 0& 1& 0\\ 1& 0& 0\end{array}\right]}^{-1}\left(\begin{array}{c}1\\ 0\\ 0\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ 1\end{array}\right),$$

as claimed.

Example 3. 

Consider the problem:

$$\begin{array}{cc}\underset{x}{\mathrm{minimize}}\hfill & x_1^2+\frac{1}{2}{x}_2^2+x_1{x}_2\hfill \\ \mathrm{subject}\mathrm{to}\hfill & -x_1\le 1,\hfill \\ \hfill & x_2\le 1.\hfill \end{array}$$

(47)

The solution to this problem is the point $(x_1^{*},x_2^{*})=(0,0)$ with ${\lambda }_1^{*}=0$ and ${\lambda }_2^{*}=0$. Hence, $I\left(x^{*}\right)=\varnothing$. Moreover,

$$Q=\left(\begin{array}{cc}2& 1\\ 1& 1\end{array}\right)andc=\left(\begin{array}{c}0\\ 0\end{array}\right).$$

By choosing $h^1={(1,1)}^T$ and $h^2={(0,0)}^T$, the system (41) reduces to the following linear system for problem (47):

$$\mathsf{\Psi }(x,\lambda )=\left(\begin{array}{c}2x_1+x_2-{\lambda }_1\\ x_1+x_2+{\lambda }_2\\ -{\lambda }_1\\ {\lambda }_2\end{array}\right).$$

Solving $\mathsf{\Psi }(x,\lambda )=0$ yields $(x_1^{*},x_2^{*},{\lambda }_1^{*},{\lambda }_2^{*})=(0,0,0,0)$, as claimed.

To illustrate the second approach, we rewrite the exact formula (45) for the solution of problem (47) in the form:

$$\left(\begin{array}{c}{x}_1^{*}\\ x_2^{*}\end{array}\right)={\left[\begin{array}{cc}2& 1\\ 1& 1\end{array}\right]}^{-1}\left(\begin{array}{c}0\\ 0\end{array}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right),$$

(48)

as claimed.

5.1.4. Third Approach to Solving the QP Problem

In this subsection, we present another approach to solving the QP problem. A formula that we obtain for the solution of the QP problem is also based on the construction of the 2-factor-operator.

First, we replace the inequality constraints in the QP problem with equality constraints of the form:

$$Ax-b+y^2=0,$$

where $y^2={(y_1^2,\dots ,y_m^2)}^T$. We then define the Lagrangian as follows:

$$\tilde{\mathcal{L}}(x,y,\lambda )=\frac{1}{2}{x}^{T}Qx+c^{T}x+\sum _{i=1}^m{\lambda }_i(a_{i}x-b_i+y_i^2).$$

(49)

Introduce the notation:

$$\mathsf{\Lambda }=\mathrm{diag}{\left({\lambda }_j\right)}_{j=1}^m,Y=\mathrm{diag}{\left(y_j\right)}_{j=1}^m,e={(1,1,\dots ,1)}^T.$$

Then the point $(x^{*},y^{*},{\lambda }^{*})$ is a solution of the following system:

$$\tilde{\mathsf{\Phi }}(x,y,\lambda )=\left(\begin{array}{c}Qx+c+A^T\lambda \\ 2\mathsf{\Lambda }Ye\\ Ax-b+y^2\end{array}\right)=0.$$

(50)

The Jacobian matrix of the system (50) is given by:

$${\tilde{\mathsf{\Phi }}}^{\prime }(x,y,\lambda )=\left(\begin{array}{ccc}Q& O_{n\times m}& A^T\\ O_{m\times n}& 2\mathsf{\Lambda }& 2Y\\ A& 2Y& O\end{array}\right).$$

Then with $h={(h_1,h_2,h_3)}^T,h_1\in {\mathbb{R}}^n,h_2,h_3\in {\mathbb{R}}^m$, we obtain

$${\tilde{\mathsf{\Phi }}}^{\prime }(x,y,\lambda )h=\left(\begin{array}{c}Q{h}_1+A^T{h}_3\\ 2\mathsf{\Lambda }{h}_2+2Y{h}_3\\ A{h}_1+2Y{h}_2\end{array}\right).$$

Assuming that LICQ and MSOSC hold, matrix ${\tilde{\mathsf{\Phi }}}^{\prime }(x^{*},y^{*},{\lambda }^{*})$ is singular if and only if the strict complementarity condition does not hold. In other words, the set of indices of the weakly active constraints,

$$I_0\left(x^{*}\right)=\{j=1,\dots ,m\mid {\lambda }_j^{*}=0,y_j^{*}=0\},$$

is not empty.

Let $P_1$ be the matrix of the orthoprojector onto $\mathrm{Im}{\tilde{\mathsf{\Phi }}}^{\prime }(x^{*},y^{*},{\lambda }^{*})$, and $P_2$ be the matrix of the orthoprojector onto ${\left(\mathrm{Im}{\tilde{\mathsf{\Phi }}}^{\prime }(x^{*},y^{*},{\lambda }^{*})\right)}^{\perp }$. Note that $P_1$ will be a projector onto the linear part of the mapping $\mathsf{\Phi }$, while $P_2$ will be a projector onto the quadratic part of $\mathsf{\Phi }$.

Introduce vector $\widehat{h}=({\widehat{h}}_1,{\widehat{h}}_2,{\widehat{h}}_3)$ such that $P_2{\tilde{\mathsf{\Phi }}}^{\prime }(x^{*},y^{*},{\lambda }^{*})\widehat{h}=0$. Then,

$$\left\{\begin{array}{cc}2\mathsf{\Lambda }{\widehat{h}}_2+2Y{\widehat{h}}_3\hfill & =0\hfill \\ A{\widehat{h}}_1+2Y{\widehat{h}}_2\hfill & =0\hfill \end{array}\right.$$

or

$${\left({\widehat{h}}_2\right)}_i=\left\{\begin{array}{cc}k,\hfill & k\in \mathbb{R}\setminus \left\{0\right\},\mathrm{when}{y}_i^{*}=0,\mathrm{and}{\lambda }_i^{*}=0,\hfill \\ 0,\hfill & \mathrm{when}{y}_i^{*}=0,{\lambda }_i^{*}\ne 0,\hfill \\ w,\hfill & w\in \mathbb{R}\setminus \left\{0\right\},\mathrm{when}{y}_i^{*}\ne 0,{\lambda }_i^{*}=0,\hfill \end{array}\right.$$

$${\left({\widehat{h}}_3\right)}_i=\left\{\begin{array}{cc}t,\hfill & t\in \mathbb{R}\setminus \left\{0\right\},\mathrm{when}{y}_i^{*}=0,\mathrm{and}{\lambda }_i^{*}=0,\hfill \\ 0,\hfill & \mathrm{when}{y}_i^{*}\ne 0,{\lambda }_i^{*}=0,\hfill \\ r,\hfill & r\in \mathbb{R}\setminus \left\{0\right\},\mathrm{when}{y}_i^{*}=0,{\lambda }_i^{*}\ne 0,\hfill \end{array}\right.$$

and ${\widehat{h}}_1$ is defined by:

$$A{\widehat{h}}_1+2Y{\widehat{h}}_2=0.$$

(51)

Observe that $P_2{\tilde{\mathsf{\Phi }}}^{\prime }(x^{*},y^{*},{\lambda }^{*})\widehat{h}=0$, i.e., $\widehat{h}\in \mathrm{Ker}{P}_2{\tilde{\mathsf{\Phi }}}^{\prime }(x^{*},y^{*},{\lambda }^{*})$.

Define H as a diagonal matrix with elements in the rows corresponding to the components of the vector ${\widehat{h}}_2$, and K as a diagonal matrix with elements of the vector ${\widehat{h}}_3$, so that:

$$H=\mathrm{diag}\left({\left({\widehat{h}}_2\right)}_i\right),K=\mathrm{diag}\left({\left({\widehat{h}}_3\right)}_i\right),i=1,\dots ,m.$$

Then:

$${\tilde{\mathsf{\Phi }}}^{″}(x,y,\lambda )\widehat{h}=\left(\begin{array}{ccc}O& O_{n\times m}& O\\ O_{m\times n}& 2K& 2H\\ O_{m\times n}& 2H& O\end{array}\right).$$

The 2-factor-operator for the mapping $\tilde{\mathsf{\Phi }}$ is defined as:

$$\tilde{\mathsf{\Psi }}(x,y,\lambda )=P_1\tilde{\mathsf{\Phi }}(x,y,\lambda )+P_2{\tilde{\mathsf{\Phi }}}^{\prime }(x,y,\lambda )\widehat{h}$$

or

$$\tilde{\mathsf{\Psi }}(x,y,\lambda )=\left(\begin{array}{c}Qx+c+A^T\lambda \\ 2\mathsf{\Lambda }{\widehat{h}}_2+2Y{\widehat{h}}_3\\ A{\widehat{h}}_1+2Y{\widehat{h}}_2\end{array}\right).$$

We choose a vector ${\widehat{h}}_1$ according to (51) so that matrix:

$$\tilde{\mathsf{\Psi }}(x^{*},y^{*},{\lambda }^{*})=\left(\begin{array}{ccc}Q& O_{n\times m}& A^T\\ O_{m\times n}& 2K& 2H\\ O_{m\times n}& 2H& O_{n\times m}\end{array}\right)$$

is nonsingular. Then $(x^{*},y^{*},{\lambda }^{*})$ can be determined using the following formula:

$$\left(\begin{array}{c}{x}^{*}\\ y^{*}\\ {\lambda }^{*}\end{array}\right)={\left[\begin{array}{ccc}Q& O_{n\times m}& A^T\\ O_{m\times n}& 2K& 2H\\ O_{m\times n}& 2H& O_{m\times m}\end{array}\right]}^{-1}\left(\begin{array}{c}-c\\ O_m\\ -A{\widehat{h}}_1\end{array}\right).$$

5.2. Identification of the Active Constraints

In this section, we address the issue of identifying the active constraints and propose strategies for numerically identifying the set of active constraints $I\left(x^{*}\right)$.

We begin by considering the mapping $h\left(z\right):{\mathbb{R}}^n\to {\mathbb{R}}^n$, where $h\in C^2\left({\mathbb{R}}^n\right)$. We can also represent h as an n-vector of functions $h_1$, …, $h_n$, such that $h\left(z\right)={(h_1\left(z\right),\cdots ,h_n\left(z\right))}^T$.

Theorem 8. 

Let $h\in C^2({\mathbb{R}}^n\to {\mathbb{R}}^n)$ be 2-regular at the point $z^{*}$, and let $N_{\epsilon }\left(z^{*}\right)$ be a sufficiently small neighborhood of $z^{*}$ in ${\mathbb{R}}^n$. Assume that there exists a function $\eta \left(z\right):N_{\epsilon }\left(z^{*}\right)\to \mathbb{R}$ such that $\eta \left(z^{*}\right)=0$ and for all $z\in N_{\epsilon }\left(z^{*}\right)$, we have:

$$c_1\parallel z-z^{*}{\parallel }^2\le \eta \left(z\right)\le c_2\parallel z-z^{*}\parallel ,$$

(52)

where $c_1,c_2>0$ are independent constants.

Then there exists a sufficiently small δ such that $0<\delta <\epsilon$, and for any $1\le i\le n$ and any point $z\in N_{\delta }\left(z^{*}\right)$, the following holds:

• Either $|h_i{\left(z\right)|>\eta \left(z\right)}^{1/3}$, which implies that $h_i\left(z^{*}\right)\ne 0,$
• Or $|h_i{\left(z\right)|<\eta \left(z\right)}^{1/3}$, which implies that $h_i\left(z^{*}\right)=0.$

Proof. 

The proof is similar to the one in [5]. □

Let:

$$\varrho \left(x\right)=\underset{i=1,\dots ,m}{min}d\left(g_i^{\prime }\left(x\right),\mathrm{span}(g_1^{\prime }\left(x\right),\dots ,g_{i-1}^{\prime }\left(x\right),g_{i+1}^{\prime }\left(x\right),\dots ,g_m^{\prime }\left(x\right)\right),$$

where $d(a,S)$ denotes the distance between a vector a and a set S. It turns out that if we take $\eta \left(x\right)=max\left\{{\parallel g\left(x\right)\parallel }^{1/2},\varrho \left(x\right)\right\}$, and g is 2-regular at $x^{*}$, then inequality (52) holds with $z=x$.

Theorem 8 can be used for the numerical determination of the set of active constraints $I\left(x^{*}\right)$ in the QP problem. To apply Theorem 8, we need to define a function $\eta (·)$ that satisfies the conditions of the theorem. Recall that for QP problem (3), we denote the Lagrange function defined in (32) by $\mathcal{L}(x,\lambda )$.

Under the assumptions of LICQ and MSOSC, the following holds for $x\in N_{\epsilon }\left(x^{*}\right)$ and $\lambda \in N_{\epsilon }\left({\lambda }^{*}\right)$:

$$c_1\parallel x-x^{*}\parallel \le \parallel {\mathcal{L}}_x^{\prime }(x,\lambda )\parallel +\sum _{i=1}^m|min\{{\lambda }_i,b_i-a_{i}x\}|\le c_2(\parallel x-x^{*}\parallel +\parallel \lambda -{\lambda }^{*}\parallel ),$$

where $\epsilon >0$ is sufficiently small (see, for example, [18]). Hence, the required function $\eta (x,\lambda )$ can be defined by:

$$\eta (x,\lambda )=\parallel {\mathcal{L}}_x^{\prime }(x,\lambda )\parallel +\sum _{i=1}^m|min\{{\lambda }_i,b_i-a_{i}x\}|.$$

Then, according to Theorem 8, for every $i=1,\dots ,m$, if:

$$|b_i-a_i{x|\le \eta (x,\lambda )}^{1/2},$$

then it follows that $i\in I\left(x^{*}\right)$.

Moreover, if we introduce the function:

$$\tilde{\eta }(x,\lambda )=\parallel {\left(L_A\right)}_x^{\prime }(x,\lambda )\parallel +\sum _{i=1}^m|a_{i}x-b_i|,$$

where: $L_A(x,\lambda )=\frac{1}{2}{x}^{T}Qx+c^{T}x+{\displaystyle \sum _{i=1}^m}{\lambda }_i(a_{i}x-b_i)$, then $\tilde{\eta }(x,\lambda )$ satisfies the estimate:

$$c_1\parallel (x,\lambda )-(x^{*},{\lambda }^{*})\parallel \le \tilde{\eta }(x,\lambda )\le c_2\parallel (x,\lambda )-(x^{*},{\lambda }^{*})\parallel$$

for $(x,\lambda )\in N_{\epsilon }(x^{*},{\lambda }^{*})$, where $\epsilon >0$ is a sufficiently small number.

Then, for any $i\in I\left(x^{*}\right)$, if:

$$|{\lambda }_i|\le \tilde{\eta }{(x,\lambda )}^{1/2},$$

then $i\in I_0\left(x^{*}\right)=I\left(x^{*}\right)\setminus I_{+}\left(x^{*}\right)$. Here, $I_0\left(x^{*}\right)$ represents the set of constraints that are weakly active, i.e, for which the associated multipliers are equal to zero, while $I_{+}\left(x^{*}\right)$ denotes the set of constraints that are strongly active at the point $x^{*}$, i.e., the associated Lagrange multipliers are positive.

5.3. General Case

Consider the Lagrange function in the form:

$$\mathcal{L}(x,{\lambda }_0,\lambda )={\lambda }_0\left(\frac{1}{2}{x}^{T}Qx+c^{T}x\right)+\sum _{i=1}^m{\lambda }_i(a_{i}x-b_i).$$

In this case, if $x^{*}$ is a solution of problem (3), then there exist multipliers ${\lambda }_0^{*}$ and ${\lambda }^{*}$, not all zeros, such that ${\lambda }_i^{*}\ge 0$, $A{x}^{*}\le b$, and the point $(x^{*},{\lambda }_0^{*},{\lambda }^{*})$ is a solution of the following system:

$${\mathsf{\Phi }}_0(x,{\lambda }_0,\lambda )=\left(\begin{array}{c}{\lambda }_0(Qx+c)+A^T\lambda \\ \mathsf{\Lambda }(Ax-b)\\ {\lambda }_0+{\displaystyle \sum _{i=1}^m}{\lambda }_i-1\end{array}\right)=0,\mathsf{\Lambda }=\mathrm{diag}{\left({\lambda }_i\right)}_{i=1,\dots ,m}.$$

(53)

Introduce the notation:

$$\xi (x,{\lambda }_0,\lambda )=\parallel {\lambda }_0(Qx+c)+\sum _{i=1}^m{\lambda }_i{a}_i^T\parallel +\sum _{i=1}^m\left|{\lambda }_i(a_{i}x-b_i)\right|+{\lambda }_0+\sum _{i=1}^m{\lambda }_i-1.$$

(54)

We are making the following assumption for the rest of the section.

Assumption A1. 

Assume that there exists $C_1>0$ and a sufficiently small $\epsilon >0$ such that for any $(x,{\lambda }_0,\lambda )\in N_{\epsilon }(x^{*},{\lambda }_0^{*},{\lambda }^{*})$, the following holds:

$$\xi (x,{\lambda }_0,\lambda )\ge C_1{\parallel x-x^{*}\parallel }^2.$$

Remark 5. 

It is easy to see that for any $(x,{\lambda }_0,\lambda )\in N_{\epsilon }(x^{*},{\lambda }_0^{*},{\lambda }^{*})$,

$$\xi (x,{\lambda }_0,\lambda )\le C_2\parallel (x,{\lambda }_0,\lambda )-(x^{*},{\lambda }_0^{*},{\lambda }^{*})\parallel ,$$

where $C_2$ is an independent constant.

As follows from Assumption 1 and Theorem 8, for those indices $i=1,\dots ,m$ that satisfy the inequalty:

$$|a_{i}x-b_i|<\xi {(x,{\lambda }_0,\lambda )}^{1/4}$$

we make a conclusion that $i\in I\left(x^{*}\right)$.

We can illustrate Assumption 1 with the following examples, where Assumption 1 holds.

Example 4. 

This example illustrates a choice of the function ξ in a more general setting.

Consider mapping F defined by either:

$$F(x,\lambda )=\left(\begin{array}{c}{x}^2-{\lambda }^2\\ x\lambda \end{array}\right)$$

or

$$F(x,\lambda )=\left(\begin{array}{c}x-\lambda \\ x\lambda \end{array}\right).$$

In each of the two cases, $(x^{*},{\lambda }^{*})=(0,0)$.

Introduce function $\xi$ defined as:

$$\xi (x,\lambda )=\parallel F(x,\lambda )\parallel .$$

It follows that for any $(x,\lambda )\in N_{\epsilon }(x^{*},{\lambda }^{*})$, the inequality:

$$\xi (x,\lambda )\ge {C\parallel x-0\parallel }^2$$

holds, where C is an independent constant.

Example 5. 

Consider the problem:

$$\begin{array}{cc}\underset{x}{\mathrm{minimize}}\hfill & x_2^2-x_1\hfill \\ \mathrm{subject}\mathrm{to}\hfill & 2x_1\le 0,\hfill \\ \hfill & x_1\le 0.\hfill \end{array}.$$

(55)

The solution to this problem is the point $(x_1^{*},x_2^{*})=(0,0)$, so $I\left(x^{*}\right)=\{1,2\}$. Moreover, the system (53) in this example is given by:

$${\mathsf{\Phi }}_0(x,{\lambda }_0,\lambda )=\left(\begin{array}{c}-{\lambda }_0+{\lambda }_1+{\lambda }_2\\ 2{\lambda }_0{x}_2\\ 2{\lambda }_1{x}_1\\ {\lambda }_2{x}_1\\ {\lambda }_0+{\lambda }_1+{\lambda }_2-1\end{array}\right)=0.$$

(56)

We also introduce the function $\xi (x,{\lambda }_0,\lambda )$, which can be defined using Equation (54), but in this case, we define it as $\xi (x,{\lambda }_0,\lambda )=\left|{\mathsf{\Phi }}_0(x,{\lambda }_0,\lambda )\right|$.

Under Assumption 1, we use the function $\xi$ to determine the set $I\left(x^{*}\right)$. We also take into account the fact that the constraints in the problem are linear and the rank of the matrix

$$\left(\begin{array}{cc}2& 0\\ 1& 0\end{array}\right)$$

is 1. This implies that the constraints are linearly dependent. Consequently, we can eliminate, for instance, the second constraint from problem (55) and simplify system (56) to the following one:

$${\mathsf{\Phi }}_0(x,{\lambda }_0,\lambda )=\left(\begin{array}{c}-{\lambda }_0+{\lambda }_1\\ 2{\lambda }_0{x}_2\\ 2{\lambda }_1{x}_1\\ {\lambda }_0+{\lambda }_1-1\end{array}\right)=0.$$

Now, by introducing:

$$P_1=\left(\begin{array}{cccc}1& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 1\end{array}\right),P_2=\left(\begin{array}{cccc}0& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 0\end{array}\right), \mathrm{and} h=\left(\begin{array}{c}0\\ 0\\ 1\\ 1\end{array}\right),$$

we construct the modified 2-factor-system:

$${\tilde{\mathsf{\Phi }}}_0(x,{\lambda }_0,\lambda )=P_1{\mathsf{\Phi }}_0(x,{\lambda }_0,\lambda )+P_2{\mathsf{\Phi }}_0^{\prime }(x,{\lambda }_0,\lambda )h=\left(\begin{array}{c}-{\lambda }_0+{\lambda }_1\\ 2x_2\\ 2x_1\\ {\lambda }_0+{\lambda }_1-1\end{array}\right)=0.$$

This system implies that the solution is $x^{*}=0$.

Now we will demonstrate the application of the approach described in Section 5.1.2 to problem (55). By removing the first constraint, we obtain a regular QP problem with $A_A=(1,0)$. Additionally, in this example,

$$Q=\left(\begin{array}{cc}0& 0\\ 0& 1\end{array}\right),c=\left(\begin{array}{c}-1\\ 0\end{array}\right).$$

Then application of Equation (45) derived in Section 5.1.2 yields:

$$\left(\begin{array}{c}{x}_1^{*}\\ x_2^{*}\\ {\lambda }^{*}\end{array}\right)={\left[\begin{array}{ccc}0& 0& 1\\ 0& 1& 0\\ 1& 0& 0\end{array}\right]}^{-1}\left(\begin{array}{c}1\\ 0\\ 0\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ 1\end{array}\right),$$

as claimed.

There are various directions in which the approach proposed in this paper can be extended. The next example illustrates a degenerate QP problem, in which MSOSC does not hold at the solution. However, an approach proposed in this paper can be applied to find a solution to this problem. Moreover, the solution set is locally not unique.

There are various directions in which the approach proposed in this paper can be extended. The next example illustrates a degenerate QP problem in which MSOSC does not hold at the solution. However, the approach proposed in this paper can still be applied to find a solution to this problem. It is worth noting that the solution set in this case is locally not unique.

Example 6. 

Consider the problem:

$$\begin{array}{cc}\underset{x}{\mathrm{minimize}}\hfill & x_1{x}_2-x_1\hfill \\ \mathrm{subject}\mathrm{to}\hfill & x_1\le 0.\hfill \end{array}.$$

(57)

The solution to this problem is the set of points $X^{*}=\{(0,x_2^{*})\mid x_2^{*}\in \mathbb{R}\}$. We observe that the objective function in this example is satisfied as an equality for any $x^{*}\in X^{*}$. Additionally, the system (53) for this example consists of one linear equation and three quadratic equations:

$${\mathsf{\Phi }}_0(x,{\lambda }_0,\lambda )=\left(\begin{array}{c}{\lambda }_0{x}_2-{\lambda }_0+{\lambda }_1\\ {\lambda }_0{x}_1\\ {\lambda }_1{x}_1\\ {\lambda }_0+{\lambda }_1-1\end{array}\right)=0.$$

Denote the projection of the point x onto the set $X^{*}$ by $P_{X^{*}}\left(x\right)$. Also, define the notation $\xi (x,{\lambda }_0,\lambda )=\parallel {\mathsf{\Phi }}_0(x,{\lambda }_0,\lambda )\parallel$. For any point $(x,{\lambda }_0,\lambda )\in N_{\epsilon }(x^{*},{\lambda }_0^{*},{\lambda }^{*})$, we have the inequality:

$$\xi (x,{\lambda }_0,\lambda )\ge \alpha \parallel x-P_{X^{*}}\left(x\right)\parallel ,$$

where $\alpha >0$ and $\epsilon >0$ is sufficiently small.

Consider, for example, the point $x^{*}={(0,1)}^T$.

In problem (57), we replace the inequality $x_1\le 0$ with the equation $x_1+y^2=0$, where $y\in \mathbb{R}$. We then introduce the Lagrange function in the form of (49) as follows:

$$L(x,y,\lambda )=(x_1{x}_2-x_1)+\lambda (x_1+y^2).$$

If ${\lambda }^{*}$ is a Lagrange multiplier corresponding to the solution $(x_1^{*},x_2^{*},y^{*})=(0,x_2^{*},0)$, then the point $(x_1^{*},x_2^{*},y^{*},{\lambda }^{*})$ is a solution of the following system:

$$\mathsf{\Phi }(x_1,x_2,y,\lambda )=\left(\begin{array}{c}{x}_2-1+\lambda \\ x_1\\ 2\lambda y\\ x_1+y^2\end{array}\right)=0.$$

(58)

The Jacobian matrix of this system is given by:

$${\mathsf{\Phi }}^{\prime }(x_1,x_2,y,\lambda )=\left(\begin{array}{cccc}0& 1& 0& 1\\ 1& 0& 0& 0\\ 0& 0& 2\lambda & 2y\\ 1& 0& 2y& 0\end{array}\right).$$

This Jacobian matrix becomes singular at $(x_1,x_2,y,\lambda )=(0,x_2,0,\lambda )$. To overcome this singularity, we can apply the approach described in the paper. Specifically, we notice that ${\mathsf{\Phi }}^{\prime }(x_1^{*},x_2^{*},y^{*},{\lambda }^{*})h=0$ for $h=(0,1,1,-1)$. Moreover, the point $(x_1^{*},x_2^{*},y^{*},{\lambda }^{*})=(0,1,0,0)$ is one of the solutions of the system defined in (58), corresponding to a solution of the QP problem (57).

Additionally,

$${\mathsf{\Phi }}^{\prime }(x_1,x_2,y,\lambda )\left(\begin{array}{c}0\\ 1\\ 1\\ -1\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ 2\lambda -2y\\ 2y\end{array}\right),{\mathsf{\Phi }}^{″}(x_1,x_2,y,\lambda )\left(\begin{array}{c}0\\ 1\\ 1\\ -1\end{array}\right)=\left(\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& -2& 2\\ 0& 0& 2& 0\end{array}\right).$$

The 2-factor-operator of $\mathsf{\Phi }(x_1^{*},x_2^{*},y^{*},{\lambda }^{*})$ with respect to the vector $h=(0,1,1,-1)$, which is defined similarly to the operator in Equation (11), is given by:

$${\mathsf{\Phi }}^{\prime }(x_1^{*},x_2^{*},y^{*},{\lambda }^{*})+{\mathsf{\Phi }}^{″}(x_1^{*},x_2^{*},y^{*},{\lambda }^{*})h=\left(\begin{array}{cccc}0& 1& 0& 1\\ 1& 0& 0& 0\\ 0& 0& -2& 2\\ 1& 0& 2& 0\end{array}\right).$$

Note that the 2-factor-operator is nonsingular and the system:

$$\mathsf{\Phi }(x_1,x_2,y,\lambda )+{\mathsf{\Phi }}^{\prime }(x_1,x_2,y,\lambda )h=\left(\begin{array}{c}{x}_2-1+\lambda \\ x_1\\ 2\lambda y+2\lambda -2y\\ x_1+y^2+2y\end{array}\right)=0$$

has the point $(x_1^{*},x_2^{*},y^{*},{\lambda }^{*})=(0,1,0,0)$ as its regular solution.

6. Conclusions

The paper focused on applying the p-regularity theory to nonlinear equations with quadratic mappings and quadratic programming (QP) problems. The first part of the paper used the special structure of the nonlinear equation and the construction of a 2-factor operator to derive a formula for the solution of the equation. In the second part, the QP problem was reduced to a system of linear equations using a 2-factor-operator. The solution of the system is a local minimizer of the QP problem with a corresponding Lagrange multiplier. The formula for the solution of the linear system was given. The paper also described a procedure for identifying the active constraints, which was used in constructing the linear system.

The paper primarily focuses on the case where the matrix $F^{\prime }\left(x^{*}\right)$ is degenerate at the solution of the nonlinear equation $F\left(x\right)=0$. However, the matrix $F^{\prime }\left(x^{*}\right)$ does not need to be degenerate. While we do not explicitly address the identification of degeneracy at a solution point, it is possible to determine the degeneracy of the matrix $F^{\prime }\left(x^{*}\right)$ by examining the behavior of the mapping F in a small neighborhood of the solution $x^{*}$. Specifically, a function ${\nu }_p\left(x\right)$ can be defined, such that:

$$c_1\parallel x-x^{*}{\parallel }^p\le {\nu }_p\left(x\right)\le c_2\parallel x-x^{*}\parallel$$

for some natural number p and constants $c_1$ and $c_2$. Based on the conclusion about the degeneracy of the matrix $F^{\prime }\left(x^{*}\right)$, an appropriate method can be chosen to solve the system of equations $F\left(x\right)=0$, as stated in the following theorem.

Theorem 9. 

Let $F\in {\mathcal{C}}^p({\mathbb{R}}^n,{\mathbb{R}}^n)$ be such that $F\left(x^{*}\right)=0$, and let there exist $x\in N_{\epsilon }\left(x^{*}\right)$, where $\epsilon >0$ is sufficiently small. Then we have the following two cases:

• In the first case, for all $i\in \{1,2,\dots ,n\}$, we have:

  $$d\left(f_i^{\prime }\left(x\right),\mathrm{span}\left(f_1^{\prime }\left(x\right),\dots ,f_{i-1}^{\prime }\left(x\right),f_{i+1}^{\prime }\left(x\right),\dots ,f_m^{\prime }\left(x\right)\right)\right)>{\nu }_p{\left(x\right)}^{1/p+1}.$$
  In this case, $\det F^{\prime }\left(x^{*}\right)\ne 0$, indicating that F is not degenerate at $x^{*}$.
• In the second case, there exists an index $i\in \{1,2,\dots ,n\}$ such that:

  $$d\left(f_i^{\prime }\left(x\right),\mathrm{span}\left(f_1^{\prime }\left(x\right),\dots ,f_{i-1}^{\prime }\left(x\right),f_{i+1}^{\prime }\left(x\right),\dots ,f_m^{\prime }\left(x\right)\right)\right)<{\nu }_p{\left(x\right)}^{1/p+1}.$$
  In this case, $\det F^{\prime }\left(x^{*}\right)=0$, indicating that F is degenerate at $x^{*}$.

Certainly, the construction of the function ${\nu }_p\left(x\right)$ is an important consideration. One approach to constructing such a function is provided in the following lemma, specifically for the case of $p=2$.

Lemma 3. 

Let $F\in {\mathcal{C}}^2({\mathbb{R}}^n,{\mathbb{R}}^n)$ and assume that either ${\left(F^{\prime }\left(x^{*}\right)\right)}^{-1}$ exists, or for any $h\in Ker{F}^{\prime }\left(x^{*}\right)$, there exists ${\left(F^{″}\left(x^{*}\right)h\right)}^{-1}$ with $\parallel h\parallel =1$. Then, there exists a sufficiently small $\epsilon >0$ such that the following inequality holds for all $x\in N_{\epsilon }\left(x^{*}\right)$:

$$\parallel F\left(x\right)\parallel \ge C\parallel x-x^{*}{\parallel }^2,$$

where C is a positive constant.

Based on this lemma, one can choose the function ${\nu }_2\left(x\right)=\parallel F\left(x\right)\parallel$.

It is worth noting that the proposed approach also covers the case where the system of equations consists of both linear and quadratic equations. Moreover, the approach can be extended to solve multilinear equations with polynomials of degree p, given by the equation:

$$F\left(x\right)=Q_p{\left[x\right]}^p+Q_{p-1}{\left[x\right]}^{p-1}+\dots +Q_1\left[x\right]+Q_0=0,$$

where $Q_k{\left[x\right]}^k$ is k-multilinear mapping for $k=1,\dots ,p$. Additionally, polynomial programming problems can be formulated as follows:

$$\begin{array}{cc}\mathrm{minimize}\hfill & f\left(x\right)\hfill \\ \mathrm{subject}\mathrm{to}\hfill & f_i\left(x\right)\le 0,i=1,\dots ,m,\hfill \end{array}$$

where $f_i\left(x\right)$ are polynomial mappings.

There are various possible directions for future research work, based on the results obtained in this paper. While the focus of the current work was on obtaining exact formulas for the solutions of nonlinear equations with quadratic mappings and quadratic programming problems, it would be interesting to generalize the proposed approaches to other classes of problems, including systems of equations with both linear and quadratic mappings. Another direction would be an extension of presented methods to polynomial equations and polynomial programming problems. Another direction of future research could be focusing on numerical studies and the implementation of the methods described in the paper.