An Improved Three-Term Conjugate Gradient Algorithm for Constrained Nonlinear Equations under Non-Lipschitz Conditions and Its Applications

Abstract

This paper proposes an improved three-term conjugate gradient algorithm designed to solve nonlinear equations with convex constraints. The key features of the proposed algorithm are as follows: (i) It only requires that nonlinear equations have continuous and monotone properties; (ii) The designed search direction inherently ensures sufficient descent and trust-region properties, eliminating the need for line search formulas; (iii) Global convergence is established without the necessity of the Lipschitz continuity condition. Benchmark problem numerical results illustrate the proposed algorithm’s effectiveness and competitiveness relative to other three-term algorithms. Additionally, the algorithm is extended to effectively address the image denoising problem.

1. Introduction

Consider the following constrained nonlinear monotone equations of the form:

$$E\left(x\right)=0,x\in \mathbb{E},$$

(1)

where $E:{\mathbb{R}}^n\to {\mathbb{R}}^n$ is a monotonic and continuous mapping, and $\mathbb{E}\subseteq {\mathbb{R}}^n$ is a convex set. The monotonic property of the mapping is defined as

$$\langle E\left(x\right)-E\left(y\right),x-y\rangle \ge 0,\forall x,y\in {\mathbb{R}}^n.$$

(2)

Numerous practical and theoretical problems can be transformed into nonlinear equations, such as those arising from nonlinear mathematical physics [1,2], compressed sensing [3,4], economic equilibrium [5], and optimal power flow control [6]. This broad applicability has driven extensive research into efficient solution methods. Among the various numerical methods that have been developed, derivative-free methods have gained significant attention due to their unique advantages. These methods include spectral gradient methods [7,8,9], two-term conjugate gradient methods [10,11,12,13,14,15], and three-term conjugate gradient methods [16,17,18,19,20]. To be specific, these methods leverage the structure of first-order optimization methods, inheriting the advantages of simplicity and low storage requirements, making them highly effective for solving a wide range of practical problems. However, it has been observed that the convergence properties of the aforementioned derivative-free methods often require mapping to satisfy the Lipschitz continuity condition, which is a stringent theoretical requirement. Hence, our goal in this paper stems from the need to develop a more robust algorithm that operates under the non-Lipschitz continuity condition.

Before presenting our new algorithm, it is essential to review the three-term conjugate gradient method designed for unconstrained optimization problems, specifically those of the form $min\left\{f\left(x\right)\right|x\in {\mathbb{R}}^n\}$. Here, $f:{\mathbb{R}}^n\to \mathbb{R}$ represents a continuously differentiable function, with its gradient at any point $x_k\in {\mathbb{R}}^n$ denoted by $g_k:=\nabla f\left(x_k\right)$. The iterative formula for the three-term conjugate gradient method can be formulated as follows:

$$x_{k+1}=x_k+{\alpha }_k{d}_k,d_k=-g_k+{\tilde{\beta }}_k{d}_{k-1}+{\tilde{\theta }}_k{y}_{k-1},k\ge 1,d_0=-g_0,$$

where ${\alpha }_k$ is the step length determined by a specific line search formula, ${\tilde{\beta }}_k$ and ${\tilde{\theta }}_k$ are scalar parameters, and $y_{k-1}=g_k-g_{k-1}$. The choice of ${\tilde{\beta }}_k$ and ${\tilde{\theta }}_k$ is critical, as different values of these parameters lead to different variants of the three-term conjugate gradient method [21,22,23]. Recently, leveraging the memoryless BFGS approach, Li [24] developed a three-term Hestense–Stiefel (HS)-type conjugate gradient for unconstrained optimization problems. This method’s search direction closely approximates that of the memoryless BFGS method, offering improved performance and robustness. Additionally, Li [25] introduced a three-term Polak–Ribière–Polyak (PRP)-type conjugate gradient method, which modified the search direction by replacing $\langle d_{k-1},y_{k-1}\rangle$ with $\parallel g_{k-1}{\parallel }^2$, thereby enhancing the efficiency in solving optimization problems. Furthermore, through comprehensive analysis [24,25], Li [26] developed a family of three-term conjugate gradient methods for unconstrained optimization problems. A notable feature of these methods is that their search directions consistently satisfy the sufficient descent property, ensuring reliable and effective convergence. Hence, our goal for this paper was to extend and modify these methods for solving nonlinear monotone equations with constraints.

Drawing inspiration from three-term conjugate gradient methods [24,25,26] and the projection technique, our goal was to extend these methods and propose an improved three-term conjugate gradient projection algorithm to solve the problem (1) without requiring the Lipschitz continuity condition. The advantages of our proposed algorithm are multifaceted, addressing several key challenges in solving nonlinear equations with convex constraints: minimal requirements, eliminates the need for line search formulas, global convergence without Lipschitz continuity, effective and competitive performance, and extension to image denoising. The remainder of this paper is structured as follows: In Section 2, we detail the process of the proposed algorithm. Section 3 is dedicated to establishing the convergence analysis of the proposed algorithm. Section 4 and Section 5 present numerical experiments for nonlinear monotone equations with convex constraints and the image denoising problem, respectively. Finally, the conclusions are given in Section 6. Throughout the paper, the symbols $\parallel ·\parallel$ and $\langle ·,·\rangle$ denote the Euclidean norm and the product of two vectors. For convenience, we abbreviate $E\left(x_k\right)$ to $E_k$.

2. Algorithm

In this section, we detail the formulation of our proposed algorithm, outlining its mathematical foundation and the derivation of key parameters. We start by defining the search direction and associated parameters that ensure efficiency and robustness. In addition, we provide a step-by-step description of the algorithm and discuss the theoretical underpinnings of the designed search direction.

To facilitate our formulation, we define several key parameters as follows: We introduce a notation ${\tilde{y}}_{k-1}$, which is given by [27]

$${\tilde{y}}_{k-1}=y_{k-1}+v_{k-1}{\parallel E_{k-1}\parallel }^{a_1}{d}_{k-1},$$

where $v_{k-1}=a_2+max\{0,-{\displaystyle \frac{\langle d_{k-1},y_{k-1}\rangle }{\parallel d_{k-1}{\parallel }^2}}\}{\parallel E_{k-1}\parallel }^{-a_1}$ and $y_{k-1}=E_k-E_{k-1}$ with $a_1,a_2>0$. These parameters play a crucial role in the formulation of our proposed search direction. After making a careful modification, we propose the following search direction:

$$d_k=\left\{\begin{array}{cc}-E_0,& k=0,\\ -E_k+{\beta }_k{d}_{k-1}+{\theta }_k{\tilde{y}}_{k-1}& k\ge 1.\end{array}\right.$$

(3)

Here, the coefficients ${\beta }_k$ and ${\theta }_k$ are defined by the following expressions:

$${\beta }_k={\displaystyle \frac{\langle E_k,{\tilde{y}}_{k-1}\rangle }{{\varpi }_k}}-{\displaystyle \frac{\parallel {\tilde{y}}_{k-1}{\parallel }^2\langle E_k,d_{k-1}\rangle }{{\varpi }_k^2}}$$

and

$${\theta }_k={\displaystyle \frac{{\delta }_k\langle E_k,d_{k-1}\rangle }{{\varpi }_k}},$$

where ${\varpi }_k=b_1{\left(\parallel d_{k-1}\parallel +\parallel {\tilde{y}}_{k-1}\parallel \right)}^2+b_{2}max\left\{\parallel E_{k-1}{\parallel }^2,\langle d_{k-1},{\tilde{y}}_{k-1}\rangle \right\}$ with $0\le {\delta }_k\le \overline{\delta }<1$ and $b_1,b_2>0$. Note that the inclusion of ${\varpi }_k$ can be mathematically justified by its role in ensuring the sufficient descent property and trust-region characteristics. These properties are essential for the global convergence of the algorithm.

Before detailing our algorithm, it is essential to define the projection operator, which ensures the feasibility of our solutions. The projection operator is defined as follows:

$$T_{\mathbb{E}}\left[x\right]=argmin\left\{\parallel x-y\parallel |y\in \mathbb{E}\right\},x\in {\mathbb{R}}^n.$$

Projecting x onto the closed convex set $\mathbb{E}$ guarantees that the subsequent iterative point determined by our algorithm remains within the set $\mathbb{E}$. Additionally, this operator possesses a well-known non-expansive property, which can be expressed as

$$\parallel T_{\mathbb{E}}\left[x\right]-T_{\mathbb{E}}\left[y\right]\parallel \le \parallel x-y\parallel ,\forall x,y\in {\mathbb{R}}^n.$$

(4)

Now, we illustrate the steps of our algorithm designed to efficiently solve nonlinear monotone equations subject to convex constraints. For convenience, Algorithm 1 is referred to as Algorithm ITTCG.

Algorithm 1 Improved Three-Term Conjugate Gradient Algorithm

Step  0. Choose $\sigma ,\rho \in (0,1)$, $\xi \in (0,2)$, $a_1,a_2,b_1,b_2,\epsilon >0$, $\overline{\delta }\in (0,1)$, and an initial point $x_0\in {\mathbb{R}}^n$. Set $k:=0$. Step  1. Set $d_k=-E_k$. Step  2. Set the trial point $z_k=x_k+{\alpha }_k{d}_k$, where the step length $\alpha =max\left\{{\rho }^i\right|i=0,1,\dots ,\}$ satisfies $$-\langle E\left(z_k\right),d_k\rangle \ge \sigma {\alpha }_k\parallel E\left(z_k\right)\parallel \parallel d_k{\parallel }^2.$$ (5) Step  3. If $z_k\in \mathbb{E}$ and $\parallel E\left(z_k\right)\parallel \le \epsilon$, $x_{k+1}:=z_k$ and stop. Otherwise, continue to Step 4. Step  4. Compute the next iterative point as $$x_{k+1}=T_{\mathbb{E}}\left[x_k-\xi {\tau }_{k}E\left(z_k\right)\right],{\tau }_k={\displaystyle \frac{\langle E\left(z_k\right),x_k-z_k\rangle }{\left|\right|E\left(z_k\right){\left|\right|}^2}}.$$ Step  5. If $\parallel E\left(x_{k+1}\right)\parallel \le \epsilon$, stop. Otherwise, compute the search direction $d_{k+1}$ by (3). Step  6. Set $k:=k+1$ and go to Step 2.

Remark 1. 

Based on the definitions of ${\tilde{y}}_{k-1}$ and $v_{k-1}$, we can derive the following expression:

$$\begin{array}{ccc}\langle d_{k-1},{\tilde{y}}_{k-1}\rangle & =\hfill & \langle d_{k-1},y_{k-1}\rangle +v_{k-1}\parallel E_{k-1}{\parallel }^{a_1}{\parallel d_{k-1}\parallel }^2\hfill \\ & \ge \hfill & \langle d_{k-1},y_{k-1}\rangle +a_2\parallel E_{k-1}{\parallel }^{a_1}\parallel d_{k-1}{\parallel }^2-{\displaystyle \frac{\langle d_{k-1},y_{k-1}\rangle }{\parallel d_{k-1}{\parallel }^2}}\parallel E_{k-1}{\parallel }^{-a_1}\parallel E_{k-1}{\parallel }^{a_1}{\parallel d_{k-1}\parallel }^2\hfill \\ & =\hfill & a_2\parallel E_{k-1}{\parallel }^{a_1}{\parallel d_{k-1}\parallel }^2>0.\hfill \end{array}$$

This derivation shows that $\langle d_{k-1},{\tilde{y}}_{k-1}\rangle$ is always positive. Consequently, this implies that the definitions of ${\beta }_k$ and ${\theta }_k$ are valid and feasible within the context of our algorithm.

The following lemma indicates that the search direction determined by Algorithm ITTCG meets both the sufficient descent and trust-region properties. These properties are crucial for establishing the global convergence of Algorithm ITTCG.

Lemma 1. 

Let the sequences $\left\{d_k\right\}$ and $\left\{E_k\right\}$ be determined by Algorithm ITTCG. Then, we have the following results:

$$\langle E_k,d_k\rangle \le -c_1{\parallel E_k\parallel }^2$$

(6)

and

$$c_1\parallel E_k\parallel \le \parallel d_k\parallel \le c_2\parallel E_k\parallel ,$$

(7)

where $c_1=1-{\displaystyle \frac{{\left(1+\overline{\delta }\right)}^2}{4}}$ and $c_2=1+{\displaystyle \frac{1+\overline{\delta }}{4b_1}}+{\displaystyle \frac{1}{16b_2^2}}$.

Proof. 

(i) We will show that (6) holds. For $k=0$, we have $\langle E_0,d_0\rangle =-\parallel E_0{\parallel }^2\le -c_1{\parallel E_0\parallel }^2$. For $k\ge 1$, using the search direction defined in (3), we obtain

$$\begin{array}{ccc}\langle E_k,d_k\rangle & =\hfill & -\parallel E_k{\parallel }^2+{\beta }_k\langle E_k,d_{k-1}\rangle +{\theta }_k\langle E_k,{\tilde{y}}_{k-1}\rangle \hfill \\ & =\hfill & -\parallel E_k{\parallel }^2+{\displaystyle \frac{\langle E_k,{\tilde{y}}_{k-1}\rangle \langle E_k,d_{k-1}\rangle }{{\varpi }_k}}-{\displaystyle \frac{\parallel {\tilde{y}}_{k-1}{\parallel }^2{\langle E_k,d_{k-1}\rangle }^2}{{\varpi }_k^2}}+{\displaystyle \frac{{\delta }_k\langle E_k,d_{k-1}\rangle \langle E_k,{\tilde{y}}_{k-1}\rangle }{{\varpi }_k}}\hfill \\ & =\hfill & -\parallel E_k{\parallel }^2+(1+{\delta }_k){\displaystyle \frac{\langle E_k,d_{k-1}\rangle \langle E_k,{\tilde{y}}_{k-1}\rangle }{{\varpi }_k}}-{\displaystyle \frac{\parallel {\tilde{y}}_{k-1}{\parallel }^2{\langle E_k,d_{k-1}\rangle }^2}{{\varpi }_k^2}}.\hfill \end{array}$$

(8)

In addition, using the inequality $2\langle e_k,l_k\rangle \le \parallel e_k{\parallel }^2+{\parallel l_k\parallel }^2$ with $e_k={\displaystyle \frac{1+{\delta }_k}{2}}{E}_k$ and $l_k={\displaystyle \frac{\langle E_k,d_{k-1}\rangle }{{\varpi }_k}}{\tilde{y}}_{k-1}$, we obtain

$$(1+{\delta }_k){\displaystyle \frac{\langle E_k,d_{k-1}\rangle \langle E_k,{\tilde{y}}_{k-1}\rangle }{{\varpi }_k}}\le {\displaystyle \frac{{\left(1+{\delta }_k\right)}^2}{4}}\parallel E_k{\parallel }^2+{\displaystyle \frac{{\langle E_k,d_{k-1}\rangle }^2}{{\varpi }_k^2}}{\parallel {\tilde{y}}_{k-1}\parallel }^2.$$

(9)

Substituting (9) into (8), we have

$$\langle E_k,d_k\rangle \le -\parallel E_k{\parallel }^2+{\displaystyle \frac{{\left(1+{\delta }_k\right)}^2}{4}}\parallel E_k{\parallel }^2\le -\left(1-{\displaystyle \frac{{\left(1+\overline{\delta }\right)}^2}{4}}\right){\parallel E_k\parallel }^2.$$

(ii) We will show that (7) holds. For $k=0$, we have $c_1\parallel E_0\parallel \le \parallel d_0\parallel =\parallel E_0\parallel \le c_2\parallel E_0\parallel$. For $k\ge 1$, from the definition of ${\varpi }_k$ and using the inequality ${(e-l)}^2=e^2-2el+l^2={(e+l)}^2-4el\ge 0$, we obtain

$${\varpi }_k\ge 4b_1\parallel d_{k-1}\parallel \parallel {\tilde{y}}_{k-1}\parallel .$$

Using this relation and the definitions of ${\beta }_k$ and ${\theta }_k$, we obtain

$$|{\beta }_k|\le {\displaystyle \frac{\parallel E_k\parallel \parallel {\tilde{y}}_{k-1}\parallel }{4b_1\parallel d_{k-1}\parallel \parallel {\tilde{y}}_{k-1}\parallel }}+{\displaystyle \frac{\parallel {\tilde{y}}_{k-1}{\parallel }^2\parallel E_k\parallel \parallel d_{k-1}\parallel }{{\left(4b_1\parallel d_{k-1}\parallel \parallel {\tilde{y}}_{k-1}\parallel \right)}^2}}\le \left({\displaystyle \frac{1}{4b_1}}+{\displaystyle \frac{1}{16b_1^2}}\right){\displaystyle \frac{\parallel E_k\parallel }{\parallel d_{k-1}\parallel }}$$

and

$${\theta }_k\le {\displaystyle \frac{{\delta }_k\langle E_k,d_{k-1}\rangle }{4b_1\parallel d_{k-1}\parallel \parallel {\tilde{y}}_{k-1}\parallel }}\le {\displaystyle \frac{\overline{\delta }\parallel E_k\parallel \parallel d_{k-1}\parallel }{4b_1\parallel d_{k-1}\parallel \parallel {\tilde{y}}_{k-1}\parallel }}\le {\displaystyle \frac{\overline{\delta }\parallel E_k\parallel }{4b_1\parallel {\tilde{y}}_{k-1}\parallel }}.$$

Combining these inequalities with the definition of $d_k$, we obtain

$$\begin{array}{ccc}\parallel d_k\parallel & \le \hfill & \parallel E_k\parallel +\left({\displaystyle \frac{1}{4b_1}}+{\displaystyle \frac{1}{16b_1^2}}\right){\displaystyle \frac{\parallel E_k\parallel }{\parallel d_{k-1}\parallel }}\parallel d_{k-1}\parallel +{\displaystyle \frac{\overline{\delta }\parallel E_k\parallel }{4b_1\parallel {\tilde{y}}_{k-1}\parallel }}\parallel {\tilde{y}}_{k-1}\parallel \hfill \\ & \le \hfill & \left(1+{\displaystyle \frac{1+\overline{\delta }}{4b_1}}+{\displaystyle \frac{1}{16b_2^2}}\right)\parallel E_k\parallel .\hfill \end{array}$$

Additionally, together with (6), we have

$$-\parallel E_k\parallel \parallel d_k\parallel \le \langle E_k,d_k\rangle \le -c_1{\parallel E_k\parallel }^2,$$

which implies that $\parallel d_k\parallel \ge c_1\parallel E_k\parallel$. □

3. Convergence Analysis

In this section, we analyze the global convergence of the proposed algorithm without assuming the Lipschitz continuity condition. We assume that $E\left(x\right)\ne 0$ for any $x\notin {\mathbb{E}}_{*}$, where ${\mathbb{E}}_{*}$ represents the solution set of problem (1). If $E\left(x\right)=0$ for some $x\in {\mathbb{E}}_{*}$, this indicates that the solution to problem (1) has already been achieved.

The following lemma indicates that the line search Formula (5) of the proposed algorithm is well-defined.

Lemma 2. 

Let the sequences $\left\{d_k\right\}$ and $\left\{x_k\right\}$ be generated by Algorithm ITTCG. Then, in each iteration, there exists a step length ${\alpha }_k$ that satisfies the line search Formula (5).

Proof. 

We begin by contradiction and assume that there exists $k_0\ge 0$ such that the line search formula (5) does not hold for any non-negative integer i, i.e.,

$$-\langle E(x_{k_0}+{\rho }^i{d}_{k_0}),d_{k_0}\rangle <\sigma {\rho }^i\parallel E(x_{k_0}+{\rho }^i{d}_{k_0})\parallel \parallel d_{k_0}{\parallel }^2.$$

Given the continuity of E and the fact that $0<\rho <1$, we take the limit as $i\to \infty$ and obtain the relation $\langle E\left(x_{k_0}\right),d_{k_0}\rangle \ge 0$. This contradicts with $\langle E\left(x_{k_0}\right),d_{k_0}\rangle \le -c_1{\parallel E\left(x_{k_0}\right)\parallel }^2<0$ from (6). Therefore, there must be a step length ${\alpha }_k$ that satisfies the line search formula. □

The following lemma indicates that the sequence $\left\{x_k\right\}$ generated by Algorithm ITTCG is monotonic with respect to the solution from the set ${\mathbb{E}}_{*}$ of problem (1).

Lemma 3. 

Let the sequences $\left\{x_k\right\}$ and $\left\{z_k\right\}$ be generated by Algorithm ITTCG, then we have

$$\parallel x_{k+1}-x_{*}{\parallel }^2\le \parallel x_k-x_{*}{\parallel }^2-{\sigma }^2(2\xi -{\xi }^2){\parallel x_k-z_k\parallel }^4,x_{*}\in {\mathbb{E}}_{*}.$$

(10)

Moreover, the sequence $\left\{x_k\right\}$ is bounded.

Proof. 

From the inequality (2), we have

$$\begin{array}{ccc}\langle E\left(z_k\right),x_k-x_{*}\rangle & =\hfill & \langle E\left(z_k\right),x_k-z_k\rangle +\langle E\left(z_k\right),z_k-x_{*}\rangle \hfill \\ & \ge \hfill & \langle E\left(z_k\right),x_k-z_k\rangle +\langle E\left(x_{*}\right),z_k-x_{*}\rangle \hfill \\ & =\hfill & \langle E\left(z_k\right),x_k-z_k\rangle \hfill \\ & \ge \hfill & \sigma {\alpha }_k^2\parallel E\left(z_k\right)\parallel \parallel d_k{\parallel }^2,\hfill \end{array}$$

(11)

where the second inequality follows from the definition of $z_k$ and the search line Formula (5). Additionally, using the definition of ${\tau }_k$ and the inequality (11), we have

$${\tau }_k={\displaystyle \frac{\langle E\left(z_k\right),x_k-z_k\rangle }{\parallel E\left(z_k\right){\parallel }^2}}\ge {\displaystyle \frac{\sigma {\alpha }_k^2\parallel E\left(z_k\right)\parallel \parallel d_k{\parallel }^2}{\parallel E\left(z_k\right){\parallel }^2}}={\displaystyle \frac{\sigma {\alpha }_k^2{\parallel d_k\parallel }^2}{\parallel E\left(z_k\right)\parallel }}.$$

(12)

Utilizing the inequalities (4), (11), and (12), we have

$$\begin{array}{ccc}\parallel x_{k+1}-x_{*}{\parallel }^2& =\hfill & \parallel T_{\mathbb{E}}[x_k-\xi {\tau }_{k}E\left(z_k\right)]-T_{\mathbb{E}}\left[x_{*}\right]{\parallel }^2\hfill \\ & \le \hfill & \parallel x_k-\xi {\tau }_{k}E\left(z_k\right)-x_{*}{\parallel }^2\hfill \\ & =\hfill & \parallel x_k-x_{*}{\parallel }^2-2\xi {\tau }_k\langle E\left(z_k\right),x_k-x_{*}\rangle +{\xi }^2{\tau }_k^2{\parallel E\left(z_k\right)\parallel }^2\hfill \\ & \le \hfill & \parallel x_k-x_{*}{\parallel }^2-2\xi {\tau }_k\langle E\left(z_k\right),x_k-z_k\rangle +{\xi }^2{\tau }_k^2{\parallel E\left(z_k\right)\parallel }^2\hfill \\ & =\hfill & \parallel x_k-x_{*}{\parallel }^2-2\xi {\tau }_k^2\parallel E\left(z_k\right){\parallel }^2+{\xi }^2{\tau }_k^2{\parallel E\left(z_k\right)\parallel }^2\hfill \\ & =\hfill & \parallel x_k-x_{*}{\parallel }^2-(2\xi -{\xi }^2){\tau }_k^2{\parallel E\left(z_k\right)\parallel }^2\hfill \\ & \le \hfill & \parallel x_k-x_{*}{\parallel }^2-(2\xi -{\xi }^2){\left({\displaystyle \frac{\sigma {\alpha }_k^2{\parallel d_k\parallel }^2}{\parallel E\left(z_k\right)\parallel }}\right)}^2{\parallel E\left(z_k\right)\parallel }^2\hfill \\ & =\hfill & \parallel x_k-x_{*}{\parallel }^2-(2\xi -{\xi }^2){\sigma }^2{\alpha }_k^4{\parallel d_k\parallel }^4\hfill \\ & =\hfill & \parallel x_k-x_{*}{\parallel }^2-(2\xi -{\xi }^2){\sigma }^2{\parallel x_k-z_k\parallel }^4,\hfill \end{array}$$

(13)

which implies that the sequence $\{\parallel x_k-x_{*}\parallel \}$ is monotonically non-increasing and convergent. Hence, the sequence $\left\{x_k\right\}$ is bounded. □

To be specific, if the sequence $\left\{x_k\right\}$ is finite, then the last iterative point is the solution to problem (1). If the sequence $\left\{x_k\right\}$ is infinite, we assume this to prove the following result:

Theorem 1. 

Let the sequences $\left\{x_k\right\}$, $\left\{z_k\right\}$, $\left\{d_k\right\}$, and $\left\{E_k\right\}$ b generated by Algorithm ITTCG, then we have

$$\underset{k\to +\infty }{lim}inf\parallel E_k\parallel =0.$$

(14)

Proof. 

We begin by contradiction and assume that there exists a constant ${ϵ}_1>0$ such that $\parallel E_k\parallel >{ϵ}_1$ for any $k\ge 0$. This, combined with (7), yields

$$\parallel d_k\parallel \ge c_1\parallel E_k\parallel >c_1{ϵ}_1,\forall k\ge 0.$$

(15)

According to the continuity of E and the boundedness of $\left\{x_k\right\}$, the sequence $\left\{E_k\right\}$ is also bounded. That is, there exists a non-negative constant ${ϵ}_2$ such that $\parallel E_k\parallel \le {ϵ}_2$ for any $k\ge 0$. This, combined with (7), yields

$$\parallel d_k\parallel \le c_2\parallel E_k\parallel \le c_2{ϵ}_2,\forall k\ge 0.$$

(16)

The inequalities (15) and (16) imply that the sequence $\left\{d_k\right\}$ is bounded.

Moreover, from (10), we deduce that

$$\sum _{k=0}^{\infty }{\parallel x_k-z_k\parallel }^4\le {\displaystyle \frac{1}{{\sigma }^2(2\xi -{\xi }^2)}}\sum _{k=0}^{\infty }\left(\parallel x_k-x_{*}{\parallel }^2-\parallel x_{k+1}-x_{*}\parallel \right)\le {\displaystyle \frac{\parallel x_0-x_{*}{\parallel }^2}{{\sigma }^2(2\xi -{\xi }^2)}},$$

which implies that

$$\underset{k\to \infty }{lim}\parallel x_k-z_k\parallel =\underset{k\to \infty }{lim}{\alpha }_k\parallel d_k\parallel =0.$$

Together with the boundedness of the sequence $\left\{d_k\right\}$, it follows that

$$\underset{k\to \infty }{lim}{\alpha }_k=0.$$

(17)

Given the boundedness of the sequences $\left\{x_k\right\}$ and $\left\{d_k\right\}$, there exists two convergent subsequences $\left\{x_{k_n}\right\}$ and $\left\{d_{k_n}\right\}$ such that

$$\underset{n\to \infty ,n\in \mathcal{K}}{lim}{x}_{k_n}=\overline{x},\underset{n\to \infty ,n\in \mathcal{K}}{lim}{d}_{k_n}=\overline{d},$$

where $\mathcal{K}$ is an infinite index set. The inequality (6) yields

$$-\langle E_{k_n},d_{k_n}\rangle \ge c_1{\parallel E_{k_n}\parallel }^2.$$

By allowing $n\to \infty$ in the above inequality, the continuity of E shows that

$$-\langle E\left(\overline{x}\right),\overline{d}\rangle \ge c_1{\parallel E\left(\overline{x}\right)\parallel }^2>c_1{ϵ}_1^2>0.$$

(18)

Next, considering the line search Formula (5), we have

$$-\langle E(x_{k_n}+{\rho }^{-1}{\alpha }_{k_n}{d}_{k_n}),d_{k_n}\rangle <\sigma {\rho }^{-1}{\alpha }_{k_n}\parallel E(x_{k_n}+{\rho }^{-1}{\alpha }_{k_n}{d}_{k_n})\parallel \parallel d_{k_n}{\parallel }^2.$$

By allowing $n\to \infty$ in the above inequality, the continuity of E implies that

$$-\langle E\left(\overline{x}\right),\overline{d}\rangle \le 0,$$

which contradicts with (18). Therefore, the desired result holds. □

4. Numerical Experiments

In this section, we conducted numerical experiments to demonstrate the effectiveness and competitiveness of Algorithm ITTCG. We compared it with two existing three-term algorithms: Algorithm HTTCGP [18] and Algorithm ZYL [28]. All experiments were performed on an Ubuntu 20.04.2 LTS 64 bit operating system, utilizing an Intel(R) Xeon(R) Gold 5115 CPU at 2.40 GHz.

The parameters for Algorithm ITTCG were configured as follows: $\sigma ={10}^{-4}$, $\rho =0.74$, $\xi =1.3$, $a_2=0.001$, $b_1=0.3$, $b_2=1$, $\epsilon ={10}^{-6}$, $\overline{\delta }=0.1$, and ${\tau }_k$ is computed by

$${\tau }_k=min\left\{\overline{\delta },max\left\{0,1-{\displaystyle \frac{\langle y_{k-1},s_{k-1}\rangle }{\parallel y_{k-1}{\parallel }^2}}\right\}\right\}.$$

The parameters for Algorithms HTTCGP and ZYL were set according to their respective references. We selected benchmark problems with dimensions $n=[1000500010,00050,000$$100,000]$. The benchmark problems were formulated as $E\left(x\right)={\left(E_1\left(x\right),E_2\left(x\right),\dots ,E_n\left(x\right)\right)}^{\mathrm{T}}$ with $x={(x_1,x_2,\dots ,x_n)}^{\mathrm{T}}$. For each benchmark problem, we utilized the following initial points: $x_1={(1,1,\dots ,1)}^{\mathrm{T}}$, $x_2={({\displaystyle \frac{1}{3}},{\displaystyle \frac{1}{3^2}},\dots ,{\displaystyle \frac{1}{3^n}})}^{\mathrm{T}}$, $x_3={({\displaystyle \frac{1}{2}},{\displaystyle \frac{1}{2^2}},\dots ,{\displaystyle \frac{1}{2^n}})}^{\mathrm{T}}$, $x_4={(0,{\displaystyle \frac{1}{n}},{\displaystyle \frac{2}{n}},\dots ,{\displaystyle \frac{n-1}{n}})}^{\mathrm{T}}$, $x_5=(1,{\displaystyle \frac{1}{2}},\dots ,{\displaystyle \frac{1}{n}})$, $x_6=({\displaystyle \frac{1}{n}},{\displaystyle \frac{2}{n}},\dots ,{\displaystyle \frac{n}{n}})$, $x_7=(1-{\displaystyle \frac{1}{n}},1-{\displaystyle \frac{2}{n}},\dots ,1-{\displaystyle \frac{n}{n}})$, $x_8=rand(n,1)$. For each benchmark problem, each algorithm was terminated when $\parallel E_k\parallel \le ϵ$ or the number of iterations exceeded 2000.

Problem 1. Set

$$\begin{array}{ccc}\hfill E_1\left(x\right)& =& e^{x_1}-1,\hfill \\ \hfill E_i\left(x\right)& =& e^{x_i}+x_i-1,\mathrm{for}i=2,3,\cdots ,n,\hfill \end{array}$$

and $\mathbb{E}={\mathbb{R}}_{+}^n$.

Problem 2. Set

$$E_i\left(x\right)=e^{x_i}-1,\mathrm{for}i=1,2,\cdots ,n,$$

and $\mathbb{E}={\mathbb{R}}_{+}^n$. Clearly, this problem has a unique solution $x^{*}={(0,0,\cdots ,0)}^{\mathrm{T}}$.

Problem 3. Set

$$\begin{array}{ccc}\hfill E_1\left(x\right)& =& 2x_1+sin\left(x_1\right)-1,\hfill \\ \hfill E_i\left(x\right)& =& 2x_{i-1}+2x_i+sin\left(x_i\right)-1,\mathrm{for}i=2,3,\cdots ,n-1,\hfill \\ \hfill E_n\left(x\right)& =& 2x_n+sin\left(x_n\right)-1,\hfill \end{array}$$

and $\mathbb{E}={\mathbb{R}}_{+}^n$.

Problem 4. Set

$$E_i\left(x\right)={\displaystyle \frac{i}{n}}{e}^{x_i}-1,\mathrm{for}i=1,2,\cdots ,n,$$

and $\mathbb{E}={\mathbb{R}}_{+}^n$.

Problem 5. Set

$$E_i\left(x\right)=2x_i-sin\left(x_i\right),\mathrm{for}i=1,2,\cdots ,n,$$

and $\mathbb{E}=[-2,+\infty )$.

Problem 6. Set

$$E_i\left(x\right)={\left(e^{x_i}\right)}^2+3sin\left(x_i\right)cos\left(x_i\right)-1,\mathrm{for}i=1,2,\cdots ,n,$$

and $\mathbb{E}={\mathbb{R}}_{+}^n$.

Problem 7. Set

$$\begin{array}{ccc}\hfill E_1\left(x\right)& =& x_1-e^{cos\left({\displaystyle \frac{x_1+x_2}{2}}\right)},\hfill \\ \hfill E_i\left(x\right)& =& x_i-e^{cos\left({\displaystyle \frac{x_{i-1}+x_i+x_{i+1}}{i}}\right)},\mathrm{for}i=2,3,\cdots ,n-1,\hfill \\ \hfill E_n\left(x\right)& =& x_n-e^{cos\left({\displaystyle \frac{x_{n-1}+x_n}{n}}\right)},\hfill \end{array}$$

and $\mathbb{E}={\mathbb{R}}_{+}^n$.

Problem 8. Set

$$\begin{array}{ccc}\hfill E_1\left(x\right)& =& x_1+sin\left(x_1\right)-1,\hfill \\ \hfill E_i\left(x\right)& =& -x_{i-1}+2x_i+sin\left(x_i\right)-1,\mathrm{for}i=2,3,\cdots ,n-1,\hfill \\ \hfill E_n\left(x\right)& =& x_n+sin\left(x_n\right)-1,\hfill \end{array}$$

and $\mathbb{E}=\{x\in {\mathbb{R}}^n:x\ge -3\}.$

The numerical results of benchmark problems solved by Algorithms ITTCG, HTTCGP, and ZYL are presented in

Table 1. Numerical results for Problem 1.

Init (n)	ITTCG	HTTCGP	ZYL
	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm
x1(1)	8.30$\times {10}^{-3}$/7/1/0.00$\times {10}^0$	2.89$\times {10}^{-3}$/7/1/0.00$\times {10}^0$	3.77$\times {10}^{-3}$/90/18/9.93$\times {10}^{-7}$
x2(1)	7.84$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	1.11$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	5.27$\times {10}^{-4}$/25/8/1.84$\times {10}^{-7}$
x3(1)	1.53$\times {10}^{-3}$/78/16/5.89$\times {10}^{-7}$	1.56$\times {10}^{-3}$/84/17/4.60$\times {10}^{-7}$	4.51$\times {10}^{-3}$/261/62/6.28$\times {10}^{-7}$
x4(1)	1.16$\times {10}^{-3}$/53/11/3.86$\times {10}^{-7}$	1.44$\times {10}^{-3}$/69/15/2.46$\times {10}^{-7}$	1.84$\times {10}^{-3}$/100/20/5.05$\times {10}^{-7}$
x5(1)	1.93$\times {10}^{-3}$/69/14/6.42$\times {10}^{-7}$	8.06$\times {10}^{-4}$/38/8/6.62$\times {10}^{-7}$	6.76$\times {10}^{-3}$/288/68/8.66$\times {10}^{-7}$
x6(1)	1.49$\times {10}^{-3}$/53/11/3.94$\times {10}^{-7}$	1.52$\times {10}^{-3}$/69/15/3.14$\times {10}^{-7}$	2.03$\times {10}^{-3}$/100/20/5.09$\times {10}^{-7}$
x7(1)	1.35$\times {10}^{-3}$/53/11/3.44$\times {10}^{-7}$	1.78$\times {10}^{-3}$/69/15/1.28$\times {10}^{-7}$	2.96$\times {10}^{-3}$/100/20/5.60$\times {10}^{-7}$
x8(1)	1.28$\times {10}^{-3}$/53/11/8.69$\times {10}^{-7}$	2.57$\times {10}^{-3}$/108/21/9.53$\times {10}^{-7}$	1.95$\times {10}^{-3}$/100/20/4.68$\times {10}^{-7}$
x1(5)	1.35$\times {10}^{-3}$/7/1/0.00$\times {10}^0$	8.77$\times {10}^{-4}$/7/1/0.00$\times {10}^0$	1.23$\times {10}^{-2}$/90/18/8.21$\times {10}^{-7}$
x2(5)	5.94$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	5.96$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	4.38$\times {10}^{-3}$/25/8/1.84$\times {10}^{-7}$
x3(5)	6.18$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	4.95$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	3.91$\times {10}^{-3}$/25/8/2.00$\times {10}^{-7}$
x4(5)	5.79$\times {10}^{-3}$/44/9/3.64$\times {10}^{-7}$	8.06$\times {10}^{-3}$/69/15/2.23$\times {10}^{-7}$	1.18$\times {10}^{-2}$/105/21/4.48$\times {10}^{-7}$
x5(5)	8.99$\times {10}^{-3}$/69/14/6.60$\times {10}^{-7}$	5.43$\times {10}^{-3}$/38/8/6.39$\times {10}^{-7}$	3.46$\times {10}^{-2}$/288/68/8.67$\times {10}^{-7}$
x6(5)	5.24$\times {10}^{-3}$/44/9/3.72$\times {10}^{-7}$	8.65$\times {10}^{-3}$/69/15/4.01$\times {10}^{-7}$	1.04$\times {10}^{-2}$/105/21/4.49$\times {10}^{-7}$
x7(5)	5.48$\times {10}^{-3}$/44/9/3.98$\times {10}^{-7}$	7.85$\times {10}^{-3}$/69/15/3.15$\times {10}^{-7}$	1.16$\times {10}^{-2}$/105/21/4.60$\times {10}^{-7}$
x8(5)	6.42$\times {10}^{-3}$/53/11/5.88$\times {10}^{-7}$	8.85$\times {10}^{-3}$/69/15/9.92$\times {10}^{-9}$	1.16$\times {10}^{-2}$/105/21/4.40$\times {10}^{-7}$
x1(10)	1.28$\times {10}^{-3}$/7/1/0.00$\times {10}^0$	1.25$\times {10}^{-3}$/7/1/0.00$\times {10}^0$	1.50$\times {10}^{-2}$/90/18/8.80$\times {10}^{-7}$
x2(10)	8.37$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	7.26$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	5.07$\times {10}^{-3}$/25/8/1.84$\times {10}^{-7}$
x3(10)	7.53$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	7.21$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	5.80$\times {10}^{-3}$/25/8/2.00$\times {10}^{-7}$
x4(10)	8.52$\times {10}^{-3}$/44/9/1.54$\times {10}^{-7}$	1.28$\times {10}^{-2}$/69/15/3.77$\times {10}^{-7}$	1.82$\times {10}^{-2}$/105/21/6.34$\times {10}^{-7}$
x5(10)	1.33$\times {10}^{-2}$/69/14/6.62$\times {10}^{-7}$	6.92$\times {10}^{-3}$/38/8/6.36$\times {10}^{-7}$	5.57$\times {10}^{-2}$/288/68/8.67$\times {10}^{-7}$
x6(10)	9.67$\times {10}^{-3}$/44/9/1.58$\times {10}^{-7}$	1.39$\times {10}^{-2}$/69/15/5.06$\times {10}^{-7}$	1.91$\times {10}^{-2}$/105/21/6.34$\times {10}^{-7}$
x7(10)	9.01$\times {10}^{-3}$/44/9/1.69$\times {10}^{-7}$	1.37$\times {10}^{-2}$/69/15/4.45$\times {10}^{-7}$	1.92$\times {10}^{-2}$/105/21/6.42$\times {10}^{-7}$
x8(10)	1.15$\times {10}^{-2}$/53/11/2.05$\times {10}^{-8}$	1.39$\times {10}^{-2}$/69/15/2.94$\times {10}^{-8}$	1.83$\times {10}^{-2}$/105/21/6.17$\times {10}^{-7}$
x1(50)	5.84$\times {10}^{-3}$/7/1/0.00$\times {10}^0$	3.71$\times {10}^{-3}$/7/1/0.00$\times {10}^0$	5.84$\times {10}^{-2}$/95/19/5.74$\times {10}^{-7}$
x2(50)	2.75$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	2.56$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	1.56$\times {10}^{-2}$/25/8/1.84$\times {10}^{-7}$
x3(50)	2.46$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	2.31$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	1.60$\times {10}^{-2}$/25/8/2.00$\times {10}^{-7}$
x4(50)	5.49$\times {10}^{-2}$/81/17/5.07$\times {10}^{-7}$	4.94$\times {10}^{-2}$/69/15/9.57$\times {10}^{-7}$	6.51$\times {10}^{-2}$/110/22/5.60$\times {10}^{-7}$
x5(50)	4.53$\times {10}^{-2}$/69/14/6.64$\times {10}^{-7}$	2.60$\times {10}^{-2}$/38/8/6.34$\times {10}^{-7}$	1.84$\times {10}^{-1}$/288/68/8.67$\times {10}^{-7}$
x6(50)	5.33$\times {10}^{-2}$/76/16/8.44$\times {10}^{-7}$	5.10$\times {10}^{-2}$/74/16/6.93$\times {10}^{-8}$	6.71$\times {10}^{-2}$/110/22/5.60$\times {10}^{-7}$
x7(50)	5.21$\times {10}^{-2}$/76/16/4.33$\times {10}^{-7}$	4.64$\times {10}^{-2}$/69/15/9.88$\times {10}^{-7}$	6.74$\times {10}^{-2}$/110/22/5.62$\times {10}^{-7}$
x8(50)	3.59$\times {10}^{-2}$/48/10/0.00$\times {10}^0$	5.82$\times {10}^{-2}$/78/17/7.67$\times {10}^{-7}$	6.84$\times {10}^{-2}$/110/22/5.55$\times {10}^{-7}$
x1(100)	9.61$\times {10}^{-3}$/7/1/0.00$\times {10}^0$	9.45$\times {10}^{-3}$/7/1/0.00$\times {10}^0$	1.29$\times {10}^{-1}$/95/19/7.76$\times {10}^{-7}$
x2(100)	6.76$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	5.55$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	3.12$\times {10}^{-2}$/25/8/1.84$\times {10}^{-7}$
x3(100)	5.52$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	5.36$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	3.02$\times {10}^{-2}$/25/8/2.00$\times {10}^{-7}$
x4(100)	7.97$\times {10}^{-2}$/52/11/0.00$\times {10}^0$	1.12$\times {10}^{-1}$/74/16/6.18$\times {10}^{-8}$	1.42$\times {10}^{-1}$/110/22/7.92$\times {10}^{-7}$
x5(100)	9.69$\times {10}^{-2}$/69/14/6.64$\times {10}^{-7}$	5.44$\times {10}^{-2}$/38/8/6.33$\times {10}^{-7}$	3.61$\times {10}^{-1}$/288/68/8.67$\times {10}^{-7}$
x6(100)	7.86$\times {10}^{-2}$/52/11/0.00$\times {10}^0$	1.10$\times {10}^{-1}$/74/16/8.59$\times {10}^{-8}$	1.40$\times {10}^{-1}$/110/22/7.92$\times {10}^{-7}$
x7(100)	8.00$\times {10}^{-2}$/52/11/0.00$\times {10}^0$	1.15$\times {10}^{-1}$/74/16/7.44$\times {10}^{-8}$	1.82$\times {10}^{-1}$/110/22/7.94$\times {10}^{-7}$
x8(100)	1.02$\times {10}^{-1}$/62/13/5.59$\times {10}^{-7}$	1.25$\times {10}^{-1}$/78/17/9.11$\times {10}^{-7}$	1.43$\times {10}^{-1}$/110/22/7.88$\times {10}^{-7}$

,

Table 2. Numerical results for Problem 2.

Init(n)	ITTCG	HTTCGP	ZYL
	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm
x1(1)	3.05$\times {10}^{-3}$/5/1/0.00$\times {10}^0$	1.26$\times {10}^{-4}$/5/1/0.00$\times {10}^0$	6.58$\times {10}^{-4}$/36/11/1.82$\times {10}^{-7}$
x2(1)	9.14$\times {10}^{-5}$/4/1/0.00$\times {10}^0$	1.24$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	4.13$\times {10}^{-4}$/25/8/1.84$\times {10}^{-7}$
x3(1)	4.95$\times {10}^{-4}$/22/8/9.13$\times {10}^{-7}$	4.36$\times {10}^{-4}$/23/8/1.61$\times {10}^{-7}$	1.46$\times {10}^{-3}$/94/31/5.23$\times {10}^{-7}$
x4(1)	1.06$\times {10}^{-3}$/47/15/2.08$\times {10}^{-7}$	1.23$\times {10}^{-3}$/59/19/9.26$\times {10}^{-7}$	2.11$\times {10}^{-3}$/117/38/7.52$\times {10}^{-7}$
x5(1)	5.70$\times {10}^{-4}$/24/8/6.49$\times {10}^{-8}$	5.69$\times {10}^{-4}$/28/9/6.46$\times {10}^{-8}$	1.43$\times {10}^{-3}$/81/26/9.56$\times {10}^{-7}$
x6(1)	9.49$\times {10}^{-4}$/40/13/5.57$\times {10}^{-7}$	7.91$\times {10}^{-4}$/38/13/8.06$\times {10}^{-9}$	2.14$\times {10}^{-3}$/121/39/4.72$\times {10}^{-7}$
x7(1)	1.06$\times {10}^{-3}$/47/15/2.08$\times {10}^{-7}$	1.15$\times {10}^{-3}$/59/19/9.26$\times {10}^{-7}$	2.06$\times {10}^{-3}$/117/38/7.52$\times {10}^{-7}$
x8(1)	1.03$\times {10}^{-3}$/39/14/1.88$\times {10}^{-7}$	9.06$\times {10}^{-4}$/40/13/4.75$\times {10}^{-7}$	1.42$\times {10}^{-3}$/76/24/8.46$\times {10}^{-7}$
x1(5)	5.79$\times {10}^{-4}$/5/1/0.00$\times {10}^0$	4.40$\times {10}^{-4}$/5/1/0.00$\times {10}^0$	4.36$\times {10}^{-3}$/36/11/4.07$\times {10}^{-7}$
x2(5)	3.91$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	4.16$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	2.82$\times {10}^{-3}$/25/8/1.84$\times {10}^{-7}$
x3(5)	5.10$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	4.63$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	3.11$\times {10}^{-3}$/25/8/2.00$\times {10}^{-7}$
x4(5)	6.11$\times {10}^{-3}$/45/15/3.16$\times {10}^{-7}$	6.60$\times {10}^{-3}$/55/18/1.08$\times {10}^{-12}$	1.27$\times {10}^{-2}$/126/41/8.02$\times {10}^{-7}$
x5(5)	2.69$\times {10}^{-3}$/24/8/9.08$\times {10}^{-8}$	2.99$\times {10}^{-3}$/28/9/8.18$\times {10}^{-8}$	8.06$\times {10}^{-3}$/81/26/9.57$\times {10}^{-7}$
x6(5)	4.58$\times {10}^{-3}$/44/14/5.94$\times {10}^{-7}$	4.47$\times {10}^{-3}$/43/15/6.72$\times {10}^{-8}$	1.00$\times {10}^{-2}$/106/34/9.43$\times {10}^{-7}$
x7(5)	4.87$\times {10}^{-3}$/45/15/3.16$\times {10}^{-7}$	6.44$\times {10}^{-3}$/55/18/1.08$\times {10}^{-12}$	1.11$\times {10}^{-2}$/126/41/8.02$\times {10}^{-7}$
x8(5)	6.10$\times {10}^{-3}$/50/17/7.67$\times {10}^{-8}$	4.00$\times {10}^{-3}$/38/13/7.39$\times {10}^{-7}$	1.15$\times {10}^{-2}$/123/40/6.51$\times {10}^{-7}$
x1(10)	6.70$\times {10}^{-4}$/5/1/0.00$\times {10}^0$	6.54$\times {10}^{-4}$/5/1/0.00$\times {10}^0$	4.92$\times {10}^{-3}$/36/11/5.75$\times {10}^{-7}$
x2(10)	5.04$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	5.16$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	3.03$\times {10}^{-3}$/25/8/1.84$\times {10}^{-7}$
x3(10)	5.41$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	6.07$\times {10}^{-4}$/4/1/0.00$\times {10}^0$	3.99$\times {10}^{-3}$/25/8/2.00$\times {10}^{-7}$
x4(10)	9.40$\times {10}^{-3}$/52/16/7.66$\times {10}^{-7}$	8.07$\times {10}^{-3}$/56/18/5.41$\times {10}^{-10}$	1.80$\times {10}^{-2}$/129/42/6.00$\times {10}^{-7}$
x5(10)	4.74$\times {10}^{-3}$/24/8/9.43$\times {10}^{-8}$	4.19$\times {10}^{-3}$/28/9/8.42$\times {10}^{-8}$	1.18$\times {10}^{-2}$/81/26/9.57$\times {10}^{-7}$
x6(10)	8.57$\times {10}^{-3}$/51/16/8.19$\times {10}^{-7}$	8.58$\times {10}^{-3}$/58/19/2.93$\times {10}^{-7}$	1.59$\times {10}^{-2}$/112/36/4.86$\times {10}^{-7}$
x7(10)	1.06$\times {10}^{-2}$/52/16/7.66$\times {10}^{-7}$	9.62$\times {10}^{-3}$/56/18/5.41$\times {10}^{-10}$	1.94$\times {10}^{-2}$/129/42/6.00$\times {10}^{-7}$
x8(10)	1.30$\times {10}^{-2}$/50/17/5.50$\times {10}^{-7}$	9.49$\times {10}^{-3}$/58/19/1.09$\times {10}^{-7}$	2.12$\times {10}^{-2}$/120/39/8.17$\times {10}^{-7}$
x1(50)	3.13$\times {10}^{-3}$/5/1/0.00$\times {10}^0$	1.94$\times {10}^{-3}$/5/1/0.00$\times {10}^0$	2.00$\times {10}^{-2}$/39/12/2.06$\times {10}^{-7}$
x2(50)	1.69$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	1.39$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	1.03$\times {10}^{-2}$/25/8/1.84$\times {10}^{-7}$
x3(50)	1.71$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	1.64$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	1.12$\times {10}^{-2}$/25/8/2.00$\times {10}^{-7}$
x4(50)	3.51$\times {10}^{-2}$/60/19/4.83$\times {10}^{-7}$	3.23$\times {10}^{-2}$/57/18/4.68$\times {10}^{-14}$	6.65$\times {10}^{-2}$/129/42/9.70$\times {10}^{-7}$
x5(50)	1.48$\times {10}^{-2}$/24/8/9.72$\times {10}^{-8}$	1.68$\times {10}^{-2}$/28/9/8.62$\times {10}^{-8}$	4.04$\times {10}^{-2}$/81/26/9.57$\times {10}^{-7}$
x6(50)	3.64$\times {10}^{-2}$/54/18/1.53$\times {10}^{-7}$	2.29$\times {10}^{-2}$/39/14/3.40$\times {10}^{-7}$	5.72$\times {10}^{-2}$/117/38/5.84$\times {10}^{-7}$
x7(50)	3.77$\times {10}^{-2}$/60/19/4.83$\times {10}^{-7}$	3.44$\times {10}^{-2}$/57/18/4.68$\times {10}^{-14}$	6.29$\times {10}^{-2}$/129/42/9.70$\times {10}^{-7}$
x8(50)	3.82$\times {10}^{-2}$/58/18/2.83$\times {10}^{-7}$	2.89$\times {10}^{-2}$/41/14/4.10$\times {10}^{-7}$	6.26$\times {10}^{-2}$/129/42/8.15$\times {10}^{-7}$
x1(100)	4.75$\times {10}^{-3}$/5/1/0.00$\times {10}^0$	3.72$\times {10}^{-3}$/5/1/0.00$\times {10}^0$	3.23$\times {10}^{-2}$/39/12/2.91$\times {10}^{-7}$
x2(100)	3.49$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	2.62$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	1.60$\times {10}^{-2}$/25/8/1.84$\times {10}^{-7}$
x3(100)	3.94$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	3.03$\times {10}^{-3}$/4/1/0.00$\times {10}^0$	1.60$\times {10}^{-2}$/25/8/2.00$\times {10}^{-7}$
x4(100)	4.69$\times {10}^{-2}$/41/15/3.94$\times {10}^{-8}$	3.74$\times {10}^{-2}$/41/14/3.81$\times {10}^{-7}$	1.06$\times {10}^{-1}$/132/43/8.32$\times {10}^{-7}$
x5(100)	2.63$\times {10}^{-2}$/24/8/9.75$\times {10}^{-8}$	2.77$\times {10}^{-2}$/28/9/8.64$\times {10}^{-8}$	6.34$\times {10}^{-2}$/81/26/9.57$\times {10}^{-7}$
x6(100)	5.50$\times {10}^{-2}$/47/18/3.93$\times {10}^{-7}$	4.14$\times {10}^{-2}$/43/15/4.35$\times {10}^{-8}$	1.04$\times {10}^{-1}$/129/42/5.76$\times {10}^{-7}$
x7(100)	4.78$\times {10}^{-2}$/41/15/3.94$\times {10}^{-8}$	4.06$\times {10}^{-2}$/41/14/3.81$\times {10}^{-7}$	1.10$\times {10}^{-1}$/132/43/8.32$\times {10}^{-7}$
x8(100)	6.30$\times {10}^{-2}$/48/17/3.01$\times {10}^{-7}$	5.47$\times {10}^{-2}$/45/16/4.28$\times {10}^{-15}$	1.07$\times {10}^{-1}$/129/42/7.87$\times {10}^{-7}$

,

Table 3. Numerical results for Problem 3.

Init(n)	ITTCG	HTTCGP	ZYL
	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm
x1(1)	5.36$\times {10}^{-3}$/250/33/5.87$\times {10}^{-7}$	5.72$\times {10}^{-3}$/293/36/9.74$\times {10}^{-7}$	8.26$\times {10}^{-3}$/439/73/9.64$\times {10}^{-7}$
x2(1)	3.93$\times {10}^{-3}$/180/23/7.99$\times {10}^{-7}$	5.81$\times {10}^{-3}$/307/38/8.37$\times {10}^{-7}$	7.07$\times {10}^{-3}$/367/61/9.94$\times {10}^{-7}$
x3(1)	4.19$\times {10}^{-3}$/208/27/8.37$\times {10}^{-7}$	6.63$\times {10}^{-3}$/349/43/8.55$\times {10}^{-7}$	7.49$\times {10}^{-3}$/385/64/8.74$\times {10}^{-7}$
x4(1)	4.51$\times {10}^{-3}$/222/29/6.50$\times {10}^{-7}$	8.57$\times {10}^{-3}$/433/54/9.46$\times {10}^{-7}$	7.66$\times {10}^{-3}$/391/65/9.70$\times {10}^{-7}$
x5(1)	4.73$\times {10}^{-3}$/237/31/6.04$\times {10}^{-7}$	6.64$\times {10}^{-3}$/337/42/8.37$\times {10}^{-7}$	7.42$\times {10}^{-3}$/391/65/9.80$\times {10}^{-7}$
x6(1)	4.61$\times {10}^{-3}$/222/29/6.49$\times {10}^{-7}$	8.58$\times {10}^{-3}$/410/51/9.15$\times {10}^{-7}$	8.01$\times {10}^{-3}$/397/66/9.25$\times {10}^{-7}$
x7(1)	5.08$\times {10}^{-3}$/250/33/5.41$\times {10}^{-7}$	6.35$\times {10}^{-3}$/330/41/8.54$\times {10}^{-7}$	7.97$\times {10}^{-3}$/421/70/9.17$\times {10}^{-7}$
x8(1)	6.45$\times {10}^{-3}$/308/41/5.09$\times {10}^{-7}$	8.47$\times {10}^{-3}$/413/51/9.89$\times {10}^{-7}$	9.79$\times {10}^{-3}$/480/80/9.21$\times {10}^{-7}$
x1(5)	3.76$\times {10}^{-2}$/243/32/6.03$\times {10}^{-7}$	3.64$\times {10}^{-2}$/299/37/8.62$\times {10}^{-7}$	5.50$\times {10}^{-2}$/445/74/8.19$\times {10}^{-7}$
x2(5)	2.20$\times {10}^{-2}$/173/22/8.68$\times {10}^{-7}$	3.98$\times {10}^{-2}$/338/42/1.00$\times {10}^{-6}$	4.94$\times {10}^{-2}$/403/67/9.85$\times {10}^{-7}$
x3(5)	2.43$\times {10}^{-2}$/201/26/8.32$\times {10}^{-7}$	3.75$\times {10}^{-2}$/319/39/9.75$\times {10}^{-7}$	4.98$\times {10}^{-2}$/415/69/9.21$\times {10}^{-7}$
x4(5)	2.67$\times {10}^{-2}$/229/30/8.15$\times {10}^{-7}$	4.77$\times {10}^{-2}$/418/52/3.03$\times {10}^{-7}$	5.21$\times {10}^{-2}$/421/70/7.23$\times {10}^{-7}$
x5(5)	2.62$\times {10}^{-2}$/216/28/5.92$\times {10}^{-7}$	3.86$\times {10}^{-2}$/316/39/7.17$\times {10}^{-7}$	5.17$\times {10}^{-2}$/415/69/8.21$\times {10}^{-7}$
x6(5)	2.92$\times {10}^{-2}$/229/30/8.14$\times {10}^{-7}$	5.12$\times {10}^{-2}$/417/52/9.34$\times {10}^{-7}$	5.52$\times {10}^{-2}$/415/69/8.50$\times {10}^{-7}$
x7(5)	3.48$\times {10}^{-2}$/271/36/7.63$\times {10}^{-7}$	7.09$\times {10}^{-2}$/560/70/9.12$\times {10}^{-7}$	5.42$\times {10}^{-2}$/439/73/7.95$\times {10}^{-7}$
x8(5)	4.21$\times {10}^{-2}$/309/41/5.34$\times {10}^{-7}$	5.09$\times {10}^{-2}$/413/51/5.52$\times {10}^{-7}$	6.69$\times {10}^{-2}$/522/87/9.37$\times {10}^{-7}$
x1(10)	4.56$\times {10}^{-2}$/236/31/6.73$\times {10}^{-7}$	6.37$\times {10}^{-2}$/338/42/3.92$\times {10}^{-7}$	8.78$\times {10}^{-2}$/451/75/9.13$\times {10}^{-7}$
x2(10)	3.39$\times {10}^{-2}$/187/24/9.18$\times {10}^{-7}$	6.63$\times {10}^{-2}$/338/42/9.20$\times {10}^{-7}$	7.80$\times {10}^{-2}$/415/69/9.61$\times {10}^{-7}$
x3(10)	3.83$\times {10}^{-2}$/208/27/9.16$\times {10}^{-7}$	6.19$\times {10}^{-2}$/335/42/6.78$\times {10}^{-7}$	7.91$\times {10}^{-2}$/427/71/9.89$\times {10}^{-7}$
x4(10)	4.54$\times {10}^{-2}$/236/31/7.88$\times {10}^{-7}$	6.21$\times {10}^{-2}$/359/45/4.72$\times {10}^{-7}$	6.89$\times {10}^{-2}$/397/66/6.98$\times {10}^{-7}$
x5(10)	4.12$\times {10}^{-2}$/223/29/5.94$\times {10}^{-7}$	5.37$\times {10}^{-2}$/302/37/5.30$\times {10}^{-7}$	7.13$\times {10}^{-2}$/409/68/9.04$\times {10}^{-7}$
x6(10)	4.33$\times {10}^{-2}$/236/31/7.88$\times {10}^{-7}$	6.24$\times {10}^{-2}$/369/46/3.53$\times {10}^{-7}$	6.95$\times {10}^{-2}$/397/66/7.00$\times {10}^{-7}$
x7(10)	5.05$\times {10}^{-2}$/264/35/8.21$\times {10}^{-7}$	9.77$\times {10}^{-2}$/568/71/5.46$\times {10}^{-7}$	7.02$\times {10}^{-2}$/397/66/9.17$\times {10}^{-7}$
x8(10)	5.65$\times {10}^{-2}$/309/41/7.30$\times {10}^{-7}$	6.72$\times {10}^{-2}$/387/48/8.03$\times {10}^{-7}$	8.77$\times {10}^{-2}$/486/81/7.25$\times {10}^{-7}$
x1(50)	1.75$\times {10}^{-1}$/236/31/7.21$\times {10}^{-7}$	2.01$\times {10}^{-1}$/292/36/6.60$\times {10}^{-7}$	3.00$\times {10}^{-1}$/427/71/9.67$\times {10}^{-7}$
x2(50)	1.31$\times {10}^{-1}$/180/23/9.41$\times {10}^{-7}$	2.50$\times {10}^{-1}$/350/44/9.18$\times {10}^{-7}$	2.97$\times {10}^{-1}$/427/71/9.94$\times {10}^{-7}$
x3(50)	1.44$\times {10}^{-1}$/209/27/4.76$\times {10}^{-7}$	2.80$\times {10}^{-1}$/412/52/8.85$\times {10}^{-7}$	3.02$\times {10}^{-1}$/415/69/7.96$\times {10}^{-7}$
x4(50)	1.60$\times {10}^{-1}$/229/30/8.95$\times {10}^{-7}$	2.65$\times {10}^{-1}$/394/50/8.74$\times {10}^{-7}$	2.90$\times {10}^{-1}$/416/69/9.13$\times {10}^{-7}$
x5(50)	1.62$\times {10}^{-1}$/237/31/6.63$\times {10}^{-7}$	1.95$\times {10}^{-1}$/288/36/9.73$\times {10}^{-7}$	3.12$\times {10}^{-1}$/439/73/8.44$\times {10}^{-7}$
x6(50)	1.52$\times {10}^{-1}$/229/30/8.95$\times {10}^{-7}$	2.64$\times {10}^{-1}$/387/49/8.40$\times {10}^{-7}$	2.90$\times {10}^{-1}$/416/69/9.17$\times {10}^{-7}$
x7(50)	1.80$\times {10}^{-1}$/264/35/8.82$\times {10}^{-7}$	2.63$\times {10}^{-1}$/381/48/7.31$\times {10}^{-7}$	2.79$\times {10}^{-1}$/410/68/9.11$\times {10}^{-7}$
x8(50)	2.24$\times {10}^{-1}$/324/43/6.03$\times {10}^{-7}$	3.16$\times {10}^{-1}$/451/56/7.36$\times {10}^{-7}$	3.41$\times {10}^{-1}$/481/80/9.57$\times {10}^{-7}$
x1(100)	4.19$\times {10}^{-1}$/222/29/8.67$\times {10}^{-7}$	5.07$\times {10}^{-1}$/261/32/7.78$\times {10}^{-7}$	8.75$\times {10}^{-1}$/433/72/8.25$\times {10}^{-7}$
x2(100)	3.24$\times {10}^{-1}$/180/23/9.00$\times {10}^{-7}$	5.38$\times {10}^{-1}$/265/33/7.48$\times {10}^{-7}$	8.16$\times {10}^{-1}$/415/69/8.73$\times {10}^{-7}$
x3(100)	3.84$\times {10}^{-1}$/216/28/4.70$\times {10}^{-7}$	6.33$\times {10}^{-1}$/320/40/4.66$\times {10}^{-7}$	7.70$\times {10}^{-1}$/415/69/9.78$\times {10}^{-7}$
x4(100)	4.77$\times {10}^{-1}$/243/32/9.84$\times {10}^{-7}$	5.49$\times {10}^{-1}$/378/48/8.73$\times {10}^{-7}$	7.55$\times {10}^{-1}$/416/69/7.31$\times {10}^{-7}$
x5(100)	4.32$\times {10}^{-1}$/230/30/7.85$\times {10}^{-7}$	6.88$\times {10}^{-1}$/344/43/9.43$\times {10}^{-7}$	9.14$\times {10}^{-1}$/451/75/9.94$\times {10}^{-7}$
x6(100)	4.56$\times {10}^{-1}$/243/32/9.84$\times {10}^{-7}$	6.98$\times {10}^{-1}$/347/44/9.61$\times {10}^{-7}$	8.11$\times {10}^{-1}$/416/69/7.31$\times {10}^{-7}$
x7(100)	4.67$\times {10}^{-1}$/244/32/5.07$\times {10}^{-7}$	6.55$\times {10}^{-1}$/317/40/6.90$\times {10}^{-7}$	8.56$\times {10}^{-1}$/422/70/9.36$\times {10}^{-7}$
x8(100)	6.35$\times {10}^{-1}$/345/46/8.22$\times {10}^{-7}$	9.66$\times {10}^{-1}$/475/59/7.73$\times {10}^{-7}$	9.91$\times {10}^{-1}$/493/82/8.45$\times {10}^{-7}$

,

Table 4. Numerical results for Problem 4.

Init(n)	ITTCG	HTTCGP	ZYL
	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm
x1(1)	2.21$\times {10}^{-3}$/89/20/2.89$\times {10}^{-7}$	2.83$\times {10}^{-3}$/119/27/8.70$\times {10}^{-8}$	2.78$\times {10}^{-3}$/118/38/8.33$\times {10}^{-7}$
x2(1)	1.57$\times {10}^{-3}$/51/19/1.61$\times {10}^{-7}$	1.56$\times {10}^{-3}$/56/21/9.69$\times {10}^{-7}$	2.28$\times {10}^{-3}$/96/33/6.03$\times {10}^{-7}$
x3(1)	1.23$\times {10}^{-3}$/39/15/8.04$\times {10}^{-7}$	1.53$\times {10}^{-3}$/54/18/3.13$\times {10}^{-7}$	2.20$\times {10}^{-3}$/93/32/7.55$\times {10}^{-7}$
x4(1)	4.91$\times {10}^{-3}$/223/39/4.76$\times {10}^{-7}$	4.59$\times {10}^{-3}$/237/35/7.19$\times {10}^{-7}$	2.80$\times {10}^{-3}$/123/39/8.83$\times {10}^{-7}$
x5(1)	9.47$\times {10}^{-4}$/29/12/4.29$\times {10}^{-7}$	1.40$\times {10}^{-3}$/52/19/6.57$\times {10}^{-8}$	2.37$\times {10}^{-3}$/102/35/8.01$\times {10}^{-7}$
x6(1)	4.47$\times {10}^{-3}$/212/36/8.45$\times {10}^{-7}$	5.37$\times {10}^{-3}$/279/43/1.69$\times {10}^{-7}$	2.77$\times {10}^{-3}$/123/39/8.74$\times {10}^{-7}$
x7(1)	1.12$\times {10}^{-3}$/37/14/1.06$\times {10}^{-7}$	1.41$\times {10}^{-3}$/53/19/8.14$\times {10}^{-7}$	2.05$\times {10}^{-3}$/87/30/7.34$\times {10}^{-7}$
x8(1)	6.41$\times {10}^{-3}$/328/44/6.49$\times {10}^{-7}$	6.04$\times {10}^{-3}$/320/44/7.16$\times {10}^{-7}$	4.16$\times {10}^{-3}$/207/50/6.30$\times {10}^{-7}$
x1(5)	2.23$\times {10}^{-2}$/231/36/1.93$\times {10}^{-7}$	1.77$\times {10}^{-2}$/216/33/5.67$\times {10}^{-7}$	1.50$\times {10}^{-2}$/132/42/7.97$\times {10}^{-7}$
x2(5)	5.03$\times {10}^{-3}$/39/16/7.13$\times {10}^{-7}$	6.34$\times {10}^{-3}$/50/18/4.49$\times {10}^{-7}$	1.02$\times {10}^{-2}$/102/35/6.85$\times {10}^{-7}$
x3(5)	5.63$\times {10}^{-3}$/39/16/8.49$\times {10}^{-7}$	7.93$\times {10}^{-3}$/68/26/5.31$\times {10}^{-7}$	1.03$\times {10}^{-2}$/96/33/6.96$\times {10}^{-7}$
x4(5)	3.01$\times {10}^{-2}$/373/49/6.04$\times {10}^{-7}$	4.30$\times {10}^{-2}$/567/65/2.84$\times {10}^{-7}$	1.33$\times {10}^{-2}$/133/42/8.46$\times {10}^{-7}$
x5(5)	4.56$\times {10}^{-3}$/36/14/2.40$\times {10}^{-7}$	7.41$\times {10}^{-3}$/62/23/9.87$\times {10}^{-7}$	9.95$\times {10}^{-3}$/99/34/6.30$\times {10}^{-7}$
x6(5)	3.07$\times {10}^{-2}$/376/50/1.06$\times {10}^{-7}$	4.29$\times {10}^{-2}$/570/62/1.23$\times {10}^{-7}$	1.31$\times {10}^{-2}$/141/45/7.46$\times {10}^{-7}$
x7(5)	6.42$\times {10}^{-3}$/45/17/3.98$\times {10}^{-7}$	6.82$\times {10}^{-3}$/61/22/7.17$\times {10}^{-7}$	1.05$\times {10}^{-2}$/96/33/5.87$\times {10}^{-7}$
x8(5)	4.89$\times {10}^{-2}$/637/69/9.82$\times {10}^{-7}$	4.17$\times {10}^{-2}$/562/63/6.41$\times {10}^{-8}$	1.70$\times {10}^{-2}$/189/44/9.43$\times {10}^{-7}$
x1(10)	2.08$\times {10}^{-2}$/93/24/3.59$\times {10}^{-7}$	3.40$\times {10}^{-2}$/265/34/3.37$\times {10}^{-7}$	2.11$\times {10}^{-2}$/118/38/7.24$\times {10}^{-7}$
x2(10)	1.13$\times {10}^{-2}$/42/16/1.55$\times {10}^{-7}$	6.60$\times {10}^{-3}$/35/13/3.54$\times {10}^{-7}$	2.09$\times {10}^{-2}$/108/37/5.11$\times {10}^{-7}$
x3(10)	1.12$\times {10}^{-2}$/45/17/3.65$\times {10}^{-7}$	1.33$\times {10}^{-2}$/55/20/9.85$\times {10}^{-7}$	1.98$\times {10}^{-2}$/108/37/5.70$\times {10}^{-7}$
x4(10)	6.45$\times {10}^{-2}$/464/57/8.15$\times {10}^{-8}$	1.00$\times {10}^{-1}$/811/87/5.54$\times {10}^{-7}$	2.51$\times {10}^{-2}$/138/44/7.43$\times {10}^{-7}$
x5(10)	1.15$\times {10}^{-2}$/37/15/9.00$\times {10}^{-7}$	9.85$\times {10}^{-3}$/51/19/2.68$\times {10}^{-7}$	1.91$\times {10}^{-2}$/102/35/5.79$\times {10}^{-7}$
x6(10)	6.39$\times {10}^{-2}$/461/56/9.13$\times {10}^{-7}$	1.17$\times {10}^{-1}$/1002/101/2.35$\times {10}^{-7}$	2.36$\times {10}^{-2}$/138/44/9.78$\times {10}^{-7}$
x7(10)	1.23$\times {10}^{-2}$/43/17/4.03$\times {10}^{-7}$	1.32$\times {10}^{-2}$/53/21/7.96$\times {10}^{-7}$	2.08$\times {10}^{-2}$/109/37/8.60$\times {10}^{-7}$
x8(10)	8.19$\times {10}^{-2}$/633/66/5.97$\times {10}^{-7}$	1.11$\times {10}^{-1}$/1005/92/2.54$\times {10}^{-7}$	4.14$\times {10}^{-2}$/276/54/8.07$\times {10}^{-7}$
x1(50)	1.73$\times {10}^{-1}$/401/46/1.44$\times {10}^{-7}$	2.90$\times {10}^{-1}$/724/73/8.61$\times {10}^{-7}$	9.25$\times {10}^{-2}$/186/46/7.62$\times {10}^{-7}$
x2(50)	4.29$\times {10}^{-2}$/57/20/2.84$\times {10}^{-7}$	4.22$\times {10}^{-2}$/65/22/4.43$\times {10}^{-7}$	6.37$\times {10}^{-2}$/108/37/8.56$\times {10}^{-7}$
x3(50)	3.75$\times {10}^{-2}$/57/19/3.18$\times {10}^{-7}$	3.22$\times {10}^{-2}$/46/18/8.01$\times {10}^{-7}$	6.17$\times {10}^{-2}$/108/37/8.41$\times {10}^{-7}$
x4(50)	3.34$\times {10}^{-1}$/788/76/5.24$\times {10}^{-7}$	8.21$\times {10}^{-1}$/2118/166/6.17$\times {10}^{-7}$	8.29$\times {10}^{-2}$/139/43/9.48$\times {10}^{-7}$
x5(50)	3.79$\times {10}^{-2}$/45/17/2.58$\times {10}^{-7}$	5.17$\times {10}^{-2}$/70/24/4.35$\times {10}^{-7}$	6.62$\times {10}^{-2}$/108/37/8.69$\times {10}^{-7}$
x6(50)	3.40$\times {10}^{-1}$/805/78/9.22$\times {10}^{-7}$	6.01$\times {10}^{-1}$/1542/130/6.18$\times {10}^{-7}$	7.45$\times {10}^{-2}$/133/41/7.02$\times {10}^{-7}$
x7(50)	4.43$\times {10}^{-2}$/58/21/1.15$\times {10}^{-7}$	4.87$\times {10}^{-2}$/75/26/3.42$\times {10}^{-8}$	6.55$\times {10}^{-2}$/112/38/9.59$\times {10}^{-7}$
x8(50)	4.03$\times {10}^{-1}$/968/91/8.15$\times {10}^{-7}$	7.75$\times {10}^{-1}$/2002/154/3.01$\times {10}^{-7}$	1.63$\times {10}^{-1}$/371/61/9.71$\times {10}^{-7}$
x1(100)	3.19$\times {10}^{-1}$/422/51/5.34$\times {10}^{-8}$	6.89$\times {10}^{-1}$/943/93/3.29$\times {10}^{-7}$	2.17$\times {10}^{-1}$/267/58/9.06$\times {10}^{-7}$
x2(100)	5.56$\times {10}^{-2}$/42/16/6.16$\times {10}^{-7}$	8.42$\times {10}^{-2}$/75/25/3.72$\times {10}^{-7}$	1.19$\times {10}^{-1}$/118/40/4.98$\times {10}^{-7}$
x3(100)	5.52$\times {10}^{-2}$/42/16/6.54$\times {10}^{-8}$	1.17$\times {10}^{-1}$/103/33/7.65$\times {10}^{-7}$	1.17$\times {10}^{-1}$/118/40/5.08$\times {10}^{-7}$
x4(100)	5.75$\times {10}^{-1}$/771/77/2.88$\times {10}^{-7}$	1.34$\times {10}^0$/1932/156/7.33$\times {10}^{-7}$	1.24$\times {10}^{-1}$/133/39/7.09$\times {10}^{-7}$
x5(100)	8.06$\times {10}^{-2}$/66/23/7.79$\times {10}^{-8}$	7.31$\times {10}^{-2}$/58/20/1.15$\times {10}^{-7}$	1.17$\times {10}^{-1}$/118/40/5.29$\times {10}^{-7}$
x6(100)	5.64$\times {10}^{-1}$/774/80/6.88$\times {10}^{-7}$	1.28$\times {10}^0$/1883/150/7.31$\times {10}^{-8}$	1.25$\times {10}^{-1}$/133/39/7.55$\times {10}^{-7}$
x7(100)	8.95$\times {10}^{-2}$/76/24/2.56$\times {10}^{-7}$	8.48$\times {10}^{-2}$/71/25/6.40$\times {10}^{-7}$	1.10$\times {10}^{-1}$/109/37/9.37$\times {10}^{-7}$
x8(100)	8.61$\times {10}^{-1}$/1225/105/7.94$\times {10}^{-7}$	1.92$\times {10}^0$/2868/204/7.36$\times {10}^{-7}$	2.37$\times {10}^{-1}$/320/49/9.81$\times {10}^{-7}$

,

Table 5. Numerical results for Problem 5.

Init(n)	ITTCG	HTTCGP	ZYL
	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm
x1(1)	3.96$\times {10}^{-4}$/12/5/5.48$\times {10}^{-8}$	5.80$\times {10}^{-4}$/24/10/3.72$\times {10}^{-7}$	5.63$\times {10}^{-4}$/31/10/6.08$\times {10}^{-7}$
x2(1)	2.51$\times {10}^{-4}$/10/4/1.65$\times {10}^{-8}$	3.69$\times {10}^{-4}$/18/7/5.57$\times {10}^{-8}$	3.90$\times {10}^{-4}$/25/8/3.43$\times {10}^{-7}$
x3(1)	5.97$\times {10}^{-4}$/20/7/8.62$\times {10}^{-7}$	8.13$\times {10}^{-4}$/33/11/8.36$\times {10}^{-7}$	1.04$\times {10}^{-3}$/52/17/5.45$\times {10}^{-7}$
x4(1)	1.06$\times {10}^{-3}$/42/16/6.01$\times {10}^{-7}$	8.85$\times {10}^{-4}$/44/14/1.09$\times {10}^{-7}$	1.57$\times {10}^{-3}$/88/29/8.47$\times {10}^{-7}$
x5(1)	6.14$\times {10}^{-4}$/26/9/6.17$\times {10}^{-8}$	1.25$\times {10}^{-3}$/61/19/6.07$\times {10}^{-7}$	1.39$\times {10}^{-3}$/79/26/9.94$\times {10}^{-7}$
x6(1)	1.04$\times {10}^{-3}$/40/16/4.36$\times {10}^{-7}$	7.10$\times {10}^{-4}$/35/11/4.13$\times {10}^{-7}$	1.69$\times {10}^{-3}$/88/29/9.04$\times {10}^{-7}$
x7(1)	1.08$\times {10}^{-3}$/42/16/6.01$\times {10}^{-7}$	9.01$\times {10}^{-4}$/44/14/1.09$\times {10}^{-7}$	1.56$\times {10}^{-3}$/88/29/8.47$\times {10}^{-7}$
x8(1)	7.76$\times {10}^{-4}$/32/11/5.11$\times {10}^{-7}$	1.13$\times {10}^{-3}$/56/17/3.91$\times {10}^{-7}$	1.45$\times {10}^{-3}$/79/26/7.43$\times {10}^{-7}$
x1(5)	2.17$\times {10}^{-3}$/12/5/1.39$\times {10}^{-7}$	3.04$\times {10}^{-3}$/24/10/8.33$\times {10}^{-7}$	3.11$\times {10}^{-3}$/34/11/2.17$\times {10}^{-7}$
x2(5)	1.18$\times {10}^{-3}$/10/4/1.65$\times {10}^{-8}$	1.87$\times {10}^{-3}$/18/7/5.57$\times {10}^{-8}$	2.36$\times {10}^{-3}$/25/8/3.43$\times {10}^{-7}$
x3(5)	9.94$\times {10}^{-4}$/8/3/2.00$\times {10}^{-7}$	1.70$\times {10}^{-3}$/14/5/7.19$\times {10}^{-8}$	2.78$\times {10}^{-3}$/25/8/4.88$\times {10}^{-7}$
x4(5)	8.32$\times {10}^{-3}$/37/13/5.39$\times {10}^{-7}$	5.83$\times {10}^{-3}$/47/16/5.29$\times {10}^{-7}$	9.94$\times {10}^{-3}$/91/30/9.14$\times {10}^{-7}$
x5(5)	3.12$\times {10}^{-3}$/26/9/6.15$\times {10}^{-8}$	3.98$\times {10}^{-3}$/39/14/5.16$\times {10}^{-7}$	7.61$\times {10}^{-3}$/79/26/9.95$\times {10}^{-7}$
x6(5)	4.18$\times {10}^{-3}$/37/13/5.25$\times {10}^{-7}$	5.90$\times {10}^{-3}$/56/18/6.52$\times {10}^{-7}$	8.74$\times {10}^{-3}$/91/30/9.26$\times {10}^{-7}$
x7(5)	4.40$\times {10}^{-3}$/37/13/5.39$\times {10}^{-7}$	4.87$\times {10}^{-3}$/47/16/5.29$\times {10}^{-7}$	1.31$\times {10}^{-2}$/91/30/9.14$\times {10}^{-7}$
x8(5)	5.04$\times {10}^{-3}$/37/13/5.20$\times {10}^{-7}$	4.02$\times {10}^{-3}$/40/13/3.87$\times {10}^{-8}$	8.56$\times {10}^{-3}$/91/30/7.14$\times {10}^{-7}$
x1(10)	2.64$\times {10}^{-3}$/12/5/2.15$\times {10}^{-7}$	5.83$\times {10}^{-3}$/27/11/4.48$\times {10}^{-8}$	4.55$\times {10}^{-3}$/34/11/3.08$\times {10}^{-7}$
x2(10)	1.68$\times {10}^{-3}$/10/4/1.65$\times {10}^{-8}$	2.94$\times {10}^{-3}$/18/7/5.57$\times {10}^{-8}$	3.62$\times {10}^{-3}$/25/8/3.43$\times {10}^{-7}$
x3(10)	1.27$\times {10}^{-3}$/8/3/2.00$\times {10}^{-7}$	2.51$\times {10}^{-3}$/14/5/7.19$\times {10}^{-8}$	3.17$\times {10}^{-3}$/25/8/4.88$\times {10}^{-7}$
x4(10)	7.97$\times {10}^{-3}$/40/14/2.70$\times {10}^{-7}$	8.08$\times {10}^{-3}$/49/17/7.74$\times {10}^{-7}$	1.51$\times {10}^{-2}$/97/32/4.85$\times {10}^{-7}$
x5(10)	4.56$\times {10}^{-3}$/26/9/6.15$\times {10}^{-8}$	9.67$\times {10}^{-3}$/42/16/1.01$\times {10}^{-7}$	1.26$\times {10}^{-2}$/79/26/9.95$\times {10}^{-7}$
x6(10)	1.00$\times {10}^{-2}$/40/14/2.69$\times {10}^{-7}$	7.89$\times {10}^{-3}$/45/16/4.43$\times {10}^{-7}$	1.55$\times {10}^{-2}$/97/32/4.88$\times {10}^{-7}$
x7(10)	8.85$\times {10}^{-3}$/40/14/2.70$\times {10}^{-7}$	9.63$\times {10}^{-3}$/49/17/7.74$\times {10}^{-7}$	1.61$\times {10}^{-2}$/97/32/4.85$\times {10}^{-7}$
x8(10)	7.50$\times {10}^{-3}$/37/13/9.78$\times {10}^{-7}$	1.09$\times {10}^{-2}$/59/20/4.61$\times {10}^{-8}$	1.50$\times {10}^{-2}$/91/30/9.04$\times {10}^{-7}$
x1(50)	8.56$\times {10}^{-3}$/12/5/6.49$\times {10}^{-7}$	2.03$\times {10}^{-2}$/27/11/1.00$\times {10}^{-7}$	1.79$\times {10}^{-2}$/34/11/6.88$\times {10}^{-7}$
x2(50)	5.63$\times {10}^{-3}$/10/4/1.65$\times {10}^{-8}$	9.17$\times {10}^{-3}$/18/7/5.57$\times {10}^{-8}$	9.69$\times {10}^{-3}$/25/8/3.43$\times {10}^{-7}$
x3(50)	4.62$\times {10}^{-3}$/8/3/2.00$\times {10}^{-7}$	6.75$\times {10}^{-3}$/14/5/7.19$\times {10}^{-8}$	9.82$\times {10}^{-3}$/25/8/4.88$\times {10}^{-7}$
x4(50)	2.99$\times {10}^{-2}$/40/14/3.65$\times {10}^{-7}$	3.13$\times {10}^{-2}$/47/17/4.20$\times {10}^{-8}$	5.09$\times {10}^{-2}$/100/33/8.67$\times {10}^{-7}$
x5(50)	1.85$\times {10}^{-2}$/26/9/6.15$\times {10}^{-8}$	2.43$\times {10}^{-2}$/39/13/7.07$\times {10}^{-8}$	4.00$\times {10}^{-2}$/79/26/9.95$\times {10}^{-7}$
x6(50)	3.18$\times {10}^{-2}$/40/14/3.65$\times {10}^{-7}$	2.60$\times {10}^{-2}$/42/14/8.30$\times {10}^{-8}$	5.30$\times {10}^{-2}$/100/33/8.68$\times {10}^{-7}$
x7(50)	2.55$\times {10}^{-2}$/40/14/3.65$\times {10}^{-7}$	3.04$\times {10}^{-2}$/47/17/4.20$\times {10}^{-8}$	5.10$\times {10}^{-2}$/100/33/8.67$\times {10}^{-7}$
x8(50)	2.91$\times {10}^{-2}$/40/14/3.61$\times {10}^{-7}$	3.75$\times {10}^{-2}$/63/20/7.11$\times {10}^{-8}$	4.94$\times {10}^{-2}$/97/32/9.80$\times {10}^{-7}$
x1(100)	1.97$\times {10}^{-2}$/14/6/1.09$\times {10}^{-8}$	3.18$\times {10}^{-2}$/27/11/1.42$\times {10}^{-7}$	3.09$\times {10}^{-2}$/34/11/9.72$\times {10}^{-7}$
x2(100)	9.06$\times {10}^{-3}$/10/4/1.65$\times {10}^{-8}$	1.52$\times {10}^{-2}$/18/7/5.57$\times {10}^{-8}$	1.74$\times {10}^{-2}$/25/8/3.43$\times {10}^{-7}$
x3(100)	7.64$\times {10}^{-3}$/8/3/2.00$\times {10}^{-7}$	1.07$\times {10}^{-2}$/14/5/7.19$\times {10}^{-8}$	1.55$\times {10}^{-2}$/25/8/4.88$\times {10}^{-7}$
x4(100)	4.34$\times {10}^{-2}$/40/14/7.05$\times {10}^{-7}$	4.65$\times {10}^{-2}$/45/16/3.74$\times {10}^{-7}$	8.37$\times {10}^{-2}$/103/34/5.77$\times {10}^{-7}$
x5(100)	2.98$\times {10}^{-2}$/26/9/6.15$\times {10}^{-8}$	4.12$\times {10}^{-2}$/41/14/4.50$\times {10}^{-7}$	6.93$\times {10}^{-2}$/79/26/9.95$\times {10}^{-7}$
x6(100)	4.34$\times {10}^{-2}$/40/14/7.05$\times {10}^{-7}$	4.32$\times {10}^{-2}$/41/14/6.54$\times {10}^{-7}$	8.85$\times {10}^{-2}$/103/34/5.78$\times {10}^{-7}$
x7(100)	5.00$\times {10}^{-2}$/40/14/7.05$\times {10}^{-7}$	4.74$\times {10}^{-2}$/45/16/3.74$\times {10}^{-7}$	9.16$\times {10}^{-2}$/103/34/5.77$\times {10}^{-7}$
x8(100)	4.77$\times {10}^{-2}$/40/14/7.04$\times {10}^{-7}$	6.10$\times {10}^{-2}$/54/19/6.43$\times {10}^{-7}$	8.39$\times {10}^{-2}$/103/34/5.47$\times {10}^{-7}$

,

Table 6. Numerical results for Problem 6.

Init(n)	ITTCG	HTTCGP	ZYL
	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm
x1(1)	2.71$\times {10}^{-4}$/6/1/0.00$\times {10}^0$	2.30$\times {10}^{-4}$/6/1/0.00$\times {10}^0$	1.25$\times {10}^{-3}$/57/9/6.00$\times {10}^{-7}$
x2(1)	1.63$\times {10}^{-4}$/9/1/0.00$\times {10}^0$	1.48$\times {10}^{-4}$/9/1/0.00$\times {10}^0$	6.92$\times {10}^{-4}$/43/7/1.85$\times {10}^{-7}$
x3(1)	1.63$\times {10}^{-3}$/96/12/3.75$\times {10}^{-15}$	2.41$\times {10}^{-3}$/151/19/2.45$\times {10}^{-7}$	2.70$\times {10}^{-3}$/174/28/9.86$\times {10}^{-7}$
x4(1)	2.26$\times {10}^{-3}$/114/14/5.42$\times {10}^{-7}$	3.11$\times {10}^{-3}$/163/20/0.00$\times {10}^0$	4.31$\times {10}^{-3}$/219/35/5.14$\times {10}^{-7}$
x5(1)	1.66$\times {10}^{-3}$/81/11/7.63$\times {10}^{-7}$	2.88$\times {10}^{-3}$/152/19/3.07$\times {10}^{-8}$	3.36$\times {10}^{-3}$/182/29/4.94$\times {10}^{-7}$
x6(1)	2.33$\times {10}^{-3}$/115/15/0.00$\times {10}^0$	3.59$\times {10}^{-3}$/200/24/0.00$\times {10}^0$	4.57$\times {10}^{-3}$/237/38/2.96$\times {10}^{-7}$
x7(1)	2.25$\times {10}^{-3}$/114/14/5.42$\times {10}^{-7}$	3.02$\times {10}^{-3}$/163/20/0.00$\times {10}^0$	4.16$\times {10}^{-3}$/219/35/5.14$\times {10}^{-7}$
x8(1)	2.19$\times {10}^{-3}$/95/12/1.03$\times {10}^{-7}$	3.90$\times {10}^{-3}$/185/23/2.29$\times {10}^{-7}$	4.09$\times {10}^{-3}$/186/30/5.18$\times {10}^{-7}$
x1(5)	1.31$\times {10}^{-3}$/6/1/0.00$\times {10}^0$	1.12$\times {10}^{-3}$/6/1/0.00$\times {10}^0$	7.51$\times {10}^{-3}$/63/10/1.25$\times {10}^{-7}$
x2(5)	1.05$\times {10}^{-3}$/9/1/0.00$\times {10}^0$	8.05$\times {10}^{-4}$/9/1/0.00$\times {10}^0$	4.20$\times {10}^{-3}$/43/7/1.85$\times {10}^{-7}$
x3(5)	3.70$\times {10}^{-4}$/3/1/0.00$\times {10}^0$	4.07$\times {10}^{-4}$/3/1/0.00$\times {10}^0$	3.86$\times {10}^{-4}$/3/1/0.00$\times {10}^0$
x4(5)	1.65$\times {10}^{-2}$/179/22/3.86$\times {10}^{-7}$	1.36$\times {10}^{-2}$/163/21/9.04$\times {10}^{-7}$	2.00$\times {10}^{-2}$/244/39/4.27$\times {10}^{-7}$
x5(5)	1.26$\times {10}^{-2}$/136/18/0.00$\times {10}^0$	1.13$\times {10}^{-2}$/143/18/2.74$\times {10}^{-7}$	1.59$\times {10}^{-2}$/194/31/8.94$\times {10}^{-7}$
x6(5)	1.58$\times {10}^{-2}$/185/23/4.39$\times {10}^{-7}$	1.13$\times {10}^{-2}$/141/18/2.33$\times {10}^{-7}$	2.13$\times {10}^{-2}$/255/41/4.07$\times {10}^{-7}$
x7(5)	1.55$\times {10}^{-2}$/179/22/3.86$\times {10}^{-7}$	1.34$\times {10}^{-2}$/163/21/9.04$\times {10}^{-7}$	1.94$\times {10}^{-2}$/244/39/4.27$\times {10}^{-7}$
x8(5)	7.83$\times {10}^{-3}$/81/10/0.00$\times {10}^0$	1.40$\times {10}^{-2}$/154/20/8.32$\times {10}^{-7}$	1.93$\times {10}^{-2}$/225/36/4.26$\times {10}^{-7}$
x1(10)	9.42$\times {10}^{-4}$/6/1/0.00$\times {10}^0$	8.92$\times {10}^{-4}$/6/1/0.00$\times {10}^0$	6.75$\times {10}^{-3}$/63/10/1.76$\times {10}^{-7}$
x2(10)	8.31$\times {10}^{-4}$/9/1/0.00$\times {10}^0$	8.03$\times {10}^{-4}$/9/1/0.00$\times {10}^0$	4.85$\times {10}^{-3}$/43/7/1.85$\times {10}^{-7}$
x3(10)	5.54$\times {10}^{-4}$/3/1/0.00$\times {10}^0$	4.58$\times {10}^{-4}$/3/1/0.00$\times {10}^0$	3.31$\times {10}^{-4}$/3/1/0.00$\times {10}^0$
x4(10)	1.20$\times {10}^{-2}$/105/13/1.67$\times {10}^{-9}$	1.65$\times {10}^{-2}$/157/20/5.74$\times {10}^{-7}$	2.66$\times {10}^{-2}$/244/39/4.95$\times {10}^{-7}$
x5(10)	1.61$\times {10}^{-2}$/122/16/0.00$\times {10}^0$	1.56$\times {10}^{-2}$/154/19/0.00$\times {10}^0$	1.93$\times {10}^{-2}$/182/29/4.08$\times {10}^{-7}$
x6(10)	1.10$\times {10}^{-2}$/90/11/2.02$\times {10}^{-9}$	1.44$\times {10}^{-2}$/134/17/2.39$\times {10}^{-7}$	2.34$\times {10}^{-2}$/226/36/7.42$\times {10}^{-7}$
x7(10)	1.21$\times {10}^{-2}$/105/13/1.67$\times {10}^{-9}$	1.60$\times {10}^{-2}$/157/20/5.74$\times {10}^{-7}$	2.61$\times {10}^{-2}$/244/39/4.95$\times {10}^{-7}$
x8(10)	1.21$\times {10}^{-2}$/98/12/0.00$\times {10}^0$	1.66$\times {10}^{-2}$/131/17/0.00$\times {10}^0$	2.84$\times {10}^{-2}$/255/41/8.67$\times {10}^{-7}$
x1(50)	3.23$\times {10}^{-3}$/6/1/0.00$\times {10}^0$	2.94$\times {10}^{-3}$/6/1/0.00$\times {10}^0$	2.81$\times {10}^{-2}$/63/10/3.94$\times {10}^{-7}$
x2(50)	2.65$\times {10}^{-3}$/9/1/0.00$\times {10}^0$	2.75$\times {10}^{-3}$/9/1/0.00$\times {10}^0$	1.46$\times {10}^{-2}$/43/7/1.85$\times {10}^{-7}$
x3(50)	1.44$\times {10}^{-3}$/3/1/0.00$\times {10}^0$	1.66$\times {10}^{-3}$/3/1/0.00$\times {10}^0$	1.39$\times {10}^{-3}$/3/1/0.00$\times {10}^0$
x4(50)	4.61$\times {10}^{-2}$/103/13/1.41$\times {10}^{-7}$	6.50$\times {10}^{-2}$/157/20/3.17$\times {10}^{-7}$	1.04$\times {10}^{-1}$/250/40/5.94$\times {10}^{-7}$
x5(50)	8.27$\times {10}^{-2}$/186/24/9.60$\times {10}^{-8}$	6.28$\times {10}^{-2}$/148/19/3.83$\times {10}^{-7}$	7.85$\times {10}^{-2}$/194/31/5.49$\times {10}^{-7}$
x6(50)	4.52$\times {10}^{-2}$/103/13/1.43$\times {10}^{-7}$	5.55$\times {10}^{-2}$/134/17/2.22$\times {10}^{-16}$	1.00$\times {10}^{-1}$/238/38/5.59$\times {10}^{-7}$
x7(50)	4.50$\times {10}^{-2}$/103/13/1.41$\times {10}^{-7}$	6.59$\times {10}^{-2}$/157/20/3.17$\times {10}^{-7}$	1.06$\times {10}^{-1}$/250/40/5.94$\times {10}^{-7}$
x8(50)	4.76$\times {10}^{-2}$/96/12/1.33$\times {10}^{-7}$	5.49$\times {10}^{-2}$/117/15/8.48$\times {10}^{-7}$	1.03$\times {10}^{-1}$/238/38/6.93$\times {10}^{-7}$
x1(100)	8.14$\times {10}^{-3}$/6/1/0.00$\times {10}^0$	6.15$\times {10}^{-3}$/6/1/0.00$\times {10}^0$	4.80$\times {10}^{-2}$/63/10/5.57$\times {10}^{-7}$
x2(100)	5.83$\times {10}^{-3}$/9/1/0.00$\times {10}^0$	4.69$\times {10}^{-3}$/9/1/0.00$\times {10}^0$	2.41$\times {10}^{-2}$/43/7/1.85$\times {10}^{-7}$
x3(100)	3.14$\times {10}^{-3}$/3/1/0.00$\times {10}^0$	2.90$\times {10}^{-3}$/3/1/0.00$\times {10}^0$	2.63$\times {10}^{-3}$/3/1/0.00$\times {10}^0$
x4(100)	1.27$\times {10}^{-1}$/166/21/2.91$\times {10}^{-7}$	1.31$\times {10}^{-1}$/173/22/5.65$\times {10}^{-7}$	1.81$\times {10}^{-1}$/250/40/9.26$\times {10}^{-7}$
x5(100)	1.04$\times {10}^{-1}$/146/19/0.00$\times {10}^0$	1.04$\times {10}^{-1}$/148/19/5.90$\times {10}^{-7}$	1.18$\times {10}^{-1}$/175/28/1.08$\times {10}^{-7}$
x6(100)	1.25$\times {10}^{-1}$/166/21/3.23$\times {10}^{-7}$	9.50$\times {10}^{-2}$/134/17/2.71$\times {10}^{-15}$	1.71$\times {10}^{-1}$/238/38/9.99$\times {10}^{-7}$
x7(100)	1.19$\times {10}^{-1}$/166/21/2.91$\times {10}^{-7}$	1.23$\times {10}^{-1}$/173/22/5.65$\times {10}^{-7}$	1.76$\times {10}^{-1}$/250/40/9.26$\times {10}^{-7}$
x8(100)	9.91$\times {10}^{-2}$/111/14/9.76$\times {10}^{-9}$	1.06$\times {10}^{-1}$/126/16/3.64$\times {10}^{-15}$	2.06$\times {10}^{-1}$/273/44/9.01$\times {10}^{-7}$

,

Table 7. Numerical results for Problem 7.

Init(n)	ITTCG	HTTCGP	ZYL
	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm
x1(1)	9.76$\times {10}^{-3}$/90/19/1.33$\times {10}^{-7}$	1.35$\times {10}^{-2}$/120/24/3.94$\times {10}^{-7}$	3.80$\times {10}^{-2}$/381/90/9.43$\times {10}^{-7}$
x2(1)	8.69$\times {10}^{-3}$/84/18/8.04$\times {10}^{-7}$	1.28$\times {10}^{-2}$/133/27/6.50$\times {10}^{-7}$	4.29$\times {10}^{-2}$/434/103/6.56$\times {10}^{-7}$
x3(1)	1.10$\times {10}^{-2}$/98/21/6.30$\times {10}^{-7}$	1.32$\times {10}^{-2}$/132/26/9.64$\times {10}^{-7}$	4.17$\times {10}^{-2}$/426/101/6.50$\times {10}^{-7}$
x4(1)	9.65$\times {10}^{-3}$/102/22/8.39$\times {10}^{-7}$	1.07$\times {10}^{-2}$/109/22/5.40$\times {10}^{-7}$	4.01$\times {10}^{-2}$/418/99/5.99$\times {10}^{-7}$
x5(1)	7.44$\times {10}^{-3}$/76/16/9.56$\times {10}^{-7}$	9.36$\times {10}^{-3}$/96/19/6.21$\times {10}^{-7}$	4.00$\times {10}^{-2}$/401/95/7.48$\times {10}^{-7}$
x6(1)	1.01$\times {10}^{-2}$/101/22/9.38$\times {10}^{-7}$	1.17$\times {10}^{-2}$/117/23/2.55$\times {10}^{-7}$	4.17$\times {10}^{-2}$/418/99/5.98$\times {10}^{-7}$
x7(1)	8.83$\times {10}^{-3}$/89/19/3.19$\times {10}^{-7}$	1.24$\times {10}^{-2}$/129/25/2.52$\times {10}^{-7}$	3.56$\times {10}^{-2}$/357/84/5.14$\times {10}^{-7}$
x8(1)	8.88$\times {10}^{-3}$/91/20/2.65$\times {10}^{-7}$	1.71$\times {10}^{-2}$/167/33/4.74$\times {10}^{-7}$	4.15$\times {10}^{-2}$/417/99/9.79$\times {10}^{-7}$
x1(5)	4.95$\times {10}^{-2}$/94/20/5.32$\times {10}^{-7}$	5.72$\times {10}^{-2}$/111/22/5.98$\times {10}^{-7}$	1.94$\times {10}^{-1}$/369/87/5.58$\times {10}^{-7}$
x2(5)	5.82$\times {10}^{-2}$/113/25/1.86$\times {10}^{-7}$	6.32$\times {10}^{-2}$/129/27/5.34$\times {10}^{-7}$	2.32$\times {10}^{-1}$/458/109/6.29$\times {10}^{-7}$
x3(5)	5.97$\times {10}^{-2}$/112/25/8.38$\times {10}^{-7}$	6.88$\times {10}^{-2}$/137/29/4.35$\times {10}^{-7}$	2.40$\times {10}^{-1}$/458/109/6.29$\times {10}^{-7}$
x4(5)	5.91$\times {10}^{-2}$/111/24/2.77$\times {10}^{-7}$	4.96$\times {10}^{-2}$/94/19/7.52$\times {10}^{-7}$	2.31$\times {10}^{-1}$/446/106/5.82$\times {10}^{-7}$
x5(5)	5.37$\times {10}^{-2}$/103/23/1.59$\times {10}^{-7}$	8.01$\times {10}^{-2}$/166/31/5.77$\times {10}^{-7}$	2.40$\times {10}^{-1}$/458/109/6.07$\times {10}^{-7}$
x6(5)	6.43$\times {10}^{-2}$/122/27/6.59$\times {10}^{-7}$	4.75$\times {10}^{-2}$/94/19/6.85$\times {10}^{-7}$	2.28$\times {10}^{-1}$/446/106/5.81$\times {10}^{-7}$
x7(5)	5.11$\times {10}^{-2}$/96/21/3.29$\times {10}^{-7}$	5.95$\times {10}^{-2}$/115/23/8.71$\times {10}^{-7}$	2.29$\times {10}^{-1}$/454/108/6.09$\times {10}^{-7}$
x8(5)	5.29$\times {10}^{-2}$/103/22/5.52$\times {10}^{-7}$	5.48$\times {10}^{-2}$/111/23/7.31$\times {10}^{-7}$	2.06$\times {10}^{-1}$/405/96/7.94$\times {10}^{-7}$
x1(10)	1.14$\times {10}^{-1}$/118/26/7.92$\times {10}^{-7}$	1.24$\times {10}^{-1}$/132/27/4.22$\times {10}^{-7}$	4.45$\times {10}^{-1}$/474/113/7.04$\times {10}^{-7}$
x2(10)	1.04$\times {10}^{-1}$/108/24/5.78$\times {10}^{-7}$	1.06$\times {10}^{-1}$/112/23/4.30$\times {10}^{-7}$	3.78$\times {10}^{-1}$/395/94/6.07$\times {10}^{-7}$
x3(10)	1.04$\times {10}^{-1}$/109/24/5.61$\times {10}^{-7}$	9.55$\times {10}^{-2}$/101/21/4.62$\times {10}^{-7}$	3.72$\times {10}^{-1}$/395/94/6.07$\times {10}^{-7}$
x4(10)	1.15$\times {10}^{-1}$/123/27/4.37$\times {10}^{-7}$	1.13$\times {10}^{-1}$/122/25/6.26$\times {10}^{-7}$	4.19$\times {10}^{-1}$/446/106/7.30$\times {10}^{-7}$
x5(10)	1.29$\times {10}^{-1}$/135/30/4.75$\times {10}^{-7}$	1.22$\times {10}^{-1}$/132/27/1.79$\times {10}^{-7}$	3.48$\times {10}^{-1}$/370/88/9.63$\times {10}^{-7}$
x6(10)	1.24$\times {10}^{-1}$/130/29/5.79$\times {10}^{-7}$	1.19$\times {10}^{-1}$/129/27/7.05$\times {10}^{-7}$	4.25$\times {10}^{-1}$/446/106/7.30$\times {10}^{-7}$
x7(10)	1.07$\times {10}^{-1}$/112/25/2.50$\times {10}^{-7}$	1.20$\times {10}^{-1}$/127/27/4.10$\times {10}^{-7}$	4.28$\times {10}^{-1}$/444/106/9.47$\times {10}^{-7}$
x8(10)	1.05$\times {10}^{-1}$/109/24/3.00$\times {10}^{-7}$	1.24$\times {10}^{-1}$/133/27/9.07$\times {10}^{-7}$	4.46$\times {10}^{-1}$/466/111/6.03$\times {10}^{-7}$
x1(50)	6.01$\times {10}^{-1}$/118/26/8.42$\times {10}^{-7}$	5.36$\times {10}^{-1}$/106/22/8.97$\times {10}^{-7}$	2.45$\times {10}^0$/468/112/7.75$\times {10}^{-7}$
x2(50)	5.22$\times {10}^{-1}$/102/23/3.61$\times {10}^{-7}$	6.67$\times {10}^{-1}$/130/27/2.68$\times {10}^{-7}$	2.42$\times {10}^0$/464/111/8.61$\times {10}^{-7}$
x3(50)	5.24$\times {10}^{-1}$/102/23/3.92$\times {10}^{-7}$	6.79$\times {10}^{-1}$/133/27/9.03$\times {10}^{-7}$	2.39$\times {10}^0$/464/111/8.57$\times {10}^{-7}$
x4(50)	5.01$\times {10}^{-1}$/98/22/6.48$\times {10}^{-7}$	7.64$\times {10}^{-1}$/150/31/2.72$\times {10}^{-7}$	2.35$\times {10}^0$/452/108/8.91$\times {10}^{-7}$
x5(50)	5.68$\times {10}^{-1}$/111/25/4.20$\times {10}^{-7}$	7.90$\times {10}^{-1}$/155/32/7.32$\times {10}^{-7}$	2.41$\times {10}^0$/464/111/8.34$\times {10}^{-7}$
x6(50)	5.04$\times {10}^{-1}$/98/22/6.48$\times {10}^{-7}$	7.17$\times {10}^{-1}$/141/29/5.02$\times {10}^{-7}$	2.34$\times {10}^0$/452/108/8.91$\times {10}^{-7}$
x7(50)	6.01$\times {10}^{-1}$/115/26/4.87$\times {10}^{-7}$	6.63$\times {10}^{-1}$/128/27/9.58$\times {10}^{-7}$	2.40$\times {10}^0$/460/110/9.28$\times {10}^{-7}$
x8(50)	5.11$\times {10}^{-1}$/98/22/7.88$\times {10}^{-7}$	6.79$\times {10}^{-1}$/131/28/5.48$\times {10}^{-7}$	2.30$\times {10}^0$/432/103/7.76$\times {10}^{-7}$
x1(100)	9.81$\times {10}^{-1}$/97/22/8.53$\times {10}^{-7}$	1.08$\times {10}^0$/112/23/5.11$\times {10}^{-7}$	4.53$\times {10}^0$/456/109/9.55$\times {10}^{-7}$
x2(100)	1.19$\times {10}^0$/118/27/8.34$\times {10}^{-7}$	1.33$\times {10}^0$/129/26/4.22$\times {10}^{-7}$	4.91$\times {10}^0$/497/119/6.39$\times {10}^{-7}$
x3(100)	1.25$\times {10}^0$/127/29/2.67$\times {10}^{-7}$	1.38$\times {10}^0$/143/30/9.45$\times {10}^{-7}$	4.77$\times {10}^0$/481/115/6.26$\times {10}^{-7}$
x4(100)	1.06$\times {10}^0$/107/24/3.28$\times {10}^{-7}$	1.17$\times {10}^0$/122/25/7.53$\times {10}^{-7}$	4.52$\times {10}^0$/456/109/8.83$\times {10}^{-7}$
x5(100)	9.93$\times {10}^{-1}$/101/23/6.39$\times {10}^{-7}$	1.14$\times {10}^0$/119/24/4.93$\times {10}^{-7}$	4.92$\times {10}^0$/493/118/6.21$\times {10}^{-7}$
x6(100)	1.06$\times {10}^0$/107/24/3.29$\times {10}^{-7}$	1.25$\times {10}^0$/128/26/3.82$\times {10}^{-8}$	4.56$\times {10}^0$/456/109/8.83$\times {10}^{-7}$
x7(100)	9.94$\times {10}^{-1}$/98/22/9.49$\times {10}^{-8}$	1.25$\times {10}^0$/128/26/3.81$\times {10}^{-7}$	4.85$\times {10}^0$/485/116/6.04$\times {10}^{-7}$
x8(100)	1.02$\times {10}^0$/102/23/3.43$\times {10}^{-7}$	1.09$\times {10}^0$/113/23/5.86$\times {10}^{-7}$	4.82$\times {10}^0$/485/116/6.34$\times {10}^{-7}$

and

Table 8. Numerical results for Problem 8.

Init(n)	ITTCG	HTTCGP	ZYL
	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm	Time/Nfunc/Niter/Norm
x1(1)	4.04$\times {10}^{-3}$/44/14/6.03$\times {10}^{-7}$	1.97$\times {10}^{-3}$/26/8/4.64$\times {10}^{-8}$	2.25$\times {10}^{-3}$/29/7/5.31$\times {10}^{-7}$
x2(1)	1.03$\times {10}^{-2}$/159/23/6.90$\times {10}^{-7}$	1.31$\times {10}^{-2}$/205/28/9.90$\times {10}^{-7}$	1.02$\times {10}^{-2}$/160/28/5.49$\times {10}^{-7}$
x3(1)	9.97$\times {10}^{-3}$/164/24/8.26$\times {10}^{-7}$	1.26$\times {10}^{-2}$/215/30/7.18$\times {10}^{-7}$	9.64$\times {10}^{-3}$/159/28/5.57$\times {10}^{-7}$
x4(1)	7.67$\times {10}^{-3}$/128/19/9.28$\times {10}^{-7}$	8.69$\times {10}^{-3}$/140/20/5.23$\times {10}^{-7}$	9.64$\times {10}^{-3}$/159/30/4.97$\times {10}^{-7}$
x5(1)	9.63$\times {10}^{-3}$/159/23/7.20$\times {10}^{-7}$	1.32$\times {10}^{-2}$/213/30/6.66$\times {10}^{-7}$	9.20$\times {10}^{-3}$/155/27/4.53$\times {10}^{-7}$
x6(1)	7.92$\times {10}^{-3}$/128/19/9.28$\times {10}^{-7}$	8.39$\times {10}^{-3}$/144/20/5.73$\times {10}^{-7}$	1.03$\times {10}^{-2}$/159/30/4.88$\times {10}^{-7}$
x7(1)	7.57$\times {10}^{-3}$/128/19/7.06$\times {10}^{-7}$	9.54$\times {10}^{-3}$/155/22/5.75$\times {10}^{-7}$	9.33$\times {10}^{-3}$/155/29/9.48$\times {10}^{-7}$
x8(1)	1.15$\times {10}^{-2}$/187/27/5.73$\times {10}^{-7}$	1.32$\times {10}^{-2}$/221/30/5.32$\times {10}^{-7}$	7.11$\times {10}^{-3}$/120/20/9.25$\times {10}^{-7}$
x1(5)	2.10$\times {10}^{-2}$/47/15/3.88$\times {10}^{-7}$	1.00$\times {10}^{-2}$/26/8/1.04$\times {10}^{-7}$	1.21$\times {10}^{-2}$/33/8/6.70$\times {10}^{-8}$
x2(5)	5.09$\times {10}^{-2}$/165/24/7.30$\times {10}^{-7}$	6.85$\times {10}^{-2}$/230/32/9.16$\times {10}^{-7}$	4.22$\times {10}^{-2}$/144/25/8.58$\times {10}^{-7}$
x3(5)	5.39$\times {10}^{-2}$/177/26/9.37$\times {10}^{-7}$	6.29$\times {10}^{-2}$/213/30/9.98$\times {10}^{-7}$	4.46$\times {10}^{-2}$/148/26/4.02$\times {10}^{-7}$
x4(5)	4.28$\times {10}^{-2}$/133/20/9.06$\times {10}^{-7}$	4.55$\times {10}^{-2}$/147/21/7.30$\times {10}^{-7}$	4.27$\times {10}^{-2}$/137/26/4.54$\times {10}^{-7}$
x5(5)	5.68$\times {10}^{-2}$/183/27/7.21$\times {10}^{-7}$	6.18$\times {10}^{-2}$/197/27/5.19$\times {10}^{-7}$	4.93$\times {10}^{-2}$/155/27/7.52$\times {10}^{-7}$
x6(5)	4.07$\times {10}^{-2}$/133/20/9.04$\times {10}^{-7}$	5.31$\times {10}^{-2}$/175/25/7.40$\times {10}^{-7}$	4.29$\times {10}^{-2}$/137/26/4.73$\times {10}^{-7}$
x7(5)	4.01$\times {10}^{-2}$/133/20/5.99$\times {10}^{-7}$	5.52$\times {10}^{-2}$/182/26/5.60$\times {10}^{-7}$	4.15$\times {10}^{-2}$/135/26/9.26$\times {10}^{-7}$
x8(5)	6.15$\times {10}^{-2}$/206/30/5.21$\times {10}^{-7}$	7.30$\times {10}^{-2}$/239/33/2.79$\times {10}^{-7}$	4.02$\times {10}^{-2}$/126/21/5.03$\times {10}^{-7}$
x1(10)	3.57$\times {10}^{-2}$/47/15/5.49$\times {10}^{-7}$	1.98$\times {10}^{-2}$/26/8/1.47$\times {10}^{-7}$	2.13$\times {10}^{-2}$/33/8/9.47$\times {10}^{-8}$
x2(10)	9.53$\times {10}^{-2}$/165/24/8.88$\times {10}^{-7}$	1.21$\times {10}^{-1}$/215/30/5.80$\times {10}^{-7}$	8.14$\times {10}^{-2}$/144/25/6.81$\times {10}^{-7}$
x3(10)	9.90$\times {10}^{-2}$/172/25/5.14$\times {10}^{-7}$	1.15$\times {10}^{-1}$/202/28/3.43$\times {10}^{-7}$	8.20$\times {10}^{-2}$/136/24/8.97$\times {10}^{-7}$
x4(10)	7.54$\times {10}^{-2}$/133/20/4.98$\times {10}^{-7}$	7.96$\times {10}^{-2}$/144/21/3.17$\times {10}^{-7}$	8.38$\times {10}^{-2}$/141/27/4.63$\times {10}^{-7}$
x5(10)	9.17$\times {10}^{-2}$/165/24/9.01$\times {10}^{-7}$	1.07$\times {10}^{-1}$/193/27/7.56$\times {10}^{-7}$	8.60$\times {10}^{-2}$/150/26/5.92$\times {10}^{-7}$
x6(10)	7.73$\times {10}^{-2}$/133/20/4.98$\times {10}^{-7}$	1.12$\times {10}^{-1}$/206/29/3.42$\times {10}^{-7}$	8.49$\times {10}^{-2}$/141/27/4.68$\times {10}^{-7}$
x7(10)	7.29$\times {10}^{-2}$/126/19/8.04$\times {10}^{-7}$	1.17$\times {10}^{-1}$/204/29/4.02$\times {10}^{-7}$	8.39$\times {10}^{-2}$/140/27/8.91$\times {10}^{-7}$
x8(10)	1.13$\times {10}^{-1}$/200/29/7.31$\times {10}^{-7}$	1.32$\times {10}^{-1}$/222/30/3.73$\times {10}^{-7}$	7.25$\times {10}^{-2}$/126/21/6.66$\times {10}^{-7}$
x1(50)	1.83$\times {10}^{-1}$/50/16/3.53$\times {10}^{-7}$	9.11$\times {10}^{-2}$/26/8/3.28$\times {10}^{-7}$	1.12$\times {10}^{-1}$/33/8/2.12$\times {10}^{-7}$
x2(50)	5.21$\times {10}^{-1}$/171/25/8.48$\times {10}^{-7}$	5.41$\times {10}^{-1}$/177/25/7.99$\times {10}^{-7}$	4.51$\times {10}^{-1}$/143/25/5.60$\times {10}^{-7}$
x3(50)	5.16$\times {10}^{-1}$/171/25/8.84$\times {10}^{-7}$	7.61$\times {10}^{-1}$/251/35/5.12$\times {10}^{-7}$	4.13$\times {10}^{-1}$/131/23/8.83$\times {10}^{-7}$
x4(50)	4.21$\times {10}^{-1}$/138/21/5.04$\times {10}^{-7}$	4.63$\times {10}^{-1}$/150/22/8.39$\times {10}^{-7}$	4.68$\times {10}^{-1}$/150/29/9.54$\times {10}^{-7}$
x5(50)	5.25$\times {10}^{-1}$/171/25/8.61$\times {10}^{-7}$	7.06$\times {10}^{-1}$/231/33/7.37$\times {10}^{-7}$	4.45$\times {10}^{-1}$/143/25/4.76$\times {10}^{-7}$
x6(50)	4.20$\times {10}^{-1}$/138/21/5.04$\times {10}^{-7}$	4.90$\times {10}^{-1}$/164/24/3.00$\times {10}^{-7}$	4.77$\times {10}^{-1}$/150/29/9.67$\times {10}^{-7}$
x7(50)	4.06$\times {10}^{-1}$/131/20/6.58$\times {10}^{-7}$	4.65$\times {10}^{-1}$/152/22/7.21$\times {10}^{-7}$	5.08$\times {10}^{-1}$/156/30/4.61$\times {10}^{-7}$
x8(50)	5.92$\times {10}^{-1}$/195/28/8.22$\times {10}^{-7}$	8.47$\times {10}^{-1}$/281/39/2.74$\times {10}^{-7}$	4.11$\times {10}^{-1}$/132/22/6.17$\times {10}^{-7}$
x1(100)	3.58$\times {10}^{-1}$/50/16/4.99$\times {10}^{-7}$	1.80$\times {10}^{-1}$/26/8/4.64$\times {10}^{-7}$	2.12$\times {10}^{-1}$/33/8/3.00$\times {10}^{-7}$
x2(100)	1.05$\times {10}^0$/177/26/5.90$\times {10}^{-7}$	1.42$\times {10}^0$/244/34/4.53$\times {10}^{-7}$	8.12$\times {10}^{-1}$/137/24/6.64$\times {10}^{-7}$
x3(100)	1.04$\times {10}^0$/178/26/5.25$\times {10}^{-7}$	9.33$\times {10}^{-1}$/158/22/9.92$\times {10}^{-7}$	7.95$\times {10}^{-1}$/131/23/7.50$\times {10}^{-7}$
x4(100)	7.89$\times {10}^{-1}$/131/20/5.53$\times {10}^{-7}$	1.33$\times {10}^0$/226/33/7.44$\times {10}^{-7}$	9.26$\times {10}^{-1}$/149/29/6.70$\times {10}^{-7}$
x5(100)	1.06$\times {10}^0$/177/26/5.21$\times {10}^{-7}$	1.47$\times {10}^0$/242/34/5.15$\times {10}^{-7}$	8.52$\times {10}^{-1}$/137/24/7.67$\times {10}^{-7}$
x6(100)	7.95$\times {10}^{-1}$/131/20/5.53$\times {10}^{-7}$	1.26$\times {10}^0$/206/30/2.16$\times {10}^{-7}$	9.38$\times {10}^{-1}$/149/29/6.70$\times {10}^{-7}$
x7(100)	7.34$\times {10}^{-1}$/124/19/8.94$\times {10}^{-7}$	8.96$\times {10}^{-1}$/151/22/7.80$\times {10}^{-7}$	9.80$\times {10}^{-1}$/158/31/6.50$\times {10}^{-7}$
x8(100)	1.20$\times {10}^0$/202/29/5.27$\times {10}^{-7}$	1.35$\times {10}^0$/231/32/9.96$\times {10}^{-7}$	7.90$\times {10}^{-1}$/132/22/9.92$\times {10}^{-7}$

. In these tables, “Init(n)” refers to the initial points and the dimension multiplied by 1000. The detailed results are formatted as Time/Nfunc/Niter/Norm, where “Time” represents the CPU time in seconds, “Nfunc” represents the number of function evaluations, “Niter” represents the number of iterations, and “Norm” represents the norm of the function at the approximate optimal point. These tables illustrate that all three algorithms were capable of solving the benchmark problems across various initial points and dimensions. Notably, Algorithm ITTCG exhibited superior performance in most cases. To clearly demonstrate the performance of Algorithm ITTCG, we utilized the performance profiles developed by Dolan and Moré [29]. These profiles visually compared the performance in terms of CPU time, Nfunc, and Niter, as shown in Figure 1, Figure 2 and Figure 3. From these figures, we can observe that Algorithm ITTCG won about 59%, 77%, and 80% of the experiments in terms of CPU time, Nfunc, and Niter, respectively. The results indicate that Algorithm ITTCG outperformed Algorithm HTTCGP and ZYL on the given benchmark problems.

5. Applications in Image Denoising

Image denoising, a well-known inverse problem in the field of compressive sensing, poses significant challenges due to various sources of image noise. This noise can originate from faulty pixels in camera sensors, errors in hardware storage locations, or transmission through noisy channels. Some pixels in the image are contaminated by Gaussian noise, known as additive white Gaussian noise (AWGN), or impulse noise, known as salt-and-pepper noise. Our primary focus is on images affected by salt-and-pepper noise. This type of noise is particularly challenging because it can obscure important image details and edges, which are critical for various image processing applications such as medical imaging, remote sensing, and object recognition. In the works [30,31], a robust two-phase scheme was proposed to detect and remove salt-and-pepper noise. The first stage involves using an adaptive median filter to identify noisy pixels. The adaptive median filter is effective because it can handle varying noise densities and preserve image edges better than standard median filters. Once the noisy pixels have been detected, the second stage employs variational methods to restore the image. Variational methods are advantageous because they formulate image restoration as an optimization problem, balancing between data fidelity and the smoothness of the image. To enhance readability and comprehensiveness, we now provide an in-depth and concise explanation of this method.

Given an original image x with dimensions $m\times n$, let $x_{i,j}$ represent the grayscale level at the pixel location $(i,j)\in \mathcal{A}=\{1,2,\dots ,m\}\times \{1,2,\dots ,n\}$. To facilitate image processing and analysis, we often consider the neighborhood of each pixel. Let ${\mathcal{V}}_{i,j}$ denote the neighborhood of $(i,j)$, defined as ${\mathcal{V}}_{i,j}=\{(i,j-1),(i,j+1),(i-1,j),(i+1,j)\}$. This represents the four direct neighbors of the pixel at $(i,j)$: left, right, up, and down. A common type of noise is salt-and-pepper noise, which randomly alters the pixel values to either the minimum or maximum grayscale level, creating a “salt-and-pepper” appearance. When the image x is corrupted by salt-and-pepper noise, the observed noisy image is presented by y. The grayscale level at pixel location $(i,j)$ in the noisy image y is given by the following probabilistic model:

$$y_{i,j}=\left\{\begin{array}{cc}{x}_{i,j},\hfill & \mathrm{with}\mathrm{probability}1-r,\hfill \\ s_{min},\hfill & \mathrm{with}\mathrm{probability}p,\hfill \\ s_{max}\hfill & \mathrm{with}\mathrm{probability}q,\hfill \end{array}\right.$$

where $[s_{min},s_{max}]$ is the range of $x_{i,j}$, and $r=q+p$ represents the overall noise level. To obtain the denoised image $u^{*}$, we employ a comprehensive two-phase scheme. In the first stage, we apply an adaptive median filter to the noisy image y. This process results in an intermediate image, denoted as $\tilde{y}$. Based on the differences between the noisy image y and the filtered image $\tilde{y}$, we define the noise candidate set as follows:

$$\mathcal{N}=\{(i,j)\in \mathcal{A}:{\tilde{y}}_{i,j}\ne y_{i,j}\mathrm{and}{y}_{i,j}=s_{min}\mathrm{or}{s}_{max}\}.$$

In the second stage, we proceed with the recovery of the noisy pixels identified in the set $\mathcal{N}$. For each pixel $(i,j)\in \mathcal{N}$, if it is not contaminated by noise, we retain its original value, i.e., $u_{i,j}^{*}=y_{i,j}$. For noisy pixels $y_{i,j}$, $(i,j)\in \mathcal{N}$, we need to perform recovery. We set $u_{m,n}^{*}=y_{m,n}$ for $(m,n)\in {\mathcal{V}}_{i,j}\setminus \mathcal{N}$, ensuring that neighboring non-noisy pixels are preserved. For the pixels $(m,n)\in {\mathcal{V}}_{i,j}\cap \mathcal{N}$, which are in the neighborhood and are also candidates for noise, we must also recover their values. To restore the image, we aim to minimize the following function:

$$\underset{u}{min}E\left(u\right)=\sum _{(i,j)\in \mathcal{N}}\left\{\sum _{(m,n)\in {\mathcal{V}}_{i,j}\setminus \mathcal{N}}2{\varphi }_{\alpha }(u_{i,j}-y_{m,n})+\sum _{(m,n)\in {\mathcal{V}}_{i,j}\cap \mathcal{N}}{\varphi }_{\alpha }(u_{i,j}-u_{m,n})\right\},$$

where ${\varphi }_{\alpha }$ is an even edge-preserving potential function with parameter $\alpha$. We know from [11] that if ${\varphi }_{\alpha }$ is convex, then $\nabla E\left(u\right)$ is monotone.

We utilized the well-known grayscale test images: lighthouse (512$\times$512), peppers (256$\times$256), boat (512$\times$512), Kiel (512$\times$512), fruits (256$\times$256), brain (256$\times$256), clown (512$\times$512), couple (512$\times$512), trucks (512$\times$512), baboon (256$\times$256), Barbara (512$\times$512), and cameraman (256$\times$256). Each image was affected by 30% salt-and-pepper noise, and the experiments were repeated 10 times with different noise samples. The detailed numerical results are presented in

Table 9. Efficiency comparison for different algorithms.

Image	ITTCG	HTTCGP	ZYL
	$\overline{\mathit{Niter}}$/$\overline{\mathit{Time}}$/$\overline{\mathit{PSNR}}$/$\overline{\mathit{SSIM}}$	$\overline{\mathit{Niter}}$/$\overline{\mathit{Time}}$/$\overline{\mathit{PSNR}}$/$\overline{\mathit{SSIM}}$	$\overline{\mathit{Niter}}$/$\overline{\mathit{Time}}$/$\overline{\mathit{PSNR}}$/$\overline{\mathit{SSIM}}$
lighthouse	28.2/5.73/30.85/0.97	50.5/11.51/31.32/0.97	58.7/14.24/31.12/0.97
peppers	19.4/1.18/33.24/0.96	51.5/3.19/33.77/0.96	29.4/1.94/33.37/0.96
boat	14.4/3.19/33.96/0.98	48.0/11.00/34.46/0.98	24.4/5.98/34.08/0.98
kiel	25.5/5.25/27.81/0.97	49.0/10.93/27.94/0.97	47.5/11.37/27.88/0.97
fruits	21.3/1.33/30.01/0.94	69.3/4.34/30.25/0.95	27.1/1.80/30.01/0.94
brain	14.2/0.90/31.07/0.87	23.9/1.51/31.13/0.87	18.7/1.28/31.10/0.87
clown	14.1/3.15/36.62/0.99	42.4/9.68/37.17/0.99	24.9/6.02/36.80/0.99
couple	14.4/3.15/34.26/0.99	36.1/8.38/34.60/0.99	24.2/5.90/34.33/0.99
trucks	12.1/2.77/34.07/0.98	25.0/5.89/34.13/0.98	18.4/4.66/34.08/0.98
baboon	32.9/1.72/24.68/0.87	35.8/1.85/24.67/0.87	72.1/4.47/24.68/0.87
barbara	15.6/3.31/29.03/0.96	46.1/10.39/29.07/0.96	27.2/6.47/29.04/0.96
Cameraman	19.0/1.13/29.95/0.96	37.6/2.33/30.14/0.96	38.9/2.54/30.13/0.96

, where $\overline{\mathrm{Niter}}$, $\overline{\mathrm{Time}}$, $\overline{\mathrm{PSNR}}$, and $\overline{\mathrm{SSIM}}$ represent the number of average iterations, the average CPU time in seconds, the average peak signal-to-noise ratio, and the average structural similarity index, respectively. Additionally, we display the noisy images with 30% salt-and-pepper noise and the images restored using the ITTCG, HTTCGP, and ZYL algorithms (see Figure 4 and Figure 5). From the results in Table 9, and Figure 4 and Figure 5, we can draw the following conclusions: (i) All images affected by 30% salt-and-pepper noise were successfully recovered by the ITTCG, HTTCGP, and ZYL algorithms. (ii) With a similar average structural similarity index, Algorithm ITTCG generally required less CPU time, fewer iterations, and achieved a lower peak signal-to-noise-ratio than the HTTCGP and ZYL algorithms, indicating that Algorithm ITTCG was efficient and competitive in image denoising.

6. Conclusions

In this paper, we proposed a projection-based improved three-term conjugate gradient algorithm for solving constrained nonlinear monotone equations. Its search direction automatically satisfies the sufficient descent and trust-region properties. The global convergence of the proposed algorithm is established under the assumption that the mapping is continuous and monotonic. A notable theoretical advantage of the proposed algorithm is that it does not require Lipschitz continuity of the mapping, unlike traditional algorithms for similar problems. Numerical results on benchmark problems demonstrated the effectiveness and competitiveness of the proposed algorithm. Furthermore, the proposed algorithm could successfully recover noise images.