On the Derivation of a Fast Solver for Nonlinear Systems of Equations Utilizing Frozen Substeps with Applications

Abstract

In this manuscript, we propose a multi-step framework for solving nonlinear systems of algebraic equations. To improve the solver’s efficiency, the Jacobian matrix is held constant during the second sub-step, while a specialized strategy is applied in the third sub-step to maximize convergence speed without necessitating additional Jacobian evaluations. The proposed method achieves fifth-order convergence for simple roots, with its theoretical convergence established. Finally, computational experiments are conducted to illustrate the performance of the proposed solver in addressing nonlinear equation systems.

1. Preliminary Discussion

1.1. Nonlinear Equations

Nonlinear sets of equations are integral to numerous scientific and engineering applications, serving as fundamental tools in modeling and analysis [1,2,3,4]. These equations, due to their complexity, often defy exact analytical solutions (denoted by r), making their resolution an active area of research in applied mathematics. As a result, numerical iterative methods have become essential in approximating their solutions [5,6]. Iterative numerical methods play a vital role in resolving nonlinear sets of algebraic equations due to the inherent complexity and computational demands associated with such systems. The challenge becomes even more pronounced when dealing with high-dimensional systems where the number of equations and variables is substantial. Iterative methods provide a systematic approach to approximate the solution of nonlinear systems through successive refinements, enabling the computation of accurate results with manageable computational resources. Their ability to converge towards a solution makes iterative techniques indispensable in numerical analysis [7,8,9].

1.2. The Need in Differential Equations

Nonlinear partial differential equations (PDEs) [10,11] frequently arise in diverse scientific and engineering fields, such as fluid mechanics. These equations are generally challenging to solve analytically due to the lack of closed-form solutions, requiring the development and application of numerical solvers. One common approach involves discretizing the continuous problem using methods like finite difference (FD), which reformulates the PDE into a set of nonlinear algebraic equations. To tackle such resulting systems efficiently, advanced iterative solvers, such as the higher-order methods furnished in this study, can be employed. These solvers utilize higher-order procedures and exploit enhanced convergence features to provide fast and efficient solutions while substantially reducing calculational effort compared to conventional approaches.

Higher-order iterative solvers for solving nonlinear PDEs consist of several crucial stages. The process begins with discretizing the PDE, converting it into a large and often sparse nonlinear algebraic system. Subsequently, an initial approximation for the solution is provided, serving as a starting point for the iterative refinement.

Another advantage of iterative methods is their scalability to large-dimensional problems [12]. Consider, for instance, a system of nonlinear equations arising from modeling a three-dimensional fluid dynamics problem, such as simulating airflow over an aircraft. Such systems can involve millions of variables and equations derived from discretizing PDEs governing fluid flow. Iterative methods, on the other hand, decompose the problem into smaller, more manageable steps. By iteratively improving the solution starting from an initial guess, these methods can efficiently handle large systems while taking advantage of sparsity and other problem-specific structures to reduce computational overhead (see  [13] for more).

1.3. Our Aim and Contribution

This study is driven by the necessity to develop a robust iterative framework that emphasizes computational efficiency and precision. By drawing on foundational ideas from prior research [14,15], our approach introduces a strategy that circumvents the need for second-order Fréchet derivative calculations. This not only simplifies the computational process but also contributes to advancing numerical methodologies in the field. The core aim is to design a solver capable of handling both real and complex roots, achieving high convergence rates while minimizing computational demands such as matrix inversions and function evaluations. To enhance the solver’s efficiency, the Jacobian matrix is frozen during the second sub-step, while a strategy is employed in the third sub-step to maximize convergence speed without requiring additional Jacobian computations. This approach achieves a fifth-order convergence rate for simple roots, and its theoretical convergence has been rigorously analyzed.

1.4. Existing Approaches

The problem under consideration is formulated as follows [16,17]:

$$H\left(s\right)=0,$$

(1)

where

$$H\left(s\right)={(f_1\left(s\right),f_2\left(s\right),\dots ,f_N\left(s\right))}^T,$$

and $f_i\left(s\right)$, $1\le i\le N$ are component functions defined over a convex domain $D\subset {\mathbb{R}}^N$. To build on the current state of knowledge, we revisit and assess several classical iterative methods for solving (1).

One widely studied technique is Newton’s method (NM), which operates as follows [18,19]:

$$\begin{array}{cc}\hfill & H^{\prime }\left(w^{\left(\alpha \right)}\right){\delta }^{\left(\alpha \right)}=-H\left(w^{\left(\alpha \right)}\right),\alpha =0,1,2,\dots ,\hfill \\ \hfill & w^{(\alpha +1)}=w^{\left(\alpha \right)}+{\delta }^{\left(\alpha \right)}.\hfill \end{array}$$

(2)

NM is well known for its quadratic convergence rate, provided the initial vector $w^{\left(0\right)}$ lies sufficiently near to the simple zero. Nevertheless, its reliance on Jacobian computations can pose challenges, leading to the development of alternative methods.

It is noted that iterative methods provide flexibility in handling systems with nonlinearity and lack of symmetry, which are common in practical applications. For example, nonlinear systems often arise in optimization, structural engineering, and machine learning, where the governing equations are complex and highly sensitive to initial conditions. Iterative approaches, such as Newton’s solver and its other alternatives, can adapt to these challenges. This adaptability ensures that iterative methods remain effective across a wide spectrum of applications, making them an essential tool in computational mathematics.

Derivative-free techniques such as Steffensen’s method (SM) have been proposed as follows [20]:

$$\begin{array}{cc}\hfill & {\lambda }^{\left(\alpha \right)}=H\left(w^{\left(\alpha \right)}\right)+w^{\left(\alpha \right)},\hfill \\ \hfill & w^{(\alpha +1)}=w^{\left(\alpha \right)}-{[w^{\left(\alpha \right)},{\lambda }^{\left(\alpha \right)},H]}^{-1}H\left(w^{\left(\alpha \right)}\right),\alpha =0,1,2,\dots ,\hfill \end{array}$$

(3)

where the divided difference operator (DDO) is central to its implementation. For vectors $\theta$ and z, the DDO is defined as:

$${[\theta ,z;H]}_{i,j}={\displaystyle \frac{-H_i({\theta }_1,\dots ,{\theta }_{j-1},z_j,\dots ,z_m)+H_i({\theta }_1,\dots ,{\theta }_j,z_{j+1},\dots ,z_m)}{-z_j+{\theta }_j}}.$$

(4)

The author in [6] furnished an improvement to the NM that reaches local third order (here abbreviated as TM) as follows:

$$\left\{\begin{array}{c}{\gamma }^{\left(\alpha \right)}=w^{\left(\alpha \right)}-H^{\prime }{\left(w^{\left(\alpha \right)}\right)}^{-1}H\left(w^{\left(\alpha \right)}\right),\hfill \\ w^{(\alpha +1)}={\gamma }^{\left(\alpha \right)}-H^{\prime }{\left(w^{\left(\alpha \right)}\right)}^{-1}H\left({\gamma }^{\left(\alpha \right)}\right).\hfill \end{array}\right.$$

(5)

Despite the utility of the above methods, certain drawbacks persist, such as the formation of dense matrices during DDO computations, which limit their scalability to large systems.

1.5. Outlines

Motivated by these challenges, this manuscript proposes an innovative multistep solver with fifth-order convergence rates. The solver is particularly designed to eliminate the need for second-order Fréchet derivatives. The structure of the manuscript is given as follows. Section 2 introduces the design of a proposed multistep solver, highlighting its computational efficiency through reduced LU decompositions. Section 3 presents a detailed theoretical error analysis to confirm the convergence properties. Section 4 applies the presented solver to benchmark test cases. Section 5 concludes with a summary of observations and suggests potential directions for future investigations.

2. A Novel Iterative Framework

This section introduces a refined iterative methodology derived from the foundational techniques outlined in Section 1. The proposed scheme utilizes multiple steps while maintaining a fixed Jacobian matrix to achieve improved computational performance. We first consider the the following three-step strategy:

$$\left\{\begin{array}{c}{y}^{\left(\alpha \right)}=w^{\left(\alpha \right)}-H^{\prime }{\left(w^{\left(\alpha \right)}\right)}^{-1}H\left(w^{\left(\alpha \right)}\right),\hfill \\ {\mu }^{\left(\alpha \right)}=y^{\left(\alpha \right)}-H^{\prime }{\left(y^{\left(\alpha \right)}\right)}^{-1}H\left(y^{\left(\alpha \right)}\right),\hfill \\ w^{(\alpha +1)}={\mu }^{\left(\alpha \right)}-H^{\prime }{\left({\mu }^{\left(\alpha \right)}\right)}^{-1}H\left({\mu }^{\left(\alpha \right)}\right).\hfill \end{array}\right.$$

(6)

Evidently, (6) is a three-step structure of (2) and reaches the eighth order of convergence, but requires Jacobian computation three times and makes no meaningful advance in contrast to (2). To improve this, we follow the Traub’s idea in (5) and consider a frozen structure as follows:

$$\left\{\begin{array}{c}{y}^{\left(\alpha \right)}=w^{\left(\alpha \right)}-H^{\prime }{\left(w^{\left(\alpha \right)}\right)}^{-1}H\left(w^{\left(\alpha \right)}\right),\hfill \\ {\mu }^{\left(\alpha \right)}=y^{\left(\alpha \right)}-H^{\prime }{\left(w^{\left(\alpha \right)}\right)}^{-1}H\left(y^{\left(\alpha \right)}\right),\hfill \\ w^{(\alpha +1)}={\mu }^{\left(\alpha \right)}-H^{\prime }{\left(w^{\left(\alpha \right)}\right)}^{-1}H\left({\mu }^{\left(\alpha \right)}\right).\hfill \end{array}\right.$$

(7)

The scheme (8) is not that useful since it follows (5) and reaches only the fourth convergence speed. To furnish a useful technique, a DDO is designed carefully in the third substep. For such a target, we first consider:

$$\left\{\begin{array}{c}{y}^{\left(\alpha \right)}=w^{\left(\alpha \right)}-H^{\prime }{\left(w^{\left(\alpha \right)}\right)}^{-1}H\left(w^{\left(\alpha \right)}\right),\hfill \\ {\mu }^{\left(\alpha \right)}=y^{\left(\alpha \right)}-H^{\prime }{\left(w^{\left(\alpha \right)}\right)}^{-1}H\left(y^{\left(\alpha \right)}\right),\hfill \\ w^{(\alpha +1)}={\mu }^{\left(\alpha \right)}-DDO{1}^{-1}{H}^{\prime }{\left(w^{\left(\alpha \right)}\right)}^{-1}H\left({\mu }^{\left(\alpha \right)}\right),\hfill \end{array}\right.$$

(8)

where for the sake of simplicity in the scalar case, the DDO1 is given by:

$$DDO1={\displaystyle \frac{A_{1}H\left({\mu }^{\left(\alpha \right)}\right)-A_{2}H\left(y^{\left(\alpha \right)}\right)}{{\mu }^{\left(\alpha \right)}-y^{\left(\alpha \right)}}}.$$

(9)

Here, $A_1$ and $A_2$ are two appropriate parameters that must be set using the knowledge of the previous substeps to gain as must possible of convergence order. We choose:

$$A_1=A_2=H^{\prime }{\left(w^{\left(\alpha \right)}\right)}^{-1}.$$

(10)

Since the Jacobian $H^{\prime }\left(w^{\left(\alpha \right)}\right)$ has already been computed in the first substep, it does not add extra new functional or derivative evaluations to the structure of the proposed method. Hence, the final formulation is expressed as follows (CS5):

$$\left\{\begin{array}{c}{y}^{\left(\alpha \right)}=w^{\left(\alpha \right)}-H^{\prime }{\left(w^{\left(\alpha \right)}\right)}^{-1}H\left(w^{\left(\alpha \right)}\right),\hfill \\ {\mu }^{\left(\alpha \right)}=y^{\left(\alpha \right)}-H^{\prime }{\left(w^{\left(\alpha \right)}\right)}^{-1}H\left(y^{\left(\alpha \right)}\right),\hfill \\ w^{(\alpha +1)}={\mu }^{\left(\alpha \right)}-{\left[{\displaystyle \frac{H^{\prime }{\left(w^{\left(\alpha \right)}\right)}^{-1}H\left({\mu }^{\left(\alpha \right)}\right)-H^{\prime }{\left(w^{\left(\alpha \right)}\right)}^{-1}H\left(y^{\left(\alpha \right)}\right)}{{\mu }^{\left(\alpha \right)}-y^{\left(\alpha \right)}}}\right]}^{-1}{H}^{\prime }{\left(w^{\left(\alpha \right)}\right)}^{-1}H\left({\mu }^{\left(\alpha \right)}\right).\hfill \end{array}\right.$$

(11)

In practical computations, direct matrix inversion is computationally expensive and should be avoided. Instead, LU decomposition is employed to solve the associated linear systems. Consequently, during the implementation of CS5, several distinct linear systems are solved at each iteration to estimate the resolution of the nonlinear problem (1):

$$\left\{\begin{array}{c}{H}^{\prime }\left(w^{\left(\alpha \right)}\right)C_1^{\left(\alpha \right)}=H\left(w^{\left(\alpha \right)}\right),\hfill \\ H^{\prime }\left(w^{\left(\alpha \right)}\right)C_2^{\left(\alpha \right)}=H\left(y^{\left(\alpha \right)}\right),\hfill \\ H^{\prime }\left(w^{\left(\alpha \right)}\right)C_3^{\left(\alpha \right)}=H\left({\mu }^{\left(\alpha \right)}\right),\hfill \\ [y^{\left(\alpha \right)},{\mu }^{\left(\alpha \right)},H]C_4^{\left(\alpha \right)}=H\left({\mu }^{\left(\alpha \right)}\right),\hfill \end{array}\right.$$

(12)

Rewriting the iterative schemes for practical implementation, CS5 is reformulated as:

$$\left\{\begin{array}{c}{y}^{\left(\alpha \right)}=w^{\left(\alpha \right)}-C_1^{\left(\alpha \right)},\hfill \\ {\mu }^{\left(\alpha \right)}=y^{\left(\alpha \right)}-C_2^{\left(\alpha \right)},\hfill \\ w^{(\alpha +1)}={\mu }^{\left(\alpha \right)}-C_4^{\left(\alpha \right)}.\hfill \end{array}\right.$$

(13)

This acceleration improves computational efficiency, particularly for complex and high-dimensional problems. The efficiency of a numerical algorithm is a cornerstone of computational mathematics, dictated by its convergence rate and structural design relative to computational costs. The quality of a solver is evaluated based on these factors, considering the number of function evaluations, Jacobian computations, and LU decompositions. This will be discussed later at the beginning of Section 4.

3. Investigating the Convergence Rate

Here, we conduct a convergence analysis to determine the convergence rates of the solver (13), or equivalently (11). Before presenting the main analytical order, the N-D Taylor series is furnished. The convergence rate for the solver CS5 is derived using the N-dimensional Taylor series. Let ${\mathrm{err}}^{\left(\alpha \right)}=w^{\left(\alpha \right)}-r$ show the error at the $\alpha$-th loop. As detailed in [21], the following relationship holds:

$${\mathrm{err}}^{(\alpha +1)}=\left(F\right){\mathrm{err}}^{{\left(\alpha \right)}^p}+O\left({\mathrm{err}}^{{\left(\alpha \right)}^{p+1}}\right),$$

(14)

where F exhibits the properties of a p-linear map, and $F\in \mathcal{H}({\mathbb{R}}^N,{\mathbb{R}}^N,\cdots ,{\mathbb{R}}^N)$. Additionally, the following holds:

$${\mathrm{err}}^{{\left(\alpha \right)}^p}=\left(\underset{p\mathrm{terms}}{\underbrace{{\mathrm{err}}^{\left(\alpha \right)},{\mathrm{err}}^{\left(\alpha \right)},\dots ,{\mathrm{err}}^{\left(\alpha \right)}}}\right).$$

(15)

Assuming that $H:D\subseteq {\mathbb{R}}^N\to {\mathbb{R}}^N$ is Fréchet differentiable over D [22], the m-th derivative of H at $u\in {\mathbb{R}}^N$ for $m\ge 1$ can be expressed as an m-linear map, i.e.,

$$H^{\left(m\right)}\left(u\right):{\mathbb{R}}^N\times \cdots \times {\mathbb{R}}^N\to {\mathbb{R}}^N,$$

(16)

such that $H^{\left(m\right)}\left(u\right)(v_1,\dots ,v_m)\in {\mathbb{R}}^N$. For $r+h\in {\mathbb{R}}^N$ near r, the Taylor series expansion is given by [22]:

$$H(r+h)=H^{\prime }\left(r\right)\left[h+\sum _{m=2}^{p-1}{\mathrm{\Xi }}_m{h}^m\right]+O\left(h^p\right),$$

(17)

wherein

$${\mathrm{\Xi }}_m={\displaystyle \frac{1}{m!}}{\left[H^{\prime }\left(r\right)\right]}^{-1}{H}^{\left(m\right)}\left(r\right),m\ge 2.$$

(18)

It follows that ${\mathrm{\Xi }}_m{h}^m\in {\mathbb{R}}^N$ and ${\left[H^{\prime }\left(r\right)\right]}^{-1}\in \mathcal{H}\left({\mathbb{R}}^N\right)$. Moreover, for $H^{\prime }(·)$, the following expansion holds:

$$H^{\prime }(r+h)=H^{\prime }\left(r\right)\left[I+\sum _{m=2}^{p-1}m{\mathrm{\Xi }}_m{h}^{m-1}\right]+O\left(h^p\right),$$

(19)

whereas I is the unit matrix, and $m{\mathrm{\Xi }}_m{h}^{m-1}\in \mathcal{H}\left({\mathbb{R}}^N\right)$.

Theorem 1. 

Let $H:D\subseteq {\mathbb{R}}^N\to {\mathbb{R}}^N$ be suitably smooth in the Fréchet sense at each node in D, and suppose $H\left(r\right)=0$ at some $r\in {\mathbb{R}}^N$. Furthermore, assume that $H^{\prime }\left(s\right)$ is smooth and invertible at r. Then, the sequence ${\left\{w^{\left(\alpha \right)}\right\}}_{\alpha \ge 0}$ via CS5 locally tends to r with 5th-rate.

Proof. 

To establish the convergence rate, we use the expansions from (17) and (19) to obtain the following expressions:

$$H\left(w^{\left(\alpha \right)}\right)=H^{\prime }\left(r\right)\left[{\mathrm{err}}^{\left(\alpha \right)}+{\mathrm{\Xi }}_2{{\mathrm{err}}^{\left(\alpha \right)}}^2+{\mathrm{\Xi }}_3{{\mathrm{err}}^{\left(\alpha \right)}}^3+{\mathrm{\Xi }}_4{{\mathrm{err}}^{\left(\alpha \right)}}^4+{\mathrm{\Xi }}_5{{\mathrm{err}}^{\left(\alpha \right)}}^5\right]+O\left({{\mathrm{err}}^{\left(\alpha \right)}}^6\right),$$

(20)

and

$$H^{\prime }\left(w^{\left(\alpha \right)}\right)=H^{\prime }\left(r\right)\left[I+2{\mathrm{\Xi }}_2{\mathrm{err}}^{\left(\alpha \right)}+3{\mathrm{\Xi }}_3{{\mathrm{err}}^{\left(\alpha \right)}}^2+4{\mathrm{\Xi }}_4{{\mathrm{err}}^{\left(\alpha \right)}}^3+5{\mathrm{\Xi }}_5{{\mathrm{err}}^{\left(\alpha \right)}}^4\right]+O\left({{\mathrm{err}}^{\left(\alpha \right)}}^5\right),$$

(21)

where ${\mathrm{\Xi }}_i={\displaystyle \frac{1}{i!}}{\left[H^{\prime }\left(r\right)\right]}^{-1}{H}^{\left(i\right)}\left(r\right)$ for $i\ge 2$, and ${\mathrm{err}}^{\left(\alpha \right)}=w^{\left(\alpha \right)}-r$. Using these expansions, we derive:

$$y^{\left(\alpha \right)}=r+{\displaystyle \frac{1}{3}}{\mathrm{err}}^{\left(\alpha \right)}+{\displaystyle \frac{2}{3}}{\mathrm{\Xi }}_2{{\mathrm{err}}^{\left(\alpha \right)}}^2+O\left({{\mathrm{err}}^{\left(\alpha \right)}}^3\right).$$

(22)

A similar Taylor expansion using (20)–(22) gives the error at the end of the second sub-step of CS5 as follows:

$${\mu }^{\left(\alpha \right)}-r=\left({\mathrm{\Xi }}_2^2\right){{\mathrm{err}}^{\left(\alpha \right)}}^3+\left(-3{\mathrm{\Xi }}_2^3+3{\mathrm{\Xi }}_3{\mathrm{\Xi }}_2\right){{\mathrm{err}}^{\left(\alpha \right)}}^4+\cdots +O\left({{\mathrm{err}}^{\left(\alpha \right)}}^6\right).$$

(23)

Subsequently, we obtain:

$$H\left({\mu }^{\left(\alpha \right)}\right)=H^{\prime }\left(r\right)\left[{\mathrm{\Xi }}_3{{\mathrm{err}}^{\left(\alpha \right)}}^3+{\mathrm{\Xi }}_4{{\mathrm{err}}^{\left(\alpha \right)}}^4+\cdots \right]+O\left({{\mathrm{err}}^{\left(\alpha \right)}}^6\right).$$

(24)

By employing these expressions in the 3rd sub-step of CS5, we obtain:

$${\mathrm{err}}^{(\alpha +1)}=2{\mathrm{\Xi }}_2^4{{\mathrm{err}}^{\left(\alpha \right)}}^5+O\left({{\mathrm{err}}^{\left(\alpha \right)}}^6\right),$$

(25)

which demonstrates that the method achieves a fifth-order, requiring only two LU decompositions for every iterates. This completes the proof.    □

4. Numerical Tests

The primary objective here is to demonstrate the applicability and efficacy of the proposed CS5 scheme. Some remarks are given as:

• Computational experiments were performed using Wolfram software 13.3 [23,24], with all programming conducted in a multiple-precision arithmetic framework using 2500-digit precision except the last test. This setup allowed us to rigorously observe and verify the higher speeds reflected in the computational pieces of evidence.
• Linear systems arising during the process were efficiently solved through LU factorization using $\mathtt{LinearSolver}\left[\right]$.
• All numerical tests were carried out in a consistent computational environment to ensure reliability and reproducibility.
• For comparative analysis, the proposed higher-order method was benchmarked against existing approaches, including NM and SM, for resolving nonlinear collections of equations.

Prior to presenting the numerical results, we highlight the efficiency indices (EI) of the solvers under comparison. Attaining fifth-order convergence with the CS5 scheme requires solving three linear systems per iteration cycle. However, computational efficiency is enhanced by requiring only two LU factorizations per cycle, followed by matrix-vector multiplications, thereby eliminating the need for repeated matrix inversions within each iteration. The efficiency index, as defined in [20], is calculated as:

$$EI=\sqrt[\mathcal{C}]{\mathfrak{p}},$$

(26)

whereas

• $\mathcal{C}$ shows the cost, and 
• $\mathfrak{p}$ shows the rate in terms of function evaluations.

For evaluating the EI of the CS5 scheme, we detail the required functional evaluation costs in $\nu$-dimensional functions and their Jacobians as follows: For $H\left(w\right),H\left(w\right),H\left(w\right)$, $\nu$ evaluations are required for each. For $H^{\prime }\left(w\right)$, ${\nu }^2$ evaluations are needed. Similarly, for the divided difference operator (DDO1), ${\nu }^2$ evaluations are also involved.

The results summarized in

Table 1. Comparison of various techniques based on their efficiency indices.

Techniques	NM	SM	PM
$\mathfrak{p}$	2	2	5
Functional evaluations	$\nu +{\nu }^2$	$\nu +{\nu }^2$	$3\nu +2{\nu }^2$
No. of LU factorization	1	1	2
The classic EI	$2^{{\displaystyle \frac{1}{\nu +{\nu }^2}}}$	$2^{{\displaystyle \frac{1}{\nu +{\nu }^2}}}$	$5^{{\displaystyle \frac{1}{3\nu +2{\nu }^2}}}$

clearly show that when changing $\nu$, the EI of the proposed solver surpasses that of its competitors. A comparative analysis of different solvers, visualized in Figure 1, underscores the superiority of the CS5 scheme.

Example 1. 

The nonlinear set $H\left(s\right)=0$, which encompasses a real root, is analyzed in detail as described in the following

$$H\left(s\right)=\left\{\begin{array}{c}sin(s_1+s_2-s_3),\hfill \\ cos(s_1-s_2),\hfill \\ s_1^2+s_2^2-s_3,\hfill \end{array}\right.$$

(27)

where its simple root is:

$$\begin{array}{ccc}r\hfill & \simeq \hfill & {(3.7177,2.1470,18.431)}^{*}.\hfill \end{array}$$

(28)

The computational results are presented in Figure 2, Figure 3 and Figure 4. Here, different starting points have been compared for any of the figures to reveal the usefulness of the new method. Furthermore, the norm of residual is expressed through the use of the ${\parallel ·\parallel }_2$ notation, providing a concise representation of the error magnitude.

A simple yet effective implementation for CS5 in Example 1 has now been provided in the Mathematica environment to make the process easier for readers as follows (noting that the notations may not be necessarily the same as the mathematical notations in the derivation of the method):

ClearAll["Global‘*"];

digits = 2500; size = 3;

f[{x1_, x2_, x3_}] :=

 {

  Sin[x1 + x2 - x3],

  Cos[x1 - x2],

  x1^2 + x2^2 - x3

  };

J[{x1_, x2_, x3_}] = D[f[{x1, x2, x3}], {{x1, x2, x3}}];

x = SetAccuracy[{6, 4, 2}, digits];

$$

max = 6; fx = f[x];

$$

point1[j_] := Join[x1[[;; j]], w[[j + 1 ;;]]];

point2[j_] := Join[x1[[;; j - 1]], w[[j ;;]]];

$$

Do[{

  f1x = J[x];

  yy = LinearSolve[f1x];

  w = x - yy[fx];

  fw = f[w];

  x1 = w - yy[fw];

  fx1 = f[x1];

  T = Transpose@

    SparseArray@

    Table[(f[point1[j]] - f[point2[j]])/(x1[[j]] - w[[j]]), {j,

      size}];

  fun = LinearSolve[T];

  x = x1 - fun[(fx1)];

  fx = f[x];

  L[i] = Norm[fx, 2];

  }, {i, 1, max}

 ]; // AbsoluteTiming

$$

s3 = Table[N[L[k], 5], {k, 1, max}]

$$

N[x, 5]

Example 2. 

The performance and accuracy of the presented scheme is validated by solving the following nonlinear test problem: $H(s_1,s_2)=0$, where $H:(0,30)\times (0,30)\to {\mathbb{R}}^2$. The function H is defined as:

$$H\left(s\right)=H(s_1,s_2)=\left(s_1^2-s_2-19,{\displaystyle \frac{s_2^3}{6}}-s_1^2+s_2-17\right).$$

(29)

The iterative process begins with the initial vector $w^{\left(0\right)}={(40.5,-5.5)}^{*}$, with the objective of converging to the exact solution $r={(5,6)}^{*}$.

Numerical experiments, presented in

Table 2. A presentation of the numerical outcomes obtained from the application of various methods in Example 2.

Methods	$\parallel \mathit{H}\left({\mathit{w}}^{\left(6\right)}\right)\parallel$	$\parallel \mathit{H}\left({\mathit{w}}^{\left(7\right)}\right)\parallel$	$\parallel \mathit{H}\left({\mathit{w}}^{\left(8\right)}\right)\parallel$	$\parallel \mathit{H}\left({\mathit{w}}^{\left(9\right)}\right)\parallel$
NM	$8.69\times {10}^1$	$1.88\times {10}^1$	$2.08$	$3.78\times {10}^{-2}$
SM	$2.04\times {10}^6$	$1.93\times {10}^6$	$1.82\times {10}^6$	$1.70\times {10}^6$
CS5	$7.11\times {10}^{-6}$	$2.72\times {10}^{-34}$	$2.24\times {10}^{-176}$

, highlight the robustness of the proposed scheme in solving a variety of nonlinear equations. These computational tests reveal that the CS5 scheme converges to the solutions in only a few iterations while delivering high accuracy at every iterate. Furthermore, the numerical rate consistently aligns with the analytically expected higher speeds, confirming the reliability and efficiency of the method.

Example 3. 

To have a test problem of higher dimensions, we take into account

$$\left\{\begin{array}{c}{s}_i{s}_{i+1}-1=0,i=1,2,\dots ,N-1,\hfill \\ s_N{s}_1-1=0,\hfill \end{array}\right.$$

(30)

in which the simple root is $r={(1,\dots ,1)}^T$ for odd N. Here we test the performance of CS5 for $N=199$ in three different initial guesses.

The results for this problem are illustrated in Figure 5, re-verifying the effectiveness of the proposed procedure for higher dimensional test problems. The fast convergence for CS5 in machine precision can be observed for all the considered initial guesses.

Discussion

In this section, we illustrate the performance of the proposed solver, CS5, in solving systems of nonlinear equations. Our observations indicate that the CS5 scheme demonstrates rapid convergence for both low-dimensional and high-dimensional systems. Moreover, in several cases, it exhibits a superior convergence radius compared to classical methods such as the NM and the SM. The proposed scheme comprises three steps, utilizing a single Jacobian matrix for the first two steps and a DDO in the final sub-step to achieve local fifth-order convergence.

5. Summary

In this work, we have introduced and analyzed a novel iteration solver aimed at resolving collections of nonlinear equations efficiently and accurately. Building upon existing techniques, we have furnished an enhanced multi-step scheme, denoted as CS5, that incorporates fixed Jacobian approximations to balance computational cost and convergence performance. Through careful design and theoretical underpinnings, these method has demonstrated fifth-order convergence, achieving higher efficiency while requiring fewer computations per iteration compared to traditional approaches. The analysis has relied on the Taylor series expansion and Frèchet differentiability, which have provided the foundation for establishing convergence rates.

Throughout this study, we have emphasized the balance between computational efficiency and accuracy, showcasing that our methods align well with the principles of numerical optimization. The proposed iterative schemes not only reduce the need for matrix inversion by adopting LU decomposition techniques but also maintain a high degree of flexibility in practical implementation. This balance ensures their suitability for a wide range of nonlinear problems, making them valuable tools in computational mathematics. We have demonstrated the capacity of a fifth-order iterative method to offer competitive solutions despite the existence of higher-order methods. By strategically freezing or updating the derivative operator at critical stages, our method has exhibited the ability to expand the convergence domain while maintaining simplicity and practical relevance. Future directions might be focused on several aspects. First, it is of theoretical interest to investigate how freezing the DDO at the third substep for CS5 can impact the convergence order when it is frozen an arbitrary multi-step format. Additionally, studying the semi local convergence of the scheme in order to relax the assumption of the smoothness of the function can be investigated further.