Solving Nonlinear Equation Systems via a Steffensen-Type Higher-Order Method with Memory

Abstract

This article introduces a multi-step solver for sets of nonlinear equations. To achieve this, we consider and develop a multi-step Steffensen-type method without memory, which does not require evaluations of the Fréchet derivatives, and subsequently extend it to a method with memory. The resulting order is $\sqrt{5}+2$, utilizing the identical number of functional evaluations as the solver without memory, thereby demonstrating a higher computational index of efficiency. Finally, we illustrate the advantages of the proposed scheme with memory through various test problems.

1. Introductory Notes

Consider a set of square algebraic nonlinear problems as [1,2]:

$$H\left(\eta \right)=0,$$

(1)

wherein $H\left(\eta \right)={({\nu }_1\left(\eta \right),{\nu }_2\left(\eta \right),\dots ,{\nu }_{\omega }\left(\eta \right))}^T$ and ${\nu }_i\left(\eta \right)$, $1\le i\le \omega$ are coordinate functions. Assume that $H\left(\eta \right)$ is an enough differentiable function of $\eta$ within a convex open set denoted by $D\subseteq {\mathbb{R}}^{\omega }$. Now we first revisit some pioneering iterative methods to address (1). Newton’s method (NM), widely used for such problems, is formulated as follows [3]:

$$\left\{\begin{array}{c}{H}^{\prime }\left(g^{\left(\chi \right)}\right){\delta }^{\left(\chi \right)}=-H\left(g^{\left(\chi \right)}\right),\chi =0,1,2,\dots ,\hfill \\ g^{(\chi +1)}={\delta }^{\left(\chi \right)}+g^{\left(\chi \right)}.\hfill \end{array}\right.$$

(2)

This approach achieves second-order convergence, provided that the starting vector $g^{\left(0\right)}$ is adequately near to the true root $\theta$. To overcome certain constraints associated with the NM, alternative solvers [4,5] have been developed, including Steffensen’s method (SM), which operates without the need for derivative calculations [6]. The formulation of the SM for addressing nonlinear systems is outlined in [7]:

$$\left\{\begin{array}{c}{\lambda }^{\left(\chi \right)}=H\left(g^{\left(\chi \right)}\right)+g^{\left(\chi \right)},\hfill \\ g^{(\chi +1)}=-{[g^{\left(\chi \right)},{\lambda }^{\left(\chi \right)};H]}^{-1}H\left(g^{\left(\chi \right)}\right)+g^{\left(\chi \right)},\chi =0,1,2,\dots ,\hfill \end{array}\right.$$

(3)

which utilizes the divided difference operator (DDO). The first-order DDO of H for high-dimensional knots $\varsigma$ and $\zeta$ is defined via ($1\le i,j\le \omega$):

$${[\varsigma ,\zeta ;H]}_{i,j}={\displaystyle \frac{-H_i({\varsigma }_1,\dots ,{\varsigma }_{j-1},{\zeta }_j,\dots ,{\zeta }_{\omega })+H_i({\varsigma }_1,\dots ,{\varsigma }_j,{\zeta }_{j+1},\dots ,{\zeta }_{\omega })}{-{\zeta }_j+{\varsigma }_j}}.$$

(4)

More generally, the DDO for H on ${\mathbb{R}}^{\omega }$ can be defined as [7]:

$$[·,·;H]:D\subset {\mathbb{R}}^{\omega }\times {\mathbb{R}}^{\omega }\to \mathcal{L}\left({\mathbb{R}}^{\omega }\right),$$

(5)

that satisfies $[\zeta ,\varsigma ;H](-\varsigma +\zeta )=H\left(\zeta \right)-H\left(\varsigma \right),\forall \varsigma ,\zeta \in D.$ Using $h=\zeta -\varsigma$, the first-order DDO can also be given by [8]:

$$[\varsigma +h,\varsigma ;H]={\int }_0^1{H}^{\prime }(\varsigma +th)dt.$$

(6)

However the definitions (4)–(6) mainly yield in dense matrices for the representation of the DDO, which restrict the applicability of the SM for tackling (1) to some extent.

The author in [9] provided another procedure to compute the DDO in a similar way as follows:

$$\left\{\begin{array}{c}[g^{\left(\chi \right)}+F^{\left(\chi \right)},g^{\left(\chi \right)};H]=\hfill \\ (H(g^{\left(\chi \right)}+F^{\left(\chi \right)}{e}^1)-H\left(g^{\left(\chi \right)}\right),\dots ,H(g^{\left(\chi \right)}+F^{\left(\chi \right)}{e}^{\omega })-H\left(g^{\left(\chi \right)}\right)){F^{\left(\chi \right)}}^{-1},\hfill \end{array}\right.$$

(7)

with $F^{\left(\chi \right)}=\mathrm{diag}({\nu }_1\left(g^{\left(\chi \right)}\right),{\nu }_2\left(g^{\left(\chi \right)}\right),\dots ,{\nu }_{\omega }\left(g^{\left(\chi \right)}\right))$. Traub (TM) introduced an improvement to NM with local cubic convergence [9]:

$$\left\{\begin{array}{c}{\gamma }^{\left(\chi \right)}=g^{\left(\chi \right)}-H^{\prime }{\left(g^{\left(\chi \right)}\right)}^{-1}H\left(g^{\left(\chi \right)}\right),\hfill \\ g^{(\chi +1)}={\gamma }^{\left(\chi \right)}-H^{\prime }{\left(g^{\left(\chi \right)}\right)}^{-1}H\left({\gamma }^{\left(\chi \right)}\right).\hfill \end{array}\right.$$

(8)

Another well-known and effective method for resolving (1) is the fourth-order Jarratt technique (JM) [10,11], which is described as:

$$\left\{\begin{array}{c}{y}^{\left(\chi \right)}=g^{\left(\chi \right)}-{\displaystyle \frac{2}{3}}{H}^{\prime }{\left(g^{\left(\chi \right)}\right)}^{-1}H\left(g^{\left(\chi \right)}\right),\hfill \\ g^{(\chi +1)}=g^{\left(\chi \right)}-{\displaystyle \frac{1}{2}}{(3H^{\prime }\left(y^{\left(\chi \right)}\right)-H^{\prime }\left(g^{\left(\chi \right)}\right))}^{-1}\hfill \\ ·(3H^{\prime }\left(y^{\left(\chi \right)}\right)+H^{\prime }\left(g^{\left(\chi \right)}\right))H^{\prime }{\left(g^{\left(\chi \right)}\right)}^{-1}H\left(g^{\left(\chi \right)}\right).\hfill \end{array}\right.$$

(9)

An improvement of (3) was furnished in [12,13] as:

$$\left\{\begin{array}{c}{z}^{\left(\chi \right)}=g^{\left(\chi \right)}-{[g^{\left(\chi \right)},w^{\left(\chi \right)};H]}^{-1}H\left(g^{\left(\chi \right)}\right),\hfill \\ g^{(\chi +1)}=z^{\left(\chi \right)}-{[g^{\left(\chi \right)},w^{\left(\chi \right)};H]}^{-1}H\left(z^{\left(\chi \right)}\right),\hfill \end{array}\right.$$

(10)

where

$$w^{\left(\chi \right)}=g^{\left(\chi \right)}+\vartheta H\left(g^{\left(\chi \right)}\right),\vartheta \in \mathbb{R}.$$

(11)

The distinction in (10), as opposed to (3), lies in its utilization of two iterations and consequently two $\omega$-D function assessments, enabling it to achieve a rate superior to quadratic. The core concept is to stabilize the DDO during each cycle and subsequently augment the substeps to maximize order enhancement and contribute to the computational efficiency index (CEI) of the solver.

This manuscript is motivated by the objective of creating a multi-step fast iterative solver that improves both accuracy and efficiency in addressing nonlinear equation sets via (10). By eliminating the reliance on Fréchet derivatives, our proposed method based on an extension over TM seeks to alleviate the complexities and computational demands associated with derivative-involved approaches, thereby advancing the domain of this field. The primary objective is to furnish a higher-order derivative-free Steffensen-type solver capable of addressing nonlinear systems, encompassing both complex and real solutions. Our intention is to enhance computational efficiency by reducing the frequency of matrix inversions and functional evaluations, in accordance with the principles of numerical analysis. In this work, functional evaluations refer to both function and derivative evaluations, which differ from the concept typically used in the Calculus of Variations.

This article is constructed as comes next. Section 2 explores the memorization technique utilized in the Steffensen-type scheme for addressing nonlinear sets of equations. Section 3 formulates a multistep approach comprising several substeps to achieve rapid convergence while minimizing the number of LU decompositions using a with memory structure to accelerate the convergence as much as possible. Section 4 presents an error analysis and assesses the rate of convergence. Subsequently, Section 5 examines the CEI of various methods, focusing on the number of functional assessments and the flops-type index. Furthermore, Section 6 demonstrates the applicability and advantages of the proposed method with memory through its application to several problems. Finally, Section 7 offers concluding remarks.

2. With Memorization of the Iterative Methods

In this context, our objective is to enhance the CEI of (10) without adding additional steps or further DDOs for every iterate; see [14]. To achieve this, we utilize the concept of memory-based methods, which suggest that the speed of convergence and overall efficiency of iterative techniques can be enhanced by retaining and utilizing previously calculated function values.

Noting that the error equation for (10) (the notations here will be pointed out further in Section 3):

$${\epsilon }^{(\chi +1)}=(I+\vartheta A^{\prime }\left(\theta \right))(2I+\vartheta A^{\prime }\left(\theta \right))C_2{\epsilon }^{{\left(\chi \right)}^3}+\mathcal{O}\left({\epsilon }^{{\left(\chi \right)}^4}\right),$$

(12)

contains a term expressed as follows:

$$I+\vartheta H^{\prime }\left(\theta \right)=0.$$

(13)

Here the non-zero scalar $\vartheta$ in (13) significantly influences both the convergence domain and the enhancement of the convergence speed. When addressing a nonlinear set of problems, and since $\theta$ is unknown, one could approximate $H^{\prime }\left(\theta \right)$ to bring the entire relation in (13) close to zero. Thus, we can express this as

$$\vartheta \simeq -H^{\prime }{\left(\overline{\theta }\right)}^{-1},$$

(14)

where $\overline{\theta }$ represents an estimation of the root (for each iteration).

It is crucial to elaborate on how one estimates the matrix $\vartheta :=A^{\left(\chi \right)}$, $(\chi \ge 1)$ by utilizing certain approximations of $-H^{\prime }\left(\theta \right)$ derived from the available data [15].

To enhance the performance of (10) through the principle of memory-based methods [13], we take into account the following iterative expression without memory shown as PM1:

$$\left\{\begin{array}{cc}\hfill w^{\left(\chi \right)}& =g^{\left(\chi \right)}+\vartheta H\left(g^{\left(\chi \right)}\right)\hfill \\ \hfill q^{\left(\chi \right)}& =g^{\left(\chi \right)}-{[g^{\left(\chi \right)},w^{\left(\chi \right)};H]}^{-1}H\left(g^{\left(\chi \right)}\right),\hfill \\ \hfill z^{\left(\chi \right)}& =q^{\left(\chi \right)}-{[g^{\left(\chi \right)},w^{\left(\chi \right)};H]}^{-1}H\left(q^{\left(\chi \right)}\right),\hfill \\ \hfill g^{(\chi +1)}& =z^{\left(\chi \right)}-{[g^{\left(\chi \right)},w^{\left(\chi \right)};H]}^{-1}H\left(z^{\left(\chi \right)}\right).\hfill \end{array}\right.$$

(15)

This solver reads the following error equation:

$${\epsilon }^{(\chi +1)}=(I+\vartheta H^{\prime }\left(\theta \right))(2I+\vartheta H^{\prime }\left(\theta \right))C_2^2{\epsilon }^{{\left(\chi \right)}^4}+\mathcal{O}({\epsilon }^{{\left(\chi \right)}^5}).$$

(16)

Without loss of generality, we focus on the scalar case to analyze the dynamical behavior of the iterative methods in the complex plane, rather than extending to the multi-dimensional case for (19). Visualizing the fractal attraction basins of iterative methods for polynomial equations of various degrees in the complex plane is critical for several reasons [16,17], particularly when shading the plot based on the number of iterations required for convergence. In this context, different polynomial roots correspond to distinct regions of attraction. By mapping these basins, one can illustrate where initial guesses converge to specific roots. This step is essential in our work, demonstrating how the use of memory and small free parameter values can expand the convergence radii, thereby enlarging the region for selecting initial approximations.

Shading the plot based on the number of iterations needed for convergence offers insights into the solver’s effectiveness. Regions where the method converges rapidly can be identified as more stable or efficient, whereas areas requiring more iterations (or failing to converge) suggest potential inefficiencies or instability. Such analyses are illustrated in Figure 1, Figure 2, Figure 3 and Figure 4 over the domain $[-2,2]\times [-2,2]$, with a maximum iteration count of 150 and a tolerance of ${10}^{-2}$ for the residual as the stopping criterion. They reveal that the higher the order is for Steffensen-type methods, the larger the attraction basin is. Note that PM1 and PM2 both are Steffensen-type methods. In these new methods, the number of iterations to get the root is lower than SM; due to this, they have lighter and fewer shaded areas in their attraction basins. Moreover, the convergence radii could be enlarged by selecting small values for the free non-zero scalar $\vartheta$. Thus, memorization will not only contribute to a higher-efficiency index but also to larger attraction basins, which means higher stability for such a solver in contrast to the Steffensen-type method without memory.

Figure 1, Figure 2, Figure 3 and Figure 4 also reveal that by observing how the number of iterations varies across the complex plane, one can assess the convergence properties of any methods. The fractal boundaries highlight regions where small changes in the initial guess can yield drastically different outcomes (i.e., converging to different roots or diverging). This sensitivity is crucial to understand, especially when implementing these methods in practical applications where precision of the initial guess might be limited. They are used later in the paper by implying to select a small value for the free nonzero parameter (carried forward the relation (37)) and how memorization can enhance the convergence domain.

3. Derivation of the Scheme

To facilitate the implementation of the memory-based scheme (15), we will first examine

$$\vartheta :=A^{\left(\chi \right)}=-{[w^{(\chi -1)},g^{(\chi -1)};H]}^{-1}\approx -H^{\prime }{\left(\theta \right)}^{-1},$$

(17)

and

$$\left\{\begin{array}{c}{N}^{(\chi -1)}{\delta }^{\left(\chi \right)}=-H\left(g^{\left(\chi \right)}\right),\hfill \\ N^{(\chi -1)}{\gamma }^{\left(\chi \right)}=-H\left(q^{\left(\chi \right)}\right),\hfill \\ N^{(\chi -1)}{\psi }^{\left(\chi \right)}=-H\left(z^{\left(\chi \right)}\right).\hfill \end{array}\right.$$

(18)

Consequently, we now present the subsequent scheme (PM2) as our main contribution, ($A^{\left(\chi \right)}=-[w^{(\chi -1)}$,$g^{(\chi -1)}{;H]}^{-1}$, $\chi \ge 1$):

$$\left\{\begin{array}{c}{w}^{\left(\chi \right)}=g^{\left(\chi \right)}+A^{\left(\chi \right)}H\left(g^{\left(\chi \right)}\right),\chi \ge 1,\hfill \\ q^{\left(\chi \right)}=g^{\left(\chi \right)}+{\delta }^{\left(\chi \right)},\chi \ge 0,\hfill \\ z^{(\chi +1)}=q^{\left(\chi \right)}+{\gamma }^{\left(\chi \right)},\hfill \\ g^{(\chi +1)}=z^{\left(\chi \right)}+{\psi }^{\left(\chi \right)}.\hfill \end{array}\right.$$

(19)

The error equation of this solver with memory will be given in the next section.

It is well known [15] when $D\subset {\mathbb{R}}^{\omega }$ represents a convex nonempty domain. Then, assume that H is three-times Fréchet smooth over D, and $[u,v;H]\in \mathcal{L}(D,D)$ for any distinct points $u,v\in D$ (where $v\ne u$). Additionally, let the starting vector $g^{\left(0\right)}$ and the zero $\theta$ be in close proximity to each other. By defining $A^{\left(\chi \right)}=-{[w^{(\chi -1)},g^{(\chi -1)};H]}^{-1}$ and setting $d^{\left(\chi \right)}:=I+A^{\left(\chi \right)}{H}^{\prime }\left(\theta \right)$, finally we can derive the equation below

$$d^{\left(\chi \right)}\sim e^{(\chi -1)}.$$

(20)

The relation (20) will be used later in Section 4 of this work.

To implement (19), it is essential to resolve a series of linear algebraic sets of equations. This entails performing a new LU factorization at each iteration, without leveraging any information from preceding steps. Nevertheless, a substantial body of the literature exists regarding the recycling of such information to derive updated preconditioners for iterative solvers [18]. The advantage of (19) lies in the fact that all linear systems share the identical coefficient matrix. Consequently, a single LU factorization suffices; by retaining this decomposition, it can be applied to several distinct right-hand side parts to obtain the resolution vectors for each sub-cycle of (19).

The solution of the nonlinear equation systems that we consider here are in $D\subseteq {\mathbb{R}}^{\omega }$ as stated in Section 1. The roots that we are seeking for are assumed to be simple zeros. Both real and complex roots can be obtained by the discussed methods (if existed). In fact, by choosing a suitable complex initial guess, a complex root (if existed), can be obtained.

4. Convergence Order

Here, we furnish a theoretical analysis of the convergence speed of the iterative scheme presented in (19). Before introducing the main contribution, we represent the $\omega$-dimensional Taylor expansion.

The rate at which the iteration without memory PM1 converges is determined via $\omega$-dimensional Taylor expansions. Let ${\epsilon }^{\left(\chi \right)}=g^{\left(\chi \right)}-\theta$ denote the error at the $\chi$-th iterate. As noted in [19]:

$${\epsilon }^{(\chi +1)}=G{\epsilon }^{{\left(\chi \right)}^p}+\mathcal{O}\left({\epsilon }^{{\left(\chi \right)}^{p+1}}\right),$$

(21)

this error equation implies G is a p-linear functional, where $G\in \mathcal{L}({\mathbb{R}}^{\omega },{\mathbb{R}}^{\omega },\dots ,{\mathbb{R}}^{\omega })$ and p is the speed. Additionally, we have:

$${\epsilon }^{{\left(\chi \right)}^p}=\left(\underset{p\mathrm{terms}}{\underbrace{{\epsilon }^{\left(\chi \right)},{\epsilon }^{\left(\chi \right)},\dots ,{\epsilon }^{\left(\chi \right)}}}\right).$$

(22)

Assume that $H:D\subseteq {\mathbb{R}}^{\omega }\to {\mathbb{R}}^{\omega }$ is sufficiently differentiable in the Fréchet sense in D. Following [10], the $\omega$-th derivative of H at $u\in {\mathbb{R}}^{\omega }$, $\omega \ge 1$, is the $\omega$-linear functional, i.e.,

$$H^{\left(\omega \right)}\left(u\right):{\mathbb{R}}^{\omega }\times \cdots \times {\mathbb{R}}^{\omega }\to {\mathbb{R}}^{\omega },$$

(23)

so that $H^{\left(\omega \right)}\left(u\right)(v_1,\dots ,v_{\omega })\in {\mathbb{R}}^{\omega }$. For $\theta +h\in {\mathbb{R}}^{\omega }$ located in a vicinity of the solution $\theta$ of (1), the Taylor expansion can be formulated as [10]:

$$H(\theta +h)=H^{\prime }\left(\theta \right)\left[h+\sum _{\omega =2}^{p-1}{M}_{\omega }{h}^{\omega }\right]+\mathcal{O}\left(h^p\right),$$

(24)

where $M_{\omega }={\displaystyle \frac{1}{\omega !}}{\left[H^{\prime }\left(\theta \right)\right]}^{-1}{H}^{\left(\omega \right)}\left(\theta \right),\omega \ge 2$. It follows that $M_{\omega }{h}^{\omega }\in {\mathbb{R}}^{\omega }$, as $H^{\left(\omega \right)}\left(\theta \right)\in \mathcal{L}({\mathbb{R}}^{\omega }\times \cdots \times {\mathbb{R}}^{\omega },{\mathbb{R}}^{\omega })$ and ${\left[H^{\prime }\left(\theta \right)\right]}^{-1}\in \mathcal{L}\left({\mathbb{R}}^{\omega }\right)$. Additionally, for $H^{\prime }$, we have:

$$H^{\prime }(\theta +h)=H^{\prime }\left(\theta \right)\left[I+\sum _{\omega =2}^{p-1}\omega M_{\omega }{h}^{\omega -1}\right]+\mathcal{O}\left(h^p\right),$$

(25)

wherein I shows the unit matrix. Additionally, $\omega M_{\omega }{h}^{\omega -1}\in \mathcal{L}\left({\mathbb{R}}^{\omega }\right)$.

Theorem 1.

Let in (1) $H:D\subseteq {\mathbb{R}}^{\omega }\to {\mathbb{R}}^{\omega }$ be adequately Fréchet differentiable at every point in D and that $H\left(\theta \right)=0$ at $\theta \in {\mathbb{R}}^{\omega }$. Additionally, let $H^{\prime }\left(\eta \right)$ be continuous and nonsingular at θ. Next, ${\left\{g^{\left(\chi \right)}\right\}}_{\chi \ge 0}$ produced by (19) with memory with a selection of an appropriate starting value has 4.23607 R-convergence order.

Proof. 

For proving the convergence speed, we consider (24) and (25) to write

$$H\left(g^{\left(\chi \right)}\right)=H^{\prime }\left(\theta \right)\left[{\epsilon }^{\left(\chi \right)}+M_2{{\epsilon }^{\left(\chi \right)}}^2+M_3{{\epsilon }^{\left(\chi \right)}}^3\right]+\mathcal{O}\left({{\epsilon }^{\left(\chi \right)}}^4\right).$$

(26)

In the context of the scheme (15), when operating without memory (i.e., PM1) and applying (24)–(26), we ultimately derive

$${\epsilon }^{(\chi +1)}=(I+\vartheta H^{\prime }\left(\theta \right))(2I+\vartheta H^{\prime }\left(\theta \right))C_2^2{\epsilon }^{{\left(\chi \right)}^4}+\mathcal{O}\left({\epsilon }^{{\left(\chi \right)}^5}\right).$$

(27)

Now by considering the with memorization in (19), we shall express (27) in its asymptotic form as follows:

$$\begin{array}{c}\hfill {\epsilon }^{(\chi +1)}\sim d_1^{\left(\chi \right)}{{\epsilon }^{\left(\chi \right)}}^4,\end{array}$$

(28)

where ∼ shows for the error equation without the asymptotical term. A variety of symbolic computations conducted by considering that the coefficients of the error terms in our $\omega$-D scenario are all matrices, and applying (20), along with the understanding that multiplication does not allow for commutativity, we can deduce

$$\begin{array}{c}\hfill d_1^{\left(\chi \right)}\sim {\epsilon }^{(\chi -1)},\forall \chi \ge 1.\end{array}$$

(29)

Consequently, one arrives at

$$\begin{array}{c}\hfill {d_1^{\left(\chi \right)}}^2\sim {{\epsilon }^{(\chi -1)}}^2,\forall \chi \ge 1.\end{array}$$

(30)

By imposing (29) and (30) into (28), we obtain:

$$\begin{array}{c}\hfill {\epsilon }^{(\chi +1)}\sim {{\epsilon }^{(\chi -1)}}^1{{\epsilon }^{\left(\chi \right)}}^4.\end{array}$$

(31)

This demonstrates that ${\displaystyle \frac{1}{p}}+4=p$, where the convergence R-order is expressed as

$$p=\left(\sqrt{5}+2\right)\simeq 4.23607.$$

(32)

The proof is concluded. □

Before ending this section, it is pointed out that with a simple change the structure of PM2, it is possible to provide another iteration scheme with memory of a similar kind as follows (PM3), (this time: $A^{\left(\chi \right)}={[w^{(\chi -1)},g^{(\chi -1)};H]}^{-1}$, $\chi \ge 1$):

$$\left\{\begin{array}{c}{w}^{\left(\chi \right)}=g^{\left(\chi \right)}-A^{\left(\chi \right)}H\left(g^{\left(\chi \right)}\right),\chi \ge 1,\hfill \\ q^{\left(\chi \right)}=g^{\left(\chi \right)}+{\delta }^{\left(\chi \right)},\chi \ge 0,\hfill \\ z^{(\chi +1)}=q^{\left(\chi \right)}+{\gamma }^{\left(\chi \right)},\hfill \\ g^{(\chi +1)}=z^{\left(\chi \right)}+{\psi }^{\left(\chi \right)}.\hfill \end{array}\right.$$

(33)

If we aim to further improve the convergence order, two possible approaches can be considered. First, we could construct a solver by incorporating additional subsets and then apply the memorization procedure. Second, one might explore faster methods to accelerate the free parameter using alternative types of interpolation, which, however, falls beyond the scope of this paper.

5. Computational Efficiency Comparisons

For the proposed solver with memory, PM2 (or equivalently PM3), only a single LU factorization is required, followed by matrix-vector multiplications, which enhances computational efficiency by eliminating the need to compute multiple matrix inverses in each iteration. The CEI for iterative solvers is defined as follows [7]:

$$CEI=p^{1/\mathcal{C}},$$

(34)

where $\mathcal{C}$ represents the total computational cost and p signifies the convergence speed, considering the quantity of functional evaluations. The cost of computing each scalar function is considered a unit, while the costs associated with other calculations are expressed as multiples of this unit cost. To evaluate the CEI for PM2, we first outline the number of functional evaluations (cost) required for $\omega$-dimensional functions, as detailed below (excluding the indicator $\chi$):

• In $H\left(g\right),H\left(w\right),H\left(q\right),H\left(z\right)$: $4\omega$ evaluations.
• In the DDO: ${\omega }^2$ evaluations.

Furthermore, we consider the costs to solve two triangular systems, all quantified in floating-point operations (flops). The flops necessary for executing the LU procedure amount to $\left(2{\omega }^3\right)/3$, while resolving the two related triangular systems requires approximately $2{\omega }^2$ flops. Noting that, here, we assume that the cost for one functional evaluation is roughly equal to the cost for one flop. The findings displayed in

Table 1. Efficiency indices for several Steffensen-type methods.

Compared Methods	SM	PM1	PM2
Order	2	4	4.23607
Function assessments	$\omega +{\omega }^2$	$4\omega +{\omega }^2$	$4\omega +{\omega }^2$
The classical CEI	$2^{{\displaystyle \frac{1}{\omega +{\omega }^2}}}$	$4^{{\displaystyle \frac{1}{4\omega +{\omega }^2}}}$	$4.{23607}^{{\displaystyle \frac{1}{4\omega +{\omega }^2}}}$
No. of LU decomposition	1	1	1
Assessments for LU decompositions (flops)	${\displaystyle \frac{2{\omega }^3}{3}}$	${\displaystyle \frac{2{\omega }^3}{3}}$	${\displaystyle \frac{2{\omega }^3}{3}}$
Assessments for linear systems (flops)	${\displaystyle \frac{2{\omega }^3}{3}}+2{\omega }^2$	${\displaystyle \frac{2{\omega }^3}{3}}+6{\omega }^2$	${\displaystyle \frac{2{\omega }^3}{3}}+6{\omega }^2$
Flops-type CEI	$2^{{\displaystyle \frac{1}{\frac{2{\omega }^3}{3}+3{\omega }^2+\omega }}}$	$4^{{\displaystyle \frac{1}{\frac{2{\omega }^3}{3}+7{\omega }^2+4\omega }}}$	$4.{23607}^{{\displaystyle \frac{1}{\frac{2{\omega }^3}{3}+7{\omega }^2+4\omega }}}$

demonstrate that for varying $\omega$, the CEI of our method surpasses that of competing approaches. Comparisons of different derivative-free Steffensen-type solvers with and without memory based on various choices of $\omega$ are illustrated in Figure 5 and Figure 6, confirming the superiority and improvement in both classic and flops-type efficiency indices of PM relative to its main competitors.

In real-world applications, there is a trade-off between eliminating Fréchet-derivatives and increasing the method’s overall computational complexity. Generally speaking, it relies on the specific problem, which leads to a nonlinear system of equations. However, by eliminating the Fréchet derivative, we make the solver derivative-free, which is suitable for problems, in which the derivative is not available. Moreover, the concept of memorization can be accompanied in methods without Fréchet derivatives to increase the computational complexity.

It is also remarked that in the absence of Fréchet derivatives, one might ask that what specific mechanisms ensure that the proposed Steffensen-type technique remains robust over a variety of problem sets? To tackle this, it is stated that in the absence of Fréchet derivatives, the convergence radius mainly reduced tremendously, which is one weak point of Steffensen-type techniques at first sight. However, this can simply be improved by choosing very small values for the free non-zero parameter (as seen in the attraction basins of Section 2), as well as imposing the with memorization concept.

6. Numerical Aspects

The objective of this section is to facilitate the implementation of our proposed scheme, PM2. The computations were executed using Mathematica 13.3 [20] in standard machine precision to handle the round-off errors. The linear systems were resolved employing LU decomposition via $\mathtt{LinearSolver}\left[\right]$. All computational examples were conducted in a uniform environment. We adopt the stopping criterion as follows:

$$\left|\right|H\left(g^{(\chi +1)}\right){\left|\right|}_{\infty }\le {10}^{-6}.$$

(35)

A significant challenge in implementing (19) involves the integration of $A^{\left(\chi \right)}$, which is no longer constant and must be characterized as a matrix. Here, $A^{\left(0\right)}$ is delineated follows:

$$A^{\left(0\right)}=\mathrm{diag}\left(-0.01\right).$$

(36)

In fact due to Figure 1, Figure 2, Figure 3 and Figure 4 and the results discussed previously in [13], other small values for the free parameter can also be used which can yield in different choices such as

$$A^{\left(0\right)}=\mathrm{diag}\left(-0.001\right),$$

(37)

or

$$A^{\left(0\right)}=\mathrm{diag}\left(-0.0001\right),$$

(38)

or

$$A^{\left(0\right)}=\mathrm{diag}\left(-0.00001\right).$$

(39)

In fact, for implementation of method with memory, here, for the first iterate, we start with the without memory version of the method and after that, from the second iterate, the information of the previous step can be used to update the parameters and thus with memorization of the scheme can be done. The selection of $A^{\left(0\right)}$ has a direct impact on the entire process, influencing the speed at which convergence is achieved. Here, (37) aligns with the dynamic investigations of Steffensen-type solvers with memory, where larger basins of attraction occur when the free parameter is near zero.

To validate the analytical convergence rate in the computational experiments, we determine the numeric speed of convergence using [7]

$$\rho \approx {\displaystyle \frac{ln\left(\right|\left|H\left(g^{(\chi +1)}\right)\right|{|}_2/\left|\right|H\left(g^{\left(\chi \right)}\right)\left|{|}_2\right)}{ln\left(\right|\left|H\left(g^{\left(\chi \right)}\right)\right|{|}_2/\left|\right|H\left(g^{(\chi -1)}\right)\left|{|}_2\right)}}.$$

(40)

Example 1

([14]). We investigate a nonlinear system $H\left(\eta \right)=0$, which possesses a complex root, as detailed below

$$H\left(\eta \right)=\left\{\begin{array}{c}{\eta }_{1}sin\left({\eta }_2\right)-2{{\eta }_{10}}^{{\eta }_8}+{\eta }_{10}-5{\eta }_6-10{\eta }_9,\hfill \\ 10{\eta }_1+{{\eta }_3}^2-5{{\eta }_5}^2+10{{\eta }_6}^{{\eta }_8}-sin\left({\eta }_7\right)+2{\eta }_9,\hfill \\ {cos}^{-1}(-10{\eta }_{10}+{\eta }_8+{\eta }_9)+{\eta }_{4}sin\left({\eta }_2\right)+{\eta }_3-15{{\eta }_5}^2+{\eta }_7,\hfill \\ {\eta }_1{{\eta }_2}^{{\eta }_7}-{{\eta }_8}^{{\eta }_{10}}+{{\eta }_3}^5-5{{\eta }_5}^3+{\eta }_7,\hfill \\ 10{{\eta }_1}^2-{\eta }_{10}+cos\left({\eta }_2\right)+{{\eta }_3}^2-5{{\eta }_6}^3-2{\eta }_8-4^{{\eta }_9},\hfill \\ {cos}^{-1}\left({{\eta }_1}^2\right)sin\left({\eta }_2\right)-2{\eta }_{10}{{\eta }_5}^4{\eta }_6{\eta }_9+{{\eta }_3}^2,\hfill \\ 2tan\left({{\eta }_1}^2\right)+2^{{\eta }_2}+{{\eta }_3}^2-5{{\eta }_5}^3-{\eta }_6+{{\eta }_8}^{cos\left({\eta }_9\right)},\hfill \\ {{\eta }_1}^2-{\eta }_{10}{\eta }_5{\eta }_6{\eta }_7{\eta }_8{\eta }_9+tan\left({\eta }_2\right)+2{{\eta }_3}^{{\eta }_4}-5{{\eta }_6}^3,\hfill \\ 5tan({\eta }_1+2)+cos\left({{\eta }_9}^{{\eta }_{10}}\right)+{{\eta }_2}^3+7{{\eta }_3}^4-2{sin}^3\left({\eta }_6\right),\hfill \\ 5exp({\eta }_1-2){\eta }_2+2{{\eta }_7}^{{\eta }_{10}}+8{{\eta }_3}^{{\eta }_4}-5{{\eta }_6}^3-{\eta }_9,\hfill \end{array}\right.$$

(41)

in which the precise solution is presented up to eight floating points as

$$\begin{array}{ccc}\theta \hfill & \simeq \hfill & (1.32734904+0.35029249i,1.0585993-1.7487246i,\hfill \\ \hfill & \hfill & 1.02761867-0.01413080i,3.2739500+0.1278283i,\hfill \\ \hfill & \hfill & 0.83182439+0.00175519i,-0.4853245912+0.68487764i,\hfill \\ \hfill & \hfill & 0.16936676+0.1840917i,1.5344199-0.3212147i,\hfill \\ \hfill & \hfill & {2.0863796+0.42634275i,-1.9895923+1.4783953i)}^{*}.\hfill \end{array}$$

(42)

The computational evidence and the numerical speed, denoted as $\rho$, are detailed in

Table 2. Comparisons of different methods with and without memory in Example 1.

Methods	$\parallel \mathit{H}\left({\mathit{g}}^{\left(3\right)}\right)\parallel$	$\parallel \mathit{H}\left({\mathit{g}}^{\left(4\right)}\right)\parallel$	$\parallel \mathit{H}\left({\mathit{g}}^{\left(5\right)}\right)\parallel$	$\parallel \mathit{H}\left({\mathit{g}}^{\left(6\right)}\right)\parallel$	$\parallel \mathit{H}\left({\mathit{g}}^{\left(7\right)}\right)\parallel$	$\parallel \mathit{H}\left({\mathit{g}}^{\left(8\right)}\right)\parallel$	$\parallel \mathit{H}\left({\mathit{g}}^{\left(9\right)}\right)\parallel$	$\mathit{\rho }$
NM	8.19 $\times {10}^{-1}$	$2.73\times {10}^{-2}$	$1.79\times {10}^{-5}$	$1.28\times {10}^{-11}$	$2.52\times {10}^{-23}$	$8.28\times {10}^{-47}$	$2.50\times {10}^{-94}$	2.02
SM	$7.68\times {10}^{-1}$	$1.83\times {10}^{-2}$	$7.33\times {10}^{-6}$	$5.17\times {10}^{-12}$	$1.51\times {10}^{-24}$	$2.03\times {10}^{-49}$	$3.91\times {10}^{-99}$	1.99
PM1	$1.01\times {10}^{-4}$	$6.95\times {10}^{-20}$	$1.19\times {10}^{-80}$	$2.12\times {10}^{-323}$				3.99
PM2 with (36)	$2.19\times {10}^{-3}$	$6.12\times {10}^{-16}$	$3.29\times {10}^{-69}$	$7.45\times {10}^{-295}$				4.23
PM2 with (37)	$1.44\times {10}^{-4}$	$2.99\times {10}^{-21}$	$7.96\times {10}^{-92}$	$1.09\times {10}^{-390}$				4.23
PM2 with (38)	$1.60\times {10}^{-4}$	$2.88\times {10}^{-21}$	$1.05\times {10}^{-91}$	$2.88\times {10}^{-390}$				4.23
PM2 with (39)	$1.58\times {10}^{-4}$	$2.61\times {10}^{-21}$	$8.16\times {10}^{-92}$	$9.18\times {10}^{-391}$				4.24

. We utilized 1000 fixed floating points, with the initial value set as $g^{\left(0\right)}={(1.20+0.30i,1.10-1.90i,1.00-0.10i,2.50+0.50i,0.80-0.10i,-0.40+1.00i,0.10+0.10i,1.30-0.70i,2.00+0.50i,-1.90+1.40i)}^{*}$. The choice of this is based on $g^{\left(0\right)}$[14]. Rather than $g^{\left(0\right)}$ and (37), no other parameters should be chosen and the method works based on satisfying (35). Additionally, the residual norm is expressed using the ${\parallel ·\parallel }_2$ notation. The numerical pieces of evidence seen in Table 2 support the observations in Figure 1, Figure 2, Figure 3 and Figure 4, discussing that the smaller the choice of the free parameter for PM2 would results in arriving at the convergence phase faster than consider larger values for this parameter.

Example 2

([21]). In this test, we investigate the nonlinear systems extracted through the computational resolution of the following Partial Differential Equation (PDE)

$$\left\{\begin{array}{c}{\mathfrak{u}}_{\tau }+\mathfrak{u}{\mathfrak{u}}_z=\overline{\rho }{\mathfrak{u}}_{zz},\hfill \\ \mathfrak{u}(1,\tau )=0,\tau \ge 0,\hfill \\ \mathfrak{u}(0,\tau )=0,\tau \ge 0,\hfill \\ \mathfrak{u}(z,0)={\displaystyle \frac{2\overline{\rho }\overline{\vartheta }\pi sin\left(\pi z\right)}{\xi +\overline{\vartheta }cos\left(\pi z\right)}},0\le z\le 2,\hfill \end{array}\right.$$

(43)

where the coefficient of diffusion is $\overline{\rho }$. If we take into account $\mathfrak{u}=\mathfrak{u}(z,\tau )$, then the computational resolution is represented by:

$${\varrho }_{i,j}\simeq \mathfrak{u}(z_i,{\tau }_j),$$

(44)

at the grid points $(i,j)$ on a equidistant mesh. Let ${\varpi }_1$ and ${\varpi }_2$ show the number of steps in spatial and temporal variables, respectively. The parameters are set as follows [21]: $\xi =5$, $T=1$, $\overline{\rho }=0.5$, and $\overline{\vartheta }=4$. The parameters have been chosen so as to obtain a unique and well-defined numerical solution and stay away from stiffness or irregularity for the solution of PDEs.

To address this, we may utilize the first-order backward FD method for the first differentiation in temporal $\tau$:

$${\mathfrak{u}}_{\tau }(z_i,{\tau }_j)\simeq {\displaystyle \frac{{\varrho }_{i,j}-{\varrho }_{i,j-1}}{k}},$$

(45)

wherein k denotes the time step size. For the spatial terms of the PDE, we apply the second-order central difference FD method as follows:

$${\mathfrak{u}}_z(z_i,{\tau }_j)\simeq {\displaystyle \frac{{\varrho }_{i+1,j}-{\varrho }_{i-1,j}}{2h}},$$

(46)

and

$${\mathfrak{u}}_{zz}(z_i,{\tau }_j)\simeq {\displaystyle \frac{{\varrho }_{i+1,j}-2{\varrho }_{i,j}+{\varrho }_{i-1,j}}{h^2}},$$

(47)

where h represents the spatial step size along z. Following this discretization and imposing the boundary conditions will lead a to a set of nonlinear equations that must be tackled iteratively.

The numerical solutions using PM2 are illustrated in Figure 7, and

Table 3. Computational outcomes obtained in Example 2.

Different Solvers	SM	PM1	PM2 with (37)
No. of iterates	7	2	1
Time (in seconds)	6.12	5.11	3.27

provides the comparative evidence for this case in double precision when the residual of the numerical obtained solution is less than ${10}^{-5}$. We set ${\varpi }_1={\varpi }_2=21$, i.e., 21 equally spaced nodes in both space and time direction and after imposing the initial and boundary conditions, we derive to a set of nonlinear system of the dimension $400\times 400$, where the initial vector $g^{\left(0\right)}=\mathbf{1}$. Along space, we have used central three-point second-order FD approximations, and along time, we have employed first order forward FD approximations to discretize the problem.

For an alternative set of parameters, specifically ${\varpi }_1={\varpi }_2=31$, representing 31 equally spaced points in both the spatial and temporal directions, we obtained a nonlinear system of dimension $900\times 900$ after applying the initial and boundary conditions. The initial vector is again defined as $g^{\left(0\right)}=\mathbf{1}$. Figure 8 presents the numerical simulations for this setup, further demonstrating the effectiveness of PM2 and the concept of memorization.

The numerical performance of the proposed solver is demonstrated through root-finding for various nonlinear equations. Results from the numerical tests clearly indicate that the solver achieves convergence in fewer iterations and with higher accuracy per iteration.

We finish this section by pointing out the following matter. Another concern may arise regarding how the method handles convergence issues in nonlinear systems with multiple solutions or when the initial guess significantly influences convergence behavior. To address this, the focus of this article is on simple roots, not multiple roots; while the methods can be applied to find multiple zeros, their orders of convergence drop significantly in such cases. In fact, if a nonlinear system has multiple roots (whether the multiplicity is known or unknown), specialized solvers designed for multiple roots, along with appropriate initial guesses, must be developed to maximize efficiency.

7. Concluding Summaries

In this paper, we have presented an advanced Steffensen-type iteration expression aimed at solving nonlinear systems of equations, specifically tailored to eliminate the need for computing Fréchet derivatives. This approach has exhibited significant computational efficiency by reducing the number of matrix inversions and functional evaluations, as outlined in Section 3, Section 4 and Section 5. Our examination has validated the efficacy of the proposed method. Future endeavors will focus on further improving the efficiency of the scheme and expanding its applicability to a wider variety of nonlinear challenges, including nonlinear stochastic differential equations as highlighted in [22].