A Note on Traub’s Method for Systems of Nonlinear Equations

Abstract

Traub’s method was extended here to systems of nonlinear equations and compared to Steffensen’s method. Even though Traub’s method is only of order 1.839 and not quadratic, it performed better in the 10 examples.

1. Introduction

Recently, Traub’s [1] method was compared to other superlinear methods, and to Steffensen’s scheme [2], see also Amat and Busquier [3], Chicharro et al. [4], Cordero et al. [5]. Neta [6] has shown that Traub’s method is superior in terms of its dynamics. The method is given by

$$x_{k+1}=x_k+\frac{f\left(x_k\right)}{f[x_{k-2},x_k]-f[x_{k-2},x_{k-1}]+f[x_{k-1},x_k]},k=0,1,\dots ,$$

(1)

being $x_0$, $x_{-1}$ and $x_{-2}$ their initial estimations and the divided differences are $f[x_i,x_j]=\frac{f\left(x_i\right)-f\left(x_j\right)}{x_i-x_j}$. The method is of order $p=1.839$ and it was used as a first step to obtain higher order methods for finding simple roots, see Neta [6,7]. The reasons for its stability were discussed by Cordero et al. [8].

In this article, we extend Traub’s method to solve systems of nonlinear equations and compare its performance to Steffensen’s method, which is used currently as a basis for derivative-free multistep methods because of its quadratic convergence. Chun and Neta [9] have compared Steffensen’s method to several higher order methods (see [10,11,12,13,14,15]). and found that in 21 examples, a three-step fourth order method based on Neta’s method was best.

In Section 2, we state the methods and list several examples for the comparative study. In Section 3, the numerical performance of the method is checked in terms of the number of function evaluations, CPU run-time, and the accuracy of the solution.

2. Numerical Examples

In this section, we compare two methods, namely Traub’s method and Steffensen’s method. Traub’s method for systems of nonlinear equations $F\left(x\right)=0$ where $x\in {\mathbb{R}}^n$ and $F:D\subset {\mathbb{R}}^n\to {\mathbb{R}}^n$ is given by

$$x_{n+1}=x_n-B_n^{-1}F\left(x_n\right)$$

(2)

where

$$B_n=[x_{n-2},x_n;F]-[x_{n-2},x_{n-1};F]+[x_{n-1},x_n;F]$$

and the divided difference

$${[x,y;F]}_{ij}=\frac{f_i(x_1,\dots ,x_j,y_{j+1},\dots ,y_n)-f_j(x_1,\dots ,x_{j-1},y_j,\dots ,y_n)}{x_i-y_j}$$

The method is of order 1.839, which is lower than Steffensen’s and requires only one function evaluation per step. It requires two additional initial vectors, which we took as a small number added to the components of $x_0$.

Theorem 1.

Let the function $F:D\subset {\mathbb{R}}^m\to {\mathbb{R}}^m$ be sufficiently differentiable in a convex set D containing asimplezero α of $F\left(x\right)$. Then the local convergence order of the method (2) is 1.839, and the error equation is given by

$$\begin{array}{ccc}\hfill e_{n+1}& =& 2A_3{e}_n{ϵ}_n{E}_n+\dots \hfill \end{array}$$

(3)

where $e_n=x_n-\alpha \in {\mathbb{R}}^m, {ϵ}_n=x_{n-1}-\alpha \in {\mathbb{R}}^m$, $E_n=x_{n-2}-\alpha \in {\mathbb{R}}^m$, $e_n^i=\underset{i-times}{\underbrace{(e_n,e_n,...,e_n)}},$ $A_j=(1/j!){\left[F^{\prime }\left(\alpha \right)\right]}^{-1}{F}^{\left(j\right)}\left(\alpha \right)\in L_i({\mathbb{R}}^m,{\mathbb{R}}^m),$ $F^{\left(j\right)}\in L({\mathbb{R}}^m\times \cdots \times {\mathbb{R}}^m,{\mathbb{R}}^m)$ and ${\left[F^{\prime }\left(\alpha \right)\right]}^{-1}\in L\left({\mathbb{R}}^m\right)$, and similarly for ${ϵ}_n^i$ and $E_n^i$.

Proof. 

Let $\alpha$ be the zero. Given $x_0$, $x_{-1}$ and $x_{-2}$, our method is given by (2) where $B_n={\int }_0^1{F}^{\prime }(x_n+t{h}_1)dt-{\int }_0^1{F}^{\prime }(x_n+\sigma h_2)d\sigma +{\int }_0^1{F}^{\prime }(x_n+\tau h_3)d\tau$, where $h_1=x_{n-2}-x_n$, $h_2=x_{n-2}-x_{n-1}$ and $h_3=x_{n-1}-x_n.$

Now we expand in the Taylor series

$$F\left(x_n\right)=F^{\prime }\left(\alpha \right)\left\{e_n+A_2{e}_n^2+A_3{e}_n^3+A_4{e}_n^4+\dots \right\},$$

(4)

where $e_n=x_n-\alpha$, ${ϵ}_n=x_{n-1}-\alpha$ and $E_n=x_{n-2}-\alpha .$ We would like to prove that $e_{n+1}=2A_3{e}_n{ϵ}_n{E}_n+\dots .$

Now we have

$$\begin{array}{ccc}{B}_n\hfill & =& F^{\prime }\left(x_n\right)+\frac{1}{2}{F}^{\left(2\right)}\left(x_n\right)(E_n-e_n)+\frac{1}{6}{F}^{\left(3\right)}\left(x_n\right){(E_n-e_n)}^2+\dots \hfill \\ & -& \left\{F^{\prime }\left(x_n\right)+\frac{1}{2}{F}^{\left(2\right)}\left(x_n\right)(E_n-{ϵ}_n)+\frac{1}{6}{F}^{\left(3\right)}\left(x_n\right){(E_n-{ϵ}_n)}^2+\dots \right\}\hfill \\ & +& F^{\prime }\left(x_n\right)+\frac{1}{2}{F}^{\left(2\right)}\left(x_n\right)(e_n-{ϵ}_n)+\frac{1}{6}{F}^{\left(3\right)}\left(x_n\right){(e_n-{ϵ}_n)}^2+\dots \hfill \\ & =& F^{\prime }\left(x_n\right)+\frac{1}{3}{F}^{\left(3\right)}\left(x_n\right)(e_n^2-e_n{E}_n-{ϵ}_n(e_n-E_n)+\dots .\hfill \end{array}$$

From the Taylor series expansion of $F\left(x_n\right)$ given above, we get

$$F^{\prime }\left(x_n\right)=F^{\prime }\left(\alpha \right)(I+2A_2{e}_n+3A_3{e}_n^2+4A_4{e}_n^3+\dots )$$

$$F^{\left(2\right)}\left(x_n\right)=F^{\prime }\left(\alpha \right)(2A_2+6A_3{e}_n+12A_4{e}_n^2+\dots )$$

$$F^{\left(3\right)}\left(x_n\right)=F^{\prime }\left(\alpha \right)(6A_3+24A_4{e}_n+60A_5{e}_n^2+\dots )$$

Substituting all of these expansions in $B_n$, we have

$$B_n=F^{\prime }\left(\alpha \right)\left\{I+2A_2{e}_n+2A_3(-e_n{ϵ}_n+e_n{E}_n+{ϵ}_n{E}_n)+\dots \right\}$$

The inverse is

$$B_n^{-1}=\left\{I-2A_2{e}_n-2A_3(-e_n{ϵ}_n+e_n{E}_n+{ϵ}_n{E}_n)+4A_2^2{e}_n^2+\dots \right\}{\left[F^{\prime }\left(\alpha \right)\right]}^{-1}.$$

(5)

Therefore

$$e_{n+1}=e_n-\left\{I-2A_2{e}_n-2A_3(-e_n{ϵ}_n+e_n{E}_n+{ϵ}_n{E}_n)+4A_2^2{e}_n^2+\dots \right\}\left\{e_n+A_2{e}_n^2+\dots \right\}.$$

(6)

Expanding and collecting lower order terms leads to

$$e_{n+1}=2A_3{e}_n{ϵ}_n{E}_n+\dots$$

(7)

Therefore, the method is of order 1.839.

Steffensen’s method is given by

$$x_{n+1}=x_n-C_n^{-1}F\left(x_n\right)$$

(8)

where $C_n=[x_n,y_n;F]$ and $y_n=x_n+F\left(x_n\right)$. The method is of order 2, and requires two function evaluations per step.

The criteria for comparisons are the number of function evaluations for convergence. We use a tolerance of ${10}^{-15}$ and allow a maximum of 50 iterations. We also collected the accuracy and the CPU run-time.

The systems used for our comparative study are mostly taken from [9]:

• Example 1
  This system has six solutions.

  $$\begin{array}{c}{x}_1^2+x_1{x}_2^3-9=0\hfill \\ 3x_1^2{x}_2-x_2^3-4=0\hfill \end{array}$$

  $$x_0={(-1.2,-2.5)}^T$$

  $${\alpha }_1={(-0.9012661905,-2.086587595)}^T$$

  $${\alpha }_2={(9.985950982,-2.086587595)}^T$$

  $${\alpha }_3={(2.998375993,0.1481079950)}^T$$

  $${\alpha }_4={(-3.001624887,0.1481079950)}^T$$

  $${\alpha }_5={(1.336355377,1.754235198)}^T$$

  $${\alpha }_6={(-6.734735503,1.754235198)}^T$$
• Example 2

  $$\begin{array}{c}cos{x}_2-sin{x}_1=0\hfill \\ x_3^{x_1}-1/x_2=0\hfill \\ e^{x_1}-x_3^2=0\hfill \end{array}$$

  $$x_0={(1.2,0.5,1.5)}^T$$

  $${\alpha }_1={(0.9095694944,0.6612268323,1.575834144)}^T$$

  $${\alpha }_2={(-0.9095694944,0.6612268323,0.6345845493)}^T$$
• Example 3

  $$\begin{array}{c}{x}_1^5+x_2^3{x}_3^4+1=0\hfill \\ x_1^2{x}_2{x}_3=0\hfill \\ x_3^4-1=0\hfill \end{array}$$

  $$x_0={(-8,2,8)}^T$$

  $$\alpha ={(-1,0,1)}^T$$
• Example 4

  $$\begin{array}{c}6x_1^2+x_2-\frac{37}{6}=0\hfill \\ x_1-6x_2^2-\frac{5}{6}=0\hfill \\ x_1+x_2+x_3-\frac{1}{2}=0\hfill \end{array}$$

  $$x_0={(1.5,0.5,-0.5)}^T$$

  $${\alpha }_1={(1,1/6,-2/3)}^T$$

  $${\alpha }_2={(1.028512437,-0.1803603357,-0.3481521018)}^T$$

  and two other complex conjugate solutions.
• Example 5

  $$\begin{array}{c}12x_1-3x_2^2-4x_3-7.17=0\hfill \\ x_1^2+10x_2-x_3-11.54=0\hfill \\ x_2^3+7x_3-7.631=0\hfill \end{array}$$

  $$x_0={(3,0,1)}^T$$

  $${\alpha }_1={(1.2,1.1,0.9)}^T$$

  $${\alpha }_2={(7.809384276,-3.953119569,9.915287083)}^T$$

  and two other pairs of complex conjugate solutions.
• Example 6

  $$\begin{array}{c}{x}_2{x}_3+x_4(x_2+x_3)=0\hfill \\ x_1{x}_3+x_4(x_1+x_3)=0\hfill \\ x_1{x}_2+x_4(x_1+x_2)=0\hfill \\ x_1{x}_2+x_1{x}_3+x_2{x}_3-1=0\hfill \end{array}$$

  $$x_0={(1.7,0.7,1.8,0.8)}^T$$

  $${\alpha }_1={(1/\sqrt{3},1/\sqrt{3},1/\sqrt{3},-1/\left(2\sqrt{3}\right))}^T$$

  $${\alpha }_2={(-1/\sqrt{3},-1/\sqrt{3},-1/\sqrt{3},1/\left(2\sqrt{3}\right))}^T$$
• Example 7

  $$\begin{array}{c}{x}_i{x}_{i+1}-1=0,i=1,2,\dots ,n-1\hfill \\ x_n{x}_1-1=0\hfill \end{array}$$

  $$x_0={(2,2,\dots ,2)}^T$$
  If n is odd, there are two solutions:

  $${\alpha }_1={(1,1,\dots ,1)}^T$$

  $${\alpha }_2={(-1,-1,\dots ,-1)}^T$$
  If n is even, then choose $x_n$

  $$x_1=x_3=\dots =x_{n-1}=\frac{1}{x_n}$$

  $$x_2=x_4=\dots =x_{n-2}=x_n$$
  We used $n=9$ in this example.
• Example 8
  This is another example from [16,17].

  $$\begin{array}{c}{x}_k^2{x}_{k+1}-1=0,1\le k\le n-1,\hfill \\ x_n^2{x}_1-1=0,k=n\hfill \end{array}$$

  $$x_0={(0.1,0.1,\dots ,0.1)}^T$$

  $$\alpha ={(1,1,\dots ,1)}^T$$
  We used $n=10$ in this example.
• Example 9
  The process of penetration of the magnetic field into the material is modeled by the averaged integro-differential equation:

  $$\frac{\partial U}{\partial t}-\left[1+{\int }_0^t{\int }_0^1{\left(\frac{\partial U}{\partial x}\right)}^{2}dxd\tau \right]\frac{{\partial }^{2}U}{\partial x^2}=f(x,t),$$

  (9)

  with Dirichlet boundary conditions

  $$U(0,t)=U(1,t)=0,$$

  (10)

  and the initial condition

  $$U(x,0)=U_0\left(x\right),$$

  (11)

  in the rectangle $Q_T=(0,1)\times (0,T)$, where T is a positive constant, $f=f(x,t)$ and $U_0=U_0\left(x\right)$ are given functions of their arguments. The resulting nonlinear system is [18]

  $$\begin{array}{c}{\displaystyle \frac{1}{\tau }{u}_i^{j+1}-A\left({\mathbf{u}}^{j+1}\right)\frac{u_{i+1}^{j+1}-2u_i^{j+1}+u_{i-1}^{j+1}}{h^2}-f_i^j-\frac{1}{\tau }{u}_i^j=0,}\\ i=1,\dots ,M-1.\end{array}$$

  (12)

  where

  $${\displaystyle A\left({\mathbf{u}}^{j+1}\right)=1+\tau h\sum _{\ell =1}^M\sum _{k=1}^{j+1}{\left(\frac{u_{\ell }^k-u_{\ell -1}^k}{h}\right)}^2.}$$
  This system can be written in matrix form

  $${\displaystyle \mathbf{T}\left({\mathbf{u}}^{j+1}\right){\mathbf{u}}^{j+1}=\frac{1}{\tau }{\mathbf{u}}^j+{\mathbf{f}}^j.}$$
  The vector $\mathbf{u}$ containing all the unknowns $u_1,\dots ,u_{M-1}$ at the level indicated. The matrix $\mathbf{T}$ is symmetric and tridiagonal with elements

  $${\displaystyle {\mathbf{T}}_{ir}=\left\{\begin{array}{ccc}{\displaystyle \frac{1}{\tau }+2\frac{A}{h^2},}\hfill & & r=i,\hfill \\ \\ {\displaystyle -\frac{A}{h^2},}\hfill & & r=i\pm 1.\hfill \end{array}\right.}$$
  In our experiments, we take $M=100$, so $h=0.01$. The initial solution is given by

  $$U_0\left(x\right)=x(1-x)e^{-x}.$$
  In order to get the exact solution $U(x,t)=x(1-x)e^{-x-t}$, we picked the following forcing term

  $$f(x,t)=-\left(\frac{9}{8}-\frac{3}{8}{e}^{-2}-\left(\frac{1}{8}-\frac{3}{8}{e}^{-2}\right)e^{-2t}\right)(-4+5x-x^2)e^{-x-t}-x(1-x)e^{-x-t}.$$
• Example 10
  Consider the Hammerstein integral equation [19],

  $$x\left(s\right)=1+\frac{1}{5}{\int }_0^{1}K(s,t)x{\left(t\right)}^{3}dt$$

  (13)

  where $x\in C[0,1]$, $s,t\in [0,1]$ and the kernel

  $$K(s,t)=\left\{\begin{array}{cc}(1-s)t\hfill & t\le s,\hfill \\ s(1-t)\hfill & s\le t.\hfill \end{array}\right.$$
  Solving the system numerically by using the Gauss–Legendre quadrature formula with 7 points, yield

  $$5x_i-5-\sum _{j=1}^7{a}_{ij}{x}_j^3=0$$

  where

  $$a_{ij}=\left\{\begin{array}{cc}{w}_j{t}_j(1-t_i)\hfill & j\le i\hfill \\ w_j{t}_i(1-t_j)\hfill & i\le j,\hfill \end{array}\right.$$

  and $w_j$ are the weights and $t_j$ are the nodes. The initial solution is $x^{\left(0\right)}={(-1,\dots ,-1)}^T.$ The approximate solution is

  $$x^{(k+1)}={(1.0027414204,1.0125680365,1.0235854047,1.0285959684,1.0243765818,1.0179001828,1.0026562769)}^T.$$

□

3. Numerical Experiments

We ran the following two methods:

• Steffensen’s method
• Traub’s method

on each of the following examples with the corresponding initial guesses listed.

All computations were conducted in MAPLE 2016 with 100-digit floating point arithmetic in a laptop with Windows 10 Home and processor Intel(R) Core(TM) i7-7500u CPU 2.90GHZ. The results for the number of function evaluations are given in

Table 1. Number of function evaluations used by each method for each example.

Example	Dimension	Steffensen	Traub
1	2	Div	7
2	3	18	8
3	3	Div	Div
4	3	22	7
5	3	24	7
6	5	22	17
7	9	16	7
8	10	22	17
9	99	Div	5
10	7	Div	6

. We also collected the CPU times used, which are listed in

Table 2. CPU time (in seconds) used by each method for each example.

Example	Dimension	Steffensen	Traub
1	2	Div	0.079
2	3	0.172	0.116
3	3	Div	Div
4	3	0.062	0.076
5	3	0.156	0.25
6	5	0.172	0.307
7	9	0.78	0.133
8	10	0.203	0.594
9	99	Div	459.172
10	7	Div	0.137

. The number of function evaluations and CPU times used by each method for each example are given with tolerance $\parallel F\left(x_k\right){\parallel }_{\infty }<{10}^{-15}$, and “Div” denotes that the method does not converge within 50 iterations.

As can be seen in the tables, Steffensen’s method, is the only one that diverged in several examples. Both methods converged to the first root ${\alpha }_1$, except Steffensen’s method converged to ${\alpha }_2$ in examples 4 and 5. Steffensen’s method requires more function evaluations than Traub’s method, even when converged.

In Table 2, we list the CPU times required for each method and each example. Clearly, we did not list the CPU times for Steffensen’s method in the several examples where the methods did not converge. It is no clear cut as to which method is faster, but the main result is that Traub’s method is more robust.

We now compute the error norm for each method and each example, see

Table 3. Accuracy of methods for each example.

Example	Dimension	Steffensen	Traub
1	2	Div	1.27(−20)
2	3	1.89(−16)	2.13(−21)
3	3	Div	Div
4	3	1.13(−24)	2.04(−19)
5	3	9.93(−18)	7.12(−28)
6	5	1.45(−20)	2.47(−20)
7	9	1.03(−16)	1.75(−30)
8	10	2.05(−20)	2.47(−20)
9	99	Div	2.06(−20)
10	7	Div	1.47(−22)

. We see that Traub has much smaller error than Steffensen in 3 out of the 10 examples. In two other examples, they both have the same accuracy, and only in one case the accuracy of Steffensen is higher.

Overall, Traub’s method is best, since it almost always converges (except 1 in 10) and is fastest with the fewest number of function evaluations. If we discount the examples for which the methods diverge, then we find that Traub’s method requires the least number of function evaluations to converge (4.3 iterations on average). We thus conclude that Traub’s method is a better, derivative-free method on which to build multistep methods for the solution of systems of nonlinear equations.

4. Conclusions

We compared Traub’s method to Steffensen’s method and showed that, in ten examples, the latter diverged in four cases and the former in only one. Traub’s method requires fewer function evaluations to converge and it is almost always more accurate. In future research, we plan to develop multistep derivative-free methods based on Traub’s method as the first step. We will compare the new method to currently available schemes based on Steffensen’s method as the first step.