Abstract
In this paper, we present an innovative technique that improves the convergence order of iterative schemes that do not require the evaluation of Jacobian matrices. As far as we know, this is the first technique that allows us the achievement of an increase, from p to units, in the order of convergence. This is constructed from any Jacobian-free scheme of order p. We conduct comprehensive numerical tests first in academical examples to validate the theoretical results, showing the efficiency and effectiveness of the new Jacobian-free schemes. Then, we apply them on the non-differentiable partial differential equations that models the nutrient diffusion in a biological substrate.
MSC:
65H10
1. Introduction
Let us consider the system of nonlinear equations , where and , are the coordinate functions of F, with . Solving nonlinear systems is a challenging task, typically requiring either linearization of certain nonlinear problems or the application of a fixed-point function , which leads to a fixed-point iteration scheme. Additionally, these schemes can be applied in different areas of knowledge, including engineering, chemistry, fluid dynamics, among others, making them valuable tools for solving nonlinear and nondifferentiable problems (see, for example, refs. [1,2,3]). Tackling such systems often involves employing linearization techniques to approximate solutions or employing fixed-point iteration methods based on function G. These strategies provide valuable insights and approximated solutions of the nonlinear systems.
In many of these methods, the evaluation of Jacobian matrices at one or more points per iteration is required. However, a significant challenge arises from the computation of the Jacobian matrix, especially in cases where it may not exist or, for high-dimensional scenarios, its computation becomes excessively costly or even infeasible. As a result, certain authors have attempted to overcome this issue by eliminating the dependence on the Jacobian matrix and replacing it with alternative techniques.
There are numerous methods for solving systems of nonlinear equations. Among them is Steffensen’s method which was presented by Samanskii in [4]. This stands out as one of the most renowned techniques in the literature on iterative methods without Jacobian. Its iterative scheme is given by the expression
where , being the divided difference operator of F on defined as (see [5])
In this work, we explore innovative iterative methods for the resolution of nonlinear systems of equations without the need to compute Jacobian matrices. Different researchers in this field have made significant contributions to the development of these strategies.
Several authors have developed memory-based methods that also utilize the Kurchatov divided difference to enhance the efficiency of known methods. Additionally, as part of our strategy, we adopted the mth-order vectorial divided differences proposed by Amiri et al. in [6]. These strategies, found in the literature, circumvent the complexity and costs associated with Jacobian matrices, making them particularly valuable for large-scale systems. Among the most commonly used approaches are the finite difference method, as well as polynomial interpolators, divided differences, and other clever techniques that have also been employed in the literature to avoid the direct evaluation of derivatives. These additional techniques further expand the tools available for tackling derivative-related problems in complex systems.
The advances achieved by these different procedures are reflected in numerous works by different authors. Chicharro et al. and Cordero et al. in [7,8], respectively, proposed new parametric families for solving nonlinear systems. In a similar way, the authors of manuscripts [9,10,11] replaced the Jacobian matrix by a particular divided difference operator. The technique of weight functions (in this case, matrix functions) play an important role for designing Jacobian-free schemes for solving nonlinear systems, as we can see in [12].
In this research, we present an approach that improves the convergence order from p to , applicable to Jacobian-free schemes. We replace the traditional Jacobian matrix of F with , a concept introduced by Amiri et al. in [6], where with provides a remarkable approximation of the Jacobian matrix for the function at any given point. By leveraging this approach with , we achieve a convergence order of .
This novel approach not only circumvents the complexities associated with traditional Jacobian matrix computations, but also opens new avenues for improving the convergence rate of numerical methods, making it a valuable contribution to the field of numerical analysis and optimization.
The work is developed as follows: first, in Section 1, we introduce the basic concepts necessary for the development of the paper. Then, in Section 2, we prove in a general way that the scheme has a convergence order of . To illustrate this, in Section 3, we solve some academic problems to confirm the reliability of the modified methods without Jacobian matrices. Finally, we address the solution of the problem entitled Modeling of nutrient diffusion in a biological substrate, which is formulated through a nonlinear non-differentiable second-order elliptic nonlinear partial differential equation. To address this question, we proceed to discretize the equation by means of the finite difference method, transforming it into a system of nonlinear equations. We then employ the modified Traub method with increased order of convergence, which we denote as Traub-M2, to solve the resulting system. We conclude by presenting the matrix representing the approximate solution of this system of equations.
Preliminary Concepts
First, we introduce a number of necessary concepts in order to develop the proposed Taylor series scheme and prove the order of convergence of the proposed iterative scheme.
Definition 1.
Let be a sequence in , convergent to ξ. Then, convergence is
- 1.
- linear, if there exists for , and , such that
- 2.
- sequence , which converges to ξ with order , if there exists for and such that
We let sufficiently differentiable in . The qth derivative of F in is the q-linear function such that . It is easy to see that
- is a linear operator that maps to .
- , for every permutation of .
From the above properties, we define the following notation:
- ,
- .
On the other hand, for which lies in a vicinity of a solution of , we can apply the Taylor expansion, and if the Jacobian matrix is not singular, we can express
where . We observe that since and . In addition, we can express as
where I is the identity matrix. Therefore, . From (3), we can obtain
where
On the other hand, we denote by the error in the kth iteration. Equation
where M is a p-linear function , is called the error equation, and p is the order of convergence of the sequence generated by the iterative process. We observe that is .
To estimate the convergence order, we use the Approximated Computational Order of Convergence (ACOC), whose expression we can see below.
Definition 2.
Let ξ be a zero of function F and suppose that and are three consecutive iterations close to ξ. Then, the order of convergence p can be approximated using formula
To compare the different methods obtained by applying the results proposed in this work, we employ Ostrowski’s efficiency index (see [13]), where p is the order of convergence and d is the total number of functional evaluations required by the method (per iteration). This is the most commonly used index, but not the only one. In his book [14], Traub uses an operational index that is defined as , where is the number of operations per iteration. We recall that the number of products and quotients needed to solve r linear systems with the same coefficient matrix, using the factorization , is calculated as follows:
We use a combination of both indices, , which is called the computational efficiency index.
The formula of Gennochi–Hermite (see [5]),
allows us the calculation of the Taylor expansion of the divided difference operator in terms of the successive derivatives of F,
By denoting and using the error at both points, , the Taylor expansion of the divided difference (1) can be written as
Now, we use these expressions to prove the following result.
In order to perform the Taylor development of , we use the following result, presented in the work of Amiri et al. in [6].
Theorem 1.
Let H be a nonlinear operator with coordinate functions and such that . Let us consider the divided difference operator , where and ; then, the order of the divided difference as an approximation of the Jacobian matrix is m.
Proof.
Let be the coordinate functions of . Let us consider the Taylor expansion of around :
where for and and , for is the th coordinate of error e. We can write (7) as
with for , since
so for , and we have
By introducing the multilinear operator for , we can express the Taylor series of around as follows:
so we define the error at as
Now, we let
Let us consider the Taylor expansion of around . When we apply the Gennochi–Hermite Formula (6), we obtain
Since the Taylor expansions of and around coincide for the first m terms, the order of the divided difference is exactly m. □
2. Main Result
In the following, we present a technique that allows increasing the order of convergence from p to units in Jacobian-free methods. To achieve this goal, it is necessary to introduce a second arbitrary scheme or class of two-step iterative methods whose predictor step must be Newton’s scheme, a second generic step of order p. We add a third corrector step , which depends on real numbers , and guaranteeing that any scheme with such conditions will reach convergence order , for which we state the following result.
In the following section, we introduce an iterative scheme that utilizes a Steffensen type method as a predictor. This concept is inspired in the idea of divided differences with to adapt various families of derivative-dependent iterative methods. This adaptation ensures consistent convergence orders when substituting derivatives with divided differences for different values of m, especially in the scalar case.
For the scalar case, the iterative expression of a Steffensen type method is defined as:
By applying a similar concept and replacing Jacobian matrices with divided differences in the vectorial case, we create a generic scheme with an undetermined number of steps, where the first step is an extension of the Steffensen scheme (10) to systems. This extension enables an increase in the convergence order from p to units.
In a more general way, we represent the scheme as follows:
where
and , where are coordinate functions of , converge to with convergence order for We demonstrate this assertion below.
Theorem 2.
Let be a sufficiently differentiable function at each point in a neighborhood of an open convex set Ω containing , such that is a solution of the system. Let be an initial estimate sufficiently close to ξ, and suppose that the Jacobian matrix is continuous and nonsingular at ξ.
Then, the method defined in (11) has a convergence order of , where p is the order of the arbitrary scheme , whose first step is Steffensen type method, for , and and its error equation is as follows:
Proof.
To perform a Taylor series expansion of the divided difference, we need to develop the Taylor series of , , and around :
where The derivative is
According to Theorem 1, for is a second-order approximation of the Jacobian matrix . Using (14) and (15), its Taylor series expansion is as follows:
Let us now calculate the Taylor series development of . Forcing that equality
must be satisfied, we conjecture
and, consequently,
Therefore,
Let us calculate the error equation for the first step of the scheme as
It follows that
We denote
Now, let us suppose that the error of the second step is of order p; then, the error in the kth iterate is
and therefore the Taylor development of around is
Thus, the Taylor series expansion of the Jacobian matrix using the divided difference is as follows:
In order to calculate the error equation at the last step, we first calculate the product
For this, using (17) and (21), we have
where
In addition,
where
From (27), we derive system
whose solution is , and . Simplifying the resulting error equation, we obtain
showing that the order of convergence is . This ends our proof. □
3. Numerical Results
Below, we present several systems of nonlinear equations that we solve to confirm the reliability of the methods based on the conditions stated in Theorem 2, specifically the methods that do not require the evaluation of Jacobian matrices.
Each table displays the initial approximation used to find the solutions. Additionally, we indicate the number of iterations (Iter) required for the schemes to converge, the approximate computational order of convergence (ACOC) for each method, and the approximate computational time (e-time) required (in seconds).
Calculations are performed with MATLAB programming software using variable precision arithmetics with 2000 digits of mantissa and an error tolerance of . In addition, the expression (5) is used to calculate the approximated computational order of convergence ACOC. Each table includes the CPU time used by each method to obtain the solution. It is important to note that the computer used for these calculations has the following software and hardware specifications: macOS Ventura operating system version 13.4.1, Intel Core i7 quad-core processor Chip M1 Pro, 16 GB Ram Memory LPDDR5 and year of manufacture 2021. In addition, we apply the following stopping criteria:
3.1. Computational Efficiency
Now, we present five classes of iterative methods that we use to apply our technique and perform various numerical tests in relation to specific academic problems. Then, we apply the third step of our approach to each of these methods. In a first step, we compute the computational efficiency index of each of these methods in order to be able to compare the methods of order p with the schemes modified to among the different classes of approaches.
The first scheme to be used to increase its convergence order and check its efficiency is a modification of the method presented by Cordero et al. in [15], where we eliminate the Jacobian matrix. For simplicity, we denote it as , where represents the parameter selected in that scheme, four is the order of the method and .
where
The second test scheme we employ in this paper is Traub’s method, which is introduced by Traub in [16]. The system-transferred version of this method, where the Jacobian is also replaced by the split difference and , is represented by the following expression:
where
Chun’s method was initially introduced in [17]. Now, we present a modification of this scheme, in which we replace the Jacobian matrix with the divided difference for .
being where
Ostrowski’s method is presented in [13] and transferred to systems by Grau et al. in [18]; as in the previous cases, we modify this method by replacing the Jacobian matrix by the split difference proposed in this work for .
where
Finally, we employ the family of iterative methods proposed by Cordero et al. in [19]. This class, characterized by having convergence order in three steps, allows us the application of our new technique, stated in Theorem 2, resulting in a transformation of the scheme into a four-step method, achieving an increase in its convergence order up to units. The adoption of this advanced technique promises to offer a level of accuracy and efficiency in the results obtained. This scheme takes its new form after replacing the Jacobian matrix by the proposed divided difference for ; its iterative expression in three steps is as follows:
for and
We note that in each method we use the value . After several numerical tests, we find that the iterative schemes converge faster to the solution when is near to zero.
We apply to each of them the conditions of Theorem 2 in order to increase their orders of convergence from p to .
In Table 1, we present information on the order of convergence, the number of scalar functional evaluations per iteration, the number of products/quotients per iteration, and the computational efficiency index (CI) for each method, both before and after applying our proposed third step. It is important to remember that in order to evaluate the function in one iteration, the computation of scalar functions is required. The number of scalar function evaluations is for any split-difference evaluation and for the expression . Furthermore, to each iterative class presented after applying our scheme, we refer as through , respectively, where p represents the convergence order of each modified method.
Table 1.
Computational efficiency index of order p and .
In Figure 1 and Figure 2, we can observe the computational efficiency behavior of our technique, both for the methods through and for the modified methods by incorporating our third step when applied to small and large systems.
Figure 1.
Efficiency index for methods of order p. (a) Efficiency index of order p for small systems of equations; (b) Efficiency index of order p for large systems of equations.
Figure 2.
Efficiency index for methods of order . (a) Efficiency index of order for small systems of equations; (b) Efficiency index of order for large systems of equations.
3.2. Some Academical Problems
Example 1.
The first case we present is a system of equations of size ,
The solution of System (28) is given by and the initial estimate used is .
Example 2.
The second case presented is a system of size ,
The solution of System (29) is and the initial estimate used is .
Example 3.
The third case presented is a system of size ,
whose exact solution is , and initial estimate is .
Example 4.
The fourth system presented has size ; the expression representing this system is given by
The exact solution of System (31) is , for which we use the initial estimate .
Example 5.
Finally, we consider system
It is a system of size , and we select the initial estimation to approximate the solution
Table 2, Table 3, Table 4, Table 5 and Table 6 provide strong confirmation of our theoretical findings as the application of the proposed scheme to each class yielded a convergence order of . Remarkably, the class exhibited outstanding efficiency, particularly in terms of computational time, when compared to alternative approaches that do not utilize Jacobian matrices. This observation reinforces the practical viability of our technique and its potential to significantly accelerate numerical computations, supporting its relevance in academic problem solving.
Table 2.
Numerical results for Example 1.
Table 3.
Numerical results for Example 2.
Table 4.
Numerical results for Example 3.
Table 5.
Numerical results for Example 4.
Table 6.
Numerical results for Example 5.
3.3. Application of the Finite Difference Method on a Model of Nutrient Diffusion in a Biological Substrate
Now, we focus on the detailed study of a nonlinear elliptic initial value and contour problem, which was previously treated in [20]. To enrich and improve the analysis, we introduce an additional term in the equation, namely , which makes the problem non-differentiable.The proposed partial differential equation has the ability to model a wide range of nonlinear phenomena in physical, chemical and biological systems.
In the following, we delve into an applied problem concerning the modeling of nutrient diffusion in a biological substrate. We state the problem and discuss the potential implications that solving it could have for researchers in various fields and for farmers, as well as the interpretation of its solution.
In the context of agriculture and biotechnology, let us consider a two-dimensional biological substrate that represents a cultivation area or a growth medium for microorganisms. We aim to understand how nutrients diffuse and distribute within this substrate, impacting the growth and health of the organisms present,
In this scenario, represents the concentration of nutrients at each point within the substrate. The equation reflects how nutrients diffuse within the substrate, while the term incorporates the interaction between nutrient concentration and the biochemical processes present in the substrate.
The boundary conditions are derived from the conditions at the edges of the cultivation area. The conditions at the lateral edges ( and ) could reflect the initial nutrient concentration or the constant input of nutrients into the substrate. The conditions at the upper and lower edges ( and ) could represent nutrient absorption by plant roots or interaction with microorganisms on the surface.
Solving this problem would enable researchers and farmers to understand how nutrients are distributed within the substrate and how they impact the growth and health of the organisms present, providing valuable insights for optimizing agricultural practices and enhancing biological yields.
As an illustrative example, we solve this equation for a small system, employing a block-wise approach to represent it as follows.
We create a mesh for discretizing the problem , ; we also have the mesh point with , and , such that
Therefore, by approximating the partial derivatives by central divided differences, we have
for
Now, we denote . Simplifying the notation, we obtain
with and
Equation (34) together with the boundary and initial conditions form a nonlinear system of size is given by
where
with . In addition, W denotes a vector containing the boundary conditions. We can set the nonlinear system as follows:
Let us define the differentiable part as , and the non-differentiable part as , so that equation holds.
To solve Equation (35), we employ the iterative method . We generate an initial approximation of the exact solution . During the execution of the iterative process, we use variable precision arithmetics with a precision of 100 digits. We define the stopping criterion as the difference between two consecutive iterations, such that and also . The technical specifications of the computer used to solve this case are identical to those employed in solving academical problems. In this instance, we select .
Solving the system associated with this PDE, we write the solution of the system represented in the following matrix.
After three iterations, the obtained solution satisfies , and the norm of the nonlinear operator evaluated at the last iteration is such that . The approximate solution in , resized to a matrix for , and then embedded in the solution matrix within the grid bounded by the boundary conditions, is presented in the matrix above.
We can affirm that the values obtained fall within the range . In the context of the posed problem involving the diffusion of nutrients in a two-dimensional biological substrate, these observations take on fundamental significance. The concentration of nutrients demonstrates a trend in which values are bounded between 0 and . This limitation in concentration implies a biological equilibrium in the system, where absorption, diffusion, and biochemical reactions harmonize. This characteristic suggests that the system is stable and well-regulated in terms of nutrient availability.
The coherence between the obtained values and the initial and boundary conditions of the problem reinforces the validity of the solution. This indicates that the nutrient distribution aligns with the influences of the boundary conditions, thus supporting the biological interpretation of the model.
The influence of the differential equation is reflected in the limited concentration range. This implies that biochemical interactions between nutrients and present species are influencing nutrient distribution in the substrate.
In summary, the observation that the internal values of the solution matrix lie within the range of 0 to suggests a balanced, stable, and regulated biological system in terms of nutrient concentration. The alignment with the conditions of the problem and the influence of biochemical interactions endorse the validity of the obtained results and their interpretation in the context of agriculture and biotechnology.
4. Conclusions
In this paper, we presented an innovative proposal that shows an ability to increase the order of convergence from p to , without relying on Jacobian matrices. Our validation was solidly supported by consistently obtaining this three-unit increase, as evidenced in various academic problems. This approach generated equally significant results, translating into an increase in the order of convergence by units in each of the iterative classes we used to perform our numerical tests as we already assumed. Finally, we successfully tackled the resolution of a problem modeled by a nonlinear partial differential equation describing the phenomenon of nutrient diffusion in a biological substrate.
Author Contributions
Conceptualization, A.C.; methodology, M.P.V.; software, M.A.L.-S.; validation, J.R.T.; formal analysis, M.A.L.-S.; writing—original draft preparation, M.A.L.-S.; writing—review and editing, A.C., J.R.T. and M.P.V. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Data Availability Statement
Not applicable.
Acknowledgments
The authors would like to thank the anonymous reviewers for their comments and suggestions that have improved the final version of this manuscript.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Ali, R.; Ali, A.; Iqbal, S. Iterative methods for solving absolute value equations. J. Math. Comput. Sci. 2022, 26, 322–329. [Google Scholar] [CrossRef]
- Khan, W. Numerical simulation of Chun-Hui He’s iteration method with applications in engineering. Int. J. Numer. Methods Heat Fluid Flow 2022, 32, 944–955. [Google Scholar] [CrossRef]
- Khan, W.; Arif, M.; Mohammed, M.; Farooq, U.; Farooq, F.; Elbashir, M.; Rahman, J.; AlHussain, Z. Numerical and Theoretical Investigation to Estimate Darcy Friction Factor in Water Network Problem Based on Modified Chun-Hui He’s Algorithm and Applications. Math. Probl. Eng. 2022, 2022, 8116282. [Google Scholar] [CrossRef]
- Samanskii, V. On a modification of the Newton method. Ukr. Math. J. 1967, 19, 133–138. [Google Scholar]
- Ortega, J.M.; Rheinboldt, W.C. Iterative Solution of Nonlinear Equations in Several Variables; SIAM: Philadelphia, PA, USA, 2000. [Google Scholar]
- Amiri, A.; Cordero, A.; Darvishi, M.T.; Torregrosa, J.R. Preserving the order of convergence: Low-complexity Jacobian-free iterative schemes for solving nonlinear systems. J. Comput. Appl. Math. 2018, 337, 87–97. [Google Scholar] [CrossRef]
- Chicharro, F.I.; Cordero, A.; Garrido, N.; Torregrosa, J.R. A new efficient parametric family of iterative methods for solving nonlinear systems. J. Differ. Equ. Appl. 2019, 25, 1454–1467. [Google Scholar] [CrossRef]
- Cordero, A.; Villalba, E.G.; Torregrosa, J.R.; Triguero-Navarro, P. Convergence and stability of a parametric class of iterative schemes for solving nonlinear systems. Mathematics 2021, 9, 86. [Google Scholar] [CrossRef]
- Singh, A. An efficient fifth-order Steffensen-type method for solving systems of nonlinear equations. Int. J. Comput. Sci. Math. 2018, 9, 501–514. [Google Scholar] [CrossRef]
- Sharma, J.; Arora, H. Efficient derivative-free numerical methods for solving systems of nonlinear equations. Comput. Appl. Math. 2016, 35, 269–284. [Google Scholar] [CrossRef]
- Wang, X.; Zhang, T.; Qian, W.; Teng, M. Seventh-order derivative-free iterative method for solving nonlinear systems. Numer. Algorithms 2015, 10, 545–558. [Google Scholar] [CrossRef]
- Sharma, R.; Guha, J.; Sharma, R. A efficient fourth-order wheighted-Newton method for systems of nonlinear equations. Numer. Algorithms 2013, 62, 307–323. [Google Scholar] [CrossRef]
- Ostrowski, A. Solution of Equations and Systems of Equations; Academic Press: New York, NY, USA, 1966. [Google Scholar]
- Traub, J.F. Iterative Methods for the Solution of Equations; American Mathematical Society: Providence, RI, USA, 1982; Volume 312. [Google Scholar]
- Cordero, A.; Leonardo Sepúlveda, M.A.; Torregrosa, J.R. Dynamics and Stability on a Family of Optimal Fourth-Order Iterative Methods. Algorithms 2022, 15, 387. [Google Scholar] [CrossRef]
- Traub, J. Iterative Methods for the Solution of Equations; Prentice-Hall: Englewood Cliffs, NJ, USA, 1964. [Google Scholar]
- Chun, C. Construction of Newton-like iterative methods for solving nonlinear equations. Numer. Math. 2006, 104, 297–315. [Google Scholar] [CrossRef]
- Grau-Sánchez, M.; Grau, Á.; Noguera, M. Ostrowski type methods for solving systems of nonlinear equations. Appl. Math. Comput. 2011, 218, 2377–2385. [Google Scholar] [CrossRef]
- Cordero, A.; Maimó, J.G.; Rodríguez-Cabral, A.; Torregrosa, J.R. Convergence and Stability of a New Parametric Class of Iterative Processes for Nonlinear Systems. Algorithms 2023, 16, 163. [Google Scholar] [CrossRef]
- Villalba, E.G.; Hernandez, M.; Hueso, J.L.; Martínez, E. Using decomposition of the nonlinear operator for solving non-differentiable problems. Math. Methods Appl. Sci. 2022. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

