Abstract
This paper presents an inexact version of an exponential iterative method designed for solving nonlinear equations , where the function F is only locally Lipschitz continuous. The proposed algorithm is completely new as an essential extension of the iterative exponential method for solving nonsmooth equations. The method with backtracking is globally and superlinearly convergent under some mild assumptions imposed on F. The presented results of the numerical computations confirm both the theoretical properties of the new method and its practical effectiveness.
1. Introduction
Let be locally Lipschitz continuous. Consider the nonlinear equation in the following simplest form:
where , and , . Furthermore, we assume that there exists , which is a solution to Equation (1) (i.e., ). It is worth recalling that Pang and Qi [1] presented significant motivations and a broad scope of applications for nonsmooth equations.
Some class of iterative exponential methods for solving one-dimensional nonlinear equations was proposed by Chen and Li [2]. One of the most interesting formulas, and simultaneously the simplest one, is the method which has the following form:
This method turns to the classic Newton method if we use the first-order Taylor series expansion of the expression . The above method in Equation (2) has (at least) quadratic convergence if the function is twice differentiable and in a neighborhood of the solution. A substantial extension of the exponential method in Equation (2) to the nonsmooth case in was presented in [3]. This locally and superlinearly convergent method can be written in the following two-stage form:
where the matrix is arbitrarily taken from the B-differential of F at .
On the other hand, one traditional approach for solving nonsmooth equations is the inexact generalized Newton method:
where the matrix could be arbitrarily taken from some subdifferential of F at (not only from the generalized Jacobian or B-differential but also from, for example, the ∗-differential) and for all Martínez and Qi in [4] stated the local and superlinear convergence of the generalized Jacobian-based method with Equation (4) for solving equations with semismooth function F. Other versions of such Newton-type methods for solving nonsmooth equations were also considered among the others in [5,6,7]. The main concept of the inexact Newton-type method is to substitute the standard Newton equation with an inequality as in Equation (4), where the parameter is the forcing term. Then, the Newton step may be determined inexactly, such as by the use of any iterative method with feasible inaccuracy, especially when the current approximation is far from the solution to Equation (1).
The main objective of this paper is to introduce an inexact version of the exponential method with a subdifferential for multidimensional nonsmooth cases. Therefore, we present some new iterative methods for solving nondifferentiable problems, which can be sufficiently effective in solving not only nonsmooth equations but also some important problems in nonsmooth optimization. The exponential method was introduced in [2] for finding single roots of univariate nonlinear equations and considered in [8] for solving unconstrained optimization problems. In both cases, the proposed formulas allow solving only smooth and one-dimensional problems. We discuss an algorithm which is intended to solve both nonsmooth and multi-dimensional problems. Our method allows solving Equation (1), in which the function is only Lipschitz continuous and need not be differentiable. A primary extension, shown in Equation (3), was introduced in [3], but this study focuses on the inexact approach. In this way, vector can be approximated in every iteration with some inaccuracy, which reduces the cost of determining it. However, all these methods are only locally convergent. A new approach also includes a backtracking procedure, and as a result, we obtained a global convergence of the inexact exponential method.
This paper is organized as follows. We recall some important needed notions and properties in Section 2. In Section 3, we not only introduce an algorithm but also prove its global and superlinear convergence. Section 4 features the results of numerical tests. In Section 5 at the end, we give some conclusions.
2. Preliminaries
Throughout the paper, denotes the Euclidean norm on , where we regard the vector from as a column vector. However, all theoretical results do not depend on the choice of the norm.
We assume that the function F is locally Lipschitz continuous in the traditional sense. In other words, for every , there exists a constant such that
If is a differentiable function, we denote the Jacobian matrix of F at x as and the set where F is differentiable as . The local Lipschitz continuity of the function F implies that it is differentiable almost everywhere, according to Rademacher’s theorem [9]. Then, we have
which is called the Bouligand subdifferential (B-differential for short) of F at x [10]. Furthermore, the generalized Jacobian of function F at x (in the sense of Clarke [9]) is the convex hull of the Bouligand subdifferential such that
We say that the function F is BD-regular at x if F is locally Lipschitz continuous at x and if all matrices V from are nonsingular. For BD-regular functions, Qi proved some important properties (see Lemma 2).
In turn, the set
is called the b-differential of function F at x (defined by Sun and Han in [11]). It is easy to verify that in a one-dimensional case (i.e., when ), the generalized Jacobian reduces to the Clarke generalized gradient and i.e., the B-differential and b-differential coincide). Furthermore, a ∗-differential is a non-empty bounded set for any x such that
where denotes the generalized gradient of the component function at x. Obviously, for a locally Lipschitz function, all previously mentioned differentials, namely , and , are ∗-differentials (see Gao [12]). Obviously, if all components of function F are at x, then .
The notion of semismoothness was primarily introduced by Mifflin [13] for functionals. Later, Qi and Sun [14] extended the original definition to nonlinear operators. A function F is semismooth at x if F is locally Lipschitz continuous at x and for any , the limit
exists. If function F is semismooth at x, then F is directionally differentiable at x, and is equal to the above limit:
Lemma 1
(Lemma 2.2 by Qi [15]). Suppose that the function is directionally differentiable in a neighborhood of x. The following statements are equivalent:
- (1)
- F is semismooth at x;
- (2)
- is semicontinuous at x;
- (3)
- For any , the following is true:
Moreover, we say that function F is p-order semismooth at x if for any matrix V from the generalized Jacobian , it holds that
where . If , then F is just called strongly semismooth [16].
Remark 1.
- (i)
- Clearly, semismoothness of any order implies semismoothness.
- (ii)
- The piecewise function is strongly semismooth.
Lemma 2
(Lemma 2.6 by Qi [15]). If function F is BD-regular at x, then there is a neighborhood N of x and a constant such that for any and matrix , V is nonsingular, and
If the function F is also semismooth at , then for any , the following is true:
Qi and Sun showed in [14] that if a function F is semismooth at x, then
Furthermore, if a function F is p-order semismooth at x, then
Throughout this paper, denotes the closed ball with a center x and radius r in (if the center is known, and a radius is negligible, just N is used).
Furthermore, for a given diagonal matrix whose diagonal entries are such that
then denotes a so-called exponential matrix, which has the following form:
3. The Algorithm and Its Properties
In this section, we introduce a new algorithm, and we discuss its convergence. For solving Equation (1), we can consider a method in the descriptive two-stage form
where matrix is taken from some subdifferential of F at . We suggest an approach which is a combination of two methods: the iterative exponential one and the inexact generalized Newton one with a decreasing function norm and backtracking. The proposed inexact approach is similar to the algorithm presented by Birgin et al. [17] for smooth equations, whereas the exponential formula was introduced in [3]. Because of the nondifferentiability of F, as iteration matrices, we use matrices from the B-differential instead of the Jacobians.
Remark 2.
- (i)
- The matrix may be chosen absolutely arbitrarily from known elements of the Bouligand subdifferential (in Step 1 of Algorithm 1). Appropriate matrices are usually suitable Jacobians of F.
- (ii)
- The backtracking approach used in Step 2 is a particular case of the minimum reduction algorithm.
- (iii)
- The formula in Equation (10) for obtaining could be written in a substitute form, which explicitly shows how to generate a new approximation using components of the previous approximation :
- (iv)
- Various strategies for the choice of the forcing sequence can be found in [18] but for the inexact methods for solving smooth equations. Therefore, most of these strategies would need to be modified with respect to the nondifferentiability of F. For example, we could set as a small constant or use the following rule:where . Then, Equation (8) holds with equality.
Before the convergence of Algorithm 1 is proven, we present a necessary important assumption:
| Algorithm 1: Inexact exponential method (IEM). |
Assume that , and . Let be an arbitrary starting approximation. The steps for obtaining from are as follows: Step 1: Determine and find an value that satisfies
Step 2: If
Otherwise, choose , set , and repeat Step 2. |
Assumption 1.
Function F satisfies Assumption 1 at x if for any y from a neighborhood N of x and any matrix , it holds that
Furthermore, function F satisfies Assumption 1 at x with a degree ρ if the following holds:
Remark 3.
- (i)
- It is not difficult to indicate functions that satisfy Assumption 1. We can indicate three classes of such functions. Assumption 1 is implied not only by semismoothness but also by H-differentiability (introduced by Song et al. [19]) and second-order C-differentiability (introduced by Qi [10]).
- (ii)
- If function F is BD-regular at x and satisfies Assumption 1 at x, then there is a neighborhood N of x and a constant such that for any and matrix , the following is true:
- (iii)
- Lemma 2, proven in [7], states that if function F is BD-regular at , satisfies Assumption 1 at and , where and , thenwhere N is a neighborhood of .
Now, we establish some theoretical foundations for Algorithm 1:
Lemma 3.
Assume that Algorithm 1 is carried out with determined in such way that the inequality in Equation (9) holds. If the series is divergent, then
Proof of Lemma 3.
Under Equation (9), the following is true:
Since and , convergence to 0 follows from the divergence of . □
The divergence of series implies a satisfactory decrease in over all steps such that , namely if the inequality in Equation (9) holds.
Lemma 4.
Let and x be given, and assume that there is , which satisfies , where . If function F satisfies Assumption 1 at x, then there is such that for any , there exists h, which satisfies
where .
Proof of Lemma 4.
Obviously, and .
Set
where is sufficiently small for the following inequality to be satisfied:
whenever . Under Assumption 1, such a value of exists. Since , then .
Let
for any . Then, we have
In addition, since
then
□
Lemma 4 above implies that the inexact exponential step is well-defined for the current approximation , and suitable and values exist for any .
Now, we will prove that the method described by Algorithm 1 is convergent. The first theorem regarding convergence is analogous to the one presented in [20] for the inexact generalized Newton method for solving nonsmooth equations, albeit with some significant changes in the proof:
Theorem 1.
Let be a sequence generated by Algorithm 1 with , and for each k, there is a constant such that the following holds:
and
for any matrix .
If function F satisfies Assumption 1 at , then every limiting point of the sequence is a solution to Equation (1), and
Proof of Theorem 1.
Suppose that K is a sequence of indices such that . We have two cases due to the behavior of the sequence :
- (i)
- is not convergent to 0.
Then, there is a sequence of indices and such that for all . Clearly, for all , the following is true:
In turn, according to Equation (9), we have
By adding all these inequalities side by side, we obtain the following inequality:
Therefore, . Hence, .
- (ii)
- is convergent to 0.
For k at large enough values, under the choice (according to Step 2 of Algorithm 1), there is , such that
and
Therefore, for some such that Equation (8) is satisfied, it holds that
which implies that
and finally
Because , then
Since is bounded, and tend toward zero, Assumption 1 and the local Lipschitz continuity of function F imply that the product on the right side of the above inequality tends toward zero for large enough values of k. Hence, and .
Let be arbitrary. Let be such that . Note that . Therefore, according to Equation (9), the following holds:
Therefore, if , we have that , and thus
□
The main theoretical result of this section is presented in the theorem below. Theorem 2 establishes the sufficient conditions for superlinear convergence of the inexact exponential method for solving nonsmooth equations:
Theorem 2.
Let be a sequence generated by Algorithm 1 with . If is a limiting point of , function F is BD-regular at and satisfies Assumption 1 at , then the sequence is superlinearly convergent to .
Proof of Theorem 2.
If matrix is nonsingular, then by Equation (8), we have that
From the BD-regularity of function F at , there is a such that for any and any , we have . Therefore, we have
Thus, from Theorem 1, it follows that
and every limiting point of is a solution to Equation (1). Furthermore, the construction of new approximation in Algorithm 1 and Equation (12) implies that
Since function F is BD-regular at , the inverse function theorem guarantees that there is such that , provided . Let be arbitrary. Then, the set cannot contain any solution. Therefore, not only does it not contain any limiting point, but it can also contain a finite number of approximations. Hence, there is such that, for all , we have
Let be such that
Since is a limiting point, there is such that , and hence
However, we have that because . Therefore, we also have that for all integers . A number is arbitrary, so the sequence converges to a solution to Equation (1).
Since both and , we obtain
Thus, for k values that are large enough, we have
Therefore, the inequality in Equation (9) holds with , and for k values that are large enough, we obtain
Hence, we have
Now, under the local Lipschitz continuity and BD-regularity of function F at and Assumption 1, there exists such that
for all x in a neighborhood of (see Remark 3 (iii)). Then, according to Equation (14), we have
□
Remark 4.
- (i)
- It is not hard to check that Theorem 2 is still true if our new Algorithm 1 is used with one of other subdifferentials mentioned in Section 1.
- (ii)
- The method is quadratically convergent if the function F satisfies Assumption 1 at with a degree of two.
Some versions of the inexact Newton methods have superlinear convergence under an assumption called a local error bound condition. It requires that
holds for a positive constant c in a neighborhood N of the solution to Equation (1). A local error bound holds, for example, for a piecewise linear function with non-unique solutions.
In solving smooth equations, a local error bound condition is noticeably weaker than the nonsingularity of the Jacobian (see [21]). In our nonsmooth case, this condition can be used as a weaker assumption instead of the BD-regularity. Obviously, for our inexact exponential method, both the BD-regularity and local error bound perform the same role as the nonsingularity of the Jacobian in a smooth case.
4. Numerical Experiments
Now, we present some numerical results which help us to confirm the theoretical properties of the algorithm. We solved a few different problems. Algorithm 1 was implemented in C++ with double precision arithmetic and performed using Apache NetBeans. In all examples, the crucial parameters were set to be alike as follows: , for all k and . The stopping criterion was the condition . A flowchart of Algorithm 1 is presented in Figure 1. In our opinion, the results of the numerical computations were satisfactory, as we will now demonstrate.
Figure 1.
Flowchart for Algorithm 1.
In all examples, is the starting approximation, and is the number of iterations performed by Algorithm 1. For comparison of the efficiency of the new method, we also show the results obtained in tests by the inexact generalized Newton (IGM) method presented in [20] (), which also has superlinear convergence.
Example 1.
As the first (double) test problem, we chose the usual Equation (1) with two different nonsmooth functions:
The equation with function has one solution , while the equation with function has two solutions: and . Both equations have one nondifferentiability point each: for and for .
Both functions and are locally Lipschitz at their solution points. is at because
Therefore, is also semismooth at . In turn, Qi and Sun proved in [14] that the function is semismooth at some point if and only if all of its component functions are semismooth at this point. And the component functions of are semismooth as the sum of the semismooth functions (the local Lipschitz continuities of the absolute value function and quadratic function are easy to check). Moreover, the Bouligand subdifferential of at has the form
The equations can be solved by Algorithm 1. The only point of nondifferentiability of the function is not a solution, and function satisfies Assumption 1 (it follows from Remark 3(i)) and is BD-regular at . The exemplary results are shown in Table 1 and Table 2.
Table 1.
The numerical results obtained for the equation with function from Example 1.
Table 2.
The numerical results obtained for the equation with function from Example 1 ( indicates to which solution the sequence converged.).
Example 2.
The second test problem was Equation (1) with the function (used among the others in [4]), defined by
with
Obviously, if , then function F is differentiable. Therefore, the value may be treated as the degree of nondifferentiability of function F [22]. Equation (1) has an infinite number of solutions , where are arbitrary. First, the starting approximation was . We used Algorithm 1 to solve one series of smooth problems (with ) and three series of nonsmooth problems with various parameters .
Function F is semismooth as a composite of semismooth functions (as mentioned in Example 1), and all functions are semismooth as piecewise smooth functions (see [13]).
In the smooth case, Algorithm 1 (IEM) stopped after six iterations, while Algorithm IGM generated seven successive approximations. The most interesting results we selected are presented in Table 3.
Table 3.
The numerical results obtained for the equation from Example 2 (n is the number of component functions.).
In these tests, we noticed that the choice of starting point did not affect the number of iterations very much when Algorithm 1 generated a convergent sequence. Therefore, we omitted the results for other starting points. The number of iterations was actually very similar for a small number of variables. However, Algorithm 1 showed its advantage for larger problems (i.e., when a system of nonlinear equations has at least a dozen unknowns).
Example 3.
The last test problem was the Walrasian production–price problem [23] with the demand function ξ defined in the following form:
Here, is the price vector, measures the i-th consumer’s intensity of demand for commodity j, and is the i-th consumer’s elasticity of substitution for . Furthermore, the following
is a function of the price chosen in such way to satisfy the budget constraint for each unit. Let denote the technology, denote the total endowment of the economy prior to production and denote the production plan.
Then, the Walrasian model can be written as the nonlinear complementarity problem (NCP) in the form
with the following functions:
where the operator “min” denotes the component-wise minimum of the vectors.
Function is locally Lipschitzian, semismooth and BD-regular, so we can use Algorithm 1 to solve nonsmooth Equation (1).
An approximate solution of such a problem (NCP) with , and is
We solved this problem using the inexact exponential method with various starting approximations from the neighborhood of in the form
where and . The exemplary results are shown in Table 4.
Table 4.
The numerical results obtained for the Walrasian production–price problem from Example 3 with various parameters and .
The Broyden-like method (from [24]) generated somewhat comparable results, which was not surprising since all these methods have the same order of convergence. The number of iterations was equal to 27, 9, −, 66 and 263 for the same initial points as in Table 4. In this test problem, the convergence significantly depended on the location of the starting point (parameters and ). Here, Algorithm 1 showed its advantage for initial points further from the solution.
5. Conclusions
We have proposed a new inexact exponential method for solving systems of nonsmooth equations (Equation (1)). We proved that the presented method has global and superlinear convergence under some mild assumptions when function F is locally Lipschitz continuous. In practice, Algorithm 1 (IEM) was significantly faster than the usual inexact generalized Newton (IGM) method. The used globalization technique turned out to be an effective approach to improve convergence behavior. The results of the numerical tests confirmed that the new inexact exponential method can be used to effectively solve miscellaneous nonsmooth problems such as nonsmooth equations, nonlinear complementarity problems or equlibrium problems.
Funding
This research received no external funding.
Institutional Review Board Statement
Not applicable.
Data Availability Statement
Data sharing is not applicable to this article.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Pang, J.; Qi, L. Nonsmooth equations: Motivation and algorithms. SIAM J. Optim. 1993, 3, 443–465. [Google Scholar] [CrossRef]
- Chen, J.; Li, W. On new exponential quadratically convergent iterative formulae. Appl. Math. Comput. 2006, 180, 242–246. [Google Scholar] [CrossRef]
- Śmietański, M.J. On a new exponential iterative method for solving nonsmooth equations. Numer. Linear Algebra Appl. 2019, 25, e2255. [Google Scholar] [CrossRef]
- Martínez, J.M.; Qi, L. Inexact Newton method for solving nonsmooth equations. J. Comput. Appl. Math. 1995, 60, 127–145. [Google Scholar] [CrossRef]
- Pu, D.; Tian, W. Globally convergent inexact generalized Newton’s methods for nonsmooth equations. J. Comput. Appl. Math. 2002, 138, 37–49. [Google Scholar] [CrossRef]
- Bonettini, S.; Tinti, F. A nonmonotone semismooth inexact Newton method. Optim. Meth. Soft. 2007, 22, 637–657. [Google Scholar] [CrossRef][Green Version]
- Śmietański, M.J. Inexact quasi-Newton global convergent method for solving constrained nonsmooth equations. Int. J. Comput. Math. 2007, 84, 1157–1170. [Google Scholar] [CrossRef]
- Kahya, E. A class of exponential quadratically convergent iterative formulae for unconstrained optimization. Appl. Math. Comp. 2007, 186, 1010–1017. [Google Scholar] [CrossRef]
- Clarke, F.H. Optimization and Nonsmooth Analysis; SIAM: Philadelphia, PA, USA, 1990. [Google Scholar]
- Qi, L. C-Differential Operators, C-Differentiability and Generalized Newton Methods; Applied Mathematics Report AMR 96/5; University of New South Wales: Sydney, Australia, 1996. [Google Scholar]
- Sun, D.; Han, J. Newton and quasi-Newton methods for a class of nonsmooth equations and related problems. SIAM J. Optim. 1997, 7, 463–480. [Google Scholar] [CrossRef]
- Gao, Y. Newton methods for solving nonsmooth equations via a new subdifferential. Math. Meth. Oper. Res. 2001, 54, 239–257. [Google Scholar] [CrossRef]
- Mifflin, R. Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Optim. 1977, 15, 959–972. [Google Scholar] [CrossRef]
- Qi, L.; Sun, J. A nonsmooth version of Newton’s method. Math. Program. 1993, 58, 353–367. [Google Scholar] [CrossRef]
- Qi, L. Convergence analysis of some algorithms for solving nonsmooth equations. Math. Oper. Res. 1993, 18, 227–244. [Google Scholar] [CrossRef]
- Potra, F.A.; Qi, L.; Sun, D. Secant methods for semismooth equations. Numer. Math. 1998, 80, 305–324. [Google Scholar] [CrossRef]
- Birgin, E.; Krejić, N.; Martínez, J.M. Globally convergent inexact quasi-Newton methods for solving nonlinear systems. Numer. Algorithms 2003, 32, 249–260. [Google Scholar] [CrossRef]
- An, H.; Mo, Z.; Liu, X. A choice of forcing terms in inexact Newton method. J. Comput. Appl. Math. 2007, 200, 47–60. [Google Scholar] [CrossRef]
- Song, Y.; Gowda, M.S.; Ravindran, G. On the characterizations of P- and P0-properties in nonsmooth functions. Math. Oper. Res. 2000, 25, 400–408. [Google Scholar] [CrossRef]
- Śmietański, M.J. Some superlinearly convergent inexact quasi-Newton method for solving nonsmooth equations. Optim. Meth. Soft. 2012, 27, 405–417. [Google Scholar] [CrossRef]
- Yamashita, N.; Fukushima, M. On the rate of convergence of the Levenberg-Marquardt method. Computing 2001, 15, 237–249. [Google Scholar]
- Spedicato, E. Computational Experience with Quasi-Newton Algorithms for Minimization Problems of Moderately Large Size; Report CISE-N-175; Centro Informazioni Studi Esperienze: Milan, Italy, 1 October 1975. [Google Scholar]
- Scarf, H. The Computation of Economic Equilibria; Yale University Press: New Haven, CT, USA, 1973. [Google Scholar]
- Chen, X.; Qi, L. A parametrized Newton method and a quasi-Newton method for nonsmooth equations. Comput. Optim. Appl. 1994, 3, 157–179. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
