A Conjugate Gradient Method: Quantum Spectral Polak–Ribiére–Polyak Approach for Unconstrained Optimization Problems
Abstract
:1. Introduction
2. Literature Review
- RQ1.
- When can the spectral PRP method be reduced to the standard PRP conjugate gradient approach?
- RQ2.
- How can the step length be chosen in selecting the subsequent iterative quantum points?
- RQ3.
- What is the impact of using the quantum derivative in place of the classical derivative in the proposed method?
3. A Quantum Spectral PRP CG Algorithm and Convergence Analysis
Algorithm 1 A quantum spectral PRP CG algorithm |
|
- A1.
- The level set is bounded, where the starting point is .
- A2.
- There exists a constant such that θ is continuously quantum-differentiable in the neighborhood N of Ω, and its quantum gradient is Lipschitz continuous, such that
- A3.
- The following inequality holds for λ large enough:
4. Numerical Illustration
5. Discussions
6. Conclusions and Future Prospects
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Mishra, S.K.; Ram, B. Introduction to Unconstrained Optimization with R; Springer: Berlin/Heidelberg, Germany, 2019. [Google Scholar]
- Babaie-Kafaki, S. A survey on the Dai–Liao family of nonlinear conjugate gradient methods. RAIRO Oper. Res. 2023, 57, 43–58. [Google Scholar] [CrossRef]
- Wu, X.; Shao, H.; Liu, P.; Zhang, Y.; Zhuo, Y. An efficient conjugate gradient-based algorithm for unconstrained optimization and its projection extension to large-scale constrained nonlinear equations with applications in signal recovery and image denoising problems. J. Comput. Appl. Math. 2023, 422, 114879. [Google Scholar] [CrossRef]
- Liu, J.; Du, S.; Chen, Y. A sufficient descent nonlinear conjugate gradient method for solving M-tensor equations. J. Comput. Appl. Math. 2020, 371, 112709. [Google Scholar] [CrossRef]
- Andrei, N. On three-term conjugate gradient algorithms for unconstrained optimization. Appl. Math. Comput. 2013, 219, 6316–6327. [Google Scholar] [CrossRef]
- Dai, Y.h.; Yuan, Y. An efficient hybrid conjugate gradient method for unconstrained optimization. Ann. Oper. Res. 2001, 103, 33–47. [Google Scholar] [CrossRef]
- Shanno, D.F. Conjugate gradient methods with inexact searches. Math. Oper. Res. 1978, 3, 244–256. [Google Scholar] [CrossRef]
- Johnson, O.G.; Micchelli, C.A.; Paul, G. Polynomial preconditioners for conjugate gradient calculations. SIAM J. Numer. Anal. 1983, 20, 362–376. [Google Scholar] [CrossRef]
- Wei, Z.; Li, G.; Qi, L. New nonlinear conjugate gradient formulas for large-scale unconstrained optimization problems. Appl. Math. Comput. 2006, 179, 407–430. [Google Scholar] [CrossRef]
- Mishra, S.K.; Chakraborty, S.K.; Samei, M.E.; Ram, B. A q-Polak–Ribière–Polyak conjugate gradient algorithm for unconstrained optimization problems. J. Inequal. Appl. 2021, 2021, 1–29. [Google Scholar] [CrossRef]
- Zhang, L.; Zhou, W.; Li, D.H. A descent modified Polak–Ribière–Polyak conjugate gradient method and its global convergence. IMA J. Numer. Anal. 2006, 26, 629–640. [Google Scholar] [CrossRef]
- Gouvêa, É.J.; Regis, R.G.; Soterroni, A.C.; Scarabello, M.C.; Ramos, F.M. Global optimization using q-gradients. Eur. J. Oper. Res. 2016, 251, 727–738. [Google Scholar] [CrossRef]
- Mishra, S.K.; Panda, G.; Ansary, M.A.T.; Ram, B. On q-Newton’s method for unconstrained multiobjective optimization problems. J. Appl. Math. Comput. 2020, 63, 391–410. [Google Scholar] [CrossRef]
- Lai, K.K.; Mishra, S.K.; Panda, G.; Ansary, M.A.T.; Ram, B. On q-steepest descent method for unconstrained multiobjective optimization problems. AIMS Math. 2020, 5, 5521–5540. [Google Scholar]
- Lai, K.K.; Mishra, S.K.; Panda, G.; Chakraborty, S.K.; Samei, M.E.; Ram, B. A limited memory q-BFGS algorithm for unconstrained optimization problems. J. Appl. Math. Comput. 2021, 66, 183–202. [Google Scholar] [CrossRef]
- Wan, Z.; Yang, Z.; Wang, Y. New spectral PRP conjugate gradient method for unconstrained optimization. Appl. Math. Lett. 2011, 24, 16–22. [Google Scholar] [CrossRef]
- Hu, Q.; Zhang, H.; Zhou, Z.; Chen, Y. A class of improved conjugate gradient methods for nonconvex unconstrained optimization. Numer. Linear Algebra Appl. 2023, 30, e2482. [Google Scholar] [CrossRef]
- Gilbert, J.C.; Nocedal, J. Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim. 1992, 2, 21–42. [Google Scholar] [CrossRef]
- Polyak, B.T. The conjugate gradient method in extremal problems. USSR Comput. Math. Math. Phys. 1969, 9, 94–112. [Google Scholar] [CrossRef]
- Powell, M.J.D. Restart procedures for the conjugate gradient method. Math. Program. 1977, 12, 241–254. [Google Scholar] [CrossRef]
- Yuan, G.; Lu, X. A modified PRP conjugate gradient method. Ann. Oper. Res. 2009, 166, 73–90. [Google Scholar] [CrossRef]
- Wang, C.y.; Chen, Y.y.; Du, S.q. Further insight into the Shamanskii modification of Newton method. Appl. Math. Comput. 2006, 180, 46–52. [Google Scholar] [CrossRef]
- Powell, M.J. Nonconvex minimization calculations and the conjugate gradient method. In Numerical Analysis, Proceedings of the 10th Biennial Conference held at Dundee, Scotland, 28 June–1 July 1983; Springer: Berlin/Heidelberg, Germany, 1983; pp. 122–141. [Google Scholar]
- Li, Y.; Du, S.; Chen, Y. An improved spectral conjugate gradient algorithm for nonconvex unconstrained optimization problems. J. Optim. Theory Appl. 2013, 157, 820–842. [Google Scholar]
- Liu, D.; Zhang, L.; Xu, G. Spectral method and its application to the conjugate gradient method. Appl. Math. Comput. 2014, 240, 339–347. [Google Scholar] [CrossRef]
- Liu, H.; Yao, Y.; Qian, X.; Wang, H. Some nonlinear conjugate gradient methods based on spectral scaling secant equations. J. Comput. Appl. Math. 2016, 35, 639–651. [Google Scholar]
- Tarzanagh, D.A.; Nazari, P.; Peyghami, M.R. A nonmonotone PRP conjugate gradient method for solving square and under-determined systems of equations. Comput. Math. Appl. 2017, 73, 339–354. [Google Scholar] [CrossRef]
- Zhu, Z.; Wang, H.; Zhang, B. A spectral conjugate gradient method for nonlinear inverse problems. Comput. Math. Appl. 2018, 26, 1561–1589. [Google Scholar] [CrossRef]
- Awwal, A.M.; Kumam, P.; Abubakar, A.B. Spectral modified Polak–Ribiére–Polyak projection conjugate gradient method for solving monotone systems of nonlinear equations. Appl. Math. Comput. 2019, 362, 124514. [Google Scholar] [CrossRef]
- Guo, J.; Wan, Z. A new three-term spectral conjugate gradient algorithm with higher numerical performance for solving large scale optimization problems based on Quasi-Newton equation. Int. J. Model. Simul. Sci. Comput. 2021, 12, 2150053. [Google Scholar] [CrossRef]
- Jian, J.; Yang, L.; Jiang, X.; Liu, P.; Liu, M. A spectral conjugate gradient method with descent property. Mathematics 2020, 8, 280. [Google Scholar] [CrossRef]
- Li, Y.; Du, S.; Chen, Y. Modified spectral PRP conjugate gradient method for solving tensor eigenvalue complementarity problems. J. Ind. Manag. Optim. 2022, 18, 157–172. [Google Scholar] [CrossRef]
- Lai, K.K.; Mishra, S.K.; Sharma, R.; Sharma, M.; Ram, B. A Modified q-BFGS Algorithm for Unconstrained Optimization. Mathematics 2023, 11, 1420. [Google Scholar] [CrossRef]
- Lai, K.K.; Mishra, S.K.; Ram, B. A q-Conjugate Gradient Algorithm for Unconstrained Optimization Problems. Pac. J. Opt. 2021, 17, 57–76. [Google Scholar]
- Jamil, M.; Yang, X.S. A literature survey of benchmark functions for global optimisation problems. Int. J. Math. Model. Numer. 2013, 4, 150–194. [Google Scholar] [CrossRef]
- Dolan, E.D.; Moré, J.J. Benchmarking optimization software with performance profiles. Math. Program. 2002, 91, 201–213. [Google Scholar] [CrossRef]
- Al-Saggaf, U.M.; Moinuddin, M.; Arif, M.; Zerguine, A. The q-least mean squares algorithm. Signal Process. 2015, 111, 50–60. [Google Scholar] [CrossRef]
- Cai, P.; Wang, S.; Qian, J.; Zhang, T.; Huang, G. The diffusion least mean square algorithm with variable q-gradient. Signal Process. 2022, 127, 50–60. [Google Scholar] [CrossRef]
Source | Description | Advantage | Evaluation Tool |
---|---|---|---|
Wan et al. [16] | A new spectral PRP conjugate gradient algorithm | Search direction at each iteration is a descent direction | Classical derivative |
Deng et al. [24] | An improved spectral conjugate gradient algorithm | The search direction is sufficiently close to the quasi-Newton direction | Classical derivative |
Liu et al. [25] | A global convergence of the nonlinear conjugate gradient with the spectral method | An iteration matrix is positive definite symmetrical | Classical derivative |
Liu et al. [26] | Three conjugate gradient methods based on the spectral equations | An approximation to the spectral of the Hessian of the objective function is adopted | Classical derivative |
Tarzanagh et al. [27] | A non-monotone PRP conjugate gradient method for solving square and under-determined systems of equations | A relaxed non-monotone line search technique | Derivative-free |
Serial | Quantum Spectral PRP Algorithm | ||||
---|---|---|---|---|---|
Number | Test Problem | Starting Point | |||
1 | Rosenbrock | (3 , 4 )⌃T | (0.9993397 , 0.9986580)⌃T | 4.84E-07 | 16 |
2 | Rosenbrock | (4 , 3)⌃T | (1.065881, 1.136169)⌃T | 0.004340773 | 13 |
3 | SPHERE | (1 , 2 , 3)⌃T | (4.438117E-14 , 9.803269E-14 , −1.129652E-13)⌃T | 2.43E-26 | 3 |
4 | SPHERE | (3 , 2 , 1)⌃T | (−1.293611E-06 , −1.037781E-06 , −7.819502E-07)⌃T | 3.36E-12 | 3 |
5 | ACKLEY | (0.2 , 0.2)⌃T | (−4.105524E-07 , −4.105524E-07)⌃T | 1.64E-06 | 3 |
6 | ACKLEY 2 | (−6 , −6)⌃T | (−0.00000000003559351 , −0.00000000003559351)⌃T | −200 | 2 |
7 | ACKLEY 2 | (−4 , −5)⌃T | (1.633527E-09 , 2.246892E-09)⌃T | −200 | 2 |
8 | Beale | (3 , 2)⌃T | (3.0005905 , 0.5001345)⌃T | 1.45E-10 | 11 |
9 | Beale | (4 , 1)⌃T | (3.0095993 , 0.5024505)⌃T | 1.47E-05 | 9 |
10 | Bohachevsky | (0.2 , 0.3)⌃T | (−5.841102E-07 , 3.085369E-08)⌃T | 4.92E-12 | 5 |
11 | Bohachevsky | (0.1 , 0.2)⌃T | (2.276130E-06 , −1.151592E-07)⌃T | 7.47E-11 | 5 |
12 | Booth | (40 , 5)⌃T | (0.9999947 , 3.0000062)⌃T | 6.888418E-11 | 4 |
13 | Booth | (60 , 80)⌃T | ( 0.9994183 , 3.0008450)⌃T | 1.33E-06 | 6 |
14 | DROP-WAVE | (0.1 , 0.2)⌃T | (4.767480E-14 , 2.085834E-14)⌃T | −1 | 3 |
15 | Colville | (1 , 1 , 1 , 0.8)⌃T | (1.0394380 , 1.0802997 , 0.9614360 , 0.9241323)⌃T | 0.00568415 | 7 |
16 | Colville | (0.8 , 1 , 1 , 1)⌃T | (0.9768628 , 0.9543990 , 1.0241951 , 1.0488717)⌃T | 0.00212309 | 11 |
17 | Csendes | (−4 , −5)⌃T | (−0.03013482 , −0.01924908)⌃T | 8.146356E-10 | 7 |
18 | Csendes | (0.4 , 1)⌃T | (−0.01171701 , 0.01905952)⌃T | 1.41E-10 | 5 |
19 | Cube | (−7 , 10)⌃T | (0.9805592 , 0.9422543)⌃T | 0.000408162 | 20 |
20 | Cube | (1 , −6)⌃T | ( 0.9397829 , 0.8228094)⌃T | 0.008808971 | 13 |
21 | Deckkers-Aarts | (10 , 50)⌃T | (9.597778E-07 , 1.494765E+01)⌃T | −24,776.51 | 8 |
22 | Deckkers-Aarts | (9, 40)⌃T | (−1.078035E-06 , 1.494759E+01)⌃T | −24 , 776.51 | 10 |
23 | Dixon Price | (7 , 4)⌃T | (1.001180 , −0.707635)⌃T | 1.59E-06 | 8 |
24 | Easom | (2.5 , 2.1)⌃T | (3.142261 , 3.141810)⌃T | −1 | 4 |
25 | Egg Crate | (−1.3 , −1.6)⌃T | (−5.302490E-07 , −4.244362E-07)⌃T | 1.20E-11 | 6 |
26 | Exponential | (−3 , −1)⌃T | ( 4.306445E-08 , 1.072428E-08)⌃T | −1 | 4 |
27 | Freudenstein Roth | (4 , 5)⌃T | (4.981870 , 4.000912)⌃T | 0.001151386 | 7 |
28 | Six Hump Camel | (7 , 1)⌃T | (−0.08978278 , 0.71276481)⌃T | −1.031628 | 9 |
29 | Three Hump Camel | (0.6 , 0.7)⌃T | (−2.648394E-07 , −5.719217E-07)⌃T | 6.188417E-13 | 6 |
30 | Sum Squares | (4 , 70)⌃T | (−1.110432E-05 , 1.337182E-05)⌃T | 4.81E-10 | 4 |
31 | GRAMACY and LEE | 1 | 0.9490034 | −0.5266035 | 3 |
32 | Rotated Ellipse 2 | (100 , 1.4) | (−1.438029E-07 , −1.555622E-07)⌃T | 2.25E-14 | 4 |
33 | Zakharov | (−6 , −5) | (2.640608E-07 , −3.044188E-07)⌃T | 1.92E-13 | 8 |
34 | Zirilli | (3 , 7)⌃T | (−1.04651E+00 , −2.308637E-05)⌃T | −0.352386 | 6 |
35 | Zett1 | (8 , 4)⌃T | (−2.990219E-02 , −2.078441E-08)⌃T | −0.003791237 | 9 |
36 | Wayburn Seader 3 | (5 , 6)⌃T | (5.147307, 6.839722)⌃T | 19.10588 | 5 |
37 | Wayburn Seader 2 | (1 , 1)⌃T | (0.4604724 , 1.0073360)⌃T | 19.20677 | 7 |
Serial | Spectral PRP Algorithm Given by Wan et al. [16] | ||||
---|---|---|---|---|---|
Number | Test Problem | Starting Point | |||
1 | Rosenbrock | (3 , 4 )⌃T | (0.9999998 , 0.9999995)⌃T | 4.84E-07 | 20 |
2 | Rosenbrock | (4 , 3)⌃T | (0.9999194 , 0.9998484)⌃T | 1.56E-08 | 16 |
3 | SPHERE | (1 , 2 , 3)⌃T | (−4.275638E-07 , -3.976857E-07, −3.678076E-07)⌃T | 4.76E-13 | 3 |
4 | SPHERE | (3 , 2 , 1)⌃T | (−1.125211E-13 , 9.832413E-14, −3.757411E-14)⌃T | 2.37E-26 | 3 |
5 | ACKLEY | (0.2 , 0.2)⌃T | (3.764868E-11 , 3.764868E-11)⌃T | 1.51E-10 | 4 |
6 | ACKLEY 2 | (−6 , −6)⌃T | (3.630772E-08 , 3.630772E-08)⌃T | −200 | 2 |
7 | ACKLEY 2 | (−4 , −5)⌃T | (−4.263692E-07 , 3.132365E-08)⌃T | −200 | 11 |
8 | Beale | (3 , 2)⌃T | (3.0000951 , 0.5000238)⌃T | 3.41E-09 | 12 |
9 | Beale | (4 , 1)⌃T | (3.0000004, 0.5000001)⌃T | 2.77E-14 | 10 |
10 | Bohachevsky | (0.2 , 0.3)⌃T | (3.324145E-13 , −2.460865E-14)⌃T | 0 | 5 |
11 | Bohachevsky | (0.1 , 0.2)⌃T | ( 1.990839E-06 , −3.959075E-07)⌃T | 6.20E-11 | 5 |
12 | Booth | (40 , 5)⌃T | (1 , 3)⌃T | 2.55E-17 | 3 |
13 | Booth | (60 , 80)⌃T | (1 , 3)⌃T | 8.79E-17 | 3 |
14 | DROP-WAVE | (0.1 , 0.2)⌃T | (−5.062833E-07, −5.107584E-07)⌃T | −1 | 4 |
15 | Colville | (1 , 1 , 1 , 0.8)⌃T | (0.9995939 , 0.9991930 , 1.0004239 , 1.0008541)⌃T | 6.49E-07 | 20 |
16 | Colville | (0.8 , 1 , 1 , 1)⌃T | (0.9997061 , 0.9994101 , 1.0002997 , 1.0005954)⌃T | 3.19E-07 | 25 |
17 | Csendes | (−4 , −5)⌃T | (−0.03055821, −0.01731213)⌃T | 8.71E-10 | 8 |
18 | Csendes | (0.4 , 1)⌃T | (0.01957772 , 0.01183933)⌃T | 1.60E-10 | 4 |
19 | Cube | (−7, 10)⌃T | (0.9999933, 0.9999799)⌃T | 4.51E-11 | 22 |
20 | Cube | (1 , −6)⌃T | (1 , 1)⌃T | 9.29E-17 | 20 |
21 | Deckkers-Aarts | (10 , 50)⌃T | (4.652559E-08 , 1.494511E+01)⌃T | −24,776.52 | 9 |
22 | Deckkers-Aarts | (9 , 40)⌃T | (3.694906E-07 , 1.494511E+01)⌃T | −24,776.52 | 8 |
23 | Dixon Price | (7 , 4)⌃T | (1.0000020 , −0.7071099)⌃T | 9.87E-11 | 9 |
24 | Easom | (2.5 , 2.1)⌃T | (3.141593 , 3.141593)⌃T | −1 | 5 |
25 | Egg Crate | (−1.3 , −1.6)⌃T | (−8.137993E-07 , −8.800207E-08)⌃T | 1.74204E-11 | 6 |
26 | Exponential | (−3 , −1)⌃T | (1.527478E-10 , −4.751579E-10)⌃T | −1 | 3 |
27 | Freudenstein Roth | (4 , 5)⌃T | (5.000009, 4.000000)⌃T | 1.18699E-10 | 8 |
28 | Six Hump Camel | (7 , 1)⌃T | (−0.08984676 , 0.71266325)⌃T | −1.031639 | 8 |
29 | Three Hump Camel | (0.6 , 0.7)⌃T | ( −1.088238E-10 , 1.920156E-10)⌃T | 3.97E-20 | 6 |
30 | Sum Squares | (4, 70)⌃T | (−4.084082E-06 , −4.088583E-06)⌃T | 4.83E-10 | 4 |
31 | GRAMACY and LEE | 1 | −0.5266048 | 0.9489337 | 3 |
32 | Rotated Ellipse 2 | (100 , 1.4) | ( −1.249927E-06 , −1.367064E-06)⌃T | 1.72E-12 | 5 |
33 | Zakharov | (−6 , −5) | (1.359022E-15 , −5.823407E-16)⌃T | 2.20E-30 | 8 |
34 | Zirilli | (3 , 7)⌃T | ( −1.046681E+00, 3.696484E-12)⌃T | −0.3523861 | 8 |
35 | Zett1 | (8 , 4)⌃T | (−2.989599E-02 , −6.894129E-08)⌃T | −0.003791237 | 10 |
36 | Wayburn Seader 3 | (5 , 6)⌃T | (5.146885 , 6.839598)⌃T | 19.10588 | 7 |
37 | Wayburn Seader 2 | (1 , 1)⌃T | (0.4248603 , 0.9999999)⌃T | 1.68E-14 | 10 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lai, K.K.; Mishra, S.K.; Ram, B.; Sharma, R. A Conjugate Gradient Method: Quantum Spectral Polak–Ribiére–Polyak Approach for Unconstrained Optimization Problems. Mathematics 2023, 11, 4857. https://doi.org/10.3390/math11234857
Lai KK, Mishra SK, Ram B, Sharma R. A Conjugate Gradient Method: Quantum Spectral Polak–Ribiére–Polyak Approach for Unconstrained Optimization Problems. Mathematics. 2023; 11(23):4857. https://doi.org/10.3390/math11234857
Chicago/Turabian StyleLai, Kin Keung, Shashi Kant Mishra, Bhagwat Ram, and Ravina Sharma. 2023. "A Conjugate Gradient Method: Quantum Spectral Polak–Ribiére–Polyak Approach for Unconstrained Optimization Problems" Mathematics 11, no. 23: 4857. https://doi.org/10.3390/math11234857
APA StyleLai, K. K., Mishra, S. K., Ram, B., & Sharma, R. (2023). A Conjugate Gradient Method: Quantum Spectral Polak–Ribiére–Polyak Approach for Unconstrained Optimization Problems. Mathematics, 11(23), 4857. https://doi.org/10.3390/math11234857