An Improved Blind Kriging Surrogate Model for Design Optimization Problems
Abstract
:1. Introduction
2. Improved Blind Kriging
2.1. Basics
2.2. Variable Selection
3. Infill Strategy
Algorithm 1 The combined infill strategy for the expensive constrained optimization problem |
|
4. Numerical Examples
4.1. Study in High-Order Effects of the Trend Function
4.2. Synthetic Test Problems
4.3. Structural Design Optimization
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A
References
- Amine Bouhlel, M.; Bartoli, N.; Regis, R.G.; Otsmane, A.; Morlier, J. Efficient global optimization for high-dimensional constrained problems by using the Kriging models combined with the partial least squares method. Eng. Optim. 2018, 50, 2038–2053. [Google Scholar] [CrossRef]
- Gu, J.; Zhang, H.; Zhong, X. Hybrid meta-model-based global optimum pursuing method for expensive problems. Struct. Multidiscip. Optim. 2019, 61, 543–554. [Google Scholar] [CrossRef]
- Myers, R.H.; Montgomery, D.C.; Anderson-Cook, C.M. Response Surface Methodology: Process and Product Optimization Using Designed Experiments; John Wiley & Sons: Hoboken, NJ, USA, 2016. [Google Scholar]
- Box, G.E.; Draper, N.R. Empirical Model-Building and Response Surfaces; Wiley: New York, NY, USA, 1987; Volume 424. [Google Scholar]
- Powell, M. The Theory of Radial Basis Function Approximation in 1990, Advances in Numerical Analysis II: Wavelets, Subdivision, and Radial Functions (WA Light, ed.); Oxford University Press: Oxford, UK, 1992; Volume 105, p. 210. [Google Scholar]
- Wu, Y.; Yin, Q.; Jie, H.; Wang, B.; Zhao, J. A RBF-based constrained global optimization algorithm for problems with computationally expensive objective and constraints. Struct. Multidiscip. Optim. 2018, 58, 1633–1655. [Google Scholar] [CrossRef]
- Sacks, J.; Welch, W.J.; Mitchell, T.J.; Wynn, H.P. Design and analysis of computer experiments. Stat. Sci. 1989, 4, 409–423. [Google Scholar] [CrossRef]
- Sakata, S.; Ashida, F.; Zako, M. Structural optimization using Kriging approximation. Comput. Methods Appl. Mech. Eng. 2003, 192, 923–939. [Google Scholar] [CrossRef]
- Clarke, S.M.; Griebsch, J.H.; Simpson, T.W. Analysis of support vector regression for approximation of complex engineering analyses. J. Mech. Des. 2005, 127, 1077–1087. [Google Scholar] [CrossRef]
- Berke, L.; Hajela, P. Applications of artificial neural nets in structural mechanics. In Shape and Layout Optimization of Structural Systems and Optimality Criteria Methods; Springer: Berlin/Heidelberg, Germany, 1992; pp. 331–348. [Google Scholar]
- Mai, H.T.; Kang, J.; Lee, J. A machine learning-based surrogate model for optimization of truss structures with geometrically nonlinear behavior. Finite Elem. Anal. Des. 2021, 196, 103572. [Google Scholar] [CrossRef]
- Zhang, W.; Li, X.; Ma, H.; Luo, Z.; Li, X. Federated learning for machinery fault diagnosis with dynamic validation and self-supervision. Knowl.-Based Syst. 2021, 213, 106679. [Google Scholar] [CrossRef]
- Zhang, W.; Li, X.; Ma, H.; Luo, Z.; Li, X. Universal domain adaptation in fault diagnostics with hybrid weighted deep adversarial learning. IEEE Trans. Ind. Inform. 2021, 17, 7957–7967. [Google Scholar] [CrossRef]
- Shepard, D. A two-dimensional interpolation function for irregularly-spaced data. In Proceedings of the 1968 23rd ACM National Conference, New York, NY, USA, 27–29 August 1968; pp. 517–524. [Google Scholar]
- Shields, M.D.; Zhang, J. The generalization of Latin hypercube sampling. Reliab. Eng. Syst. Saf. 2016, 148, 96–108. [Google Scholar] [CrossRef]
- Jones, D.R.; Schonlau, M.; Welch, W.J. Efficient global optimization of expensive black-box functions. J. Glob. Optim. 1998, 13, 455–492. [Google Scholar] [CrossRef]
- Huang, D.; Allen, T.T.; Notz, W.I.; Zeng, N. Global optimization of stochastic black-box systems via sequential kriging meta-models. J. Glob. Optim. 2006, 34, 441–466. [Google Scholar] [CrossRef]
- Huang, D.; Allen, T.T.; Notz, W.I.; Miller, R.A. Sequential kriging optimization using multiple-fidelity evaluations. Struct. Multidiscip. Optim. 2006, 32, 369–382. [Google Scholar] [CrossRef]
- Dong, H.; Song, B.; Wang, P.; Huang, S. A kind of balance between exploitation and exploration on kriging for global optimization of expensive functions. J. Mech. Sci. Technol. 2015, 29, 2121–2133. [Google Scholar] [CrossRef]
- Zhao, L.; Wang, P.; Song, B.; Wang, X.; Dong, H. An efficient kriging modeling method for high-dimensional design problems based on maximal information coefficient. Struct. Multidiscip. Optim. 2019, 61, 39–57. [Google Scholar] [CrossRef]
- Gutmann, H.M. A radial basis function method for global optimization. J. Glob. Optim. 2001, 19, 201–227. [Google Scholar] [CrossRef]
- Regis, R.G.; Shoemaker, C.A. Improved strategies for radial basis function methods for global optimization. J. Glob. Optim. 2007, 37, 113–135. [Google Scholar] [CrossRef]
- Regis, R.G.; Shoemaker, C.A. A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J. Comput. 2007, 19, 497–509. [Google Scholar] [CrossRef]
- Regis, R.G.; Shoemaker, C.A. Parallel stochastic global optimization using radial basis functions. INFORMS J. Comput. 2009, 21, 411–426. [Google Scholar] [CrossRef]
- Joseph, V.R.; Kang, L. Regression-based inverse distance weighting with applications to computer experiments. Technometrics 2011, 53, 254–265. [Google Scholar] [CrossRef]
- Bemporad, A. Global optimization via inverse distance weighting. arXiv 2019, arXiv:1906.06498. [Google Scholar]
- Regis, R.G. Stochastic radial basis function algorithms for large-scale optimization involving expensive black-box objective and constraint functions. Comput. Oper. Res. 2011, 38, 837–853. [Google Scholar] [CrossRef]
- Regis, R.G. Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points. Eng. Optim. 2014, 46, 218–243. [Google Scholar] [CrossRef]
- Nuñez, L.; Regis, R.G.; Varela, K. Accelerated random search for constrained global optimization assisted by radial basis function surrogates. J. Comput. Appl. Math. 2018, 340, 276–295. [Google Scholar] [CrossRef]
- Li, Y.; Wu, Y.; Zhao, J.; Chen, L. A Kriging-based constrained global optimization algorithm for expensive black-box functions with infeasible initial points. J. Glob. Optim. 2017, 67, 343–366. [Google Scholar] [CrossRef]
- Carpio, R.R.; Giordano, R.C.; Secchi, A.R. Enhanced surrogate assisted framework for constrained global optimization of expensive black-box functions. Comput. Chem. Eng. 2018, 118, 91–102. [Google Scholar] [CrossRef]
- Qian, J.; Yi, J.; Cheng, Y.; Liu, J.; Zhou, Q. A sequential constraints updating approach for Kriging surrogate model-assisted engineering optimization design problem. Eng. Comput. 2019, 36, 993–1009. [Google Scholar] [CrossRef]
- Bartoli, N.; Lefebvre, T.; Dubreuil, S.; Olivanti, R.; Priem, R.; Bons, N.; Martins, J.R.; Morlier, J. Adaptive modeling strategy for constrained global optimization with application to aerodynamic wing design. Aerosp. Sci. Technol. 2019, 90, 85–102. [Google Scholar] [CrossRef]
- Forrester, A.; Sobester, A.; Keane, A. Engineering Design via Surrogate Modelling: A Practical Guide; John Wiley & Sons: Hoboken, NJ, USA, 2008. [Google Scholar]
- Shi, R.; Liu, L.; Long, T.; Wu, Y.; Tang, Y. Filter-based adaptive Kriging method for black-box optimization problems with expensive objective and constraints. Comput. Methods Appl. Mech. Eng. 2019, 347, 782–805. [Google Scholar] [CrossRef]
- Joseph, V.R.; Hung, Y.; Sudjianto, A. Blind kriging: A new method for developing metamodels. J. Mech. Des. 2008, 130, 031102. [Google Scholar] [CrossRef]
- Kersaudy, P.; Sudret, B.; Varsier, N.; Picon, O.; Wiart, J. A new surrogate modeling technique combining Kriging and polynomial chaos expansions–Application to uncertainty analysis in computational dosimetry. J. Comput. Phys. 2015, 286, 103–117. [Google Scholar] [CrossRef]
- Zhang, Y.; Yao, W.; Ye, S.; Chen, X. A regularization method for constructing trend function in Kriging model. Struct. Multidiscip. Optim. 2019, 59, 1221–1239. [Google Scholar] [CrossRef]
- Zhang, Y.; Yao, W.; Chen, X.; Ye, S. A penalized blind likelihood Kriging method for surrogate modeling. Struct. Multidiscip. Optim. 2019, 61, 457–474. [Google Scholar] [CrossRef]
- Couckuyt, I.; Forrester, A.; Gorissen, D.; De Turck, F.; Dhaene, T. Blind Kriging: Implementation and performance analysis. Adv. Eng. Softw. 2012, 49, 1–13. [Google Scholar] [CrossRef]
- Forrester, A.I.; Keane, A.J. Recent advances in surrogate-based optimization. Prog. Aerosp. Sci. 2009, 45, 50–79. [Google Scholar] [CrossRef]
- Palar, P.S.; Shimoyama, K. On efficient global optimization via universal Kriging surrogate models. Struct. Multidiscip. Optim. 2018, 57, 2377–2397. [Google Scholar] [CrossRef]
- Joseph, V.R.; Delaney, J.D. Functionally induced priors for the analysis of experiments. Technometrics 2007, 49, 1–11. [Google Scholar] [CrossRef]
- Haftka, R.T.; Villanueva, D.; Chaudhuri, A. Parallel surrogate-assisted global optimization with expensive functions—A survey. Struct. Multidiscip. Optim. 2016, 54, 3–13. [Google Scholar] [CrossRef]
- Forrester, A.; Jones, D. Global optimization of deceptive functions with sparse sampling. In Proceedings of the 12th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Victoria, BC, Canada, 10–12 September 2008; p. 5996. [Google Scholar]
- Keane, A.; Bright, A. Passive vibration control via unusual geometries: Experiments on model aerospace structures. J. Sound Vib. 1996, 190, 713–719. [Google Scholar] [CrossRef]
- Couckuyt, I.; De Turck, F.; Dhaene, T.; Gorissen, D. Automatic surrogate model type selection during the optimization of expensive black-box problems. In Proceedings of the 2011 Winter Simulation Conference (WSC), Phoenix, AZ, USA, 11–14 December 2011; IEEE: New York, NY, USA, 2011; pp. 4269–4279. [Google Scholar]
- Suprayitno; Yu, J. C. Evolutionary reliable regional Kriging surrogate for expensive optimization. Eng. Optim. 2019, 51, 247–264. [Google Scholar] [CrossRef]
- Shao, T.; Krishnamurty, S. A clustering-based surrogate model updating approach to simulation-based engineering design. J. Mech. Des. 2008, 130, 041101. [Google Scholar] [CrossRef]
- Andrei, N.; Andrei, N. Nonlinear Optimization Applications Using the GAMS Technology; Springer: Berlin/Heidelberg, Germany, 2013. [Google Scholar]
Problems | Best Known Value | Dimensionality | No. Constraints |
---|---|---|---|
G24 | −5.508 | 2 | 2 |
G8 | −0.0958 | 2 | 2 |
Two_bars | 0.0309 | 3 | 2 |
Four_bars | 1400 | 4 | 1 |
Ibeam | 0.0131 | 4 | 2 |
G5MOD | 5126.5 | 4 | 5 |
G4 | −30,665.539 | 5 | 6 |
Hesse | −310 | 6 | 6 |
SR | 2994.42 | 7 | 11 |
Test Problems | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
G24 | G8 | Four Bars | Two Bars | Ibeam | G5MOD | G4 | Hesse | SR | ||
IBK | Best | −5.4669 | −0.095 | 1400 | 0.031 | 0.013 | 5126.5 | −30,665.5 | −310 | 3042.2 |
Median | −5.4075 | −0.091 | 1400 | 0.031 | 0.014 | 5135.2 | −30,665.5 | −310 | 3051.3 | |
Mean | −5.3832 | −0.089 | 1400 | 0.031 | 0.014 | 5167.1 | −30,665.5 | −310 | 3053.3 | |
NEF | 33.2 | 46.8 | 17.4 | 36.2 | 60 | 74.0 | 24.2 | 38.6 | 68.4 | |
FLT-AKM | Best | −5.46 | −0.095 | 1400 | 0.031 | 0.013 | 5739.9 | −30,611.7 | −306.553 | 3024.4 |
Median | −5.4173 | −0.084 | 1400 | 0.031 | 0.015 | 6157.9 | −30,416.8 | −297.34 | 3061 | |
Mean | −5.415 | −0.085 | 1400 | 0.031 | 0.018 | 6187.8 | −30,411.2 | −297.093 | 3058.5 | |
NEF | 34 | 54.8 | 17.5 | 37.8 | 74.8 | 59.2 | 35.2 | 58.6 | 59.6 | |
RCGO | Best | −0.096 | 1400 | 0.031 | 0.013 | 5323.5 | −30,628.8 | −306.428 | ||
Median | −0.096 | 1400 | 0.031 | 0.014 | 6099.8 | −30,389.5 | −294.812 | |||
Mean | N/A(10) | −0.073 | 1400 | 0.033 | 0.016 | 5992.9 | −30,395.3 | −296.239 | N/A(10) | |
NEF | 51.7 | 17.2 | 40.4 | 67.8 | 65.4(3) | 35 | 73.2 | |||
COBRA-Local | Best | - | −0.1 | - | - | - | 5126.5 | −30,665.5 | −309.94 | 2994.4 |
Median | −0.1 | 5126.51 | −30,665.2 | −297.87 | 2994.7 | |||||
[28] | Mean | −0.09 | 5126.51 | −30,665.1 | −296.25 | 2994.7 | ||||
NEF | 50 | 50 | 50 | 50 | 50 | |||||
COBRA-Global | Best | - | −0.1 | - | - | - | 5126.5 | −30,665.4 | −309.97 | 2994.7 |
Median | −0.1 | 5126.51 | −30,664.9 | −297.87 | 2994.7 | |||||
[28] | Mean | −0.09 | 5126.62 | −30,664.9 | −296.25 | 2994.7 | ||||
NEF | 50 | 50 | 50 | 50 | 50 |
Design Variables | IBK | FLT-AKM | RCGO | CMLS [48] | EORKS [49] |
---|---|---|---|---|---|
3 | 3 | 2.98 | 2.37 | 3 | |
4.31 | 4.251 | 4.19 | 4.174 | 4.209 | |
1.907 | 1.91 | 1.91 | 1.877 | 1.902 | |
2.267 | 2.275 | 2.278 | 1.721 | 2.285 | |
4.218 | 4.218 | 4.237 | 4.024 | 4.222 | |
1.706 | 1.708 | 1.705 | 1.438 | 1.709 | |
2.723 | 2.702 | 2.624 | 2.186 | 2.644 | |
Best | 25.85 | 25.46 | 24.72 | 22.79 | 25.66 |
Median | 25.27 | 25.21 | 23.46 | - | - |
Mean | 25.36 | 25.33 | 23.75 | - | - |
NFE | 55.6 | 65.7 | 72.3 | 95 | 87 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Mai, H.T.; Lee, J.; Kang, J.; Nguyen-Xuan, H.; Lee, J. An Improved Blind Kriging Surrogate Model for Design Optimization Problems. Mathematics 2022, 10, 2906. https://doi.org/10.3390/math10162906
Mai HT, Lee J, Kang J, Nguyen-Xuan H, Lee J. An Improved Blind Kriging Surrogate Model for Design Optimization Problems. Mathematics. 2022; 10(16):2906. https://doi.org/10.3390/math10162906
Chicago/Turabian StyleMai, Hau T., Jaewook Lee, Joowon Kang, H. Nguyen-Xuan, and Jaehong Lee. 2022. "An Improved Blind Kriging Surrogate Model for Design Optimization Problems" Mathematics 10, no. 16: 2906. https://doi.org/10.3390/math10162906