A High-Precision Surrogate Modeling Method Based on Parallel Multipoint Expected Improvement Point Infill Criteria for Complex Simulation Problems

Abstract

In order to overcome the problem of the low fitting accuracy of the expected improvement point infill criteria (EI) and the improved expected improvement point infill criteria (IEI), a high-precision surrogate modeling method based on the parallel multipoint expected improvement point infill criteria (PMEI) is presented in this paper for solving large-scale complex simulation problems. The PMEI criterion takes full advantage of the strong global search ability of the EI criterion and the local search ability of the IEI criterion to improve the overall accuracy of the fitting function. In the paper, the detailed steps of the PMEI method are introduced firstly, which can add multiple sample points in a single iteration. At the same time, in the process of constructing the surrogate model, it is effective to avoid the problem of the low fitting accuracy caused by adding only one new sample point in each iteration of the EI and IEI criteria. The numerical examples of the classical one-dimensional function and two-dimensional function clearly demonstrate the accuracy of the fitting function of the proposed method. Moreover, the accuracy of the multi-objective optimization surrogate model of a truck cab constructed by the PMEI method is tested, which proves the feasibility and effectiveness of the proposed method in solving high-dimensional modeling problems. All these results confirm that the Kriging model developed by the PMEI method has high accuracy for low-dimensional problems or high-dimensional complex problems.

1. Introduction

The surrogate model optimization method is considered as the most important and efficient analysis method, which has become one of the hot research topics in the engineering fields [1,2,3,4]. Bacigalupo applied the surrogate model optimization method to the field of architected materials and proposed an efficient algorithm strategy based on machine learning technology, which effectively solved the computational efficiency problem related to seeking the optimal solution of strong nonlinear and unilateral constraint problems in the large-scale multidimensional space of design parameters [5]. Tian proposed an enhanced variable-fidelity surrogate model optimization method to improve the global optimizing ability of surrogate-based optimizations [6]. At present, the widely used surrogate model construction techniques mainly include the Response Surface Methodology (RSM) [7], Radial Basis Functions (RBF) [8], Kriging interpolation method and Support Vector Machine (SVM) [9]. Among these methods, the Kriging model not only has high accuracy in fitting linear and high-order nonlinear functions but can also obtain the response value and predict the variance, which has attracted more and more attention.

As an important surrogate model construction method, the Kriging method has been widely applied to the automotive engineering, computer engineering, electromagnetics and aerospace fields [10,11,12,13,14,15]. Designing reasonable point infill criteria is the core technology of the Kriging method, which is very important to improve the efficiency and accuracy of the surrogate model. Therefore, in recent years, the Kriging model has been studied by many scholars and a series of improved methods have been put forward. Fuhg [16] reviewed and summarized the different types of Kriging adaptive schemes proposed in the literature and introduced the main principles of different adaptive technologies in detail. Kaps [17] proposed an optimization scheme based on a hierarchical Kriging for large-scale and highly nonlinear crashworthiness problems. In order to minimize the global Kriging variance, Xiao put forward a novel sampling algorithm to adaptively sample landmarks that was associated with a high information capacity [18]. To effectively analyze the interval-valued data, Bean developed an interval-valued Kriging based on the modification of the random sets theory [19]. Maia [20] combined an adaptive Kriging framework with a hybrid particle swarm optimization (PSO) algorithm to improve the computational efficiency of the optimization process. Koziel and Pietrenko-Dabrowska conducted a series of in-depth studies on the agent model of antenna electromagnetic simulation and proposed some effective agent model modeling methods, including nested Kriging, gradient enhanced Kriging, etc., in order to effectively reduce the modeling time cost [21,22,23,24]. Moreover, the EI is the most widely used point infill criterion in the Kriging model, so as to improve the accuracy and efficiency of the surrogate model. The research on the improvement of the EI point infill criterion has become a hotspot. Jones proposed an improved expectation improvement (EI) method, which can search for points with large model prediction errors in the entire region, balance global accuracy and local accuracy and achieve the purpose of optimizing the overall accuracy of the surrogate model [25]. In order to improve modeling efficiency, Kleijnen proposed an improved estimator of the Kriging predictor variance [26]. For the efficient estimation of small failure probabilities, Cadini put forward an algorithm, which combined FORM to an adaptive Kriging-based importance sampling strategy [27]. Marzat considered an algorithm strategy in which expected improvement was computed in the minimax optimization context, which allowed to further reduce the number of costly evaluations of the performance index [28].

With the increasing complexity of computational models in engineering, parallel computing and optimization have become an important way to improve the computational efficiency and accuracy of large-scale models [29,30]. As can be seen from the above, most of the existing research mainly focuses on the single-point infill criteria and less on the parallel-point infill criteria. Single-point infill criteria can only generate one sample point in each iteration, which is low in optimization efficiency and the accuracy of the surrogate model. Therefore, based on the introduction of the Kriging model EI and IEI point infill criterion, this paper proposes a parallel multipoint infill criterion, named the PMEI method. Firstly, the detailed steps of constructing the Kriging model based on the PMEI point infill criteria are introduced, which realize the effect of adding multiple points in an iterative process. Then, the classical one-dimensional function and two-dimensional function are used to test the accuracy of the PMEI method. Finally, taking the truck cab as the research object, the PMEI method is used to create the Kriging surrogate model for multi-objective optimization of the truck cab and test the accuracy of the model. The results show that the PMEI point infill criterion can improve the accuracy of the complex surrogate model, which demonstrates the capability and potential of this method in solving the high-dimensional model of engineering practice.

2. Kriging Model of Point Infill Criterion

When constructing the surrogate model, the design variables and their response values of the samples are usually obtained by the experimental design. Then, the prediction point data are obtained by fitting or interpolating the obtained sample point set. In the process of constructing a surrogate model, the randomness of sampling leads to some differences between the surrogate model and the real target value in some regions. Therefore, the point infill criterion is very important to improve the accuracy of the surrogate model.

2.1. Kriging Model

The Kriging method is mainly based on the known position and correlation of known samples to obtain the linear unbiased variable with the smallest estimation error in the unknown region [31]. Kriging equation [32] is composed of global approximation model and local model deviation:

$$G\left(x\right)=V\left(x\right)\cdot \beta +Z\left(x\right)$$

(1)

where V(x) is a polynomial function of variable x, β is the regression coefficient, V(x)·β is used to express the overall trend of the function of G(x), Z(x) is the realization of a normally distributed Gaussian random process with mean zero, and variance σ², which is used to represent the global local deviation. For n sample points, the covariance is given by the following expression:

$$Cov\left[Z\left(x_i\right),Z\left(x_j\right)\right]={\sigma }^{2}R;R=R_{ij}^N\left(x_i,x_j\right)$$

(2)

where R^N represents N × N correlation matrix and $R_{ij}^N\left(x_i,x_j\right)$ is the correlation equation between two points x_i and x_j, and its expression is shown in formula:

$$R_{ij}^N\left(x_i,x_j\right)=\exp \left(-{{\displaystyle {\sum }_{s=1}^k{\theta }_s\left(x_i^s-x_j^s\right)}}^2\right)={\displaystyle \prod _{s=1}^k\exp \left(-{\theta }_s{\left(x_i^s-x_j^s\right)}^2\right)}$$

(3)

where $x_i^s$ and $x_j^s$ represent the s-th component at points x_i and x_j, and θ_s is the undetermined parameter, which can be calculated by maximum likelihood estimation:

$$\max \underset{{\theta }_s>0}{L\left({\theta }_s\right)}=-\frac{1}{2}\left(N\ln \left({\widehat{\sigma }}^2\right)+\ln \left|R\right|\right)$$

(4)

It can be seen from the above formula that the coefficient θ_s in the correlation function must be calculated to obtain the surrogate model.

For the known sample point x and response G(x), the regression coefficient $\beta$ and random process ${\sigma }^2$ are estimated as:

$$\widehat{\beta }={\left(l^T{R}^{-1}l\right)}^{-1}{l}^T{R}^{-1}Y$$

(5)

$${\sigma }^2={\left(Y-\widehat{\beta }l\right)}^T{R}^{-1}\left(Y-\widehat{\beta }l\right)/N$$

(6)

where l is the N-dimensional unit column vector and Y is the N-dimensional column vector.

Using Kriging model, the response value $\widehat{G}\left(x_0\right)$ [33] and estimated variance ${\widehat{\sigma }}_G^2\left(x_0\right)$ can be expressed as follows:

$$\widehat{G}\left(x_0\right)=\widehat{\beta }+r{\left(x_0\right)}^T{R}^{-1}\left(Y-\widehat{\beta }l\right)$$

(7)

$${\widehat{\sigma }}_G^2\left(x_0\right)={\sigma }^2\left[1-r{\left(x_0\right)}^T{R}^{-1}r\left(x_0\right)+u{\left(x_0\right)}^T{\left(l^T{R}^{-1}l\right)}^{-1}u\left(x_0\right)\right]$$

(8)

where u(x₀) = l^TR⁻¹r(x₀) − l, r represents the correlation vector between the unknown point x₀ and the sample point, and its expression is given by:

$$r\left(x_0\right)={\left[R\left(x_0,x_1\right),R\left(x_0,x_2\right),\dots ,R\left(x_0,x_N\right)\right]}^T$$

(9)

2.2. Expected Improvement Point Infill Criteria (EI)

Jones [25] proposed the improved expected improvement (EI) method, which is one of the most widely used point infill criteria at present. When designing the experiment, there may be a point x_t where the estimated value $\widehat{G}\left(x_t\right)$ is smaller than the known best (usually minimum) value G_min. Adding this point to the existing sample set can improve the predicted value of Kriging surrogate model in the optimal value region. The improvement at point x_t is defined as $I=G_{\min }-\widehat{G}\left(x_t\right)>0$. For Kriging model, it is assumed that variable Y obeys normal distribution $N [\widehat{G}\left(x_t\right),{\sigma }^2\left(x_t\right)]$, where $\widehat{G}\left(x_t\right)$ is the predicted response value of Kriging surrogate model and ${\sigma }^2\left(x_t\right)$ is the predicted variance. Then, the expected expression [25] is expressed as:

$$E\left[I\left(x\right)\right]=\{\begin{array}{l}\left(G_{\min }-\widehat{G}\left(x_t\right)\right)·\mathsf{\Phi }\left(\frac{G_{\min }-\widehat{G}\left(x_t\right)}{\sigma \left(x_t\right)}\right)+\sigma \left(x_t\right)·\phi \left(\frac{G_{\min }-\widehat{G}\left(x_t\right)}{\sigma \left(x_t\right)}\right) ,\sigma \left(x_t\right)>0\\ 0 ,\sigma \left(x_t\right)<0\end{array}$$

(10)

where Φ(*) denotes the distribution function and φ(*) denotes the density function of the standard normal distribution. Meanwhile, if $G_{\min }-\widehat{G}\left(x_t\right)<0$, it means that the estimated value can not be improved at this point, and I = 0.

As can be seen from Equation (10), the essence of EI criteria of Kriging model is to search for the point with the smallest predicted value or the point with the largest variance in the design space. Consequently, this method ignores the search for the maximum region of the prediction target.

2.3. Improved Expected Improvement Point Infill Criteria (IEI)

In view of the shortcomings of EI criteria, Henkenjohann and Kunert [34] proposed IEI criteria to enhance the search ability of the global maximum area, and its expression is as follows:

$$I\left(x\right)=\{\begin{array}{l}{G}_{\max }-\widehat{G}\left(x_t\right) G_{\max }>\widehat{G}\left(x_t\right)\\ 0 G_{\max }<\widehat{G}\left(x_t\right)\end{array}$$

(11)

where G_max represents the current best. Therefore, the new improved expectation expression is defined by:

$$E\left[I\left(x\right)\right]=\{\begin{array}{l}\left(\widehat{G}\left(x_t\right)-G_{\max }\right)\cdot \left(1-\mathsf{\Phi }\left(\frac{\widehat{G}\left(x_t\right)-G_{\max }}{\sigma \left(x_t\right)}\right)\right)+\sigma \left(x_t\right)\cdot \phi \left(\frac{\widehat{G}\left(x_t\right)-G_{\max }}{\sigma \left(x_t\right)}\right),\sigma \left(x_t\right)>0\\ 0 ,\sigma \left(x_t\right)<0\end{array}$$

(12)

Although IEI criteria can improve the global accuracy of the surrogate model, it often needs to add more sample points to improve the local accuracy. This will lead to low iterative efficiency of surrogate model. In other words, with the same number of samples, the accuracy of the surrogate model constructed by IEI criterion is relatively low, especially in the complex high-dimensional computing model.

2.4. Parallel Multipoint Expected Improvement Point Infill Criteria (PMEI)

From the above analysis, it can see that the single EI and IEI criteria both have their own limitations. In order to complement the advantages of the two criteria, a PMEI criterion based on EI and IEI criteria is proposed. Using the EI and IEI point infill criteria can achieve the effect of adding two points simultaneously. In the process of surrogate model construction, adding multiple sample points can not only accelerate the fitting efficiency of surrogate model but also improve the fitting accuracy.

The flowchart of the PMEI criteria can be shown in Figure 1. The detailed optimization modeling processes of the PMEI criteria are described as follows. Step1, initial experiment design. For the objective function, N initial sample points of design variables are generated by optimal Latin hypercube design. Step2, sample response. According to the numerical simulation analysis, the response of the sample is calculated. Step3, construct Kriging model. The Kriging surrogate model is constructed by using the design variables and responses of the initial sample set. Step4, accuracy test of Kriging surrogate model. The accuracy of the new model is calculated according to the accuracy evaluation criteria to judge whether it meets the optimization requirements. Step5, if the accuracy evaluation criteria are not met, calculate EI criteria and IEI criteria to obtain multiple new sample points. In particular, duplicate sample points are generated between these new sample points or between the new sample point and the initial sample point. Step6, repeat step4 and step5 until the accuracy of the surrogate model meets the accuracy evaluation standard. Then, the final optimized Kriging surrogate model is obtained.

3. Numerical Tests for The PMEI Method

In order to verify the effectiveness of the PMEI point infill criteria, the Kriging models of the classical one-dimensional function and two-dimensional function are constructed by the EI, IEI and PMEI methods, respectively. The fitting accuracy and the efficiency of different methods are compared and analyzed.

3.1. One-Dimensional Function Example

The expression of one-dimensional function is: f(x) = x²*cos(3x) + (0.1x + 2)*sin(x) + (x − 1)², x ∈ [0, 9].

Six initial sample points are extracted by optimal Latin hypercube design (Opt LHD), and the one-dimensional function is optimized by EI, IEI and PMEI methods, with 10, 10 and 5 iterations, respectively. In addition, in order to be more intuitive, the accuracy of the Kriging models is evaluated by several evaluation criteria in this case. The Kriging model curves and the absolute error curves are shown in Figure 2 and Figure 3, respectively, and the R² and RMAE values of different methods are shown in Figure 4.

In addition, the determination coefficient R² [34] and the relative maximum absolute error RMAE [35] are used to evaluate the global accuracy and local accuracy of the surrogate models, respectively. The calculation expressions of R² and RMAE are as follow:

$$R_{}^2=1-\frac{{\displaystyle \sum _{i=1}^N{\left(G\left(x_i\right)-\widehat{G}\left(x_i\right)\right)}^2}}{{\displaystyle \sum _{i=1}^N{\left(G\left(x_i\right)-\mu \left(x\right)\right)}^2}}$$

(13)

$$RMAE=\max \left(\frac{\left|G\left(x_i\right)-\widehat{G}\left(x_i\right)\right|}{s\left(x\right)}\right)$$

(14)

where $G\left(x_i\right)$ is the true value of sample points, $\widehat{G}\left(x_i\right)$ is the predicted value of Kriging model, s(x) is the sample standard deviation and μ(x) is the sample mean. Obviously, if RMAE is close to 0, it indicates that the local accuracy of the model is high. Similarly, if R² is close to 1, it means that the surrogate model has more accurate global accuracy. Overall, the closer the value of R² to 1 and the smaller the values of RMAE, the better the surrogate accuracy.

As can be seen from Figure 2a, the distribution of the new sample points of the EI method is relatively uniform, which has good global fitting accuracy. However, comparing the EI fitting function with the original function curve, the search ability of the EI method in the maximum region is relatively weak, and the local fitting accuracy is low. As can be seen from Figure 2b, the new sample points of the IEI method are concentrated in the maximum region and have high local fitting accuracy in this region. Nevertheless, because there are few new sample points in other places, the global fitting accuracy of the IEI method is relatively low. Obviously, the Kriging model based on the PMEI method is in good agreement with the original function, as can be seen from Figure 2c. The new sample points of the PMEI method are not only relatively uniform but also have very high fitting accuracy in both the minimum and maximum regions. Figure 3 shows the absolute error curves of the three methods. It is obvious that the absolute errors of the PMEI Kriging model are smaller than those of the EI Kriging model and the IEI Kriging model. It can be clearly seen from Figure 4 that although the R² values of the Kriging model of the three methods are almost equal to 1, the RMAE value of the PMEI method is larger than that of the other two methods. On the whole, the PMEI complements the advantages of the EI and IEI, which not only improves the global accuracy but also improves the local accuracy of the Kriging model.

3.2. Two-Dimensional Function Example

The expression of two-dimensional function is: f(x,y) = y*sin(x) − x*sin(y), x, y ∈ [0, 5].

Five initial sample points are extracted by optimal Latin hypercube sampling method, and the two-dimensional function is optimized by EI, IEI and PMEI methods, with 6, 6 and 3 iterations, respectively. Moreover, the accuracy of the Kriging models is evaluated by four evaluation criteria in this case. The results on the predicted MSEs and absolute errors of the three criterions for two-dimensional problems are shown in Figure 5 and Figure 6, respectively, and the R² and RMAE values of different methods are shown in Figure 7.

It can be seen from the mean square error analysis results of the surrogate model generated by the three point adding criteria in Figure 5, the mean square error of the PMEI is less than the EI and IEI, which means that the surrogate model constructed by the PMEI criterion has better accuracy. Simultaneously, it can be seen from Figure 6 that the absolute error of the PMEI Kriging model is smaller than that of the EI Kriging and IEI Kriging models. Figure 7 shows the R² and RMAE values of the surrogate models established by the three methods, which shows that the PMEI method has very good accuracy, despite that the RMAE value of the PMEI is worse than the EI and IEI methods when the same number of total iterations are taken into account. In addition, the shape and characteristics of the test function and surrogate model based on the PMEI criterion are shown in Figure 8 and the distribution of the sampling points is shown in Figure 9. It can be clearly seen that the surrogate model created by the PMEI has a smooth shape and good fitting compared with the original function. The PMEI method can construct an appropriate Kriging model to replace the original function. In a word, it shows that the proposed method can provide good modeling accuracy. Therefore, the PMEI method is verified to be effective for two-dimensional problems.

4. Kriging Surrogate Model of a Truck Cab

The calculation accuracy of the PMEI has been numerically tested by the one-dimensional function and two-dimensional function, respectively, in Section 3. However, for the complex engineering calculation, the accuracy of the PMEI needs to be further verified.

The structural optimization design of truck cab crash safety is a multi-objective optimization problem, which requires a repeated finite element analysis [36,37,38,39]. In the automotive industry, A-pillar impact is usually used to design and verify the safety of vehicle structures [40,41]. Therefore, as an example, the surrogate model of a truck cab is constructed, and the accuracy of the model is tested in this paper. First, the finite element model of the truck cab is established in HyperMesh software according to the ECE R-29 standard, which is shown in Figure 10. Afterward, the A-pillar collision simulation is carried out by the LS-DYNA solver, and the deformation of the cab is shown in Figure 11.

4.1. Construction of Surrogate Model

According to the ECE R-29 standard, the crash safety performance of the cab is tested by measuring the living space of the fiftieth-percentile male manikin model after the crash test [40]. Specifically, the living space is represented by characteristic dimensions, which means the minimum distances between the manikin and any inelastic part of the cab. Therefore, the minimum distance L_min between the steering column and the manikin knee after the A-pillar impact test simulation is extracted as the characteristic dimensions of the living space of the cab, which is shown in Figure 12. Hence, it is necessary to construct Kriging surrogate models of the cab A-pillar collision safety index L_min.

Considering that the thickness of the parts has a direct impact on the crash safety of the cab, the thickness of the top 8 parts (denoted by t₁~t₈) in the energy absorption sequence during the collision simulation process are extracted as the key design variable, as shown in Figure 13. In this study, it is assumed that the value range of the design variables is 70~120% of the initial thickness; the related data are shown in

Table 1. Key design variables and value range.

Design Variable	Part Name	Initial Value	Value Range
t1	A-pillar outer plate	1.20	(0.84, 1.44)
t2	A-pillar inner plate	1.20	(0.84, 1.44)
t3	Top cover side plate	1	(0.7, 1.2)
t4	Top cover beam	1.5	(1.05, 1.8)
t5	Side wall longitudinal beam	1.00	(0.7, 1.2)
t6	Front wall inner panel	1.20	(0.84, 1.44)
t7	Front wall cover plate	1.20	(0.84, 1.44)
t8	Roof front panel	1	(0.7, 1.2)

.

According to the eight selected design variables, the surrogate model is to construct an approximate function from a series of sampling points, which are typically determined using design of experiment methods. In this study, 40 groups of sample points were generated using the optimal Latin hypercube sampling method. Then, the 40 thickness design variables are substituted into the A-pillar collision finite element model of the truck cab for the simulation calculation, and the response value L_min is obtained. For a comparative study, the surrogate models of the cab are constructed based on the EI, IEI and PMEI methods, respectively, according to the 40 sample points and response values. Then, the surrogate models are optimized with 120, 120 and 60 iterations, respectively.

4.2. Accuracy Assessment of Surrogate Model

To evaluate the accuracy of these three different surrogate schemes, 10 new validation points are also generated by using the optimal Latin cube sampling method here. According to Equations (13) and (14), respectively, the relative maximum absolute error RMAE and the determination coefficient R² were used to measure the accuracy of these three different surrogate models; the analysis results are shown in Figure 14.

The assessment results are compared in Figure 14. Apparently, the RMAE values of the surrogate model based on the PMEI criteria are less than that of the EI criteria and IEI criteria. Meanwhile, the R² values of the surrogate model based on the PMEI criteria are greater than that of the EI criteria and IEI criteria. By comparative analysis, the surrogate models constructed by the PMEI criterion have large R² values and small RMAE values, which means that they have not only good global accuracy but also better local accuracy. In conclusion, the truck cab surrogate model based on the PMEI method has high accuracy and can be applied to replace the original complex model in engineering practice. Consequently, the PMEI method of constructing a high-precision surrogate model is effective and feasible for high-dimensional modeling.

5. Conclusions

This paper proposes a PMEI method of a Kriging surrogate model based on the EI criterion and IEI criterion. Moreover, numerical examples of one-dimensional function and two-dimensional function are used to verify the accuracy of the PMEI surrogate model. Compared with the EI and IEI single-point infill criteria, the PMEI method can improve the global accuracy of the surrogate model. In order to verify the effectiveness of the PMEI criterion in constructing a high-dimensional model, the surrogate model of a truck cab is constructed by the EI, IEI and PMEI criterion, respectively. Then, the accuracy evaluation indexes R² and RMAE of different surrogate models are compared and analyzed. The research results show that the cab surrogate model created by the PMEI criterion has higher accuracy and can meet the needs for engineering practical application. Therefore, the method of constructing a high-precision surrogate model by PMEI criterion has advantages in complex engineering problems.