1. Introduction
With the expansion of computational capabilities and the surge in data accumulation, optimization challenges in various scientific and industrial domains have become increasingly complex. The evolutionary algorithms (EAs) have emerged as an outperform optimization technique within this region, capable of handling the aforementioned complexities [
1]. However, the mechanism of EAs is rooted in population-based iteration, which requires an extensive iterative phase to determine the fitness values of a large population. In many engineering fields, to evaluate these fitness functions for complex optimization tasks is a significant computational overhead, and this is classified as an expensive optimization problem (EOP) [
2,
3,
4], such as in the field of aerodynamic design optimization. To achieve consistency between a design’s aerodynamic metrics in computational fluid dynamics simulations, the real-world aerodynamic performance can be time-consuming, which may take anywhere from minutes to potentially hours for a single evaluation.
Due to the challenges mentioned, the academic community has shifted towards the surrogate-assisted evolutionary algorithm (SAEA) [
5], and the SAEA has gained significant scholarly interest [
6,
7,
8]. The SAEA has proven to be effective for EOPs [
9,
10], as it can reduce computational overheads. Based on the available dataset, either singular [
11] or composite models [
12] are used to assess the feasibility of potential solutions. Contemporary methodologies, such as polynomial regression [
13], support vector machines [
14], radial basis function networks [
14,
15], artificial neural networks [
16], and Gaussian processes (also known as Kriging model) [
17,
18], are commonly used in surrogate model.
The SAEA commonly uses global surrogate models to improve search capability [
19,
20,
21]. Liu et al. proposed harnessing Gaussian functions for surrogate model articulation to enhance the SAEA’s prowess in curating a robust global surrogate construct [
18]. By utilizing sample mapping techniques, the dimensionality of decision variables obtains a contraction [
22]. To redress the algorithm’s tapering exploratory vigor in its mature phases, Wang et al. provided a global model governance blueprint [
23], in which optimal fitness values concerned with ambiguous solutions are discerned through the amalgamated surrogate model.
However, in recent years, there has been a focus on local surrogate models. These models explore the proximity of the global optimal solution’s locus. Based on the latent potential of historic datasets, Sun et al. proposed an avant-garde strategy to use semi-supervised learning, optimally harnessing both annotated and non-annotated datasets to amplify algorithmic precision [
24]. Yu et al. provided a social learning particle swarm optimization method for granular, localized spatial exploration [
25]. Additionally, cooperation between global and local surrogate models comes into prominence as a new optimization method. Yu et al. proffered archival samples to cultivate both global and local surrogate models. The strategy meticulously calibrates a balance between the dual models to minimize the computational impositions of the genuine objective function.
Despite the widespread use of global and local surrogate models, there is a lack of effective control mechanisms during global and local surrogate model transitions. Additionally, due to constraints in single evaluation metrics, it is difficult to precisely describe the state of surrogate models, which can impact their effectiveness during transformation. Both of these reduce the effectiveness of the local surrogate model and its ability to explore locally.
Based on the above considerations, this paper firstly presents a new methodology to address situations with high levels of uncertainty. To navigate these complexities, we record points with significant uncertainty and use them to formulate a normal distribution function, which enables the self-adaptive modulation of parameters. This adaptation improves the algorithm’s ability to explore new areas and avoid becoming stuck in local optima. An archive of the region with uncertain solutions is created, guided by a normal distribution function. Archival data serve as the basis for recalibrating adaptive parameters to determine whether to transition to the local surrogate model. The composite model is used for evaluation within this surrogate space to guide the particle swarm optimization (PSO) algorithm in its optimization efforts. Its effectiveness is continuously evaluated against the adaptive parameters within the local surrogate model to determine if exiting this mode is necessary. This cyclical optimization process can balance exploration and exploitation. The main contributions of this paper can be concluded as follows.
(1) An effective evaluation criterion is proposed to estimate the confidence of a surrogate model in its predictions. The criterion combines the effectiveness of the surrogate model and the uncertainty level of the current state and can direct well the optimization behavior according to the exact state of the surrogate model.
(2) An adaptive Gaussian distribution parameter is used to select the surrogate models, which serves as a gauge of the advisability about the local surrogate model. The local exploration is enhanced, which promotes the ability to search the entire feasible solution space more effectively.
(3) The combination of the confidence criterion and the parameter controlled method cooperates as a comprehensive strategy, which can improve the overall performance and reliability of our algorithm in solving complex expensive optimization problems.
The subsequence of the paper unfolds as follows.
Section 2 introduces the surrogate model techniques used in our algorithm.
Section 3 provides a detailed description of the proposed ASAPSO algorithm. In
Section 4, we present the experimental outcomes of ASAPSO on both benchmark functions and an intricate engineering challenge, comparing them with other state-of-the-art algorithms. Finally,
Section 5 provides conclusions and discusses future works.
3. Adaptive Surrogate-Assisted Particle Swarm Optimization (ASAPSO)
In real-world applications, the challenges often arise from small datasets. This requires swift problem-solving strategies due to the constraints of limited data. To achieve this, a harmonious blend of precision and expeditious optimization processes is necessary. The recent methodology of cherry-picking the optimal fitness value for integration into the local surrogate model raises concerns.
The benefits of the integration between surrogate models and PSO in optimization problems are widely pointed out. Firstly, in terms of simplicity and ease of implementation, PSO is known for its straightforward implementation compared to other evolutionary algorithms like genetic algorithms (GAs). It requires fewer parameters to tune, which makes it easier to adapt and apply to different problems. Secondly, in terms of fast convergence speed, PSO tends to converge faster to optimal or near-optimal solutions, especially in high-dimensional search spaces. This can be particularly advantageous in applications where computational efficiency is crucial. Finally, in terms of flexibility and adaptability, PSO is highly flexible and can be easily adapted to different optimization problems by adjusting the velocity and position update rules. This flexibility allows for customization to specific problem domains without requiring significant modifications to the core algorithm.
Specifically, there is a risk of neglecting areas of uncertainty where multiple solutions exist but the optimal solution remains elusive. This vulnerability may inadvertently direct the algorithm towards a local optimum. The proposed ASAPSO algorithm in this paper introduces two key innovations. Firstly, an amalgamated surrogate model is implemented within the overarching surrogate model, endorsing the dual evaluation paradigm that emphasizes uncertainty. During the transition interplay between the global and local spheres, adaptive parameters that follow a normal distribution are introduced.
The ensemble surrogate model emerges by melding RBF, PR, and KRG surrogates under the aegis of the OWS management paradigm. The evaluation matrix of uncertain solutions hinges on computing the root mean square error disseminated across the triad of surrogate models, as illustrated in
Figure 1.
As illustrated in
Figure 1, the red region represents an area of uncertainty, hinting at a more extensive unexplored zone nearby. Within this zone is the domain tied to the optimal fitness value. The label of red arrow points to an uncertain solution guiding the algorithm in pursuit of the best result. This method encourages the algorithm to traverse both the landscape of uncertain solutions and the area containing the peak fitness value, prompting it to delve into and refine these areas after each assessment. Such a repetitive approach amplifies the algorithm’s local probing prowess.
The ensemble surrogate model is the aggregate result obtained through a weighted summation of the outputs generated by all individual members. This relationship is formulated as follows:
where
stands for the i output of each member, 1 ≤ i ≤ 3, and
represents the weight assigned to the i output, as defined:
where
signifies the root mean square error of the i model and the weighted aggregation method is employed [
32].
In transitioning between the global and local surrogate models, the specific procedure is delineated as follows. A new k-value emerges from a normal distribution with a mean derived from the ustep and a standard deviation set at 0.1 during each assessment. This is realized through the given equation.
For the first half of the surrogate model, we utilize its incremental function curve. We postulate that amidst a plethora of uncertain solutions, the probability of pinpointing the optimal fitness value point intensifies within this segment of the region. As a result, the ensuing equation is employed to chronicle the uncertainty information over intervals and to influence the amplitude of the k-value grounded on these archived data.
where un symbolizes the ensemble of uncertain solutions archived by the PSO algorithm for every iteration. The tally of uncertain solutions is monitored per iteration via Goodstep. The initial step size is set at 0.001, in accordance with the recommendations set forth in reference [
33], and is subsequently recalibrated following the determination of the number of uncertain solutions within the global surrogate model.
where ustep signifies the parameter archive storing uncertainty values and their mean value for each iteration. c is a value between 0 and 1. In a typical setting, the threshold is initialized at 0.5 to begin with balanced probabilities.
The primary objective of this adaptive adjustment is to identify whether a transition to either a local or a global surrogate model is warranted. Such adaptability broadens the exploration spectrum across the entirety of the population, thereby reducing the potential of the algorithm becoming ensnared in a local optimum. Additionally, this methodology adeptly mitigates the challenges of suboptimal convergence observed when solely depending on a static value for decision-making criteria. In the final analysis, the ustep is designated its expected value through the application of a normal distribution function and maintains a fixed standard deviation of 0.1. This deliberate decision ensures the procurement of values from the primary half of the normal distribution curve. In tandem, this study harnesses three unique surrogate models which are RBF, PR, and KRG. These models underpin the dual evaluation benchmarks of the amalgamated surrogate model and the uncertainty resolution via the OWS methodology.
The selection of individuals not only possesses optimal adaptation metrics, but also exhibits pronounced uncertainty resolutions. Such a discerning selection mechanism steers the PSO algorithm towards pinpointing the apex value. The local surrogate model integrates the foremost 75% of the ordered fitness values extracted from the global surrogate model and the latter 25% from the sorted uncertain solutions. Subsequently, the population is subjected to a de-emphasis procedure, drawing from the global surrogate model, thus shaping the population intended for the local surrogate model. Within this localized model, the amalgamated model becomes pivotal for predicting response values, thereby forecasting the fitness values that the surrogate model anticipates for points situated within this domain. If the fitness value derived from the local surrogate model, less the least initial population value, falls below 0.001, or if said fitness value is inferior to the initial population’s minimum, the algorithm persists within the global surrogate model’s ambit. On the contrary, if the value meets or surpasses this threshold, the course of action redirects to the global surrogate model. A visual representation of the algorithm’s architectural blueprint can be viewed in
Figure 2.
The whole process of ASAPSO can be represented as Algorithm 1.
Algorithm 1 The pseudo-code of ASAPSO |
Output: the global solution with its real fitness |
begin |
; |
Use Latin hypercube sampling to initialize the population POP and evaluate it |
Save POP with its fitness in DB |
while computational budget is not reaching the limitation do |
; |
Construct the PR model according to Section 2.2.1, the GP model according to Section 2.2.2, and RGB model according to Section 2.2.3; Cooperate the models by optimal weighted surrogate (OWS); |
Construct the local surrogate model by the optimal weighting approach; |
Evaluate and select uncertainty solutions; Transform the local and global surrogate models by transformation mechanism; |
Update the global best solution; |
end while |
end |
4. Experiments and Analysis
4.1. Comparative Experiment on Benchmark Functions
- (1)
Experimental environment
In this paper, the source code corresponding to both the comparative and the proposed algorithms was implemented in MATLAB 2018b. For experimental evaluations, we employed a computational platform with the following specifications: an Intel CPU operating at 3.2 GHz, a memory of 16 GB RAM, and running on the Windows 10 operating system.
For the analysis, the test function suite was derived from the CEC2014 series benchmark test functions. Numerous problems within this suite have been predominantly used in various stochastic algorithmic evolutionary approaches [
28].
4.2. Comparative Algorithm
For the experimentation, a selection was made that encompassed one foundational swarm intelligence algorithm and two enhanced versions of the initial algorithm. Unique surrogate model management methodologies are embodied by each of these algorithms, which are particle swarm optimization (PSO), a Gaussian process surrogate model-assisted evolutionary algorithm (GPEME) [
18], and committee-based active learning for surrogate-assisted PSO (CALSAPSO). This curated selection aimed to facilitate comprehensive comparative scrutiny among these algorithms.
(1) Particle Swarm Optimization (PSO). PSO is a computational approach that emulates the collective behavior observed in the social phenomena of bird flocks or fish schools. Introduced by Kennedy and Eberhart in 1995 [
34,
35], scholars have proposed these variations to either accelerate convergence [
36] or amplify population diversity, the latter being a strategy to diminish the likelihood of confinement to local optima.
(2) GPEME. This represents a Kriging-based SAEA that incorporates the lower confidence bound as its pivotal criterion for the selection of solutions worthy of evaluation, specifically tailored for median scale problems.
(3) CALSAPSO. Within the domain of the SAEA, the strategy of active learning is integrated using a triad of models. An integrated surrogate model arises from the evaluation of the root mean square error. Moreover, a localized surrogate model is crafted by scrutinizing the samples marked by the highest uncertainty, harmonized with the identification of potential optimal solutions as inferred from the integrated surrogate model. It is noteworthy that PSO is the driving force behind the optimization, and the algorithm delineated in this paper functions in harmony with the PSO framework. The parameter configuration for the comparative algorithms is illustrated in
Table 1.
4.3. Comparison Experiment
Table 2 shows the results from the ten-dimensional tests derived from the five CEC2014 test sets. Every algorithm was executed individually on each test function with the same machine, completing a total of 20 runs, while addressing the resource-intensive optimization challenge. Evaluations were confined to a peak of 120 for 10D and 240 for the 20-dimensional (20D) tasks, starting with a population size of 50. These numerical trials took place with decision variable dimensions set at D = 10 and D = 20, specifically targeting the resource-intensive optimization issue. The convergence profiles of ASAPSO, PSO, GPEME, and CALSAPSO on four functions are illustrated in
Figure 3,
Figure 4,
Figure 5, and
Figure 6, respectively. From
Figure 3,
Figure 4,
Figure 5 and
Figure 6, it can be further substantiated that ASAPSO exhibits faster convergence on almost all functions compared to other compared algorithms. In all tables, the best ranks are shown in bold.
Table 3 and
Table 4 is the Friedman test of all algorithms on 10D and 20D, respectively.
Additionally, to further validate the effectiveness of the proposed algorithm, comparative experiments were conducted on the CEC2015 expensive optimization problem test set. From
Figure 7 and
Table 5, it can be observed that the proposed algorithm exhibits optimal advantages in both convergence speed and accuracy.
The experimental results reveal that ASAPSO, due to its effective transition between local and global surrogate models, can more successfully explore the locally uncertain regions where potential optimal solutions may exist. Therefore, ASAPSO exhibits optimal search performance. Moreover, the successful operation of the transition mechanism also benefits from the accurate evaluation of the models.
4.4. Adaptive Parameter Behavioral Analysis
To further substantiate the efficacy of adaptive parameters grounded in a normal distribution, a side-by-side analysis is undertaken between the algorithm that employs adaptive parameters and its counterpart without such modulation. This evaluative process spans four test functions, holding the parameters at D = 10 and D = 20, and imposing an evaluation ceiling of 12D. The comparison charts are illustrated in
Figure 8,
Figure 9,
Figure 10 and
Figure 11. It is worth noting that within these figures, ASAPSO-WGP denotes the version of the proposed algorithm that forgoes the use of adaptive parameters. From
Figure 8,
Figure 9,
Figure 10 and
Figure 11, it can be further substantiated that the proposed algorithm exhibits faster convergence on almost all functions compared to other compared algorithms.
Through an analytical catenation encompassing the proffered algorithm, a pair of classical algorithms (namely PSO and GPEME), and an avant-garde algorithm (CALSAPSO), discernible augmentations in performance are manifested by the proposed algorithm, especially when D = 10 is under scrutiny. An in-depth perusal of the convergence trajectories unveils that our proposed algorithm not only attains a more rapid initial convergence but also preserves a modicum of local exploratory prowess during the advanced phases. When extended to D = 20, its efficacy transcends that of both PSO and GPEME, marginally eclipses CALSAPSO, and underscores its dominance over algorithms devoid of an adaptive framework. In a further series of trials, bifurcated by the presence and absence of adaptive parameters, it is evident that ASAPSO, when armored with the adaptive parameter regimen, registers superior outcomes across a multitude of functions at D = 10, albeit with nuances that are less conspicuous. It merits particular emphasis that, at D = 20, ASAPSO elucidates pronounced refinements in trio functions, namely the Elliptic function, Griewank function, and Rastrigin function. While the apex fitness value optimized for the Rosenbrock function does not particularly dazzle when juxtaposed with its parameter-absent counterpart, a deeper dive into the convergence trajectory illuminates that ASAPSO, when synergized with adaptive parameters, indeed fares better in the Rosenbrock function, evincing more pronounced oscillatory patterns. This revelation accentuates the algorithm’s amplified exploratory finesse, thereby confirming the efficacy of the strategy proposed in this paper.
4.5. Real-World Application of Airfoil Design
The recent research on airfoil aerodynamic optimization has focused on various methods and technologies to enhance the performance of airfoil designs in different applications. Researchers have been implementing genetic algorithms to optimize airfoil geometries, focusing on improving aerodynamic efficiency and lift-to-drag ratios. One study detailed the use of a genetic algorithm model to optimize airfoil geometry specifically for low Reynolds number conditions [
37]. Artificial neural networks (ANNs): Feed-forward artificial neural networks have been employed as surrogate models to predict the lift and drag coefficients of NACA airfoils. These models help in reducing computational time while maintaining optimization accuracy [
38]. Aerodynamic shape optimization: Advances in aerodynamic shape optimization include using physics-informed methods and machine learning to improve the performance of airfoils under various flow conditions. For instance, one study proposed a modified metric-based proper orthogonal decomposition approach for enhancing airfoil designs [
39].
In this study, an aerodynamic design optimization problem pertaining to airfoil geometries is employed to critically assess the efficacy of ASAPSO in practical applications. Recognizing that the evaluation of prospective airfoil configurations demands rigorous computational simulations rooted in fluid dynamics, the XFOIL platform is harnessed as an instrumental tool for the analysis and design of subsonic, isolated, single-segment airfoils. The airfoil aerodynamic optimization problem, conceived by Professor Mark Drela of the Massachusetts Institute of Technology, emerges as a quintessential application for the SAEA. This is attributed to the intricate computational fluid dynamics models imperative for prognosticating the performance metrics of potential airfoil configurations. Intrinsically, the optimization challenge revolves around the enhancement of the airfoil’s geometry to maximize the lift coefficient-to-drag coefficient ratio. Concurrently, it is paramount to curtail the number of evaluations. Delving into the technical nuances, the airfoil’s geometric intricacies are encapsulated by the Hicks–Henne function. Introduced by Hicks and Henne in the concluding years of the 1970s, this function leverages parametric techniques to modulate airfoil morphologies by characterizing the extent of variances in curvature and thickness. Subsequent to this, the derived variations are meticulously superposed onto the foundational curvature and thickness dimensions of the archetypal airfoil. Intriguingly, the Hicks–Henne typology [
32] amalgamates a foundational airfoil with a set of ten decision variables. The NACA0012 airfoil is a commonly used symmetrical airfoil, widely used in aviation fields such as low-speed stunt aircraft main wings and high-performance helicopter rotors. Due to its specific geometric shape and aerodynamic characteristics, precise computational simulations of it require the consideration of multiple factors, including fluid dynamics, pressure distribution on the airfoil surface, and lift and drag under different flight conditions. These calculations not only require accuracy, but also need to consider the effective utilization of computing resources to ensure the efficiency and accuracy of the calculations. Thus, the optimization design of NACA0012 airfoil is quietly difficult to solve. The problem is thus articulated as follows:
In the provided equation,
signifies the fitness value of the lift-to-drag ratio, while
denotes the ten decision variables formulated using a Hicks–Henne function. The parameters
represent predefined values for the Mach number, Reynolds number, and angle of attack. Within this study, two algorithms, CALSAPSO and GPEME, are evaluated alongside the newly proposed algorithm. The primary metric for gauging their performance is the lift-to-drag ratio. Results of this analysis can be found in
Table 6. It is noteworthy that the algorithm presented in this paper outperforms the others, particularly in terms of the maximum, median, and average values of the lift-to-drag ratio.
An instance of the symmetric airfoil NACA0012 is globally renowned and frequently employed as a benchmark for theoretical analyses and wind tunnel comparisons. Its geometry is parameterized at 0.5 before being simulated via the Xfoil software (0.0.8). It then undergoes an optimization process using ASAPSO, capped at 120 evaluations. In the provided visual, the initial airfoil is shown in green and its optimized counterpart in red, emphasizing the airfoil’s smooth and inherently symmetrical contour.
The comparison of the original airfoil geometry with ASAPSO airfoil geometry is given in
Figure 12. From
Figure 12, it can be illustrated that the proposed ASAPSO algorithm exhibits better optimization airfoil geometry compared to NACA0012.
The integration of surrogate models with PSO can significantly enhance the efficiency of engineering design processes, particularly in fields like aerospace, automotive, and renewable energy. By optimizing complex systems such as airfoils or turbine blades, managers can achieve better performance outcomes, a reduced time-to-market, and lower development costs. By reducing the computational cost of simulations and improving the convergence speed of optimization algorithms, the proposed method can lower the overall cost of engineering projects. This makes advanced optimization techniques more accessible to smaller firms and industries with limited resources. The optimization of aerodynamic designs, such as airfoils, directly contributes to energy efficiency. For example, optimizing wind turbine blades can lead to increased energy capture, while optimizing aircraft wing designs can reduce fuel consumption. These improvements have significant environmental benefits, contributing to the reducing of carbon emissions and promoting sustainability.