The Successive Approximation Genetic Algorithm (SAGA) for Optimization Problems with Single Constraint

Abstract

A limited feasible region restricts individuals from evolving optionally and makes it more difficult to solve constrained optimization problems. In order to overcome difficulties such as introducing initial feasible solutions, a novel algorithm called the successive approximation genetic algorithm (SAGA) is proposed; (a) the simple genetic algorithm (SGA) is the main frame; (b) a self-adaptive penalty function is considered and the penalty factor is adjusted automatically by the proportion of feasible and infeasible solutions; (c) a generation-by-generation approach and a three-stages evolution are introduced; and (d) dynamically enlarging and reducing the tolerance error of the constraint violation makes it much easier to generate initial feasible solutions. Then, ten benchmarks and an engineering problem were adopted to evaluate the SAGA in detail. It was compared with the improved dual-population genetic algorithm (IDPGA) and eight other algorithms, and the results show that SAGA finds the optimum in 5 s for an equality constraint and 1 s for an inequality constraint. The largest constraint violation can be accurate to at least three decimal fractions for most problems. SAGA obtains a better value, of 1.3398, than the eight other algorithms for the engineering problem. In conclusion, SAGA is very suitable for solving nonlinear optimization problems with a single constraint, accompanied by more accurate solutions, but it takes a longer time. In reality, SAGA searched for a better solution along the bound after several iterations and converged to an acceptable solution in early evolution. It is critical to improve the running speed of SAGA in the future.

1. Introduction

As novel and efficient optimization algorithms, heuristic algorithms have been applied widely in many kinds of optimization areas, such as the sizing, topology and geometry optimization of steel structures [1], the optimization of the seismic behavior of steel planar frames with semi-rigid connections [2] and so on [3,4,5,6,7,8]. Heuristic algorithms have remarkable advantages over mathematical programming methods; for example, the relevant derivative information of objectives, including the first-order Jacobian matrix or second-order Hessian matrix, is not needed to solve problems, and continuous or convex design regions or explicit constraints are not required. The genetic algorithm (GA) described by Holland [9] is a typical and the most-used heuristic algorithm and is composed of population initialization and calculating fitness, judgments, iterations, evaluations and updates. Moreover, other intelligent algorithms also have been proposed and utilized to solve various optimization problems, such as the simulated annealing algorithm [10], particle swarm algorithm [11], neural networks [12], ant colony algorithm [13], artificial fish swarm algorithm [14], firefly algorithm [15], cuckoo search [16], etc.

Genetic algorithms search for the optimum by imitating natural phenomena or referring to life experience, which has been demonstrated to be very efficient and low-cost in practice. In the past, many studies have studied and improved genetic algorithms to solve unconstrained and constrained optimization problems and achieved the desired effect. Nonlinear constrained optimization problems are more difficult to deal with than unconstrained optimization problems in practice, because the limited feasible region restricts individuals from evolving optionally, and inappropriate constraint handling can easily cause the algorithm to converge to an infeasible solution or a feasible solution far from the optimum, or even fail to converge. A general form of nonlinear constrained optimization problems is shown in Equation (1), where $g_i\left(x\right)$ and $h_i\left(x\right)$ are inequality constraints and equality constraints, respectively, and at least one of objectives and constraints is nonlinear.

$$\begin{array}{l}\min f\left(x\right)\\ s.t. \{\begin{array}{ll}{g}_i\left(x\right)\le 0,& i=1, 2, \dots , m\\ h_i\left(x\right)=0,& i=m+1, m+2, \dots , n\end{array}\end{array}$$

(1)

The most common way to deal with constraints is to convert the constraints to objectives by introducing penalty functions in constrained optimization problems. The objective function is represented as shown in Equation (2); $G\left(x\right)$ is the objective after conversion, $C$ is a penalty factor, $\phi \left(x\right)$ is a combined penalty item and $\epsilon$ is a small positive number that represents the tolerance error of an equality constraint.

$$\begin{array}{l}G\left(x\right)=f\left(x\right)+C\phi \left(x\right)\\ \phi \left(x\right)={\displaystyle \sum _{\mathrm{i}=1}^n{\phi }_i\left(x\right)}\\ {\phi }_i\left(x\right)=\{\begin{array}{ll}\max \left\{0, g_i\left(x\right)\right\},& i=1, 2, \dots , m\\ \max \left\{0, \left|h_i\left(x\right)\right|-\epsilon \right\},& i=m+1, m+2, \dots , n\end{array}\end{array}$$

(2)

Thus, the original constrained problem is converted to an unconstrained problem, as shown in Equation (3).

$$\min G\left(x\right)$$

(3)

Many studies on constraint handling have been conducted in the literature. Typical penalty functions include static penalty functions [17,18,19], dynamic penalty functions [20,21], annealing penalty functions [20,22,23], self-adaptive penalty functions [24,25,26], etc. The details will be explained below. Furthermore, there are some other studies on constraint handling. Deb [27] exploited GA’s population-based approach and ability to make pair-wise comparisons in a tournament selection operator to devise a penalty function approach, in which any penalty parameter is not required; then, feasible and infeasible solutions are compared carefully, once sufficient feasible solutions are found,. A niching method (along with a controlled mutation operator) is used to maintain diversity among feasible solutions. Shang et al. [28] proposed a hybrid flexible tolerance genetic algorithm (FTGA) to solve nonlinear, multimodal and multi-constraint optimization problems, in which, as one of the adaptive genetic algorithm (AGA) operators, a flexible tolerance method (FTM) is organically merged into an AGA. Li and Du [29] proposed a boundary simulation method to address inequality constraints for GAs, which can efficiently generate a feasible region boundary point set to approximately simulate the boundary of the feasible region. Meanwhile, they proposed a series of genetic operators that abandon or repair infeasible individuals produced during the search process. Montemurro et al. [30] proposed the Automatic Dynamic Penalization (ADP) method, which belongs to the category of exterior penalty-based strategies, in which in order to guide the search through the whole definition domain and to achieve a proper evaluation of the penalty coefficients, it is possible to exploit the information restrained in the population at the current generation. Liang et al. [31] proposed a genetic algorithm to handle constraints which searches the decision space of a problem through the arithmetic crossover of feasible and infeasible solutions, and performs a selection on feasible and infeasible populations, respectively, according to fitness and constraint violations. Meanwhile, the algorithm uses the boundary mutation on feasible solutions and the non-uniform mutation on infeasible solutions and maintains the population diversity through dimension mutation. Garcia et al. [32] presented a constraint handling technique (CHT) called the Multiple Constraint Ranking (MCR), which extends the rank-based approach from many CHTs by building multiple separate queues based on the values of the objective function and the violation of each constraint and overcomes difficulties found by other techniques when faced with complex problems characterized by several constraints with different orders of magnitude and/or different units. Miettinen et al. [33] studied five penalty function-based constraint handling techniques to be used with genetic algorithms in global optimization and discovered the method of adaptive penalties turns out to be most efficient, while the method of parameter-free penalties is the most reliable. Yuan et al. [34] combined the genetic algorithm with hill-climbing search steps to propose a new algorithm which can be widely applied to a class of global optimization problems for continuous functions with box constraints. A repair procedure was proposed by Chootinan and Chen [35] and embedded into a simple GA as a special operator, in which the gradient information derived from the constraint set is used to systematically repair infeasible solutions. Without defining membership functions for fuzzy numbers, Lin [36] used the extension principle, interval arithmetic and alpha-cut operations for fuzzy computations, and a penalty method for constraint violations to investigate the possibility of applying GAs to solve this kind of fuzzy linear programming problem. Hassanzadeh et al. [37] considered nonlinear optimization problems constrained by a system of fuzzy relation equations and proposed a genetic algorithm with max-product composition to obtain a near-optimal solution for convex or nonconvex solution sets. Abdul-Rahman et al. [38] proposed a Binary Real-coded Genetic Algorithm (BRGA) which combines cooperative Binary-coded GA (BGA) with Real-coded GA (RGA), then introduced a modified dynamic penalty function into the architecture of the BRGA and extended the BRGA to constrained problems. Tsai and Fu [39] presented two genetic-algorithm-based algorithms that adopt different sampling rules and searching mechanisms to consider the discrete optimization via simulation problem with a single stochastic constraint. Zhan [40] proposed a genetic algorithm based on annealing infeasible degree selection for constrained optimization problems which shows great simulation results.

In order to explore the potential of genetic algorithms adequately and develop an algorithm suitable for engineering optimization, this paper proposes a novel and efficient genetic algorithm named the successive approximation genetic algorithm (SAGA) for nonlinear constrained optimization problems with a single constraint, considering a self-adaptive penalty function, dividing solution space into feasible and infeasible solutions and introducing a generation-by-generation approaching strategy. The performance of SAGA was evaluated in detail through a series of mathematical and engineering optimization problems; then, comparing it with other existing algorithms, it was further proved that SAGA has better accuracy and robustness. Considering that existing algorithm scripts are mostly written in MATLAB while the open source language Python is growing in popularity, all scripts in this paper were completed in Python and its third-party library, including the main program of SAGA (Python and Numpy), the images of the objectives and contour lines or surfaces (Numpy, Matplotlib and Mayavi), writing the results into the text (Python and Pandas), etc.

2. Related Work on Typical Penalty Functions

Just a few typical penalty functions are listed here. The equation constraint $h_i\left(x\right)=0$ is usually converted as $\left|h_i\left(x\right)\right|-\epsilon \le 0$, where $\epsilon$ is a small positive number. This section briefly summarizes some commonly used penalty function methods.

2.1. Static Penalty Functions

The penalty function proposed by Homaifar, Qi and Lai [17] is shown in Equation (4). The constraint violation is divided into several levels; $R_i$ is a penalty factor and $M$ is the number of constraint violations after converting the equation constraints to inequation constraints.

$$G\left(x\right)=f\left(x\right)+{\displaystyle \sum _{i=1}^M\left(R_i\times \max {\left\{0, g_i\left(x\right)\right\}}^2\right)}$$

(4)

Like a unilateral penalty, Hoffmeister and Sprave [18] proposed the penalty function shown in Equation (5) and adopted the Heavyside function $H\left(x\right)=\{\begin{array}{ll}0,& x\le 0\\ 1,& x>0\end{array}$.

$$G\left(x\right)=f\left(x\right)\pm \sqrt{{\displaystyle \sum _{i=0}^{m}H\left(-g_i\left(x\right)\right)}{g}_i{\left(x\right)}^2}$$

(5)

Kuri and Quezada [19] proposed the penalty function shown in Equation (6), where $n$ is the number of constraints, $q$ is the number of constraints met and $K$ is a very large number; they suggested ${10}^9$.

$$G\left(x\right)=\{\begin{array}{ll}f\left(x\right),& if feasible\\ K-{\displaystyle \sum _{i=1}^q}\frac{K}{n},& if infeasible\end{array}$$

(6)

2.2. Dynamic Penalty Functions

Joines and Houck [20] proposed the dynamic penalty function shown in Equation (7), where $C$, $\alpha$ and $\beta$ are three constants—they suggested 0.5, 1 or 2 and 1 or 2, respectively—$t$ is the current generation and $D_i\left(x\right)$ and $D_j\left(x\right)$ are the inequality and equality penalty item, respectively.

$$\begin{array}{l}G\left(x\right)=f\left(x\right)+{\left(Ct\right)}^{\alpha }\times SVC\left(\beta , x\right)\\ SVC\left(\beta , x\right)={\displaystyle \sum _{i=1}^m{D}_i{}^{\beta }\left(x\right)}+{\displaystyle \sum _{i=m+1}^n{D}_j\left(x\right)}\\ D_i\left(x\right)=\{\begin{array}{ll}0,& g_i\left(x\right)\le 0\\ \left|g_i\left(x\right)\right|,& else\end{array}, i=1, 2, \dots , m\\ D_j\left(x\right)=\{\begin{array}{ll}0,& \left|h_i\left(x\right)\right|\le \epsilon \\ \left|h_i\left(x\right)\right|,& else\end{array}, i=m+1, m+2, \dots , n\end{array}$$

(7)

2.3. Annealing Penalty Functions

Joines and Houck [20] proposed another annealing penalty function, shown in Equation (8), where the meanings of parameters are the same as above, and $C$, $\alpha$ and $\beta$ are suggested to be 0.05, 1 and 1, respectively.

$$G\left(x\right)=f\left(x\right)+e^{{\left(Ct\right)}^{\alpha }\times SVC\left(\beta , x\right)}$$

(8)

Michalewicz and Attia [22] divided constraints into four categories: linear equality constraints, nonlinear equality constraints, linear inequality constraints and nonlinear inequality constraints; then, they proposed the annealing penalty function shown in Equation (9), where $\tau$ is a variable parameter and gradually decreases based on iterations.

$$\begin{array}{l}G\left(x\right)=f\left(x\right)+\frac{1}{2\tau }{\varphi }_i{}^2\left(x\right)\\ {\varphi }_i\left(x\right)=\{\begin{array}{ll}\max \left\{0, g_i\left(x\right)\right\},& i=1, 2, \dots , m\\ \left|h_i\left(x\right)\right|,& i=m+1, m+2, \dots , n\end{array}\end{array}$$

(9)

Carlson and Shonkwiler [23] proposed a similar annealing penalty function, shown in Equation (10), where $M$ is a distance parameter and $T$ is a time parameter that gradually decreases to zero.

$$\begin{array}{l}G\left(x\right)=Af\left(x\right)\\ A=e^{-\frac{M}{T}}\\ T=\frac{1}{\sqrt{t}}\end{array}$$

(10)

2.4. Self-Adaptive Penalty Functions

Hadj-Alouane and Bean [26] utilized the feedback of search process to propose a new self-adaptive penalty function, shown in Equation (11), where ${\alpha }_1>{\alpha }_2>1$, case1 means all best solutions belong to feasible region in the previous k iterations, and case2 means all best solutions belong to infeasible region in the previous k iterations.

$$\begin{array}{l}G\left(x\right)=f\left(x\right)+\lambda \left(t\right)\left({\displaystyle \sum _{i=1}^m{g}_i{}^2\left(x\right)+}{\displaystyle \sum _{i=m+1}^n{h}_i{}^2\left(x\right)}\right)\\ \lambda \left(t+1\right)=\{\begin{array}{ll}\frac{1}{{\alpha }_1}\times \lambda \left(t\right),& if case1\\ {\alpha }_2\times \lambda \left(t\right),& if case2\\ \lambda \left(t\right),& else\end{array}\end{array}$$

(11)

3. Methodology

The penalty functions listed above have achieved delightful results in actual application, but have also faced some difficulties, for instance, the accurate parameters are difficult to select, an initial feasible solution that satisfies many constraints needs to be introduced beforehand and so on. The proposed successive approximation genetic algorithm (SAGA) in this paper adopts a series of easy and efficient strategies and gradually searches for the global optimum based on iterations. The main frame of SAGA is still a simple genetic algorithm (SGA). This section explains each part of SAGA in detail.

3.1. The Penalty Function of SAGA

The penalty function form in Equation (2) is still adopted for constraint handling in SAGA, and a self-adaptive penalty function is utilized in which it is critical to select the penalty factor $C$. $C$ should be a larger value in the earlier evolution because there are fewer feasible solutions, and $C$ should gradually decrease as iterations go on. The penalty factor $C$ selected in the paper is referred to by Cai [41], as shown in Equation (12), where $\rho$ represents the proportion of feasible solutions, $N_{fea}$ and $N_{all}$ represent the number of feasible solutions and all solutions, respectively, and $\alpha$ is used to control the range of penalty factors, and a large positive integer is generally preferred.

$$\begin{array}{l}C\left(\rho \right)=e^{\alpha \left(1-\rho \right)}-1\\ \rho =\frac{N_{fea}}{N_{all}}\end{array}$$

(12)

The penalty factor $C$ changes with $\rho$; when $\rho$ equals zero, no feasible solutions exist and $C$ reaches the maximum; when $\rho$ equals one, all solutions are feasible and $C$ reaches the minimum and equals zero, which means no individual needs to be penalized. One great advantage of such a penalty function is that the penalty factor is not too large or small, changes dynamically with the population and does not need to be specified manually.

3.2. The Successive Approximation Strategy of SAGA

Plenty of problems have proved that the global optimum is often located on the boundary of constraints. Some algorithms simply discard infeasible solutions during the solution process, which seems simple, but it is actually difficult to search for an exact optimal solution. There are also some algorithms that consider infeasible solutions to take full advantage of all solutions, which may be complicated to operate.

It is well known that equation constraint handling is more difficult; an optimization problem with an equality constraint is as shown in Figure 1, where the area enclosed by the outer black rectangle represents the design space and the inner black ring represents the equality constraint.

Evidently, it is very difficult, even impossible, to generate a solution that is located exactly on the equality constraint by randomly initializing the population. For inequality constraints $\left|h_i\left(x\right)\right|-\epsilon \le 0$ from the conversion of equation constraints, $\epsilon$ is used to distinguish feasible and infeasible solutions. The feasible region is transformed from a line to an area, but a small positive number $\epsilon$ means this area is relatively narrow, which leads to a larger popsize required to create the possibility of at least one solution being located in the feasible region after randomly initializing the population. Contrarily, a larger $\epsilon$ cannot be used in order to guarantee higher computation accuracy. While infeasible solutions are not useless, a better solution may be produced by the combination of a feasible solution and an infeasible solution, two feasible solutions or two infeasible solutions.

The successive approximation strategy of SAGA has a dynamically changing tolerance error of constraints. In the preliminary stage of evolution, a relatively large $\epsilon$ is given, which makes it very easy for the randomly initialized population to include several feasible solutions, and better solutions can be generated based on these temporary feasible solutions. Then, $\epsilon$ gradually decreases with the iterations. Finally, $\epsilon$ reaches the precision required. The whole progress is shown in Figure 1; green points and red stars represent feasible and infeasible solutions, respectively, the area between the two blue rings represents the current temporary feasible region after the conversion of equality constraint and the gray rings represent the boundaries of the temporary feasible regions of other generations. From (b) to (d) in Figure 1, $\epsilon$ gradually decreases, the temporary feasible region gradually shrinks and all solutions gradually approach the original equation constraint.

Considering that the maximum number of generations is relatively large, it is too cumbersome to operate if $\epsilon$ is updated after every evolution. Meanwhile, the temporary feasible region shrinking too rapidly may lead to the loss of feasible solutions, so $\epsilon$ is recommended to be 0.8 times the previous value after every five generations.

3.3. Three-Stages Evolution of SAGA

For the sake of making full use of all solutions, a new population of SAGA is composed of three parts, including the individuals evolving from two feasible solutions (the first part), a feasible and an infeasible solution (the second part) and two infeasible solutions (the third part). In the meantime, the whole evolution progress is divided into three stages including the first third, the middle third and the last third of the generations.

The number of feasible solutions in the solutions generated by randomly initializing the population is very small, and the preliminary stage of evolution is more likely to produce feasible solutions by evolution from infeasible solutions, which is easily found in Figure 1, so the second and third parts account for a larger proportion of the evolution. As iterations go on, the number of feasible solutions gradually increases. The algorithm is also likely to produce feasible solutions by evolution from feasible solutions, so the proportion of the first part in the total evolutionary population gradually increases and the proportion of the third part gradually decreases. In the later stage of evolution, there are many feasible solutions, so the proportion of the first part in the total evolutionary population further increases and the proportion of the third part further decreases (set as zero in the paper). The proportions of every part in different stages are set as shown in

Table 1. The proportion of individuals produced by each part in different stages.

Stage	Feasible + Feasible	Feasible + Infeasible	Infeasible + Infeasible
1	1/5	2/5	2/5
2	2/5	2/5	1/5
3	3/5	2/5	0

.

3.4. The Whole Progress of SAGA

SAGA is still mainly composed of population initialization, judgments, iterations, updates and evaluations. Simple and efficient real encoding is adopted in SAGA.

When initializing the population, firstly, two empty sets (set1 and set2) are defined to store feasible and infeasible solutions, respectively. Then, it initializes the population several times, saves the feasible solution obtained after each initialization into set1 until the number of feasible solutions is not less than 10, and only saves the infeasible solution obtained after the last initialization into set2. Finally, it selects set1 as the initial feasible solution set, and randomly selects (popsize—the length of set1) solutions without repeating set2 as the initial infeasible solution set.

The fitness of the individual is calculated according to Equation (13), where $c_{\max }$ is a larger positive number. Roulette wheel selection is adopted in SAGA.

$$fit\left(x\right)=\{\begin{array}{ll}{c}_{\max }-G\left(x\right),& G\left(x\right)<c_{\max }\\ 0,& else\end{array}$$

(13)

The recombination approach applied by Cai [41] is adopted in SAGA, as shown in Equation (14), where $x_i$ and $x_j$ are two parents, ${\tilde{x}}_i$ and ${\tilde{x}}_j$ are two offspring and $r_{11}$, $r_{12}$, $r_{21}$ and $r_{22}$ are four random numbers between zero and one.

$$\{\begin{array}{l}{\tilde{x}}_i=\frac{r_{11}}{r_{11}+r_{12}}{x}_i+\frac{r_{12}}{r_{11}+r_{12}}{x}_j\\ {\tilde{x}}_j=\frac{r_{21}}{r_{21}+r_{22}}{x}_i+\frac{r_{22}}{r_{21}+r_{22}}{x}_j\end{array}$$

(14)

The mutation approach applied by Cai [41] is adopted in SAGA as shown in Equation (15), where $x_k$ is a parent, ${\tilde{x}}_k$ is an offspring, $u_k$ and $l_k$ are the upper and lower bounds of a variable, respectively, $T$ is the maximum number of generations and $r$ is a random number between zero and one.

$$\{\begin{array}{ll}{\tilde{x}}_k=x_k+\frac{u_k-l_k}{T}r,& r\le 0.5\\ {\tilde{x}}_j=x_k-\frac{u_k-l_k}{T}r,& r>0.5\end{array}$$

(15)

During evolution, the solutions are divided into feasible and infeasible solutions after each evolution, and they are extended to the feasible and infeasible solution sets of the last evolution, respectively, so the lengths of set1 and set2 are larger. In order to prevent many similar feasible solutions from appearing too early, which may cause the algorithm to converge prematurely to an imprecise feasible solution, three upper limits on the number of feasible solutions involved in evolution are given in the three stages, as shown in

Table 2. The upper limits on the number of feasible solutions involved in evolution.

Stage	Number
1	$0.4\times popsize$
2	$0.6\times popsize$
3	$0.8\times popsize$

. For example, in the first stage, if the length of set1 is less than 0.4 times the popsize, all feasible solutions in set1 are involved in the next evolution, and the (popsize—the length of set1) best infeasible solutions in set2 are involved in the next evolution; contrarily, if the length of set1 is more than 0.4 times the popsize, the (0.4 times the popsize) best feasible solutions in set1 are involved in the next evolution, and the (popsize—0.4 times the popsize) best infeasible solutions in set2 are involved in the next evolution, similarly to other stages. A feasible solution determines whether it is good or bad based on the fitness of itself, and an infeasible solution determines whether it is good or bad based on its constraint violation.

Finally, all individuals of every generation are re-divided into feasible and infeasible solutions by using the target precision. If feasible solutions exist, the one with the largest fitness is selected as the best individual of the current generation. If no feasible solution exists, the one with the least constraint violation is selected as the best individual of the current generation. The complete process of SAGA is as shown in Figure 2.

4. Mathematical Tests and Results of SAGA

Ten nonlinear optimization benchmarks including five problems with an equality constraint and five problems with an inequality constraint were utilized to evaluate the performance of SAGA. All the scripts were written in Python and all the tests were conducted on an Intel Core i7-11370H 3.30GHz machine with 16 GB RAM and single thread under Win10 platform.

4.1. Benchmarks

Five benchmarks with an equality constraint were marked as f1, f2, f3, f4 and f5, and five benchmarks with an inequality constraint were marked as f6, f7, f8, f9 and f10. $n$ in f5 was set as 10 in the paper. The mathematical description and optimums of the ten benchmarks are presented in

Table 3. The mathematical expressions of ten benchmarks.

Benchmark	Mathematical Form	Optimum
f1	$\begin{array}{l}\min f(x)=x_1^2+x_2^2\\ s.t. \{\begin{array}{l}h(x)=x_1+x_2-2=0\hfill \\ -10\le x_i\le 10 \left(i=1,2\right)\hfill \end{array}\end{array}$	$f\left(1,1\right)=2$
f2	$\begin{array}{l}\min f(x)=x_1^2+{\left(x_2-1\right)}^2\\ s.t. \{\begin{array}{l}h(x)=x_2-x_1^2=0\hfill \\ -1\le x_i\le 1 \left(i=1,2\right)\hfill \end{array}\end{array}$	$f\left(\pm \frac{\sqrt{2}}{2},\frac{1}{2}\right)=\frac{3}{4}$
f3	$\begin{array}{l}\min f(x)=-2x_1^3+12x_1^2-16x_1\\ s.t. \{\begin{array}{l}h(x)=x_1^2-4x_1+x_2=0\hfill \\ 0\le x_i\le 5 \left(i=1,2\right)\hfill \end{array}\end{array}$	$f\left(2-\frac{2\sqrt{3}}{3},\frac{8}{3}\right)=-\frac{32\sqrt{3}}{9}$
f4	$\begin{array}{l}\min f(x)=\left(x_1+4\right)\left(x_2+2\right)-128\\ s.t. \{\begin{array}{l}h(x)=x_1{x}_2=128\hfill \\ 0\le x_i\le 20 \left(i=1,2\right)\hfill \end{array}\end{array}$	$f\left(16,8\right)=72$
f5	$\begin{array}{l}\min f(x)=-{(\sqrt{n})}^n{\displaystyle \prod _{i=1}^n{x}_i}\\ s.t. \{\begin{array}{l}h(x)={\displaystyle \sum _{i=1}^n{x}_i^2}-1=0\hfill \\ 0\le x_i\le 1(i=1,2,\dots ,n)\hfill \end{array} \end{array}$	$f\left(\frac{1}{\sqrt{n}},\frac{1}{\sqrt{n}},\dots ,\frac{1}{\sqrt{n}}\right)=-1$
f6	$\begin{array}{l}\min f(x)=x_1^2+x_2\\ s.t. \{\begin{array}{l}g(x)=2x_1-x_2-5\le 0\\ -10\le x_i\le 10, \left(i=1,2\right)\end{array}\end{array}$	$f\left(-1,-7\right)=-6$
f7	$\begin{array}{l}\min f(x)=-\sqrt{x_1{x}_2}\\ s.t. \{\begin{array}{l}g(x)=x_1+2x_2-12\le 0\\ 0\le x_i\le 10, \left(i=1,2\right)\end{array}\end{array}$	$f\left(6,3\right)=-3\sqrt{2}$
f8	$\begin{array}{l}\min f(x)=x_1+x_2\\ s.t. \{\begin{array}{l}g(x)=2x_1^2+x_2^2-54\le 0\\ -10\le x_i\le 10, \left(i=1,2\right)\end{array}\end{array}$	$f\left(-3,-6\right)=-9$
f9	$\begin{array}{l}\min f(x)=-x_1{x}_2{x}_3\\ s.t. \{\begin{array}{l}g(x)=x_1{x}_2+2x_2{x}_3+2x_3{x}_1-12\le 0\\ 0\le x_i\le 10, \left(i=1,2,3\right) \end{array}\end{array}$	$f\left(2,2,1\right)=-4$
f10	$\begin{array}{l}\min f(x)=\frac{{\left(x_1-5\right)}^2+{\left(x_2-5\right)}^2+{\left(x_3-5\right)}^2-100}{100}\\ s.t. \{\begin{array}{l}g(x)={\left(x_1-3\right)}^2+{\left(x_2-4\right)}^2+{\left(x_3-5\right)}^2-10\le 0\hfill \\ 0\le x_i\le 10 (i=1,2,3)\hfill \end{array}\end{array}$	$f\left(5,5,5\right)=-1$

.

The contour lines or surfaces of the objectives and feasible regions of the decision variables of the ten benchmarks were plotted with Matplotlib and Mayavi. In Figure 3, the optimum of every problem is marked with a red point, of which the corresponding decision variables are also marked in a red transparent box. For f5, $n=3$ makes the problem visible. It is important to note that the figure drawn for f3 is the image of the objective and constraint. The contour surfaces of f9 and f10 are three-dimensional, so the feasible regions are not marked to be observed more visually. Meanwhile, only half of the contour surfaces of f10 are drawn. Furthermore, most optimums of the ten benchmarks are located on the tangent points (tangent planes) of the contour lines (contour surfaces) and feasible regions.

4.2. Simulation Settings

To explore the appropriate popsize of SAGA to solve the problems well, the popsize of 200, 300, 400 and 500, respectively, was used to run the scripts. In order to obtain more than 10 feasible solutions in a short time in continuous initialization, the popsize used in the initialization was set as a slightly larger number, which was 500 for most of the problems in the paper. The value of $\alpha$ in the penalty factor $C$ was set as 7. The probability of recombination and mutation were set as 0.65 and 0.05. The maximum number of generations was set as 100. The initial tolerance error ${\epsilon }_0$ was set as 0.1, $0.1\times {0.8}^{\frac{100}{5}}=0.00115$, so the final tolerance error ${\epsilon }_u$ was set as 0.001. The stop criterion was that the maximum number of generations were reached.

All scripts of SAGA were run 30 times consecutively and the results were recorded.

4.3. Simulation Results

The best result of the 30 runs is defined as the one with the least constraint violation, and the worst result is that with the largest constraint violation. The results and time consumption with different popsizes are shown in

Table 4. The objective and constraint violation of 30 runs.

Benchmark	Popsize	Objective				Constraint Violation
		Best	Worst	Mean	SD	Min.	Max.	Mean
f1	200	2.0109	7.7494	2.1933	1.0318	0.000389	0.002874	0.001385
	300	2.0052	1.9962	2.0020	0.0049	0.001442	0.001997	0.001516
	400	1.9984	2.0034	2.0015	0.0057	0.000846	0.001530	0.001448
	500	2.0065	1.9973	2.0013	0.0042	0.001443	0.001539	0.001463
f2	200	0.7492	0.7489	0.7514	0.0031	0.001095	0.001915	0.001451
	300	0.7502	0.7518	0.7510	0.0019	0.000072	0.001568	0.001419
	400	0.7515	0.7517	0.7538	0.0128	0.000121	0.001580	0.001387
	500	0.7494	0.7488	0.7505	0.0029	0.000650	0.001528	0.001403
f3	200	−6.1584	−6.1584	−6.1584	0.0000	0.000086	0.001147	0.000614
	300	−6.1584	−6.1574	−6.1562	0.0109	0.000002	0.002831	0.000655
	400	−6.1584	−6.1584	−6.1584	0.0000	0.000011	0.002355	0.000623
	500	−6.1584	−6.1584	−6.1584	0.0000	0.000067	0.000960	0.000489
f4	200	72.0107	72.1093	72.4091	1.5158	0.000208	0.010938	0.002718
	300	72.1234	72.2529	72.0858	0.1543	0.000399	0.005289	0.001888
	400	72.0028	71.9974	72.1036	0.2258	0.000428	0.002846	0.001395
	500	72.0061	72.1437	72.1569	0.2669	0.000105	0.003645	0.001254
f5	200	−0.9631	−1.0007	−0.9880	0.0135	0.000019	0.001425	0.000753
	300	−0.9529	−0.9748	−0.9886	0.0132	0.000145	0.001444	0.000803
	400	−0.9922	−1.0007	−0.9958	0.0056	0.000161	0.001220	0.000825
	500	−0.9836	−1.0016	−0.9959	0.0062	0.000391	0.001286	0.000811
f6	200	−5.9986	−6.0022	−5.9997	0.0021	0.000000	0.002314	0.001449
	300	−5.9997	−6.0021	−6.0000	0.0021	0.000000	0.002087	0.001385
	400	−5.9987	−5.9937	−5.9986	0.0075	0.000059	0.001853	0.001335
	500	−5.9970	−5.9996	−6.0000	0.0022	0.000000	0.001559	0.001283
f7	200	−4.2430	−4.2434	−4.2429	0.0004	0.001291	0.002162	0.001580
	300	−4.2428	−4.2432	−4.2429	0.0003	0.000561	0.001732	0.001457
	400	−4.2430	−4.2432	−4.2430	0.0003	0.000945	0.001691	0.001492
	500	−4.2431	−4.2430	−4.2428	0.0005	0.001318	0.001618	0.001492
f8	200	−8.9995	−9.0002	−8.9997	0.0010	0.000000	0.002099	0.001267
	300	−9.0000	−9.0000	−9.0000	0.0001	0.000071	0.004253	0.001560
	400	−8.9999	−8.9996	−9.0000	0.0002	0.000000	0.001590	0.001291
	500	−9.0000	−9.0002	−9.0000	0.0002	0.000588	0.001881	0.001349
f9	200	−3.9995	−3.9999	−3.9989	0.0027	0.000000	0.001817	0.000887
	300	−3.9984	−4.0003	−3.9954	0.0144	0.000000	0.001588	0.000773
	400	−3.9995	−4.0001	−4.0001	0.0005	0.000000	0.001541	0.000904
	500	−3.9997	−3.9981	−4.0002	0.0005	0.000000	0.001491	0.000936
f10	200	−1.0000	−1.0000	−1.0000	0.0000	0.000000	0.000000	0.000000
	300	−1.0000	−1.0000	−1.0000	0.0000	0.000000	0.000000	0.000000
	400	−1.0000	−1.0000	−1.0000	0.0000	0.000000	0.000000	0.000000
	500	−1.0000	−1.0000	−1.0000	0.0000	0.000000	0.000000	0.000000

and

Table 5. The error and time consumption of 30 runs.

Benchmark	Popsize	Error			Time (s)
		Min.	Max.	Mean	Min.	Max.	Mean
f1	200	0.0109	5.7494	0.1952	0.73	0.79	0.76
	300	0.0052	0.0038	0.0042	1.56	5.32	4.73
	400	0.0016	0.0034	0.0037	2.75	9.40	5.86
	500	0.0065	0.0027	0.0037	4.19	14.36	6.35
f2	200	0.0008	0.0011	0.0023	0.73	2.53	2.31
	300	0.0002	0.0018	0.0018	1.53	1.66	1.60
	400	0.0015	0.0017	0.0047	4.85	9.17	8.97
	500	0.0006	0.0012	0.0018	4.34	14.66	13.24
f3	200	0.0000	0.0000	0.0000	0.76	0.83	0.78
	300	0.0000	0.0010	0.0022	1.59	1.72	1.64
	400	0.0000	0.0000	0.0000	2.79	2.93	2.87
	500	0.0000	0.0000	0.0000	4.27	4.41	4.33
f4	200	0.0107	0.1093	0.4097	0.73	0.88	0.80
	300	0.1234	0.2529	0.0863	1.42	1.73	1.64
	400	0.0028	0.0026	0.1040	2.68	2.91	2.84
	500	0.0061	0.1437	0.1571	4.12	4.47	4.35
f5	200	0.0369	0.0007	0.0124	0.98	3.39	3.15
	300	0.0471	0.0252	0.0115	1.89	2.04	1.98
	400	0.0078	0.0007	0.0047	3.18	3.38	3.27
	500	0.0164	0.0016	0.0051	4.82	5.02	4.88
f6	200	0.0014	0.0022	0.0018	0.69	0.76	0.73
	300	0.0003	0.0021	0.0015	1.52	1.64	1.59
	400	0.0013	0.0063	0.0029	2.75	2.89	2.82
	500	0.0030	0.0004	0.0015	4.52	14.62	13.84
f7	200	0.0003	0.0007	0.0004	0.70	0.76	0.72
	300	0.0002	0.0006	0.0004	1.53	1.62	1.58
	400	0.0003	0.0006	0.0004	2.68	2.84	2.78
	500	0.0004	0.0004	0.0005	4.21	4.37	4.27
f8	200	0.0005	0.0002	0.0004	0.72	0.79	0.75
	300	0.0000	0.0000	0.0001	1.53	1.60	1.57
	400	0.0001	0.0004	0.0001	2.69	2.80	2.74
	500	0.0000	0.0002	0.0001	4.14	4.23	4.19
f9	200	0.0005	0.0001	0.0014	0.75	0.83	0.77
	300	0.0016	0.0003	0.0049	1.60	5.49	2.33
	400	0.0005	0.0001	0.0003	2.75	3.12	2.86
	500	0.0003	0.0019	0.0004	4.20	4.65	4.29
f10	200	0.0000	0.0000	0.0000	0.82	0.88	0.84
	300	0.0000	0.0000	0.0000	1.69	1.79	1.74
	400	0.0000	0.0000	0.0000	2.90	3.04	2.98
	500	0.0000	0.0000	0.0000	4.45	4.68	4.54

, including the best, worst, mean and standard deviation (SD) of the objectives; the minimum, maximum and mean of the constraint violation; the minimum, maximum and mean errors of the objectives; and the minimum, maximum and mean of the time consumption.

The optimum is located on the bound of the constraint for f1 to f9, and it is located in the constraint for f10. In general, for most problems, when the popsize increases, the best and worst values of the objectives of the 30 runs change slightly. In reality, the means of the 30 runs do not mean much because the objective of a solution with a large constraint violation may be closer to the theoretical optimum. The results also demonstrate that the optimums of the 30 runs have small standard deviations; in other words, the proposed SAGA performs with great robustness. Correspondingly, the relevant information of the constraint violation of a solution is more crucial, which directly determines the quality of a solution. The largest constraint violation of ten problems is 0.010938, the others are accurate to at least three decimal fractions.

The tolerance error of the constraint violation and the accuracy of the objective are taken into account at the same time. Then, the appropriate popsize for each problem is found as the bolded texts in Table 4. For the problems with an equality constraint, the suitable popsize is 300 for f1, 200 for f2 and f3, and 400 for f4 and f5. For the problems with an inequality constraint, the suitable popsize is 200 for f6 to f10, which guarantees that the constraint violation can be accurate to three fractions and the objective can be accurate to two fractions.

The errors of the objectives and time consumption corresponding to the appropriate popsize are bolded in Table 5.

All errors corresponding the appropriate popsize listed in Table 5 are very small. For the problems with an equality constraint, SAGA takes an average time of 4.73, 2.31, 0.78, 2.84 and 3.27 s to search for the optimums. For the problems with an inequality constraint, SAGA takes an average time of 0.73, 0.72, 0.75, 0.77 and 0.84 s to search for the optimums. To sum up, SAGA solves the problems with an equality constraint in 5 s and with an inequality constraint in 1 s.

For each problem with the appropriate popsize, the decision variables corresponding to the best and worst constraint violation of 30 runs are presented in

Table 6. Decision variables corresponding to the best and worst results.

Benchmark	Popsize	Theoretical	Best	Worst
f1	300	$\left(1,1\right)$	(1.0344, 0.9670)	(0.9883, 1.0097)
f2	200	$\left(\pm \frac{\sqrt{2}}{2},\frac{1}{2}\right)$	(0.6951, 0.4843)	(0.6848, 0.4709)
f3	200	$\left(2-\frac{2\sqrt{3}}{3},\frac{8}{3}\right)$	(0.8453, 2s.6668)	(0.8453, 2.6678)
f4	400	$\left(16,8\right)$	(15.8377, 8.08197)	(16.0897, 7.9552)
f5	400	$\left(\frac{1}{\sqrt{10}},\frac{1}{\sqrt{10}},\dots ,\frac{1}{\sqrt{10}}\right)$	(0.3201, 0.3166, 0.3087, 0.3062, 0.3215, 0.3106, 0.3394, 0.3159, 0.3162, 0.3060)	(0.3296, 0.3190, 0.3191, 0.3169, 0.3196, 0.3032, 0.3074 0.3186, 0.3214, 0.3087)
f6	200	$\left(-1,-7\right)$	(−0.9926, −6.9839)	(−1.0123, −7.0268)
f7	200	$\left(6,3\right)$	(5.9562, 3.0226)	(6.0283, 2.9869)
f8	200	$\left(-3,-6\right)$	(−3.0421, −5.9574)	(−3.0039, −5.9963)
f9	200	$\left(2,2,1\right)$	(1.9673, 2.0035, 1.0147)	(1.9651, 2.0510, 0.9925)
f10	200	$\left(5,5,5\right)$	(5.0000, 5.0000, 5.0000)	(5.0018, 5.0000, 4.9999)

. The decision variables of the best and worst of the 30 best individuals simulated are very close to the theoretical solution, which proves the effectiveness and reliability of SAGA once again.

4.4. Evolution Figures

In order to show the entire evolution process of SAGA more intuitively, for each problem with the appropriate popsize, the evolution processes of the best and worst objectives and corresponding constraint violations of 30 runs are plotted as shown in Figure 4 and Figure 5. For example, SAGA-300-B in a figure means the propriate popsize is 300 and the evolution shown in the figure is the best one of the 30 runs; similarly, W means it is the worst one of the 30 runs.

For the problems with an equality constraint, the results show that the objective fluctuates in the early stages of evolution and gradually levels off in the late stages. The general tendency is that SAGA converges with iterations. Refer to the evolution progress of constraint violation; it is easy to see the phenomenon in which the constraint violation constantly oscillates near zero in the early stages and converges to zero at last, which indicates that SAGA is searching for better and better optimums near the bound of the equality constraint. The above results prove once again that the successive approximation strategy of SAGA is highly efficient.

For the problems with an inequality constraint, the feasible region is an area, not a line, which makes searching for some feasible solutions much easier, so the objective fluctuates slightly in the early stages of evolution and gradually levels off in the late stages. Similarly, SAGA converges with iterations in general. The constraint violation is not less than zero, and it gradually trends towards zero in fluctuations. In actuality, constraint violation may also be less than zero in the evolution, considering the corresponding solution satisfies the inequality constraint, so the constraint violation is set as zero in the figure above. These results also demonstrate that the solutions are more and more accurate and SAGA is highly efficient.

Moreover, combined with the evolution figures of the objectives and constraint violation, it is evident that SAGA converges to an acceptable solution before the maximum number of generations is reached.

Finally, on behalf of the optimization problems with an equality constraint and an inequality constraint, the scripts of f4 and f7 with a popsize of 200 were run once again and the distributions of the population during evolution were plotted to observe the convergence progress of SAGA. Only the distributions of the first, sixth, twelfth, twentieth, fiftieth and hundredth generations are shown in Figure 6 and Figure 7 to describe the process briefly, where the orange pluses represent all individuals of the current generation, the green star represents the best individual of the current generation and the red point represents the theoretical optimum.

The results show that all individuals are randomly distributed discretely in the whole design space at first, and they gradually move to the bound of constraint with iterations. Most individuals are near the bound after the sixth generation; in other words, SAGA starts to search for a better solution along the bound which satisfies the constraint better afterwards. In the end, all individuals collect near the theoretical optimum. These results again prove that SAGA solves the problems quickly and efficiently.

What is different from Figure 6 is that the feasible region is an area; in the early stages, most individuals are distributed in the feasible region. Other conclusions are consistent with the above.

5. Comparison with the Improved DPGA

Next, the improved dual-population genetic algorithm (IDPGA) was utilized for comparing the performance of SAGA, and the computing platform was consistent with the above. Two improved strategies, including a random dynamic immigration operator and keeping the best individuals, were adopted. Different popsizes and iterations were used for equality constraint problems. The popsize and iteration were taken as 100 for inequality constraint problems. Finally, the tolerance error for equality constraint was set as 0.01.

5.1. Simulation Results of IDPGA

Similarly, the results of 30 runs about objective, constraint violation, error, time consumption and decision variable are listed in

Table 7. The objective and constraint violation of 30 runs.

Benchmark	Iterations	Popsize	Objective				Constraint Violation
			Best	Worst	Mean	SD	Min.	Max.	Mean
f1	100	400	2.0782	1.9801	2.0711	0.1137	0.000381	0.009997	0.006635
		500	2.0105	1.9801	2.0012	0.0313	0.000806	0.010000	0.007284
f2	100	100	0.7664	0.7402	0.7480	0.0114	0.001078	0.009997	0.007357
		200	0.7503	0.7400	0.7415	0.0022	0.005613	0.010000	0.009244
f3	100	400	−6.1386	−5.9414	−6.1385	0.0451	0.000119	0.009369	0.004352
		500	−6.1584	−6.1579	−6.1550	0.0106	0.000383	0.009973	0.004198
f4	100	400	72.0317	72.0023	73.1233	2.2774	0.000027	0.009801	0.005178
		500	72.0039	71.9875	72.6433	1.2700	0.000407	0.009996	0.005583
f5	400	100	−0.8991	−0.8935	−0.9653	0.0773	0.003545	0.009922	0.008490
	500		−1.0024	−1.0313	−1.0161	0.0125	0.006404	0.009994	0.009385
f6	100	100	−5.9845	−6.0100	−6.0046	0.0109	0.003496	0.010000	0.009075
f7			−4.2408	−4.2462	−4.2450	0.0015	0.003126	0.009999	0.008755
f8			−8.9998	−9.0008	−8.9935	0.0141	0.000000	0.010000	0.005242
f9			−3.9930	−4.0043	−3.9712	0.0522	0.000000	0.009956	0.003277
f10			−1.0000	−1.0000	−1.0000	0.0000	0.000000	0.000000	0.000000

,

Table 8. The error and time consumption of 30 runs.

Benchmark	Iterations	Popsize	Error			Time (s)
			Min.	Max.	Mean	Min.	Max.	Mean
f1	100	400	0.0782	0.0199	0.0820	0.26	0.40	0.30
		500	0.0105	0.0199	0.0181	0.34	0.51	0.38
f2	100	100	0.0164	0.0098	0.0081	0.05	0.06	0.05
		200	0.0003	0.0100	0.0085	0.13	0.41	0.20
f3	100	400	0.0198	0.2170	0.0199	0.27	0.35	0.30
		500	0.0000	0.0005	0.0034	0.36	0.46	0.40
f4	100	400	0.0317	0.0023	1.1244	0.24	0.34	0.28
		500	0.0039	0.0125	0.6466	0.32	0.49	0.36
f5	400	100	0.1009	0.1065	0.0465	0.51	0.61	0.53
	500		0.0024	0.0313	0.0178	0.64	0.70	0.66
f6	100	100	0.0155	0.0100	0.0097	0.05	0.06	0.05
f7			0.0018	0.0035	0.0026	0.05	0.07	0.05
f8			0.0002	0.0008	0.0071	0.05	0.06	0.05
f9			0.0070	0.0043	0.0296	0.06	0.07	0.06
f10			0.0000	0.0000	0.0000	0.07	0.08	0.07

and

Table 9. Decision variables corresponding to the best and worst results.

Benchmark	Iterations	Popsize	Theoretical	Best	Worst
f1	100	500	$\left(1,1\right)$	(0.9336, 1.0672)	(0.9962, 0.9938)
f2	100	200	$\left(\pm \frac{\sqrt{2}}{2},\frac{1}{2}\right)$	(−0.6461, 0.4231)	(−0.7005, 0.5008)
f3	100	500	$\left(2-\frac{2\sqrt{3}}{3},\frac{8}{3}\right)$	(0.8439, 2.6631)	(0.8539, 2.6765)
f4	100	500	$\left(16,8\right)$	(15.8125, 8.0948)	(16.0150, 7.9919)
f5	500	100	$\left(\frac{1}{\sqrt{10}},\frac{1}{\sqrt{10}},\dots ,\frac{1}{\sqrt{10}}\right)$	(0.3082, 0.3448, 0.3036, 0.3458, 0.3269, 0.3139, 0.2875, 0.3058, 0.3135, 0.3177)	(0.3412, 0.3044, 0.3131, 0.3162, 0.2984, 0.3005, 0.3215, 0.3163, 0.3382, 0.3253)
f6	100	100	$\left(-1,-7\right)$	(−0.8622, −6.7280)	(−0.9978, −7.0056)
f7			$\left(6,3\right)$	(5.7796, 3.1118)	(6.0030, 3.0035)
f8			$\left(-3,-6\right)$	(−3.0098, −5.9900)	(−3.0095, −5.9913)
f9			$\left(2,2,1\right)$	(2.0736, 2.0599, 0.9348)	(2.0224, 2.0225, 0.9790)
f10			$\left(5,5,5\right)$	(5.0000, 5.0000, 5.0000)	(5.0000, 5.0000, 5.0000)

.

For equality constraint problems, IDPGA behaves better with a larger popsize or iteration. For inequality constraint problems, IDPGA reaches satisfying solutions with a small popsize and iteration. The constraint violation of all results is less than the set value (0.01). Similarly, IDPGA also performs with great robustness. IDPGA takes less than 0.7 s for equality constraint problems and less than 0.1 s for inequality constraint problems to search for the optimums, and the decision variables are close to the theoretical solutions.

5.2. Evolution Figures of IDPGA

The evolution processes of the best and worst result with the appropriate popsize and iteration are plotted in Figure 8 and Figure 9. For example, IDPGA-B in a figure means the evolution shown in the figure is the best one of the 30 runs; similarly, W means it is the worst one of the 30 runs.

For ten constraint problems, the results show that the objective does not fluctuate in the early stages, but follows one direction and gradually converges to the optimum. Similarly, the constraint violation oscillates in the early stages and converges near zero at last.

5.3. Comparison of Two Algorithms

In order to compare two algorithms, the best and worst results corresponding to the appropriate popsize and iteration are selected, whose error and constraint violation are shown in Figure 10. Apparently, SAGA can obtain a more precise solution and violate the constraint less than IDPGA. However, combined with the above, IDPGA takes much less time to search than SAGA.

5.4. An Engineering Problem

An engineering optimization problem was solved by two algorithms. The problem is to minimize the weight of a stepped cantilever beam under a vertical concentrated force, as shown in Figure 11. The constraint is the upper limit on the vertical displacement of the free end. The thickness of each element is supposed to be constant, and the decision variables are the width of each element. The mathematical expression is shown in Equation (16).

$$\begin{array}{l}\min f(x)=0.06224(x_1+x_2+x_3+x_4+x_5)\\ s.t. \{\begin{array}{l}g(x)=\frac{61}{x_1^3}+\frac{37}{x_2^3}+\frac{19}{x_3^3}+\frac{7}{x_4^3}+\frac{1}{x_5^3}\le 1\hfill \\ 0.01\le x_i\le 100 \left(i=1,2,3,4,5\right)\hfill \end{array}\end{array}$$

(16)

All results of two algorithms are listed in

Table 10. The objective and constraint violation of 30 runs.

AlgoriThm	Iterations	Popsize	Objective				Constraint Violation
			Best	Worst	Mean	SD	Min.	Max.	Mean
IDPGA	100	100	1.3440	1.3343	1.3391	0.0050	0.000000	0.009885	0.006631
	200		1.3344	1.3338	1.3335	0.0012	0.007149	0.009999	0.009638
SAGA	100	200	1.3429	1.6453	1.4892	0.1757	0.000000	0.000924	0.000122
		300	1.3460	1.3476	1.4436	0.1011	0.000000	0.000737	0.000087
		400	1.3398	1.3413	1.3742	0.0450	0.000000	0.000988	0.000208
		500	1.3408	1.3419	1.3669	0.0321	0.000000	0.000911	0.000205

and

Table 11. The time consumption of 30 runs.

Algorithm	Iterations	Popsize	Time (s)
			Min.	Max.	Mean
IDPGA	100	100	0.08	0.14	0.09
	200		0.16	0.17	0.17
SAGA	100	200	0.87	1.31	1.05
		300	1.83	2.80	2.25
		400	3.09	4.01	3.46
		500	4.52	5.52	4.95

.

If a constraint is required to be strictly satisfied, IDPGA obtains a best value of 1.3440 and SAGA obtains 1.3398. In reference [42], eight algorithms achieved the best values shown in

Table 12. The best objective of 8 algorithms.

Algorithm	Value	Algorithm	Value	Algorithm	Value
SOS	1.33996	GCA-I	1.3400	PSO-DE	1.339957
CS	1.33999	GCA-II	1.3400	AHA	1.339957
MMA	1.3400	MFO	3.399880

. From our findings, SAGA obtains a better value and it also obtains a satisfying value. If the constraint is relaxed appropriately, IDPGA can obtain a value of 1.3344, which can be acceptable in engineering. Furthermore, IDPGA solves the problem much faster than SAGA.

6. Conclusions

A new algorithm named the successive approximation genetic algorithm (SAGA) has been proposed by introducing several strategies. After solving a series of mathematic and engineering problems and comparing it with nine other algorithms, we proved that SAGA is highly accurate and effective in solving optimization problems with a constraint, but with a high time cost. In the future, there are several ways in which to improve SAGA. One aspect is to speed up the searching process of SAGA. We are trying to combine the advantages of SAGA and particle swarm optimization to greatly improve its solving efficiency. On the other hand, as seen in Table 1, the proportion of individuals of each part in each stage is determined artificially after trial calculation. Although the results of the problems in the paper are all satisfying, there is no doubt that the proportion of individuals needs to be evaluated in detail in the future. Most engineering problems have many constraints, so another aspect is to extend SAGA to solving optimization problems with multiple, even many, constraints. In this case, if the equality constraints are not properly scaled, the algorithm may easily fall into local optimal even stagnation, which is the direction we are currently exploring.