An Improved Gradient-Based Optimization Algorithm for Solving Complex Optimization Problems

Abstract

In this paper, an improved gradient-based optimizer (IGBO) is proposed with the target of improving the performance and accuracy of the algorithm for solving complex optimization and engineering problems. The proposed IGBO has the added features of adjusting the best solution by adding inertia weight, fast convergence rate with modified parameters, as well as avoiding the local optima using a novel functional operator (G). These features make it feasible for solving the majority of the nonlinear optimization problems which is quite hard to achieve with the original version of GBO. The effectiveness and scalability of IGBO are evaluated using well-known benchmark functions. Moreover, the performance of the proposed algorithm is statistically analyzed using ANOVA analysis, and Holm–Bonferroni test. In addition, IGBO was assessed by solving well-known real-world problems. The results of benchmark functions show that the IGBO is very competitive, and superior compared to its competitors in finding the optimal solutions with high convergence and coverage. The results of the studied real optimization problems prove the superiority of the proposed algorithm in solving real optimization problems with difficult and indefinite search domains.

1. Introduction

In recent years, information technology has had a deep impact on human civilization [1]. Due to this advancement, a massive amount of data needs to be analyzed, more complicated real-world problems need to be solved, and enhancement of the computing efficiency of computers is needed [2]. Artificial Intelligence (AI) has been a persistently hot topic to deal with this development. AI refers to the simulation of human intelligence in machines that are programmed to think like humans and mimic their actions [3]. The term may also be applied to any machine that exhibits traits associated with a human mind such as learning and problem-solving. AI methods in MG strategies contain Reasoning and Learning (RL) and Swarm Intelligence (SI) methods [4]. The SI-based algorithms have attained significant popularity among researchers and are considered one of the highest encouraging categories of AI especially metaheuristic algorithms [5]. In general, metaheuristic optimization algorithms try to imitate the physical, biological, or even chemical procedures that take place in the environment. However, some algorithms depend on mathematical theories [6]. The most generic ones are: Genetic Algorithm, which simulates Darwin’s theory of evolution [7], Simulated Annealing (SA) algorithm, that is developed from the thermodynamic process [8], Particle swarm optimization (PSO) algorithms, which simulate the behaviour of fish school or bird flock [9], Differential Evolution (DE) algorithm, that is applied in solving problems and functions by iteratively improving a candidate solution based on an evolutionary process [10], Teaching Learning-Based Optimization (TLBO), which is based on a teaching-learning process [11], Jaya algorithm, that based on the concept that tries to reach the best solution and tries to avoid failure to move away from the worst solution [12], Cuckoo Search (CS) Algorithm, which imitates the brood parasitism conduct of some cuckoo species in conjunction with the Levy flight conduct of certain birds and fruit flies [13], flower pollination algorithm (FPA) that brings its metaphor from the pollination in the flowering cycle of some plants in nature [14]. African Vultures Optimization Algorithm (AVOA) which imitates the nature of African vultures in foraging and navigation [15].

Gradient-based optimization (GBO) is a newly developed metaheuristic optimization to solve optimization problems. It contains search directions defined by the gradient of the function at the current point. GBO algorithm is motivated by Newton’s gradient procedure including two principal processes: gradient search rule process and local escaping operator. The gradient-based approach uses the gradient search rule to improve exploring phenomena and quickens the convergent rate of GBO to obtain the optimal position within the search space. However, the local escaping process prevents GBO to avoid getting stuck into the local optima [16].

In swarm-based algorithms, inertia weight is a concept utilized to balance the influence of the current position and the attraction to the best-known position in the search space. This helps the algorithm avoid getting trapped in local optima and to explore the search space more effectively. The value of inertia weight is typically decreased over time to increase the exploration [17].

In general, the optimization algorithm procedure steps, consist of parameter selection, variables (search agents), initialization, exploration, exploitation, randomization formula of the step search, selection of step, and terminating condition [18]. Each search agent in the population interacts with other agents to locate the optimal solution [19]. Generally, the swarm-based algorithms require some common control parameters like population size and the number of iterations [20]. In addition, some algorithms have specific control parameters besides the general parameters, known as hyper-parameters. These parameters exist to improve the performance of the algorithm by tuning their values properly [21].

The search agents in a SI system are designed to have simple rules. There is no central control to give order to how individual agents should perform [22]. The agent’s real performance is local, with a degree of arbitrary. However, the relations between such agents and the other parameters in the algorithm take the edge of the occurrence of “intelligent” to mimic the global behavior, while the individual agents cannot find alone [23].

The major contributions of this paper are listed as follows:

• Utilizing modified inertia weight in the original version of GBO to adjust the accuracy of the best solution. Whereas, the inertia weight in optimization algorithm, gives more weight to previous solutions in order to converge faster but also allows for exploration of new solutions.
• Modified parameters are utilized in GBO to boost the convergence speed and provide the proper balance of global and local search capabilities.
• A novel operator (G) is introduced which supports the diversity search in the search space. Whereas, G applied to move search agents toward better solutions leading to suitable performance both in the global search and local search using new developed formula.

The prove the superiority of the proposed IGBO, its performance is compared with GBO, CS, DE, FPA, PSO, TLBO, and AVOA using a wide range of benchmark functions on a few real-world problems.

The rest of the article is organized as follows: In Section 2 background and related works are presented. Section 3 presents a brief description of the GBO while the proposed IGBO is described in Section 4. Section 5 describes the benchmark function and Section 6 explains the real-world problems. The obtained results and performance comparison using benchmark functions against the different optimization algorithms are presented in Section 7. In Section 8 the proposed IGBO is employed to solve a challenging real-world optimization problem in the field of engineering. Finally, Section 9 summarizes the study.

2. Background and Related Works

The improvement of the optimization algorithm depends on the enhancement of the procedure steps or advancements of hybrid algorithms [24]. Every year there is competition in the algorithms on benchmarks function and real-world problems based on time, accuracy, and result to decide which algorithm is efficient [25]. There are different approaches to improving algorithms such as parameters tuning, opposite strategy, inertia weight, chaos strategy, fuzzy logic strategy, adding new operators, or multi-objective theory [26].

Adding inertia weight to adjust the accuracy and convergence speed toward the optimal solution [27]. There are different methods to implement inertia such as fixed and arbitrary inertia weights, and Adaptive inertia weights [28]. Tuning parameters to enhance convergence rate by many approaches. For example, with fixed values that are appropriate to the search process, some approaches gradually change the parameters of operators through the problem search process while, in some approaches, the mechanism will update the parameter for some instances of problems. Some of these methods aim to develop an adaptive mechanism to change the parameter value according to the search process [29]. Adding extra operators to balance exploration and exploitation and enhance diversity. There are different kinds of operators such as comparison/relational operators, stream operators, subscript operators, function operators, and arithmetic operators [30].

Because of the robustness of GBO, it is applied to solve quite complex real-world problems. The authors in [31] used GBO to find the optimal design automatic voltage regulator (AVR) using the GBO algorithm. In [32], GBO is used to calculate the reliability redundancy allocation problem of a series-parallel framework. In [33], GBO is employed in the estimation of the parameters of solar cells and photovoltaic (PV) panels as an effective and precise method. GBO in [34] is utilized in the calculation of the Economic Load Dispatch (ELD) problem for different situations such as ELD with transmission losses, and mixed economic and emission dispatch. In [35], GBO was utilized with Proton Exchange Membrane Fuel Cell to estimate the optimal parameters of three distinct kinds of PEM fuel cells. The scholars in [36] used an ensemble random vector functional link model (ERVFL) incorporated with GBO to model the ultrasonic welding of a polymeric material blend. ERVFL-GBO has the best outcome which indicates its high accuracy over other tested methods.

Despite the effective performance, GBO is trapped in local solution when conducting complicated non-linear functions, thus, it can decrease its accuracy. To overcome these drawbacks, different variants of the GBO have been introduced. The researchers in [37] used a multi-objective GBO algorithm-based Weighted multi-view Clustering (MO-GBO-WMV) to find the consensus clustering among different partitioning generated from individual views and compared this approach with some other methods to demonstrate the advanced ability of this approach. In [38] introduced a multi-objective gradient-based optimizer (MO-GBO) to handle the best solution for more than one objective, where needed. In [39], by improving the GBO using $C_a$ and $C_b$ which are new chaotic numbers generated by various chaos maps. Then IGBO is used to derive the parameters of PV modules. Similarly, in [40] a novel random learning mechanism is designed to improve the performance of GBO. After that, IGBO was utilized to extract parameters of four photovoltaic models. In [41], an improved gradient-based optimizer denoted by (CL-GBO) to build DNA coding sets that contain the Cauchy and Levy mutation operators, which are utilized as readers and addresses of the libraries. In [42] the goal was to solve single and multi-Economic Emission Dispatch problems, using an elegant method depending on mix-object both Manta ray foraging optimization (MRFO) with GBO, named (MRFO–GBO) to avoid trapped into local optima as well as accelerate the solution process. In [43] enhance the performance of Grey Wolf Optimizer (GWO) by a new procedure to use GBO to produce a new algorithm called (G-GWO). This combination was applied to improve the algorithm’s exploitation and exploration and also added Gaussian walk and Levy flight. These two are arbitrary operators utilized to increase the diversity search of exploration in the G-GWO. In [44] modified GBO algorithm using operator D to improve the stability between exploration and exploitation phases in the search process. Similarly, in [45] used binary search, which is an advanced type of search algorithm that finds and fetches data from a sorted list of items. It is called the binary GBO (B-GBO) algorithm. Then B-GBO is used in feature selecting problems of machine learning and data mining.

Optimization algorithms have gained a surge in popularity and have achieved significant attention from both academic and industrial fields. Here the review some recent optimization algorithms. Arithmetic Optimization Algorithm (AOA) has developed depending on well-known Arithmetic theory [46]. In [47], the scholars introduced Aquila Optimizer (AO), which mathematically models and mimics the nature of aquila during the procedure of hunting the prey. Dwarf Mongoose Optimization (DMO) algorithm is introduced in [48] which simulate the teamwork behaviors of the dwarf mongoose. The authors in [49] have developed Ebola Optimization search Algorithm (EOSA) based on the propagation behavior of the Ebola virus disease. Gazelle Optimization Algorithm (GOA) have presented in [50], which is inspired by the survival ability in their predator-dominated environment. The authors in [51] have developed prairie dog optimization (PDO) algorithm, which mimic the behaviour of four prairie dog in foraging and burrow building. Reptile Search Algorithm (RSA) has presented in [52], which is inspired by the nature of Crocodiles in hunting. The authors in [53], introduced oppositional unified particle swarm gradient-based optimizer based on mix of oppositional learning, unified particle swarm algorithm, and GBO algorithm to solve the complex inverse analysis of structural damage problems. In [54], the scholars have developed social engineering particle swarm optimization algorithm, which consists of combination of social engineering optimizer and particle swarm optimization to deal with structural health monitoring as objective function. However, this combination of more than algorithm may lead to slower convergence and inaccuracy in some optimization problems where is the speed is compulsory.

Summarizing the previous studies, the GBO algorithm has superiority over all modern counterparts in solving problems in different fields. However, ordinary GBO has some limitations such as:

GBO will still be trapped into local optima and suffer from the imbalance between exploitation and exploration, premature convergence, and slow convergence speed under some circumstances due to incomplete judgment standard and operators.

The main function of the local escaping operator (LEO) phase algorithm is to avoid the occurrence of local optimal stagnation, but only when the random number is less than 0.5, it will enter the LEO phase.

The former GBO does not identify the optimal solution for discrete functions which have discrete search spaces and decision variables such as feature selection problems.

There is only one guidance towards the best solution during the updating process, which limits the exploitation capability and can lead to the propensity of falling into the local optimal solution.

Former GBO does not have enough internal memory to save optimal solutions among all generations, which leads to a lack the population diversity. Moreover, the performance of the algorithm is affected significantly by the space domain of the objective function. However, an intensive searching process may lead to the deterioration of multimodal objective functions.

3. Original Gradient-Based Optimizer Algorithm

GBO was inspired by Newton’s gradient-based method. It combines concepts from gradient-based and population methods to find solutions. GBO uses two principles, the gradient search rule (GSR) and the local escaping operator (LEO). The GSR improves the exploration tendency and accelerates the convergence rate in the search space, while the LEO allows GBO to escape from local optima [16].

Metaheuristic algorithms such as GBO usually generate several stochastic operators without the demand to drop data and depend on steady performance. GBO mainly uses two operators and a set of vectors to explore the entire search space [40].

3.1. Initialization

Every individual in the whole population is denoted as a vector for a simple implementation description, which can be defined as:

$${\mathrm{X}}_{\mathrm{n},\mathrm{d}}=[{\mathrm{X}}_{\mathrm{n},1},{\mathrm{X}}_{\mathrm{n},2},\dots ,{\mathrm{X}}_{\mathrm{n},\mathrm{d}}],\mathrm{n}=1,2,\dots ,\mathrm{N};\mathrm{d}=1,2,\dots ,\mathrm{D}$$

(1)

Therefore, the GBO size population is denoted by N with the D-variables.

So, the vector is created randomly and represented as below:

$${\mathrm{x}}_{\mathrm{n}}={\mathrm{X}}_{\min }+\mathrm{rand}\left(0,1\right)\times \left({\mathrm{X}}_{\max }-{\mathrm{X}}_{\min }\right)$$

(2)

where ${\mathrm{X}}_{\min }$, and ${\mathrm{X}}_{\max }$ are the lower and upper limits the variables X, and rand (0, 1) is an arbitrary number in the boundary [0, 1].

3.2. Gradient Search Rule

The GSR operator is utilized in the gradient-based technique to enhance the exploration feasibility and enhance the convergence speed to find the optimal solution in the search domain. GSR expression is represented as

$$\mathrm{GSR}=\mathrm{norm}\times {\mathsf{\rho }}_1\times \frac{2\mathsf{\Delta }\mathrm{x}\times {\mathrm{X}}_{\mathrm{n}}}{\left({\mathrm{x}}_{\mathrm{worst}}-{\mathrm{x}}_{\mathrm{best}}+\mathsf{\epsilon }\right)}$$

(3)

where $\mathrm{norm}$ is a random value chosen within normal distribution, ${\mathrm{x}}_{\mathrm{worst}}$ and ${\mathrm{x}}_{\mathrm{best}}$ are the worst and optimal solution in the whole search progress of optimization, $\mathsf{\Delta }x$ is the value of increase, and ε is an arbitrary number chosen from the boundary (0 to 0.1).

To attain the balance between the exploration and exploitation phases and follow the search feasibility development, the GSR will be improved depending on the mathematical expression below:

$${\rho }_1=2\times rand\times \alpha -\alpha$$

(4)

$$\Delta \mathrm{x}=\mathrm{rand}\left(1:\mathrm{N}\right)\times \left|\mathrm{step}\right|$$

(5)

$$step=\frac{\left(x_{best}-x_{r1}^m\right)+\delta }{2}$$

(6)

$$\mathsf{\delta }=2\times \mathrm{rand}\times \left(\left|\frac{{\mathrm{x}}_{\mathrm{r}1}^{\mathrm{m}}+{\mathrm{x}}_{\mathrm{r}2}^{\mathrm{m}}+{\mathrm{x}}_{\mathrm{r}3}^{\mathrm{m}}+{\mathrm{x}}_{\mathrm{r}4}^{\mathrm{m}}}{4}-{\mathrm{x}}_{\mathrm{n}}^{\mathrm{m}}\right|\right)$$

(7)

$$\mathsf{\alpha }=\left|\mathsf{\beta }\times \sin \left(\frac{3\mathsf{\pi }}{2}+\sin \left(\mathsf{\beta }\times \frac{3\mathsf{\pi }}{2}\right)\right)\right|$$

(8)

$$\beta ={\beta }_{min}+\left({\beta }_{max}-{\beta }_{min}\right)\times {\left(1-{\left(\frac{m}{M}\right)}^3\right)}^2$$

(9)

where $\mathrm{rand}\left(1:\mathrm{N}\right)$ is an arbitrary value with N population, $\mathrm{r}1,\mathrm{r}2,\mathrm{r}3,\mathrm{and}\mathrm{r}4(\mathrm{r}1\ne \mathrm{r}2\ne \mathrm{r}3\ne \mathrm{r}4\ne \mathrm{n}$) are integers random numbers chosen from [1, N]. $rand$ is a random number between [0, −1], ${\beta }_{min}$ = 0.2, ${\beta }_{max}$ = 1.2, $m$ is the value of the current iteration, $M$ is the total iterations, and α is a function based on$\beta .$

The $step$ is a mathematical equation, that signifies the step size, which is based on ${\mathrm{x}}_{\mathrm{best}}$ and ${\mathrm{x}}_{\mathrm{r}1}^{\mathrm{m}}$, while ${\mathsf{\rho }}_2$ is represented by:

$${\mathsf{\rho }}_2=2\times \mathrm{rand}\times \mathsf{\alpha }-\mathsf{\alpha }$$

(10)

For more enhance of the exploitation of the surrounding area of ${\mathrm{x}}_{\mathrm{n}},$ by using the direction of movement$(\mathrm{DM}$). The $\mathrm{DM}$ can be determined by:

$$\mathrm{DM}=\mathrm{rand}\times {\mathsf{\rho }}_2\times \left({\mathrm{x}}_{\mathrm{best}}-{\mathrm{x}}_{\mathrm{n}}\right)$$

(11)

The GSR can be expressed as:

$$\mathrm{GSR}=\mathrm{rand}\times {\mathsf{\rho }}_1\times \frac{2\Delta \mathrm{x}\times {\mathrm{x}}_{\mathrm{n}}}{\left(y{p}_n-y{q}_n+\mathsf{\epsilon }\right)}$$

In which

$${\mathrm{yp}}_{\mathrm{n}}=\mathrm{rand}\times \left(\frac{\left[{\mathrm{z}}_{\mathrm{n}+1}+{\mathrm{x}}_{\mathrm{n}}\right]}{2}+\mathrm{rand}\times \Delta \mathrm{x}\right)$$

(12)

$${\mathrm{yq}}_{\mathrm{n}}=\mathrm{rand}\times \left(\frac{\left[{\mathrm{z}}_{\mathrm{n}+1}+{\mathrm{x}}_{\mathrm{n}}\right]}{2}-\mathrm{rand}\times \Delta \mathrm{x}\right)$$

(13)

$${\mathrm{z}}_{\mathrm{n}+1}={\mathrm{x}}_{\mathrm{n}}-\mathrm{rand}\times \left(\frac{2\Delta \mathrm{x}\times {\mathrm{x}}_{\mathrm{n}}}{{\mathrm{x}}_{\mathrm{worst}}-{\mathrm{x}}_{\mathrm{best}}+\mathsf{\epsilon }}\right)$$

(14)

Concerning the GSR and $\mathrm{DM}$, Equations. The position $\mathrm{X}{1}_{\mathrm{n}}^{\mathrm{m}}$ at any iteration is calculated using $\mathrm{GSR}$ and $\mathrm{DM}$ as follows:

$$\mathrm{X}{1}_{\mathrm{n}}^{\mathrm{m}}={\mathrm{x}}_{\mathrm{n}}^{\mathrm{m}}-\mathrm{GSR}+\mathrm{DM}$$

(15)

Full expression of $\mathrm{X}{1}_{\mathrm{n}}^{\mathrm{m}}$ can be written as

$$\mathrm{X}{1}_{\mathrm{n}}^{\mathrm{m}}={\mathrm{x}}_{\mathrm{n}}^{\mathrm{m}}-\mathrm{norm}\times {\mathsf{\rho }}_1\times \frac{2\mathsf{\Delta }\mathrm{x}\times {\mathrm{x}}_{\mathrm{n}}^{\mathrm{m}}}{\left({\mathrm{yp}}_{\mathrm{n}}^{\mathrm{m}}-{\mathrm{yq}}_{\mathrm{n}}^{\mathrm{m}}+\mathsf{\epsilon }\right)}+\mathrm{rand}\times {\mathsf{\rho }}_2\times \left({\mathrm{x}}_{\mathrm{r}1}^{\mathrm{m}}-{\mathrm{x}}_{\mathrm{r}2}^{\mathrm{m}}\right)$$

(16)

By substituting the position of the best vector (${\mathrm{x}}_{\mathrm{best}}$) with the current vector (${\mathrm{x}}_{\mathrm{n}}^{\mathrm{m}}$) in Equation (1), the new vector ($\mathrm{X}{2}_{\mathrm{n}}^{\mathrm{m}}$) can be calculated as follows:

$$\mathrm{X}{2}_{\mathrm{n}}^{\mathrm{m}}={\mathrm{x}}_{\mathrm{best}}-\mathrm{randn}\times {\mathsf{\rho }}_1\times \frac{2\mathsf{\Delta }\mathrm{x}\times {\mathrm{x}}_{\mathrm{n}}^{\mathrm{m}}}{\left({\mathrm{yp}}_{\mathrm{n}}^{\mathrm{m}}-{\mathrm{yq}}_{\mathrm{n}}^{\mathrm{m}}+\mathsf{\epsilon }\right)}+\mathrm{rand}\times {\mathsf{\rho }}_2\times \left({\mathrm{x}}_{\mathrm{r}1}^{\mathrm{m}}-{\mathrm{x}}_{\mathrm{r}2}^{\mathrm{m}}\right)$$

(17)

based on the positions$\mathrm{X}{1}_{\mathrm{n}}^{\mathrm{m}}$,$\mathrm{X}{2}_{\mathrm{n}}^{\mathrm{m}}$ and the existing position (${\mathrm{X}}_{\mathrm{n}}^{\mathrm{m}})$, at the next iteration, the new coming solution (${\mathrm{x}}_{\mathrm{n}}^{\mathrm{m}+1}$) can be calculated as:

$${\mathrm{x}}_{\mathrm{n}}^{\mathrm{m}+1}={\mathrm{r}}_{\mathrm{a}}\times \left({\mathrm{r}}_{\mathrm{b}}\times \mathrm{X}{1}_{\mathrm{n}}^{\mathrm{m}}+\left(1-{\mathrm{r}}_{\mathrm{b}}\right)\times \mathrm{X}{2}_{\mathrm{n}}^{\mathrm{m}}\right)+\left(1-{\mathrm{r}}_{\mathrm{a}}\right)\times \mathrm{X}{3}_{\mathrm{n}}^{\mathrm{m}}$$

(18)

where${\mathrm{r}}_{\mathrm{a}}$ and ${\mathrm{r}}_{\mathrm{b}}$ are arbitrary values within the range [0, 1].

3.3. Local Escaping Operator Process

The LEO operator provides an important feature of the GBO to escape local solutions and improve the convergence rate. It helps the algorithm escape from local minima or saddle points, which can trap the algorithm and prevent it from finding the global minimum.

There are areas in the search space that can prevent the algorithm from finding the global minimum. The operator works by introducing random variations or perturbations to the model’s parameters at each iteration. This helps the algorithm explore other regions of the parameter space and potentially discover a more optimal solution. As the optimization process continues, the amount of noise added is usually reduced to allow the algorithm to arrive at a more accurate result.

By using several positions ($X{1}_n^m,X{2}_n^m$), so LEO produces an optimal solution with advanced performance$(X_{LEO}^m$). The pseudo code of the operator LEO is shown in Algorithm 1.

Algorithm 1: Operator LEO

$if rand<pr$

$if rand<0.5$

$X_{LEO}^m=X_n^{m+1}+f_1\times \left(u_1\times x_{\mathrm{best}}-u_2\times x_k^m\right)+f_2\times {\rho }_1\times \left(u_3\times (X{2}_n^m-X{1}_n^m\right)+u_2\times \left(x_{r1}^m-x_{r2}^m\right))/2$

$X_n^{m+1}=X_{LEO}^m$

else

$X_{LEO}^m=x_{\mathrm{best}}+f_1\times \left(u_1\times x_{\mathrm{best}}-u_2\times x_k^m\right)+f_2\times {\rho }_1\times \left(u_3\times (X{2}_n^m-X{1}_n^m\right)+u_2\times \left(x_{r1}^m-x_{r2}^m\right))/2$

$X_n^{m+1}=X_{LEO}^m$

End

End

where $pr$ is a probability value, $pr$ = 0.5, the values $f_1$, and $f_2$ are represent distribution arbitrary values between [−1; 1],$u_1$, $u_2$, and $u_3$ are three random numbers, created as follows:

$${\mathrm{u}}_1=\left\{\begin{array}{c}2\times {\mathrm{rand}\mathrm{if}\mathsf{\mu }}_1<0.5\\ 1\mathrm{otherwise}\end{array}\right.$$

(19)

$${\mathrm{u}}_2=\left\{\begin{array}{c}{\mathrm{rand}\mathrm{if}\mathsf{\mu }}_1<0.5\\ 1\mathrm{otherwise}\end{array}\right.$$

(20)

$${\mathrm{u}}_3=\left\{\begin{array}{c}{\mathrm{rand}\mathrm{if}\mathsf{\mu }}_1<0.5\\ 1\mathrm{otherwise}\end{array}\right.$$

(21)

where $\mathrm{rand}$is an arbitrary value in the range of [0, 1], and ${\mathsf{\mu }}_1$ is a random number from the interval [0, 1]. The previous equations can be written as:

$${\mathrm{u}}_1={\mathrm{L}}_1\times 2\times {\mathsf{\gamma }}_1+\left(1-{\mathrm{L}}_1\right)$$

(22)

$${\mathrm{u}}_2={\mathrm{L}}_1\times {\mathsf{\gamma }}_2+\left(1-{\mathrm{L}}_1\right)$$

(23)

$${\mathrm{u}}_3={\mathrm{L}}_1\times {\mathsf{\gamma }}_3+\left(1-{\mathrm{L}}_1\right)$$

(24)

where ${\mathsf{\gamma }}_1,{\mathsf{\gamma }}_2,{\mathrm{and}\mathsf{\gamma }}_3$ are equal to $\mathrm{rand}.{\mathrm{L}}_1$ is a binary parameter with a value of 0 or 1. If parameter ${\mathsf{\mu }}_1$ is less than 0.5, the value of ${\mathrm{L}}_1$ is 1, otherwise, it is 0. To determine the solution ${\mathrm{x}}_{\mathrm{k}}^{\mathrm{m}}$ in Equation (25), the following scheme is suggested.

$${\mathrm{x}}_{\mathrm{k}}^{\mathrm{m}}=\left\{\begin{array}{c}{\mathrm{x}}_{\mathrm{rand}}{\mathrm{if}\mathsf{\mu }}_2<0.5\\ {\mathrm{x}}_{\mathrm{p}}^{\mathrm{m}}\mathrm{otherwise}\end{array}\right.$$

(25)

$${\mathrm{x}}_{\mathrm{rand}}={\mathrm{X}}_{\min }+\mathrm{rand}\left(0,1\right){\text{×}(\mathrm{X}}_{\max }-{\mathrm{X}}_{\min })$$

(26)

4. Improved Gradient-Based Optimizer Algorithm

In this study, an Improved gradient-based optimizer (IGBO) is introduced to solve different real problems accurately. The main motives for enhancing the original GBO are described in the following sections.

4.1. Varying Inertia Weight

The inertia weight concept was introduced in many works of literature. During the last few decades, researchers have developed many inertias weigh strategies, and it has an important role in optimization processes using population-based metaheuristic algorithms. It provides a good balance between the local search and the global search capabilities of the algorithm. A mathematical equation is used to generate a new inertia weight [55].

In this paper time-varying, inertia weight strategy has been used in which the value of the inertia weight is determined based on the iteration number. The linearly decreasing inertia weight was introduced by many researchers and was shown to be effective in improving the fine-tuning characteristic of algorithms [28]. The weight parameter $\omega$is computed as follows:

$$\mathsf{\omega }\left(\mathrm{Itr}\right)={\left(\frac{\mathrm{MaxItr}-\mathrm{Itr}}{\mathrm{MaxItr}}\right)}^2*\left({\mathsf{\omega }}_{\max }-{\mathsf{\omega }}_{\min }\right)+{\mathsf{\omega }}_{\min }$$

(27)

where $\mathsf{\omega }\left(\mathrm{Itr}\right)$: The inertia weight at iteration $\mathrm{Itr}$

${\mathsf{\omega }}_{\min }$: The minimum inertia weight (final), ${\mathsf{\omega }}_{\max }$: The maximum inertia weight (initial), $\mathrm{MaxItr}$: Iteration by which inertial weight should be at final.

Different optimization problems need different Inertia boundary values. The need just to adjust set the minimum and the maximum boundaries (${\mathsf{\omega }}_{\min }$, ${\mathsf{\omega }}_{\max }$) which are mentioned in some literature such as (0 and 1), (−1 and 1), or (0.9 and 1.2). In this study, for some optimization problems the suitable values for ${\mathsf{\omega }}_{\min }$ and ${\mathsf{\omega }}_{\max }$ are 0.4 and 0.9 respectively, and 0.8 and 0.2 for the other optimization problem. In each generation, the inertia weight $\mathsf{\omega }\left(\mathrm{Itr}\right)$ is generated automatically using Equation (27) within the selected range [56]. In this study, the magnitude of inertia weight is selected by the trial-and-error technique, so $({\mathsf{\omega }}_{\min }=0.8$, ${\mathsf{\omega }}_{\max }=1.2)$. Then Inertia weight will be used in the first position generated by LEO.

So, the first position ${\mathrm{x}}_{\mathrm{n}}^{\mathrm{m}+1}$ can be defined as:

$${\mathrm{x}}_{\mathrm{n}}^{\mathrm{m}+1}=\mathsf{\omega }\left(\mathrm{Itr}\right)\times {\mathrm{r}}_{\mathrm{a}}\times \left({\mathrm{r}}_{\mathrm{b}}\times \mathrm{X}{1}_{\mathrm{n}}^{\mathrm{m}}+\left(1-{\mathrm{r}}_{\mathrm{b}}\right)\times \mathrm{X}{2}_{\mathrm{n}}^{\mathrm{m}}\right)+\left(1-{\mathrm{r}}_{\mathrm{a}}\right)\times \mathrm{X}{3}_{\mathrm{n}}^{\mathrm{m}}$$

(28)

The new strategy holds the simple idea that the particles should visit more positions in the search space at the beginning and have better local search ability at the end.

4.2. Adaptive Parameters

The tuning of the parameters has a huge impact on the optimization performance of the metaheuristic approach [57]. For different problems, they need different values of parameters; some problems need to enhance the diversity and the convergence speed. In another aspect, some optimization problems need to escape the higher level of diversity search that may lead to rash convergence and decreased convergence speed [58]. GBO has very important parameters to find globally the best solutions. However, GBO uses a fixed and random (rand) value. To escape the awkward parameters and achieve fine-tuning, a new parameter-setting-free technique is presented. The mechanism delivers the issue of setting five key parameters inside GSR and LEO [59].

4.2.1. Adaptive Parameters in GSR

The original GBO, the parameters ${\mathrm{r}}_{\mathrm{a}}{\mathrm{and}\mathrm{r}}_{\mathrm{b}}$designed to have a random magnitude in the interval between [0, 1]. However, by the tests, we could find advanced optimal by giving high value to${\mathrm{r}}_{\mathrm{a}}$and low values to ${\mathrm{r}}_{\mathrm{b}}$ value within the domain [0, 1]. That can be done by changing the values of these parameters with mathematical equations used in IGBO. These mathematical equations extract the best values of the parameters, so with the increase in the number of iterations, the values of ${\mathrm{r}}_{\mathrm{a}}$ will raise using Equation (27), and the value of ${\mathrm{r}}_{\mathrm{b}}$ will decrease using Equation (28). That would help boost exploration to a larger extent and is likely to select the solutions from the entire feasible range [60]. Figure 1 show the visualization of adaptive parameters in GSR. The IGBO proposed the following equations to set the adjustment:

$${\mathrm{r}}_{\mathrm{a}}=\frac{\mathrm{Itr}}{\mathrm{MaxItr}}$$

(29)

$${\mathrm{r}}_{\mathrm{b}}=1-\left(\frac{\mathrm{Itr}}{\mathrm{MaxItr}}\right)$$

(30)

So, the new position at the next iteration ${\mathrm{x}}_{\mathrm{n}}^{\mathrm{m}+1}$ can be defined as:

$${\mathrm{x}}_{\mathrm{n}}^{\mathrm{m}+1}=\left(\frac{\mathrm{Itr}}{\mathrm{MaxItr}}\right)\times \left(\left(1-\left(\frac{\mathrm{Itr}}{\mathrm{MaxItr}}\right)\right)\times \mathrm{X}{1}_{\mathrm{n}}^{\mathrm{m}}+\left(1-\left(1-\left(\frac{\mathrm{Itr}}{\mathrm{MaxItr}}\right)\right)\right)\times \mathrm{X}{2}_{\mathrm{n}}^{\mathrm{m}}\right)+\left(1-\left(\frac{\mathrm{Itr}}{\mathrm{MaxItr}}\right)\right)\times \mathrm{X}{3}_{\mathrm{n}}^{\mathrm{m}}$$

(31)

4.2.2. Adaptive Parameters in LEO

Similarly, in standard GBO the parameters ${\mathsf{\gamma }}_1,{\mathsf{\gamma }}_2{\mathrm{and}\mathsf{\gamma }}_3$ set to have a random value in the range between [0, 1]. Whereas, to avoid the local optima, these parameters tend to have different approaches to achieve that goal. The parameter ${\mathsf{\gamma }}_1$ is likely to increase its rate steadily along with the increase in iterations. In both parameters ${\mathsf{\gamma }}_2{\mathrm{and}\mathsf{\gamma }}_3$ tend to step down the values of them gradually with the increase in iterations, to make the algorithm modify the solutions to get the best one. The best solutions keep on adding the number of iterations until the last [61]. Therefore, the randomness equations of the parameters can be calculated as:

$${\mathsf{\gamma }}_1=\frac{\mathrm{Itr}}{\mathrm{MaxItr}}$$

(32)

$${\mathsf{\gamma }}_2=1-\left(\frac{\mathrm{Itr}}{\mathrm{MaxItr}}\right)$$

(33)

$${\mathsf{\gamma }}_3=1-\left(\frac{\mathrm{Itr}}{\mathrm{MaxItr}}\right)$$

(34)

So, LEO uses the three random numbers ${\mathrm{u}}_1$, ${\mathrm{u}}_2$, and ${\mathrm{u}}_3$, will be generated as follows:

$${\mathrm{u}}_1={\mathrm{L}}_1\times 2\times \left(\frac{\mathrm{Itr}}{\mathrm{MaxItr}}\right)+\left(1-{\mathrm{L}}_1\right)$$

(35)

$${\mathrm{u}}_2={\mathrm{L}}_1\times \left(1-\left(\frac{\mathrm{Itr}}{\mathrm{MaxItr}}\right)\right)+\left(1-{\mathrm{L}}_1\right)$$

(36)

$${\mathrm{u}}_3={\mathrm{L}}_1\times \left(1-\left(\frac{\mathrm{Itr}}{\mathrm{MaxItr}}\right)\right)+\left(1-{\mathrm{L}}_1\right)$$

(37)

4.3. Adaptive Operator G

Increasing the range of random integers used in a special operator in optimization algorithm can assist the diversity and exploration of the search domain. This modification led to a higher probability of finding the optimal solution in the case of optimization functions, and that depends on the nature of both the specific algorithm and optimization problem [62].

The operator G enhances the feasibility of the search in the search domain. Moreover, it improves the balancing between exploration and exploitation processes to search for the global optimal solution by increasing the G value throughout iterations instead of being chosen as a constant value or random between (0,1). Subsequently, the operating time of the proposed IGBO is decreased in contrast with the former GBO [63].

This method is enhancing the performance of optimization algorithms seeking to obtain the best solution and decreasing the search space. The convergence of the approach according to how the movement of solutions in the search domain [64]. In the GBO algorithm, the direction of movement (DM) is used to converge near the area of the solution. Therefore, the suggestion is to change the DM randomly with extra values to discover more areas in the search domain [44]. Hence, the DM of IGBO is utilized to be

$$\mathrm{DM}=\mathrm{G}\times {\mathsf{\rho }}_2\times \left({\mathrm{x}}_{\mathrm{best}}-{\mathrm{x}}_{\mathrm{n}}\right)$$

(38)

Here, the value of the operator $\mathrm{G}$ changes significantly between specific limits.

The operator $\mathrm{G}$ can be expressed as

$$\mathrm{G}={\mathrm{g}}_1+\left(\left({\mathrm{g}}_2-{\mathrm{g}}_1\right)*\left(\mathrm{rand}\right)\right)$$

(39)

where ${\mathrm{g}}_1$and${\mathrm{g}}_2$ are real numbers chosen in this study to be (1 and 10). This range selected by trial method according to the nature of IGBO algorithm and nature of the selected optimization functions used in this research.

This operator improves the search process by discovering more areas in the search space. Moreover, it optimizes the balance between exploitation and exploration. The pseudo code of the IGBO algorithm is shown in Algorithm 2.

Algorithm 2. Pseudo code of the IGBO algorithm
1	Step 1. Initialization
2	Assign values for parameters$\mathrm{pr}$,$\mathsf{\epsilon }$, ${\mathsf{\omega }}_{\min }$, ${\mathsf{\omega }}_{\max },{\mathrm{g}}_1,{\mathrm{g}}_2$and M
3	Generate an initial population ${\mathrm{X}}_0=\left[{\mathrm{x}}_{0,1}{,\mathrm{x}}_{0,2}{,\dots ,\mathrm{x}}_{0,\mathrm{D}}\right]$
4	Evaluate the objective function value $\mathrm{f}\left({\mathrm{X}}_0\right),\mathrm{n}=1,\dots ,\mathrm{N}$
5	Specify the best and worst solutions ${\mathrm{x}}_{\mathrm{best}}^{\mathrm{m}}$ and ${\mathrm{x}}_{\mathrm{worst}}^{\mathrm{m}}$
6	Step 2. Main loop
7	While m<M (iterations) do
8	for n = 1 : N (particles) do
9	for i = 1 : D (dimensions) do
10	Select randomly$\mathrm{r}1\ne \mathrm{r}2\ne \mathrm{r}3\ne \mathrm{r}4\ne \mathrm{n}$ in the range of [1, N]
11	Calculate the operator G using Equation (37)
12	Compute the direction of movement (DM) using Equation (36)
13	Gradient search Rule
14	Calculate the parameters ${\mathrm{r}}_{\mathrm{a}}{\mathrm{and}\mathrm{r}}_{\mathrm{b}}$ using Equations (27) and (28).
15	Calculate the position ${\mathrm{x}}_{\mathrm{n},\mathrm{i}}^{\mathrm{m}+1}$ using Equation (29)
16	end
17	Local escaping operator
18	Calculate the parameters ${\mathsf{\gamma }}_1,{\mathsf{\gamma }}_2{\mathrm{and}\mathsf{\gamma }}_3$ using Equations (30)–(32)
19	Created random numbers ${\mathrm{u}}_1$, ${\mathrm{u}}_2$, and ${\mathrm{u}}_3$ using Equations (33)–(35)
20	if $\mathrm{rand}<\mathrm{pr}$then
21	Calculate the position ${\mathrm{x}}_{\mathrm{LEO}}^{\mathrm{m}}$
22	At the next iteration calculate the new coming position ${\mathrm{X}}_{\mathrm{n}}^{\mathrm{m}+1}$
23	end
24	Update the positions ${\mathrm{x}}_{\mathrm{best}}^{\mathrm{m}}$ and ${\mathrm{x}}_{\mathrm{worst}}^{\mathrm{m}}$
25	end
26	m=m+1
27	end
28	Step 3. return ${\mathrm{x}}_{\mathrm{best}}^{\mathrm{m}}$

5. Test Process

This process examines IGBO and its counterparts with benchmark functions along with Friedman rank, and Real-world problems to verify their performance.

5.1. Parameter Settings

In all quantitative experiments, the initial parameters of the competitor algorithms were set according to the reference papers, as shown in

Table 1. Initial parameters of the competitor algorithms.

Algorithm	Ref.	Parameter
		Name	Value
IGBO		Probability Parameter (pr)	pr = 0.5
		Minimum Balance Parameter (${\beta }_{min}$)	${\beta }_{min}=0.2$
		Maximum Balance Parameter (${\beta }_{max}$)	${\beta }_{max}=1.2$
		Inertia weight$s \left({\omega }_{min},{\omega }_{\max }\right)$	$({\omega }_{min}=0.7,$ ${\omega }_{\max }=1.2)$
GBO	[16]	Probability Parameter (pr)	pr = 0.5
		Minimum Balance Parameter (${\beta }_{min}$)	${\beta }_{min}=0.2$
		Maximum Balance Parameter (${\beta }_{max}$)	${\beta }_{max}=1.2$
CS	[13]	Discovery rate of alien eggs/solutions (p)	p = 0.25
DE	[65]	Scale Factor ($F$)	F = 0.5
		Crossover Probability rate ($C_r$)	$C_r=0.5$
FPA	[66]	Probability switch (p)	p = 0.8
PSO	[67]	Cognitive Constant (C1)	C1 = 2
		Social Constant (C2)	C2 = 2
		Minimum Inertia weight (${\omega }_{min}$)	${\omega }_{min}=$0.7
		Maximum Inertia weight (${\omega }_{max}$)	${\omega }_{max}=$0.9
		Maximum Velocity ($V_{max}$)	$V_{max}=$0.002
TLBO	[65]	Teaching factor ($T_f$)	$T_f=1, 2$
		Teaching step$(T_s$)	$T_s$chosen randomly between [0, 1]
AVOA	[15]	Probability parameter for selecting the first best vulture$(L_1$)	$(L_1=0.8$)
		Probability parameter for selecting the second-best vulture$(L_2$)	$(L_2=0.2$)
		A parameter that determines the disruption of the exploration and exploitation phases$(k$)	$(k=2.5$)
		A parameter to determine the probability of selecting the mechanisms in the exploration phase$(p_1$)	$(p_1=0.6$)
		A parameter to determine the probability of selecting the mechanisms in the exploitation phase of the first part$(p_2$)	$(p_2=0.4$)
		A parameter to determine the probability of selecting the mechanisms in the exploitation phase of the second phase$(p_3$)	$(p_3$ = 0.6)

.

5.2. Benchmark Test Functions

The quality of optimization algorithms is commonly examined by using conventional standard literature benchmark functions [68]. There are various categories of these test functions. That variety in functions, to achieve the best representation of a wider range of real-world large-scale optimization functions and compare various metaheuristic algorithms with more convenience and flexibility [69]. Some optimization algorithms, besides the IGBO algorithm, will be tested using different benchmark functions, which are categorized into two groups: (i) Unimodal benchmark functions in

Table 2. Unimodal benchmark functions.

f. No.	Name	Dim	Range	${\mathit{f}}_{\mathit{m}\mathit{i}\mathit{n}}$
F1	Sphere	30	[−100, 100]	0
F2	Schwefel’s 2.20	30	[−100, 100]	0
F3	Schwefel’s 2.21	30	[−100, 100]	0

, (ii) Multimodal benchmark functions in

Table 3. Multimodal benchmark functions.

f. No.	Name	Dim	Range	${\mathit{f}}_{\mathit{m}\mathit{i}\mathit{n}}$
F4	Qing	30	[−500, 500]	0
f5	Alpine N. 1	30	[−10, 10]	0
F6	Xin-She Yang N. 1	30	[−5, 5]	0
F7	Salomon	30	[−100, 100]	0
F8	Xin-She Yang N. 2	30	[−2 pi, 2 pi]	0
F9	Penalized	30	[−50, 50]	0

. In every table, (Dim) denotes the number of dimensions of the functions, while the lower and upper bounds of the variables are denoted by (Range), and the global minimum of the function is represented by ($f_{min}$) [70].

5.3. Statistical Analysis

In this section, measurement criteria and statical tests are applied to prove the significance of the IGBO algorithm different from the others. These tests are analysis of variance (ANOVA), and Holm–Bonferroni test. These two statistical tests were conducted on results obtained by every algorithm from 50 independent runs with 1000 iterations each.

5.3.1. Measurement Criteria

To evaluate the results five measured criteria were used. These measurements were the worst, best, mean, median, and standard deviation (SD) [16].

5.3.2. Wilcoxon Signed-Rank Test

It is a non-parametric statistical test, which utilized to evaluate if there is a significant difference between two correlated samples or repeated measurements on the same sample. It is employed to determine whether there are a statistically significant difference two related sets of observations.

5.3.3. Friedman Rank Test

The Friedman test is a non-parametric statistical test established by Milton Friedman [64]. The test makes a comparison among the mean ranks in the related groups and identifies how the groups differed. Moreover, tells you which group of data was rated best versus worst. The Friedman test is widely supported by many statistical software packages [65]. IBM SPSS software has been used in this work to calculate Friedman Rank Test.

6. Evaluation of the Igbo Algorithm on Real-World Engineering Problems

In this section, three real-world design problems are tested. The results obtained by IGBO were compared with those of different well-known optimizers suggested in previous studies. The population and the maximum number of iterations were 30 and 500, respectively, for all problems.

6.1. Three-Bar Truss Design Problem

This engineering problem illustrates the truss’s form and the forces applied to the structure as it is explained in Figure 2. This problem has two design parameters ($x_1, x_2$). The purpose of this problem is to minimize the total weight of the structure. It also has various constraints, such as deflection, buckling, and stress [15]; mathematically stated as follows:

$$\mathrm{Minimize}\mathrm{Fitness}\left(\overrightarrow{x}\right)=\left(2\sqrt{2}{x}_{A1}+x_{A2}\right)\times \mathrm{l}$$

(40)

Subject to:

$$g_1\left(\overrightarrow{x}\right)=\frac{\sqrt{2}{x}_{A1}+x_{A2}}{\sqrt{2}{x}_{A1}^2+2x_{A1}{x}_{A2}}P-\sigma \le 0$$

(41)

$$g_2\left(\overrightarrow{x}\right)=\frac{x_{A2}}{\sqrt{2}{x}_{A1}^2+2x_{A1}{x}_{A2}}P-\sigma \le 0,$$

(42)

$$g_3\left(\overrightarrow{x}\right)=\frac{1}{\sqrt{2}{x}_{A2}+x_{A1}}P-\sigma \le 0,$$

(43)

where, $0\le x_{A1},x_{A2}\le 1,\mathrm{l}=100\mathrm{cm},p=2\frac{\mathrm{kN}}{{\mathrm{cm}}^2},\mathsf{\sigma }=2\frac{\mathrm{kN}}{{\mathrm{cm}}^2}$

6.2. I-Beam Design Problem

It is also a difficult engineering structure optimization problem. The objective of this structure is to reduce the vertical deflection of the I-beam so that the design problem identifies the optimal geometric parameters related to the cross-section as displayed in Figure 3. The design variables in this problem are: length ($\mathrm{h}$ or ${\mathrm{x}}_1$), height (l or ${\mathrm{x}}_2$), and the thicknesses (${\mathrm{t}}_{\mathrm{w}}$ or ${\mathrm{x}}_3$ and ${\mathrm{t}}_{\mathrm{f}}$ or ${\mathrm{x}}_4$) [71]. This optimization problem and its constraints are explained in the equations as follows:

$$\mathrm{Considering}:\mathrm{X}=\left[{\mathrm{x}}_1,{\mathrm{x}}_2,{\mathrm{x}}_3,{\mathrm{x}}_4\right]=\left[\mathrm{h},\mathrm{l},{\mathrm{t}}_{\mathrm{w}},{\mathrm{t}}_{\mathrm{f}}\right]$$

(44)

$$\mathrm{Minimize}\mathrm{Fitness}=\frac{5000}{\frac{1}{12}{\mathrm{t}}_{\mathrm{w}}{\left(\mathrm{h}-2{\mathrm{t}}_{\mathrm{f}}\right)}^3+\frac{1}{6}{\mathrm{lt}}_{\mathrm{f}}^3+2{\mathrm{lt}}_{\mathrm{f}}{\left(\frac{\mathrm{h}-{\mathrm{t}}_{\mathrm{f}}}{2}\right)}^2}$$

(45)

Subject to:

$${\mathrm{g}}_1\left(\mathrm{x}\right)=2{\mathrm{lt}}_{\mathrm{f}}+{\mathrm{t}}_{\mathrm{w}}\left(\mathrm{h}-2{\mathrm{t}}_{\mathrm{f}}\right)\le 300$$

(46)

$${\mathrm{g}}_2\left(\mathrm{x}\right)=\frac{180000{\mathrm{x}}_1}{{\mathrm{t}}_{\mathrm{w}}{\left(\mathrm{h}-2{\mathrm{t}}_{\mathrm{f}}\right)}^3+2{\mathrm{lt}}_{\mathrm{f}}\left[4{\mathrm{t}}_{\mathrm{f}}^2+3\mathrm{h}\left(\mathrm{h}-2{\mathrm{t}}_{\mathrm{f}}\right)\right]}+\frac{15000{\mathrm{x}}_2}{\left(\mathrm{h}-2{\mathrm{t}}_{\mathrm{f}}\right){\mathrm{t}}_{\mathrm{w}}^3+2{\mathrm{t}}_{\mathrm{f}}{\mathrm{l}}^3}\le 6$$

(47)

The variables are subject to: $10\le \mathrm{h}\le 80$, $10\le \mathrm{l}\le 50,0.9\le {\mathrm{t}}_{\mathrm{w}}\le 5,0.9\le {\mathrm{t}}_{\mathrm{f}}\le 5$

6.3. Automatic Voltage Regulator Design Problem (AVR)

AVR is one of the real-world problems which the researchers tried to solve in different aspects [72]. Moreover, AVR is one of the main components in any power system which is used to control the outcome’s voltage under different conditions of the operating process. The objective function of AVR is to estimate the optimal parameters of the Fractional Order Proportional Integrator Derivative (FOPID) controller [73]. The mathematical representation of all the AVR parts using Laplace transformation. The output transfer function of all parts of IGBO-based FOPID controlling in the AVR system is shown in Figure 4. Moreover, the parts are the FOPID controller, amplifier, exciter, generator, and sensor [74].

$${\mathrm{G}}_{\mathrm{FOPID}}\left(\mathrm{s}\right)={\mathrm{K}}_{\mathrm{P}}+{\mathrm{K}}_{\mathrm{I}}{\mathrm{s}}^{-\Delta }+{\mathrm{K}}_{\mathrm{D}}{\mathrm{s}}^{\mathsf{\mu }}$$

(48)

$$G_A\left(s\right)=\frac{K_A}{1+S{T}_A}$$

(49)

$$G_E\left(s\right)=\frac{K_E}{1+S{T}_E}$$

(50)

$$G_G\left(s\right)=\frac{K_G}{1+S{T}_G}$$

(51)

$$G_S\left(s\right)=\frac{K_S}{1+S{T}_S}$$

(52)

7. Results and Analysis

The parameters of IGBO and other algorithms are selected according to the subsection parameter settings. Then, they have been tested and executed using the MATLAB (R2021a) desktop computer running Windows 10 Enterprise 64-bit with an Intel ^® Core ^TM (Santa Clara, CA, USA) i5-8500 CPU processor and 8.00 GB RAM. All results are stored based on 50 population sizes and 50 independent runs with 1000 iterations for every run, then the results are compared using the obtained results.

In benchmark test functions, the results found by IGBO are compared with seven well-known algorithms (GBO, CS, DE, PSO, FPA, TLBO, AVOA). In addition, the results of IGBO in real-world problems are compared with the result of the same counterpart algorithms.

7.1. Benchmark Test Functions Results

This section describes the results of IGBO and the chosen algorithms using variant benchmark functions. Further, the results computes and compares the descriptive statistics in terms of best, worst, median, mean value, and standard deviation of all the algorithms. The best result for each function is highlighted in boldface.

The results of IGBO for unimodal benchmark functions are compared with other algorithms in

Table 4. Unimodal Benchmark Functions Results.

f. No	Statistics	IGBO	GBO	CS	DE	FPA	PSO	TLBO	AVOA
f1	Best	0	3.16E-281	$1.02\times {10}^{-17}$	$4.29\times {10}^{-12}$	$1.75\times {10}^{+04}$	$5.13\times {10}^{-13}$	$7.08\times {10}^{-180}$	$2.67\times {10}^{-261}$
	Worst	0	$3.45\times {10}^{-267}$	$1.33\times {10}^{+03}$	$9.24\times {10}^{-11}$	$4.53\times {10}^{+04}$	$9.39\times {10}^{-11}$	$3.49\times {10}^{-176}$	$3.41\times {10}^{-258}$
	Median	0	$1.47\times {10}^{-272}$	$2.77\times {10}^{-10}$	$2.83\times {10}^{-11}$	$2.69\times {10}^{+04}$	$9.84\times {10}^{-12}$	$4.60\times {10}^{-178}$	$1.35\times {10}^{+260}$
	Mean	0	$1.43\times {10}^{-268}$	$3.18\times {10}^{+01}$	$3.20\times {10}^{-11}$	$2.79\times {10}^{+04}$	$2.07\times {10}^{-12}$	$3.19\times {10}^{-177}$	$1.92\times {10}^{-259}$
	Std	0	$8.54\times {10}^{-270}$	$1.90\times {10}^{+02}$	$1.83\times {10}^{-11}$	$6.25\times {10}^{+03}$	$2.23\times {10}^{-11}$	$2.71\times {10}^{-178}$	$1.96\times {10}^{-125}$
f2	Best	0	$7.07\times {10}^{-140}$	$3.78\times {10}^{-07}$	$1.10\times {10}^{-06}$	$3.18\times {10}^{+02}$	$1.44\times {10}^{-06}$	$4.81\times {10}^{-89}$	$9.34\times {10}^{-137}$
	Worst	0	$5.95\times {10}^{-134}$	$5.10\times {10}^{+02}$	$4.15\times {10}^{-06}$	$7.83\times {10}^{+02}$	$3.37\times {10}^{-04}$	$3.59\times {10}^{-87}$	$1.16\times {10}^{-111}$
	Median	0	$5.91\times {10}^{-137}$	$2.33\times {10}^{+00}$	$2.04\times {10}^{-06}$	$5.64\times {10}^{+02}$	$6.61\times {10}^{-06}$	$5.91\times {10}^{-88}$	$7.70\times {10}^{-121}$
	Mean	0	$6.62\times {10}^{-135}$	$2.79\times {10}^{+01}$	$2.15\times {10}^{-06}$	$5.73\times {10}^{+02}$	$2.63\times {10}^{-05}$	$8.23\times {10}^{-88}$	$2.32\times {10}^{-113}$
	Std	0	$1.51\times {10}^{-134}$	$7.85\times {10}^{+01}$	$6.28\times {10}^{-07}$	$8.34\times {10}^{+01}$	$6.28\times {10}^{-05}$	$6.91\times {10}^{-88}$	$2.34\times {10}^{-116}$
f3	Best	$1.67\times {10}^{-121}$	$3.87\times {10}^{-130}$	$1.05\times {10}^{+01}$	$8.88\times {10}^{-01}$	$7.81\times {10}^{+01}$	$9.33\times {10}^{-03}$	$9.66\times {10}^{-73}$	$8.52\times {10}^{-120}$
	Worst	$3.12\times {10}^{-85}$	$7.71\times {10}^{-123}$	$6.06\times {10}^{+01}$	$1.81\times {10}^{+00}$	$9.22\times {10}^{+01}$	$4.73\times {10}^{-02}$	$1.58\times {10}^{-70}$	$2.52\times {10}^{-121}$
	Median	$1.62\times {10}^{-111}$	$1.04\times {10}^{-125}$	$2.57\times {10}^{+01}$	$1.24\times {10}^{+00}$	$8.68\times {10}^{+01}$	$1.78\times {10}^{-02}$	$1.03\times {10}^{-71}$	$1.58\times {10}^{-123}$
	Mean	$1.62\times {10}^{-267}$	$3.37\times {10}^{-124}$	$2.79\times {10}^{+01}$	$1.26\times {10}^{+00}$	$8.63\times {10}^{+01}$	$1.95\times {10}^{-02}$	$1.66\times {10}^{-71}$	$9.28\times {10}^{-122}$
	Std	$3.35\times {10}^{+00}$	$1.81\times {10}^{-123}$	$1.23\times {10}^{+01}$	$2.35\times {10}^{-01}$	$3.10\times {10}^{+00}$	$7.96\times {10}^{-03}$	$2.39\times {10}^{-71}$	$2.72\times {10}^{-121}$

. IGBO algorithm obtained the lowest value in the results in the functions (f1, f2) excluding the function (f3). This shows that IGBO has least variation in results of compared to the competitor’s algorithms. Hence, IGBO is a better choice.

Table 5. Wilcoxon Signed Ranks test for unimodal benchmark functions.

IGBO vs.	GBO	CS	DE	FPA	PSO	TLBO	AVOA
Z	−2.097	−4.486	−5.170	−4.867	−6.245	−1.743	−3.341
p-values	$6.21\times {10}^{-05}$	$9.04\times {10}^{-12}$	$5.56\times {10}^{-16}$	$3.55\times {10}^{-14}$	$1.29\times {10}^{-17}$	$3.49\times {10}^{-19}$	$2.70\times {10}^{-09}$

illustrate p-values obtained from Wilcoxon rank-sum statistical test with 5% accuracy. By looking at the results, it is evident that IGBO has achieved excellent results with significant differences between the proposed IGBO approach and other optimization methods.

By examining the results of unimodal functions using Friedman mean ranking test, it is obvious that the proposed IGBO algorithm performed better than other counterpart algorithms and had been able to achieve the appropriate score of 3.826 in Friedman test as it is clear from Figure 5.

Multimodal benchmark functions are complicated tasks to test the exploration ability of the optimization algorithms to find the main optimal region of search domain.

The results of IGBO for multimodal benchmark functions are compared with other algorithms in

Table 6. Multimodal Benchmark Functions Results.

f. No	Statistics	IGBO	GBO	CS	DE	FPA	PSO	TLBO	AVOA
f4	Best	$5.54\times {10}^{-14}$	$7.80\times {10}^{-11}$	$2.93\times {10}^{-16}$	430.3679	$2.39\times {10}^{+10}$	$5.92\times {10}^{-10}$	$2.71\times {10}^{-20}$	$1.26\times {10}^{-06}$
	Worst	$6.29\times {10}^{-09}$	31.783	$73,933,906$	1186.599	$1.67\times {10}^{+11}$	$6.82\times {10}^{-07}$	$2.88\times {10}^{-12}$	8.9331
	Median	$2.91\times {10}^{-12}$	2.3485	$0.0000138$	874.0392	$7.87\times {10}^{+10}$	$1.01\times {10}^{-08}$	$1.70\times {10}^{-17}$	$1.68\times {10}^{-05}$
	Mean	$2.17\times {10}^{-10}$	3.863	$14,822,130$	865.4393	$8.37\times {10}^{+10}$	$3.22\times {10}^{-08}$	$5.84\times {10}^{-14}$	0.40381
	Std	$9.48\times {10}^{-10}$	5.3199	$10,455,346$	146.055	$3.49\times {10}^{+10}$	$9.65\times {10}^{-08}$	$4.07\times {10}^{-13}$	1.6162
f5	Best	$1.83\times {10}^{-158}$	$5.60\times {10}^{-145}$	$2.60\times {10}^{-08}$	0.021702	28.1595	1.78$\times {10}^{-08}$	$1.75\times {10}^{-90}$	$9.80\times {10}^{-127}$
	Worst	$5.54\times {10}^{-149}$	$1.80\times {10}^{-134}$	$36.36$	0.028265	53.1178	$4.85\times {10}^{-06}$	$1.15\times {10}^{-88}$	$8.90\times {10}^{-116}$
	Median	$2.67\times {10}^{-154}$	$1.20\times {10}^{-139}$	$5.8848$	0.025524	44.5526	$2.11\times {10}^{-07}$	$2.67\times {10}^{-89}$	$3.20\times {10}^{-122}$
	Mean	$1.44\times {10}^{-152}$	$4.20\times {10}^{-136}$	$8.6883$	0.025537	44.4585	$4.57\times {10}^{-07}$	$3.57\times {10}^{-89}$	$1.90\times {10}^{-117}$
	Std	$7.63\times {10}^{-150}$	$2.50\times {10}^{-135}$	$8.7131$	0.001555	5.0097	$8.49\times {10}^{-07}$	$2.90\times {10}^{-89}$	$8.29\times {10}^{-91}$
f6	Best	$5.54\times {10}^{-129}$	$5.00\times {10}^{-116}$	$6.31\times {10}^{-11}$	$1.02\times {10}^{-10}$	256,286.14	0.002556	$4.71\times {10}^{-55}$	$1.01\times {10}^{-86}$
	Worst	$5.54\times {10}^{-119}$	$2.58\times {10}^{-60}$	$79,748,031$	$5.21\times {10}^{-08}$	$1.74\times {10}^{+10}$	1.4607	$1.78\times {10}^{-13}$	$8.67\times {10}^{-03}$
	Median	$5.54\times {10}^{-125}$	$4.00\times {10}^{-100}$	$0.72962$	$2.22\times {10}^{-09}$	52,558,177	0.046811	$1.41\times {10}^{-28}$	$4.51\times {10}^{-65}$
	Mean	$5.54$	$5.22\times {10}^{-62}$	$16,042,004$	$5.65\times {10}^{-09}$	487,357,738	0.12426	$3.90\times {10}^{-15}$	$8.27\times {10}^{-53}$
	Std	$5.54\times {10}^{-14}$	$3.64\times {10}^{-61}$	$11,276,834$	$9.88\times {10}^{-09}$	244,656,464	0.25385	$2.52\times {10}^{-14}$	$7.53\times {10}^{-08}$
f7	Best	$0.1997$	$2.00\times {10}^{-130}$	4.4226	0.29987	13.0581	2.2999	0.29987	0.3275
	Worst	$1.1327$	$9.00\times {10}^{-101}$	17.0238	0.40037	22.4805	4.3999	2.58987	1.86584
	Median	$0.199873$	$3.40\times {10}^{-120}$	8.4499	0.31895	17.8066	2.9999	0.39884	2.7887
	Mean	$0.234792$	$1.80\times {10}^{-102}$	8.8499	0.33611	17.8965	3.0799	0.46187	0.847516
	Std	$0.03587$	$1.30\times {10}^{-101}$	2.9535	0.040328	1.9846	0.51627	0.014142	0.048512
f8	Best	$1.015\times {10}^{-12}$	$3.51\times {10}^{-12}$	$2.64\times {10}^{-11}$	$7.13\times {10}^{-11}$	$5.64\times {10}^{-11}$	$3.51\times {10}^{-12}$	$2.79\times {10}^{-11}$	$9.01\times {10}^{-12}$
	Worst	$8.72\times {10}^{-12}$	$3.45\times {10}^{-12}$	$1.05\times {10}^{-11}$	$2.32\times {10}^{-11}$	$1.05\times {10}^{-11}$	$2.71\times {10}^{-06}$	$1.12\times {10}^{-10}$	$3.58\times {10}^{-11}$
	Median	$2.29\times {10}^{-12}$	$3.47\times {10}^{-12}$	$1.73\times {10}^{-11}$	$2.65\times {10}^{-11}$	$2.87\times {10}^{-11}$	$6.54\times {10}^{-10}$	$6.02\times {10}^{-10}$	$7.18\times {10}^{-11}$
	Mean	$2.68\times {10}^{-12}$	$3.49\times {10}^{-12}$	$1.82\times {10}^{-11}$	$2.73\times {10}^{-11}$	$2.95\times {10}^{-11}$	$8.40\times {10}^{-10}$	$7.01\times {10}^{-10}$	$8.17\times {10}^{-11}$
	Std	$5.29\times {10}^{-13}$	$1.68\times {10}^{-13}$	$1.48\times {10}^{-12}$	$1.28\times {10}^{-12}$	$7.45\times {10}^{-13}$	$6.57\times {10}^{-07}$	$7.08\times {10}^{-10}$	$6.84\times {10}^{-12}$
f9	Best	$1.52\times {10}^{-18}$	$9.45\times {10}^{-06}$	$1.24\times {10}^{-16}$	$9.53\times {10}^{-12}$	$2.63\times {10}^{-5}$	$1.52\times {10}^{-18}$	$6.29\times {10}^{-24}$	$1.40\times {10}^{-10}$
	Worst	$2.55\times {10}^{-14}$	$0.10393$	$8.93\times {10}^{-08}$	$2.42\times {10}^{-10}$	$0.21576$	1.5588	$1.45\times {10}^{-18}$	$3.31\times {10}^{-09}$
	Median	$5.19\times {10}^{-17}$	$3.40\times {10}^{-05}$	$3.29\times {10}^{-11}$	$5.25\times {10}^{-11}$	$2.14\times {10}^{-03}$	0.10367	$6.62\times {10}^{-22}$	$9.63\times {10}^{-10}$
	Mean	$9.97\times {10}^{-16}$	$0.004205$	$1.46\times {10}^{-10}$	$6.66\times {10}^{-11}$	$0.014017$	0.17841	$8.04\times {10}^{-20}$	$1.16\times {10}^{-09}$
	Std	$3.81\times {10}^{-15}$	$0.020548$	$0.005938$	$5.15\times {10}^{-11}$	$6.72\times {10}^{-02}$	0.28147	$2.69\times {10}^{-19}$	$7.09\times {10}^{-09}$

. Analysis of the results of this table shows that IGBO with its high exploration power, has provided the global optimal for (f4, f5, f6, f8), which indicates the more effective performance of IGBO.

Non-parametric Wilcoxon sign rank test has archived significant results when it is applied on multimodal benchmark functions, which shown in

Table 7. Wilcoxon Signed Ranks test for multimodal benchmark functions.

IGBO vs.	GBO	CS	DE	FPA	PSO	TLBO	AVOA
Z	−4.189	−4.803	−7.574	−7.861	−3.986	−8.956	−5.122
p-Value	$1.23\times {10}^{-11}$	$1.34\times {10}^{-12}$	$3.62\times {10}^{-15}$	$1.82\times {10}^{-17}$	$6.90\times {10}^{-09}$	$8.48\times {10}^{-18}$	$1.054\times {10}^{-13}$

. The post hoc analysis confirms the effectiveness of the proposed method and it is statistically significant.

The obtained Friedman test results for multimodal functions are sown in Figure 6. The overall rank demonstrates that IGBO algorithm is superior to its counterparts. It obtained the lower value with 3.193.

To identify the nature of the algorithms with the functions, Figure 7 shown the 3D design, which allows for a more visual and intuitive understanding of the function’s behavior and properties. The convergence curve is demonstrated in Figure 8, the evaluation of convergence capability shows the robustness of IGBO against different algorithms.

7.2. Comparison of Computational Time

The modifications to improve original GBO, make it able to find optimal from the entire feasible range with the proper balance of global and local search capabilities Moreover, they effect to boost the convergence speed of IGBO compared to its counterparts. Each algorithm runs 1000 iterations and the average of the elapsed time is considered a criterion for computational time. The proposed algorithm needs less time to find the best solution measured in seconds.

Table 8. Comparison of computational time between IGBO and other algorithms.

Benchmark Test Functions	Elapsed Time (s)
	IGBO	GBO	CS	DE	FPA	PSO	TLBO	AVOA
Unimodal functions	6.2186	7.9960	12.6258	8.2360	11.1468	9.05327	9.6487	8.9960
Multimodal functions	19.5034	20.6731	21.6094	22.1904	22.6831	20.0217	23.536	21.8772

illustrates the comparison of computational time between IGBO and other algorithms.

7.3. Result of Real-World Problems

This part show results of the parameter values of the maximum function evaluations (MFEs), that compare the proposed IGBO algorithm against counterpart algorithms. All algorithms are used to solve real-world problems as mentioned before, with 50 numbers of population and 30 runs containing 1000 iterations.

7.3.1. Three-Bar Truss Design Results

Table 9. Comparison of best solutions for the three-bar truss design.

Optimal Cost		IGBO	GBO	CS	DE	FPA	PSO	TLBO	AVOA
MFEs		30,000	30,000	30,000	30,000	30,000	30,000	30,000	30,000
Best Weight		263.8258	263.9861	264.1753	264.6827	265.1275	264.8643	266.9627	264.5241
Variables	$\mathit{x}\left(\mathbf{1}\right)$	0.78868	0.78869	0.78677	0.78868	0.78868	0.78865	0.78871	0.78678
	$\mathit{x}\left(\mathbf{2}\right)$	0.40825	0.40825	0.41366	0.40825	0.40825	0.40831	0.40815	0.40832

shows the results of the comparative algorithms for solving the three-bar truss design problem, and Figure 9 shows the convergence curve and best positions of the three-bar truss design using IGBO.

The non-parametric Wilcoxon sign rank test has achieved significant results when applied to three-bar truss problem, as demonstrated in

Table 10. Wilcoxon and Friedman tests for three-bar truss problem.

	Test Type
	Wilcoxon		Friedman
	Z	p-Value	Mean Rank	Overall Rank
IGBO vs.	--	--	--	--
IGBO	--	--	1.233	1
GBO	3.014	$1.66\times {10}^{-23}$	2.862	2
CS	−3.682	$7.45\times {10}^{-29}$	4.355	4
DE	−5.062	$3.08\times {10}^{-35}$	6.509	5
FPA	−7.608	$5.21\times {10}^{-39}$	8.680	7
PSO	−6.853	$7.94\times {10}^{-36}$	9.754	8
TLBO	−7.911	$6.18\times {10}^{-41}$	7.953	6
AVOA	−3.849	$9.21\times {10}^{-34}$	3.561	3

. Moreover, In Friedman test, IGBO algorithm has achieved the smallest rank compared with other intelligent optimization algorithms. These tests confirm the effectiveness of the proposed method and it is statistically significant.

7.3.2. I-Beam Design Results

Table 11. Comparison of the best solutions for I-beam design.

Optimal Cost		IGBO	GBO	CS	DE	FPA	PSO	TLBO	AVOA
MFEs		30,000	30,000	30,000	30,000	30,000	30,000	30,000	30,000
Best Weight		0.013069	0.013074	0.013175	0.01593	0.013082	0.013075	0.013096	0.013075
Variables	$\mathbf{\left(}\mathit{h}\mathbf{\right)}$	79.98	80	80	80	80	80	80	80
	$\mathbf{\left(}\mathit{b}\mathbf{\right)}$	49.99	50	50	50	50	50	50	50
	$\mathbf{\left(}{\mathit{t}}_{\mathit{w}}\mathbf{\right)}$	0.9	0.9	0.9	0.9	0.9	0.9	0.9	0.9
	$\mathbf{\left(}{\mathit{t}}_{\mathit{f}}\mathbf{\right)}$	2.32179	2.32179	2.32181	2.32179	2.32181	2.32179	2.32179	2.32179

shows the results of the comparative algorithms for solving the I-beam design problem, and Figure 10 shows the convergence curve and best positions of I-beam design using IGBO.

Table 12. Wilcoxon and Friedman tests for I-beam problem.

	Test Type
	Wilcoxon		Friedman
	Z	p-Value	Mean Rank	Overall Rank
IGBO vs.	--	--	--	--
IGBO	--	--	2.617	1
GBO	−5.231	1.27 $\times {10}^{-31}$	4.108	3
CS	−8.573	7.95 $\times {10}^{-41}$	7.247	8
DE	−9.161	2.66 $\times {10}^{-46}$	4.976	4
FPA	−7.042	3.81 $\times {10}^{-37}$	5.862	6
PSO	−6.268	5.91 $\times {10}^{-33}$	5.354	5
TLBO	−7.843	1.78 $\times {10}^{-38}$	6.059	7
AVOA	−6.715	2.04 $\times {10}^{-34}$	3.865	2

shows Wilcoxon sign rank test and Friedman mean rank test for I-beam problem. Wilcoxon test has produced significant outcomes when used on I-Beam Design results. Furthermore, the overall ranking using Friedman test proved that IGBO algorithm is superior to other algorithms.

7.3.3. Automatic Voltage Regulator Design Results

Due to the novelty performance and advanced ability in tuning, the proposed study optimization of the AVR system contains a FOPID controller. Figure 11 shows step response of the AVR-based IGBO-FOPID controller.

The FOPID parameters estimated at the end of the IGBO search process are shown in

Table 13. Optimized FOPID parameters.

Parameter	Value
KP	1.508478
KI	0.992512
ʎ	1.051341
KD	0.621713
µ	1.219410

.

The transfer function model for the proposed GBO-based AVR system with the incorporation of optimized variables is given in the equation below:

$$\frac{V_{ref}\left(s\right)}{V_m\left(s\right)}=\frac{0.032251s^{3.5708}+3.2251s^{2.5708}+0.19585s^{2.3513}+19.585s^{1.3513}+0.056251s+5.6251}{0.0004s^{5.3513}+0.0454s^{4.3513}+0.555s^{3.3513}+3.2251s^{2.5708}+1.51s^{2.3513}+20.585s^{1.3513}+5.6251}$$

(53)

To validate the effectiveness of the proposed optimal AVR design, its dynamic response is compared with that of the previously designed AVRs under identical operating conditions as shown in Figure 12.

Table 14. Dynamic response evaluation of proposed GBO-based AVR design.

PID/FOPID Tuning Method	Peak Value (Pv)	Percentage Overshoot (%Mp)	Rise Time (tr)	Peak Time (tp)	Settling Time (ts)
IGBO-FOPID (Proposed)	1.084	8.36	0.0899	0.176	0.932
GBO-FOPID [31]	1.110	11.3	0.0885	0.16	0.653
JOA-FOPID [74]	1.130	13.2	0.0827	0.1750	0.453
SSA-FOPID [75]	1.15	15.5	0.0981	0.209	0.551
DE-PID [76]	1.3285	32.8537	0.1516	0.3655	2.6495
PSO- PID [76]	1.3006	30.0634	0.1610	0.3824	3.3994
ABC-PID [76]	1.2501	25.0071	0.1557	0.3676	3.0939
BBO-PID [77]	1.1552	15.5187	0.1485	0.3165	1.4457
PSA-PID [78]	1.1693	16.93	0.1438	0.3159	0.8039

provides the quantitative evaluation of the dynamic response based on some of the very important dynamic response indicators such as percentage overshoot, settling time, and peak time.

Conversely,

Table 15. Comparative AVR designs are based on stability evaluation indicators.

PID/FOPID-Tuning Method	PM	DM	BW	PG
IGBO-FOPID (Proposed)	95.9	0.0866	19.3	0.971
GBO-FOPID [31]	91.1	0.0753	21.27	1.2
JOA-FOPID [74]	90.3	0.0765	20.60	1.24
SSA-FOPID [75]	89.3	0.0910	17.01	1.21
DE-PID [76]	58.4	0.0920	12.8	4.2
PSO- PID [76]	62.2	0.1030	12.182	3.75
ABC-PID [76]	69.4	0.111	12.879	2.87
BBO-PID [77]	81.6	0.122	14.284	1.56
PSA-PID [78]	79.69	0.115	14.636	1.68

describes stability criterion results with the best AVR design-based algorithms. The stability indicators are Phase Margin (PM), Delay Margin (DM), Bandwidth (BW), and Phase Gain (PG). As can be seen from the proposed GBO tuned FOPID-AVR provides the most stable design among the considered AVRs with the highest PM and BW values.

In summary, the results of the real-world problems, demonstrate that IGBO can deal with different challenging problems and various combinatorial optimization problems. Thus, IGBO is the most powerful optimization algorithm with the lowest computational costs and high convergence speed to get the optimal solution.

8. Discussion

In this section, a comprehensive understanding of the performance of the proposed IGBO algorithm and its significance in comparison to the other algorithms studied in this manuscript. The key findings from this study are summarized as follows:

• The initial comparison that was made between IGBO and several state-of-the-art algorithms such as GBO, CS, DE, FPA, PSO, TLBO, and AVOA using unimodal benchmark functions with dimensions of 30. Whereas, the optimization based on 50 population sizes and 50 independent runs with 1000 iterations for every run. the proposed IGBO algorithm demonstrated outstanding performance in most cases.
• The second comparison between IGBO and the same its counterparts’ algorithms using the multimodal benchmark functions along with 30 dimensions. The results revealed a remarkable performance from the version of IGBO.
• The implementation of evolutionary algorithms competitors to several real-world constraint mechanical design optimization problems. The IGBO algorithm has shown great performance when applied to solving real-world constraint optimization problems, and as such, it has been successfully utilized to address automatic voltage regulator design.

It is worthwhile to mention that, despite the excellent optimizing capability of the IGBO algorithm, there are some limitations associated with the proposed IGBO such as solutions for the optimal temporary spatial-variation constraint sample are very difficult to achieve and comparatively greater computational complexity than its competing algorithms.

9. Conclusions and Future Works

In this manuscript, an Improved Gradient Optimization (IGBO) algorithm has been presented for solving real-world engineering and optimization problems. The proposed IGBO performance was examined using benchmark test functions to verify its effectiveness. To validate its superior optimization capabilities, its performance is compared with the seven most widely used metaheuristic-based optimization algorithms. The results demonstrate that the proposed algorithm is better than its competing algorithms in terms of achieving the most optimal solution to the benchmark functions and real-world engineering designs. Moreover, the statistical test analysis shows that the IGBO algorithm has better performance than its original version. In addition, the solution to three real-world problems was also examined and compared with other well-known algorithms to justify the performance of the proposed IGBO algorithm. The optimization results showed that IGBO has a high exploration ability in the scanning search domain, escaping local areas, and finding the main optimal area. IGBO is superior than the seven competitor algorithms and provides far more competitive optimization results in solving unimodal and multimodal benchmark functions. Moreover, IGBO performance in evaluating three design problems showed its high ability to solve real- world optimization problems.

Finally, multi-objective and binary forms of IGBO may be established as future works for solving multi-objective and discrete optimization problems. In addition, the utilize of a chaotic map in every iteration will improve the performance to avoid local optima and accelerate convergence. Furthermore, using IGBO for solving real-world optimization problems in different applications and domains can be another valuable future work.