Enhancing Mean-Variance Mapping Optimization Using Opposite Gradient Method and Interior Point Method for Real Parameter Optimization Problems

Abstract

The aim of optimization methods is to identify the best results in the search area. In this research, we focused on a mixture of the interior point method, opposite gradient method, and mean-variance mapping optimization, named IPOG-MVMO, where the solutions can be obtained from the gradient field of the cost function on the constraint manifold. The process was divided into three main phases. In the first phase, the interior point method was applied for local searching. Secondly, the opposite gradient method was used to generate a population of candidate solutions. The last phase involved updating the population according to the mean and variance of the solutions. In the experiments on real parameter optimization problems, three types of functions, which were unimodal, multimodal, and continuous composition functions, were considered and used to compare our proposed method with other meta-heuristics techniques. The results showed that our proposed algorithms outperformed other algorithms in terms of finding the optimal solution.

1. Introduction

Optimization methods are employed to solve real parameter optimization problems in order to obtain a vector $\mathit{x}=\left(x_1\dots x_D\right)$ that yields the optimal value of the function $f\left(\mathit{x}\right)$, where $D$ is the number of dimensions of the vector. To solve optimization problems, a global solution is searched without related knowledge or the physical structure of the cost function. There are real parameter optimization problems, such as engineering optimization problems, as well as those related to scientific applications, energy savings for tissue paper mills involving energy efficiency scheduling [1], the optimal scheduling of vehicle-to-grid energy and ancillary services [2], query optimization mechanisms in cloud computing [3], energy management [4], and tools for analytics in mechanical engineering [5].

Previously, a wide variety of evolutionary algorithms have been proposed to obtain solutions from real-world continuous optimization problems. For instance, the covariance matrix adaptation evolution strategy (CMA-ES) algorithm [6] is based on correlated mutations. The covariance matrix C has been adopted to enable the proper distribution of mutations, resulting in an increased likelihood of the successful replication of the search process. However, there are several black box properties that may promptly lead to premature CMA-ES convergence, with many uncontrollable variations and uncertainties. Therefore, to increase the possibility of finding the global optimum using the restart strategy, many techniques have been proposed based on CMA-ES, such as the restart CMA evolution strategy with increasing population sizes (IPOP-CMA-ES) [7], the bi-population CMA-ES strategy (BIPOP-CMA-ES) [8], and the new bi-population CMA-ES strategy (NBIPOPaCMA-ES) [9]. Usually, a new generation is randomly sampled based on the current covariance matrix in the desired search space. Consequently, the starting point is still randomized. Therefore, the time taken to find a solution increases as the amount of data increases. Moreover, the size of the search area and the surface of the cost function have also not been considered.

Another technique that solves the optimization problem is mean-variance mapping optimization (MVMO). MVMO [10] involves an evolutionary algorithm with two interesting issues. Firstly, the MVMO process determines the range of search areas for variables within an interval [0, 1]. This process ensures that new values computed later for the optimization population are always generated within this range before the fitness value is calculated. Secondly, the statistical features used in MVMO are useful for changing the search direction. Using the mean and variance of the different solutions and a mapping function for the mutation operation yields the optimal fitness value. Additionally, for each time that the algorithm produces better fitness values, the solutions with the n-best fitness values are updated and stored in the archive for use in finding the best solution, until the specified number of iterations has been reached. MVMO is applied in chemical process applications. For example, an adaptive PID controller based on MVMO has been proposed to enhance the performance of a chemical process with variable time delay [11]. Swarm-based mean-variance mapping optimization (MVMO-S) is an extension of MVMO, which is combined with swarm intelligence. The search procedure applied to this technique begins with the particles. They are represented in the form of the consistent function of archiving and mapping to the original MVMO [12]. Another extension of MVMO is the swarm variant of hybrid mean-variance mapping optimization (MVMO-SH) [13,14,15], which is an evolutionary algorithm. The algorithm takes advantage of the statistical features (mean and variance) of the dynamic search function, using mapping functions for mutation and modification according to the mean and variance of the n-best solutions that are recorded in the solution archive. The process of generating new offspring from techniques based on MVMO also uses randomness, regardless of the number of populations in the actual search surface of the cost function.

In addition to the algorithms mentioned above, there are still many algorithms that use natural behavior concepts to solve optimization problems, such as the seagull and gray wolf optimization algorithms. The seagull optimization algorithm (SOA) is a new type of bioinspired optimization algorithm that is based on the characteristics of a seagull. The SOA is combined with another algorithm to solve energy problems, such as short-term wind speed forecasting problems [16]. The gray wolf optimization algorithm (GWO) [17] relies on the level of leadership and the hunting mechanism of the gray wolf population for the search process of the optimization algorithm. The algorithms, based on different animal behavior patterns, are also subject to their effectiveness, which cannot be significantly improved when modified. Valenta and Langer proposed 2D P colonies to model the gray wolf optimization algorithm [18], where the performance was compared with that of the original GWO algorithm. From the computer simulation, the 2D P colonies had good performance for optimization problems.

Seagull, gray wolf optimization, and other evolutionary algorithms have the characteristics of self-management, adaptation, and self-learning. However, the new population determination process requires random initialization, resulting in relatively low computing efficiency. Considering the advantages and disadvantages of these algorithms, facing the same problem, the efficiency can be improved by modifying the original algorithms with other initialization concepts to avoid the randomization process.

Furthermore, a philosophy-inspired algorithm, namely, yin–yang pair optimization (YYOP) [19], uses the concept of balancing between two opposite entities for exploration and exploitation. YYPO is a low-complexity method that maintains good performance.

An important step in an optimization algorithm is the population generation step. Random initialization is used to generate new candidate solutions and eliminate the solutions that yield low scores, so that the cost value can either be maximized or minimized. However, the geometric structure of the cost function is not assessed at any step of the solution when generating offspring or for the search process. We proposed the application of the opposite gradient search [20] technique to search for the solution on the surface of the cost function. The technique, namely the fast opposite gradient search (FOGS) method, refers to a surface search algorithm applied with the opposite gradient method (OGM) [21]. FOGS is different from other search methods. It is not based mainly on meta-heuristics; instead, it seeks the manifold to obtain the locations where the cost function has zero gradients and minimum values.

This article introduces methods to increase the problem-solving efficiency with a better initial population for real parameter optimization problems. The proposed method presents a combination of the interior point method, the opposite gradient for generating new offspring, and the mean-variance mapping optimization algorithm (IPOG-MVMO). The mapping optimization algorithm is applied for the mutation operation to generate a modification depending on the n-best solution’s mean and variance, to obtain a candidate solution on a complex surface of the function. Therefore, this proposed research contribution involves generating new offspring and obtaining the solution from the high-dimensional space of the cost function.

The manuscript is structured as follows. Section 2 describes the concepts of the proposed technique. In Section 3, we present an experimental simulation and analyze the results. Section 4 presents the discussion. Finally, Section 5 provides the conclusions and scope for future research.

2. Proposed Concept

For the proposed method, there were three main phases. Firstly, IPM was applied for local searching. The results from the first phase were introduced as points used for the second phase to create a new population, which depended on the manifold of the cost function using OGM. Finally, the last phase aimed to obtain the best solution using MVMO.

Karmarkar [22] presented the interior point method (IPM) to solve a linear programming problem in polynomial time complexity. The number of iterations taken by IPM was not greater than 100, regardless of the size of the problem. Therefore, IPM is suitable for local searching for large problems. The repetitive path pattern of searching for IPM always walks within the domain of possible answers. Since each iteration of IPM is complicated, especially in cases where the coefficient matrix is dense, IPM spends more time on each iteration.

One important concept that has been used to find the best solution or enhance the IPM is the opposite gradient method (OGM). The OGM is used to create a new set of points from the point resulting from the IPM. MVMO then uses the new points from the OGM phase to mutate to achieve a better solution, instead of randomly generating points as the original MVMO. This proposed technique, based on a combination of IPM, OGM, and MVMO, is called IPOG-MVMO and yields a set of solutions throughout the surface of the cost function. The process of IPOG-MVMO is presented in Algorithm 1.

Algorithm 1 Proposed IPOG-MVMO algorithm

Initialize parameters.

Generate an initial population using IPM from randomized points.

Improve the population using OGM.

Update the archive using MVMO.

Return the best point in the archive.

2.1. Generating Initial Population Using Local Minima Obtained from the IPM

The proposed algorithm focused on the interior point method called the barrier method [23,24], because of the proof of convergence and its complexity. The IPM was used to find the local minima that are located near an initial population. A function $F\left(\mathit{x}\right)$ was defined and was differentiable in the neighborhood of a point $\mathit{x}$. The process involved formulating a rule to test a differentiable function for the local minima that satisfied the restrictions. The procedure is described in Algorithm 2, where the output of this algorithm is assigned as the initial population for the next phase.

Algorithm 2 Finding the local minima using IPM

$Q$: an archive storing a population

$D$: number of dimensions in search space

Randomize $N$ points in the feasible domain to generate an initial population.

Set $Q$ as an empty set.

Set parameters $t:= t_0>0, \mu >1, ϵ>0$.

FOR {$1 \le j \le N$}

Set a random point ${\mathit{x}}_{j }$ as a starting point.

WHILE ($\frac{D}{t}>ϵ$) do

Compute ${\mathit{x}}_j^{*}$ by minimizing $tF\left(\mathit{x}\right)+\varnothing \left(\mathit{x}\right),$ subject to $A{\mathit{x}}_j=b$.

Set ${\mathit{x}}_j:={\mathit{x}}_j^{*}$.

Update new $t$ as $\mu t.$

ENDWHILE

Insert ${\mathit{x}}_j$ into $Q$.

ENDFOR

RETURN $Q$ as an archive storing the initial population for the next phase.

According to Algorithm 2, there are three constants, $t_0>0, \mu >1, ϵ>0$, to be defined to initiate the algorithm. In each iteration, ${\mathit{x}}_j^{*}$ is computed by minimizing $tF\left(\mathit{x}\right)+\varnothing \left(\mathit{x}\right)$ subject to constraint $A{\mathit{x}}_j=b$ and $\varnothing \left(\mathit{x}\right)=-{{\displaystyle \sum }}_{i=1}^M\log \left(b_i-a_i\mathit{x}\right)$. The solution is updated until $t\ge D/ϵ$. All local minima in a set $Q$ are used as the initial population for the next step.

2.2. Opposite Gradient Method (OGM)

This section briefly presents a concept for generating new offspring. For a vector in a $D$-dimensional space of the cost function, new offspring are generated at the position of the manifold by OGM, where its first derivative is zero. One of these locations is expected to be the best solution for the cost function. The proposed technique proceeded along the same line with the following observation. The first derivative of $F\left(\mathit{x}\right)$ is denoted by $F^{\prime }\left(\mathit{x}\right)$ and the gradient vector field is represented as the matrix of the first derivative form. Let ${\mathit{x}}^{\left(\sigma \right)}$ and ${\mathit{x}}^{\left(\gamma \right)}$ be two vectors on the manifold of the cost function. The locations with zero values of the first derivatives on dimension $i$ could be in the range between ${\mathit{x}}^{\left(\sigma \right)}$ and ${\mathit{x}}^{\left(\gamma \right)}$ such that $F_i\left({\mathit{x}}^{\left(\sigma \right)}\right) F_i\left({\mathit{x}}^{\left(\gamma \right)}\right)<0$. If $\left|F_i\left({\mathit{x}}^{\left(\sigma \right)}\right)\right|>\left|F_i\left({\mathit{x}}^{\left(\gamma \right)}\right)\right|$, and the location with zero gradient is closer to ${\mathit{x}}^{\left(\sigma \right)}$ than ${\mathit{x}}^{\left(\gamma \right)}$. On the other hand, if $\left|F_i\left({\mathit{x}}^{\left(\sigma \right)}\right)\right|<\left|F_i\left({\mathit{x}}^{\left(\gamma \right)}\right)\right|$, the location with zero gradient is closer to ${\mathit{x}}^{\left(\gamma \right)}$ than ${\mathit{x}}^{\left(\sigma \right)}$. This concept can also be applied to any other dimensions.

In particular, for each pair of two vectors obtained from set $Q$, the output of IPM in Algorithm 2, new next-generation vectors are generated from a varying jumping distance $\delta$ related to two vectors that have different signs of gradient on each dimension, as shown in Equation (1).

$$\delta =\frac{\left|F_i\left({\mathit{x}}^{\left(\gamma \right)}\right)\right|}{\left|F_i\left({\mathit{x}}^{\left(\sigma \right)}\right)\left|+\right|F_i\left({\mathit{x}}^{\left(\gamma \right)}\right)\right|}\Vert {\mathit{x}}^{\left(\gamma \right)}-{\mathit{x}}^{\left(\sigma \right)}\Vert w$$

(1)

Suppose that ${\mathit{x}}^{\left(\sigma \right)}$ and ${\mathit{x}}^{\left(\gamma \right)}$ are selected from the output IPM with the assumption that $F_i\left({\mathit{x}}^{\left(\sigma \right)}\right)F_i\left({\mathit{x}}^{\left(\gamma \right)}\right)<0$. Then, we have two new vectors, ${\mathit{x}}_{new}^{\left(\sigma \right)}$, ${\mathit{x}}_{new}^{\left(\gamma \right)}$, whose coefficients on dimension $i$ are calculated from this jumping distance, as shown in Equations (2) and (3), while the coefficients on the other dimensions are still unchanged.

$$x_{new,i}^{\left(\sigma \right)}=x_i^{\left(\sigma \right)}+\delta$$

(2)

$$x_{new,i}^{\left(\gamma \right)}=x_i^{\left(\gamma \right)}-\delta$$

(3)

In Equation (1), a weight value, $w$, is calculated at each iteration $t$ according to Algorithm 3 to guarantee that the new vectors are generated within the pre-specified range of the search area. In essence, the weight value is in the interval between 0 and 1 and involves the jumping distance, $\delta$, such that the new vectors discovered outside the search space are discarded immediately.

Algorithm3 Generating candidate vectors ${\mathit{x}}_{new}^{\left(\sigma \right)}$, ${\mathit{x}}_{new}^{\left(\gamma \right)}$ within the pre-specified search space

$w$: weight value

NP: number of iterations

$Q$: an archive storing the current population

Set ${{\rm X}}_{cand}=\varnothing$.

$w$ =$0.05w$(NP/t)/* Calculating an updated weight value */

Compute a jumping distance $\delta$ using Equation (1).

Create two new candidate vectors ${\mathit{x}}_{new}^{\left(\sigma \right)}$, ${\mathit{x}}_{new}^{\left(\gamma \right)}$ using Equations (2) and (3).

IF {${\mathit{x}}_{new}^{\left(\sigma \right)}$ is in the range}

Insert ${\mathit{x}}_{new}^{\left(\sigma \right)}$ into ${{\rm X}}_{cand}$.

ELSE

Discard ${\mathit{x}}_{new}^{\left(\sigma \right)}$.

ENDIF

IF {${\mathit{x}}_{new}^{\left(\gamma \right)}$ is in the range}

Insert ${\mathit{x}}_{new}^{\left(\gamma \right)}$ into ${{\rm X}}_{cand}$.

ELSE

Discard ${\mathit{x}}_{new}^{\left(\gamma \right)}$.

ENDIF

RETURN ${{\rm X}}_{cand}$

Two new vectors can be generated and located approximately between the parent vectors. Accordingly, the next generation of vectors is calculated to decrease the search area. The cost function values of these new vectors must also be within an acceptable range. If some new vectors providing better cost value are attained, these new vectors will replace some vectors in the previous generation that are stored in the archive. Subsequently, for each dimension $i$, all the vectors in the archive are sorted ascendingly according to their cost values and divided into two different gradient groups, which are $G_i^{+}$, a group of vectors with a positive gradient value along dimension $i$, and $G_i^{-}$, a group of vectors with a negative gradient value along dimension $i$. For each group, a vector that contains the smallest gradient is retrieved such that these vectors are used as a pair of vectors to generate the next candidate vectors in the next generation. These steps are described in Algorithm 4.

Algorithm 4 Generating a population using the opposite gradient method (OGM)

$Q$: an archive storing the initial population obtained from Algorithm 2

NP: number of iterations

$D$: number of dimensions

Set count = 1

WHILE {count ≤ NP}

FOR {$1\le i\le D$}

Set $w$= 1

Partition the population in $Q$ into $G_i^{+}$ and $G_i^{-}$.

Select ${\mathit{x}}^{\left(\sigma \right)}$, a vector with smallest gradient from $G_i^{+}$.

Select ${\mathit{x}}^{\left(\gamma \right)}$, a vector with smallest gradient from $G_i^{-}$.

Compute vectors ${\mathit{x}}_{new}^{\left(\sigma \right)}$, ${\mathit{x}}_{new}^{\left(\gamma \right)}$ from ${\mathit{x}}^{\left(\sigma \right)}$ and ${\mathit{x}}^{\left(\gamma \right)}$ using Algorithm 3.

IF ${\mathit{x}}_{new}^{\left(\sigma \right)}\in {{\rm X}}_{cand}$ and $F\left({\mathit{x}}_{new}^{\left(\sigma \right)}\right)$ is less than the highest cost value from vectors in $Q$

Discard the vector with the highest cost value.

IF $F_i\left({\mathit{x}}_{new}^{\left(\sigma \right)}\right)>0$

Insert ${\mathit{x}}_{new}^{\left(\sigma \right)}$ in $G_i^{+}$.

ELSE

Insert ${\mathit{x}}_{new}^{\left(\sigma \right)}$ in $G_i^{-}$.

ENDIF

ENDIF

IF ${\mathit{x}}_{new}^{\left(\gamma \right)}\in {{\rm X}}_{cand}$ and $F\left({\mathit{x}}_{new}^{\left(\gamma \right)}\right)$ is less than the highest cost value from vectors in $Q$

Discard the vector with the highest cost value.

IF $F_i\left({\mathit{x}}_{new}^{\left(\gamma \right)}\right)$ < 0

Insert ${\mathit{x}}_{new}^{\left(\gamma \right)}$ in $G_i^{-}$.

ELSE

Insert ${\mathit{x}}_{new}^{\left(\gamma \right)}$ in $G_i^{+}$.

ENDIF

ENDIF

ENDFOR

$Q^{\prime }=\left({{\displaystyle \cup }}_{i=1}^D{G}_i^{+}\right)\cup \left({{\displaystyle \cup }}_{i=1}^D{G}_i^{-}\right)$

Select $n$ best points with respect to the cost values to store in $Q$.

count++

ENDWHILE

RETURN $Q$ as an archive storing the initial population for the next phase

The offspring from Algorithm 4 are filled in the archive $Q$ to store the $n$-best offspring. The offspring index in the archive can be arranged according to the fitness suitability sequence and can be used as a guide for conducting the search. The size of the archive $Q$ is not changed throughout the process. Note that the archive is updated only when the new vector obtains a solution better than the existing solution in the archive.

2.3. Combining IPM, OGM, and MVMO

The combination of IPM and OGM with MVMO is presented in this section. MVMO carries out global searching and focuses on the best solution. MVMO continuously updates an archive using compact memory with a fixed storage space. The $n$-best offspring are stored in the archive and serve as a guide toward the search direction. A solution stored in the archive is replaced by new offspring with a lower cost function value. Once the proposed combination is conducted, the vector with the lowest cost function value represents the optimal solution of the given function.

The search procedure applied to MVMO [25] is initiated with a particular set of points with $D$ dimensions, obtained from Algorithm 4 in this study. MVMO has a key feature in that a mapping function is used for modifying the offspring depending on the specific mean-variance of the solutions collected in an archive. The mean ${\overline{x}}_i$ and variance $v_i$ of dimension $i$ are calculated once the update of the archive for each dimension is made with Equations (4) and (5), where $N$ is the population size stored in the archive.

$${\overline{x}}_i=\frac{1}{N}{\displaystyle \sum }_{j=1}^N{x}_i\left(j\right)$$

(4)

$$v_i=\frac{1}{N}{\displaystyle \sum }_{j=1}^N{\left(x_i\left(j\right)-{\overline{x}}_i\right)}^2$$

(5)

The size of the search space depends on the mapping function. Moreover, the shape of the function with respect to ${\overline{x}}_i$ can be controlled by two shape variables $s_{i,1}$ and $s_{i,2}$ as follows:

$$h\left({\overline{x}}_i,s_{i,1},s_{i,2},x\right)={\overline{x}}_i\left(1-e^{-x{s}_{i,1}}\right)+\left(1-{\overline{x}}_i\right)e^{-(1-x)s_{i,2}}$$

(6)

The new coefficient of each selected dimension $x_i$ of $x$ is calculated using $h$ values in terms of $h_x,h_1, h_0$ as follows:

$$x_i=h_x+\left(1-h_1+h_0\right) x_i^r-h_0$$

(7)

where $x_i^r$ is a number obtained randomly with uniform distribution in the range of [0, 1]. It can be guaranteed that the output of Equation (7) is within the range of [–100, 100]. According to Equation (6), $h_x,h_1, h_0$ are calculated using $x=x_i^r$, $x=1$, and $x=0$, respectively.

MVMO is capable of searching the global optimum with the best mean values of the solution. Two shape variables $s_{i,1}$ and $s_{i,2}$ are originated from a variable $s_i$, calculated from a scaling factor $f_s$ and variance $v_i$ to change the shape of the function as follows:

$$s_i=-f_s\ln \left(v_i\right)$$

(8)

where $v_i$ is initially set to 1 and then recalculated once the update of the archive for each dimension is made using (5). The scaling factor $f_s$ can be used to improve the accuracy when its value is greater than 1. On the other hand, the search is coarsely conducted when the value is less than 1. The factor is initiated for the search procedure with a small value. Subsequently, the size is increased for every iteration to increase the efficiency, as denoted in Equation (9):

$$f_s=f_s^{*}\left(1+rand\right)$$

(9)

where $rand$ is randomized within [0, 1], and $f_s^{*}$ can be calculated as in Equation (10):

$$f_s^{*}=f_{s_i}^{*}+{\left(\frac{j}{j_f}\right)}^2\left(f_{s_f}^{*}-f_{s_i}^{*}\right)$$

(10)

where j and $j_f$ are the current iteration number and the last iteration number, respectively. In this study, the values of $f_{s_i}^{*}$ and $f_{s_f}^{*}$ were set to 1 and 25, respectively.

As aforementioned, the shape of the mapping function can be determined, which depends upon the shape variables $s_{i,1}$ and $s_{i,2}$ of $x_i$. To improve the efficiency of the search process, the proper shape variables can be calculated using Algorithm 5.

Algorithm 5 Calculating the shape variables $s_{i,1} \mathrm{a}\mathrm{n}\mathrm{d} s_{i,2}$ for dimension $i$ at iteration $j$

Calculate the variance $v_i$ using Equation (5).

Calculate the scaling factor $f_s$ using Equations (9) and (10).

Calculate the variable $s_i$ using Equation (8).

Set $s_{i,1}=s_i \mathrm{a}\mathrm{n}\mathrm{d} s_{i,2}=s_i$.

IF $s_i>0$ then

IF $s_i>d_{i }$

Update $d_{i }$ by multiplying with $\Delta d$.

ELSE

Update $d_{i }$ by dividing with $\Delta d$.

ENDIF

IF $d_{i }>s_i$

Set $\alpha =$ $d_i$ and $\beta =$ $s_i$.

ELSE

Set $\alpha =s_i$ and $\beta =$ $d_i$.

ENDIF

IF $rand<0.5$

Set $s_{i,1}=\alpha$ and $s_{i,2}=\beta$.

ELSE

Set $s_{i,1}=\beta$ and $s_{i,2}=\alpha$.

ENDIF

ENDIF

RETURN $s_{i,1}$, $s_{i,2}$, $d_i$

The parameter $d_{i }$ is initially set to 1 and then continuously updated at every iteration using the increment factor $\Delta d$ calculated from Equation (11).

$$\Delta d=\left(1+\Delta d_0+2·\Delta d_0·\left(rand-0.5\right)\right)$$

(11)

This factor leads to the expansion or shrinkage of the value of parameter $d_{i }$, resulting in oscillation around the current $s_i$. The optimal interval of $\mathsf{\Delta }{d}_0$ is within [0.01, 0.4].

In essence, the original MVMO encounters two problems, causing a long-duration searching process and difficulty in finding the best solution [10,26]. The first problem is zero variance when solutions stored in the archive are located at the same position. The second problem is that the value of the variance is sometimes outside of the specified range. This research attempted to overcome the limitations of MVMO and improve the flexibility of global searching over the classical MVMO by generating the population with the opposite gradient method before proceeding to MVMO. Algorithm 6 presents the determination of the $n$-best solutions using MVMO within fewer generations when using the population obtained from OGM.

Algorithm 6 Generating $n$-best offspring using MVMO

$Q$: an archive storing the initial population obtained from Algorithm 4

NP: number of iterations

$D$: number of dimensions

Set the parameters $d_i,\Delta d_0$, $f_{s_i}^{*}$, $f_{s_f}^{*}$.

WHILE {count ≤ NP}

FOR {$1 \le i \le D$}

Calculate ${\overline{x}}_i$ using Equation (4).

Apply Algorithm 5 to obtain two shape variables $s_{i,1} \mathrm{a}\mathrm{n}\mathrm{d} s_{i,2}$.

Calculate $h_x,h_1, h_0$ using Equation (6).

Generate new $x_i$ using Equation (7).

ENDFOR

Calculate the cost value of ${\mathit{x}}_{new}$.

IF $F({\mathit{x}}_{new})$ is less than the highest cost value from vectors in $Q$.

Discard the vector with the highest cost value.

Insert ${\mathit{x}}_{new}$ in $Q$.

ENDIF

ENDWHILE

3. Experiments

The IPOG-MVMO algorithm and the other algorithms, which were NBIPOPaCMA [9], PVADE [27], and MVMO, were applied to 28 benchmark functions on real-parameter optimization [28] using a computer with a Core i7 2.10 GHz CPU and 6 GB RAM for comparative purposes. All 28 benchmark functions have been described by Liang et al. [29,30]. The functions were composed of three types: unimodal (F1–F5), multimodal (F6–F20), and composition functions (F21–F28). All algorithms began running from the same set of initial points. The global optimal point of each test function was at the origin of the vector space. The error value ε can be calculated by Equation (12):

$$\epsilon =\left|f_j\left(\mathbf{0}\right)-f_j\left({\mathit{x}}^{*}\right)\right|$$

(12)

where $f_j\left(\mathbf{0}\right)$ is the cost value at the exact optimal point of the j-th benchmark function and $f_j\left({\mathit{x}}^{*}\right)$ is the cost value of the optimal point obtained by a selected algorithm. The error value is rounded to zero if it is less than 0.00000001 or $1\times {10}^{-8}$.

3.1. IPOG-MVMO Parameter Set-Up

By using IPOG-MVMO, all functions have been tested with $N=15D$ points, where $D$ is the number of dimensions. The parameters selected for IPOG-MVMO are presented in

Table 1. The parameters of the proposed algorithm experiment.

Parameters	Values
The number of dimensions (D) in each problem	$\left\{10,30,50\right\}$
Search range	${\left[-100,100\right]}^D$
$f_{s_i}^{*}$	1
$f_{s_f}^{*}$	25
$d_i$	1
$\Delta d_0$	0.05
$N$	$15D$
Size of the archive	15
NP	100,000
Number of experimental trials	50

.

The parameters used for IPOG-MVMO are presented in Table 1. The number of dimensions (D) in each problem, and the search range, were defined according to the evaluation criteria of CEC2013 benchmark problems. The number of points ($N$) varied depending on the number of dimensions, to improve the flexibility of the algorithms to deal with more difficult problems. The number of iterations (NP) and the number of experimental trials were defined to achieve reliable results. Moreover, other parameters were successfully tested in the original MVMO process.

3.2. Experimental Results

The best value, worst value, mean, and standard deviation of the error between the cost value of the solution found by an algorithm and the exact cost value were recorded over 50 trials. With these four metrics, the best value was used for the comparison and was the focus of this study. The results for 10, 30, and 50 dimensions are summarized in

Table 2. The comparative results of the errors obtained from unimodal functions F1–F5.

F1

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

0

0

0

0

0

1.91 × 10−8

0

4.30 × 10−8

0

0

0

0

PVADE

0

0

0

0

0

0

0

0

0

0

0

0

MVMO

0

0

0

0

0

0

0

0

0

0

0

0

IPOG-MVMO

0

0

0

0

0

0

0

0

0

0

0

0

F2

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

0

0

0

0

0

3.45 × 10−8

1.61 × 10−8

3.78 × 10−9

0

0

0

0

PVADE

0

6.57 × 102

1.39 × 101

1.12 × 102

1.35 × 103

4.23 × 104

9.03 × 103

7.40 × 103

3.55 × 104

4.15 × 105

2.07 × 105

9.44 × 104

MVMO

0

0

0

0

5.59 × 10−4

9.21 × 10−4

7.65 × 10−4

9.49 × 10−5

5.59 × 10−4

9.21 × 10−4

7.65 × 10−4

9.49 × 10−5

IPOG-MVMO

0

0

0

0

0

5.27 × 10−7

0

0

0

0

0

0

F3

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

0

0

0

0

0

7.42 × 10−8

2.21 × 10−8

1.68 × 10−9

1.17 × 10−5

5.01 × 100

1.41 × 100

2.19 × 100

PVADE

0

4.85 × 10−3

9.7 × 10−5

7.15 × 10−4

0

1.55 × 103

4.01 × 101

2.31 × 103

6.14 × 105

2.63 × 107

1.89 × 107

1.16 × 107

MVMO

0

0

0

0

1.00 × 103

1.11 × 107

7.95 × 105

1.81 × 106

2.88 × 10−6

1.29 × 106

6.08 × 104

2.72 × 105

IPOG-MVMO

0

0

0

0

0

1.17 × 10−6

3.65 × 10−7

4.62 × 10−7

1.75 × 10−5

6.24 × 10−4

2.11 × 10−4

2.23 × 10−4

F4

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

0

0

0

0

7.75 × 10−8

1.71 × 10−8

1.39 × 10−8

4.81 × 10−8

7.75 × 10−8

1.71 × 10−8

1.39 × 10−8

4.81 × 10−8

PVADE

0

2.12 × 10−1

4.21 × 10−3

3.13 × 10−2

5.53 × 10−7

1.33 × 10−3

1.93 × 10−4

3.31 × 10−4

5.53 × 10−7

1.33 × 10−3

1.93 × 10−4

3.31 × 10−4

MVMO

0

1.18 × 10−8

0

0

5.37 × 10−6

3.91 × 10−3

4.62 × 10−4

7.56 × 10−4

5.37 × 10−6

3.91 × 10−3

4.62 × 10−4

7.56 × 10−4

IPOG-MVMO

0

0

0

0

0

1.42 ×10−6

3.65 ×10−7

5.64 ×10−7

0

1.42 ×10−6

3.65 ×10−7

5.64 ×10−7

F5

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

0

1.15 × 10−8

0

0

0

3.34 × 10−8

1.39 × 10−8

6.18 × 10−8

0

0

0

0

PVADE

0

8.13 × 10−5

1.26 × 10−5

1.11 × 10−4

0

3.39 x10−8

1.02 × 10−8

1.14 × 10−8

1.45 × 10−4

1.6 × 10−4

1.02 × 10−3

1.11 × 10−3

MVMO

0

4.19 × 10−8

0

0

1.22 × 10−8

3.01 × 10−8

2.34 × 10−8

2.12 × 10−8

0

0

0

0

IPOG-MVMO

0

0

0

0

0

2.79 × 10−8

1.02 ×10−8

1.46 ×10−8

0

0

0

0

,

Table 3. The comparative results of the errors obtained from multimodal functions F6–F9.

F6

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

0

0

0

0

0

2.13 × 10−8

0

3.96 × 10−9

0

0

0

0

PVADE

0

7.35 × 100

7.81 × 100

3.89 × 100

0

2.67 × 101

4.96 × 10−1

3.73 × 100

2.57 × 101

1.48 × 102

8.52 × 101

3.15 × 101

MVMO

0

0

0

0

0

3.91 × 101

1.04 × 101

9.71 × 100

0

4.34 × 101

3.05 × 101

2.01 × 101

IPOG-MVMO

0

0

0

0

0

2.81 × 10−8

0

8.11 × 10−9

0

5.76 × 100

1.21 × 100

2.26 × 100

F7

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

0

1.62 × 101

1.40 × 102

4.80 × 102

5.67 × 10−3

5.69 × 101

7.01 × 100

1.71 × 101

3.05 × 10−3

1.83 × 101

3.94 × 100

8.05 × 100

PVADE

3.11 × 10−5

1.10 × 101

3.02 × 10−1

1.53 × 100

4.21 × 10−1

2.64 × 101

4.60 × 100

5.75 × 100

3.92 × 104

4.41 × 101

1.89 × 101

9.26 × 100

MVMO

6.33 × 10−3

6.34 × 10−3

6.13 × 10−3

0

2.48 × 100

1.92 × 101

8.17 × 100

3.62 × 100

1.52 × 101

4.91 × 101

3.04 × 101

7.87 × 100

IPOG-MVMO

0

0

0

0

4.22 × 10−5

4.62 × 10−4

2.11 × 10−4

2.22 × 10−4

1.32 × 10−5

4.81 × 10−4

2.21 × 10−4

2.15 × 10−4

F8

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

3.49 × 10−8

1.68 × 10−8

1.04 × 10−8

8.57 × 10−9

2.08 × 101

2.19 × 101

2.13 × 101

2.85 × 101

7.77 × 10−1

2.11 × 101

5.87 × 100

9.20 × 100

PVADE

1.92 × 101

2.01 × 101

2.01 × 101

6.11 × 10−2

2.04 × 101

2.14 × 101

2.61 × 101

4.08 × 101

2.10 × 101

2.12 × 101

2.11 × 101

3.49 × 10-2

MVMO

1.94 × 101

2.11 × 101

1.98 × 101

6.54 × 10−2

2.08 × 101

2.10 × 101

2.19 × 101

5.41 × 102

2.10 x101

2.12 × 101

2.11 × 101

3.94 × 10-2

IPOG-MVMO

0

6.12 × 10−8

1.54 × 10−7

1.02 × 10−7

2.08 × 101

2.10 × 101

2.09 × 101

2.37 × 102

8.35 × 102

2.16 × 101

1.96 × 101

4.34 × 100

F9

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

1.92 × 10−8

2.47 × 101

3.82 × 100

9.56 × 10−9

2.02 x100

1.78 × 101

4.30 × 100

4.81 × 100

0

7.41 × 101

1.50 × 101

3.30 × 100

PVADE

8.11 × 10−3

3.36 × 100

1.34 × 100

9.67 × 10−1

2.27 × 101

3.09 × 101

2.74 × 101

1.77 × 100

1.99 × 101

3.34 × 101

2.60 × 101

3.05 × 100

MVMO

4.92 × 10−1

2.45 × 100

8.26 × 10−1

6.66 × 10−1

7.11 × 100

1.92 × 101

1.32 × 101

2.65 × 100

2.48 × 101

4.17 × 101

3.33 × 101

4.39 × 100

IPOG-MVMO

0

9.45 × 10−8

6.95 × 10−8

4.45 × 10−8

7.54 × 10−7

1.19 × 10−6

9.57 × 10−7

2.19 × 10−7

4.82 × 10−2

6.73 × 100

1.64 × 100

2.57 × 1000

,

Table 4. The comparative results of the errors obtained from multimodal functions F10–F13.

F10

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

1.02 × 10−8

6.71 × 10−8

3.24 × 10−8

1.84 × 10−9

1.01 × 10−8

3.55 × 10−8

1.94 × 10−8

8.32 × 10−8

0

4.69 × 10−2

9.37 × 10−3

2.09 × 10−2

PVADE

0

1.79 × 10−1

5.01 × 10−2

3.79 × 10−2

1.73 × 10−2

1.89 × 10−1

7.64 × 10−2

3.53 × 10−2

4.77 × 10−2

1.15 × 100

5.99 × 10−1

3.42 × 10−1

MVMO

9.95 × 10−3

3.58 × 10−2

1.58 × 10−2

2.11 × 10−2

7.40 × 10−3

7.35 × 10−2

2.78 × 10−2

1.64 × 10−2

0

0

0

0

IPOG-MVMO

0

3.87 × 10−8

2.89 × 10−8

7.34 × 10−9

0

4.01 × 10−8

1.42 × 10−8

1.78 × 10−8

0

6.66 × 101

2.78 × 10−3

3.91 × 10−2

F11

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

0

6.72 × 10−8

3.27 × 10−8

2.11 × 10−9

9.95 × 10−1

3.98 × 100

2.70 x100

1.25 × 100

7.37 × 10−2

1.89 × 101

3.94 × 100

8.36 × 100

PVADE

0

1.21 × 101

4.04 × 100

2.33 × 100

1.03 × 100

7.34 × 100

3.13 × 100

1.49 × 100

6.76 × 101

2.34 × 102

1.68 × 102

4.08 × 101

MVMO

0

6.02 × 100

2.32 × 100

1.29 × 10−2

1.01 × 100

8.05 × 100

4.34 × 100

1.90 × 100

2.59 × 101

7.57 × 101

4.55 × 101

1.22 × 101

IPOG-MVMO

0

1.13 × 10−8

4.87 × 10−8

3.13 × 10−9

0

1.03 × 10−7

2.13 × 10−8

4.03 × 10−8

1.10 × 10−3

4.90 × 10−2

1.21 × 10−2

1.81 × 10−2

F12

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

0

1.03 × 100

3.43 × 10−1

1.89 × 10−9

6.70 × 10−8

3.98 × 100

2.27 × 100

1.70 × 100

7.37 × 10−2

1.02 × 100

3.30 × 10−1

3.96 × 10−1

PVADE

9.87 × 10−1

1.59 × 101

5.86 × 100

3.68 × 100

1.59 × 100

3.24 × 101

2.13 × 101

3.79 × 100

2.20 × 102

3.12 × 102

2.56 × 102

2.01 × 101

MVMO

1.89 × 100

1.55 × 101

5.89 × 100

1.32 × 100

1.59 × 101

5.97 × 101

3.36 × 101

1.14 × 101

3.98 × 101

1.34 × 102

7.77 × 101

2.33 × 101

IPOG-MVMO

0

3.49 × 10−5

4.78 × 10−8

7.56 × 10−9

0

2.72 × 10−8

8.87 × 10−8

1.21 × 10−8

7.74 × 10−5

2.45 × 10−3

9.99 × 104

1.05 × 10−3

F13

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

0

4.54 × 100

7.57 × 10−1

1.45 × 100

0

3.07 × 100

2.77 × 100

1.36 × 100

6.46 × 10−3

1.48 × 101

3.10 × 100

6.54 × 100

PVADE

0

2.08 × 101

8.35 × 100

4.34 × 100

7.60 × 101

1.56 × 102

1.21 × 102

1.86 × 101

8.10 × 101

1.62 × 102

1.18 × 102

1.69 × 101

MVMO

1.87 × 100

1.82 × 101

8.84 × 100

9.01 × 100

3.06 x101

9.95 × 101

5.90 × 101

1.67 × 101

7.20 × 101

2.05 × 102

1.33 × 102

3.43 × 101

IPOG-MVMO

0

3.58 × 10−5

1.08 × 10−5

1.01 × 10−5

0

1.79 × 10−6

3.23 × 10−7

6.07 × 10−7

6.43 × 10−4

6.30 × 100

1.54 × 100

2.59 × 100

,

Table 5. The comparative results of the errors obtained from multimodal functions F14–F17.

F14

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

2.13 × 101

7.68 × 102

3.44 × 102

2.11 × 102

7.12 × 102

1.35 × 103

8.70 × 102

3.40 × 102

3.16 × 101

4.59 × 101

3.96 × 101

5.94 × 100

PVADE

4.01 × 101

5.13 × 102

1.67 × 102

1.07 × 102

2.04 × 103

4.01 × 103

3.11 × 103

4.70 × 102

2.14 × 103

3.90 × 103

3.07 × 103

3.80 × 102

MVMO

3.21 × 100

2.08 × 101

8.01 × 100

7.59 × 100

9.40 × 101

1.66 × 103

8.56 × 102

4.12 × 102

2.19 × 103

4.84 × 103

3.89 × 103

6.02 × 102

IPOG-MVMO

2.03 × 10−5

2.67 × 10−5

2.39 × 10−5

4.78 × 10−6

2.37 × 101

6.35 × 102

3.09 × 102

3.11 × 102

9.23 × 10−5

6.10 × 101

5.05 × 101

1.70 × 100

F15

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

6.87 × 100

2.38 × 102

1.17 × 102

8.89 × 101

5.53 × 102

1.67 × 103

7.65 × 102

2.85 × 102

1.27 × 10−1

9.68 × 101

6.51 × 101

3.94 × 101

PVADE

3.71 × 102

1.02 × 103

7.84 × 102

1.68 × 102

2.13 × 103

4.94 × 103

3.32 × 103

4.08 × 102

4.54 × 103

6.17 × 103

5.36 × 103

3.54 × 102

MVMO

1.89 × 102

6.57 × 102

5.58 × 102

9.01 × 101

1.70 × 103

4.45 × 103

3.03 × 103

5.41 × 102

5.75 x103

8.65 × 103

6.64 × 103

5.92 × 102

IPOG-MVMO

2.58 × 10−5

7.28 × 10−2

3.21 × 10−2

3.13 × 10−2

2.19 × 101

1.22 × 103

2.28 × 102

2.37 × 102

3.69 × 10−3

6.67 × 101

2.03 × 101

2.76 × 101

F16

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

0

1.97 × 101

5.68 × 100

7.23 × 100

5.68 x10−2

1.23 × 100

9.14 × 10−1

1.86 × 10−1

1.18 x10−3

3.51 × 100

1.20 × 100

1.65 × 100

PVADE

4.75 × 10−1

1.26 × 100

8.93 × 10−1

1.88 × 10−1

1.42 × 100

3.13 × 100

2.32 × 100

3.01 × 10−1

2.67 × 100

3.84 × 100

3.38 × 100

2.86 × 10−1

MVMO

3.52 × 10−1

6.49 × 10−1

5.39 × 10−1

1.62 × 10−1

5.37 × 10−1

1.62 × 100

1.08 × 100

2.88 × 10−1

6.60 × 10−1

1.70 × 100

1.14 × 100

2.37 × 10−1

IPOG-MVMO

0

7.64 × 10−1

1.63 × 10−1

3.18 × 10−1

8.08 × 10−2

1.34 × 100

5.40 × 100

1.09 × 101

3.12 × 10−1

3.25 × 100

2.11 × 100

2.24 × 10−1

F17

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

0

9.71 × 101

1.89 × 101

2.64 × 101

3.41 × 101

4.26 × 101

3.53 × 101

1.87 × 100

2.10 × 100

5.55 × 101

1.30 × 101

2.37 × 101

PVADE

1.15 × 101

1.75 × 101

1.39 × 101

1.39 × 101

7.44 × 101

1.17 × 102

9.52 × 101

1.10 × 101

1.44 × 102

3.17 × 102

2.37 × 102

1.10 × 101

MVMO

1.03 × 101

1.18 × 101

1.11 × 101

7.65 × 10−1

4.38 × 101

8.14 × 101

6.72 × 101

8.50 × 100

8.89 × 101

1.43 × 102

1.11 × 102

1.34 × 101

IPOG-MVMO

0

3.07 × 101

9.85 × 100

8.85 × 100

3.25 × 101

6.60 × 101

4.31 × 101

4.89 × 100

4.50 × 100

6.66 × 101

3.48 × 101

3.30 × 101

,

Table 6. The comparative results of the errors obtained from composition functions F18–F20.

F18

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

0

5.74 × 101

1.58 × 101

1.51 × 101

3.86 × 101

1.72 × 102

6.38 × 101

4.45 × 101

8.60 × 10−2

9.86 × 101

2.54 × 101

4.13 × 101

PVADE

1.13 × 101

3.26 × 101

2.40 × 101

3.90 × 100

1.40 × 102

1.89 × 102

1.66 × 102

1.12 × 101

3.43 × 102

4.22 × 102

3.65 × 102

1.12 × 101

MVMO

1.51 × 101

2.01 × 101

1.72 × 101

2.58 × 100

4.58 × 101

7.88 × 101

5.94 × 101

7.59 × 100

8.06 × 101

1.63 × 102

1.07 × 102

1.73 × 101

IPOG-MVMO

0

1.01 × 10−1

3.27 × 10−2

3.71 × 10−2

1.88 × 101

3.38 × 101

2.94 × 101

9.87 × 100

1.20 × 10−2

4.18 × 100

1.34 × 100

1.92 × 100

F19

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

0

7.38 × 10−1

4.34 × 10−1

2.01 × 10−1

1.14 × 100

2.54 × 100

2.40 × 100

4.41 × 10−1

1.60 × 10−1

2.85 × 100

8.65 × 10−1

1.16 × 100

PVADE

3.27 × 10−1

9.69 × 10−1

5.98 × 10−1

1.25 × 10−1

3.10 × 100

1.06 × 101

6.49 × 100

1.74 × 100

1.06 × 101

3.10 × 101

2.12 × 101

4.74 × 100

MVMO

3.52 × 10−1

6.07 × 10−1

5.19 × 10−1

1.44 × 10−1

1.10 × 100

3.13 × 100

1.93 × 100

4.09 × 10−1

2.79 × 100

7.55 × 100

4.82 × 100

1.24 × 100

IPOG-MVMO

0

1.90 × 10−1

2.59 × 10−2

6.02 × 10−2

3.90 × 10−1

2.88 × 100

1.06 × 100

4.73 × 10−1

1.52 × 10−1

3.28 × 100

1.53 × 100

1.52 × 100

F20

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

2.11 × 100

4.85 × 100

3.26 × 100

9.87 × 10−1

1.01 × 101

1.56 × 101

1.29 × 101

5.98 × 10−1

7.85 × 10−1

2.53 × 101

7.01 × 100

1.05 × 100

PVADE

9.96 × 10−1

3.52 × 100

2.12 × 100

5.61 × 10−1

1.04 × 101

1.50 × 101

1.34 × 101

1.89 × 10−1

1.95 × 101

2.55 × 101

2.36 × 101

1.90 × 100

MVMO

1.83 × 100

2.78 × 100

2.34 × 100

4.76 × 10−1

8.81 × 100

1.19 × 101

1.04 × 101

5.85 × 10−1

1.87 × 101

2.23 × 101

2.01 × 101

6.84 × 10−1

IPOG-MVMO

1.12 × 100

2.74 × 100

1.95 × 100

4.01 × 10−1

6.84 × 100

1.28 × 101

1.01 × 101

6.23 × 10−1

2.10 × 100

1.78 × 101

2.30 × 100

4.57 × 10−1

,

Table 7. The comparative results of the errors obtained from composition functions F21–F24.

F21

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

3.64 × 100

1.42 × 101

1.05 × 101

4.47 × 100

2.00 × 102

2.00 × 102

2.00 × 102

0

1.00 × 102

2.00 × 102

1.91 × 102

1.43 × 101

PVADE

2.53 × 102

3.81 × 102

2.90 × 102

4.06 × 101

2.00 × 102

4.43 × 102

3.17 × 102

6.22 × 101

8.36 × 102

1.22 × 103

9.56 × 102

1.44 × 102

MVMO

3.35 × 10−5

3.90 × 102

2.26 × 102

9.73 × 101

2.00 × 102

4.43 × 102

2.95 × 102

1.07 × 102

1.00 × 102

1.13 × 103

2.45 × 102

1.91 × 102

IPOG-MVMO

2.68 × 100

1.50 × 101

1.22 × 101

4.11 × 100

2.00 × 102

4.45 × 102

2.57 × 102

1.15 × 101

1.00 × 102

5.24 × 102

2.29 × 102

3.48 × 101

F22

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

3.31 × 100

1.00 × 101

8.81 × 100

3.05 × 100

3.54 × 102

1.08 × 103

8.03 × 102

3.12 × 102

1.89 × 102

3.86 × 103

1.45 × 103

6.01 × 102

PVADE

2.34 × 101

5.07 × 102

2.57 × 102

1.11 × 102

1.72 × 103

3.38 × 103

2.49 × 103

3.86 × 102

6.11 × 103

1.01 × 104

7.72 × 103

8.44 × 102

MVMO

5.02 × 100

1.00 × 102

3.27 × 101

1.91 × 101

2.35 × 102

2.00 × 103

8.19 × 102

4.28 × 102

1.17 × 103

4.94 × 103

2.76 × 103

8.38 × 102

IPOG-MVMO

3.00 × 100

1.76 × 101

6.20 × 100

2.48 × 102

2.00 × 102

2.83 × 103

6.26 × 102

2.55 × 102

3.92 × 102

4.68 × 103

2.35 × 103

6.94 × 102

F23

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

3.59 × 100

4.14 × 100

3.96 × 100

2.01 × 101

3.52 × 102

3.59 × 103

1.29 × 103

1.35 × 103

1.37 × 103

1.31 × 104

4.29 × 103

5.00 × 103

PVADE

1.18 × 101

1.12 × 102

9.14 × 101

1.78 × 101

4.74 × 103

7.25 × 103

5.81 × 103

5.04 × 102

7.91 × 103

1.54 × 104

1.17 × 104

1.48 × 103

MVMO

5.63 × 101

8.71 × 102

4.95 × 102

2.03 × 102

1.69 × 103

4.45 × 103

3.09 × 103

5.28 × 102

6.17 × 103

1.12 × 104

8.64 × 103

1.32 × 103

IPOG-MVMO

3.54 × 100

3.14 × 100

3.89 × 100

3.01 × 10−1

4.89 × 102

4.28 × 103

2.79 × 103

9.41 × 102

4.97 × 102

7.29 × 103

5.79 × 103

9.41 × 102

F24

Algorithms

$D$ = 10

$D$ = 30

$D$ = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

3.59 × 100

3.70 × 100

3.74 × 100

5.01 × 10−2

1.39 × 102

3.01 × 102

2.42 × 102

8.00 × 101

2.35 × 102

3.86 × 102

3.27 × 102

7.60 × 101

PVADE

1.01 × 102

2.11 × 102

1.94 × 102

1.30 × 101

2.01 × 102

2.61 × 102

2.03 × 102

1.39 × 100

2.40 × 102

3.28 × 102

2.78 × 102

1.83 × 101

MVMO

1.01 × 102

2.07 × 102

1.83 × 102

3.74 × 101

2.04 × 102

2.20 × 102

2.11 × 102

3.77 × 100

2.28 × 102

2.61 × 102

2.45 × 102

8.05 × 100

IPOG-MVMO

9.85 × 10−1

4.42 × 100

2.07 × 100

1.44 × 10−2

1.05 × 102

4.62 × 102

2.17 × 102

1.46 × 100

1.05 × 102

4.62 × 102

2.17 × 102

1.46 × 101

and

Table 8. The comparative results of the errors obtained from composition functions F25–F28.

F25

Algorithms

D = 10

D = 30

D = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

7.31 × 100

7.49 × 100

7.44 × 100

5.01 × 10−2

2.05 × 102

3.01 × 102

2.66 × 102

4.26 × 101

2.34 × 102

3.86 × 102

3.23 × 102

7.86 × 101

PVADE

1.90 × 102

2.13 × 102

2.04 × 102

3.77 × 100

2.00 × 102

2.56 × 102

2.30 × 102

2.04 × 101

3.17 × 102

3.92 × 102

3.54 × 102

1.72 × 101

MVMO

1.01 × 102

2.01 × 102

1.94 × 102

2.25 × 101

2.00 × 102

2.71 × 102

2.51 × 102

9.39 × 100

3.00 × 102

3.50 × 102

3.26 × 102

1.13 × 101

IPOG-MVMO

7.09 × 100

7.57 × 100

7.37 × 100

4.88 × 10−2

2.09 × 102

2.75 × 102

2.55 × 102

2.84 × 101

2.36 × 102

2.77 × 102

2.57 × 102

8.88 × 101

F26

Algorithms

D = 10

D = 30

D = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

4.04 × 100

4.29 × 100

4.14 × 100

1.41 × 10−1

1.55 × 102

3.26 × 102

2.42 × 102

7.92 × 101

1.95 × 102

4.84 × 102

3.43 × 102

1.42 × 102

PVADE

1.04 × 102

2.00 × 102

1.84 × 102

3.33 × 101

2.00 × 102

3.11 × 102

2.18 × 102

4.01 × 101

2.00 × 102

3.96 × 102

3.47 × 102

6.01 × 101

MVMO

1.02 × 102

1.15 × 102

1.08 × 102

3.52 × 100

2.00 × 102

2.00 × 102

2.00 × 102

4.47 × 10-3

2.00 × 102

2.00 × 102

2.00 × 102

3.21 × 10−3

IPOG-MVMO

3.94 × 100

4.46 × 100

4.12 × 100

1.29 × 10−1

1.89 × 102

3.11 × 102

2.12 × 102

2.29 × 101

1.94 × 102

2.96 × 102

2.12 × 102

2.29 × 101

F27

Algorithms

D = 10

D = 30

D = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

1.13 × 101

1.26 × 101

1.22 × 101

5.70 × 10−1

4.00 × 102

5.85 × 102

4.75 × 102

6.75 × 101

4.00 × 102

2.15 × 103

1.48 × 103

9.01 × 102

PVADE

2.95 × 102

5.35 × 102

3.64 × 102

8.98 × 101

3.06 × 102

3.61 × 102

3.26 × 102

1.14 × 101

8.27 × 102

1.36 × 103

1.11 × 103

1.85 × 102

MVMO

2.48 × 102

2.88 × 102

2.60 × 102

4.03 × 100

3.43 × 102

6.89 × 102

4.74 × 102

9.19 × 101

9.09 × 102

1.31 × 103

1.08 × 103

1.10 × 102

IPOG-MVMO

5.75 × 100

1.11 × 101

7.34 × 100

2.45 × 10−1

3.95 × 102

5.17 × 102

4.34 × 102

2.45 × 101

5.95 × 102

1.17 × 103

7.34 × 102

2.45 × 102

F28

Algorithms

D = 10

D = 30

D = 50

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

Best

Worst

Mean

Std.

NBIPOPaCMA

3.60 × 100

1.11 × 101

9.63 × 100

3.31 × 100

3.00 × 102

3.00 × 102

3.00 × 102

0

4.00 × 102

4.00 × 102

4.00 × 102

0

PVADE

8.45 × 101

3.01 × 102

2.38 × 102

4.52 × 101

3.00 × 102

3.00 × 102

3.00 × 102

2.23 × 10−5

4.00 × 102

3.54 × 105

4.64 × 102

4.39 × 102

MVMO

9.87 × 101

2.90 × 102

1.88 × 102

1.00 x102

3.00 × 102

3.00 × 102

3.00 x102

5.36 × 10−4

4.00 × 102

4.00 × 102

4.00 × 102

1.29 × 10−2

IPOG-MVMO

3.00 × 100

8.72 × 100

4.49 × 100

2.37 × 10−1

3.00 × 102

3.00 × 102

3.00 × 102

2.37 × 10−5

4.00 × 102

4.00 × 102

4.00 × 102

1.13 × 10−2

, respectively. To clearly illustrate the comparison, only the mean of the error was selected and plotted, as is shown in Figure 1, Figure 2 and Figure 3.

To solve unimodal functions F1–F5, the proposed algorithm was able to search the solution without error in every case except F3 ($D$ = 50), where NBIPOPaCMA yielded the better solution, as is shown in Table 2. Moreover, when considering the mean of the error, IPOG-MVMO yielded a zero mean in 10 out of 15 cases. In other words, in these 10 cases, IPOG-MVMO guaranteed the best solution for every trial in every case. It seems that IPOG-MVMO is a suitable technique for solving unimodal problems.

The multimodal functions F6–F20 were analyzed in three groups based on their complexity. To solve multimodal functions in the $D$ = 10 group with the lowest complexity, IPOG-MVMO attained results with no error, except for F14, F15, and F20. Nevertheless, these results were still satisfactory when compared with other algorithms. For the $D$ = 30 group, IPOG-MVMO performed worse than the other algorithms only in two cases, which were F8 and F16. Moreover, IPOG-MVMO also attained error-free results in F6 and F10–F13. For the $D$ = 50 group with the highest complexity, IPOG-MVMO yielded lower performance than the other algorithms in only five cases, which were F8, F9, F16, F17, and F20.

To solve the composition functions F21–F28, the proposed algorithm also outperformed the other algorithms in all cases of $D$ = 10. The algorithm was able to attain better results than the other algorithms in F22 and F24 for $D$ = 30, and in F23, F24, and F26 for $D$ = 50, respectively. Note that this type of function could not provide zero errors in all cases due to its complexity.

The superior results for each problem are highlighted in bold where the best value of IPOG-MVMO was obtained and was better than the other methods; although sometimes, more than one method provided the best value. Clearly, among the 28 × 3 cases, there were 68 cases in which IPOG-MVMO yielded the best result. Likewise, when considering the mean value, IPOG-MVMO yielded the best result in 59 out of the 28 × 3 cases. The results indicated that applying IPM and OGM to MVMO provided a solution that was closer to the optimal solution.

Some examples for the performance comparison were selected, as presented in Figure 4. We also present the average error with respect to the number of iterations (NP) in Figure 4. The average error was computed over 50 trials. The results showed that applying IPM and OGM to MVMO can accelerate the search process to achieve a solution that was closer to the optimal solution within a smaller number of iterations.

4. Discussion

4.1. Optimal Solutions

Twenty-eight different functions of three types were taken to verify our proposed method. The experimental results in terms of error are shown in Table 2, Table 3, Table 4, Table 5, Table 6, Table 7 and Table 8. When considering the best solution out of all the methods, it was found that IPOG-MVMO achieved the best result in 68 cases out of 84 cases, or 80.95%. Next, three types of functions were further analyzed separately. For the unimodal functions F1–F5, the proposed method offered the best result in 14 out of 15 cases, or 93.33%. This is quite acceptable when applying our method to this type of function. In the case of the multimodal functions F6–F20, there were 45 cases in total. Among these cases, IPOG-MVMO yielded the best result in 37 cases, or 82.22%. This was slightly lower than for the unimodal function due to the complexity of the problem. The remaining functions were the composition functions, F21–F28. The proposed method yielded the best result in 17 out of 24 cases, or 70.83%, which was lower than for unimodal and multimodal functions. Subsequently, the results can be discussed from a different perspective, where we mainly focused on the number of dimensions. There were 28 cases for each dimension and our method yielded the best result in 27, 22, and 19 cases, or 96.43%, 78.57%, and 67.86%, respectively.

In essence, zero error indicates whether the algorithm can reach the exact solution within the pre-defined number of iterations. IPOG-MVMO provided zero error in 33 out of 84 cases.

Table 9. Likelihood of obtaining zero error in 50 trials.

Algorithms	Unimodal Functions			Multimodal Functions			Composition Functions
	$\mathit{D}$	$\mathit{D}$	$\mathit{D}$	$\mathit{D}$	$\mathit{D}$	$\mathit{D}$	$\mathit{D}$	$\mathit{D}$	$\mathit{D}$
NBIPOPaCMA	100	80	60	33.33	13.33	20	0	0	0
PVADE	100	60	20	26.67	6.67	0	0	0	0
MVMO	100	20	40	13.33	6.67	13.33	0	0	0
YYPO	40	40	20	6.67	6.67	6.67	0	0	0
IPOG-MVMO	100	100	80	80	33.33	13.33	0	0	0

presents the likelihood that IPOG-MVMO and four comparative methods could reach the exact solution in terms of function type and the number of dimensions. YYPO, a method inspired by the philosophy of creating a balance between two concepts, was also conducted and is included in this table. For unimodal functions, there was only one case at $D$ = 50 in which the error was non-zero, whereas the percentage dropped when the number of dimensions was higher in multimodal functions. Moreover, NBIPOPaCMA presented a higher likelihood of obtaining zero error at $D$ = 50; however, IPOG-MVMO still provided the best results at $D$ = 10 and $D$ = 30. For the composition function, the best solution over 50 trials did not reach the exact solution for any algorithm. Since the function type and the number of dimensions corresponded to the problem difficulty and complexity, it can be inferred that although our IPOG-MVMO method outperformed the other comparative methods, the performance gradually decreased as the problem difficulty and complexity rose. However, using different parameter settings might result in greater performance for solving black box problems.

When considering the mean value of error in Table 2, Table 3, Table 4, Table 5, Table 6, Table 7 and Table 8, it was found that IPOG-MVMO yielded the best result in 59 out of 84 cases, or 70.24%. Typically, the zero mean rarely occurs because the method must reach the exact solution in every trial. Our proposed method achieved zero mean in 13 cases, although most were unimodal functions. According to the analysis of the function type, IPOG-MVMO yielded the best result in 12 cases, or 80%, for the unimodal functions, 34 cases, or 75.56%, for the multimodal functions, and 13 cases, or 54.17%, for the composition functions. It can be seen that the type of function also affected the performance in terms of the mean value. In the case of the number of dimensions, IPOG-MVMO yielded the best result in 25 cases, or 89.29%, 18 cases, or 64.29%, and 16 cases, or 57.14%, in 10, 30, and 50 dimensions, respectively. The number of dimensions was another factor that impacted our algorithm.

Finally, the worst value of error presented a solution to be expected over a specific number of trials. IPOG-MVMO achieved the best result in 48 cases, or 57.14%, which was higher than the other methods. Again, the method still worked well in 11 cases, or 73.33%, of unimodal functions, while it yielded the best result in 29 cases, or 64.44%, and eight cases, or 33.33%, in multimodal and composition functions, respectively. In terms of the number of dimensions, our method achieved the best result in 20 out of 28 cases in 10 dimensions. For the cases with 30 dimensions, our method yielded the best result in 12 cases, or 42.86%, and the percentage became higher when the number of dimensions was 50, with a 57.14% rate of obtaining the best result. More parameter settings and experiments are required to identify the relationship between the number of dimensions and the optimal solutions in the worst-case scenario.

4.2. Stability

The standard deviation of error can be used to measure the stability of the proposed method compared with the other methods. Zero standard deviation means that the method can provide the same solution over a limited number of trials. Under the experiments, IPOG-MVMO obtained the best result in 46 out of 84 cases, or 54.76%, and there were 12 cases of zero standard deviation, 10 of which were uniform functions. Based on the function type, the unimodal function was the type with the highest stability, and the best result was obtained in 11 out of 15 cases, whereas the multimodal functions yielded the best result in 26 out of 45 cases. Unfortunately, the composition functions method yielded the best result only in nine out of 24 cases, which was greater than the other methods, but most of the cases were in 10 dimensions. For the number of dimensions, the method yielded the best result in 20 cases (71.43%), 11 cases (39.29%), and 15 cases (53.57%), in 10, 30, and 50 dimensions, respectively. Without taking the composition functions into consideration, it seems that the number of dimensions had a minor impact on the stability of the method, because, as previously mentioned, the composition method worked well only in 10 dimensions.

5. Conclusions

In this study, IPOG-MVMO, a combination of the interior point method (IPM), the opposite gradient method (OGM), and mean-variance mapping optimization (MVMO), was proposed to identify a solution for the continuous real-world optimization problem. IPM and OGM were combined with the original MVMO in the area of an IPM local search and then a new population was created using the opposite gradient concept. This procedure ensured that the newly created population was located somewhere close to the minimum value so that a better solution could be obtained using MVMO. For experiments, creating a new population close to the minimum value to obtain an accurate and fast solution by using the local search strength of IPM and OGM’s solution convergence was a key step in this study. When these two approaches were combined with traditional MVMO, the best cost was achieved, which was better than using a single technique, i.e., MVMO. It was important to generate a new population near the minimum in the optimization problem, which can be applied to optimization problems in many areas, such as energy saving for tissue paper mills through energy efficiency scheduling, and the optimal scheduling of energy and ancillary services. Future research will focus on the hybridization of IPOG-MVMO and other techniques, and improving the algorithms to ensure compatibility with combinatorial real-world problems.