Dynamic Shannon Performance in a Multiobjective Particle Swarm Optimization

Abstract

Particle swarm optimization (PSO) is a search algorithm inspired by the collective behavior of flocking birds and fishes. This algorithm is widely adopted for solving optimization problems involving one objective. The evaluation of the PSO progress is usually measured by the fitness of the best particle and the average fitness of the particles. When several objectives are considered, the PSO may incorporate distinct strategies to preserve nondominated solutions along the iterations. The performance of the multiobjective PSO (MOPSO) is usually evaluated by considering the resulting swarm at the end of the algorithm. In this paper, two indices based on the Shannon entropy are presented, to study the swarm dynamic evolution during the MOPSO execution. The results show that both indices are useful for analyzing the diversity and convergence of multiobjective algorithms.

1. Introduction

Multiobjective optimization (MOO) consists either in minimizing or in maximizing a set of objective functions subject to some constraints. In these problems, the objective functions are conflicting, leading to several vectors of decision variables. Each vector represents a possible solution that solves the problem with different trade-offs among the design objectives. Evolutionary and social-based algorithms have attracted the attention of many researchers, because they are frequently superior to conventional mathematical techniques due to their stochastic proprieties [1].

The MOO is inspired by biological phenomena and adopts a population that evolves during several generations. The PSO nature metaphor mimics the behavior of birds flocking or fish schooling [2]. Each bird or fish is represented by a particle with two components, namely by its position and velocity. A set of particles forms the swarm that evolves during several iterations giving rise to a powerful optimization method.

The particle swarm optimization’s (PSO) simplicity and success led to its application in problems where more than one optimization criterion is considered. Many techniques, such as those borrowed from genetic algorithms (GA) [3,4], have been developed to find a set of solutions belonging to the Pareto front. Since the multiobjective PSO (MOPSO) proposal [5], the algorithm was used in a wide range of applications [6,7], and a considerable number of variants of refined MOPSO were developed in order to improve its performance [8,9].

The performance of multiobjective algorithms is usually analyzed at the end of their execution, and its success is measured by means of several metrics proposed in the literature [10]. Additionally, in ambiguous situations, the use of nonparametric tests can be adopted [11,12]. Some of the proposed indices are based on Shannon entropy. Wang et al. [13] presented a method revealing interesting results: (i) the computational effort increases linearly with the number of solutions, (ii) the metric qualifies the combination of uniformity and coverage of the Pareto set, and (iii) it determines when the evolution has reached its maturity. LinLin and Yunfang [14] proposed a technique to measure the performance of multiobjective problems that not only indicates when the algorithm should be stopped, but can also compare the performance of multiobjective algorithms. The technique adopts entropy that is evaluated regarding the solution density in a gridded space.

Other studies tried to unravel the population dynamics during time evolution [15,16,17,18,19,20]. Farhang-Mehr and Azarm [15] developed an entropy-based metric to assess the diversity in a MOEA during the run time. To measure the entropy, a grid of cells was used, where solutions belonging to the same cell are considered identical. They orthogonalize and project them into a plane to count 3D nondominated solutions. Deb and Jain [16] proposed two multiobjective running metrics, one for measuring the convergence and the other for assessing the diversity among solutions. Myers and Hancock [17] suggested the use of the Shannon entropy to evaluate the run-time performance of a GA to solve labeling problems. The entropy measured in the parameter space is used to provide useful information about the algorithm state. Myers and Hancock concluded that populations with entropy smaller than a given threshold become saturated and their diversity disappears. Pires et al. [18] studied the signal propagation during the evolution of a GA. The mutation operator signal suffers a perturbation during some generations, and the corresponding fitness variation is analyzed. Pires et al. adopted the Shannon entropy to study the dynamics of MOPSO [19] and nonsorting GA II [20] during their execution. Wu et al. proposed a MOEA considering individual density (cell density) where the Shannon entropy was used to estimate the evolution state [21].

Taking these ideas in consideration, this paper studies the dynamics and self-organization of solutions during the MOPSO execution. The study analyzed two entropy indices considering three optimization functions with different swarm and archive sizes.

The main contributions of the paper are:

• New diversity indices inspired by physics and biologic systems.
• A good agreement of measures between the indices.
• Identification of stagnating states during the evolution.

Section 2 describes the method adopted in the work and includes a brief description of the main entropy concepts. Section 3 presents the indices for measuring the population diversity. Section 4 formulates the functions to be optimized and analyzes the simulations results. Finally, Section 5 outlines the main conclusions and the perspectives toward future work.

2. Methodology and Entropy Concepts

A careful look into MOPSO reveals a need to understand the dynamics during successive iterations with a particular focus on the particles’ convergence to the nondominated front and particle diversity. For this purpose, the Shannon entropy is used in the follow-up, and a set of tests is performed considering different optimization functions and archive sizes. Since the MOPSO algorithm is stochastic, a battery of tests is required to generate a representative statistical sample [22].

The entropy is associated with several concepts and interpretations [23]. Boltzmann used entropy to describe systems that evolve from ordered to disordered states. Spreading was used by Guggenheim to indicate the diffusion of an energy system from a small to a large volume. Lewis stated that in a spontaneous expansion gas in an isolated system, information regarding the particles’ locations decreases, while the missing information or uncertainty increases.

Shannon [24] developed a theory to quantify the information loss in the transmission of a given message. The study was carried out in a communication channel and focused on the physical and statistical constraints that limit the message transmission. Shannon defined entropy H as a measure of information, given by:

$$H\left(X\right)=-K\sum _{x\in X}{p}_i\left(x\right)log{p}_i\left(x\right).$$

(1)

This expression considers a discrete random variable $x\in X$ characterized by the probability distribution $p\left(x\right)$. The parameter K is a positive constant, often set to 1, and is used to express H in a given unit of measure.

The Shannon entropy can be easily extended to multidimensional random variables. For a pair of random variables $(x,y)\in (X,Y)$, entropy is defined as:

$$H(X,Y)=-K\sum _{x\in X}\sum _{y\in Y}{p}_i(x,y)log{p}_i(x,y).$$

(2)

3. Entropy Indices for Assessing the MOPSO

In this section, two indices for measuring the entropy in a MOPSO are presented. The first one captures the particle diversity, while the second addresses the front diversity.

3.1. Particle Diversity

The index to measure the particle diversity was proposed previously [25]. The index follows a particular interpretation of entropy. Indeed, entropy can express the spreading of a system energy from a `better located’ state to a `more distributed’ one. Taking this idea in mind, the minimum spanning tree that connects all the archive particles, A, where each connection belongs to the set of edges that connects all the $\#A$ particles with the minimal edge distance, was considered. Let $d_i$ be one of these edges, where $i\in \{1,2,\dots ,\#A-1\}$, and $p_i$ is a probability given by the following Equation:

$$p_i=\frac{d_i}{{\displaystyle \sum _{j=1}^{\#A-1}}{d}_j}.$$

(3)

The particle diversity index is based on this point of view and can be represented as:

$$H\left(X\right)=-\sum _{i=1}^{\#A-1}{p}_{i}log{p}_i.$$

(4)

3.2. Front Level Heterogeneity

In ecology, the diversity measure of different populations species is equated with the uncertainty that occurs when selecting randomly one individual species from the populations [26]. The information content, or population diversity, can be defined in several ways [27], and one of them is explained in the follow-up.

Consider a population with s species in proportion to $p_i=\frac{n_i}{\#A}$, i.e., $\{p_1,p_2,\dots ,p_s\}$, where $n_i$ is the number of elements of the ith species and s denotes the total number of species. Then the population diversity is given by Shannon and Weaver’s formula [28]:

$$H^{\prime }\left(X\right)=-\sum _{i=1}^s{p}_{i}log{p}_i.$$

(5)

Taking this idea in mind, Expression (5) is used to measure the Shannon front level diversity, where $n_i$ and s are the number of particles in each front and the total number of fronts, respectively. This index is called front level heterogeneity in order to avoid confusion with the particle diversity index. In MOPSO, at later evolution stages, when only the nondominated front exists in the swarm, the entropy heterogeneity $H^{\prime }\left(X\right)$ is zero.

4. Simulations Results

This section presents the results obtained for 3 optimization problems $P_i,i=1,2,3$, and different archive sizes. The dynamic behavior of algorithms was studied using the two proposed indices.

The optimization problems $P_1$ and $P_2$ (known as DTLZ2 and DTLZ4 [29]) are defined by Equations (6) and (7), and problem $P_3$ (known as UF8 in CEC 2009 special session competition [30]) is formulated in Equation (8) as follows:

$$\left\{\begin{array}{ccc}min{P}_1\hfill & =& [f_{11}\left(X\right),f_{12}\left(X\right),f_{13}\left(X\right)]\hfill \\ f_{11}\left(X\right)\hfill & =& [1+g_1\left(X\right)]cos(x_1\pi /2)cos(x_2\pi /2)\\ f_{12}\left(X\right)\hfill & =& [1+g_1\left(X\right)]cos(x_1\pi /2)sin(x_2\pi /2)\hfill \\ f_{13}\left(X\right)\hfill & =& [1+g_1\left(X\right)]sin(x_1\pi /2)\hfill \\ g_1\left(X\right)\hfill & =& 1+9{\displaystyle \sum _{i=3}^m}{(x_i-0.5)}^2\hfill \end{array}\right.$$

(6)

$$\left\{\begin{array}{ccc}min{P}_2\hfill & =& [f_{21}\left(X\right),f_{22}\left(X\right),f_{23}\left(X\right)]\hfill \\ f_{21}\left(X\right)\hfill & =& [1+g_2\left(X\right)]cos(x_1^{\alpha }\pi /2)cos(x_2^{\alpha }\pi /2)\hfill \\ f_{22}\left(X\right)\hfill & =& [1+g_2\left(X\right)]cos(x_1^{\alpha }\pi /2)sin(x_2^{\alpha }\pi /2)\hfill \\ f_{23}\left(X\right)\hfill & =& [1+g_2\left(X\right)]sin(x_1^{\alpha }\pi /2)\hfill \\ g_2\left(X\right)\hfill & =& 1+9{\displaystyle \sum _{i=3}^m}{(x_i^{\alpha }-0.5)}^2\hfill \end{array}\right.$$

(7)

$$\left\{\begin{array}{ccc}min{P}_3\hfill & =& [f_{31}\left(X\right),f_{32}\left(X\right),f_{33}\left(X\right)]\hfill \\ f_{31}\left(X\right)\hfill & =& cos(0.5x_1\pi )cos(0.5x_2\pi )+\frac{2}{|J_1|}{\displaystyle \sum _{j\in J_1}}{g}_j\left(X\right)\hfill \\ f_{32}\left(X\right)\hfill & =& cos(0.5x_1\pi )sin(0.5x_2\pi )+\frac{2}{|J_2|}{\displaystyle \sum _{j\in J_2}}{g}_j\left(X\right)\hfill \\ f_{33}\left(X\right)\hfill & =& sin(0.5x_1\pi )+\frac{2}{|J_3|}{\displaystyle \sum _{j\in J_3}}{g}_j\left(X\right)\hfill \\ g_j\left(X\right)\hfill & =& {\left(x_j-2x_{2}sin(2\pi x_1+\frac{\mathrm{j}\pi }{m})\right)}^2\hfill \\ J_1\hfill & =& \left\{j\right|3\le j\le m,\mathrm{and}j+2\mathrm{is}\mathrm{multiple}\mathrm{of}3\}\hfill \\ J_2\hfill & =& \left\{j\right|3\le j\le m,\mathrm{and}j+1\mathrm{is}\mathrm{multiple}\mathrm{of}3\}\hfill \\ J_3\hfill & =& \left\{j\right|3\le j\le m,\mathrm{and}j\mathrm{is}\mathrm{multiple}\mathrm{of}3\}\hfill \end{array}\right.$$

(8)

where m is the number of parameters, $f_{ij}$ is the objective $j\in \{1,2,3\}$ of problem $i\in \{1,2,3\}$, and g represent some auxiliary functions in order to simplify the expressions. For $\{P_1,P_2\}$, the parameter intervals are set to $x_i\in [0,1]$. In Equation (7), the parameter value $\alpha =100$ allows a meta-variable mapping, $x_i\to x_i^2$, between the two functions [29]. For $\left\{P_3\right\}$, $x_i\in [0,1]$ if $i\le 2$ and $x_i\in [-2,2]$ if $2<i\le m-2$. For $\{P_1,P_2,P_3\}$, the number of parameters is set to $m=\{12,12,30\}$.

These problems are to be optimized using a MOPSO [5,6], where the search is driven by a population of particles that move using the equations:

$$v_i^{t+1}=w·v_i^t+{\varphi }_1·\mathrm{rand}(0,1)·(b_i-x_i^t)+{\varphi }_2·\mathrm{rand}(0,1)·(g_i-x_i^t),$$

(9)

$$x_i^{t+1}=x_i^t+v_i^{t+1},$$

(10)

where t is the iteration number, w denotes the inertia coefficient, and the positions $x_i$ and velocities $v_i$ are codified by means of real numbers. In order to start with an high exploration rate of the search space, w is initialized with the value $0.7$ and decreases linearly with t to $0.25$. In the stages where w is near the value $0.25$, more importance is given to the local search rather then the global one. In the particle motion, the same influence is given to the local best particle position $b_i$ and the position of the `best’ particle $g_i$. Therefore, the cognitive and social coefficients are set to the values ${\varphi }_1={\varphi }_2=0.8$. In a nondominated set, there is no best solution. Consequently, to choose a particle with similar characteristics while incorporating uncertainty, a particle determines its `global best’, or guide, by randomly selecting three particles from the archive and picking up the nearest particle $g_i$.

The archive is updated, at the end of each iteration, using a $(\mu +\lambda )$ strategy among the archive, $\#\mu$, and swarm, $\#\lambda$, solutions. Therefore, the best $\mu$ solutions are chosen among the archive and population solutions. The solutions are selected according to the maximin sorting scheme [12].

Four swarm sizes with $\#N=\{250,300,350,400\}$ particles and four archive sizes of $\#A=\{50,100,150,200\}$ particles are adopted, resulting in a total of $4^2$ different experiments. For each experiment, 21 distinct runs were performed, their entropies evaluated, and the medians of the particles and the populations’ diversities and heterogeneities $H_{\mathrm{m}}=\mathrm{median}\left(H\right)$ and $H_{\mathrm{m}}^{\prime }=\mathrm{median}\left(H^{\prime }\right)$ at each iteration t taken as representing the entropy evolution at that instant.

This section presents the entropy evolution for those experiments addressing the problems $\{P_1$, $P_2$, $P_3\}$.

4.1. Results of DTLZ Problems Optimization

The first optimization functions to be considered belong to problem $P_1$, with three objectives described by Equation (6). Expressions (4) and (5) are adopted to monitoring the MOPSO evolution. The results are depicted in Figure 1 and Figure 2 for archive sizes of $\#A_1=50$ and $\#A_4=200$ particles, respectively. The charts show the median entropies $H_{\mathrm{m}}$ and $H_{\mathrm{m}}^{\prime }$ versus the iteration t, for experiments with $\#N=\{250,300,350,400\}$ particles. The curves with the `solid’ and `dotted’ lines represent $H_m$ and $H_m^{\prime }$, respectively.

It can be observed that the two entropy signals are inversely correlated. In what concerns $H_{\mathrm{m}}^{\prime }$, it is verified that it starts with a low value and increases during the first iterations. Afterwards, $H_{\mathrm{m}}^{\prime }$ remains almost stationary during some interactions and finally decreases to zero. This means that at the early stages, the number of fronts increases, remains constant during a certain number of iterations, and then decreases until only one front remains (i.e., the nondominated front), when $H_{\mathrm{m}}^{\prime }=0$. As the archive size increases, the initial transient takes more iterations, because as the number of archive size increases, it gets more difficult to find a larger number of nondominated particles in the same period. On the other hand, since the number of particles is larger, it is possible for more fronts to emerge. Therefore, $H_{\mathrm{m}}^{\prime }$ takes more iterations to approach zero as the archive size increases.

Figure 3 and Figure 4 present the entropy indices for the $P_2$ problem. A behavior similar to the one exhibited by $P_1$ is visible.

4.2. Results of $P_3$ Optimization

Figure 5 and Figure 6 present the entropy indices for the $P_3$ problem. Here, $H_{\mathrm{m}}^{\prime }$ starts by decreasing, showing that the number of fronts begins to reduce until only one front remains. Since the initial value of $H_{\mathrm{m}}^{\prime }$ is low, the number of initial fronts is also small. On the other hand, the diversity between the particles begins with a high value and increases slightly during the iterations.

The number of fronts, i.e., the entropy front level diversity, can increase or decrease at early stages depending on the optimization problem.

4.3. Correlation Coefficient

The results reveal a correlation between $H_{\mathrm{m}}$ and $H_{\mathrm{m}}^{\prime }$. The Pearson correlation coefficient r measures the strength and direction of a relationship between two indices, during $T={10}^3$ iterations:

$$r=\frac{\#A{\displaystyle \sum _{t=1}^T}{H}_{\mathrm{m}}\left(t\right)H_{\mathrm{m}}^{\prime }\left(t\right)-{\displaystyle \sum _{t=1}^T}{H}_{\mathrm{m}}\left(t\right){\displaystyle \sum _{t=1}^T}{H}_{\mathrm{m}}^{\prime }\left(t\right)}{\sqrt{\left[\#A{\displaystyle \sum _{t=1}^T}{H}_{\mathrm{m}}^2\left(t\right)-{\left({\displaystyle \sum _{t=1}^T}{H}_{\mathrm{m}}\left(t\right)\right)}^2\right]\left[\#A{\displaystyle \sum _{t=1}^T}{H^{\prime }}_{\mathrm{m}}^2\left(t\right)-{\left({\displaystyle \sum _{t=1}^T}{H}_{\mathrm{m}}^{\prime }\left(t\right)\right)}^2\right]}}.$$

(11)

Table 1. Pearson correlation coefficient r between $H_{\mathrm{m}}$ and $H_{\mathrm{m}}^{\prime }$.

Archive Size (${\mathit{A}}_{\mathbf{p}}$)	Swarm Size (${\mathit{N}}_{\mathbf{p}}$)	${\mathit{r}}_1$	${\mathit{r}}_2$	${\mathit{r}}_3$
50	250	$-0.97$	$-0.92$	$-0.51$
	300	$-0.97$	$-0.92$	$-0.66$
	350	$-0.88$	$-0.96$	$-0.50$
	400	$-0.93$	$-0.89$	$-0.43$
100	250	$-0.99$	$-0.99$	$-0.81$
	300	$-0.99$	$-0.92$	$-0.69$
	350	$-0.97$	$-0.96$	$-0.73$
	400	$-0.96$	$-0.95$	$-0.75$
150	250	$-0.98$	$-0.98$	$-0.82$
	300	$-0.98$	$-0.99$	$-0.74$
	350	$-0.97$	$-0.99$	$-0.80$
	400	$-0.98$	$-0.97$	$-0.69$
200	250	$-0.99$	$-0.98$	$-0.81$
	300	$-0.98$	$-0.98$	$-0.72$
	350	$-0.97$	$-0.98$	$-0.75$
	400	$-0.96$	$-0.98$	$-0.77$

presents the correlation between the particle diversity and front heterogeneity entropies. Column 1 indicates the archive size, column 2 stands for the swarm size, and the symbols $r_1$, $r_2$, and $r_3$ represent the correlation between $H_{\mathrm{m}}$ and $H_{\mathrm{m}}^{\prime }$ for the problems $P_1$, $P_2$, and $P_3$, respectively. An almost perfect negative relationship can be observed between them for each DTLZ problem. The relationship for the $P_3$ problem signals is moderate for the archive size of $\#A=50$ particles, but it is stronger for the other archive sizes. These values of r demonstrate that the 2 indices are in good agreement, concluding that the diversity solution is highly correlated in the number of fronts.

4.4. Archive Evolution

In order to show the particle distribution of the archive, the position of the particles archive is plotted at $t=\{0,3,80,110,400,1000\}$ iterations, for problem $P_2$, with $\#N=250$ and $\#A=200$. The iterations are chosen at different stages of the evolution, namely, at the beginning ($t=1$), when the diversity index drops down ($t=4$), at the end of this stagnation phase ($t=80$), after an abrupt increase of the index ($t=110$), after some iterations ($t=400$), and at the end of the run ($t=1000$). The plots represented in Figure 7 illustrate the result for one single run.

It can be observed that the entropy achieves the maximum value when the archive solutions are well dispersed.

5. Conclusions and Future Work

Two indices based on entropy were proposed to characterize the MOPSO dynamics. This work measures the diversity of the archive during the evolution, adopting one possible interpretation of entropy. The first index, the particle diversity, is used to measure the archive diversity between particles. The second index, borrowed from ecology, measures the species heterogeneity, in this case the front level heterogeneity. The indices were evaluated using different approaches, but both entropy indices were in good agreement, revealing that solution diversity is correlated with the number of fronts. The particle diversity indices when stagnated reveal that the algorithm has converged. On the other hand, the front level heterogeneity, when reaching zero, indicates that there is only one front in the archive.

For most MOPSO reported in the literature, the performance evaluation is analyzed at the end of the algorithm, by comparing the final front extension, spreading, and diversity. The indices formulated in this paper can be used to analyze the convergence rate during the time evolution. In future work, the indices will be used to identify stained stages of MOPSO and introduce mechanisms to promote the dispersion of particles during evolution optimization.