Measuring the Performance of Ant Colony Optimization Algorithms for the Dynamic Traveling Salesman Problem

Abstract

Ant colony optimization (ACO) has proven its adaptation capabilities on optimization problems with dynamic environments. In this work, the dynamic traveling salesman problem (DTSP) is used as the base problem to generate dynamic test cases. Two types of dynamic changes for the DTSP are considered: (1) node changes and (2) weight changes. In the experiments, ACO algorithms are systematically compared in different DTSP test cases. Statistical tests are performed using the arithmetic mean and standard deviation of ACO algorithms, which is the standard method of comparing ACO algorithms. To complement the comparisons, the quantiles of the distribution are also used to measure the peak-, average-, and bad-case performance of ACO algorithms. The experimental results demonstrate some advantages of using quantiles for evaluating the performance of ACO algorithms in some DTSP test cases.

1. Introduction

Ant colony optimization (ACO) is a mainstream swarm intelligence algorithm inspired by the foraging behavior of real ant colonies [1,2,3]. In ACO, a colony of artificial ants constructs solutions guided by artificial pheromone trails and some heuristic information. ACO has been successfully applied in various complex optimization problems [4,5,6]. However, in most real-world applications, the optimization problem has a dynamic environment [7]. In particular, the  environment of the optimization problem, including objective function, constraints, search space, input variables, and so on, may change over time.

Major challenges occur when addressing dynamic optimization problems because repeated optimization of the changing optimum is required. The optimal solution in which the algorithm has converged to it before the dynamic change may not be the optimal solution after the dynamic change. Consequently, the optimization process needs to be restarted in order to search for a new optimal solution. However, it may not be very efficient to restart the optimization process from scratch in complex problems (e.g., $\mathcal{NP}$-hard combinatorial problems), as they require huge computational efforts and time [8].

A more efficient way to address dynamic optimization problems is to use knowledge from previously optimized environments to speed up the re-optimization process [9]. With ACO algorithms, this knowledge can be transferred via the pheromone trails generated in the previous environments. However, the pheromone trails of the previous environment may contain information directing the search process to the previous optimal solution, which may represent a poor local optimum in the current environment [10]. Several strategies have been integrated to enhance the adaptation capabilities of ACO in dynamic optimization problems, such as the generation of new ants using immigrant schemes [11,12], population-based pheromone update strategies [13], multiple ant colonies [14], and memetic computing [15].

In this work, the traveling salesman problem (TSP) is used as the base optimization problem to generate dynamic test cases using the benchmark generator proposed in [10]. A conventional ACO [16] and a population-based ACO [13] that use pheromone evaporation and memory archive, respectively, to adapt to dynamic changes are applied to the generated dynamic TSP (DTSP) test cases.

The most common method to evaluate the performance of ACO in dynamic environments is used in the experiments, that is, to  obtain the mean and standard deviation (for multiple independent executions) for the generated DTSPs. Statistical tests are performed to compare the algorithms. In [17], it is argued that, for stochastic algorithms (e.g., ACO algorithms), this way of comparison may not be very informative. This is because the distribution of executing ACO algorithms multiple times may be asymmetrical and, hence, outlier values may affect the mean value. Consequently, inadequate information regarding the average performance of the algorithms may be provided. In contrast, quantiles of the distribution may address this issue with the use of median values as an average performance measurement.

The rest of the paper is structured as follows. Section 2 provides a description of the TSP as well as a description of the generation of dynamic test cases. Section 3 describes the application of the ACO metaheuristic to the TSP. In addition, the pheromone update policies in dynamic environments for the two ACO algorithms used in the experiments are described. Section 4 presents the experimental setup and results. An analysis of the obtained results is provided. Section 5 concludes this work.

2. The Dynamic Traveling Salesman Problem

2.1. Problem Definition

The TSP is a well-known combinatorial optimization problem proven to be $\mathcal{NP}$-hard [18]. The problem can be described as follows: a salesperson must visit multiple cities once and only once, starting from one city and returning finally, to the same city [19]. Typically, a fully connected weighted graph $G=(N,A)$ is used to model the TSP, where $N=\{1,\dots ,n\}$ is the set of n cities and $A=\left\{\right(i,j)\in N:i\ne j\}$ is a set of arcs connecting these cities. Each arc $(i,j)$ is associated with a non-negative value $d_{ij}$, which is the distance between cities i and j.

The TSP solution $s=[1,\dots ,n]$ is a permutation of city indices, and the objective function of the TSP is defined as follows:

$$\varphi \left(s\right)=\sum _{i=1}^{n-1}{d}_{s\left(i\right)s(i+1)}+d_{s\left(n\right)s\left(1\right)}.$$

(1)

2.2. Generating Dynamic Environments

There are two main dynamic versions of the TSP in which: (a) the values associated with the arcs are changing [12,20,21,22]; and (b) the cities to be visited are changing [11,13,23]. The dynamic benchmark generator (Available at https://github.com/Mavrovouniotis/ACODTSP (accessed on 23 February 2023)) used in [10] is adopted to generate DTSP test cases. Each DTSP test case is characterized by the type of change (either the weights or the nodes change), the magnitude of change (either small, medium, or severe), and the frequency of change (either fast or slow).

For the DTSP with weight changes, a factor that increases or decreases the value of arcs connecting cities i and j is assigned every environmental period T, i.e.,  $d_{ij}\left(T\right)$, where $T=\lceil t/f\rceil$, f is the frequency of a dynamic change, and t is the algorithmic iteration count. The value of the arc $(i,j)$ changes as follows:

$$d_{ij}(T+1)=\left\{\begin{array}{cc}{d}_{ij}\left(0\right)+{\mathcal{R}}_{ij},\hfill & \mathrm{i}\mathrm{f} \mathrm{a}\mathrm{r}\mathrm{c}(i,j)\in A_{rnd}\left(T\right),\hfill \\ d_{ij}\left(T\right),\hfill & \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e},\hfill \end{array}\right.$$

(2)

where ${\mathcal{R}}_{ij}$ is a random number drawn from a normal distribution with a zero mean and standard deviation proportional to the initial value of the arc $d_{ij}\left(0\right)$, $A_{rnd}\left(T\right)\subset A$ is a set of randomly selected arcs with size $\lceil mn(n-1)\rceil$ in which their values are subject to change at environmental period index T, and $m\in (0,1)$ is the magnitude of change. Therefore, the quality value of solution s for the DTSP depends on the parameter t and, thus, is evaluated as $\varphi (s,t)$. In other words, the  quality of a solution s at time t may be different at time $t+1$.

For the DTSP with node changes, a newly generated set of cities, i.e., $N_{rnd}\left(T\right)$, is initialized, with n random cities in the range of the actual set of cities N. A dynamic change is generated for every f function evaluation, in which exactly $\lceil mn\rceil$ cities are randomly chosen from $N_{rnd}\left(T\right)$ to replace exactly $\lceil mn\rceil$ randomly chosen cities from N, where f and m define the frequency and magnitude of a dynamic change, respectively, as above.

3. Addressing the Dynamic Traveling Salesman Problem

3.1. Ant Colony Optimization Metaheuristic

ACO has been designed for graph problems and, thus, it can be applied directly to the TSP, which is modeled as a weighted graph [24,25,26]. All arcs are associated with a pheromone trail value ${\tau }_{ij}$ and some heuristic information value ${\eta }_{ij}$. Initially, the pheromone trails are assigned with equal values, i.e., ${\tau }_{ij}\leftarrow {\tau }_0,\forall \in A$, where ${\tau }_0$ is the initial pheromone value. The heuristic information at environment T is defined as ${\eta }_{ij}=1/d_{ij}\left(T\right)$. A colony of $\omega$ artificial ants is initially positioned in different randomly selected cities. Each ant k will select the next city to visit based on the existing pheromone trails ${\tau }_{ij}$ and the defined heuristic information ${\eta }_{ij}$ of arc $(i,j)$. Each ant will visit all cities once, in order to represent a feasible DTSP solution. Then, the pheromone trails associated with the $\omega$ solutions constructed by the ants will be reinforced according to their quality value.

3.1.1. Constructing Solutions

With a probability $(1-q_0)$, where $0\le q_0\le 1$ is a parameter of the decision rule, the k-th chooses the next city j from city i, from a probability distribution that is defined as follows:

$$p_{ij}^k=\left\{\begin{array}{cc}\frac{{\left[{\tau }_{ij}\right]}^{\alpha }{\left[{\eta }_{ij}\right]}^{\beta }}{{\sum }_{l\in {\mathcal{N}}_i^k}{\left[{\tau }_{il}\right]}^{\alpha }{\left[{\eta }_{il}\right]}^{\beta }},\hfill & \mathrm{i}\mathrm{f}j\in {\mathcal{N}}_i^k,\hfill \\ 0,\hfill & \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\hfill \end{array}\right.$$

(3)

where ${\tau }_{ij}$ and ${\eta }_{ij}$ are the existing pheromone trail and heuristic information values, respectively, ${\mathcal{N}}_i^k$ is the set of cities that ant k has not visited yet, and $\alpha$ and $\beta$ are the two parameters that determine the relative influence of the pheromone trail and heuristic information, respectively. With  probability $q_0$, ant k chooses the next city, i.e., j, with  the maximum probability as follows [27]:

$$j=\underset{j\in {\mathcal{N}}_i^k}{argmax}{\left[{\tau }_{ij}\right]}^{\alpha }{\left[{\eta }_{ij}\right]}^{\beta }.$$

(4)

This selection process will continue until each ant has visited all cities once and only once, as shown in Algorithm 1. The constructed solutions will be evaluated based on Equation (3).

3.1.2. Updating Pheromone Trails

The pheromone update procedure consists of two parts: (1) pheromone evaporation, in which all pheromone trails are reduced by a constant rate; and (2) pheromone deposit, in which ants increase the pheromone trails associated with their constructed solutions.

In this work, the pheromone policy of the $\mathcal{MAX}-\mathcal{MIN}$ Ant System ($\mathcal{MM}$AS) algorithm is used, which is one of the best-performing ACO variants [28,29]. Firstly, in the pheromone update procedure of $\mathcal{MM}$AS, an evaporation is performed on all pheromone trails as follows:

$${\tau }_{ij}\leftarrow \left(1-\rho \right){\tau }_{ij},\forall (i,j)\in A,$$

(5)

Algorithm 1 Construct Solutions (t)

1: for each ant k do   2:    $s^k\leftarrow \varnothing$   3:    $r\leftarrow$ random $\{1,\dots ,n\}$   4:    $s^k\leftarrow s^k\cup r$   5:    while $s^k$ length not equal with n do   6:      $j\leftarrow$ select next city $\in {\mathcal{N}}_i^k$   7:      $s^k\leftarrow s^k\cup j$   8:    end while   9:    $C^k\leftarrow \varphi (s^k,t)$ 10: end for

where $0<\rho \le 1$ is the rate of evaporation.

Thereafter, the best ant is allowed to deposit pheromone as follows:

$${\tau }_{ij}\leftarrow {\tau }_{ij}+\Delta {\tau }_{ij}^{best},\forall (i,j)\in s^{best},$$

(6)

where $s^{best}$ is the solution generated by the best ant and $\Delta {\tau }_{ij}^{best}=1/C^{best}$, where $C^{best}=\varphi (s^{best},t)$ is the quality value of DTSP solution $s^{best}$. The best ant to deposit pheromone may be either the best-so-far ant, in which case $\Delta {\tau }_{ij}^{best}=1/C^{bs}$, where $C^{bs}=\varphi (s^{bs},t)$ is the solution quality of the best-so-far ant, or the iteration-best ant, in which case $\Delta {\tau }_{ij}^{best}=1/C^{ib}$, where $C^{ib}=\varphi (s^{ib},t)$ is the solution quality of the best ant of the iteration. By default, the iteration-best ant is used to update the pheromone trails and, occasionally, the best-so-far ant. The pheromone trail values in $\mathcal{MM}$AS are kept to the interval $[{\tau }_{min},{\tau }_{max}]$, where ${\tau }_{min}$ and ${\tau }_{max}$ are the minimum and maximum pheromone trails limits, respectively. The overall $\mathcal{MM}$AS pheromone update is presented in Algorithm 2, lines 1–20.

3.2. Adapting in Dynamic Environments

ACO algorithms have proven effective in addressing dynamic optimization problems because they are very robust algorithms, according to Bonabeau et al. [30]. However, when a dynamic change occurs, the ACO’s pheromone trails of the previous environment will become outdated because they will be associated with the previous optimum. A straightforward way to tackle the dynamic changes in the DTSP using a conventional ACO, such as the $\mathcal{MM}$AS, is to restart the optimization process whenever changes occur. In particular, the pheromone trail limit values ${\tau }_{max}$ and ${\tau }_{min}$ are reset back to their initial values, and all the pheromone trails are re-initialized to the initial ${\tau }_0$ value, as in Equation (5), whenever a dynamic change is detected.

There are two concerns when restarting an ACO: (1) the dynamic changes are not always detectable [31], and (2) a portion of the previous pheromone trails that can help discover the new optimum faster is erased [32]. For the first concern, the changes in the DTSP can be detected by re-evaluating a constant solution (or detector) on every iteration (note that more than one detector can be used). If a change occurs in the quality of the detector, then a dynamic change is recorded. In this way, a change is always detected for the DTSP with node changes but not for the DTSP with weight changes. This is because the weights affected by the dynamic change may not belong to that particular detector, and hence, its quality will not be affected [10]. For the second concern, the pheromone trails of the previous environment will be most probably useful when the changing environments are correlated, e.g.,  when the dynamic changes are small to medium, the new environment is more likely to be similar to the previous environment. As a consequence, some of the previous pheromone trails will still be useful to the new environment.

Algorithm 2 Pheromone Update  ($t,s^{ib},s^{bs}$)

1: if$\mathcal{MM}$AS is selected then   2:    ${\tau }_{max}\leftarrow 1/\rho C^{bs}$   3:    ${\tau }_{min}\leftarrow {\tau }_{max}\left(1-\sqrt[n]{0.05}\right)/\left(\left(cand-1\right)·\sqrt[n]{0.05}\right)$   4:    for $\forall (i,j)\in A$ do   5:      ${\tau }_{ij}\leftarrow (1-\rho ){\tau }_{ij}$   6:    end for   7:    $s^{best}\leftarrow s^{ib}\left|\right|s^{bs}$   8:    $\Delta {\tau }_{ij}^{best}\leftarrow 1/C^{ib}\left|\right|1/C^{bs}$   9:    for each arc$(i,j)\in s^{best}$ do 10:      ${\tau }_{ij}\leftarrow {\tau }_{ij}+\Delta {\tau }_{ij}^{best}$ 11:    end for 12:    for $\forall (i,j)\in A$ do 13:      if ${\tau }_{ij}>{\tau }_{max}$ then 14:         ${\tau }_{ij}\leftarrow {\tau }_{max}$ 15:      end if 16:      if ${\tau }_{ij}<{\tau }_{min}$ then 17:         ${\tau }_{ij}\leftarrow {\tau }_{min}$ 18:      end if 19:    end for 20: end if 21: if P-ACO is selected then 22:    $\Delta \tau \leftarrow ({\tau }_{max}-{\tau }_0)/K$ 23:    if $t==K$ then 24:      $pop\left(t\right)\leftarrow pop\left(t\right)\setminus s^r$ 25:      for each arc$(i,j)\in s^r$ do 26:         ${\tau }_{ij}\leftarrow {\tau }_{ij}-\Delta \tau$ 27:      end for 28:    end if 29:    $pop\left(t\right)\leftarrow pop\left(t\right)\cup s^{ib}$ 30:    for each arc$(i,j)\in s^{ib}$ do 31:      ${\tau }_{ij}\leftarrow {\tau }_{ij}+\Delta \tau$ 32:    end for 33: end if

Therefore, a more efficient way is to allow the pheromone evaporation defined in Equation (5) to remove the useless pheromone trails and utilize the useful pheromone trails of the previous environment. Specifically, the useless pheromone trails will reach the ${\tau }_{min}$ value due to the constant deduction of pheromone evaporation, whereas the useful pheromone trails will be reinforced up to the ${\tau }_{max}$ due to the pheromone deposit in Equation (6). In other words, ACO will be able to adapt to the newly generated environment.

Figure 1 demonstrates the performance of $\mathcal{MM}$AS with and without pheromone re-initialization and proves our claims above. Specifically, it can be easily observed that when the dynamic changes are small (i.e., $m=0.1$), adapting to the changes maintains better offline performance (see Equation (9)) in most environmental changes (except when $T=3$), whereas when the dynamic changes are severe (i.e., $m=0.75$), re-initializing the pheromone trails maintains better offline performance in most environmental changes (except when $T=7$ and $T=9$). More details regarding the experimental setup of Figure 1 are given later in Section 4.

A more advanced way is to use the pheromone update policy of the population-based ACO (P-ACO) that has been specifically designed to address optimization problems with dynamic environments [13]. The overall P-ACO pheromone procedure is presented in Algorithm 2, lines 21–33. In particular, an archive, i.e., $pop\left(t\right)$, of iteration-best ants is maintained. For every iteration t, the iteration-best ant enters $pop\left(t\right)$, and a positive constant pheromone update is added to the trails associated with the arcs that belong to its solution as follows:

$${\tau }_{ij}\leftarrow {\tau }_{ij}+\Delta \tau ,\forall (i,j)\in s^{ib},$$

(7)

where $\Delta \tau =({\tau }_{max}-{\tau }_0)/K$ is the constant pheromone value added, $s^{ib}$ is the solution of the iteration-best ant, K is the size of $pop\left(t\right)$, and ${\tau }_{max}$ and ${\tau }_0$ are the maximum and initial pheromone trail values, respectively.

Whenever a dynamic change occurs, the solutions represented by the current ants in the population list may become invalid (e.g., when existing nodes are removed and replaced with new nodes). Therefore, these solutions are repaired heuristically using the KeepElite principle [33]. Specifically, the affected nodes are removed from the solution, reconnecting the successor and predecessor nodes. The new nodes are inserted in the best possible position of the solution, causing, in this way, the least increase in the solution quality. At the same time, the corresponding pheromone trails associated with the arcs of the affected nodes are updated.

When $pop\left(t\right)$ is full, the current iteration-best ant needs to replace an existing ant, say r, in $pop\left(t\right)$, following a negative constant update to its corresponding pheromone trails, which is defined as follows:

$${\tau }_{ij}\leftarrow {\tau }_{ij}-\Delta \tau ,\forall (i,j)\in s^r,$$

(8)

where $\Delta \tau =({\tau }_{max}-{\tau }_0)/K$ is the constant deducted pheromone value (the same with the added value) and $s^r$ is the solution of the ant to be replaced, typically the oldest entry of the population list $pop\left(t\right)$. Note that pheromone evaporation is not used in the P-ACO algorithm. However, it is able to adapt to the changes because outdated pheromone trails are removed directly from the population list.

The overall application of $\mathcal{MM}$AS and P-ACO algorithms for the DTSP is described in Algorithm 3. It is worth mentioning that in both $\mathcal{MM}$AS and P-ACO, the solution of the best-so-far ant is repaired using KeepElite, and the heuristic information is updated, reflecting the newly generated distances as in Equation (2), whenever a dynamic change occurs (see Algorithm 3, lines 16–33).

Algorithm 3 ACO for DTSP

1: $t\leftarrow 0$   2: $C^{bs}\leftarrow \infty$   3: for $\forall (i,j)\in A$do   4:    ${\tau }_{ij}\leftarrow {\tau }_0$   5:    ${\eta }_{ij}\leftarrow 1/d_{ij}\left(0\right)$   6: end for   7: while termination condition not met do   8:    Construct Solutions(t)                  % Algorithm 1   9:    $s^{ib}\leftarrow$ find best ant at iteration t 10:    if $\varphi (s^{ib},t)$ is better than $\varphi (s^{bs},t)$ then 11:      $s^{bs}\leftarrow s^{ib}$ 12:      $C^{bs}\leftarrow C^{ib}$ 13:    end if 14:    Pheromone Update($t,s^{ib},s^{bs}$)      % Algorithm 2 15:    if dynamic change occurs then 16:      if $\mathcal{MM}$AS || P-ACO then 17:         Repair $s^{bs}$ 18:         for $\forall (i,j)\in A$ do 19:           ${\eta }_{ij}\leftarrow 1/d_{ij}\left(T\right)$ 20:         end for 21:         if P-ACO then 22:           for $\forall (i,j)\in A$ do 23:              ${\tau }_{ij}\leftarrow {\tau }_0$ 24:           end for 25:           for each ant $k\in pop\left(t\right)$ do 26:              Repair $s^k$ 27:              for each arc$(i,j)\in s^k$ do 28:                ${\tau }_{ij}\leftarrow {\tau }_{ij}+\Delta \tau$ 29:              end for 30:           end for 31:         end if 32:      end if 33:    end if 34:    $t\leftarrow t+1$ 35: end while

4. Experimental Results

4.1. Experimental Setup

The dynamic benchmark generator, described in Section 2, is used to generate the DTSP test cases. The following three stationary TSP benchmark instances are obtained from TSPLIB (Available at http://comopt.ifi.uni-heidelberg.de/software/TSPLIB95/ (accessed on 3 April 2023)): kroA100, rat575, and pr1002. The magnitude of change m is set to slightly (i.e., $m=0.1$), medium (i.e., $m=0.25$ and $m=0.5$), and severely (i.e., $m=0.75$) changing environments. The frequency of change f is set to $10n$ iterations (i.e., $f=1000$ for kroA100, $f=5750$ for rat575, and $f=\mathrm{10,020}$ for pr1002) to allow sufficient time for the ACO algorithms to converge—and it is scalable for each problem instance. For each type of DTSP (i.e., weights or nodes changes), a total of four test cases are generated from each static benchmark instance. Ten environmental changes are allowed for each DTSP test case.

The modified offline performance [34] is used as the performance metric, which is defined as follows:

$${\overline{P}}_{offline}=\frac{1}{E}\sum _{t=1}^E\varphi (s^{b{s}^{\prime }},t),$$

(9)

where E is the number of observations taken, and $\varphi (s^{b{s}^{\prime }},t)$ is the solution quality value of the best-so-far solution since the last dynamic change.

The common ACO parameters of $\mathcal{MM}$AS and P-ACO are set to $\alpha =1$, $\beta =5$, and $\omega =25$. The evaporation rate of $\mathcal{MM}$AS is investigated with values $\rho =\{0.1,0.2,0.5,0.8\}$, and it is set to $\rho =0.8$. Furthermore, the exploitation strength of the decision rule is set to $q_0=0.0$, and the initial pheromone trail value ${\tau }_0=1/\rho C^{nn}$, where $C^{nn}=\varphi (s^{nn},0)$ is the solution quality generated by the nearest-neighbor heuristic. The maximum and minimum pheromone trail limits are set to ${\tau }_{max}=1/\rho C^{best}$ and ${\tau }_{min}={\tau }_{max}\left(1-\sqrt[n]{0.05}\right)/\left(\left(cand-1\right)·\sqrt[n]{0.05}\right)$, respectively, where $cand$ is the number of different choices available to an ant at each step. The frequency with which the best-so-far ant deposits pheromone occasionally is set for every 25 iterations [29]. The population list size of the P-ACO is investigated with values $K=\{2,3,5,10\}$, and it is set to $K=3$. Furthermore, the initial pheromone trail value is set to ${\tau }_0=1/(n-1)$, the exploitation strength of the decision rule is set to $q_0=0.5$, and the maximum pheromone trails limit is to ${\tau }_{max}=1$ [35].

4.2. Statistical Comparisons Using Mean and Standard Deviation

Since ACO algorithms are non-deterministic stochastic metaheuristics, 50 independent runs were executed on the same set of random seed numbers to perform statistical comparisons.

Table 1. Mean and standard deviation results of ACO algorithms for the DTSP with node changes.

Algorithm $\mathit{m}\Rightarrow$	0.1	0.25	0.5	0.75
	kroA100
$\mathcal{MM}$AS	22,223.86 ± 102.9	22,492.60 ± 108.1	22,537.36 ± 95.2	22,472.96 ± 70.7
P-ACO	22,118.66 ± 49.0	22,350.54 ± 67.1	22,419.70 ± 54.7	22,320.62 ± 48.8
	rat575
$\mathcal{MM}$AS	6573.22 ± 25.1	6577.54 ± 66.1	6399.26 ± 22.5	6401.76 ± 15.3
P-ACO	6581.46 ± 25.9	6443.86 ± 20.4	6566.00 ± 43.8	6392.60 ± 19.8
	pr1002
$\mathcal{MM}$AS	305,793.32 ± 1719.4	315,315.60 ± 1633.72	314,533.80 ± 1364.8	315,447.28 ± 1419.8
P-ACO	308,342.64 ± 733.1	315,763.52 ± 456.0	313,705.60 ± 445.2	315,500.44 ± 490.4

Bold values indicate statistical significance.

and

Table 2. Mean and standard deviation results of ACO algorithms for the DTSP with weight changes.

Algorithm $\mathit{m}\Rightarrow$	0.1	0.25	0.5	0.75
	kroA100
$\mathcal{MM}$AS	20,543.36 ± 164.7	20,600.40 ± 62.0	20,222.00 ± 60.6	19,869.66 ± 58.2
P-ACO	20,462.34 ± 36.4	20,663.38 ± 64.7	20,196.60 ± 44.7	19,900.18 ± 37.9
	rat575
$\mathcal{MM}$AS	6621.04 ± 23.4	6465.78 ± 23.1	6404.72 ± 22.1	6399.72 ± 12.7
P-ACO	6758.06 ± 70.5	6675.62 ± 25.3	6643.98 ± 15.0	6618.88 ± 13.2
	pr1002
$\mathcal{MM}$AS	266,132.52 ± 1625.4	264,653.36 ± 1902.2	268,062.60 ± 1104.2	268,531.12 ± 1255.6
P-ACO	282,904.76 ± 592.3	276.911.88 ± 394.2	275,933.72 ± 382.7	275,841.04 ± 494.7

Bold values indicate statistical significance.

shows the mean and standard deviation of the ${\overline{P}}_{offline}$ results obtained by the $\mathcal{MM}$AS and P-ACO algorithms for the DTSP with node and weight changes, respectively. In  addition, the statistical test comparisons are reported using the Wilcoxon rank-sum statistical test with a significance level of 0.05. Figure 2, Figure 3 and Figure 4 show plots of ${\overline{P}}_{offline}$ against the environmental changes, for kroA100, rat575, and pr1002, respectively. From the comparisons, the following observations can be drawn.

First, P-ACO performs significantly better than $\mathcal{MM}$AS in most DTSPs with node changes. This is because the P-ACO has been designed to address DTSPs with node changes. In particular, when ants are currently stored in the population list when a dynamic change occurs, they will be repaired heuristically using change-related information (i.e., the inserted and removed nodes). Therefore, a constant amount of pheromones will be removed directly from the outdated trails associated with the arcs connecting the nodes to be removed. At the same time, the pheromone trails associated with the arcs connecting the nodes to be inserted will receive a constant amount of pheromone. This specific pheromone update will have a direct effect on the solution construction of the next iteration. On the other hand, the pheromone evaporation of $\mathcal{MM}$AS requires more iterations to express its effect (e.g., more than the frequency used in these experiments [10]). Also, the removal of outdated pheromone trails is not applied locally to the affected areas, as in the P-ACO, because the evaporation is applied globally on all arcs.

Second, $\mathcal{MM}$AS performs significantly better than P-ACO in most DTSPs with weight changes. This is because repairing the ants stored in the population list has no effect when a dynamic change of this type occurs. In particular, the feasibility of the stored solutions is not affected as in the case of the DTSP with node changes. Only the solution quality will change (which will be re-evaluated); however, because a constant pheromone update is performed, it has no effect on the P-ACO’s behavior. However, since the pheromone trails in P-ACO are removed directly because of the constant pheromone removal, there is a higher risk of erasing knowledge from previous environments. On the other hand, the pheromone evaporation in $\mathcal{MM}$AS is able to better adapt to these types of dynamic changes because the decrease in the pheromone trails is performed gradually.

4.3. Quantile Comparisons Using Peak, Average, and Bad-Case Performances

Although the mean and standard deviation values presented in the previous section are useful in performing statistical comparisons between ACO algorithms, they are not very informative [17]. For example, they do not provide any indication of the expected solution quality value in case ACO is executed once, which is the case in real-world applications. On the other hand, the quantiles of the distribution values (from multiple independent runs) address this issue as follows: if the quantile $Q_p$ of the solution quality value of an ACO algorithm is $C^{bs}$, then the probability of achieving a solution quality value better or equal to $C^{bs}$ is greater than or equal to p.

Table 3. Experimental results of ACO algorithms for the DTSP with node changes.

Algorithm	kroA100				rat575				pr1002
$\mathit{m}\Rightarrow$	0.1	0.25	0.5	0.75	0.1	0.25	0.5	0.75	0.1	0.25	0.5	0.75
	$Q_{0.10}$
$\mathcal{MM}$AS	22,127	22,390	22,440	22,385	6573	6459	6375	6381	302,494	311,643	311,175	312,291
P-ACO	22,084	22,266	22,356	22,268	6546	6418	6397	6367	307,513	315,147	313,007	314,951
	$Q_{0.50}$
$\mathcal{MM}$AS	22,186	22,467	22,520	22,486	6570	6483	6395	6402	306,249	315,903	314,514	316,560
P-ACO	22,102	22,341	22,416	22,319	6583	6445	6432	6390	308,586	315,880	313,964	315,455
	$Q_{0.90}$
$\mathcal{MM}$AS	22,400	22,670	22,676	22,556	6613	6518	6431	6420	309,279	318,927	317,478	318,266
P-ACO	27,212	22,432	22,506	22,386	6610	6467	6481	6419	309,059	316,302	314,222	316,100

Bold values indicate the best results of each comparison.

and

Table 4. Experimental results of ACO algorithms for the DTSP with weight changes.

Algorithm	kroA100				rat575				pr1002
$\mathit{m}\Rightarrow$	0.1	0.25	0.5	0.75	0.1	0.25	0.5	0.75	0.1	0.25	0.5	0.75
	$Q_{0.10}$
$\mathcal{MM}$AS	20,415	20,531	20,147	19,804	6594	6493	6380	6383	260,009	258,675	266,618	267,049
P-ACO	20,424	20,590	20,147	19,854	6658	6643	6624	6603	28,2195	276,460	275,443	275,157
	$Q_{0.50}$
$\mathcal{MM}$AS	20,483	20,590	20,223	19,873	6618	6465	6402	6399	267,186	265,953	268,035	268,321
P-ACO	20,453	20,654	20,196	19,895	6767	6680	6644	6620	282,958	276,900	275,998	275,900
	$Q_{0.90}$
$\mathcal{MM}$AS	20,881	20,689	20,308	19,944	6658	6499	6439	6417	270,615	269,053	269,799	270,295
P-ACO	20,519	20,753	20,245	19,956	6833	6706	6662	6634	283,574	277,404	276,372	276,496

Bold values indicate the best results of each comparison.

show the quantiles $Q_{0.10}$, $Q_{0.50}$, and $Q_{0.90}$ of $\mathcal{MM}$AS and P-ACO for the DTSP with node and weight changes, respectively. Quantile $Q_{0.10}$ indicates the peak performance, $Q_{0.50}$ indicates the average performance, and $Q_{0.90}$ indicates the bad-case performance of algorithms. In addition, Figure 5, Figure 6 and Figure 7 show the box plots of $\mathcal{MM}$AS and P-ACO drawn between the first and third quartile of the distribution on kroA100, rat575, and pr1002, respectively, for the different DTSP test cases.

Another issue that the use of quantiles may address is when the distribution of executing ACO algorithms multiple times is asymmetrical (i.e., having several outliers such as the distribution of $\mathcal{MM}$AS in Figure 5 for kroA100 with weight changes, and P-ACO in Figure 6 for rat575 with weight changes), affecting the mean comparisons [17]. Therefore, the quantile $Q_{0.50}$, that defines the median of the distribution, may be a more adequate choice to measure the average performance of ACO algorithms than the mean values shown in Section 4.2. This effect is demonstrated when comparing the mean results of ACO algorithms in Table 1 (and Table 2) with the corresponding median results in Table 3 (and Table 4). Specifically, it can be observed that the comparisons of the DTSP with weight changes are consistent, whereas the comparisons of the DTSP with node changes have several inconsistencies as follows. P-ACO has better mean values than $\mathcal{MM}$AS in most DTSP cases of pr1002, while $\mathcal{MM}$AS has better median values than P-ACO on the same DTSP cases. The difference regarding the distribution of P-ACO and $\mathcal{MM}$AS can be observed in Figure 7, in which P-ACO has a more symmetrical and concentrated distribution than $\mathcal{MM}$AS.

From the quantile comparisons in Table 3 and Table 4, it can also be observed that: first, the peak performances of both ACO algorithms are better, as expected, than  the average performance or bad-case performance, and second, the average performance (i.e., $Q_{0.50}$) comparisons in Table 4 for the DTSP with weight changes are consistent with the peak (i.e., $Q_{0.10}$) and bad-case (i.e., $Q_{0.90}$) performance comparisons since $\mathcal{MM}$AS outperforms P-ACO in all cases. In contrast, when comparing the average performance results with the peak performance results for the DTSP with node changes in Table 3, there are some inconsistencies. For example, $\mathcal{MM}$AS has a better peak performance than P-ACO on the pr1002 test cases, whereas P-ACO has a better average performance than $\mathcal{MM}$AS on the same test cases.

In summary, $\mathcal{MM}$AS may be a better choice for DTSP with weight changes when the independent execution is performed in parallel. This is because it has a better peak performance than P-ACO, and hence, it is expected to obtain a solution at least as good as $Q_{0.10}$. In contrast, P-ACO may be a better choice when a single execution is performed because there are more chances in obtaining a solution as good as $Q_{0.50}$.

5. Conclusions

In this work, two ACO variations are applied to optimization problems in a dynamic environment. The TSP is used as the base problem to systematically generate dynamic test cases. Two types of dynamic changes are considered: (1) when nodes change and (2) when weights change. From the experiments, the follow concluding remarks can be drawn. First, P-ACO performs significantly better than $\mathcal{MM}$AS in most DTSPs with node changes. Second, $\mathcal{MM}$AS is significantly better than P-ACO in DTSP with weight changes. Third, P-ACO has better average performance than $\mathcal{MM}$AS in most DTSP with node changes. Fourth, $\mathcal{MM}$AS has a better peak performance in most DTSPs with weight changes.

In general, complex problems arising in practical applications, such as training a neural network, require huge computational power. As a consequence, the computation power for optimizing complex problems as well as performing several independent executions (due to the stochastic nature of swarm and evolutionary computation algorithms) in parallel may not be available. Therefore, optimization algorithms with better average performances may be the best choice rather than optimization algorithms with better peak performance.

For future work, it will be interesting to apply and evaluate ACO algorithms in more practical applications, such as the agile earth observation satellites’ scheduling problem [36,37]. In fact, this particular problem can be solved as a constrained TSP [38] and consists of several unexpected environmental changes that may affect the scheduled tasks [39,40]. Hence, the adaptation capabilities and evaluation of the ACO demonstrated in this work may be suitable to address the aforementioned problem.