**5. Discussion**

In this section, we compare the performance of VNS against the multi-start SA used by [10] for the four complex instances under consideration, taking into account both the quality of the solution reached by means of the fitness function and the four original objectives and execution times.

Each instance was executed 10 times using a Intel(R) Xeon(R) E3-1240 PC with 3.50 GHz and 16 GB of RAM, running Windows 10.

Table 2 shows the minimum, mean and maximum values of the fitness function in the optimal solutions achieved by the VNS and SA. The mean values for SA are clearly higher than for VNS in all four instances under consideration. We can thus conclude that SA slightly outperforms VNS with respect to the quality of the solutions reached.


**Table 2.** Optimal fitness values in Phase 3 for SA and VNS.

Let us analyze in depth the optimal solutions derived by both metaheuristics in the four instances under consideration.

Tables 3–6 show the values of the initial solution and solutions reached by VNS and SA for the four instances under consideration, respectively, in terms of the four original objectives, optimizing the ATCo work and rest periods and positions, the similarity to the previous template-based solution, the number of control center changes (rest periods), and the distribution of the ATCo workload.

Note that Tables 3–6 provide mean values, and the respective best values for each column are highlighted in bold.

**Table 3.** Objective functions in the initial solution and solutions reached by SA and VNS for Instance 1.


**Table 4.** Objective functions in the initial solution and solutions reached by SA and VNS for Instance 2.


**Table 5.** Objective functions in the initial solution and solutions reached by SA and VNS for Instance 3.


**Table 6.** Objective functions in the initial solution and solutions reached by SA and VNS for Instance 4.


Looking at sub-object. 1, we find that VNS outperforms SA for 8 out the 12 mean values shown in Tables 3–6; whereas SA is better than VS for sub-object. 2 in the first instance but not for instances 3 and 4.

As expected, the initial solution outperforms the solutions reached by SA and VNS in terms of similarity to previous template-based solution. However, SA outperforms VNS with respect to this objective in all four instances under consideration.

Regarding the number of rest periods, SA outperforms VNS for all four instances under consideration. For all four instances, the number of rest periods in VNS is [10.67, 17]% higher than for SA.

Finally, looking at the standard deviation representing the ATCo workload dispersion, we find that SA outperforms VNS for the first instance, whereas VNS is better for Instances 2, 3 and 4. In all four cases, the ATCo workload dispersions were very similar in both metaheuristics.

These tables were shown to CRIDA experts, who analyzed the quality of the solutions reached by VNS and SA for the four complex instances. They concluded that, although the solutions derived by SA slightly outperform those reached using VNS in terms of the fitness function, the quality of both solutions were very similar four the four instances under consideration and that the key factor was then the time it took to reach that solutions, i.e., the computation times.

Tables 7 and 8 show the minimum, mean and maximum computation times (in minutes) in Phases 2 and 3, respectively, for the original multi-start SA proposed by [10], an improved non multi-start SA using a constraint implementation in the code rather than regular expressions and with parallelization (4 threads), and the proposed adaptation of VNS (also using a constraint implementation in the code and with parallelization). The metaheuristic with the lowest computation time is highlighted in bold.

**Table 7.** Computation times (in minutes) in Phase 2.



**Table 8.** Computation times (in minutes) in Phase 3.

The first thing that we found is that the computation times for the improved SA are much lower than for the original SA in both phases and for the four instances under consideration, as was expected. Besides, computation times for Phase 3 are quite a lot higher than in Phase 2, accounting for the biggest share of the accumulated computation times shown in Table 9.

**Table 9.** Mean accumulated computation times (in minutes).


If we focus on Phase 2, VNS and SA outperform each other in two out of the four instances. However, although the mean computation times for Instances 1 and 3 are similar, VNS clearly outperforms SA in Instance 2, whereas the opposite applies for Instance 4 (SA is more than 70 min faster than VNS). Note that Instance 4 involves a larger number of open sectors and required ATCos.

In Phase 3, VNS clearly outperforms SA in instance 1 but it is only slightly better for Instances 2 and 3. In Instance 4, SA clearly outperforms VNS with a difference of close to 200 min in the mean computation times.

Looking at the mean accumulated computation times in Table 9, we find that VNS clearly outperforms SA in Instances 1 and 2, they are similar for Instance 3, but SA is quite a lot better in Instance 4.

We have analyzed other sectorizations provided by CRIDA in order to verify whether or not the performance of VNS is sensitive to the instance dimension, as in the case of Instance 4, and we have confirmed this hypothesis: the improved SA outperforms VNS when the dimensionality (number of open sectors and required ATCos) is high in all cases, whereas VNS is clearly better in any other situation with a low and medium number of dimensions.

Note that although the mean computation time accumulated in Instance 4 is about 5 h (297.43 min) with the improved SA, which could be considered a high computation time, the complexity of this instance is the highest (9 simultaneously open sectors and 28 ATCos) that could materialize in Spanish airports. Consequently, the maximum computation time of this instance, 50.58 min + 360.72 min 7 h could be considered, as an upper bound for the computation times. In most cases, solutions are reached in the less than an hour.

Taking into account that the pre-tactical phase takes place one to six days before the day of operations, the CRIDA experts considered this a good upper bound for computation times.
