**5. Simulations Results**

We verify the applicability of the techniques presented in this manuscript to the pilot sequence assignment problem, as well as the impact of using them in terms of user SINR, system spectral efficiency and minimum and maximum user spectral efficiency. We developed a MATLAB script to create a dataset and made it public at github.com/ evertonalex/utfpr-ppgi-pilotcontamination, accessed on 12 May 2022.

We consider a seven hexagonal cell cluster geographically disposed such as presented in Figure 1. Each user in the cell is randomly positioned. However, to establish some physical limits, user positions can only be integers numbers pairs and must be contained within the limits of the hexagon, i.e., let (*<sup>x</sup>*, *y*) ∈ Z<sup>2</sup> be some user position related to its base station which is fixed at the origin, *x*, *y* ∼ U{1, *<sup>R</sup>*}, and the following conditions are also met:

$$1. \quad |x| \le R$$

$$\begin{array}{rcl} \text{2.} & |y| \le \frac{\sqrt{3}}{2} \, R\\ \text{3.} & \frac{\sqrt{3}}{2} \, |x| + |y| \end{array}$$

$$3. \quad \frac{\sqrt{3}}{2}|x| + |y| \ge \frac{\sqrt{3}}{2}R$$

**Figure 1.** Base stations, users and cells geographic disposition in our simulation scenario. User positions were randomly generated.

After their positions are randomly generated with respect to the origin, each user position is translated to their cell position. Moreover, this process replaces any users that overlap the same position with a new random one. Table 2 shows the parameters used in the aforementioned scenario.

**Table 2.** System parameters used in the computational simulations.


*5.1. Heuristic Parameters Optimization*

At this point, when the performance results are combined with the execution time results, such as observed in Figure 2 and 3, more iterations do not represent a more significant increase in performance. Hence, the best approach is to use as few iterations as possible in each of the algorithms due to the high cost and low increase in performance. All the simulations use the optimal parameters for each heuristic, and the number of iterations is set to 50.

**Figure 2.** Spectral efficiency average by iterations.

**Figure 3.** Time to complete the execution of a single instance in the dataset by each algorithm. We used a Google Cloud solution with 32 virtual CPUs, 32GB of RAM, running MATLAB on Ubuntu 20.

Since each heuristic has its own set of parameters which may impact the overall performance, we investigated the algorithms performance in the following scenario: *K* = 20 users and *R* = 1000 m. The findings, for each heuristic, are presented in Table 3. The values which result in the best performance are marked in bold type.


**Table 3.** Set of tested heuristic parameters. The parameters in bold are the ones which resulted in the best system spectral efficiency.

It is worth noting that although we tested different parameters for each heuristic, the results in terms of system spectral efficiency are where all are within a less than 1% margin from each other. For instance, the increment in the results for GA when *G*max goes from 100 to 200 is around 0.0019%. Additionally, the increment from using a 0.5 mutation rate to a 0.75 is around 0.0067%. In both cases, the difference means less than a bit per second per Hertz, which mean they are irrelevant system-wise.

This pattern is also observed in the BPSO, SPV–PSO and SPVG–PSO (with or without VNS) results. The difference between the parameters is within the margin of error for the dataset size used in our work.

Since the difference between the algorithms performance in terms of average SE is marginal, we further analyzed the minimal spectral efficiency case. The argumen<sup>t</sup> here is that increasing the minimal SE may result in a reduction in terms of outage probability.

Thus, Table 4 presents the minimal SE for different combinations of maximum number of iterations and for each algorithm. It is worth noting that only the optimal heuristic parameters were used in this simulation. Furthermore, it is clear that the tested heuristics are divided into two groups when we compare their performance: the low performing group includes RA, BPSO and GA while the high performing group is composed of SPV– PSO, SPV–VNS–PSO, SPVG–PSO and SPVG–VNS–PSO.

**Table 4.** Minimal spectral efficiency (in bps/Hz) using the optimal parameters for each heuristic (*ω* = 0.25, *v*max = 2, *VNS* = 0.30, *Tm* = 0.50 and *K* = 20).


Besides the aforementioned fact, one can also imply that the average variation in performance among all algorithms in the high performance bracket when the number of iterations is increased from 50 to 100 is less than 2%. Additionally, even though Table 4 shows that RA, BPSO and GA had the same performance despite the change in the number of iterations, the average difference in performance was in the order of 0.0013% when the number of iterations rise from 50 to 100.

Finally, as it is commonplace, increasing the number of iterations directly impacts the computational time of all the algorithms but RA. Hence, we present Figure 3 which shows the average time needed to compute the solution.

At this point, it is clear when the performance results are combined with the execution time results that the best approach is to use as few iterations as possible in each of the algorithms due to the high cost and low increase in performance. From now on, all the simulations use the optimal parameters for each heuristic, and the number of iterations is set to 50.

As one may observe, each algorithm has a set of parameters that need to be adapted to the discussed problem. In order to find the best parameters, we considered a macro-cell scenario scenario with *K* = 20 users per cell of *R* = 1000 m in radius and a maximum number of iterations *I* = 50 to all heuristics.

For the PSO-based algorithms, we first analyzed the inertia coefficient *ω*. Figure 4 shows the results which demonstrate that *ω* = 0.25 is the best configuration for both BPSO and SPVG–PSO, while *ω* = 0.1 is the best option for SPV–PSO. Even though the performance differences of the tested parameters values are marginal, the best values were used to evaluate other aspects of the proposed solutions.

**Figure 4.** Results of tests to find the better parameter to inertia (*K* = 20, *R* = 1000 m and *v*max = 1).

The second parameter tested in the PSO-based algorithms was the maximum velocity (*<sup>v</sup>*max). Values from one to four were used, and the best results were different for each algorithm: *v*max = 4 for the BPSO, *v*max = 3 to SPV–PSO and *v*max = 2 to SPVG–PSO. The tests results are shown in Figure 5. As it was observed with the inertia coefficient, the increment in system performance were less than 0.1% when compared to the random allocation process.

**Figure 5.** Result of tests to find the better parameter to *v*max (*K* = 20, *R* = 1000 m and (*ω*) = 0.25).

The third and forth parameters tested were the local and global solution weights (*<sup>c</sup>*1 and *c*2). These parameters are not found in the SPVGs variants, so they were only tested for BPSO and SPV–PSO.

All the previous analysis considered that *c*1 = *c*2 = 1, so we compared the results with those solutions. Increasing the local solution weight *c*1 = 2 and maintaining the global weight *c*2 = 1 downgrades the BPSO performance in 0, 001%, while increasing the SPV– PSO performance in 0.02%. Further increasing the local solution *c*1 = 5 and *c*2 = 1 made the BPSO algorithm's performance around 0.004% higher than the baseline (*<sup>c</sup>*1 = *c*2 = 1), while an increase of 0.03% was found for the SPV–PSO when compared to the baseline. Finally, changing the parameters to *c*1 = 1 and *c*2 = 5 increased the BPSO performance in 0.005% while decreased the SPV–PSO performance in 0.02%

This test showed that *c*1 and *c*2 values are not significant in terms of performance since their impact are in the one thousandth of a bit/s/Hz mark.

The last PSO-based parameters optimization was the VNS rate. We tested four different configurations, and the results are presented in Figure 6. They demonstrate that a rate of 0.9 is the best configuration for both SPV–PSO–VNS and SPVG–PSO–VNS.

Finally, the GA algorithm parameter was tested. Figure 7 shows the GA's performance for different mutation rates, and the *Tm* = 0.9 was found to be the best parameter value.

**Figure 6.** Results of tests to find the better parameters to VNS ( *K* = 20, *R* = 1000 m, (*ω*) = 0.25 and *v*max = 1).

**Figure 7.** Results of tests to find the better parameters to *Tm* (*K* = 20, *R* = 1000 m).

#### *5.2. Cell Size Impact on Performance*

We conducted an analysis of the impact of cell size on all of the algorithms performance. We set up a scenario with *K* = 20 users per cell and the maximum number of iterations per algorithm to 50. To evaluate performance, Figure 8 shows the cumulative distribution function using our dataset.

As one may see, the impact of cell size on the CDF is really marginal, which may indicate that the system is not interference bounded. This is corroborated with two facts.

No intracell interference exists due to the orthogonality of the pilot sequences within a cell. The path-loss and shadowing effects deteriorate the interfering signals from other cells.

**Figure 8.** Empirical CDF for different cell sizes with *K* = 20 users, *G*max = 50 generations and *I* = 50 iterations.

Although the optimization problem aims to maximize the system spectral efficiency, we analysed the minimum spectral efficiency among all users in the system which may be seen as a fairness measure in the system. Table 5 presents the numerical values of the average lowest spectral efficiency in the system considering the 1000 instances in the dataset. The difference between the results for the RA, BPSO and GA algorithms were marginal, so they are listed in the same column.

An important remark to make here is: for a cell with a 100 m radius, the SPV–PSO has an average lowest spectral efficiency 1.7% higher than the RA, BPSO and GA algorithms.


**Table 5.** Minimum spectral efficiency per user in different cell sizes. The best value for each algorithm is marked in bold type.

In relation to the average maximum spectral efficiency, the results are presented in Table 6. It is clear that RA, BPSO and GA algorithms have a worse performance than the other ones. In fact, the average maximum spectral efficiency more than doubles when the same cell radius is compared.

Finally, in terms of cell size, it is clear that for all the results presented here, the impact in terms of CDF, average maximum and minimum spectral efficiency is marginal which corroborates the argumen<sup>t</sup> that the system is not interference bounded.


**Table 6.** Maximum spectral efficiency per user in different cell sizes. The best value for each algorithm is are marked in bold type.

#### *5.3. System Loading Impact on Performance*

Figure 9 describes the empirical CDFs for different system loading scenarios: *K* = 20, *K* = 40 and *K* = 60. These values represent three system loading scenarios which represent low, medium and high user density. The main objective here is to determine whether the higher density of users in the system implies more interference among them.

It is evident that RA, BPSOand GA suffer an impact in performance as the system loading increases for the same cell size. However, that significant difference in performance is not observed for the other algorithms, meaning that they can handle user density better than the former three algorithms. Numerically, there is an 11 dB CDF shift from *K* = 20 users to *K* = 40 and a 7 dB shift from *K* = 40 to *K* = 60. Meanwhile, the performance difference between *K* = 20 and *K* = 60 is less than a 0.1 dB shift.

**Figure 9.** Empirical CDF for *K* = 20, *K* = 40 and *K* = 60 users in an *R* = 1000 m cell size scenario with the maximum number for iterations set to 50 for all algorithms.
