6.4.4. *AAMO* Efficiency Assessment

In this section, several statistical analyses were performed on different indicators to demonstrate the efficiency of the proposed *AAMO* approach. Wilcoxon signed-rank tests were performed (a non-parametric test was chosen since data in each condition do not follow a normal distribution) to compare samples from two populations. More precisely, it tests the indicator differences between approaches for a given fleet size.

Firstly, in relation to TT (see Figure 23), the evidence confirms two expected results: (i) All approaches benefit from adding robots to the fleet. A Wilcoxon difference test was performed regarding TT and the fleet size for each approach. All comparisons present a significant decrease in TT when fleet size increases (*p*-value < 0.05); (ii) Since it only takes care of connectivity, the EbC approach shows the worst performance regardless the fleet size. Wilcoxon tests showed a significant result (*p*-value < 0.001) for all comparisons between approaches given a fleet size.

Additionally, all *AAMO* instances show competitive TT results even slightly outperforming other approaches in the case of *AAMO:10*. In particular, a Wilcoxon difference test showed that *AAMO:10* has a smaller TT than *MinPos* for 2 and 5 robots (resp. *W* = 169, *p*-value < 0.01, and *W* = 159, *p*-value < 0.05).

**Figure 23.** Total Exploration Time (TT) under non-ideal communication conditions.

Secondly, concerning the PL indicator (see Figure 24), the EbC approach present the worst performance, coherently. Again, the Wilcoxon test showed significant results (*p*-value < 0.001). Likewise, all *AAMO* instances show competitive results too.

Besides, and as was pointed above, the TT and PL results show that the lack of ideal communication conditions negatively affects the *MinPos* approach more than the *Yamauchi* approach. Wilcoxon tests showed a trend for 4 and 5 robots (*p*-value < 0.1) concerning TT, and a sigfinicant difference in PL for 4 robots (*p*-value < 0.05).

Up to this point, the *AAMO* approach has shown results as good as the *MinPos* approach. Next, the indicators related to connectivity are analysed in order to properly assess the potential advantages of the *AAMO* approach in the presence of more realistic communication conditions.

The DLR indicator trend is shown in Figure 25. As can be seen, while the performance of the *MinPos* and *Yamauchi* approaches are the worst, the EbC performance is remarkably the best. These visual results were confirmed by Wilcoxon tests between approaches for each fleet size. DLR indicator is significantly bigger (*p*-value < 0.001) for MinPos and Yamauchi than AAMO and EbC approaches, except for 8-sized fleets where these results are significant only when compared to EbC. Moreover EbC has a significant smaller DLR indicator (*p*-value < 0.05) than all the others approaches except for the 5 and 8 robots cases, in which any statistical difference can be found between AAMO approaches and EbC.

**Figure 24.** Path length (PL) under non-ideal communication conditions.

Similarly, the *AAMO* approach results represent a very good improvement with respect to both *MinPos* and *Yamauchi* approaches. The chart in Figure 25 reveals that our approach outperforms both Yamauchi and minPos approaches independently of the fleet size on average. Nevertheless, the smaller fleet, the greater outperforming. The explanation can arise correctly from intuition: when the environment is bounded, the probability of being disconnected tends to decrease as the fleet size increase. Therefore, the benefits of our approach tend to be smaller when the fleet size increases. Either way, it is always meaningful. Please note that even in the largest fleet size case, the DLR of *AAMO* represents an improvement of 20% on average compared to the corresponding *Yamauchi* or *MinPos*.

Furthermore, the relation between TT, DLR and *HO-Threshold* is noticeable. The more effort demanded by the human operator (higher threshold), the slower but higher connected the *AAMO* performs. This claim is confirmed by Wilcoxon tests that showed a significantly bigger (*p*-value < 0.05) TT indicator for AAMO:20 than for AAMO:10, and also show that the DRL indicator is significantly smaller (*p*-value < 0.05) for AAMO:20 than for AAMO:10, regardless the fleet size.

**Figure 25.** Disconnection Last Ratio (DLR) under non-ideal communication conditions. The bigger the *HO-Threshold*, the smaller DLR. This fact holds showing an oscillatory behaviour as the fleet size increase.

Regarding the oscillation registered, it could suggest the existence of the following rational pattern. When fleet size is even, the easier way to avoid isolation situations is keeping in pairs (connected with at least another teammate). Contrarily, when the fleet size is odd, not all robots can keep in pairs. In case the fleet has divided, at least one sub group must be composed of three robots. Therefore, this

oscillatory behaviour could hint at the fact that odd-sized fleets need to make little more effort to avoid robot isolation situations and are consequently subject to bigger DLR results as well.

Likewise, it is interesting to analyse the DLR indicator and network topology together. This way it is possible to get a closer notion about the interaction between robots along the exploration. Figure 26 is devoted to showing the number of connected components present in the network, averaged over time.

Please note that for the AAMO:20 instance—run on 2-Robot fleet—the DLR is about 40% (see Figure 25), coinciding with the percentage achieved by the 2CC of the same fleet size in Figure 26. In other words, the fleet holds a network composed of one single connected component during 60% (100%–40%) of total exploration time. Consistently, this is equivalent to say that during this portion of the time none robot has been disconnected.

Additionally, and as a matter of fact, the chart shows that as the fleet size increase it is more challenging to keep the whole fleet connected: 1CC stack is decreasing in size as the fleet size increase. Nevertheless, it also shows that simultaneously with the adding of new robots, the fleet is more and more cohesive (in relative terms). This fact may be corroborated looking at the upper part of the chart where the stacks corresponding to the greatest number of connected components are plotted. The following pattern can be observed: the number of connected components (given by *nCC*) increase slower than the fleet size *n*. Again, the fact that the *Maze* scenario is bounded may explain this phenomenon to a large extent.

**Figure 26.** Network topology composition under non-ideal communication conditions averaged over time. Depending on the number of connected components and the fleet size, it is possible to study the existence of sufficient conditions to fall into isolation situations. For instance, for a 3-Robot fleet, the 2CC or 3CC topologies imply having at least one robot isolated while for a 5-Robot fleet this implication is related to 3CC, 4CC, or 5CC topologies, and so on.

Although all this information gives an approximated notion about how disconnected is the fleet (group perspective) along explorations, it is not enough to hint what is happening at the individual level. Thus, it is also interesting to study the worst case of the individual disconnections last. This way it is easier to evaluate both coordination capabilities (how long a robot is unable to coordinate its actions with any other teammates) and risky situations (how long the fleet present single points of failure). Recall that the key motivations in considering communication constraints are strongly related with the rework avoidance: (i) When robots are unconnected they have fewer possibilities to coordinate their actions hence they could visit the same regions unnecessarily. Hence, keep them connected is a way to favour the efficiency; (ii) In the presence of damages or inner failures the exploration strategy should take those events into account preventing the need of re-exploration.

In Figure 27 the trend followed by Maximum Disconnection Last Ratio MDLR indicator is depicted showing that the bigger *HO-Threshold*, the shorter disconnection periods (Wilcoxon tests showed that the MDLR indicator is significant smaller (*p*-value < 0.05) for AAMO:20 than for AAMO:10, for 2 and 3 robots, and tends to be smaller (*p*-value = 0.09) for 4 robots) and that the last of isolation situations is at most equivalent to half of the DLR values for every fleet size and *HO-Threshold* value as well. In other words, the isolation situations regard more than one single robot and this in turn, reveals that under the *AAMO* approach the robots often intent to rejoin each other.

**Figure 27.** Maximum Disconnection Last Ratio (MDLR) under non-ideal communication conditions. MDLR shows the longest individual isolation period registered by some fleet member along the exploration. The trend is oscillatory following the same pattern as the DLR indicator.

At last but not least, it is worth to discuss the trend of OSR as the fleet size increase. The results obtained by the different *AAMO* instances are depicted in Figure 28. In Section 6.3 the OSR levels were achieved mostly thanks to simultaneous sensing actions, conversely, in these simulation runnings, the OSRs achieve higher levels due to non-ideal communication conditions. As was expected, the more the mapping information of the robots is out-of-date with respect to each other, the higher the OSR. However, in any communication conditions, the same upper bound is achieved. This suggests that the size and bounded condition of the *Maze* environment could be limiting the over-sensing phenomenon when fleet size increase beyond five robots.

To sum up and concerning the *Maze* scenario and the baseline stated in Section 6.3, the conclusions of this section are: (i) The *AAMO* approach can be employed as a strategy to coordinate multi-robot systems that are dedicated to exploration tasks; (ii) As was expected, the *HO-Threshold* value directly impacts on the connectivity level that the fleet is able to hold during the mission; (iii) Likewise, the relation between *HO-Threshold* values and the TT and DLR/MDLR indicators is the expected: the bigger the *HO-Threshold* value, the worse TT performance, but the better DLR/MDLR ratios; (iv) Although all instances of the *AAMO* approach present TT degradation with respect to the baseline, in any case it is not significantly due to the computation of the proposed task-to-robots distribution; (v) All *AAMO* instances outperform the baseline concerning the DLR and MDLR indicators; (vi) With the exception of DLR/MDLR, all instances of the *AAMO* approach outperform the *EbC* approach; (vii) The topology of the fleet networks shown during exploration is consistent with the *HO-Threshold* values, for all *AAMO* instances.

**Figure 28.** Over-sensing ratio (OSR) under non-ideal communication conditions. The *Yamauchi* and *MinPos* approach results (coloured in purple and green, respectively) obtained under ideal communication conditions are placed together to make the comparison easier.

The *AAMO* approach shows effectiveness and flexibility (through the *HO-Threshold* setup) to tackle the multi-robot exploration problem. Particularly concerning the efficiency related to both completion time and connectivity level maintenance, the approach appears as an intermediate solution that presents much better TT performance than the most restrictive approach *EbC* and better connectivity level along exploration than the approaches that do not take care about communication issues.
