*4.2. Baseline Method:* EvoColor

No standard design method exists to address the class of missions we consider in this study. Little related work exists—see Section 2—and refers only to mission-specific methods that follow the neuro-evolutionary approach. Indeed, as no extensive comparison has ever been performed between neuro-evolutionary methods across multiple missions, a state of the art in neuro-evolutionary robotics has not been identified, yet. Together with the results obtained with TuttiFrutti, in the following we will present also those obtained by EvoColor—a method based on neuro-evolution for the automatic design of swarms of e-pucks that can display and perceive colors. The results we will present should not therefore be considered as a comparison between TuttiFrutti and the state of the art in neuro-evolutionary robotics. In this context, our results should rather be considered as a comparison between TuttiFrutti and a reasonable instance of the neuro-evolutionary approach.

EvoColor is an adaptation of EvoStick [65]—a standard neuro-evolutionary method previously used as a yardstick in studies on the automatic design of robot swarms [8,10,15,53]. To the best of our knowledge, EvoStick is the only neuro-evolutionary method that has been tested via simulations and robot experiments on multiple missions without undergoing any per-mission modification. EvoColor produces control software for swarms of e-pucks that operate with RM 3—see Section 3.1. The control software has the form of a fully connected feed-forward artificial neural network with 41 input nodes (*in*), 8 output nodes (*out*) and no hidden layers. In this topology, the input nodes and output nodes are directly connected by synaptic connections (*conn*) with weights (*ω*) in a range of [−5, 5]. The activation of each output node is determined by the weighted sum of all inputs nodes filtered through a standard logistic function. EvoColor selects appropriate synaptic weights using an evolutionary process based on elitism and mutation. Just as in TuttiFrutti, the evolutionary process is conducted through simulations performed in ARGoS3, version beta 48, together with the argos3-epuck library. The evolution ends when an *a priori* defined budget of simulations is exhausted. Table 3 summarizes the topology of the neural network, the novelties with respect to EvoStick and the parameters used in the evolutionary process.


**Table 3.** Network topology and parameters of the evolutionary process in EvoColor. Novelties with respect to EvoStick are highlighted in gray. They concern the capability of displaying and perceiving colors. The neural network operates according to RM 3, see Table 1.

\* The number of generations is computed according to the budget of simulations. \*\* The population obtained in the last generation is post-evaluated to select the best individual.

The readings of the proximity (*prox*) and ground (*gnd*) sensors are passed directly to the network. Information about the number of neighboring peers (*n*) is provided through the function *z* (*n*) ∈ [0, 1], with *z* (*n*) = <sup>1</sup> <sup>−</sup> <sup>2</sup> <sup>1</sup>+*e*(*n*) . The vector *Vn* and each vector in *Vc*∈{*R*,*G*,*B*,*C*,*M*,*Y*} are translated into scalar projections onto four unit vectors that point at 45°, 135°, 225°, and 315° with respect to the front of the robot. Then, each projection is passed to the network through an independent input node. The last input of the network comes from a bias node. Four output nodes encode tuples (*v* ) of negative and positive components of the velocity of the wheels. Each tuple is obtained from two independent output nodes and is defined as *v* = ([−12, 0], [0, 12]). The velocity of a wheel (*v*) is computed as the sum of the two elements in a tuple (*v* ). Similarly, the color displayed by the RGB LEDs of the robot is selected by comparing the value of the output nodes that correspond to colors in the set {∅, *<sup>C</sup>*, *<sup>M</sup>*,*Y*}. The color displayed corresponds to the maximum value found across the four colors.

EvoColor differs from EvoStick in two aspects: the reference model and how the output of the neural network is mapped to the velocity of the robots. First, EvoColor is based on RM 3 and EvoStick on RM 1.1. In accordance to RM 3, EvoColor does not integrate the capability of the e-pucks for sensing the intensity of ambiance light—originally integrated in EvoStick. The second difference between EvoColor and EvoStick is how the output of the neural network is mapped to the velocity of the e-pucks. In EvoColor, we introduce a velocity mapping based on tuples to facilitate the evolution of standstill behaviors—as we expect robots need them to perform STOP and AGGREGATION.

In EvoStick, the control software maps directly a single output node of the neural network into velocity commands (*<sup>v</sup>* = [−12, 12]) for each wheel (*vk*∈{*l*,*r*})—a robot can stand still only if the velocity of the two wheels is set exactly to 0. A standstill behavior is then difficult to achieve since only one pair of values in the output nodes maps exactly to *vl* = 0 and *vr* = 0; Moreover, the output nodes can not maintain a steady value because they are subject to the injection of sensory noise. In EvoColor, the control software maps the sum of elements of a tuple (*v* ) into velocity commands for each wheel *vk*∈{*l*,*r*}—each tuple is defined by two output nodes and provides a negative and a

positive component to compute the velocity. We expect that this mapping facilitates the evolution of standstill behaviors: first, robots can stand still if the elements of each tuple (*v* ) are any pair of values of equal magnitude—steady values are not required provided that the output nodes that encode the same tuple vary proportionally; second, the sum of the positive and negative components can cancel out the sensory noise injected in the output nodes that encode a tuple—given a proper tuning of the synaptic weights. If one compares EvoColor with EvoStick, the first has more freedom to tune neural networks that lead to standstill behaviors.
