Automatic Task Selection from Targets Recognition for Swarm Mobile Robots with Specialized Agents ^†

Abstract

Considering specialized agents in swarm robotics, with robots dedicated to specific tasks, requires that formation control and efficient transition in the leadership of the swarm is achieved. Here, a task switching approach is formulated by evolving the definition of specialization to match with targets recognition in the environment, such as detecting special landmarks via embedded sensors. Specialization zones are defined around each detected target corresponding to a task to be dealt with by a specific robot. Entering within the zone of influence surrounding a target triggers the switching of the leader of the formation. The framework is also further refined by making the targets, and therefore the corresponding zone of influence, dynamic, which leads to the consideration of combined specialization areas. The proposed system is validated in simulation to demonstrate that the group of robots effectively coordinate themselves around targets and dynamically allocate the appropriate specialized agent.

1. Introduction

The coordination of a group of mobile robots is widely studied in the literature through rules that govern the local interaction between robots. The coordination of a fleet of robots can be defined as a collective behaviour [1] resulting from the interactions between the robots. Soysal and Sahin [2] present a gathering model of swarm robots, which results from switching between three basic steps, namely random walk, wait and approach. Each step is activated and controls the robots behaviour for a given duration. Then it expires to allow the second step to start and control the robots for another predefined period of time. Shucker and Bennett [3] simulate the coordination of a scalable physical system that deploys a large number of robots as a distributed remote sensing array. Hettiarachchi et al. [4] propose a physics-based framework to address a connectivity problem of collaborative swarm robotics as an autonomous control problem. O’Grady et al. [5] propose a decentralized technique for controlling a group of robots where the robots can be physically connected to create stable morphological structures. A multi-robot system is adopted by Beckers et al. [6] to regroup a large set of randomly distributed objects into a single cluster. The authors address three behaviors for the robots based on the assumption that the system activates only one robot at any time. Wawerla et al. [7] provide a multi-robot based solution to the construction industry. Inspired by mammals or insects that release pheromone to the environment for communications, Payton et al. [8] propose “Pheromone robotics” to coordinate the navigation of a large number of robots. Alternatively, classical potential fields motion coordination for multi-robot exploration is addressed by Howard et al. [9]. Each robot in such system serves as a sensor node and is subject to virtual potential forces. Sabattini et al. [10] also present a decentralized approach based on potential fields to control multi-robot formation. Many other works also address the coordination of swarm systems [11,12,13] for different purposes, but they generally build on the assumption that individual robots among the swarm possess equal coordination properties and capabilities.

In this paper, the framework for the coordination and task-based switching of the leadership among specialized robots that was previously introduced in [14] is extended to address more advanced scenarios. The problem is formulated by evolving the definition of specialized zones of influence distributed over the workspace of a group of robots. The concept of specialized agents assigned to specific tasks is extended to enforce that specialized agents with different and specific capabilities lead the swarm when handling a task, itself detected and recognized in the workspace using on-board sensors. The proposed approach is also revisited to account for dynamic zones of influence, which better reflects realistic mobile robotic intervention scenarios.

2. Overall Proposed Framework

The proposed automatic task selection system combines two cascaded stages, as shown in Figure 1. The first stage is an Automatic Task Selection Unit (ATSU) and the second one controls the robot dynamics and fleet formation. The ATSU operates in two main modes, a normal and an automatic mode. In the normal mode, the robots look for a specific predefined task identified by an operator. In the automatic mode, the swarm of robots uses embedded sensors to explore its environment and identify tasks at hand. In both modes, the group of robots proceeds in three successive phases characterized by three independent areas surrounding a given target location, as shown in Figure 2. The three phases considered are: a Search phase, a Task phase and an Execution phase.

In the Search phase, the robots adopt a cooperative leader-follower formation where a by-default group leader is generating the path that the group of robots follows until RGB-D sensors on-board the robots recognize a target within reach of the formation. The system switches to the Task phase as soon as the center point of the swarm enters the zone of influence (i.e., the task switching border in Figure 2) that surrounds the detected target. The robots then change their formation according to the specialty that the task requires. While continuing the navigation towards the target position as a group, the proper specialized agent transitions to become the leader of the formation. When the fleet approaches the task execution border (Figure 2), the specialized agent, which became the formation leader, approaches towards the target and performs the task associated with the detected target. Once completed, the mission automatically switches back to the Search phase to further explore the environment and search for a new task.

Figure 2. Three phase zones (Search, Task, and Execution phases).

Figure 2. Three phase zones (Search, Task, and Execution phases).

3. Operation Modes and Phases

As described above, the proposed framework can operate under two modes of operation and three successive phases. The specific formulation of each is detailed in this section. The normal mode of operation is used when a number of tasks to be performed at specific locations in the workspace are known a priori. The operator, or a supervisory layer, identifies a specific target to reach, on which the group of robots is to execute a specific task. Then, the cooperative group of robots proceeds through the three main phases defined above. After the task is completed, the automatic task selection unit (ATSU) prompts the operator or supervisory layer and waits for another specific task to be performed. As such, the normal operation mode provides the system with the capability to be semi-automatically driven, under the assistance of an operator. In the automatic mode, when the number and specificities of tasks to be executed are unknown, the ATSU automates the task detection, recognition and execution processes. The group of robots aims to automatically detect targets and perform the entire set of tasks that are available on the workspace, based on the nature of targets that are discovered, and without consulting the system operator.

3.1. Search Phase

In this work, a number n of specialized robotic agents are considered. These agents are chosen to perform $m$ different tasks. Assuming that these agents begin their navigation on the workspace with $n=\left\{1\right\}$ acting as the default group leader during the search phase, the path of the group of robots is planned starting from the initial positions of the robots and designed to allow for the group of robots to navigate and survey the entire tasks space while attempting to sequentially perform all tasks available. The references generation, formation control and stabilization of the system in the Search phase is detailed in [14].

3.2. Task Phase

When on-board sensors recognize a target and estimate its position, T_pose = (x_t,y_t), the sensors measure the distance between the target and the group of robots while they keep approaching that target and until they enter the circular zone of influence surrounding it. A variable radius, D_TZ, characterizes the zone of influence around each target. It defines a switching edge that triggers the transition of the group of robots to a new formation in preparation for executing the specialized task. When the formation enters the zone of influence, the ATSU determines which agent is appropriate to perform this specific task, as detailed in Algorithm 1. The system switches to the Task phase and assigns this agent to become the new group leader. During the task phase, the group of robots smoothly changes its formation while robots continue approaching the target. Within the zone of influence of the target, the new leader reference is determined based on the recognized position of the target, T_pose = (x_t,y_t), and the current leader position, $\left(x_{l_t},y_{l_t}\right)$. The follower robots stop on the border of the execution zone and wait until the leader reaches the target.

Algorithm 1: Automatic Task Selection Unit method to select appropriate leader robot

$$\begin{gathered} \mathrm{Step}1:\mathrm{RGB}\mathrm{Sensor}\mathrm{Output}=TaskState=\left(T_{pose},T_{type}\right) \\ \mathrm{Step}2\mathrm{a}:\mathrm{ATSU}\mathrm{checks}\mathrm{if}\mathrm{robots}\mathrm{have}\mathrm{reached}\mathrm{the}\mathrm{task}’\mathrm{s}\mathrm{zone}\mathrm{of}\mathrm{influence}: \end{gathered}$$ $\mathrm{Calculate}\mathrm{the}\mathrm{Euclidean}\mathrm{distance}\left(\mathrm{ED}\right)\mathrm{between}\mathrm{the}\mathrm{task}\mathrm{position}\left(T_{pose}\right)\mathrm{and}\mathrm{the}\mathrm{current}$ $\mathrm{Centre}\mathrm{point}\mathrm{of}\mathrm{the}\mathrm{current}\mathrm{formation},P_{center},\mathrm{and}\mathrm{compare}\mathrm{it}\mathrm{with}\mathrm{the}\mathrm{radius}\mathrm{of}\mathrm{the}$ $\mathrm{Task}\mathrm{Zone}(D_{TZ})$ $\mathrm{if}ED\le D_{TZ}$                                    Then Enable Switching to the Task Phase.                     else                                    exit $\mathrm{Step}2\mathrm{b}:\mathrm{ATSU}:\mathrm{Check}\mathrm{the}\mathrm{Task}\mathrm{Type}\left(T_{type}\right);$                              Select the suitable agent to perform the current task                              Assign the selected robot to be the new leader $\mathrm{Return}\mathrm{the}\mathrm{new}\mathrm{group}\mathrm{leader}\mathrm{ID}\left(L\right).$           Step 3: Assign the new group leader as the reference: $\mathrm{ATSU}:\mathrm{if}L=1$ $\left(x_{ref},y_{ref}\right)=$$\left(x_{l_t|1},y_{l_t|1}\right)\mathrm{else}\mathrm{if}L=2$ $$\begin{gathered} \left(x_{ref},y_{ref}\right)=\left(x_{l_t|2},y_{l_t|2}\right) \\ \mathrm{else}\mathrm{if}L=3 \end{gathered}$$ $\left(x_{ref},y_{ref}\right)=\left(x_{l_t|3},y_{l_t|3}\right) ;$ $$\begin{gathered} \mathrm{Return}\mathrm{the}\mathrm{current}\mathrm{leader}\mathrm{reference}\left(x_{ref},y_{ref}\right) \\ \mathrm{Step}4:\mathrm{Stop}\mathrm{the}\mathrm{followers}\mathrm{at}\mathrm{the}\mathrm{border}\mathrm{of}\mathrm{the}\mathrm{execution}\mathrm{zone}. \end{gathered}$$

3.3. Execution Phase

The on-board sensing system keeps measuring the distance between the target position, T_pose = (x_t,y_t), and the current group’s leader position, $\left(x_{l_t},y_{l_t}\right)$, while the leader keeps approaching the target until it enters the execution zone. The group of robots then reconfigure themselves in a special formation that will distribute the follower robots along the execution border. Over the execution period, the followers are not only to provide a cooperative coordination but also they can work as stationary sensors to assist the leader while it performs the task. From the moment the robots hit the execution zone, the ATSU compares the current Euclidean distance between the leader and the target position until it becomes equal to a critical distance of execution. Then the ATSU sends a switching signal to switch the group formation to the execution phase. As such, during the execution phase, the group of robots adopts a special formation where the follower robots localize themselves along the execution border and encircle the target together.

4. Overlapping Zones of Influence and Dynamic Targets

While zones of influence surrounding targets would typically be distinct, as shown in Figure 3, overlapping zones of influence may result from collocated targets having influence over a fairly large radius, or targets being dynamic which makes their respective zone of influence to interact with each other. Such a situation is depicted in Figure 4, where the blue and red targets share a mutual section of the workspace under their respective zone of influence. The proposed framework dynamically handles such situations via the ATSU. Whenever overlapping zones of influence are considered after detection of one of more targets, the operator or supervisory layer is consulted to determine which target should be executed first. The operator commands the system to one of the recognized targets and the robots continue their operation while concentrating on the selection. Once that target is resolved, they proceed directly to the second recognized task located in the same zone. When all targets have been visited, the group of robots returns to the Search mode for a new task to be identified while they patrol the workspace.

The automatic mode also supports cases where targets are moving over the workspace. To address this situation, when the sensors collect different positions for a target over time, the robots switch to a target following phase instead of the Task phase. Target following ensures the robots keep following the target until it stops. The target stopping assumption is made to address the case where a target is moving faster than the group of robots, which would compromise completion of the task. Once the target stops moving, the execution of the task proceeds as usual.

5. Simulation Results

The proposed framework is validated in simulation using Simulink. For simulation purpose n = 3 robots acting as specialized agents and m = 3 different types of targets are considered. Each robot is specialized to only execute one type of tasks. Matching robots and tasks are color-coded for clarity in Figure 3, Figure 4 and Figure 5. Robot paths are depicted by matching color lines, and tasks are depicted by colored square dots surrounded by dotted line circles that represent their respective zone of influence. In the Search phase, robots navigate over the workspace and keep their cooperative formation while following their default leader (e.g., red agent in Figure 3a) until on-board sensors recognize a target. The system switches subsequently to the Task and Execution phases, as shown in Figure 3b,c.

In the simulation, it is assumed that when the proper specialized robot reaches the matching target position, then the task is completed. The target and zone of influence symbols change to black, indicating that the given task has been executed, as shown in Figure 3d. The system then returns to the Search phase, and the default leader robot transitions back to its role of leading the group of agents while the other robots become followers again and until a new specialized task is detected.

Figure 3. (a) Robots in Search phase; (b) robots switch to Task phase; (c) robots switch to Execution phase; and (d) task completed.

Figure 3. (a) Robots in Search phase; (b) robots switch to Task phase; (c) robots switch to Execution phase; and (d) task completed.

Figure 4 depicts a scenario where zones of influence overlap as more than one target attract the robots toward the same area. The ATSU then consults the operator to decide which target has to be executed first. Here the red task is selected by the operator to be performed first. Once the priority task is complete (turned to black), the robots switch automatically to execute the second task, where the new leader (blue) is smoothly brought ahead of the group of robots and the other robots become followers, as shown in Figure 4b. Finally, Figure 5 demonstrates the scenario of dynamic targets where the robots follow one of the green targets as it moves away from the agents and that until the target stops. At that point the matching green specialized robot executes the task.

Figure 4. (a) Overlapping zones of influence with task 1 (red) selected as having priority which drives the red robot to become the leader until task completion; and (b): second priority task (blue) being performed with blue robot transitioned to leader position.

Figure 4. (a) Overlapping zones of influence with task 1 (red) selected as having priority which drives the red robot to become the leader until task completion; and (b): second priority task (blue) being performed with blue robot transitioned to leader position.

Figure 5. Scenario with dynamic target: (a) green moving target recognized; (b) green target being pursued by group of robots with green agents transitioning toward leader position; and (c) green target stopped and task executed by matching green leader robot.

Figure 5. Scenario with dynamic target: (a) green moving target recognized; (b) green target being pursued by group of robots with green agents transitioning toward leader position; and (c) green target stopped and task executed by matching green leader robot.

6. Conclusions

A generic conceptual framework to ensure coordination and formation control of a group of mobile agents is detailed in this paper. Here, specialized agents are considered, with robots dedicated to perform specific tasks, according to their on-board equipment and mechanical capability. A task switching approach is formulated that evolves the definition of specialization to match with targets recognition. The proposed Automatic Task Selection Unit (ATSU) is implemented to operate in two possible modes, either under external supervision or in a fully automated way. Both modes perform each specialized task in three phases that ensure the search for a target, its detection and recognition via embedded sensors, and finally the execution of a given task compatible with the resources available on a selected agent which also serves as the group leader for the duration of a specialized assignment. The various simulation scenarios demonstrate that efficient navigation and smooth transition of the leadership in between the group of robots is achieved with the proposed framework. The ATSU is able to deal with a variable number of targets, including several targets requiring the same specialized intervention, and allows for prioritization to be easily implemented in between targets thanks to an embedded supervisory layer. The proposed sequence of three modes of operation provides the necessary support for the ATSU to deal with the exploratory, pursuit, and task execution components typically found in swarm robotic scenarios. The fact the overlapping zones of influence and dynamic target considerations are added to the approach also opens the door to general operational contexts that can be reliably and efficiently handled by the framework. Future developments will involve the development of robust target and task detection and recognition from embedded sensors and integration of the proposed framework with more advanced supervisory functionalities, as well as experimental validation on real robots.

Authors Contributions

The authors jointly developed the concept; O. Al-Buraiki and H. Morales Busiquia implemented the framework and performed experiments; all authors analyzed the data and wrote the paper.