Decentralized Multi-Robot Collision Avoidance: A Systematic Review from 2015 to 2021

Abstract

An exploration task can be performed by a team of mobile robots more efficiently than human counterparts. They can access and give live updates for hard-to-reach areas such as a disaster site or a sewer. However, they face some issues hindering them from optimal path planning due to the symmetrical shape of the environments. Multiple robots are expected to explore more areas in less time while solving robot localization and collision-avoidance issues. When deploying a multi-robot system, it is ensured that the hardware parts do not collide with each other or the surroundings, especially in symmetric environments. Two types of methods are used for collision avoidance: centralized and decentralized. The decentralized approach has mainly been used in recent times, as it is computationally less expensive. This article aims to conduct a systematic literature review of different collision-avoidance strategies and analyze the performance of innovative collision-avoidance techniques. Different methods such as Reinforcement Learning (RL), Model Predictive Control (MPC), Altruistic Coordination, and other approaches followed by selected studies are also discussed. A total of 17 studies are included in this review, extracted from seven databases. Two experimental designs are studied: empty/open space and confined indoor space. Our analysis observed that most of the studies focused on empty/open space scenarios and verified the proposed model only through simulation. ORCA is the primary method, against which all the state-of-the-art techniques are evaluated. This article provides a comparison between different methods used for multi-robot collision avoidance. It discusses if the methods used are focused on safety or path planning. It also sheds light on the limitations of the studies included and possible future directions.

1. Introduction

The use of autonomous mobile robots has increased in various fields of life in recent times. Mobile robots are being used in industry, medicine, search and rescue, and other applications. Most of these applications require a team of robots to fulfill the task efficiently and in less time than their human counterparts [1]. However, they face some issues hindering them from optimal path planning due to the symmetrical shape of the environment. Multiple robots are expected to explore more areas in less time while solving robot localization and collision-avoidance issues. In a scenario where a team of mobile robots is operating, it is necessary to keep them safe from collisions and the surroundings. Collision avoidance for a multi-robot system forms the focus of many current studies. When deploying a multi-robot system, it is ensured that the hardware parts do not collide with each other or the surroundings, especially in symmetric environment. There are two main approaches used to address collision avoidance in multi-robot systems: centralized and decentralized [2]. The centralized approach is efficient only for small groups of robots, and for large groups, the decentralized approach is more effective, as it is less expensive computationally. A decentralized method that uses a triangular grid pattern was introduced by Yang [3], using previously explored maps and information from neighbors to avoid collisions.

The agents need a communication system to inform them about their current position and velocity in a decentralized approach. Various types of communication setups have been introduced in recent years. The Alternating Direction Method of Multipliers (ADMM) used by Rey et al. [4] works so that each agent communicates with the neighboring agents, creating a local coordinate system. The setup can manage transmission from all the agents simultaneously. Being a decentralized network, it can manage frequent setup changes and failures. Rodríguez et al. [5] used the velocity information of each agent to create a cooperative communication setup. The relative velocity of agents was then used to decrease an individual agent’s detection region, resulting in collision avoidance. A scheme suggesting the use of a shared memory-driven device for coordination and communication was presented by Pesce et al. [6]. Each agent utilizes the shared device to learn from the collective private observations and share relevant information with other agents. Deep reinforcement learning and Voronoi cell setup were proposed in Wang et al. [7] and Nguyen et al. [8] to ensure cooperative behavior among the agents. Another approach towards collision avoidance is by using a local or global path planner. These approaches use path-planning strategies such as GMapping [9] to avoid collisions [10,11]. Socially aware path-planning strategies have gained popularity recently. Avoidance of robot-to-pedestrian collisions, human-like speed of motion, and navigation through dense environments require carefully planned trajectories [12,13]. When planning a trajectory, the structure of the environment also plays a vital role in collision avoidance. It is essential to explore the environment carefully to maintain a low time and energy cost while reaching the target. Some task planner techniques have been presented to address this issue [14]. However, very few studies have discussed cluttered and confined environments [15,16].

In the process of exploring the databases for this SLR study, it was observed that most of the studies on collision avoidance are conducted for aerial vehicles [17,18,19], surface vehicles [20,21,22], ships [23,24,25,26,27] and underwater vehicles [15,28,29] However, the focus of this SLR is only on ground vehicles. By carefully scanning the titles and abstracts of the studies, any study using other than ground vehicles was excluded. In this SLR, 17 studies from 2015 to date are selected after systematic screening. All these studies address the collision-avoidance issue for multi-robots (ground vehicles) directly or indirectly. These studies propose new algorithms such as CADRL or innovate classic techniques such as ORCA to avoid collisions. Although various studies were conducted before 2015, they rarely addressed the issue of collision avoidance in a decentralized manner for confined spaces. Many studies attempted to resolve the said issue for no more than two agents. In this SLR, we select only those studies which use decentralized approaches for a large group of agents in confined space scenarios. Some studies experimented with up to a hundred agents in different scenarios [2], while others considered techniques such as ECBS for a group of agents [30].

This paper aims to retrieve the most relevant studies on collision avoidance for multi-robot systems. A thorough literature review has been conducted to provide a critical understanding of the collision-avoidance issue for a group of autonomous ground mobile robots. This paper answers the following research questions:

• RQ1. What is the number of primary studies on collision avoidance in the multi-robot system since 2015?
• RQ2. How do researchers report the implemented techniques for avoiding collisions in a multi-robot system?
• RQ3. How does a collision-avoidance technique solve coordination issues?
• RQ4. What are the limitations of the current research on collision avoidance in multi-robot systems?
• RQ5 What are the main directions of future research and possible solutions in multi-robot systems?

2. Protocol for Identifying Related Works of Collision Avoidance for Multi-Robot

The method used by Okoli et al. [31,32] is used as a guideline to answer RQ1 by extracting the maximum possible details from the studies on collision avoidance for multi-robots. This method involves planning, selection, extraction, and execution of the review. The inclusion criteria for the studies was that the study must address the issue of collision avoidance for multi-robot systems (more than two agents) and should be evaluated for multiple scenarios at least through simulation. All the works included provided at least one successful solution for avoiding collisions. They experimented with as many as a hundred agents in circular, hallway, and deadlock arrangements [2,13,33]. Some of the works indirectly addressed the issue of planning a collision-free path for a team of robots [34,35].

Eight electronic databases were selected after considering expert opinions on searching literature. At the same time, another critical feature in selection was the availability of the most reliable and relevant data. The following is the list of selected databases: Scopus, Web of Science, Springer Link, IEEE Xplore, ACM Digital Library, and ProQuest. The academic search engine Google Scholar was also used to expand the search, as it is one of the most comprehensive sources. The keywords used for the search string were “Collision Avoidance,” “Decentralized,” “Multi-robot,” “Drone,” “Aerial,” and “Rotor.” Appropriate Boolean operators were also utilized to ensure meaningful search results. “AND”,” OR” and “NOT” operators were used to narrow down the search, while the “-” operator was used in Google Scholar, which is equivalent to NOT. As every database has its search algorithm, the keywords and Boolean operators were accordingly modified. Besides the operators, each database was searched with a slightly different arrangement of keywords.

The screening process followed to retrieve the most relevant articles involved three steps: initially, 852 papers were retrieved from all the databases. After reviewing for duplication 692 papers were left. To narrow them down further, each study’s title in this review was scanned to ensure it was related to the topic. Papers that had keywords in their titles pointing to different issues than multi-agent collision avoidance were excluded. After careful screening, 121 papers were left from 692, which were included for further investigation. For further screening, the abstract and keywords were scanned more deeply for a better understanding of the papers which were more relevant to the topic. The inclusion criteria for the papers for second screening are shown in

Table 1. Inclusion and exclusion criteria for article screening.

Inclusion Criteria	Exclusion Criteria
Refer to a collision-avoidance technique for a multi-agent system	Addressing collision avoidance for agent other than ground robots
A decentralized approach is followed	The technique followed is not based on decentralized methods
The solution proposed performs better than the conventional approaches	The technique proposed is only tested for two agents
Consider multiple scenarios for the experiments	Using the keywords without enough information
Published as journals articles, conference papers, and book chapters	Focus on path planning or trajectory optimization as the main topic
Are written in English	Include experience papers, forums, comments, tutorials, opinions, or discussions.
Published in or after 2015

.

A second screening resulted in 42 papers which underwent full-text screening for quality appraisal and to ensure their relevance to collision avoidance in a multi-robot system. We analyzed whether papers included put the right amount of emphasis on the topic and adequately answered the research questions. The main focus of the analysis was each paper’s aim, description, and methodology. A comprehensive summary of the screened papers was assembled to answer specific research questions thoroughly. This final screening resulted in 17 papers out of 852, which fulfilled the entire criteria for inclusion in this review. The screening process for the number of articles included from each database after each step is shown in Figure 1.

3. Primary Studies on Collision Avoidance in the Multi-Robot System from 2015 to 2021

The main focus of the analysis was each paper’s aim, description and methodology, where comprehensive summaries of the screened papers were assembled to answer specific research questions thoroughly. The overview of the systematic screening for each source is presented in

Table 2. Overview of the systematic screening.

Source	Initially Retrieved	After Removing Duplicates	After First Screening	After Second Screening	After Third Screening
Scopus	108	87	22	11	5
WoS	78	17	9	5	4
Springer Link	219	206	25	3	0
IEEE Xplore	169	134	34	14	3
ACM Digital Library	36	34	8	1	0
ProQuest	242	214	23	8	5
Total	852	692	121	42	17

.

We present our analysis for the distribution of year published (Figure 2), countries (Figure 3), publication types (Figure 4), and statistical results from the papers. When distributing over databases, it was observed that most of the publications, including the keywords, were held by ProQuest. The literature search performed in this review started in 2015, showed no papers published in 2016 or 2018 on this particular topic.

After careful screening, the resulting 17 studies were considered the primary works on decentralized collision avoidance for multi-robots during the last seven years. The papers were divided into two classes: analytical and empirical. Careful analysis revealed that most of the studies used a combination of analytical and empirical approaches. Furthermore, 2 of the 15 studies used the theoretical approach to validate their results. They compiled some theorems and proved them to validate their proposed model [36]. The rest of the studies used both approaches to prove their proposed solutions. The validation of the proposed algorithms was conducted using different simulation setups. Only one of the studies used real-time experiments to validate the model. None of the studies used the quantitative approach, as it was unsuitable for the issue under consideration. Three studies innovated and applied existing models and used a mixed approach to validate their proposition [37,38,39]. No specific studies focused on purely qualitative approaches or case studies, as these methods are not suitable for the topic. The topic of collision avoidance in a multi-robot system needs a more practical approach than a theoretical one and needs experiments to apply and validate the results.

4. Characteristics of Collision Avoidance in Multi-Robot System Techniques

This section discusses RQ2, involving the main characteristics of various collision-avoidance techniques using reinforcement learning (RL) and non-reinforcement learning (non-RL) for a multi-robot system. It was observed that only a few studies address the said issue directly, while others propose a path-planning algorithm that avoids collisions. Each study considered the hardware specifications of the agent to implement the proposed model effectively. Different real-world scenarios were evaluated to narrow the simulation and real transition gap. Following is a summary of each related study. The summary of the technique experimental design, benchmark, strength, and weakness on all selected studies are summarized in

Table 3. Synthesis analysis of decentralized collision avoidance.

References	Technique	No of Agents	Experimental Design	Benchmark	Strength	Weakness
Non-RL approaches
[33]	Continuous Best-Response Approach	4	Indoor but open space maps	ORCA	Shorter paths	Prolongation issue
[30]	Push and rotate with bounded-suboptimal ECBS	4	Open Space	ORCA	Higher success rate	N/A
[39]	Voronoi cells superimposed with RVO cones	4, 10, 25, 70	Open space	ORCA	Passive collision avoidance guarantees	Not optimized
[40]	Safe Zone Discrete-ORCA-MPC	4	Open space	PID, PB-NC-DMPC	Non-holonomic robots	Not suitable for the unknown scenario
[41]	Alturistic coordination	8	Indoor and confined spaces	PID	Local adjustments capability	Ignore low-level control parameters
[36]	Modified MR-DFS strategy	2	Open space	MR-DFS	Reduces edge traversals	Only static obstacles
[5]	Velocity-based detection regions	Up to 20	Open space	Constant detection radii	Yield faster and less conservative trajectories	N/A
[34]	Spot Auction-based Robotic Collision Avoidance Scheme (SPARCAS	Up to 500	Open space	M*	Prioritization and dynamic handling	Not optimized
[42]	Independent virtual center points	2, 6, 8	Open space	Monte Carlo	An arbitrary number of agents	N/A
RL approaches
[13]	CADRL	2, 4, 6, 8	Open space	ORCA	Higher path quality	Stuck in a dense setting
[37]	RL-ORCA	Up to 42	Open space	ORCA	Congestion and deadlock capability	N/A
[2]	Hybrid-RL	Up to 100	Open space with obstacles	NH-ORCA, RL	High success rate and generalization capability	Not socially compliant
[12]	Multi-sensor Fusion and DRL	4	Open and confined space	A*, D*	Dense environment	Oscillatory behavior in a highly spacious environment
[43]	Deep Q learning—CNN	4	Open and confined space	A*, D*		Unnecessary movement occurs
[44]	End-to-end DRL policy	Team of 3	Open space	Ind-PPO, MAPPO	Navigate through a dense environment	Unable to address deadlock situation
[38]	CBF-MARL	2	Open space	MADDPG	Safety guarantees	Only static obstacles

.

For non-RL approaches, eight different studies are selected with various agents in both open and confined spaces. Most studies used ORCA as their benchmark evaluation. Cap et al. [33] address the problem of finding a collision-free trajectory for an agent in a dynamic environment. The setup considered is an infrastructure with agents already performing tasks when a new task is assigned to an individual agent. The proposed algorithm implements a token system and plans a global trajectory considering all the agents. Meanwhile, Dergachev et al. [30] suggest coordinating sub-groups of the agents that appear to be deadlocked, using locally confined multi-agent pathfinding (MAPF) solvers. The limitation of this model is its assumption that each agent has prior knowledge of the environment and its failure to perform in uncertain situations. Arul et al. [39] used buffered Voronoi cells (BVC) and reciprocal velocity obstacles (RVO) to develop a collision-avoidance method for dense environments. To calculate a local collision-free path for each agent, first, a suitable direction is computed by superimposing BVC and RVO cones together. However, this method does not guarantee deadlock resolution, similar to earlier decentralized methods—more studies focus on alternative approaches to avoid the need for global communication among robots [33,41,45].

Mao et al. [40] presented a collision-avoidance approach by considering the non-holonomic constraints of the agents. The proposed method is cheaper than PB-NC-DMPC, as it does not use central coordination or rely on communication among the robots. Another study by Wei et al. [41] proposed altruistic coordination where each robot is ready to make concessions whenever in congested situations. It is demonstrated that when robots face a congested situation, they can implement waiting, moving forward, dodging, retreating, and turning-head strategies to make local adjustments. Another approach using robot graph exploration is proposed by Nagavarapu et al. [36], in which no direct communication is needed to avoid collisions. A data structure is proposed to provide efficient information exchange. Modified Multi-Robot Depth First Search (MR-DFS) strategy is used to achieve better execution than other tree strategies. Zhang et al. [42]. suggest a technique using two cooperative strategies to decrease the effective detection regions of the vehicles, for a random number (large number) of agents, using velocity information. Another approach using prioritization is presented by Das et al. [34], where agents intentionally disclose their information in order to become prioritized. Competitive robots take part in spot auctions, where they show their willingness to pay the price to obtain access to the desired location. The results show that the proposed method can manage dynamic arrival without compromising the path-length optimality too much. Zhang et al. [42] propose an obstacle-avoidance method that incorporates virtual center points, implemented in a distributed manner, which is set based on the current state of the nearby robots and the agent itself. The stability of the system is proved using a Lyapunov function. Two control modes—the obstacle-free mode and obstacle avoidance mode—are used for robots, which are switched carefully using a direct signal.

Researchers have applied several reinforcement learning approaches for decentralized collision avoidance. Chen et al. [13] developed an innovative method that applies deep reinforcement learning to offload the online computation to an offline learning procedure to predict interaction patterns. A value network that uses each agent’s joint configuration is developed to estimate the time to the goal. This value network also finds a collision-free velocity vector by admitting the efficient queries while considering other agents’ motion uncertainty. However, some robots become stuck when the obstacle field is dense such that various traps and dead ends are formed. One effective method to resolve the dead-end issue is presented by [44]. Meanwhile, Li et al. [37] presented a continuous action space-based algorithm. In this method, only the positions and velocities of nearby agents are observed by each agent. A solution of simple convex optimization with safety constraints from ORCA is implemented to resolve the multi-robot collision-avoidance problem. The training process of the proposed approach is much faster than other RL-based collision-avoidance algorithms. Fan et al. [2] designed a decentralized sensor-level collision-avoidance policy. The movement velocity for an agent’s steering commands is directly drawn from raw sensor measurements. The technique used here is policy-gradient-based reinforcement learning. The said technique is integrated into a hybrid control framework to improve the policy’s robustness and effectiveness. Liang et al. [12] used a depth camera with 2D LIDAR as multiple perception sensors to detect nearby dynamic agents and calculate collision-free velocities. The previously unseen virtual and dense real-world environment is directly transferable from the navigation model learned by the agents. However, in the case of glass or nonplanar surfaces, the sensors fail to perform accurately.

Bae et al. [43] suggested a combination of Deep q-learning and CNN algorithm. This combination enhances the learning algorithm and analyzes the situation more efficiently. Depending on the given situation, the agents can act independently or collaboratively. The memory regeneration technique is used to reuse experience data and reduce correlations between samples to improve data efficiency. The presented method uses image-processing techniques such as object recognition [46,47] to obtain the robot’s location. However, an unnecessary movement occurs in an environment where the generated path is simple or without obstacles. Lin et al. [44] propose a method with a geometric centroid of the robot team, which avoids collisions while maintaining connectivity using Deep RL. The proposed model can sometimes fail to predict the dead-end scenario effectively, which can cause the agent formation to take extra time to reach the goal. Cai et al. [38] suggest a combination of Multi-robot Reinforcement Learning MARL and decentralized Control Barrier Function (CBF) shields based on available local information. They developed a Multi-robot Deep Deterministic Policy Gradient (MADDPG) to Multi-robot Deep Deterministic Policy Gradient with decentralized multiple Control Barrier Functions (MADDPGCBF). Each agent has its unique CBFs according to the proposed approach. These CBFs involve cooperative CBFs and non-cooperative CBFs, which deal with the respective types of agents.

4.1. Experimental Setup for Decentralized Multi-Robot Collision Avoidance

This section discusses the main characteristics of the experimental approach and hardware specifications of the agent to implement the proposed model effectively. Furthermore, different real-world scenarios were tested to narrow the simulation gap to real-time transition. Based on the selected papers, researchers considered several essential scenarios: circle, random, roundabout, dense crowds, narrow corridors, obstacle shapes, intersection, edge traversals, deadlock situations, static, dynamic obstacle, and indoor environments. The environmental setup scenarios’ platform, and the number of robots used by each study are summarized in

Table 4. Environment setup for decentralized collision avoidance.

Simulation Setup	Platform	No of Robots	References
Circle and random	Stage, OpenAI Gym	3, 70, 100	[2,37,39,44]
Roundabout	ROS, Python	10–500	[34]
Dense crowds	ROS, Gazebo, OpenAI Gym	4	[12,44]
Narrow corridors	ROS, Gazebo	4	[43]
Obstacle shapes	OpenAI Gym	3	[44]
Intersection	Matlab	4	[40]
Edge traversals	ROS, Matlab	2–10	[12,36]
Deadlock situations	BW4T simulator	1 to 8	[41]
Static and non-cooperative	Python	2, 4, 6, 8	[13,43]
Dynamic obstacle	C++	4	[43]
Three scenarios	ROS, Turtlebot	4, 6, 8	[42]
Three environments (Hall, warehouse, office)	Maven, R	up to 35	[33]
Four-grid map	C++	5 to 40	[30]
Vehicle sensing	N/A	4, 8, 12, 16, 20	[5]

. The distributions of the papers per number of robots and environment are illustrated in Figure 5 and Figure 6.

4.2. Evaluation Measures

Our analysis presents nine main evaluation parameters used by previous studies as presented in

Table 5. Measurement matrices for multi-robot collision avoidance.

Measures	Definition	References
Success rate	$\frac{m}{N}$, m is the # times reach the goal without colliding with an obstacle, and N is the total attempts.	[12,30,33,36,41,44]
Travel Distance	Sum of linear movement segments over times intervals, over the total number of attempts.	[12,30,34,37,38,39,40,41,42,43]
Time cost	Mean time to reach the goal over all attempts.	[13,39,41,42,44,48]
Velocity	Average velocity over the total attempts until a goal is reached or a collision occurs.	[12,33,43]
Position error	Difference between the position of actual reached position and desired position.	[37,41]
Minimum separation distance	Euclidean distance to the current waypoint on the agent’s geometric path, πcur.	[13,30]
Cumulative reward	Total discount factor times rewards of all agents at each time step.	[38]
Planning time	Time taken to compute the offline path and the time of the online spot.	[34]
Search scope	Total of search ranges to move to the target point.	[43]

. We observed that the four main parameters for multi-robot collision avoidance are success rate, travel distance, time cost, and velocity. These parameters are crucial to evaluate the efficiency and successful implementation of the approaches. The other measurements are position error, minimum separation distance, cumulative reward, planning time and search scope.

5. Collision-Avoidance Techniques to Solve Coordination Issues

To answer RQ3, we analyzed the primary works presenting collision-avoidance techniques for multi-robot systems while considering the coordination issues. The techniques studied in this work are focused on presenting solutions to the said problem for a group of more than two agents. Many studies on collision avoidance in a multi-robot system made a list when first searched. Most of these studies addressed manipulators or warehouse scenarios, which were unsuitable for this work. Studies that addressed the type of agents other than ground robots were excluded, which resulted in around 121 studies. These studies were further screened by excluding those that focused on aspects of the multi-robot system other than collision avoidance, such as trajectory optimization or navigation issues [10,49]. Studies that addressed only the coordination issue without considering collision avoidance were excluded [7]. The final screening resulted in 17 carefully selected studies for relevance to the topic, each presenting a unique collision-avoidance strategy.

Collision avoidance is an essential part to be considered when dealing with multi-robot systems. It includes collisions with obstacles and among the agents. Two types of methods most often used are centralized and decentralized approaches. The decentralized approach is computationally inexpensive and enables the agent to be more independent. Another division is classical and reactive approaches [50]. However, this review provides information about effective decentralized techniques, classical or reactive. This review is designed according to the guidelines provided by Okoli [31], which resulted in the final selection of 17 papers. All the studies included are focused on finding and developing an effective strategy to avoid collisions in a multi-robot system. Several methods, including deep reinforcement learning, fuzzy logic, and supervised learning, are used as a base to validate or apply the proposed strategies. Most of the studies are focused on application in an open-space environment with static or dynamic obstacles, while confined-space scenarios are not often studied. Environments such as AC ducts and sewers need to be explored even more, as these environments offer different types of hurdles for a multi-robot system compared to open-space environments. Only 2 of the 17 studies addressed the deadlock situation [2,5], which can appear when agents need to swap their positions or cross a narrow entrance, while others fail to perform in such scenarios [45]. Some of the crucial criteria for decentralized multi-robot collision avoidance are summarized as the following:

Coordination strategy: Several efficient coordination systems have produced successful collision avoidance for multi-robot systems. The velocity obstacle method allows robots to transmit and receive each other’s states and intentions via an altruistic coordination network. A token-passing technique based on a synchronized shared memory holds all robots’ current trajectories, which learns a value function that completely encodes collaborative behaviors. By utilizing the processed data from the LIDAR sensor, agents coordinate with others via a robot team that works as a centroid or beacon to exchange information among the robots. In the decentralized control barrier function, local information received through the agent’s sensors can be shared by other robots nearby.

Traversable region detection region: A unique application of the locally confined multi-robot pathfinding (MAPF) solvers is suggested by Dergachev et al. [30]. This approach presents a way to build a grid-based MAPF instance to avoid deadlocks. Through the learned policies, the robots use local observations of each robot and traversable detection region to collaboratively plan the move to accomplish the team’s navigation task. A proper data-structure-based technique is essential for providing efficient information exchange, as suggested by Nagavarapu et al. [36]. Combining 2D multiple perception sensors using LIDAR and depth cameras enabled the agents to sense the dynamic agents in the surroundings and compute collision-free velocities. An approach using Voronoi cells and RVO cones provides an efficient calculation of collision-free direction for each agent. On the other hand, exploiting the velocity information in the proposed technique resulted in less complicated and collision-free trajectories by the traversable region-detection region. Furthermore, incorporating virtual center points implemented in a distributed manner should be set based on the current state of nearby robots and the agent itself.

Optimal multi-robot trajectories: While ensuring optimal collision-free navigation, agents are also required to maintain a coordination link within their coordinated network to lower the communication overhead in decentralized systems. One of the essential factors is detecting nearby dynamic agents and calculating collision-free velocities. Early detection and trajectory prediction will result in a higher success rate with less time and trajectory length taken to the goal. Detection of the obstacle’s direction and assuming it to be the leader, the agent can be trained to maintain a predefined distance using formation control.

Adaptability: One of the essential criteria for successful application is adaptability, especially in deadlock situations that often occur in multi-robot systems. Few studies faced a problem where agents become stuck when deployed in a dense environment. In some studies, the problem of failing to navigate in an uncertain environment also arose. In some scenarios, only static obstacles are considered while training the agents, making it challenging to address inter-agent collisions. Overall, it is observed that not a single study was able to address all the issues faced when designing a collision-avoidance algorithm. However, deadlock and inter-agent collisions are two main issues that need to be addressed to develop an efficient collision-avoidance model.

6. Limitations of the Current Research on Collision Avoidance in Multi-Robot Systems

To answer RQ4, we discussed the challenges with proposed methods in previous studies. The most common issues mentioned were dealing dynamic obstacles [40], congested situations [41], narrow passages [33], convergence time, deviation [5] and deadlocks [2,42]. These issues are reported to occur when obstacles are other agents, more than one agent at one spot, the path not being wide enough, finding the optimal path, and more than one agent wanting to cross the same spot. The listed issues are addressed by testing the proposed algorithms in multiple scenarios with different levels of complexity, and promising results are reported. The summary of challenges and future studies on decentralized multi-robot collision avoidance is presented in

Table 6. Summary of challenges and future studies on multi-robot collision avoidance.

Challenges	Context
Decision-Making Dynamic Obstacles	Other robots in the system
Congested situations	Obstacle rich environment
Narrow passages	A path where two robots cannot pass each other without one stopping
Convergence time	Rush to reach the goal
Less deviation	Deviation from optimal path to avoid collisions
Deadlocks	Same goal and one path available

.

Decision-Making Dynamic Obstacles: There can be two types of obstacles in a multi-robot system: static and dynamic. Dynamic obstacles include other agents in the team as well. These obstacles can cause a problem as each agent can make their own decision in a decentralized approach; more than one agent can decide to take the same action. This problem was explored by Mao et al. [40] using linear equations of motion to design the state-space.

Congested situations: Congested situations are not widely studied when considering a multi-robot system. It is highlighted by Wei et al. [41] that even when a researcher considers a congested scenario, the agents are still connected to a shared database such as a reservation table [51] and conflict-avoidance table [52]. Multiple scenarios with very congested situations were successfully tested by Chen et al. [13] using Multi-Robot Cooperative Pathfinding (MRCP) based on the ideas from Ryan et al. [53].

Narrow passages: When developing a collision-avoidance strategy for multiple agents, it is preferred to consider some real-life scenarios that an agent can encounter. One such scenario is a narrow path, where crossing or taking-over other agents is quite tricky. This can become even more complex if the path has obstacles. Some solutions such as prioritized planning [54] and complete polynomial algorithms [55,56] were presented, but they fail in complex environments. An innovative solution for this issue is presented by Čáp et al. [33], in which they suggest using an auction system for prioritized path planning.

Convergence time: Convergence is defined as a point when an agent completes an optimal trajectory without collisions. One strategy for a lower convergence time using Lyapunov-based analysis [57] was proposed and was extended to multiple cooperative avoidance control strategies [58]. The main drawback for these techniques was that they fail to perform in situations with an arbitrarily large number of agents [59,60,61], or the paths are too conservative [58,62,63]. This challenge was the main focus in [5], where they proposed a collision-avoidance strategy with optimal convergence time using velocity-based detection regions.

Less deviation: Optimal collision-free trajectory is required to be as close to the ideal trajectory as possible. Standard deviation measures this aspect; lesser deviation indicates that the trajectory is closer to the ideal path. A method was proposed to ensure the smaller deviation by [58], but it was not suitable for a large number of agents. By developing collision avoidance for fully-actuated Lagrangian systems, the authors in [5] successfully addressed the issue of deviation from the ideal path.

Deadlocks: For a multi-robot system, if the environment has intersections, a small entrance into a room, or a narrow-shared path, there is a high chance of a deadlock situation arising. This situation occurs when more than one agent wants to cross each other, but there is not enough space for them to pass. Some centralized methods [64,65,66] are proposed, but they cannot be implemented effectively to a large group of agents. Taking inspiration from the hybrid control framework proposed by Egerstedt et al. [67], a solution for operating many agents in the scenarios mentioned above is proposed by Fan et al. [2]. They successfully trained up to 100 agents. Another solution for deadlocks is presented by Dergachev et al. [30] using MAPF.

7. Future Directions for Multi-Robot the Collision-Avoidance Studies

Above all the issues raised above, there are several key highlights for future studies on decentralized multi-robot collision avoidance, as summarized in

Table 7. Summary of future studies on multi-robot collision avoidance.

Future Studies	Context
Adaptability	Ready to be implemented in any environment
Complex scenarios	Situations with large number of robots and narrow paths
Dense obstacle field	Obstacle-rich situation with both static and dynamic obstacles
Outdoor Settings	Situations such as roads, sidewalks and related environment
Heterogeneous Agents	A team of ground and aerial robots or water and aerial robots
Curvilinear Paths	Paths with sharp and blind turns
Larger state space	The situation as close as possible to real-world situation
Safety and performance	Only focusing on safety and ignoring the path quality

. To answer RQ5, the following eight future directions are suggested:

Adaptability in an uncertain environment: Most researchers aim to make their collision-avoidance algorithm easily generalized to any uncertain environment. Many tasks involving multi-robots, such as search and rescue, duct exploration, and cleaning a chemical spill, have a high percentage of uncertainty in the environment under consideration. Furthermore, 2 of the studies from selected 17 admitted to omitting this issue from their current study and aim to address this issue in their following research [13,27,40].

More complex scenarios: In Čáp et al. [33], paths with wide enough space for passing more than one agent are tested, such as a warehouse, office room, and other areas. So, they want to expand their work by improving the proposed method to apply to more complex scenarios such as confined spaces. At the same time, Fan et al. [2] admitted that their approach could not compete with a global path planner. Incorporating classical mapping methods (e.g., SLAM [68]) and global path planners (e.g., RRT and A) with their algorithms will be the goal in the future.

Dense obstacle field: The future direction of exploring environments with more obstacles and even dynamic obstacles is stated by Chen et al. [13]. They declare their proposed algorithm a collision-avoidance algorithm but not a path-planning one. It performs only in environments with few obstacles and open space.

Outdoor Settings: Mobile robot systems can be used for outdoor exploration, such as in Fan et al. [2] and Erdmann et al. [54]. However, Mao et al. [40] only considered indoor settings with static obstacles when designing a collision-avoidance algorithm. Overcoming the freezing robot issue and implementation in an outdoor setting is declared the future goal by Liang et al. [12].

Heterogeneous Agents: In practical applications, multi-robot systems may consist of heterogenous agents such as turtle-bot and fetch robot. Most of the studies analyzed in this review have focused on homogenous agents. Only Zhang et al. [42] stated that their future work would aim to implement the proposed collision-avoidance problem to heterogeneous agents.

Curvilinear Paths: When moving on curvilinear paths, more collisions are possible as agents may slip. A piece-wise linear approximation can resolve this issue to a curved edge between two vertices resulting in multiple virtual vertices with linear edges. This issue of inter-agent collision considering curvilinear paths is the future direction chosen by Nagavarapu et al. [36].

Larger state space: There may be complex dynamics when an environment is closer to real-world situations. Complex dynamics lead to larger state space. It is difficult and time-consuming to differentiate between relevant and irrelevant actions in ample state space to achieve an effective collision-avoidance solution. An optimal solution to deal with ample state space and collision avoidance for the multi-robot system is addressed by Li et al. [37].

Balance safety and performance: Studies are needed to examine the safety of the agents and how to keep them from collisions, without considering optimal trajectories too much, as highlighted by Cai et al. [38]. Further studies should consider algorithms that can balance the multi-robot system’s safety and performance.

8. Conclusions

This study conducted a systematic review of different RL and non-RL decentralized collision-avoidance techniques for multi-robots. The main goal of this study was to summarize state-of-the-art decentralized collision-avoidance techniques for multi-robots. The studies in this review were selected from well-reputed public databases such as Springer, IEEE Xplore, WoS and other sources. The outputs of these databases were filtered through a series of screening steps to exclude any irrelevant studies. It was observed that most of the studies used non-RL approaches to address the issue. Furthermore, China has conducted the most studies on decentralized collision avoidance for multi-robots in the last seven years.

In general, the summarized studies related to the topic were analyzed based on using RL and non-RL approaches. Algorithms such as Discrete ORCA-MPC, CADRL, MR and MR-DFS and other approaches have been used to solve the problem successfully. Then, the success rate of the technique proposed, and the coordination strategy used, were analyzed and discussed. It was observed that only 2 out of 17 studies considered confined-space scenarios, while others implemented collision-avoidance strategies in relatively open and empty space. Few studies implemented their algorithms in real-world scenarios, while others focused on simulations. Scenarios such as deadlocks or swapping in a narrow path are not widely studied. The most used algorithms are different modified versions of ORCA implemented in relatively open spaces. All of the techniques proposed, plus benchmark algorithms, strengths and weaknesses are presented and discussed as references for future research. The data used in this review are up to date as per our knowledge and any omission is unintentional.