*Review* **Review of Collision Avoidance and Path Planning Algorithms Used in Autonomous Underwater Vehicles**

**Rafał Kot**

Faculty of Mechanical and Electrical Engineering, Polish Naval Academy, 81-127 Gdynia, Poland; r.kot@amw.gdynia.pl

**Abstract:** The rapid technological development of computing power and system operations today allows for increasingly advanced algorithm implementation, as well as path planning in real time. The objective of this article is to provide a structured review of simulations and practical implementations of collision-avoidance and path-planning algorithms in autonomous underwater vehicles (AUVs). The novelty of the review paper is to consider not only the results of numerical research but also the newest results of verifying collision-avoidance and path-planning algorithms in real applications together with a comparison of the difficulties encountered during simulations and their practical implementation. Analysing the last 20 years of AUV development, it can be seen that experiments in a real environment are dominated by classical methods. In the case of simulation studies, artificial intelligence (AI) methods are used as often as classical methods. In simulation studies, the APF approach is most often used among classical methods, whereas among AI algorithms reinforcement learning and fuzzy logic methods are used. For real applications, the most used approach is reactive behaviors, and AI algorithms are rarely used in real implementations. Finally, this article provides a general summary, future works, and a discussion of the limitations that inhibit the further development in this field.

**Keywords:** collision avoidance; path planning; obstacle detection; autonomous underwater vehicle; artificial intelligence; autonomous navigation

#### **1. Introduction**

In recent years, a significant increase in the number of unmanned underwater, ground, and air vehicles (UxVs) in various environments has been observed. Most often, these vehicles are autonomous with different levels of autonomy. When speaking of autonomous vehicles, most will refer to air or ground vehicles. Less widespread and commercialised, but equally important are autonomous underwater vehicles (AUVs). This type of vehicle can be used for ocean exploration, performance of various industrial operations, or military missions. Due to the limited adaptability of humans, diving to very deep depths may be impossible or require a lot of time and preparation. In some cases, like observing the seafloor or looking for damage to pipes running along the bottom of a water reservoir, using AUV can bring many benefits to humans. To perform underwater operations properly and safely, it is necessary to equip the AUVs with all the essential systems, such as obstacle detection systems, motion- or path-planning systems, collision-avoidance systems, trajectory-planning systems, mapping systems, mission planners, path-tracking systems, etc. The underwater environment poses many difficulties for the vehicles' movement, i.e., high signal attenuation, limited possibilities of spatial orientation in relation to reference points, the presence of sea currents, limited access to light, and disturbing signal reflections from the bottom and surface of the water. Therefore, some motion-planning solutions implemented for robots or other groups of autonomous vehicles may not be appropriate for AUVs. The rapid technological development of computing power and system operations today allow for more and more advanced algorithm implementation as

**Citation:** Kot, R. Review of Collision Avoidance and Path Planning Algorithms Used in Autonomous Underwater Vehicles. *Electronics* **2022**, *11*, 2301. https://doi.org/10.3390/ electronics11152301

Academic Editor: Zhiyun Lin

Received: 4 July 2022 Accepted: 21 July 2022 Published: 23 July 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

well as path planning in real time. As one study [1] indicated, increasing the computing power through hardware can significantly increase the data-processing speed and system efficiency compared to the implementation of more efficient software. Reliable performance is considered a crucial condition for AUV design. Although simulation analysis of collisionavoidance algorithms and path planning usually brings satisfactory results, real vehicle implementation has caused many difficulties in past studies [2]. As scientific and technical knowledge advance, researchers attempt to systematise current achievements in this field. In [3], motion-planning approaches for robots from the years 1980–2010 were analysed, indicating how the number of heuristic algorithms changed in individual decades compared to conventional algorithms. The basics of classical and heuristics-based methods for robots were discussed in [4]. The authors of [5] analysed 80 articles from 2011–2015 on motion planning in a dynamic environment for robots and systematised individual solutions in terms of parameters such as smooth path, safety, path length, run-time, accuracy, stability, control, computation cost, and efficiency. The study [6] reviewed 3D path-planning algorithms for robots, focusing on the universality of individual algorithms implementation in aerial, ground, and underwater robots. Reference [7] elaborated on heuristic algorithms for robots, showing examples of their efficient and inefficient operation. Reference [8] reviewed sampling-based motion-planning algorithms for robots, and [9] focused on probabilistic roadmap approaches in dealing with dynamic collision, narrow passages, multi-targets, and manipulation planning for deformable linear objects. Other studies also summarized the achievements in the field of quadrupedal robots [10], AGVs (usually road vehicles) [11,12], coverage path planning for robots [13], and searching methods for environmental monitoring [14]. Together with single robots, multi-robot performance has also been studied. Reference [15] discussed navigation techniques for the single and multi-robot systems in both static and dynamic environments. In addition, reference [16] analysed the techniques of movement of the robot group depending on the shape of the formation (cluster/line flocking). The authors of [17] reviewed the available literature in terms of the multi-robot movement in normal and faulty situations, and [18] focused on methods determining desired trajectories for multi-robots. In the path-planning studies, some researchers have been focusing on solutions designed for specific types of vehicles depending on the environment in which the vehicle is moving. In [19,20], the algorithms designed for unmanned aerial systems (UASs) were analysed, and in [21–23] algorithms for unmanned aerial vehicles (UAVs) were discussed. Reference [24] discussed vision-based algorithms for UAVs, and [25] for multiple UAVs. Many reviews in the field of underwater robotics have also been published. In [26], the technological development and limitations that are encountered in marine robotics are analysed. The authors of [27] define the trends in construction, instrumentation, and systems for unmanned underwater vehicles (UUVs). The UUVs can be remotely operated vehicles (ROVs) and AUVs. In [28], the main systems for both a single AUV and multi-AUV are described, as well as the possible directions of AUVs' further development. In terms of motion control of a UUV, reference [29] reviews the development of control methods and the problems encountered during the research. In turn, reference [30] provides detailed literature that analyses path-planning-related solutions for AUVs. The main surveys connected with motion planning for robots are presented in Table 1.

The objective of this article is to provide a structured review of simulations and practical implementations of collision-avoidance and path-planning algorithms in AUVs together with a comparison of the difficulties encountered during simulations and their practical implementation. The paper is organised as follows. In Section 2, the most essential pathplanning and collision-avoidance algorithms for AUVs are explained. Section 3 includes a summary of simulation and practical research in this area. In Section 4, the general summary, future works, and limitations that inhabit the further development in this field are discussed. Final conclusions are presented in Section 5.


**Table 1.** Surveys connected with motion planning for robots.


**Table 1.** *Cont*.

#### **2. Fundamentals of Path-Planning Algorithms**

This section presents the main path-planning algorithms for AUVs. The description of each method includes an explanation of the fundamental principles. It also analyses the advantages and disadvantages, environmental limitations, and potential difficulties in real applications. Each approach contains several application examples from different fields of robotics.

#### *2.1. A\* Algorithm*

In 1986, the heuristic method [31] A\* was proposed, which consists of dividing the known area into individual cells and calculating the total cost of reaching the target for each of them. Given the total distance from the starting point to the currently analysed cell and the distance from the analysed cell to the target, the total path cost is calculated with the following formula:

$$F(n) = G(n) + H(n),\tag{1}$$

where:

*F*(*n*) is the total path cost;

*G*(*n*) is the cost of reaching from the starting point to the analysed cell; and

*H*(*n*) is the cost of reaching from the analysed cell to the target.

The A\* algorithm focuses on finding the shortest path to a destination in the presence of static obstacles assuming that both the environment and the location of the obstacles are known. In this method, however, an increased amount of computation is needed for large areas analysis or areas with many obstacles. This significantly increases the path planning time. For the marine vehicles, AUVs and autonomous surface vehicles (ASV), the A\* algorithm method is in most cases used in combination with the visibility graph algorithm [32–34]. In [35], simulations of two grid-based methods in the 3D environment were compared. Compared to the Dijkstra algorithm, the multi-directional A\* proved to be more efficient in terms of the number of nodes and the total length of the path. The practical implementation of this method is associated with the problem of variability of the underwater environment and the lack of precise knowledge about the location of obstacles. Another challenge for practical implementation of this method in AUV may be the unforeseen effect of sea currents.

#### *2.2. Artificial Potential Field*

The artificial potential field (APF) method has its origins in 1986 [36] and assumes the presence of a repulsive field around the obstacle and an attractive area around the target that affect a moving vehicle (e.g., AUV). As a result of the interaction of these virtual fields, the resultant force is determined according to which way the vehicle is moving. In this method, prior knowledge of the environment and the location of obstacles is not needed. It can be used regardless of whether the obstacles in the environment in which the AUV is moving are static or dynamic and whether they have regular or irregular shapes. The crucial condition for this algorithm to be efficient is accurate obstacle detection. The advantage of the APF method is the ease of implementation and low computational requirements, which makes it possible to control AUVs and avoid collisions in close to real time. Despite the many abovementioned advantages, the possibility of local minima or trap situations is considered as a significant disadvantage [37]. Under certain conditions, e.g., a U-shaped obstacle, when the resultant force acting on the AUV is balanced, the algorithm will control the movement of the AUV in a closed infinite loop without reaching the goal. In addition, the passage of the AUV between closely spaced obstacles may not be possible or cause oscillation due to the alternating influence of force fields from opposing obstacles [37,38]. Also, the AUV tends to demonstrate unstable movements when passing around obstacles. In order to solve the local minima problem, APF is combined with other methods, such as [39,40]. In the simulation-based study [41] for AUVs a solution was proposed based on the introduction of a virtual obstacle in the place where the local minimum occurs. Another way to avoid a trap situation can be by using random movements to lead the AUV out of the adverse area (randomised potential field) [42]. Despite the limitations of this method, it is used to control a swarm of robots [43–46]. Simulation-based studies also prove the possibility of controlling multi-AUV formations [47] and its application in mission planners [48]. In [49], the potential field-based method was used in the practical implementation in NPS ARIES AUV in a real-world environment. To avoid the negative impact of APF method limitations in real implementations, it is necessary to use the global path-planner module based on, e.g., heuristic methods.

#### *2.3. Rapidly Exploring Random Tree*

Introduced in 1998 [50], the motion-planning algorithm named rapid-exploring random tree (RRT) is a sampling-based method. Any node in the *Xrand* spatial is randomly determined from the initial point *Xinit* (Figure 1). Then, depending on the given direction of movement and the maximum length of the section from the analysed point, the intermediate *Xnear* node is determined. The longer the sections' connecting nodes, the higher the risk of moving in the wrong direction for a long time and encountering obstacles on the designated path. If any obstacle is detected between waypoints, further route calculation in that direction is ignored. If no obstacles are detected, a random point in the *Xrand* spatial is determined, followed by a new intermediate point *Xnew*.

**Figure 1.** An illustration of the RRT mechanism.

As a result of subsequent iterations (time steps), new edges and path points are determined. The method is easy to process and ensures the finding of a collision-free path (if there is one) in an unknown environment, both 2D and 3D. RRT can be used in both static and dynamic environments [51]. A modification of this method was used in the SPARUS-II AUV and tested in a real-world environment [52]. The study [53] shows the validity of the modified RRT\* algorithm for mapping in a 3D environment for multiple AUVs. In [54], an RRT-based approach was used to solve the problem of local minima in the fast warm starting module in the Aqua2 AUV. It was noted that this method is not always efficient in an environment where the path to the target leads through a narrow opening or gap. Another limitation of this method is the need to provide information about large areas of the environment, which is not always possible in practical implementations due to the technical limitations of sensors [55–57]. Additionally, the calculated path is suboptimal [58] which requires the use of additional optimisation algorithms.

#### *2.4. Artificial Neural Network*

An artificial neural network (ANN) is a machine learning method based on the mathematical mapping of information processing by the human brain. The general structure consists of three layers: input layer, hidden layers, and output layers (Figure 2). The neurons in each layer are connected to all the neurons in the neighboring layer and process the inputs based on the weights in between. The data from each neuron in the input layer multiplied by weights is sent as input to hidden layers where each neuron is assigned a value (bias). Depending on the activation function and bias value, only neurons with a value higher than the threshold value are activated. The output from each layer is also the input for the next layer. The data transfer between the layers is performed only by active neurons. The ANN method requires learning, i.e., indicating the accuracy of the output data. Based on the determination of the expected results, the neural network adjusts the weights between the individual neurons in each iteration in such a way that the output data is as close to the expectations as possible.

**Figure 2.** Neural network architecture.

The method has learning ability and is applicable in systems that implement complex functions, supporting many outputs based on data from multiple sensors. The algorithm is also efficient for the systems where the input data is incomplete or distorted, or in cases in which there poorly modelled, nonlinear dynamical systems [59–61]. The main drawback of this algorithm is the need for long-term training to achieve satisfactory results [62]. In the case of practical implementation for AUV, online learning does not bring satisfactory results due to slow learning speed and long training time [63]. Therefore, offline training for controllers is essential before the AUV can be used in a real-world environment. Reference [64] presents a collision-avoidance controller for static and dynamic obstacles based on neural networks that do not require training. The resulting behaviour of the AUV was defined by neuron weights. It should be noted, however, that this type of controller is suitable for simple AUV operational cases. In [65], the neural network method was used in real implementation in AUV-UPCT. The vehicle required prior learning in ROV mode. In [66], the neural network algorithm was used in a simulation study to control and avoid collisions for multiple AUVs in a 3D environment. In [67], the neural network was used to process real images of the underwater environment (simple, coloured monocular camera) to determine the free space enabling the escape of the AUV from the cluttered environment.

#### *2.5. Genetic Algorithm*

Genetic algorithm (GA) refers to research and optimisation methods inspired by the natural evolution process where only the fittest organisms have a chance for survival. In general, the algorithm starts looking for a solution to a problem by generating a random population of possible solutions. Depending on the applied function, a selection is made during which the least suitable solutions are eliminated [68]. For path planning, the main criterion is the energy cost required to run each path [69]. Then, through an operation called crossover, further potential solutions to the problem (offspring) are created, combining the best solutions from the previous generation (parents). Additionally, in the mutation process, random modifications of the best solutions are created. Then, the GA runs the selection again, adding to the population's successive possible solutions that are closer and closer to the correct result until a specific final condition is reached, e.g., the number of generations. The main advantage of the GA method is the possibility of fast and global stochastic searching for optimal solutions [70]. In addition, the algorithm is easy to implement and can be used to solve complex problems, such as determining the optimal AUV path in the presence of static and dynamic obstacles in the underwater environment. Due to the random nature of searching for solutions, the algorithm reduces the risk of the local minima problem [71]. In general, the method does not require large numbers of calculations to solve the path-planning problem. However, the route may be suboptimal if too few generations are executed. As the number of generations increases, the route becomes closer and closer

to the optimal one due to the constant elimination of the least optimal solutions in the population. It entails an increase in computational costs. Similarly, in a rapidly changing environment or when the environment is very extensive, the amount of computation needed to determine a solution increases significantly. In [69], a modification of the GA was presented in order to determine the energy's optimal path. The modification involved the introduction of iterations consisting of additional runs of the algorithm with different initial conditions and the operator based on the random immigrants' mechanism, which sets the level of randomness of the developing population. Reference [72] discussed the framework of the collision-avoidance system based on the GA. The simulation test proved the proper functioning of the method for static and dynamic obstacles. Improved GA was also used in a simulation-based study [73] to optimise energetic routing in a complex 3D environment in the presence of static and dynamic obstacles. The algorithm with improved crossover and mutation probability and the modified fitness function was tested in simulation [70] in comparison with the traditional GA method. The simulation confirmed that the improved GA allows a shorter and smoother path with fewer generations. The method worked very well in the energy optimisation of routing. It should be noted that the efficiency of this method in real testing depends on the correct detection and determination of the obstacle location as well as the target.

#### *2.6. Fuzzy Logic*

The fuzzy logic (FL) method [74,75] is based on the evaluation of the input data depending on the fuzzy rules which can be determined by using the knowledge and experience of experts. In AUV obstacle avoidance systems, the fuzzy controller sensor processes data containing information about the surrounding environment and makes decisions based on it. Then, appropriate signals are transmitted to the actuators. The first stage of the algorithm's operation is fuzzification, i.e., assigning the input data to the appropriate membership function. Each of the functions is based on a descriptive classification of the input data, e.g., low, medium or high collision risk. Then, in the fuzzy inference process, a data evaluation is performed based on "if prerequisite then result" statements rule. Ultimately, the defuzzification process determines specific system output values (e.g., actuator control signal values). The method can be used for both static and dynamic obstacles. However, when an AUV operates in an unknown environment, the usability of the algorithm directly depends on the implemented rules, and therefore also on the knowledge and experience of experts. The main advantage of the algorithm is its usability in the case of incomplete information about the environment, even containing noises or errors [76]. The method is easy to implement and provides satisfactory results in real-time processing but requires a precise definition of membership functions and fuzzy rules [77]. Additionally, as the number of inputs increases, the amount of input data necessary for the system to process increases. An important issue affecting the effectiveness of the algorithm is also the AUV speed and the complexity of the environment. A fuzzy system usually has at least two inputs. In a very dynamic environment, the use of additional inputs can increase the efficiency of the controller [78]. In some cases, the necessity of performing complex maneuvers to avoid a collision may cause the AUV to be diverted far from the optimal path. For this reason, it is necessary to use additional algorithms that control the path in terms of energy. A simulation study [71] showed that the use of GA to optimise fuzzy logic path planner allows one to achieve greater efficiency and reduce cross-track errors and total traveled path. A modification of this method was also used in a practical implementation in a study [79] where the Bandler and Kohout product of fuzzy relations was used for preplanning in horizontal plane maneuvers. The fuzzy logic method was also used in [80] to control a single unmanned surface vehicle (USV) and in the simulation [81] to control virtual AUVs in the leader–follower formation of AUVs.

#### *2.7. Reinforcement Learning*

The reinforcement learning (RL) method is based on research about the observation of animal behaviour. As a result of strengthening the pattern of behavior by receiving a stimulus by the animal, it increases the tendency to choose actions [82]. Machine learning based on this idea was very intensively developed in the second half of the 19th century. Depending on the requirements of the environment model, the simplicity of data processing, and the iterative nature of calculations, several methods of solving RL problems have been developed, e.g., dynamic programming, Monte Carlo methods, and temporal difference learning. The most crucial method of solving RL problems is the temporal difference method. Depending on the occurrence of the policy function, it is divided into several algorithms, e.g., Actor-Critic, Q-learning or Sarsa method [82]. The actor-critic method consists of autonomous learning of the correct problem solving depending on the received reward or punishment. By performing action, the agent influences the environment by observing the effects of the action. Depending on the implemented reward function, different ways of influencing the environment are assessed. The objective of this method is to learn how to perform such actions in order to get the greatest reward. In the case of path planning, the agent will receive the greatest reward if moving toward the given destination. In general, an agent includes a combination of neural network's actor and critic. The interaction between them is a closed-loop learning situation [83]. The actor chooses the action for performing to improve the current policy. The critic observes the effects of this action and tries to assess them. The assessment is then compared with the reward function. Based on the error rate, the critic network is updated to predict actor network behaviour in the future better. The abovementioned approach was used in [84] to control four robots. Each of them was able to avoid collisions with other robots and with obstacles. In the study [85], the RL approach was used in a two-dimensional simulation of cooperative pursuit of unauthorized UAV by using a team of UAVs in an urbanized environment. In [86], a simulation of smart ship control algorithm was presented. Moreover, in simulation studies [87,88] the RL approach was used for path planning and motion control of AUVs. The method can be used for both static and dynamic obstacles in an unknown environment. It is characterised by a strong focus on problem solving and shows high environmental adaptability. The RL algorithm can learn to solve very complex problems as long as the correct reward function is used. The inability to manually change the parameters of the learned network is considered as the main drawback of this method. To change the operation of the algorithm, the network must be redesigned, and a long learning process must be performed. Also, checking this method's operation by simulation does not guarantee its correct operation in a real-world environment.

#### *2.8. Comments*

Among the classical and heuristic methods discussed above, it is difficult to find one that will work efficiently in various conditions and at the same time meet all the requirements for complete control and for a collision-avoidance system for the AUV. In an ideal scenario, the implemented method should assure high efficiency, real-time response, safety, path optimisation in terms of energy, low complexity, high accuracy, the shortest possible path, and the ability to operate properly in an unknown environment in relation to both static and dynamic obstacles. Each of the methods has its advantages and disadvantages depending on the given conditions. A situation in which the algorithm works very well and one in which the effectiveness of the algorithm will be very low can be found for each of the discussed methods [7]. Researchers still work on improving the parameters of individual methods by modifying them. In order to improve efficiency and eliminate defects, combinations of various algorithms are also used.

#### **3. Chronological State of the-Art**

After analysing the available literature in the field of path planning and collision avoidance for AUVs, a detailed classification of the studies was carried out in terms of chronology and in terms of the technique of testing the effectiveness of the presented algorithms. In each of the 4 intervals (until 2010, 2011–2015, 2016–2020 and from 2021 onward), the studies were divided into 2 groups, as follows.


In each of the abovementioned groups, the research was ordered from oldest to newest in the analyzed period of time.

#### *3.1. Until 2010*

The need for a collision-avoidance system for AUVs has been recognised in the last century. At that time, many researchers proved algorithms that could detect and avoid collisions for static and moving obstacles [64,89]. Studies carried out in early 2000 showed a very intensive development of AUV collision-avoidance and path-planning algorithms. The capabilities of most of them, however, were tested only with simulation-based methods. The list of algorithms validated in numerical research until 2010 is presented in Table 2. At that time, intensive research on obstacle detection and classification were carried and analysed for physical sensor use and in simulation-based methods. In some cases, the algorithms were explicitly created for practical implementation. In [90] a virtual force field (VFF)-based system for the RAIS AUV was introduced and tested by using numerical simulations and hardware-in-loop simulation methods. In [91], researchers proposed a collision-avoidance algorithm through the pitch angle change. This algorithm was intended for physical implementation in the Taipan AUV. The authors of [92] proposed a collision avoidance algorithm based on the prediction of gradient lines (static obstacle) and deformed safety zone (for dynamic obstacles) intended foran AUV called DeepC. The approach was based on determining the trajectory by using the graph method combined with the A\* method. The Redermor test platform equipped with forward-looking sonar (FLS), ten echosounder, and side scan sonar in the [93] study was used to collect sonar data during underwater tests. The authors have been able to classify the obstacles and prove that it is possible to define the boundary level around the obstacle. The implementation of more advanced hardware equipped with FPGAs allowed [1] to significantly increase the speed of data processing and power efficiency. At that time, solutions were also designed to control AUVs moving in formation. The accuracy of the proposed method was tested only in simulation-based analysis. In the study [94], the method based on the potential function and behaviour rules was proven in simulations for 3 AUVs moving in formation. Also, the authors of [81] proposed a well-functioning algorithm based on FL to control the velocity of the AUV formation. An algorithm based on the artificial potential field method [95] was also developed, which has been proven, under simulation conditions, to work efficiently for static obstacles.


**Table 2.** List of algorithms validated in numerical research until 2010.

An example of practical implementation of collision-avoidance algorithms can be found in [49]. In this study, AUV avoided the sunken barge by changing the altitude by using a method based on the artificial Pptential field. Another practical implementation [38] was also performed with MEREDITH. 2D vector field histogram (VFH) based method functionality was tested in a real-world environment. The vehicle was able to avoid a large obstacle which was a breakwater. Also, in 2001–2009, a project code-named Autosub was carried out. At that time, several vehicles from the Autosub brand were made and practically tested during many under-ice missions. The collision-avoidance capability has been proven practically in this mission. In [105], a collision-avoidance module was triggered, and the vehicle correctly managed to avoid the obstacle. At that time, the algorithm worked only in the vertical plane. Over the years, the project's team gained a lot of experience and gradually improved their AUVs capability. Reference [106] presented an improved collision-avoidance system capable of turning in a horizontal plane. The upgraded Autosub-I AUV, after detecting an obstacle, performs a maneuver to change the depth and then turn and move in the horizontal plane. Then, the AUV retreats and tries to avoid the obstacle. After the maneuver, it returns to the previously taken path. As the authors claimed, the algorithm copes well with large obstacles. However, small-sized obstacles and a complex environment pose a significant challenge to the vehicle. The same algorithm was also implemented in another vehicle from the Autosub group—Autosub3. The vehicle operated for four days under the ice and covered 510 km during the mission, proving practical adaptation to even difficult operations [107]. In Table 3 the list of algorithms in real applications until 2010 is presented.


**Table 3.** List of algorithms in real applications until 2010.

#### *3.2. 2011–2016*

In the past decade, researchers focused more often on analysing the combinations of algorithms. In Table 4 the list of algorithms validated in numerical research between 2011–2016 is presented. In the simulation study [77], the particle swarm optimization (PSO) algorithm was used to regulate the membership function in the FL-based method in the AUV motion controller system. In the study [108], the method based on the combination of FL for pre-planning and reactive techniques is simulated. The authors consider collision avoidance only in the horizontal plane. The study [109] analyses the combination of potential field and edge detection to avoid collisions (only in the vertical plane). The method is designed for applications in which the AUV is responsible for inspecting the area (by using side scan sonar), where a straight course line should be kept. In [34], a combination of graph methods with the A\* algorithm was used to determine the path. However, as concluded, that combination does not allow one to determine the optimal path in a completely unknown environment. In [110], the APF-based algorithm was used for local collision avoidance, and the velocity synthesis algorithm was used to optimise the AUV path. Also, modifications of the already existing algorithms were developed. The multipoint potential field method was analysed in [111]. The vehicle is steered in the direction

of the field's lowest resultant value, which is generated in many directions. Additionally, in that time, more and more focus was given to path planning and global collision avoidance. Several control systems and collision-avoidance and path-planning studies have also been developed for multi-AUV formations. In a simulation study [47], a multi-layer region control concept was used to control and avoid both static and dynamic obstacles, and a PD-based regulator controlled the shape of the formation. The authors in [66] used neural networks to simulate the multi-AUV control system in the presence of static obstacles. Other issues closely related to collision avoidance and path planning by AUV were investigated in studies at that time. The authors of [111] performed a simulation in laboratory conditions of AUV with hardware-in-loop and software-in-loop methods with real sensor data (ultrasonic sensor- for distance to the obstacle, pan and tilt angle inputs by the user) and real implementation of actuators. This method is beneficial and effective for verifying the operation of algorithms in real time.


**Table 4.** List of algorithms validated in numerical research between 2011–2016.

In [65], researchers successfully tested a AUV-UPCT control system based on the neural network method in Menor Lagoon. The vehicle has the ability to move in 3D, and obstacle avoidance is implemented only in the horizontal plane. In the study [115], the successful maneuver to avoid a group of four obstacles was accomplished by making a 90◦ turn and bypassing the obstacle, which was the island and the points around it. However, one test resulted in AUV looping due to the local planner not being fully integrated with mission planner. The study [52] tested the possibilities of the SPARUS-II AUV in a real-world environment. In the simulation conditions, a computation time equal to 1 s was assumed, whereas in real-world conditions, it was necessary to change this to 1.5 s. This indicates that the method works quickly but is not intended for a dynamically changing environment. The study [116] proposed an algorithm based on reactive behaviours that allows it to correctly navigate and avoid obstacles in both vertical and horizontal planes. Due to the fact that the collision avoidance in both planes is not parallel, there may be cases of a

non-optimal AUV trajectory. The study discusses the situation in which the AUV, to avoid an obstacle, gradually ascends to the minimum depth and then avoids the obstacle by using horizontal plane behaviours. the list of algorithms in real applications between 2011–2016 is presented in Table 5.


**Table 5.** List of algorithms in real applications between 2011–2016.

#### *3.3. 2016–2020*

In the next five years (2016–2020), there was an increased interest in collision avoidance and control systems for single AUV and multi-AUV formations. Researchers focused their simulation studies on modifying existing methods to improve efficiency. List of algorithms validated in numerical research between 2016 and 2020 is presented in Table 6. In the survey [117], an APF algorithm modification was introduced, consisting of the disappearance of the repulsive force effect on AUV in the case of increasing the distance from the obstacle. For this method, an obstacle is detected only when at less than 80 m distance. In the study [118], the path optimisation determined by the neural network was solved using the Radau pseudospectral method. The authors of [73] presented a solution to modify the mutation operator through a grey wolf optimiser for energy optimisation in path planning. In the modified APF-based method in [119], the amount of data necessary for the correct operation of the algorithm was reduced to provide information only about the distance between the vehicle and the obstacle. The study [78] introduced a three-input FL controller that can provide greater efficiency for fast-moving obstacles than the generally used two-input controllers. In the survey [70], a modified GA is presented by adaptively adjusted crossover and mutation probability and by assessing the diversity of the new population. The collision-avoidance and controls systems for AUV formations were also points of focus at that time. In [120], authors used the fuzzy artificial potential function and leader–follower algorithm for flocking control of group AUV. Another solution based on the neural network method for controlling AUVs and APF for collision detection and avoidance was simulated in the study [121]. Due to hardware and environmental limitations, real-time in-system simulation was performed. The results proved to be similar to the simulation run in the MATLAB environment without using AUVs. Researchers also modified well-known routing optimisation algorithms for multiple AUVs and their energy consumption [122]. Also, many other studies have been focusing on the control and avoidance of collisions for multiple AUV formations, e.g., [123–125], which shows that in the time period considered in this paragraph, there was a significant increase in interest not only in collision avoidance and path planning algorithms for individual AUVs but also a lot of focus has been given to the research of AUV-formation control algorithms. In [126], the authors developed a

practical implementation of the 2D control method based on the Dijkstra algorithm with its testing on the ROV. The authors claimed that the method can also be easily implemented in AUVs. In the study [127], a real-world test of the biomimetic AUV (BAUV) was carried out. The study, however did not bring satisfactory results due to the insufficient accuracy of the navigation system. Also, some false obstacles were detected due to reflections from testing the BAUV in shallow water. The examples of BAUV and AUV are shown in Figure 3. Another study [128] analysed the possibilities of determining the optimal and collision-free path for the Inspection-AUV (I-AUV) with a manipulator in cluttered environments. The authors of [129] confirmed that it is possible to build a collision-avoidance controller for AUVs by using a hydroacoustic obstacle-detection system. Reference [130] analyses the collision avoidance problem for various types of underwater objects (usually vehicles) and various environmental conditions in cases of one-way communication.

**Figure 3.** BAUV examples made at the Polish Naval Academy [own source].


**Table 6.** List of algorithms validated in numerical research between 2016 and 2020.


**Table 6.** *Cont*.

The list of algorithms in real applications between 2016–2020 is presented in Table 7. In the study performed in [79], horizontal maneuvers are preferred over vertical ones. If it is impossible to calculate the path by using horizontal maneuvers, the vertical approach is activated. This approach allows one to avoid collisions in 3D, although in some cases the path on which the AUV moves may not be optimal or is far from optimal. The authors [140] tested AUV in a real-world environment with simulated obstacles in the form of a labyrinth. Therefore, obstacle detection and sensor input processing have been simplified. Nonetheless, the algorithm recalculated the path in around 5 s, which means the AUV might not be able to react appropriately in an emergency. The study [141] discussed an obstacle-avoidance algorithm, changing the AUV altitude when an obstacle appears in

the scanning area. When performing evasive maneuvers, the AUV ascends vertically, and a downward-looking camera and laser monitor the obstacle, memorising the position of the obstacle's peak of the obstacles. After executing the maneuver, the AUV returns to the before-avoidance altitude. In the study [142], the reactive algorithm enforces turning when an obstacle is detected. If not, the AUV is controlled parallel to a wall, keeping a constant distance from it. In [143], the reactive algorithm uses a combination of behaviours depending on the situation: gain altitude, reduce altitude, move forward, move backwards, and escape from hot water. The study includes photographing the bottom of the test reservoir containing hydrothermal springs. When avoiding collisions with seafloor elements, the AUV makes a vertical plane movements only. In [144], extensive simulation and practical tests (under various conditions, including regular and irregularly shaped obstacles) were conducted to confirm the correct operation of the collision-avoidance algorithm implemented on the SPARUS-II AUV. The research proved the possibility of an effective operation of the algorithm in the field of online planning with full 3D control. The authors [54] tested the Aqua2 AUV in a pool with static obstacles. Although offline path planning was successfully accomplished in 3D, and online planning was only realized in the horizontal plane.

**Table 7.** List of algorithms in real applications between 2016–2020.


*3.4. 2021–Now*

In the last two years, simulation methods for route planning and collision avoidance are still being developed. Another aspect that is becoming more important is accurate navigation, together with algorithms for reducing the navigation errors and also methods of tracking the previously determined path [63,145–148]. There is still continuing interest in control and collision avoidance systems for multi-AUV formations [149–151]. To improve the efficiency of simulation algorithms, combinations or improvements to the existing methods are implemented. The list of algorithms validated in numerical research from 2021 is presented in Table 8.


**Table 8.** List of algorithms validated in numerical research from 2021.

In [158], a real-world environment test was conducted in which the AUV, after detecting a fishing net, was to make a turn and then return to the original path. In a result, 13 out of 16 attempts were successful. For the three unsuccessful attempts, the vehicle did not react accurately due to the too high detection threshold to exclude false detections. The MRF-net was used to detect obstacles in real time. The method is suitable for detection of a single static obstacle. The study [159] examined the capabilities of the PI AUV, which is intended for damage inspection of underwater pipes. It was tested whether the AUV correctly detects a leak from the pipe. Also, the anti-collision system operation was tested in which the avoidance expert system takes over the control if an obstacle is detected at a distance of 80 m from the AUV. Depending on the obstacle in relation to the AUV, the appropriate maneuver (turn left/right or bypass left/right) is selected in the horizontal plane. The position of dynamic obstacles was predicted by using the Kalman filter. The authors [160] introduced the path-planning and collision-avoidance systems for the spherical underwater robot SUR IV. The combination of ant colony and PSO approaches allowed the researchers to avoid collisions in 3D, which was confirmed by an experiment in a real water environment. To detect obstacles in this study, a camera was used. It must be noted that placing the SUR IV in a real water environment with low transparency could interfere with the correct detection of the environment and obstacles and as a result, malfunction of the entire system. The list of algorithms in real applications from 2021 is presented in Table 9.



#### **4. Discussion**

This section summarises the results of the literature analysis, containing statistics on the use of path planning and collision-avoidance methods. Quantitative statistics include a division into simulation-tested and real-environment validated methods from 2001 to 2020. It also contains classification according to classical and artificial intelligence methods. The next part of this chapter discusses future works and limitations that inhabit the further development in this field.

#### *4.1. Summary*

After analysing the literature on motion planning and collision avoidance for AUVs, it is clear that a significant increase in publications in this area occurred only in the last decade (Figure 4). The rise in simulation tests of new, improved, or combined algorithms for path planning and collision avoidance is accompanied by an increase in the number of practical implementations in AUV motion controllers and tests in a real environment. However, the practical implementations of the control and collision-avoidance algorithms are not developed as fast as the simulation methods. A practical approach to the study requires dealing with design and technological problems, which are often time-consuming and costly. Additionally, performing tests in a real environment requires thorough preparation and logistical organisation. Analysing the last 20 years of AUV development, it can also be noticed that experiments in the physical environment are dominated by classical methods (Figure 5). In the case of simulation studies, artificial intelligence methods are used as often as classical methods. Although artificial intelligence algorithms have origins in the previous century and allow for solving very complex problems, their use by researchers has not been dominated in further studies on the development of AUV. Figure 6 shows that in simulation studies, the APF approach is most often used among classic methods, whereas among artificial intelligence algorithms RL and FL methods predominate (Figure 7). For real applications, the most used approach is reeactive behaviors (Figure 8), as can be seen in Figure 9. Artificial intelligence algorithms are rarely used in real implementations. This probably results from the fact that these methods need a lots of data to train the artificial intelligence system. Summarizing the above analysis, it can be stated that in the field of collision avoidance and path planning in underwater environment algorithms, researchers still show great interest in classical methods. Thanks to the research to date, many disadvantages of classic methods have been eliminated, which, combined with the ease of implementation, is still an alternative to the often complex methods of artificial intelligence.

The currently designed AUVs are diverse in terms of intended use. Local collisionavoidance algorithms are usually connected to a global system that analyses and optimises the path. This creates the need to develop many side areas of studies, such as the selection of the path-planning method, preplanning, algorithms and drivers for tracking the designated path, navigation and mapping related to determining the most accurate position of obstacles and the AUV. Moreover, in recent years the development of I-AUVs has also been noticeable. I-AUVs are usually equipped with an arm with several degrees of freedom and are designed to perform inspections and repairs, for example of gas and oil installations [161]. When carrying out specific underwater missions, I-AUVs must have collision-avoidance algorithms for the correct task execution and safe movement. For the I-AUV with several arms, it is also necessary to use collision-avoidance algorithms between arms. Implementing that component to the control system, however, significantly increases the complexity of the controller system. Another type of AUV is a spherical underwater robot (SUR). All control elements, electronic devices, and power sources are within that spherical shape [162–164]. Due to the shape and center of gravity, SURs are characterised by high stability and maneuverability, allowing for a tiny turning radius (even 0 degrees). The obstacle-avoidance and control systems for the multiple AUV formations are also studied more often recently. In such formations, the control is, however, very complex due to the need to simultaneously control multiple vehicles, their communication and cooperation, as well as to perform specific maneuvers when avoiding obstacles and then

returning all formations to the previous positions. Currently, AUV control systems mostly use artificial intelligence methods to define the environment and the situation they are in, as well as to find solutions in the event of encountering problems or obstacles. Due to the limitations of individual methods in their basic versions, to increase the efficiency of the AUV motion control system, modifications or combinations of methods are used. That approach in some cases allows one to use the advantages to eliminate the disadvantages of the chosen methods.

**Figure 4.** Number of publications in the field of simulation methods and practical implementations of control and collision avoidance systems for AUV in the last 20 years.

**Figure 5.** Percentage of classical methods used in simulations studies and real applications in the last 20 years.

**Figure 6.** Classical methods validated in numerical research.

**Figure 7.** Artificial intelligence methods validated in numerical research.

**Figure 8.** Classical methods in real applications.

**Figure 9.** Artificial intelligence methods in real applications.

#### *4.2. Future Works*

Over the last few decades, multiple path-planning methods and collision-avoidance methods have been proposed and simulated in the field of AUV technology. Nevertheless, there are only a few practical AUV implementations with an effective and efficient system of optimal path planning and collision avoidance. In the coming years, it will be necessary to focus more on practical implementations by using the existing knowledge gained in simulation studies. Technological development provides better possibilities for more accurate detection, and more precise navigation or increasing the computing speed. The appropriate use of evolving technology is crucial in the further development of collision avoidance and path-planning systems for AUVs. Many problems in this field have already been resolved. However, there are still issues that require extensive research. AUVs are still not able to achieve high speeds during collision-avoidance maneuvers. Additionally, despite the developing technology, the navigational uncertainties still have to be compensated with algorithms that approximate the exact location. Recently, research on the methods of following a designated path and compensating for navigational error has been developing very intensively. Another aspect that requires more analysis and further work is navigation of the AUV in shallow water. This applies particularly to AUVs that use obstacle-detection sonars, where ground and water surface reflections create fictional obstacles.

#### *4.3. Bottlenecks of Future Development*

Artificial intelligence methods usually require a large amount of computation when processing 3D data in a complex environment. The vehicle's speed depends on how quickly the system makes decisions when detecting an obstacle or a group of obstacles. Increasing the computing power allows for more efficient operation of the control system and faster data processing and consequently making faster decisions on how to avoid collisions. Ensuring high computing power, however, requires increasing the vehicle's size, which limits the vehicle's maneuvering parameters. Furthermore, highly efficient systems that provide high computing power consume more energy and require more financial outlay. The solution to this problem could be, for example, the creation of wireless charging stations for underwater vehicles; however, there are still limitations related to the specificity of the underwater environment, which inhibits the development of such infrastructure [23]. Thus, the optimal selection of the computing power for path planning and collision-avoidance drivers is crucial. Another aspect that limits the development of AUV in collision avoidance and path planning is the rough conditions in the underwater environment. The greater the operating depth, the higher the technical requirements for the AUV design. The lack of reference points, poor visibility and high signal attenuation make it challenging to navigate AUVs. Moreover, sea currents make it difficult to predict the environmental impact on AUVs. Another issue is the nonlinear dynamics of AUVs. Methods based on machine learning can achieve the ability to correctly control taking into account the dynamics of the vehicle. However, this requires a very long training time and does not guarantee correct vehicle control in situations that differ slightly from those covered by the training. For this reason, vehicle dynamics plays a vital role in practical implementations.

#### **5. Conclusions**

This article presented quantitative and qualitative comparisons of path-planning and collision-avoidance systems verified by using numerical tests and also real applications. We performed a literature review and analysis, summarizing the achievements, future works, and aspects limiting the further development of AUVs, especially in terms of path planning and collision avoidance.

Although since the 1990's many of the AUV's collision avoidance-related issues, such as the local minima problem, mapping the optimal path in an unknown environment, 3D path planning, re-planning and real-time decision-making, have been addressed and resolved by using mainly numerical studies, there are still many issues, which have not been entirely resolved, e.g., moving in a complex environment at high speed, nonlinear dynamics of the AUV, detection disruptions related to shallow water and a limited environment, or accurate navigation in the underwater environment. Moreover, it is difficult to indicate the best method or group of methods for collision avoidance, based on the state-of-theart analysis. Depending on the circumstances, various methods were used with success for different cases. Quite often, hybrid methods showed good efficiency. Each method has disadvantages that reduce its effectiveness in a challenging underwater environment. The presented results show that the most frequently used algorithms tested in the simulated environment are APF, RL, and FL. These algorithms appear to be the most suitable for use in a real environment. Currently, in real implementations, the most frequently used method is the application of behavioural rules based on reacting to changes in the environment in a specific way. Although artificial intelligence methods have long dominated the field of collision avoidance for robots [3], in AUV, they are used in numerical research as often as classical methods. However, in real applications, mainly classical methods are still used.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Abbreviations**

The following abbreviations are used in this manuscript:


#### **References**


**Agnieszka Lazarowska 1,\*,† and Andrzej Zak ˙ 2,†**


**Abstract:** The paper introduces a proposal of an Autonomous Navigation System for Unmanned Surface Vessels. The system architecture is presented with a special emphasis on collision avoidance and maneuver auto-negotiation. For the purpose of maneuver auto-negotiation, the concept of multi-agent systems has been applied. The algorithm developed for the task of collision avoidance is briefly described and the results of the simulation tests, confirming the effectiveness of applied method, are also given. Presented outcomes include solutions of test scenarios from the perspectives of different ships taking part in the considered situations, confirming the applicability of the collision avoidance algorithm in the process of maneuver auto-negotiation.

**Keywords:** autonomous navigation system; collision avoidance; maneuver auto-negotiation; multi-agent system; obstacle avoidance; real-time path planning; unmanned surface vehicle

#### **1. Introduction**

Navigation is a vital task in the operation of every autonomously moving object. This relates to self-driving cars, autonomous mobile robots, drones and unmanned ships. Autonomous navigation of a craft has to cover perception, obstacle detection and avoidance and path planning and following. Marine units that move autonomously can be divided into autonomous merchant vessels and smaller crafts—Unmanned Surface Vehicles (USVs). The International Maritime Organization (IMO) introduced a term for autonomous merchant vessels—the Maritime Autonomous Surface Ship (MASS), which is used to express *a ship which, to a varying degree, can operate independent of human interaction* [1].

Over the last few years, many research and development projects dedicated to MASSs have been carried out. Examples of such projects, listed in Table 1 and in Figure 1, include the Maritime Unmanned Navigation through Intelligence in Networks (MUNIN) [2], ReVolt [3], Advanced Autonomous Waterborne Applications (AAWA) [4], Autosea [5], Autoferry [6], Yara Birkeland [7], Safer Vessel with Autonomous Navigation (SVAN) [8] and Mayflower Autonomous Ship (MAS) [9].

Projects on autonomous ships development

**Citation:** Lazarowska, A.; Zak, A. A ˙ Concept of Autonomous Multi-Agent Navigation System for Unmanned Surface Vessels. *Electronics* **2022**, *11*, 2853. https://doi.org/10.3390/ electronics11182853

Academic Editor: Arturo de la Escalera Hueso

Received: 28 June 2022 Accepted: 5 September 2022 Published: 9 September 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).


**Table 1.** Research projects on autonomous ships.

The technology of USVs has also developed dynamically in recent years. Examples of recently developed USVs are listed in Table 2. USVs application areas include defense area, such as surveillance, search and rescue, reconnaissance and strike missions, but also ocean surveying, such as collecting oceanographic data (bathymetry, pollution monitoring). A review of USVs with a special emphasis on the design aspects of the GNC system for these marine crafts can be found in [10].

**Table 2.** Examples of recently developed USVs.


Recently, different classification societies, maritime organizations and companies have been developing classifications of ships based on their degrees of autonomy [20]. The Maritime Safety Committee (MSC) of the IMO defined 4 degrees of ship autonomy, as shown in Table 3 [1].

**Table 3.** Degrees of ship autonomy according to IMO.


The aim of this paper is to present the developed concept of the Autonomous Navigation System for the USV. The main contributions of the research presented in this paper are:


Section 2 introduces the general structure of the Autonomous Navigation System for USVs. Section 3 presents the concept of the Multi-agent system for USV maneuver autonegotiation. In Section 4, the collision avoidance algorithm developed for the application in the ANS for USVs has been briefly described. In Section 5, simulation tests results obtained with the use of the collision avoidance algorithm are shown, with a special emphasis of solutions achieved from the perspectives of different USVs taking part in the considered encounter situations. Discussion on the results is given in Section 6 and the paper is summarized in Section 7.

#### **2. Autonomous Navigation System Structure**

The Autonomous Navigation System is responsible for the navigation of the USV. The main tasks of the Autonomous Navigation System of every vehicle include: perception of the environment, path planning and path following (vehicle control).

One of the main subsystems of the ANS is the Collision Avoidance (CA) module. The CA module is responsible for the collision risk assessment on the basis of information obtained from the system that fuses data from various navigational sensors. The second task of the CA module is to ensure the safe navigation of the USV, both on the open sea and in restricted waters. The main element of the CA module is the collision avoidance algorithm, responsible for determining a safe maneuver or a safe trajectory for the vehicle, when a risk of collision has been detected. Besides the CA module, the route planning module is also applied. This subsystem is responsible for the calculation of a global path between the initial and final position, e.g., two harbors, including the craft's mission, but also weather conditions (therefore known also as weather routing). Figure 2 presents a general structure of the Autonomous Navigation System for USVs.

*USV Autonomous Navigation System*

**Figure 2.** Autonomous Navigation System for USVs.

The module responsible for navigational data reception and fusion is the Advanced Sensor Module—ASM, also known as the Situation Awareness module or Sensor Fusion module. Applied sensors include: nautical charts, long-range radars, the Automatic Identification System (AIS), stereo vision systems, short-range radars, lidars and the Global Positioning System (GPS). In relation to the ANS of USVs, the term Obstacle Detection and Avoidance (ODA) system is also used, which reflects a combined version of the Situation Awareness and Collision Avoidance modules. Motion Control system is responsible for the calculation of appropriate control forces to steer the USV along the calculated trajectory or to execute a maneuver determined by the CA module. Table 4 presents a comparative analysis of ANSs proposed in different research projects on autonomous ships, listed in Figure 1 and Table 1. The ANS structure applied in the research presented in this paper was developed based on the solutions proposed in the mentioned projects.

**Table 4.** ANSs in research projects on autonomous ships.


#### **3. Multi-Agent System for USV Maneuver Auto-Negotiation**

Multi-agent systems are regarded as a powerful concept for solving real-world problems, especially in complex, dynamic environments, enabling to achieve increased autonomy within control systems [21]. Table 5 presents a summary of multi-agent systems proposed in the recent literature in relation to ships and different types of unmanned vehicles: USVs, UUVs (Unmanned Underwater Vehicles) and UAVs (Unmanned Aerial Vehicles). The comparison lists the control objects, the purpose of proposed systems, the type of tasks solved by the system, i.e., cooperation and/or competition, the type of applied method and information about the optimization of the task. The last column contains additional remarks, concerning advantages, limitations or other comments with regard to the relation between the mentioned methods and the approach described in this paper. Some of the proposed multi-agent systems are intended for solving a different task than collision avoidance, considered in this paper, such as formation control [22,23], training a team of USVs [24], searching water region by a team of UUV [25] and/or consider a different type of vehicle, such as UAV [26], UUV or a car [27].

The concept of the maneuver auto-negotiation system in relation to autonomous cars was proposed in [27], to UAV in [26] and to ships in [28–30]. In [28], collision avoidance and maneuver auto-negotiation is based on geometrical relationships. Two ships encounters are considered in this approach and the solutions might not be COLREGs compliant. In [29], a concept stage of the task has been presented. The authors assumed the application of evolutionary sets of safe trajectories, where the best set of reciprocal trajectories is calculated, but results of the collision avoidance approach are not presented in the paper. In [30], the author proposes a structure of the maneuver auto-negotiation system with the concentration on the data acquisition aspect. Safe trajectories are assumed to be calculated as evolutionary sets of safe trajectories. This approach has been presented in the author's previous works [x, y]. The author proposed a control flow, in which one of the vessels is a leader ship. The leader is responsible for data gathering, determination and optimization of maneuvers and distribution of the results among other participants of the encounter.

The literature analysis shows that the development of a multi-agent system for collision avoidance with maneuver auto-negotiation is an open research problem, as algorithms for the calculation of multiple trajectories are rare in the recent literature. There exists a demand in the industry for the development of an effective, robust and reliable multi-agent system, which will enable for the calculation of a set of safe trajectories for all ships or USVs taking part in an encounter. The development of such a solution is the aim of the research presented in this paper.


**Table 5.** Multi-agent systems for ships and unmanned vehicles.

Extending the ANS architecture for the purpose of maneuver auto-negotiation was based on multi-agent systems. Multi-agent systems are classified as an approach of Distributed Artificial Intelligence (DAI) [31]. A multi-agent system is composed of a number of agents, which interact with each other and the environment in order to achieve their goals. Such approach has been applied for modeling and solving problems by cooperation between local solvers and the design of complex systems. An agent is an object, being in a certain situation, that has the ability to perceive the environment and influence it through an autonomous action to achieve a defined goal. In the approach presented in this paper an agent is an USV, perceiving the environment through navigational equipment such as AIS, radar with ARPA, GPS, log, gyrocompass, lidar and stereovision systems. The USV also interacts with other USVs in the surroundings and acts on the environment by changing its course and/or speed.

Assumed agent architecture and control flow is shown in Figure 3 [25,32]. The algorithms related to detecting obstacles are included in the layer of reflex action. The planning layer implements obstacle avoidance algorithms [33–35] and these responsible for the control of the vehicle along a set trajectory. At the level of cooperation, the vehicle

exchanges data with other vehicles and conducts negotiations about planned collision avoidance actions.

**Figure 3.** Agent architecture and control flow.

The agent architecture applied in this research is the InteRRaP model [36], which means Integration of Reactive behavior and Rational Planning. This architecture utilized the BDI model of reasoning, which comes from Beliefs, Desires (Goals) and Intentions [37,38]. In such an architecture, two types of interaction between different layers are possible. These are: bottom-up activation and top-down execution. In the first type, when a lower layer is not able to deal with the current situation, it passes control to a higher layer. In the second type of interaction, a higher layer uses the functionality assured by a lower layer in order to reach its goal. In the InteRRaP architecture, when new data from sensors arrive at the lowest reflex action layer, it evaluates whether it can deal with the situation. When the layer is not able to perform the task, the control will be passed in the bottom-up activation manner to the planning layer. This layer will also asses the task and when it will be able to deal with the it, after which it will apply a top-down execution. Otherwise, it will pass control to the highest cooperation layer. Such an architecture, in comparison with other layered architectures of multi-agent systems, has as an advantage that only the lowest layer has direct access to actuators. This excludes the occurrence of conflicting decisions between different layers. A different layered approach was proposed in [39], where appropriate filtering and suppression mechanisms have to be applied in order to prevent the system from conflicts among the actions of different layers.

Agents communicate with each other, while maintaining their autonomy of action and decision-making, which translates into actions taken by individual agents. Such interaction between agents leads to the modification of knowledge and actions of individual agents based on the behavior of other agents. A single agent is equipped with a strategy, leading it to achieve the assumed goal, which in this case is to reach a specific geographical position the end point of the trajectory. However, as part of a multi-agent system, it must implement collaborative strategies to avoid collisions with other agents. The key feature of agents in a multi-agent system is their collaboration and competition. Therefore, agents exchange information on individual actions taken in order to avoid a collision.

The collision avoidance process with the use of the proposed autonomous multi-agent navigation system for USVs can be performed in one of the two following ways, both of which are possible with the use of the applied collision avoidance algorithm:


#### **4. USV Collision Avoidance Algorithm**

Collision avoidance and safe path planning algorithm applied in the introduced concept of Autonomous Multi-agent Navigation System is a deterministic approach. The algorithm's operation is based on the search through a database with stored candidate trajectories in order to find the best safe path solving the currently considered situation; therefore, the approach has been called the Trajectory Base Algorithm (TBA). This algorithm has been chosen for further development due to its competitive results in relation to other deterministic and non-deterministic methods. A comparative analysis of different methods, along with the TBA, can be found in [40].

The main advantages of this approach are: the solution repeatability for every run of calculations with the same input data, achievement of COLREGs compliant solutions with minimal path length and run time of the algorithm enabling the application of this approach for real-time path planning.

The input variables, shown in Figure 4, are:


The inputs from the database of trajectories are: candidate trajectories composed of a number of waypoints (x and y coordinates of the USV position). The output variables are: safe trajectories composed of a number of waypoints (x and y coordinates of the USV position) and the values of the USVs course at the consecutive parts of the trajectories.

**Figure 4.** Parameters defining a navigational situation at sea.

Figure 5 presents a flowchart of the TBA. As mentioned above, input data are the courses, speeds, bearings and distances of all vessels taking part in an encounter. After the reception of input data describing the current navigational situation, the first candidate trajectory is retrieved from the database for the evaluation procedure. The evaluated candidate trajectory is divided into a number of steps. Afterwards, in every step, an own ship is moved into a new instantaneous position along the evaluated trajectory. Target ships are also moved into their next instantaneous positions, resulting from their motion parameters and a trajectory selected for the implementation. Then, the procedure of checking, whether the vessels positions do not lead to a collision, is carried out. When the whole trajectory is verified as a safe path, which means that it does not cause a collision with any of encountered ships during the vessel's movement along it, it becomes the final best solution. The optimization criterion applied in the collision avoidance algorithm, defined by Equation (1), is the minimal path length. It is calculated as a sum of the lengths of line segments composing the safe path, where *i* = 1, . . . , *e* are the waypoints composing the path:

$$f(t) = dist(t) = \sum\_{i=1}^{\varepsilon - 1} \sqrt{(\mathbf{x}\_{i+1} - \mathbf{x}\_i)^2 + (y\_{i+1} - y\_i)^2} \to \min \tag{1}$$

The information about the selected trajectory, as a part of cooperation activities between agents, is transferred to other agents in a multi-agent system to be used in their decision-making. Further calculations are terminated. A more detailed description of the TBA can be found in [41].

**Figure 5.** A flowchart of the Trajectory Base Algorithm.

#### **5. Results of Collision Avoidance Algorithm**

The TBA proposed in this paper for USVs collision avoidance has been tested comprehensively in order to validate its reliability and robustness. Various tests of this algorithm have been carried out, including:


The results of these test can be found in [42].

One of the methods to assure a proper distance between the USVs or other marine crafts in a collision avoidance algorithm is to model other USVs (target ships) with the use of a ship domain. A ship domain is defined as an area around a ship, which a navigator wants to keep free from static and dynamic obstacles. In the presented approach, a ship domain with a hexagon shape, as proposed in [43], has been applied. The dimensions of the target ship domain used in the tests were: distance towards the bow: 1.3 nm, distance of amidships: 0.6 nm, distance towards the stern: 0.5 nm, distance towards the starboard side: 0.6 nm and distance towards the port side: 0.5 nm.

For the purpose of the algorithm's application in the USV Autonomous Multi-Agent Navigation System, the evaluation of solutions consistency from different ships' perspectives has been carried out. The results of these experiments are presented here. The algorithm was implemented in the MATLAB programming language and tested on a PC with Intel Core i7-10750H CPU, 32 GB RAM, 64-bit Windows 10 operating system. Out of many test cases, four scenarios have been chosen for the presentation in this paper. These test cases have been chosen as standard scenarios, which present the solutions compliance with the COLREGs. Similar test scenarios were presented in the cooperative path planning approach proposed in [44]. Input data of USVs, including courses in degrees, speeds in knots, bearings in degrees and distances between the marine crafts in nautical miles, are given in Tables 6–9. Tables 10–13 present output data, such as USV courses in degrees at consecutive parts of the calculated safe path, path lengths in nautical miles and run time of the algorithm in seconds. Figures 6–9 show graphical solutions of test cases 1–4. Initial positions of USVs are marked by digits representing the number of an USV (USV No. in tables). The numbers given along the consecutive positions of the USVs show the time of the USV arrival at a given point in minutes. Presented results have been discussed in the following section.

**Table 6.** Input data of test case 1—head-on.


**Table 7.** Input data of test case 2.


**Table 8.** Input data of test case 3.



**Table 9.** Input data of test case 4.

**Figure 6.** Solution of test case 1.

**Table 10.** Output data of test case 1.


**Table 11.** Output data of test case 2.


**Figure 7.** Solution of test case 2.

**Figure 8.** Solution of test case 3.

**Figure 9.** Solution of test case 4.

**Table 12.** Output data of test case 3.


**Table 13.** Output data of test case 4.


#### **6. Discussion**

*6.1. Analysis of Results*

Results of TBA applied in the presented concept for USV collision avoidance allow to state the following remarks:


It should be underlined here that results of collision avoidance algorithms, regarding calculation of multiple trajectories, are very rare in the literature on that topic. Examples of approaches other that the one presented in this paper can be found in [44,45].

Test case 1 presents a head-on situation. In this encounter situation, according to rule 14 of COLREGs, both vessels should alter her course to starboard side in order to pass on the port side of the other vessel. Figure 1 presents paths calculated by the algorithm for both vessels. As can be observed there, both USVs execute maneuvers to their starboard sides. Maneuvers are also compliant with rule 8b of COLREGs—they are large enough to be easily noticed and interpreted by the other vessel.

Test case 2 presents an encounter of three USVs and is composed of head-on (between vessels 0–2) and crossing situations (between vessels 0–1 and 1–2) between the different pairs of vessels. The behavior of vessels in a crossing situation is defined by rule 15 of COLREGs. According to this rule, the vessel that has the other vessel on her starboard side should keep out of the way and avoid crossing ahead of the other vessel. As can be observed in Figure 2, the USV no. 0 keeps out of the way of the USV no. 1 and does not cross ahead on the other vessel. In the same manner, the USV no. 1 keeps out of the way of the USV no. 2. The solution is also compliant with rule 14 of COLREGs—both USVs no. 0 and no. 2 alter their course to starboard, passing on the port side of the other vessel. Similarly, as in test case 1, maneuvers of all USVs also fulfill rule 8b.

In test case 3, a crossing situation occurs between vessels 0–1 and 0–2, whereas a head-on encounter takes place between vessels 1–2. Likewise, in test case 2, solutions are calculated by the TBA for all of the USVs fulfill rules 8b, 14 and 15.

Test case 4 is an encounter between four USVs, where crossing situations occur between the vessels 0–1, 1–2, 2–3 and 3–0, and head-on situations take place between vessels 0–2 and 1–3. Analyses of safe paths returned by the TBA enable to state that the solutions are compliant with COLREGs rules 8b, 14 and 15.

The obtained results confirm the prospect to apply the proposed collision avoidance algorithm in the Autonomous Multi-Agent Navigation System for Unmanned Surface Vessels, as it has been proven that the solutions calculated for different vessels in the same encounter situation constitute safe paths (do not lead to a collision between the marine crafts). Therefore, they can constitute a solution of maneuver auto-negotiation procedure.

#### *6.2. Comparison with Other Collision Avoidance Algorithms*

The collision avoidance algorithm has been compared with other approaches proposed for the application in maneuver auto-negotiation systems or presenting results from different ships' perspectives. The results of this analysis are shown in Tables 14 and 15. The comparison includes: applied method, type of the method, i.e., deterministic or non-deterministic, consideration of static navigational restrictions, dynamic obstacles and COLREGs, applied optimization criterion. The run time of the algorithm and repeatability of a solution for every run of calculations with the same input data are also specified in the tables. The adopted calculation approach has also been defined for every algorithm, whether the trajectories are calculated by only one vessel: the leader ship or by all of marine crafts taking part in the encounter. The main advantages of the approach proposed in this paper, in relation to other existing methods, are as follows:



**Table 14.** Comparison of collision avoidance algorithms applicable in maneuver auto-negotiation systems—part 1.

**Table 15.** Comparison of collision avoidance algorithms applicable in maneuver auto-negotiation systems—part 2.


#### **7. Conclusions**

The main contributions of the research presented in this paper in relation to the previous works of the authors are the application of the developed deterministic collision avoidance algorithm within the framework of a multi-agent system for the purpose of USV maneuver auto-negotiation and also the carried out tests of the algorithm in order to assess the possibility to apply collaborative strategies to avoid collisions with other agents in the proposed multi-agent system by the calculation of multiple trajectories for all USVs taking part in the considered navigational scenario.

A concept of the Autonomous Navigation System for Unmanned Surface Vessels, with a special emphasis on the task of collision avoidance, was presented first. A general structure of the ANS system was developed, based on the idea of multi-agent systems. An agent is an autonomous entity that perceives the environment using sensors and acts on the environment through effectors (actuators). The agent's interaction can be based on cooperation or competition. The goal of the USV, modeled as an agent, is to reach the end point of the trajectory. However, in order to achieve this objective, it has to apply collaborative strategies to avoid collisions with other agents. For this purpose, agents exchange information on planned actions. Proposed collision avoidance algorithm was validated by simulation experiments with a special emphasis on the verification of its applicability for the purpose of maneuver auto-negotiation. The achieved results prove that the trajectories obtained for different agents do not lead to a collision; therefore, the

algorithm is suitable for the application in the Autonomous Multi-Agent Navigation System for the USV.

Further research include the development of the communication protocol between agents, allowing them to exchange the information efficiently. After that, experiments in real operating conditions are planned to be carried out.

**Author Contributions:** Conceptualization, A.L. and A.Z.; methodology, A.L. and A. ˙ Z.; software, ˙ A.L.; validation, A.L. and A.Z.; formal analysis, A.L. and A. ˙ Z.; investigation, A.L. and A. ˙ Z.; resources, ˙ A.L. and A.Z.; data curation, A.L. and A. ˙ Z.; writing—original draft preparation, A.L. and A. ˙ Z.; ˙ writing—review and editing, A.L. and A.Z.; visualization, A.L. and A. ˙ Z.; supervision, A. ˙ Z.; project ˙ administration, A.L. and A.Z.; funding acquisition, A.L. and A. ˙ Z. All authors have read and agreed ˙ to the published version of the manuscript.

**Funding:** This research was funded within the framework of the program of the Minister of Science and Higher Education under the name "Regional Excellence Initiative" in the years 2019–2022; project number 006/RID/2018/19; the amount of financing: 11 870 000 PLN.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Abbreviations**

The following abbreviations are used in this manuscript:


#### **References**

