*4.3. Summary of Hazard Avoidance Algorithm*

The above study considers the problem of reactive avoidance of proximity in the design of path-planning algorithms. In the face of complex environments, a hybrid algorithm using hierarchical processing and a grid-based model prevents repeated searches and makes paths more efficient. In realistic marine environments, the effects of wind, waves, currents, and other factors are also considered to ensure safety and to produce globally

optimal paths. Based on AIS data, COLREGs and machine learning algorithms combine historical data with real-time data for route planning to ensure safe navigation and avoid multi-vessel head-on collisions, cross-collisions, and random collisions. Table 3 compares different algorithms for hazard avoidance and considers the factors of the optimal path.

**Figure 5.** Path planning of improved DQN [102].



Note: consider (T), no consider (F), DOA (dynamic obstacle avoidance), \* (start).

#### **5. Cluster Path Planning**

USVs cluster path planning technology is rapidly developing as a core key technology. USVs cluster path planning is the key to cluster path generation, obstacle avoidance, collision avoidance, and other navigation and coordination tasks assignment. It is essentially a multi-constrained combinatorial optimization algorithm. The cluster intelligence optimization algorithm focusing on the bionic approach and the multi-task assignment strategy are essential assets to solve the cluster path planning problem [111].

### *5.1. Bionic Algorithm*

For large-scale, high-dimensional, and non-linear USV cluster path planning problems, it is a good choice to use a bionic optimization algorithm. In [112], a firefly-based Approach (FA) for robot cluster path planning is proposed, where Firefly social behavior optimizes group behavior. Since the path planning problem is an NP complexity problem, multi-objective evolutionary algorithms are an effective way to solve this problem. In [113], a cluster path-planning algorithm based on ant colony optimization (ACO) [114] in a dynamic environment is proposed to perform path-planning optimization and establish the path-planning optimization objective function in a multi-tasking scenario. In [115], based artificial bee colony (ABC) algorithm [116], an efficient artificial bee colony (EABC) algorithm is proposed. It solves online path planning collision avoidance for multiple mobile robots by selecting appropriate objective functions for the target, obstacles, and robots. It utilizes elite individuals to maintain good evolution, improve performance and shorten path length. The method improves the quality of path planning for clusters, but it tends to fall into local optimality. For USV clusters caught in a local optimum, ref. [117] proposed an improved particle swarm optimization(PSO) algorithm based on an adaptive sensitivity decision operator that is more adapted to 3D path. It solves the drawback that traditional PSO is prone to fall into local optimality [118], improves the convergence accuracy and cooperative operation of clusters, and predicts the globally optimal planning. In [119], it proposed a decision support algorithm to use an artificial fish swarming algorithm (AFSA) [120] based on AIS data for path-planning decisions in USV clusters. It can calculate the optimal avoidance turn to time and avoidance angle, and the algorithm possesses excellent robustness and converges quickly. In [121], a hybrid improved artificial fish swarm algorithm (HIAFSA) was proposed to address the problem of falling into local optima. It combines the A\* algorithm with AFSA for further optimization in global suboptimal paths. It introduces decay functions to enhance the visual range and motion steps to improve the convergence speed. The algorithm offers access to local optimum avoidance, convergence speed, and accuracy. In [122], the salp swarm algorithm (SSA) [123] optimization algorithm was used to divide the task area using Voronoi diagrams to achieve collaborative path planning for USV clusters. This optimization algorithm avoids repeated searches, improves search efficiency and accuracy, and effectively avoids collisions between obstacles and USVs. However, the method does not consider the influence of COLREGs and the actual environment at sea. To consider USV clusters in realistic environmental scenarios at sea, ref. [124] proposed a simulated annealing-bacterial foraging optimization algorithm (SA-BFO). The bacterial foraging optimization algorithm (BFO) is a population intelligence optimization algorithm with good search efficiency and robustness [125]. The hybrid approach, based on COLREGs, can perform real-time avoidance of dynamic obstacles using USV cooperative control. It is able to plan collision-free paths efficiently, moving away from local optima and converging to global optima.

#### *5.2. Multi-Objective Task Assignment Algorithm*

When faced with multiple target missions, it is difficult for a single USV to perform complex tasks in unknown underwater environments. Then, USV clustering can improve the performance of the system, shorten the mission time and increase the probability of the search success. Multi-objective search task assignment is the key to USV clustering path collaboration [126,127]. As shown in Figure 6, multiple search tasks are assigned to multiple specific USVs. Currently, reinforcement learning algorithms (RL) are more suitable for task assignment, and path planning [128,129]. In [130], combining HER with DQN and introducing a reward mechanism can increase the validity of the sample and the speed of convergence. In [131], a pheromone mechanism using ACO on the classical Q-learning approach is investigated. It solves the information-sharing problem in reinforcement learning systems and improves the efficiency of robot cluster path planning. In [132], a hexagonal area search (HAS-DQN) was proposed. It solves the collaborative path planning problem for UAVs, maximizes the data collected by UAVs, minimizes the total energy consumption, and extends the lifetime. However, due to the effects of offshore winds, waves, currents, etc., the above multi-task and multi-objective optimization decisions based on reinforcement learning do not work well when applied to USVs. In [133], an improved self-organizing map (SOM) [134] with collision avoidance capability was proposed, based on a fast marching square (FMS) [135] path planning algorithm. It is adapted to multi-task assigned USVs cluster for complex tasks, such as maritime patrol search and rescue and environmental detection. The algorithm first assigns tasks to each USV and enables the function of fast task assignment and optimized execution sequences. Not only does it achieve avoidance of all static and dynamic obstacles, but it also takes into account COLREGs. However, the algorithm does not consider the complex terrain and time-varying currents under the sea. In [136], an improved the K-means algorithm to accommodate unsupervised learning of competing strategies was proposed. The algorithm first assigns different tasks to multiple USVs and performs the task assignment by the SOM algorithm. It can autonomously undertake complex maritime missions in a limited environment and verifies its effectiveness by simulating it in a real marine environment. In [137], an improved SOM combined with spectral clustering (SC), is used to solve collaborative path planning for USVs cluster and multi-task allocation. The method allows for the selection of the globally optimal path with minimal energy consumption in a collision-free manner. To make the generated routes smoother, a dual smoothing strategy with B-samples and indirect adaptive disturbance observer-based line-of-sight (IADO-LOS) [138] are used to achieve the precise path following of the USV. The method also accounts for the effect of ocean currents on ship driving and is more in line with the realistic environment at sea.

**Figure 6.** USV cluster multi-objective task assignment.

#### *5.3. Summary of Cluster Path Planning Algorithm*

The above study uses the bionic optimization concept for path planning algorithms in USV clusters to optimize multi-objective paths for multi-task, multi-constraint models. Using AIS big data, COLREG and machine learning algorithms are used to combine historical and real-time data for path planning to ensure safe navigation while completing multi-tasking assignment strategies at sea. Table 4 compares different algorithms for the USV cluster and considers the factors of the optimal path.


**Table 4.** Characteristics of different algorithms for clusters.

Note: consider (T), no consider (F), Muti (muti-objective).

#### **6. Conclusions**

Path planning is a hot and complex area of research in the USV field and a key technology to ensure its autonomy in the marine environment. This paper mainly reviews and analyzes the literature related to USV path planning in recent years. Path planning is divided into four aspects: global path planning, local path planning, hazard avoidance with approximate responses, and path planning under clustering. The advantages and disadvantages of different algorithms for global and local path planning are analyzed in Sections 2 and 3. As shown in Table 5, global path planning can cover the whole map and find the global optimal solution. However, the computational complexity is high, the operation efficiency is slow, and it cannot cope with dynamic environmental changes. The local path planning algorithm is simple, responsive, and can adapt to the dynamic environment. However, it cannot handle the dead ends and obstacles in global path planning, and it is easy to fall into local optimal solutions. In addition, most of the improved algorithms for global and local path planning ignore the influence of real environments, such as ocean currents and wind waves. Modeling USV kinematics and dynamic constraints is important for path planning when considering the effects of wind, waves, and currents on safe avoidance and energy consumption. Using USV clusters with multi-task assignment and path collaboration strategies is also the right choice when facing multiple tasks.

**Table 5.** Global path planning strategy and local path planning strategy contrast and analysis.


#### **7. Prospects**

Firstly, the future direction of current path planning optimization algorithms is multialgorithm fusion, based on traditional optimization algorithms combined with deep reinforcement learning, digital twins, and other artificial intelligence optimization algorithms, which holds the promise of dynamic and online path planning.

Secondly, the evaluation system of the path and the collision risk assessment models need to be further improved. Under the premise of ensuring safety, it considers factors such as path length, smoothness, time consumption, and efficiency, and also further considers the motion performance of angular velocity, acceleration, and turning angle of the ship.

Finally, USV cluster collaboration for path planning is also a hot research topic for the future. How to make USV clusters perform multiple tasks while performing integrated obstacle avoidance is also a problem that needs to be tackled in the future. Improving the perception of the navigation area and the autonomous decision-making capability is a significant part of the solution to this problem.

**Author Contributions:** Conceptualization, B.X.; methodology, B.X., M.Y. and B.L.; writing—original draft preparation, M.Y., B.X. and Z.L.; writing—review and editing, B.X., M.Y. and B.L.; supervision, B.X., Y.S., Z.L. and Y.T.; project administration, B.X., Y.T. and Y.S.; funding acquisition, B.X. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by Shanghai Science and Technology Committee (STCSM) Local Universities Capacity-building Project (No. 22010502200).

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** The data are available on request.

**Acknowledgments:** The authors would like to express their gratitude for the support of Fishery Engineering and Equipment Innovation Team of Shanghai High-level Local University.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

MDPI St. Alban-Anlage 66 4052 Basel Switzerland www.mdpi.com

*Journal of Marine Science and Engineering* Editorial Office E-mail: jmse@mdpi.com www.mdpi.com/journal/jmse

Disclaimer/Publisher's Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Academic Open Access Publishing

mdpi.com ISBN 978-3-0365-9781-2