*2.3. Genetic Algorithm (GA)*

The genetic algorithm (GA) was proposed by John Holland, and adheres to the principles of genetics and natural selection [30]. The main benefit of GA is that it is available for complex problems and is a general approach to solving search methods. However, classical GA suffered from low convergence and function-dependent optimization [31]. The slow processing speed makes it unsuitable for real-time route planning in dynamic and unknown situations. GA improvements have focused on improving the crossover and variation operators, choosing better coding methods, and adjusting genetic parameters. In [32], a new crossover operator to solve the complicated case of the USV path planning problem was utilized. With the aim of minimizing the overall flight cost, the algorithm randomly selects the child paths from the parent and generates paths to find the globally optimal solution and improve the convergence speed. In [33], an artificial adaptation function and two custom genetic operators were proposed, dealing with the path planning problem of USVs in the static case. It is shown that the method enables fast optimal solutions in a static environment but is hard to obtain in dynamic obstacle avoidance. In [34], a hybrid repetition-free string selection operator and heuristic multiple variables operators was proposed to find optimal paths and extend the population search. It obtains smooth flight paths and reduces the computational time of the algorithm. In [35], an adaptive genetic algorithm (AGA) was proposed that focuses on solving complex optimization problems. The hierarchical approach to selecting operators reduces time and speeds up convergence. The algorithm can shorten the length of the global path, improve quality, and find globally optimal routes in static environments. In [36], an improved particle swarm genetic algorithm (IPSO-GA) was proposed for UAV path planning, which optimizes continuously by swarm mating variation operations. The algorithm saves time and improves efficiency compared to greedy algorithms. It is more suitable for computationally intensive and complex situations to achieve collision avoidance of dynamic obstacles, but the quality of generating globally optimal paths in complex environments. In [37], a multi-objective enhanced genetic algorithm (EGA) was developed. Using crossing and variational operators, it takes full account of path smoothness and path length in ensuring that the generated paths are safe and feasible, and improves running time and efficiency. However, the method needs further improvement for dynamic obstacle avoidance. In [38], an improved multi-objective genetic algorithm (IMOGA) was proposed. Decreasing the path length is achieved by removing the redundant nodes of the running path. The optimized path quality is achieved by improving the mobility of the mobile robot, reducing the number of turning movements, and improving smoothness. However, the method does not consider the dynamic obstacle avoidance problem in the underwater environment.

#### *2.4. Neural Network (NN)*

The Neural Network (NN), introduced by McCulloch Pitts in 1944 [39], also becomes the Artificial Neural Network (ANN). It also describes the NN method to navigate in an unknown environment [40]. In [41], an NN based the COLREG rule and the risk of avoiding collisions with static obstacles was considered, to ensure the safety of MASS. However, the generated global path is not optimal by the limited environmental information. In [42], a CNN to extract information and collect visual data through two-dimensional images was proposed. Furthermore, considering COLREG, the algorithm changes the motion vector to avoid collisions. However, the data do not update in real-time and the path planning for dynamic obstacles is not accurate enough. In [43], a residual convolutional neural network (R-CNN) algorithm was investigated. It improves the capability of real-time path planning path quality by collecting previous global information, training by reinforcement, and avoiding dynamic obstacles in a real-time environment. At the same time, it can generate optimal paths on global paths. In [44], a GNN model to use features extracted from graphs by NN was proposed. With consideration of the structure and node characteristics of the graph, it improves the smoothness and speeds up the computation. However, using the GNN model directly on large-scale graphs can be computationally challenging [45]. In [46], a graph convolution network (GCN) was proposed. It is a semi-supervised learning method based on graphical data. Using a model of GCN, nodes with previously unknown data are created to capture global information to predict traffic states [47] and to achieve global path planning. The method has few weight parameters and fast convergence. However, it has a poor performance when it is applied to maritime USVs and works for planning globally optimal paths on the ground. In [48], a bio-inspired neural network (BINN) algorithm was proposed and applied to full-coverage path planning for robots on land. It is able to find the shortest path by a full-coverage approach while avoiding static obstacles. Applied to USV, it generates paths with too large turning angles and not smooth enough paths. To adapt full-coverage path planning for USV over the sea, ref. [49] proposed a full-coverage neural network (CCNN) algorithm for USV path planning. It can greatly reduce the computation time and improve coverage efficiency. The turning angle is further reduced to make the path smoother and to avoid all dynamic obstacles to obtain the globally optimal path.

#### *2.5. Summary of Global Path Planning Algorithms*

The above research uses tools such as shortening the path length and smoothing the generated paths when designing path planning algorithms to make the planned paths more compatible with navigation practice. The use of hierarchical processing of data in constraints to avoid repeated searches and the introduction of genetic crossover operators make the paths more convergent and efficient while reducing the solution time of the algorithm to meet practical needs. The safety distance parameter is entered, following COLREGs, to improve the safety and robustness of the ship. The global path algorithm is dependent on perceived environmental information and cannot optimally deal with unknown and dynamic obstacles. As shown in Table 1, it compares different algorithms for global path planning and considers various factors for optimal paths.


**Table 1.** Characteristics of different algorithms for global path planning.

Note: consider (T), no consider (F), dynamic obstacle avoidance (DOA), \* (start).

#### **3. Local Path Planning**

In contrast to the global path planning algorithm, local path planning is the optimal path from the current node to a targeted node. It uses sensors to obtain data from the surrounding location environment, senses the obstacle distribution in the region, and performs local path planning in a real-time path search. Due to the lack of global environment information, it is easy for the pathway to fall into local optimum. Typical local path planning algorithms include artificial potential field (APF), fast extended random tree (RRT), velocity obstacle (VO), and dynamic window method (DWA). As shown in Figure 2, the typical local algorithms are analyzed for their advantages and disadvantages and the corresponding optimization algorithms.

**Figure 2.** Local path planning algorithm.

#### *3.1. Artificial Potential Field (APF)*

The artificial potential field (APF) method is a well-known USV local path planning algorithm proposed by Khatib [50]. Although the APF method has the advantages of a single structure, fast speed, and good real-time performance, it suffers from problems such as local optimum and poor path smoothing. In [51], the APF method is applied to the local path planning of USVs to solve the problem of avoiding static obstacles during navigation and ensuring the safety of the ship's trajectory. However, when the target points near obstacles are difficult to detect, there is poor performance of the generated global paths. To solve the problem of dead loops, ref. [52] proposed an improved APF approach using map expansion to reprogram local routes. When an infinite loop occurs, the map is transformed and starts searching for travelable paths, but does not take into account the problem of unreachability due to obstacles being too close to the target point. In [35], a vector artificial potential field method (VAPF) was proposed that improves the use of the space vector method to calculate the total force of AUVs. It allows USVs to reach static targets and track dynamic targets. The improved algorithm improves computational efficiency and reduces the cumbersome time in AUV dynamic avoidance. It can avoid dynamic obstacles effectively in real-time and reduces the cost of AUVs. However, when the target points near obstacles are difficult to detect, there is poor performance of the generated global paths. In [53], the discrete artificial potential field (DAPF) method was proposed. The results show that the algorithm not only efficiently and quickly computes safe routes in static and dynamic obstacle environments but also solves local optimization methods. Using the path optimization algorithm can reduce the path length and improve the smoothness. In [54], the Ant Colony Potential Field Algorithm (APF-ACO) global obstacle avoidance scheme was used, where the real-time information from a LiDAR sensor can avoid local obstacles, and a method to create a potential field without local extrema was proposed. The length of the path is optimized to reduce steering during driving and to ensure the robustness and safety of the ship itself. In [55], a model prediction strategy and an artificial potential field (MPAPF) was proposed. It overcomes the drawbacks of the traditional APF and other algorithms and solves the difficulty of becoming stuck in the local optimum. At the same time, it considers the head-on and cross-encounter situations of ships, effectively avoiding dynamic obstacles in real-time and ensuring the safety of USVs under complex conditions.

#### *3.2. Rapidly Expanding Random Tree (RRT)*

The rapidly expanding Random Tree (RRT), formulated by Lavalle [56], is a samplingbased path planning algorithm. As the creation of new nodes is random, node creation will stop once the node determines the target location. However, the RRT algorithm provides a preferred path that generates many nodes, which consumes time and generates a not sufficiently smooth path. In [57], the algorithm reduces the randomness of the final path by deselecting the parent node.The shortest path is found by increasing the number of iterations, but it can fall into local optimality. In [58], RRT\* trajectory optimization is used to create smoother paths and reduce path lengths. However, convergence to the optimal path solution is slow and consumes considerable memory and time. In [59], a Q-Learning-based RRT\* (Q-RRT\*) method was proposed that reduces the path length and accelerates convergence to the optimal solution by extending the search. However, it yields paths that are not globally optimal and obstacle avoidance for dynamic obstacles requires further consideration. In [60], the RRT-Connect algorithm was proposed, which is a simple and efficient stochastic algorithm. The greedy heuristic algorithm needs to speed up the search capability and the convergence process by searching from the starting and target points but is prone to local optima. To address the case of falling into a local optimum, ref. [61] proposed a path extension-based heuristic sampling RRT\* algorithm (EP-RRT\*). Based on RRT-Connect, the sampling-based asymptotically optimal path planning algorithm is proposed, which can quickly explore the global environment and find feasible paths. In [62], a novel probabilistic smoothed bi-directional RRT (PSBi-RRT) algorithm was

presented. A greedy algorithm and biasing strategy require searching for nearby points. As a result, it avoids collisions in the environment, significantly reduces running time, and decreases the likelihood of local optimization. In [63], a method combining APF and RRT\* (P-RRT\*) was presented. By introducing randomized gradient descent (RGD), the algorithm sufficiently reduces the number of iterations and the time and converges to the optimal path solution more quickly. The algorithm solves the local minima problem, but the real-time performance needs to be enhanced. In [64], an improved heuristic Bi-RRT algorithm based on the Bi-RRT algorithm was presented. Not only can dynamic obstacles in the real-time environment be avoided, but further improvements are also achieved in the efficiency of algorithm operation and path length, and smoothness.

#### *3.3. Velocity Obstacle (VO)*

To address the dynamic obstacle avoidance of USVs, Fiorini and Shiller [65] proposed the velocity obstacle (VO) approach in 1998 and incorporated the velocity characteristics of obstacles into the scope. However, the algorithm failed to consider the dynamic changes in the velocity of the obstacle and the dynamic performance of the USV. In [66,67], a VO-based dynamic avoidance algorithm was designed for USV to avoid moving obstacles. Since COLREGs are not considered, the experiments verify that USV is ineffective for moving and unknown obstacle avoidance, and the safety of the generated paths needs further enhancement. In [68], an enhanced velocity obstacle-based method with a particular triangular obstacle geometry model for autonomous dynamic obstacle avoidance of USVs was proposed. COLREGs should be considered to optimize the heading angle and improve the portability of the ship. It also reduces the USV path length and fuel cost by ensuring accurate obstacle avoidance. In [69], a hybrid VO-APF algorithm for obstacle avoidance of dynamic obstacles was raised, which enhances the real-time performance of obstacle avoidance and integrates COLREGS rules and path optimization functions. However, the algorithm can easily fall into a local optimum and cannot generate a globally optimal path. To tackle the situation of falling into a local optimum, in [70], a path planning method, based on the COLREGs, was displayed, combining the VO algorithm with the ACO algorithm to realize dynamic obstacle transitions relative to the stationary state of the USV at a given instant. The method can effectively avoid dynamic obstacles and obtain a reasonable risk avoidance strategy that provides the globally optimal path. However, owing to the uncertainty of the obstacle motion state, the predicted obstacle information i nevitably appears inaccurate. Based on the Generalized Velocity Obstacle (GVO) algorithm [71,72] proposed a collision avoidance system (GVO-CAS). The method, based on COLREG, visualizes the collision course and velocity for collision avoidance of multiple dynamic obstacles at close range. The algorithm takes into account the ship dynamics to improve collision avoidance performance and provides the globally optimal path. In [73], a novel efficient path planning algorithm was presented, the Constraint Locked Sweep Method and Velocity Obstacles (CLSM-VO). The algorithm optimizes global search performance, increases computational efficiency, and enhances generated path smoothness. It is suitable for complex environments with multiple dynamic obstacles and enables fast and effective dynamic obstacle avoidance by providing USV initial yaw angle and constraint layer optimization.

#### *3.4. Dynamic Window Approach (DWA)*

The Dynamic Window Approach (DWA) is a commonly used path planning method, originally proposed by FOX et al. [74,75] in 1997. The algorithm samples multiple velocity values in velocity space and simulates the trajectory generated by the robot at each velocity. To avoid the influence of dynamic obstacles, in [76], a "dynamic collision model" was introduced for the conventional DWA to predict possible future collisions, taking into account obstacle movements. The DWA algorithm for real-time path planning reduces wasted space and time by simulating the obstacle shape based on the obstacle. In [77], a local virtual target dynamic window (DW-VG) obstacle avoidance method was proposed. Introducing virtual target points selects the optimal amount of motion control. As a result, the algorithm can avoid all dynamic obstacles and predict an optimal path. In [78], the DWA-VO algorithm was proposed. The algorithm can speed up the search and improve the efficiency of the operation. It only considers local moving and unknown obstacles, which solves the information error caused by dynamic obstacles encountered by USV. It fails to identify accurately the global obstacles and leads to a local optimum. To address the condition of being trapped in a local minimum, ref. [79] proposed a path planning system with DWA and A\* algorithms, where USV can avoid unknown obstacles by selecting the optimal speed. USV can adapt to global and local maritime obstacles by choosing the best velocity to avoid unknown obstacles and avoid local optimum problems. In [80], an improved dynamic window approach (IDWA) was proposed to improve the efficiency of USV navigation. Combined with the idea of non-uniform theta\*, the reverse search is performed from the target node to reduce the path length and improve the navigation efficiency and the real-time performance of the USV. The algorithm prevents the USV from falling into local minima and enhances the ability of the USV to avoid dynamic obstacles at close distances. In [81], the ant colony optimization (ACO) and dynamic window approach (DWA) were combined to solve local path planning. The global path length is optimized by enhancing the search capability and removing redundant points through a dual population heuristic function. Adaptive navigation strategies are used to achieve obstacle avoidance for unknown dynamic obstacles and to solve the local optimum problem.

#### *3.5. Summary of Local Path Planning Algorithms*

The above research focuses on solving local optimization problems. Using the sensor's ability to sense the maritime environment, it can efficiently and quickly avoid dynamic obstacles, making the generated paths smoother, and can calculate safe routes. By using space vectors, employing extended search, and introducing greedy algorithms, the length of the path is reduced while reducing the solution time of the algorithm to meet practical needs. Using a dynamic collision model and considering the speed variation of dynamic obstacles, it can avoid all obstacles in real time. However, local paths ignore the effects of wind and wave currents in real sea conditions. There is no good solution to the problem in the face of complex situations. Table 2 compares different algorithms for local path planning and considers the factors of the optimal path.


**Table 2.** Characteristics of different algorithms for local path planning.


**Table 2.** *Cont*.

Note: consider (T), no consider (F), dynamic obstacle avoidance (DOA), global obstacle avoidance (GOA), \* (start).

#### **4. Proximity Responsive Hazard Avoidance**

Proximity-responsive hazard avoidance is where a USV, following a set course, needs to adjust its response in time to effectively avoid a collision with an obstacle if the uncertainty of a dynamic obstacle is detected suddenly. In contrast, close-range collision avoidance requires consideration of the vessel's shape and turning angle. Hazard avoidance and multi-vessel collision avoidance in complex marine conditions are also hot topics of current research.

#### *4.1. Hazard Avoidance in a Complex Environment*

For USV path planning in complex environments, an increasing number of hybrid algorithms are used to deal with the above problem. In [82], a multi-layer path planner (MPP) was proposed, which combines the fast marching method (FMM) [83] and the SDCE model to simulate a static environment. Collision Avoidance (CA) decisions are used to introduce safety parameters to achieve obstacle avoidance for dynamic and unknown obstacles and reefs in the coastal environment. The method does not consider the effects of wind, waves, and currents. In [84], a combined GA and APF approach was presented. It can perform path planning for various complex dynamic obstacle situations and find the best path among the obstacles on three sides around the starting point. However, the method only plans for simple geometries and coastlines, which does not work well in actual maritime environments. To consider realistic factors at sea, ref. [85] improved the multi-objective route planning method of PSO-GA and proposed a risk assessment model for wind and waves. Meanwhile, the length of the path, smoothness, and speed of the ship are taken into account. The genetic algorithm is used for cross-calculation, which improves operational efficiency. However, it does not consider the effect of ocean currents. If you want to consider the impact of ocean currents on path planning, building a kinematic USV model is a key choice, as shown in Figure 3.

In [86], an improved imperial competition algorithm (ICA) was proposed. Based on a three-dimensional current model and the influence of static obstacles at sea, the locally optimal solution is solved effectively and the AUV can obtain the global optimal path. However, the method cannot be adapted to dynamic obstacles. In [87], the steering angle of the ship is incorporated into the cost by building a cost model of the effect of ocean currents on the energy consumption of the AUV. The algorithm uses an improved D\* algorithm and considers the kinematic and dynamical equations to plan the path with the minimum energy consumption, but the path is not optimal. In [88], a VF-RRT\* algorithm was proposed, which improves its effectiveness in the current environment by introducing a virtual field sampling function and an ocean current constraint function. As a result, it greatly reduces the navigation time and energy consumption of the USV. In [89], an improved sparrow search algorithm (SSA) was designed, which considers the effects of time-varying characteristics of ocean currents, wind and wave factors, and dynamic obstacles on the navigation of AUVs in complex ocean environments. Convergence speed and search capability are improved by building a 3D underwater model and optimizing the coefficient weights. The algorithm can avoid all the obstacles underwater and find the globally optimal path in the complex ocean environment.

**Figure 3.** USV kinematic model.

Using machine learning algorithms is another way to deal with path planning in complex environments. In [90,91], a deep Q network (DQN) is developed for ship path tracking and heading control in calm seas without considering the actual sea environment. In [92], an improved DQN algorithm for vehicle path tracking using a nonlinear dynamic model was shown. It also ensures the safety and robustness of ship navigation in a windy and rough marine environment. Due to the complex environment of wind, waves, and currents, USV need to constantly change their speed and direction, considering the dynamics and kinematics of USV. In [93], a combination of Q-learning and neural networks (Q-NN) was proposed for path planning in unknown and harsh environments. It is able to avoid obstacles in real time and considers the kinematics of USVs, but it does not consider the effect of ocean currents. In [94], harsh environments such as wind, waves, and currents on USV path planning were considered. The optimization using the least squares strategy considers the dynamics and kinematics of the USV. However, the method is mainly applicable to static obstacle avoidance and textcolorred needs further breakthroughs for dynamic and unknown obstacles. In [95], an improved deep reinforcement learning algorithm (DRL) was presented. A two-dimensional ocean current model and a planar kinematic model are developed with energy cost as an important constraint. The global optimal path is solved by utilizing an improved reward function, action set, and state set and using B spline smoothing optimization.

#### *4.2. Multi-Vessel Collision Hazard Avoidance*

The multi-vessel collision problem is a common traffic accident at sea [96,97]. Currently, AIS ship trajectory data analysis is mainly used to require compliance with COLREG to prevent ship collisions as much as possible. The distance to the point of closest approach (DCPA) and the time to the point of closest approach (TCPA) are important indicators of a ship's collision risk [98]. Figure 4 shows the positional relationship between the ship and the target ship. The DCPA Collision Hazard Model is used to determine the distance from the vessel to the hazardous vessel for collision hazard assessment. In [99] DCPA and TCPA were combined to calculate hazard classes and analyses collision hazard classes in the context of ship safety domains, ship domains, and dynamic boundaries. A full-coverage algorithm is used to enable collision avoidance for multiple vessels. In [100], a full-coverage path planning algorithm based on deep reinforcement Q-learning neural network (DQN) was proposed, which uses convolutional layers to perceive the raster map and construct and train deep reinforcement learning. However, the traditional DQN suffers from low learning efficiency and low coverage. In [101,102], a method to improve its action selection

mechanism was proposed. Training data is selected based on the priority of the task. A dynamic reward mechanism ensures that valuable data is trained and reduces the risk of dangerous USV action selection during the learning process. The method improves the convergence and efficiency of full-area coverage, and can effectively avoid other vessels. As shown in Figure 5, the improved DQN algorithm can achieve full coverage, which is able to avoid all obstacles and has fewer repeated paths. It is also feasible to apply this algorithm to collision avoidance of multiple vessels. In [103], a sliding window algorithm (SWA) was raised for accurately detecting ship collision avoidance behavior from AIS trajectory data, but the real-time performance is not strong enough. In [104], AIS and spatial-temporal was used to analyze the massive data of vessel traffic conditions in past waters, and used simulated vessel traffic demand and temporal characteristics to predict vessel trajectories. The method achieves global ship collision avoidance and cannot monitor the risk of ship collision in real-time. For the issue of weak real-time monitoring of the risk of ship collision, in [105], using AIS data and ACO-APF algorithm, real-time dynamic avoidance is performed to plan a safe route. It achieves real-time dynamic collision avoidance for multiple vessels and effectively reduces the risk of collision vessels. In [106], a practical rule-aware time-varying conflict risk (R-TCR) for multi-ship collision avoidance was raised, considering vessel maneuverability and COLREGs. In [107], a synergy ship domain (SSD) was proposed to construct a collision risk model in a two-ship situation, which can identify the risk of ship collision in real time. By simulating frontal collision, side collision, and cross-collision, it identifies the dynamic changes in collision risk. For the local optimal problem in multi-objective complex situations, ref. [108] investigated collision avoidance considering COLREGs for solving multi-ship encounter problems. To obtain an optimal decision to avoid ship collisions, multiple objective decision-making (MODM) was combined with CGA. The algorithm keeps the ship moving safely as it avoids dynamic obstacles. In the case of multiple objectives, it takes a global view rather than just considering optimizing one target. In [109], a RACO combination of the ant colony algorithm (ACO) with the rolling window method (RWM) [110] was proposed. To address the complexity of collisions, the algorithm considers three possible approaches: frontal collision, side collision, and random collision. By introducing secondary safety distances and complying with COLREGs, it achieves real-time collision avoidance safely and effectively and finds the globally optimal path.

**Figure 4.** Vessel collision avoidance route map based on DCPA.
