Surveillance Unmanned Ground Vehicle Path Planning with Path Smoothing and Vehicle Breakdown Recovery

Abstract

As unmanned ground vehicles (UGV) continue to be adapted to new applications, an emerging area lacks proper guidance for global route optimization methodology. This area is surveillance. In autonomous surveillance applications, a UGV is equipped with a sensor that receives data within a specific range from the vehicle while it traverses the environment. In this paper, the ant colony optimization (ACO) algorithm was adapted to the UGV surveillance problem to solve for optimal paths within sub-areas. To do so, the problem was modeled as the covering salesman problem (CSP). This is one of the first applications using ACO to solve the CSP. Then, a genetic algorithm (GA) was used to schedule a fleet of UGVs to scan several sub-areas such that the total distance is minimized. Initially, these paths are infeasible because of the sharp turning angles. Thus, they are improved using two methods of path refinement (namely, the corner-cutting and Reeds–Shepp methods) such that the kinematic constraints of the vehicles are met. Several test case scenarios were developed for Goheung, South Korea, to validate the proposed methodology. The promising results presented in this article highlight the effectiveness of the proposed methodology for UGV surveillance applications.

1. Introduction

In recent years, autonomous vehicles have gained increasing attention in numerous applications. Autonomous surveillance systems are a representative example of such vehicles [1,2,3]. Typically, autonomous surveillance vehicles can be either unmanned aerial vehicles (UAVs) [4], unmanned ground vehicles (UGVs) [5], or unmanned marine vehicles (UMVs) [6]. The type of vehicle is dependent on the size of the environment, the presence of obstacles, and the resources available. Additionally, the vehicles can be used individually [7] or in fleets consisting of several vehicles [8].

In our paper, UGV fleet surveillance will be the focus. Specifically, a fleet management strategy that can generate the global paths for a fleet of several UGVs is proposed. The proposed methods yield the shortest paths to scan a large area while simultaneously producing feasible routes that obey the kinematic constraints of the UGVs.

In the literature on UGV path planning for surveillance, several different topics have been explored. Some researchers have explored the possibility of wirelessly controlling UGVs to conduct surveillance tasks [9,10,11]. In these applications, data were transmitted wirelessly through Wi-Fi, Bluetooth, or radio transmission while an operator manually controlled the vehicle to traverse the environment. This can reduce the manpower needed to perform surveillance tasks, but these approaches lack any intelligent decision-making and rely on human intervention. Therefore, other studies improved on this concept by applying some optimization techniques to find optimal paths in known environments. In one study, a heuristic path-planning algorithm was developed for road network surveillance [3]. In this study, the road network was modeled as a graph, where the sensor scanning radius was able to capture a large amount of the surrounding environment while traversing the roads. However, road network surveillance is simple when compared to surveying large open areas. In these cases, the environment should be discretized into a graph-like structure for optimization. One study focuses on this issue by developing a path-planning algorithm for autonomous surveillance that is computationally more efficient than the A* algorithm [12]. The authors also considered static obstacles in their study. Their heuristic algorithm was developed to provide the shortest path that is the furthest from obstacles to minimize the possibility of collisions.

In path-planning applications, a large area is typically divided into several smaller sub-areas (decomposition), and the optimal path within each sub-area can be calculated [13,14]. Solving for the minimum cost path within each sub-area can be modeled as the traveling salesman problem (TSP). However, in UGV surveillance applications, the sensors onboard the vehicle, such as LiDAR, typically have a large scanning range that can cover a portion of the surrounding area [15]. In these cases, the problem can be better modeled as the coverage path-planning problem (CPP). However, only a subset of nodes within an area (if it is discretized into a graph) need to be visited due to the large scanning range of the sensor. This problem is commonly referred to as the covering salesman problem (CSP) [16].

To solve the CSP within a sub-area, several methods have been proposed. One of the more trivial methods is the random walk (RW). In the RW algorithm, a linear or spiral path can be calculated for a random travel direction. Then, the next trajectory is randomly selected when the boundary of the sub-area is reached [17]. The RW typically yields sub-optimal results and lacks logical decision-making strategies. The genetic algorithm (GA) has also seen great success in solving the TSP or CSP [18,19,20]. By mimicking the evolutionary process through crossover and mutation, the GA uses the concept of survival of the fittest to continuously improve a population of solutions with respect to the problem’s objective [21]. Additionally, the ant colony optimization (ACO) algorithm has been used to solve the TSP [22,23,24], but little research has applied ACO to the CSP [25]. In the ACO, a colony of ants is dispersed into a graph to find the shortest path from the nest (start) to the food (goal). Over several iterations, the best path is exploited through a pheromone rewarding scheme [26]. ACO has been applied in several other related applications, such as vehicle routing [27], mobile robotics [28], and task scheduling [29], thus making the ACO algorithm a good starting point for solving the CSP.

When a fleet of UGVs is used, an effective scheduling algorithm should be used to assign the UGVs to sub-areas. Compared to using a single UGV, fleet operations can reduce the overall travel time required and distribute the workload amongst the vehicles [30]. The scheduling algorithm is also responsible for generating the total path for all vehicles that begin at a depot (start location) and typically end at the depot or any other specified ending location. In the literature, several approaches have been proposed to schedule the vehicles, such as the GA [31,32], and in other similar applications, tabu search (TS) [33] and simulated annealing (SA) [34]. The GA has shown great success and can effectively handle complex constraints commonly seen in scheduling problems. Other studies have had success using more trivial approaches such as assigning sub-areas to vehicles until the vehicle constraints are violated and then using zig-zag patterns within each sub-area to generate the coverage path [14].

Typically, the paths produced by any route optimization algorithm do not consider the kinematics of the vehicles. Since including the kinematic constraints in the optimization process can significantly increase the computation time, it is typically avoided. This means that the paths produced are initially infeasible and thus require further processing to modify them such that the vehicle turn radius is properly accounted for.

One study uses Bezier curves to estimate a smooth centerline for UGVs to follow on roadways [35]. The problem with Bezier curves is that the curvature is difficult to control, and the paths produced by the Bezier curve in [35] are dependent on the boundary of the areas where they are generated. To gain better control of the curves, a Dubins curve can be used [36]. The Dubins curve can find the optimal path between a pair of waypoints with desired trajectories while considering a minimum turning radius. However, the UGV in the Dubins curve can only travel forward. This means that the Dubins curve often produces large turns to reach the target location and trajectory. One way to avoid such large turns is to allow forward and backward motion. The Reeds–Shepp path can be used to find the minimum cost path between a pair of points with starting and ending trajectories while considering a minimum turn radius and allowing forward and backward motions. The Reeds–Shepp path is used to enhance the feasibility of a path by modifying its waypoints with Reeds–Shepp curves [37]. However, the Reeds–Shepp path can add additional traveling when it is used to post-process a path.

Due to unforeseen events, UGVs may become temporarily unavailable during operation. When this occurs, the remaining portion of the path assigned to the broken vehicle should be distributed amongst the operating vehicles. There are several different approaches explored in the literature to achieve this. In one study, a first-responder (FR) approach is proposed where the first vehicle that completes the scheduled paths can service the remainder of the broken vehicle’s path [38]. However, this study does not use any optimization approaches. Therefore, it was improved in a future study by the same authors. In their subsequent study, the FR was improved by implementing the cooperative autonomy for resilience and efficiency (CARE) algorithm that uses a game–theoretic structure to reassign the path of the out-of-service vehicle amongst the fleet [39]. A different approach is to use the sub-area scheduling methodology, but the sub-area paths can be updated according to the coverage progress and the available vehicles can be updated by removing the broken UGVs [32].

From the above literature, several different approaches to solving the CSP have been discussed. However, very few papers have explored using the ACO algorithm to generate the optimal path for the CSP. Furthermore, no studies have used the ACO algorithm to generate paths in fleet UGV surveillance applications. In this study, the ACO algorithm will be used to generate optimal scanning paths within sub-areas for surveillance applications, while a scheduling algorithm will be used to distribute the vehicles amongst the several sub-areas. ACO was also selected due to its exceptional global optimization performance and seamless transition for parallel computation. Since the ACO algorithm has not been used for UGV surveillance applications, this is one of the contributions of the research presented.

The UGV paths can then be post-processed such that the kinematic constraints of the vehicle are considered. Several different approaches have been discussed in the above literature; however, each has its pros and cons. In this study, a combination of two different approaches will be used to improve the path curvature. The first approach is the corner-cutting method, and the second approach is a Reeds–Shepp curve. As previously discussed, the Reeds–Shepp curve adds additional traveling with its forward and backward motions. Therefore, it is used alongside an additional algorithm (the corner-cutting method) to minimize the distance added by the Reeds–Shepp curve. Since this combination of path refinement strategies has not been explored in existing studies, this is another contribution of the following research.

Additionally, several recovery strategies for breakdown scenarios have been proposed in the literature for vehicle fleet scheduling. In this study, the higher-level path generation approach from the authors’ previous work was adapted to the UGV surveillance problem for real-time scheduling. This is the final contribution of our study since existing research has not considered the breakdown scenario in UGV surveillance applications.

A study for the proposed approaches was conducted in Goheung, South Korea, and the results will be presented to highlight the effectiveness of the developed technology. The results presented in this article were generated from simulations. The authors took measures to ensure that the simulations could replicate the physical aspects of the project (such as including the sensor scanning range and vehicle turn radius). Additionally, GPS data were supplied by our research partners to define the testing environment that included the outer boundary, several pre-defined sub-areas, and the location of static obstacles (buildings, etc.). The remainder of this article is organized as follows: Section 2 covers the route generation methodology, Section 3 discusses the study parameters, Section 4 shows the results for several test scenarios in the study, and Section 5 concludes the article.

2. Route Generation Methodology

The developed methodology uses a multi-level approach that pre-processes routes in sub-areas on the lower level and schedules the vehicles to create the total path on the higher level. The graph formulation and the lower and higher-level path generation algorithms will be discussed in detail in this section. A pseudocode representation of the developed approaches can be seen in Algorithm 1.

Algorithm 1 Total Algorithmic Flow: Lower-Level and Higher-Level Path Generation

Input: Sub-area graphs with candidate start locations, total area graph, current vehicle locations, and vehicle route progress

## Run lower-level path generation on all sub-areas to create several routes for each area

for sub_area in sub_areas do

for start_point in sub_area do

run lower-level path generation algorithm to create route

resolve infeasible turns in route

## Run higher-level path generation to create the total route for each vehicle

## The higher-level path generation waits for vehicles to finish the assigned route, or for a vehicle to breakdown so it can re-generate the routes

while True do

run higher-level path generation algorithm to create total paths

wait for all_vehicles_finsihed or vehicle_breakdown

if all_vehicles_finsihed do

set False

elif vehicle_breakdown do

update current_vehicle_locations and vehicle_route_progress

continue

Output: coverage paths for each vehicle

2.1. Graph Formulation

To use the proposed methodology, the targeted area can be modeled as a graph. Any method can be used in this process; however, the resolution has an impact on the quality of the graph produced. As a result, the optimal graph resolution can be selected through trial and error. Based on previous experience, the resolution can be set to twice the length of the vehicle.

The graph should consist of many edges such that the optimizer has the freedom to select the most direct path to nodes that are far from the current node. Therefore, the graph can be strongly connected for all nodes that allow a direct connection. In some cases, the direct path between a pair of nodes crosses over a boundary or static obstacle. In these cases, a direct connection cannot be made. The graph can be modeled as an undirected graph where all nodes are strongly connected if the direct path does not cross a boundary or static obstacle. To generate the graph within the serviceable area (used by the lower-level path generation), a grid of nodes spaced at the specified resolution was created within each sub-area. Then, nodes inside of obstacles were removed, and direct connections (edges) between all remaining nodes were made. Edges that passed through obstacles or outer boundaries were omitted since these direct routes were infeasible. By doing so, a strongly connected graph with direct connections between all nodes was created. Then, each sub-area’s graph can be connected to neighboring sub-areas to create the total graph used by the higher-level path generation. By connecting all sub-areas, the higher-level path generation algorithm can find the shortest path between sub-areas.

Additionally, there should be some deadhead (non-serviceable) distance between the outer boundary of the targeted area and static obstacles. The buffer distance reduces the possibility of collisions and creates some space to allow the vehicle to turn (if necessary). An example of a graph created in a small area can be seen in Figure 1. In Figure 1, the graph representation of the serviceable area does not extend directly to the border. This is the effect of the buffer distance.

For the following methodology, a large area can be divided into several smaller areas. Since several vehicles are dispatched to cover the entire area, it can be divided into several smaller areas that can be assigned to each vehicle. This can be carried out manually or automatically; however, it is recommended that there is at least the same number (or more) of sub-areas as vehicles available.

2.2. Lower-Level Path Generation: Ant Colony Optimization (ACO)

In the following approach, the lower-level path generation method was used to pre-compute several routes within each sub-area. The paths are dependent on the start location selected; however, several start locations can be used to create a collection of candidate paths. Eventually, the higher-level path generation method will select the optimal start location to yield the least total distance.

The lower-level path generation method is based on the ACO algorithm used to solve the CSP. The basic principles of the ACO and its modifications are explained in this section.

2.2.1. Foundation and Basic Principles

The ACO algorithm was developed for solving discrete optimization problems in a relatively short amount of time [26]. The ACO algorithm falls under the metaheuristic algorithm umbrella. The ACO algorithm uses a simulated colony of ants (search agents) to find near-optimal solutions to complex optimization problems. The ants exploit good solutions through a pheromone update scheme while they explore the solution space through a probability selection stage in graph-like structures.

The ACO can best be explained through an example. In a graph, the ants begin at a “nest” location (start), and they must find the shortest path to the “food” location (end). Initially, all ants travel along a random path until the food is reached. However, the edges in the shortest path found by the colony are rewarded with a pheromone increase, and the remaining edges undergo pheromone evaporation after each iteration. In the following iterations, the ants are more likely to select edges with higher pheromone levels. This can be visualized in Figure 2.

The node selection probability can be seen in Equation (1), and the pheromone updating scheme can be seen in Equation (2). In Equation (1), $p_j$ is the probability of travelling to node $j$ from node $i$, ${\tau }_{ij}$ is the pheromone level on edge $ij$, $d_{ij}$ is the distance of edge $ij$, $\alpha$ is the pheromone influence factor, $\beta$ is the distance influence factor, and $A$ is the set of candidate nodes to visit. In Equation (2), $L$ is the distance of the best-found solution and $x_{ij}$ is a binary indicator used to reward edge $ij$ if it belongs to the best-found path.

$$p_j={\displaystyle \frac{{\tau }_{ij}^{\alpha }{d}_{ij}^{-\beta }}{{{\displaystyle \sum }}_{kϵA}{\tau }_{ik}{d}_{ik}^{-1}}}$$

(1)

$${\tau }_{ij}=\left(1-\rho \right){\tau }_{ij}+{\displaystyle \frac{1}{L}}{x}_{ij}$$

(2)

As seen in Equation (1), the pheromone levels and edge distance play an important role when it comes to the edge selection stage. As a result, the pheromone and distance influence factors should be carefully tuned to obtain optimal results. Additionally, the ants travel in a stochastic manner. This means that they randomly select the next edge to travel along with a probability that is proportional to the pheromone levels and edge distance.

Also, the $L$ parameter in Equation (2) does not have to be the length. This is the fitness of the path created. In some cases, achieving the minimum distance is not the objective of the optimization problem, so this can be changed accordingly.

2.2.2. Modifications for the Coverage Salesman Planning Problem (CSP)

Since the ACO algorithm is being used to find the shortest distance path that covers the desired area, there are some changes required to the traditional ACO algorithm. Specifically, the traditional ACO algorithm can be used to find the shortest path from point a to point b. However, this is not the case for this research. Since the sensors onboard the surveillance vehicles have a specific scanning radius, a portion of the surrounding area is scanned at each instance. In graph theory, this is modeled as the CSP, where only a portion of nodes should be traversed to cover the desired area.

To make this modification in the ACO algorithm, the set $A$ in Equation (1) can be modified. Typically, $A$ would contain all nodes that are adjacent to node $i$. However, $A$ was modified to include only adjacent nodes that have not been covered by the sensor. This means that the set of scanned nodes should be updated after each selection made by the ants. Additionally, the termination condition is the desired area coverage. Therefore, the ants will continue to travel through the target area until the desired area coverage is met. One of the challenges with this approach is finding an appropriate coverage ratio input. When the desired coverage input is 100%, the ants travel significantly more to cover every node. However, when the coverage is near 100% (around 95%), the ants yield more efficient paths with less overlapping. This highlights the tradeoff between the desired area coverage and the path distance. Therefore, the coverage ratio should be selected such that an appropriate coverage ratio is obtained in a relatively short path. The node coverage scheme is visualized in Figure 3, where a hypothetical sensor radius is highlighted in yellow.

2.3. Higher-Level Path Generation: Genetic Algorithm (GA)

In the following approach, a higher-level path generation method was used to distribute the vehicles to scan several sub-areas. Each vehicle begins at a starting location (which can be different for each vehicle) and travels to a sub-area, scans the sub-area, and then either continues to the next sub-area or travels to the ending location.

The higher-level path generation method is based on the GA with some minor modifications. Specifically, modifications to the crossover and mutation operators were made to meet the constraints of this study. The basic principles of the GA and the modifications to the crossover and mutation operators are explained in this section. The higher-level path generation algorithm was adapted from the authors’ previous research [32].

2.3.1. Foundation and Basic Principles

Like the ACO algorithm, the GA was developed to solve discrete optimization problems [21] but has also demonstrated success in continuous optimization. The GA belongs to the evolutionary class of metaheuristics. The GA mimics the evolutionary process using crossover and mutation operators, and survival of the fittest. Over the course of many generations, the solutions will continue to improve until the termination condition is met. The overall flow of the GA is visualized in Figure 4.

The GA requires a solution to be encoded into a structure called a chromosome, where each gene contains important information used to calculate the fitness of the solution. A population consists of a collection of chromosomes, and the fitness of a chromosome is used to quantify how good a solution is.

Initially, a population of randomized chromosomes is created, and then the fitness of each member is evaluated. This means that the average fitness in the first generation is bad since there has not been any constructive reward system applied. To improve the fitness of the population, GA operations such as selection, crossover, and mutation can be used.

To select parents, a roulette-style selection is commonly used. This means that chromosomes with a better fitness are more likely to be selected for reproduction. A roulette-style selection can be used to choose two unique parents to perform the crossover and mutation.

The crossover operation occurs when a crossover point is selected in the chromosomes, and the genes from each chromosome are spliced together with respect to the crossover point. This process creates two unique children that can then be mutated. The mutation randomly changes one (or several) of the genes in a chromosome with respect to the problem constraints. Typically, the probability of crossover is larger than the probability of mutation. A large crossover probability allows the population to efficiently explore the solution space, while the low mutation probability allows each chromosome to search the local vicinity of the solution space.

2.3.2. Chromosome Structure

For this research, the GA was used to assign vehicles to the desired sub-areas. The objective of the optimization was to minimize the total travel distance required to scan the entire area. Therefore, the chromosome structure should reflect the sub-areas, the vehicle assigned to the sub-area, the corresponding sub-area’s starting location, and the order of operation (if one vehicle is assigned to multiple areas). Several constraints are applied to the chromosomes to ensure that a feasible solution can be generated. The length of a chromosome is equal to 3 times the number of available areas (in accordance with the chromosome structure, which follows the sub-area, starting point, and vehicle assignment), where each sub-area should appear once, and each vehicle should appear at least once. For each sub-area, the starting point can be selected out of a set of candidate starting locations, each with their own unique path created by the lower-level path generation. Additionally, the vehicle assigned to each area can be selected out of a set of available vehicles. The chromosome structure is visualized in Figure 5.

2.3.3. Crossover Operation

Since the crossover operation mixes the genes of two solutions, any index in the chromosome cannot be selected as the crossover point. This is because crossover should change the order in which the sub-areas are serviced. Therefore, the vehicle-area assignment and starting point should be preserved.

To achieve this, the chromosome can be encoded into “chromosome blocks” that group every three genes together (sub-area, starting point, and vehicle assignment). The crossover point can be selected such that the chromosome blocks are preserved by selecting indexes that are divisible by three. The chromosome blocks and possible crossover points can be seen in Figure 6.

2.3.4. Mutation Operation

The mutation operators are meant to explore neighboring solutions by making small changes in the chromosomes. For this research, two mutation operators are proposed: vehicle and starting point mutation. The vehicle mutation occurs when a new vehicle is randomly selected for the corresponding chromosome block, and a starting point mutation is carried out when a new starting point is randomly selected for the chromosome block.

The mutation can only happen once per chromosome block and the mutation operator used is randomly selected with an equal probability of being a vehicle or starting point mutation. This is to prevent over-searching in the local solution space. To perform the mutation, the chromosome blocks are iterated over, and a random number is generated. If it is less than a predefined mutation probability, mutation will occur. However, the mutation operator is randomly selected as either a vehicle or starting point mutation. The vehicle mutation has the potential to remove vehicles from the chromosome. This means that some vehicles can be excluded from operation in some solutions. This presents a unique challenge since all vehicles should be used in operation. To solve this issue, vehicle mutation is only performed if changing the current vehicle assignment does not completely remove a vehicle from operation. The mutation process can be seen in Figure 7.

2.3.5. Objective Function

The higher-level path generation uses the pre-computed paths calculated by the lower-level path generation algorithm. Therefore, only one of the start points for each area is selected. However, the GA can select the optimal starting location such that the total vehicle distance is minimized.

The objective function for the higher-level graph generation can be seen in Equations (3) and (4). In Equation (3), $f$ is the objective function, which is to minimize the sum of distances traveled by each vehicle $v_i$. The vehicle distance can be calculated in Equation (4), where $d_k$ is the distance from the vehicle’s current location to the next sub-area’s starting location. When $k=0$, $d_k$ is the distance from the vehicle starting location to the starting location of the first sub-area assigned to vehicle $i$. When $k=l$, $d_k$ is the distance from the ending location of sub-area $l$ to the vehicle ending location.

$$f=\min {{\displaystyle \sum }}_{i=0}^j{v}_i$$

(3)

$$v_i={{\displaystyle \sum }}_{k=0}^l{d}_k$$

(4)

The objective function can be modified according to the problem requirements. In this case, the total distance (sum of all vehicle distances) was to be minimized. However, some challenges in the UGV surveillance domain are related to selecting an appropriate objective function to achieve the desired outcome. Several other objectives can be achieved, such as minimizing the overall travel time (maximum vehicle travel time), or balancing the workload across all UGVs. Selecting an appropriate objective function also requires communication between research partners such that the objectives are mathematically and physically feasible.

2.4. Feasible Path Generation: Turn Radius Consideration

The routes created by the lower and higher-level path generation methods do not consider the turn radius of the vehicle. As a result, the paths are initially difficult for the vehicles to follow. Therefore, additional methods were applied to make the optimal paths feasible with respect to the vehicle’s kinematics. Feasible path generation is a post-processing event that modifies the paths produced by the lower and higher-level path generation methods. Feasible path generation consists of two methods that can be applied in different use cases: the corner-cutting method and the Reeds–Shepp method.

2.4.1. Corner-Cutting Method

In some cases, the angle between two consecutive line segments is not too large. This means that the turn is not too sharp. When this is the case, the sharp corner can simply be rounded off with respect to the vehicle’s turn radius. This process is the corner-cutting method and is visualized in Figure 8. However, this approach may not be ideal for vehicles with a large turn radius. In these cases, the corner-cutting method can reduce the area coverage significantly by skipping a large portion of the original path. Therefore, this method should be used with caution when vehicles have a large turn radius.

It should be noted that the corner-cutting method is only feasible in cases where the angle is not too small. If the corner-cutting method is applied to sharp turns, too much of the desired path will be removed, thus reducing the total area coverage seen by the sensors. To prevent the corner-cutting method from removing too much of the desired path, a threshold was defined. If the portion of the desired path being removed is longer than twice the vehicle turn radius, then an additional method can be used. A visual representation of this condition can be seen in Figure 9.

2.4.2. Reeds–Shepp Method

The Reeds–Shepp path was developed to find the shortest path between two points that considers a minimum turn radius and allows forward and backward motion [40]. The Reeds–Shepp curve also requires starting and ending trajectories so that the turn radius can be properly incorporated.

The Reeds–Shepp curve is used when the portion of the desired path being removed by the corner-cutting method is larger than twice the turn radius. This is the only condition where the Reeds–Shepp curve is used since it is a comparatively complex maneuver involving forward and backward motions that add additional travel to the path.

To use the Reeds–Shepp path to resolve sharp turns, the start/end point must be where the two path segments meet each other, the start trajectory is the trajectory of the path before the turn, and the end trajectory is the trajectory of the path after the turn. Using these values, the Reeds–Shepp algorithm can be used to calculate the shortest forward/backward curved path that can be added to resolve the sharp turn. The Reeds–Shepp algorithm used in this research was borrowed from Atsushi Sakai [41]. The process of using the Reeds–Shepp algorithm is visualized in Figure 10. As seen in Figure 10, the Reeds–Shepp path makes sharp turns feasible by adding some additional distance to perform the forward/backward turns such that the desired trajectories can be achieved while considering the vehicle turn radius. Like the corner-cutting method, the Reeds–Shepp method may not be ideal for vehicles with a large turn radius. In these cases, the added maneuver will be large, increasing the travel distance, and potentially colliding with obstacles. To limit the possibility of collisions, the distance buffer between the outer boundary of the area and obstacles should be increased in the graph generation stage. However, this reduces the coverage ratio since the vehicle cannot get close enough to the border, and it may restrict movement in tight spaces.

In the study, the corner-cutting method was applied where applicable, and the Reeds–Shepp method was used when the corner-cutting method removed too much of the desired path. This means that the resulting path consists of smooth turns (from the corner-cutting method) and three-point turns (from the Reeds–Shepp method).

2.5. Real-Time Scheduling for Breakdown Scenarios

In the real world, unexpected events such as vehicle breakdowns can be detrimental to servicing operations. In these cases, the remaining portion of the path assigned to a broken vehicle may be forgotten if the breakdown scenario is not properly addressed. When a vehicle breakdown occurs, the higher-level path generation can be used to reassign the remaining paths amongst the unbroken vehicles.

To effectively handle the breakdown scenario, all vehicles must keep track of path coverage progress. This is because the sections of the paths that have been covered do not need to be covered again by the other vehicles. Therefore, when a vehicle breaks, only the remaining portion of the paths will be used in the higher-level path generation. The higher-level path generation has the capacity to select the optimal start point (beginning or end of the remaining path) for all paths while reassigning vehicles to the lower-level paths, such that the sum of all vehicles’ distances is minimized. The path-updating scheme is visualized in Figure 11, and a pseudocode representation of this process can be seen in Algorithm 2.

Algorithm 2 Real-Time Scheduling for Breakdown Scenarios

Input: Paths assigned to each vehicle, vehicle locations, broken vehicle(s)

## Calculate the remaining portion of each vehicle route

## This portion updates the paths for the serviceable areas at the point of a breakdown

for vehicle in vehicles do

if current_location is servicing

use current_location to split servicing_path

update start_points for servicing_path

##Run the higher-level path generation to redistribute the remaining serviceable areas

## The broken vehicle is not considered for this operation

run higher_level_path_generation without broken_vehicle(s)

Output: coverage paths for each vehicle without the broken vehicle

3. Study Parameters

To validate the proposed methodology, a study was conducted in Goheung, South Korea. In this study, several autonomous surveillance vehicles were used. The vehicle and area specifications will be discussed in this section.

3.1. Vehicle Specifications

Several identical autonomous surveillance vehicles were used in the following study. The vehicles are 4.00 m long and 1.65 m wide. The vehicle dimensions can be seen in Figure 12, and the simulation parameters can be found in

Table 1. Vehicle simulation parameters.

Parameter	Value
Travel Speed	15 km/h
Minimum Turning Radius	5.60 m
Scanning Range	25.0 m

.

3.2. Area Specifications

For the following study, a large area with several static obstacles (buildings) was considered. For logistical reasons, the large area was divided into several smaller sub-areas that the vehicles should scan. This process can be performed manually or automatically, but it was performed manually to preserve the pre-defined areas outlined by our research partners. A comparison of the satellite image and the GPS data representation of the targeted area can be seen in Figure 13.

For this study, the buffer distance used for generating the graph was set to 2.24 m, which is 0.4 times the vehicle’s turn radius. This buffer distance allows the vehicle to perform turning maneuvers near obstacles while ensuring that the vehicle does not collide with the obstacles. In this study, the buffer distance is significantly less than the scanning radius of 12.5 m (scanning range: 25 m), so this does not have any effect on the coverage. However, this parameter can vary in different case studies and should be carefully selected. Additionally, a resolution of 8 m was selected for generating the graphs. This is also equivalent to twice the length of the vehicle. To put this into perspective, there is 58,049 m² of scannable area for the case study in Goheung. Like the buffer distance, the resolution is still smaller than the scanning radius, so the overall coverage is not affected by this. This value (8 m) was selected through trial and error to yield a graph partition that could generate ideal results within a timely manner since finer resolutions increased the computation time.

Because the lower-level path generation pre-processes several paths within each sub-area, several starting points in each sub-area were selected. The starting points were selected at positions where adjacent sub-areas join each other, as well as some of the expected vehicle starting/ending locations. The starting points in each sub-area can be seen in Figure 13c. Each sub-area has several possible starting points. The candidate start points for an area are visualized using colored dots, where the starting points in the same area share the same color. When the higher-level path generation algorithm is executed, it can select one of the colored dots for the corresponding sub-area to begin the scanning route, which is pre-processed using the lower-level path generation algorithm. Pre-processing several paths for each sub-area can minimize the total distance by selecting optimal starting points for each sub-area. The lower-level paths were also pre-processed to save computation time in real-time applications.

4. Results and Discussion

The parameters used to create the paths can be found in

Table 2. Parameters used in the lower- and higher-level path generation algorithms.

Algorithm	Parameter	Value
Lower-Level Path Generation (ACO)	α (Pheromone Influence)	0.1
	β (Edge Distance Influence)	0.3
	ρ (Pheromone Evaporation Rate)	0.4
	Number of Ants *	24
	Iterations	250
Higher-Level Path Generation (GA)	Generations	60
	Population Size	20
	Crossover Probability	0.9
	Mutation Probability	0.05
	CPUs *	5

* These parameters are used for parallel computing.

. In the table, there are two parameters that rely on parallel computing processes, which include the number of ants in the lower-level path generation and the number of CPUs used in the higher-level path generation. In the lower-level path generation, a single ant was simulated by each CPU in parallel. By doing so, the computation time for each iteration is significantly reduced. In the higher-level path generation, the CPUs were used to compute chunks of the population in parallel. Specifically, five CPUs were used to compute four solutions each (for a population size of 20) in parallel. When more CPUs are used, more time is required for resource allocation. When fewer CPUs are used, each CPU computes too many solutions. Therefore, the number of CPUs was optimized to yield the least computation time such that the resource allocation time and population distribution were effectively balanced. Additionally, the ACO parameters were selected based on various simulated experiments to yield satisfying results within a manageable timeframe. As for the GA, these parameters were selected based on a general rule-of-thumb (high crossover and low mutation) combined with previous project experience with route optimization and scheduling [28,33].

To evaluate the effectiveness of the proposed methods, several different test scenarios were proposed. A summary of the test scenarios can be found in

Table 3. Test case scenarios for the proposed research.

Scenario	Number of Vehicles	Operating Conditions	Starting Locations
1	3	Normal	Different Starting Locations 1
2	3	Normal	Same Starting Location 2
3	3	Normal	Same Starting Location 3
4	5	Normal	Same Starting Location 3
5	3	1 Vehicle Breakdown After 700 m	Same Starting Location 3
6	5	2 Vehicles Breakdown After 500 m	Same Starting Location 3

¹ The starting locations are depicted in Figure 14a. ² The starting location is depicted in Figure 14b. ³ The starting location is depicted in Figure 14c.

, and the respective vehicle starting locations can be seen in Figure 14. As highlighted in Table 3, two breakdown scenarios were used to observe the feasibility of the real-time scheduling capabilities of the higher-level path generation. In these scenarios, the rest of the path left by the out-of-service vehicle should be covered by one (or several) of the remaining vehicles.

The total distance and overall travel time for all scenarios are summarized in

Table 4. The results from each test scenario. The total distance is the sum of all the vehicle distances for each scenario, and the overall time is the maximum travel time for all vehicles in each scenario.

Scenario	Vehicle	Distance (m)	Time (min)	Total Distance (m)	Overall Time (min)	Computation Time (s)
1	1	1364	5.40	5962	9.44	80
	2	2239	8.96
	3	2359	9.44
2	1	2718	10.87	6239	10.87	81
	2	1348	5.39
	3	2173	8.69
3	1	2293	9.17	5246	9.17	85
	2	1157	4.63
	3	1796	7.18
4	1	1396	5.58	6943	6.60	82
	2	1649	6.60
	3	1621	6.48
	4	1120	4.48
	5	1157	4.63
5	1	2479	9.92	5769	10.10	83
	2 *	765	3.06
	3	2525	10.10
6	1	1429	5.71	7014	12.64	81
	2 *	549	2.2
	3	3160	12.64
	4 *	563	2.25
	5	1313	5.25

* Breakdown vehicles.

. As seen in Table 4, the starting location for the vehicles has an influence on the total distance and overall operation time. This is evident in Scenarios 1 to 3 (the only difference in these scenarios is the starting location, but different statistics are seen). However, the vehicle starting/ending locations were not optimized in this study. The vehicle’s starting/ending locations are simply places where the vehicles are stored and thus used as an input to the developed path generation methods. Also, an arbitrary ending point was selected for this study. Therefore, all vehicles will park in the same location at the end of the route. This may not be the case in real-world scenarios; however, modifying the ending starting/ending locations for the vehicles can easily be changed in the proposed methodology. Another observation is that the total travel distance increases when more vehicles are used, but the overall travel time is decreased.

Comparing the breakdown scenarios to the normal ones, more significant differences are observed. Specifically, comparing Scenario 3 to 5, the only difference is that Vehicle 2 breaks down, but the total distance and overall travel time increase. In Scenario 5, Vehicle 2 breaks down and Vehicles 1 and 3 split the remainder of Vehicle 3′s assigned path. This effect is more drastic in cases with more vehicles, as highlighted in Scenarios 4 and 6. In Scenario 6, Vehicles 2 and 4 break down, initially leaving the remaining three vehicles to service the areas left by the broken vehicles., However, only Vehicle 3 is assigned the remainder of the path originally intended for the broken vehicles. The proposed algorithm has the flexibility to select multiple vehicles to service the remainder of the path, but assigning it only to Vehicle 3 is optimal in this case. As a result, the workload is drastically increased for Vehicle 3, resulting in a significant increase in the overall time. Since the objective of the higher-level path generation is to minimize the total distance, the overall travel time was not of interest. This results in similar total distances between the normal and breakdown scenarios but an increase in the overall time.

Having more sub-areas could lead to more flexibility in the higher-level path generation algorithm, thus reducing the overall time by effectively balancing the workload. However, this was not considered for this study.

The paths made by the proposed methods were made feasible using the corner cutting or Reeds–Shepp method. A sample of the scanning path can be seen in Figure 15. As seen in the figure, each method can be applied when it is appropriate, resulting in a path that consists of smooth turns and forward–backward motions. The total paths for all vehicles in all 6 scenarios can be seen in Figure 16.

5. Conclusions

In this research, a two-stage routing algorithm was proposed to generate optimal paths for a fleet of autonomous surveillance vehicles: the lower-level and higher-level path generation algorithms. First, the total area was divided into several sub-areas for logistical purposes. Then, optimal paths were pre-processed for several different start points within each sub-area using a modified version of the ACO algorithm fit for the CSP problem. This process is called lower-level path generation. Then, a GA was used to assign the vehicles to the sub-areas while selecting the optimal starting point such that the total distance was minimized. This process is called higher-level path generation. The paths generated by these approaches were initially infeasible due to the harsh turn angles. However, a feasible path generator was developed to modify the paths such that the turn radius of the vehicle was properly accounted for. To do so, a corner-cutting method was used for less sharp angles, and the Reeds–Shepp path was used for more harsh angles. Additionally, breakdown scenarios can be effectively handled by the higher-level path generation. This was achieved by updating the remaining portion of all paths and redistributing them amongst the operating vehicles. A study was conducted in Goheung, South Korea, for a fleet of autonomous surveillance vehicles. Several different scenarios were proposed that involved different start locations, numbers of vehicles, and vehicle breakdowns. The results presented in this article highlight the effectiveness of the lower-level, higher-level, and feasible path generator for autonomous UGV applications.

In this study, several novelties were achieved. Firstly, this study is one of the first to successfully use the ACO algorithm for the CSP model. This was achieved by modeling the sensor scanning radius such that the nodes surrounding the vehicles’ current locations are also visited. This resulted in paths that do not need to physically visit all nodes, but instead, only a subset of nodes should be visited to achieve the desired coverage. Secondly, this study proposes a method of real-time scheduling for breakdown scenarios that redistributes the workload amongst all vehicles upon vehicle breakdown. This was achieved by executing the higher-level path generation algorithm with the remaining vehicles and their respective paths at the point of a breakdown. Finally, a path refinement algorithm was proposed to make the vehicle paths feasible with respect to the kinematic constraints. This was achieved by using a corner-cutting method (with respect to the vehicle turn radius) at each corner where this operation would not cause too much simplification. Then, the Reeds–Shepp algorithm was used where the corner-cutting method could not be applied. The Reeds–Shepp path would add some additional maneuvers such that the shortest forward and backward path can provide a smooth transition at sharp corners. Existing research has proposed some solutions to this issue; however, the corner-cutting and Reeds–Shepp combination is one of the novel approaches developed in this research.

For this research, the vehicle battery was not considered. This is because the operation time for the battery is 4 hr. The results presented in this article show that the longest operation time was 12 min; thus, there will always be enough power for the selected study. However, this may not always be the case for different studies. Therefore, future studies can incorporate an effective constraint-handling strategy into the higher-level path generation. Also, an arbitrary depot location was selected for this study (to test the feasibility of the proposed methods). This can be modified in future studies to model the appropriate ending location for all vehicles. Finally, future studies can include generating local path-tracking algorithms for autonomous surveillance vehicles such that they can follow the paths produced by the developed approaches.