1. Introduction
Marine oil spills have a considerable economic and ecological impact on the ecosystem of marine life. Incidents with oil spills occur with some regularity during the exploration, production, and transport of petroleum products [
1]. Between 2010 and 2023, there were 90 spills with 7 tonnes or more, resulting in 192,000 tonnes of oil spills in the environment, with 91% of this amount spilled in 32 incidents alone [
2]. Such incidents require immediate, simple, effective, and eco-friendly actions to minimize environmental damage. This level of importance in having a fast intervention is detailed in
Figure 1, with satellite radar images from Copernicus Sentinel-1 over the Mediterranean Sea, north of the French island of Corsica, after a collision between two merchant ships. The first radar image shows that the oil spill was about 20 km long. Twelve hours later, imagery from the same position shows that the oil spill had lengthened to about 35 km. And 24 h later, the spill had grown to about 60 km long. Therefore, it is clear to us that there is a need to work on autonomous solutions that allow intervention in the first hours of the oil spill to contain the spread and, at the same time, to start the mitigation process.
First-line responses typically include physical (e.g., controlled burning; absorbing) and chemical (e.g., dispersing) oil removal, largely constrained by maritime conditions. Though these treatments are essential to control the diffusion and drift of oil rapidly, they are unsuitable for ecological restoration. Using microorganisms with a natural capacity to degrade petroleum is highly advantageous in an environmentally friendly process and allows the decomposition of complex petroleum hydrocarbons [
3]. The success of bioremediation, to improve petroleum removal and reduce clean-up time and cost, relies on two major approaches: (i) the addition of nutrients to stimulate the growth of the microorganisms that break down oil (biostimulation) and (ii) the addition of pre-grown microbial cultures/consortia to enhance microbial populations (bioaugmentation). This mitigation strategy involves the introduction of microorganisms in the affected environment.
Figure 1.
Oil spill incident detected by the Satellite radar images from Copernicus Sentinel-1 in the Mediterranean Sea [
4].
Figure 1.
Oil spill incident detected by the Satellite radar images from Copernicus Sentinel-1 in the Mediterranean Sea [
4].
The state-of-the-art of oil spill response could be driven by mechanical recovery [
5], chemical recovery [
6], thermal recovery [
7], biological recovery [
3], vacuum and centrifuge [
8], and solidifying [
9,
10]. The mechanical approach consists of deploying booms, with the help of skimmers, to store the oil spill in a controlled area. Different skimmers are available, such as ones based on suction, oleophilic materials, or weirs to remove oil from the water’s surface. In the second phase, after the oil recovery, it is transferred using pumps and hoses [
5]. The thermal response involves a controlled burn of the floating oil [
7]. In that case, an ignition device is released from a vessel (or other access points). The ignition is achieved by releasing a burning, gelled fuel from the aerial way (i.e., using a helicopter). The chemical response uses dispersants—chemicals applied to oil slicks to accelerate oil dispersion into the water column. The idea is not to remove the oil from the water, but to limit the amount of oil forming a slick on the water surface by driving that oil into a dissolved phase [
11]. Biological recovery consists of the use of native organisms with bioremediation capacity. The innovative solution aims to be environmentally friendly by using this organism to naturally degrade oil spills, avoiding the introduction of additional chemical or biological additives. Another approach is the vacuum and centrifuge method, which uses oil and water to obtain near-purity water. There is a debate about whether the resulting water should be returned to the sea or not due to the amount of oil that could be present at the end of the process. The solidifying of the oil could also be an option. With dry ice pellets and hydrophobic polymers, the liquid oil is solidified into a floatable rubber-like material that allows for a more convenient collection and recycling method.
Technologies that allow a safe, immediate, effective, and eco-friendly operation of oil spill removal, in situ, have been under development in recent years. A specific area that grew from that development was the application of autonomous vehicles for ocean exploration and conservation, with two important projects being developed in 2010: Seaswarm [
12] and Protei [
13]. The autonomous vehicles designed within the scope of these projects reduce the operational time and protect the health and safety of the cleaning crew by working as an organized fleet or “swarm” of vehicles that rely on ocean-skimming and oil removal techniques. This can be viewed as a limitation of both approaches, since oil considerably impacts the mechanical parts.
This evolution in oil spill mitigation techniques has primarily occurred in hardware development. The research on advanced software-based navigation algorithms still lacks sufficient attention and support. However, some cases are worth mentioning, like the approach developed by [
14], which uses a multi-resolution navigation algorithm that seamlessly integrates the concepts of local navigation and global navigation based on the sensory information for oil spill cleaning in dynamic and uncertain environments, using autonomous vehicles as a single agent. The developed algorithm provides complete coverage of the search area for the clean-up of the oil spills. It does not have the problem of having “
local minima”, which is commonly encountered in potential field-based methods through the adaptive decision-making and online re-planning of vehicle paths. The problem of optimal path following with a team of ASVs and autonomous underwater vehicles (AUVs) for oil plume tracking is discussed in [
15], with an approach based on a Genetic Algorithm (GA) to optimize the trajectory of both vehicles. In [
16], a cooperative intervention involving UAVs, AUVs, and remotely operated vehicles (ROVs) during the Exercise Cathach is detailed. During the field tests, different research groups demonstrated new robotic systems, sensors, and networking technologies that could be applied during maritime incidents like oil spills.
In this paper, we propose to improve previously described techniques by developing autonomous and coordinated actions that can increase the efficiency of the bioremediation process by deploying microorganisms and nutrients in oil spill-detected areas with a team of heterogeneous robots.
Figure 2 presents the concept approach for oil spill mitigation with a team of heterogeneous autonomous vehicles. Each autonomous vehicle is equipped with a release system of microorganisms and nutrients, and is responsible for spreading on top of the oil spill to contain and start the mitigation process. The proposed approach combines two distinguished but, at the same time, complementary research areas: marine biology for the development of native microbial consortia with bioremediation capacity (not addressed in this paper) [
17,
18], and robotics, in the development of multi-robot frameworks able to ensure the capability of intervention at different levels (air/sea) cost-effectively with a team of heterogeneous autonomous vehicles. This paper is an extended version of the work published in [
1,
19,
20]. The paper is outlined as follows:
Section 2 presents the multi-robot approach for oil spill mitigation with a team of autonomous vehicles. The algorithm developed for each vehicle is presented in detail with the respective mathematical formulation. We present both vehicles, ASV and UAV, detailing each vehicle’s features and the developments to provide the functionalities to perform oil spill combat.
Section 4 details the release systems developed for each vehicle and discusses the requirements imposed on each.
Section 5 presents the oil spill simulation environment developed to validate the multi-robot framework for oil spill mitigation and support the field tests.
Section 6 presents the field tests performed to validate the developed release systems integrated into the autonomous vehicles and the path planning developed for both vehicles (ASV and UAV).
Section 7 presents the remarks and future lines of research to improve oil spill mitigation with a team of heterogeneous autonomous vehicles.
2. Multi-Robot Approach for Oil Spill Mitigation
The in situ bioremediation combat system for oil spill disasters will be performed by a team of unmanned vehicles capable of transport and release of lyophilized bacteria consortia (bioaugmentation) and nutrients (biostimulation) over the oil spill, which will require that each autonomous vehicle have the capability of performing onboard data acquisition, image processing, and mission control. The containers and release systems will be integrated with unmanned aerial vehicles (UAVs) and autonomous surface vehicles (ASVs), enabling air and surface oil spill combat capabilities. A dedicated software integrated into the vehicle control unit will control the release system. Furthermore, a customized multi-robot processing framework, depicted in
Figure 3, will facilitate mission planning and control and the acquisition and processing of environmental data, providing valuable insights on oil spill combat throughout the mission. During the oil spill combat, each autonomous vehicle will have distinct roles: the UAV will be responsible for oil spill detection and aerial release of microbial consortia in areas inaccessible to the ASV. At the same time, the ASV will handle oil spill detection and validation and release the microbial consortia in border areas to contain the oil spill spread. The proposed approach requires the specification, development, and implementation of the two main blocks: hardware, containers, and release systems for bacteria and nutrients, able to be integrated into the autonomous vehicles (detail in
Section 3); and a multi-robot framework for mission control, data acquisition, and processing.
The vehicle’s estimated pose is provided by a localization block composed of an Inertial Measurement Unit (IMU) and a Global Navigation Satellite Systems (GNSS) receiver that are being fused by an Extended Kalman Filter (EKF). The monocular vision block provides the oil spill detection, where the image of the scenery will be acquired and processed (as described in [
21]) to obtain clusters of oil spill spots. This block needs to output multiple points (in the image reference frame) representing the contour of the oil spill areas detected. Using the information provided by these two blocks, it is possible to obtain the georeferenced points of the contour. These points are then provided to the Oil Spill Local State block. The path planning block then uses this previous data, described in
Section 2.1 (for the ASV) and
Section 2.2 (for the UAV), where the control actions are generated to control the vehicle’s movement along with the mission, to combat the spread of the oil slick. A data distribution service (DDS) communication middleware is also implemented, which is used to share the information of each robot to all network agents. The DDS protocol is based on a publish-subscribe paradigm with a data-centric approach, where the Global Data Space identifies the data circulating within the system. A stable and reliable communication medium is achieved, given that DDS can control and optimize the use of resources, such as network bandwidth, CPU memory, and time. Therefore, each vehicle will provide the following information: acquired image (visible and thermographic cameras), position and attitude, estimated oil spill position, path planning, and vehicle status (battery status, tank level of lyophilized powder).
2.1. ASV Oil Spill Combat-Based Artificial Potential Field and Normal Vectors
The ASV mission will require navigating the stroke border areas of the oil spill. Therefore, the path planning method explored focused on having a cost function capable of covering all areas while avoiding oil spills. The algorithm developed for the ASV trajectory computation [
21] is an adaptation from the standard Potential Field algorithm [
22]. The ASV is a nonholonomic vehicle because it is equipped with two independent thrusters, making it impossible to perform sideways movement, which will require distinct attractive and repulsive forces to be applied to the front and rear bodies of the vehicle. These repulsive forces, represented in
Figure 4, are computed by the distance between obstacle points and the vehicle’s body, the sum of those repulsive and attractive forces constitute the resultant force, represented in
Figure 4, that determines the motion of the vehicle.
Considering an oil spill environment, the standard Potential Field algorithm cannot be applied, since there is no real goal defined, and the objective is not simply to avoid getting too close to obstacles but to follow the contour boundaries of the obstacles that describe the oil spill while maintaining a safe distance from it. Therefore, a new algorithm based on a new curvature-based approach was created, denominated by Normal Vectors Control Points + Artificial Potential Field (APF). This algorithm is responsible for computing a newly enlarged contour, resorting to normal vectors at points from the original contour. These new control points from the enlarged contour are put through the standard potential field algorithm as goals. This ensures the vehicle moves through all the points while maintaining a safe distance from the original contour (representing the oil spill).
The algorithm starts by taking three consecutive points from the contour, N − 1(
,
), N(
,
), and N + 1(
,
), and computing the Euclidean distance
between N − 1 and N + 1; Equation (
1).
The variables
and
are obtained by simply taking the differences of the positions in
x and
y, respectively, and dividing them by the Euclidean distance calculated previously, as represented in Equations (
2) and (
3).
Now, with
and
computed, it is relatively easy to obtain a new point
, distanced
from N(
,
), as represented in Equations (
4) and (
5). The parameter
can be configured before the mission, and represents the repulsive energy between the ASV and the oil spill.
By repeating this procedure through every successive combination of three points in the contour (, N, and ), a new set of points is obtained, distanced from the original contour points, that describe a new, enlarged contour, representing then, the intended ASV trajectory.
These equations describe the simple procedure of taking three consecutive points from the original contour,
,
N, and
(gray points in
Figure 5) to compute the line segment that passes through
and
(represented in black). Next, the normal vector to that line segment at
N is obtained, represented in red in the figure. The last step is to compute a new point, represented in green, on that normal, distanced
from the original point
N.
2.2. UAV Oil Spill Combat with Optimal Coverage
The oil spill mitigation maneuver proposed for the UAV is based on an Axis-parallel “zig-zag” algorithm [
20]. This maneuver achieves complete coverage of the spill surface while deploying microorganisms and nutrients (bioremediation) capable of mitigating and containing the spillage. To compute the UAV path planning, we need to know the optimal maneuver height for the UAV that imposes the distance between the spreader nozzle and the surface, as well as the angle of dispersion of the powder spreader nozzle. Assuming that the dispersion of the nozzle forms the cone depicted in
Figure 6, its action area on the water surface can be estimated as the largest square to fit in the cone base, represented in blue in the figure. The length of the defined square,
L, can be obtained if the maneuver height for the UAV,
H, and the angle of dispersion of the powder spreader nozzle,
, are known. If the height of UAV,
H, and the angle of dispersion of the powder spreader nozzle,
, are known, the length of the defined square,
L, can be obtained. For that, the diagonal of the cone base,
D, is computed, with Equations (
6) and (
7), resulting in the computation of the action square length.
With the dimensions of the action area estimated, the UAV’s trajectory can now be computed. The goal is to ensure that the trajectory achieves the complete coverage of the oil spill surface, distributing lyophilized powder over its entire surface. The trajectories, represented by the red lines in
Figure 7, are obtained by the UAV path planning algorithm by dividing the points that define the spill into several horizontal layers, represented in the same figure in green, with the corresponding width from the action area square.
The red markers (*) in
Figure 7 represent the UAV waypoints corresponding to the minimum and maximum contour points along the X-axis in each previously obtained horizontal layer. This ensures minimal overlap and, consequently, minimal waste of resources while achieving complete oil spill coverage.
4. Release Systems
Considering each vehicle’s payload constraints and operation conditions, the release system must be developed and adapted for each platform.
For aerial vehicles, the idea is to develop a suitable system that can be integrated with unmanned and manned vehicles and execute missions autonomously. To maximize the bioremediation substance’s density and increase the efficiency of the aerial missions, the release system uses lyophilized consortia instead of a mixture of bacteria and water.
Based on these requirements, the system must have a versatile control system composed of an embedded microcontroller able to support several communication protocols (CAN, RS232, and RS485), a tank for the lyophilized powder, a valve, and a motor driver to control the quantity of bacteria mixed on the airflow, and an optional GNSS and IMU system, for the autonomous preloaded tasks (
Figure 10).
The first UAV release system developed was based on STORK I features. This prototype consists of a vertical cylinder tank for 1 kg of lyophilized bacteria, a regulation valve actuated by a servo motor, a turbine that generates airflow in the tank, and a nozzle to spread the bacteria. The functionality of the system relies on the vacuum forces inside the reservoir. By actuating the turbine and opening the valve, the airflows create suction on the tank, blowing the powder through the nozzle and releasing the bacteria into the oil spill, as depicted in
Figure 11.
A second version was developed with a maximum of 7 kg of lyophilized bacteria to increase the release system’s capacity. This system can be integrated into pairs to the GRIFO-X, presenting an overall of 14 kg of consortia (
Figure 11).
The developed ASV release system (
Figure 10) comprises a water pump that is responsible for suctioning water and creating a powerful flow and a micro-pump that injects a bacteria concentrate into the same flow. Attached to the sprinkler, a motor is responsible for throwing the mixed water flow to the oil spill in the correct orientation.
Figure 12 shows the water pump, micro-pump, and other components mounted in a water-proof box.
The release system assembled in ASV ROAZ is detailed in
Figure 12, with the white hose that connects the water pump to the sprinkler. During the field tests, even with the wind, the system released a mixed solution with a median distance of 12 m from ROAZ, the expected distance defined in the initial requirements.
5. Oil Spill Simulation Environment
A critical step in developing a strategy for oil spill mitigation is evaluating the cooperative behavior between both heterogeneous vehicles in the presence of an oil spill. Therefore, a simulation environment was developed with the following requirements: (1) provide a realistic 3D environment able to simulate the water buoyancy effect, the effect of the sun in the water, and the oil spill; (2) direct integration with the perception layer onboard on each vehicle, allowing in the future to switch between simulated cameras with real cameras (visual and thermographic cameras); and (3) straightforward integration with the vehicle’s actions (waypoint) generated by the multi-robot path planning framework layer (see
Figure 2).
Based on these requirements, the chosen simulator was the 3D simulator Gazebo [
30]. Gazebo is an open-source project developed by the robotic community for various applications. It is fully integrated with the framework ROS [
31]. It provides an extensive library of common sensors used in robotics, such as cameras, GPS, and IMU, and provides the water waves and buoyancy effect.
The simulated oil spill environment is composed of the following items: the autonomous vehicles, the ASV ROAZ II and the UAV STORK I, both of them imported from the mechanical design; a realistic ocean environment representation from [
32]; and an oil spill darker color mesh created on top of the previously existent ocean mesh. The overall results are detailed in
Figure 13. The developed environment was also used during the field tests to provide the oil spill image to both autonomous vehicles, as depicted in Figure 17.
6. Field Tests
The field tests were performed in different scenarios and places. To enumerate them: harbor of Leixões, Medas, and Lisbon Naval Base in Alfeite—Portugal, and also in the harbor of Coruña Spain (see
Figure 14 and
Figure 15). To provide a quasi-real-world disaster, the field tests were carried out with a combination of an oil spill in a simulation environment in a real-world localization (see Figure 17).
The existence of a link between the physical world and the simulation scenario allowed the UAV and ASV robots to localize the oil spill autonomously, path-planning the vehicle trajectory, and perform the mitigation maneuver.
The ASV mission consists of a maneuver to surround the oil spill and use the sprinkler to spread the mixed water into the oil spill. The UAV mission consists of an efficient Axis-parallel “zig-zag” maneuver, where the lyophilized bacteria consortia are spread across the oil spill, as detailed in the algorithm in
Section 2.2.
The preliminary tests occurred in the Leixões Harbor in Porto, Portugal, with a single simulated oil spill placed close to shore. The UAV computed trajectory was formed by 18 waypoints covering the oil spill, while nine waypoints formed the ASV trajectory. This low number of waypoints represented the ASV maneuver due to the minimal distance of 30 m between each waypoint. The reduced turning angle of this specific surface vehicle imposed this condition. A slight drift occurred from the intended trajectory due to harsh weather conditions, strong wind, and water currents. The vehicle trajectory concerning the simulated oil spill is present in
Figure 16.
A second field test was performed in the Medas Douro River in Porto, Portugal. Two simulated oil spills were placed in the river’s middle to increase difficulty. The ASV mission planner generated 36 waypoints to cover the two zones, and the UAV mission had 20 waypoints corresponding only to one zone.
Figure 15 in the left shows both vehicles performing the cooperative autonomous mitigation task in the real scenario.
Figure 15 and Figure 18 show the vehicles performing the cooperative autonomous mitigation task in the Puerto de A Coruña.
The final tests occurred in Puerto de A Coruña, Spain, where two simulated oil spills were placed close to shore, as shown by
Figure 17.
Figure 18 shows the results obtained by this mitigation mission. The waypoints were generated and sent to the vehicles (39 waypoints for the ASV and 26 waypoints for the UAV), as represented by the orange markers, the white lines are the detected oil spills, and the blue lines represent the real trajectory of the vehicles. As depicted in
Figure 18, both autonomous vehicles carried out the mission successfully. In the particular case of the ASV, the oil spill contour mission takes approximately 15 min, covering a total distance of 1270 m, and is carried out at an average speed of 1.5 m/s, and a safety distance of 2.5 m related to the oil spill. In the case of the UAV, the intervention was inside the oil spill to ensure total coverage. The mission from takeoff to landing lasted 18 min, with a total distance covered of 2600 m, a safety distance of 3 m from the water, and an average operating speed of 3 m/s. Both vehicles started the mission from the harbor of Coruña. However, the algorithm executed in each autonomous vehicle is prepared to generate the maneuver from the moment they have information about the simulated oil spill and the estimated position.
During the field tests, both vehicles were able to perform the mission, even when we moved the oil spill position, with the perception layer being able to detect a new oil spill and share the information with the vehicle’s path planning layer. A critical improvement to the field tests is having a more coordinated task between the ASV and UAV to ensure that both vehicles are not covering the same oil spill area. We experienced one situation where the UAV spread the powder inside the range of the ASV water pump sprinkler, which means that the ASV spread water can damage the UAV.
A real-time application for 3D visualization and monitoring the ground station’s mission was developed in RVIZ–ROS, as depicted in
Figure 17. The oil spill included in the scenario is represented by the black areas, where the blue squares draw the detected contours. The red lines and the squares show the path planning and waypoints for the ASV ROAZ (red vehicle), and the green lines and squares are the path planning and waypoints of UAV STORK (green vehicle near ROAZ).
The field test videos are available in the supplementary movie [
33].