A Digital Twin Infrastructure for NGC of ROV during Inspection

Abstract

Remotely operated vehicles (ROVs) provide practical solutions for a wide range of activities in a particularly challenging domain, despite their dependence on support ships and operators. Recent advancements in AI, machine learning, predictive analytics, control theories, and sensor technologies offer opportunities to make ROVs (semi) autonomous in their operations and to remotely test and monitor their dynamics. This study moves towards that goal by formulating a complete navigation, guidance, and control (NGC) system for a six DoF BlueROV2, offering a solution to the current challenges in the field of marine robotics, particularly in the areas of power supply, communication, stability, operational autonomy, localization, and trajectory planning. The vehicle can operate (semi) autonomously, relying on a sensor acoustic USBL localization system, tethered communication with the surface vessel for power, and a line of sight (LOS) guidance system. This strategy transforms the path control problem into a heading control problem, aligning the vehicle’s movement with a dynamically calculated reference point along the desired path. The control system uses PID controllers implemented in the navigator flight controller board. Additionally, an infrastructure has been developed that synchronizes and communicates between the real ROV and its digital twin within the Unity environment. The digital twin acts as a visual representation of the ROV’s movements and considers hydrodynamic behaviors. This approach combines the physical properties of the ROV with the advanced simulation and analysis capabilities of its digital counterpart. All findings were validated at the Point Rouge port located in Marseille and at the port of Ancona. The NGC implemented has proven positive vehicle stability and trajectory tracking in time despite external interferences. Additionally, the digital part has proven to be a reliable infrastructure for a future bidirectional communication system.

1. Introduction

Oceanic exploration has emerged as one of the most captivating frontiers in the quest for scientific knowledge and technological advancement, as only 5–10% of the seabed has been explored, containing secrets crucial to understanding the planet’s history, climate, and biodiversity [1].

However, challenges like high pressure, low temperatures, and darkness posed by the complex environment create obstacles to scientists, particularly in acquiring high-quality underwater images for photogrammetry [2].

Due to a limited field of view, techniques based on divers present significant dangers and inaccurate information. Specialized certification, stringent safety protocols, and robust physical endurance are also required for these explorations [3]. Meanwhile, traditional methods for underwater mapping include the use of satellite-based sensors or sensors mounted on aircraft for remote sensing, which can cover larger areas and provide accurate and repeatable data. However, these methods suffer from optical problems in terms of spectral bands, lower spatial resolutions, and weather dependency. To overcome these limitations, a hybrid approach combines remote sensing techniques with detailed data acquired from divers, validating the results [4,5]. This approach applies primarily to shallow seabed areas.

As a result, underwater robots equipped with cameras and other sensors are becoming increasingly popular [6], filling the gap between remote and in situ methods. They offer versatility, precision, and safety features to explore environments that would otherwise be inaccessible.

Among all the unmanned underwater vehicles (UUVs), ROVs have the most potential for acquiring underwater images because (a) of their small size, flexibility, and affordable cost; (b) human operators remotely control them, but they can also gain the benefits of an autonomous underwater vehicle (AUV) by implementing special guidance and control strategies; (c) they can adjust their speed and distance from a target as required, thereby protecting the environment; (d) they can maneuver with six DoFs, unlike most AUVs, which typically have three DoFs.

However, the use of an ROV poses some challenges, such as the vehicle’s autonomy in collecting images. ROVs could certainly go to great depths during their surveys, which would require them to be operational for long periods. Additionally, maintaining proper buoyancy and balance while scanning the seabed is crucial, yet not always assured, particularly in the presence of strong sea currents.

The primary objective of this article is to propose a potential approach to address the challenges of ocean exploration using an ROV, aiming to (1) solve the NGC problem, including problems in communication, localization, target identification, and trajectory planning; (2) provide a virtual environment for both training and testing purposes that can be integrated with the real system to enable a bidirectional flow of communication.

Regarding (1), conventional communication methods based on electromagnetic transmission prove inadequate for underwater applications. Therefore, this work presents an acoustic communication system using a USBL configuration. The guidance system is designed to generate commands for the ROV’s thrusters, allowing autonomous control of the vehicle’s direction and orientation. The use of an LOS guidance strategy ensures that a straight trajectory is maintained. The response to (2) is provided by a high-fidelity digital twin of both the vehicle and the environment in which the tests were conducted. The simulator collects spatial information from the ROV, mimicking its real behavior. Thus, the digital twin is designed for the BlueROV2, integrating communication via PyMAVLink, which is the open protocol used by the ROV. This integration allows for the high-level development of both the control software and guidance systems, capitalizing on the requirements of the BlueROV2 platform. In this manner, the system addresses communication challenges in a hostile and highly unstructured environment, such as the sea, intending to increase reliability in communications and effectiveness in autonomous and semi-autonomous operations of the BlueROV2. This serves as a first step towards the development of a comprehensive two-way communication digital model.

The research conducted was carried out at LabMACS, which is affiliated with the Università Politecnica delle Marche, in collaboration with the CNR of Ancona and the CNRS of Marseille.

This paper is structured as follows: Section 2 presents related research in the area of navigation systems and a brief overview of remotely operated vehicles, their features, and applications; it also examines advancements in research in digital twins for the marine field. Section 3 describes the materials and software methods used, as well as the communication between them, and the mathematical model behind the ROV’s locomotion, especially the reference systems, the rotational matrix, the kinematic and dynamic models, the parameters of the vehicle, and the space-state model. Furthermore, it analyzes the digital environment, the simulation, and the implemented control system, including the LOS guidance and the high-level logic. All the results are shown in Section 4. Finally, Section 5 summarizes the key findings of the study and their implications, interpreting the results and introducing some considerations about future developments.

2. Related Work

2.1. Overview of Navigation, Guidance, and Control of ROVs in Underwater Applications

Despite sharing several core components, such as thrusters, a tether cable, pilot controls, lights, and several sensors for navigation tasks [7], ROVs are designed in different shapes and sizes based on specific requirements. These variations have made ROVs indispensable tools in numerous domains. One notable application is the fully automated inspection of fish farm nets to ensure their integrity [8]. This particular domain requires computer vision and image processing algorithms (as in [9,10]), a tetherless configuration, sensors for measuring environmental parameters, and specific guidance laws (e.g., LOS [11]), to accurately detect damages in nets. Another application is in ship-hull inspections, exemplified by the hybrid propulsion ROV ‘Iznos’ [12]. Harbor structures also benefit from ROV inspections, especially corrosion checks caused by salty seawater. In these scenarios, a manually operated ROV equipped with visual systems captures high-quality images, supported by onboard navigation sensors [13].

ROVs are versatile in a range of conservation tasks, from preserving, cleaning, restoring, and maintaining archaeological sites [14,15,16] to monitoring ecosystems. For instance, as seen in [17], ROVs are essential in analyzing Posidonia oceanica evolution to track water quality and environmental changes. ROVs also contribute to animal behavior monitoring, as exemplified by their use in shark research [18], although AUVs are more commonly used due to their greater autonomy.

ROVs have proven useful in defense and recovery operations, specifically in underwater surveillance, underwater mine hunting, and anti-submarine warfare [19], or in recovery missions for abandoned ghost fishing gear [20].

These examples show that the use of ROVs offers benefits from cost, safety, and reliability perspectives. However, there is a need to develop autonomous guidance and control systems for ROVs since AUVs are preferred over ROVs in exploration and surveillance, despite the latter’s greater maneuverability. Furthermore, the current state-of-the-art in this field highlights the lack of a clear and well-defined methodology, which requires the study of solutions to address the various challenges associated with underwater exploration and intervention. Current research could contribute to aspects such as the autonomy of the ROV and its energy efficiency, buoyancy, path following, data transmission, and communication limitations [21,22,23,24].

Achieving autonomy in ROVs is a challenging task, primarily due to difficulties in underwater communication and data transmission, which are mostly related to the physical medium. Using the same transmission medium for both navigation and communication can present a bottleneck in guidance and control. Acoustic communication is commonly used in water, despite acoustic shadows and multipath Doppler effects. Moreover, sensors need to be correctly calibrated, as sound propagation in water depends on temperature and salinity [21]. Meanwhile, optical communication provides a high data transfer rate but cannot perform long-distance propagation [25].

Specifically, research is focused primarily on controlling stable motion, which has led to the importance of analyzing the hydrodynamic characteristics of ROVs since their complex structures with sensors, arms, tether cable, and other devices can greatly influence their stability and maneuverability [26,27,28]. The vehicle represents a nonlinear system, and its movements are coupled with each other, increasing the difficulties in guidance and control. Thus, researchers have tried various control methods, from PID control to fuzzy control, sliding mode control (SMC), and neural network control [29]. However, PID control is commonly suitable for linear systems [30,31,32,33], while fuzzy control is recommended for nonlinear systems with high coupling degrees, but it has parameter uncertainty and inaccuracy [34]. SMC is also suitable for complex nonlinear systems, providing robustness to uncertainty, but it reveals a chattering problem [29,35]. Other control algorithms are based on neural networks [36] or on predictive schemes to solve optimization functions [37].

2.2. Overview of Digital Twin Techniques in Marine Robotics

Digital twins (DTs) represent an essential technology in marine robotics, providing advanced tools for simulation, monitoring, and predictive analysis, which increase the safety of underwater operations. A DT is a virtual replica of a physical entity, process, or system that allows for the assessment of risks and optimization of strategies before deployment in the real world [38,39]. By simulating different scenarios, operators can plan and test responses to potential emergencies or system failures without the risks associated with physical trials. It consists of five elements: a physical part, a virtual part, connections, data, and services.

DTs have two primary challenges: creating a high-fidelity virtual representation of the physical object, and rapidly updating the collected data for the digital twin’s diagnosis and decision-making. Regarding the first, researchers have developed photogrammetry-based reconstruction workflows. However, the use of DTs as a continuous monitoring framework is still relatively unexplored and represents a significant opportunity for advancement in this field [40]. In [41], technologies for DTs are shown, which include data-driven, statistical, or machine-learning strategies.

DTs have found applications in a wide range of sectors, including manufacturing, supply chain management, preventive maintenance, agriculture, healthcare, and weather modeling [42,43]. However, their development in underwater applications has not reached the same level of maturity as in these other fields. This is partly due to the difficulty of modeling the complex marine environment, which includes simulating hydrodynamic effects and the interactions between water particles and the vehicle [44]. Ref. [45] uses the digital twin approach to address challenges related to position accuracy and trajectory tracking in the presence of environmental disturbances by providing a reliable simulation platform for testing control algorithms. However, how to communicate with the real-world application is not explicitly addressed.

Indeed, a critical issue is the need for higher communication bandwidth to enable the application of digital twins in marine environments due to the requirements of condition sensors to support decision-making [46]. In addition, the upstream process of creating the digital replica, which consists of acquiring underwater images, presents a big challenge [46,47]. Underwater vehicles equipped with high-quality cameras and sensors can collect data about underwater assets. After the data collection, photogrammetry software can process them to generate 3D point cloud models to create digital twins. However, as mentioned earlier, the data acquisition process is not simple, mainly due to the lack of visibility in deep water [48]. Therefore, DTs in marine robotics remain a relatively unknown field. Nonetheless, with advancements in technology and increased research, the potential for digital twin technology to transform underwater applications cannot be ignored, particularly in the context of real-time adaptive control based on the feedback of a digital twin, which is useful for mitigating the impact of environmental disturbances [46]. Recent advancements have incorporated visual feedback methods into digital twin technologies. However, these techniques often encounter limitations due to uncertainties associated with visual feedback, such as flow conditions and the high barriers to training. In [49], the authors propose an innovative hierarchical intuitive control method that utilizes virtual reality (VR) and haptic simulators to address these challenges. This approach leverages a distributed sensor system for the flexible collection of subsea environmental data, which are then integrated into a BlueROV2 Digital Twin (DT) module to enhance augmented reality simulations.

3. Materials and Methods

The aim of the work is to enable the ROV to navigate autonomously to specific points of interest. This involved an initial selection of materials and methods based on the given specifications and constraints. Specifically, the proposed architecture and materials for constructing and validating the NGC system and its digital model are presented. This includes the communication between various components, the mathematical model used, the guidance strategy, and the control logic.

3.1. Underwater Setup

The central figure in this work is the BlueROV2, an underwater vehicle belonging to the ROV category and manufactured by BlueRobotics, as shown in Figure 1. It is characterized by high maneuverability and stability, which is facilitated by the Heavy Configuration Retrofit Kit with eight T200 brushless thrusters for complete control across all six DoFs and active pitch and roll feedback stabilization [50]. The vehicle can reach a maximum speed of 1.5 m/s.

The ROV utilizes a navigator flight controller and BlueOS for surface communication and control. The navigator is purpose-built for ROVs and is equipped with an advanced processor and onboard sensors, including an inertial measurement unit (IMU) and a magnetometer, and it runs on the real-time operating system NuttX. The Raspberry Pi 4 has a 64-bit quad-core processor, dual-display capabilities with resolutions reaching 4K, up to 4 GB of RAM, a dual-band 2.4/5.0 GHz wireless LAN, Bluetooth 5.0, Gigabit Ethernet, and USB 3.0 options. The processor is paired with the navigator and is responsible for all computing tasks within the ROV. The open-source BlueOS software runs the ArduSub vehicle control software and monitors the system. The Raspberry Pi 4 is connected to the Fathom ROV Tether and uses Ethernet technology to transmit telemetric data to the surface [51]. The low-level motor control is managed by the Raspberry Pi 4, as is the estimation of the ROV’s attitude, which is achieved through the use of an Extended Kalman Filter. The position data are obtained directly from the USBL sensor, obviating the necessity for state estimation. The mathematical model was employed to calibrate the forward PID controllers and to conduct a final comparison between real and simulated behaviors in order to evaluate the efficacy of the implemented control algorithm.

The tether is a 300-m flexible polyurethane cable with neutral buoyancy and robust breaking strength [52].

The BlueROV2 differs from traditional ROV models in its modular design, which allows it to be adapted to different mission requirements through a range of attachable sensors and tools, such as scanning sonar or navigation systems. The ROV used is equipped with an HD and wide-angle camera (1080 p, 30 fps, 200 ms latency), specialized for low-light conditions and underwater usage, and an advanced lighting system of up to 6000 lumens to improve visibility in the depths of underwater environments.

Furthermore, there is the SeaTrac Lightweight system, which manages the acoustic positioning of the vehicle to reduce the bottleneck of the tether, as described in the literature. This setup enables the tether to manage power, data, and commands, while the SeaTrac provides navigation and positioning, sending position updates every 4 s. ROV’s orientation and depth are transmitted via the tether every 100 ms. Additionally, video from the cameras and health data, such as battery status, presence of water, and light status, are transmitted through the cable at a frequency of once per second.

This system comprises two beacons: the X150 USBL beacon and the X010 transponder beacon. The principle behind this solution is that a single USBL beacon can be used to track the position of 1–14 underwater devices, each equipped with a transponder beacon, and to establish bidirectional communication with them in real-time at depths of up to 300 m. Each X010 transponder is equipped with sensors that measure environmental pressure and temperature. This enables the calculation and continuous monitoring of the beacon’s depth, contributing to the automatic refinement of the local VoS value and minimizing errors in ranging calculations [53]. The X150 micro-USBL beacon provides information on the remote beacon’s relative position during data exchange, using an ASCII-based command protocol. The beacon is equipped with a 9 DoF AHRS and a Doppler sensor that uses data from the onboard MEMS gyroscope, accelerometer, and magnetometer to determine pitch, roll, and yaw relative to magnetic north and the direction of gravity [54].

The configuration is shown in Figure 2.

3.2. Surface Setup

The USBL data provide information on the relative position of the ROV. To determine its geographic coordinates, these data need to be integrated with information from a surface GPS located at the X150’s position. The GPS receiver used in this project is a water-resistant BU-353S4 model. It has a SiRF Star IV chip with a −163 dBm tracking sensibility, a fast TTFF to a low-level signal and supports NMEA0183 protocol.

The ROV is powered by the Outland Technology Power Supply system via a tether instead of a battery. This consists of a Topside Power Supply Unit, an ROV Power Supply Enclosure, and a High-Power Tether Cable with Connectors. The Topside Power Supply Unit is a compact case containing a built-in FXTI board and a safety GFI that shuts down the system if an unsafe event or current leakage is detected. The ROV Power Supply Enclosure is mounted on the lower part of the ROV and converts 400 V power into 15 V at the end of the tether. The High-Power Tether Cable with Connectors is a tether designed with five twisted pairs. Two of these pairs are used for isolated power transmission, another two pairs are used for an earth-to-ground connection, and the last pair is used for Fathom-X communication. A switcher was used to establish communication between the tether and the laptop without using the FXTI board of the OTPS [55]. Essentially, the ROV tether converts Ethernet into a single twisted pair of wires through the Fathom-X. This component introduces a maximum delay of 60 ms. Since underwater vehicles operate with slow dynamics, even in the worst-case scenario, this delay is manageable and does not require compensation.

A Dell Latitude 7480 laptop with 15.5 GB of memory and 512.1 GB of hard drive capacity was used, with an Intel^® Core™ i5-7200U CPU (Intel Corporation, Santa Clara, CA, USA) @ 2.50 GHz × 4 processor. It runs on the 64-bit Ubuntu 20.04.6 LTS Operating System and a Mesa Intel® HD Graphics 620 (KBL GT2).

The software used included the following: ArduSub, compatible with Pixhawk for comprehensive vehicle management and control; MAVLink, a lightweight messaging protocol for sending and receiving messages; QGroundControl 4.2.6 version, an open-source, cross-platform ground control station software, created for planning, monitoring, and controlling missions of any vehicle that supports MAVLink communication; ROS Noetic version, an open-source collection of libraries and tools for robotic applications’ developers and users; Unity3D 2021.3.18f1 version, a real-time 3D cross-platform game engine and development framework used to create interactive 3D applications.

3.3. Communication and Connections Setup

The completed setup, shown in Figure 2, comprises the laptop placed on top of the Topside Power Supply Unit, running the Python code for the control and localization of the ROV. The code communicates with QGC using UDP on port 14,440 and publishes ROV NED coordinates to the ROS topic, which the C# Unity code subscribes to using UDP on port 14,550.

Thus, ROS acts as a bridge between Python and C# to update the ROV digital twin position in the environment. The system obtains GPS information using the UART protocol and communicates bidirectionally with the ROV via Ethernet and MAVLink. Meanwhile, the Raspberry Pi 4 inside the ROV runs ArduSub software and processes data to send to the navigator controller.

Additionally, the X010 beacon of the Seatrac Lightweight, mounted on the BlueROV2, communicates with the X150 beacon, which is connected to one of the laptop’s USB ports.

3.4. Mathematical Model for Design and Control

The mathematical equations of the system are based on the Fossen model, according to the methodology described in [56]. To simplify the analysis of the mathematical model, some assumptions have been introduced.

Assumption 1. 

The fixed ground reference system is considered inertial.

Assumption 2. 

Water is an ideal fluid, characterized by being incompressible, non-viscous, and non-rotational.

Assumption 3. 

The ROV is considered a rigid body that is completely submerged in water.

Remark 1. 

When the ROV is fully submerged, the wave-induced disturbance is neglected, assuming that the ROV operates below the wave-affected zone.

Assumption 4. 

The ROV is considered to have symmetry in both the xz and xy planes, with the center of gravity located within these planes of symmetry.

Assumption 5. 

The center of gravity and the center of buoyancy are placed along the same vertical axis in the ROV-fixed reference system. Specifically, the center of buoyancy is placed at the origin of the reference system.

Assumption 6. 

The ROV has 6 DoFs.

Remark 2. 

In most cases, the vehicle is positively buoyant, so it can rise to the surface if propulsion is lost.

Assumption 7. 

The ROV’s speed is very low (less than 2 m/s), so lift forces can be excluded.

Assumption 8. 

The dynamics of the tether connected to the ROV are not modeled.

Remark 3. 

These model assumptions are very common in the submarine environment [56]. Actually, the ocean current is not constant but varies very slowly in space and time. In the control field, the forces generated by the currents and the tether are considered disturbances that vary slowly over time, which can be compensated through a robust controller.

Two fixed frames are considered: the Earth-fixed frame, defined with the $x,y,z$ axes aligned according to the NED coordinate system, with the origin fixed at the head of the USBL system, the X axis oriented towards the true north, the Y axis pointed towards east, and the Z axis extended downward, perpendicular to the Earth’s surface; the body-fixed frame, defined with the $X_b,Y_b,Z_B$ axes, with its origin located at the vehicle’s center of gravity and oriented as shown in Figure 3. Both frames are right-handed.

A vectorial representation is used for positions, velocities, and forces, defined as follows:

$$\mathit{\eta }={(x,y,z,\varphi ,\theta ,\psi )}^T$$

(1)

$$\nu ={(u,v,w,p,q,r)}^T$$

(2)

$$\mathit{\tau }={(X,Y,Z,K,M,N)}^T$$

(3)

where $\mathit{\eta }$ is the vector of the vehicle’s positions with respect to the Earth-fixed frame, $\nu$ is the vector of the vehicle’s velocities with respect to the body-fixed frame, and $\mathit{\tau }$ is the vector of the vehicle’s forces and moments with respect to the body-fixed frame. $\varphi$ and $\psi$ are defined in the interval $[-\pi ,\pi )$, while $\theta$ is in the interval $(-\frac{\pi }{2},\frac{\pi }{2})$ due to the singularity of the rotational matrix.

It is possible to decompose the previous vectors into two vectors, representing the linear variables and the angular ones:

$$\mathit{\eta }=\left[\begin{array}{c}P\\ \Theta \end{array}\right],\nu =\left[\begin{array}{c}v\\ \omega \end{array}\right],\mathit{\tau }=\left[\begin{array}{c}f\\ m\end{array}\right]$$

(4)

3.4.1. Kinematic Equations

As two coordinate systems are used in the ROV model, the variables must be transformed from the body-fixed frame b to the Earth-fixed frame n, through the rotational matrix $R_b^n(\Theta )$ and the transformation matrix of angular velocities $T_{\Theta }(\Theta )$. Hence, the kinematic equation can be written in vector form as follows:

$$\dot{\mathit{\eta }}=J\left(\mathit{\eta }\right)\nu \iff \left[\begin{array}{c}\dot{p}\\ \dot{\Theta }\end{array}\right]=\left[\begin{array}{cc}{R}_b^n(\Theta )& 0_{3x3}\\ 0_{3x3}& T_{\Theta }(\Theta )\end{array}\right]\left[\begin{array}{c}{v}_b\\ {\omega }_b\end{array}\right]$$

(5)

3.4.2. Dynamic Equations

The dynamic model of an underwater vehicle can be described through nonlinear Newton–Euler equations in the body-fixed frame, as follows:

$$\mathbf{M}\dot{\nu }+\mathbf{C}\left(\nu \right)\nu +\mathbf{D}\left(\nu \right)\nu +\mathbf{g}\left(\mathit{\eta }\right)=\mathit{\tau }+{\mathit{\tau }}_d$$

(6)

where

• $\mathbf{M}\in {\mathbb{R}}^{6x6}$ is the matrix of inertia and added mass.
• $\mathbf{C}\in {\mathbb{R}}^{6x6}$ is the centripetal and Coriolis matrix.
• $\mathbf{D}\in {\mathbb{R}}^{6x6}$ is the hydrodynamic damping matrix.
• $\mathbf{g}\in {\mathbb{R}}^{6x1}$ is the vector of gravitational and buoyancy forces.
• $\mathit{\tau }\in {\mathbb{R}}^{6x1}$ is the vector of forces and moments applied to the vehicle.
• ${\mathit{\tau }}_d\in {\mathbb{R}}^{6x1}$ represents environmental disturbances.

The $\mathbf{M}$ matrix represents the force and moment due to the acceleration of the ROV (rigid body mass) and water (added mass) around the vehicle. Considering Assumption 4 and Assumption 5 (i.e., $x_b=0,y_b=0,z_b\ne 0$, $x_g=0,y_g=0,z_g=0$, $I_{xy}=I_{xz}=I_{yz}=0$) and that the movements between the degrees of freedom of the ROV in hydrodynamics are decoupled, the matrix can be calculated as follows:

$$\mathbf{M}={\mathbf{M}}_{RB}+{\mathbf{M}}_A=\left[\begin{array}{cccccc}m-X_{\dot{u}}& 0& 0& 0& 0& 0\\ 0& m-Y_{\dot{v}}& 0& 0& 0& 0\\ 0& 0& m-Z_{\dot{w}}& 0& 0& 0\\ 0& 0& 0& I_x-K_{\dot{p}}& 0& 0\\ 0& 0& 0& 0& I_y-M_{\dot{q}}& 0\\ 0& 0& 0& 0& 0& I_z-N_{\dot{r}}\end{array}\right]$$

(7)

where m is the mass of the vehicle, $I_i$ is the inertial moment of the i axis, $I_{ij}$ is the inertial product on the $ij$ plane, ${\mathbf{r}}_g:={[x_g,y_g,z_g]}^T$ is the center of gravity, ${\mathbf{r}}_b:={[x_b,y_b,z_b]}^T$ is the center of buoyancy.

The Coriolis force matrix can also be decomposed into a term concerning the rigid body and a term concerning the added mass, similar to before:

$$\begin{array}{c}C\left(\nu \right)=C_{RB}\left(\nu \right)+C_A\left(\nu \right)\end{array}$$

(8a)

$$\begin{array}{c}=\left[\begin{array}{cccccc}0& 0& 0& 0& (m+Z_{\dot{w}})w& -(m-Y_{\dot{v}})v\\ 0& 0& 0& (-m-Z_{\dot{w}})w& 0& (m-X_{\dot{u}})u\\ 0& 0& 0& (m-Y_{\dot{v}})v& (-m+X_{\dot{u}})u& 0\\ 0& (m-Z_{\dot{w}})w& (-m+Y_{\dot{v}})v& 0& (I_z-N_{\dot{r}})r& (-I_y+M_{\dot{q}}q)\\ (-m+Z_{\dot{w}})w& 0& (m-X_{\dot{u}})u& -(I_z+N_{\dot{r}})r& 0& (I_x-K_{\dot{p}})p\\ (m-Y_{\dot{v}})v& (-m+X_{\dot{u}})u& 0& (I_y-M_{\dot{q}})q& (-I_x+K_{\dot{p}})p& 0\end{array}\right]\end{array}$$

(8b)

The hydrodynamic damping matrix can be decomposed into a term representing the skin friction (linear) and another one representing the damping due to vortex shedding (nonlinear), as

$$\begin{array}{c}D=D_l+D_{nl}\end{array}$$

(9a)

$$\begin{array}{c}=-\mathrm{diag}(X_u+X_{\left|u\right|u}|u|,Y_v+Y_{\left|v\right|v}|v|,Z_w+Z_{\left|w\right|w}|w|,K_p+K_{\left|p\right|p}|p|,M_q+M_{\left|q\right|q}|q|,N_r+N_{\left|r\right|r}|r\left|\right)\end{array}$$

(9b)

which represents a diagonal matrix with the given elements along the diagonal.

The restoring force $g\left(\mathit{\eta }\right)$ is the net buoyancy, where $W=mg$ is the weight of the ROV, and $B=\rho gV$ is the buoyancy. The vector is defined as follows:

$$g\left(\mathit{\eta }\right)=\left[\begin{array}{c}(W-B)sin\theta \\ -(W-B)cos\theta sin\varphi \\ -(W-B)cos\theta cos\varphi \\ -z_{b}Wcos\theta sin\varphi \\ -z_{b}Wsin\theta \\ 0\end{array}\right]$$

(10)

The vehicle is actuated by eight propellers. So the forces and moments can be determined by

$$\mathit{\tau }=\left[\begin{array}{c}\mathbf{f}\\ \mathbf{r}\times \mathbf{f}\end{array}\right]=\left[\begin{array}{c}{F}_x\\ F_y\\ F_z\\ F_z{l}_y-F_y{l}_z\\ F_x{l}_z-F_z{l}_x\\ F_y{l}_x-F_x{l}_y\end{array}\right]=T\left(\alpha \right)F=T\left(\alpha \right)Ku$$

(11)

where $\mathbf{f}={[F_x,F_y,F_z]}^T$ is the force vector, $\mathbf{r}={[l_x,l_y,l_z]}^T$ is the moment arms vector, $T={[t_1,t_2,t_3,t_4,t_5,t_6,t_7,t_8]}^T\in {\mathbb{R}}^{6x8}$ is the thrust configuration matrix, $\alpha \in {\mathbb{R}}^8$ is the vector of azimuth angle, $\mathbf{u}={[u_1,u_2,u_3,u_4,u_5,u_6,u_7,u_8]}^T$, whose elements $u_i$ are the control inputs of each thruster, and $K=\mathrm{diag}{[K_1,K_2,K_3,K_4,K_5,K_6,K_7,K_8]}^T$, whose elements $K_i$ are the thrust coefficients, which are scalar factors.

Finally, the state of the system $\mathbf{x}\in {\mathbb{R}}^{12}$ is selected as the position and velocity of the vehicle, defined as follows:

$$\mathbf{x}=\left[\begin{array}{c}\mathit{\eta }\\ \nu \end{array}\right]$$

(12)

$$\dot{\mathbf{x}}=\left[\begin{array}{c}\dot{\mathit{\eta }}\\ \dot{\nu }\end{array}\right]=\mathbf{f}(\mathbf{x},\mathbf{u},{\mathit{\tau }}_d,t)=\left[\begin{array}{c}\mathbf{J}\left(\mathit{\eta }\right)\nu \\ {\mathbf{M}}^{-1}[{\mathbf{K}}_p\left(\mathbf{A}\mathbf{u}\right)+{\mathit{\tau }}_d-\mathbf{C}\left(\nu \right)\nu -\mathbf{D}\left(\nu \right)\nu -\mathbf{g}\left(\mathit{\eta }\right)]\end{array}\right]$$

(13)

However, the effects of marine currents must be considered, modifying the dynamic equation as follows:

$${\mathbf{M}}_{\mathbf{RB}}\dot{\nu }+{\mathbf{C}}_{\mathbf{RB}}\left(\nu \right)\nu +{\mathbf{M}}_{\mathbf{A}}\dot{{\nu }_r}+{\mathbf{C}}_{\mathbf{A}}\left({\nu }_r\right){\nu }_r+\mathbf{D}\left({\nu }_r\right){\nu }_r+\mathbf{g}\left(\mathit{\eta }\right)=\mathit{\tau }+{\mathit{\tau }}_d$$

(14)

where ${\nu }_r=\nu -{\nu }_c$ is the relative velocity vector, and ${\nu }_c={[u_c,v_c,w_c,0,0,0]}^T$ is the irrotational marine current velocity vector. Since the Coriolis and centripetal matrices are independent of linear velocity, the equation can be rewritten as

$$\mathbf{M}\dot{{\nu }_r}+\mathbf{C}\left({\nu }_r\right){\nu }_r+\mathbf{D}\left({\nu }_r\right){\nu }_r+\mathbf{g}\left(\mathit{\eta }\right)=\mathit{\tau }+{\mathit{\tau }}_d$$

(15)

3.4.3. Parameters of a BlueROV2

Specifically, in

Table 1. Moment arms of the 8 thrusters of the BlueROV2.

Thrust	${\mathit{l}}_{\mathit{xi}}$ (m)	${\mathit{l}}_{\mathit{yi}}$ (m)	${\mathit{l}}_{\mathit{zi}}$ (m)
T1	0.156	0.111	0.085
T2	0.156	−0.111	0.085
T3	−0.156	0.111	0.085
T4	−0.156	−0.111	0.085
T5	0.120	0.218	0
T6	0.120	−0.218	0
T7	−0.120	0.218	0
T8	−0.120	−0.218	0

, the moment arms of the 8 thrusters relative to the center of gravity of the BlueROV2 are calculated.

The rotation angles of the horizontal thrusters from T1 to T4 are, respectively, $\pi /4$, $-\pi /4$, $-3\pi /4$, and $3\pi /4$. The thrusters from T5 to T8 are vertical thrusters without horizontal rotations.

The physical and hydrodynamic parameters of the BlueROV2 are summarized in

Table 2. BlueROV2 Parameters.

Parameter	Symbol	Value	Unit
Mass	m	11.5	kg
Buoyancy	B	114.80	N
Weight	W	112.82	N
Center of Gravity	${\mathbf{r}}_g$	(0, 0, 0)	m
Center of Buoyancy	${\mathbf{r}}_b$	(0, 0, −0.02)	m
Inertia Moment	I	diag(0.26, 0.23, 0.37)	kg·m2
Added Mass Parameters	$X_{\dot{u}}$	−5.5	kg
	$Y_{\dot{v}}$	−12.7	kg
	$Z_{\dot{w}}$	−14.57	kg
	$K_{\dot{p}}$	−0.12	kg·m2
	$M_{\dot{q}}$	−0.12	kg·m2
	$N_{\dot{r}}$	−0.12	kg·m2
Linear Damping Parameters	$X_u$	−4.03	N·s/m
	$Y_v$	−6.22	N·s/m
	$Z_w$	−5.18	N·s/m
	$K_p$	−0.07	N·s/rad
	$M_q$	−0.07	N·s/rad
	$N_r$	−0.07	N·s/rad
Nonlinear Damping Parameters	$X_{u|u}$	−18.18	N·s2/m2
	$Y_{v\left|v\right|}$	−21.66	N·s2/m2
	$Z_{w\left|w\right|}$	−36.99	N·s2/m2
	$K_{p\left|p\right|}$	−1.55	N·s/rad2
	$M_{q\left|q\right|}$	−1.55	N·s2/rad2
	$N_{r\left|r\right|}$	−1.55	N·s2/rad2

.

3.5. Control System

3.5.1. Line of Sight Guidance

Line of sight (LOS) guidance is a strategic approach in navigation categorized as a three-point nonlinear guidance law for underactuated vehicles. The term “line of sight” is derived from the tactical requirement that the interceptor’s movement be directed along the LOS vector, which connects the reference point and the target, as shown in Figure 4 [57].

This approach transforms the path-following control problem into a heading control problem by aligning the vehicle’s movement with a dynamically calculated reference point along the intended path. This alignment is achieved through constant adjustments to the vehicle’s heading, ensuring that it remains on course toward the next waypoint, to simplify the navigation process [57,58]. Therefore, the LOS law takes the waypoints as input and calculates the desired heading angle to minimize the distance to the waypoint. Considering the $k^{th}$ waypoint position $P_k=[x_k,y_k]$, while $P\left(t\right)=\left[x\right(t),y(t\left)\right]$ is the current position of the vehicle at time t with respect to the Earth-fixed frame, it is possible to calculate the distance $\Delta P=[\Delta x,\Delta y]=\sqrt{{(x_k-x\left(t\right))}^2+{(y_k-y\left(t\right))}^2}$ and the reference angle $\alpha \left(t\right)=arctan(y_k-y\left(t\right),x_k-x\left(t\right))$.

The strategy is particularly valuable in applications where precise and straight-line movement is essential, like surveying, mapping, and targeted exploration.

3.5.2. High-Level Logic

High-level logic refers to a more abstract representation of the logic used to control a system, which uses state and transition diagrams to model system behavior. In Figure 5, the high-level logic of the LOS guidance law is represented.

The guidance law takes the system into 5 different states during the survey. The first state is the initial one, where reference coordinates, waypoints, and error variables are initialized. The second state aims to reduce the depth error until it falls below the specified threshold of 0.5 m. This threshold was a good compromise between the sensor error and the speed at which the ROV descends. With an error of 0.5 m, the movement is slow enough for the vehicle to begin the process of rotation adjustment. If the error is still above the threshold, this stage will be repeated. In the third state, the yaw error is calculated from the reference yaw angle and the current yaw measurement. If the absolute value of this error exceeds the predefined threshold of 5 degrees, the state is repeated. Otherwise, it proceeds to the execution of the fourth block. Similarly, the considerations made for the depth threshold also apply to the yaw. The block calculates the distance between the current position and the desired point by using the square root of the sum of the differences between the measured and desired components along the x and y axes. The end state is reached if this calculated distance is below the predefined threshold of 2.5 m and all previously designated waypoints have been reached. Otherwise, it returns to the initial state to process the next reference point. This threshold for the distance was chosen based on the acoustic navigation system’s accuracy. The USBL documentation states an error of up to 2 m plus 1 m for GPS.

Depth and orientation errors serve as inputs to two separate PID controllers, implemented into the BlueROV2’s flight controller, while the distance error is the input of a personalized PID controller, implemented in the system.

3.6. Digital Twin

The digital model depicts the system consisting of the ROV, in Figure 6, the swimming pool, in Figure 7, and the environment representing the dock of the port Point Rouge of Marseille.

Additionally, a reconstruction of the seabed with points of interest (marked as red squares) has been integrated into the digital twin of the Marseille harbor. Figure 8 shows the entire operational virtual environment.

The simulation in Unity focuses on replicating the behavior of the BlueROV2. This is achieved through dedicated scripts that establish a connection to the ROS topic. The primary purpose of these scripts is to wait for and receive messages containing information about the movements and orientation of the ROV. Once received, these messages are converted into a format (Vector3) compatible with Unity, and orientation is extracted as a quaternion. With the data received, the simulation updates the matrices that manage the position and orientation within the Unity environment. This allows for an accurate reflection of the real-world movements and orientation changes in the ROV in the virtual context. Essentially, every movement or change in the direction of the ROV in the real world is translated and replicated in the simulation, providing a realistic experience of the operation of the ROV.

4. Results

The NGC software was tested in two divergent aquatic contexts: a controlled environment within the swimming pools of “Università Politecnica delle Marche” and “Lycée Marseilleveyre” as well as the natural environments of the Ancona and Marseille Point Rouge ports. This section particularly focuses on presenting results collected during field tests conducted at the port and evaluating their efficacy under sea conditions, including navigation precision, response to control inputs, stability under varying sea states, and data communication efficiency. The software’s comprehensive version, which integrates the digital twin, was not tested in the sea. However, its performance was validated through simulation-based results.

4.1. Performances Evaluation in the Real Environment

The first step was to test the correct communication between the components. Figure 9 shows that the ROV is accurately positioned on the QGC map in the Port of Ancona, indicating that the data from the acoustic system have been received correctly.

Information such as latitude, longitude, time, depth, and ROV orientation was recorded and stored in a file during the mission. These data were subsequently cleaned and used to generate graphs in MATLAB to evaluate the results.

A trajectory graph of the ROV was created, as shown in Figure 10. In order to anonymize the real coordinates of the ROV, they have been translated while ensuring the meaningfulness of the data remain uncompromised.

The vertical axis represents the depth, which appears to fluctuate throughout the trajectory. This suggests that the ROV is responding to control commands to reach its depth, conditioned by environmental factors. In addition, it can be observed that the ROV first adjusts its depth and then initiates movement towards the target. The trajectory in the latitude and longitude planes shows the path taken by the ROV. It should be a linear path as an LOS strategy has been used, but external deviations such as currents can affect the trajectory. However, the control strategy can overcome these disturbances. The endpoint of the trajectory, compared to the desired end location, will indicate the accuracy of the NGC system in reaching the target coordinates.

The graph in Figure 11 shows how the depth of the ROV varies over time, which is useful for analyzing the system’s dynamics. This graph highlights a short transient response, showing how quickly the control system can respond to the command and start the descent. After the initial descent, the depth line becomes relatively horizontal, with small fluctuations around the desired depth value (the red line). This graph indicates that the ROV has reached the desired depth and is maintaining it, which may reflect the ”hold depth” capability of the control system.

Separate graphs for roll, pitch, and yaw over time, shown in Figure 12, analyze the ROV’s balance and orientation control. Before testing, the ROV was balanced using floats, and this lateral balance is evident from the roll graph, which fluctuates around a relatively stable mean. When the motors start, a small torque is generated because the center of mass is not at the same level. Apart from this initial fluctuation, the graph suggests a system actively maintaining side-to-side equilibrium. However, there are more noticeable oscillations in pitch, probably due to sensors such as the camera, which could cause an imbalance in the front-to-back orientation. Furthermore, the umbilical cable always causes a little disturbance. The yaw graph shows adjustments being made to align with the target heading. In particular, as shown also in the trajectory graph, there are some changes in orientation, proving how effectively the control algorithm handles external disturbances by recalculating the ROV’s orientation to bring it back to the desired position. After passing the waypoint, around 50 s, only the depth control, roll, and pitch remain active. Hence, in the final few meters, the forward speed is significantly reduced, making the ROV more susceptible to disturbances. Thus, the vehicle started at a speed of 0.5 m/s and then decelerated as it approached the desired point. At these speeds, the USBL delay of 4 s to update the new position is not a significant issue. However, it is still important to properly tune the PID controller for advancement. A proportional gain that is excessively high, combined with the aforementioned delay, can result in the vehicle exceeding the desired point by a considerable distance.

4.2. Digital Twin Performance Analysis

The system’s operational capabilities assessment was limited to simulations conducted by ArduSub and Simulink. Although these simulations cannot replicate the unpredictability of a real aquatic environment, they provide insight into the theoretical performance of the system. However, it was possible to conduct in-water tests to confirm the successful connection of the ROV to QGroundControl and ROS. This aspect of the system performed as expected, demonstrating reliable communication links. The user input waypoints to specify locations to visit to reach points of interest with target objects. Simulink was used to simulate the behavior of the ROV, and the movements were plotted on the graph shown in Figure 13, where waypoints are represented by red dots and the simulated ROV trajectory by the green line. The actual path followed by the ROV, depicted in blue, was also plotted for comparison with the simulated trajectory to analyze the performance of the two systems. A detailed analysis of the results demonstrated a significant alignment between the simulated and actual paths. The differences are mostly due to the measurement error of the USBL and slightly to the forces generated by the tether. During the entirety of the simulation, Unity depicted the behavior of the ROV.

5. Conclusions

The paper focuses on the challenges of marine robotics and explores the field by integrating advanced technological solutions with practical applications. The BlueROV2 was selected as the vehicle of choice and was paired with the Seatrac USBL navigation system. The implementation of the line of sight control strategy with a linear trajectory was found to be highly effective in achieving the project’s objectives, ensuring precise and efficient movement of the vehicle despite external environmental disturbances. The incorporation of a digital twin, which mirrors the behavior of the ROV in real-time using intermediary software, represents an advancement in remote vehicle monitoring and control. This digital replication not only reproduced the system’s functionality but also provided a tool for testing and simulation. The improvements our digital twin architecture provided translate into significant practical benefits. Faster inspections mean reduced operational costs and minimal disruption, while lower error rates enhance the safety and reliability of the inspections. The integration of communication with the USBL compared to [49] allows for a real-time position and, therefore, the testing of new NGC algorithms more easily in simulation and in the real environment. The NGC system’s reliability and effectiveness have been demonstrated through sea trials and digital simulations, showing a high level of control over the ROV’s movements. It efficiently maintained stability, navigated to precise coordinates, and adjusted its orientation according to the mission’s requirements. Although the acoustic navigation system transmitted data at a minimum interval of every 4 s, this limitation did not pose a significant problem in this application. However, it could reduce the system’s responsiveness in more dynamic or demanding scenarios. Moreover, the USBL exhibits reduced accuracy in non-optimal environmental conditions, indicating sensitivity to external factors such as distances from surfaces or unknown interference, potentially leading to errors higher than 2 m or, in the worst case, loss of communication.

Future investigations should concentrate on developing and integrating more accurate communication systems with higher bandwidths and lower latency or on real-time integration of data from various sensors to improve the reliability of the system, as well as implementing data fusion algorithms to estimate the position between one measurement and another. It would also be of interest to apply a more robust control algorithm and to integrate the modeling of the tether for a more comprehensive understanding of the tether’s impact on the ROV’s movements and dynamics.