1. Introduction
Bridges, buildings, roads, dams, and pipelines are key components of a nation’s civil infrastructure, influencing urban development, economic growth, and industrial prosperity. To ensure the safety, reliability, and longevity of these structures, standardized design criteria have been established. Despite these measures, however, structural deterioration and failure remain significant concerns [
1].
Structural degradation can result from various factors, including inadequate design durability, poor construction quality control, high levels of pollution, and insufficient inspection and maintenance. Consequently, infrastructure may become functionally obsolete long before the end of its expected lifespan [
2]. Therefore, regular and thorough inspection of civil infrastructure is essential for maintaining its quality and functionality. However, conducting these inspections is costly and requires specialized training to use various technologies. Manual inspections can be prone to errors and, in some cases, may even cause damage to the infrastructure. Moreover, traditional inspection methods often disrupt the normal operation of the infrastructure and expose workers to hazardous environments. As a result, these conventional methods are limited by time constraints and access challenges. To overcome these limitations, emerging technologies and advanced monitoring techniques for structural condition assessment have been proposed. Structural Health Monitoring (SHM) is designed to provide a continuous assessment of the condition of a structure or infrastructure throughout its lifespan. It evaluates the state of the materials, individual components, and the entire system that makes up the structure. SHM monitors, verifies, and reports on any changes in the structure’s condition, providing engineers with reliable data to support informed decision-making and effective management.
In recent years, Structural Health Monitoring (SHM) has become a widely recognized and commonly used tool in structural engineering across many countries. Factors such as shorter construction timelines, increased traffic loads, the introduction of high-speed trains with new dynamic and fatigue challenges, the use of new materials and construction methods, and the need for time-saving solutions all contribute to the growing demand for better control, making SHM an essential tool for ensuring quality and safety.
The rapid advancements in sensor technology, data acquisition and communication, signal analysis, and data processing have further enhanced the effectiveness of SHM.
SHM provides reliable, real-time data on the condition of structures. Civil infrastructure, including bridges, buildings, towers, pipelines, tunnels, dams, and other critical structures, must be maintained to ensure their safety and reliability for everyday use. Maintaining the structural integrity of these assets is vital for the well-being of society. In this context, structural health can be defined as a structure’s current capacity to provide its intended level of service safely and cost-effectively while withstanding expected hazards throughout its service life.
Health monitoring of civil infrastructure involves identifying the location and severity of damage to structures such as buildings or bridges by analyzing measured parameters. This process includes the design and configuration of sensor setups, cable connections, enclosures, central units, data loggers, and communication systems, with detailed schematics created for the entire system. The selection of sensors for SHM is highly dependent on the specific type of structure being inspected [
3].
Currently, infrastructure monitoring demands significant human effort, along with specialized and costly equipment [
4]. While Non-Destructive Evaluation (NDE) sensors have a broad range of applications in infrastructure inspection and maintenance, these tasks still require highly trained personnel. Manual data collection using NDE sensors can introduce errors. These challenges in conventional infrastructure inspection can be addressed with fully autonomous or teleoperated robotic inspection systems [
5,
6,
7,
8]. Such robotic solutions offer the potential to reduce inspection time, improve safety, and lower maintenance costs for civil infrastructure [
9]. As a result, robotics and automation technologies are increasingly being explored for use in the inspection, maintenance, and rehabilitation of infrastructure.
Understanding the critical importance of inspection for the maintenance and assessment of structural health is especially vital when dealing with the marine environment. Structures built or placed in marine settings are particularly susceptible to degradation over time, necessitating regular inspections to ensure they remain functional, safe, and economically viable [
10]. Traditional methods used for underwater inspections often face limitations due to outdated techniques, which highlight the benefits of newer solutions that leverage robotic technologies and image-based underwater inspection methods [
11].
Structures in the marine environment are a crucial component of the infrastructure network. Due to the harsh and corrosive conditions, these structures are particularly vulnerable to rapid aging and deterioration compared to those on land. As a result, the need for efficient inspection methods is even more pressing. The ideal inspection strategy involves assessing the structure at the right time and location, using the appropriate tools, all while minimizing costs [
12].
Subsea infrastructure systems, and, more broadly, any structure exposed to or submerged in water, can be broadly categorized into four main groups:
- -
Offshore structures used for oil and natural gas extraction from the seabed, along with communication cables.
- -
Waterfront facilities such as piers, retaining walls, and docks.
- -
Pipelines that transport oil and natural gas to processing facilities.
- -
Naval vessels, including ships and submarines.
In the 1980s, numerous bridge collapses were attributed to underwater failures, with scour being the most common natural phenomenon responsible. Scour occurs when the erosive action of flowing water wears away the bed and banks of flood channels, weakening underwater infrastructure and leading to structural collapse [
13]. The scour process poses a significant threat to the integrity of hydraulic structures and bridges, potentially resulting in catastrophic failure if the foundation is undermined.
A series of bridge failures during floods, caused by pier scour, highlighted the urgent need to develop better methods for protecting bridges from the destructive effects of scour [
10]. Inspection procedures vary depending on the type of structure, but there are several general inspection methods and widely recognized practices that apply to maritime inspections.
Over the past few decades, three main methods have been commonly used for conducting underwater inspections, namely, wading inspection, self-contained diving (SCUBA), and surface-supplied diving. The term “underwater inspection” was traditionally understood as a hands-on process that required the use of scuba gear and other diving equipment, making it reliant on human presence underwater [
14].
Wading inspection is the primary method used for underwater inspection of structures over shallow, wadable streams. The substructure and waterway are assessed using tools such as a probing rod, sounding rod, or line, along with waders and, if necessary, a boat. These inspections can typically be carried out by regular bridge inspection teams equipped with waders, a life preserver, or a boat [
14].
SCUBA, which stands for Self-Contained Underwater Breathing Apparatus, is the most widely used type of self-contained diving equipment. This system allows divers to breathe underwater without the need for surface-supplied air, offering greater mobility and a much wider horizontal range compared to the limitations of an umbilical hose attached to surface-supplied diving equipment (SSDE) [
15].
For bridge inspections, SCUBA diving is particularly suited to sites where environmental and waterway conditions are favorable and where the dive duration is relatively short. However, extreme caution is necessary when visibility is low, currents are strong, or when drift and debris may pose hazards at any depth in the water column [
16].
Monitoring marine structures is often challenged by poor visibility, limited access, and high costs. However, the demand for underwater inspections is expected to increase significantly in the coming decades, driven by the growth of the offshore renewable energy sector and the expansion of major oil and gas companies into deeper and more hostile environments [
17]. Inspections can generally be categorized into three types [
18]:
- I.
Routine Inspections, which consist of a basic check for obvious defects rather than a detailed, in-depth examination. Their purpose is to ensure that structures are functioning as intended and to identify any major maintenance or safety concerns. These inspections do not require specialized NDT (non-destructive testing) tools, but a photographic record is typically created as part of the process.
- II.
Principal Inspections, which involve a visual examination of all parts of the structure, including areas that may be difficult to access. The goal is to evaluate the need for repairs, monitor changes in the condition of structural components, and verify that routine maintenance has been performed properly. While specialized NDT tools are not necessary, digital photographs are usually taken to document any damage.
- III.
Special Inspections, which are conducted to assess the nature, extent, and cause of structural damage. These inspections require a thorough evaluation of the affected areas and typically involve non-destructive testing using specialized equipment and sometimes even destructive testing if necessary. Due to their comprehensive and detailed nature, special inspections are often costly and labor-intensive.
The frequency and specific requirements for each type of inspection can vary across different jurisdictions [
19].
Common underwater NDT techniques include
- -
Electromagnetic;
- -
Ultrasonic (US);
- -
Radiography;
- -
Acoustic Emission (AE);
- -
Vibration Analysis.
Currently, underwater inspections are conducted using various methods, such as divers, Remotely Operated Vehicles (ROVs), manned submersibles, and Atmospheric Diving Suits (ADSs) [
20]. Diver- and ROV-based inspections are the most widely used. Divers offer remarkable dexterity and can cover extensive areas, reaching depths of up to 50 m. They can perform detailed, close-up assessments with specialized equipment when necessary, or conduct broader visual inspections.
ROVs, equipped with a variety of sensors, typically carry one or more cameras or video systems. They can be deployed to much greater depths and remain in operation for longer durations than divers. However, ROVs are considerably more expensive, less flexible, and more prone to failure compared to diver-based methods. As a result, operating ROVs requires careful attention and highly skilled operators, often with formal training or certifications. Despite these limitations, diver-based inspections remain the most common and effective approach [
18].
The effectiveness of visual inspections largely depends on the diver’s ability to observe and accurately document defects. This process is vulnerable to factors such as fatigue, lapses in concentration, knowledge gaps, and subjectivity, all of which can reduce the reliability and accuracy of the inspection [
21]. Furthermore, divers may experience impaired judgment in cold water or due to inert gas narcosis at greater depths when using air as a breathing medium [
16]. To enhance the reliability of visual inspections, photographs are typically taken and included in the inspection report [
22]. Any use of specialized equipment also requires additional training in its operation.
Underwater vehicles are generally categorized into two types:
- -
Manned Underwater Vehicles
- -
Unmanned Underwater Vehicles (UUVs)
In the U.S. Navy, the term “UUV” is often used interchangeably with “Autonomous Underwater Vehicles” (AUVs), although this definition is not universally adopted across the industry. According to [
23], an “unmanned undersea vehicle” is defined as “A self-propelled submersible that operates either fully autonomously (through preprogrammed or real-time adaptive mission control) or with minimal supervisory control, and is typically untethered, except possibly for data links such as fiber-optic cables”.
The term AUV is commonly used to refer to an untethered underwater vehicle, which operates without a physical connection to the surface. AUVs can follow either pre-programmed or logic-driven courses. The key distinction between AUVs and ROVs (Remotely Operated Vehicles) is the presence or absence of a direct, hard-wired communication link between the vehicle and the surface. While AUVs are typically untethered, they can also be connected to the surface for direct communication via an acoustic modem, or (when at the surface) through RF (radio frequency) or optical links [
24].
In recent decades, the offshore energy industry has experienced significant growth due to increasing energy demands. Subsea pipeline and power transmission cable installations have become widespread worldwide, but potential breakages can lead to both equipment damage and environmental harm. Currently, most offshore pipeline inspections are carried out using Towed or Remotely Operated Vehicle (ROV) systems. In these systems, the submersible component responsible for the underwater inspection is powered and controlled via a tethered cable [
25].
ROVs can be categorized into three main power types [
26]:
- -
Surface-powered vehicles, which are tethered to the surface, where the power source is located. The vehicle does not store power onboard, and all energy is supplied through the tether.
- -
Vehicle-powered vehicles, which store their power on board in the form of batteries, fuel cells, or other energy sources needed for propulsion and operation.
- -
Hybrid systems, which combine both surface-supplied and onboard power. For example, a battery-powered submersible might be charged via the tether during periods of lower power demand.
The National Institute of Standards and Technology [
26] defines several modes of operation for unmanned vehicles:
- -
Fully autonomous: In this mode, the unmanned system (UMS) completes its mission without human intervention, within a defined scope of operation.
- -
Semi-autonomous: The UMS and the human operator collaboratively plan and conduct the mission, with varying levels of human–robot interaction.
- -
Tele-operation: The human operator controls the vehicle’s motors and actuators, or assigns incremental goals, using video feedback and/or other sensory input. Communication occurs through tether or wireless links.
- -
Remote control: The human operator directly controls the UMS actuators without receiving video or sensory feedback. The operator issues continuous commands via a tether or radio link.
Communication between the vehicle and its operator can be established through various methods, depending on the distance and type of communication. These include the following [
27]:
- -
Hard-wire communication (electrical or fiber optic);
- -
Acoustic communication (via underwater analog or digital modem);
- -
Optical communication (for surface communication);
- -
Radio frequency (RF) communication (for surface or near-surface communication).
A good ROV possesses several key features, namely, minimal tether diameter; surface power for providing unlimited endurance as opposed to battery-powered systems with limited operational time; small size, enabling operation within tight spaces or structures; and extremely high data throughput capacity for sensor data transmission. It is important to note that ROV systems often involve trade-offs between factors such as cost, size, deployment resources, platform compatibility, and operational requirements. Some of the most widely used sensors on rovers are sonars, which create a 3D representation of the inspected area. However, these systems are often hindered by their size, weight, high cost, and the need for specialized personnel to operate them. The solution presented in [
28] offers a sophisticated approach to underwater inspection, specifically engineered to withstand harsh underwater conditions while performing complex tasks. Its rotating head provides exceptional flexibility for key attachments, enabling the imaging sonar, ultra-high-definition camera, and grabber to rotate and achieve optimal angles for specific mission objectives. Despite the impressive performance of the ROV in [
28], its advanced features and onboard sensors make it costly and difficult to operate without expert personnel. A review of underwater robots is provided in [
29], and a robot designed for shallow water inspection is detailed in [
30]. This paper aims to explore more affordable alternatives for underwater inspection and to evaluate the potential of underwater photogrammetry for damage detection. This study addresses the existing gap in underwater inspection. The research was carried out using a rover, equipped with a commercial low-cost camera but still possessing the necessary features for effective photogrammetry. Experimental photogrammetry tests were conducted on a selected object, both in and out of the sea environment, to validate the proposed application. The results thus far confirm the validity of the approach and support the further development of photogrammetry using this system for underwater inspections.
2. Design Improvements for the Rover
The underwater robot used in this research is the FIFISH V6 (
Figure 1), an advanced underwater rover powered by six thrusters, offering exceptional maneuverability. It can move forward, backward, up, and down while also rotating with roll, pan, and tilt capabilities. Despite its versatility, it maintains excellent stability when capturing images or video with its Sony 4K UHD camera, which offers a 166° field of view. The robot can operate at depths of up to 100 m (see
Table 1 and
Table 2 for details). An interesting feature of the V6 is Posture Lock, whereby the user can tilt the V6 to an angle, e.g., 45 degrees, then maintain that angle while moving in the desired direction; this can be especially useful when performing hull inspections, etc.
A 100 m cable connects the FIFISH V6 to its controller, which in turn communicates via Wi-Fi with a smartphone or tablet, allowing for real-time video streaming. Up to three devices can be connected simultaneously, enabling multiple users to view the footage in real time.
While the onboard camera is effective for maneuvering and capturing photos and videos, it is not suitable for photogrammetry. To perform detailed inspections, an external camera must be added to the ROV. One limitation of the onboard camera is its fisheye lens, which causes image distortion and curvature at the edges. To address this issue, a custom support structure has been designed for mounting the external sensor. To ensure proper fitting and functionality, a 3D model of the ROV must be created for the sensor support design using reverse engineering techniques, which can be defined as the process of digitizing a physical object through 3D scanning and subsequent processing by CAD software. This process allows for the replication or modification of an existing object, as well as the creation of a new product that can interface with it; in fact, it was used to create suitable support for an external device for the ROV.
A schematic overview of the steps involved in the reverse engineering process is illustrated in the diagram in
Figure 2.
The 3D geometry of an object can be generated through photogrammetry by capturing a series of 2D photographs from various viewpoints. Photographs are taken at 32nd equally rotationally transformed locations about the center of the model at 13 height levels, giving (316) images in total for the model. Between each photo, the camera position is rotated by 10 degrees around the center point of the model with the camera focused approximately on the model’s center. A Canon EOS 500D camera with an 18–55 mm F/3.5 and a 3168 × 4752 pixels resolution lens is used in this setup. By determining the camera’s position and matching corresponding points across the images, the 3D model is created using a triangulation process [
31], as shown in
Figure 3.
Figure 3a illustrates the resulting point cloud, while
Figure 3b displays the triangular meshes derived from the data.
A second method used for the reverse engineering of the FIFISH V6 was 3D scanning. To capture the point cloud of the ROV, a Creality CR-Scan 01 3D scanner was employed, with the results displayed in
Figure 4. It adopts an intelligent large-scale alignment algorithm to ensure automatic matching without manual grid alignment, calibration, or the use of markers. The CR-Scan can be used in two different modes, namely, hand-scan mode and turn mode, and multiple poses can be recorded, which are automatically aligned by the scanner and made into a complete model. Objects are captured by the scanner with an accuracy of up to 0.1 mm. The 3D scanner also ensures high-precision details when scanning larger objects.
The comparison demonstrated that photogrammetry can be a relatively inexpensive and cost-effective approach for generating 3D point clouds for reverse engineering, enabling the creation of CAD-ready models suitable for FEA simulations or 3D printing.
For the specific case, the 3D model was utilized to design a support structure for installing the sensor. The support was initially designed for a camera but can accommodate any sensor that the rover can support without altering its fluid dynamics. The installation of the support with the sensor is shown in
Figure 5 for the final set-up.
3. Experimental Tests
The experimental tests focus on the photogrammetry of an anchor both in and out of the water, with the goal of validating the underwater photogrammetry conducted using the FIFISH V6 rover and the Brave 7 camera.
The first phase of the study involved assessing the handling of the rover. Before conducting the photogrammetric tests, it was essential to understand the rover’s performance under various conditions. Since the sea can be rough and weather conditions may be unfavorable, it was crucial to ensure that the ROV could maintain stability in the water and respond accurately to maneuvering commands to capture high-quality photogrammetry. To achieve this, the handling of the FIFISH V6 was tested in different weather conditions along the Formia–Gaeta coastline, considering both favorable and adverse conditions, as well as the impact of seasonal variations. First experimental tests are shown in the video.
The first maneuvering test of the rover took place on 18 September 2022 under favorable weather conditions (
Figure 6a), while the second test was conducted on 1 February 2023 in less favorable weather conditions (
Figure 6b). The tests showed that the rover is fully maneuverable in both favorable and unfavorable weather conditions. Strong currents and rough seas had no impact on maneuverability, enabling smooth movements in all directions, including 360° rotations, pans, and tilts. Maneuverability was also checked, carrying the camera mounted on the designed and built support; this was an important step since there is no information about the payload of the ROV and its ability to navigate carrying extra loads.
The next task was to test the performance of the camera underwater. The successful overlapping of images is a crucial point in photogrammetry, and a potential disadvantage of underwater photogrammetry is the inability to correctly place the photos due to the lack of geo-referencing. To overcome this problem, markers were created. These markers made it easier to overlap the images in post-processing. Markers were 3D-printed and placed on a suitable base with connecting rods, as shown in
Figure 7. To test photogrammetry inside and outside the marine environment, an anchor was selected. It was placed on the base with the markers (
Figure 7b), and an experimental campaign was carried out (
Figure 7a). Indeed, the method requires the use of markers to be placed before the experiments; alternatively, they can be embedded in the structure to be examined. Photos were taken both outside and inside the water at six different height levels, with the camera positioned at rotationally equally spaced intervals around the anchor’s center. Each set of images is captured every 10 degrees, resulting in a total of 180 images for the model. The camera is always aimed at the center of the model during each rotation.
Figure 8a displays a typical photograph of the anchor outside the water taken at a single height, with the camera being systematically moved to new locations. With careful planning, the 180 photos necessary for model reconstruction can be captured within an hour. The point clouds from the images were generated using the open-source Meshroom software (V 2021.1.0), while point cloud cleaning and the creation of triangular meshes were carried out using Geomagic Studio. An example of the point clouds generated from this photo set is shown in
Figure 8b.
Figure 9a shows a typical photograph of the anchor taken at a single height, with the camera systematically moved to new locations underwater. In this instance, the FIFISH V6 equipped with the Brave 7 camera add-on was used to capture the images. With careful planning, the 367 photos needed for model reconstruction can be captured within two hours.
Figure 9b illustrates the point cloud generated from the set of photos. By using the open-source software Cloud Compare, the anchor images are extrapolated from the point clouds. In order to compare the tests, the point clouds were generated in 3D Zephyr, and the meshes were generated, as shown in
Figure 10 and
Figure 11. Preliminary results were presented in [
32]. The first comparison was made by considering the number of points obtained from the point clouds and number of elements obtained from the meshes (
Table 3).
The second comparison was made using the confidence level of the dense point clouds. With the confidence levels shown in
Figure 12 and
Figure 13, it is possible to appreciate the accuracy of the two tests; the photogrammetry of the anchor inside the marine environment is less accurate than the photogrammetry of the anchor outside the marine environment due to water pollution, moving sand, and poor light.
Finally, the last comparison was made; the meshes were imported and the measurements of the anchor outside and inside the marine environment were conducted as it is possible to appreciate in
Figure 14 and
Figure 15. Dimensional comparison is reported schematically in
Table 4. Experimental tests were conducted at depths varying from 1.5 to 7 m. During the tests, the turbidity of the water was related to the environmental conditions and the period of the year. Since the depth, no lighting setup was considered. The rover was tested in good and bad weather conditions; in particular, the wavy water did not affect the rover maneuverability but the accuracy of the results because of increased turbidity. The smallest object to be appreciated is of an order of millimeters, but this value can vary according to the water conditions. In conclusion, the photogrammetry obtained using the ROV demonstrates a level of quality comparable to that achieved by manually operating the Brave 7 camera outside of the marine environment. The results were convincing, supporting the potential for developing an underwater inspection system based on autonomous photogrammetric techniques. It is worth noting that the application of the method requires the use of markers that can be conveniently prepared and placed on the object or structural element to be inspected in a pre-testing phase.
The confidence factor (
Figure 12 and
Figure 13) in digital photogrammetry is a measure of the reliability and accuracy of the reconstructed 3D points and the overall model. It is calculated using mathematical and statistical tools applied during the optimization process, particularly with the bundle adjustment algorithm. It is associated with key values such as residual errors, standard deviations, RMS (Root Mean Square Error), and the uncertainty of 3D point coordinates. These metrics help to assess the quality of the reconstruction and highlight areas where precision may be compromised. Low values typically indicate high accuracy and reliability, while higher values point to potential issues, such as errors in camera calibration, poor image overlaps, or mismatches in feature detection. Addressing these issues ensures a more accurate and reliable 3D model. In marine environment, it can be observed that the confidence factor values are very low, indicating a high degree of accuracy achieved during experimentation. This suggests that the methodologies and conditions applied in such settings are particularly effective in ensuring precise and reliable 3D reconstructions.
Despite advancements in 3D digital mapping technologies, the accuracy and resolution of 3D data provided by the sensors in underwater inspection remain highly dependent on various external factors, with distance to the object being the most critical, as reported in [
33]. As a rule, the greater the distance between the sensor and the object, the lower the accuracy and resolution. This simple geometric principle underpins many reconnaissance operations at sea, such as searching for a specific underwater target. In such cases, assumptions and prior knowledge about the object are crucial. While submerged optical sensors offer unmatched spatial resolution, they are rarely used for surveying large seafloor areas. Their effectiveness is limited to only a few meters from the object due to factors like water turbidity (sometimes as little as a few centimeters in certain seas or lakes) and the absorption and scattering of light. However, it has been shown in [
34] that when the surveying area is small, lower-cost cameras, such as GoPro, can produce results similar to those produced by higher-cost custom camera systems, provided that the proper camera settings are selected.