1. Introduction
As cosmopolitan animals, deep-sea gorgonians are found in a wide variety of geographical locations and depth ranges, settling primarily on rocky substrates such as continental shelves, slopes, canyons, and seamounts [
1]. Their typical fan shape adds an important three-dimensional component to the ecosystem, adding complexity to the seascape, attracting numerous epibenthic species and thus concentrating a high level of biodiversity [
2,
3]. However, gorgonians are highly vulnerable to impacts caused by fishing gear due to their fragility, low recovery capacity, and low growth rates [
4,
5,
6]. As a result, they are included in the EU Habitats Directive as components of the 1170 Reefs and are cataloged as Vulnerable Marine Ecosystems (VME).
Gorgonians are passive suspension feeders and play an important role in biogeochemical cycles by transferring energy from the water column to the benthos [
7,
8]. The activity of the polyps, which includes expansion and retraction, depends mainly on food availability and local currents [
9,
10]. Octocorals are suspension feeders that primarily filter small organic particles from the water, including phytoplankton, ciliates, bacterioplankton [
11,
12], and mesozooplankton [
13]. Although corals are known to feed on zooplankton and other particulate material, further studies are necessary to elucidate the specific natural prey per species.
Their fans may exceed one meter in height and are usually located perpendicular to the direction of the prevailing current [
14,
15]. This orientation strategy maximizes the volume of water flowing through the polyps, increasing their access to food.
The feeding behavior of deep-sea corals is influenced by environmental factors and food supply. Gorgonians seem to benefit from the acceleration of the current flow [
16]. Some studies show that the optimal feeding rates for gorgonians are at moderate flow velocities. Feeding rates initially increase with flow velocity, reach a peak in the range of 8 to 15 cm·s
−1, and then decrease with low and high flow velocities [
17,
18,
19]. However, the optimal feeding speeds vary according to the species evaluated, and these types of studies are scarce.
The study of deep ecosystems is complex due to the difficulty of accessing them. Methodologies that require the extraction and death of specimens from the seafloor must be limited to avoid damage to these valuable ecosystems. For this reason, image-based studies are essential for increasing knowledge of deep marine ecosystems. Within this approach, the use of underwater Remotely Operated Vehicles (ROVs) or Remotely Operated Towed Vehicles (ROTVs) is a common practice. However, remotely operated vehicles record data at a specific time, and their use requires large oceanographic vessels with high costs and time-consuming operations.
Monitoring the activity rhythms of deep-sea coral species can provide information about basic biology, such as feeding and reproduction processes. Identifying the relationships between this activity and physical variables enables the replication of the behavior of the target species and modeling of their response to future conditions. The activity rhythms of gorgonian polyps during feeding processes are not well understood in most deep-sea species. Several similar studies have focused on cold-water coral reefs (CWC) dominated by
Madrepora oculata Linnaeus, 1758, and
Desmophyllum pertusum (Linnaeus, 1758). These reefs can be found at moderate depths in northern latitudes and are transferable to captive experiments due to their survival in aquariums. These factors facilitate the study of polyp activity or rhythms in the CWC [
20,
21,
22]. In deeper areas, the feeding rhythm of the
Paragorgia arborea and its relationship with hydrographic conditions have been studied in detail [
23,
24,
25].
In order to understand the dynamics and behaviors of deep-sea species, it is necessary to develop methodologies that allow the capture of data from deep-sea habitats over long periods of time. This approach can be achieved with deep multiparameter observatories or oceanographic landers. This tactic is useful for evaluating benthic species and habitat characterization in Marine Protected Areas (MPAs) and VME using underwater cameras [
26,
27,
28]. It is a technological challenge to achieve systems with numerous sensors that work at great depths and remain in operation for long periods of time. Moreover, reducing the costs will make it possible to have more sampling sites and increase our observation capacity.
As our ability to observe deep habitats advances, the need to develop image-processing methodologies based on automation techniques becomes more evident. Deep-learning-based techniques have boosted ocean data processing. Using these algorithms, image annotation can be carried out automatically, achieving the automatic identification and classification of species that inhabit the deep sea [
29,
30,
31], and even advancing in the mapping of deep-sea habitats [
32,
33,
34,
35]. The use of a deep learning model for the semantic segmentation of images, also known as pixel-level classification, is a task in computer vision that involves classifying each pixel in an image into one of several predefined classes. This technique seems useful for applications such as self-driving cars and medical image analysis, but its application in underwater images is still scarce. However, this technique has proven to be a significant advance in the process of automating the labeling of underwater images [
36,
37,
38].
Marine Protected Areas are important conservation tools to mitigate the effects of climate change and other anthropic pressures on marine ecosystems, as well as to protect biodiversity, emphasizing VME characteristic species, as cold water corals and sponges grounds. Studies on deep-sea corals help us better understand their biology and the role they play in ecosystems. Given their high vulnerability to anthropogenic impacts, such as deep-sea mining or fishing, this information is essential to understand in detail the effects of these human activities on them [
39]. Therefore, the aims of this study are to (1) describe the activity of the gorgonian
Placorgia sp. polyps, (2) to develop an automatic methodology to monitor feeding activity using deep learning, and (3) to relate gorgonian behavior to environmental conditions and food availability.
2. Materials and Methods
2.1. Study Area
Le Danois Bank, also known as “El Cachucho”, the first offshore Marine Protected Area (MPA) in Spain [
40], is located in the Cantabrian Sea, off the northern coast of Spain. It is a large offshore bank and seamount surrounded by slopes and a complex system of channels and canyons (
Figure 1a). This MPA was later included in the OSPAR Network of MPAs in 2009 and the Natura 2000 Network as a Special Area of Conservation (SAC) in 2011. The management plan includes specific measures for fishing activities, oil exploration, minerals and military activity. The bank and its intra-slope basin cover an area of 234,000 hectares, with depths ranging from 425 to 4000 m. The main circulation patterns show the presence of anticyclonic flow at the seamount summit. However, the entire system is affected by strong and variable along-slope currents, making it a complex and dynamic environment [
41]. These characteristics make the MPA a crucial biodiversity hotspot in the Cantabrian Sea [
42].
The area is home to deep gorgonian forests, which were crucial in its designation as MPA. The gorgonian
Placogorgia sp. forest, where this study is focused, is located in a hard bottom seamount in the southwest of the bank, with a bathymetric range of 500 to 600 m and a surface area of 2.88 hectares (
Figure 1b). The gorgonian colonies in this area are fan-shaped and grow at different densities, curvatures, and with varying numbers of branches (
Figure 1c).
2.2. Placogorgia sp. Colonies Morphometric Characterization
The use of photogrammetric techniques has greatly advanced the study of benthic species by enabling three-dimensional reconstructions of marine habitats. This approach provides a precise measurement of the size and orientation of specific specimens and allows for accurate evaluation of the habitat’s three-dimensional component, which is directly related to the biodiversity associated with it [
43,
44,
45]. Estimation of gorgonian population structure (i.e., maximum height, density), morphometrics (i.e., maximum width, fan area), and biomass are valuable data for description of gorgonian forests [
46,
47]. It has also been useful in studying basic aspects of the biology of sessile species, such as growth rates and degradation suffered in coral reefs [
48,
49,
50,
51,
52,
53].
In 2019, a detailed characterization of the 3D sizes of this gorgonian population was carried out [
47], generating valuable information about the group of gorgonians that inhabit this area, being for the moment the only aggregation of this species present in the entire protected area. The study was carried out through the analysis of the video transects obtained at the Le Danois Bank, using the Politolana underwater towed vehicle during the July 2017 ECOMARG survey. Video-sections were decomposed in thousands of geo-positioned overlapping images processed using photogrammetric Pix4D Mapper Pro software (Pix4D SA, Prilly, Switzerland). This software carries out an advanced automatic triangulation based purely on image content and an optimization technique. The triangulation algorithm is based on binary local key points, searching for matching points by analyzing all images. Those matching points, as well as approximate values of image position and orientation provided by the Politolana telemetry system, are used in a bundle adjustment to reconstruct the most probable position and orientation of the camera for every acquired image. For this study, the focal length, principal point and radial/tangential distortions were set as initial theoretical values, while the final internal and exterior orientation parameters of the camera were determined by bundle adjustment processing. The distance between parallel lasers beams (separated 20 cm) is used as reference scale. Using three-dimensional reconstructions, the size and orientation of each of the three specimens were calculated by digitizing the contour of the fan in the point cloud. The orientation and surface area covered by the fan were directly related to the specimens’ ability to capture food and withstand the existing currents in the area.
2.3. Image and Oceanographic Data Registration
In May 2022, a lander was deployed in a gorgonian forest at a depth of 515 m using a new vehicle called LanderPick [
54]. The LanderPick concept (
Figure 2a) consists of a specific towed vehicle that deploys and recovers lightweight oceanographic landers. The system allows the lander to be positioned with high accuracy at the desired geographical point using the Super Short Base Line (SSBL) system and the Dynamic Positioning System of the oceanographic vessel. LanderPick uses a real-time video camera, lasers, and thrusters to choose its orientation and facing camera to the specific target (
Figure 2b). It also allows the recovery of landers without the need for automatic releases or abandoning material at the bottom. The low-cost design makes it possible to reduce the price of deployments operations, thereby increasing the observation capabilities of deep-sea ecosystems.
The camera system is based on a Raspberry Pi4 computer with a HQ Sony IM477 12.3 megapixels sensor. For data storage an external USB 3.0 250 Gb SSD was chosen. The time-lapse controller is based on an Arduino Pro Mini. This programmable module maintains the low-cost concept. The programming language is Python for the Rapsberry and C for the Arduino-based time-lapse PLC controller. For lighting, it was necessary to develop a system of spotlights based on high luminosity COB LED chips experimentally adjusted to 40 W to maintain low power consumption. To power the system two independent 12 v 150 Ah AGM batteries mounted in fiberglass containers flooded with dielectric oil with an elastic pressure compensation membrane were used. The camera was located 0.8 m above the sea floor with an angle of 30° and configured in time-lapse mode with an interval of 30 min. To describe the hydrographical conditions, the lander is equipped with an upward-looking Acoustic Doppler Current Profiler (ADCP) RDI Workhorse Sentinel 300 kHz recording the vertical structure of the currents every 20 min, a SeaBird37 CTD (Conductivity-Temperature-Depth) recording at 20-min intervals, and an RBR solo-T thermistor with a high sampling rate of 5 s.
The camera was placed 1.5 m away from a group of colonies of
Placogorgia sp. The three complete specimens selected for this study are recorded in the camera’s Field Of View (FOV) (
Figure 2c). From 6 to 29 May 2022, photographs were taken every 30 min, generating a dataset of 1074 images. On the other hand, the oceanographic variables were recorded from 6 May to 22 October 2022, corresponding to the total duration of the deployment.
2.4. Image Processing
In this section, a semantic segmentation model is employed to assign each pixel of the images to specific classes. The selected model is based on ConvNeXt [
55], a neural network architecture that has demonstrated its effectiveness in segmentation tasks, similar to the one being addressed in this study. The pre-defined classes are class 1, which corresponds to active polyps, and class 2, which includes inactive polyps, including intermediate polyp states where the polyps are partially contracted. Therefore, class 1 only comprises the part of the scene where the polyps are fully expanded, enabling the observation of gorgonian behavior at different spatial scales: overall scene, per gorgonian, and intra-specimens (apical, central, and basal zone of the gorgonian fan).
2.4.1. Training and Test Dataset
To perform this task, it was decided to work with approximately one-fifth of the total images, given that gorgonians are static elements and do not exhibit significant changes across the image set. Therefore, a 70-image dataset was selected for training (50 images) and validation (20 images), while a set of 30 randomly selected images was used to test the best model. Evaluation of the model’s performance was carried out using metrics such as precision and the Jaccard index or IoU (Intersection-Over-Union) [
56].
2.4.2. Data Augmentation
To expand the data variability and improve the model’s generalization capacity [
57,
58], data augmentation techniques were applied to the training and validation dataset. Given the small number of samples and the similarity between them, brightness and contrast variation, rotation, and cropping were selected as data augmentation methods. A preliminary visual analysis of the image set revealed differences in gorgonian position due to movement, as well as in illumination and blur, primarily due to water turbidity and camera configuration.
2.4.3. Model
ConvNeXt is a deep learning backbone that is used for semantic segmentation. It is an up-to-date convolutional network and an adaptation of old convolutional networks such as Resnet, with newer techniques like vision transformers. This adaptation enables ConvNext to become one of the most powerful tools for image segmentation. ConvNeXt is a convolutional neural network or CNN-type neural network architecture that differs from other architectures because it uses Convolution blocks with channel groups, improving the network’s learning capacity [
59,
60].
The ConvNeXt architecture consists of several Convolution blocks with channel groups, followed by pooling layers and fully connected layers. The Convolution blocks with channel groups are an improvement to conventional Convolution layers, as they divide the input channels into groups and only connect them to a subset of the output channels, reducing the number of parameters in the network and improving its learning capacity. The ConvNeXt architecture is characterized by having Convolution blocks with channel groups and a residual Convolution block structure, improving the network’s learning capacity and reducing overfitting. Moreover, it can have multiple Convolution blocks with channel groups followed by pooling layers and fully connected layers, achieving better performance in computer vision tasks.
The ConvNeXt block is illustrated in
Figure 3, with the convolution layer being followed by Linear Normalization and Gaussian Error Linear Unit (GELU). To obtain the best model, the selected model was trained with different variations of the training parameters.
The block above applies a mathematical operation called convolution, which is a process of multiplying a small matrix, called a kernel or filter, over a section of the input image. This operation is performed at all possible positions of the image, generating a new image called a feature map.
After the convolutional layer, the ConvNeXt network applies the Linear Normalization layer, which normalizes the activations of the feature map using the Frequency Division Normalization (FDN) technique. This layer helps to reduce the covariance between channels of the previous layer and improve the generalization ability of the network.
Finally, the network applies the Gaussian Error Linear Unit (GELU) activation layer, which is a non-linear function used to introduce non-linearities in the network. Where
is the input to the activation function and
is the cumulative distribution function of the standard normal distribution. It is a mathematical function that assigns a cumulative probability to a given value of the standard normal random variable, also known as the error function.
2.5. Temporal Dynamics Analysis
The annotated images obtained through automatic segmentation have been analyzed according to predefined categories. The data on polyp activity have been independently analyzed for each gorgonian specimen to determine whether there are any differences in behavior among the three specimens or if they exhibit a common pattern. The temporal patterns of extension and retraction, as well as the duration of each period, have been compared. By using a semantic segmentation model, an analysis of the states of the polyps within each gorgonian specimen has been conducted to identify any differences in activity based on their location within the specimen. Each specimen has been divided into three sections (apical, central, and basal), and the activity of each zone has been described and compared.
The feeding activity of the Placogorgia sp. forest has been defined by analyzing time series that describe the activity of the polyps and integrating the percentage of extended/retracted polyps from the overall scene. This method allows for a more accurate description of the feeding activity and also enables the behavior of the specimens to be related to the unique environmental variables recorded in the area.
Two temporal scales have been used to explore the existence of patterns in polyp activity and its relation to the environmental dynamics; (1) the behavior of the polyps based on the time of day (diel), that is, the averaged percentage of active polyps over 30-min sampling periods throughout the day, and (2) the half-hourly behavior over the whole period covering a time series of 22 days.
To address this analysis, a Singular Spectrum Analysis (SSA) approach was used. SSA is a time series analysis method that decomposes and forecasts time series data using tools from time series analysis, multivariate statistics, dynamical systems, and signal processing, and employs the singular value decomposition as its main mathematical tool [
61]. SSA decomposes the original time series into a trend component and oscillatory components that can be associated with periodicity and noise. Time series data is analyzed using a trajectory matrix constructed from lagged vectors. The matrix is a Hankel matrix and can be decomposed into eigenvectors and eigenvalues. By selecting a certain number of eigenvectors, a lower-dimensional subspace can be defined, onto which the data is projected, the resulting matrix is an approximation to the original trajectory matrix [
62]. It should be noted that is a model-free method, meaning it can be applied to all types of time series data and is not restricted to monthly or quarterly data, unlike some other methods that have only been programmed for monthly or quarterly series [
63]. SSA analysis was implemented using the Rssa library [
64] in R [
65].
4. Discussion
Octocorals are vulnerable components of deep-sea benthic communities, however their natural diets and feeding conduct are little known due to the difficulty of accessing them. Gorgonians are passive filter feeders and their polyps have a very limited ability to capture food and because of this, they depend mainly on environmental conditions that favor their feeding. The environmental conditions that determine the availability of food for octocorals in deep-sea are mainly the conditions of currents. In this regard, it is known that the feeding process is optimal when currents of moderate speed occur in the area and in a perpendicular direction to its fan-shaped body. Very slow currents do not bring food to their polyps, while very strong currents do not allow the polyps to capture that food. For this reason, in this study the behavior of polyps has been analyzed against the available environmental data in search of evidence that corroborates these statements.
Differences in polyp activity can be produced due to changes in temperature and current speed, but especially in response to increasing nutritional stimuli [
9]. The gorgonians remain with open polyps, that is, in active mode while they are feeding. The tentacles of the polyps catch passing particles and carry them to the stomach where they are digested. After digestion, the polyp expels the remains through the same opening. When gorgonians contract, it is mainly because there are not optimal conditions for feeding, that is, there is less availability of food or environmental conditions are not favorable for trapping the particles on which they feed [
66,
67].
Precisely defining the fan-shaped growth, i.e., the size, shape and orientation of the gorgonian can provide information about the local current near the seafloor, due to the existence of a relationship of growth with hydrodynamic conditions. Thanks to the very high resolution three-dimensional reconstruction, the specific geometry of these 3 gorgonian colonies can be related to the recorded oceanographic data. It can be seen how these three specimens are arranged with their concave side facing the prevailing current flows in the site. This is contrary to what was stated in [
47], given that although the geometric data of the gorgonian population was available, there was no record of environmental data at the site. Gorgonians grow by forming a concave side that helps them create small micro-currents and turbulence in the path of the water flow. These small turbulences help in the feeding process of the polyps that are located mainly on that side of the fan.
The semantic segmentation model presented in this work shows high performance in terms of accuracy and IoU and can generate accurate and detailed image segmentations. One of the main advantages of semantic segmentation is that it allows for a more fine-grained understanding of an image, compared to traditional object detection methods. This is because it not only detects objects in an image, but also segments them into their individual parts and classifies each part according to its semantic content. This model has immense potential to be used in modeling applications. Thanks to the segmentation model implemented, it has been possible to extract the conduct data of the gorgonians according to the state of their polyps automatically. Therefore, a robust methodology has been established for the image data processing of future long time photo-landers.
All the polyps in the colonies, regardless of the area where they are located, open and close in synchrony. This agrees with previous studies carried out in
Paramuricea clavata [
10]. But it must be taken into account that the low temporal resolution, one photo every 30 min, may mean that the existing data is not the most suitable for studying this phenomenon.
There are different processes within the water column related to the time of day, for example the migratory movements of zooplankton to surface waters to feed at night and their return to the deep layers during daylight hours. These patterns mean that species have developed differential strategies to optimize their feeding and protection against predators. The biological processes that occur in shallow waters depending on the periods of light and darkness are well known, however little is known about the daily cycles in the deep-sea environment.
Placogorgia sp. has been found to be most active in feeding during the day, probably in this hours the food concentrations are highest, and remain least active at night. Research in this area is ongoing to better understand of the trophic ecology of gorgonians and their role in benthic−pelagic coupling processes [
68].
As passive suspension-feeders, gorgonians are highly dependent on prevalent currents to flow water through the colony. Several studies show that to optimise the feeding process the moderate currents are needed. For example for
Pseudopterogorgia acerosa colonies the colony-feeding rates are highest during moderate flow velocities (10–15 cm·s
−1), and reduced at low (0–5 cm·s
−1) and high (20–25 cm·s
−1) velocities [
17]. The polyp activity data obtained for
Placogorgia have been analyzed throughout the entire time series. These data show times when gorgonians remain active for longer periods of time, and also show other times when gorgonians contract more frequently. In addition, it can be observed how the periods of time in which the gorgonians remain with their retracted polyps are stable throughout the series. Digestion time is dependent upon temperature but in this case the temperate are very constant along the sampling period around 10.8–11.0 Celsius degree. The average length of polyp retracted periods is 3.4 h very similar to digestion time of other species of gorgonians [
10].
To address the relation between current dynamics and food availability with the gorgonian feeding behaviour, the SSA approach has been used. The percentage of active polyps of the trend reconstructed time series are compared with the same reconstructed series for current speed, current direction and ADCP return signal intensity. ADCP return signal intensity is used here like an approximation of backscatter. The concentration of suspended sediment is assumed to be directly proportional to acoustic intensity (voltage recorded by the ADCP) and logarithmically proportional to acoustic attenuation due to water and sediment concentration. Linear regressions analyses revealed low correlations between the number of open colonies and all environmental variables (current speed 0.37 and direction and acoustic backscatter 0.11). But the graphs (
Figure 11) show that feeding activity was positively correlated with all of them. It should be noted, for example, in the middle of the observation period, especially low values are recorded in the activity of polyps, from May 17 to 20, which correspond to days in which very low current speeds (mean 7.4 cm·s
−1 and mode 4.3 cm·s
−1) of the observation periods are recorded, and which also coincide with infrequent current directions (values from 45° to 90° deg). In these days ADCP return signal intensity also registered low values.
Both the reconstructed time series of the percentage of active polyps and the selected environmental variables exhibit a sharp decrease at a specific point in time (between May 18th to 20th). This suggests a decline in food availability on those dates and a possible relationship between the low food availability and the prevailing environmental conditions at that time.
The number of open polyps of the Placogorgia sp. colonies, indicative of active feeding, do not show any seasonal pattern, since the short period of time observed does not allow recording seasonal changes in environmental variables.