Remote Measurement of Tide and Surge Using a Deep Learning System with Surveillance Camera Images

Abstract

The latest progress in deep learning approaches has garnered significant attention across a variety of research fields. These techniques have revolutionized the way marine parameters are measured, enabling automated and remote data collection. This work centers on employing a deep learning model for the automated evaluation of tide and surge, aiming to deliver accurate results through the analysis of surveillance camera images. A mode of deep learning based on the Inception v3 structure was applied to predict tide and storm surges from surveillance cameras located in two different coastal areas of Italy. This approach is particularly advantageous in situations where traditional tide sensors are inaccessible or distant from the measurement point, especially during extreme events that require accurate surge measurements. The conducted experiments illustrate that the algorithm efficiently measures tide and surge remotely, achieving an accuracy surpassing 90% and maintaining a loss value below 1, evaluated through Categorical Cross-Entropy Loss functions. The findings highlight its potential to bridge the gap in data collection in challenging coastal environments, providing valuable insights for coastal management and hazard assessments. This research contributes to the emerging field of remote sensing and machine learning applications in environmental monitoring, paving the way for enhanced understanding and decision-making in coastal regions.

1. Introduction

The rise in global mean sea level, attributed to human-induced climate change [1], is resulting in a global escalation in the frequency of coastal flooding, with a multitude of negative impacts for coastal communities, public safety, and economies [2].

The components of flooding include astronomical and meteorological water levels. The astronomical water level is determined by the tidal cycle, which is primarily caused by the gravitational attraction between the Moon and the Earth. Tides follow a regular pattern and can cause a periodic rise and fall in the water level along coastlines. On the other hand, meteorological water levels are influenced by weather events such as storms, heavy rainfall, or strong winds. These factors can lead to a temporary increase in water level in coastal areas, known as “storm surge”. Storm surge can significantly contribute to coastal flooding during extreme weather events. Both of these components can result in an elevation of the water level and contribute to flooding in coastal areas. It is important to consider both factors when evaluating and managing the risk of flooding [3].

Tidal patterns have significant impacts on various aspects of human life and the ecosystem [4]. The positive outcomes of tides include preserving the marine ecosystem, facilitating fishing and harvesting [5], controlling pollution, generating power, and influencing weather circulation [6]. Tides play a vital role in many coastal ecosystems: they influence the habitats, aid in reproductive activities, and support the food chain [7]. In fishing, tides are used to catch fish and harvest seafood, optimizing economic returns [8]. Additionally, they contribute to the generation of renewable energy through tidal energy, harnessing hydraulic forces [9]. Tidal currents impact the circulation of weather, leading to the creation of more favorable climate conditions and contributing to the balance of global temperatures. [10]. They also play a vital role in the movement of sediment along the beach. Phases of high and low tide influence the deposition of sand, gravel, and other particles on the beach [11]. Understanding tides offers opportunities for their optimal utilization in modern society [12].

Significant emphasis must be placed on coastal protection measures in response to storms and the ensuing storm surge.

A storm surge is an anomalous increase in water levels caused by a storm that exceeds the expected astronomical tide. It represents the fluctuation in water level directly influenced by the storm’s presence [13].

The primary driver behind storm surges comes from the powerful winds associated with cyclones [14]. The wind patterns surrounding the cloud-free area of a cyclone’s ‘eye’ are determined by a cyclonic circulation through diabatically produced potential vorticity (PV) anomalies in the lower–mid troposphere [15,16]. In offshore areas, this circulation remains largely undisturbed, resulting in minimal signs of storm surge [17]. When the hurricane approaches nearshore areas, the cyclonic circulation is influenced by seabed friction [18], determining the water flow inland [19]. Regarding coastal protection and emergency management, it is important to emphasize that the storm surge can penetrate far inland from the coastline [20,21].

To capture the characteristics of forcing during extreme events and the resulting impacts on the coastal environment, a combination of direct methods (e.g., wave buoys, tide gauges, LiDAR surveys) and indirect methods (e.g., webcam or satellite images) have long been utilized. In recent times, the popularity of indirect methods has increased significantly due to their ability to analyze large volumes of data using artificial intelligence techniques.

Machine learning and deep learning [22], two branches of artificial intelligence [23], are proving to be versatile tools in the examination of coastal environments [24]. This, in turn, can facilitate the informed and sustainable planning and management of coastal regions [25]. Such efforts contribute to safeguarding marine ecosystems and enhancing the tourism appeal of beaches [26]. Machine and deep learning enable the analysis of beach areas through the processing of satellite imagery [27] or footage from surveillance cameras [28]. For instance, machine learning algorithms can be employed to recognize and categorize various aspects of coastal landscapes, such as beach morphology, that are influenced by tides and currents [29]. Additionally, deep learning facilitates the creation of specialized algorithms tailored to the detection of specific coastal features, enhancing the precision and efficiency of monitoring and analysis systems [30]. This, in turn, positively influences beach management by enabling a more precise evaluation of a beach’s condition and a more timely response to potential emergencies [31].

In this study, we present the results of experiments conducted using an algorithm for automated tide and surge measurement from images captured by a surveillance camera system. Once an image is acquired and submitted to the system, the algorithm is capable of providing the corresponding tidal height and, consequently, the surge [28]. These results are highly useful in areas where the tide sensor is remotely located, necessitating time corrections [32], and in situations of extreme events, such as Mediterranean Hurricanes, where surge measurement becomes valuable.

The tool presented in this paper represents an upgrade of the Convolutional Neural Network (CNN) for Tide Assessment developed in LEUCOTEA [28]. The forecasts produced by neural networks were compared to the observational data derived from tide gauge records. The previous code was written in MATLAB and used the GoogLeNet model [33]. This latest update is translated into Python [34] and is based on the Inception V3 model developed by Google [35].

The employment of a comprehensive observational system spanning multiple platforms provides the opportunity for real-time feedback on the occurrence of oceanic and atmospheric events. Moreover, this approach would enable the execution of analyses that are typically reliant on in situ sensors [36], thereby circumventing logistical challenges and facilitating the seamless creation of information databases [37].

The paper was organized as follows:

• Section 2 contains the methodology section, where the study areas, datasets, and deep learning models are reported;
• Section 3 presents the results and discussion, including predictions and hyperparameters;
• Section 4 contains the conclusions.

2. Methodology

In this section, we present a detailed overview of the methodologies and techniques employed in this study. We outline the specific approaches adopted for the remote measurement of tides and surges, as well as details of the deep learning system utilized for analysis. Additionally, we discuss the preprocessing steps applied to the surveillance camera images and the training procedures undertaken to optimize the performance of the model. The following subsections provide a comprehensive overview of the methodologies used in this research endeavor.

2.1. Data Acquisition and Territorial Framework

In the preliminary step, the process begins by choosing a coastal region for investigation and assembling a pertinent set of images to effectively portray the entire spectrum of potential tidal values. A substantial quantity of images is essential for conducting a thorough analysis. In our case, we chose two locations in Italy, and the model was trained and tested using images from surveillance cameras.

2.1.1. Site 1: Santa Lucia

The first site is Santa Lucia, located on the Maddalena peninsula in the municipality of Siracusa (SR), southeastern Sicily (Figure 1a). It features numerous distinct characteristics, various residential settlements, several commercial activities, as well as areas of natural and archaeological interest. In 2004, the Plemmirio Marine Protected Area was established to safeguard the surrounding marine environment and its abundant marine fauna. The management of the Marine Area provided the images from the surveillance camera.

The coastal region of south-eastern Sicily extends over approximately 300 km and is experiencing significant erosion, with the shoreline retreating at a rate of about 5 m/year [38]. This region has faced severe storms over the past few decades, experiencing waves that have reached approximately 6 m in height and storm surges surpassing 1 m [39]. The examined coastal region displays a mix of small rocky headlands and low-lying beach systems, often adjacent to coastal lagoons. The impacts of numerous tsunamis have been recorded by studying accumulations of boulders [40,41], high-energy deposits [42], and the stratigraphy of the lagoon areas [43,44]. Furthermore, the region has also been impacted by severe storms, resulting in the displacement of boulders and cobbles along the coastline [40]. Similar to other Mediterranean coastal areas, these storm occurrences have undergone impacts similar to those witnessed in past extreme marine events, such as tsunamis.

We also conducted an analysis using data recorded by the Catania wave buoy from the National Wave Network (Rete Ondametrica Nazionale (RON)). The most powerful storm in southeastern Sicily since 1990 was identified, characterized by a substantial wave height (Hs) of around 6.2 m and a peak period (Tp) of 11.3 s. Several medicanes, or Mediterranean hurricanes, have affected the coasts of south-eastern Sicily, and in recent decades, these events have had a greater impact compared to typical seasonal storms. Between 2014 and 2023, ten cyclones passed in the Ionian Sea, with four of them—Qendresa in 2014, Zorbas in 2018, Apollo in 2021, and Helios in 2023 [45]—strongly impacting the coast of south-eastern Sicily. While the Ionian Sea is not typically prone to the formation of tropical cyclones, medicanes result from a process known as Tropical Transition (TT), where an extratropical system transforms into a tropical system or induces a hybrid cyclone [46,47]. However, some studies suggest that climate change may alter medicanes in the future, potentially reducing their frequency but intensifying their impact [48]. Nevertheless, in the Ionian basin, except for 2019, a medicane has crossed the region each year since 2014.

2.1.2. Site 2: Lignano Sabbiadoro

The second site is Lignano Sabbiadoro, a municipality located in the Friuli Venezia Giulia region (Figure 1b). Here, the images, acquired from a webcam placed on the roof of a restaurant overlooking the sea, were provided by the Panomax portal (https://lignano.panomax.com/ (accessed on 4 May 2023)) for tourism in the Friuli Venezia Giulia Region. Despite only having around 7000 inhabitants, it is one of the most important seaside resorts in the Northern Adriatic Sea, with around 3.5 million visitors (ISTAT, 2019). Lignano is located on the peninsula made up of the eastern lobe of the Tagliamento river delta and separates the Adriatic Sea from the Marano and Grado lagoon. The coast, conventionally divided from the Tagliamento river mouth (to the south) to the Lignano inlet (to the north) into the three sectors of Riviera, Pineta and Sabbiadoro, is made up of a single sandy beach approximately 8 km long. The beach has a curvilinear shape: from south to north, the direction rotates progressively clockwise from around 10° N to 45° N at the locality of Sabbiadoro, where the webcam is located, and then rotates in the opposite direction near the tidal inlet. The width of the beach varies from 23 m to 181 m. The seabed facing Sabbiadoro beaches is characterized by the presence of the ebb tidal delta of the Lignano inlet, whose morphological high is evident up to 1600 m from the shoreline, beyond which the depth progressively increases from 2 to 10 m in 1500 m, resulting in an average slope of 3.2‰ [49].

The sediment budget of the shoreface in Sabbiadoro has been positive over the last 20 years [49], although the beach has shown an erosive trend partly compensated by nourishments and partly by the attempt to protect the beach using seasonal barriers made up of plastic bags filled with sand taken in situ.

The tides in the Northern Adriatic Sea are semi-diurnal, with average mean spring tide and mean neap tide ranges of 78 cm, 105 cm [50], and 22 cm [51], respectively.

The local wind climate is affected by two main winds, “Bora” and “Sirocco”. Although the Bora wind (from ENE) is predominant in terms of frequency and strength [52,53], the Sirocco wind (coming from the SSE) is statistically significant, although it holds a subordinate position in terms of strength.

The wave regime tends to exhibit a bimodal pattern due to the prevailing wind conditions. As per data collected from the wave buoy OGS DWRG1 (positioned offshore at coordinates 13.24 E, 45.56 N; 16 m depth), the average significant wave height (Hs) remains below 0.5 m. Instances of Hs exceeding 0.5 m make up 25% of the entire dataset, with prevailing waves originating from the SE (10.7%) and ENE (10.5%). The Sirocco wind contributes to the highest recorded waves, reaching an Hs of 4.4 m [53]. The simultaneous influence of spring tides, seiches, winds, and low atmospheric pressure has the potential to cause a significant elevation in sea level, resulting in a locally recognized surge referred to as “acqua alta”.

The storm surges in the Northern Adriatic Sea are mainly related to the Sirocco and secondly to the Bora due to their different fetches. Considering the beach of Lignano, the fetch of the Bora is around 50 km while the Sirocco acts along the entire Adriatic basin for around 800 km, being able to produce greater and more persistent storm surges at the end of the basin [54].

Extreme storm events in the northern Adriatic have been the subject of multiple studies, most of which are associated with the city of Venice due to its important artistic and cultural heritage. The major extreme events were recorded between late October and December (1966, 1979, 2018, and 2019), while the 1986 event occurred in February [55]. Cavaleri et al. [56] studied the event of 29 October 2018, identifying extreme storm surge values above 150 cm, limited by the out-of-phase astronomical tide, and assuming that much more catastrophic consequences would occur in the case of concomitant in-phase factors. Moreover, the authors observed a significant increase in the nearshore sea level associated with the wave set-up and surface wind stress, caused by the progressive decrease in depth shoreward. Ferrarin et al. [57] analyzed the meteorological characteristics of the November 2019 event, comparing it with previous extreme events in 1966, 1979, and 2018. Unlike previous events, characterized by high storm surges associated with a low astronomical tidal range, the 2019 event showed how the in-phase concomitance of even non-extreme factors can lead to an exceptional rise in sea level. Mel et al. [58] analyzed the event at the end of November 2022, associated with a persistent low pressure over central Italy which generated both Sirocco winds along the Adriatic and Bora winds over the northern Adriatic, causing storm surges of 173 cm (refer to the Punta della Salute gauge datum, ZMPS) and wave heights of up to 4.5 m at the CNR platform, located 12 m offshore from the Venice lagoon.

The barrier islands of the adjacent Marano and Grado lagoon were also affected by such extreme events, and were frequently overwashed and breached with the formation of multiple washover fans [59].

2.2. Creation of the Dataset

The dataset creation constitutes the initial step and is essential for training the Convolutional Neural Network (CNN). This task is typically performed manually by the research team. In our case, we tested the system using two different datasets of images during significant meteorological events. In each frame, a value was assigned based on the real data recorded during the event from the tide gauge sensors. Starting from the historical data that were acquired, the overall range of tidal values in the considered area is divided into intervals, with each representing a membership class. The amplitude value must be as representative as possible of the event type to consider the excursion values of the site. For site 1, a minimum value of −81.4 cm and a maximum value of 89.9 cm were considered, while for site 2, a minimum value of −20 cm and a maximum value of 145 cm were considered. The image datasets were split into groups based on the membership class of the corresponding tidal value. To address class imbalances (Figure 2), a function was implemented to calculate the respective weights for each class. These weights were then converted into a dictionary where the keys represent the unique classes in the training data and the values correspond to the class weights. This approach helps prevent the model from being overly influenced by more represented classes at the expense of less represented ones. The characteristics of each dataset are listed in

Table 1. Dataset and site information.

Name Dataset	Basement	N° Imgs	Train	Test	N° Classes	Site	Coordinates UTM Wgs84
Santa Lucia	Rock	3.266	2.605	661	32	Santa Lucia (SR), Italy	37°02′03.19″ N 15°18′54.41″ E
Lignano	Sand	430	248	101	34	Lignano Sabbiadoro (UD), Italy	45°41′18.36″ N 13°08′51.08″ E

.

Since there were no sensors at the specific sites, the nearest ones were used, and all corrections for tidal lag were applied: in the case of Santa Lucia, Capo Passero’s (SR) tide gauge was used, while Trieste’s tide gauge was used for Lignano. For Santa Lucia, the images of the impact of the cyclone Helios (8–11 February 2023) were analyzed (Figure 3a). In the case of Lignano, a severe storm surge that occurred between 21 November and 23 November 2022 was analyzed (Figure 3b).

2.3. Installation and Requirements

It is necessary to set up the entire script by creating an environment with the necessary and suitable requirements for its proper functioning. The open-source TensorFlow 2 library was used for model training [60]. For convenience and to ensure a better performance, we proceeded to install the system on the Google Collaboratory platform, which temporarily provides high-performance workstations [61].

In our case, the features provided by the Google Colab workstation were as follows:

• CPU: Intel Xeon 2.00 GHz (×2)
• GPU: NVIDIA Tesla T4 16 GB
• Driver Version: 525.85.12
• CUDA Version: 12.0
• RAM: 12.7 GB

2.4. Training Model Process and Testing

We opted for the Inception v3 model as our foundation, a convolutional neural network specifically designed for image analysis and object detection [35]. Originally developed as a module for GoogLeNet, it is the third iteration of Google’s Inception Convolutional Neural Network. Similar to how ImageNet serves as a classified visual object database [62], Inception assists in object classification within the realm of computer vision. The Inception v3 architecture has found widespread use across various applications [63], and is frequently employed in a “pre-trained” state from ImageNet. One notable application lies within the life sciences field, where it contributes to leukemia research [64,65,66].

The next step involves training our dataset to enable the system to recognize high tides based on previously organized classes. The images were automatically resized to a dimension of 224 × 224 pixels, and augmentation techniques, including rescale, horizontal and vertical flip, width and height shift, and zoom, were applied. The duration of the computational processes relies on several factors, chiefly the computational capacity. Given that the training of models encompasses CNN and image processing procedures, substantial resource utilization is expected, particularly involving the CPU, GPU, and RAM. We delegated the training to the GPU instead of the CPU, which has specialized functional units such as the ‘tensor core’ and parallel computing, thereby speeding up processing times [67]. The configuration of hyperparameters represents the second factor affecting both the time required and the accuracy achieved by the trained model [68]. Hyperparameters in a deep learning model are not learned directly through the training process; instead, they are selected manually by the user or through automated hyperparameter search techniques. These parameters play a pivotal role in controlling the model’s behavior during training and influencing its generalization capabilities. Examples of hyperparameters include the learning rate, the number of epochs, and the batch size.

Optimal hyperparameter selection is essential to ensure a deep learning model achieves strong generalization capabilities without succumbing to issues like overfitting or underfitting [69]. The hyperparameters used were chosen after repeated fine-tuning and they are as follows: batch Size = 8, Starting LR = 0.003, Epochs = 35.

Image analysis follows a step-by-step procedure incorporating three layers: convolutional layers, pooling layers, and fully connected layers. Two-dimensional convolutional layers are employed to process two-dimensional signals, like images. These layers apply a convolution kernel to the image, executing convolutions at each position between the kernel and the corresponding image segment. The kernel then shifts by a set number of pixels, referred to as the stride. It is important to consider the stride value, as a small stride can lead to redundant information. To control the output size, zero padding is introduced, which adds a border of zeros (of size l) around the image. The convolutional operations are complemented by an activation function known as the Rectified Linear Unit (ReLU), which is typically applied as an activation layer. This is a common activation function in deep learning neural networks that introduces non-linearity, facilitating the learning of complex models. The advantages of ReLU include its simplicity and computational efficiency, its ability to mitigate the vanishing gradient problem in deep neural networks, and the promotion of sparsity in the representation of outputs, which can be beneficial in certain contexts [70].

Moreover, to normalize each activation across different channels, a cross-channel normalization operation was employed. It improves model performance by ensuring a consistent scale of features, expediting convergence during training, enhancing stability, and acting as an implicit regularization. This process contributes to more efficient and robust model learning [71]. The CNN architecture also incorporates pooling layers, which serve to reduce the dimensionality of the input by subsampling, either through mean-pooling or max-pooling applied to patches of the image. Much like convolutional layers and pooling layers, these work on different parts of the image and incorporate a stride parameter. In the context of this specific deep learning network, 2D max-pooling layers were employed, extracting the maximum values from within the patches [72]. To handle inputs with the same height and width, depth concatenation layers were employed, which concatenate the inputs along the third dimension, representing the channels. The CNN concludes with a fully connected layer, establishing connections between each element of the preceding layer and every element within the softmax layer. The softmax layer, which estimates relative probabilities, is instrumental in determining the ultimate and most probable value. The structural framework of the deep neural network is crafted around Inception modules [33].

The Inception V3 model was trained using a deep learning technique called “transfer learning” [73]. This involves using a pre-trained deep learning model on a large dataset and updating the last layers of the model on a specific dataset of interest [74]. This approach allows the knowledge gained during the pre-training of the model to improve the model’s ability to generalize to new data [75]. These modules enable the network to select from various convolutional filter sizes within each block, enhancing its flexibility and adaptability.

The original architecture of Inception v3 was modified to tailor it to this application; the last layer was removed, and a Global Average 2D pooling layer was implemented to allow the CNN to better adapt to the specific case study. This is a special layer employed in Convolutional Neural Networks to compress the input tensor space. Unlike traditional pooling layers with fixed-size windows, Global Average Pooling calculates the average of all values in the feature map, generating a single mean value for each channel. This process reduces the spatial dimensions of the feature map while preserving crucial information about the features. The use of this layer is often considered to reduce the number of parameters in the model, prevent overfitting, and improve generalization. Additionally, it makes the network less sensitive to variations in the position and size of objects in images. Our modified architecture has 313 layers, of which 184 are trainable.

During the training process, the CNN was trained using 70% of the available images for training purposes and the remaining 30% for validation, which is considered by the literature as the optimal partition of the dataset for training these types of CNNs [35,76].

Figure 4 shows the training curves for the model in both case studies.

3. Results and Discussion

The CNN outputs deliver the probabilities corresponding to tide classes linked to a specific video frame captured by the webcams. In Figure 5a,b, the probabilities of tide classes related to a given image are illustrated. To depict tide phases across a continuous temporal spectrum, various snapshots were automatically extracted from the video recordings, with tide classes assigned to each image. The CNN achieved an accuracy exceeding 90%, and the Categorical Cross-Entropy Loss function yielded a value below 1 at the conclusion of the iterations. Detailed results can be found in

Table 2. Results from training.

Location	Accuracy	Loss
Santa Lucia	99.55%	0.25
Lignano	94.06%	0.88

. The evaluation of the CNN output metrics involved the examination of the confusion matrix (depicted in Figure 6a,b), which illustrates the relationship between predicted classes (CNN output) and true classes (spatial reference from field surveys). The confusion matrix stands as a commonly employed metric in the domain of classification problem-solving. Its versatility extends to both binary and multiclass classification scenarios. This matrix furnishes a tabulated representation of counts stemming from predicted and actual values [77].

As tide classes are mutually exclusive and encompass all potential tide phases, the most probable predicted class was chosen as a dependable representation of tide values. It is worth noting that during storms and medicanes, exceptional values may arise, wherein predicted classes may be linked to higher values than those typically associated with common tide phases.

In addition to considering the accuracy and loss metrics mentioned above, F1 scores in the various classes are presented below for a deeper understanding of the performance of the CNN model. The F1 score combines accuracy and recall through its harmonic mean and offers insight into the model’s effectiveness in handling class imbalances [78].

In both cases, the obtained weighted average is significantly higher, over 90%, indicating a good overall performance for all classes.

In particular, for the Santa Lucia site, the F1 scores show a weighted average of 0.9955. Most of the classes in this series achieve perfect scores (1.0). There is a slight decrease in the F1 score for class 2 (0.9709) and class 20 (0.9286). Although these scores are slightly lower than the results obtained by the other classes, they are still considered good and indicate the substantial ability of the model to correctly classify images belonging to the respective classes.

For the Lignano site, the weighted average is 0.9339. Specifically, classes 0 and 9 show lower F1 scores (0.5), suggesting difficulty in accurately predicting these particular tidal classes. This could be attributed to their limited representation in the training data. In contrast, many other classes, such as 2, 4, 5, and 6, achieve perfect scores (1.0), highlighting the excellent performance of the model within these tidal classes. Other classes, such as 1, 3, 24, 25, and 27, show moderate F1 scores ranging from (0.8 to 0.8571), indicating areas where improvements can be made.

Overall, the F1 scores for both sites demonstrate the effectiveness of the model in accurately classifying tidal heights into different categories, revealing areas of strength and scope for additional improvement.

The widespread use of deep learning models integrated with monitoring sensors, and they are becoming a low-cost tool for engineering and oceanographic studies [4,79,80,81]. On the other hand, using surveillance cameras allows for an increase in the density of data in coastal areas without monitoring stations. Remote sensing data combined with machine learning models have usually been applied to obtain physical features of the coastal areas, like water depth [82,83], storm surge [10,11,84], and hydrological parameters [85]. To date, few studies have focused on the assessment of meteo-marine parameters through deep learning and video monitoring [86,87,88]. Here, classification techniques using the Inception V3 model allowed us to obtain new observations of tide phases characterizing sandy and rocky coasts.

In summary, the outcomes of the study are influenced by factors such as image quality and landscape complexity. Convolutional neural networks (CNNs) have shown impressive effectiveness in classifying data across diverse applications, but it is important to note that using CNNs requires substantial computational resources. However, the advantages include the high precision in image classification, demonstrated by the elevated F1 scores at specific sites. The efficiency of CNNs is notable due to their parallel processing, making them suitable for large datasets. Additionally, CNNs automate image identification and classification, reducing the need for human intervention, and their customization feature allows for adaptation to specific application requirements through fine-tuning with representative image data.

However, from the evidence, it can be concluded that training the model is sufficient to implement and correlate it to the new environment. Training is necessary only once, after which it will be possible to perform classification of all the images that are inserted to recognize the tide height.

Using a temporal denomination for the images, it is possible to recreate the time-sheet of the extreme event to provide a better understanding of its temporal progression (Figure 7) and also for potential comparison with real data. Regarding Lignano, the images do not cover the entire 24 h period as the camera does not record during the night. One of the benefits of using systems like this is that they can reduce the workload of the operator, who has to manually analyze and view the images, and obtain almost instantaneous results. This approach could be relevant for the study of extreme events and for measuring hydrodynamic parameters in order to model the high frequency in the coastal dynamic. Similar approaches to the high-frequency dynamic are also applied to predict coastal changes, like shoreline movements [89], sea surface temperature changes [90], and tropical cyclone forecasting [91,92].

The field of application is diverse and could be of interest for those institutions that deal with the research and monitoring of beaches. Classification techniques could be integrated with segmentation techniques for beach monitoring [93,94,95] in order to obtain a reliable assessment of sedimentary balance.

4. Conclusions

In this study, we introduced a deep learning-based system for the automatic measurement and classification of tides in surveillance camera images. Deep learning techniques offer a cost-effective solution for coastal monitoring by increasing the availability of data that are sparsely distributed along coastlines. Furthermore, video footage serves as a valuable resource for gaining insights into high-energy occurrences, exemplified by the impact of Helios in southeastern Sicily in February 2023. Leveraging convolutional neural networks (CNN) in these scenarios enables the evaluation of hydrodynamic characteristics such as storm surge, which are challenging to assess in the field during actual events.

Accumulating a significant volume of data is essential for the development of prediction models that forecast the intensity of future extreme marine events. The application of deep learning in coastal monitoring facilitates the expansion of available datasets, thus proving valuable in this context.

The findings suggest that, even with a limited test dataset, the algorithm is capable of accurately recognizing the value of the tide or the relative surge during extreme events. The trained models achieved accuracy values exceeding 90% and a loss below 1. The generated confusion matrices also exhibited excellent results, with predicted values rarely deviating from the reference diagonal. Furthermore, by comparing the time-series of actual values with those predicted by the CNN for the two intense weather events, it is evident that the curves follow very similar patterns.

The precision of the outcomes relies on multiple factors, encompassing image quality and the intricacy of objects within the images. Overall, convolutional neural networks have proven highly effective in tasks involving classification and measurement across diverse applications. The incorporation of deep learning technology could yield even more advantages. Introducing such a system would offer the chance to receive instantaneous feedback on the consequences of atmospheric events in real-time. In addition, it would empower remote analysis, eliminating the necessity of in-person visits. This not only addresses logistical challenges but also facilitates the establishment of comprehensive databases of information. Utilizing systems like this could also alleviate the workload of operators who would otherwise need to manually analyze and review images, enabling almost instantaneous results.