Extremal-Aware Deep Numerical Reinforcement Learning Fusion for Marine Tidal Prediction

Abstract

In the context of global climate change and accelerated urbanization, coastal cities face severe threats from storm surges, and accurately predicting coastal water level changes during storm surges has become a core technological demand for disaster prevention and reduction. Storm surges are caused by atmospheric pressure and wind conditions, and their destructive power is closely related to the morphology of the coastline. Traditional tide level prediction models often face difficulties in boundary condition parameterization. Tide level changes result from the combined effect of various complex processes. In past prediction studies, harmonic analysis and numerical simulations have dominated, each with their own limitations. Although machine learning applications in tide prediction have garnered attention, issues such as data inconsistency or missing data still exist. The physical–data fusion approach aims to overcome the limitations of single methods but still faces some challenges. This paper proposes a Deep-Numerical-Reinforcement learning fusion prediction model (DNR), which adopts ensemble learning. First, deep learning models and the numerical model Finite-Volume Coastal Ocean Model (FVCOM) are used to predict tide levels at different tide stations, and then a fusion approach based on the improved reinforcement learning model DDPG_dual is applied for model assimilation. This reinforcement learning fusion model includes a module specifically designed to handle tide extreme points. In the case of the Typhoon Mangkhut storm surge, the DNR model achieved the best results for tide level predictions at six tide stations in the South China Sea.

1. Introduction

In recent decades, coastal cities have rapidly developed into thriving centers of population and economic activity, benefiting from convenient ports, rich tourism resources, and concentrated industries. However, with global climate change intensifying and urbanization accelerating, these coastal regions have become increasingly vulnerable to storm surges. When powerful weather events, such as typhoons or hurricanes, coincide with astronomical high tides, the resulting surge can cause extensive flooding, infrastructure damage, and heavy losses of life and property. History provides numerous examples of the destruction caused by such events. In 1953, a devastating flood submerged a vast area of land in the Netherlands, killing 1836 people [1]. Similarly, when Hurricane Katrina struck New Orleans in 2005, flooding severely breached levees, killing over 1400 people and inflicting economic damage exceeding USD 125 billion in 2005 dollars [2]. Super Typhoon Haiyan in 2013 left around 6000 dead in the Philippines, with economic losses reaching approximately USD 802 billion in 2013 dollars [3]. These tragic events highlight the critical importance of accurately forecasting coastal water levels during storm surges to improve disaster preparedness and response efforts.

As sea levels rise and storm intensity increases, coastal regions face heightened risks from storm surges. Reliable prediction of coastal water levels has thus emerged as a crucial requirement for disaster prevention and mitigation efforts. Accurate forecasts offer critical early warnings, providing essential time for local authorities and residents to undertake protective measures, including evacuations, reinforcement of coastal defenses, and relocation of vital resources. Therefore, enhancing research on storm surge forecasting technologies is essential to ensuring the safety and sustainable development of coastal cities.

Storm surges are primarily driven by atmospheric conditions, including wind intensity and surface pressure variations [2,4,5]. They are highly sensitive to storm characteristics such as intensity, trajectory, size, and speed. Additionally, the destructive potential of storm surges is closely linked to coastal morphology. Under typhoon or tropical cyclone conditions, seawater accumulates abnormally due to the combined influence of strong onshore winds and rapid atmospheric pressure drops. As storm surges approach the shoreline, irregular coastlines, complex underwater topographies, and man-made structures significantly alter wave propagation. In practice storm surge heights near irregular coastlines can be 20–40% [6] higher than those along open coasts. Shallow continental shelf regions also experience an intensified “funneling effect”, while infrastructure like breakwaters, sea-crossing bridges, and ports further complicate wave patterns through turbulence and reflection.

Typically, observed water levels consist of tidal (astronomical) and non-tidal (non-astronomical) components. The tidal component arises from gravitational interactions among the Sun, Earth, and Moon, while the non-tidal component encompasses the influences of meteorological and oceanographic factors [7]. Historically, tidal predictions relied mainly on harmonic analysis and numerical simulations. Numerical models effectively handle complex terrain but often face limitations in accurately defining boundary conditions [8]. Harmonic analysis methods require extensive parameters and struggle under irregular meteorological influences such as storm surges and wind-driven effects [9,10].

Recently, machine learning (ML) techniques have attracted attention in tidal prediction studies due to their capability to capture nonlinear relationships from large-scale oceanographic datasets. Neural networks and Support Vector Machines (SVMs) have demonstrated potential by learning patterns from historical tide level records. Studies indicate that ML approaches can match or even surpass traditional physical models in certain scenarios. Nonetheless, ML models still face limitations related to data inconsistencies and missing values. Particularly during extreme events not represented in historical data, ML models’ predictive accuracy significantly decreases. Marine observation data often originate from multiple sources, which can be buoys, satellites, and shore-based stations whose sensing footprints and acquisition cadences differ markedly. In practice, this means that measurements are taken at unequal sampling intervals and over spatial supports that range from point like records to broad areal swaths. When such heterogeneous streams are merged to form model input features, the resulting dataset unavoidably embodies unbalanced temporal resolution and non-uniform spatial coverage. Collaborating them on a common grid or timeline requires interpolation, resampling, and compositing steps that introduce spatiotemporal misalignment These effects are amplified during rapidly evolving processes, such as convection and tidal fronts. Under these conditions, even modest lags among multiple sources to systematic biases in the inputs supplied to learning algorithms, propagating errors in downstream representations and predictions. In addition, marine extreme events, such as typhoons and storm surges, frequently coincide with sensor failures and data transmission interruptions, producing gaps that are not random but tightly coupled to the very extremes of interest. This event-dependent missingness concentrates data loss at precisely the times and locations that characterize critical system states while leaving periods of normal conditions comparatively intact. As a consequence, the empirical distribution seen by the model underrepresents extreme regimes and overrepresents nominal regimes. Training on such imbalanced, non-randomly missing data prevents models from adequately learning the structure and amplitude of extreme states and instead encourages parameter estimates that minimize error on normal-condition samples. The practical outcome is a systematic attenuation of predicted extremes and a tendency to overfit routine variability, thereby degrading generalization precisely when robust performance is most needed [11].

To overcome these limitations, hybrid physical–data fusion approaches have emerged, combining traditional numerical methods with data-driven ML techniques [12,13,14]. These methods integrate machine learning models with numerical tidal simulations to enhance prediction accuracy, especially under extreme conditions. For instance, neural networks have been used to correct errors in hydrodynamic models like Hycom, leveraging combined knowledge to generate more reliable forecasts [15]. Despite their potential, these fusion approaches still encounter challenges due to data quality, model complexity, and limited training samples during extreme events. Consequently, their ability to predict outlier scenarios remains constrained.

Existing tidal prediction methods face significant challenges under extreme storm surge events. Harmonic analysis and statistical models struggle with irregular meteorological disturbances, numerical models are highly sensitive to boundary conditions, and machine learning models suffer from sharply reduced accuracy when encountering events not covered by historical data. To overcome these limitations, this study develops an innovative reinforcement learning-based framework, a multi-model collaborative prediction framework—the Deep-Numerical-Reinforcement Fusion Model (DNR). The DNR model employs ensemble learning by first using deep learning methods, and the numerical FVCOM model independently to forecast tidal levels at different stations. While deep learning methods excel in capturing periodic tidal patterns, their accuracy declines in predicting extreme tide levels [16]. Numerical models, on the other hand, have demonstrated robust predictive capabilities for extreme events. Subsequently, the outputs from both methods were assimilated through a reinforcement learning-based data fusion approach. Specifically, an improved reinforcement learning algorithm, DDPG_dual, was developed to enhance model integration. This variant of the Deep Deterministic Policy Gradient (DDPG) algorithm incorporates a dual-channel actor–critic structure and customized reward functions specifically designed to improve extreme tide predictions, surpassing conventional reinforcement learning approaches. The primary contributions of this work include the following:

• Developing a hybrid tidal prediction model DNR based on ensemble learning, combining the strengths of numerical methods and data-driven approaches to achieve higher accuracy.
• Introducing a novel reinforcement learning-based data assimilation framework for multi-model fusion, which adaptively integrates numerical models and machine learning predictions, thereby enhancing the robustness and accuracy of tidal and storm surge forecasting under extreme events.
• Based on the fundamental reinforcement learning framework, we have inventively introduced a dual-channel attention mechanism network. This architecture features one channel dedicated to processing regular tidal levels and another focused on extreme tidal levels. Compared to the baseline reinforcement learning model, our approach demonstrates significant performance improvement.

The rest of the paper is organized as follows, Section 2 introduces the related work, including marine tidal level prediction methods and tidal level prediction fusion methods. Section 3 presents the proposed methodology, starting with an overview of the DNR model that is followed by a detailed description of its individual sub-modules—particularly the integrated reinforcement learning module DDPG_dual. Section 4 describes the experiments conducted to validate the proposed model, using the storm surge caused by Typhoon Mangkhut in 2018 as a case study. Finally, Section 5 concludes the paper.

2. Related Work

2.1. Ocean Tidal Level Prediction

Ocean tidal level prediction plays a crucial role in oceanographic research and coastal engineering. Existing prediction methods can be broadly categorized into traditional approaches and modern data-driven techniques. Traditional tidal prediction methods primarily include harmonic analysis [17], empirical statistical models [18], and numerical simulation models [19]. More recently, machine learning and deep learning techniques have begun to emerge as powerful tools for tidal prediction.

Harmonic analysis decomposes tidal fluctuations into sinusoidal or cosinusoidal components with predefined astronomical periods. Using historical tidal observations, harmonic methods estimate amplitude and phase parameters based on theories such as tidal decomposition [20] and the harmonic analysis algorithm [21]. Empirical statistical models, including moving averages and Autoregressive Integrated Moving Average (ARIMA) models [22], construct prediction frameworks through statistical analysis of historical data. Numerical models rely on tidal hydrodynamic theories to simulate the propagation and evolution of tides by using advanced oceanographic simulation platforms such as FVCOM and ADCIRC. Despite their strengths, these methods have inherent limitations: harmonic analysis is effective for long-term periodic prediction but struggles with irregular and extreme events; empirical statistical models are straightforward and computationally efficient but fail to represent complex physical mechanisms; numerical simulations are effective in representing intricate coastal geometries but require significant computational resources and are highly sensitive to boundary conditions.

Machine learning approaches, such as Support Vector Machines (SVMs) [23], Random Forest, and K-Nearest Neighbors (KNN), have also been explored for tidal prediction. More sophisticated deep learning models, including Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, Recurrent Neural Networks (RNNs), and Temporal Convolutional Networks (TCNs), have recently emerged, demonstrating superior capability in capturing nonlinear tidal dynamics. In the late 1990s, artificial neural networks were applied to the prediction of ocean tidal levels, and the backpropagation network was successfully utilized to predict the tidal level variations in Taichung Harbor [24,25]. In [26], the author demonstrated that artificial neural networks could achieve prediction accuracy comparable to traditional methods with reduced data requirements. However, these early neural network models primarily depended on harmonic parameters and were thus limited in predicting non-astronomical tide variations. A subsequent study [27] employed the Radial Basis Function (RBF) and Generalized Regression (GR) neural networks to predict tidal levels in a shallow coastal region of Portugal, achieving correlation coefficients of 99% for 6-h forecasts but only 83% for 24-h predictions. Additional studies [28,29] further validated the effectiveness of neural networks but typically employed simpler feedforward structures, limiting their performance for short-term hourly forecasting and leaving room for improvement. More recently, a study [30] employed a Bidirectional LSTM (Bi-LSTM) neural network to effectively generate long-term tidal predictions, successfully validating its performance across multiple coastal stations in the U.S.

The development of machine learning methods in tidal prediction has demonstrated a significant evolution from shallow models to deep architectures and from reliance on astronomical parameters to autonomous feature extraction. Early research primarily focused on traditional machine learning algorithms such as Support Vector Machines and Random Forests, which initially showed potential but were limited by their linear modeling capabilities. Since the 1990s, artificial neural networks (ANNs) began to be applied, reducing data requirements yet still struggling to effectively predict non-astronomical factors influencing tidal variations.

In recent years, with advancements in computer hardware performance, deep learning has gained prominence. Models such as RNNs and LSTMs, through hierarchical feature extraction and temporal modeling, have significantly enhanced the capability to capture nonlinear tidal dynamics. Currently, future trends are expected to focus on hybrid modeling that integrates physical mechanisms with data-driven approaches, as will be discussed in the subsequent section.

2.2. Tidal Level Prediction Fusion Methods

Given the complexity of hydrodynamic processes, single-model approaches often struggle to achieve consistently accurate tidal predictions. Consequently, hybrid or fusion models have emerged as a research focus, aiming to integrate the strengths of various prediction techniques to enhance forecast accuracy. Numerous studies have combined neural networks with other methods to form hybrid models. For instance, harmonic analysis and Kalman filters have been integrated with neural networks to improve prediction reliability [12]. Subsequently, a hybrid forecasting method combining harmonic analysis and neural networks was proposed [13], demonstrating improved predictive capability at multiple U.S. coastal tidal stations. Similarly, wavelet-based neural networks employing dynamic nonlinear activation functions were developed [14], significantly outperforming traditional neural networks. Further research combined harmonic analysis with wavelet networks, achieving approximately 20% improvement over standalone wavelet networks [31]. However, few existing studies have explored combining multiple model outputs using reinforcement learning for model fusion.

Ensemble learning, a widely adopted technique in machine learning, aggregates multiple models to achieve superior prediction accuracy and robustness compared to single-model approaches. In recent years, ensemble methods have gained popularity in tidal prediction due to their ability to effectively handle complex and variable data scenarios. The fundamental principle underlying ensemble learning is to combine multiple predictive models—each possibly trained with different algorithms or parameter settings—to leverage their complementary strengths and mitigate their individual limitations [32]. Ensemble learning has been successfully applied across various domains, including image recognition, natural language processing, and financial forecasting.

Common ensemble learning methods include bagging, boosting, and stacking [33]. Bagging (Bootstrap Aggregating) involves creating multiple training subsets via bootstrap sampling, training separate models on each subset, and then aggregating their predictions to reduce variance and improve generalization [34]. Boosting sequentially trains weak predictive models, combining their outputs to form a strong predictor that systematically reduces bias and enhances accuracy [35]. Stacking employs a two-layer framework, where multiple base models—either homogeneous or heterogeneous—generate initial predictions, which are subsequently combined by a meta-learner to produce the final forecast [33]. During training, predictions from base models are used as inputs, paired with observed data, to train the meta-learner. At the inference stage, the outputs of base models serve as inputs for the meta-learner, which then generates the ultimate prediction.

Despite these advancements, applying ensemble learning to tidal prediction, particularly for extreme storm surge scenarios, remains relatively unexplored. In this study, we propose a novel fusion model that integrates deep learning and numerical modeling outputs through reinforcement learning to improve the accuracy and robustness of tidal level predictions during storm surge events.

3. Methodology

The architectural design of the DNR model is schematically presented in Figure 1. This integrated framework comprises three distinct modules operating in synergy. The foundational layer consists of two specialized base models: a Temporal Convolutional Network (TCN) and the FVCOM hydrodynamic model. These models independently generate tidal predictions for identical spatiotemporal coordinates, with their outputs and corresponding ground truth values aggregated to construct a training dataset for the meta-learning DDPG_dual module. The operational workflow proceeds as follows: Initially, TCN processes temporal–contextual features to derive tidal forecasts, while FVCOM leverages hydrodynamic principles for complementary predictions. Subsequently, the DDPG_dual meta-learner dynamically fuses these heterogeneous outputs to produce the final unified prediction. The technical details of each component are elaborated below.

3.1. FVCOM Hydrodynamic Model

FVCOM is a three-dimensional, unstructured-grid, prognostic modeling system designed for free-surface flow simulations [36]. Its computational efficiency and geometric flexibility enable precise representation of complex coastal and marine environments, including irregular bathymetries, islands, and engineered structures. The model’s governing equations incorporate fundamental hydrodynamic principles, solving coupled systems for momentum, mass continuity, temperature, salinity, and density fields. The core formulation is governed by the following equation [37]:

$$\partial U={\displaystyle \frac{\partial u}{\partial t}}+u{\displaystyle \frac{\partial u}{\partial x}}+v{\displaystyle \frac{\partial u}{\partial y}}+w{\displaystyle \frac{\partial u}{\partial z}}$$

(1)

$$\partial V={\displaystyle \frac{\partial v}{\partial t}}+u{\displaystyle \frac{\partial v}{\partial x}}+v{\displaystyle \frac{\partial v}{\partial y}}+w{\displaystyle \frac{\partial v}{\partial z}}$$

(2)

$$\partial W={\displaystyle \frac{\partial w}{\partial t}}+u{\displaystyle \frac{\partial w}{\partial x}}+v{\displaystyle \frac{\partial w}{\partial y}}+w{\displaystyle \frac{\partial w}{\partial z}}$$

(3)

$$\partial U=-{\displaystyle \frac{1}{\rho }}{\displaystyle \frac{\partial \left(P_{ah}+P_{nh}\right)}{\partial x}}+{\displaystyle \frac{\partial }{\partial z}}\left(K_m{\displaystyle \frac{\partial u}{\partial z}}\right)+M_u+cv$$

(4)

$$\partial V=-{\displaystyle \frac{1}{\rho }}{\displaystyle \frac{\partial \left(P_{ah}+P_{nh}\right)}{\partial y}}+{\displaystyle \frac{\partial }{\partial z}}\left(K_m{\displaystyle \frac{\partial v}{\partial z}}\right)+M_v-cu$$

(5)

$$\partial W=-{\displaystyle \frac{1}{\rho }}{\displaystyle \frac{\partial P_{nh}}{\partial y}}+{\displaystyle \frac{\partial }{\partial z}}\left(K_m{\displaystyle \frac{\partial w}{\partial z}}\right)+M_w$$

(6)

where x, y, z, u, v, and w represent the three coordinate axes in the Cartesian coordinate system and their corresponding velocity components, respectively. $\rho$ denotes density; $P_{ah}$ represents hydrostatic pressure; $P_{nh}$ denotes non-hydrostatic pressure; c is the Coriolis parameter. $K_m$ is the vertical eddy diffusion coefficient. $M_u$, $M_v$, and $M_w$ are the momentum components in the three directions. This dynamic process is constructed on an unstructured grid to simulate wave propagation. To achieve this, a refined grid was established for the simulated water area, as shown in Figure 2 [38].

3.2. Temporal Convolutional Network

Temporal Convolutional Network (TCN) is a deep learning model specifically designed for processing sequential data [39], combining the parallel computing advantages of Convolutional Neural Networks (CNNs) with the long-term dependency modeling capabilities of Recurrent Neural Networks (RNNs). The structure of TCN is shown in Figure 3.

The entire TCN is composed of 4 stacked structures described above, with the output of each TemporalBlock layer serving as the input to the next TemporalBlock layer. The core TemporalBlock implementation in TCN is as follows: TCN employs causal convolution to ensure the model only relies on current and past time step data when predicting the current time step, without using future information. By applying padding operations to the convolution kernel, it guarantees that the t-th time step in the output tidal level prediction sequence only convolves with the t-th and prior time steps in the input tidal level sequence. This preserves the causality of time series prediction and prevents future information from leaking into the current prediction moment. This process can be expressed as follows:

$${hidden}_r=Dc\left(Wn\left(pr{e}_{t-k+1},pr{e}_{t-k+2},\dots ,pr{e}_t\right)\right)$$

(7)

where $W_n\left(\right)$ performs weight normalization to avoid vanishing and exploding gradients, enhancing model generalization. $Dc\left(\right)$ implements causal convolution, and $hidde{n}_r$ represents the corresponding hidden layer representation. To maintain consistent input–output lengths, TCN uses Chomp1d to trim redundant padded portions from convolution results. To capture long-range dependencies, TCN employs dilated convolution [39], which inserts dilation between kernel elements to exponentially expand the receptive field without increasing parameter count, enabling the model to capture long-distance dependencies. The formula for dilated convolutions is expressed as follows [39]:

$${hidden}_{out}=\sum _{i=0}^{k-1}{w}_i·{\left({hidden}_{in}\right)}_{t-d·i}$$

(8)

where $w_i$ represents the convolutional weights, and d is the dilation rate, which determines how many steps between inputs are used in the convolution. When $d=1$, the convolution involves consecutive inputs. $hidde{n}_{out}$ refers to the output of the hidden layer, and ${\left({hidden}_{in}\right)}_{t-d·i}$ represents the input sequence from the previous layer, where the input is gathered according to the dilation rate d. Finally, a residual connection is used to directly add the input to the output of the convolutional layer. This helps alleviate the vanishing gradient problem, accelerates training, and, through the activation function, produces the output of the TemporalBlock. This process is expressed as follows:

$${predict}_t=ReLU\left(Ds({pre}_{t-k+1},\dots ,{pre}_t)\right)+TB$$

(9)

where $predic{t}_t$ is the output predicting the tidal level at time $t+1$, and ${pre}_{t-k+1},\dots ,{pre}_t$ are the tidal input sequence of length k, consisting of the tidal levels at time t and earlier. $Ds\left(\right)$ is a downsample layer, which is essentially a 1D convolutional layer that is used only when the input and output dimensions of the hidden layer are not equal. The purpose is to ensure dimensionality matching when performing residual connections. $TB$ stands for Temporal Block, which can be specifically described by the following equation:

$$TB=TemporalBlock({pre}_{t-k+1},{pre}_{t-k+2},\dots ,{pre}_t)$$

(10)

where ${pre}_{t-k+1},\dots ,{pre}_t$ are the tidal input sequence of length k.

It is important to note that, in this study, $predic{t}_t$ actually predicts the tidal level at time $t+1$. Tidal level prediction can be viewed as an autoregressive prediction, where the target tidal level prediction output is defined as the result of shifting the input tidal levels one time step forward [39].

3.3. DDPG_Dual Moudle

Deep Deterministic Policy Gradient (DDPG) is a deep reinforcement learning method that provides an intelligent solution for combining predictions from base models. Specifically, DDPG is an algorithm used for reinforcement learning in continuous action spaces [40]. It combines deep neural networks with deterministic policy gradient methods, effectively addressing decision-making problems in high-dimensional states and continuous action spaces. The core idea is to enable efficient learning of agents in complex environments through the collaborative optimization of the policy network and value network. The structure of the DDPG algorithm consists of four key components: the current policy network (Actor), the target policy network (Target Actor), the current value network (Critic), and the target value network (Target Critic). The Actor network outputs deterministic actions based on the current state, while the Critic network evaluates the value of these actions. The target networks provide stable training objectives through soft updates, reducing oscillations during the policy update process. The experience replay mechanism improves training efficiency by randomly sampling from historical experiences, breaking data correlations.

The DDPG_dual structure used in this study is shown in Figure 4. The online network and the target network are constructed separately, with both networks having identical structures. The online network is used for real-time decision making, while the target network is used for training and learning. The Actor network and Critic network are deployed on both the online and target networks. The Actor network is responsible for calculating actions based on the environmental state. The design of the Critic network is intended to assist in evaluating actions, as the environment cannot always provide concise feedback for each action step. In DDPG_dual, the action $a\left(t\right)$ is formed through a dual-channel network and an attention gate control network. The dual-channel network consists of the mainstream network and the extreme flow network. Note that, the dashed red region are the two parallel branches which are a main stream and an extreme stream. The mainstream network handles regular tidal level changes, while the extreme flow network specializes in processing extreme regions such as wave peaks and troughs. The outputs of these two networks are dynamically fused through the attention gate control network. The Actor network receives the current environmental state $S\left(t\right)$ and outputs the final tidal level prediction, $Prediction$. The score output from the Critic network, $CombineValue$, comprehensively considers the influences of the current state $S\left(t\right)$ and action $a\left(t\right)$. This is also formed through the dual-channel network and the attention gate control network. The action $a\left(t\right)$ is fed back into the environment to obtain the reward information $r\left(t\right)$ and transition to the next state $S(t+1)$. The data generated through this process are stored in the experience replay buffer for parameter updates across the entire DDPG_dual.

The loss function of the Critic network in DDPG_dual consists of the general mean squared error and the extreme point mean squared error, which are defined as follows:

$${critic}_{loss}=MSE\left({current}_Q,{target}_Q\right)+2·{extreme}_{loss}$$

(11)

where $curren{t}_Q$ represents the current Critic network’s Q-value estimate for the current state–action pair, which is the calculated immediate reward and the estimated future reward. $targe{t}_Q$ is the target Critic network’s Q-value estimate for the next state–action pair. $extrem{e}_{loss}$ is the loss for extreme points, designed to enable the network to better combine the base model’s predictions for tidal extreme points. $extrem{e}_{loss}$ is defined as follows:

$${extreme}_{loss}=MSE\left({current}_Q·mask,{target}_Q·mask\right)$$

(12)

where $mask$ is the extreme point mask, which is set to 1 when an extreme point is predicted, and 0 otherwise. The loss function for the Actor network corresponding to $criti{c}_{loss}$ is then defined as follows:

$${actor}_{loss}=-Mean\left(critic\left(s,Actor\left(s\right)\right)\right)$$

(13)

where s represents all states obtained from the experience replay buffer, and $Actor\left(s\right)$ denotes the action taken for the corresponding state $Mean\left(\right)$ is the mean function. The calculation of $Targe{t}_Q$ is performed using the following formula:

$${target}_Q=reward+1·(1-done)·\lambda ·{nextTarget}_Q$$

(14)

where $Reward$ is the current reward, $done$ indicates whether it is a terminal state, $\lambda$ is the discount factor, and the final term $nextTarge{t}_Q$ represents the target network’s Q-value estimate for the next state and action.

The overall workflow of DDPG_dual is as follows: it receives tidal level prediction value sequences from two base models, FVCOM, and TCN and uses them as input for the environmental state. DDPG_dual then focuses on learning how to effectively combine these predictions and ultimately outputs the optimized tidal level prediction results.

4. Experiments

The super typhoon “Mangkhut” in 2018 was a notable storm of the Pacific typhoon season, recognized for its intense strength, broad impact, and devastating power, resulting in severe disasters across multiple regions [41]. This experiment used this case as the experimental object. Observation data from six tidal stations in the South China Sea were selected for study. The specific study area is shown in Figure 5.

Data Sources. This study selected a total of six tidal stations in the South China Sea region. The names of each station and their corresponding longitude and latitude coordinates are shown in

Table 1. Stations’ longitude and latitude coordinates.

Tidal Station	Longitude	Latitude
BHI	109.030° E	21.280° N
DFG	108.375° E	19.064° N
DWS	113.430° E	21.560° N
QLN	110.493° E	19.338° N
SWI	115.220° E	22.460° N
WZU	109.070° E	21.210° N

.

The data used in this study come from the water level observation records of six tidal stations in the South China Sea, covering the period from January to September 2018 (a total of 9 months). The data from each tidal station for the period from January to August 2018 were used for training the neural network, while the data from September were reserved for validation. All tidal stations recorded water levels every hour, resulting in 24 recordings per day. During the data processing phase, we pre-processed the data from the six tidal stations, disgrading some missing values, and then fed the processed data into the neural network for training. After training the neural network, we used it to predict the water levels for 312 h starting from 00:00 on 3 September 2018. At the same time, the FVCOM model performed the same prediction task, generating corresponding prediction results for the six tidal stations. Among the six sets of prediction results obtained, we selected the prediction data from five tidal stations and divided them into a training set and a test set in an approximately 80:20 ratio for use in the reinforcement learning fusion module. The remaining 312 h of prediction data from one tidal station were used entirely to validate the generalization ability of the DNR model.

Evaluation Metrics. To evaluate the performance of our proposed model, we adopted the Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Pearson correlation coefficient, and Nash–Sutcliffe Efficiency (NSE), which are commonly used metrics for tidal level prediction tasks. Additionally, we conducted a detailed comparison with other models.

4.1. Experimental Setup

This study was equipped with 32 GB of RAM and an NVIDIA RTX 4070Ti GPU with 12 GB of VRAM for hardware, as well as an Intel 14th-generation processor for the CPU. On the software side, VSCode was used as the development environment. For more detailed setup information, please refer to

Table 2. Experimental operating environment.

Name	Type/Version
CPU	Intel(R) Core(TM) i7-14700 KF
GPU	NVIDIA GeForce RTX 4070Ti (12 GB)
Memory	32 GB
IDE	VSCode 1.96.2
Cuda	11.8
Python	3.7.16

.

The key settings of the FVCOM model’s nml configuration file are shown in

Table 3. FVCOM configuration.

Item	Configuration Value
START_DATE	2018-9-01 00:00:00
END_DATE	2018-9-15 23:00:00
STARTUP_TYPE	Coldstart
NC_FIRST_OUT	2018-9-01 00:00:00
NC_OUT_INTERVAL	3600
WIND_ON	F
EXTSTEP_SECONDS	3.00
MIN_DEPTH	0.05000000

. Considering that the FVCOM simulation has an initial cold start period, the simulation was run in advance for a period of time, and the first 48 h of the prediction results were discarded. That is, the actual prediction started from 00:00 on 3 September 2018, with prediction results output every hour. In Table 3, START_DATE represents the start time of the FVCOM prediction, and END_DATE denotes the end time of the prediction. STARTUP_TYPE is used to specify the startup mode of the model. NC_FIRST_OUT refers to the start time for outputting prediction results in the NetCDF file, while NC_OUT_INTERVAL defines the time interval for outputting prediction values in the NetCDF file. As for WIND_ON, it indicates whether the wind field file is enabled, with “F” meaning that the wind field file is not enabled.

As shown in Table 3, the model employed a time step of 3 s. The grid design incorporated a minimum resolution of approximately 500 m, with simulated current velocities not exceeding 4 m/s in the model outputs. The CFL condition analysis confirms that values remained substantially below 1. Furthermore, the wet/dry treatment threshold was set to 0.05 m following FVCOM manual recommendations, significantly enhancing the model’s stability in nearshore simulations. Referencing the grid resolution settings from existing FVCOM studies [42], our grid generation followed these principles: coastal zones, estuaries, and bays characterized by high gradients were refined to 500 m, while open sea areas gradually transitioned to 30 km. The geometric fidelity of shorelines and isobaths was maintained using high-resolution topographic and shoreline mapping data provided by RESDC.

The general hyperparameter settings for the TCN deep learning model are shown in

Table 4. TCN hyperparameter setting.

Item	Configuration Value
batch_size	400
epochs	400
lr	0.001
gamma	0.9
device	cuda
dilation	2
n_inputs	48

. The hyperparameters were configured based on the TCN paper and experimental observations from this study. The number of training epochs was set to 400, because the loss curve indicated convergence after approximately 300 epochs, and models trained with 400 epochs exhibited comparable performance to those trained with 500 epochs.

The common hyperparameters for the other deep learning models used for comparison can be set according to the settings in the table above. For all deep learning models, we selected water level data from January to August 2018 as the training set. The model input was uniformly set as the 48 h of observed data prior to the prediction, while the output focused on predicting the water level for the next hour. Based on this setup, approximately 5500 data points were generated for each station to train the model. After training the model, we used a rolling prediction approach for water level forecasting. Specifically, the first 48 h of data before 00:00 on 1st September were used to predict the water level for the next hour. Then, this prediction result was combined with the previous 47 h of actual data to form a new input for predicting the water level for the second hour. This process continued until 48 h of water level predictions were completed. Next, the 48 h observed water level data from 1st to 2nd September were used to predict the water levels from 3rd to 4th September, and so on, until predictions were made through 23:00 on 15th September, totaling 360 h of predictions. After comparing the prediction performance of the four deep learning models and the ARIMA model, we selected the best-performing model. We then used approximately 80% of the predicted data from five tidal stations for training the fusion model. Finally, we tested the fusion model using the remaining 48 h of prediction data from these five stations, as well as the full prediction data from another station that was not involved in fusion model training.

The hyperparameter settings for the fusion model, DDPG_dual, are shown in

Table 5. DDPG_dual hyperparameter setting.

Item	Configuration Value
state_dim	24
action_dim	2
lr_actor	0.0001
lr_critic	0.001
replay_buffer	10000
tau	0.01
gamma	0.99

. The state dimension is 24, which means the input to the fusion network consists of 24 data points, combining 12-h predicted water levels from both FVCOM and TCN. The fusion network is responsible for outputting two weights, which are used to perform a weighted fusion of the final set of predictions from the FVCOM and TCN models, resulting in the final prediction from the fusion model.

4.2. Experimental Result

This section provides a detailed analysis of the experimental results, focusing primarily on the four metrics mentioned earlier. Comparative analysis of similar models was conducted in both the deep learning prediction model phase and the fusion model phase, with the aim of comprehensively evaluating the performance and effectiveness of the proposed fusion model.

Firstly, we used the FVCOM model to predict the tidal levels at six tidal stations. Since FVCOM starts simulations in cold-start mode, to ensure the accuracy of the prediction results, we discarded the first 48 h of data after the simulation began and selected the subsequent 312 h of data as the FVCOM prediction results. Among these, for the five stations used for fusion model training, we specifically selected the last 48 h of data, which were not involved in the fusion model training, for display (the previous 250 h of prediction data were used for fusion model training). For station BHI, which was not involved in fusion model training, we displayed its complete 312-h prediction results. In Figure 6, the predicted results for each station are shown by green lines, while the corresponding actual tidal level data is presented by red lines.

(1) Quantitative Comparative Analysis of Models: This section focuses on the performance comparison between two categories of models: first, the comparison between deep learning models and the ARIMA model, and second, the performance comparison among different fusion models.

First, we compared the performance between the deep learning models and the ARIMA model. The deep learning models included TCN (Temporal Convolutional Network), RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory Network), and VIT (Vision Transformer). We used these models to predict the tidal levels at six tidal stations, with the performance metrics of the models for each station shown in

Table 6. Performance comparison between deep learning models and the ARIMA model.

Station	Metrics	TCN	RNN	LSTM	VIT	ARIMA
BHI	MAE	20.477	37.292	22.729	59.721	48.279
	RMSE	26.170	44.188	30.215	85.630	61.482
	Pearson	0.967	0.902	0.951	0.282	0.647
	NSE	0.864	0.613	0.919	−0.452	0.252
DFG	MAE	12.400	27.388	13.615	31.694	75.456
	RMSE	13.772	33.014	16.257	38.692	83.463
	Pearson	0.988	0.671	0.963	0.386	−0.536
	NSE	0.870	0.253	0.819	−0.026	−3.775
DWS	MAE	16.123	22.175	16.946	18.131	37.963
	RMSE	19.597	27.984	22.229	21.785	45.458
	Pearson	0.925	0.670	0.844	0.830	0.784
	NSE	0.675	0.338	0.582	0.599	−0.748
QLN	MAE	10.175	18.086	11.829	11.331	32.935
	RMSE	11.727	22.330	14.970	13.718	38.225
	Pearson	0.921	0.910	0.844	0.908	0.586
	NSE	0.779	0.198	0.640	0.697	−1.351
SWI	MAE	15.548	20.746	21.052	21.783	40.048
	RMSE	18.935	24.544	27.810	26.488	47.908
	Pearson	0.795	0.800	0.662	0.755	0.511
	NSE	0.499	0.159	−0.080	0.020	−2.205
WZU	MAE	22.663	57.744	29.235	37.038	108.677
	RMSE	24.385	73.654	40.648	42.533	125.184
	Pearson	0.990	0.354	0.891	0.885	−0.255
	NSE	0.853	−0.338	0.593	0.554	−2.865

. For each station, the numbers marked in red represent the minimum value for the corresponding model on that metric (note that for MAE, which is Mean Absolute Error, and RMSE, which is Root Mean Square Error, smaller values are better). The numbers marked in blue represent the maximum value for the corresponding model on that metric (where Pearson correlation coefficient and NSE, the Nash–Sutcliffe Efficiency, are better when larger).

Overall, the TCN model demonstrated the best performance in this study, achieving the smallest Mean Absolute Error (MAE) at most stations while also attaining the highest correlation coefficient and Nash–Sutcliffe Efficiency (NSE). This outstanding performance is primarily attributed to the unique architecture of the TCN: by stacking convolutional layers with different dilation rates, the TCN can effectively capture patterns at various scales and long-term dependencies within sequential data. This structural feature makes the TCN particularly effective in modeling long-term temporal correlations. Additionally, the convolutional operations used in TCN are shift-invariant, meaning that regardless of the time position of the input sequence, the learned features remain valid. This characteristic further enhances the robustness of the TCN model, enabling it to better adapt to temporal shifts and dynamic changes in sequential data, thereby significantly improving prediction accuracy. The LSTM model followed closely, exhibiting second-best performance, while the traditional RNN model ranked behind the LSTM. This outcome is expected, as LSTM, an improved version of RNN, has a distinct advantage in handling long-term dependencies in sequential data. In contrast, the VIT model performed poorly in this tidal level forecasting task. Originally designed for image recognition, the VIT model’s core idea is to divide an image into multiple patches and arrange them into a sequence, similar to the processing of natural language. While some adjustments can be made to the network, the VIT model still struggled to achieve satisfactory prediction results for sequential data like tidal level forecasting. The ARIMA model performed the worst in this study. Across all data metrics, the TCN model, with its excellent performance and robustness, emerged as the preferred choice for the deep learning prediction module.

After obtaining the prediction data from FVCOM and the deep learning model TCN, we conducted a series of tests and comparisons on five reinforcement learning fusion models: DDPG_dual, DDPG, PPO, DQN, and DDQN. Specifically, we selected approximately 80% of the prediction data from five stations for training the fusion models and used the last 48 h of prediction data for fusion testing. Additionally, for the station that did not participate in the fusion model training (BHI), we used 312 h of prediction data for fusion testing. The results are presented in

Table 7. Comparison of fusion models, FVCOM, and TCN.

Tidal Station	Metrics	TCN	FVCOM	DDPG Dual	DDPG	DQN	DDQN	PPO
BHI	MAE	16.269	73.340	13.020	21.279	27.131	20.573	227.365
	RMSE	21.198	82.871	15.835	26.520	33.827	26.652	242.178
	Pearson	0.983	0.664	0.990	0.980	0.965	0.972	0.726
	NSE	0.691	0.410	0.979	0.940	0.901	0.939	−4.040
DFG	MAE	12.400	33.760	5.813	6.135	9.956	7.325	64.842
	RMSE	13.772	42.078	6.995	7.387	11.970	9.112	71.449
	Pearson	0.988	0.534	0.987	0.985	0.957	0.974	0.761
	NSE	0.870	−0.214	0.967	0.963	0.902	0.943	−2.499
DWS	MAE	16.123	43.223	11.546	12.060	15.593	14.240	71.875
	RMSE	19.597	49.453	14.326	14.820	21.456	18.871	76.265
	Pearson	0.925	0.137	0.937	0.905	0.801	0.853	0.729
	NSE	0.675	−1.069	0.826	0.814	0.611	0.699	−3.921
QLN	MAE	10.175	29.998	7.405	9.665	13.735	9.281	52.292
	RMSE	11.727	35.235	8.905	11.136	16.160	10.935	55.294
	Pearson	0.921	0.050	0.948	0.896	0.797	0.905	0.749
	NSE	0.779	−0.997	0.872	0.801	0.580	0.808	−3.919
SWI	MAE	15.548	31.146	13.045	13.413	15.723	14.333	67.429
	RMSE	18.935	38.267	17.241	16.085	18.607	17.537	70.648
	Pearson	0.795	0.103	0.819	0.810	0.758	0.788	0.641
	NSE	0.499	−1.045	0.585	0.639	0.517	0.571	−5.969
WZU	MAE	22.663	62.344	10.627	11.158	18.067	14.204	124.842
	RMSE	24.385	71.735	12.623	13.569	21.754	18.411	134.848
	Pearson	0.990	0.458	0.994	0.896	0.940	0.959	0.698
	NSE	0.853	−0.269	0.961	0.955	0.883	0.916	−3.485

. For comparison purposes, the performance metrics of the FVCOM and TCN models are also listed in the table. As in Table 6, the numbers marked in red indicate the minimum value achieved by the corresponding model on that metric, while the numbers marked in blue indicate the maximum value achieved.

Overall, the DDPG_dual model demonstrated the best predictive performance across all tide stations, outperforming both the TCN model and the DDPG model. This can primarily be attributed to the addition of a specialized module for handling extremes in the actor and critic networks of the DDPG_dual model compared with the DDPG model. This module is more effective in predicting wave peaks and troughs, leading to predictions that are closer to the real data. This will be further validated in the subsequent ablation experiments.

The DDPG model showed strong performance at the five training stations, second only to the DDPG_dual model. However, at the BHI station, its Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) still lagged behind the TCN model. Notably, the DDPG_dual model also achieved the best results when conducting fusion predictions at the unseen BHI station. This suggests that the generalization ability of the DDPG_dual model is stronger compared with the DDPG model. It is worth emphasizing that, within the spatiotemporal scope influenced by Typhoon Mangkhut, the proposed ensemble learning DNR model effectively integrated the prediction sequences from two base models and achieved accurate tidal level forecasting. This also further validates robustness of the proposed method under extreme weather conditions.

A comparison with previously mentioned related studies reveals that our proposed DNR model achieved a correlation coefficient above 0.9 even over a 48-h prediction horizon, outperforming the correlation coefficient reported in that study [27] for 24-h tidal predictions. Compared with the results of this study [14], the correlation coefficient of our model’s 48-h tidal prediction (with an average of 0.9403 across six stations) remains close to that of the 50-h tidal prediction curve (average 0.9511) reported in their study under the influence of Typhoon Mangkhut. Overall, the performance of our proposed model can be considered satisfactory.

(2) Qualitative Comparison: We also plotted the prediction curves of each model. Figure 7 shows the prediction curves of the deep learning models compared against the ARIMA model, while Figure 8 presents the prediction curves of the various fusion models. In Figure 8, for each subplot, the red solid line represents the real tide level curve, and the black solid line represents the prediction curve of the DDPG_dual model. From the figure, it is clear that the DDPG_dual model generally fits the real curve quite well. In most cases, compared to the DDPG model, it is able to more accurately capture the peaks and troughs of the real curve, demonstrating superior predictive performance. However, there are certain time points where the DDPG_dual model’s predictions still show some shortcomings. For example, around the 10th hour at the SWI station and around the 13th h at the DWS station, there are significant discrepancies between the predicted and actual tide levels. The DQN and DDQN models, due to their discrete action selection, have certain limitations in their fusion performance compared to the DDPG model. The PPO fusion model, being the most complex of the models tested, not only requires longer training times but also has a slower convergence rate. Although some patterns in the tide level changes can be observed in its predictions, the prediction errors are large, making it difficult to meet the precision requirements for practical applications.

(3) Ablation Study: To further enhance the performance of the fusion model, we made two key improvements to the original DDPG model. On one hand, we introduced a dual-channel mechanism to strengthen the model’s ability to process input information, thereby improving the fusion effect. On the other hand, we optimized the original reward mechanism to encourage the model to focus more on accurately handling extreme values during training.

Specifically, we named the original DDPG model as DDPG_O_1. After introducing the dual-channel mechanism, the model was named DDPG_dual. Further adjustments to the reward mechanism based on DDPG_dual resulted in a model named DDPG_d_e. To validate the effectiveness of the newly added structures, we tested the above models using the same dataset. The average absolute errors (MAE) for each model are shown in

Table 8. Model prediction error.

Station	DDPG_d_e	DDPG_d_e	DDPG_Dual	DDPG_O_1
	(256)	(128)
DFG	5.818	6.683	7.589	6.135
DWS	11.552	13.334	13.260	12.060
QLN	7.406	7.703	8.209	9.665
SWI	13.066	11.783	12.885	13.413
WZU	10.626	11.468	15.315	11.158
BHI	13.020	16.396	12.485	21.279
average	10.248	11.228	11.624	12.285

. Here, DDPG_d_e (128) and DDPG_d_e (256) represent models with hidden layer neuron counts of 128 and 256, respectively. The error units are unified in centimeters (cm).

As seen in Table 8, the basic fusion model DDPG had an average error of 12.285 cm. After incorporating the dual-channel attention mechanism, the average error dropped to 11.624 cm. Further improving the reward mechanism reduced the average error to 11.228 cm. When the hidden layer was adjusted to 256 neurons, the average error decreased even further to 10.248 cm. Overall, these adjustments effectively enhanced the fusion performance of the DDPG model, resulting in approximately a 16% reduction in prediction error. During the optimization of DDPG_dual, we implemented the aforementioned improvements and plotted their prediction curves (Figure 9). The results demonstrate that the optimal model outperformed others in closely matching the extreme values of the ground truth curve. For instance, at tidal station DWS, it showed significantly better alignment with actual measurements at the first trough, third trough, and second peak. Similar improvements were observed at station SWI. Moreover, the best performing model maintained superior consistency with real observations across all remaining tidal stations.

5. Conclusions

This paper focuses on the problem of ocean tide level prediction under storm surge conditions, with the core objective of accurately predicting the ocean tide level with minimal error. Given the limitations of a single model in terms of predictive capability, this paper innovatively proposes an ensemble learning-based DNR model. The model leverages the improved reinforcement learning model DDPG_dual to combine the predictions of base models, successfully integrating the advantages of deep learning models and the FVCOM model. Compared to base models, the DNR model achieved significant improvements in prediction accuracy. Through validation with real-world cases, Typhoon Mangkhut, the DNR model demonstrated exceptional predictive capabilities. In particular, under storm surge conditions and when facing unseen environments (such as the BHI station), the DNR model outperformed single models, fully proving its adaptability and reliability in complex environments.Moreover, through ablation studies, this paper further confirms that the two key improvements made to DDPG significantly reduce the tide prediction errors, providing strong theoretical support for model optimization.

We attribute the effectiveness of the DNR model primarily to the following factors: The FVCOM numerical model, based on hydrodynamic equations, captures the physical principles of tidal wave propagation but remains sensitive to boundary conditions; the TCN neural network excels at extracting nonlinear temporal features from historical data but lacks physical constraints. The integration of both preserves physical consistency while enhancing the capability to model complex patterns. The DDPG_dual framework leverages an actor–critic network to dynamically evaluate the predictive reliability of each base model across spatiotemporal conditions, generating adaptive fusion weights that outperform traditional static weighting methods. Furthermore, the policy gradient update mechanism in reinforcement learning learns fusion rules through environmental interactions rather than direct fitting of training data, reducing dependence on annotated datasets and mitigating overfitting in small-sample scenarios.

The DNR model requires extended data duration, which implies that longer monitoring periods are needed to accumulate sufficient observations for optimal model calibration and stable convergence; in particular, capturing seasonal cycles and infrequent extremes typically demands sustained records rather than short-term datasets. In addition, FVCOM computations are time consuming. Running the model, especially at the spatial and temporal resolutions employed in this paper, entails substantial runtime. Future research could explore further ways to integrate the DNR model with other advanced prediction techniques or data sources to address even more complex and dynamic marine environments. This includes the impact of extreme weather events, changes in marine ecosystems, and other factors on tide prediction. Additionally, further investigations can be made to optimize the model’s parameter settings and training strategies, improving the model’s computational efficiency and generalization ability. This will ensure that the model can more quickly and accurately provide tide prediction information in real-world applications, offering stronger support for ocean disaster warnings and emergency responses.