Ocean Wave Height Series Prediction with Numerical Long Short-Term Memory

Abstract

This paper investigates the possibility of using machine learning technology to correct wave height series numerical predictions. This is done by incorporating numerical predictions into long short-term memory (LSTM). Specifically, a novel ocean wave height series prediction framework, referred to as numerical long short-term memory (N-LSTM), is introduced. The N-LSTM takes a combined wave height representation, which is formed of a current wave height measurement and a subsequent Simulating Waves Nearshore (SWAN) numerical prediction, as the input and generates the corrected numerical prediction as the output. The correction is achieved by two modules in cascade, i.e., the LSTM module and the Gaussian approximation module. The LSTM module characterizes the correlation between measurement and numerical prediction. The Gaussian approximation module models the conditional probabilistic distribution of the wave height given the learned LSTM. The corrected numerical prediction is obtained by sampling the conditional probabilistic distribution and the corrected numerical prediction series is obtained by iterating the N-LSTM. Experimental results validate that our N-LSTM effectively lifts the accuracy of wave height numerical prediction from SWAN for the Bohai Sea and Xiaomaidao. Furthermore, compared with the state-of-the-art machine learning based prediction methods (e.g., residual learning), the N-LSTM achieves better prediction accuracy by 10% to 20% for the prediction time varying from 3 to 72 h.

1. Introduction

Ocean waves are irregular combinations of multiple waves with multiple wave heights, periods and travel directions [1]. Ocean waves cause huge losses to the lives and properties of people when big waves reach the coast. The predictions of significant wave height play an important role in marine engineering such as fisheries, exploration, power generation and marine transportation [2].

In the research literature, many efforts have been made in predicting significant wave heights [3,4]. Numerical wave models are widely applied to global sea-state predictions [5,6]. The principle of a numerical wave model is to obtain the wave height, period and other information by solving the wave spectrum equation of ocean physical processes. Dentale et al. [7] compare the wave height buoy observations and model predictions and find that the numerical prediction is a reliable method of wave height prediction. The third generation models such as Wave Model (WAM) [8,9], WAVEWATCH-III (WWIII) [10,11] and Simulating Waves Nearshore (SWAN) [12,13] are among the most advanced numerical models. The WAM model and WMIII model are similar in structure. WMIII uses more complicated dissipation source terms and wind input terms than the WAM model [14]. Liu et al. [15] use the data of the South Indian Ocean to compare the performance of WAM and WMIII. The experiments show that the two methods perform well in the prediction of significant wave height. The SWAN model is developed for addressing the complex wave conditions in coastal regions [16]. Various operations validate the effectiveness of the SWAN model. Specifically, Liang et al. [17] validate the SWAN performance with respect to the buoy measurements in the Northwest Pacific, Northeast Pacific and Northwest Atlantic. The experimental results show that the SWAN model accurately simulates waves in coastal regions when boundary conditions are accurate. However, the fixed energy spectrum equations with fixed expressions hardly characterize the complex and changeable ocean environment in a comprehensively fine way. Specifically, the numerical predictions of waves under extreme ocean conditions are not satisfactory.

Machine learning is a data-driven methodology and has been recently applied to wave height prediction [18,19,20]. Based on long-term accurate wave height measurement data obtained through buoys [21], satellites [22] and scatterometers [23], machine learning methods [24,25] predict wave heights in the future by learning the inherent data variability. Deo et al. [26] explore a three-layered feed forward network to obtain the output of significant wave heights. Berbic et al. [27] use the artificial neural network (ANN) and support vector machine (SVM) to predict the significant wave height between 0.5 and 5.5 h. Experiments validate that the ANN and SVM predictions in such intervals outperform numerical models. Dixit and Londhe [28] use the neuro wavelet technique to predict extreme wave heights. Experiments validate that multi-level decomposition of wave data helps to improve the prediction accuracy. Recurrent neural networks (RNN) [29] and its variant long and short-term memory network (LSTM) [30] have unique advantages in addressing forecasting problems. Zhang et al. [31] use the LSTM model to learn the relationship between the meteorological factors and the rainfall, allowing the correction of the model forecast rainfall. They also have merits in wave height forecasting. Mandal and Prabaharan [32] use a recurrent neural network with rprop update algorithm to predict wave height. Felix and Marina [33] apply RNN-LSTM to predicting significant wave height and the method performs well in forecasting within 24 h. Kaloop et al. [34] integrate wavelet, particle swarm optimization (PSO) and extreme learning machine (ELM) methods into a wavelet PSO-ELM model for coastal and deep-sea wave height estimation. The evaluations show that the method has high prediction accuracy. The disadvantage of the machine learning methods is that the long-term forecasting capability is significantly reduced and the generalization ability of the model is limited.

Both the numerical model and machine learning have advantages and disadvantages for the wave height forecast. Recently, some research is devoted to combining the numerical model prediction methods and machine learning methods to improve the prediction accuracy. One typical manner for combining numerical prediction and machine learning is to use a machine learning model to learn the difference between the numerical predictions and real measurements. Deshmukh et al. [35] use a wavelet neural network for learning the error between SWAN predictions and real measurements. Wang et al. [36] develop a multi-factor extreme learning machine which characterizes the difference between SWAN wave height numerical predictions and real measurements and refers to the strategy as residual learning. Corrections for the numerical predictions are obtained through the learned residuals. Campos et al. [37,38] use neural networks to perform a nonlinear ensemble averaging of the global ocean wave integrated forecasting system data and predict significant wave heights through residual forecasting. Though effectively improving the prediction accuracy of numerical models, the residual learning method tends to exhibit poor performance when the wave height changes abruptly. One reason for this limitation is that residual learning considers a machine learning model and a numerical model as two separate procedures. Learning the difference between their predictions cannot inherently exploit their joint prediction capability.

In order to explore the potential of machine learning for lifting wave height numerical prediction accuracy, we introduce a numerical long short-term memory (N-LSTM) framework for correcting the wave height predictions from the numerical model. The model is composed of an LSTM module and a Gaussian approximation module in cascade. The LSTM module characterizes the measurement-prediction correlation for wave height sequences, resulting in the condition on which the Gaussian approximation module models the wave height distribution. In contrast to error and residual learning [35,36] which just models the difference between measurement and prediction, our N-LSTM comprehensively characterizes the correlation between measurement sequences and numerical prediction sequences via long short-term memory. It explores temporal characteristics and thus renders accurate corrections for numerical prediction sequences. Furthermore, the measurements in our method are used to provide wave height feature information and we can use very few measurements to predict future wave heights for a longer time. As long as there is no missing a lot of measured data, our method can be used. Our N-LSTM is motivated by the autoregressive recurrent networks [39] in terms of model structure but exhibits major differences from two operational aspects. First, our N-LSTM incorporates numerical predictions which are not considered in the autoregressive recurrent network. Second, our N-LSTM makes predictions by correcting the wave height numerical prediction, contrasting the straightforward prediction in the autoregressive recurrent networks. Experimental evaluations validate the effectiveness of our proposed model in predicting significant wave heights by correcting significant wave height predictions from SWAN for the Bohai Sea and Xiaomaidao.

The organization of the paper is listed as follows: Section 2 presents the structure of the numerical long short-term memory (N-LSTM) network and describes the training method for the N-LSTM network. Section 3 describes the significant wave height correction based on an N-LSTM network. Section 4 evaluates the effectiveness of our method qualitatively and quantitatively. Section 5 discusses the effectiveness of our method under large wave conditions and the probability correction. Section 6 concludes the paper.

2. Numerical Long Short-Term Memory (N-LSTM) Network

2.1. Overall N-LSTM Model

In this section, we present a numerical long short-term memory (N-LSTM) network for correcting the numerical wave height prediction series. The N-LSTM processes time-sequential data. It consists of an LSTM module and a Gaussian approximation module. Figure 1 shows the structure of the N-LSTM.

At the time $t$, the input of the overall N-LSTM and also the LSTM module is a combined significant wave height representation ${\mathit{x}}_t$, which is formed of the current wave height measurement $m_t$ at the time $t$ and a numerical prediction $n_{t+1}$ for the significant wave height at the subsequent time $t$ + 1. The LSTM input ${\mathit{x}}_t$ is denoted as follows:

$${\mathit{x}}_t={\left[m_t,n_{t+1}\right]}^{\ast },$$

(1)

where $\ast$ denotes the transpose operation. The LSTM updates the memory cell state ${\mathit{c}}_t$ and the output of the LSTM module ${\mathit{h}}_t$. ${\mathit{c}}_t$ and ${\mathit{h}}_t$ are fed back for updating the LSTM at the subsequent time $t$ + 1. In addition, ${\mathit{h}}_t$ is used as the input of the Gaussian approximation module. The output of the overall N-LSTM and also the Gaussian approximation module is ${\widehat{n}}_{t+1}$, which is the corrected numerical prediction with respect to the numerical prediction $n_{t+1}$.

2.2. LSTM Module

The subsection describes the internal structure of the LSTM module. The LSTM module consists of an input gate ${\mathit{i}}_t$, a forget gate ${\mathit{f}}_t$ and an output gate ${\mathit{o}}_t$. Figure 2 shows the architecture of an LSTM module.

At the time $t$, the inputs of the LSTM module contain the combined significant wave height representation ${\mathit{x}}_t$, the memory cell state ${\mathit{c}}_{t-1}$ and the output of the LSTM module ${\mathit{h}}_{t-1}$. ${\tilde{\mathit{c}}}_{\mathrm{t}}$ denotes memory information. The LSTM computing process is as follows:

$${\mathit{i}}_t=\mathrm{sig}\left({\mathit{W}}_i{\mathit{x}}_t+{\mathit{U}}_i{\mathit{h}}_{t-1}+{\mathit{b}}_i\right),$$

(2)

$${\mathit{f}}_t=\mathrm{sig}\left({\mathit{W}}_f{\mathit{x}}_t+{\mathit{U}}_f{\mathit{h}}_{t-1}+{\mathit{b}}_f\right),$$

(3)

$${\mathit{o}}_t=\mathrm{sig}\left({\mathit{W}}_o{\mathit{x}}_t+{\mathit{U}}_o{\mathit{h}}_{t-1}+{\mathit{b}}_o\right),$$

(4)

$${\tilde{\mathit{c}}}_t=\tanh \left({\mathit{W}}_c{\mathit{x}}_t+{\mathit{U}}_c{\mathit{h}}_{t-1}+{\mathit{b}}_c\right),$$

(5)

$${\mathit{c}}_t={\mathit{i}}_t○{\mathit{c}}_{t-1}+{\mathit{f}}_t○{\tilde{\mathit{c}}}_t,$$

(6)

$${\mathit{h}}_t={\mathit{o}}_t○\tanh \left({\mathit{c}}_t\right),$$

(7)

where $\mathit{W}$, $\mathit{U}$ denote weight matrices and $\mathit{b}$ denote bias vectors. Sig and tanh are smooth step functions and hyperbolic tangent functions, respectively. $○$ denotes the Hadamard product. The output of the LSTM module ${\mathit{c}}_t$ and ${\mathit{h}}_t$ will be the input of the LSTM module at a subsequent time and ${\mathit{h}}_t$ is also the input of the Gaussian approximation module. ${\mathsf{\Theta }}_{\mathrm{LSTM}}$ summarizes the parameters of the LSTM module as follows:

$${\mathsf{\Theta }}_{\mathrm{LSTM}}=\left\{{\mathit{W}}_i,{\mathit{U}}_i,{\mathit{b}}_i,{\mathit{W}}_f,{\mathit{U}}_f,{\mathit{b}}_f,{\mathit{W}}_o,{\mathit{U}}_o,{\mathit{b}}_o,{\mathit{W}}_c,{\mathit{U}}_c,{\mathit{b}}_c\right\}$$

(8)

The LSTM structure extracts long-term useful features in sequences and has been validated effectively in a variety of prediction problems. Different from the original LSTM which tends to memorize historical measurements to predict subsequent measurements, the LSTM module within our N-LSTM explores a combined significant wave height representation in terms of both real measurements and the numerical predictions. Being capable of capturing time-varying characteristics, the LSTM module characterizes the temporal correlation between the real measurements and the numerical predictions. Therefore, the LSTM module within the N-LSTM takes advantage of both real measurements and the numerical predictions along with their temporal correlation characteristics and is bound to exhibit greater prediction capability than the original LSTM which just relies on memorizing historical measurements.

2.3. Gaussian Approximation Module

The LSTM module learns the correlation between historical measurements and numerical prediction. The learning uncertainty arises subject to noise from historical measurements and numerical prediction. To alleviate such deficiency, probabilistic models are used for addressing uncertainty. The widely used Gaussian function is exploited for modeling the conditional distribution of the significant wave height, and accordingly a Gaussian approximation module is introduced. Figure 3 shows the architecture of a Gaussian approximation module.

At the time $t$, the input of the Gaussian approximation module is the output of the LSTM module ${\mathit{h}}_t$. Specifically, the Gaussian approximation module commences by processing ${\mathit{h}}_t$ via two layers in parallel as shown in Figure 3. The top layer is a linear mapping which operates as follows:

$$\mu \left({\mathit{h}}_t\right)={\mathit{w}}_{\mu }^{*}{\mathit{h}}_t+b_{\mu },$$

(9)

where ${\mathit{w}}_{\mu }^{*}$ and $b_{\mu }$ are the weight and bias of the linear mapping, respectively. The bottom layer is a linear mapping, followed by a softplus activation. The softplus activation is used to ensure the variance σ > 0. The linear mapping and softplus activation with respect to the bottom layer operate as follows:

$$\sigma \left({\mathit{h}}_t\right)=\log \left(1+\exp \left(\mu ({\mathit{w}}_{\sigma }^{*}{\mathit{h}}_t+b_{\sigma })\right)\right),$$

(10)

where ${\mathit{w}}_{\sigma }^{*}$ and $b_{\sigma }$ are the weight and bias of the linear mapping, respectively. The softplus activation layer ensures that $\sigma \left({\mathit{h}}_t\right)$ is positive. We denote the parameter set ${\mathsf{\Theta }}_P$ of the Gaussian approximation module as follows:

$${\mathsf{\Theta }}_P=\left\{{\mathit{w}}_{\mu },b_{\mu },{\mathit{w}}_{\sigma },b_{\sigma }\right\}$$

(11)

The Gaussian approximation module terminates by modeling a conditional probabilistic distribution for significant wave height in terms of a Gaussian distribution. The outputs $\mu \left({\mathit{h}}_t\right)$ and $\sigma \left({\mathit{h}}_t\right)$ of the two layers (9) and (10) are modeled as the mean and variance of the Gaussian distribution, respectively. The conditional probabilistic distribution for significant wave height is given as follows:

$$P\left(r_{t+1}|{\mathit{h}}_t; \mathsf{\Theta }\right)=\frac{1}{\sqrt{2\pi }\sigma \left({\mathit{h}}_t\right)}\exp \left(-\frac{{\left(r_{t+1}-\mu \left({\mathit{h}}_t\right)\right)}^2}{2{\left(\sigma \left({\mathit{h}}_t\right)\right)}^2}\right),$$

(12)

where $r_{t+1}$ denotes the real wave height observations at the time t + 1 and Θ represents the overall N-LSTM parameter set including ${\mathsf{\Theta }}_{\mathrm{LSTM}}$ and ${\mathsf{\Theta }}_P$ as follows:

$$\mathsf{\Theta }=\left\{{\mathsf{\Theta }}_{\mathrm{LSTM}},{ \mathsf{\Theta }}_P\right\}$$

(13)

The Gaussian approximation module models the conditional probabilistic distribution of the subsequent significant wave height given the learned LSTM (i.e., the learned ${\mathit{h}}_t$), rather than straightforwardly predicting significant wave height. It should be noted that the unconditional significant wave height distribution might not follow Gaussian distribution but an arbitrary one. It is observed in the literature of probability that the distribution of an arbitrary (non-Gaussian) process can be approximated by a weighted combination of Gaussian distribution functions. In the light of this observation, the unconditional distribution $P\left(r_{t+1}\right)$ of the significant wave height process can be expressed as a weighted combination of Gaussian distribution $P\left(r_{t+1}|{\mathit{h}}_t\right)$ as follows:

$$P\left(r_{t+1}\right)={{\displaystyle \int }}_{{\mathit{h}}_t}P\left(r_{t+1}|{\mathit{h}}_t\right)P\left({\mathit{h}}_t\right)d{\mathit{h}}_t,$$

(14)

where $P\left({\mathit{h}}_t\right)$ behaves as the weight for combination. This justifies the effectiveness to use the Gaussian function to model the conditional distribution of significant wave height.

2.4. Sampling for Correcting Numerical Predictions

The prediction for significant wave height at the time $t$ + 1 is obtained by sampling from the conditional probability distribution of prediction (12) and it is presented as follows:

$${\widehat{n}}_{t+1}~P\left(·|{\mathit{h}}_t; \mathsf{\Theta }\right)$$

(15)

The benefits of the sampling procedure are two-fold. First, the sampling procedure produces a future result by not only learning historical measurements but also correcting numerical predictions with respect to historical measurements. This is enabled by the conditional probability distribution which encodes both measurements and numerical predictions within ${\mathit{h}}_t$. Second, the sampling strategy alleviates the accumulation of prediction errors which cause considerable prediction uncertainty. At each time step, an LSTM without sampling the conditional probabilistic distribution predicts a fixed value. The predicted value might be contaminated by noise. If the noisy predicted values keep recycled for updating LSTM, error accumulation inevitably arises. The random sampling of the conditional distribution neutralizes such error accumulation and renders predictions more robust than those straightforwardly regressed from an LSTM.

2.5. N-LSTM Training

One sample for training the N-LSTM consists of a real significant wave height observation sequence ${\mathit{r}}_{1:T}=\left\{r_1,r_2,\cdots r_{\tau },r_{\tau +1},\cdots ,r_T\right\}$ and a significant wave height numerical prediction sequence ${\mathit{n}}_{1:T}=\left\{n_1,n_2,\cdots n_{\tau },n_{\tau +1},\cdots ,n_T\right\}$, where T indicates the sequence length. The training procedure with respect to the sample is under the assumption that the input of real observations is available for the measurement time $t\in \left\{1, 2, \cdots ,\tau \right\}$, but the unavailable time $t\in \left\{\tau +1,\cdots ,T\right\}$. In this scenario, the significant wave height predictions obtained as described in Section 2.4 are used for representing the measurement time $t\in \left\{\tau +1,\cdots ,T\right\}.$

In addition, we construct the significant wave height measurement sequences ${\mathit{m}}_{1:T}=\left\{m_1,m_2,\cdots ,m_{\tau },m_{\tau +1},\cdots ,m_T\right\}$, with the significant wave height measurement $m_t$ defined as follows:

$$m_t=\{\begin{array}{cc}{r}_t,& \mathrm{for} t\in \left\{1, 2, \cdots ,\tau \right\}\\ {\widehat{n}}_t,& \mathrm{for} t\in \left\{\tau +1, \cdots ,T\right\}\end{array},$$

(16)

where $r_t$ is the real observation of the significant wave height at time $t$ and ${\widehat{n}}_t$ is computed according to (14) and obtained as follows:

$${\widehat{n}}_t~P\left(·|{\mathit{h}}_{t-1}; \mathsf{\Theta }\right)$$

(17)

Given a training dataset including I significant wave height numerical prediction sequences ${\left\{{\mathit{m}}^{\left(i\right)}\right\}}_{i=1,\cdots ,I}$ and I wave height real observation sequences ${\left\{{\mathit{n}}^{\left(i\right)}\right\}}_{i=1,\cdots ,I}$, we train the N-LSTM by maximizing the log-likelihood as follows:

$$\widehat{\mathsf{\Theta }}=\arg \underset{\mathsf{\Theta }}{\max } L\left(\mathsf{\Theta }\right)=\mathrm{argmax}{\displaystyle \sum }_{i=1}^I{\displaystyle \sum }_{t=1}^T\log P\left(r_{t+1}^{\left(i\right)}|{\mathit{h}}_t^{\left(i\right)}; \mathsf{\Theta }\right)$$

(18)

We use a stochastic gradient descent algorithm to optimize Equation (18) by calculating the gradient of $\widehat{\mathsf{\Theta }}$. The Adam adaptive stochastic gradient descent optimizer is used to minimize the loss in our work [40]. The overall procedure of training the N-LSTM network is presented in Algorithm 1.

Algorithm 1 The training procedure of the N-LSTM

1: Input: The significant wave height numerical predictions ${\left\{{\mathit{n}}^{\left(i\right)}\right\}}_{i=1,\cdots ,I}$ and the real observations ${\left\{{\mathit{r}}^{\left(i\right)}\right\}}_{i=1,\cdots ,I}$. 2: Output: The parameters $\widehat{\mathsf{\Theta }}$ of the autoregressive recurrent network. 3: for $i=1,2,\cdots ,I$ do 4: for $t=1,2,\cdots ,T$ do 5: Compute ${\mathit{h}}_t$ according to (2)–(7). 6: Compute $\mu \left({\mathit{h}}_t\right)$ and $\sigma \left({\mathit{h}}_t\right)$ according to (9) and (10). 7: Construct the conditional probabilistic distribution $P\left(r_t|{\mathit{h}}_t;\mathsf{\Theta }\right)$ according to (12). 8: end for 9: Maximize the log-likelihood according to (18). 10: Train the log-likelihood by stochastic gradient descent algorithm. 11: end for

3. Numerical Wave Height Series Correction Based on N-LSTM

Given the significant wave height real observations ${\mathit{r}}_{1:\tau }=\left\{r_1,r_2,\cdots ,r_{\tau }\right\}$ and numerical predictions ${\mathit{n}}_{1:T}=\left\{n_1,n_2,\cdots n_{\tau },n_{\tau +1},\cdots ,n_T\right\}$, the learned N-LSTM corrects the significant wave height numerical predictions $\left\{n_{\tau +1},n_{\tau +2},\cdots ,n_T\right\}$ and produces corrected numerical predictions $\left\{{\widehat{n}}_{\tau +1},{\widehat{n}}_{\tau +2},\cdots ,{\widehat{n}}_T\right\}$. Figure 4 shows the significant wave height prediction procedure of N-LSTM.

The significant wave heights at the time $t\in \left\{1, 2, \cdots ,\tau \right\}$ does not require prediction because real observations are available in the time interval. The significant wave height real observations ${\mathit{r}}_{1:\tau }$ and numerical predictions ${\mathit{n}}_{1:\tau }$ at the time $t\in \left\{1, 2, \cdots ,\tau \right\}$ are used for computing ${\mathit{h}}_{\tau }$ according to (7). The real observation is no longer available starting from the time $\tau +1$. At the time $\tau +1$, the corrected prediction ${\widehat{n}}_{\tau +1}$ is produced Gaussian approximation module in terms of sampling the conditional distribution as follows:

$${\widehat{n}}_{\tau +1}~P\left(·|{\mathit{h}}_{\tau }; \mathsf{\Theta }\right)$$

(19)

At the time $t\in \left\{\tau +1,\cdots ,T\right\}$, the corrected prediction ${\widehat{n}}_t$ is used as the measurement $m_t$, i.e.,:

$$m_t={\widehat{n}}_t$$

(20)

Specifically, the input of the N-LSTM is given as follows:

$${\mathit{x}}_t={\left[{\widehat{n}}_t,n_{t+1}\right]}^{*},\mathrm{for} t\in \left\{\tau +1,\cdots ,T\right\}$$

(21)

The corrected numerical prediction series $\left\{{\widehat{n}}_{\tau +1},{\widehat{n}}_{\tau +2},\cdots ,{\widehat{n}}_T\right\}$ are obtained by iterating the N-LSTM procedure as illustrated in Figure 1. In order to render more accurate predictions, the sampling at each time $t\in \left\{\tau +1,\cdots ,T\right\}$ is performed multiple times and the average is given as the resulted prediction ${\widehat{n}}_t$.

4. Results

4.1. Data Collection and Evaluation Criterions

The study area is the Bohai Sea and the northern part of the Yellow Sea (35°–41° N, 117°–123° E), as shown in Figure 5. The submarine topography in the study area is relatively flat, with an average water depth of about 18 m. The maximum tidal range is about 2.7 m. The study area is dominated by wind waves and is greatly affected by the monsoon.

In our work, the wave height numerical predictions are obtained from the third-generation numerical wave model SWAN. The grids in this study are with a spatial resolution of 0.1° × 0.1°. The source term in the SWAN model uses the default formula package. The model considers the dissipation caused by whitecapping, bottom friction and depth induced wave breaking [41]. Other configurations of the SWAN model are carried out according to [42]. The real wave height observations for validating our method are collected from buoys located in the Bohai Sea, China, and Xiaomaidao Nearshore Sea, Shandong. The buoy in the Bohai Sea is located in the open waters inside the Bohai Sea with a water depth of 20 m. The position of the buoy on Xiaomaidao is shown in Figure 5 and the water depth is about 24 m. All the obtained data are recorded hourly with a period for two years from January 2017 to December 2018. We use a total of 17,510 sample pairs to train and test our model. We randomly select 10% of the time-continuous data pairs as the test set and the remaining 90% of the data pairs are used as the training set. All training data and test data are divided into real significant wave height observation sequences and numerical model prediction sequences according to Section 2.5. The number of sequences is related to the predicted time T.

We adopt the root mean square error (RMSE), mean absolute percentage error (MAPE) and skill score (SS) to evaluate the effectiveness of the proposed method. The RMSE and MAPE metric are defined as follows:

$$\mathrm{RMSE}=\sqrt{\frac{1}{T}{\displaystyle \sum }_{t=1}^T{\left(r_t-{\widehat{n}}_t\right)}^2},$$

(22)

$$\mathrm{MAPE}=\frac{1}{T}{\displaystyle \sum }_{t=1}^T\left|\frac{r_t-{\widehat{n}}_t}{r_t}\right|$$

(23)

The skill score shows the relative improvement of the proposed models over the numerical model. It is defined as follows:

$$\mathrm{SS}=\frac{{{\displaystyle \sum }}_{t=1}^T\left|r_t-n_t\right|-{{\displaystyle \sum }}_{t=1}^T\left|{\widehat{n}}_t-n_t\right|}{{{\displaystyle \sum }}_{t=1}^T\left|r_t-n_t\right|}$$

(24)

The performance of the prediction methods is better when the RMSE and MAPE are closer to 0 and the SS is closer to 1.

In our experiment, the epoch and learning rate are set to 10,000 and 0.001, respectively. The number of hidden units of the LSTM module in the N-LSTM is set to 50. The hyper-parameter setting of LSTM is the same as that of N-LSTM. The major hyper-parameter of the ELM methods is the number of hidden layers, which is set to 36 for both ELM and MFELM. We carry out hyper-parameter optimization through manual search. We conduct our experiments on a computing platform with three Intel Xeon CPUs E5-2690 at 2.60 GHz. N-LSTM and LSTM are implemented by PyTorch under the environment of Python. ELM residual learning method and MFELM residual learning method are implemented by using MATLAB R2019a.

4.2. Empirical Evaluations

We compare the performance of our N-LSTM method with the LSTM method, the ELM residual learning method [36] and the MFELM residual learning method [43] through the significant wave height prediction results of the numerical model in the future time. The numerical model is used as a baseline method for evaluating the improvement of the predictions from the numerical model. The LSTM behaves as one module in our N-LSTM. The other important module in the N-LSTM is the Gaussian approximation. The LSTM is used for validating the effectiveness of the overall N-LSTM (especially with the involvement of the Gaussian approximation) over the sole LSTM. The ELM and MFELM share some similar ideas to our N-LSTM in terms of correcting numerical predictions. They straightforwardly model the difference between the numerical predictions and the real observations via residual learning. In particular, the MFELM residual learning method exploits multiple factors, such as wind speed and wind direction, and achieves state-of-the-art performance for wave height predictions. In contrast to the ELM and MFELM residual correction methods, the N-LSTM does not model the residuals in a straightforward way but comprehensively incorporates the correction procedure into an LSTM followed by a Gaussian approximation. The empirical comparison between the N-LSTM and the (MF)ELM validates the advantage of the comprehensive correction procedure over the straightforward correction procedure.

We correct the significant wave height numerical model predictions for 3 h, 6 h, 12 h, 24 h, 48 h, 72 h and the time step is one hour. Each of these forecast leads comes from independent N-LSTM models.

Table 1. Significant wave height prediction results for the buoy position in the Bohai Sea.

Validation Method	Method	3 h	6 h	12 h	24 h	48 h	72 h
RMSE	Numerical model	0.4730	0.4325	0.3716	0.2728	0.2811	0.2807
	LSTM	0.1372	0.1835	0.2163	0.1554	0.1706	0.2051
	ELM residual learning	0.2160	0.2525	0.2796	0.2409	0.2340	0.2427
	MFELM residual learning	0.1888	0.2133	0.2194	0.2126	0.2104	0.2103
	N-LSTM (Our method)	0.0581	0.1248	0.1479	0.1237	0.1500	0.1869
MAPE	Numerical model	0.2688	0.2466	0.2267	0.2717	0.1544	0.1420
	LSTM	0.0728	0.1025	0.1281	0.1522	0.0917	0.1092
	ELM residual learning	0.1172	0.1393	0.1569	0.2789	0.1219	0.1209
	MFELM residual learning	0.1039	0.1167	0.1192	0.2611	0.1026	0.1039
	N-LSTM (Our method)	0.0319	0.0691	0.0830	0.1233	0.0833	0.0946
SS	LSTM	0.7412	0.5854	0.4343	0.5142	0.4360	0.2848
	ELM residual learning	0.5598	0.4383	0.3128	0.1135	0.2189	0.1497
	MFELM residual learning	0.6362	0.5280	0.4626	0.2215	0.3267	0.2776
	N-LSTM (Our method)	0.8890	0.7221	0.6530	0.5670	0.5266	0.3655

and

Table 2. Significant wave height prediction results for Xiaomaidao.

Validation Method	Method	3 h	6 h	12 h	24 h	48 h	72 h
RMSE	Numerical model	0.1529	0.1181	0.1248	0.1547	0.1777	0.3243
	LSTM	0.0610	0.0927	0.0838	0.1353	0.1593	0.2326
	ELM residual learning	0.0839	0.0402	0.0915	0.1119	0.1546	0.2713
	MFELM residual learning	0.0766	0.0440	0.0775	0.0849	0.1176	0.2944
	N-LSTM (Our method)	0.0174	0.0574	0.0765	0.0831	0.1166	0.1858
MAPE	Numerical model	0.7639	0.4938	0.6443	0.7174	0.9697	1.2006
	LSTM	0.2955	0.3278	0.4170	0.6332	0.8954	0.8170
	ELM residual learning	0.4171	0.1446	0.4574	0.4220	0.5910	0.8979
	MFELM residual learning	0.3815	0.1808	0.2968	0.3629	0.6095	1.1040
	N-LSTM (Our method)	0.0774	0.2191	0.3765	0.3516	0.6260	0.6229
SS	LSTM	0.6131	0.2213	0.4069	0.1398	0.1206	0.3317
	ELM residual learning	0.4540	0.7309	0.2968	0.3527	0.2518	0.1946
	MFELM residual learning	0.5005	0.6012	0.4339	0.5247	0.4092	0.0908
	N-LSTM (Our method)	0.8987	0.5825	0.4453	0.5321	0.4218	0.4891

show wave height prediction results at the buoy positions of the open water in Bohai Sea and the nearshore water by Xiaomaidao, respectively. The bold entries denote the best results. From Table 1, we observe that our proposed N-LSTM is significantly better than the LSTM method, the ELM residual learning method and the MFELM residual learning method in each evaluation criterion. This reflects that correcting the significant wave height predictions from the numerical model in the future time steps according to the real observations and the numerical model in the previous time steps is useful, and our model learns the relationship between the numerical model predictions and real observations effectively. In addition, we observe that the accuracy of all prediction methods decreases with the increase of the prediction time. In particular, the accuracies of the ELM related methods are considerably reduced when the prediction time increases more than 24 h. However, our method maintains good predictions in 24 h and 48 h and the skill scores are both higher by 50%. It drops to 36% in the 72-h prediction and is still significantly better than the other methods. This validates that our N-LSTM maintains good results in long-term predictions. This is because the LSTM structure can save long-term information that is not available in the ELM methods. From Table 2, the N-LSTM achieves the best performance compared with the other methods except the six-hour prediction. The comparison methods have different wave height prediction accuracies at different locations. The original LSTM outperforms two ELM methods for the Bohai Sea. The MFELM outperforms the original LSTM for Xiaomaidao. It confirms that our method effectively improves the accuracy of nearshore wave height prediction and validates that our method has robustness in predicting wave heights in different sea areas.

Figure 6 shows the significant wave height prediction results at various time steps. We evaluate the prediction results of several methods qualitatively through Figure 6. We see that the performance of our method is better than the other three machine learning methods. In particular, Figure 6d,e show that as the significant wave height increases, the error between the predictions of the numerical model and the real observations increases too. We can see that our N-LSTM method is more accurate than the other three machine learning methods in these cases. More detailed validations will be introduced in Section 4.1. In addition, the principle of the ELM residual learning method and the MFELM residual learning method is to predict the residuals between the real observations and the predictions from the numerical model. It is difficult to accurately predict the residual when the residuals change suddenly (the significant wave height has a peak). Our N-LSTM method characterizes the temporal correlation between real observations and numerical predictions rather than straightforwardly use their difference such that it achieves better performance.

5. Discussion

5.1. Analysis of Significant Wave Height Predictions between 1.25 m and 4 m

The World Meteorological Organization (WMO) sea state code describes the wave height and corresponding characteristics [44]. When the significant wave height is below 1.25 m, the sea state is calm or slight. The numerical model has a good performance in this case. The probability of the significant wave height above 4 m appearing on the coast is extremely low. The significant wave height sometimes occurs between 1.25 m and 4 m. The sea state at this time is rough and will affect marine production. This section mainly discusses the model prediction of significant wave height between 1.25 m and 4 m.

Figure 7 shows the scatter plot of the significant wave height prediction results in which the real observations are between 1.25 m and 4 m. In the plot, the diagonal indicate 100% accuracy. The scattered points close to the diagonal reflect accurate predictions and those far from the diagonal inaccurate ones. In Figure 7, the numerical model prediction results are concentrated on the upper left of the diagonal. It means that the correct results for the numerical model are always smaller than the real observations. The prediction results obtained by our method are closer to the diagonal than the results obtained by the ELM residual learning method and the MFELM residual learning method. In particular, the predictions of the numerical model have a large deviation and the prediction effectiveness of the other two ELM methods is poor when the significant wave heights are higher than 2 m. In contrast, our method maintains a good prediction of this situation.

5.2. Analysis of Probabilistic Wave Height Predictions

In addition to calculating the prediction results of the wave height numerical model, our method obtains the confidence interval of the prediction wave heights in the form of Monte Carlo sampling [45] and obtain probabilistic forecasts of wave heights in the future. Figure 8 shows probabilistic predictions for the forecasting time of 1–3 h for the nowcast and 48 h for the long forecast and made 600 h of consecutive predictions. The black scatters shows the real observations. The green line shows the numerical model predictions. We plot prediction results from our method as red line along with eighty percent confidence intervals (shaded area). The confidence interval represents the probability interval that real wave height observations fall between the predictions.

Figure 8 indicates that our N-LSTM predictions accurately approach most of the real observations for the one-h probability forecast, no matter the significant wave heights have a peak or stay stable. In addition, the corrected confidence intervals are smooth and their amplitude is small. This reflects that the probabilistic forecast is useful in the one-h prediction. The confidence interval increases as the forecast time step increases and the amplitude fluctuate greatly at the peak. In general, the confidence interval reasonably presents the characteristics of uncertainty, because the confidence interval contains most of the real observations. Accurate probability prediction of significant wave height plays an important role in early warning of Marine disasters.

6. Conclusions

This paper presents a numerical long short-term memory (N-LSTM) network for correcting the significant wave height predictions from the numerical model. The N-LSTM takes a combined significant wave height representation, which is formed of real observations from a buoy in the Bohai Sea and the predictions of the SWAN at the same location as the input and produces the corrected numerical prediction. The N-LSTM characterizes the temporal correlation between the real measurement and the numerical prediction and models the conditional probabilistic distribution of the significant wave height real observations and numerical predictions. Compared with traditional machine learning prediction methods, our method reduces the accumulation of error in the prediction process. The experimental results show that the N-LSTM network has better prediction accuracy. Besides, our N-LSTM method performs better when the significant wave height is over 1.25 m and the numerical model predicts poor performance in this case. Our method N-LSTM make probabilistic predictions for significant wave heights and the confidence interval of the prediction covers most of the real observations. Therefore, our N-LSTM method is capable of coping with not only difficult wave situations but also prediction uncertainties, and renders a robust prediction strategy.

Furthermore, we find that the wave height prediction of the numerical model sometimes has a lag. Since our model only uses the significant wave height as the input data, the prediction may fail. In future work, we will add sea surface wind, atmospheric pressure and other factors as input and go deep in to investigate the impact of the wave type [46,47] and meteorological elements.