A Hybrid Extended Kalman Filter Based on Parametrized ANNs for the Improvement of the Forecasts of Numerical Weather and Wave Prediction Models

Abstract

A hybrid optimization filter for weather and wave numerical models is proposed and tested in this study. Parametrized Artificial Neural Networks are utilized in conjunction with Extended Kalman Filters to provide a novel postprocess strategy for 10 m wind speed, 2 m air temperature, and significant wave height simulations. The innovation of the developed model is the implementation of Feedforward Neural Networks and Radial Basis Function Neural Networks as estimators of an exogenous parameter that adjusts the covariance matrices of the Extended Kalman Filter process. This hybrid system is evaluated through a time window process leading to promising results, thus enabling a decrease in systematic errors alongside the restriction of the error variability and the corresponding forecast uncertainty. The obtained results showed that the average reduction of the systematic error exceeded 75%, while the corresponding nonsystematic part of that error decreased by 35%.

1. Introduction

The necessity for accurate weather and wave predictions has increased dramatically in recent decades as a result of several affected applications, including ocean surface and wildfire modeling. Numerical Weather and Wave Prediction (NWWP) models are extensively utilized for these purposes and are employed by operational and research institutions to successfully simulate global environmental conditions. Yet, when anticipating a local region of interest, NWWP models fail to produce satisfactory forecasts. This is owing to the multiparametric character of the numerous components involved, such as the significant dependency on the initial and lateral boundary conditions, the inability to capture subscale phenomena, and the variability of subgrid scale physiographic characteristics (e.g., land use, topography, etc.), as well as the inherent errors in the numerical methods and the parameterization of atmospheric/wave processes [1].

To address these concerns, one possible answer could be to improve the model’s resolution; however, the effectiveness of this strategy is uncertain, and the computational cost will undoubtedly rise significantly. An alternative technique would be to improve the initial conditions using assimilation systems and postprocessing algorithms [2,3,4]. These tools, such as Artificial Neural Networks (ANNs) and Kalman Filters (KFs), are used to improve the final output from statistical models. Such approaches construct a cost function, which is defined as the “distance” (bias) between the model’s direct outputs and the corresponding recorded observations to minimize the simulation’s systematic inaccuracy.

Under this framework, a combined method of Extended Kalman Filters (EKFs) and Artificial Neural Networks is presented in this study for the enhancement of the forecasts of NWWP models. Specifically, the numerical WAve Model (WAM) and the numerical Weather Research and Forecasting (WRF) model are utilized, thus regarding the environmental parameters of significant wave height, 10 m wind speed, and 2 m air temperature.

To accomplish this, the suggested methodology aims to eliminate the systematic part of the forecast error and reduce the remaining nonsystematic noise. Several strategies have been developed to achieve the first objective. Particularly, Kariniotakis and Pinson [5] and Kariniotakis et al. [6] suggested an ANN approach, whereas Vanem [7], Giebel [8], Resconi [9], and others proposed methodologies based on heavy statistical models. This study, however, implements the Kalman Filter process due to the reduced CPU memory requirements, which constitute a crucial advantage for every application.

The Kalman Filter [10,11,12] and its nonlinear expansion, EKF [13,14], are valuable tools for reducing systematic errors, as they are considered to be the statistically ideal sequential strategy that integrates recursive observations with current predictions using weights that minimize the relevant biases [15]. Nevertheless, in several cases, such filters fail to decrease the variability in the forecast error [16], thus producing poor final forecasts due to the remaining uncertainty. This is partly owing to the filter’s initial assumptions, which often require fixed covariance matrices, which are defined before the implementation of the method. However, this approach is inadequate for dynamic conditions, as the measurement inaccuracy varies due to external influences.

To avoid this issue and reduce the remaining nonsystematic part of the forecast error, this work employs a modified estimation rule to update the covariance matrices based on an exogenous (memory) parameter that is determined using parametrized Artificial Neural Networks (pANNs). The initial methodology implements a parametrized Feedforward Neural Network (pFFNN). More specifically, the pFFNN is trained with multiple parameter values, and the optimal one is selected according to a preset goal. The chosen value is then applied to the EKF to adapt the covariance matrices and provide improved results for the NWWP model in use. This Hybrid Extended Kalman Filter (HEKF) was successfully used for significant wave height forecasts, thus improving the direct outputs of the WAM model [17].

The main drawback of this strategy, though, is the increased computational cost caused by the FFNN training algorithms; therefore, an alternative ANN structure is required. As a result, the novelty of the presented study is the introduction of Radial Basis Function Neural Networks (RBF NNs) as parameter estimators (pRBF NNs). The various architectures are compared for several environmental factors derived from different time periods and regions of Greece, thus aiming to examine the efficiency of the methodology in terms of accuracy and computational (time) cost.

The evaluation of the method was carried out through a time window process. More thoroughly, the first case study focuses on the area of Crete in the Aegean Sea from 2007 to 2009 regarding the significant wave height, while the second and the third case studies concern the area of Thessaloniki in northern Greece for 2020, thus regarding 10 m wind speed and 2 m air temperature simulations.

The structure of this work is organized as follows: In Section 2, the utilized numerical weather and wave prediction models are described alongside a detailed presentation of the proposed methodology. Section 3 covers the fundamentals of the Extended Kalman Filter and the suggested adaptative estimation rule for the covariance matrices, while in Section 4, the various parametrized Artificial Neural Networks are presented. Moving forward, the developed Hybrid Extended Kalman Filter is described in Section 5, whereas in Section 6, the time window process is analyzed, as well as the extracted results for every case study. Finally, the conclusions drawn from this study are discussed in Section 7.

2. Models and Methodology

The utilized weather and wave numerical models are described in this section alongside a thorough analysis of the proposed methodology, thus highlighting its core components.

2.1. Numerical WAve Model

One of the world’s most widely applied and well-tested models for wave forecasting is the WAve Model. WAM [18] is a 3rd generation model that solves the wave transport equation directly, with no assumptions regarding the structure of the wave spectrum. The numerical model can be utilized on any given grid with an arbitrarily set time and space resolution—in both deep and shallow waters—thus taking into consideration the effect of wave refraction induced by depth variations and wave currents.

A modification of the standard WAM model is implemented in this study, which is based on the ECMWF (European Centre for Medium Range Weather Forecasts) version CY46R1 [19,20], which establishes new parameterizations for the spectral dissipation. These parameterizations are merged and calibrated using the discrete interaction approximation for nonlinear interactions. The applied enhancements result in more accurate modeling of swell dissipation, as well as better forecasting of extreme wave values over swell-dominated locations such as the southeast region of the Mediterranean Sea.

WAM produces a variety of outputs, including full wave spectra at predefined grid points, significant wave height, mean wave direction and frequency, wind and swell wave height and mean direction components, and wind stress fields corrected for wave-induced stress and drag coefficients at each grid point at selected output times. However, this research focuses on the wave parameter of Significant Wave Height (SWH), which is utilized to a variety of cutting-edge applications (renewable energy sources, research and rescue, shipping, etc.) and is defined as the mean wave height of the highest third of the waves, which can be computed by the following:

$$SWH=4\sqrt{m_0}$$

(1)

where $m_0={{\displaystyle \int }}_0^{2\pi }{{\displaystyle \int }}_0^{\infty }{f}^{0}F\left(f,\theta \right) df d\theta$. $F$ represents the spectral density with respect to frequency $f$ and direction $\theta$.

2.2. Weather Research and Forecasting Model

The Weather Research and Forecasting model is a numerical nonhydrostatic system that produces atmospheric simulations intended for research and operational use. The WRF model [21] has been successfully utilized in a variety of applications such as numerical weather prediction, air chemistry, hydrology, wildland fires, intense weather events and regional climate [22,23,24,25].

The simulation procedure has two stages. Initially, the model’s domain is configured by assimilating the input data and preparing the initial conditions, followed by the implementation of the prediction system, which runs within WRF’s software framework and handles I/O and parallel computing communications. Specifically, WRF simulations start with the WRF Preprocessing System (WPS), which analyzes and combines geographical information (such as topography and land use) to establish model domains.

Afterward, it collects, reorganizes, and interpolates the required initial guess atmospheric data (e.g., a global analysis or model forecast) into the user’s domains, thus generating the model’s vertical levels and lateral boundary conditions [26]. Then, the WRF model is ready to run through the embedded forecast system that includes the dynamical solver and physics packages for atmospheric processes (e.g., microphysics, radiation, planetary boundary layer, etc.).

The nonhydrostatic WRF model with the Advanced Research dynamic solver is utilized by the Department of Meteorology and Climatology in the Aristotle University of Thessaloniki, Greece in order to produce operational numerical weather predictions (http://meteo3.geo.auth.gr/WRF/, accessed on 19 June 2024). The modeling system was originally developed, calibrated, and evaluated in the framework of the Wave4Us project [27]. It has been employed in studies of intense weather events in the Mediterranean Sea [28,29,30,31] and the hydrography of the Thermaikos Gulf in northern Greece [32].

The forecasts have been produced in three one-way nested domains that cover (a) Europe, the Mediterranean Sea Basin, and Northern Africa (D01); (b) a large part of the central and eastern Mediterranean, including the whole of Greece (D02); and (c) northern Greece and parts of the surrounding regions (D03) at horizontal grid spacings of 15 km × 15 km, 5 km × 5 km, and 1.667 km × 1.667 km (longitude–latitude), respectively. High-resolution USGS (United States Geological Survey) data at 30 × 30 arcsec have been used for the definition of the land–sea mask, land use, and topography. The initial and boundary conditions of the outer model domain (D01) are based on the operational analyses and predictions of the 12:00 UTC forecast cycle of GFS/NCEP (Global Forecast System/ National Centers for Environmental Prediction) at 0.25° × 0.25° (longitude–latitude). The boundary conditions of D02 (D03) are provided by the forecasts of D01 (D02). The sea surface temperatures are forced by the global analyses of the NCEP at 1/12° × 1/12°, and they do not change during each forecast cycle of the WRF model. The model employs 39 sigma levels with enhanced vertical resolution at the lowest troposphere, and its top is located at 50 hPa. The physical parameterizations include the schemes of Eta Ferrier [33], Betts-Miller-Janjic [34,35], and Mellor-Yamada-Janjic [34] for the microphysics, cumulus convection (only in D01 and D02), and boundary and surface layer processes, respectively. The longwave and shortwave radiations are parameterized by the RRTMG (Rapid Radiative Transfer Model application for Global climate models; [36]) scheme, while the soil processes are represented by the Unified NOAH (NCEP/Oregon State University/Air Force/Hydrologic Research Lab) land surface model [37] using four layers to a depth of 2 m.

The WRF model generates a number of outcomes, such as mean sea level pressure, air temperature, wind azimuth, and speed for the model forecasts and at the anemometer’s height for the observations, as well as the relative humidity. This study, though, focuses on the environmental parameters of 10 m wind speed (WS) and 2 m air temperature (Tem), which are predicted by the domain D02. These two parameters are of primary importance in meteorology, climatology, weather forecasting, and numerous applications (in engineering, agriculture, renewable energy sources, etc.). The wind speed is characterized by significant spatiotemporal variability and abrupt changes (due to fronts, convergence lines, thunderstorms, etc.), especially near the land/sea surface, thus making its accurate prediction a difficult task, even in high-resolution numerical modeling. The 2 m air temperature is also highly affected by the physiographic characteristics (land use, topography, etc.) of the surface. The insufficient representation of these features (for example, due to the horizontal resolution or recent land use changes), especially in regions that they vary significantly, may induce systematic biases to the 2 m air temperature.

2.3. Methodology

The main objective of this work is to present a combined implementation of Extended Kalman Filters and Artificial Neural Networks that aims to improve the forecasting capabilities of the numerical weather and wave models discussed in the previous sections. Two different types of ANNs have been utilized, particularly the Feedforward Neural Networks and the Radial Basis Neural Networks, to determine an exogenous memory parameter based on a predefined history data set (History data set), which contains both the model’s forecasts and the recorded observations.

The parametrized Artificial Neural Networks specify the optimal parameter, which is afterward applied to the Extended Kalman Filter to adaptively estimate the covariances matrices and to produce the enhanced predictions (Forecast data set) of the environmental parameter under study. The developed Hybrid Extended Kalman Filter seeks to decode the prediction error of the simulation expressed as an ANN, (FFNN or RBF NN) the size (number of neurons/clusters) of which is defined through the network’s initial training. Therefore, the HEKF is used as a learning algorithm for the selected network based on a smaller data set (Range data set).

During that process, the HEKF employs the Extended Kalman Filter for eliminating potential systematic errors, in conjunction with parametrized Artificial Neural Networks for pursuing the reduction in the remaining white noise and the accompanying variability in the nonsystematic part of the model error. The suggested methodology is described in the Method’s Diagram (Figure 1)

In every stage of the method, both the model’s predictions and the recorded observations are necessary. The forecasts are available from the numerical models WAM and WRF, while the recorded observations were derived from various stations in Greece (Figure 2). Particularly, the wind speed and the temperature observations were obtained from Stations AUTH and AXIOS located on the campus of the Aristotle University of Thessaloniki (AUTH) and the Axios river delta, respectively (Figure 3). On the other hand, the corresponding observations for the significant wave height were obtained from the Heraklion Station in the area of Crete (Figure 4). The key features of each station are summarized in

Table 1. Station characteristics.

Stations	Area	Latitude	Longitude	Data Type	Forecast Interval
Heraklion	Crete	35.4342	25.0792	Time Series	3 h
AUTH	Thessaloniki—AUTH’s Campus	40.63177	22.95755	Time Series	1 h
AXIOS	Thessaloniki—Axios Delta	40.55157	22.74187	Time Series	1 h

.

3. Adaptive Extended Kalman Filter

Kalman Filtering (KF) is a recursive estimation algorithm that produces an improved prediction for an unknown state vector $x_t$ at time $t$, which is given as the information of an observable vector $y_t$, under the premise that those variables have a linear association. In most real-world applications, though, the model’s dynamics are nonlinear, thus necessitating a different filtering strategy. One such approach is the Extended Kalman Filter, which is a Kalman Filter extension that uses Taylor’s approximation for linearization [38].

To set the framework for the construction of the EKF, consider a nonlinear system characterized by the following state measurement model:

$$x_t=h_{t-1}\left(x_{t-1}\right)+w_{t-1}$$

(2)

$$y_t=g_t\left(x_t\right)+v_t$$

(3)

where $w_{t-1}$ and $v_t$ are Gaussian nonsystematic parts of the model error with covariance matrices $Q_{t-1}$ and $R_t$, respectively, while $h_{t-1}\left(•\right)$ and $g_t\left(•\right)$ are nonlinear, time dependent transition $C^1$ functions. The state space model of Equations (2) and (3) is linearized around the previous or most current state estimate, which can be either the priori estimate ${\widehat{x}}_{t-1}$ or the posteriori estimate ${\widehat{x}}_{t/t-1}$, thereby depending on the functional being considered. However, if the linearization process fails to approximate the linear model accurately, the EKF may diverge [39].

More specifically, the EKF includes the following stages:

• Definitions. Jacobian matrices of the nonlinear transition functions.
• $H_{t-1}=\nabla h_{t-1}|{\widehat{x}}_{t-1}$
• $G_{t/t-1}=\nabla g_{t/t-1}|{\widehat{x}}_{t/t-1}$
• Initialization. For $t=0$, set

$${\widehat{x}}_0=E\left[x_0\right],$$

$${\mathrm{P}}_0=E\left[\left(x_0-E\left[x_0\right]\right){\left(x_0-E\left[x_0\right]\right)}^T\right],$$

$${\mathrm{R}}_0,Q_0$$

where ${\mathrm{P}}_0$ is the state error covariance.

• Iteration. For $t=1,2,\dots$, compute the following:

• Prediction Step:

  $${\widehat{x}}_{t/t-1}=h_{\mathrm{t}-1}\left({\widehat{x}}_{t-1}\right)$$

  $$P_{t/t-1}=H_{t-1}{P}_{t-1}{H}_{t-1}^T+Q_{t-1}$$
• Correction Step:

  $${\widehat{x}}_{t/t}={\widehat{x}}_{t/t-1}+K_t\left[y_t-g_t\left({\widehat{x}}_{t/t-1}\right)\right]$$

  $$P_{t/t}=\left[I-K_t{G}_{t/t-1}\right]P_{t/t-1}$$

where $K_t=P_{t/t-1}{G}_{t/t-1}^T{\left[G_{t/t-1}{P}_{t/t-1}{G}_{t/t-1}^T+R_t\right]}^{-1}$. The $K_t$ is the Kalman Gain and is the filter’s most essential parameter, since it determines how the filter will respond to any potential changes [40]. For instance, in the case that the value of the Kalman Gain is modest, this can be considered as an indication that the uncertainty in the measurements is high; thus, only a small fraction of the observation will be used to predict the new state.

A challenging issue in Kalman filtering is the choice of the covariance matrices, as a poor selection might substantially impair its performance or possibly lead the filter to diverge. Some researchers have employed fixed covariance matrices that are estimated before the application of the algorithm [41,42], whereas others have altered them during the process, thus utilizing the last seven values of $w_t=x_t-x_{t-1}$ and $v_t=y_t-x_t$ [14,43,44].

3.1. Residual-Based Method

This work, however, adopts an approach based on the study of Shahrokh and Huang [45], who proposed an estimating method for adaptably adjusting the covariances matrices by applying a constant (memory) parameter $\alpha$.

In every step of the filtering process, the covariance matrix $R_t$ can be estimated using the following formula:

$$R_t=S_t-G_{t/t-1}{P}_{t/t-1}{G}_{t/t-1}^T$$

(4)

where $S_t$ is the covariance matrix of the innovation, which expresses the difference between the actual measurement and its predicted value based on ${\widehat{x}}_{t/t-1}$. $P_{t/t-1}$ is a $\mathsf{d} imState\times dimState$ matrix, while $G_{t/t-1}$ is a $1\times dimState$ vector, with $dimState$ being the dimension of the state vector $x_t$. One the other hand, $S_t$ is a square matrix, where its size depends on the dimension of the measurement vector $y_t$.

However, Equation (4) does not guarantee that $R_t$ will be positive, as it should be as a covariance matrix; hence, the residual-based adaptive approach is used to ensure positivity. More specifically, the innovation covariance matrix $S_t$ is approximated as follows:

$$S_t \approx \overline{{\mathrm{S}}_{\mathrm{t}}}=E\left[{\epsilon }_t{\epsilon }_t^T\right]=E\left[v_t{v}_t^T\right]-G_{t/t-1}{P}_{t/t-1}{G}_{t/t-1}^T$$

where $R_t=E\left[v_t{v}_t^T\right]$, and ${\epsilon }_t$ is the residual. The residual is determined as the difference between the measurement’s actual value and its projected value based on ${\widehat{x}}_{t/t}$. Therefore, the measurement noise covariance matrix $R_t$ can be computed by the following:

$$R_t=E\left[{\epsilon }_t{\epsilon }_t^T\right]+G_{t/t-1}{P}_{t/t-1}{G}_{t/t-1}^T$$

(5)

To implement (4), the expectation operation ${\epsilon }_t{\epsilon }_t{ }^T$ is estimated by averaging itself, and also—instead of the moving window—a memory factor, $0<\alpha \le 1$, is utilized. Hence, the Equation (5) is transformed to the following:

$$R_t=\alpha R_{t-1}+\left(1-\alpha \right)\left({\epsilon }_t{\epsilon }_t^T+G_{t/t-1}{P}_{t/t-1}{G}_{t/t-1}^T\right)$$

(6)

3.2. Innovation-Based Method

During the EKF implementation, the process noise can be found through the following formula:

$$w_{t-1}=x_t-h_{t-1}\left(x_{t-1}\right)$$

Thus, using the prediction step, the expected process noise can be defined as follows:

$$\begin{array}{l} {\hat{w}}_{t-1}={\widehat{x}}_{t/t}-h_{t-1}\left({\widehat{x}}_{t/t-1}\right) \hfill \\ ={\widehat{x}}_{t/t}-{\widehat{x}}_{t/t-1}={\mathrm{K}}_{\mathrm{t}}\left[y_t-g_t\left({\widehat{x}}_{t/t-1}\right)\right]\hfill \\ ={\mathrm{K}}_{\mathrm{t}}{d}_t\hfill \end{array}$$

where $d_t$ expresses the measurement innovation. Therefore, an approximation of the process noise covariance matrix can be computed using the following equation:

$$\begin{array}{l}\mathrm{E}\left[{\hat{w}}_{t-1}{\hat{w}}_{t-1}^T\right]=\mathrm{E}\left[{\mathrm{K}}_{\mathrm{t}}\left(d_t{d}_t^T\right){\mathrm{K}}_t^T\right]={\mathrm{K}}_{\mathrm{t}}\mathrm{E}\left[d_t{d}_k^T\right]{\mathrm{K}}_t^T\hfill \\ {\hat{Q}}_{t-1}={\mathrm{K}}_{\mathrm{t}}E\left[d_t{d}_k^T\right]{\mathrm{K}}_t^T\hfill \end{array}$$

Similar to the residual-based method, the expectation operation $d_t{d}_t{ }^T$ is estimated by averaging itself, and the same memory factor $\alpha$ is introduced to adaptively compute the variable $Q$ over time, thus using the following formula:

$$Q_t=\alpha Q_{t-1}+\left(1-\alpha \right)\left({\mathrm{K}}_t{d}_t{d}_t^T{\mathrm{K}}_t^T\right)$$

(7)

Equations (6) and (7) are linear expressions of the parameter $a$ that dictate the degree of effect of the prior and subsequent condition. In particular, if the value of the parameter is close to 1, earlier predictions gain more weight, which forces the covariance matrix to take longer to adapt to changes. In their study [45], the value of the (memory) parameter was chosen to be fixed and equal to 0.3.

Τhe selection of the memory parameter, however, cannot be specified without taking into consideration the available data, as critical information required for the study of dynamical systems is lost. To this end, the proposed study implements parametrized Artificial Neural Networks to define that parameter. More thoroughly, the parametrized ANNs are trained for various values of the parameter $a$, and the optimal one is selected to update the covariances matrices based on Equations (6) and (7).

4. Artificial Neural Networks

Artificial Neural Networks are supervised Machine Learning algorithms that draw inspiration from the process of the human brain, thus aiming to discover the fuzzy association between independent input variables and their corresponding dependent targets. The ANNs consist of processing elements, also known as nodes (neurons), and connections. Nodes have their input, which is transformed through a transfer function into output, thus allowing them to communicate with other nodes and to process information to the entire structure. Following how information travels into the network, various ANNs can be constructed [46]. This paper, however, focuses on Feedforward Neural Networks and Radial Basis Function Neural Networks, thus evaluating and comparing their properties as parameter estimators.

4.1. Parametrized Feedforward Neural Networks

Feedforward Neural Networks are artificial models that map input data sets to corresponding outputs and usually consist of multiple layers and neurons that are fully connected in a direct graph. This study presents a three-layer FFNN with one hidden layer of the log-sigmoid function, $\phi \left(x\right)=\frac{1}{1+e^{-x}}$, as an activation function [47] and one linear output layer.

The proposed parametrized Feedforward Neural Network (Figure 5) accepts an input vector $X^i={\left(X_{t+i-m}^i,\dots ,X_{t+i-1}^i,a\right)}^T$, where $i=0,\dots ,N$, which contains the value of the parameter $a$, along with $m$ previous predictions of the numerical weather model, and produces an estimation for the recorded observation at the time $t+i$. More specifically, $N$ is the total number of the available input vectors, and $m=2,\dots ,t-1$ is the size of the model’s previous predictions.

The pFFNN can be trained for various values of that parameter, thus determining which one best portrays the correlation between the input vector and the related output. Here, we choose $a\in \left[0, 1\right]$. Furthermore, as the size of the network is not specified, the suggested structure is also trained for a number of neurons ranging from 5 to 12.

The main objective of this process is to determine the ideal combination of neurons and the value of the (memory) parameter that optimizes the network’s performance, i.e., that minimizes the Mean Squared Error (MSE):

$$MSE=\frac{1}{N}{\displaystyle \sum }_{i=0}^N{(e{r}^i)}^2$$

where $e{r}^i=\left(T^i-\widetilde{o^i}\right)$ is the training error between the $i$th target ($T^i$) and the corresponding output from the pFFNN $\widetilde{o^i}=V \phi \left(D {{\rm X}}^i+b^T\right)+c$. Here, $D$ is a $M\times n$ matrix, with $n$ being the size of the input vector, i.e., $n=m+1$, and $M$ being the number of neurons in the hidden layer. Additionally, $V$ and $b$ are $1* M$ matrices, while $c$ is a scalar.

The determination of these variables is known as the “training” or “learning” stage and is carried out through optimization techniques that seek to minimize the MSE [48]. Several optimization algorithms have been utilized for training ANNs [49,50]. However, regarding the pFFNN’s training, this work implements the Levenberg–Marquardt (LM) method, as the number of the network’s variables is relatively small and also because the MSE is used as a performance index [51].

The Levenberg–Marquardt algorithm combines the steepest descent method with the Gauss–Newton algorithm to update the weights of the network during the training process. To be more precise, the updating rule in the $k$ iteration is given by the following formula [52]:

$$W_{k+1}=W_k-\left(J_k^T{J}_k+\mu {\rm I}\right)J_{k}e{r}_k$$

In this case, we have the following:

• $W_k$ is a column vector that contains all the network’s variables (weights and biases).
• $J_k$ is the Jacobian matrix.
• $e{r}_k$ express the vectorized training error, i.e., $e{r}_k={\left[e{r}_k^0,..,e{r}_k^N\right]}^T$.
• The quantity $\left(J_k^T{J}_k+\mu {\rm I}\right)$ represents an approximation of the Hessian matrix introduced by the LM, i.e., $H=\left(J_k^T{J}_k+\mu {\rm I}\right)$.
• ${\rm I}$ is the identity matrix.
• $\mu$ is a combination parameter that takes only positive values and ensures the reversibility of the $J_k^T{J}_k$ matrix.

Calculating the Jacobian is a critical factor for the LM implementation, as it can become a difficult and time-consuming procedure due to the complexity of the network’s structure (the number of neurons and hidden layers). The corresponding Jacobian matrix for the proposed parametrized FFNN has the following form:

$$J_k=\left(\begin{array}{cccccccccccccc}\frac{\partial e{r}_k^0}{\partial d_{1,1}}& \cdots & \frac{\partial e{r}_k^0}{\partial d_{n,1}} & \dots & \frac{\partial e{r}_k^0}{\partial d_{1,M}}& \dots & \frac{\partial e{r}_k^0}{\partial d_{n,{\rm M}}} & \frac{\partial e{r}_k^0}{\partial b_1}& \dots & \frac{\partial e{r}_k^0}{\partial b_M}& \frac{\partial e{r}_k^0}{\partial v_1}& \dots & \frac{\partial e{r}_k^0}{\partial v_M}& \frac{\partial e{r}_k^0}{\partial b_{out}}\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮\\ \frac{\partial e{r}_k^i}{\partial d_{1,1}}& \cdots & \frac{\partial e{r}_k^i}{\partial d_{n,1}}& \dots & \frac{\partial e{r}_k^i}{\partial d_{1,M}}& \dots & \frac{\partial e{r}_k^i}{\partial d_{n,{\rm M}}}& \cdots & \frac{\partial e{r}_k^i}{\partial b_1}& \frac{\partial e{r}_k^i}{\partial b_M}& \frac{\partial e{r}_k^i}{\partial v_1}& \dots & \frac{\partial e{r}_k^i}{\partial v_M}& \frac{\partial e{r}_k^i}{\partial b_{out}}\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮\\ \frac{\partial e{r}_k^N}{\partial d_{1,1}}& \cdots & \frac{\partial e{r}_k^N}{\partial d_{n,1}}& \cdots & \frac{\partial e{r}_k^N}{\partial d_{1,M}}& \cdots & \frac{\partial e{r}_k^N}{\partial d_{n,{\rm M}}}& \frac{\partial e{r}_k^N}{\partial b_1}& \cdots & \frac{\partial e{r}_k^N}{\partial b_M}& \frac{\partial e{r}_k^N}{\partial v_1}& \cdots & \frac{\partial e{r}_k^N}{\partial v_M}& \frac{\partial e{r}_k^N}{\partial b_{out}}\end{array}\right)$$

wherein this case, we have the following:

• $d_{s,l}$ express the synaptic weight between the $s$ input and the $l$ neuron in the hidden layer. Here, $s=1,\dots ,n$, and $l=1,\dots ,M$.
• $b_l$ and $v_l$ are the bias of the $l$ neuron and the corresponding synaptic weight between this neuron and the output layer.
• $\frac{\partial e{r}_k^i}{\partial d_{s,l}}=-v_l\phi \prime \left(ne{t}_l^i\right)X_s^i$ and $\frac{\partial e{r}_k^i}{\partial b_l}=v_1\phi \prime \left(ne{t}_l^i\right)$, with $ne{t}_l^i$ being the input of the $l$ neuron based on the $i$ training pattern, i.e., $ne{t}_l^i={\displaystyle \sum }_{s=1}^n{d}_{s,l}{X}_s^i+b_l$, where $X_s^i$ is the $s$ input of the $X^i$ input vector.
• $\frac{\partial e{r}_k^i}{\partial v_l}=-v_l\phi \left(ne{t}_l^i\right)$ and $\frac{\partial e{r}_k^i}{\partial b_{out}}=-1$, where $b_{out}$ is the bias of the output layer.

When the Jacobian matrix is calculated, the training process (Figure 6) for the pFFNN can be described by the next diagram [52].

Beyond the training algorithm, another aspect to take into account is overfitting. Overfitting is a phenomenon that occurs when an ANN memorizes the characteristics of the training data set, thus preventing thereby the creation of models that effectively generalize from observed data to unseen data [53]. To address this issue, this study applies the early stopping strategy [54]. During the learning process, the training data set is used to update the network’s weights at each epoch, and afterward, the network’s performance is calculated on the validation data set. If the computed error increases for several epochs, the training process is stopped, and the weights that produced the minimum validation error are the optimal network’s variables (Figure 7). The overfitting handle is a crucial part of the training process.

To summarize the above procedure, for identifying the optimal combination of neurons and memory parameter $a$, the following tables present the major elements of the suggested network (

Table 2. Basic elements of the parametrized Feedforward Neural Network.

ANN	Feedforward Neural Network
Neurons	5 to 12
Activation Function	Log-Sigmoid
Hidden Layers	1
Output Layer	One Linear Output
$\mathrm{Parameter} a$	0:0.1:1
Training Algorithm	$\mathrm{Levenberg}–\mathrm{Marquardt}, \mu =0.01$
Overfitting	Early Stop

), together with Algorithm 1.

Algorithm 1: pFFNN

Create the training data based on History data:

{Inputs, Targets} → {Model’s Forecast, Observations}

for$\mathrm{each} \mathrm{parameter} a$ do

Insert parameter into the Input data

for each Neuron do

Create the network. % number of neurons, initial values for the weights and biases, number of epochs (total and for Early Stop).

while train ≤ maxtrain % For every network, conduct multiple trainings

Split the Input data randomly, and create the Train and the Validation data sets.

Train network using Levenberg–Marquardt backpropagation algorithm and Early Stop.

performance → Network’s performance % MSE.

if performance < Initial Value

Initial Value → performance

Store the best results for each combination. Number of neurons, performance, parameter, train time.

endif

train → train + 1

endwhile

train → 1

endfor

readjust Initial Value

endfor

Define the optimal parameter and number of neurons

if multiple indices in the Total MSE vector show similar results %, their absolute difference is below a threshold position → the index with minimum train time else position → the index with minimum MSE end

Best Neurons → Neuron (position) and Best Parameter → parameter (position)

4.2. Parametrized Radial Basis Function Neural Networks

A major drawback of the application of Feedforward Neural Networks is the gradient-based training algorithms that significantly increase the computational cost. Therefore, a different network structure is necessary. On such architecture is the Radial Basis Function Neural Network [55], which represent a unique type of ANN and have been extensively utilized in various applications [56,57,58] due to their simple, straightforward configuration and training algorithms that are characterized by high accuracy and low computational cost [59].

This study presents a parametrized RBF NN consisting of an input layer, one hidden layer with a Gaussian $\phi \left(x\right)=e^{-x^2}$, radial basis transfer functions [47,60,61], and one linear output layer, as illustrated in Figure 8.

Unlike the corresponding pFFNN structure, which conducts an inner product operation between the weights and the input and adds a bias, pRBF NN computes the Euclidean distance between the input vector and each weight and multiplies the bias. Hence, the so-called net input for the $j$ neuron in the hidden layer is calculated as follows:

$$netinpu{t}_j=\left|\left|X^i-C_j\right|\right|b_j$$

where $\left|\left|\circ \right|\right|$ expresses the Euclidean distance. The weight $C_j$ acts as the center point of the $j$th node, while the corresponding offset variable $b_j$, also known as the width, scales the Gaussian distribution, thus instigating it to stretch or compress. The calculated input is then transformed through the activation function and produces the neuron’s output. Each output is multiplied by the corresponding synaptic weight ($v_j$), and their sum generates the network’s direct response:

$$\widetilde{o^i}={\displaystyle \sum }_{j=1}^K{v}_j\phi \left(\left|\left|X^i-C_j\right|\right|b_j\right)$$

(8)

where number $K$ represents the total number of centers.

In RBF NN modeling, the selection of centers and widths is critical for their successful implementation. More specifically, it is important to define well-distributed centers through the network’s inputs and biases that allow for high overlap among the basis functions. A common established approach for identifying the centers of an RBF network is the k-means++ clustering technique [62].

The k-means++ algorithm is a modification of the k-means [63] that ensures the finding of a set of centers that achieves a $O\left(\log \left(k\right)\right)$ approximation for the ideal center set [64]. However, k-means++ does not automatically dictate the ideal number of clusters, as their number must be specified before the usage of the method; thus, it is difficult to define their optimal size.

To avoid this issue and determine the network’s size, the suggested parametrized RBF NN is trained for several clusters ranging from 10 to 70. Furthermore, as the network receives an input vector $X^i={\left(X_{t+i-m}^i,\dots ,X_{t+i-1}^i,a\right)}^T$, it can also be trained for various values of the memory parameter $a$ specifying the combination that maximizes the network’s performance, i.e., that minimizes the Sum Squared Error (SSE):

$$SSE={{\displaystyle \sum }}_{i=0}^N{\left(e{r}^i\right)}^2$$

(9)

When the centers are established, the width of each neuron cluster can be computed using the following strategy: For each neuron, locate the $r_C$ input vectors from the training set that are closest to the associated center and then compute the average distance between the center and its neighbors [51]:

$$dis{t}_j=\frac{1}{r_c}\sqrt{{\displaystyle \sum }_{p=1}^{r_c}\left|\left|X^p-C_j\right|\right|}$$

where $X^1$ and $X^2$ are the closest and the next-closest input vectors to the center $C_j$. Then, the corresponding width is calculated by the following formula:

$$b_j=\frac{1}{\sqrt{2} dis{t}_j}$$

After the estimation of the hidden nodes and their widths, the linear interconnection of the hidden and output layer permits the use of linear regression techniques for weight calculation rather than gradient descent approaches [65]. To illustrate the training procedure, let us represent the network’s response for all input vectors $X^i$ in Equation (7) as follows:

$$Y=\Phi V$$

where $\Phi =[\phi {\left({{\rm X}}^i,C_j,b_j\right]}_{i,j} \left(i=0,\dots ,N, j=1,\dots ,K\right)$ is the output matrix of the radial functions $\phi$, and $V=\left[v_1,\dots ,v_K\right]{ }^T$ is the vector of synaptic weights that connect the hidden layer with the output layer. Hence, the vector of weights that minimizes the SSE is given by the following:

$$\widehat{V}={({\Phi }^{{\rm T}}\Phi )}^{-1}{\Phi }^{{\rm T}}T$$

with $T$ being the column vector of the measured target values, i.e., $T={\left[T^0,\dots ,T^i,\dots ,T^N\right]}^T$.

As in the case of pFFNNs, the risk of overfitting remains high; therefore, the way to deal with it is crucial for the creation of an RBF network with enhanced generalization capabilities. More specifically, the weight decay or ridge regression strategy [66] is utilized instead of early stopping. The goal of this approach is to constrain the magnitude of network weights rather than the quantity of them by adding a parameter $\lambda >0$ in the Sum-Squared Error, which penalizes large weights. Based on this amendment, the SSE (9) is transformed to the following:

$$SSE={\displaystyle \sum }_{i=0}^N{\left(e{r}^i\right)}^2+\lambda {\displaystyle \sum }_{j=1}^K{v}_j$$

whereas the corresponding optimal vector $\widehat{V}$ is defined as follows:

$$\widehat{V}*={({\Phi }^{{\rm T}}\Phi +\lambda {{\rm I}}_{{\rm K}})}^{-1}{\Phi }^{{\rm T}}T$$

The described process is summarized by the following tables, which present the key elements of the proposed pRBFNN (

Table 3. Basic elements of the parametrized Radial Basis Function Neural Network.

ANN	Radial Basis Function Neural Network
Centers (Clusters)	$10 \mathrm{to} 70 \mathrm{with} step1$= 10
Clustering	K-means++
Activation Function	Gaussian
Hidden Layers	1
Output Layer	One Linear Output
$\mathrm{Parameter} a$	0:0.1:1
Training Algorithm	Linear Least Square
Overfitting	$\mathrm{Regularization}, \lambda =0.06$

), as well as Algorithm 2.

Algorithm 2: pRBFNN

Create the training data based on History data:

{Inputs, Targets} → {Model’s Forecast, Observations}

for$\mathrm{each} \mathrm{parameter} a$ do

Insert parameter into the Input data

for each Cluster do

Create the network. % number of clusters, regularization parameter λ = 0.06.

while train ≤ maxtrain % For every network, conduct multiple trainings

Split the Input data randomly, and create the Train and the Validation data sets.

Define the Centers from the Train data set using K-means++ algorithm, and calculate the widths. Train network using LLS.

performance → Network’s performance % SSE.

if performance < Initial Value

Initial Value → performance

Store the best results for each combination. Number of clusters, performance, parameter, train time.

endif

train → train + 1

endwhile

train → 1

endfor

readjust Initial Value

endfor

Define the optimal parameter and number of clusters

if multiple indices in the Total SSE vector show similar results %, their absolute difference is below a threshold position → the index with minimum train time else position → the index with minimum SSE end

Best Clusters → Cluster (position) and Best Parameter → parameter (position)

5. Hybrid Extended Kalman Filter

Aiming to improve the predictions of the numerical weather and wave prediction models, the Hybrid Extended Kalman Filter is utilized to decode the forecast error, which is defined as the difference between the recorded observation and the corresponding model’s forecast (For) [17]. In particular, a nonlinear function is considered based on the model’s previous estimation, which in time $t$ is given by the following:

$$Ob{s}_t-Fo{r}_t=m_t\left(N_t,Fo{r}_{t-1}\right)+v_t$$

or

$$y_t=m_t\left(N_t,Fo{r}_{t-1}\right)+v_t$$

(10)

where $m\left(•\right)$ is the nonlinear correlation, and $N_t$ is the vectorial parameter that needs to be estimated by the hybrid filter. In this work, $m\left(•\right)$ is expressed as a neural network, which can be an FFNN or an RBF NN; thus, the variable $N_t$ includes the network’s weights and biases. More specifically, if an FFNN is employed, $m_t\left(N_t,Fo{r}_{t-1}\right)=V \phi \left(D,Fo{r}_{t-1},b\right)+c$, and $N_t=\left[D,V,b,c\right]$, while if an RBF NN is applied, $m_t\left(N_t,y_{t-1}\right)=V \widehat{\phi }\left(Fo{r}_{t-1},C,b\right)$, and $N_t=\left[V,b\right]$, where $\widehat{\phi }={\left[\phi \left(Fo{r}_{t-1},C_1,b_1\right),\dots ,\phi \left(Fo{r}_{t-1},C_K,b_K\right)\right]}^T$. Note here that the centers $C$ of the hidden nodes are determined before the application of the hybrid filter through the k-means++ algorithm; hence, they are not included in the vector $N_t$.

Therefore, the Hybrid Extended Kalman Filter is used as a learning rule intending to determine the network’s optimal parameters. Similar modified Extended Kalman Filter algorithms have been frequently utilized as neural network training methods [67,68,69,70] due to their fast and efficient implementation [71]. These approaches are closely connected to sequential versions of the Gauss–Newton algorithm and, unlike the comparable batch variant, they do not need inversion of the approximate Hessian matrix [51].

To illustrate the procedure of the hybrid filter, the state space model must be established. As a measurement equation, the appropriate choice for this study is Equation (10), whereas the selection of the matching state equation is uncertain, as the evolution of the state vector $N_t$ in time is unknown. Hence, we presume that its change would be random and, as a result, we set the system’s nonlinear transition function $h_{t-1}=1$. Thus, the state measurement model is the following:

$$N_t=N_{t-1}+w_{t-1}$$

$$y_t=m_t\left(N_t,Fo{r}_{t-1}\right)+v_t$$

After the dynamic system is defined, the Hybrid Extended Kalman Filter conducts the following steps:

1.

Definition. Jacobian matrix of the nonlinear transition function $m\left(•\right)$.

$$F_{t/t-1}=\nabla m_{t/t-1}|{\widehat{N}}_{t/t-1}$$

2.

Initialization. For $t=0$, set

$$\begin{gathered} \begin{array}{l} {\widehat{N}}_0=E\left[N_0\right],\hfill \\ { \mathrm{P}}_0=E\left[\left(N_0-E\left[N_0\right]\right){\left(N_0-E\left[N_0\right]\right)}^T\right],\hfill \end{array} \\ {\mathrm{R}}_0,Q_0 \end{gathered}$$

3.

Iteration. For $t=1,2,\dots$, compute the following:

3.1.

Prediction Step:

$${\widehat{N}}_{t/t-1}={\widehat{N}}_{t-1}$$

$$P_{t/t-1}=P_{t-1}+Q_{t-1}$$

3.2.

Correction Step:

$$d_{\mathrm{t}}=y_t-m_t\left({\widehat{N}}_{t/t-1}\right)\left(\mathrm{innovation}\right)$$

$$K_t=P_{t/t-1}{F}_{t/t-1}^T{\left[F_{t/t-1}{P}_{t/t-1}{F}_{t/t-1}^T+R_t\right]}^{-1}$$

$${\widehat{N}}_{t/t}={\widehat{N}}_{t/t-1}+K_t\left[y_t-m_t\left({\widehat{N}}_{t/t-1}\right)\right]\left(\mathrm{First\; State\; Correction}\right)$$

$${\epsilon }_{\mathrm{t}}=y_t-m_t\left({\widehat{N}}_{t/t}\right)\left(\mathrm{residual}\right)$$

Update the measurement noise covariance matrix based on (6):

$$R_t=\alpha R_{t-1}+\left(1-\alpha \right)\left({\epsilon }_t{\epsilon }_t^T+F_{t/t-1}{P}_{t/t-1}{F}_{t/t-1}^T\right)$$

$$K_t=P_{t/t-1}{F}_{t/t-1}^T{\left[F_{t/t-1}{P}_{t/t-1}{F}_{t/t-1}^T+R_t\right]}^{-1}\left(\mathrm{Re-Calculate\; the\; Kalman\; Gain}\right)$$

$$P_{t/t}=\left[I-K_t{F}_{t/t-1}\right]P_{t/t-1}$$

Update the process noise covariance matrix based on (7):

$$Q_t=\alpha Q_{t-1}+\left(1-\alpha \right)\left({\mathrm{K}}_t{d}_t{d}_t^T{\mathrm{K}}_t^T\right)$$

$${\widehat{N}}_{t/t}={\widehat{N}}_{t/t-1}+K_t\left[y_t-m_t\left({\widehat{N}}_{t/t-1}\right)\right]\left(\mathrm{Second\; State\; Correction}\right)$$

Applying the algorithm outlined in Algorithm 2, we obtain as $N_t$ the estimated systematic error at time $t$ of the numerical model in use. This value represents the optimal weights and biases of the selected ANN, which produces a direct output added to the model’s forecast for time $t+1$, thus resulting in a more accurate prediction:

$$Improved Fo{r}_{t+1}=Fo{r}_{t+1}+m_t\left(N_t,Fo{r}_t\right)$$

Note here that, during the training process, $N_t$ is a column vector; therefore, each variable of the network should be adapted accordingly. Furthermore, the size of the network is determined based on Algorithms 1 and 2 correspondingly. The hybrid filter’s procedure is summarized in Algorithm 3.

Algorithm 3: Hybrid Extended Kalman Filter
$\mathrm{Obtain} \mathrm{the} \mathrm{optimal} \mathrm{parameter} a$ and the number of hidden notes:		Algorithm 1 or Algorithm 2
Create the training data set for the Hybrid Filter based on the Range data:		{Inputs, Targets} → {Model’s Forecast, Observations}
% if Algorithm 2 is used, then the centers C are determined by k-means++:		$\mathrm{For} \mathrm{Algorithm} 2 \mathrm{the} \mathrm{centers} C$$\mathrm{are} \mathrm{not} \mathrm{included} \mathrm{in} N_t$
Create the initial values for the Filter:		$N_0$$, P_0$$, Q_0$$.R_0$
	for each training data do
	Implement Hybrid Extended Kalman Filter; % if Algorithm 2 is used, then take the absolute value of the widths (b).
	end
$\mathrm{Obtain} \mathrm{the} \mathrm{optimal} \mathrm{state} \mathrm{vector} N_t$
Create the forecasting data based on the Forecast data:		{Inputs, Targets} → {Model’s Forecast, Observations}
Compute the Improved Forecasts

6. Time Window Process

This section presents a time window process application for the various environmental parameters under study. Specifically, the proposed methodology was applied for Significant Wave Height forecasts concerning the area of Heraklion in 2007–2009 and for Wind Speed and Temperature forecasts concerning the Thessaloniki region in 2020. The aim of this procedure was dual. On the one hand, the stability of the suggested hybrid Filter was tested over different regions and time periods, while on the other hand, the utilized ANNs topologies were compared through their computational cost.

Each time window applies Algorithms 1–3 based on specified history data sets and forecasting intervals, which are defined prior to the usage of the method. These hyper-parameters were not randomly selected but were determined via multiple sensitivity tests, and the results are shown in

Table 4. Time Window Process hyperparameters.

SWH	Time Windows	History Data	Range Data	Forecasting Data
Heraklion_2007	4	278	72	12
Heraklion_2008	4	278	72	12
Heraklion_2009	4	400	72	15
WS_Thessaloniki	Time Windows	History Data	Range Data	Forecasting Data
AUTH_2020	3	328	72	24
AXIOS_2020	2	328	72	24
TEM_Thessaloniki	Time Windows	History Data	Range Data	Forecasting Data
AUTH_2020	3	303	72	18
AXIOS_2020	2	303	72	18

.

Studying the above, it is clear that there is not a fixed history data set for the ANN’s training that meets all environmental parameters, and the same goes for the predicting intervals. However, that is not the case regarding the hybrid filter’s training, as the conducted tests show that the optimal value for the number of observations is equal to 72. The proposed Time Window Process is described by the Algorithm 4.

Algorithm 4: Time Window Process
Load the data:		{Inputs, Targets} → {Model’s Forecast, Observations}
Normalize the data:		{Inputs, Targets} → [–1, 1]
Define the History, the Range, and the Forecast data sets
Define the maximum number of time windows:		$Max\_tw$
Initialize the necessary matrix and vectors to store the results
Define the set of parameters:		$\mathrm{parameter} a$ → [0:step:1]
Define the set of Neurons or Clusters depending on the ANN structure:		Neuron → [5,...,12] or Cluster→ [10:step1:70]
	for $k=0,\dots ,Max\_tw$
	$Tr$$\to 1+k:History Data+k$	Training data for the ANN %s one step in time.
	Run the Algorithm 1 or 2	Obtain the optimal parameter and network size.
	$T{r}_1$ →$Tr \left(end\right)+1: Tr \left(end\right)+Range$	Training data for the HEKF.
	$T{r}_2$ →$T{r}_1 \left(end\right)+1:T{r}_1\left(end\right)+Forecast$	Forecasting data for the improved predictions and the evaluation.
	Run the Algorithm 3 Denormalize the data	Apply HEKF, and obtain the optimal vector $N_t$ and improved forecasts. Improved forecasts, corresponding model’s forecasts, and recorded observations.
	Evaluate the method based on $T{r}_2$ and store the results	Bias, RMSE, and Ns indices. Store Time Series, parameter, and network’s size.
	end
Store the aggregate results from every time window

6.1. Evaluation of the Method

The following metrics were employed to assess the method’s efficacy for the different environmental parameters:

• $Bias$ which serves as a measure of variations between observations and model forecasts, is defined as follows:

  $$\mathbf{Bias}=\frac{{{\displaystyle \sum }}_{t=1}^{Forecast}obs\left(t\right)-for\left(t\right)}{Forecast}$$
• RMSE stands for the Root Mean Squared Error and is a metric that quantifies the error variability by describing the model’s overall predictive behavior, and it is computed by the following equation:

  $$\mathbf{RMSE}=\sqrt{\frac{{{\displaystyle \sum }}_{t=1}^{Forecast}{\left(obs\left(t\right)-for\left(t\right)\right)}^2}{Forecast}}$$
• NS, or the Nash–Sutcliffe model efficiency coefficient, varies between $-\infty$ and 1. A perfect match between the model’s forecasts and observations is indicated by a value of 1, but a zero value indicates that the model’s accuracy is only equal to that of the reference model, which in this case is the mean value of the observations $\overline{obs}$. The value of NS is defined as follows:

  $$\mathbf{NS}=1-\frac{{{\displaystyle \sum }}_{t=1}^{Forecast}{\left(obs\left(t\right)-for\left(t\right)\right)}^2}{{{\displaystyle \sum }}_{t=1}^{Forecast}{\left(obs\left(t\right)-\overline{obs}\right)}^2}$$

For every indicator, Forecast is the range of the forecasting interval (number of valid pairs with predictions–observations), $obs\left(t\right)$ expresses the observed value at time $t$, and $for\left(t\right)$ is the improved prediction of the proposed method, or the model’s forecast, at the same time.

6.2. Results

The results derived from the suggested approach are presented in this section. More specifically, the following

Table 5. Evaluation indicators. Average results for the area of Crete. Significant Wave Height.

Average Results Heraklion 2007				Average Results Heraklion 2008
	Bias	RMSE	NS		Bias	RMSE	NS
Model	0.2098	0.2431	−0.8012	Model	0.3972	0.4725	0.6349
FFNN	0.0361	0.1549	0.2746	FFNN	−0.0524	0.3735	0.7521
RBFNN	−0.0595	0.2152	−0.3982	RBFNN	0.1050	0.3572	0.7893
		Average Results Heraklion 2009
			Bias	RMSE	NS
		Model	0.3860	0.4775	−0.4780
		FFNN	−0.3800	0.5343	−0.8811
		RBFNN	−0.1104	0.3499	0.2022

,

Table 6. Evaluation indicators. Average result for the area of Thessaloniki. Wind speed.

Average Results AUTH 2020				Average Results AXIOS 2020
	Bias	RMSE	NS		Bias	RMSE	NS
Model	−2.2196	2.8300	−27.0153	Model	−1.0463	2.3152	−6.4568
FFNN	−0.3183	1.4920	−6.8193	FFNN	−0.4603	2.1559	−5.1157
RBFNN	−0.7639	2.3510	−18.2353	RBFNN	−0.6843	2.1861	−5.0844

and

Table 7. Evaluation indicators. Average results for the area of Thessaloniki. Temperature.

	Average Results AUTH 2020				Average Results AXIOS 2020
	Bias	RMSE	NS		Bias	RMSE	NS
Model	−2.9420	3.1923	−1.8749	Model	−2.4957	2.6977	−1.8940
FFNN	−0.0646	1.4494	0.4343	FFNN	0.0047	1.0186	0.3165
RBFNN	2.2706	4.6103	−3.9203	RBFNN	5.1325	5.3080	−8.8425

present the average values of the evaluation indicators obtained from the various time windows, while the following time series diagrams present the corresponding time series diagrams. In addition, for every case study, the produced computational cost for each ANN structure is provided. Detailed results from each time window are presented in Appendix A, Appendix B and Appendix C.

6.2.1. Results Derived from the Crete Region: Heraklion Station—Significant Wave Height

The results from the Crete region show that the suggested methodology improved the predictions of the WAM-cycle 4 numerical model considerably. In particular for 2007, the hybrid filter based on the FFNN produced better results, as the minimum enhancement noticed was 36% and concerned the RMSE index, while the corresponding improvement based on the RBFNN structure was only 11%. Similar conclusions can be extracted by studying the time series diagram (Figure 9—Crete 2007), as the FFNN’s improved forecasts satisfactorily describe the distribution of recorded observations.

Moving on to the next period, it is unclear which combination offered the best outcomes. The highest improvement reported was 86% in the Bias index after implementing the FFNN structure, while the biggest enhancements noticed for the RMSE and NS were 24% and 23%, respectively, through the RBFNN. Similar indications can be obtained from the time series diagram (Figure 9—Crete 2008), as the application of the hybrid filter, based on both structures, produced outputs closer to the recorded observations; thus, it is uncertain which architecture is the proper choice. However, if the time efficiency (Figure 10) is taken into consideration, then the RBFNN is the suitable selection, as its computational cost was almost 32 s.

During 2009, the RBFNN was the optimal choice, since it improved all the evaluation indicators noticeably and also because the FFNN structure produced lower-quality outcomes compared to the model. That can be seen, additionally, through the time series (Figure 9—Crete 2009), where the implementation of the RBFNN hybrid filter led to an adequate convergence of the modeled PDFs to the observations.

6.2.2. Results Derived from the Thessaloniki Region: Stations AUTH and AXIOS—Wind Speed

The results derived from the Thermaikos Gulf, regarding the wind speed, showed that the hybrid filter based on the FFNN improved the forecasts of the WRF model greatly. More specifically, in Station AUTH, the Bias index decreased by 86%, while the values of the RMSE and NS indicators enhanced by 47% and 75%, respectively. Moreover, the distribution of improved predictions (Figure 11—AUTH) exhibited the same tendency, as their morphology in a range of 72 observations was identical to that of the recorded observations. The first time corresponds to the 6th forecast hour (due to WRF model spin-up), and the following times are in hourly intervals (the same applies to the air temperature diagrams).

On the other hand, the improvement at Station AXIOS was less significant. In this scenario, the Bias dropped by 56%, but the RMSE and NS improved only by 7% and 21%, respectively. The same pattern is visible in the time series diagram (Figure 11—AXIOS), as between the 27th and 48th observation, the improved forecasts failed to approximate the distribution of the recorded observations successfully. Furthermore, it should be noted that the consistent behavior of the FFNN hybrid filter considerably rose the computational cost (Figure 12) of the procedure, since the total time was 53.4 s, whereas the corresponding time for the RBFNN was 15 s.

6.2.3. Results Derived from the Thessaloniki Region: Stations AUTH and AXIOS—Temperature

The implementation of the proposed approach for the environmental parameter of temperature considerably boosted the forecasting capabilities of the WRF model. Studying the time series diagrams (Figure 13) from Stations AUTH and AXIOS, it is observed that, once more, the hybrid filter based on the FFNN generated estimations that matched the available observation perfectly. Additionally, the enhancement noticed in the assessment indicators is of the same magnitude. Particularly, in Station AUTH, the Bias index showed the highest improvement of 98%, while the RMSE showed the lowest improvement of 55%. The corresponding indicators at station AXIOS improved by 99% and 62%, respectively, while the NS index enhanced by 83%. Although, it should be highlighted here that the selection of the FFNN hybrid filter caused a major decrease in the time efficiency of the process, since the total computational cost (Figure 14) exceeded 78 s.

7. Conclusions

A hybrid optimization model for various environmental forecasts has been proposed in this study by utilizing an Extended Kalman Filter enhanced by parametrized Artificial Neural Networks. The suggested system seeks the following:

• To eliminate the systematic forecast error in the direct outputs of the Numerical Weather and Wave Prediction models WRF and WAM;
• To reduce the remaining nonsystematic part of the error and the related forecast uncertainty.

The initial objective was achieved by the implementation of the Extended Kalman Filter, as well as the second goal by the parametrized Artificial Neural Networks. More thoroughly, two different ANN structures were used to define an exogenous parameter, which was employed by the EKF to adaptively estimate the process noise and the measurement noise covariance matrices.

The proposed model was evaluated through a Time Window Process for Significant Wave Height forecasts obtained from the numerical wave model WAM and for wind speed and temperature forecasts derived from the numerical non-hydrostatic system WRF. For each case study, a Feedforward Neural Network and a Radial Function Neural Network were trained for various values of that parameter based on a specified history data set, and the optimal one was selected. Afterward, the Hybrid Extend Kalman Filter was applied, using the selected parameter, to produce improved forecasts for the environmental parameter under study.

The obtained results showed that the suggested hybrid filter greatly improved the statistics of the weather numerical models. Specifically, the systematic errors recorded were practically eliminated, as the Bias index decreased at a percentage of more than 75%, while the RMSE indicator and, therefore, the associated forecast uncertainty was reduced by almost 35%. Moreover, in most case studies, the hybrid filter based on the FFNN structure produced more consistent and balanced final forecasts compared to the corresponding RBFNN’s predictions; thus, it can be considered as an optimal architecture. However, it should be noted that the FFNN’s selection worsened the time efficiency of the system, since the overall computational cost exceeded 320 s.

Furthermore, the Hybrid Extended Kalman Filter based on parametrized ANNs exhibited a stable behavior independent of forecasting horizons and environmental parameters, thus avoiding the limitations of traditional Kalman Filters, which replace initial systematic deviations with similar over- and underestimation periods, thus resulting in reduced mean error values but no significant increase in predictions.

Overall, the combined application of Artificial Neural Networks and Extended Kalman Filters managed to enhance the predictive capacity of the NWWP models in use, regardless of the environmental parameter under study, thus providing a reliable and effective tool for improving weather forecasts. In future stages, the suggested hybrid model can be expanded to similar simulations in domains like economy and biology, where the optimization afforded by the Hybrid Extended Kalman Filter can aid to tailor predictions to specific demands.