Improving Weather Forecasts for Sailing Events Using a Combination of a Numerical Forecast Model and Machine Learning Postprocessing

Abstract

Accurate predictions of wind and other weather phenomena are essential for making informed strategic and tactical decisions in sailing. Sailors worldwide utilize current state-of-the-art forecasts, yet such forecasts are often insufficient because they do not offer the high temporal and geographic resolution required by sailors. This paper examines wind forecasting in competitive sailing and demonstrates that traditional wind forecasts can be improved for sailing events by using an integration of traditional numerical modeling and machine learning (ML) methods. Our primary objective is to provide practical and more precise wind forecasts that will give sailors a competitive edge. As a case study, we demonstrate the capabilities of our proposed methods to improve wind forecasting at Lake Kinneret, a popular sailing site. The lake wind pattern is highly influenced by the area’s topographic features and is characterized by unique local and mesoscale phenomena at different times of the day. In this research, we simulate the Kinneret wind during the summers of 2015–2021 in up to one-kilometer resolution using the Weather Research and Forecasting (WRF) atmospheric model. The results are used as input for convolutional neural network (CNN) and multilayer perceptron (MLP) ML models to postprocess and improve the WRF model accuracy. These advanced ML models are trained using training datasets based on the WRF data as well as real data measured by the meteorological service, and subsequently, a validation process of the trained ML model is performed on unseen datasets against site-specific meteorological service observations. Through our experimental analysis, we demonstrate the limitations of the WRF model. It uncovers notable biases in wind direction and velocity, particularly a persistent northern bias in direction and an overestimation of wind strength. Despite its inherent limitations, this study demonstrates that the integration of ML models can potentially improve wind forecasting due to the remarkable prediction accuracy rate achieved by the CNN model, surpassing 95%, while achieving partial success for the MLP model. Furthermore, a successful CNN-based preliminary forecast was effectively generated, suggesting its potential contribution to the future development of a user-friendly tool for sailors.

1. Introduction

In the realm of atmospheric science, wind-pattern forecasting holds a pivotal role in understanding and predicting weather phenomena. The atmosphere is characterized by variables such as temperature and humidity, which influence atmospheric pressure and wind movement. Wind is generated by air flowing from high to low pressure, and its direction is influenced by the Coriolis effect. Local wind patterns are affected by topography, which can cause wind enhancement, deceleration, or changes in direction. Downslope winds are caused by air being forced down a slope, while sea breezes are caused by temperature differences between water and land. These local wind patterns can have significant impacts on wind velocity and direction.

Numerical weather prediction (NWP) uses physical simulation models to forecast weather. The main stages of NWP are data synthesis, simulation, postprocessing, and data assimilation. Improvements in NWP have come from better measurements and increased processing power. However, long-range and short-range weather forecasting remains challenging due to the chaotic nature of meteorological systems. For competitive sailing, high-resolution NWP models are used for site characterization and short-term forecasting. Site characterization helps sailors prepare for events, while short-term forecasting provides information on wind direction and velocity.

Sailing is a competitive sport that requires physical fitness, technical skill, and tactical decision-making. Success in sailing depends on accurate wind forecasts, as wind velocity and direction affect boat speed and route selection. High-resolution wind forecasts are particularly important for sailing events, as they can help sailors make informed decisions about their race strategy. NWP models are commonly used for sailing forecasts, but they can be limited in their ability to provide the necessary spatial and temporal resolution. As a result, new prediction methods are being developed to meet the specific requirements of sailing events.

Machine learning (ML) is a field of computer science that enables computers to learn from data without being explicitly programmed. ML algorithms construct mathematical models based on sample data, allowing them to make predictions or decisions. Deep learning, a type of ML based on artificial neural networks, has gained prominence due to its ability to identify patterns and relationships in complex data. ML techniques are finding increasing applications in natural science, including tasks such as language processing, speech recognition, image analysis, and disease outbreak prediction.

Accurate wind forecasts are crucial for sailors’ strategic and tactical decisions. To enhance wind predictions, the research employs machine learning (ML) techniques, particularly convolutional neural networks (CNNs) and multilayer perceptron (MLP). These models are trained using data from the Weather Research and Forecasting (WRF) atmospheric model and real data from the Israeli Meteorological Service (IMS) to improve wind velocity and direction predictions on Lake Kinneret. The research aims to provide improved wind forecasts for competitive sailing, where accurate weather information can give sailors a competitive advantage.

In this research, we aimed to improve the temporal prediction of wind velocity and direction at a specific point in Lake Kinneret. We employed an ML model to postprocess the WRF atmospheric model predictions to improve prediction accuracy. The WRF atmospheric model output was used as an input for the training process of the ML model in conjunction with the site observations (provided by the Israeli Meteorological Service, IMS) as the validation set. This methodology was applied to two ML models (CNN and MLP).

The remainder of this paper is structured as follows: Section 2 describes related background and prior work, Section 3 presents our combined numerical forecast model and ML method, Section 4 showcases the experimental results, and Section 5 summarizes our conclusions.

2. Related Background

This section describes the related background to our study and reviews related work. We describe wind physical characteristics, wind forecasting methods, and competitive sailing principles and provide an overview of ML models. Last, we provide an overview of related works.

2.1. Wind Characteristics

The atmosphere is characterized by many variables. Temperature and relative humidity particularly control the atmospheric pressure gradient to a large extent [1]. According to Pascal’s Law, air flows from higher to lower pressures to equilibrate, which results in air movement (i.e., wind). The parameters controlling wind are affected by many factors, the major ones being latitude-dependent radiation and geographic differences in heat fluxes. When flowing from high to low pressure, air is diverted by the Earth’s rotation and forms cyclonic weather systems, where air movement is parallel to the pressure isobars form. This global process is attributed to the Coriolis effect [2].

Unlike synoptic system-based winds, which are generated by global processes and sprawl over a large scale (1000 km or more), local wind patterns are highly dependent on land topography [3,4]. For instance, the wind shadow effect causes a reduction in wind velocity and increased fluctuations in the velocity airflow, usually due to land obstructions such as topography, flora, and urbanization [5]. On the contrary, narrow valleys or building avenues can result in forced air movements, a phenomenon known as the wind funnel effect that increases wind velocity [6]. In addition, mountainous topography is related to downslope wind, which results in air forced down the slope. The air is compressed by the increasing air pressure toward the lower altitude and, therefore, is accelerated. The downslope wind is usually caused by warmer and drier conditions due to the wind’s enhancement [7]. Another common and important phenomenon is the sea breeze, a local-scale effect that occurs along coastlines. The sea breeze is caused by temperature differences between a body of water and its adjacent land. During the day, the water, characterized by higher specific heat, is cooler than the land, which results in a landward wind breeze due to the local pressure gradient and air [8].

2.2. Wind Forecasting

Currently, weather predictions are mostly conducted by numerical analysis using physical simulation models. The major global models used are the GFS (Global Forecast System) and ECMWF (European Centre for Medium-Range Weather Forecasting), whereas local simulations are conducted using other models such as WRF (a regional model that is used in this research) and ICON (Icosahedral Nonhydrostatic). NWP relies on physical conservation laws, specifically the ideal gas law. It calculates the future state of weather systems by computing spatial and temporal derivatives of the weather system [9]. Numeric forecasting is a cyclical process, and its main stages are as follows:

• Synthesis of measured environmental and simulated data that describes the state of the atmosphere is used as input (initial and boundary conditions) for simulation software.
• Four-dimensional (3D space + time dimension) simulation of the future atmospheric state is performed using atmospheric model simulators.
• Postprocessing of simulation output produces a forecast.
• Data assimilation of model output and newly measured atmospheric properties are used to create initial conditions for the next run of the model.

The forecasting cycle has been used since the 1960s [10,11]. Since then, immense improvements in predictability have been achieved by better measurement of atmospheric quantities [8] and increased processing power, enabling high-resolution simulation [12]. Atmospheric modelers focused most of their attention on stage 4 of the prediction [10,13,14,15,16] cycle (data assimilation). However, even this improved weather predictability does not meet operational forecasting needs. Currently, the ability to perform both long-range weather prediction and short-range weather forecasts or nowcasting remains limited. Meteorological systems are chaotic due to atmospheric instability, which introduces many variables and degrees of freedom into the forecasting equations [17,18].

Small-scale forecasting plays an ever-greater role in competitive sailing. The use of very high-resolution data, attributed to current simulation tool capabilities, has allowed models to exhibit higher performance. Prior research using super-high-resolution WRF simulation improved wind-direction prediction but struggled with reliability on specific days due to initialization and lateral boundary conditions [19,20,21].

Forecasting for sailing events is divided into site characterization, which aids sailors in event preparation by understanding typical weather conditions [22,23,24], and high-resolution short-term forecasting [20,24]. Pezzoli and Bellasio [23] recently proposed a method that groups typical days at a venue into clusters, aiding tactical decisions through a call book based on these clusters.

2.3. Competitive Sailing

Sailing, a globally popular competitive sport since the 1900 Olympics, is also a common recreational activity. Sailing competitions involve multiple races, as illustrated in Figure 1, with scores based on finishing order, and courses vary in layout and scoring systems. Each course is composed of several legs with different angles relative to wind direction. The upwind leg is crucial in sailing competitions because it requires zigzagging across the course, which spreads competitors and exposes them to varying wind conditions. Sailors must choose the best route to maximize their success, as upwind legs account for 60% of racing time. Sailing success depends on physical fitness, technical ability, and tactical skills, including strategy, tactics, boat speed, and boat handling. Accurate wind forecasts are crucial due to the impact of wind velocity on equipment trimming and upwind route selection [20,25,26,27].

The specific requirements of high temporal and spatial resolution forecasting for sailing events are similar in other fields, such as wind farms [28], marine operations [29], and artillery [30]. However, these requirements are inherently hard to satisfy using the standard numerical weather prediction process [31,32], which calls for the implementation of new prediction methods. Research on sailing event forecasting has focused on improving numerical models and providing athletes with local weather information [26,33]. Dedicated weather forecast teams are common in major sailing events to support decision-making and race strategy preparation.

2.4. Machine Learning

Machine learning (ML) is a field of computer science that uses algorithms that improve automatically through experience [34]. It is a subdomain within the broader field of artificial intelligence (AI) that encompasses various challenges such as classification, regression, clustering, and anomaly detection. Machine learning algorithms build a mathematical model based on sample data, known as training data, to make predictions or decisions without being explicitly programmed to do so [35]. Over the last decade, deep learning, a branch of machine learning using artificial neural networks, has excelled in applications ranging from natural language processing to predicting local pandemic outbreaks, significantly enhancing its use in natural sciences [36,37,38,39,40,41]. This work employs deep learning models, including CNN and MLP, which leverage multiple layers and activation functions like ReLU, Sigmoid, and Tanh to extract features and process complex input data relationships for accurate predictions or classifications [42,43,44].

An MLP model is a feedforward artificial neural network that consists of multiple layers of nodes, or neurons, arranged in a directed graph. The layers are typically organized into an input layer, one or more hidden layers, and an output layer. Each neuron in the MLP model takes a weighted sum of its inputs and applies a non-linear activation function to produce an output. The weights of the connections between the neurons are adjusted during the training process to minimize a loss function. The mathematical description of the MLP model can be expressed as follows:

• Input layer: The input layer consists of n input neurons, each of which receives an input value xi where 1 ≤ i ≤ n.
• Hidden layers: The MLP model can have one or more hidden layers. Each hidden layer consists of m neurons. The output of the j-th neuron in the l-th hidden layer is given by the following:

$$h_j^l=\sigma ({\sum }_{i=1}^n{w}_{ij}^l{x}_i+b_j^l)$$

(1)

where w^l_ij is the weight of the connection between the i-th input neuron and the j-th neuron in the i-th hidden layer, b_j^l is the bias of the j-th neuron in the i-th hidden layer, and $\sigma$ is the activation function.

3.

Output layer: The output layer consists of k output neurons. The output of the k-th neuron can be expressed according to Equation (1).

A CNN is a different type of feedforward artificial neural network that is specifically designed to process data that have a grid-like structure, such as images. CNNs are commonly used for image classification, object detection, and other computer vision tasks. The mathematical description of a CNN model can be expressed as follows:

• Input layer: The input layer of a CNN model typically consists of a multi-dimensional array of values representing the input.
• Convolutional layers: CNNs use convolutional layers to extract features from the input image. A convolutional layer consists of a set of filters, which are small matrices of weights. The filters are convolved with the input tensor, producing a feature map. The output of the l-th convolutional layer is given by the following:

$$F^l=\sigma (w^l\times X^{l-1}+b^l)$$

(2)

where $w^l$ is the set of filters in the l-th convolutional layer, $X^{l-1}$ is the output of the previous layer, $b^l$ is the bias term for the l-th convolutional layer, $\times$ denotes matrix multiplication, and $\sigma$ is the activation function.

3.

Pooling layers: Pooling layers are used to reduce the dimensionality of the feature maps produced by the convolutional layers. Pooling layers typically use a max pooling or average pooling operation.

4.

Fully connected layers and an output layer: The final layers of a CNN model are typically fully connected layers and an output layer, both of which are similar to the layers in an MLP model.

2.5. Prior Works

In prior works, ML methods have been introduced into the numerical weather prediction cycle [45], and attempts were made to use ML methods in different stages of numerical weather prediction. Some examples include improving remote sensing of the environment [46], data assimilation [47], and subgrid scaling of physical processes that were previously performed using approximation formulas [48]. In 2019, the first attempt was made to completely abandon physical simulation and use an ML model for weather prediction [36].

One of the earliest implementations of ML in the field of numerical weather prediction involved Kalman filters [49,50]. The filter, which improves forecasting in non-linear forecasting by performing non-Gaussian estimation of the system, was used to improve the observation data and data assimilation [51,52]. Later, when multimodel ensemble forecasting was developed, ensemble Kalman filters nearly became the norm for the representation of forecast uncertainty.

Recent examples of ML methods for weather forecasting include analyzing the wind signal of the prior minute by using a neural network to predict wind shifts during the following minute in order to support competitive sailors [53]. Precise short-term wind forecasting was performed using a measured wind signal decomposed using a wavelet transform as an input for a fuzzy ARTMAP neural network (which is superior to generic neural networks in learning new features while forecasting); the forecasted signal was then recomposed to wind signal forecast using wavelet transform [54]. Estimation of a nondirect model output (road surface temperature) by using quantile regression forests on multiple prediction models’ output was suggested by Kirkwood et al. [55].

These earlier implementations of machine learning in weather predictions caught the attention of both AI moguls such as Google and Huawei and weather prediction agencies such as the ECMWF, which recently published their versions of an AI weather forecast. Google implemented a unique data-interpolation method on 0.25° weather reanalysis data to create the input for a 16-layer graph neural network in an encoder–decoder configuration to create a six-hour global forecast (which can then be used as input to create longer-range forecasts). That approach achieved more accurate forecasts than prior operational forecasts both in terms of everyday prediction and in terms of extreme weather prediction [56]. Huawei implemented a standard encoder–decoder design for vision transformers on reanalysis data with an Earth-specific positional bias version of the shifting window that also performed better than prior NWP methods [57]. Both Google and Huawei were also able to significantly reduce computer resources when performing their forecasts.

3. Combined Numerical Forecast Model and Machine Learning

In this section, we present our integrated numerical forecast model in conjunction with the ML approach. In the present study, our objective was to enhance the temporal forecasting of wind velocity and direction at a designated location within Lake Kinneret. To achieve this goal, we implemented an ML framework to refine predictions derived from the WRF atmospheric model, thereby augmenting prediction accuracy. As illustrated in Figure 2, the output of the WRF atmospheric model served as the input for the training process of the ML model, incorporating site-specific observations obtained from the IMS as the validation dataset. We begin this section by describing the research site, followed by a detailed discussion of the WRF simulation model, the training and validation dataset, and finally, our proposed ML model.

3.1. Research Site

Lake Kinneret, also known as the Sea of Galilee and shown in Figure 3, is the world’s lowest freshwater lake, with an altitude range of 208–215 m below sea level. It is located in the northern part of the Dead Sea rift, between the Galilee and Golan Mountain regions (N 35°35, E 32°50). The average daily annual air temperatures range between 17 and 18 °C in the winter and up to 40 °C during the summer, whereas the lake’s average surface water temperatures are 15–16 °C in the winter and reach 31 °C in the summer. The Kinneret’s surface area is about 166 km², with a maximum length and width of 22 and 12 km, respectively [58,59,60].

The Kinneret wind system during the summer is characterized by strong westerly winds in the afternoons, which makes it a popular sailing site. The daily wind cycle starts with the morning’s lake breeze, from approximately 07:00 to 11:00, blowing from the lake toward the shores and usually light, up to 4–5 m/s [61]. The mechanism that drives the afternoon winds is the westerly Mediterranean Sea breeze, which develops in the morning to early noon and blows toward Lake Kinneret. The high prevalence of the Mediterranean Sea breeze is a local phenomenon that relies on the stable presence of the Persian trough that dominates during summer [62]. When the westerlies originating from the Mediterranean reach the lake, they blow significantly stronger, reaching velocities of 8–15 m/s [63,64].

The wind velocity enhancement process in the Kineret is substantial, and whereas seasonal average winds at the Mediterranean are in the 4–6 m/s range [63], they increase approximately threefold when reaching the Kinneret. Due to the lake’s shape, the westerly wind reaches the lake first at the west bank, at the area of Vadi Arbel, where the strongest winds are experienced. The wind gradually decreases toward the middle of the lake and the eastern coast.

Three factors affect the temporal and geographic distribution of the Kinneret’s wind. The primary factor is the downslope wind. The westerly Mediterranean breeze (~45 km west of the Kinneret) climbs the Galilee Mountains and then blows downward toward the rift. This process is intensified by differences in air temperature between the colder, humid, and mountain-descending air and the warm, dry rift air. Furthermore, the pressure gradient is intensified by the afternoon deepening of the Persian trough [65]. The second factor is a wind funnel mechanism: The wind is strengthened by the Galilee valleys, most of which are oriented east–west [66,67]. The third factor is the inversion layer height, which is a hot air layer located above the surface air. The inversion layer acts as a top barrier for the westerly breeze, compressing the air above the higher surface at the mountain ridges and increasing wind velocity.

The altitude of the inversion layer is a determinant of Kinneret wind velocity in two opposing mechanisms. Whereas an inversion layer of 700–800 m (or more) above sea level will dismiss the compressing effect, an inversion of 400 m or less may block the westerly winds at the mountain ridges and prevent the wind from reaching the lake. Sometimes, the height of the inversion layer is such that wind is formed over the westerly coast but is blocked by the inversion layer at the Golan Heights on the eastern coast, which causes the wind to change direction northward and/or southward toward the rift, instead of resuming the westerly direction [68].

3.2. WRF Physical Modeling

The WRF model version 4.0 was installed and operated using UEMS (https://www.wrfems.info/index.php, accessed on 26 March 2024) version 19.8 to produce atmospheric forecasts. UEMS is a collection of scripts that simplifies model configuration, defines new domains, and downloads input data for the WRF preprocessor (WPS). WRF is a next-generation mesoscale numerical weather prediction system designed for both atmospheric research and operational forecasting applications. Since the aim of this study was to test a postprocessing method, we used the default WRF configuration. The use of a default WRF configuration results in lower accuracy of WRF predictions, but it offers the advantage of saving optimization efforts and allows for testing the robustness of ML postprocessing. To achieve a high-resolution forecast (illustrated in Figure 3b), we ran three nested domains in increasing resolutions (9, 3, and 1 km grid spacing). We used one-way nesting, where information only flowed from the outer domain to the inner domain [69].

Model runs were conducted in 2021, and the initialization and boundary conditions were extracted from the GFS 0.25 deg archive [70]. The WRF was run every summer day from June to August of the years 2015–2021. The model was initialized once per day at midnight and ran for 18 h. The first 10 h were used as a spin-up period, so ML modeling was conducted from 10:00 to 18:00. The ML model was trained with only the input from the 9 km resolution domain (WRF9) and the 1 km resolution domain (WRF1). The attributes of WRF1 and WRF9 resolutions are summarized in

Table 1. WRF domains used by the ML models.

Parameter	WRF9	WRF1
Model shape (x, y, z)	89 × 89 × 45	78 × 51 × 45
Area (km2)	69,696	3850
Output intervals	3 h	30 min
Number of samples	1340	7600

.

3.3. Building the Training and Validation Datasets

The datasets utilized for ML training and validation comprise data generated by the WRF numerical model, serving as input features for the ML models, and the IMS observations at Kefar Nahum (32.88329, 35.57923) as labels. This measurement station is located very close to the coastline and was chosen because the buoy station suffers from frequent malfunctions. Because the high-resolution domain output frequency was 30 min and the low-resolution domain output frequency was once per three hours, we interpolated the coarse-resolution simulation to 30 min output resolution using linear interpolation. This adjustment was made to allow for a quality comparison between simulations and ML models. For the ML model inputs, we used the meridional (east–west) and zonal (north–south) 10 m wind vectors from the WRF output, denoted as U10 and V10, respectively. The data labels (observations) were taken from the IMS Kfar Nahum meteorological station, shown in Figure 3, by the IMS data repositories. This is the closest station to the Kineret sailing competition area, and it was chosen because it has the largest and most reliable dataset of the Kineret stations. The complex topographic area of this station is advantageous when trying to develop a local-scale forecasting method. The sampling rate at the IMS station was six samples h⁻¹. The U10/V10 and IMS observations from the Kfar Nahum were downscaled to model the output interval using a moving average. The observed data were randomly partitioned into two datasets for training and validation, with a ratio of 1:4 in favor of the training dataset. Because we employed classification ML models, we began by converting the IMS observations into five equally sized wind-interval classes. This resulted in extreme wind conditions being represented by only a few samples. In a later stage, we further refined the approach by converting the IMS observations into equal bins of classes, significantly improving predictability.

3.4. Machine Learning Models

In this study, we investigate two ML models, convolutional neural network (CNN) and multilayer perceptron (MLP), which are commonly employed in image-classification tasks. We then present the ML models, followed by the hyperparameter optimization.

3.4.1. ML Models

In this study, we explore the CNN and MLP models for individually forecasting the meridional and zonal components of wind speed (U and V, respectively). Each ML model utilizes both U and V components of the atmospheric model as input and is evaluated on both high- and low-resolution WRF data. WRF data were extracted for the grid point closest to the labeling point at each resolution. For WRF1, at 35.583° E, 32.883° N, located 351 m east of Kfar Nahum, and for WRF9, at 35.610° E, 32.901° N, located 3570 m northeast of Kfar Nahum.

The use of CNN for weather prediction and parameterization is similar to the method implied by Larraondo, Inza, and Lozano [71], which was used to predict rainfall in different locations. Our CNN model architecture, illustrated in Figure 4a, consists of multiple convolution layers originally designed to process image data effectively. The first CNN model layer consists of a 2D convolutional layer structure with 64 output channels, each employing a 3 × 3 convolutional kernel, followed by a rectified linear unit (ReLU) activation function. A max pooling with a 2 × 2 pool size and stride of 2 × 2 was also applied. The second layer consists of a 2D convolutional layer structure with 128 channels, employing a 3 × 3 convolutional kernel, a ReLU activation function. A max pooling with a 2 × 2 pool size and stride of 2 × 2 was subsequently applied. Finally, the output feature maps are flattened to a 1D vector, producing 61,952 output shapes for WRF9 and 29,184 for WRF1 input data. The last layer is a fully connected layer that employs a SoftMax activation function that sorts the categorial classification into five classes. Figure 4b illustrates the MLP architecture employed in this study, comprising several dense, fully connected layers for effective feature extraction and classification. Each layer consists of multiple perceptrons, which are the fundamental building blocks of the network. A perceptron operates on multiple input features, each appropriately weighted; it aggregates these inputs and applies an activation function to generate an output. During the learning process, the perceptron undergoes weight adjustments to minimize the disparity between its output and the desired target output, facilitating the acquisition of patterns and the ability to make predictions. The first layer in the MLP model is a fully connected dense layer, with the number of perceptrons set to the size of the input shape’s dimension (Table 1) and a sigmoid activation function. The second and third dense layers comprise 32 and 10 perceptrons, respectively, and use sigmoid activation. Similar to the CNN, the last layer consists of five fully connected SoftMax activation functions used to classify the model output into five categories. To regularize the model and avoid overfitting, L2 regularization with a penalty of 0.00001 is applied in every dense layer [72].

3.4.2. Hyperparameters Optimizations

The hyperparameters of each model, including the learning rate, number of epochs, activation function type, and layer sizes, were optimized using grid search optimization [73]. This method systematically explores a predefined set of hyperparameter values, creating a grid of all possible combinations. Each combination is then evaluated for model performance, allowing the identification of the optimal set of hyperparameters that maximize prediction accuracy. The exhaustive grid search approach is effective for fine-tuning models, and the selected hyperparameter values from the 324 examined combinations are presented in

Table 2. Value ranges for model hyperparameters used through the grid search process.

Hyperparameter	Value
Learn rate	0.0001, 0.0002, 0.0003, 0.0004
Epochs	100, 300, 500
Activation function	Sigmoid, ReLU, Tanh
CNN number of filters in 1st layer	64, 128, 256
CNN number of filters in 2nd layer	64, 128, 256
MLP dense 2nd layer size	16, 32, 64
MLP dense 3rd layer size	5, 10, 20

and illustrated in Figure 4. Notably, the best prediction accuracy was achieved with a learning rate of 0.0001 and 100 epochs.

4. Simulation and Experimental Results and Discussion

This section describes our experimental analysis and discusses the results obtained. We begin by presenting the tools used in our experimental environment, followed by an evaluation of the prediction accuracy of the WRF model. Lastly, we provide the experimental results of our ML models.

4.1. Simulation and Experimental Environment Tools

The analysis environment for the physical model and the plotting tools in this study was implemented using Python [74]. The numerical foundation for the data-frame structure and computations utilized the NumPy library [75]. Further analysis was conducted using the Pandas [76] and SciPy libraries [77]. The NetCDF4 library facilitated operations such as learning, reading, writing, and modifying the multidimensional NetCDF output from the WRF. Experimental visualizations were generated using plots, graphs, and charts derived from the Matplotlib library [78]. Machine learning models were constructed using the Keras framework using TensorFlow and the Sklearn library for classification and preprocessing [79,80]. Statistical tests were performed using SciPy libraries, JMP, and Excel.

4.2. WRF Simulation Analysis

Our analysis of both the WRF atmospheric model predictions and wind observations indicates that wind speeds above 10 kts predominantly originate from the western sector, as illustrated in the wind rose in Figure 5. Additionally, it is worth noting that the WRF1 model predicts a direction approximately 60° more northerly and at higher velocities compared to the observed wind speeds and that WRF9 predicts winds that are lower than the observation and more varied in their direction. In the lower wind speed range, both WRF1 and WRF9 predict winds from the western sector, whereas a substantial portion of the wind measurements come from the eastern sector. Additionally, the WRF1 model forecasts a higher frequency (approximately 32%) of 10–15 kts winds compared to the 12% observed and roughly 5% more wind velocities in the range of 15–20 kts. For stronger winds, the WRF1 simulations generally indicate a W-NW direction, with a minor increase in the northern bias, as indicated by the higher frequency of the NW direction becoming dominant, whereas the observed data still indicate a SW–WSW direction. To conclude, the wind-rose investigation highlights the inaccuracies of the WRF simulation models. First and foremost, WRF1 presents a solid preference for the northern wind component throughout the day. This is important in two contexts:

• Both resolutions of WRF fail to predict the morning to midday lake breezes, which are the ESE to SE winds that are visible at IMS wind roses, as illustrated in Figure 5.
• Like typical sea breeze, these measurements are from the onshore direction and are characterized by low velocities.

Due to the typical wind cycle [64], we expected both the WRF simulation and IMS observations for the noon winds to present stronger and more westerly. Although the velocity difference is less dramatic than the direction discussed earlier, it also plays a crucial role from a competitive sailing perspective. For clarity, even a relatively small velocity difference between two boats is crucial when it comes to boat speed and angle relative to the wind origin. The 60° direction difference between WRF1 and observations and the diversion of velocity and WRF1 velocity predictions are clearly significant and show that the basic WRF (WRF1) simulation can be greatly improved, especially for demanding applications such as sailing competitions where a relatively small wind velocity difference could result in victory [81].

The regression of direction between the simulation and observation, as depicted in Figure 6, shows that the WRF1 simulation was not able to identify any change in wind direction (r² = 0.01, p ≤ 0.01) and that the WRF9 simulation was not able to predict wind direction in much the same way (r² = 0.00, p = 0.99). As speculated previously, this inability to identify changes in wind direction is probably caused by the failure of the models to simulate the morning lake breeze [82]. The WRF1 simulation predicted changes in wind velocity better (r² = 0.47, p < 0.001), while the WRF9 simulation was not able to predict the velocity changes (r² = 0.01, p < 0.001). It is possible that WRF1 was able to resolve velocity changes because they are driven by larger scale phenomena such as intrusion of Mediterranean sea breeze. This conclusion is strengthened by the absence of wind direction forecasts lower than 200°, which are present in the observations. The WRF1 simulation also overestimates wind velocity and direction, which also illustrates that the simulated wind is more northerly than the observations.

The generation process of 2D linear regression for wind direction can be a challenging task. This is because degree units exhibit a circular characteristic; for example, 2° and 358° would appear at different edges along the axis, although they represent the equivalent wind direction. However, in our examined data, this did not pose a problem because there were no observations or simulated wind from the north.

The WRF9 wind velocity simulation was unable to resolve the wind velocity patterns. The simulation was always close to 5 knots, while the observations ranged between 2 and 14 knots. The WRF1 simulated wind velocity, which is illustrated in Figure 7a, is an overestimate compared with the measurements. While the slope of the regression line is 0.77, so the deviation overestimate decreases as wind velocity increases, the intercept of the regression line is 4.9 knots, so the overestimate in the lower wind range is high. The maximum wind measured at the station reaches only 16 kts, whereas the simulated velocity contains numerous predictions for 15–20 kts as well as a few days of predicted 20 kts, although those may be interpreted as artifacts. Both observations and simulations show two prevalent wind velocity modes, which are illustrated by the two hotspots in Figure 7a,b. The prevalent low wind mode is at wind velocities of about 1.5–4.5 kts in the observations and about 4–9 kts in the simulations (left yellow patches). The higher velocity mode is at about 9–12 kts for the measurements and about 12–15 kts for the WRF1 forecast (right light blue patches). We assume that the wind modes in both the forecast model and the observations are due to the lower morning lake breeze and the stronger afternoon wind. Systematic deviations, such as those we previously described, can potentially be recognized by ML methods to improve the prediction of the simulation model.

There are multiple possible explanations for the low prediction accuracy of the WRF, all of which result from our decision not to optimize the WRF simulation. One possibility is that the data derived from the initial conditions of the GFS-27 model (27 km resolution) may not be entirely suitable for our research site. Another potential factor could be that the terrain characteristics supplied to the model within the WRF model might not have been of a high enough resolution for accurate predictions. Furthermore, a more accurate parameterization scheme for subgrid processes using a different parameterization scheme for processes such as convection could achieve better forecasting. In this research, we intentionally avoided optimizing the WRF simulation at the cost of lower WRF accuracy in order to test ML postprocessing as an alternative to optimizing the WRF simulation. The goal was to emphasize the use of the ML model, shifting the focus away from WRF reconfiguration.

The use of high-resolution simulation, especially when focusing on competitive sailing forecasts, is expected to yield better results. However, the high-resolution simulation is not a good fit for this application. Lower-resolution numeric predictions (e.g., WRF9 or even GFS with 13 km resolution), on the other hand, can be easily obtained from open databases and do not require heavy computational resources for processing and simulations.

4.3. ML Models Prediction Accuracy

4.3.1. Data Preprocessing

The outputs of the atmospheric modeling using WRF served as input for the ML model. Additionally, each measured wind vector U10/V10 was converted independently into five categorical classes. For each wind component, quantization into categorical classes was conducted, ensuring that the dataset’s classes had equal sizes in terms of the number of observations. This step was crucial to prevent bias and represent all classes equally during the ML model training process. The dataset was split, with 80% allocated for training and the remaining portion for validation. Data preprocessing included data augmentation to expand the dataset with synthetic data by 4×x, enhancing the training dataset size [83]. This augmentation method involves adding Gaussian noise to the input dataset, generating an extended dataset based on existing data. Data augmentation not only addresses overfitting concerns [84] but has also proven effective in weather and climatology studies [85,86]. It should be noted that data augmentation increased the size of our training dataset by 4×, and thereby, the training time increased accordingly.

4.3.2. Prediction Accuracy of ML Models

Both the CNN and MLP models were compiled with categorical cross-entropy loss using the Adam optimizer and the optimal hyperparameters discussed in Section 3.4.2.

Table 3. Final ML model evaluation results.

(a) High Resolution
Model	U10 avg. Accuracy	V10 avg. Accuracy	U10 st. dev.	V10 st. dev.
CNN	93.7%	94.8%	1.5	0.2
MLP	72.0%	54.8%	1.6	2.6
(b) Low resolution
Model	U10 avg. accuracy	V10 avg. accuracy	U10 st. dev.	V10 st. dev.
CNN	98.2%	97.7%	0.14	0.46
MLP	97.7%	97.5%	0.14	0.60

summarizes the prediction accuracy and standard deviation achieved by our models for both low- and high-resolution datasets. The CNN model achieved more than 93% and 94% prediction accuracy for U10 and V10, respectively, for the high resolution. In the case of low resolution, the CNN model achieved more than 98% and 97% prediction accuracy for U10 and V10, respectively. The MLP model achieved over 97% prediction accuracy for both the U10 and V10 vector components in the low-resolution dataset. However, for the high-resolution dataset, the MLP model achieved 72% and 64% accuracy rates for the U10 and V10 wind components, respectively. A possible explanation for the lower accuracy of the MLP model in the case of high-resolution data could be that the high-resolution data encompasses a smaller geographic area compared with the low-resolution domain and that features that are present in the larger area help in identifying local phenomena. As a final step, cross-validation tests [87,88] were performed on the low-resolution forecasts for both CNN and MLP and evaluated with a high accuracy rate of 97% and above for both ML models (see

Table 4. Cross-validation results summary.

Model	Min Accuracy	Max Accuracy	St. dev.	Post Trimming Average
CNN	97.3%	98.3%	0.42	97.9
MLP	97.2%	98.0%	0.33	97.6

).

As part of our experimental analysis, we performed cross-validation (CV, Table 4) for both the CNN and MLP models. The CV was conducted by randomly selecting the training and validation datasets using the same 80/20 split between the training and validation datasets. This process was repeated five times, and the mean was extracted after removing the best and worst outliers [87,88].

To assess the ML models’ performance comprehensively, we conducted further analysis by constructing a confusion matrix (CM) and evaluating the ML model accuracy, as illustrated in Figure 8 for the low-resolution model. In an ideal CM, each class aligns with its counterpart, reflecting a similar prevalence across the diagonal cells. The CM is vital in evaluating the performance of our ML models. First, it provides a detailed breakdown of a model’s predictions, revealing true positives, true negatives, false positives, and false negatives for each class. In addition, it allows for a comprehensive performance assessment, error analysis, and detection of class imbalances. In essence, the CM offers nuanced insights beyond basic accuracy metrics, guiding decisions on model refinement and deployment. As shown in Figure 8a,b, the CM illustrates favorable outcomes for both the CNN and MLP models, revealing accurate predictions, small percentages of misclassifications, and a well-balanced representation across various classes.

An additional analysis was conducted to investigate the convergence of the CNN and MLP models during the training process, as illustrated in Figure 8c. Notably, 50 epochs are sufficient to attain stable and converged training accuracy. The CNN exhibits a slightly faster convergence rate, surpassing a prediction accuracy ratio of 90% after 20 epochs and achieving 95% within 30 epochs. In comparison, the MLP reached 90% accuracy at 32 epochs and 95% by 45 epochs.

The fact that we used categorial ML models for our wind prediction involves an inherent loss of information. To evaluate the loss of the combined model using the numerical model in conjunction with the ML model, we decategorized the combined forecast output by using the median velocity in each category and direction (U10 and V10) versus the measured wind velocity during that forecast. Figure 9 illustrates the decategorized predicted wind velocity and direction versus the observations for our CNN ML model.

The forecast of both wind velocity and wind direction is significantly improved. The wind velocity r² improved from 0.46 to 0.88 by using the ML postprocessing, and the wind direction further improved from r² = 0.0 to r² = 0.52. When looking at the wind direction, the inherent data loss from categorizing continuous quantities is most visible. Further improvement can be achieved by ignoring the lowest category in both U10 and V10 forecasts (lowest line of predictions on the plot) out of the 25 forecasted wind directions (5 × 5 categories for U and V, respectively). In this case, the slope is further improved from 0.04 to 0.74. This observation highlights an inherent challenge in the postprocessing method we developed; it may not be the most suitable approach for predicting extreme weather events. The fact that many of the forecasted wind direction points are scattered around 150° indicates that the CNN model was able to identify typical morning conditions and forecast the local morning sea breeze, a typical Kineret phenomenon that was not resolved by the numerical forecast. The wind direction forecast is accurate even when focusing on the afternoon predictions (Figure 9b, yellow points). Most of the sailing competitions in Lake Kineret are conducted during the strong westerly afternoon winds, and the fact that the combined forecast was able to predict wind direction within the narrow-angle of afternoon wind directions indicates that the combined forecast can provide sailors much more effective means for winning races.

5. Conclusions and Future Studies

This study aims to apply ML forecast models to provide a relevant, reliable, and accurate tool for tactical and strategic use in competitive sailing. This study demonstrated that ML models can be efficiently used for the postprocessing of numerical weather prediction models with their default configuration, thereby eliminating the necessity for optimizing the model parameters. The site used as a case study in this research is unique in its properties and introduces a challenging problem for prediction. Atmospheric models are known to produce high-accuracy predictions; however, the low performance of the atmospheric model in our research (WRF) is due to the decision to use the default WRF configuration and avoid the tedious process of model configuration optimization. Nevertheless, we successfully developed a categorical wind predictor with five categories for both CNN and MLP, with accuracy rates of 90% and above for most forecasts. In addition, we generated a preliminary wind velocity and direction forecast based on a CNN customized for sailors using wind speed units (kts).

We suggest further investigations in the following areas:

• Further optimization of the number of categories used by the postprocessing ML mode.
• Using spatial wind measurements conducted by synthetic aperture radar (SAR) to further enhance the accuracy and robustness of the processing of the ML model forecast.
• Further exploration into using ML models to enhance the accuracy of the GFS with 13 km resolution. Such a model is accessible online and will eliminate the need for numerical modeling.

Lastly, this research presents promising directions for advancing the area of integrated weather forecasts with ML models and will serve as a foundation for future developments in this field.