A Nationwide Flood Forecasting System for Saudi Arabia: Insights from the Jeddah 2022 Event

Abstract

Saudi Arabia is threatened by recurrent flash floods caused by extreme precipitation events. To mitigate the risks associated with these natural disasters, we implemented an advanced nationwide flash flood forecast system, boosting disaster preparedness and response. A noteworthy feature of this system is its national-scale operational approach, providing comprehensive coverage across the entire country. Using cutting-edge technology, the setup incorporates a state-of-the-art, three-component system that couples an atmospheric model with hydrological and hydrodynamic models to enable the prediction of precipitation patterns and their potential impacts on local communities. This paper showcases the system’s effectiveness during an extreme precipitation event that struck Jeddah on 24 November 2022. The event, recorded as the heaviest rainfall in the region’s history, led to widespread flash floods, highlighting the critical need for accurate and timely forecasting. The flash flood forecast system proved to be an effective tool, enabling authorities to issue warnings well before the flooding, allowing residents to take precautionary measures, and allowing emergency responders to mobilize resources effectively.

1. Introduction

Floods are among the most frequent and dangerous natural disasters globally, and their impacts are likely to increase as climate change progresses and the population grows [1,2,3,4], regardless of location, climate, or level of development of the affected country [5,6,7]. Flash floods, in particular, have escalated drastically due to climate change and ongoing urbanization [8,9], with disasters reported worldwide.

The effect of climate change is just one factor contributing to the rising impacts of floods [10]. A combination of societal factors [11] and methodological limitations in modeling and forecasting make cities more vulnerable to flooding, especially in arid environments [12,13,14].

Flash flood forecasting is challenging [12] and is made more so by some features that distinguish it from riverine flood forecasting and add to its uncertainty [6,15,16]. Flash floods are driven mainly by convective precipitation events, with high spatiotemporal variability limiting their predictability [15,17], also because of the inherently three-dimensional and turbulent nature of the atmosphere [18]. Furthermore, flash floods are characterized by fast dynamics and high specific discharges [15,19], which require high-resolution models to be captured efficiently. Bannister et al. [20] provided a review of some of the most significant challenges in predicting convective events driving flash floods, including the needs for large volume of water-related observations (e.g., from radar, microwave, and infrared instruments), the necessity to analyze hydrometeors of various nature at multiple scales [21], the growing importance of local-forecast errors due to the scale at which these events happen, the requirement to account for the flow-dependent multivariate balance between atmospheric water and both dynamical and mass fields [18], and the inherently non-Gaussian nature of atmospheric water variables [20]. This complexity is further heightened by the fact that convective storms are influenced by local topography, land use, and surface conditions, which can vary widely and affect storm development [20].

Although some countries offer operational flash flood monitoring services based on weather radar rainfall estimates [22,23,24], the lag time between a rainfall burst and flood impact is too short to grant a meaningful period of anticipation in affected domains [12,25]. The importance of providing accurate inundation forecasts at the scale of specific critical locations within a broad region underscores the need for increased anticipation and finer flood impact modeling than that on which current operational forecasts are based. Indeed, flash flood impacts occur at the scale of individual properties, and accurate warning needs to account for the specific processes of the urban environment, where stormwater follows multiple flow paths and interacts with artificial structures [26]. In addition, near-real-time inundation forecasts may require performing ensembles of simulations with a latency of just a few minutes [27] and a forecast horizon short enough to be significant but large enough to grant time for preparedness.

Simulating the discharge and accurately determining inundation are also challenging for large-scale applications. Most rainfall-runoff models operate on spatial scales on the order of 1 km. Still, the high number of calibration parameters may become a constraint because the systematic observational data are generally missing. The scale at which flash floods develop is too small for conventional rain or discharge measurement gauges to sample them effectively [28]. Moreover, flash floods are relatively rare at the local scale, with flood impacts varying in location, especially in urban environments, making it hard to collect inundation references. Even from a modeling perspective, solving a bidimensional inundation tends to become too computationally expensive for implementation in operational systems.

Nowadays, however, new numerical methods, the deployment of high-performance computing [29,30,31,32], and new and highly detailed datasets often derived from remote sensing [33,34,35] are offering new opportunities to improve the models’ accuracy and make them sufficiently reliable to enhance our understanding of the generation mechanisms of flash floods.

Although Saudi Arabia is in a region characterized as arid/semi-arid [36], flash floods are quite frequent there, and they are becoming increasingly so [37,38,39,40,41,42,43]. The events often cause heavy human casualties, as well as significant damage to infrastructure, even though the country lacks permanent year-round rivers.

The Arabian Peninsula (AP) features a diverse climatic range, with higher temperatures and inconsistent rainfall patterns [44,45], with areas mostly characterized by winter snow precipitation in the Asir Province to those featuring intense humidity around the Arabian Gulf and parts characterized by extreme heat, such as the Rub Al Khali desert, or monsoon-dominated climate in the Qara Mountains of Dhofar [45]. Rainfalls are characterized mostly by irregular, heavy rainstorms, on only a few days in a year and only in some areas, with the southwestern area being mostly controlled by orography and showing local convective precipitation [46,47]. Various authors highlighted shifts in temperature and evaporation rates [48] and precipitation intensity [48,49,50,51], resulting in more impactful and frequent flooding. Historical analysis showcased an average increase in precipitation intensity of about 15 to 20% in some areas, such as Jeddah [49], with rising trends in daily maximum precipitation, suggesting an increasing flood risk [52] across AP overall [44,50].

Flooding impacts due to these changes in climate drivers are further exacerbated by rapid population growth, urbanization, and transformation of large areas of sandy desert into concrete-built impervious surfaces that delay rainwater infiltration [53] and affect the natural drainage [52,53,54,55,56,57,58]. Flash floodwaters travel at high speed without a predetermined path and carry a high debris load [59]. Accumulated debris within drainage conduits in urban areas and in minor ephemeral wadis originating from previous runoff events, furthermore, can locally increase the flood potential [60]. Risk further increases due to the fact that many urban settlements and other infrastructure developments were built over the years on dry wadi beds during a long time of aridity [56,57,61,62].

The nature of these flows demands a warning system that can provide ample lead time and local-scale modeling. However, the generally limited availability of data records and field observations poses unique challenges to researchers and practitioners.

This paper introduces the Flash Flood Forecasting System of the National Center for Meteorology of the Kingdom of Saudi Arabia. In collaboration with Saudi Business Machines, LENOVO Super-Computers, the University of Connecticut, Weather & Marine Engineering Technologies, and the University of Athens, the Center implemented a first-of-its-kind operational instance of a flash flood forecast system over the Arabian Peninsula. As the rapid onset of flash flood events hinders mitigation measures and limits timely decisions that could save lives and avert property losses, the principal objective of the current study was to investigate the predictive capability of the operational tool during a flash flood that struck in western parts of Saudi Arabia after torrential rain on 24 November 2022 (Figure 1). During this event, the city of Jeddah in Makkah Province recorded 179 mm of rain in six hours, the heaviest ever recorded in the area. The result was widespread flash floods across the urban and rural areas of Jeddah, with floodwaters inundating numerous streets and closing major routes, including the main highway between Jeddah and Makkah. In the aftermath, power outages and water supply issues were also reported in parts of the city. In the following sections, we describe the forecasting system and models, validate the precipitation, hydrological, and flood inundation forecasts, and discuss the forecast system’s performance for 24 November 2022. We close with recommendations for future system development.

2. Materials and Methods

The National Center for Meteorology (NCM) Flash Flood Forecasting System integrates numerical weather forecasts from the Weather Research and Forecasting (WRF) model [63] with the Coupled Routing and Excess STorage (CREST [64] and a 2D hydrodynamic model (HEC-RAS; see Figure 2). The atmospheric component runs at cloud-resolving scales (1.6 km) to incorporate local features and strong convection. The hydrological and hydrodynamic models run at variable spatiotemporal resolution: rainfall-runoff generation and routing are both at 500 m-by-hourly, and floodplain dynamics are at 30 m-by-hourly and 2.5 m-by-hourly. The significant computational requirements dictate the domain differences, with CREST running over large natural basins, while HEC-RAS runs over small basins with dense infrastructure and high exposure to floods. The Flood Forecasting System runs over multiple nested domains that are operationally selected based on the amount of exposure—based on the presence of human beings and their livelihoods, assets, infrastructure, and resources in places where flood hazards could occur. Each domain runs using different cores, depending on the size of the domains. The system takes about 1 to 2 h to produce results, and it is triggered every 6 h, with a forecast window of 120 h (5 days).

The following sections describe the domains and setups for the atmospheric, hydrological, and hydrodynamic models.

2.1. Atmospheric Model Domain and Setup

During the past decade, high-resolution and convection-permitting numerical weather prediction (NWP) models have improved weather forecasting systems to the point where they can capture heavy precipitation events [65]. Among the many existing NWP operational and research applications, WRF is the most popular atmospheric model used [66]. This popularity is attributed to its ability to employ a wide range of physical parameterizations (such as microphysics, convection, planetary boundary layer, radiation, and surface physics), its capability to run at fine scales, and its support from a large and active user community. Furthermore, the continuous development and enhancement of WRF, along with its proven scalability and reliable output, contribute to its widespread adoption.

For the NCM operational system, WRF runs in two-way nesting interactive nests (Figure 3). The nests are used to enhance regional weather simulations by increasing spatial resolution in specific areas. This improves the representation of localized phenomena (e.g., thunderstorms, complex terrain effects) and overall forecast accuracy, providing computational efficiency by focusing resources where high resolution is most needed.

The coarse nest covers a large domain, including parts of Africa, Asia, and Europe, with a spatial resolution of 4.8 km, while the fine nest covers the Arabian Peninsula with a spatial resolution of 1.6 km (Figure 3). We apply a level-division scheme with 48 vertical levels. The model setup and the resolutions employed were chosen to balance computational efficiency with the need for detailed forecasts. The 4.8 km resolution is used to capture larger-scale atmospheric processes and to provide a comprehensive overview of a wider region. At the same time, the 1.6 km resolution is employed to capture finer-scale atmospheric features, such as convective processes (explicitly), which are critical for accurately forecasting heavy precipitation events. The available resources allow this model setup to operate in four daily cycles (00 UTC, 06 UTC, 12 UTC, 18 UTC), with the outer domain configured to provide a 10-day forecast and the inner one a 5-day forecast (+12 h spin-up time). The model’s predictability depends on many factors, such as initial and boundary conditions, physics parameterization, etc. When focusing on forecasting strong and localized convective activity, the best forecast is expected within a 24-h window. Beyond this period, the forecasting capabilities gradually decline, affecting both the spatiotemporal distribution and intensity of the event. Patlakas et al. (2023) [67] provided more information regarding the configuration and performance of the modeling system.

2.2. The Hydrological and Hydrodynamic Model Domain and Setup (CREST-HEC2D)

Figure 4 shows the settings for the integrated CREST-HEC2D system for the whole domain and for the area affected by the November 2022 event.

CREST is a distributed hydrological model that integrates precipitation-runoff and channel routing processes at high spatiotemporal resolution (30 m–1 km/hourly), fitting most land cover types. The model is currently applied for real-time flood forecasting and historical analysis over selected locations in Connecticut, USA [64,68,69]. For the operational Flash Flood Forecast System, the model considers land cover information (“MOD12Q1”) retrieved from the Moderate Resolution Imaging Spectroradiometer (MODIS; [70] and integrates precipitation-runoff and channel routing processes to produce discharge at hourly timesteps based on the input climatic forcing from the atmospheric module described above. The rainfall-runoff module runs at 500 m resolution, following the input variable of the finest resolution (in this case, MOD12Q1). The direct runoff is then downscaled to the Copernicus 30m GLO-30m digital elevation model (DEM) grids [71] for routing.

We produced flood inundation characteristics using HEC-RAS 5.0.7 for Linux. The implemented system takes full advantage of a standard calibration model for each domain that is automatically updated based on input streamflow data. To perform such automation, we developed a coupling algorithm in Python to provide a set of computational subroutines that allows complete automation of the HEC-RAS model, from preparing the input flow conditions to simulating the event and gathering the results into meaningful maps. This automation reduces the need for excessive manipulation of files and manual adjustments in the model itself, which presents a significant advantage when essential decisions must be made relatively quickly.

The control of HEC-RAS computations involves three main components:

1. Data Retrieval Functionality (DRF). This component retrieves the input flow data and synchronizes with CREST’s completion of its runs. After each run, the data retrieval packet is triggered independently for each domain. It collects results for each outlet, assigns flows to boundary conditions in HEC-RAS, and sets unsteady flow rates. This process embeds the simulation’s start and end time, along with flow data, into the HECRAS input files.

2. Operation Functionality (OF). The OF is responsible for managing operations such as opening, closing, and running HEC-RAS. It performs the unsteady flow simulation in three steps: reading data from the modified input plan file, executing the RasUnsteady.exe program, which reads hydraulic properties tables from the pre-processor, along with boundary conditions and flow data, and finally, conducting the unsteady flow calculations.

3. Post-Processing Functionality (PostProc). PostProc handles the production of results and the web display. Since HEC-RAS 5.0.7 for Linux does not directly provide inundation maps at the resolution of the input DEM (Digital Elevation Model), the post-processing functionality retrieves inundation depth at the computational cell level, interpolates it to the DEM resolution using common Python 3.10 packages (numpy and scipy), and generates the final 2D inundation map. To streamline production, the interpolation process uses triangulation, which is repeated at each run. Additionally, for each domain, the post-processing routine stores the indices of the vertices of the enclosing simplex and the interpolation weights for each grid point. This information, computed only once, allows for the calculation of interpolated values for each grid point using the stored barycentric coordinates and values of the function at the vertices of the enclosing simplex. This reduces the computational load of the pipeline. The final map is then used as input for the web-gis display. For this, we have implemented a streamlined procedure based on Leaflet, a popular choice for a wide range of mapping applications known for its ease of use, flexibility, and extensive plugin ecosystem.

HEC-RAS runs considering the COP-30m DEM over most areas across the country. To represent topography correctly over nine selected cities, the system considers high-resolution DEMs (2.5 m) with a high accuracy of 0.5 m in both horizontal and vertical directions, realized with DigitalGlobe’s satellites [72,73]. To better represent the impacts of infrastructure on inundation dynamics, we manually added urban features, such as houses, buildings, and existing drainage structures, to the DEM when they were available from the local authorities. We used a mesh of variable sizes to construct the HEC-RAS domains, setting a reduced mesh resolution in more homogenous areas and a highly detailed description of critical terrain features within the urban areas. HEC-RAS manning coefficients were defined based on landcover retrieved from the European Space Agency (ESA) WorldCover 10 m 2021 product, which provides a global land cover map for 2021 at 10 m resolution based on Sentinel-1 and Sentinel-2 data [74]. As calibrating Manning’s values requires a large ground-truth dataset, which is not available at the country scale, we implemented standard Manning’s values based on the literature [75,76] but selected the higher end of the range, as it is generally agreed that for 2D flows, a shallower flow depth corresponds to higher Manning’s n values [75,76].

To verify the system’s effectiveness and rank the results for the 2022 event, improving upon existing databases and making them more comprehensive, we compiled a digitized Flood Inventory with flood events georeferenced by latitude and longitude. We collected historical flood events in Saudi Arabia, including the flooding time, cities affected, and flood impacts, from FloodList (https://floodlist.com/tag/saudi-arabia, accessed on 26 June 2024), a website funded by Copernicus that has recorded flooding and flooding news since 2008. We supplemented the historical flood list using the flood archives for Saudi Arabia from existing projects, and we retrieved events included in the Emergency Disasters Database (EM-DAT, http://www.emdat.be/, accessed on 26 June 2024), compiled and managed by the Center for Research on the Epidemiology of Disasters (CRED). EM-DAT includes only those events in which ten or more people were killed, a hundred or more people were affected, a state of emergency was declared, or there was a call for international assistance. The database is freely accessible online and has a sorting tool to define location, time, and disaster category. We selected all reported events categorized under “heavy rainfall” or “flood.” Where actual coordinates were provided for the locations of events, we kept that information. Where events were categorized only by city or province name, we geolocated them using the Nominatim geocoder (developed and maintained by the OpenStreetMap community and used as a default geocoding service for OSM data). The event counts for each domain are provided in

Table 1. N. of events by region (as reported in Figure 4) collected in the flood inventory.

Region	N. of Events
SouthWestCoast	61
MidWestCoast	55
Hail_Buraydah	36
Riyadh	25
NorthWestCoast	23
EastCoast	22
NorthBorder	22
NorthWestCoastDown	15
Hafar	10
NorthWestCoastUp	4
Total	276

.

2.3. Validation Approach

The quality of the models was evaluated following recommended statistics [77], including dimensionless statistics and absolute error index statistics. For this, we considered the Pearson correlation coefficient that describes the collinearity between the observed and model-simulated variates, the Nash-Suttclife efficiency (NSE), which is able to account for differences in the observed and model-simulated means and variance, and the root mean square error (RMSE), mean error (ME), and mean absolute error (MAE) as measures of the mean error in the flow unit itself. Additionally, for the rainfall, we investigated the bias ratio expressed as the differences relative to observations between raw and bias-corrected precipitation, and we evaluated an object-based verification approach to directly address the skill of forecasts of a localized rainfall event such as the one in Jeddah.

The literature suggests that model performance can be judged satisfactory with NSE > 0.50, Pearson’s values greater than 0.5 being acceptable, with higher values indicating less error variance [78], and RMSE and MAE values less than half the standard deviation of the measured data [78].

Object-based verification for rainfall analysis is particularly important due to the need to verify highly localized and episodic phenomena, such as flooding [79,80], as it provides valuable diagnostic information about errors related to the location, spatial characteristics, and geometric patterns of precipitation. Such information is crucial for hydrologic applications, particularly in distributed hydrological modeling, where variations in the location and spatial patterns of precipitation events influence the modeled runoff volumes. Regarding the comparison, the analysis considers the Intersection over Union (IoU), which serves as a widely used measure for assessing the precision and determining localization discrepancies within object detection models. It quantifies the extent of overlap between two bounding boxes: one predicted and the other ground truth. A high IoU score signifies substantial overlap between the predicted and ground truth boxes, while a low score indicates minimal overlap. An IoU score of 1 represents a flawless alignment between the projected and ground truth boxes, whereas a score of 0 indicates no overlap between them.

2.3.1. WRF

To test the effectiveness of the system, we compared the WRF-forecast precipitation with (i) precipitation estimates from the Integrated Multi-Satellite Retrievals for Global Precipitation Measurement (GPM) Mission (IMERG-Late), (ii) precipitation gauge measurements (Figure 5), and (iii) radar-rainfall estimates.

The GPM-IMERG-Late [81] dataset is derived from sophisticated algorithms that combine data from many satellite-microwave and infrared precipitation estimates, precipitation gauge analyses, and other estimators. The dataset’s spatial resolution is 0.1 × 0.1 deg, with available temporal frequencies of 30 min.

Across the country, a network of METEOSERVIS MR2-0.2s rain gauges measures liquid precipitation with a resolution of 0.2 mm. These gauges rely on the “tipping bucket” principle, with a catching area of 200 cm². The range of proper operation is between 0 and 900 mm/hr, and the measuring error depends on the intensity of the rain. The values reported by the manufacturer are under 1% for rain intensity up to 20 mm/hr, <4% between 20 and 60 mm/hr, and <6% between 60 and 200 mm/hr.

For each event, rainfall maps are available based on radar observations from a VAISALA weather radar’s WRM200 network. These devices operate in a frequency range of 5.5 to 5.7 GHz, with an average power of 300 W and peak power of 250 kW (0.12% duty cycle). The transmitters can produce pulses with frequencies ranging between 50 Hz and 2.4 kHz, with two modes of operation, STAR or LDR. The radar receiver employs a dual-stage, dual-channel IF downconverter and digitizer, with a noise figure of less than 2 dB, sensitivity of 3 dB, and dynamic range exceeding 100 dB. Reflectivity values exceeding 80 dB (or 100 dB with waveguide filters) are rejected.

2.3.2. CREST

Given the scarcity of available flood records, our primary objective in tuning the hydrological model was to replicate the shape and peak of observed floods accurately. We divided the tuning procedure into two phases to represent the rapid onset and recession of desert floods accurately. First, we conducted an automatic calibration using the CREST model’s built-in function [64] based on the Shuffled Complex Evolution University of Arizona (SCEUA) algorithm [82]. This algorithm primarily optimizes routing parameters to improve the representation of the shape of the flood hydrograph. The second phase involved a sensitivity analysis within the model to adjust land surface parameters, ensuring they accurately matched the peak flood values. The analysis applied a proportional factor to the initial land surface parameters based on soil properties, land cover types, and canopy profiles [64]. The CREST model’s tuning process utilized one flood record from a 1985 storm over the Wadi Aliath area south of Jeddah (Masood, 2016). We obtained the input forcing data for the calibration process from WRF historical simulations.

The tuned parameters used in the hydrological model are listed in

Table 2. The tuned routing and land surface parameters of the hydrological model (CREST).

Parameter	Tuned Value	Description	Reference
CoeM	111.32	The overland runoff velocity coefficient	[84] https://xinyi-shen.uconn.edu/wp-content/uploads/sites/2836/2019/11/CREST-User-Manual-v2.1.3.pdf, accessed on 26 June 2024
ExpM	0.55	The overland flow speed exponent
CoeR	1.82	The speed multiplier used to convert overland flow to channel flow
CoeS	0.58	The speed multiplier used to convert overland flow to interflow
KS	0.95	The overland reservoir discharge parameter
KI	0.33	The interflow reservoir discharge parameter
CoeSAT	0.80	The coefficient applies to adjust the saturation moisture	[83]

. The first six parameters are routing parameters. The last parameter, namely CoeSAT, is used to adjust the saturation moisture derived from digital soil mapping data [83] to achieve a comparable runoff coefficient against the observed record in 1985. All routing parameters are within a reasonable range [84], with a high KS value representing the sharp flash flood hydrograph pattern in the desert area.

2.3.3. HECRAS

Since no hydrologic observations were available to the authors, we used discharge obtained from the gauge-adjusted radar-rainfall data as a reference to assess the skill of the WRF-based flood forecasts. Observation data for the event comprised reports from people (“crowdsourced”) on localized flood incidents at the street or house level. As highlighted by other authors [85], such observations are often anonymous and/or not georeferenced. Nonetheless, they carry some significant landmarks that allow for pinpointing specific locations from which the images and video were taken. Refs. [86,87] provide a thorough review of studies that used citizen science connected to floods, benchmarking difficulties and benefits of data collection and integration into models. One must consider that retrieving inundation depth/extent from crowdsourced information is subject to biases. Determining water level values from photographs of flooded areas is uncertain due to several factors: (i) the exact size of reference objects in the image may be unknown; (ii) the height of floodwater may vary across different areas of the image; and (iii) objects might be partially visible because they can be submerged in water [88]. Nevertheless, citizen science data can be adequate to provide information for evaluating the performance of the model [85,88,89,90,91,92] and to determine flood extent, as this is generally less uncertain due to its binary nature [85,86,93].

3. Results

3.1. CREST Tuning

Figure 6 presents the optimally simulated flood hydrograph following the tuning process. The simulation successfully captured the rapid response of the flood to precipitation events, showing a good overall match with the recorded flood data in both the shape and peak values. The produced peak value closely resembled the observed one in amount (−3.6%) and timing (−1.0 h). Both values are indicators of very good performance of the model, as the timing error should ideally be within a small time window, such as ±1–2 h for short-term flood events, and a good value for peak flow percentage should ideally be within ±10–15% of the observed peak flow [77,78,94].

Incorporating more observational data, nonetheless, is encouraged to refine the model calibration further and improve evaluation metrics.

3.2. Validation of WRF Precipitation Predictions

The WRF estimates for the spatial distribution of 24-h accumulated precipitation agreed with radar observations and the IMERG-Late dataset, as shown in Figure 7. In all three products, the accumulated precipitation reached a value of 50 mm. However, it is essential to note that IMERG data consistently overestimated rainfall with respect to radar by a factor of two.

Figure 8 presents a comparison of WRF cumulative precipitation time series. As station observations indicated, a significant amount of rain (up to 180 mm) fell between 03 and 09 UTC on 24 November. In Abhur Municipality, Jeddah Janub Municipality, and NCM Bani Malik, the error of WRF prediction was within 25%. A higher error was recorded for the NCM main office, with an underprediction of 50%. The time series in Figure 8 and the statistical metrics in

Table 3. WRF statistical evaluation against in situ station observations. Readers should refer to Figure 8 for the overall rainfall statistics of the various products, as compared to measured rainfall.

	WRF	IMERG-Late	Radar
Root mean square error	7.35	7.99	7.89
Mean error	0.21	2.87	−0.01
Mean absolute error	2.28	4.04	2.48
Pearson	0.60	0.58	0.43
Bias ratio	1.10	2.37	1.00

demonstrate the model’s high performance, as the Pearson coefficient was around 0.60. The mean absolute error (MAE) and root mean square error (RMSE) were the lowest for WRF, followed by radar and IMERG, respectively. The bias suggested that WRF overpredicted precipitation by 10% in all stations, while the IMERG dataset did so by 137%. Since the radar product underwent bias adjustment, the bias had a value of 1. Focusing on the model behavior, potential errors and discrepancies in forecasting capacity can be attributed to the highly localized features of the convective activity and the errors induced by the initial and boundary conditions and the physical parameterizations employed in the operational system.

Object analysis has been carried out to compare WRF (forecast) and IMERG dataset (observation), re-gridding all results to 10 km resolution. The analysis is performed by re-griding the model to match the observation resolution (10 km). Figure 9 shows the WRF objects (a), the IMERG (b), and WRF objects (colored) with the IMERG outlines (c). The result identified similar rainfall patterns regarding coverage of the rainfall event, as well as its intensity and location.

The area of interest, Jeddah, is highlighted in blue in Figure 9, and it is labeled as object no. 1.

Table 4. Object analysis statistics between WRF (FCST) and IMERG (OBS). In the table, 1 grid point covers a 10 × 10 km pixel.

Obj	Centroid Distance (km2)	Angle Diff (degrees)	FCST Area (km2)	OBS Area (km2)	Intersection Area (km2)	Union Area (km2)	Symmetry Diff (km2)
1	112	1.1	17,080	22,000	12,020	27,060	15,040

reports the statistics for this object. When comparing the centroid of the area captured by WRF to the one from IMERG, the distance was about 112 km, with a tilt between the axis of 1.1 degrees. The forecast object area covers about 1708 pixels, whereas the observation area was 2200. The intersect in both the 1202 grid and the unmatched area (symmetry difference) is a total of 1504 pixels.

For this analysis, IoU is around 0.44, which is consistent, however, with the high overestimation of IMERG compared to radar data. Even though the analysis shows there is a shift in the WRF forecast offshore, the model managed to forecast the rain event of the area covering Jeddah city as a whole. These results highlight how, in addition to capturing the statistics of the rain event itself (Table 4), WRF was also able to capture the overall spatial properties of the storm over the city of Jeddah, providing valuable information for the warning system.

3.3. Validation of CREST-HEC2D Discharge and Inundation Depths

The NCM Flash Flood Forecasting System provided discharge and inundation depths at a one-hour resolution. The flood peak discharge timing, from the hydrological model, was overall consistent (Figure 10), with a slight delay of the peak (<20 min) forecast by WRF for some locations. The timing consistency was also valid for the inundation depth. The WRF forecast captured the maximum depth consistently compared to the radar hindcast. The inundation volume forecast by WRF was slightly higher than that of the radar hindcast, consistent with the differences in peak discharge from CREST (Figure 10). The WRF forecast was triggered on November 23 (the day before the event) at 12.30; the peak discharge was forecast around 8.30 for November 24; and the peak inundation depth nearby was forecast around 9 on November 24, with timing consistent with the peak rainfall measured from the rain gauges (between 03 and 09 UTC on 24 November).

The spatial depiction of the maximum flood depth (HEC2D), both WRF-based and radar-based (shown in Figure 11a), provided similar overall statistics. The WRF-based forecast produced a slightly higher inundation depth (median = 1.3 m vs. 0.8 m for the radar data), consistent with the peak discharge differences shown in Figure 10. The radar-based inundation captured a lower variability range (0 m–1.5 m overall, compared to a general range of 0 m–3 m for WRF-based simulations). Across the domain (Figure 11b), correspondence between simulated depths from WRF and the radar data was good, with some outliers of inundation >2 m for the WRF-based simulation, as compared to inundations <0.5 m for the radar-based analysis. The comparison of the inundation depth quantiles for the November 24 storm (Figure 11c) indicated agreement between the WRF-forecast peak inundation depths and the radar-derived ones (Figure 11c). The differences between the produced inundation characteristics (percentile of depths) when using WRF versus the radar-based hydrographs were minimal (Figure 11c), with most of the domains presenting excellent agreement.

Apart from the qualitative comparison among all the datasets, we performed a statistical comparison of the model data, further highlighting their good performance (

Table 5. CREST-HEC2D statistical evaluation. Statistics were calculated by comparing the WRF-based forecast with the radar-based hindcast. Values are shown for inundation depth considering the whole domain, while for discharge, they refer to the selected locations displayed in Figure 11.

	Discharge		Inundation Depth
	Average	Max	Average	Max
Root mean square error	98.63 m3/s	121.84 m3/s	0.16 m	0.90 m
Mean absolute error	43.75 m3/s	61.09 m3/s	0.09 m	0.90 m
Pearson	0.95	0.97	0.72	0.95
NSE	0.76	0.82	—	—

). The statistical analysis showed good to very good accordance between the WRF forecast and radar hindcast discharge for the CREST model (correlation: 0.95–0.97; NSE: 0.76–0.82), and the results were consistent for the HEC2D model.

We also found good agreement in comparing the forecast depths with reported flooded locations (Figure 12). We estimated the water depth at each location by relating it to objects shown in the pictures, such as car tires, walls, traffic signs, or significant landmarks. Most locations displayed good visual correlation with the WRF-forecast depths. In Figure 12, images A and B show inundation reaching the car tires, indicating less than 0.5 m of water, comparable to the simulated depth. Some areas also showed very high inundation (>1 m), and citizen-sourced videos showcased major flooding (C, D).

4. Discussion

Because flood discharge observations were unavailable to the authors, we utilized flood simulation forced by gauge-adjusted radar-rainfall as a reference to assess the accuracy of the flood forecasts based on the WRF. The results demonstrated that WRF-derived precipitation enhanced the flood early warning system more significantly than radar-based precipitation. The comparison of the flood quantiles for the 24 November flood event, as depicted in Figure 13, showed close alignment of the peak discharge quantiles forecast by the WRF model with those derived from the radar-based approach. The differences between the flood characteristics (hydrographs peaks) when using WRF and the radar-based predictions were minimal (see Figure 13). Overall, the forecast river discharge for the event was within the range of >95%, especially in the southern part of the city.

We retrieved hourly inundation values for 50 historical floods in the area from 2000 to 2017 for over 7000 hourly depth maps and randomly sampled the inundation extents in areas downstream of the CREST outlets (Figure 14c). Consistent with the high flow quantiles, the southern part of Jeddah experienced the highest quantiles of inundation depths during the November 2022 event (Figure 14b), with the highest near the coastline (zone 9 in Figure 14).

4.1. Performance of the Flood Warning System in Jeddah in November 2022

Although some regions are more prone to flooding than others, implementing flood warning systems over major cities is essential to deliver information vital to safeguarding property and preserving lives. As installing gauges and telemetry equipment in such a vast environment is not feasible, effective flood warning strategies require skilled personnel and well-designed protocols to offer early warning about the likelihood, timing, and severity of floods. Figure 15 showcases the warning system provided during the November 2022 event. Authorities have (as they did then) access to a global warning map (Figure 15a), with each domain color-coded across the country based on warning levels expected from the most recent forecast cycle. The web system is updated every six hours, allowing access to the three most recent warning maps.

Clicking on the domain makes the warnings visible both in a static way (Figure 15b) and, after receiving feedback for the event, dynamically (Figure 15c–e). The warning maps trade off geographic realism for equalizing the importance of each region under investigation and allow highlighting locations where inundation depth is increasing. Warning levels depend on three depth thresholds: 0.2 m, 0.5 m, and 1 m. Anything below 0.2 m is issued a “low” warning. Conversely, anything above 1 m prompts a “high” warning.

For the static approach (Figure 15b), the meter-scale, hourly flood-depth maps are transformed into a maximum inundation. Each hexagon is then attributed the mode of the absolute maximum inundation, considering all pixels within its coverage, and the mode is rounded down to the closest warning level. This procedure is repeated for the dynamic approach (Figure 15c–e), aggregating the hourly depths over a 12-h window and providing the maximum inundation every 12 h.

Given that the forecast window covers five days, defining the beginning and end of the forecast inundation is critical. For this, as well as for the static maximum warning maps, the system is constructed to allow the retrieval of the following at each available timestep: (i) the most common inundation level inside each hexagon; (ii) the earliest expected peak time (that is when the water reaches its maximum flood depth); (iii) the latest recession time (when the water depth recedes below the specified warning level); and (iv) the flow velocity (the hotspot in Figure 15f).

To obtain a clearer idea of the exposure for each location, trained personnel in charge of the warning system can select various base maps, including Google Traffic, Google Imagery, and OpenMP cartography, as background.

A key characteristic that significantly affects flood inundation simulations is the resolution of the input terrain [95,96]. While its exceptional accuracy makes LiDAR the most preferred terrain for flood studies, its widespread adoption is hindered by its high production costs. Consequently, regional flood studies often turn to global terrain data, which are more accessible and affordable. For this work, the COP-30 DEM provided the most suitable and computationally inexpensive source of data, which outweighed its medium resolution (30 m; Figure 16a). Overall, since the landscape under investigation was primarily free of vegetation, COP30 worked well in representing topography, including buildings and infrastructure, and thus captured the evolution of the flood (Figure 16a). Comparisons of the WRF-forecast-based inundation area with a satellite view taken over Jeddah by PLÉIADES NEO SATELLITE on November 25 at 8.30 UTC showed that the highest floodwater accumulation provided by the flood forecast system corresponded with locations where sediment was delivered to the sea in the aftermath of the event, confirming the likelihood of the flooding for these areas. Looking at the inundation derived using the higher-resolution terrain (2.5 m, Figure 16b), the system provided a larger flooded area than the coarser-resolution DEM (Figure 16a). The goodness of the terrain model is known to depend on the severity of events and modeling purposes [97]; overall, the finer-resolution DEM appears to have performed better for reproducing small flows associated with smaller inundation depths, such as the widespread inundation with depths of less than 0.25 m, highlighted in Figure 16b. Nonetheless, overall, the flood patterns were similar between the two models.

Because of a lack of independent flood inundation data on the event, only anecdotal evidence was available to compare the warning data. Nonetheless, the comparison (Figure 17) suggested that the system performed well in identifying high-risk areas, using both the 30 m DEM and the 2.5 m DEM for the simulation. Despite not having much crowdsourced feedback for many historical events in the area, it is noted that some high-warning areas were identified as high risk for the November 2022 event, and they historically reported flooding in 2017 and 2011. Consistent with what was highlighted in the previous section, the high-resolution terrain (aw3d) provided more extensive coverage of high warning areas than the coarser-resolution DEM (Figure 17b,e versus Figure 17a,d). The combination of the two DEM resolutions allowed for labeling with high and medium warnings in most of the flooded locations and areas where traffic jams indicated possible flooding (Figure 17c,f).

4.2. Remarks

Even if a forecast system functions technically, many “potential deficiencies” exist at each stage that must be kept in mind to avoid hindering the effectiveness of the warning model [98]. It is essential not only for local disaster management and authorities to receive and notice a warning in time but also to understand its content [99,100]. Together with the implemented warning system, we provided basic training to its users, including a fundamental understanding of hydrology and hydraulics, hydraulic modeling, and manipulation of terrain data for properly addressing urban features within hydraulic models, and deep training on how to interpret the results of the model simulations and, if needed, retrieve past warnings. Having conveyed a proper grasp of the system, it was also possible to highlight its limitations, which were inherent in building a countrywide flood system with very scarce field-based data for validation. Trained personnel had access to both past warning maps and the maps for the 2022 event and could highlight areas where the model did not provide sufficient coverage. One must consider that the initial selection of the outlets for the hydrological model and, therefore, the upstream boundary conditions of the hydraulic model were arbitrary and determined by overlaying the urban area extent with the river network in each subbasin. Using Google Maps, the locations of the outlets were then adjusted to areas where we could find extensive infrastructure and where we expected more difficulties in simulating hydraulic flows. Based on the feedback from the trained personnel, we could increase the outlet density for the current system.

A central point to consider is that the streamflow simulation of the CREST model would be further enhanced by expanding the calibration process for various regions across the country. At present, CREST calibration relies on limited observational records from restricted locations. As a result, the current parameters may not comprehensively capture flood patterns across all of Saudi Arabia for every flood. We believe that we can extend the calibration procedure to more areas by securing additional flood observation records or acquiring detailed feedback from actual events. This would provide the model with richer localized information, ensuring more precise and representative streamflow simulations.

Finally, we conducted roundtables with the involved personnel to obtain information and learn their questions, uncertainties, and concerns in connection with three descriptive methods: flow quantiles versus amounts, inundation depths, and warning maps. Their feedback and our qualitative observation revealed a disjuncture between, on the one hand, understanding and using warnings based on a statistical (inundation percentiles) method and, on the other, a preference for concrete references in describing risk in terms of warnings based on actual depths. This was consistent with studies conducted in other environments [101].

5. Conclusions

This research assessed the performance of an operational national-scale flash flood forecasting system for the Kingdom of Saudi Arabia during a severe precipitation event in Jeddah on 24 November 2022, the most intense ever documented in the region. This event resulted in widespread flash floods affecting both urban and rural areas of Jeddah.

The study compared the atmospheric component forecast with the NASA satellite precipitation product (IMERG-Late) and radar-based rainfall estimates, adjusted for bias using in situ gauge observations. Because hydrological observations were absent, we utilized as a reference the discharge predicted from our system using input gauge-adjusted radar-rainfall estimates, which represent the benchmark precipitation, to evaluate the accuracy of the flood forecasts based on the Weather Research and Forecasting (WRF) model. Additionally, the study evaluated the effectiveness of the warning system by comparing its forecasts to publicly reported information on localized flood incidents at the street or neighborhood level obtained through “crowdsourcing.”

The findings indicated a temporal and spatial agreement between the forecast precipitation from WRF and the bias-adjusted radar-rainfall estimates. In contrast, the IMERG-Late data tended to overestimate precipitation in alignment with previous satellite validation studies. The comparison of flood quantiles for the November 24 event revealed a close correspondence between the WRF-driven flood peak discharge properties and those derived based on the gauge-adjusted radar-rainfall estimates. Minimal differences were observed in flood characteristics (such as hydrograph peak, timing, and volume) when using WRF-forecast precipitation versus radar-based benchmark data. The simulated inundation effectively captured broad flooding patterns at the city level, with over 95% of reported incidents in city districts falling within the areas for which the operational system provided high or extreme warnings on November 23—more than 12 h in advance.

The importance of the Flash Flood Forecast System for Saudi Arabia cannot be overstated. In a region prone to sudden and intense rainfall events, the system is a proactive tool for disaster risk reduction. Providing accurate and timely forecasts empowers communities to prepare and respond effectively, ultimately saving lives and minimizing the impact of flooding on infrastructure. As demonstrated by its success during the Jeddah event, the system presents an efficient solution for forecasting the potential impacts of extreme rainfall events, particularly those that can cause significant flash floods in the urban areas of Saudi Arabia.