Precipitation Forecast Verification of the FFGS and SisPI Tools during the Impact of the Tropical Storm Isaias over the Dominican Republic ^†

Abstract

During 2020, the Dominican Republic received the impact of several tropical organisms, among those that generated the greatest losses in the country, Tropical Storm Isaias stands out because of the significant precipitation and flooding it caused. The study analyzes the ability of the products of Flash Flood Guidance System (FFGS) and the Nowcasting and Very Short Range Prediction System (Spanish acronym SisPI) for the quantitative precipitation forecast (QPF) of the rains generated by Isaias on 30 and 31 July 2020 over the Dominican Republic. Various traditional verification methods are used in the study. The results show that both numerical weather-based systems are powerful tools for the QPF, and also to contribute to the prevention and mitigation of disasters caused by the extreme hydro-meteorological event analyzed.

1. Introduction

As the Dominican Republic is in the path of tropical cyclones, it is frequently affected by these hydrometeors. To monitor and forecast hydrometeorological events, the country has various systems based on numerical weather models, for example the Haiti−Dominican Republic Flash Flood Guidance System (HDRFFGS, referred to as FFGS; https://public.wmo.int/en/projects/ffgs (accessed on 23 June 2021)) and the Nowcasting and Very Short Term Forecast System (known by its Spanish acronym SisPI). Results described in other studies have shown that weather forecasting systems from FFGS and SisPI have good ability to forecast precipitation and different meteorological variables, however in particular for the Dominican Republic there has not been an exhaustive evaluation of them.

The work presented here aims to verify the quantitative forecast of precipitation generated by FFGS and SisPI during the passage of Tropical Storm Isaías through the Dominican Republic. The evaluation is carried out on the HIRESW ARW (forecast) and HIRESW NMMB (forecast) products of the FFGS (https://www.nco.ncep.noaa.gov/pmb/products/hiresw/ (accessed on 23 June 2021)); and on the 3 km resolution domain of the SisPI. The precipitation measured at the surface stations and the precipitation estimate from the Global Precipitation Mission (GPM) [1] are used in the verification. A verification based on traditional and categorical methods is applied [2]. The document is structured as follows: the Materials and Methods section describes the FFGS, the SisPI and their respective precipitation forecast products used, followed by a brief description of Tropical Storm Isaias as well as the data and verification methods used. The Results and Discussion section presents the results and our analysis according to the evaluation carried out; and finally a Conclusions section is presented.

2. Materials and Methods

2.1. Flash Flood Guidance System

The FFGS product was developed by the Hydrologic Research Center (HRC) in San Dieg (CA, USA). The primary objective of the FFGS is to provide forecasters and disaster management agencies with real-time informational guidance products related to the threat of small-scale flash floods in a specific country or region. The FFGS provides products to support the development of flash flood warnings associated with rainfall events through the use of remotely sensing such as, radar and satellite-based rainfall estimates, numerical weather predictions and hydrological approaches.

The FFGS implemented in the Dominican Republic integrates products from the numerical forecast model WRF, specifically from two dynamic cores (NonHydrostatic Mesoescale Model − NMM and Advanced Research WRF − ARW). Recently products from the AROME model were also incorporated (https://donneespubliques.meteofrance.fr/?fond=produit&id_produit=241&id_rubrique=51 (accessed on 23 June 2021)). This study evaluates the WRF products of the high-resolution window for Puerto Rico, which is elaborated by the National Center for Environmental Prediction (NCEP) High-Resolution Window (HIRESW) Forecast System (http://www.nco.ncep.noaa.gov/pmb/products/hiresw/ (accessed on 23 June 2021)). Both WRF cores have 5 km of spatial resolution, while the microphysics is WRF single-moment microphysics class 6 (WSM6) for ARW and Ferrier-Aligo for NMM. These outputs are available with the 0600 and 1800 UTC initializations. Figure 1 shows the simulation domains that include the Dominican Republic.

2.2. Nowcasting and Very Short Term Forecast System

SisPI is a numerical forecasting system that uses the WRF with the ARW core. It was developed by the Center for Atmospheric Physics of the Cuban Meteorological Institute (INSMET, in Spanish) [3,4] and implemented in the National Meteorological Office (ONAMET) of the Dominican Republic in 2019. The objective of SisPI is to provide numerical precipitation to support meteorological warning and act as a source for hydrological forecasting systems. In this case, the WRF is run for three nested domains of 27, 9 and 3 km of spatial resolution with the following configurations in microphysics: WSM5, WSM5, double moment Morrison; cumulus: Grell-Freitas, Grell-Freitas, not activated and Planetary Boundary Layer (PBL): Mellor-Yamada-Janjic, Mellor-Yamada-Janjic and Mellor-Yamada-Janjic, respectively. The SisPI runs four times using the GFS outputs at 0000, 0600, 1200 and 1800 UTC as initial and boundary conditions. Figure 2 shows the simulation domains for the Dominican Republic. The operational outputs can be consulted at http://186.149.199.244/sispi.php (accessed on 3 July 2021).

2.3. Isaias

Isaías was the ninth named tropical storm and the second hurricane of the 2020 cyclone season. This tropical cyclone was formed from a tropical wave that left the coasts of Africa on 23 July 2020. Several days before reaching a closed circulation and being named as a tropical cyclone by the National Hurricane Center (NHC), Isaías had already registered tropical storm force winds. Isaias impacted the Dominican Republic with its center entering through the San Pedro de Macorís province around noon on Thursday, 30 July with maximum sustained winds of up to 95 km/h, and then moving northwest over the Dominican territory during the afternoon and night until it exited the national territory around midnight on Thursday. During its transit over the Dominican Republic, this tropical storm caused heavy rains accompanied by electrical storms and winds with tropical storm force (https://www.nhc.noaa.gov/data/tcr/AL092020_Isaias.pdf (accessed on 4 July 2021)). The track followed by Isaias is shown in Figure 3.

Although its center traveled through Dominican territory for about 12 h, from noon on Thursday 30 to midnight the same day, the meteorological effects began to be felt indirectly a day before and continued to affect the country a day after its passage, due to the huge cloud field that accompanied this storm. The accumulated total of precipitation throughout this period of impact (both direct and indirect) translated into a maximum of 327.6 mm of rainfall recorded at the Sabana de la Mar meteorological station, followed by Samana where an accumulated 300.4 mm was recorded, during 29–31 July being, therefore, the east and northeast of the country were the areas that suffered the greatest rainfall impact due to the passage of this tropical storm (http://onamet.gob.do/index.php/pronosticos/temporada-ciclonica?download=4088:temporada-ciclonica-2020 (accessed on 7 July 2021)).

2.4. Data Used and Verification Methods

The verification process was carried out using 24-h precipitation records from 57 surface meteorological stations. These data were used to evaluate the satellite precipitation estimate generated by the GPM product in order to be able to use it as verification reference for the QPFs. In particular the Integrated Multi-satellite Retrievals for GPM (Integrated Multi-satellite Retrievals − IMERG) Final Precipitation L3 product (version 06) was used with temporal resolution of 30 min and spatial resolution of 0.1 degree × 0.1 degree [1].

Regarding verification methods, the following were used: (i) the traditional method based on point-to-point computing of different statistical metrics and (ii) the categorical verification method based on grid-to-grid computing of the hits, misses, correct rejection and false alarms parameters (see [2] for more details, and also [6] for an example of usage). The first method was applied with the station meteorological surface and the GPM data, while the second one was used with the FFGS products and SisPI, taking as precipitation reference the estimated by the GMP product. The bias, RMSE, critical success index (CSI), probability of detection (POD), and false alarms index (FAR) metrics were computed. The verification was applied to the FFGS forecast products and SisPI for 30 July 2020, using in both cases the outputs initialized at 0600 and 1800 UTC.

3. Results Discussion

An analysis of the results is presented below. As a first step, a comparison between the precipitation measurements of surface meteorological stations and the precipitation estimation of the GPM product is performed. The purpose of this is to know the correspondence between the real precipitation measured by the stations and the estimation of the same variable from the GPM. In this way, we would have an idea of how well the precipitation is represented via satellite over the Dominican Republic for this case study, which is very important since the estimation of precipitation by satellite is used for the spatial verification of the quantitative forecasts of precipitation employed by FFGS and produced by SisPI. This first part is followed by the presentation of the results of the evaluation by categories.

3.1. Comparison between GPM and Surface Stations

Figure 4 shows the 24 h accumulated rainfall estimated by the GPM product. The region where values of precipitation higher than 250 mm/24 h were reported is highlighted with a red square (includes Sabana del Mar 279.4 mm/24 h, Samana 276.1 mm/24 h). The precipitation estimated by GPM in the same region takes values between 100 and 200 mm/24 h showing a clear underestimation. A more detailed analysis can be done by observing Figure 5, where the real values (a) and the estimated values are presented on the coordinates of the stations (b). Indeed, it is observed that an underestimation predominates over the entire Dominican territory. Figure 5c,d present the bias and RMSE values. Notice that for Sabana del Mar and Samana a bias of −100 mm is obtained.

From another point of view, in Figure 5a,b it can be observed that although the quantitative errors of the estimation are high, principally in those places where heavy rain occured, in terms of where precipitation was reported or not, we can say that a correspondence exists between the GPM products and the data of the meteorological surface stations. Notice that GPM estimated the higher values in the same places that the highest preciptations were reported and a similar behavior occurs for the lower precipitation values.

Taking into account the above, it can be concluded that it is feasible to use the estimation of the precipitation of the GPM product to carry out a spatial verification in order to have a better coverage of the country, especially in those areas where there are no on-ground observations.

3.2. Categorical Verification

In order to know and characterize the behavior of the forecast products used by the FFGS as well as the forecast generated by SisPI in the quantitative prediction of rain for 24 h, the evaluation by categories is applied for different precipitation thresholds: 0.1, 50, 100, 150 and 200 mm. With the parameters hits, misses, corrected negative and false alarms calculated, the critical success index is obtained.

Table 1. CSI values for the 24 h rainfall forecast of the HIRESW-ARW, HIRESW-NMMB and SisPI.

Threshold	HIRESW-ARW 0600/1800		HIRESW-NMMB 0600/1800		SisPI 0600/1800
0.1	0.725	0.851	0.824	0.852	0.668	0.702
50	0.457	0.484	0.263	0.269	0.397	0.472
100	0.138	0.274	0.042	0.136	0.317	0.254
150	0.038	0.160	0.003	0.132	0.177	0.129
200	0.025	0.1	0.0	0.048	0.041	0.009

shows the values of this index for each forecast and each mentioned threshold. Notice that in the rain or no rain category (0.1 threshold) the HIRESW-NMMB presents the best performance. However, when we look at the categories that comprise the extreme values, the HIRESW-ARW and the SisPI have better skill.

A complete picture of the behavior of the CSI, the POD and the FAR for each initialization 0600 and 1800 UTC and, for each forecast term is shown in Figure 6 Again, the forecast obtained with HIRESW-NMMB, which reaches values of POD greater than 0.8 in various forecast periods and values greater than 0.4 for the CSI, stands out. It is also the system that produces the fewest false alarms. On the other hand, the most discrete values are obtained with SisPI, which presents as the greatest deficiency a high value of the false alarm index. In relation to this difference in false alarms between HIRESW-ARW, HIRESW-NMMB and SisPI, it can be pointed out as a possible influencing factor the fact that the SisPI does not have a data assimilation system.

Figure 7 and Figure 8 allow an approach to the behavior of the evaluation metrics from the spatial point of view. Figure 7 shows the spatial distribution of hits, misses, corrected negative and false alarms for the forecast for 30 July 2020 1400 UTC and 31 July 2020 0500 UTC, generated by the three systems with initialization 0600 UTC. It is evident that the highest number of hits is exhibited by the HIRESW-NMMB, at the same time that HIRESW-ARW and SisPI show higher areas of misses and false alarms.

In Figure 8, where the same metrics are shown but for the runs initialized at 1800 UTC and the forecast periods, an increase in the hitting areas is observed, with HIRESW-NMMB still being the system with the best results.

4. Conclusions

In the work presented, a first verification of the quantitative forecast of precipitation which is obtained from the HIRESW-ARW and HIRESW-NMMB forecast products used by the FFGS and the forecast generated by SisPI was carried out. In particular, the evaluation was applied to the Tropical Storm Isaias that affected the Dominican Republic during 29–31 July 2020. Although it is necessary to carry out an evaluative exercise considering more case studies, for the meteorological situation analyzed in this study, some preliminary conclusions can be put forth:

• In general, the three evaluated forecast systems showed good ability to forecast the rainy areas of Tropical Storm Isaias.
• For the forecast of rain occurrence, the HIRESW-NMMB presents the best results, with a probability of detection that reaches a value of 0.8.
• The HIRESW-ARW and SisPI systems have better performance than the HIRESW-NMMB for heavy rain values.
• SisPI presents the highest FAR index values.