*Article* **Evaluating the Latest IMERG Products in a Subtropical Climate: The Case of Paraná State, Brazil**

**Jéssica G. Nascimento 1,\*, Daniel Althoff <sup>2</sup> , Helizani C. Bazame <sup>1</sup> , Christopher M. U. Neale <sup>3</sup> , Sergio N. Duarte <sup>1</sup> , Anderson L. Ruhoff <sup>4</sup> and Ivo Z. Gonçalves <sup>1</sup>**


**Abstract:** The lack of measurement of precipitation in large areas using fine-resolution data is a limitation in water management, particularly in developing countries. However, Version 6 of the Integrated Multi-satellitE Retrievals for GPM (IMERG) has provided a new source of precipitation information with high spatial and temporal resolution. In this study, the performance of the GPM products (Final run) in the state of Paraná, located in the southern region of Brazil, from June 2000 to December 2018 was evaluated. The daily and monthly products of IMERG were compared to the gauge data spatially distributed across the study area. Quantitative and qualitative metrics were used to analyze the performance of IMERG products to detect precipitation events and anomalies. In general, the products performed positively in the estimation of monthly rainfall events, both in volume and spatial distribution, and demonstrated limited performance for daily events and anomalies, mainly in mountainous regions (coast and southwest). This may be related to the orographic rainfall in these regions, associating the intensity of the rain, and the topography. IMERG products can be considered as a source of precipitation data, especially on a monthly scale. Product calibrations are suggested for use on a daily scale and for time-series analysis.

**Keywords:** remote sensing in hydrology; precipitation; performance evaluation; GPM

#### **1. Introduction**

Precipitation plays a fundamental role in the hydrological cycle. It is considered the main water source input in the soil water balance and runoff and is used as an input in hydrological and climatological modeling. In the management of water resources, knowledge of the volume and intensity of precipitation is essential for the prediction of floods and droughts, the distribution of water for urban and industrial uses, and the planning of irrigation in agriculture and hydraulic infrastructure.

Precipitation can be measured by gauges, sensors onboard satellites, and radars [1–3]. Precipitation gauges are fundamental instruments, and their observations are considered as a reference in many studies [4]. However, to represent spatiotemporal variability of intensity and type of occurrence of precipitation, a dense measuring network is necessary with long-period information, which unfortunately is not the reality in many regions of the world [5]. In South American countries, monitoring by gauges is limited in terms of infrastructure, maintenance, density, and frequency of observations [6,7].

Regarding indirect methods, weather radars provide precipitation estimates with high spatial and temporal resolution but have limited accuracy in mountainous regions and cold

**Citation:** G. Nascimento, J.; Althoff, D.; C. Bazame, H.; M. U. Neale, C.; N. Duarte, S.; L. Ruhoff, A.; Z. Gonçalves, I. Evaluating the Latest IMERG Products in a Subtropical Climate: The Case of Paraná State, Brazil. *Remote Sens.* **2021**, *13*, 906. https:// doi.org/10.3390/rs13050906

Academic Editors: Magaly Koch and Weili Duan

Received: 27 January 2021 Accepted: 23 February 2021 Published: 28 February 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

climates [5,8]. On the other hand, satellite estimates of precipitation provide vast spatial and temporal coverage and are freely available. Over the last two decades, several satellite precipitation products have been developed, such as Tropical Rainfall Measuring Mission (TRMM) [9]; Climate Prediction Center Morphing Method (CMORPH) [10]; Global Satellite Mapping of Precipitation (GSMaP) [11]; Climate Hazards Group Infrared Precipitation with Stations (CHIRPS) [12]; and Multi-Source Weighted-Ensemble Precipitation (MSWEP) [13].

The TRMM Multi-satellite Precipitation Analysis (TMPA) algorithm combines precipitation estimates from satellite systems with data measured on the Earth's surface, to provide a calibrated final product and results in the "best" satellite estimate [14]. A successor to TRMM, the Global Precipitation Measurement (GPM) was launched in 2014, on a joint mission between NASA (National Aeronautics and Space Administration) and JAXA (Japan Aerospace Exploration Agency) and continues to offer products to this day. The GPM constellation consists of the first Dual-frequency phased array Precipitation Radar and a GPM Microwave Imager, which represent the most advanced versions compared to the Precipitation Radar (PR) and the TRMM Microwave Imager (TMI), onboard the TRMM satellite [15]. Relevant improvements in the GPM products include an increase in latitudinal coverage (global coverage of 60◦ N/S) and the detection of heavy rain, light rain, and snow [5,6]. In the era of GPM, the Integrated Multi-satellite Retrievals for GPM (IMERG) algorithm operates, intending to calibrate, unite, and interpolate satellite precipitation estimates with data from gauges [16].

Many studies have analyzed and compared the TRMM data to those obtained by gauge stations [6,17–20], which allowed for advances in the application of remote sensing in determining the volume and behavior of precipitation in several countries. Given that precipitation represents the most important parameter of the hydrologic cycle, it can directly affect physical, chemical, and biological processes. The spatial and temporal distribution of the precipitation is key for the understanding of hydrologic responses in watersheds, for example, runoff, streamflow, and flooding. Thus, the accuracy of precipitation data is essential for the good performance of hydrological models and inconsistencies in its estimates can negatively affect the hydrologic investigation.

Previous studies have reported some limitations of the IMERG products [21] related to its low performance in estimating the precipitation over North China, where snowfall events can affect the precipitation estimates from satellite products. The authors also reported the lower accuracy of the satellite precipitation estimates in areas of high altitudes or in arid and semiarid climates. Evaluating the IMERG precipitation compared to groundbased data, [22] observed a tendency for restimating the precipitation using the sub-daily products and overestimating the maximum rainfall in monthly variations over the Indian subcontinent. In a seasonal analysis, [21,23] found that IMERG performed better in warm and wet seasons. In a subtropical climate, [24] concluded that IMERG performed well for detecting precipitation events with a limitation in representing the amount of rainfall at sub-daily scales, while in tropical climates [25,26] demonstrated that it is essential to calibrate the IMERG products to reduce random and systematic errors.

On the other hand, studies comparing the IMERG to different satellite products, highlighted the better accuracy of IMERG in estimating precipitation. [27] found better performance of IMERG in estimating precipitation on a global scale compared to TMPA products. The better performance of IMERG's products was also observed by [28] concerning the TMPA in estimating precipitation in all regions of Brazil. Recently, [29] observed better performance of IMERG-Late version 6 products compared to IMERG-Early, GSMaP-NRT, GSMaP-MVK, TMPA-RT, and PERSIANN-CCS products on a global scale The better performance of IMERG in relation to other remote sensing products, reported in previous studies, supports the analysis of the performance of its products in different climates and geographic scales.The quality of IMERG products was improved over time by increasing the number of passive microwave samples [30]. The latest version of the IMERG algorithm (version 6), made available to the public in October 2019, combines the reanalysis of precipitation estimated by satellites between 2000–2014 by TMPA and in the subsequent

period by GPM, resulting in 19 years of information so far. Its products have a spatial resolution of 0.1◦ and a 30-min temporal resolution [31]. In this way, trend analysis and analysis of extreme events could be carried out with greater accuracy. In addition, the performance of climatological and hydrological models could be improved with greater detail of recent precipitation information. Thus, studies evaluating the performance of IMERG version 6 products are important and promising at regional scales. Brazil is a country of continental dimensions, with different micro-climates and rainfall patterns throughout its territory. Among its states, Paraná, located in the south of the country in the Paraná River Basin region, is the most important socio-economically hydrologic region in Brazil [32]. This basin has the largest hydroelectric infrastructure in the country, which is responsible for approximately 44.6% of the electric energy production and transmission system in Brazil [33]. Thus, the determination of precipitation in this region is essential for forecasting with hydrological and climatological monitoring models. Also, it is important for this region to detect anomalies related to excess or deficit of precipitation, which can compromise the supply of energy and hydraulic structures.

Thus, the purpose of this study is to provide a comprehensive assessment of IMERG (version 6) precipitation estimates over a subtropical region, specifically the objectives of the study are (1) to evaluate the performance of IMERG's daily and monthly products, and (2) to evaluate IMERG's performance in detecting monthly anomalies in Paraná using observations from a dense network of precipitation gauges. This study is expected to provide a reference for the use of IMERG products in monthly and daily temporal resolutions and further contribute to improvements of the satellite precipitation algorithm. The remainder of this article is organized into the following topics: Section 2 describes the study area, the precipitation data sets used, the metrics used, and the detailed methodology; Section 3 presents the main results and the discussion, and Section 4 reports the main conclusions of the study.

#### **2. Material and Methods**

#### *2.1. Study Region*

Statistics (IBGE).

2.2.1. Observed Data: Ground Gauge

from 1 June 2000 to 31 December 2018.

*2.2. Data* 

The study area is the state of Paraná in the south region of Brazil (Figure 1). Paraná occupies an area of 199,315 km<sup>2</sup> and covers 399 municipalities [34]. According to the Köppen classification, carried out by [35], the state is in the transition from tropical and subtropical climates where the humid subtropical Cfa (hot summer) and Cfb (warm summer) predominate across 61.7% and 37.0% of the state, respectively. Because of its extensive area, there is a great diversity in terms of climate, soil types, vegetation, and agricultural use. The main biomes that constitute the state are the Atlantic Forest and the Cerrado. The predominant crops are maize, soy, and sugar cane. Most of Paraná's relief is found at altitudes above 600 m (Figure 1), subdivided into four Morpho-sculptural Units [36]. *Remote Sens.* **2021**, *13*, x FOR PEER REVIEW 4 of 20

**Figure 1.** Study area and gauges used to validate the Integrated Multi-satellite Retrievals for GPM (IMERG) estimates of precipitation at daily and monthly time-steps. **Figure 1.** Study area and gauges used to validate the Integrated Multi-satellite Retrievals for GPM (IMERG) estimates of precipitation at daily and monthly time-steps.

mm. Summer is the season with the highest rainfall in South America, including the subtropical region [37,38]. The geographic mesoregions of Paraná, demarcated by the Brazilian Institute of Geography and Statistics (IBGE) and shown in Figure 2, were considered for the regional analysis of the performance of the IMERG products over the study area.

**Figure 2.** Mesoregions of Paraná state demarcated by the Brazilian Institute of Geography and

The precipitation data used were acquired from 511 gauges (Figure 1) of the National Water Agency (ANA), through the Hidroweb Portal (http://www.snirh.gov.br/hidroweb/serieshistoricas, accessed on 1 March 2020), which is available to the public after consistency checks. The analysis of consistency of precipitation data is aimed at the identification and correction of errors related to data collection, as well as filling gaps in the monthly precipitation series. All available daily data were used for analysis on a daily scale, while, for analysis on a monthly scale, monthly totals were excluded when there were more than 5% of daily failures in the corresponding month. The daily data were accumulated (totaled) to produce the monthly information. The analyzed time series was

Precipitation in Paraná varies spatially, with an annual average between 1300–2200 mm. Summer is the season with the highest rainfall in South America, including the subtropical region [37,38]. The geographic mesoregions of Paraná, demarcated by the Brazilian Institute of Geography and Statistics (IBGE) and shown in Figure 2, were considered for the regional analysis of the performance of the IMERG products over the study area. mm. Summer is the season with the highest rainfall in South America, including the subtropical region [37,38]. The geographic mesoregions of Paraná, demarcated by the Brazilian Institute of Geography and Statistics (IBGE) and shown in Figure 2, were considered for the regional analysis of the performance of the IMERG products over the study area.

(IMERG) estimates of precipitation at daily and monthly time-steps.

**Figure 1.** Study area and gauges used to validate the Integrated Multi-satellite Retrievals for GPM

Precipitation in Paraná varies spatially, with an annual average between 1300–2200

*Remote Sens.* **2021**, *13*, x FOR PEER REVIEW 4 of 20

**Figure 2.** Mesoregions of Paraná state demarcated by the Brazilian Institute of Geography and **Figure 2.** Mesoregions of Paraná state demarcated by the Brazilian Institute of Geography and Statistics (IBGE).

#### Statistics (IBGE). *2.2. Data*

#### 2.2.1. Observed Data: Ground Gauge

*2.2. Data*  2.2.1. Observed Data: Ground Gauge The precipitation data used were acquired from 511 gauges (Figure 1) of the National Water Agency (ANA), through the Hidroweb Portal (http://www.snirh.gov.br/hidroweb/serieshistoricas, accessed on 1 March 2020), which is available to the public after consistency checks. The analysis of consistency of precipitation data is aimed at the identification and correction of errors related to data collection, as well as filling gaps in the monthly precipitation series. All available daily data were used for analysis on a daily The precipitation data used were acquired from 511 gauges (Figure 1) of the National Water Agency (ANA), through the Hidroweb Portal (http://www.snirh.gov.br/hidroweb/ serieshistoricas, accessed on 1 March 2020), which is available to the public after consistency checks. The analysis of consistency of precipitation data is aimed at the identification and correction of errors related to data collection, as well as filling gaps in the monthly precipitation series. All available daily data were used for analysis on a daily scale, while, for analysis on a monthly scale, monthly totals were excluded when there were more than 5% of daily failures in the corresponding month. The daily data were accumulated (totaled) to produce the monthly information. The analyzed time series was from 1 June 2000 to 31 December 2018.

#### scale, while, for analysis on a monthly scale, monthly totals were excluded when there were more than 5% of daily failures in the corresponding month. The daily data were 2.2.2. Satellite Data: IMERG

accumulated (totaled) to produce the monthly information. The analyzed time series was from 1 June 2000 to 31 December 2018. Precipitation data from remote sensing were acquired as daily and monthly temporal resolution and 0.1◦ spatial resolution, from the satellite constellation of the GPM, IMERG version 6 product, distributed by Goddard Earth Sciences Data and Information Services Center Distributed Active Archive Center (GES DISC DAAC), available online on http://mirador.gsfc.nasa.gov/com, accessed on 15 March 2020. The daily and monthly products "Final Run" were used, and the time series analyzed was coincident with that of the gauges (1 June 2000 to 31 December 2018). The IMERG algorithm operates to intercalibrate, merge, and interpolate all satellite microwave precipitation estimates, microwave-calibrated infrared estimates, gauge observations, and other data from potential sensors from the TRMM and GPM eras [31]. The "Final Run" product includes microwave-infrared estimates without gauge adjustment and the calibrated product based on the Global Precipitation Climatology Centre monthly gauge analysis [30]. In general, the "Final Run" products present bias correction and more accurate results than the other products supplied in near real-time (Early and Late Run) [39].

#### 2.2.3. Performance Analyses

The IMERG products were compared to ground-based gauge using statistical indices. For this, the IMERG data were sampled at the exact location of the gauge stations. The performance of the IMERG products assessed using the coefficient of determination (R<sup>2</sup> ), the Kling-Gupta efficiency index (KGE), the error skewness (SK), the mean error (MBE), the mean absolute error (MAE), and the root of the mean square error (RMSE). The data qualitative assessment was performed using the categorical skills metrics: probability of detection (POD), false alarm ratio (FAR), and critical success index (CSI). Such metrics are used in several studies to assess the performance of satellite products [19,39–41]. The equations for the metrics used are shown in Table 1. The rainfall threshold was considered as amounts higher than 1 mm day−<sup>1</sup> , as used by [42].

**Table 1.** Summary of statistical indices used to evaluate the satellite precipitation products.


\* where, Oi is the observed data of gauges of order i; Pi is the estimated order data (IMERG) of order i; r = rPearson, *α* is the ratio between simulated variance and observed variance, and *β* is the ratio between simulated mean and observed mean; P is the average of the estimated data (IMERG); O is the average of the observed data (gauges); M<sup>o</sup> is the median error of satellite precipitation estimates; Hits are the days when IMERG and the station recorded rain; FalseAlarm are the days when IMERG recorded rain, but the gauges did not; Misses are the days when IMERG did not register rain, but the gauges did.

> R <sup>2</sup> measures how much of the variability of one variable can be explained by the other variable. KGE is an objective index which assesses error in terms of correlation, variability, and bias. MBE is a simple average of the error and informs if the estimate on average over- or underestimate the observed data. MAE and RMSE are error metrics in the same units as the observed variable and represent the average magnitude of the error. In this case, RMSE penalizes large errors by squaring the differences between those observed and those estimated. SK represents the distributions of errors. A positive [negative] SK value indicates a median error is smaller [larger] than the mean error, i.e., there is a higher frequency of errors smaller [larger] than the mean error. If the mean error (MBE) is centered close to 0, a positive [negative] SK value can indicate a higher frequency of underestimations [overestimations]. Errors were calculated by subtracting observed from predicted values. POD indicates the fraction of rain events detected correctly with the total number of events detected by satellite; FAR measures the fraction of occurrences of unreal events among the total number of events detected by satellite; CSI denotes the proportion of rain events correctly detected by satellite to the total number of observed events.

> Additionally, the IMERG's performance for estimating the precipitation was evaluated using a confusion matrix, for the daily and monthly products. The columns in this matrix shows the frequency distribution of IMERG precipitation amount within each ground

gauge precipitation amount class. In the perfect scenario, the matrix should be antidiagonal and present a value equal to 1 along the antidiagonal line and 0 for the all the other elements [43].

#### 2.2.4. Analysis of Anomalies

For the analysis of anomalies, the monthly values of the gauges and the monthly products of IMERG were used. The investigation of the volume of precipitation for each month concerning its average (2000–2018), was carried out by calculating the normalized anomalies of precipitation with standard deviation (Equation (1)) [44].

$$\chi\_{\text{Anomaly}} = \frac{(\chi\_{\text{i}} - \overline{\chi}\_{2000-2018})}{\sigma\_{2000-2018}} \tag{1}$$

where, Xi is the month of the year analyzed, X is the monthly average of the 2000–2018 series, and σ is the monthly standard deviation of the time series. The mean (X) and standard deviation (σ) were calculated using Equations (2) and (3), respectively.

$$\sigma = \sqrt{\frac{\sum\_{i=1}^{n} \left(\chi - \overline{\chi}\right)^{2}}{n}} \tag{2}$$

$$\overline{X} = \frac{\sum\_{\mathbf{i}=1}^{n} (\mathbf{X}\_{\mathbf{i}})}{\mathbf{n}} \tag{3}$$

Significant anomalies (95% confidence interval) were XAnomaly values ranging between ±1.96 as adopted by [45–47]. Thus, a XAnomaly > |1.96| gives us a 95% confidence to assume the observations as an anomaly, i.e., different than the historical mean.

A schematic diagram is presented in Figure 3, with the data and performance for the overall evaluation process of this study. *Remote Sens.* **2021**, *13*, x FOR PEER REVIEW 7 of 20

**Figure 3.** Schematic diagram of the source, resolution, and period of the data and metrics used for **Figure 3.** Schematic diagram of the source, resolution, and period of the data and metrics used for evaluation of the precipitation estimative by IMERG products.

#### evaluation of the precipitation estimative by IMERG products. **3. Results**

son (Figure 4).

and those estimated by IMERG.

observation close to 0 mm month−1.

#### **3. Results**  *3.1. Temporal and Spatial Distribution of Precipitation*

*3.1. Temporal and Spatial Distribution of Precipitation*  Figure 4 shows the average monthly precipitation (mm month−1) observed by gauges and estimated by IMERG, between June 2000 and December 2018. The volume and spatial distribution of observed and estimated precipitation were consistent over all months of Figure 4 shows the average monthly precipitation (mm month−<sup>1</sup> ) observed by gauges and estimated by IMERG, between June 2000 and December 2018. The volume and spatial distribution of observed and estimated precipitation were consistent over all months of the year. The rainfall distribution density curve, which relates the precipitated volume to the

the year. The rainfall distribution density curve, which relates the precipitated volume to

The highest frequency of precipitation observations occurs between 125–200 mm month−1 from October to March which is the wet season and is spatially well distributed in Paraná (Figure 4). In January and February, precipitation above 300 mm month−1 occurs in the coastal area. In October, the same behavior is observed in the southwest of the state. The IMERG data overestimated the high values of monthly precipitation recorded by the gauges, presenting a higher frequency of the monthly precipitation peaks in the wet sea-

The dry season occurs between April and September, during which the precipitation decreases from the south to the north across the state, towards the Tropic of Capricorn, with a greater frequency observed between 0–125 mm month−1. In this period, the months of July and August stand out as the driest months of the year, reaching a frequency of

observation frequency, had a similar distribution of the observed data from the gauges and those estimated by IMERG. *Remote Sens.* **2021**, *13*, x FOR PEER REVIEW 8 of 20

**Figure 4.** Average monthly rainfall observed during the study period for gauges and by IMERG, and density curve of monthly observations. **Figure 4.** Average monthly rainfall observed during the study period for gauges and by IMERG, and density curve of monthly observations.

*3.2. Daily and Monthly Evaluation of IMERG Products*  The relationship between IMERG daily and monthly precipitation and the data observed by gauges are presented in Figure 5. For the daily values, it is observed that IMERG overestimates the values, with a low coefficient of determination (R2 = 0.19). At the monthly scale, precipitation values are closer to those observed, with R2 of 0.73. The highest frequency of precipitation observations occurs between 125–200 mm month−<sup>1</sup> from October to March which is the wet season and is spatially well distributed in Paraná (Figure 4). In January and February, precipitation above 300 mm month−<sup>1</sup> occurs in the coastal area. In October, the same behavior is observed in the southwest of the state. The IMERG data overestimated the high values of monthly precipitation recorded by the gauges, presenting a higher frequency of the monthly precipitation peaks in the wet season (Figure 4).

The dry season occurs between April and September, during which the precipitation decreases from the south to the north across the state, towards the Tropic of Capricorn, with a greater frequency observed between 0–125 mm month−<sup>1</sup> . In this period, the months of July and August stand out as the driest months of the year, reaching a frequency of observation close to 0 mm month−<sup>1</sup> .

#### *3.2. Daily and Monthly Evaluation of IMERG Products*

The relationship between IMERG daily and monthly precipitation and the data observed by gauges are presented in Figure 5. For the daily values, it is observed that IMERG

**Figure 5.** Regressions between daily and monthly data observed by gauges and IMERG.

overestimates the values, with a low coefficient of determination (R<sup>2</sup> = 0.19). At the monthly scale, precipitation values are closer to those observed, with R<sup>2</sup> of 0.73. served by gauges are presented in Figure 5. For the daily values, it is observed that IMERG overestimates the values, with a low coefficient of determination (R2 = 0.19). At the monthly scale, precipitation values are closer to those observed, with R2 of 0.73.

The relationship between IMERG daily and monthly precipitation and the data ob-

**Figure 4.** Average monthly rainfall observed during the study period for gauges and by IMERG,

and density curve of monthly observations.

*3.2. Daily and Monthly Evaluation of IMERG Products* 

*Remote Sens.* **2021**, *13*, x FOR PEER REVIEW 8 of 20

**Figure 5.** Regressions between daily and monthly data observed by gauges and IMERG. **Figure 5.** Regressions between daily and monthly data observed by gauges and IMERG.

Based on the general summaries of the metrics used in this study, presented in Figure 6, IMERG showed better performance for estimating monthly precipitation, compared to the daily product. The high KGE value (0.81) indicates a strong correlation between the monthly products of IMERG and the precipitation gauge data, which shows its ability to quantify monthly precipitation in the humid subtropical region. A lower accuracy is observed for daily products (KGE = 0.43), indicating a lower correlation between satellite precipitation data and pluviometers. This is due mainly to the high variability of precipitation over small areas on a daily scale. Our findings indicate that both IMERG products (daily and monthly) overestimate the observed precipitation, with a bias (MBE) of 0.19 and 5.98 mm, in the daily and monthly products, respectively. Regarding the errors, IMERG presented MAE of 5.64 and 35.90 mm and accuracy (RMSE) of 13.10 and 50.12 mm for daily and monthly products, respectively. Based on the general summaries of the metrics used in this study, presented in Figure 6, IMERG showed better performance for estimating monthly precipitation, compared to the daily product. The high KGE value (0.81) indicates a strong correlation between the monthly products of IMERG and the precipitation gauge data, which shows its ability to quantify monthly precipitation in the humid subtropical region. A lower accuracy is observed for daily products (KGE = 0.43), indicating a lower correlation between satellite precipitation data and pluviometers. This is due mainly to the high variability of precipitation over small areas on a daily scale. Our findings indicate that both IMERG products (daily and monthly) overestimate the observed precipitation, with a bias (MBE) of 0.19 and 5.98 mm, in the daily and monthly products, respectively. Regarding the errors, IMERG presented MAE of 5.64 and 35.90 mm and accuracy (RMSE) of 13.10 and 50.12 mm for daily and monthly products, respectively.

**Figure 6.** Error distribution and performance criteria of the IMERG products at daily and at monthly time scales. **Figure 6.** Error distribution and performance criteria of the IMERG products at daily and at monthly time scales.

0). In a daily resolution, IMERG demonstrated a limited performance to detect rain events, with CSI of 0.39, a detection probability of 64%, and the risk of false alarm of 50%.

To better understand the accuracy of the IMERG products in different ranges of precipitation observations, a confusion matrix is shown in Figure 7. At the daily scale, the IMERG generally underestimate rainfall events, as seen by a large number of rainfall events greater than 1 mm) predicted in the range 0–1. This agrees with the positive SK value at daily scale (Figure 6). Considering the positive values of the error metrics showed in Figure 6 (MBE, MAE and RMSE), the positive errors of IMERG daily products could

In contrast, monthly precipitation estimates of the IMERG generally overestimate events as observed in gauge stations (negative SK in Figure 6). Only for months with higher rainfall rates (>200 mm) the IMERG product underestimates observed data. The underprediction at both daily and monthly scale can be attributed to the IMERG spatial

offset its negative errors.

Regarding the ability to detect rain at the monthly resolution, IMERG had a nearly perfect performance, with POD equal to 1, CSI of 0.99, and FAR of 0.01 (very close to ideal, 0). In a daily resolution, IMERG demonstrated a limited performance to detect rain events, with CSI of 0.39, a detection probability of 64%, and the risk of false alarm of 50%. *Remote Sens.* **2021**, *13*, x FOR PEER REVIEW 10 of 20

> To better understand the accuracy of the IMERG products in different ranges of precipitation observations, a confusion matrix is shown in Figure 7. At the daily scale, the IMERG generally underestimate rainfall events, as seen by a large number of rainfall events greater than 1 mm) predicted in the range 0–1. This agrees with the positive SK value at daily scale (Figure 6). Considering the positive values of the error metrics showed in Figure 6 (MBE, MAE and RMSE), the positive errors of IMERG daily products could offset its negative errors. and temporal resolution. Heavy rainfall events usually occur in shorter time intervals and small areas. Thus, sparse heavy rainfall events can be underestimated in the daily scale and not be accounted for in the monthly scale, even though the "Final run" relies on the calibration using monthly observations.

**Figure 7.** Confusion matrix for different precipitation ranges at daily and monthly scales. **Figure 7.** Confusion matrix for different precipitation ranges at daily and monthly scales.

The spatial distribution of the metrics for the Paraná regions (Figure 2) at daily and monthly scale are summarized in Table 2. Less accuracy was observed in the Southwest, West, Central-West, and Metropolitan regions in daily and monthly products. **Table 2.** Summary of error metrics on a daily and monthly scale of the satellite precipitation products in Paraná regions. The metrics were calculated based on mean areal precipitation.  **RMSE MAE MBE RMSE MAE MBE**  In contrast, monthly precipitation estimates of the IMERG generally overestimate events as observed in gauge stations (negative SK in Figure 6). Only for months with higher rainfall rates (>200 mm) the IMERG product underestimates observed data. The underprediction at both daily and monthly scale can be attributed to the IMERG spatial and temporal resolution. Heavy rainfall events usually occur in shorter time intervals and small areas. Thus, sparse heavy rainfall events can be underestimated in the daily scale and not be accounted for in the monthly scale, even though the "Final run" relies on the calibration using monthly observations.

**Region (mm day−1) (mm month−1)**  Central-South 13.70 6.19 0.02 48.50 36.30 0.83 Central-West 13.40 5.79 0.14 51.20 37.80 4.44 Central-East 11.90 5.24 0.18 43.20 31.90 5.52 The spatial distribution of the metrics for the Paraná regions (Figure 2) at daily and monthly scale are summarized in Table 2. Less accuracy was observed in the Southwest, West, Central-West, and Metropolitan regions in daily and monthly products.

Metropolitan 13.10 5.99 0.44 56.90 41.80 13.20 Northwest 12.40 5.14 0.31 50.50 36.90 9.37 **Table 2.** Summary of error metrics on a daily and monthly scale of the satellite precipitation products in Paraná regions. The metrics were calculated based on mean areal precipitation.


IMERG pixel covers an extensive area, large variability in precipitation is masked in areas

monthly products and over the southwest region for daily products where there are abrupt changes in elevation (Figure 1). The SK presented homogeneous spatial distribution

The spatial distribution of the error metrics for estimating daily and monthly rainfall by IMERG, for each gauge in the state of Paraná, are shown in Figure 8. Corroborating the results presented in Table 2, the MBE and MAE values are spatially well distributed in the study area with close values to the averages shown in Figure 5, for daily and monthly data. However, less accurate metrics were observed in the coastal areas (eastern part of the state), where IMERG presents, greater disagreements in some stations. Since the IMERG pixel covers an extensive area, large variability in precipitation is masked in areas where orographic effects are prevalent. This is evident near the coastal region for the daily monthly products and over the southwest region for daily products where there are abrupt changes in elevation (Figure 1). The SK presented homogeneous spatial distribution in a daily scale, and a greater number of negative values over the west region (ranging between 0 and −0.5), which suggest an overestimation of precipitation in this area. The KGE was also homogeneous across Paraná, with values close to 0.43 for the daily and 0.81 for the monthly products, as shown in Figure 8. *Remote Sens.* **2021**, *13*, x FOR PEER REVIEW 11 of 20 in a daily scale, and a greater number of negative values over the west region (ranging between 0 and −0.5), which suggest an overestimation of precipitation in this area. The KGE was also homogeneous across Paraná, with values close to 0.43 for the daily and 0.81 for the monthly products, as shown in Figure 8.

**Figure 8.** Spatial distribution of performance metrics on a daily and monthly scale. **Figure 8.** Spatial distribution of performance metrics on a daily and monthly scale.

The distribution of POD, CSI, and FAR (Figure 9) showed good performance of IMERG's products in detecting monthly rain events throughout Paraná, with values very close to ideals in the entire area. The performance of IMERG's products in estimating daily rainfall also was homogeneous across the state area, with the worst performance on the coast, in agreement with the statistical metrics. The distribution of POD, CSI, and FAR (Figure 9) showed good performance of IMERG's products in detecting monthly rain events throughout Paraná, with values very close to ideals in the entire area. The performance of IMERG's products in estimating daily rainfall also was homogeneous across the state area, with the worst performance on the coast, in agreement with the statistical metrics.

**Figure 9.** Spatial distribution of the qualitative metrics of IMERG performance on a daily and monthly scale. **Figure 9.** Spatial distribution of the qualitative metrics of IMERG performance on a daily and monthly scale.

Concerning the daily and monthly RMSE, higher values are found in the southwest and coastal areas of Parana, which correspond to areas with high volumes of rain in the autumn and summer, respectively, and the highest volumes of annual rainfall in Paraná (Figure 4). Thus, the precipitation estimates by IMERG performed better in the drier areas of the state. This behavior is better confirmed by a seasonal assessment of the errors. In Figure 10 the performance metrics for the monthly IMERG product grouped by seasons are presented. For example, IMERG showed better metrics in the winter (JJA = June, July, and August), which is the period with lowest rainfall rates. Likewise, the monthly predictions during autumn (MAM = March, April, and May) and spring (SON = September, October, and November) showed better scores than during summer (DJF = December, January, and February). Concerning the daily and monthly RMSE, higher values are found in the southwest and coastal areas of Parana, which correspond to areas with high volumes of rain in the autumn and summer, respectively, and the highest volumes of annual rainfall in Paraná (Figure 4). Thus, the precipitation estimates by IMERG performed better in the drier areas of the state. This behavior is better confirmed by a seasonal assessment of the errors. In Figure 10 the performance metrics for the monthly IMERG product grouped by seasons are presented. For example, IMERG showed better metrics in the winter (JJA = June, July, and August), which is the period with lowest rainfall rates. Likewise, the monthly predictions during autumn (MAM = March, April, and May) and spring (SON = September, October, and November) showed better scores than during summer (DJF = December, January, and February).

The orographic effect of the coastal region is not captured in any of the seasons. Only in winter (JJA) the errors were smaller because of the lower rainfall rates. Higher RMSEs were already expected during the wet season, because of the higher magnitude in rainfall rates. However, summer (DJF), also showed the highest positive bias, i.e., IMERG overestimates rainfall observed in gauge stations. The SK presented an overall tendency of median errors higher than the mean error (negative SK). This can be attributed to the IMERG product generally overestimating monthly rainfall rates (Figure 7) and resulting in negative SK, especially in seasons or regions with MBE closer to 0.

#### *3.3. Rainfall Anomalies between 2000 and 2018*

The spatial distribution of the performance of IMERG's monthly resolution products in detecting anomalies observed by the gauges is shown in Figure 11. In general, IMERG showed a limited performance for detecting anomalies across the state, considering +/−1.96 monthly standard deviation. The best performance was observed in the south-central region of Paraná, with POD above 0.75, CSI above 0.50, and FAR below 0.50. The worst performance occurred in the northeast region, the region with the lowest annual rainfall, and in the coastal and southwest regions, which correspond to the regions with the highest annual rainfall in the state (Figure 1), agreeing with the worst performance of the daily and monthly product metrics (Figure 8).

**Figure 10.** Seasonal distribution of the performance metrics for IMERG at a monthly scale. **Figure 10.** Seasonal distribution of the performance metrics for IMERG at a monthly scale.

*Remote Sens.* **2021**, *13*, x FOR PEER REVIEW 14 of 20

*3.3. Rainfall Anomalies between 2000 and 2018*  **Figure 11.** Spatial distribution of performance metrics for anomaly detection. **Figure 11.** Spatial distribution of performance metrics for anomaly detection.

The spatial distribution of the performance of IMERG's monthly resolution products in detecting anomalies observed by the gauges is shown in Figure 11. In general, IMERG showed a limited performance for detecting anomalies across the state, considering +/−1.96 monthly standard deviation. The best performance was observed in the southcentral region of Paraná, with POD above 0.75, CSI above 0.50, and FAR below 0.50. The In the analysis of the boxplot of the metrics (POD, CSI, and FAR) used to assess the performance of IMERG in detecting anomalies, a relative deviation from the mean and extreme values of CSI and FAR were observed (Figure 11). Therefore, eight stations were selected randomly, in different regions of the state, to detect anomalies by the gauges and by IMERG, between the years 2000 and 2018, shown in Figure 12. In the analysis of the boxplot of the metrics (POD, CSI, and FAR) used to assess theperformance of IMERG in detecting anomalies, a relative deviation from the mean and extreme values of CSI and FAR were observed (Figure 11). Therefore, eight stations were selected randomly, in different regions of the state, to detect anomalies by the gauges and by IMERG, between the years 2000 and 2018, shown in Figure 12.

worst performance occurred in the northeast region, the region with the lowest annual rainfall, and in the coastal and southwest regions, which correspond to the regions with the highest annual rainfall in the state (Figure 1), agreeing with the worst performance of the daily and monthly product metrics (Figure 8). Positive anomalies were detected by IMERG monthly products and gauges stations in all regions. In general, the gauges stations detected the anomalies with greater magnitude compared to those estimated by remote sensing. On the other hand, only IMERG detected negative anomalies, for example in the gauge stations 2349036, 2449040, and 2452062.

**Figure 12.** Anomalies for 9 stations selected at random. The dashed lines represent the values of

Positive anomalies were detected by IMERG monthly products and gauges stations in all regions. In general, the gauges stations detected the anomalies with greater magnitude compared to those estimated by remote sensing. On the other hand, only IMERG detected negative anomalies, for example in the gauge stations 2349036, 2449040, and

−1.96 and 1.96 σ (monthly standard deviation).

*4.1. Temporal and Spatial Distribution of Precipitation* 

2452062.

**4. Discussion** 

In the analysis of the boxplot of the metrics (POD, CSI, and FAR) used to assess the performance of IMERG in detecting anomalies, a relative deviation from the mean and extreme values of CSI and FAR were observed (Figure 11). Therefore, eight stations were selected randomly, in different regions of the state, to detect anomalies by the gauges and

**Figure 11.** Spatial distribution of performance metrics for anomaly detection.

by IMERG, between the years 2000 and 2018, shown in Figure 12.

**Figure 12.** Anomalies for 9 stations selected at random. The dashed lines represent the values of −1.96 and 1.96 σ (monthly standard deviation). **Figure 12.** Anomalies for 9 stations selected at random. The dashed lines represent the values of −1.96 and 1.96 σ (monthly standard deviation).

#### **4. Discussion**

#### Positive anomalies were detected by IMERG monthly products and gauges stations in all regions. In general, the gauges stations detected the anomalies with greater magni-*4.1. Temporal and Spatial Distribution of Precipitation*

tude compared to those estimated by remote sensing. On the other hand, only IMERG detected negative anomalies, for example in the gauge stations 2349036, 2449040, and 2452062. **4. Discussion**  *4.1. Temporal and Spatial Distribution of Precipitation*  During the summer months, the Paraná State region typically receives a humidity air mass that moves from the Amazon to the southwestern Atlantic region, defined as the South Atlantic Convergence Zone (SACZ) [48], which is directly connected to the South American monsoon system along a northwest–southeast axis. The precipitation observed in the southwest region in October, on the other hand, is related to convective and frontal complexes [32,49]. The definition of the summer as the wet season and the spatial and temporal distribution in the study area presented here is corroborated by previous studies [32,49–51], confirming the precipitation estimates by IMERG through remote sensing approaches.

#### *4.2. Daily and Monthly Evaluation of IMERG Products*

Intensity and volume on a daily scale provide important information in hydrological applications, such as frequency analysis, daily precipitation event detection, and irrigation planning. The overestimation by IMERG daily products compared to ground data was observed by [29] when evaluating IMERG, GSMaP, and PERSIANN products over the whole globe (~26, 44, and 23% of bias, respectively). However, the authors observed higher correlation for IMERG products (~0.6) compared to GSMaP and PERSIANN (~0.5 and 0.4, respectively). Also, [28] related the overestimation of TMPA, IMERG-F, and GSMaP over South Brazil, but a slightly lower values of RMSE and mean error (0.86 and 0.09 mm day−<sup>1</sup> , respectively) for GSMaP compared to TMPA (1.31 and 0.99 mm day−<sup>1</sup> , respectively) and IMERG (1.31 and 1.01 mm day−<sup>1</sup> , respectively). Regarding the underestimation of light precipitation and overestimation of heavy precipitation by IMERG presented in this study, [22] observed a tendency of IMERG daily products underestimate the frequency of rainfall events (<1 mm day−<sup>1</sup> ) and overestimate the frequency of intense rainfall events (>10 mm day−<sup>1</sup> ) across the Tibetan Plateau region. In this way, due to the limited performance of the daily products, the use of this data requires attention by the user and previous calibrations of the products at this temporal scale.

Regarding the comparison of the precipitation estimate on a monthly and daily scale, [18] also observed better performance of the monthly estimates of the precipitation when analyzing the TRMM products in Brazil. According to the authors, the monthly estimates are less affected by systematic errors than daily estimates. The IMERG products are calibrated using monthly data of in-situ gauging stations of the Global Precipitation Climatology Centre (GPCC) network [52], which can also explain the better performance of products on a monthly basis as compared to the daily scale. The better performance of the monthly precipitation product throughout Brazil also was confirmed by [20]. The authors observed an average of CC of 0.93 and RMSE of 23.20 mm when comparing IMERG version 5 in the state of Paraná. The better performance observed by [20], compared to that found in this study, may be due to the methods used for interpolation of observed capture data when assimilating them to the IMERG forecast fields. Typically, interpolation methods do not capture a large spatial variability of rain and the estimation is complex due to the spatial discontinuity [53]. The interpolation leads to the smoothing of high and low peaks of precipitation in a region, improving the relationship between observed and modeled values. With respect to the performance of IMERG for estimating seasonal precipitation, [22] showed different performance of the estimative in dry and wet seasons over sub-regions of Mainland China. The authors showed greater correlation of IMERG's products and ground gauges over the dry season for two sub-regions, as observed in this study, and the opposite for the other six. Thus, the investigation of the performance of satellites products in a regional scale is very relevant.

However, the IMERG overestimated the rainfall over regions of heavy rainfall as well as moderate rainfall. In coastal areas, even the difference of a short distance between the ocean and coastal mountains can induce failure of the satellite sensor to discriminate between the adjacent pixels of land and water, generating signal contamination and resulting in poor performance in estimating precipitation [17]. The limitation of the precipitation estimates by satellites (Global Precipitation Climatology Project–GPCP) in orographic regions was also observed in the Andes by [54]. According to the authors, the spatial resolution (2.5◦ ) and dependence on passive microwaves and infrared precipitation recoveries used by satellites may imply the worst performance of representing precipitation in locations with orographic-type precipitation. Despite presenting higher spatial resolution (0.1◦ ), IMERG products also showed less accuracy for estimating rainfall in orographic regions.

The estimative of the precipitation in the mountainous region tends to be underestimated towards the ocean (east area), where orographic rains occur. On the opposite side, after the sudden elevation change, it tends to overestimate the precipitation. Corroborating these results, [55] also observed trends of underestimation of precipitation by the TRMM (2A25 version 7) product in regions of orographic rainfall and overestimation in regions of valleys or flat areas in the southeastern Appalachians. According to the authors, this behavior occurs due to the spatial resolution and the correction of soil disorder made by the satellite. Another possibility for underestimating precipitation in the coastal region is that precipitation occurs while the top of the cloud is still relatively warm. Satellites are unable to fully identify rain, as heat exceeds infrared thresholds and the lower amount of ice in the air makes detection by passive microwave sensors difficult, and thus satellite products detect only part of the precipitation [2,56,57].

In addition, in mountainous regions, precipitation is extremely variable and there are changes in rainfall distribution over short distances [58], which can result in a representation of precipitation with less accuracy in these areas, since satellite products have the limitation of estimating precipitation considering the pixel size, when compared to gauges that measure the precipitation in-situ.

In the coastal area, the highest peak of precipitation occurs in the summer and is related to the predominant role of the Atlantic Tropical Mass [59], which finds the mountain as a physical barrier, culminating in orographic rains, with great volume over a short duration interval. Such an event may not be fully captured by satellites, as observed by [60] on the west coast of the United States, in the analysis of the precipitation of six satellite products (AFWA, TMPA–3B42, TMPA–3B42RT, CMORPH, PERSIANN, and NRL) and [33] in China, with IMERG version 5 products. Despite notable improvements in version 6 of IMERG's algorithm over previous versions as detailed by [30,58] observed lower accuracy of IMERG version 6 for estimating rainfall on the coastline of the Adriatic Sea in Europe, due to complex orography terrain. Additionally, [61] related the limitation of estimating precipitation in the Ebro River basin in Spain, in an area where the weather was dominated by the advection of wet maritime air masses. Thus, the measurement of the precipitation over coastal locations continues to be a challenge and deserves further research.

#### *4.3. Rainfall Anomalies between 2000 and 2018*

The northeastern area of the state is concentrated in the central part of the Paraná River basin, close to the climate transition line (subtropical and tropical) that separates several active climate systems in an area of greater atmospheric instability [32]. Thus, the precipitation estimate by the satellite may present a greater limitation (overestimation or underestimation) in this area due to atmospheric conditions.

As previously mentioned, high-intensity orographic rainfall occurs along the coast, which is estimated with less accuracy by satellite-based algorithms, resulting in poor performance in detecting anomalies. The southwestern region of Paraná has favorable conditions for the formation of severe storms and hail, which occur very quickly [62–64]. In September and October, the highest amounts of hail formation are observed in days per month [64], which exactly coincides with the period of greatest rainfall in the region (Figure 3). Thus, as on the coast of Paraná, the sensitivity of IMERG in detecting anomalies in this region may have lower performance compared to other regions.

According to [38], positive anomalies in rainfall can occur during the summer under conditions of El Niño Southern Oscillation (ENSO) in the south of Brazil. Over the years considered in this study, two El Niño events classified as "moderate," between 2002–2003 and 2009–2010, and one classified as "strong," between 2015–2016 (Golden Gate Weather Services, 2020) occurred. During these periods, positive anomalies were detected by IMERG, and gauges in all stations shown in Figure 12.

In the other years, IMERG and gauges detected some anomalies in all nine stations analyzed. However, only IMERG detected negative anomalies. [39] reported that IMERG version 5 products tend to underestimate precipitation amounts for rainfall rates 40–75 mm day−<sup>1</sup> , but overestimate precipitation amounts for high rainfall rates (>80 mm day−<sup>1</sup> ). Thus, the anomalies detected in this study by the satellite could occur due to the under or overestimation of rain events. Thus, the use of IMERG products in anomaly studies should consider their variable performance for this purpose, requiring calibrations and prior data assessments.

#### **5. Conclusions**

In this study, the performance of IMERG version 6 products in estimating daily and monthly precipitation were evaluated in comparison with the data observed by 511 gauges distributed in the state of Paraná in Brazil. The results showed better metrics for monthly precipitation. In summary, the main findings of this study were as follows:


Based on the results presented here, IMERG version 6 can be used as a source of monthly precipitation data over the territory of Paraná. However, on a daily scale, prior calibration of the product is recommended to ensure the positive performance of the estimate on this time scale, especially for mountainous areas. Future improvements to IMERG version 6 products may increase its accuracy and favor its application for the detection of rain in coastal areas and anomalies. Also, studies that consider seasonal analyses and other time scales (hourly and half-hourly), areas with complex topographies, and the other products of IMERG version 6 (Early and Late Run) are recommended.

**Author Contributions:** Conceptualization, all the authors; methodology, J.G.N., D.A. and H.C.B.; software and validation D.A.; writing—original draft preparation, J.G.N. D.A. and H.C.B.; writing review and editing, all the authors. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Informed Consent Statement:** Not applicable.

**Acknowledgments:** This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES–In English: Coordination of Improvement of Higher Education Personnel)—Finance code 001. Additional funding from the Daugherty Water for Food Global Institute at the University of Nebraska is also appreciated.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

