Evaluation of Three Gridded Precipitation Products to Quantify Water Inputs over Complex Mountainous Terrain of Western China

Abstract

This study evaluates the capacity of three gridded precipitation products (MSWEP V2.2, TRMM-3B42 V7, and GPM-IMERG V6) to detect precipitation in the Min Jiang watershed, a data-scarce and mountainous region in western China. A set of statistical and contingency indices is calculated for the precipitation products and compared with rain gauge observations at 23 ground stations from July 2000 to May 2016. Consistency between gridded and ground precipitation datasets is examined at different temporal (i.e., daily, monthly, seasonally, and annually) and spatial (i.e., site level, sub-regional level, and watershed level) resolutions. We identify possible reasons for discrepancies among precipitation datasets. Our results indicate that: (1) the MSWEP product is best suited for the study of long-term mesoscale rainfall, rather than short-term light or extreme rainfall; (2) the IMERG product represents stable performance for the simulation of rainfall spatial variability and detection capability; and (3) Composition of the datasets, climatic systems, and regional topography are key factors influencing the consistency between gridded and ground precipitation datasets. Therefore, we suggest using MSWEP V2.2 and GPM-IMERG V6 as potential precipitation data sources for hydrometeorological studies over the Min Jiang watershed. The findings of this study inform future hydrometeorological and climate applications in data-scarce regions with complex terrain.

1. Introduction

Precipitation is a primary component of the global water cycle and energy balance. Reliable long-term records of precipitation at high spatiotemporal resolutions can support a wide range of applications, such as hydrological model optimization [1,2], water resources management [3,4], drought and flood forecast and warning [5,6,7,8], agricultural irrigation planning [9,10], hydrometeorology analysis [11,12], and climate studies [13,14,15,16]. Information on precipitation from imagery and measurements provides an understanding of a region’s climate and is needed to facilitate rainfall prediction (e.g., about potential flooding and droughts) [17]. Accurate precipitation datasets with the high spatiotemporal resolution and long-term records are difficult to achieve [18], particularly in mountainous areas, mainly due to the inadequate and uneven distribution of ground observation stations [19,20,21]. Complex topography and variable rainfall characteristics add particular challenge in regions where ground-based precipitation monitoring is limited [22].

The development of infrared (IR) and microwave (MW) satellite sensors promotes the contemporary application of gridded precipitation products globally or at the quasi-global scales [9,23,24]. These include satellite, reanalysis, and merged datasets, such as Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) [25], Global Satellite Mapping of Precipitation (GSMaP) [26,27], Climate Hazards Group Infrared Precipitation with Station Data (CHIRPS) [28], Tropical Rainfall Measuring Mission (TRMM) Multi-satellite Precipitation Analysis (TMPA) [29], Global Precipitation Measurement Integrated Multi-satellite Retrievals (GPM-IMERG), and Multi-Source Weighted-Ensemble Precipitation (MSWEP) [30]. As a result of the relative advantages of high spatiotemporal resolution and wide coverage, long-term gridded precipitation products have received extensive interest from a range of fields that benefit from forecasting of extreme rainfall [15,31] and hydrological variability [32], especially with the ability for precipitation remote sensing [33,34] in data-scarce areas.

However, measurement errors inherent in satellite-based datasets [35] and uncertainties originating from mechanical error and algorithm tolerances [36,37] have caused high regional variability, restricting direct applications of gridded datasets. Therefore, evaluating multiple gridded precipitation products’ performance, suitability, and reliability at distinct spatiotemporal scales is essential for further research and practical applications [38,39,40,41]. To date, long-term gridded datasets are rare not only in mountainous areas but also different landscapes, and the use of satellite-related products in meteorological studies has been limited by their systematic errors and relatively short records (e.g., <25 years for TRMM and IMERG) [42,43]. To compensate, reanalysis datasets with more accurate combined precipitation estimates have been widely used by the climatology community [44]. Reanalysis precipitation datasets also have their own uncertainties in different areas [45]. This leaves open an opportunity to evaluate and distinguish between these datasets to provide more refined inputs for hydrological, meteorological, and ecological studies, especially in data-scarce regions with complex terrain and convective systems [2,46,47].

In recent years, numerous studies have been devoted to investigating the factors that affect the performance of gridded datasets in data-scarce regions by comparing them with ground observations. Detection accuracy of datasets is subjected to satellite-based data resources, various retrieval algorithms, gauge adjustment procedures [48], and sensors’ ability to capture climatic systems [17,19,49,50,51]. For example, the drought monitoring performance of MSWEP V2.1 over eastern China was significantly superior to its performance in western China [52]. MSWEP V1.1 was identified as superior performance for hydrological applications in the Yellow River Basin, China [53]. Furthermore, its hydrological performance was consistently better than Climate Prediction Center Morphing Technique (CMORPH), TRMM, ERA-Interim (ERAI), and Global Precipitation Climatology Centre (GPCC) at multi-spatial scales in Ethiopia [13,37,54]. However, few studies have been conducted to evaluate the accuracy and applicability of the newly released MSWEP V2.2. TRMM and IMERG products have been directly compared in extensive studies, which quantified their performance and their errors and biases [55,56], such as the GPM sensors could detect light and solid precipitation more accurately than TRMM sensors [55]. Further, several studies compared the performance of different IMERG products (such as Early, Late, and Final Runs), which indicated that the IMERG Final Run captured well the seasonality of precipitation, while the Early and Late Runs showed larger deviations [18,57,58]. These results can be used as a basis for selecting proper gridded precipitation products.

One factor for biases is probably the treatment of temporal heterogeneity when comparing products, i.e., that temporal differences may be smoothed and offset by the quantile-matching approach [59,60]. Extensive studies using these intervals revealed that gridded precipitation data is highly correlated with observations on monthly and annual scales [61,62]. Common intervals of comparison are daily, monthly, and annual [17,63], while the study of intermediate time scales is limited. In addition, topographical factors influence the evaluation of gridded precipitation datasets through gauge numbers and elevation over complex terrain [23,64,65,66]. Evaluation of three long-term (spanning more than half a century) gridded precipitation products over the Tibetan Plateau recommended Asian Precipitation-Highly Resolved Observational Data Integration Towards Evaluation of Water Resources (APHRODITE) for climate change study because of its long-time coverage and high quality [67]. Previous research demonstrated that topography has a profound impact on reliable measurement of precipitation over mountainous regions [67], which could be partly overcome with stochastic weather generators [68]. The impact analysis of the corresponding topographic features is not mentioned [67]. Overall, in-depth verification and comparison of multiple datasets are required to explore further the reasons for the relative performance of different datasets and topographical characteristics [65].

The objective of this study is to systematically quantify the performance of the latest versions of MSWEP, TRMM, and IMERG gridded precipitation products in a geographically diverse watershed by comparing them to ground observations and to analyze the reasons for their strengths and weaknesses. To accomplish this, we compared the products at a basin average scale using data from 2000 through 2016. We use indices including continuous and categorical statistical metrics to appraise the agreement and differences of mean spatial patterns and variability between the precipitation datasets and observations. Results inform the improvement of future versions of precipitation retrieval algorithms for each product. We also discuss factors influencing the consistency between ground and gridded precipitation data. Precipitation data improvements could increase the forecast accuracy of hydrological and meteorological models, in return reducing the disaster risk in vulnerable areas, especially regions that currently have insufficient in situ precipitation data.

2. Materials and Methods

2.1. Study Watershed

The Min Jiang or Min River is located in southwest China, between 102°59′E and 104°31′E longitude and 29°53′N and 33°15′N latitude. Min Jiang drains a watershed of 13.6 × 10⁴ km² and is an important tributary to the Yangtze River [69]. It originates in the slopes of the Min Mountains at the junction of Sichuan and Gansu provinces, then flows south through Songpan, Dujiangyan, and Leshan, before joining the Yangtze River at Yibin (Figure 1). The total length of the Min Jiang is about 730 km, over which it drops 3650 m in elevation [70]. The average annual precipitation across the Min Jiang Watershed ranges from 500 to 850 mm, and the steep terrain of the transition zone between the Sichuan Basin and Tibetan Plateau frequently experiences heavy precipitation [71]. The wettest portion of the watershed is the Wolong Nature Reserve and Dujiangyan area in the lower part of the watershed, where average annual precipitation reaches 1000–1500 mm. The annual average air temperature is 5–15 °C [72], which shows a decreasing trend from southeast to northwest.

The Min Jiang watershed is characterized by the monsoon climate of the Western Sichuan Plateau and the subtropical climate of the Eastern Sichuan Basin [73,74]. The multi-terraced complex terrain and interactions between various circulation systems result in heterogeneous climatic and rainfall systems in this region [75]. Further, due to its unique geographic location and altitude constraints, the vertical spatial difference in its climate behavior is evident [76]. To explore the influence of elevation on precipitation detection, two major sub-regions within the study area were separated, including a high altitude (1500–3500 m) sub-region and a low altitude (300–1500 m) sub-region (Figure 1). The overall climate characteristics present dual zoning characteristics in both the horizontal and vertical directions. As a result, the watershed presents a challenging environment for characterizing precipitation through meteorological observatory networks, and there remains a strong need for understanding the characteristics and mechanisms of precipitation in the area. The Upper Min Jiang is the major water supply source for the Chengdu Plain with a population of 25 million and functions as a key ecological barrier in southwest China [77]. It is, therefore, a suitable area to develop studies regarding the validation and calibration of gridded precipitation products in data-scarce mountainous watersheds [72].

2.2. Ground Observations of Precipitation

Due to the multi-terraced terrain and interactions between different climate circulation systems [75], rain gauge stations are sparse within the Min Jiang Watershed, and the access to ground-based observations of precipitation data is rather limited. In addition, the quality of regional-scale synoptic precipitation data is uncertain. Daily ground observation precipitation datasets from 2000 to 2016 were obtained from the China Meteorological Administration (CMA) Data Center. The quality of the data from each available site was confirmed and reinforced by checking the percentage of days with missing data and migrating adjacent gauge data as required [78]. Stations with no missing data and long-term coverage of daily records (during the period of July 2000 to May 2016) were selected.

A total of 23 rain gauge stations were identified as the reference for the intercomparison of gridded precipitation datasets. Figure 1 shows the spatial distribution of the ground observation rain gauges that were located between 101°30′E and 105°30′E longitude and between 27°30′N and 34°00′N latitude, which contains a number of representative sites within ~1° of the Min Jiang watershed. The stations were located at elevations from 341 to 3492 m (Table S1), which afford a wide range of altitudes to provide vertical distinctions.

2.3. Gridded Precipitation Datasets

We evaluated three gridded precipitation products, including MSWEP (Multi-Source Weighted-Ensemble Precipitation), TRMM (Tropical Rainfall Measuring Mission), and IMERG (Integrated Multi-satellite Retrievals for GPM). The latest version of MSWEP (version 2.2) covers the period from 1979 to 2017 with full global coverage at 0.1° spatial resolution and temporal resolution of 3 h, daily, and monthly. It seeks to optimally merge data from various gauges, satellites, and reanalysis estimates over the entire globe [79]. MSWEP has several specific advantages, such as global coverage and correction for distributional biases to eliminate spurious drizzle and restore attenuated peaks. The latest Version-7 post-real-time 3B42 (hereafter referred to as TRMM-3B42 V7), a product of TMPA, is one of the best TRMM-era multi-satellite precipitation datasets [57]. The TRMM-3B42 V7 is a quasi-global (50°S–50°N) precipitation dataset with 0.25° spatial resolution and temporal resolutions of 3 h, daily, and monthly. The TRMM microwave imager, IR and MW sensors, and a visible and infrared scanner radiometer support its rainfall estimating. This precipitation product has been proven to be one of the best TRMM-era multi-satellite precipitation estimates [80,81]. To extend TRMM, the GPM Core Observatory was launched on 28 February 2014. As a successor of TRMM, the IMERG Final Run (IMERG-F) is a Level 3 GPM product that combines all the passive microwave and infrared estimates from TRMM and GPM eras and is calibrated based on monthly gauge analysis from the Global Precipitation Climatology Centre (GPCC) [35,55,82]. IMERG provides rainfall and snowfall information at 0.1° spatial and half-hour temporal resolutions. The recent version of IMERG V6 uses retrospective processing back through the TRMM era and integrates the advantages of many previous algorithms [83], such as the PERSIANN Cloud Classification System [84], TMPA satellite-gauge combination procedure [43], and CMORPH Kalman filter Lagrangian time interpolation algorithm [35] and covers the period beginning in June 2000 [85,86]. IMERG precipitation estimates from 2000 to 2018 indicated that IMERG is robust in the transition period around 2014 [35].

In terms of the independence of comparison between gauge and three gridded precipitation products, GPCC uses data from 194 International Exchange Stations over China, accounting for <9% of the total rain gauge stations across China to tune the IMERG product [87]. In addition, both TRMM and GPM research-level products are calibrated by monthly gauge data from GPCC, and MSWEP V2.2 is calibrated by daily gauge-based information [52]. These global precipitation products are widely used, providing synoptic estimates of precipitation over our large focal watershed and having a complete data record for our study period of 2000–2016. Before evaluating and comparing the detection capabilities of these three products, we converted relevant data from the Coordinated Universal Time (UTC) to the Local Standard Time (LST) [88]. The gridded precipitation products with their spatial and temporal resolution are described briefly in

Table 1. General characteristics of the precipitation products.

Precipitation Dataset	Temporal Coverage	Spatial Coverage	Spatial Resolution	Temporal Resolution
Ground Stations (CMA)	1951–present	China	Station	Daily
MSWEP V2.2	1979–2017	Global	0.1°	3 h
TRMM-3B42 V7	1998–present	50°N–50°S	0.25°	3 h
GPM-IMERG V6	2000–present	60°N–60°S	0.1°	30 min

.

2.4. Comparisons between Gridded Precipitation Data and Ground Observations

Sparse coverage of meteorological stations introduces additional errors from spatial interpolation while extrapolating data from meteorological stations to grids [51,89]. Therefore, for the comparison between observations and different gridded precipitation products, all three datasets were resampled at the point scale [90] by converting grid cells that contained at least one rain gauge station to station degree with the bilinear interpolation method [91,92,93]. The bilinear interpolation method [93,94] can be used to resample gridded data for a specific location [95], which in this study was each rain gauge. The logic is based on taking weighted averages of four adjacent pixels for the target location [95]. Once we derived these weighted averages, we captured general comparisons among the precipitation datasets by taking watershed averages. Performance of detecting spatial distribution required quantitative analysis at the daily and monthly scales. To tackle this, we evaluated the products on their ability to produce accurate rainfall intensities at the station scale.

Both the MSWEP and TRMM precipitation products are available at 3-h resolution, yet IMERG contains data every 30 min. Moreover, CMA accumulates daily precipitation ending at UTC+0800 [96], which is different from the precipitation accumulation conventions used by the three gridded datasets. Therefore, we adjusted this accumulation cut-off to the same standard daily scale in CST (China Standard Time).

To measure the accuracy of the gridded products, we calculated a number of continuous statistical metrics, including the correlation coefficient (CC), root mean square error (RMSE), mean absolute error (MAE), and relative bias (RB) [97,98]:

$$CC=\frac{{{\displaystyle \sum }}_{i=1}^N\left(P_i-\overline{P}\right)\left(O_i-\overline{O}\right)}{\sqrt{{{\displaystyle \sum }}_{i=1}^N{\left(P_i-\overline{P}\right)}^2{{\displaystyle \sum }}_{i=1}^N{\left(O_i-\overline{O}\right)}^2}}$$

(1)

$$RMSE=\sqrt{\frac{{{\displaystyle \sum }}_{i=1}^N{\left(P_i-O_i\right)}^2}{N}}$$

(2)

$$MAE=\frac{{{\displaystyle \sum }}_{i=1}^N\left|P_i-O_i\right|}{N}$$

(3)

$$RB=\frac{{{\displaystyle \sum }}_{i=1}^N\left(P_i-O_i\right)}{{{\displaystyle \sum }}_{i=1}^N{O}_i}$$

(4)

where $P_i$ denotes the gridded product precipitation,$O_i$ denotes the ground observation precipitation, $\overline{P}$ and $\overline{O}$ are their respective mean values, and $N$ is the number of collocated observations.

The value of CC represents the degree of agreement in the linear relationship between the two variables [99]. RMSE and MAE measure the average error and average difference, respectively. MAE is a linear measure of error, whereas RMSE is a quadratic loss function that emphasizes extremes [50]. The RB metric quantifies the tendency of the gridded products to over- or underestimate the observed values, which can be negative (underestimation) or positive (overestimation). Values of 0 for RMSE, MAE, and RB indicate relatively ideal product performance. RRMSE and RMAE, respectively, indicate the normalized value of RMSE and MAE, defined as the RMSE and MAE divided by the mean value of the gridded precipitation datasets. Using RRMSE and RMAE allows the errors to be reflected in the form of percentages, facilitating comparison.

Further, the modified Kling-Gupta efficiency (KGE) score was used to assess the comprehensive performance of each gridded precipitation dataset [13]. The KGE score combines the Pearson’s correlation coefficient with ratios of bias (β) and variability (γ):

$$KGE=1-\sqrt{{\left(CC-1\right)}^2+{\left(\beta -1\right)}^2+{\left(\gamma -1\right)}^2}$$

(5)

$$with \beta =\frac{\overline{P}}{\overline{O}} and \gamma =\frac{{\left(CV\right)}_p}{{\left(CV\right)}_O}$$

(6)

where CV is the coefficient of variation (i.e., the ratio of standard deviation to the mean value). The optimal values of KGE, β, and γ are 1.

Systematic and random error components of all datasets were assessed through a suitable error decomposition technique [49]:

$$Random.Err.=\frac{{{\displaystyle \sum }}_{i=1}^N\left(P_i-P_i^{*}\right)}{N}$$

(7)

$$System.Err.=\frac{{{\displaystyle \sum }}_{i=1}^N\left(P_i^{*}-O_i\right)}{N}$$

(8)

$$P_i^{*}=a\times O_i+b$$

(9)

where $P_i^{*}$ was calculated using a linear regression function in which $a$ is the slope and $b$ is the intercept [61]. We then used the percentage of systematic and random error components to describe the performance of the products. The scores ranged from 0 to 100 for errors representing the good and poor ends, respectively. A strong performance should yield lower systematic error components than random error components.

Monthly precipitation time series were examined at the watershed scale among different precipitation datasets. Then, all metrics described above were computed for the entire watershed and for each sub-region. Further, we used Taylor diagrams to demonstrate the statistical relationships between ground and gridded precipitation products and analyze factors influencing the consistency. Taylor diagrams have been applied widely to evaluate complex aspects of models and to discern the relative skill of several models [78]. For this purpose, two major sub-regions within the study area were isolated for detailed analysis of topographical influence factors (Figure 1). Taylor diagrams were then created at sub-regional scales.

2.5. Performance of Gridded Data in the Detection of Precipitation Events

Evaluation of precipitation detection ability among the three gridded precipitation products and ground observation data were performed for each station. Several widely used categorical statistical indices were adopted, including, respectively, the probability of detection (POD), false alarm ratio (FAR), BIAS, and critical success index (CSI):

$$POD=\frac{H}{H+M}$$

(10)

$$FAR=\frac{F}{H+F}$$

(11)

$$BIAS=\frac{H+F}{H+M}$$

(12)

$$CSI=\frac{H}{H+F+M}$$

(13)

where H is the number of gridded precipitation data that are consistent with the gauge monitoring data; M is the number of data points in which gridded precipitation data showed no precipitation event (daily precipitation less than 0.5 mm represents no precipitation event) but the gauge station did; and F is the number of datapoints in which the gridded precipitation data showed a precipitation event but the gauge station did not.

These contingency metrics were assessed using various thresholds (0.5, 10, 25, 50, 100 mm/day) of daily ground observed precipitation over the Min Jiang watershed. The rainfall intensity was divided according to the division standards of rainfall intensity in the China Meteorological department and [20]. The POD metric represents a “hit rate” and describes the fraction of the observed events correctly estimated by the precipitation product based on the given threshold values [100]. The FAR metric is the percentage of precipitation events identified by the gridded products when there was no rainfall [101]. The BIAS metric indicates the degree of deviation from real rainfall events. The CSI shows the overall detection ability of the gridded precipitation compared to the observed precipitation [102]. The ideal values for POD, BIAS, and CSI are 1, and the ideal value of FAR is 0.

3. Results

3.1. Evaluation of Gridded Precipitation Products

3.1.1. A General Comparison of the Precipitation Datasets

Figure 2 depicts scatterplots of the watershed-scale precipitation from the three gridded precipitation products versus the ground observations. All three products achieved their highest R² values at the monthly scale, with IMERG and TRMM (R² = 0.99) outperforming MSWEP (R² = 0.98; Figure 2g). All three products performed worst with daily values, but MSWEP showed an especially large spread of points with a radial pattern (R² = 0.14).

The continuous statistical metrics (CC, RRMSE, RMAE, RB, and modified KGE) of the comparisons between gridded precipitation products and ground observation data for different timescales (daily, 3 days, 5 days, 7 days, 10 days, 14 days, monthly, and annual) are plotted in Figure 3.

MSWEP performed better than the other products at the temporal scales from five days to monthly (CC > 0.85, RRMSE < 50%, RMAE < 30%, RB < −0.02, KGE > 0.85). Its performance peaked at the 14-day resolution (KGE = 0.95). However, MSWEP experienced severe declines in performance at time scales shorter than that, with much lower values of CC and KGE, and higher deviation and error values, reinforcing and quantifying the results from Figure 2. For the temporal scales at which the MSWEP product scored the least, IMERG performed better than TRMM. However, TRMM always had the best values of RB, indicating less deviation from the observed precipitation values.

As a supplement to Figure 3, Figure S1 presents the temporal distributions of the systematic and random error components of the comparisons. Due to the higher systematic error component for MSWEP at the daily, 3 days, and annual scales, MSWEP shows worse overall performance in short-term detection (Figure 3). It is the same reason that leads to a turning point in IMERG’s performance on the monthly-to-year scales.

3.1.2. Detailed Comparison at the Monthly Scale

The spatial distribution of the relative error in monthly average precipitation is illustrated in Figure 4. All gridded products produced precipitation rates with negative relative error values and higher variability in lower elevation areas, confirming that topography influences the gradients of rainfall relative error in this region. MSWEP was observed to have the lowest relative error when averaged across all stations (Figure 4d and Table S1), with a mean relative error value of 0.03%. It was followed by IMERG at 1.38%. TRMM showed a higher average relative error (4.00%), indicating a higher positive deviation from the ground observation data.

Figure 5 is a time series of the absolute error derived from the monthly station-average precipitation between the ground observation data and the three gridded products. This figure reveals the obvious impact of seasonal fluctuations in precipitation. During the wet season (May through September), during which rainfall intensity was higher, TRMM notably overestimated precipitation while MSWEP predominantly underestimated it. IMERG fluctuated between overestimation and underestimation of precipitation. All three gridded precipitation products underestimated the precipitation rate at lower rainfall intensities, and MSWEP had the smallest values of absolute error.

Histograms of monthly relative errors in the study area (2000–2016) are presented in Figure S2, which confirm that all the gridded products slightly overestimated precipitation values for wet season months and underestimated the dry season values from October through April. The relative error achieved by IMERG was better than TRMM but not as good as MSWEP. MSWEP had the best performance during all months except April, May, July, September, and October. We can conclude that MSWEP has better correlations as compared to the other products at the monthly scale.

To account for the impact of seasonal fluctuations for each product showing in Figure 5 on the inter-products comparison, we adopt the violin plots to illustrate the probability distributions of monthly average precipitation for the gridded precipitation products (Figure 6). The temporal variation of monthly precipitation (shown by the probability distribution in the violin plots) was similar among site groups, and MSWEP had the best performance among median values. The violin plot distribution shows substantial differences in monthly rainfall amounts and patterns. The TRMM and IMERG products underestimated the precipitation of the dry season, while MSWEP correctly represented the timing of dry season. The TRMM and IMERG products tended to overestimate precipitation during the wet season [103].

Plots of the CC and KGE comprehensive indices of MSWEP, TRMM, and IMERG in different months are shown in Figure 7. All gridded products had good agreement with ground precipitation. However, their patterns were distributed differently between dry and wet seasons. IMERG showed better skill at detecting the precipitation of the wet season (all CC values exceeded 0.9; all KGE values exceeded 0.8), while MSWEP exhibited the best skill in detecting the precipitation of the dry season. TRMM fell in the middle for CC and KGE, reflecting its stable detecting ability. All of the gridded products were more accurate when detecting higher rainfall.

3.1.3. Detailed Comparison at the Daily Scale

The spatial distribution of the mean daily rainfall intensity across the Min Jiang watershed during the study period is shown in Figure 8. Because of the low density of observation stations, Figure 8a was obtained by Kriging Interpolation of the ground observation precipitation data. The notable spatial patterns of precipitation, such as the high precipitation areas situated in the southwest, were captured realistically by all gridded products. The precipitation gradients in the northern parts of the watershed were well-captured by the TRMM and IMERG products, whereas MSWEP indicated a wider belt of low precipitation in the north. The mean precipitation of TRMM was 2.57 mm/day, which agreed well with the ground observations. It also produced the most accurate daily average precipitation values and spatial distribution, which may be due in part to lower native spatial resolutions of TRMM, which are more similar to the resolution of the ground stations [13,104].

As calculated at the watershed average scale, the various daily statistical indices of gridded precipitation are presented in

Table 2. Summary of daily statistical results of the precipitation products in the study area.

Statistical Indices	CMA	MSWEP	TRMM	IMERG
Mean (mm)	2.57	2.52	2.56	2.52
Median (mm)	0.96	0.94	0.89	0.86
Lower Quartile (mm)	0.17	0.18	0.14	0.15
Upper Quartile (mm)	3.26	3.09	3.20	3.06
Range (mm)	[0, 42.61]	[0, 42.51]	[0, 45.00]	[0, 55.21]
Skewness	3.00	3.20	3.23	3.39
Kurtosis	12.44	13.99	14.91	17.19
Standard Deviation (mm)	4.02	4.01	4.17	4.15
Coefficient of Variation (%)	156.08	159.38	162.44	164.30

. Most of the statistical indices were well-captured by MSWEP, which showed similar skewness, kurtosis, SD, and CV. TRMM was worse at representing the precipitation pattern, with a higher standard deviation (4.17) and coefficient of variation (162.44%).

3.2. Precipitation Thresholds Detection Comparison

Evaluation of Contingency. The statistical values showing the performance of the three gridded products at different thresholds are tabulated in

Table 3. The detection capacities of rainfall at different magnitudes. The POD, FAR, BIAS, and CSI contingency statistics obtained using various precipitation thresholds (0.5, 10, 25, 50, 100 mm/day) of daily observed precipitation over the Min Jiang watershed. M, T, and I represent the MSWEP, TRMM, and IMERG products, respectively.

Precipitation Intensity (mm/day)	POD			FAR			BIAS			CSI
	M	T	I	M	T	I	M	T	I	M	T	I
0.5	0.63	0.64	0.65	0.48	0.35	0.35	1.22	0.99	1.00	0.40	0.47	0.48
10	0.18	0.54	0.54	0.80	0.47	0.45	0.93	1.04	0.98	0.10	0.37	0.38
25	0.08	0.46	0.47	0.90	0.56	0.55	0.80	1.05	1.05	0.05	0.29	0.30
50	0.03	0.33	0.39	0.95	0.63	0.60	0.67	0.89	0.98	0.02	0.21	0.24
100	0.02	0.25	0.29	0.96	0.59	0.66	0.49	0.61	0.85	0.01	0.18	0.19

. When the daily precipitation threshold value was 0.5 mm/day, the IMERG product had the highest POD (0.65) and CSI (0.48) values, while the MSWEP had the lowest POD (0.63) and CSI (0.40) values. The same characteristics have been found with different thresholds [90]. At the same time, when the daily precipitation threshold value changed, the IMERG also registered the best FAR and BIAS values, which indicates that the IMERG product has minimal volatility with different thresholds [18,104]. Overall, the performance of all contingency metrics illustrated that the POD and CSI values of all three products decreased as the precipitation threshold increased, and FAR and BIAS were worse [105]. These three gridded products were not able to adequately and accurately capture precipitation events of different magnitudes.

Evaluation of Rainfall Intensity Distribution. For the probability density function (PDF) difference of daily precipitation, both TRMM and IMERG showed similar PDF structures with the ground observation data (within ±1%; Figure 9). Although MSWEP incorrectly estimated the frequency of small precipitation events (<0.5, 0.5–10 mm/day), it provided an approximate match for the frequency of large precipitation events. In general, all products had a similar ability to capture occurrence frequency distributions for a range of rainfall intensities [2,106].

4. Discussion

4.1. Factors Influencing the Consistency between Ground and Gridded Precipitation Data

• Different Precipitation Data Products

The results of our performance evaluations of the MSWEP, TRMM, and IMERG precipitation products showed that they all performed well in the data-scarce, topographically diverse Min Jiang watershed. It also confirmed that the degree of consistency between directly measured ground data and gridded precipitation data is influenced by the type of precipitation product, topographical features, and climate patterns.

As shown in Figure 2 and Figure 3, the CC, RRMSE, RMAE, and KGE metrics confirmed that IMERG was more accurate than the other products for daily and annual scales [101]. TRMM showed the smallest deviations and differences, and was slightly inferior to IMERG. These statistical deviations could be associated with the differences in detection hardware, retrieval algorithms, and correction mechanisms. The TMI (TRMM Microwave Imager) uses nine radiometric channels from 10 GHz to 85.5 GHz, whereas the GMI (GPM Microwave Imager; used by IMERG) added four channels for a total range of 10 GHz to 183 GHz [107]. The enhanced capability of IMERG to detect light rain and solid precipitation can be attributed mainly to these improvements in its sensors [104]. In addition, the TRMM satellite carries a single-frequency precipitation radar, whereas the GPM Core Observatory carries a dual-frequency precipitation radar (DPR), which provides details about particle drop size distributions and improves the performance of precipitation estimates [48,103]. Additionally, compared with the TRMM PR, the GPM DPR distinguishes rain types more clearly and classifies more precipitation events as “stratiform” and “convective” with no precipitation events classified as “missing”. DPR reduces the misclassification of clouds and noise signals as precipitation type “other” (from 10.14%) to 0.51% for all swath data [48].

With 5-day to monthly temporal resolutions, MSWEP outperformed the other two precipitation products. Figure 2 and Figure 3 revealed that MSWEP products are more suitable for the study of long-term mesoscale rainfall than for short-term light or extreme rainfall, which matches performance results from India [13]. Figure S1 revealed that a larger systematic error component may lead to worse performance for all products. The sources of systematic error can generally be attributed to their design objectives [105], which are retrieval algorithms and merging weights design of various data resources. MSWEP can be used as high spatiotemporal input data of hydrological and meteorological applications for medium temporal scales, notably in the monthly scale, even in data-scarce regions [54]. MSWEP V2.2 has made progress by merging a wide range of the gauge, satellite, and reanalysis data [52], and it was calibrated by daily gauge-based information, thus low-density distributions of ground observations will introduce a certain amount of error. Compared with TRMM and MSWEP, IMERG products have higher spatial and temporal resolutions (0.1° and 30 min, respectively), which enhanced its ability to detect extreme rainfall and reduced its downscaling errors [104].

• Seasonality

Due to the seasonal volatility of precipitation showed in Figure 6 and Figure 7, evaluation of seasonal precipitation (wet and dry accumulated precipitation) distributions is presented by the scatter plots showed in Figure 10. The values of R² indicated that the wet season detection had better agreement with the ground observation data than did the dry season detection. The TRMM and IMERG products overestimated precipitation, whereas MSWEP underestimated rainfall during the wet season. Overall, the IMERG dataset had the highest correlation values (0.73 and 0.98, respectively) for both dry and wet seasons.

Table 4. Statistical indices between each gridded precipitation product and ground observation data at the watershed scale divided between the dry and wet seasons.

Statistical Indices	Dry Season			Wet Season
	MSWEP	TRMM	IMERG	MSWEP	TRMM	IMERG
CC	0.81	0.75	0.87	0.96	0.97	0.99
RMSE	22.92	32.56	32.97	21.49	33.13	16.75
MAE	20.69	29.41	30.88	19.17	27.27	14.05
RB	−0.09	−0.14	−0.15	0.00	0.04	0.02
KGE	0.72	0.67	0.59	0.95	0.95	0.93
RE	30.25	16.41	13.09	98.89	35.57	32.16
SE	69.75	83.59	86.91	1.11	64.43	67.84

presents different statistical indices of the tested precipitation products, as compared to ground observation data at the watershed scale for the dry and wet seasons. The error characteristics and continuous indices clearly show that MSWEP was the best gridded precipitation product for both seasons, with the smallest error values and the highest scores on comprehensive indicators. Further, the wet season detection notably outperformed dry season detection across the board, which also shows that gridded precipitation had better detection ability for higher rainfall. Yilmaz and Derin [108] explained the differences in the wet and dry seasons produced by the IR and MW datasets, which are used in the algorithms of TRMM and IMERG products. Their results showed that the MW datasets were responsible for significant precipitation underestimation (underreported by about 75%) in the cold season. Given the much better detection during the wet season, gridded precipitation products are more suitable for regions with greater rainfall.

Gridded products exhibited strengths and weaknesses for detection capabilities at different temporal scales. For example, the daily results showed much lower CC values relative to observations than the monthly evaluation results for all three products. This phenomenon generally occurs because errors that are discernible on a daily scale are canceled out when averaged out across an entire month [97]. With 5-day to monthly temporal resolutions, MSWEP outperformed the other two precipitation products. IMERG was more accurate than the other products for daily and annual scales. The 5-day resolution served as a turning point for all three datasets, which was caused by the higher systematic error component.

• Topography

To explore the influence of different elevation bands on precipitation detection, two major sub-regions within this area were differentiated for spatial comparison: (a) high altitude (1500–3500 m) and (b) low altitude (300–1500 m). The two sub-regions had similar numbers of observation stations. Figure 1 and

Table 5. Daily relative errors from three precipitation products against ground observation data and daily average precipitation at different elevation levels.

Statistical Indices	High/Low Altitude	CMA	MSWEP	TRMM	IMERG
Mean (mm)	H	2.42	2.25	2.58	2.35
	L	2.74	2.81	2.55	2.72
Relative Error (%)	H	-	−7.00	6.51	−2.83
	L	-	2.52	−0.92	−0.93

show the distribution of altitudes and daily mean precipitation values, which reveals that low altitude areas had higher rainfall values in a flat setting. Table 5 displays the values of relative error for the three precipitation products for low- and high-altitude regions. As shown in Table 5, the IMERG product had the lowest negative relative error. The results in Table 5 were quite different from those in Figure 4. Taken together, we conclude that the relative error values of MSWEP across the whole study region were produced by pronounced errors in both the high and low altitude areas that canceled each other out.

Taylor diagrams comparing monthly precipitation from the three products at different elevation levels are presented in Figure 11. IMERG had the best comprehensive metrics. Figure S3 presents box plots of the daily statistical metrics distributions for different products across different elevation ranges. It reveals that gridded datasets in high-altitude areas performed better than in lower areas, which contradicted our assumption.

One of the reasons behind this discrepancy is that high altitude observations are closer to the spaceborne radar equipment, which shows a marked height dependence. It is possible that precipitation data in high-altitude areas were taken at heights where the attenuation effects of radar could be reduced. Ground observation precipitation in high-altitude areas is less influenced by temperature, interception, surface vegetation, etc., so the true ground observation values are better observed by the satellite sensors [109]. According to [55,110], the sparse gauge network in western China, with its mountainous areas, may affect the performance of gridded precipitation products. However, our hypothesis excluded this factor since it was based on comparing two subdivisions with similar numbers of observations.

Secondly, rainfall processes are complex in the watershed of western China. Ground observation stations located on the windward side of the mountains are subject to rainfall patterns dominated by orographic uplift. Moreover, most of the low-altitude observations are located in the transition zone between the Sichuan Basin and the Tibetan Plateau, right where orographic uplift and the latent heat release trigger the initiation and development of convective systems [111,112]. The satellite sensors have worse performance in convective systems [38,113], which does not trouble high altitude areas with relatively simpler weather systems [55,110]. Many studies have also confirmed that precipitation radar generally works better for stratiform rainfall with horizontally homogenous radar echoes than for convective rainfall, which tends to present with vertical cores of intense radar echoes [113,114]. The examination over the upper Yellow River and Gansu–Ningxia water system also demonstrate better POD than other areas in the Yellow River Basin [53]. The greater proximity to the satellites and the interactions between convective systems and regional topography could be the main reasons for superior performance at high altitudes.

Previous studies on the combined effects of topography and rainfall types in high and low elevation ranges have produced opposing conclusions [57,63,67,104]. It may be due to the differences in gauge density, rainfall types, and regional topography compared to Min Jiang watershed. To summarize, the stronger performance of all precipitation estimates at higher altitudes (Table 5 and Figure 11) likely results from a combination of three factors: the characteristics of the retrieval algorithms, cloud microphysics, and specific interactions with regional topography [104].

4.2. Benefits and Limitations of Using Gridded Precipitation Data in this Data Scare Region

• Potential Benefits

In the extensive and complex topography and climate conditions of western China, comprehensive in situ observations of precipitation are greatly limited by high spatial and temporal variation. Many previous studies have confirmed that small regions with complex terrain can produce large variations in precipitation, which cannot be accurately reflected by the sparse monitoring stations [46]. Gridded precipitation products have the potential to eliminate these deviations stemming from in situ stations, assess climatic trends, simulate the process of pollutant migration, evaluate the ecological impacts of climate change, and bolster hydrologic simulations with high spatiotemporal resolutions [115,116,117]. This study noted that MSWEP, TRMM, and IMERG have different strengths and limitations in data-scarce regions, depending on the specific precipitation detection conditions. Taken as a group, that may be viewed as a strength in terms of practical applications to address different spatiotemporal scales. For temporal comparison, MSWEP is the ideal choice for datasets that resolve at 5-day to monthly scales, whereas IMERG is better at the extremes (e.g., daily or annual scales). MSWEP is best suited for the precipitation detection and hydrological modeling of long-term mesoscale rainfall in western China, whereas the IMERG product has stable performance for applications with short-term light and extreme rainfall [104]. Low rain gauge availability, the noise introduced by complex topography, and bias introduced by climatic conditions may affect the adjustment and reanalysis of gridded precipitation products with ground station datasets [118]. Similarly, the infrared and microwave retrieval algorithms also have limitations [64,108,119]. These results should help guide algorithm development and encourage the development of a better understanding of the performance of the physical mechanism for each product. These accurate precipitation datasets with the high spatiotemporal resolution and long-term records can be utilized in the distributed time-variant gain-based framework to quantified the effects of climate and human activities on hydrological alterations [77], thereby reducing the disaster risk in vulnerable areas, especially regions that currently have insufficient in situ data.

For detection capacities of rainfall across a range of intensities, IMERG has the best performance among all three precipitation products. It also has the highest spatial and temporal resolutions, which may help to represent the spatial patterns, volumes, and extreme precipitation events in western China.

• Potential Limitations

With the exception of a few well-equipped research locations within the overall watersheds [120], the distribution of precipitation characteristics in complex terrain cannot currently be confirmed with direct measurements beyond sparse gauges, making it challenging to get a true estimate of the performance of these gridded products [46] within the Min Jiang watershed. The performance of all contingency metrics revealed that the POD and CSI values of all three products decreased as the precipitation threshold increased, and FAR and BIAS were worse. This indicates a limitation in the ability of these satellite sensors to detect precipitation intensity, a conclusion also drawn in [56,103]. Meanwhile, these products are unable to reflect local heavy rainfall characteristics accurately because their grids were downscaled by homogenization data. The performance of gridded products during extreme precipitation in data-scarce regions remains to be validated, especially at higher temporal resolutions, such as 30-min resolution.

Furthermore, gridded precipitation estimates are also subject to uncertainties resulting from cloud top reflectance, thermal radiance, retrieval algorithms, and correction mechanisms [18]. For example, both TRMM and GPM products compared here are calibrated by monthly gauge data from GPCC, and MSWEP V2.2 is calibrated by daily gauge-based information as well [18]. Even for different levels of IMERG products, the retrieval algorithms and rain gauge correction methods are various [108,119]. The complete IMERG products (for all Runs) include calibration to gauge data, such as climatologically for Early and Late products and by month for the Final product [108,119]. Additionally, The Early and Late Runs are executed in the Precipitation Processing System (PPS) Real Time (RT) processing system, while the Final run is executed in the PPS processing system. All the input data, algorithms, and PPS introduce differences and errors for all the Runs [108,119].

To extend TRMM, the GPM Core Observatory was launched on 28 February 2014. Therefore, there can be discrepancies of the precipitation data between the TRMM and GPM eras. However, we do not observe an evident breakpoint in precipitation time series around 2014, indicating that the consistency of the IMERG dataset is robust during the transition period, which is in line with previous findings in China [35,85]. Thus, identifying sources of errors should be achieved to further improve satellite sensors and algorithms development [58]. Follow-up studies are required to clarify the reasons that lead to the varied performance of gridded precipitation datasets when compared with ground observations in the study region, which is beyond the scope of this study. The core task of this work is to make the recommendation of potential precipitation datasets for hydrometeorological studies in data-scarce mountainous areas in western China.

This study did not consider the error effects of the interpolation method that we used, although we selected one that is relatively well-equipped to change the spatial scale from gridded products to the station scale [40]. The impact of different interpolation methods on accuracy differences presents an opportunity for future study. When extrapolating our findings to other regions, attention should be paid to the density and spatial distribution of gauge stations, monsoon influences, rainfall systems (convective or topography rainfall), seasonal wind patterns, and the location of in situ stations on the leeward sides of mountains and valleys [108,119]. To support the practical application of high-quality data products, they need to be comprehensively evaluated and carefully selected before integrating one into any specific application.

5. Conclusions

We evaluated three global-scale gridded precipitation products for their ability to estimate precipitation over a range of spatial and temporal scales within the Min Jiang watershed of western China, a data-scarce region having complex topography and large spatiotemporal variability in precipitation. We compared the remotely-sensed precipitation data products against ground-based measurement data during the period of 2000–2016 and explored the primary factors influencing differences in performance. Ground-based observational measurements representing “true” rates of observed precipitation are needed to provide an independent check on the degree of errors in gridded precipitation datasets. Future studies are required to completely identify factors influencing the consistency between ground and gridded precipitation data. The main conclusions are summarized as follows:

• The three precipitation products that were evaluated each exhibited strengths and weaknesses for detection capabilities at different temporal scales. The wet season detection notably outperformed dry season detection across the board, indicating that gridded precipitation had better detection ability for higher rainfall.
• Gridded precipitation from MSWEP outperformed TRMM and IMERG on moderate (5-day through monthly) time scales. The 5-day resolution served as a turning point for all three datasets: at shorter time scales, the quality of MSWEP detection deteriorated severely. The MSWEP product was deemed to be the most suitable for the study of long-term mesoscale rainfall, rather than short-term light or extreme rainfall, in this watershed.
• The IMERG data product was superior to the other two in its performance stability and had the best performance of rainfall detection. Overall, IMERG represents a good choice when stable performance is required for precipitation pattern simulation, uniform distribution of spatial scale indices, and detection capability.
• All products provided more accurate estimates of precipitation in high-altitude elevation bands compared to low-altitude areas.
• The composition of the datasets, climatic systems, and regional topography are the three primary factors affecting consistency between ground and gridded precipitation data.