Evaluation and Hydrological Application of CMADS against TRMM 3B42V7, PERSIANN-CDR, NCEP-CFSR, and Gauge-Based Datasets in Xiang River Basin of China

Abstract

Satellite-based and reanalysis precipitation products provide a practical way to overcome the shortage of gauge precipitation data because of their high spatial and temporal resolution. This study compared two reanalysis precipitation datasets (the China Meteorological Assimilation Driving Datasets for the Soil and Water Assessment Tool (SWAT) model (CMADS), the National Centers for Environment Prediction Climate Forecast System Reanalysis (NCEP-CFSR)) and two satellite-based datasets (the Tropical Rainfall Measuring Mission 3B42 Version 7 (3B42V7) and the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks–Climate Data Record (PERSIANN-CDR)) with observed precipitation in the Xiang River basin in China at two spatial (grids and the whole basin) and two temporal (daily and monthly) scales. These datasets were then used as inputs to a SWAT model to evaluate their usefulness in hydrological prediction. Bayesian model averaging was used to discriminate dataset performance. The results show that: (1) for daily timesteps, correlations between reanalysis datasets and gauge observations are >0.55, better than satellite-based datasets; The bias values of satellite-based datasets are <10% at most evaluated grid locations and for the whole baseline. PERSIANN-CDR cannot detect the spatial distribution of rainfall events; the probability of detection (POD) of PERSIANN-CDR at most evaluated grids is <0.50; (2) CMADS and 3B42V7 are better than PERSIANN-CDR and NCEP-CFSR in most situations in terms of correlation with gauge observations; satellite-based datasets are better than reanalysis datasets in terms of bias; and (3) CMADS and 3B42V7 simulate streamflow well for both daily (The Nash-Sutcliffe coefficient (NS) > 0.70) and monthly (NS > 0.80) timesteps; NCEP-CFSR is worst because it substantially overestimates streamflow; PERSIANN-CDR is not good because of its low NS (0.40) during the validation period.

1. Introduction

Precipitation is one of the primary drivers of the hydrological cycle and, thus of great importance in hydrological simulation [1], which is a major water resources management tool for forecasting floods and droughts. The accuracy of hydrological simulation depends on the spatial and temporal resolution of precipitation data [2]. Precipitation is more difficult than other atmospheric variables, such as temperature and relative humidity, to measure accurately because of its great spatial and temporal variability. Precipitation data are usually observed and collected using rainfall gauges and meteorological radar networks, but these measurement devices are usually geographically sparse and inadequate to fully capture the spatial and temporal variability of precipitation [3,4]. This situation is serious in China because of the country’s complex topography and relatively unevenly distributed economic resources [5]. Satellite-based and reanalysis precipitation datasets have been effective in complementing traditionally obtained precipitation data as remote sensing and computing technologies have developed [6,7,8].

Satellite-based precipitation measurement technology uses visible data, infrared imaging, and passive microwave detection to gather precipitation data [9,10]. However, satellite-based datasets inevitably contain errors due to the measurement technology [11], the sampling method [12], and the retrieval algorithms [13]. Reanalysis datasets are created from a combination of observed data and model forecasts [14]. The accuracy of reanalysis datasets is determined by the observed forcing data, the data assimilation method, and the prediction model(s) used [15]. Many studies have shown that the accuracy of reanalysis datasets is highly related to both the observing system and the assimilated data [16,17]. Hodges et al. [18] showed that newer reanalysis datasets, including the European Centre for Medium-Range Weather Forecasts (ECMWF) Interim Re-Analysis (ERA-Interim, https://www.ecmwf.int/en/forecasts/datasets/archive-datasets/reanalysis-datasets/era-interim), the National Aeronautics and Space Administration Modern Era Retrospective-Analysis for Research and Applications (NASA’s MERRA, https://climatedataguide.ucar.edu/climate-data/nasa-merra), and the NCEP-CFSR (http://globalweather.tamu.edu) perform better than older datasets (such as the 25-year Japanese Reanalysis (JRA-25, http://jra.kishou.go.jp/JRA-25/index_en.html)) in identifying recurrent extratropical cyclones because of the improvements in models, observations, and data assimilation in numerical weather prediction model (NWP) systems. Ebisuzaki and Zhang [19] compared NCEP-CFSR to a set of operational analyses for 2007 and found that NCEP-CFSR captured daily variability in precipitation better than the older reanalyses. The performance of NCEP-CFSR was attributed to major improvements in modeling, observation, and the method of data assimilation. Dee et al. [20] found that observed data have a significant effect on the initialization of an NWP model and thus on the quality of reanalysis data. They also found that successive generations of atmospheric reanalysis data have improved in quality as a result of better models, better input data, and better assimilation methods. Smith et al. [21] showed that even when the model and data assimilation method do not change, observational data density, type, and quality change over time. These observational changes can introduce spurious errors into reanalysis data. Model bias can also act on the data to introduce errors, as can the method of observations. There are many widely used satellite-based and reanalysis datasets, such as PERSIANN-CDR, 3B42V7, and NCEP-CFSR, available on the internet. Detailed information about these datasets, such as resolution, coverage, and data sources, is shown in

Table 1. Overview of precipitation datasets.

Datasets	Spatial Resolution	Temporal Resolution	Available Period	Coverage	Source of Data
Gauge	Point	Daily	1987–2013	Xiang River Basin	China Meteorological Administration
CMADS	${0.25}^{\circ }$	Daily	2008–2016	East Asia	www.cmads.org
3B42V7	${0.25}^{\circ }$	Daily	1998–present	${50}^{\circ }$ S–${50}^{\circ }$ N	Goddard Space Flight Centre
NCEP-CFSR	38 km	Daily	1979–present	Global	National Centers for Environment precipitation
PERSIANN-CDR	${0.25}^{\circ }$	Daily	1983–present	${60}^{\circ }$ S–${60}^{\circ }$ N	University of California, Irvine, CA, USA

. However, because of the errors inherent in satellite-based datasets and the high dependency of reanalysis datasets on the observation system, these datasets may not be suitable for hydrological applications in East Asia [22]. The CMADS were developed by Dr. Xianyong Meng from the China Agricultural University (CAU) and has received worldwide attention [5,23,24,25,26,27,28,29,30]. It using STMAS assimilation techniques as well as big data projection and processing methods to compensate for the fact that few specialized meteorological products were developed for East Asia [22].

In this study, four precipitation products that include two precipitation reanalysis datasets (NCEP-CFSR and CMADS) and two satellite-based precipitation datasets (3B42V7 and PERSIANN-CDR) were analyzed and evaluated in a hydrological application for the Xiang River basin, a humid watershed in central China. These datasets all have high spatial and temporal resolution. 3B42V7 is the latest release of the post-real time product (ftp://disc2.nascom.nasa.gov/ftp/data/s4pa//TRMM_L3/). Precipitation estimates from 3B42V7 have been evaluated in many studies [31,32,33,34]. PERSIANN-CDR is a new retrospective multi satellite-based precipitation dataset for long hydrological and climate studies [35], which is available online (ftp://data.ncdc.noaa.gov/cdr/persiann/files/). The dataset is produced by the PERSIANN algorithm [36] using gridded satellite (GridSat-B1) infrared data. Studies of precipitation estimates from PERSIANN-CDR are relatively few, and most of them compare PERSIANN-CDR predictions with ground-based precipitation observations [37,38,39]. only a small number are related to hydrological applications of PERSIANN-CDR [40]. The daily NCEP-CFSR data, which are in a format that the SWAT model can use, are available online (http://globalweather.tamu.edu). NCEP-CFSR datasets are widely used in many studies [41,42,43]. CMADS, which we use in this study, is a new reanalysis product which can be downloaded from the internet (www.cmads.org). There are very few studies that use CMADS because of its novelty, and they investigate northern arid areas such as the Juntanghu watershed [26,27,44], the Manas River basin [28], and the Qinghai-Tibet Plateau [30]. However, the error characteristics of precipitation products vary with climatic regions, seasons, surface conditions, storm regimes, and altitudes [45] which necessitates the analysis and evaluation of hydrological applications of precipitation dataset products in different regions. To the best of our knowledge, this is the first study of a hydrological application of CMADS in Central China.

This paper is organized as follows. Section 2 describes the materials and methods used in the study. Section 3 presents a detailed evaluation of the results given by the precipitation products, and a further discussion is given in Section 4. Lastly, Section 5 provides a short conclusion based on the results of our study.

2. Materials and Methods

2.1. Study Area

The Xiang River basin was selected as our study area. Xiang River is one of the largest tributaries of Yangtze River, flowing northward towards Dongting Lake, the second largest freshwater lake in China. The Xiang River Basin is located in Hunan Province, between between 24.5–28.25${}^{\circ }$ N and 110.5–114.25${}^{\circ }$ E. The outlet of the Xiang River Basin is Xiangtan station and the area of the basin is 82,375 km${}^2$. The basin is dominated by subtropic monsoon climate, with a meaning annual precipitation of 1400 to 1700 mm and an average annual temperature of 17 ${}^{\circ }$C. Most of the rainfall occurs between April and June. The basin suffers from frequent floods and droughts due to the uneven seasonal distribution of rainfall. The primary terrain of the Xiang River Basin is plain while the elevation of this area ranges from 1 m to more than 2000 m (based on China National Height Datum). The overview of Xiang River basin is shown in Figure 1. For more information about the study area, readers are referred to Zhu et al. [40].

2.2. Meteorological Data

In this section, the data used in the study, including gauge observations, that are required for the SWAT model, as well as the four precipitation dataset products, are briefly described. The spatial distribution of the CMADS, the locations of the precipitation and discharge gauges used in the study are shown in Figure 1. The spatial distribution of other datasets in the studied basin refers to Zhu et al. [40].

2.2.1. Satellite-Based and Reanalysis Precipitation Estimates

PERSIANN is a satellite-based precipitation retrieval algorithm based on infrared brightness temperature imagery generated by geostationary satellites [36]. The PERSIANN-CDR dataset is generated by the PERSIANN algorithm using gridded satellite (GridSat-BI) infrared data. NCEP Stage IV radar data is used to train the Artificial Neural Networks model and create nonlinear regression parameters. The model prediction (precipitation estimates) is then calibrated using the monthly Global Precipitation Climatology Project (GPCP) version 2.2 product that contains precipitation gauge data generated by the GPCP mission in order to increase the reliability of the PERSIANN-CDR data [35].

The Tropical Rainfall Measuring Mission (TRMM) is a joint mission between the U.S. National Aeronautics and Space Administration (NASA) and the Japan Aerospace Exploration Agency (JAXA) to study rainfall for weather and climate research. To increase the accuracy of the precipitation estimates, 3B42V7 integrates microwave and infrared measurements and incorporates the new Global Precipitation Climatology Center monthly precipitation data [40,46]. The TRMM satellite stopped collecting data on 15 April 2015 (https://trmm.gsfc.nasa.gov/). The Global Precipitation Measurement Mission Integrated MultisatellitE Retrievals for Global Precipitation Measurement (GPM IMERG), which has more accurate spatiotemporal resolution (half-hourly and 0.1) is a successor to TRMM [47].

CFSR is a global coupled atmosphere–land–ocean–sea-ice assimilation system developed at NCEP [48]. Its spatial resolution is approximately 38 km. CFSR includes the coupling of atmosphere and ocean during the generation of the 6-h guess field, an interactive sea-ice model, and assimilation of satellite radiance data by grid point statistical interpolation over the entire period (https://rda.ucar.edu/#!pub/cfsr.html). All available conventional and satellite observations were included in CFSR.

CMADS is constructed using multiple technologies and scientific methods, including loop nesting of data, projection of resampling models, and bilinear interpolation (www.cmads.org). Data sources for CMADS include nearly 40,000 regional automatic stations under the oversight of China’s 2421 national automatic and business assessment centers [29]. CMADS precipitation data use Climate Prediction Center morphing technique (CMORPH) global precipitation products and data from the National Meteorological Information Center of China. The spatial resolution of CMADS V1.1 (www.cmads.org), used in this study, is 0.25${}^{\circ }$.

2.2.2. Ground Gauge Observations

The daily meteorological observations used to drive the SWAT model include precipitation, maximum and minimum temperatures, solar radiation, wind speed, and relative humidity from 1987 to 2013 at 8 meteorological stations. The data were obtained from China Meteorological Administration. The daily discharge record from 1970 to 2013 was available from Xiangtan station. The locations of these meteorological stations and runoff gauges are shown in Figure 1. An overview of all datasets is given in Table 1.

2.3. Straightforward Comparison

A straightforward comparison was made on two scales: a grid and the whole basin. At the grid scale, only the precipitation estimates for those grids where gauges are located were evaluated. Grid squares were created with the same native resolution for all datasets to identify the grid–gauge pair. Pairwise statistical analyses were conducted between satellite-based/reanalysis precipitation estimates for the grid square and the observations from the gauge located in the grid square [49]. For the whole basin, areal precipitation from precipitation estimates and gauge observations were calculated and compared. The comparisons for both spatial scales were made for daily and monthly timesteps.

Diagnostic Statistics

Seven statistical indexes are used to quantify the accuracy of precipitation predictions: the correlation coefficient (CC), the root mean squared error (RMSE), the mean error (ME), relative bias (BIAS), the probability of detection (POD), the false alarm ratio (FAR), and the critical success index (CSI). The values of the indices are calculated by the following Equations [40,50]:

$$\mathrm{CC}=\frac{{\sum }_{i=1}^n(G_i-\overline{G})(S_i-\overline{S})}{\sqrt{{\sum }_{i=1}^n{(G_i-\overline{G})}^2}\sqrt{{\sum }_{i=1}^n{(S_i-\overline{S})}^2}}$$

(1)

$$\mathrm{RMSE}=\sqrt{\frac{1}{n}\sum _{i=1}^n{(S_i-G_i)}^2}$$

(2)

$$\mathrm{ME}=\frac{1}{n}\sum _{i=1}^n(S_i-G_i)$$

(3)

$$\mathrm{BIAS}=\frac{{\sum }_{i=1}^n(S_i-G_i)}{{\sum }_{i=1}^n{G}_i}\times 100\%$$

(4)

$$\mathrm{POD}=\frac{H}{H+M}$$

(5)

$$\mathrm{FAR}=\frac{F}{H+F}$$

(6)

$$\mathrm{CSI}=\frac{H}{H+M+F}$$

(7)

where $G_i$ is the observed precipitation from gauges, $S_i$ is the precipitation estimates from PERSIANN-CDR, 3B42V7, NCEP-CSFR, and CMADS; H is the observed precipitation correctly detected; M is the observed precipitation not detected; F is the precipitation detected but not observed.

CC reflects the degree of linear correlation, ranging from –1 to 1. The result get the best when the value is equal to 1. ME reflects the average difference between precipitation products and gauge observation. The range of ME is $\left[0,+\infty \right)$, with the perfect value of 0. RMSE reflects the average error between precipitation products and gauge observations, imparting bigger weights to larger errors. The range of RMSE is $\left[0,+\infty \right)$, and the perfect value of this index is 0. $BIAS$ measures the relative degree of the systematic error of the precipitation estimation, ranging from 0 to positive infinite. The perfect value of BIAS is 0. POD gives the fraction of rain occurrences that are detected. It ranges from 0 to 1, with the perfect value of 1. FAR measures the fraction of rain detections that are wrongly detected. The value field of this index is $\left[0,1\right]$, and the perfect value is 0. CSI gives the fraction of observed and/or detected rain but is correctly detected. The value field of this index is $\left[0,1\right]$, and the perfect value is 1. The precipitation threshold between wet day and dry day is 1 mm in this study.

2.4. Ensemble Bayesian Model Averaging

We used Bayesian model averaging (BMA) to determine which precipitation product is most accurate in simulating streamflow in comparison with streamflow gauge observations by comparing the weights of simulated streamflows predicted by the precipitation products.

The BMA method is as follows. Assume that $f=f_1,\cdots ,f_K$ is a set of predictions obtained from K different models, and $\Delta$ represents the quantity of interest. In BMA, each ensemble member forecast, $f_k$, $k=1,\cdots ,K$, is associated with a conditional probability density function (pdf), $g_k\left(\Delta |f_k\right)$, which can be interpreted as the conditional pdf of $\Delta$ on $f_k$, given that $f_k$ is the best forecast in the ensemble. The BMA predictive model for dynamic ensemble forecasting can be expressed as a finite mixture model [51]:

$$p\left(\Delta |f_1,\cdots ,f_k\right)=\sum _{k=1}^K{w}_k{g}_k\left(\Delta |f_k\right)$$

(8)

where, $w_k$ denotes the posterior probability of forecast k being the best one. The $w_k$s are nonnegative and add up to 1. They can be viewed as weights reflecting an individual model’s relative contribution to predictive skill over the training period [51]. $g_k\left(\Delta |f_k\right)$ of the different ensembles can be approximated by a normal distribution centered at a linear function of the original forecast, $a_k+b_k{f}_k$ and standard deviation $\sigma$.

$$\Delta |f_k\sim N\left(a_k+b_k{f}_k,{\sigma }^2\right)$$

(9)

The values for $a_k$ and $b_k$ are bias-correction terms derived by linear regression of $\Delta$ on $f_k$ for each of the K ensemble members.

The values for $w_k,k=1,\cdots ,K$ and ${\sigma }^2$ in Equations (8) and (9) are estimated by maximum likelihood (ML) from a calibration data set. Assuming the forecast errors in space and time are independent, the log-likelihood function for the BMA predictive model is:

$$l\left(w_1,\cdots ,f_K,\Delta \right)=\sum _{s,t}^{n}log\left(\sum _{k=1}^K{w}_k{g}_k\left({\Delta }_{st}|f_{kst}\right)\right)$$

(10)

where, n denotes the total number of measurements in the training data set, s and t denote the number of each dimension of the training data set, ${\Delta }_{st}$ denotes quantities of interest in the training data set, and $f_{kst}$ denotes predictions from K different models in the training data set. However, there are no analytical solutions conveniently maximizing Equation (10). In this study, the DiffeRential Evolution Adaptive Metropolis (DREAM) adaptive Markov chain Monte Carlo (MCMC) algorithm is used to estimate the parameters in Equation (10).

Specific to this study, the ensemble models are the simulated streamflows forced by gauge observations and precipitation estimates from PERSIANN-CDR, 3B42V7, NCEP-CFSR, and CMADS, while the $\Delta$ is the observed runoff. According to others’ studies, the bias-correction of the value of ensemble models can be ignored ($a_k=0,b_k=1$ in Equation (9)) when used in hydrological studies [52]. After maximizing, the probability $w_k$ can denote the relatively applicability of those datasets.

2.5. Model Creation

SWAT was used to create the hydrological model. Details of SWAT, and the model creation are given in Zhu et al. [40].

2.6. Model Calibration and Validation

Two model calibration strategies have mainly been used in previous studies: (1) the SWAT model is calibrated separately for different precipitation datasets; and (2) the best model parameters are obtained from calibrating the SWAT model using observed (gauge) precipitation data and observed streamflow data, and the model is then used for hydrological simulation with other precipitation datasets, such as (in this study) PERSIANN-CDR, NCEP-CFSR, 3B42V7, and CMADS. The first calibration strategy is used in this study because the SWAT model is a semi-distributed model and some sensitive parameters are empirically determined. Model parameters are surrounded by substantial uncertainties because they are inherently non-unique in inverse modeling, and thus it may be that many different sets of parameters will produce the same output signal. In other words, there are no best parameters for a hydrological model because of the inherent uncertainty, but the first calibration strategy guarantees a relatively good simulation result. However, this strategy can confuse because it may result in model predictions that are not comparable. Luckily, some researchers find that there is no obvious difference to the simulated streamflow obtained with the second strategy [40].

The model parameters were calibrated and validated using daily streamflow observed by gauges and simulated streamflow determined by PERSIANN-CDR, NCEP-CFSR, 3B42V7, and CMADS [53,54]. NS, given in Equation (11), and BIAS are used to evaluate the performance of the simulations.

$$\mathrm{NS}=1-\frac{{\sum }_{i=1}^n{\left(Q_{oi}-Q_{si}\right)}^2}{{\sum }_{i=1}^n{\left(Q_{oi}-\overline{Q_o}\right)}^2}$$

(11)

where, $Q_{oi}$ is observed streamflow; $Q_{si}$ is simulated streamflow; and $\overline{Q_o}$ is the mean of observed streamflow.

In view of the overlapping periods of precipitation estimates and streamflow records for the Xiang River basin, the modeling period chosen was from January 2008 to December 2013. The total period was divided into three parts: a warming-up period (1 year from January 2008 to December 2008), a calibration period (calibration; 3 years from January 2009 to December 2011), and a validation period (validation; 2 years from January 2012 to December 2013).

3. Results

3.1. Comparison of Precipitation Estimates

As described in Section 2, precipitation estimates from CMADS, 3B42V7, NCEP-CFSR, and PERSIANN-CDR were compared with daily and monthly gauge observations for the Xiang River basin. The comparisons between precipitation estimates and gauge observations were performed in two spatial scales: grid and the whole basin. At grid scale, the gauge precipitation was compared with the precipitation estimates of the grid where the gauge is located. At the whole basin scale, areal precipitation calculated from precipitation estimates and gauge observations was compared. It should be noted that the comparison method for grid scale used in this paper may introduce errors and uncertainties because the estimate is the average value within a grid while the gauge observation is the value of a point located in the grid.

3.1.1. Spatial Distribution of Annual Precipitation Estimates

The spatial distributions of annual precipitation derived from CMADS, 3B42V7, NCEP-CFSR, PERSIANN-CDR and gauges are shown in Figure 2. The spatial distribution of observed precipitation is interpolated from gauge observations shown in Figure 1 using Kriging method. The results show that the annual precipitation estimates for these four datasets have similar spatial distribution patterns, consistent with observed annual precipitation. The annual precipitation in this area decreases from south to north, and from east to west. As errors and uncertainties may be introduced by interpolation, we did not compare the precipitation magnitudes between estimates and interpolations for each grid in this section. The comparison of precipitation magnitudes was proceeded between stations and the grids where these stations are located in the following sections.

3.1.2. Comparison at Monthly Scale

The temporal distribution patterns of all the precipitation datasets and gauge observations for the included grid squares and the whole basin across different months are similar (Figure 3). In the rainy season (from April to September), NCEP-CFSR overestimates the monthly average precipitation, while CMADS underestimates it. Precipitation estimates from PERSIANN-CDR and 3B42V7 are consistent with gauge observations.

Values of CC, RMSE, ME, and BIAS between monthly precipitation estimates and observed (gauge) precipitation for grid station locations and the whole basin are summarized in

Table 2. Monthly statistical indexes.

Datasets	CC	RMSE (mm)	ME (mm)	BIAS
CMADS	0.96	49.57	−33.29	−34.24
PERSIANN-CDR	0.79	72.27	−27.75	−27.01
3B42V7	0.93	40.14	−7.27	−5.90
NCEP-CFSR	0.61	159.76	85.86	39.68
		(a) Station Dx
Datasets	CC	RMSE (mm)	ME (mm)	BIAS
CMADS	0.97	23.84	−13.21	−13.69
PERSIANN-CDR	0.67	66.34	−2.74	−2.56
3B42V7	0.92	33.24	3.46	3.05
NCEP-CFSR	0.73	101.79	64.97	37.21
		(b) Station Chzh
Datasets	CC	RMSE (mm)	ME (mm)	BIAS
CMADS	0.97	27.04	−14.85	−15.54
PERSIANN-CDR	0.76	62.41	−7.48	−7.26
3B42V7	0.94	29.41	0.44	0.40
NCEP-CFSR	0.66	98.07	41.59	27.35
		(c) Station Yzh
Datasets	CC	RMSE (mm)	ME (mm)	BIAS
CMADS	0.55	81.06	−17.80	−20.09
PERSIANN-CDR	0.72	62.25	−7.20	−7.26
3B42V7	0.87	38.98	−5.59	−5.55
NCEP-CFSR	0.67	142.98	98.02	47.95
		(d) Station Chn
Datasets	CC	RMSE (mm)	ME (mm)	BIAS
CMADS	0.89	41.28	−28.12	−41.37
PERSIANN-CDR	0.65	63.98	1.16	1.19
3B42V7	0.86	36.86	−2.01	−2.14
NCEP-CFSR	0.62	146.09	87.04	47.53
		(e) Station Hy
Datasets	CC	RMSE (mm)	ME (mm)	BIAS
CMADS	0.84	82.35	−63.46	−73.29
PERSIANN-CDR	0.73	84.28	−51.63	−52.46
3B42V7	0.92	52.79	−37.18	−32.94
NCEP-CFSR	0.77	105.36	49.44	24.78
		(f) Station Ny
Datasets	CC	RMSE (mm)	ME (mm)	BIAS
CMADS	0.93	31.18	−5.11	−5.00
PERSIANN-CDR	0.79	55.38	−9.27	−9.47
3B42V7	0.88	40.13	−0.33	−0.31
NCEP-CFSR	0.74	102.18	57.05	34.74
		(g) Station Shf
Datasets	CC	RMSE (mm)	ME (mm)	BIAS
CMADS	0.83	55.42	−24.80	−25.83
PERSIANN-CDR	0.76	60.80	−18.90	−18.55
3B42V7	0.90	37.88	−5.08	−4.39
NCEP-CFSR	0.84	72.31	36.28	23.10
		(h) Station Zhzh
Datasets	CC	RMSE (mm)	ME (mm)	BIAS
CMADS	0.98	31.86	−25.41	−28.67
PERSIANN-CDR	0.93	30.07	−9.14	−8.71
3B42V7	0.98	16.78	−0.37	−0.33
NCEP-CFSR	0.81	93.90	61.17	34.91
		(i) Areal average

. The performance of the datasets in terms of correlation with monthly observed precipitation differs from the daily estimates. There is no obvious correlation between the reanalysis datasets and the satellite-based datasets. CMADS and 3B42V7 perform better than PERSIANN-CDR and NCEP-CFSR in most locations. A comparison of Table 2 with

Table 3. Daily statistical indexes.

Datasets	CC	RMSE (mm)	ME (mm)	BIAS (%)	POD	FAR	CSI
CMADS	0.59	10.13	−1.09	−34.24	0.72	0.26	0.58
PERSIANN-CDR	0.36	12.78	−0.91	−27.01	0.47	0.48	0.33
3B42V7	0.51	11.77	−0.24	−5.90	0.62	0.49	0.39
NCEP-CFSR	0.55	12.76	2.82	39.68	0.47	0.07	0.45
(a) Station Dx
Datasets	CC	RMSE (mm)	ME (mm)	BIAS (%)	POD	FAR	CSI
CMADS	0.60	7.83	−0.43	−13.69	0.74	0.25	0.59
PERSIANN-CDR	0.32	11.15	−0.09	−2.56	0.51	0.45	0.36
3B42V7	0.50	10.25	0.11	3.05	0.65	0.51	0.39
NCEP-CFSR	0.58	9.98	2.13	37.21	0.52	0.09	0.50
(b) Station Chzh
Datasets	CC	RMSE (mm)	ME (mm)	BIAS (%)	POD	FAR	CSI
CMADS	0.62	7.94	−0.49	−15.54	0.69	0.27	0.55
PERSIANN-CDR	0.39	10.97	−0.25	−7.26	0.49	0.45	0.35
3B42V7	0.50	10.33	0.01	0.40	0.62	0.52	0.37
NCEP-CFSR	0.55	9.56	1.37	27.35	0.52	0.10	0.49
(c) Station Yzh
Datasets	CC	RMSE (mm)	ME (mm)	BIAS (%)	POD	FAR	CSI
CMADS	0.55	8.13	−1.20	−53.50	0.69	0.34	0.51
PERSIANN-CDR	0.34	10.97	−0.24	−7.26	0.49	0.46	0.35
3B42V7	0.44	10.43	−0.18	−5.55	0.66	0.55	0.37
NCEP-CFSR	0.56	11.24	3.22	47.95	0.51	0.09	0.48
(d) Station Chn
Datasets	CC	RMSE (mm)	ME (mm)	BIAS (%)	POD	FAR	CSI
CMADS	0.55	7.32	−0.92	−41.37	0.69	0.33	0.51
PERSIANN-CDR	0.31	10.54	0.04	1.19	0.46	0.48	0.32
3B42V7	0.42	9.98	−0.07	−2.14	0.65	0.54	0.37
NCEP-CFSR	0.55	10.32	2.86	47.53	0.49	0.11	0.46
(e) Station Hy
Datasets	CC	RMSE (mm)	ME (mm)	BIAS (%)	POD	FAR	CSI
CMADS	0.58	10.50	−2.08	−73.29	0.71	0.36	0.51
PERSIANN-CDR	0.28	13.63	−1.70	−52.46	0.54	0.47	0.36
3B42V7	0.39	13.29	−1.22	−32.94	0.70	0.55	0.38
NCEP-CFSR	0.60	11.33	1.62	24.78	0.58	0.12	0.54
(f) Station Ny
Datasets	CC	RMSE (mm)	ME (mm)	BIAS (%)	POD	FAR	CSI
CMADS	0.59	9.12	−0.17	−5.00	0.67	0.30	0.53
PERSIANN-CDR	0.31	11.82	−0.30	−9.47	0.48	0.44	0.35
3B42V7	0.43	11.29	−0.01	−0.31	0.65	0.50	0.39
NCEP-CFSR	0.62	9.86	1.87	34.74	0.51	0.11	0.48
(g) Station Shf
Datasets	CC	RMSE (mm)	ME (mm)	BIAS (%)	POD	FAR	CSI
CMADS	0.65	9.33	−0.81	−25.83	0.69	0.26	0.56
PERSIANN-CDR	0.33	12.73	−0.62	−18.55	0.46	0.42	0.34
3B42V7	0.48	12.32	−0.17	−4.39	0.63	0.51	0.38
NCEP-CFSR	0.66	9.95	1.19	23.10	0.56	0.09	0.53
(h) Station Zhzh
Datasets	CC	RMSE (mm)	ME (mm)	BIAS (%)	POD	FAR	CSI
CMADS	0.70	5.77	−0.83	−28.67	0.77	0.21	0.64
PERSIANN-CDR	0.49	7.85	−0.30	−8.71	0.62	0.36	0.46
3B42V7	0.60	7.35	−0.01	−0.33	0.71	0.31	0.54
NCEP-CFSR	0.78	6.08	2.01	34.91	0.62	0.04	0.60
(textbfi) Areal average

shows that precipitation estimates have a greater linear correlation with gauge observations on the monthly scale than the daily scale. This conclusion is consistent with the research of Omranian and Sharif [47]. The monthly BIAS values between precipitation estimates and gauge observation are similar to the daily values. Satellite-based estimates perform better than reanalysis estimates. CMADS tends to underestimate precipitation while NCEP-CFSR tends to overestimate precipitation.

The spatial distributions of monthly precipitation estimates’ RMSE are shown in Figure 4. The results show that the spatial distribution patterns of RMSE for the evaluated satellite-based and reanalysis datasets are similar. The gauges located in mountainous regions (Dx and Ny) have much larger RMSE than those located in plain regions, which illustrates that the precipitation estimates derived from both satellite-based datasets and reanalysis datasets perform better in plain regions than in complex orographic areas.

3.1.3. Comparison at Daily Scale

Values of the seven diagnostic indexes (CC, RMSE, ME, BIAS, CSI, FAR, and POD) between daily precipitation estimates provided by the reanalysis and satellite datasets and observed (gauged) precipitation for the station locations and for the whole basin are given in Table 3. The results show that reanalysis datasets (CMADS and NCEP-CFSR) are better than satellite-based datasets (PERSIANN-CDR and 3B42V7 ) in terms of correlation with gauge observations for all evaluated grid squares and for the whole basin. The correlations between precipitation estimates from reanalysis datasets, CMADS and NCEP-CFSR, and observed precipitation are >0.55 for all evaluated grid squares and for the whole basin. The performance of CMADS is similar to NCEP-CFSR in terms of correlation with observed precipitation. The satellite-based dataset precipitations, PERSIANN-CDR and 3B42V7, have relatively low linear correlation with observed precipitation for all grid squares considered. The same results do not hold for the whole basin. The correlation between the 3B42V7 precipitation and observed precipitation over the whole basin is 0.60. It can be deduced from the results that the correlation between dataset precipitation estimates and gauge observations become stronger as the spatial resolution decreases. This conclusion is consistent with the research of Omranian and Sharif [47]. However, the satellite-based datasets (PERSIANN-CDR and 3B42V7) give better estimates than the reanalysis datasets (CMADS and NCEP-CFSR) at most grid locations except for BIAS at station Dx and Ny. All dataset precipitation estimates show much larger BIAS values for this station than for other stations. This result is because station Dx and Ny are located in a mountainous region while other stations are on plains Figure 1. For the whole basin, 3B42V7 shows much lower BIAS than PERSIANN-CDR. CMADS clearly tends to underestimate precipitation. NCEP-CFSR tends to greatly overestimate precipitation at the selected grid squares and for the whole basin because fewer observations were available to initialize the model. NCEP-CFSR is one of a new generation of numerical weather prediction models, and it is very sensitive to the observed data that is used to initialize it [19]. The values of POD, FAR, and CSI show that PERSIANN-CDR cannot accurately detect rainfall events at the grid square level. CMADS and 3B42V7 have a greater probability of detecting rainfall events at the grid scale and for the whole basin but NCEP-CFSR is more accurate in detecting rainfall events.

The spatial distributions of the $RMSE$ of daily precipitation estimates are similar as that of monthly precipitation estimates (Figure 5). Mountainous areas (Dx and Ny) have larger RMSE than plain areas. In addition, CMADS has the minimum RMSE than other datasets, ranging from 7.32 mm to 10.50 mm.

3.2. Comparison of Streamflow Simulations

Simulation accuracy in mature hydrological models is mainly determined by meteorological inputs, especially precipitation. Both the total volume and the spatial and temporal distribution of precipitation significantly influence the output of a hydrological model. This section assesses the performance of the different precipitation datasets as drivers (or forcers) of a hydrological model and evaluates the capability of the precipitation datasets to capture the spatial and temporal characteristics of precipitation over the basin as shown by the performance of the hydrological model. The SWAT model is used because it well describes the hydrology of this area [40]. The evaluation is made for both daily and monthly timesteps.

3.2.1. Comparison at Monthly Scale

A comparison of simulated monthly streamflow using the precipitation datasets of observed data, CMADS, 3B42V7, NCEP-CFSR, and PERSIANN-CDR with observed streamflow is shown in Figure 6. The evaluation statistics (NS efficiency coefficient and BIAS) of the monthly simulations are shown in

Table 4. NS coefficient and BIAS of monthly simulated streamflow (The number outside and inside the parenthesis is for the calibration period and validation period respectively).

Datasets	NS in Calibration	NS in Validation	BIAS in Calibration (%)	BIAS in Validation (%)
Gauge	0.95	0.94	−2.06	−1.36
CMADS	0.92	0.80	−12.06	2.17
3B42V7	0.94	0.88	−7.20	3.69
NCEP-CFSR	−0.12	−0.12	36.49	31.31
PERSIANN-CDR	0.89	0.63	−8.91	1.66

. As in the daily simulations, NCEP-CFSR greatly overestimates streamflow and is the worst of all the precipitation datasets, having the lowest NS values (−0.12 during calibration and −0.12 during validation) and extremely large BIAS values (36.49% during calibration and 31.31% during validation). Of the other precipitation products, 3B42V7 performs better than others in terms of both NS (0.94 during calibration and 0.88 during validation) and BIAS (−7.20 during calibration and 3.69 during validation). The NS and BIAS for CMADS are 0.92 and −12.06% (calibration) and 0.80 and 2.17% (validation), which indicates that the performance of CMADS is acceptable. PERSIANN-CDR performs well during calibration (NS = 0.89) but not during validation (NS = 0.63). However, the BIAS of PERSIANN-CDR is relatively good, intermediate between CMADS and 3B42V7.

3.2.2. Comparison at Daily Scale

The simulated flows predicted by the SWAT model, using observed (gauge) data and data from CMADS, 3B42V7, NCEP-CFSR, and PERSIANN-CDR, are shown in Figure 7. The NS efficiency coefficient and the BIAS values of the simulations are shown in

Table 5. NS coefficient and BIAS of daily simulated streamflow.

Datasets	NS in Calibration	NS in Validation	BIAS in Calibration (%)	BIAS in Validation (%)
Gauge	0.86	0.72	−1.58	3.07
CMADS	0.83	0.70	−12.06	2.20
3B42V7	0.83	0.73	−10.84	−0.17
NCEP-CFSR	−0.46	−0.46	36.58	31.40
PERSIANN-CDR	0.71	0.40	−16.66	−6.44

. The calibration and validation periods for the simulations were 1 January 2009–31 December 2011 (calibration) and 1 January 2012–December 2013 (validation). NS and BIAS for the simulation using observed precipitation were 0.86 and −1.58% (calibration) and 0.72 and 3.07% (validation). The flow hydrograph for the simulation using observed precipitation is highly consistent with that for observed streamflow. The results indicate that the SWAT model well predicts daily hydrological response in the Xiang River basin. The NS efficiency coefficients show that the precipitation products, except NCEP-CFSR, predict the streamflow well during calibration. The NS efficiency coefficients for CMADS, 3B42V7, PERSIANN-CDR, and NCEP-CFSR are 0.83, 0.83, 0.71 and −0.46 respectively. The NS efficiency coefficients for PERSIANN-CDR during validation was 0.40, illustrating that the performance of PERSIANN-CDR is not stable during the entire simulation. The NS efficiency coefficients for simulations using observed data (0.72), data from 3B42V7 (0.73), and from CMADS (0.71) during validation are acceptable. The BIAS values show that for simulated streamflow, compared with observed streamflow, observed (gauge) precipitation data gives the best results (−1.58% during calibration and 3.07% during validation), followed by 3B42V7 (−10.84% during calibration and −0.17% during validation) and CMADS (−12.06% during calibration and 2.20% during validation). However, simulation using NCEP-CFSR data greatly overestimates streamflow, with BIAS values of 36.58% (calibration) and 31.40% (validation). The hydrographs (Figure 7) of the simulations show that most datasets, except NCEP-CFSR, which overestimates streamflow during almost the entire simulation period, produce good baseflow predictions during both calibration and validation. However, when streamflow is high, most datasets (observation, PERSIANN-CDR, CMADS, and 3B42V7) underestimate the streamflow to different extents. To eliminate the inherent contribution of the SWAT model to the underestimation of high streamflow, the simulation using observed precipitation is taken as a baseline to judge the performance of the other precipitation datasets in predicting high streamflow (Figure 8). The result shows that NCEP-CFSR overestimates high streamflow substantially, while PERSIANN-CDR overestimates high streamflow slightly. However, there is no obvious overestimation or underestimation for CMADS and 3B42V7 compared with simulation based on gauge precipitation. This implies that CMADS and 3B42V7 perform as well as gauge in capturing temporal features of high streamflow.

NS and BIAS assess the efficiency and water balance predictions of the model and data. Observed data performs best because it has the highest NS efficiency coefficient and lowest BIAS. The differences between NS efficiency coefficients and BIAS values for 3B42V7 and CMADS are insignificant, so it is difficult to discriminate between them in hydrological terms. Bayesian model averaging (BMA) was thus used to distinguish between 3B42V7 and CMADS precipitation. BMA is commonly used to handle conceptual model uncertainty in the analysis of environmental systems and to derive predictive distributions of model output [52]. Comparison of the BMA model weights can show which precipitation dataset performs better in hydrological simulation. A detailed description of BMA analysis was given in Section 2.4. The BMA model weights for simulated flow forced by CMADS and 3B42V7 are shown in

Table 6. BMA weights of simulated streamflow forced by different precipitation products.

	3B42V7	CMADS
Weights	0.47	0.53

. The results show that CMADS performs better than 3B42V7 when synthetically considering the consistency of simulated and observed streamflow.

3.3. Analysis of Anomalies in Hydrological Simulation

NCEP-CFSR and PERSIANN-CDR are relatively poor in simulating hydrology. The reasons for the poor performances of these two datasets are different. NCEP-CFSR overestimates precipitation during almost the entire simulation period (Table 3). This overestimation is probably the main reason for the extremely low NS values of NCEP-CFSR since calibration with low flows tends to give higher NS values. The cumulative distribution curves of areal precipitation derived from PERSIANN-CDR and that of gauge precipitation for calibration period and validation period are shown in Figure 9. It should be noted that only the precipitation larger than 20 mm is shown in the chart to illustrate the results more clearly. The results show that the relative position of cumulative distribution curves for PERSIANN-CDR and gauge observations is different between calibration period and validation period. This means that the high precipitation characteristics varies between calibration period and validation period. The SWAT model parameters calibrated with precipitation during calibration period are thus not suitable for validation period.

Table 5 shows that relative water volume during validation is greater than during calibration. This is probably caused by the workings of the SWAT model itself. The performance of the SWAT runoff module depends on the calibration. In this study, there was an extreme peak flow during the calibration period. Thus the SWAT parameters were calibrated to fit to high flows. Consequently, the simulated streamflow during validation, using the parameters determined during calibration, tends to be high.

4. Discussion

Hydrological simulation is thoroughly influenced by the inputs to the hydrological models. Clearly, there is some linkage between the precipitation estimates and the hydrological simulation. However, a precipitation dataset that shows good linear correlation with gauge observations does not necessarily produce a good hydrological simulation. For example, NCEP-CFSR was best linearly correlated with gauge observations but produced the worst hydrological simulation because of its substantial overestimation and relatively low probability of detecting rainfall events. A comparison of the CC and BIAS values for CMADS and NCEP-CFSR shows that these two precipitation datasets perform similarly. However, CMADS produces a much better hydrological simulation than NCEP-CFSR. This shows that the POD of precipitation estimates has a significant effect on hydrological simulation. Streamflow responds to rainfall events. If a precipitation dataset does not detect most of the rainfall events, it cannot adequately capture streamflow. The influence of FAR on the hydrological simulation cannot be determined from the results; however, it can be analyzed conceptually. If other indexes of precipitation datasets, such as CC, BIAS, and POD, are kept constant, a lower value of FAR will indicate a better hydrological simulation. In addition, the estimates of a precipitation dataset are not always consistent with its hydrological predictions. For example, the BIAS for precipitation estimates and for hydrological simulations are not consistent. The value of BIAS for the CMADS precipitation estimate was −28.67% at the whole basin scale but −12.06% for the simulated streamflow in both daily and monthly timesteps. Many factors may contribute to a difference between precipitation estimates and their corresponding hydrological outputs, such as: (1) areal precipitation is calculated by the Theissen polygon method, which does not consider the impact of topography whereas the SWAT model considers the elevation of precipitation grids; (2) the transformation of precipitation to streamflow is a very complicated nonlinear process, so error will not be transferred from precipitation to streamflow linearly; and (3) there are simplifications in the SWAT model, such as the assumptions in the universal soil loss equation for estimating sediment loss, the assumptions in calculating flow velocity in a river, and the ignoring of some hydrological processes that are considered to have relatively small impact on total hydrology.

The spatial resolution of the areas of comparison (from grids of different sizes to the whole basin) can substantially affect the results of the evaluation. [47] used the GPM IMERG dataset and found that the spatial resolution of the areas compared had a significant effect on the results. The dataset gives better results when the temporal and spatial resolutions are downscaled. However, the spatial resolution of a precipitation dataset has a significant impact on the hydrological simulation. Many studies have shown that in hydrological modeling with high spatial and temporal resolutions, datasets can better characterize streamflow [55]. Thus precipitation datasets with higher spatial and temporal resolution are needed to provide good hydrological simulations. However, as mentioned above, increased spatial and temporal resolution of these datasets worsens the model performance when compared to observation datasets, which can adversely affect the simulation. High-resolution datasets also increase model processing time. From a practical engineering perspective, a more efficient way to combine the input data preparation and the hydrological modeling, that considers both modeling accuracy and modeling efficiency, needs to be further studied [56]. In this study, CMADS performs better in modeling accuracy and is more usable because of its SWAT compatible data structure. Hence, considering the modeling accuracy and modeling efficiency, CMADS are more applicable in practical streamflow simulation.

5. Conclusions

The performance of two reanalysis precipitation datasets (CMADS and NCEP-CFSR) and two satellite-based precipitation datasets (PERSIANN-CDR and 3B42V7) was evaluated at two spatial scales (a grid square and the whole basin) and two timesteps (daily and monthly), and the ability of these datasets to simulate streamflow is assessed for both temporal scales. The results show that: (1) for daily timesteps, the reanalysis datasets perform better than satellite-based datasets in terms of correlation with gauge observations, while satellite-based datasets perform better than reanalysis datasets in most situations in terms of bias. The correlations between reanalysis datasets and gauge observations at both spatial scales are >0.55. The absolute bias values of the two satellite-based datasets are <10% at most grid squares and also for the whole basin. CMADS underestimates precipitation while NCEP-CFSR overestimates it. PERSIANN-CDR cannot accurately detect the spatial distribution of precipitation events compared with other datasets. The POD of PERSIANN-CDR at most grid squares is <0.50; (2) CMADS and 3B42V7 perform better than PERSIANN-CDR and NCEP-CFSR in most situations in terms of correlation with gauge observations and satellite-based datasets perform better than reanalysis datasets in terms of bias; (3) CMADS and 3B42V7 simulate both daily (NS > 0.70) and monthly (NS > 0.80) streamflow well; CMADS performs a little better than 3B42V7 at a daily timestep according to the weights of BMA model, and vice versa for a monthly timestep; NCEP-CFSR performs worst because of its substantial overestimation; PERSIANN-CDR performs badly because of its poor capability to capture the characteristics of streamflow during validation.

Some other studies have shown that precipitation products tend to underestimate flood peaks by directly comparing modeled streamflow driven by precipitation products to observed streamflow. In this study, we eliminated the effects of the model structure on underestimation by comparing modeled streamflow driven by precipitation data from products with streamflow driven by observed precipitation, and we found that there is no obvious underestimation of flood peaks when using precipitation products such as CMADS and 3B42V7 in the Xiang River basin. On the whole, CMADS has great potential in hydrological application in the studied area because that (1) the accuracy of simulated streamflow forced by CMADS is good in the studied area; (2) the dataset is well organized and can be used as inputs of SWAT model directly; (3) as a reanalysis dataset, CMADS can be used in areas with sparse gauges and improved in spatiotemporal resolution in further versions with relatively small cost. (4) Compared with satellite-based datasets, reanalysis datasets such as CMADS usually have much longer time series.