Evaluation of ERA5 Precipitation Accuracy Based on Various Time Scales over Iran during 2000–2018

Abstract

In regional studies, reanalysis datasets can extend precipitation time series with insufficient observations. In the present study, the ERA5 precipitation dataset was compared to observational datasets from meteorological stations in nine different precipitation zones of Iran (0.125° × 0.125° grid box) for the period 2000–2018, and measurement criteria and skill detection criteria were applied to analyze the datasets. The results of the daily analysis revealed that the correlation between ERA5 and observed precipitation were larger than 0.5 at 90% of stations. Also, The daily standard relative bias indicated that precipitation was overestimated in zone 6. As detection criteria, the frequency bias index (FBI) and proportion correct (PC) showed that the ERA5 data could capture daily precipitation events. Correlation confidence comparisons between the ERA5 and observational time series at daily, monthly, and seasonal scales revealed that the correlation confidence was higher at monthly and seasonal scales. The standard relative bias results at monthly and seasonal scales followed the daily relative bias results, and most of the ERA5 underestimations during the summer belonged to zone 1 in the coastal area of the Caspian Sea with convective precipitation. In addition, some complex mountainous regions were associated with overestimated precipitation, especially in northwest Iran (zone 6) in different time scales.

1. Introduction

A reliable precipitation dataset with a sufficient spatiotemporal distribution is essential for different water management and agricultural sectors. Meanwhile, a lack of sufficient precipitation data with a proper distribution in catchments that are extended on highlands or in arid–semi-arid areas result in a poor climate representation and inadequate surface precipitation measurements [1,2,3]. In general, observed precipitation data from weather stations typically provide the required data for hydro-climate studies [4]. Due to the precipitation role in various studies, gridded precipitation data have been applied as an accurate proxy for observed data in different parts of the world [5,6]. In recent decades, gridded precipitation datasets have been mainly available in two categories: remotely sensed and reanalysis datasets. Precipitation reanalysis datasets are ordinarily developed by incorporating an extensive range of ground-based observations and numerical weather prediction products, with or without using a data assimilation technique [7,8].

The ECMWF (European Center for Medium-Range Weather Forecasts) provides one of the most usable and operational datasets and has extended reanalysis data at different computational levels. The ECMWF reanalysis is defined as ERA-15 (15-year European Centre for Medium-Range Weather Forecasts (ECMWF) Re-Analysis), ERA-40 (40-year European Centre for Medium-Range Weather Forecasts (ECMWF) Re-Analysis), ERA-interim (European Centre for Medium-Range Weather Forecasts (ECMWF) interim reanalysis) [9], and ERA5 (fifth major global reanalysis produced by ECMWF). The ERA5 precipitation dataset, as a new ECMWF product, was developed using the 4D-VAR assimilation system and is available for the period of 1979 to the present [10,11]. The performance of ECMWF precipitation reanalysis datasets has been examined on regional and global scales and compared with observational datasets in different parts of the world, with results suggesting that the available coordination can be improved by updating each generation [12,13,14]. The reanalysis precipitation datasets typically provide suitable conditions for estimating precipitation [15]. Examples of ECMWF precipitation datasets validation in Iran is a study that compared daily ECMWF precipitation data with network rain gauge data during 2001 [16] and a study that assessed monthly ERA-Interim precipitation data [17].

Although both models exhibited insufficient performance and some underestimations for the Caspian Sea coastal areas, these data provided reliable simulations in many regions of Iran. Also, the performance of ERA5 precipitation data with satellite precipitation data over various climatic zones was compared in Turkey with respect to other areas [18]. In addition, the performance of different daily precipitation products—CAPA (Canadian Precipitation Analysis), ERA-Interim, ERA5, JRA-55 (Japanese 55-year Reanalysis), MERRA-2 (Modern-Era Retrospective Analysis for Research and Applications), and NLDAS-2 (North American Land Data Assimilation System Phase 2)—were evaluated to detect extreme events over the ARBC (Assiniboine River basin in Canada). The results indicated that these precipitation datasets displayed better performance in spring and autumn [7]. In addition, ERA5 and ERA-Interim precipitation with two other climatological parameters (2 m air temperature and snowfall) were evaluated in the region of Arctic Sea ice from 2010 to 2016 [19]. The comparison results of ERA5 and ERA-Interim for the snowfall and precipitation data, ERA5 were closer to the buoy-measured snowpack’s snow water equivalent. Investigation of ERA5 precipitation and several climatic parameters over the Canadian Prairies showed that the simulation performed well with a relatively low precipitation bias [20]. Furthermore, other studies have evaluated the application of ERA5 data in various fields [3,21,22] Considering the critical role of daily precipitation reanalysis products in climate change and hydro-climate studies [23,24], this study evaluates the performance of ERA5 precipitation data based on observational dataset for nine precipitation zones in Iran at daily, monthly, and seasonal scales from 2000 to 2018.

2. Study Area

Iran is in Western Asia (25°30′ to 40° N and 44° to 63°30′ E) and has an area of 1,648,195 km². It is limited by the Caspian Sea to the north and the Persian Gulf and Oman Sea to the south. The Alborz Mountain range extends from south of the Caspian Sea and toward the Persian Gulf and Oman Sea, while the Zagros Mountains are in the west. The elevation varies from below sea level to >5600 m above sea level (Figure 1). The diversity in the roughness of the terrain creates various precipitation zones at different latitudes and altitudes. Iran is a semi-arid region with an average annual precipitation of 245 mm/y. Desert areas with <50 mm/y and the coastal Caspian Sea area with >1500 mm/y are characterized as the hotspots of dry and wet regions in Iran, respectively [25,26]. Generally, the precipitation characteristics of Iran central and eastern parts are as follows: low annual, severe, and extreme short-term precipitation.

In contrast, the north side of the Alborz Mountains and western slopes of the Zagros Mountains receive more precipitation than the rest of the country. Non-uniform distribution throughout the year is a characteristic of precipitation in Iran. Almost all the annual precipitation is concentrated in a few months of the year; therefore, many regions are deprived of having precipitation for several months. The arrangement of precipitation lines in different regions is affected by the conditions that produce precipitation. Elevation plays a significant role due to the expansion of the Zagros Mountains from the northwest to the southeast, and in the Caspian Sea coastal parts from the west to the east, especially in the western part of the sea [25].

In general, the precipitation regime in Iran can be classified into three general categories as follows:

(1)

The dynamic low-pressure precipitationsystem. Large-scale transient eddies often move over the Mediterranean Sea and sometimes over the Black Sea, traveling from the southwest to the northeast across the western strip of the study area, thus causing precipitation. Dynamical forcing and the effect of the mountains have a very significant role in the amount of precipitation delivered from these systems as well as the spatial distribution of precipitation. In this regard, the Zagros Mountain range has an effective role in precipitation distribution, causing a significant difference between precipitation at high altitudes and with the windward direction on the left with wind shelter areas on the right.

(2)

High-pressure systems over northern Iran and the Caspian Sea. Due to the great depth of the Caspian Sea, the sea heats up during the warm season (June–September) and causes heat energy storage. The input of cold northern air in September and October results in considerable precipitation in these areas. The role of the Alborz Mountain range in the precipitation distribution is very significant on the northern and southern slopes of the Alborz chain, most of the precipitation occurs on the northern slopes of the Alborz Mountains and in the coastal areas of the Caspian Sea, while the amount of precipitation on the southern slopes and in the Central parts of Iran decrease rapidly.

(3)

Southeast Asian monsoon. In some years, the infiltration of this system in the southeastern part of Iran can cause convective rain (often from June to August).

Separate precipitation verification is essential for different regions because of the precipitation spatial distribution. Consequently, regarding the average annual precipitation, topographic conditions, and the role of the Alborz and Zagros Mountains in the formation of precipitation areas, the study area was divided into nine precipitation zones (Figure 1). Each zone is characterized by a different amount of precipitation and spatial description, as described below:

Zone 1 

covers the south of the Caspian Sea in the north and includes the northeastern slopes of the Alborz Mountains. Precipitation is distributed throughout the year, and autumn is the rainy season.

Zone 2 

covers the southern slopes of the Alborz Mountains to the northeast of the Zagros Mountains. It also includes some plains on the southern side of the Alborz Mountains, such as Zanjan, Qazvin, and Tehran. Orographic precipitation plays a primary role in most months, and convective clouds cause summer precipitation. Generally, the average annual precipitation decreases from the east to the west. The amount of precipitation is much lower than zone 1.

Zone 3 

is a mountainous region in northeastern Iran and includes the Gorgan Plain in the west and the Cape Dagh Mountains in the northeast, with a semi-arid region that extends to the south and southwest. This zone is mostly affected by the Siberian high pressure in winter. Precipitation is concentrated in winter and spring, and the average annual precipitation decreases in southward.

Zone 4 

covers the east side of the Zagros Mountains, including the Arak plain and the area toward the Kerman Plain. The average annual precipitation in this region is between 150 mm and 300 mm.

Zone 5 

includes two vast deserts and extends to Khash in southeastern of Iran. This zone is characterized by a hot and severely dry climate with the lowest precipitation.

Zone 6 

in the northwest is a mountainous region (Azerbaijan) with a winter precipitation regime.

Zone 7 

covers large parts of the Zagros Mountains western slopes. This region is one of the wettest regions in Iran, with the highest precipitation occurring in the Mediterranean system.

Zone 8 

is in the southwestern of the study area includes a mountainous area and flat region. Precipitation mainly occurs in the Red Sea Trough.

Zone 9 

includes the northern parts of the Persian Gulf and Oman Sea, which is characterized by very hot and humid conditions.

3. Materials and Methods

3.1. Observational Data

In this study, daily precipitation time series were collected from the IRIMO (Islamic Republic Meteorological Organization), which operates a synoptic rain gauge network in Iran. The number of rain gauges has increased by 37% since the early 21st century. After screening the precipitation data of all rain gauges, 151 rain gauges with precipitation data from 2000 to 2018 were selected. The locations of these rain gauges are shown in Figure 1. The spatial distribution of the stations represents the variation of precipitation in the different geographical areas. Most stations cover the mountainous regions from the northwest to the southwest and the southern of the Caspian Sea (medium wet and wet regions). There is inadequate coverage in the central and western parts of the semi-arid regions.

3.2. Re-Analysis Dataset

The ERA5 product is the latest generation of the ECMWF dataset that provides hourly precipitation at a resolution of 0.25° × 0.25° from 1979 to the present. For this study, the hourly ERA5 reanalysis dataset at resolution of 125.0° × 0.125° was retrieved for the period of 2000–2018 from https://cds.climate.copernicus.eu/cdsapp#!/search?type=dataset&text=ERA (accessed on 14 September 2021).

3.3. Methods

The daily ERA5 precipitation data were retrieved by a methodology that is downloadable from the below address (https://confluence.ecmwf.int/display/CKB/ERA5%3A+How+to+calculate+daily+total+precipitation, accessed on 14 September 2021) for the period of 2000–2018, and then monthly and seasonal precipitation values were calculated from daily data. The inter-station comparison method was used on daily, monthly, and seasonal scales. This means that the reference precipitation was extracted from any grid cell for each station where the station is located within it (i.e., the nearest grid intersected at the station). This approach is popular for assessing precipitation products [15,27].

3.3.1. Evaluation Criteria

Metrics and detection criteria [28,29] were used in this study. The first group included the Pearson correlation coefficient (r), which was used as one of the metric criteria for the observed and reanalysis dataset; the mean absolute error (MAE) and root mean square error (RMSE), were used to focus on errors; and the relative bias (BIAS) and Kling–Gupta efficiency (KGE), are synthesis indices described by the correlation component and bias. The second group included the frequency bias index (FBI) and proportion correct (PC), which were used to investigate the relationship between the ERA5 precipitation occurrence and the observed precipitation. All these indices are known criteria and have been used elsewhere [23,30].

3.3.2. Metrics Criteria

The Pearson correlation coefficient (r) is related to the degree of linear correlation, which ranges from −1 to 1. When the variance is low, the r-value is high [15,31] (Equation (1)).

$$\mathrm{r}={{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}({\mathrm{O}}_{\mathrm{i}}-\overline{\mathrm{O}})({\mathrm{R}}_{\mathrm{i}}-\overline{\mathrm{R}})/\sqrt{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}{({\mathrm{O}}_{\mathrm{i}}-\overline{\mathrm{O}})}^2{({\mathrm{R}}_{\mathrm{i}}-\overline{\mathrm{R}})}^2}$$

(1)

The MAE (Equation (2)) correlates with the mean error magnitude, while the RMSE (Equation (3)) depends on the standard deviation of reference data. The optimal values of the MAE and RMSE are zero and showed a perfect fit for the reanalysis and observation datasets [15]. Because RMSE value depends on the range of data changes, it is impossible to compare the RMSE values obtained from the two different comparisons. Therefore, to compare the RMSE value at various points, the RMSE was normalized (NRMSE) and compared at different stations.

$$\mathrm{MAE}={\mathrm{n}}^{\left(-1\right)}{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}\left|{\mathrm{R}}_{\mathrm{i}}-{\mathrm{O}}_{\mathrm{i}}\right|$$

(2)

$$\mathrm{RMSE}=\sqrt{\left({\mathrm{n}}^{\left(-1\right)}{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}{\left({\mathrm{R}}_{\mathrm{i}}-{\mathrm{O}}_{\mathrm{i}}\right)}^2\right)}$$

(3)

$$\mathrm{NRMSE}=\sqrt{({\mathrm{n}}^{\left(-1\right)}{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}{\left({\mathrm{R}}_{\mathrm{i}}-{\mathrm{O}}_{\mathrm{i}}\right)}^2)}/{{\displaystyle \sum }}_{\mathrm{i}}^{\mathrm{n}}{\mathrm{O}}_{\mathrm{i}}$$

(4)

The relative bias (Equation (5)) was used to explore difference between the two datasets, with negative values being assigned to the overestimation of the reanalysis dataset. A relative bias of zero indicates the ideal performance of precipitation datasets [15]. This study is noteworthy in that it compares the daily, monthly, and seasonal relative biases.

$$\mathrm{Bias}=\left({{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}\left({\mathrm{O}}_{\mathrm{i}}-{\mathrm{R}}_{\mathrm{i}}\right)/\left({{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{O}}_{\mathrm{i}}\right)\right)\times 100\%$$

(5)

$$\mathrm{Standard}\mathrm{Bias}\mathrm{i}=\frac{\mathrm{Biasi}}{\mathrm{Biasmax}-\mathrm{Biasmin}}$$

(6)

where n is the total number of samples; O_i is the observed precipitation during the month i (th); R_i is the precipitation during the month i (th) from the reanalysis precipitation dataset; and $\overline{\mathrm{O}}$ and $\overline{\mathrm{R}}$ are the average values of O_i and R_i over n months, respectively [7,28].

The KGE was first presented by [32] and is a qualified variant of the Nash parameter, in which the variation of the underestimates of the NSE (Nash-Sutcliffe efficiency) are refined by a convenient function (Equation (7)). The KGE is defined as a combination of the Pearson co-coefficient, bias, and variability. The KGE ranges from −∞ to 1 [23,33], where 1 indicates the best fit of the data and 0 indicates the worst fit, with a value of >0.6 being considered acceptable [6].

$$\mathrm{KGE}=1-\sqrt{({\left(\mathrm{r}-1\right)}^2+{\left(\mathsf{\beta }-1\right)}^2+{\left(\mathsf{\gamma }-1\right)}^2)}$$

(7)

where r is the correlation component; $\mathsf{\beta }=\left({\text{µ}}_{\mathrm{r}}/{\text{µ}}_{\mathrm{o}}\right)$ is a bias component; $\mathsf{\gamma }=\left(\partial \mathrm{r}⁄{\mathsf{\mu }}_{\mathrm{r}}\right)/\left(\partial \mathrm{o}⁄{\mathsf{\mu }}_{\mathrm{r}}\right)$ is a variability component, and µ_r and µ_o are the reanalysis and observed means, respectively; and $\partial \mathrm{r}$ and $\partial \mathrm{o}$ are the standard deviations of the reanalysis and observed data, respectively.

3.3.3. Detection Criteria

The FBI is the occurrence characteristic of reanalysis precipitation frequency and observational precipitation events (Equation (8)). The frequency bias does not show the measurement accuracy magnitude of the reanalysis precipitation, instead it represents the accuracy of the occurrence. The FBI score ranges from zero to ∞, and the best score is 1. The precipitation occurrence of the reanalysis dataset is more or less than the precipitation events in the observational dataset, and the FBI score is above or below 1 [30,34].

$$\mathrm{FBI}=\left(\mathrm{a}+\mathrm{b}\right)/\left(\mathrm{a}+\mathrm{c}\right)$$

(8)

where a is the reanalysis of precipitation (the observational data detect precipitation); b is the occurrence of precipitation that was predicted but did not occur in the real data (“false alarm”), and c is the occurrence of precipitation that was not predicted (“miss”). To determine the precipitation event, the threshold for precipitation is essential.

The PC shows the fraction of all correct predictions, where a PC score of 1 indicates ideal performance [30]. Here, N = (a + b + c + d), where d indicates a prediction of no precipitation.

$$\mathrm{PC}=\left(\mathrm{a}+\mathrm{d}\right)/\mathrm{N}$$

(9)

In this study, the FBI and PC were calculated when the precipitation amount during 24 h was >0.1 mm. Reference [30] proposed this threshold for the validation of daily precipitation estimates in Turkey.

4. Results

4.1. Daily Evaluation

4.1.1. Daily Metrics

The metric criteria of the daily ERA5 reanalysis dataset and the rain gauge precipitation data for each zone are provided in

Table 1. The results of the criteria of the daily ERA5 reanalysis dataset and the rain gauge precipitation data in different precipitation zones over 2000–2018.

Index		Parameter	Zone 1	Zone 2	Zone 3	Zone 4	Zone 5	Zone 6	Zone 7	Zone 8	Zone 9
Statistical Error Metrics	Correlation	Max	0.73	0.71	0.71	0.84	0.75	0.68	0.87	0.73	0.78
		Mean	0.65	0.62	0.61	0.66	0.56	0.58	0.71	0.64	0.69
		Median	0.64	0.63	0.62	0.66	0.56	0.58	0.70	0.65	0.69
		Min	0.58	0.50	0.49	0.47	0.45	0.50	0.59	0.55	0.61
	MAE	Max	4.02	1.47	2.29	1.03	0.94	1.94	1.99	1.07	0.55
		Mean	3.00	1.02	0.79	0.55	0.40	1.11	1.14	0.74	0.38
		Median	3.15	0.95	0.59	0.53	0.38	0.95	1.11	0.68	0.40
		Min	2.02	0.73	0.42	0.29	0.12	0.79	0.59	0.41	0.20
	NRMSE	Max	1.24	0.87	1.05	0.83	0.92	1.33	0.78	0.73	0.52
		Mean	0.93	0.73	0.71	0.51	0.58	0.93	0.59	0.60	0.40
		Median	0.89	0.70	0.64	0.50	0.57	0.86	0.60	0.58	0.40
		Min	0.82	0.60	0.45	0.29	0.37	0.58	0.37	0.52	0.27
	Standard Relative BIAS	Max	0.14	0.16	0.05	0.25	0.08	0.02	0.17	−0.08	0.09
		Mean	−0.02	−0.09	−0.12	0.02	−0.16	−0.29	−0.07	−0.20	0.04
		Median	−0.05	−0.07	−0.09	0.00	−0.16	−0.24	−0.07	−0.19	0.05
		Min	−0.09	−0.36	−0.42	−0.11	−0.75	−0.73	−0.42	−0.35	−0.07
	KGE	Max	0.54	0.65	0.58	0.52	0.57	0.61	0.72	0.67	0.53
		Mean	0.40	0.45	0.47	0.28	0.38	0.48	0.51	0.54	0.21
		Median	0.38	0.46	0.47	0.30	0.40	0.50	0.52	0.54	0.22
		Min	0.27	0.28	0.36	−0.07	0.09	0.37	0.14	0.36	−0.37
Detection indices	FBI	Max	1.06	1.10	1.11	1.11	1.18	1.13	1.09	1.11	1.10
		Mean	1.04	1.07	1.06	1.07	1.10	1.07	1.06	1.07	1.05
		Median	1.04	1.07	1.05	1.06	1.10	1.06	1.06	1.06	1.05
		Min	1.03	1.05	1.04	1.04	1.04	1.04	1.03	1.04	1.03
	PC	Max	0.98	0.98	0.96	0.99	1.00	0.98	0.99	0.97	0.99
		Mean	0.96	0.96	0.94	0.96	0.94	0.95	0.96	0.94	0.96
		Median	0.96	0.96	0.95	0.97	0.94	0.94	0.96	0.95	0.96
		Min	0.95	0.94	0.88	0.90	0.90	0.92	0.91	0.87	0.93

. Box-and-whisker plots are shown in Figure 2. As expected, most of the boxplots differed due to the different statistical criteria used.

r: According to Figure 2a, the mean correlation coefficient between the reanalysis and observational datasets ranged from 0.56 in zone 5 (central Iran in the wind shelter area of the Alborz and Zagros Mountains) to 0.71 in zone 7 (Zagros precipitation zone).

MAE and NRMSE: The highest daily MAE belonged to the stations with the highest daily precipitation, particularly in the coastal area of the Caspian Sea (zone 1). This was because the MAE is a function of the precipitation range at each station, and the stations with the highest precipitation were in zone 1. The lowest scattered MAE was observed in zone 9 (coastline of the Persian Gulf and Oman Sea) due to the low precipitation in this zone (Table 1). The MAE values in zone 5 (central Iran), zone 4 (Zagros wind shelter region), and zone 7 (Zagros windward area) exhibited a near normal distribution between the selected stations. The error values were associated with positive and negative skewness in the other zones (Figure 2b). The highest NRMSE was in zone 1. The statistical distribution of the NRMSE is shown in Figure 2c. The RMSE between the daily observed data and the daily reanalysis data in zone 6 was higher than that in the other zones.

Relative bias: The accuracy of the daily ERA5 estimations was investigated using relative bias. The standard relative bias showed that the highest mean overestimation of daily precipitation was in zone 6 (Table 1). This means that the amount of ERA5 precipitation at some stations in northwestern Iran (zone 6) was associated with high uncertainty. The highest positive mean of the standard relative bias belonged to zone 9, indicating an underestimation between the observed precipitation and ERA5 precipitation. Figure 2d shows that the difference between the observed and reanalysis daily precipitation in zone 6 varied considerably, confirming the NRMSE results for zone 6. The highest negative (overestimated) values belonged to zones 5 and 6, respectively, and the highest positive value was in zone 4.

KGE: The statistical distribution of the KGE in different zones revealed that the lowest mean KGE values were in zones 4 and 9. In addition, Figure 2e shows that most outlier values of this parameter belonged to these zones. Zone 8 exhibited the highest performance among the zones.

4.1.2. Daily Detection Criteria Results

FBI and PC: Table 1 shows that the FBI ranged from 1 to 1.18 and that the ERA5 precipitation data could detect precipitation events at the daily scale. The PC ranged from 0.87 to 1 for all stations, confirming the FBI results. Figure 2f,g show that the lowest variations of the FBI and PC criteria were observed in zone 1 (wet region), whereas the highest variations were observed in zones 5 (a dry region) and 6 (mountainous region).

4.1.3. Spatial Distributions of Daily Metrics

Figure 3 presents the spatial distributions of the daily metrics. Based on the correlation, the highest performance had a denser coverage in the west and southwest of Iran (zone 7 on the west side of the Zagros Mountains). The main catchments of Iran are the Karun and Karkheh, which are in southwestern Iran. The ERA5 data could improve investigations of surface water resources in such regions. The results were adjusted with the outcomes of the accuracy evaluation of the ECMWF previous generation implemented by [16]; however, the ERA5 data contrary to the past ECMWF generation could reflect more variance in the observed precipitation in the Caspian Sea coastal stations (zone 1). As previously mentioned, the precipitation range influenced the RMSE; therefore, this cannot show how much the RMSE indicates the adequacy of the model. Instead, the NRMSE was used to compare the precipitation amount.

The highest NRMSE was observed in zones 1 and 6, which included the stations with the highest precipitation, such as Bandar Anzali, Noshahr, Astara, and Ramsar in the western area near the Caspian Sea (zone 1). This subject was mentioned by [16,25] in relation to the evaluation of satellite precipitation. The MAE behaved according to the climate of Iran and the precipitation distribution across the country showed a similar pattern over various precipitation zone. Since MAE is associated with precipitation intensity [35], whereby low MAE values correspond to low precipitation areas. The lowest daily MAEs were observed in the central and eastern regions of Iran, while the highest values were observed in the Caspian Sea coastal areas (zone 1). In the central Zagros Mountains and typically in zone 6, the MAEs were larger than the eastern and southern regions of Iran which have dry conditions.

The standard relative bias varied widely in overestimating daily ERA5 precipitation. Both overestimations and underestimations of ERA5 precipitation were observed across all precipitation zones. A high standard negative relative bias (overestimation) was observed in most precipitation zones (zone 6 and at some stations in zones 5 and 7). A positive standard relative bias (underestimation) was observed in zone 9. It should be noted that in stations with overestimated relative bias values, the correlation coefficients were not low. Significant values of precipitation uncertainty have been reported for the validation of ERA5 precipitation in some studies. For example, when assessing the accuracy of ERA5 precipitation data in Turkey [18], which has a similar climate to some parts of Iran. In addition, considerable relative biases of ERA5 precipitation estimates have been reported for China [14]. Complex topographic conditions and convective situations can effectively lead to high errors, as reported in the evaluation of ERA5 daily precipitation in the United States [23]. This assumption can also be valid for stations with high errors in Iran. Specific topographic conditions and shelter conditions in the central regions of Iran can cause differences in the outcomes of ERA5 precipitation and observational precipitation data. The best KGEs belonged to zones 7 and 8 in southwestern Iran (winter and spring precipitation regimes). The worst KGEs were observed over low precipitation areas along the coastline of the Oman Sea.

4.1.4. Spatial Distribution of Daily Detection

Based on Figure 3, the spatial distribution of the FBI indicates that the scores in the western highland region and the coastal area of the Caspian Sea were >1. Due to the areas that matched with the high and medium precipitation regions of Iran, the number of precipitation events captured by the ERA5 data could have been affected by topographic and convective conditions. In general, it can be concluded that the ERA5 daily precipitation could well match the frequency of prediction and actual events, enhancing the FBI in the interior and the eastern regions of the study area. The PC represented the proper fraction of all correct predictions in the study area and was >0.93 in most places.

The following section describes the spatial and temporal distributions of observed and ERA5 precipitation data at monthly and seasonal scales. Monthly and seasonal relative biases were also determined.

4.2. Monthly Evaluation

4.2.1. Spatial and Temporal Distributions of Observed and ERA5 Precipitation

The spatial and temporal distributions of observed and ERA5 mean monthly precipitation are shown in Figure 4a,b, shows that the observed data and ERA5 precipitation were in general agreement. The most important feature of precipitation in Iran is the temporal distribution of precipitation which is not uniform throughout the year. The regions with the highest annual precipitation were the Caspian Sea coastal area in zone 1, the Zagros region in zone 7, and at high elevation in Azerbaijan. The central regions (zone 6) and southern coastal strip (zone 9) were areas of relatively low precipitation. As a result, there were overall decreasing trends in precipitation from the west to east and from the north to the south. Based on the monthly precipitation, June, July, and August had the lowest levels of precipitation (Figure 4a). Spatially, the areas with the highest and lowest levels of monthly ERA5 precipitation were in general agreement with the observed precipitation. Most of the precipitation on the slopes of the Zagros Mountains in northwestern and western Iran was the result of Mediterranean synoptic systems combined with orographic precipitation within the cold months (October to May). During the warm months (June–August), precipitation mainly occurs in the north of the Oman Sea by extending the Southeast Asian monsoon to the west. Finally, the southern coast of the Caspian Sea experienced the rainiest months of all areas.

4.2.2. Monthly Metrics

Figure 5 shows the graphical distribution of the correlation, MAE, and NRMSE statistical criteria on a monthly scale. As shown in Figure 5a, the monthly correlation increased compared to the daily correlation, but the outlier and extreme values were observed in some zones. As expected, the highest MAE was observed in zone 1 (Figure 5b), while the highest NRMSE was among the stations in zone 3 (Figure 5c).

4.2.3. Spatial and Temporal Distributions of the Monthly Standard Relative Bias

The monthly relative bias was used to define the monthly ERA5 precipitation estimation compared with the observed precipitation at each station (Figure 6). As expected, the highest relative bias was obtained with respect to (i) the temporal distribution of precipitation from November to April and (ii) the spatial distribution for the two northern and western strips of Iran. The latter nearly included the outer slopes of the mountainous areas, particularly over zone 6, including the elevated areas of Azerbaijan, zone 7 including the west slopes of Zagros, and zone 8 including the mountainous and plain conditions. The large difference between the observed and simulated precipitation data at some stations in the study area could be explained by their geographical distribution and complex internal factors such as the mountainous nature of the region, variation in altitude, the orientation of roughness, and convective processes.

The standard relative bias for the spatial distribution in January (Figure 6) indicates that the highest relative bias in the negative mode (overestimation) was in the northwest (zone 6) and to the west of Zagros (zone 7), thus covering northwestern–southwestern Iran. In February, the standard relative bias followed the same pattern as January. However, in some other zones, overestimation was observed for some stations, which may have been due to the specific environmental conditions of these stations. For example, the Yazd station in zone 5 was associated with overestimation errors because it is located behind the elevations. In March, the standard relative bias distribution pattern was similar to February, with high relative bias values condensed in zone 6 and some in zone 7. In addition, due to the intensification of the large-scale climate system affecting Iran and the influence of precipitation systems from the west, northwest, northeast, and sometimes southeast regions from January to March, the temporal distribution of precipitation in the central and southern regions of the country was considered in these months. The standard relative bias values obtained at some stations (e.g., Bam station in zone 4) indicate a low estimate of reanalysis precipitation in these areas. In April, some local stations in zones 5 (Yazd) and 3 (Gorgan and Shahrud) showed the most extensive overestimation. In May, the standard relative bias rate decreased significantly due to reduced precipitation levels. Owing to the special local topographic conditions of some stations, the error rate was significant, which was also observed by [17]. Overall, in June, July, and August, due to the Azores high established over a large part of the region, precipitation was generally very limited. The bias rate for simulating the reanalysis precipitation was very low. In September, the standard relative bias distribution increased in zone 1 (e.g., at the Bandar Anzali station). October was associated with the highest difference between the ERA5 precipitation and observed precipitation, mainly in the northern and northwestern regions. November and December were also associated with overestimations in zones 6, 7, and 8. The underestimated ERA5 precipitation was observed at some stations in the western coastal parts of the Caspian Sea in zone 3, and on the southern side of the Alborz Mountains in zone 4.

As shown in Figure 6, ERA5 overestimated the overall precipitation zones on a monthly scale. However, for some stations, the amount of precipitation was estimated to be less than the observed data. In general, the accuracy declined in mountainous areas and on the coast of the Caspian Sea. Lower accuracy in simulation using ERA5 can be attributed to several factors, such as local convective precipitation on the windward side of the Alborz Mountains (zone 1), the mountainous region of zones 6 and 7, and the parameterization of precipitation in ERA5. Other studies have also compared the reanalysis dataset of ERA-Interim [17], which confirmed a slightly reduced performance in the coastal area of the Caspian Sea and in the Zagros Mountains.

4.3. Seasonal Evaluation

4.3.1. Spatial and Temporal Distributions of Observed and ERA5 Seasonal Precipitation

Figure 7 shows the seasonal distributions of the observed and ERA5 precipitation data in the study area. The seasonal distribution of ERA5 precipitation also showed a similar pattern to the distribution of monthly precipitation. The distribution of seasonal precipitation showed that the rule of the precipitation regime of Iran is entirely seasonal, and most of the seasonal precipitation is experienced in the northern and western strips of the country, indicating the relationship between the overall distribution of precipitation and the moisture supply sources and synoptic tracks.

Due to the predominance of inclusive precipitation systems in autumn, winter, and early spring in the western strip of the country, this region received significant precipitation in these seasons, whereas the number of precipitation events decreased in the last month of spring (May) and with the arrival of summer. Most of the precipitation in the coastal area of the Caspian Sea was concentrated in autumn between September and November. The spatial distribution of seasonal precipitation showed that the highest precipitation occurred in winter (December–February) and in most of the zones. In spring (March–May), the precipitation in the north, northwest, northeast, and west quarters of the country (referring to zones 1, 2, 6, 7 and 8) was significant, and ERA5 was also able to predict precipitation events. The summer (June–August) precipitation was only captured in zone 1 and some parts of zone 9, while the other zones experienced no precipitation or deficient precipitation. In autumn (September–November), according to the explanations stated in the previous sections, the amount of precipitation increased over a large part of the study area.

4.3.2. Spatial and Temporal Distributions of Seasonal Standard Relative Bias

The spatial distribution of the seasonal standard relative bias is shown in Figure 8. The seasonal changes of the standard relative bias were similar to its monthly changes and encompassed an extensive range of positive and negative values at all stations. Based on the ERA5 results, a considerable number of stations were dispersed throughout the negative and positive phases over a wide range of all zones during the winter. In spring, the standard relative bias varied from −0.62 to 0.38, and the highest negative value was observed in northern and northwestern Iran. In addition, the highest positive values belonged to zones 4 and 7. In summer, due to the precipitation regime and low precipitation events in the observed data and ERA5, the standard relative bias was zero or near zero for approximately 65% of the stations. Therefore, it could be concluded that summer precipitation presented a low range of the standard relative bias over the most areas in comparison with the other seasons. The standard relative bias of autumn precipitation varied from −0.34 to +0.66, and the high positive value mainly had condensed coverage in some parts of north areas. At the same time, this was expected according to the monthly results of the standard relative bias.

5. Discussion

To evaluate the accuracy of ERA5 performance in Iran, the country was classified into nine precipitation zones. Owing to the dispersion of stations and topographic and climatic conditions in each zone, the difference in the densities of the selected stations was inevitable. At the daily timescale, the correlation between ERA5 and observed precipitation was >0.5 at 90% of the stations. The maximum MAE was in zone 1 to the south of the Caspian Sea, which agrees with the results of [16], but the range of the calculated errors decreased in this zone. The highest NRMSE was in zone 6. It is noteworthy that the highest daily error decreased in the ERA5 precipitation. The standard relative bias showed that daily ERA5 precipitation overestimated daily precipitation in some widespread areas. This result indicated that ERA5 slightly agreed with the large-scale precipitation event model in different zones, especially in zone 6. However, ERA5 showed satisfactory performance in simulating extreme precipitation, which is consistent with the findings of [21]. The KGE confirmed the correlation coefficient results and the relative bias criteria indicated the appropriateness of the daily precipitation ratio for the observed precipitation in a large area of the country, particularly in western parts of Iran. To evaluate the daily precipitation event detection in low-precipitation areas, mainly in the central and eastern parts, the FBI and PC results indicated that ERA5 could capture daily precipitation events. The FBI values at most stations were close to the ideal values; however, over the western parts of the Caspian Sea coastal areas, where the main precipitation regime occurs in autumn, the ability to detect precipitation events decreased. Overall, despite the low scores in one or two indices for some stations, based on the daily evaluations, the ERA5 reanalysis dataset at many stations across all zones revealed an appropriate correlation with the observed precipitation.

The monthly and seasonal evaluations of ERA5 showed that the spatiotemporal distributions of monthly and seasonal precipitation were similar to observed precipitation data. In other words, the ERA5 precipitation data were consistent with the observed precipitation data in the seasonal distribution (wet and dry seasons) and followed similar patterns. Although the ERA5 precipitation was often overestimated in most areas, it adequately demonstrated the trend of changing precipitation in different months and seasons.

In the warm season (June–September), precipitation occurred only in the northern part of Iran (zone 1). The difference between the observed precipitation and ERA5 datasets was between +0.3 to −0.7. During the warm season in the Caspian Sea coastal area and zone 1, the error was higher than other zones, especially zones 6 and 7, which are highland regions. The effect of convection in the warm season was probably one of the main sources of error in this region. In addition, many zones were dry, the amount of precipitation was low, and the error rate in these areas was low.

Autumn, winter, and spring (late September to May) coincided with the passage of low-pressure systems as the main forcing of precipitation. Examination of ERA5 errors in these seasons showed that the amount of ERA5 precipitation was not significantly different from the observed precipitation in autumn. This was due to the onset of precipitation in the western half of Iran, from the high elevations of Azerbaijan in the northwest (zone 6) and the windward side of the Zagros Mountains (zone 7) to the southwest. Another reason was the influence of these systems in eastern and central Iran, especially zones 4 and 2. However, due to the specific conditions and the impact of local topographic conditions, precipitation estimations were associated with more errors at some stations.

With the onset of winter and increased activity of incoming precipitation systems from the west, especially to the west pars of the Zagros Mountains, the error rate increased with increasing precipitation, such that the total winter precipitation was associated with an overestimation error. However, at some stations (e.g., Koohrang), the ERA5 precipitation was estimated to be less than the observed amount due to the high altitude and special local conditions. In spring, most of the northwest region of Iran (zone 6) is affected by the atmospheric system in the Black Sea, and most of the precipitation occurs because of the dominance of this system. Therefore, the amount of precipitation error in spring in the northwest region and zone 6 was higher than that in other regions.

Owing to the climatic conditions of Iran and the expansion of low precipitation areas from the east of the Zagros Mountains toward the central, eastern, and southern coasts of Iran, the distribution of the ERA5 precipitation error was reduced compared to observed data due to the absence of precipitation events. Meanwhile, the southern regions of the country along the coastal line of the Persian Gulf and Oman Sea (zone 9) were associated with less error than other regions. In contrast, the ERA5 precipitation estimation errors for stations located in plain and flat areas within the windward field and areas located behind the mountain and desert margins (zones 3, 4, 5, and 8) were lower than those in zones 6 and 7. However, the important point is the significant error rate in the northwestern regions of Iran (zone 6). Therefore, it is possible that the role of convective precipitation during the spring and warm season, which often occurs locally at a small scale, was not ideally simulated in the ERA5 network with a 0.125 resolution. In addition, the error in the ERA5 precipitation simulation could also relate to the angle of synoptic systems at height, the existence of orographic conditions, and the wind shelter location of some areas. Moreover, the mechanism of humidity injection may be among the main causes of error in precipitation simulations in mountainous areas and complex topographies such as the northwest–west regions and northern strip areas of Iran.

According to other studies performed to evaluate previous generations of ECMWF precipitation data, such as those based on Era-Interim [17] and the current generation of ERA5 [20], simulation is not accurate enough in some specific regions due to natural phenomena and various other factors. As in the present study, there were some errors in areas such as the coastal areas of the Caspian Sea and in high elevation areas in western and northwestern of Iran. In addition, the results of the ERA5 evaluation for Iran agreed with those of [18], who found that ERA5 consistently overestimated the observed precipitation in regions of complex topography in Turkey. Turkey has a climatic and topographic condition similar to western Iran, which was also associated with overestimated precipitation at some stations.

Overall, the performance of the ERA5 reanalysis dataset was better than the previous generation of ECMWF Era-Interim data reported by [16,17] at the monthly scale. Furthermore, due to the high correlation between the monthly ERA5 precipitation and observed data, the monthly and seasonal ERA5 precipitation date showed better performance at the daily scale. However, the daily data performed better in estimating daily precipitation in comparison to the previous generation data.

It is noteworthy that ERA5 precipitation needs to be corrected to minimize deviations at daily, monthly, and seasonal scales, which was also the case for previous ECMWF products and NCEP/NCAR reanalysis datasets and satellite products. In addition, given the importance of reanalysis data in water resource studies, the present study results could help fill observed data gaps in various catchments.

6. Conclusions

We evaluated the accuracy of ERA5 precipitation in Iran with the 151 precipitation stations during 19 years at the 0.125° × 0.125° resolution. Although previous research has applied different precipitation products such as satellite precipitation and the prior generation of ECMWF products over Iran, the privilege of our analysis was based on daily, monthly, and seasonal analysis of ERA5 in the different characteristic precipitation zones. To evaluate the accuracy of a newly ECMWF precipitation dataset, named ERA5, in reproducing daily precipitation in the first step of the study, metrics and detection criteria [28,29] were used. The metrics criteria included the Pearson correlation coefficient (r), the mean absolute error (MAE) and root mean square error (RMSE), the relative bias (BIAS), and Kling–Gupta efficiency (KGE). The detection criteria included the frequency bias index (FBI) and proportion correct (PC). The last two measures are useful for detecting the relationship between the ERA5 precipitation occurrence and the observed precipitation. Also, in the next step, the (r), (MAE), (RMSE), and standard relative bias evaluation criteria were applied to monthly and seasonal precipitation. In general, the detection of daily precipitation occurrence by the ERA5 dataset was acceptable, but in the mode of measure of daily precipitation, ERA5 precipitation was overestimated in a few widespread ranges particularly over the districts by complex topographic and convective situations. Also, significant underestimation was observed in some stations especially in the coast of Caspian Sea. Because of the study area’s climatic conditions and the absence of precipitation events, from the east of the Zagros Mountains toward Iran’s central, eastern, and southern coasts, the ERA5 precipitation distribution error was reduced compared to observed data. In addition, spatial distributions of seasonal precipitation were similar to observed precipitation data in wet and dry seasons. In the latter, a considerable correlation was noticed between the monthly and seasonal ERA5 precipitation and observed data. However, in winter, with the activity of incoming precipitation systems from the west, especially to the western parts of the Zagros Mountains, the overestimation error rate was increased by increasing precipitation. Overall, based on our objectives and results, bias corrected ERA5 precipitation dataset reanalysis has a strong potential for use as proxies to weather station data. It can provide alternative precipitation data for future works in climatic and hydrologic sections. This potential is particularly promising in regions where weather station coverage is limited.