Evaluation of ECMWF-SEAS5 Seasonal Temperature and Precipitation Predictions over South America

Abstract

Nowadays, a challenge in Climate Science is the seasonal forecast and knowledge of the model’s performance in different regions. The challenge in South America reflects its huge territory; some models present a good performance, and others do not. Nevertheless, reliable seasonal climate forecasts can benefit numerous decision-making processes related to agriculture, energy generation, and extreme events mitigation. Thus, given the few works assessing the ECMWF-SEAS5 performance in South America, this study investigated the quality of its seasonal temperature and precipitation predictions over the continent. For this purpose, predictions from all members of the hindcasts (1993–2016) and forecasts (2017–2021) ensemble were used, considering the four yearly seasons. The analyses included seasonal mean fields, bias correction, anomaly correlations, statistical indicators, and seasonality index. The best system’s performance occurred in regions strongly influenced by teleconnection effects, such as northern South America and northeastern Brazil, in which ECMWF-SEAS5 even reproduced the extreme precipitation anomalies that happened in recent decades. Moreover, the system indicated a moderate capability of seasonal predictions in medium and low predictability regions. In summary, the results show that ECMWF-SEAS5 climate forecasts are potentially helpful and should be considered to plan various strategic activities better.

1. Introduction

South America (SA) is a continent with a broad latitudinal extension and diversity of biomes, favoring different climates within its territory [1]. In addition, climate conditions directly influence the main socioeconomic activities developed in the region. These activities include agriculture, power generation, fishing, tourism, and the textile industry. Consequently, information derived from seasonal climate predictions is vital for different sectors of South American society.

The chaotic internal dynamics of the atmosphere limit the prediction of detailed evolution of meteorological events from a few days to two weeks [2]. However, the statistical behavior of the weather, expressed by its temporal and spatial averages, can be predicted on time scales of a season or longer [3]. The same physical principles and equations ground the mathematical models used for weather and climate prediction [4]. The primary difference is that climate models need additional information on the climate system components, such as oceans, land, cryosphere, atmospheric chemistry (including aerosols, ozone, and greenhouse gases), and a more detailed representation of the stratosphere [4].

Pioneering studies [5,6,7] laid the theoretical basis for the progress of numerical climate prediction [8]. Climate prediction derives mainly from the predictability of the boundary conditions, such as sea surface temperature (SST), sea ice, soil moisture, and snow cover, and from the significant influence of these variables in determining future atmospheric conditions [9,10,11,12]. For instance, it is possible to predict SST anomalies associated with El Niño (or its counterpart, La Niña) a few months in advance, which allows for predicting their atmospheric impacts [13] and anticipating future problems with climate extremes.

Robust seasonal climate forecasts can benefit decision-makers in numerous socioeconomic activities. For example, seasonal climate predictions are essential for the electricity sector, as they benefit activities related to the generation, transmission, and distribution of energy [14]. Seasonal climate predictions may also benefit agriculture, another relevant South American economic sector. Agricultural activity is fundamental to Brazil’s economy, representing 21.4% of the Gross Domestic Product (GDP), and has significant importance in global markets [15]. Moreover, agriculture is one of the activities most sensitive to climatic effects, given that its productivity depends directly on the temperature and precipitation conditions of the region where it is performed [16].

Seasonal climate predictions provide strategic information for risk management and drought [17,18] and flood [19] mitigation, benefiting farmers and infrastructure sectors [17]. In addition, seasonal climate forecasts are potentially helpful for farmers’ decision-making, improving their ability to mitigate drought and plan crop types, resource use, crop insurance, and agricultural contracts [17]. Concerning SA, Drumond et al. [20] found a higher occurrence of dry events in large portions of the eastern continental area during the last decade. In this changing scenario, extreme events prediction may also benefit other sectors vulnerable to natural hazards, such as wildfires and heat waves [21,22]. However, due to the strong influence of teleconnection effects, the tropical and subtropical regions possess the skill of drought prediction at a seasonal time scale, while in extratropical regions, drought forecast is still subject to considerable uncertainty limiting the assurance of its reliability [17]. In this context, reducing the uncertainty about the land surface properties among current land analyses and a more realistic representation of the land initial states result in a better simulation of the land-atmosphere coupling, which plays a crucial role in improving drought prediction on seasonal scales [23].

Global and regional climate models can reproduce and predict the main characteristics of precipitation, temperature, and circulation over SA, but they still fail to simulate regional climate over terrains with a complex topography [1,24,25,26,27,28,29]. The global models’ coarse spatial resolutions restrict their ability to simulate precipitation and represent the orography and land-atmosphere interactions [30]. In addition, the low density of rain-gauges and measurements on complex terrain limits data acquisition for model validation and may lead to misinterpretation of performance [31].

The European Centre for Medium-Range Weather Forecasts—System 5 (ECMWF-SEAS5) constitutes the state-of-the-art global modeling of seasonal climate forecasts [32]. Recently, Ferreira and Reboita [1] and Gubler et al. [33] applied cluster analysis to seasonal precipitation predictions from ECMWF-SEAS5 hindcasts (the system’s climatological model constituted by predictions of past conditions) in order to validate the system’s performance in representing the different regimes of precipitation over SA. Both works showed remarkable skill in predicting the spatial distribution of precipitation. However, there is still a scarcity of studies that assess the quality of ECMWF-SEAS5 seasonal rainfall predictions for specific domains in SA, particularly the prognostic forecasts made as of 2017.

Every seasonal climate prediction model is affected by bias since the modeled climate differs from the observed one to a greater or lesser extent. This model bias can be estimated from a previous set of past climate simulations, known as hindcasts. Thus, the hindcasts constitute an average state of the modeled climate, and their systematic error can be used to correct the model forecasts. Within this framework, this study aims to validate the ECMWF-SEAS5 seasonal precipitation and temperature predictions over SA, considering the set of hindcasts and forecasts. We apply bias correction to the seasonal forecasts considering the hindcasts’ systematic error and assess the quality of ECMWF-SEAS5 seasonal climate predictions in different subdomains of SA. Hence, we indicate the regions whose forecasts have better confidence. We stress that there are still no studies evaluating the performance of ECMWF-SEAS5 prognostic predictions (forecasts) for SA. Moreover, the few existing works analyzing retrospective predictions (hindcasts) have employed different assessment methods than those used here. In this sense, the shortage of studies evaluating the ECMWF-SEAS5 performance over SA justifies this work’s relevance, given that the system represents the latest generation of the continuous evolution of global seasonal climate modeling, and several sectors of SA may benefit from more accurate seasonal climate predictions.

2. Materials and Methods

2.1. ECMWF-SEAS5 Data

The ECMWF-SEAS5 forecasting system consists of a 51-member ensemble starting every month (on the first day) and integrated for approximately seven months (215 days). In addition, SEAS5 uses retrospective seasonal forecasts from past decades to verify and calibrate the forecasting system compared to historical records. This set of hindcasts (or reforecasts, as they are forecasts run retrospectively) has a 25-member ensemble starting on the first day’s month from 1993 to 2016. For more information, Johnson et al. [32] present a detailed description of the system components and physical parameterizations employed in ECMWF-SEAS5.

Seasonal climate forecasts start from an observed state of all Earth system components and then evolve over a few months. Thus, errors present at the beginning of the forecast persist or grow during the integration of the model, reaching magnitudes comparable to the forecast signals [34,35,36]. In this scenario, the coupled general circulation model components must be consistent with each other at the initial time of the forecasts to avoid the influence of initialization shock, which is associated with the departure of the model climatology from the observed [37]. Therefore, adjusting all the coupled model components with observations during the forecast initiation is a procedure that mitigates the initialization shock, reduces the forecast error at different time scales, and improves the model’s predictability [37].

The errors in climate prediction may be random, and the ensemble technique quantifies their effect. ECMWF-SEAS5 employs an ensemble method called “burst mode”. All members are initialized on the same date of origin in this procedure but with slightly different initial state conditions (different perturbations) to sample the observations’ uncertainties [32]. Regarding the ECMWF-SEAS5 atmospheric module, initial undisturbed atmospheric conditions start at member 0 of the ensemble. The initial conditions for all other members have perturbations applied to some fields to represent the uncertainty of the atmosphere’s initial state. Disturbed fields include upper air layers and limited soil moisture, soil temperature, snow, sea-ice temperature, and surface temperature [32].

Furthermore, other errors in climate predictions are systematic, and by comparing retrospective forecasts (hindcasts) and observations, it is possible to correct these systematic flaws [34,35,36]. These systematic errors, also known as biases, arise from the difference between the atmospheric and oceanic states simulated by the model and those of the observed climate and can vary by season, region, and forecast lead time. The model’s systematic bias can be estimated by creating a set of forecasts from past years compared to historical records. In this way, hindcasts are a fundamental process in the seasonal climate prediction system, as they provide the model’s climatological error, which allows a more accurate interpretation of the real-time forecasts. The hindcasts are created with a version of the forecast system that is as close as possible to that used for real-time forecasts to ensure that they provide a reasonable estimate of the expected bias. Since the magnitude of the bias may be comparable to the year-to-year variation in seasonal average weather, this bias must be considered when interpreting real-time forecasts. [38].

This study employed ECMWF-SEAS5 surface data (seasonal forecast daily data on single levels) of total rainfall (accumulated every 24 h since the forecast began) and 2-m temperature (instantaneous values every six hours), covering the hindcast data from January 1993 to December 2016 and forecast data from January 2017 to December 2021. These data have 1° $\times$ 1° horizontal resolution (available on the page https://cds.climate.copernicus.eu/cdsapp#!/dataset/seasonal-original-single-levels?tab=form, accessed on 5 January 2022).

2.2. NOAA CPC Data

Seasonal precipitation forecasts from ECMWF-SEAS5 were validated against the National Oceanic and Atmospheric Administration—Climate Prediction Center analysis (NOAA CPC) [39]. The NOAA CPC Unified Gauge-Based Analysis of Global Daily Precipitation (CPC-Global) dataset is derived from thousands of rain-gauges across the globe, cooperative observation networks, and meteorological agencies; data quality control is carried out through comparisons with historical records, surface measurements, radar and satellite observations, and predictions from numerical models [39]. CPC daily precipitation has a spatial resolution of 0.5°, with a time series of daily data available since 1979 and updated daily (available at https://ftp.cpc.ncep.noaa.gov/precip/CPC_UNI_PRCP/GAUGE_GLB/RT/, accessed on 23 April 2022). This study used CPC daily precipitation data from January 1993 to March 2022.

ECMWF-SEAS5 2-m temperature predictions were validated against the CPC Global Daily Temperature data, with a resolution of 0.5° (available at https://psl.noaa.gov/data/gridded/data.cpc.globaltemp.html, accessed on 23 April 2022). These data correspond to the daily maximum and minimum 2-m surface temperature registered between 06Z-06Z. For comparison, daily averages of the 2-m temperature from ECMWF-SEAS5 were calculated with the estimated daily maximum and minimum in the same hourly interval. Before these steps, CPC data were interpolated to the spatial resolution of ECMWF-SEAS5 by the bilinear method [40].

2.3. Construction of the Seasonal Means and Domain of Study

ECMWF-SEAS5 produces predictions with a forecast length of seven months (215 days). However, here we focus on the predictions relative to the first trimester from the second month after the forecast start, also known as lead time forecast 1 (assuming the month of initialization is the lead time 0). For instance, if the forecast started on 1 February, March, April, and May (trimester MAM) correspond to the first three months from the lead time 1.

Daily arithmetic means of CPC and all of the ECMWF-SEAS5 members (25 members for the hindcasts and 51 for the forecasts) were calculated to compute monthly and trimonthly means, yielding maps and statistics for all trimesters of the year. However, for brevity, only the seasons of March-April-May (MAM), June-July-August (JJA), September-October-November (SON), and December-January-February (DJF) are shown here. Thus, the seasonal mean results from 600 integrations in the hindcasts (25 members for 24 years) and 255 integrations in the forecasts (51 members for five years).

ECMWF-SEAS5 trimonthly means were also spatially compared to CPC data, considering their average values in six subdomains of South America (Figure 1): Amazon (AMZ; 5–15° S and 52–68° W), Northeast Brazil (NEB; 2.5–13° S and 35–45° W), Southeast Brazil (SEB; 19.5–25.5° S and 40–52.5° W), South Brazil (SB; 25–32.5° S and 47.5–60° W), Argentine Pampas (AP; 32.5–40° S and 52.5–65° W), and Northern South America (NSA; 7.5–2.5° S and 55–70° W). The subdomain selection was based on Reboita et al. [41]. Furthermore, considering the skill score evaluation, the interannual variability of predicted precipitation anomalies in three subdomains was evaluated: the North (NB2, 0–7.5° S, and 50–57.5° W), indicated by yellow subdomain (1), Northeast (NEB2, 5–12.5° S, and 40–47.5° W), illustrated in the yellow subdomain (2), and South Brazil (SB2, 25–32.5° S, and 50–55° W), indicated by yellow subdomain (3) (Figure 1).

2.4. Statistical Analysis

2.4.1. Hindcasts Seasonal Prediction Skill Score

The seasonal anomaly skill score evaluates the precipitation and temperature predictions from the ECMWF-SEAS5 hindcasts. These skill scores consist of mean temporal correlations between predicted and observed trimonthly anomalies [42], considering the climatological mean (1993–2016) from the ECMWF-SEAS5 and CPC datasets. Given that the samples are composed of 24 pairs of values (1993–2016) for each season, a two-tailed Student’s t-test with 95% confidence (α = 0.05) and 22 degrees of freedom provides the critical Pearson value of ≈0.40, indicating that correlations equal to or above this threshold are statistically significant [43]. Thus, only skill scores equal to or above 0.3 are presented.

2.4.2. Forecast Bias Correction

Since hindcasts provide the systematic errors of the model, it is essential to account for the climatological biases of retrospective predictions for the correction and better use of real-time forecasts. In this sense, a simple bias correction was applied to the ECMWF-SEAS5 seasonal forecasts, following the methodology of Reboita et al. [44]. First, we computed each season’s CPC climatological mean (1993–2016). Then, the seasonal difference $\left({\mathrm{Dif}}_{\mathrm{i}}\right)$ between ECMWF-SEAS5 and CPC was calculated for each year of the hindcast period (1993–2016) as:

$${\mathrm{Dif}}_{\mathrm{i}}=\mathrm{ECMWF}\mathrm{SEAS}5{\mathrm{Hindcast}}_{\mathrm{i}}-{\mathrm{CPC}}_{\mathrm{i}}$$

(1)

From ${\mathrm{Dif}}_{\mathrm{i}}$, the climatological average $\left({\mathrm{ADif}}_{\mathrm{Season}}\right)$ was calculated for each season. For the bias correction $\left(\mathrm{BC}\right)$ of ECMWF-SEAS5 forecasts, the difference was subtracted from the seasonal average:

$${\mathrm{BC}}_{\mathrm{i}}=\mathrm{ECMWF}\mathrm{SEAS}5{\mathrm{Forecast}}_{\mathrm{i}}-{\mathrm{ADif}}_{\mathrm{Season}}$$

(2)

2.4.3. Statistical Indicators

Statistical indicators were employed to verify the accuracy of ECMWF-SEAS5 in the different subdomains analyzed. The metrics used were: Pearson’s coefficient of correlation (r), coefficient of determination (R²), relative error (RE), Willmott’s index of agreement (d), and the Kling–Gupta Efficiency (KGE). Pearson’s coefficient of correlation (r) measures the degree of correspondence between variables from two datasets, assuming values from −1 to 1. A correlation close to 1 (−1) indicates a very high positive (negative) correlation [45]. The coefficient of determination (R²) measures the fit of the predicted data with the observed one, assuming values between 0 and 1. The higher the value of R², the better the fit of forecasts concerning the observations [45]. The relative error (RE) represents the proportion between the model’s error and the climate variability over SA.

Willmott’s index of agreement (d) evaluates the accuracy of predictions regarding the observations and may assume values between 0 and 1. Values close to 0 indicate disagreement between predicted and observed data, and values close to 1 indicate better prediction accuracy [46]. The d index is given by Equation (3):

$$\mathrm{d}=1-\frac{{{\displaystyle \sum }}_{\mathrm{t}=1}^{\mathrm{n}}{\left({\hat{\mathrm{y}}}_{\mathrm{t}}-{\mathrm{y}}_{\mathrm{t}}\right)}^2}{{{\displaystyle \sum }}_{\mathrm{t}=1}^{\mathrm{n}}{\left(\left|{\hat{\mathrm{y}}}_{\mathrm{t}}-{\overline{\mathrm{y}}}_{\mathrm{t}}\right|+\left|{\mathrm{y}}_{\mathrm{t}}-{\overline{\mathrm{y}}}_{\mathrm{t}}\right|\right)}^2}$$

(3)

where $\mathrm{n}$ is the number of samples, $\mathrm{t}$ is the period, ${\hat{\mathrm{y}}}_{\mathrm{t}}$ is the value predicted by ECMWF-SEAS5, ${\mathrm{y}}_{\mathrm{t}}$ is the value obtained by CPC, and ${\overline{\mathrm{y}}}_{\mathrm{t}}$ is the average of values obtained by CPC.

Kling–Gupta Efficiency (KGE) [47,48] combines Pearson’s correlation, bias ratio, and variability ratio, giving information about the model’s performance in estimating the observed variable. KGE is defined according to Equation (4):

$$\mathrm{KGE}=1-\sqrt{{\left(\mathrm{r}-1\right)}^2+{\left(\frac{{\mathsf{\sigma }}_{\mathrm{sim}}}{{\mathsf{\sigma }}_{\mathrm{obs}}}-1\right)}^2+{\left(\frac{{\mathsf{\mu }}_{\mathrm{sim}}}{{\mathsf{\mu }}_{\mathrm{obs}}}-1\right)}^2}$$

(4)

where $\mathrm{r}$ is the linear correlation between observed and simulated values, ${\mathsf{\sigma }}_{\mathrm{sim}}$ is the simulations’ standard deviation, ${\mathsf{\sigma }}_{\mathrm{obs}}$ is the observations’ standard deviation, ${\mathsf{\mu }}_{\mathrm{sim}}$ is the mean of simulations, and ${\mathsf{\mu }}_{\mathrm{obs}}$ is the mean of observations.

2.5. The Seasonality Index (SI)

Walsh and Lawler [49] proposed the Seasonality Index (SI), identifying rainfall regimes based on monthly rainfall distribution. Higher SI values indicate an asymmetry in the distribution of precipitation throughout the year, while values close to zero indicate that there is little or no seasonal variation in rainfall.

The SI is the sum of the absolute values of the differences between the amount of rain each month and the annual average of total precipitation, divided by the annual precipitation. To obtain an average SI, we calculated the ${\mathrm{SI}}_{\mathrm{i}}$ for each year in the hindcasts period (1993–2016) by Equation (5):

$${\mathrm{SI}}_{\mathrm{i}}=\frac{1}{{\mathrm{R}}_{\mathrm{i}}}{{\displaystyle \sum }}_{\mathrm{n}=1}^{12}\left|{\mathrm{X}}_{\mathrm{in}}-\frac{{\mathrm{R}}_{\mathrm{i}}}{12}\right|$$

(5)

where ${\mathrm{R}}_{\mathrm{i}}$ is the total annual precipitation in the year $\mathrm{i}$, ${\mathrm{X}}_{\mathrm{in}}$ is the monthly precipitation in the month $\mathrm{n}$ of the year $\mathrm{i}$, and ${\mathrm{SI}}_{\mathrm{i}}$ corresponds to the SI for each year.

3. Results and Discussion

3.1. Temperature

This section presents the ECMWF-SEAS5 2-m temperature predictions and the comparison with CPC fields. Thus, it is possible to evaluate the model simulations regarding seasonal climate variability and spatial patterns of temperature. Figure 2 shows the seasonal mean temperature predictions produced by ECMWF-SEAS5, and the 600 retrospective seasonal forecasts give the mean value generated from the 25 ensemble members for the 24 years of hindcasts.

ECMWF-SEAS5 hindcasts have good performance in representing the seasonal temperature variation in the South American continent. However, except for MAM (Figure 2a), bias maps show that ECMWF-SEAS5 underestimates temperatures across SA most of the year. The system underestimates by up to 1 °C in portions of AMZ and NSA, whilst in SEB and SB, the cold bias is up to 2 °C (1 °C) during spring (summer).

The NEB presents mixed biases and the slightest negative deviations, overestimating up to 1 °C in most inland regions (Figure 2a,b). Similarly, the Pampas and central Argentina point to overestimates of up to 2 °C in MAM (Figure 2a). In contrast, the AMZ region presents a cold bias throughout the year. Besides, the system’s cold bias also covers AP, indicating underestimates of up to 2 °C in SON (Figure 2c).

Other studies also verified that global climate models (GCMs) underestimate air temperature prediction in SA [41,42,50,51]. For example, Kim et al. [50] analyzed the ECMWF-SEAS4 and Climate Forecast System version 2 (CFSv2) hindcasts and observed a systematic cold bias of both models in SA. Reboita et al. [41] obtained similar results, finding a CFSv2 cold bias over SEB and NEB from MAM to SON. Osman and Vera [51] investigated the performance of nine models from the Climate Historical Forecast Project (CHFP) and concluded that models intensify the cold bias in periods of both El Niño-Southern Oscillation (ENSO) phases. Finally, Chou et al. [42] found a systematic cold bias of the Eta regional model over the entire South American continent. However, the authors observed a warm bias in the AP region between MAM and MJJ, similar to the present results (Figure 2a).

GCMs also have constraints in simulating the complex topography of the South American continent, especially over regions such as the Andes and La Plata basin [24]. Moreover, systematic air temperature errors are associated with the initial soil moisture conditions since this variable modifies the temperature amplitude at the surface level, as the increase in soil moisture induces a reduction in temperature differences between day and night [52].

Skill scores allow evaluating the ECMWF-SEAS5 capability to represent the seasonal rainfall variability over SA. Skill scores correspond to the temporal correlations between the trimonthly temperature anomalies predicted by the hindcast ensemble mean and those obtained by CPC, considering both datasets’ climatological mean (1993–2016) [42]. Studies show that the threshold of skills scores equal to 0.3 indicates the usefulness of seasonal forecasts, regarding correlations below 0.3 to potentially harmful predictions, while correlations above 0.6 suggest beneficial forecasts even for application at a local scale [33,42]. For this reason, this study presents only skill scores equal to or greater than 0.3.

The best ECMWF-SEAS5 seasonal temperature anomaly skill scores occur in regions strongly influenced by the ENSO. For example, in the summer months, ECMWF-SEAS5 can predict the temperature in regions such as AMZ and NSA, whose statistically significant correlation values are above 0.7 (Figure 3d), corroborating other equivalent findings [33]. Due to the Pacific SST influence over such regions, temperatures are expected to rise (decrease) during El Niño (La Niña) [13,33].

The SA tropical region gives the best correlations throughout the year, with values above 0.6 over NSA, AMZ, and NEB areas. The NEB region provides good skill score results practically throughout the year. The excellent performance of ECMWF-SEAS5 in this Brazilian sector shows that its skillfulness does not derive only from the strong Pacific influence on the region but also from other modes of variability such as the Tropical Atlantic Dipole [13,32,53,54,55,56,57].

ECMWF-SEAS5 presents a moderate performance in SEB and SB in SON and DJF, where the system’s marginal performance shows statistically significant correlations above 0.4 (Figure 3c,d). On the other hand, AP indicates no appreciable performance over the entire year. In contrast, central Argentina shows modest correlations of up to 0.5 in JJA and DJF (Figure 3b,d).

Figure 4 compares the DJF temperature from the ECMWF-SEAS5 forecasts ensemble mean without (Figure 4a) and with (Figure 4b) the simple bias correction obtained with the hindcasts climatological error. An expressive improvement in the forecast bias and reduction in the model’s systematic errors can be seen across most SA (Figure 4b3). In this sense, we emphasize the usefulness of evaluating the hindcast and forecast datasets separately since the former provides the climatological error of the model, allowing for better use and interpretation of the real-time forecasts.

Figure 5 presents the ECMWF-SEAS5 forecasts ensemble mean with bias correction. The 255 prognostic seasonal forecasts provide the mean value computed from the 51 ensemble members for the five years of forecasts. In general, the bias correction technique reduces model errors throughout the year over most of the continent. For example, in MAM, the error correction reduces both the cold bias north of 20° S and the warm bias below this latitude (Figure 2a and Figure 5a). However, the warm bias in the NEB persists even after correction, as does the cold bias in central Brazil in SON (Figure 5c). During this season, the cold bias reduction in central Argentina is also evident. Similarly, in DJF, a cold bias reduction occurs over most Brazil. In summary, simple bias correction allows a better interpretation of the model predictions and indicates the satisfactory performance of ECMWF-SEAS5 seasonal temperature real-time forecasts over SA.

3.2. Precipitation

ECMWF-SEAS5 reproduces the seasonal patterns of precipitation in SA and shows good agreement with the CPC fields. As a result, ECMWF-SEAS5 captures the South Atlantic Convergence Zone (SACZ) [58] in the summer (Figure 6d). However, CPC shows smaller precipitation amounts than ECMWF-SEAS5, indicating that the forecast system overestimates rainfall in the region by up to 1 mm day⁻¹ (Figure 6d).

In autumn and winter, the dry season begins in SEB [1], reducing precipitation in central SA (Figure 6a,b). In those seasons, the precipitation maxima occur in NSA and AMZ, and ECMWF-SEAS5 reasonably represents these patterns. In contrast, during spring and summer, precipitation increases in most of SA [1], and the system appropriately represents the seasonal rainfall variability.

In SB, seasonal rainfall predictions show little variability during the year, similar to the CPC fields. However, ECMWF-SEAS5 underestimates rainfall most of the year and overestimates it in spring (Figure 6c).

In SON and DJF, ECMWF-SEAS5 overestimates rainfall up to 1 mm day⁻¹ in AMZ and SEB. Moreover, portions of AMZ, SEB, and western SA with the highest precipitation overestimates coincide with the temperature underestimates in those sectors, a thermodynamically expected feature since precipitation energy is used for water evaporation instead of heating the atmosphere via sensible heat [41]. Conversely, sectors like NEB and north Brazil have dry biases throughout the year. For example, from MAM to SON, dry biases in the central-east NEB and north Brazil are more prominent, showing that ECMWF-SEAS5 underestimates the rainy season of those areas [1,42].

In conclusion, seasonal rainfall predictions from ECMWF-SEAS5 hindcasts suggest systematic overestimates (underestimates) in AMZ and SEB during the rainy (dry) season and systematic underestimates in NSA and NEB throughout the year.

Regarding the seasonal precipitation anomaly skill scores, NSA has statistically significant skill scores above 0.5 in SON and DJF (Figure 7c,d), agreeing with previous findings for the same region [42]. Likewise, NSA has moderate skill scores in MAM and JJA (Figure 7a,b), which is also analogous to other outcomes [33,42]. Again, this precipitation anomaly forecast skill stems from the greater climate predictability in the tropical region and the robust response of these sectors to the Pacific SST [8,13].

NEB presents significant skill scores above 0.5 in the wetter season of MAM (Figure 7a), corroborating other studies [33,42]. This satisfactory performance is due to the strong relationship between tropical Atlantic SST and rainfall in NEB during MAM [13,53,54,55,56,57].

ECMWF-SEAS5 seasonal precipitation anomaly correlations showed best values in NSA, NEB, and SB and low scores in AMZ, SEB, AP, and central SA. Nevertheless, the ECMWF-SEAS5 performance in NEB and SB indicates that its capability derives not only from ENSO mechanisms but also from other modes of climate predictability that contribute to a better seasonal forecasting skill, such as the Tropical Atlantic Dipole and Southern Annular Mode.

Another region with relevant skill scores is SB, in which SON and DJF have significant values above 0.5 (Figure 7c,d). Even though Chou et al. [42] have not found notable skill scores for this sector (except at SON and OND seasons), the southern subtropical region has already shown relevant predictability associated with ENSO teleconnection effects [13,33,59]. Furthermore, our results suggest the potential benefit of ECMWF-SEAS5 precipitation predictions to increase farmers’ capability for drought and flood mitigation, especially in rainfed agriculture.

SEB has low skill scores throughout the year, similar to Chou et al. [42]. However, MAM and JJA show significant moderate values in some portions of the region, including central Brazil (Figure 7b,c). SEB presents the lowest predictability due to the little dependence on the SST anomalies and the great variety and variability of transient atmospheric systems that drive the rainfall over this sector [1,8,13].

Since ECMWF-SEAS5 showed good anomaly precipitation forecasting skills in NSA, NEB, and SB, subdomains of these sectors were selected to evaluate the system’s ability to simulate their interannual variability of seasonal rainfall anomalies, considering the climatological mean of 1993–2016. ECMWF-SEAS5 has an adequate capacity to simulate the interannual variability of seasonal rainfall anomalies in the NB2 (Figure 8). Despite underestimating precipitation, the system represented the oscillations of seasonal anomalies in the summer and spring seasons, as in DJF and SON from 2008 to 2018 (Figure 8a2), and captured the most prominent negative rainfall deviations, such as the drought events in 1997/1998 and 2015 [60]. In addition, predictions in winter coherently simulated seasonal anomalies’ variability, as in JJA from 2000 to 2012. On the other hand, the system did not adequately reproduce negative anomalies in MAM from 2011 to 2015 (Figure 8a1).

In the NEB2 subdomain, ECMWF-SEAS5 presents a drier bias, as illustrated in other results (Figure 6). However, the model’s performance in simulating the seasonal anomalies of MAM from 1998 to 2004 and from 2009 to 2020 is notable (Figure 8b1). Generally, GCMs have an excellent predictive capacity in NEB during MAM due to the region’s strong relationship between tropical Atlantic SST and rainfall [53,54,55,56,57]. Besides, it approached the largest negative deviations in 1998, 2002, and 2004. Despite representing the negative bias in ENSO periods like MAM in 1998 [61,62], the system smooths the negative anomalies in MAM of 2011 and 2012 and SON of 2015. From 2011 onwards, ECMWF-SEAS5 exhibits a wetter bias, failing to reproduce mainly the drier DJF anomalies (Figure 8b2).

In the SB2 subdomain, ECMWF-SEAS5 captures some CPC maximums, as in JJA and SON of 1997 (Figure 8c). However, the model remained stable throughout the forecasts’ integration during the different seasons and failed to simulate some precipitation maxima as in JJA of 2002 and 2011 (Figure 8c1). Despite simulating the interannual variability of anomalies with relative correspondence in SON, the system does not satisfactorily reproduce the phenomenon in other seasons. For example, the most pronounced negative anomalies, such as those occurring from 2003 onwards, are not well represented by the ECMWF-SEAS5 predictions. In SB2, it is clear that ECMWF-SEAS5 does not correctly represent the drier and wetter anomalies.

GCMs have constraints to predicting rainfall over Southern Brazil and Southeastern SA due to several aspects, such as a lower ability to simulate rainfall from frontal systems and orographic processes in subtropical latitudes [63], inappropriate representation of the physical processes involved in mesoscale convective systems [64], limitation in reproducing cyclogenesis associated with adiabatic processes [65], and underestimation of low-level jets [63,66]. Although results show a good ability of ECMWF-SEAS5 to predict precipitation anomalies in SB (Figure 7c,d), more regionalized analyses indicate that the system fails to capture the drier extremes. In this context, regional downscaling techniques may improve the model’s predictive ability in the sector.

After evaluating the quality of ECMWF-SEAS5 hindcasts seasonal predictions for SA, it is worth analyzing how the forecasts behave concerning the system’s climatological model. Despite systematic overestimation (underestimation) errors in regions such as the AMZ, SEB, and SB (NEB), hindcasts showed that ECMWF-SEAS5 has a satisfactory performance in representing the South American seasonal climate patterns.

Figure 9 presents the ECMWF-SEAS5 seasonal precipitation forecasts after bias correction obtained with the hindcast climatological error. The technique notably reduces the systematic errors of underestimating rainfall in NEB and NSA during the year and the wet bias in AMZ.

Nevertheless, the ECMWF-SEAS5 wet behavior over the continent remains even after bias correction. Equatorial portions maintain their wet bias throughout the year, and the rainfall overestimation in the continental part of SACZ during DJF persists (Figure 9d3). The latter sector has low or negligible predictability associated with SST variations, given that SACZ is dominated mainly by internal variability [67,68]. Despite that, bias correction reduces the overestimation of rainfall in SEB and the Amazonian portion of SACZ.

Here we highlight some limitations and contributions of the present study. Among the drawbacks, we stress that data used as observations for validation also have uncertainties due to the low density of rain-gauges and the interpolation techniques employed [69]. Such conditions may lead to misinterpretation of the forecast performance, which becomes even more problematic in SA, given the restricted rain-gauge network in regions with complex topography over the continent [31]. In this regard, the validation of climate models can also assist in evaluating the quality of observational data [70]. However, we highlight the similarity of the results found here with those obtained by validating ECMWF-SEAS5 with data from pluviometric stations [33], such as the model’s high performance in predicting rainfall in the north of Uruguay in DJF. In this sense, the current study corroborates previously found results [33] and complements the analyses by including different evaluation methods and the validation of real-time forecasts.

The current work also contributes by presenting the regions of SA with better forecast performance by ECWMF-SEAS5, indicating equal or superior performance to that yielded by a regional model with dynamical downscaling [42]. We emphasize that it is not intended to analyze the different models and studies in a competitive or exclusionary way but to point out deficiencies and potentialities that can assist in the cooperative use of different models and seasonal climate predictions.

Using different metrics to verify the model’s proficiency is also essential. An example occurs with the significant correlations of precipitation anomalies found in the NEB and SB since regionalized analyses showed that ECMWF-SEAS5 was not efficient in capturing some anomalous rainfall events in these sectors. Thus, anomaly correlations alone may not be sufficient for model performance evaluation.

The present study advances the analyses of ECMWF-SEAS5 dexterity in SA by examining the quality of real-time forecasts corrected by the hindcast climatological error. Here, the relevance of distinctly evaluating the forecast ensemble is emphasized to identify systematic model errors and correct them in the prognostic predictions, ensuring better interpretation and use of seasonal forecasts. Our results show that a simple bias correction significantly reduces the model’s systematic biases and reiterates its satisfactory performance over most of SA. In general, the biases present after correction occur in regions notably critical for climate predictability, such as the continental sector of SACZ.

Finally, we underline that the period of forecasts is relatively short, and complementary methods of evaluating their performance are needed. Considering the recent period of forecasts (2017–2021), we stress the action of La Niña and its effects on precipitation and temperature in SA. Within this framework, global models have lower dexterity in simulating the different types of La Niña, and limitations in predicting its teleconnection effects, given that errors persist even when SST biases are eliminated from the simulations [71]. In this context, we note that evaluating the performance of forecasts and hindcasts is essential since the latter has a more extensive set of predictions, allowing the model’s assessment with more robust statistics. On the other hand, the retrospective predictions present more idealized conditions than the prognostic forecasts since the hindcasts are better calibrated and submitted to complete data ingestion for the initial conditions [34,35,36].

3.3. Seasonality Index (SI)

The Seasonality Index (SI) measures the monthly rainfall variability throughout the year, assessing seasonal contrasts in rainfall quantities rather than dryness or wetness of a month in an absolute sense [49]. Although the method uses rainfall distribution for all months, a seasonal pattern is detected when the SI value is above 0.6 [72].

The SI index can be easily calculated, allowing its application to monthly accumulated climatic and hydrological variables such as evaporation and river discharge. However, its limitation relies on the description of average rainfall regimes, so there is no evidence of the seasonality of individual years, and the monthly unit of measurement may be too coarse to evaluate regions where seasons last only a few weeks [49].

Figure 10 shows the SI for SA, considering the ECMWF-SEAS5 hindcasts and the CPC rainfall. Again, the maps indicate many similarities between the two types of outputs, showing that ECMWF-SEAS5 satisfactorily represents the seasonal contrasts of precipitation on the continent.

In northeastern Brazil, a sector comprising Semiarid Brazil [73], ECMWF-SEAS5 points to a broader area extension with 0.80 ≤ SI ≤ 0.99, whose classification is markedly seasonal with a long drier season. Moreover, cluster analyses [1,33] also identified this region marked by interannual rainfall variability, with arid and rainy years, whose variability is due to teleconnection mechanisms [73], showing that ECMWF-SEAS5 can reproduce the phenomena. Moreover, SI also identifies other SA portions with a drier climate, such as the Atacama Desert region, with SI > 0.99, which corresponds to a category where most rain occurs in three months or less, a pattern similarly found in Sub-Saharan Africa [49].

Discrepancies occur in southern Argentina, where ECMWF-SEAS5 indicates a region with 0.20 ≤ SI ≤ 0.39, classified as equable but with a wetter season, where CPC yields 0.4 ≤ SI ≤ 0.59, categorized as rather seasonal with a short drier season. In addition, in western SA, covering the region east of the Atacama Desert, ECMWF-SEAS5 shows values of 1.0 ≤ SI ≤ 1.19 (indicating that most rain occurs in three months or less), while the CPC gives values of 0.8 ≤ SI ≤ 0.99 (markedly seasonal with a long drier season). These differences are assumed to be due to systematic errors in ECMWF-SEAS5, as the system gives the highest overestimates in the mountain range during summer (Figure 6d).

Differences aside, the two outputs generate very similar fields. The SEB region falls into the categories of seasonal (0.6 ≤ SI ≤ 0.79) and rather seasonal with a short drier season (0.4 ≤ SI ≤ 0.79), where CPC presents the same SI spatial distribution, showing that ECMWF-SEAS5 can simulate the influence of the South American monsoon system over the region [1,58].

For the SB domain, ECMWF-SEAS5 also shows a similar field to CPC, with SI ≤ 0.19 (categorized as very equable), characteristic of a region with homogeneous precipitation distribution throughout the year, previously identified in cluster analyses [1,33]. For the AMZ, there is also correspondence between the two fields, with values of 0.2 ≤ SI ≤ 0.39 (equable with a definite wetter season) in the northwest and 0.4 ≤ SI ≤ 0.59 (rather seasonal with a short drier season) in the central, eastern, and southwest AMZ, results similar to those of Sapucci et al. [74] that applied SI to different precipitation databases in the Amazon basin. In addition, ECMWF-SEAS5 also captures the variability of seasonal precipitation regimes in tropical SA, with values of SI ≤ 0.59 in the northwestern part of the continent and 0.4 ≤ SI ≤ 0.79 in the far north of SA.

3.4. Statistical Indicators

Table 1. Mean statistical indicators of temperature and precipitation anomaly predictions for each subdomain through the 1993–2021 period. For each indicator, the best results are in bold. Asterisks indicate statistically significant r values at a 95% significance level using Student’s t-test.

Temperature Statistical Indicators 1993–2021
Region	r	R2	d	RE (%)	KGE
SEB	0.32	0.10	0.57	5.58	0.73
AMZ	0.61 *	0.37	0.75	4.66	0.78
NEB	0.62 *	0.38	0.77	2.30	0.66
SB	0.22	0.05	0.51	9.27	0.80
AP	0.27	0.07	0.51	14.66	0.78
NSA	0.65 *	0.42	0.79	6.00	0.68
Precipitation Statistical Indicators 1993–2021
Region	r	R2	d	RE (%)	KGE
SEB	0.25	0.06	0.35	21.80	0.88
AMZ	0.23	0.05	0.38	24.80	0.80
NEB	0.54 *	0.30	0.67	24.09	0.87
SB	0.56 *	0.32	0.61	22.42	0.52
AP	0.43 *	0.19	0.48	23.23	0.30
NSA	0.41 *	0.17	0.54	18.63	0.71

presents the statistical indicators for each SA domain analyzed, considering the hindcast and forecast datasets. For temperature predictions, NSA shows the best results, with higher r (0.65), R² (0.42), and d (0.79). According to Gilewski and Nawalany [75], correlations above 0.6 indicate satisfactory model performance, implying that only the AMZ, NEB, and NSA sectors correctly predicted temperature anomalies. On the other hand, SB and AP show lower values of r, d, and higher magnitudes of RE.

Regarding the indicators for precipitation anomaly predictions, NEB and SB provide the best r (around 0.55) and d (around 0.6), suggesting an acceptable performance [75] of the ECMWF-SEAS5 seasonal rainfall predictions in these sectors. However, AMZ, SEB, and NSA also indicate good KGE (between 0.71 and 0.88). In contrast, AP presents the lowest R² (about 0.19) and KGE (0.30).

Notably, all regions presented less than 25% relative errors, within the acceptable range of 30% for precipitation simulation errors [76]. Precipitation predictions driven by observed data can reproduce rainfall at the regional and seasonal scales with errors due to uncertainties in observational datasets, which can be high, especially in remote areas or mountainous regions [76]. Again, we stress the need to use different metrics to assess model capability, as single indicators may not be sufficient to identify model potentialities and flaws in different regions of SA. For example, SB showed the best r and R² values in seasonal rainfall predictions, but regionalized analyses indicated that the model fails to capture the local interannual variability of precipitation anomalies. Similarly, the NSA indicated good skill score values for rainfall anomaly predictions during the austral spring and summer, but statistical indicators suggest both modest prediction performance (r~0.4) and lower magnitudes of associated errors (RE~18.6%).

4. Conclusions

The present work aimed to evaluate the quality of seasonal precipitation and 2-m temperature predictions from ECMWF-SEAS5 over SA. Hence, their hindcasts from 1993 to 2016 and forecasts from 2017 to 2021 were validated.

The seasonal mean precipitation prediction fields indicated an appropriate simulation of the seasonal patterns, satisfactorily representing the monsoon system and the rainfall variability in the SA interior. Nevertheless, hindcasts have systematically overestimated AMZ, SEB, and SB rainfall predictions. Conversely, the system has systematic underestimation errors in predicting rainfall over NEB.

The temperature fields indicate that ECMWF-SEAS5 has a systematic cold bias over most SA. However, exceptions occur in NEB and AP, where the system overestimates the temperature. In addition, the cold bias tends to be accentuated in the austral summer months, covering regions such as AP.

The skill score assessment showed that the best seasonal correlations of precipitation and temperature anomaly occur in high climate predictability areas such as the tropical latitudes of NSA and NEB, but also subtropical latitudes like SB. In other words, the most reliable ECMWF-SEAS5 seasonal climate predictions occur in the equatorial Amazon, northern SA, and northeastern Brazil tropical regions. However, there is also an indication of potential forecast skill in regions of medium and low climate predictability, such as in Southern Brazil in SON and DJF and Southeastern Brazil in MAM and JJA. In addition, analyses focused on the best performing sectors indicated that the system was able to capture extreme precipitation anomalies in the north and northeastern Brazil, such as in the years 1997/1998, 2012, and 2015. In contrast, ECMWF-SEAS5 did not adequately reproduce the dry anomalies in southern Brazil. Due to its systematic wet bias over most of SA, seasonal predictions come closer to the wet anomalies in northern and northeastern Brazil but overestimate or smoothen the dry ones in the country’s south. Given this, the system presents a potential predictive capacity for extreme drought events, mainly in the more tropical sectors of SA, and post-processing techniques can improve its applicability for such purposes [77].

Regarding the novelty of the present work, we highlight the analysis of the quality of ECMWF-SEAS5 real-time forecasts over SA. Applying the simple bias correction technique through the hindcast climatological error suggests a considerable reduction of the systematic errors in the forecasts. In general, the cold bias of the model is reduced over most of the continent, but temperature underestimation remains (although with less intensity) in the central sectors of SA. Similarly, bias correction in real-time precipitation forecasts illustrates an expressive reduction of the ECMWF-SEAS5 wet bias over SA, but systematic overestimation error persists in problematic regions for climate predictability, such as the continental sector of SACZ. Furthermore, we emphasize that the short series of prognostic forecasts and the influence of climate phenomena during the period make additional evaluations of the forecasts necessary, such as statistical and sensitivity tests of the different ensemble members and evaluation of the members’ dispersion from the ensemble mean. In this context, it is relevant that future studies investigate these issues for more assured reliability of ECMWF-SEAS5 real-time forecasts over different SA subdomains.

It is also relevant to highlight the need for using different metrics to evaluate the performance of ECMWF-SEAS5 seasonal climate predictions over SA. The results here indicate that, even in regions with satisfactory model performance, such as southern Brazil, the predictions still fail to capture anomalous events. Similar findings have been previously identified [33], reiterating the importance of assessing model accuracy in a more plural and localized manner.

Seeking physical explanations for the model errors is beyond the scope of this work. Despite numerous advances and constant development of climate modeling, errors are intrinsic to the process. However, several studies point to different causes for the deviations, such as deficiency in SST simulation, errors in the initialization of soil moisture conditions, and inappropriate physical parameterization [3,34,35,36,78]. In addition, improved extreme events prediction requires a deep understanding of drought and flood mechanisms, refined observations from data assimilation, better parameterizing techniques, efficient ensemble methodologies, and proper uncertainty quantification [17].

Furthermore, the realistic representation of South American climate features remains a challenge for numerical models, and they still fail in simulating regional climate over terrains with a complex topography [21,22]. Notwithstanding, the findings here allow us to conclude that ECMWF-SEAS5 effectively predicts seasonal precipitation and temperature over South America and has potential application to guide decision-making. Hence, we strongly encourage using EMCWF-SEAS5 seasonal climate forecasts by the different South American socioeconomic sectors.