Evaluation of General Circulation Models over the Upper Ouémé River Basin in the Republic of Benin

Abstract

This study assessed the performance of eight general circulation models (GCMs) implemented in the upper Ouémé River basin in Benin Republic (West Africa) during the Fifth Assessment Report on Climate Change. Historical rainfall simulations of the climate model of Rossby Regional Centre (RCA4) driven by eight Coupled Model Intercomparison Project (CMIP5) GCMs over a 55-year period (1951 to 2005) are evaluated using the observational data set. Apart from daily rainfall, other rainfall parameters calculated from observed and simulated rainfall were compared. U-test and other statistical criteria (R², MBE, MAE, RMSE and standard of standard deviations) were used. According to the results, the simulations correctly reproduce the interannual variability of precipitation in the upper Ouémé River basin. However, the models tend to produce drizzle. Especially, the overestimation of April, May and November rains not only explains the overestimation of seasonal and annual cumulative rainfall but also the early onset of the rainy season and its late withdrawal. However, we noted that this overestimation magnitude varies from one model to another. As for extreme rainfall indices, the models reproduced them poorly. The CanESM2, CNRM-CM5 and EC-EARTH models perform well for daily rainfall. A trade-off is formulated to select the common MPI-ESM-LR, GFDL-ESM2M, NorESM1-M and CanESM2 models for different rainfall parameters for the reliable projection of rainfall in the area. However, the MPI-ESM-LR model is a valuable tool for studying future climate change.

1. Introduction

Climate change resulting from anthropogenic greenhouse gas (GHG) emissions has adverse impacts on human lives, properties and the environment. Humans can only protect themselves from the inherent damage and manage the consequences of climate change by thinking about the development of projections of future changes. General circulation models (GCMs) are the most reliable means of estimating future climate change in an atmosphere where the concentration of greenhouse gases continues to increase significantly [1,2,3]. A climate model can be used to simulate past or future climate (climate projection). In addition to empirical laws and statistical adjustments, a climate model is based on fundamental physical laws (i.e., principles of conservation of mass, energy and moments) that can be mathematically transcribed [4]. Because of the large number of processes and interactions between the atmosphere, ocean and biosphere, the climate model is extremely complex. However, this sophisticated tool is limited by the computing power and time available today [5]. Thus, it does not allow taking into account fine-scale physical processes (e.g., local convection that determines point precipitation), which are necessary for a good representation of the local climate. To overcome this major drawback, researchers have developed regional climate models (RCMs). These models are applied in a limited area domain with lateral boundary conditions (LBCs) provided by a global climate model or reanalysis. The high-resolution RCM therefore simulates small-scale features from lower-resolution boundary information [6]. This allows the addition of small-scale information related to climate change projections [7]. However, the results of these climate models remain dependent on fairly large uncertainties [8,9]. In Africa, climate models are relatively satisfactory in predicting temperature changes. On the other hand, uncertainties remain on the results of rainfall projections. Giannini [10] has shown that there is a significant disagreement between GCMs on the future evolution of rainfall in West Africa [11,12]. In the IPCC report, no conclusions are drawn regarding the rainfall regime in West Africa. Climate projections of rainfall are therefore still uncertain for West Africa. However, West Africa is one of the most vulnerable regions of the continent, often subject to the adverse effects of climate change [13,14]. It is therefore essential to evaluate climate simulation tools in West Africa, especially considering that rainfall forecasts are of major importance for most vulnerabilities, mitigation and adaptation. In Benin, better rainfall forecasting is needed in the upper Ouémé River basin for better decision-making. Indeed, the upper Ouémé River basin is an important agricultural reservoir for the country where agriculture is the main source of economic wealth creation. Moreover, it is located at the foot of Mount Tanéka in the commune of Copargo where the source of the Ouémé River is located, whose floods cause catastrophic flooding in most of the southern regions of the country. The main objective of our research is to verify whether the climate model of the Rossby Regional Centre (RCA4) is capable of representing the main past characteristics of the medium climate and its variability over the period 1951–2005. Particular attention is paid to precipitation. The evaluation of the RCA4 regional model involves the statistical comparison of certain rainfall parameters over the upper Ouémé River basin. Therefore, the RCA4 simulations driven by certain CMIP5 GCMs with a resolution between 100 and 300 km [15] are compared with observations, which makes it possible to estimate the biases or deviations of the GCMs. This research has several advantages. The interest and motivation to carry out this study are related to several aspects. First of all, the subject is part of a CORDEX (COordinated Regional Climate Downscaling EXperiment) scientific research context. The main objectives are to better understand regional and local climate phenomena, not only to evaluate models but also to improve scaling techniques [16]. Then, in the more or less long term, the results of the study can be used as a tool for planners and decision makers and to predict the impact of climate change risks on human activities and ecosystems in order to improve response capacities.

2. Materials and Methods

2.1. Study Site

Benin is a West African country with estimated population of 9,983,884 inhabitants [16], located (Figure 1) between longitudes 1° E and 3°40′ E and latitudes 6°30′ N and 12°30′ N. It is bordered by the Atlantic Ocean in the south, the Federal Republic of Nigeria in the east, Togo in the west, Burkina Faso in northwest and Niger in the north. The only high mountain chain is no more than 600 m in elevation and is located in western north of the country. The climate is characterized by a bimodal precipitation regime in the south and unimodal rainfall regime in the north while the central part of the country is under a transitional precipitation regime. The mean maximum temperature for the whole country varies between 28 and 33.5 °C, while the mean minimum temperature fluctuates between 24.5 and 27.5 °C. In the southern part of the country, the mean annual precipitation varies from 1500 mm in the east to 900 mm in the west, while in the northern region it varies from 700 mm in the far north to 1200 mm in the middle north. The main important rivers are Mékrou, Alibori and Sota in the north, and Ouémé (the major river of the country), Mono and Couffo in the middle and southern parts of the country.

The upper Ouémé River basin, which is the study area, is located between 9° N and 10° N latitude and 1.5° E and 2.8° E longitude. The climate is characterized by the alternation of a unique rainy season and a dry season of approximately equivalent duration. The dry period ranges from November to March. All the rivers of this basin are characterized by an intermittent regime.

The mesoscale site of the upper Ouémé River basin (Benin) is part of the network of mesoscale observation sites of the AMMA-CATCH observatory. Thus, the upper Ouémé River basin is one of the most instrumented regions in West Africa. Different instruments are deployed there to monitor the water cycle and vegetation dynamics.

2.2. Observational Data

In this research, rainfall series for “essential factor of differentiation in tropical environment” [17,18] have been used. A network of eight stations out of the twenty of the rainfall networks managed by the Benin Meteo Agency (Figure 1), given the length of their series, missing data and aspects of temporal homogeneity, have been exploited. The stations having at least data on the period 1951–2005 and less than 20% missing rainy season data [19] were selected. The complete list of these stations and their geographical coordinates are presented in

Table 1. List of selected stations in the upper Ouémé River basin.

Stations	Benin Meteo Agency Code	Long. (Degrees)	Lat. (Degrees)	Elevation (m)
Bassila	D037	1.667	9.017	384
Bembèrèkè	D024	2.662	10.223	491
Birni	D026	1.517	9.983	430
Djougou	D030	1.662	9.692	439
Ina	D027	2.727	9.969	358
Kouandé	D019	1.683	10.333	442
Parakou	D034	2.612	9.357	392
Tchaourou	D038	2.600	325	325

. The stations of Kouandé and Tchaourou do not belong directly to our study site but have an influence on it.

2.3. Models

We evaluated a set of CMIP5 GCM-driven RCM simulations over West Africa. The RCM simulation outputs used here result from dynamic downscaling of eight CMIP5 GCMs’ data using the RCA4, originally developed by the Swedish Meteorological and Hydrological Institute (SMHI) as part of the CORDEX initiative. Simulated outputs from RCA4 are available over Africa at a spatial resolution of 0.44° × 0.44° (~50 × 50 km) with temporal coverage from 1951 to 2005 in the Coordinated Regional Climate Downscaling Experiment (CORDEX) program context. Only historical simulations were used in this study. The resulting outputs from the eight CMIP5 models’ data, dynamically downscaled by RCA4 during the fifth report on climate change (AR5) were obtained on the research website (http://www.cordex.org/data-access/esgf/). A single ensemble run was performed for each model, although some models had multiple ensemble runs. The set used was r1i1p1. In this research, we considered GCMs that simulate data for each of the 365 days of each year. However, the HadGEM2-ES model, which simulates data for only 360 days out of 365 of each year, was also considered and the missing data were generated using the R package forecast language.

Table 2. Presentation of global climate models selected for the study.

Research Centers	Acronym(s) for Centers	Model Name	Resolution (Long. × Lat.; Degrees)
Canadian Centre for Climate Modelling and Analysis Canada	CCCma	CanESM2	2.81 × 2.79
Centre National de Recherches Météorologiques, Centre Européen de Recherche et de Formation Avancée en Calcul Scientifique France	CNRM-CERFACS	CNRM-CM5	1.41 × 1.4
EC-EARTH consortium published at Irish Centre for High-End Computing Netherlands/Ireland	ICHEC	EC-EARTH	1.13 ×1.12
Atmosphere and Ocean Research Institute (The University of Tokyo), National Institute for Environmental Studies, and Japan Agency for Marine-Earth Science and Technology Japan	MIROC	MIROC5	1.41 × 1.39
Met Office Hadley Centre England	MOHC	HadGEM2-ES	1.875 × 1.25
Max Planck Institute for Meteorology Germany	MPI-M	MPI-ESM-LR	1.875 × 1.85
Bjerknes Centre for Climate Research, Norwegian Meteorological Institute Norway	NCC, NMI	NorESM1-M	1.875 × 1.85
Geophysical Fluid Dynamics Laboratory USA	NOAA-GFDL	GFDL-ESM2M	2.5 × 2.0

provides details of the GCMs’ data used in the downscaling experiment.

2.4. Model Performance

The analysis of the performance of the models with respect to the climatic observations needs a comparison between the observed values and those simulated by the climate models under consideration. Thus, observed and simulated daily rainfalls were compared. However, the annual, seasonal, quarterly and monthly totals as well as the moving averages calculated from these observed and simulated daily precipitations were also compared. To make the analysis much more complete, the rainfall indices (

Table 3. Definition of indices.

Indices	Name	Indices Calculation	Definition	Unit
Frequency Indices (adapted from WMO 2009)
R10mm	Number of heavy rainfall days	RRij $\ge$ 10 mm	Annual count of days when days rainfall $\ge$ 10 mm	Days
R20mm	Number of very heavy rainfall days	RRij $\ge$ 20 mm	Annual count of days when days rainfall $\ge$ 20 mm	Days
R25mm	Number of extremely heavy rainfall days	${\mathrm{RR}}_{\mathrm{ij}}\ge 25 \mathrm{mm}$	Annual count of days when days rainfall $\ge$ 25 mm	Days
CDD	Consecutive dry days	${\mathrm{RR}}_{\mathrm{ij}}<1 \mathrm{mm}$	Maximum number of consecutive days with RR < 1 mm	Days
CWD	Consecutive wet days	${\mathrm{RR}}_{\mathrm{ij}}\ge 1 \mathrm{mm}$	Maximum number of consecutive days with RR $\ge$ 1 mm	Days
Intensity Indices (adapted from WMO 2009)
SDII	Simple daily intensity index	${\mathrm{SDII}}_{\mathrm{j}}=\frac{{\sum }_{\mathrm{w}=1}^{\mathrm{W}}{\mathrm{RR}}_{\mathrm{wj}}}{\mathrm{W}}$	Annual mean rainfall when PRCP > 1 mm	mm/day
RX1day	Maximum daily rainfall	$\mathrm{Rx}1{\mathrm{day}}_{\mathrm{j}}=\max \left({\mathrm{RR}}_{\mathrm{ij}}\right)$	Maximum 1-day Rainfall	mm
RX3day	Maximum 3-day rainfall	Rx3dayj = max(RRij)	Maximum 3-day rainfall	mm
RX5day	Maximum 5-day rainfall	Rx5dayj = max(RRij)	Maximum 5-day rainfall	mm
R95p	Very wet day	$\mathrm{R}95{\mathrm{P}}_{\mathrm{j}}={\displaystyle {\displaystyle \sum }_{\mathrm{w}=1}^{\mathrm{W}}}{\mathrm{R}}_{\mathrm{wj}}$	Annual total rainfall when RR > 95 percentile	mm
R99p	Extremely wet day	$\mathrm{R}99{\mathrm{P}}_{\mathrm{J}}={\displaystyle {\displaystyle \sum }_{\mathrm{w}=1}^{\mathrm{W}}}{\mathrm{R}}_{\mathrm{wj}}$	Annual total rainfall when RR > 99 percentile	mm

) were calculated from the observed data and those simulated by the models. These indices were then compared.

Statistical criteria were used to compare the simulated against the observed data. It is recommended to use the mean absolute error (MAE) in model evaluation studies [20,21]. However, the root-mean-square error (RMSE) is more appropriate to represent model performance than the MAE when the error distribution should be Gaussian [22], and the use of RMSE is beneficial in certain circumstances. However, these authors recognize that when evaluating different models using a single metric, the differences in error distributions become larger. Accordingly, a combination of measures including (but certainly not limited to) RMSE and MAE is often necessary to evaluate model performance. In the present research, we used U-test [23] and statistical criteria such as the Pearson correlation coefficient calculation [3], the deviations of the standard deviations [3,24] as well as the mean bias (MBE) calculation [19], the MAE [19,20,22] and the RMSE for the comparison between simulated and observed daily rainfall; for the precipitation indices, annual, seasonal, quarterly and monthly total rainfall, MBE, MAE and RSME were used. The choice of these different criteria mentioned above for the evaluation of the results of the models is based on the fact that each criterion helps to appreciate a certain performance of the models. The procedure for the evaluation of GCMs based on their historical simulation of precipitation is outlined below:

• The use of U-test will help to assess whether the numerical general circulation models (GCMs) are sufficiently reliable.
• The correlation is used to quantify the linear relationship between observed rainfall and those simulated by the models. The calculation of the Pearson correlation coefficient is based on the calculation of the covariance between the two variables. The correlation coefficient is actually the standardization of the covariance.
• The difference between the standard deviations of observed rainfall and those simulated by the models provides information on the simulation of interannual variability by the models.
• MBE makes it possible to quantify the average systematic error (mean bias) of each model. The better the accuracy of the estimation model, the lower the mean bias. A positive bias indicates a tendency to overestimate the model, conversely a negative bias indicates that the estimated values are underestimated compared to those actually observed. The average bias (MBE) is calculated from the following formula:

  $$MBE={\displaystyle \sum }_{j=1}^M\frac{\left[{\sum }_{i=1}^N\frac{\left(X_{Sim i,j}-X_{Obs i,j}\right)}{N}\right]}{M}$$

  (1)

  where X is the value of the parameter considered for each year (period from April to October) for the period of 55 years (1951–2005); Obs corresponds to observations and Sim to simulations. N is the number of years (i.e., 55 years) and M is the number of stations considered for each model on our study site.
• MAE allows to quantify the amplitude of the average errors regardless of the positive or negative biases (magnitude of the differences between the simulated and observed values). The mean absolute error (MAE) is defined as follows:

  $$MAE={\displaystyle \sum }_{j=1}^M\frac{\left[{\sum }_{i=1}^N\frac{\left|X_{Sim i,j}-X_{Obs i,j}\right|}{N}\right]}{M}$$

  (2)
• RMSE allows to include evidence of not only the errors in the mean value but also those in the interannual variability of the simulated versus observed data. If the RMSE is almost identical to the mean absolute error (and not zero), a systematic error affects the mean value of the simulated values and the error on the variability is therefore low or zero. On the other hand, if the RMSE is greater than the MAE (and not zero), the simulated values are affected both by an error in the mean value of the data but also by an error in the interannual variability. RMSE is often used to describe the dispersion of observations and simulations (the lower the RMSE is, the more that the prediction models are related the observations). The RMSE is calculated according to the following formula:

  $$RMSE={\displaystyle \sum }_{j=1}^M\frac{\sqrt{\frac{{\sum }_{i=1}^N{\left(X_{Sim i,j}-X_{Obs i,j}\right)}^2}{N}}}{M}$$

  (3)
  Statistical analyses were performed using the statistical software R 3.5.1 (R Development Core Team, http://www.R-project.org) with packages stats and forecast.

3. Results

3.1. U-Test Results

First, the U-test or rank test (that tests the equality of two distributions) was performed, in order to assess the temporal reproduction of precipitation by the models. For this test, p-value took the value $2.2\times {10}^{-6}$ in all stations. The results of this test showed that there was a significant difference between the simulated and observed data at the 5% level at the eight stations. The boxplot of observed and simulated data (Figure 2) clearly shows that the models tend to produce drizzle (overestimate low precipitation and underestimate heavy rainfall). Indeed, the heavy rains simulated by the models are much more concentrated between 20 and 50 mm. Observed rainfall above 50 mm is well taken into account by the models. This is much more evident with the model CanESM2.

The estimated rainfall values between 0 and 20 mm have a high density (Figure 3), which shows a natural tendency of climate models to estimate rainfall levels much lower than those observed in the field. In other words, the models predict more rainy days than observed. Extreme values are less predictable. In the figure below, we have, on the abscissa, the rainfall height expressed in millimeters (mm).

3.2. Correlation Test Results

Table 4. Correlation coefficients between observed and simulated precipitation for each station by general circulation model.

	CanESM2	CNRM-CM5	EC-EARTH	HadGEM2-ES	MIROC5	MPI-ESM-LR	NorESM1-M	GFDL-ESM2M
Bassila	0.184	0.191	0.176	−0.041	0.164	0.161	0.169	0.181
Bembèrèkè	0.169	0.179	0.180	−0.042	0.168	0.169	0.173	0.183
Birni	0.198	0.223	0.207	−0.066	0.190	0.196	0.196	0.200
Djougou	0.206	0.213	0.189	−0.050	0.169	0.195	0.198	0.203
Ina	0.178	0.208	0.195	−0.049	0.191	0.194	0.169	0.190
Kouandé	0.192	0.211	0.203	−0.058	0.195	0.204	0.179	0.199
Parakou	0.155	0.182	0.165	−0.042	0.146	0.164	0.159	0.159
Tchaourou	0.154	0.175	0.171	−0.044	0.140	0.174	0.149	0.172

below gives the values of the correlation coefficient for all stations between observed and simulated rainfall values. Table 4 shows a weak correlation between the observed and the simulated values. Indeed, according to this table, it is observed that the correlation coefficients are low for all the stations. They are positive for all models except for the model HadGEM2-ES, where they are particularly weaker and negative at all stations. The correlation coefficients are higher for the model CNRM-CM5.

3.3. Interannual Variability

The simulated interannual rainfall variability of all stations studied is underestimated. Indeed, the deviations of the standard deviations are negative (

Table 5. Deviations of standard deviations of observed and simulated precipitations by models over the period 1951–2005.

	CanESM2	CNRM-CM5	EC-EARTH	HadGEM2-ES	MIROC5	MPI-ESM-LR	NorESM1-M	GFDL-ESM2M
Bassila	−4.038	−3.467	−3.299	−3.124	−3.014	−2.47	−3.168	−2,825
Bembèrèkè	−5.562	−4.511	−4.711	−4.583	−4.581	−4.07	−4.572	−4,615
Birni	−3.853	−3.247	−2.991	−2.864	−2.959	−2.32	−2.833	−2.808
Djougou	−4.508	−3.996	−3.805	−3.589	−3.685	−3.01	−3.633	−3.513
Ina	−4.966	−4.041	−4.087	−4.022	−4.033	−3.47	−3.981	−3.998
Kouandé	−3.833	−3.032	−2.983	−2.819	−2.892	−2.29	−2.729	−2.832
Parakou	−5.574	−4.839	−4.645	−4.631	−4.630	−3.97	−4.579	−4.541
Tchaourou	−4.369	−3.731	−3.307	−3.245	−3.184	−2.69	−3.267	−3.052
mean	−4.588	−3.858	−3.729	−3.609	−3.622	−3.04	−3.595	−3.523

).

According to this table, the results for the MPI-ESM-LR model appear to be more satisfactory than the results obtained with the other models because the absolute values of the deviations of the standard deviations is lower for all the stations. The highest values are obtained for the model CanESM2. Considering the mean values of these deviations, the MPI-ESM-LR, GFDL-ESM2M, NorESM1-M, HadGEM2-ES, MIROC5, EC-EARTH, CNRM-CM5 and CanESM2 models are in the order of the most satisfactory to least satisfactory.

3.4. Statistic Criteria Results

The mean bias (MBE) between simulated and observed precipitation is positive for all models at all stations where rains appear to be overestimated, except for the CanESM2 model which underestimates them at Bembèrèkè (−0.764), Ina (−0.615) and Tchaourou (−0.170) and the model CNRM-CM5 at Bembèrèkè (−0.026).

Table 6. Mean values of MBE, MAE and RMSE between observed and simulated precipitation by models for all stations.

	CanESM2	CNRM-CM5	EC-EARTH	HadGEM2-ES	MIROC5	MPI-ESM-LR	NorESM1-M	GFDL-ESM2M
MBE	0.349	0.948	1.389	1.395	1.773	1.774	1.173	1.326
MAE	4.645	4.966	5.309	6.441	5.630	5.579	5.210	5.228
RMSE	9.249	9.467	9.661	10.834	9.905	10.171	9.720	9.834

below shows the mean bias, mean absolute error (MAE), and root-mean-square error (RMSE) for each model. According to this table, the average bias (MBE) is positive for all models. Thus, all models overestimate rainfall for all stations.

The lower the results criterion (RMSE) values, the more satisfactory the results of the simulation. For all stations, the lowest values of this parameter are obtained with the model CanESM2, followed by the model CNRM-CM5. Considering the mean value of this parameter for all studied stations, the simulation results are from the most satisfactory to the least satisfactory according to the following order of the models: CanESM2, CNRM-CM5, EC-EARTH, NorESM1-M, GFDL-ESM2M, MIROC5, MPI-ESM-LR and HadGEM2-ES. Absolute errors are more pronounced than the mean biases in particular for the model HadGEM2-ES. For all models, RMSE > MAE, therefore the errors are not only due to biases in the mean, but also due to the models having difficulty in reproducing interannual variability.

3.5. Rainfall Cumulation Evolution

In this section, annual, seasonal and quarterly precipitation is analyzed.

Over the period 1951–2005, we observed annual rainfall amounts to between about 1000 and 1500 mm. Simulated annual rainfall amounts varied between 602.62 and 1962.136 mm for CanESM2, 927.014 and 2083.879 mm for CNRM-CM5, 1023.141 and 2516.632 mm for EC-EARTH, 912.202 and 2472.769 mm for HadGEM2-ES, 1049.185 and 2540.838 mm for MIROC5, 1039.449 and 2813.003 mm for MPI-ESM-LR, 793.2987 and 2437.91 mm for GFDL-ESM2M and 879.57 and 2176.05 mm for NorESM1-M.

Figure 4 below shows that the data simulated by the eight models generally reproduce the interannual rainfall dynamics quite correctly. However, in terms of amplitude, all models overestimated the annual rainfall accumulation at all stations, except the CanESM2 model which underestimates it in Bembèrèkè, Parakou and Tchaourou. However, as shown in

Table 7. Results of statistical criteria for annual rainfall accumulation.

	MBE	MAE	RMSE
CanESM2	−28.84	276.02	340.22
CNRM-CM5	233.82	313.81	370.75
EC-EARTH	393.69	438.35	499.13
HadGEM2-ES	359.82	423.53	483.06
MIROC5	465.35	499.47	552.99
MPI-ESM-LR	484.26	518.63	588.60
NorESM1-M	324.20	391.89	451.12
GFDL-ESM2M	330.62	400.24	476.57

below, the average bias for all stations suggests an overestimation of annual rainfall totals for all models, except for the CanESM2 model where there is a very small underestimation (28.84 mm). The MAE and RMSE values are lower for CanESM2. Overall, the CanESM2 model appears to be the best model for simulating annual rainfall totals. It is followed by the CNRM-CM5 model. In all cases, RMSE > MAE, so the errors are not only related to a systematic bias in the average of the simulated totals but also to errors in the interannual variability of the rainfall totals. When the observed data are around 1000 mm, the CanESM2 model reproduces them better. This is the case at the sites of Bassila, Birni, Djougou, Kouandé and Tchaourou. In the case where the observed data are between 1000 and 1500 mm, the other models reproduce the trends of the actual observations. However, the EC-EARTH model is the most accurate, followed by the MIROC5 model.

This evolution of annual rainfall accumulation is also observed at the level of seasonal rainfall accumulation (Figure 5). This evolution of seasonal cumulations suggests that the overestimation of annual cumulations by the models is due to the overestimation of seasonal cumulations by the models. Since the average bias, MAE and RMSE are lower for the CanESM2 model (

Table 8. Results of statistical criteria for seasonal cumulation.

	MBE	MAE	RMSE
CanESM2	2.62	266.37	330.19
CNRM-CM5	257.92	320.84	376.33
EC-EARTH	392.78	431.29	489.14
HadGEM2-ES	371.49	426.76	483.15
MIROC5	375.24	422.41	478.15
MPI-ESM-LR	484.71	512.75	578.13
NorESM1-M	325.92	381.52	441.24
GFDL-ESM2M	357.47	411.49	486.63

), this model represents seasonal cumulation well. The CanESM2 model is followed by the CNRM-CM5 model. The very low value (2.619 mm) of the CanESM2 model shows the good accuracy with which it simulates seasonal accumulation.

Continuing the analysis with total quarterly precipitation (Figure 6, Figure 7, Figure 8 and Figure 9), the following observations were noted:

In the first quarter (January–February–March: JFM), at all observation sites, all models are representative except the MIROC5 model, which has higher overestimations of quarterly precipitation totals. The average bias is lower for all models except MIROC5, where it is higher (

Table 9. Results of statistical criteria for first quarter total precipitation.

	MBE	MAE	RMSE
CanESM2	−22.35	34.57	44.99
CNRM-CM5	23.13	33.20	44.74
EC-EARTH	1.44	38.09	49.80
HadGEM2-ES	14.78	37.53	49.82
MIROC5	73.78	80.61	96.35
MPI-ESM-LR	−11.79	36.22	46.40
NorESM1-M	5.59	38.78	49.16
GFDL-ESM2M	−25.02	34.71	46.49

). The CanESM2, MPI-ESM-LR and GFDL-ESM2M models underestimate the first quarter totals, their bias being negative, while all other models overestimate them. The simulation accuracy is better for EC-EARTH, NorESM1-M, MPI-ESM-LR and HadGEM2-ES models in descending order.

In the second quarter (April–May–June: AMJ), the total of quarterly simulated data from all models follows trends in total of quarterly observed precipitation (Figure 7). However, these cumulations are overestimated by all models, with the average bias being positive everywhere (

Table 10. Results of statistical criteria for second quarter total precipitation.

	MBE	MAE	RMSE
CanESM2	175.86	221.80	274.20
CNRM-CM5	130.02	156.93	185.79
EC-EARTH	317.09	319.75	344.76
HadGEM2-ES	302.55	308.02	340.29
MIROC5	246.64	260.30	293.04
MPI-ESM-LR	257.90	268.62	305.08
NorESM1-M	369.46	373.58	401.31
GFDL-ESM2M	242.52	252.48	286.50

). With the lower values of MBE, MAE and RMSE, the CNRM-CM5 model appears more representative. It is followed by the CanESM2 model.

In the third quarter (July–August–September: JAS), the CanESM2 model underestimates precipitation totals (MBE < 0) while all other models overestimate them (

Table 11. Results of statistical criteria for third quarter total precipitation.

	MBE	MAE	RMSE
CanESM2	−100.75	172.99	209.05
CNRM-CM5	66.37	159.27	196.17
EC-EARTH	70.35	168.39	206.76
HadGEM2-ES	44.49	170.21	210.89
MIROC5	88.36	178.54	218.08
MPI-ESM-LR	131.96	190.90	229.99
NorESM1-M	369.46	373.58	201.33
GFDL-ESM2M	73.15	179.87	228.97

). However, the NorESM1-M and MPI-ESM-LR models have the highest biases. Overall, all models are representative. The quarterly totals of the data simulated by these models perfectly follow the trends of the quarterly totals of observed precipitation (Figure 8). The RMSE values are close for all models.

In the fourth quarter (October–November–December: OND), the CanESM2 model underestimates cumulative precipitation (MBE < 0) while all other models overestimate it (MBE > 0). The MBE and RMSE values are higher for the MPI-ESM-LR model (

Table 12. Results of statistical criteria for fourth quarter total precipitation.

	MBE	MAE	RMSE
CanESM2	−81.61	85.41	104.73
CNRM-CM5	60.57	102.69	120.39
EC-EARTH	4.82	82.89	105.89
HadGEM2-ES	27.56	79.68	100.46
MIROC5	56.03	87.87	109.27
MPI-ESM-LR	106.17	125.90	157.62
NorESM1-M	54.37	75.70	95.54
GFDL-ESM2M	39.98	94.17	119.69

). The lowest mean bias is obtained with the EC-EARTH model (MBE = 4.821). This model simulates the fourth quarter totals more accurately. However, the NorESM1-M model has the smallest RMSE value and is therefore the most representative. The oscillations observed in the fourth quarter (Figure 9) suggest that models overestimate the end of season rains, which often occur in October.

In total, the overestimation observed in the second quarter added to that of the fourth quarter at the level of all models explains the overestimations observed at the seasonal level and even at the annual level.

Thus, during the pre-installation period of the West African monsoon (pre-onset) and to some extent during the monsoon recession period, the models overestimate rainfall. In the period when the monsoon is well established (phase onset) the different models better represent the rainfall regime.

3.6. Seasonal Cycles Evolution

At this level of detail, the precipitation 11-day moving average over the period 1956–2005 (Figure 10) follows those of the observations.

Over the first three months, all models except MIROC5 and NorESM1-M represented the daily average precipitation. MIROC5 overestimates the daily average precipitation from mid-February and NorESM1-M overestimates it from mid-March.

From April 1 to the end of June, all models overestimated the daily average precipitation. However, the evolution of the daily average precipitation from the CNRM-CM5 model is much closer to the observations. The average daily precipitation for July and August is better represented by all models. However, from September to December, the CanESM2 and NorESM1-M models underestimate the average daily precipitation at all stations while the CNRM-CM5, MIROC5, MPI-ESM-LR, GFDL-ESM2M and EC-EARTH models represent it well in Bembèrèkè, Ina and Parakou but overestimate it in Bassila, Birni, Djougou, Kouandé and Tchaourou.

In summary, it is noted as indicated above that the start of the rainy season is very poorly simulated by models that have an early start with a high overestimation of rainfall. The end is also poorly simulated with early monsoon withdrawal for CanESM2 and NorESM1-M models and rather late withdrawal for CNRM-CM5, MIROC5, MPI-ESM-LR, GFDL-ESM2M and EC-EARTH models. It should also be noted that, unlike the unimodal observation regime, in a systematic way the models have a bimodal seasonal cycle with a first precipitation peak in May. Thus, the coastal regime is maintained in the models.

The results found in this section confirmed those previously found above at the upper time scales.

3.7. Observed and Simulated Precipitation Indices Comparison

The analysis of

Table 13. Average of the statistical criteria of the indices for all the stations studied.

	CanESM2 Model			CNRM-CM5 Model			EC-EARTH Model
	MBE	MAE	RMSE	MBE	MAE	RMSE	MBE	MAE	RMSE
CWD	32.99	32.99	33.82	43.18	43.18	43.84	47.41	47.41	47.99
SDII	−7.82	7.82	8.38	−7.26	7.27	7.82	−6.92	6.93	7.52
CDD	−3.27	6.32	8.47	−2.29	6.33	8.07	−6.59	7.87	9.68
RX1day	−50.81	51.64	56.79	−39.20	45.87	52.73	−39.36	40.05	49.81
R3day	−44.75	46.25	53.86	−32.48	43.99	53.00	−31.04	38.64	47.26
RX5day	−43.20	46.23	56.27	−32.58	43.48	53.99	−28.20	40.37	51.19
R10mm	1.37	14.26	17.28	12.00	15.55	17.81	19.54	21.54	24.22
R20mm	−16.76	16.83	17.89	−15.52	15.53	16.63	−14.61	14.65	15.97
R25mm	−13.33	13.33	14.17	−12.64	12.64	13.55	−12.23	12.22	13.20
R95P	−100.79	103.84	121.96	−57.82	80.10	99.00	−43.70	71.49	88.88
R99P	−32.86	35.24	43.26	−16.86	35.13	45.62	−19.52	31.35	40.53
	HadGEM2-ES Model			MIROC5 Model			MPI-ESM-LR Model
	MBE	MAE	RMSE	MBE	MAE	RMSE	MBE	MAE	RMSE
CWD	45.79	45.79	46.42	48.08	48.08	48.62	47.54	47.54	48.19
SDII	−6.59	6.91	7.52	−7.11	7.11	7.69	−6.44	−6.47	7.10
CDD	−6.59	7.65	9.47	−7.57	7.87	9.92	−6.01	7.51	9.33
RX1day	−39.76	44.19	50.81	−34.93	39.73	46.33	−31.43	39.75	47.09
R3day	−31.09	38.64	47.22	−25.89	36.07	43.84	−21.66	37.99	48.15
RX5day	−28.92	38.19	48.72	−24.49	37.33	47.29	−19.91	37.62	49.22
R10mm	20.00	22.27	24.99	17.12	20.34	22.77	26.27	27.37	30.34
R20mm	−14.09	14.21	15.45	−14.48	14.54	15.80	−11.77	12.11	13.55
R25mm	−12.26	12.26	13.23	−12.22	12.22	13.11	−11.16	11.18	12.23
R95P	−44.48	70.05	88.43	−36.09	66.21	84.60	−13.28	68.18	88.06
R99P	−16.56	33.04	42.16	−10.66	30.75	39.49	−2.70	32.90	45.21

and

Table 14. Average of the statistical criteria of the indices for all the stations studied (continued).

	NorESM1-M Model			GFDL-ESM2M Model
	MBE	MAE	RMSE	MBE	MAE	RMSE
CWD	43.32	43.32	43.98	44.73	44.73	45.57
SDII	−6.87	6.87	7.47	−6.86	6.87	7.48
CDD	−5.11	7.59	9.57	−4.32	6.97	8.95
RX1day	−36.27	40.92	48.04	−34.95	38.91	46.24
R3day	−28.02	37.13	45.19	−25.89	35.08	44.26
RX5day	−23.76	37.45	46.96	−22.10	37.99	47.75
R10mm	16.38	19.30	21.80	17.89	20.92	24.43
R20mm	−13.95	14.03	15.33	−13.39	13.80	15.05
R25mm	−12.02	12.02	12.96	−11.62	11.65	12.67
R95P	−43.05	69.39	87.67	−34.84	68.56	85.09
R99P	−11.69	30.74	40.38	−9.27	29.95	38.88

shows that all models overestimate the CWD and the number of days of heavy rain (R10mm) for all studied stations because the MBE values are positive. On the other hand, the average intensity per rainy day (SDII), CDD, RX1day, R3day, RX5day, number of very heavy rainy days (R20mm), number of extremely heavy rainy days (R25mm), the annual total rainfalls on very rainy days (R95P) and the annual total rain on extremely rainy days (R99P) are underestimated by the models (MBE < 0).

The RMSE values suggest that most indices’ errors are essentially related to a systematic bias in the average of the simulated indices and little or no dependence on errors in interannual variability. However, the slight increase in RMSEs relative to MAEs for the CDD and RX3day indices for most models reveals an additional difficulty in reproducing the interannual variability of number of days of heavy precipitation and the maximum number of consecutive dry days. For the rest, the RMSE and MAE values seem almost identical. The number of rainy days is the most biased index.

In the previous table, the background green color indicates the best estimated indices by each model.

4. Discussion

In this research work, we evaluated the performance of eight general traffic models through the simulations of the RCA4 regional model. Particular attention was paid to precipitation data and some of their characteristics. The performance of a model varies according to the climate parameter taken into account for its evaluation. This is the case, for example, in the studies carried out in Pakistan [8,25], Southeast Asia [2] and West Africa [1]. Similarly, for the same climate parameter, the performance of a model can vary from one statistical criterion to another over the same region. Thus, in the framework of this study, where special attention was paid to precipitation, almost all the models reproduced the interannual variability of precipitation in the upper Ouémé River basin well. This result has already been obtained for West Africa with the regional climate model developed by the Hadley Centre [8]. However, the models overestimated daily rainfall and the number of low rainfall days, especially in the months of April and May. This result therefore explains the simulation of the early start of the rainy season observed with the 11-day moving average calculated over the period 1951–2005. However, the models reproduce well the rainfall in the middle of the rainy season, i.e., from July to September. The late withdrawal of the rainy season observed in the model simulations is explained by the overestimation of the number of rainy days, especially in November. These overestimations of daily rainfall, especially in the months of April, May and November, therefore, explain the models’ overestimation of the cumulative annual and, especially, half-yearly totals. However, the extent of the overestimation varies from one model to another. As for the other rainfall parameters, the models poorly reproduced the rainfall indices (R10mm, R20mm, R25mm, R95p, R99p), the number of consecutive dry days (CDD) and the average intensity per rainy day (SDII). As far as the performance of the models is concerned, it varies from one rainfall parameter to another over the upper Ouémé valley. However, when we look at all the parameters studied, the best performing results of the RCA4 set are MPI-ESM-LR, GFDL-ESM2M, NorESM1-M, and CanESM2. These results are somewhat similar to those obtained for the whole of West Africa [8]. Taking into account the different results obtained in this research when evaluating model performance, the MPI-ESM-LR model can be recommended for future climate change studies on the upper Ouémé River basin. The results of this study can be modified when particular types of assessments are carried out. In addition, model outputs (model ensemble) could be combined because this can increase the skill, reliability and consistency of climate model simulations [26].

5. Conclusions

By comparing the results of the models, we concluded that the CanESM2 model gives lower RMSE values than those produced by the other models, with the correlation coefficient being lower for all models. This model is more effective for simulating daily rainfall, while the MPI-ESM-LR model is more efficient in reproducing the natural variability of the series. For the reproduction of daily rainfall, the CanESM2, CNRM-CM5, EC-EARTH, NorESM1-M, GFDL-ESM2M, MIROC5, MPI-ESM-LR and HadGEM2-ES models can be classified in this order from the most satisfactory to the least satisfactory. However, in the order of the most satisfactory to the least satisfactory, we have for the reproduction of the natural variability of the series the models MPI-ESM-LR, GFDL-ESM2M, NorESM1-M, HadGEM2-ES, MIROC5, EC-EARTH, CNRM-CM5 and CanESM2. Taking into account the precipitation indices, it is rather the following order from most satisfactory to least satisfactory that is obtained: MPI-ESM-LR, GFDL-ESM2M, NorESM1-M, CanESM2, CNRM-CM5, MIROC5, HadGEM2-ES and EC-EARTH.

Thus, it is not easy to say that one model is better than another because all models have advantages and limitations. The best way is to work with several methods and compare the results. Moreover, scenario results must be analyzed and interpreted carefully. In our opinion, general circulation models still require a great deal of research in order to produce more satisfactory results in the upper Ouémé River basin with respect to the simulation of daily precipitation. However, by combining the results obtained for the various parameters considered in this study, it can be concluded that the MPI-ESM-LR model is more satisfactory for rain simulation in the upper Ouémé River basin, followed in decreasing order of satisfaction by the GFDL-ESM2M, NorESM1-M, CanESM2, CNRM-CM5, MIROC5, HadGEM2-ES and EC-EARTH models.