Performance measures are first discussed separately under each regionalization method and are used to compare associated model performance to that obtained using calibrated values. Subsequently, overall performance is discussed and suitability of regionalized parameters in making watershed response predictions in comparison with the calibrated model parameter is evaluated. Details are presented in ensuing subsections.
3.1. Global Averaging
Table 3 shows model parameter values as obtained through global averaging. Default values of the parameters are presented for reference purposes. For the Illinois River watershed, the ground water revaporation (GW_REVAP) value obtained from global averaging was close to that obtained through calibration (
Table 1) while the global surface runoff lag (surlag) value obtained was closer to the calibrated value for Cadron Creek watershed (
Table 1). In some cases, global values obtained differed substantially from calibrated values. For example, while the global average channel hydraulic conductivity (CH_K2) value was 44 mm/hr, the calibrated value for the Cadron Creek watershed was 0.025 (
Table 1). Similarly, while the change in curve number (CN2) obtained through global averaging was positive (reflecting an increase), the value obtained through calibration was negative (reflecting a decrease) for all the test watersheds (
Table 1).
Table 4 shows model performance statistics obtained using regionalized parameters in comparison to default and calibrated models for the periods October 1998–September 2000, October 2000–September 2004, and for the whole period October 1998–September 2004. Performance obtained using global averaged parameters is shown in column C3. The stations Benton and Sheridan in the Upper Saline watershed did not have data for the period October 1998–September 2000, and thus were only evaluated for the period October 2000–September 2004.
For the Illinois River watershed, NS values obtained using global average parameters ranged between 0.49 and 0.81. In two of the cases, (October 1998–September 2000, Siloam Springs and Savoy) model performance obtained using global average parameters was better than that obtained through calibration. In one case (October 1998–September 2004, Savoy) model performance was very similar for both global averaged and calibrated parameters. For the most part, however, model performance was comparable to calibrated model when global average parameters were used. For example during the period October 2000–September 2004, the NS value obtained from global averaging at Savoy was 0.61 compared to 0.64 obtained through calibration. Similarly, the NS value obtained at the Elm springs station using global averaged parameters was 0.81 compared to 0.90 obtained through calibration (October 1998–September 2000).
Table 3.
Model parameter values obtained using global averaging.
Table 3.
Model parameter values obtained using global averaging.
Parameter description | Parameter | Default | Global average |
Base flow recession factor, days | ALPHA_BF | 0.048 | 0.590 |
Ground water delay, days | GW_DELAY | 31 | 41 |
Ground water revaporationh coefficient | GW_REVAP | 0.020 | 0.056 |
Threshold depth for ground water flow to occur, mm | GWQMN | 0.00 | 14 |
Deep aquifer recharge fraction | rchrg_dp | 0.050 | 0.272 |
Threshold depth for revaporation to occur, mm | REVAPMN | 1 | 15 |
Snow fall temperature, ºC | sftmp | 1.0 | 1.1 |
Surface runoff lag coefficient, days | surlag | 4.00 | 3.64 |
Plant uptake compensation factor | EPCO | 1.000 | 0.830 |
Soil evaporation compensation factor | ESCO | 0.950 | 0.730 |
Average slope length, m | SLSUBBSN | Varies1 | −2.9% |
Curve Number (AMChhII) | CN2 | Varies2 | 2.8% |
Effective channel hydraulic conductivity, mm/hr | CH_K2 | 0.5 | 44.0 |
Manning's n | ch_n | 0.014 | 0.3‡ |
Soil albedo | sol_alb | 0.100 | 0.370 |
Available water capacity, m/m | SOL_AWC | Varies3 | 19.0% |
Within the Upper Saline watershed, the model generally performed better with global averaged parameters than with both default and calibrated parameters at the various stations, based on
Table 4. For example, global average-based NS for the Hurricane station was 0.73 compared to 0.5 obtained through calibration in the period October 2000–September 2004. Smaller differences in NS values were, however, also observed during the same time period, for example with the Benton station where the NS value obtained through global averaging was 0.56 compared to 0.53 obtained through calibration.
For the Cadron Creek watershed, the NS values obtained using global averaged parameters were the same for the periods October 2000–September 2004, and October 1998–September 2004 (0.55). Model performance in this watershed for the period October 1998–September 2000 was not as good as that obtained using calibrated parameters (0.49 vs. 0.75). However, model performance was better with global values in the period October 2000–September 2004 and over the entire period.
Table 4.
Global average and regression-based parameters monthly performance in comparison to performance based on calibrated parameters.
Table 4.
Global average and regression-based parameters monthly performance in comparison to performance based on calibrated parameters.
| | | Nash-Sutcliffe Coefficientŧ |
| | | C1 | C2 | C3 | C4 |
Time period | Watershed | Station | Default | Calibration | Global average | Regression |
Oct. 1998–Sept. 2000 | Upper Saline | Benton | -- | -- | -- | -- |
| | Sheridan | -- | -- | -- | -- |
| | Hurricane | −1.96 | −0.44 | −0.79 | −0.95 |
| | | | | | |
| Illinois River | Siloam Springs | 0.58 | 0.78 | 0.81 | 0.61 |
| | Savoy | 0.61 | 0.64 | 0.69 | 0.55 |
| | Elm Springs | −0.20 | 0.90 | 0.81 | 0.82 |
| | | | | | |
| Cadron Creek | Guy | −0.39 | 0.75 | 0.49 | 0.63 |
| | | | | | |
Oct. 2000–Sept. 2004 | Upper Saline | Benton | 0.18 | 0.53 | 0.56 | 0.54 |
| | Sheridan | 0.41 | 0.45 | 0.69 | 0.61 |
| | Hurricane | 0.53 | 0.50 | 0.73 | 0.66 |
| | | | | | |
| Illinois River | Siloam Springs | 0.31 | 0.77 | 0.66 | 0.75 |
| | Savoy | 0.51 | 0.64 | 0.61 | 0.60 |
| | Elm Springs | 0.37 | 0.77 | 0.49 | 0.83 |
| | | | | | |
| Cadron Creek | Guy | 0.28 | 0.45 | 0.55 | 0.53 |
| | | | | | |
Oct. 1998–Sept. 2004 | Upper Saline | Benton | -- | -- | -- | -- |
(whole period) | | Sheridan | -- | -- | -- | -- |
| | Hurricane | 0.42 | 0.45 | 0.67 | 0.60 |
| | | | | | |
| Illinois River | Siloam Springs | 0.46 | 0.79 | 0.74 | 0.71 |
| | Savoy | 0.55 | 0.64 | 0.65 | 0.58 |
| | Elm Springs | 0.13 | 0.85 | 0.68 | 0.83 |
| | | | | | |
| Cadron Creek | Guy | 0.22 | 0.49 | 0.55 | 0.55 |
Figure 2 shows stream flow hydrographs for selected stations within the test watersheds for the period October 2000–September 2004. For the Illinois River Watershed, the Siloam Springs station was selected as it was located at the watershed outlet. For the Upper Saline Watershed, the Hurricane station was selected as it had the longest period of record. The station at Guy (Cadron Creek Watershed) was used as it was the only station for that watershed. Simulated stream flow obtained using global average parameters in the Illinois River watershed was similar to that obtained through calibration. In both cases (calibrated, global average), simulated stream flow matched the observed values and trends relatively well, based on
Figure 2, with deviations mostly being observed during the dry periods. For the Upper Saline watershed, stream flow obtained using global average parameters matched the observed flow slightly better during both peak and low flow periods than did values obtained using calibrated parameters. For the Cadron Creek watershed, the hydrograph obtained using global parameters is very similar to that obtained from the calibrated model, based on
Figure 2. In both cases, however, simulated peak flows did not match the observed peak flows very well.
Figure 2.
A comparison between global average-based and calibration-based stream flow hydrographs for selected stations within the test watersheds for the period October 2000–September 2004.
Figure 2.
A comparison between global average-based and calibration-based stream flow hydrographs for selected stations within the test watersheds for the period October 2000–September 2004.
3.2. Regression-based Evaluation
Figure 3 shows regression plots, equations and statistics for model parameters for which significant relationships were obtained. Equations obtained were best for ground water delay (GW_Delay) and soil available water capacity (SOL_AWC), for which R
2 values obtained were 0.98 and 0.99 (p = 0.04 and 0.0008), respectively. Regressions were worst for the soil evaporation compensation factor (ESCO) for which R
2 was 0.33 (p = 0.17). For curve number, two possible equations were obtained, one involving the actual curve number value (CN2) and one involving the percentage change in value (CN%). As independent variables were significant in both cases (p = 0.01 and 0.08), and CN2 was highly correlated with CN% (r = 0.857; p = 0.014), the equation involving CN% was used. Values obtained were applied to default HRU values to obtain regression-based curve numbers for each HRU.
A percent change in SLSUBBSN and SOL_AWC values was determined by considering area weighted averages of these parameter values. These weighted averages were used in the regression analyses; values obtained from regression were then compared to area-weighted default values and the percentage change of the parameter from the default value computed. The percent changes in parameter values obtained were then used to calculate regression-based values for the HRUs.
Table 5 shows model parameters as obtained using regression. Default values are presented for reference purposes. Where no suitable equation was found, the corresponding global average value was used. These values were input into the SWAT model for the respective watersheds. Model runs were then performed and performance measures computed as previously described. For Cadron Creek watershed, the regression-based base flow recession factor obtained was 1.05, which exceeded the upper limit (1) for that parameter. This value was reset to one (1) so that parameter bounds were not exceeded. Values obtained for GW_DELAY were close to those obtained through calibration for both the Illinois and Upper Saline watersheds (
Table 1). As with the global averages, most of the other regression-based values differed from those obtained through calibration.
Performance statistics for each watershed as obtained using regression-based parameters are also shown in
Table 4 (column C4). In general, the model performed well when regression-based parameters were used with NS values ranging between 0.53 and 0.83. Generally, model performance obtained using regression-based parameters was comparable to that obtained through calibration, based on
Table 4. Model performance sometimes exceeded that obtained using calibrated parameters, for example with the Upper Saline watersheds for the period October 2000–September 2004. The model with regression-based parameters also consistently outperformed the default model outputs in all the periods evaluated, based on
Table 4.
Figure 3.
Regression plots, equations and statistics for model parameters for which significant relationships were obtained. Dotted lines represent 95% confidence intervals. R2 denotes the coefficient of determination, 0 ≤ R2 ≤ 1. P denotes the probability of obtaining a value of the test statistic (in this case F) that is greater than that observed.
Figure 3.
Regression plots, equations and statistics for model parameters for which significant relationships were obtained. Dotted lines represent 95% confidence intervals. R2 denotes the coefficient of determination, 0 ≤ R2 ≤ 1. P denotes the probability of obtaining a value of the test statistic (in this case F) that is greater than that observed.
Table 5.
Model parameter values obtained using regression.
Table 5.
Model parameter values obtained using regression.
| | | Regression-based values |
Parameter description | Parameter | Default | Illinois River | Upper Saline | Cadron Creek |
Base flow recession factor, days | ALPHA_BF | 0.048 | 0.610 | 0.640 | 1.000‡ |
Ground water delay, days | GW_DELAY | 31 | 84 | 71 | 72 |
Ground water revaporationh coefficient | GW_REVAP | 0.020 | 0.056* | 0.056* | 0.056* |
Threshold depth for ground water flow to occur, mm | GWQMN | 0 | 14* | 14* | 14* |
Deep aquifer recharge fraction | rchrg_dp | 0.050 | 0.272* | 0.272* | 0.272* |
Threshold depth for revaporation to occur, mm | REVAPMN | 1 | 15* | 15* | 15* |
Snow fall temperature, ºC | sftmp | 1.0 | 1.1* | 1.1* | 1.1* |
Surface runoff lag coefficient, days | surlag | 4.00 | 3.64* | 3.64* | 3.64* |
Plant uptake compensation factor | EPCO | 1.000 | 0.830* | 0.830* | 0.830* |
Soil evaporation compensation factor | ESCO | 0.950 | 0.530 | 0.790 | 0.570 |
Average slope length, m | SLSUBBSN | Varies1 | −16.0% | −10.0% | 0.0% |
Curve Number (AMChh II) | CN2 | Varies2 | −24.2% | −0.7% | −24.0% |
Effective channel hydraulic conductivity, mm/hr | CH_K2 | 0.5 | 44.0* | 44.0* | 44.0* |
Manning’s n | ch_n | 0.014 | 0.3*‡ | 0.3*‡ | 0.3*‡ |
Soil albedo | sol_alb | 0.100 | 0.370* | 0.370* | 0.370* |
Available water capacity, m/m | SOL_AWC | Varies3 | −33.0% | −20.0% | 50.0% |
Figure 4 shows graphical plots for selected stations within the watershed. Simulated stream flow obtained using regression-based parameters in the Illinois watershed compared well to that obtained through calibration, based on
Figure 4. In both cases, calibrated stream flow matched the observed values and trends relatively well. For the Upper Saline watershed, stream flow obtained using regression-based parameters was comparable to that obtained through calibration. In both cases, the model tended to overestimate flow during base flow conditions. Simulations obtained using regression-based parameters matched the observed peak flows somewhat better than did values obtained using calibrated parameters. This would explain why better performance statistics were obtained using regression-based parameters than what was obtained during calibration. For the Cadron Creek watershed, the hydrograph obtained through calibration was better than that obtained using regression-based parameters, particularly during low flow periods. However, where deviations from observed values were large, these were generally more pronounced when calibrated parameters were used than when regression-based parameters were used, thus the lower NS values obtained for calibrations than for regression-based simulations.
Figure 4.
A comparison between regression-based and calibration-based stream flow hydrographs for selected stations within the test watersheds for the period October 2000– September 2004.
Figure 4.
A comparison between regression-based and calibration-based stream flow hydrographs for selected stations within the test watersheds for the period October 2000– September 2004.
3.3. Performance Analyses of Parameter Regionalization Methods
In general, the best (highest) NS coefficients and (lowest) Dv values were obtained using calibration parameters (
Table 4 and
Table 6). With the exception of Upper Saline at Hurricane (October 1998– September 2000) and Upper Saline at Benton (annual) model performance obtained using regression-based parameters ranged between acceptable and good, based on published criteria [
41] (as cited in [
42]),[
43]. Using regression-based parameters, 29% of the NS values obtained were greater than 0.75 (indicating good model performance) compared with 42% obtained through calibration and 8% obtained through global averaged parameters. In 10 out of 24 cases, NS and Dv values obtained through regression-based parameters were better than those obtained using calibrated parameters. This was especially true when performance was considered on an annual basis (
Table 6). For example, the NS and Dv values obtained at the Elm Springs gauge in the Illinois River watershed using regression-based parameters were 0.94 and 0.10, respectively, while corresponding values obtained through calibration were 0.79 and 0.16 respectively. This suggests that regression-based parameters could provide suitable alternatives where calibrations were not possible due to paucity of data.
Table 6.
Annual NS and Dv values for the period October 1998 to September 2004.
Table 6.
Annual NS and Dv values for the period October 1998 to September 2004.
| | Nash-Sutcliffe Coefficientŧ / Dv* |
Watershed | Station | Calibration | Global Average | Regression |
Upper Saline | Benton† | 0.16/0.31 | −0.51/0.43 | −1.07/0.51 |
| Sheridan | 0.87/0.09 | 0.61/0.28 | 0.44/0.34 |
| Hurricane | 0.50/0.30 | 0.38/0.41 | 0.40/0.50 |
| | | | |
Illinois River | Siloam Springs | 0.78/0.16 | 0.55/0.24 | 0.92/0.09 |
| Savoy | 0.68/0.1 | 0.48/0.18 | 0.82/0.04 |
| Elm Springs | 0.79/0.16 | 0.55/0.25 | 0.94/0.10 |
| | | | |
Cadron Creek | Guy | 0.92/−0.02 | 0.69/0.21 | 0.89/0.06 |
In general, NS values obtained using global averaged parameters were mostly in the acceptable range. Values of NS obtained using global parameters exceeded those obtained using both calibration and regression-based parameters 9 and 12 out of 24 times respectively. However, NS values obtained using global parameters exceeded the acceptable range 2 out of 24 times, compared to 10 and 7 out of 24 times for calibrated and regression-based parameters respectively. For example, the NS value obtained at the Siloam Springs gauge during October 1998 to September 2000 was 0.81 when global parameters were used, compared to 0.78 and 0.61 obtained using calibrated and regression based parameters for the same time period, respectively. Values not meeting either criterion (acceptable or good) were comparable across the methods. At the Hurricane gauge, for example, monthly NS values were less than zero for all methods; annual NS values obtained at the Benton gauge were also negative for both global average and regression-based methods, while calibrated NS values showed unsatisfactory (NS < 0.4) model performance. In all cases NS values obtained using global averaged parameters were better than those obtained using the default model. This suggests that global averaged parameters could also be used to provide alternative parameters where calibrations cannot be conducted. The global averaged parameters have the potential to provide better model performance than that obtained with the default model, making it a method worth considering.
Figure 5 shows a comparison of cumulative plots as obtained using the various regionalization methods for the period October 2000–September 2004. A shorter period was used for these plots so as to ensure the cumulative plots were legible. The particular period was selected because it was the period for which data availability was consistent among stations. For the Illinois River watershed results obtained using both regression-based and global averaged parameters were comparable to those obtained using calibration parameters. However, stream flow obtained using global parameters was higher than that obtained through calibration, while stream flow obtained using regression-based parameters was lower. In all cases (calibration, global average, regression-based) the model overestimated stream flow, when compared with observed data, with global average parameters having the highest level of overestimation. This is attributable to high curve numbers associated with global parameters, as global averaging called for a 2.8% increase in curve numbers.
For the Upper Saline watershed, regression-based and global-averaged parameters resulted in approximately similar patterns. These also closely matched the trends in the observed data. With regard to flow volumes, however, values obtained using calibration matched observed values most closely; the model tended to overestimate stream flow when either global averaged or regression-based parameters were used, this being particularly so during dry periods (
Figure 2 and
Figure 4). For the Cadron Creek watershed, plots obtained using regression-based parameters closely matched those obtained using calibration parameters; both plots closely matched observed data. The plot obtained using global averaged parameters showed deviation from all the other plots. This was because the model tended to overestimate low flows when global parameters were used, yet observed underestimation of peak flows was not sufficient to compensate for this overestimation (
Figure 2).
Figure 6 shows scatter plots for annual stream flows for the period October 1998 to September 2004 (top) and a comparison of average annual stream flow values (bottom) as obtained using the various regionalization methods. For the Cadron Creek watershed, simulated stream flow obtained using regression-based parameters matched observed data relatively well, while that obtained using global average parameters overestimated the stream flow. For the Illinois River watershed, regression-based stream flow matched observed stream flow well for low flows but deviated during high flow conditions. The model, however, overestimated annual stream flows in this watershed when global average parameters were used. Plots obtained for the Upper Saline watershed showed consistent over estimation by all regionalization methods. Overall, results obtained for annual performance were consistent with those obtained from assessments at the monthly time-step.
Figure 5.
Inter-comparison of stream flow hydrographs as obtained using the various regionalization methods, and calibration for the period October 2000–September 2004.
Figure 5.
Inter-comparison of stream flow hydrographs as obtained using the various regionalization methods, and calibration for the period October 2000–September 2004.
Figure 6.
Scatter plots for annual stream flow volumes for the period October 1998 to September 2004 (top) and comparison of average annual volumes (bottom) as obtained using the various regionalization methods.
Figure 6.
Scatter plots for annual stream flow volumes for the period October 1998 to September 2004 (top) and comparison of average annual volumes (bottom) as obtained using the various regionalization methods.