Evaluating the Effects of Parameter Uncertainty on River Water Quality Predictions

Abstract

Due to the high uncertainty of model predictions, it is often challenging to draw definitive conclusions when evaluating river water quality in the context of management options. The major aim of this study is to present a statistical evaluation of the Hydrologic Simulation Program FORTRAN (HSPF), which is a water quality modeling system, and how this modeling system can be used as a valuable tool to enhance monitoring planning and reduce uncertainty in water quality predictions. The authors’ findings regarding the sensitivity analysis of the HSPF model in relation to water quality predictions are presented. The application of the computer model was focused on the Ave River watershed in Portugal. Calibration of the hydrology was performed at two stations over five years, starting from January 1990 and ending in December 1994. Following the calibration, the hydrology model was then validated for another five-year period, from January 1995 to December 1999. A comprehensive evaluation framework is proposed, which includes a two-step statistical evaluation based on commonly used hydrology criteria for model calibration and validation. To thoroughly assess model uncertainty and parameter sensitivity, a Monte Carlo method uncertainty evaluation approach is integrated, along with multi-parametric sensitivity analyses. The Monte Carlo simulation considers the probability distributions of fourteen HSPF water quality parameters, which are used as input factors. The parameters that had the greatest impact on the simulated in-stream fecal coliform concentrations were those that represented the first-order decay rate and the surface runoff mechanism, which effectively removed 90 percent of the fecal coliform from the pervious land surface. These parameters had a more significant influence compared to the accumulation and maximum storage rates. When it comes to the oxygen governing process, the parameters that showed the highest sensitivity were benthal oxygen demand and nitrification/denitrification rate. The insights that can be derived from this study play a critical role in the development of robust water management strategies, and their significance lies in their potential to contribute to the advancement of predictive models in the field of water resources.

1. Introduction

The difficulty in accurately representing water quantity and quality compared to a proper environment leads to expected uncertainty in water simulation models [1,2]. Despite the extensive knowledge obtained from laboratory experiments, the extrapolation of the water process has proven to be challenging. When selecting a model, it is crucial to give great attention to assigning parameters that have appropriate and physically meaningful values. These values should accurately describe the watershed and its water processes within the environment [3,4,5]. Due to the difference in scale between the modeling environment and the real world, many model parameters become challenging or even hard to calculate, and therefore, they often have to be estimated or evaluated using secondary information sources. As a result, these parameters are typically associated with significant levels of uncertainty [6]. The evaluation of a model’s performance in relation to observed water quality data is hindered by data scarcity, accuracy, and frequency. This is primarily because collecting and analyzing data can be costly and time-consuming, making it difficult to collect a sufficient amount of data for a comprehensive evaluation of the model’s performance. Water quality data are highly susceptible to noise and bias due to the various procedures involved in sampling, handling, and measurement [7]. It is worth mentioning that these data are often sourced from sampling programs that have fixed frequencies and locations [8]. Investigating the influence of changed boundary conditions on the aquatic system [9] often involves the use of water quality models. These models are beneficial for studying sources of pollution, especially non-point sources, which can be difficult to measure due to their inherent limitations. One of the primary reasons for model uncertainty is the lack of high-quality data to support and validate the model’s performance. It is commonly recognized that sensitivity and uncertainty analysis of model parameters are crucial steps in the development and evaluation of models [10,11]. In recent years, there has been a widespread recognition that hydrological models containing numerous parameters are to yield predictions that are equally valid across different parameters [12], rendering the search for a single “best” parameter set within the parameter space fruitless [13]. Environmental modeling has benefited from the application of Monte Carlo-based methodologies of uncertainty and sensitivity analysis, such as the ones used by researchers [14,15,16]. These methodologies have found numerous applications, including the modeling of surface water quality. By conducting uncertainty analysis in environmental modeling, valuable insights can be gained regarding the accuracy of the results, and a deeper understanding of the sources of error involved in the modeling process can be achieved [17,18]. Decision makers are able to evaluate the risk associated with using model results for decision making by assessing the range of uncertainty in model predictions [19,20]. This is important because the model outputs that predict future conditions are inherently uncertain. The purpose of an uncertainty analysis is to assess the full range of potential model outcomes and their corresponding probabilities, whereas a sensitivity analysis focuses on measuring the impact of varying model inputs on the resulting outputs. The quality of water in river systems plays a crucial role in maintaining environmental health and managing resources, as it has significant impacts on both ecological and human communities. The accurate prediction of water quality parameters plays a crucial role in the successful management and mitigation of pollution in river basins. Despite the advancements in hydrological models, inherent uncertainty poses a substantial obstacle when it comes to making accurate and dependable predictions. The Hydrologic Simulation Program FORTRAN (HSPF – v12.5) is a widely utilized software for simulating water quantity and quality in watersheds; however, its predictions are highly sensitive to both input parameters and calibration processes. This study aims to address the issue of parameter uncertainty in the HSPF model by employing a comprehensive statistical evaluation framework.

Effective water resource management, environmental protection, and policy-making all heavily rely on the accurate prediction of river water quality, making it a paramount aspect to consider. The primary aim of this study is to acquire a comprehensive understanding of the effects of parameter uncertainty on the predictions of river water quality. By evaluating these uncertainties, we aim to enhance the reliability and robustness of water quality models, providing a foundation for more informed decision-making.

The sense-making apparatus of this study comprises multiple critical factors that are encompassed within its theoretical framework. Regarding the hydrological process, the model is calibrated to offer a comprehensive understanding of how water moves through and undergoes transformations within the watershed, which is of utmost importance when it comes to precise modeling. And regarding the interaction of parameters, water quality outcomes are influenced by various parameters, such as flow rates, temperature, and land use, which interact with each other complexly. When it comes to dealing with model uncertainty, one effective approach is to use probabilistic methods, which not only help in quantifying uncertainty but also enable the estimation of confidence intervals around model predictions. For sensitivity analysis, identification of key parameters will determine which parameters have the most significant impact on model outputs, thus guiding efforts to reduce uncertainty and providing insight on focusing resources on improving the data and understanding the most influential parameters.

This paper presents a study conducted in the Ave River watershed in Portugal, focusing on watershed water quality modeling. This study aims to identify the parameters in the selected model that require accurate characterization for effective application of the water quality model. The hydrological model used was the Hydrologic Simulation Program—FORTRAN (HSPF). Based on the original Stanford Watershed Model IV, the HSPF model is an integration of three previously developed models: the Agricultural Runoff Management Model [21,22], the Nonpoint Source Runoff Model (NPS) [23], and the Hydrological Simulation Program (HSP) including HSP Quality [24,25]. The purpose of this consolidation was to create a comprehensive model for hydrological simulation and management. HSPF is a simulation model that is semi-distributed in nature, meaning it replicates the movement of water and contaminants through physically homogenous areas within a watershed known as Hydrologic Response Units (HRUs). It is presumed that Hydrologic Response Units (HRUs) will exhibit similar hydrological responses when exposed to specific meteorological inputs, such as precipitation, potential evapotranspiration, and temperature. HSPF has the capability to continuously simulate hydrological, hydraulic, and water quality processes across pervious and impervious land surfaces, soil profiles, and streams, as well as well-mixed impoundments [26,27,28,29]. This research seeks to improve the reliability of water quality predictions and provide a valuable tool for water resource management.

2. Materials and Methods

2.1. Study Area and Analysis Procedure

The Ave River watershed, which is in the northern region of Portugal, covers an area of approximately 1388 km².

Spanning across a distance of 90.9 km, the river is adorned with two significant tributaries: the Este River, with an area of 247 km², and the Vizela River, covering an area of 342 km². The region witnesses an average annual precipitation of 1522 mm, with an average temperature of 13.9 °C and an average annual flow rate of 30.6 m³ s⁻¹. The basin’s land use occupation comprises various categories, including 46.6% forest land, 42.6% agricultural land, 10.7% urban land, and 0.2% wetland. The data on meteorological and water quality for the Ave River watershed from the years 1990–2000 were obtained from SNIRH, which stands for Sistema Nacional de Informação de Recursos Hídricos (National Information System for Water Resources). To gather the data, we downloaded a Digital Elevation Model (DEM, 30 m) from the SRTM project and obtained the Land Use and Soil Data from the Agência Portuguesa do Ambiente-Atlas do Ambiente, which falls under the Ministry of Environment, Spatial Planning, and Energy. Land use categories were aggregated into 6 major categories (forest land, agricultural land, urban land, wetland, shrub land, and pastures).

There are a total of five meteorological stations that make up the measurement network. These stations are strategically distributed throughout the catchment area. There are two hydrometric gauges, namely 15E03 (Ave River) and 15E01 (Este River), both of which are situated 5.3 km upstream from the river mouth. Industries and wastewater treatment plants contribute multiple point discharges to the river basin. The delineation of the watershed was performed in order to characterize the specific stations where observed data were available. These stations include station 15E03 Ave River and station 15E01 Este River. The watershed was segmented based on the meteorological stations, as shown in Figure 1.

In order to run the model effectively, a comprehensive dataset was needed. This dataset encompassed meteorological data, fecal coliform measurements, and nutrient loadings originating from various sources as cattle, hogs, wildlife, and industry. These data were collected over time to capture the dynamic nature of the system. Before assessing the uncertainty and sensitivity of the model, we conducted calibration and validation of flow and water quality parameters. The water quantity and quality of the Ave River were calibrated for the period of 1990–1994 and then validated for the period of 1995–2000. Similarly, the Este River was calibrated for the period of 1994–1997 and validated for the years 1998–2000, using a complete series of observed data whenever available. The continuous and high-quality data provided by these periods are of importance for the robust calibration and validation of models. The lack of temporal consistency and incomplete data in the more recent records for the Ave River, caused by network maintenance and station removal, present a significant challenge in achieving reliable calibration and validation, since it was hard to collect 10-year of homogeneous and consistent data.

In order to evaluate the quality of the data for each station, an assessment was conducted on several parameters including water temperature, dissolved oxygen (DO), biochemical oxygen demand (BOD₅), nitrate (NO₃), orthophosphate (PO₄), and fecal coliforms (FC). The Supplementary Data Material contains the calibration and validation plots for all constituents, which can be referred to as Figures S1–S6.

2.2. Statistical Criteria Evaluation

Several statistical measures were utilized to quantify the performance of the model in both the calibration and validation simulations. The authors of [30] provide a comprehensive overview of various techniques for assessing quantitative performance. These techniques comprise direct model comparison, the concurrent examination of real and modeled values, the consideration of key residual criteria, the utilization of residual methods with data transformations, and the evaluation of performance through correlation and model efficiency measures. A brief description of the statistical criteria used follows.

The deviation of runoff volumes D_v, also known as the percentage bias, is the simplest goodness-of-fit criterion [31]. Its value is calculated using the following equation:

$$D_v\left[\%\right]={\displaystyle \frac{\sum _i^n\left(O_i-S_i\right)}{\sum _i^n{O}_i}}\times 100$$

(1)

where,

O_i = observed monthly values for the i-th month;

S_i = simulated monthly values for the i-th month and;

n = total number of months.

According to the model simulation performance rating, the deviation of volumes [4] is classified as very good for values that are below 15%. For values between 15% and 25%, the rating is good. Lastly, values between 25% and 35% are deemed satisfactory.

The Coefficient of Determination, also known as R², is a measure that quantifies the proportion of variance between two variables that can be explained by a linear regression model. It can vary between 0 and 1. The higher the value, the better the fit.

$$R^2={\left[{\displaystyle \frac{\sum _{i=1}^n\left[\left(O_i-\overline{O}\right)\left(S_i-\overline{S}\right)\right]}{{\left[\sum _{i=1}^n{\left(O_i-\overline{O}\right)}^2\right]}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}{\left[\sum _{i=1}^n{\left(S_i-\overline{S}\right)}^2\right]}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}}}\right]}^2$$

(2)

where,

Ō = mean of the observed monthly value;

$\overline{S}$ = mean of the simulated monthly value;

According to [32,33], the authors suggest that for monthly simulated constituents, a coefficient of determination above 0.6 is considered satisfactory for achieving model performance. However, it is also acceptable if the value is as low as 0.5.

The Nash–Sutcliffe coefficient, also referred to as the efficiency criterion (E), is a statistical measure of association. It quantifies the percentage of the observed variance that can be explained by the model. The Nash–Sutcliffe coefficient is estimated using Equation (3):

$$E=1-{\displaystyle \frac{{\sum }_{i=1}^n{\left(S_i-O_i\right)}^2}{{\sum }_{i=1}^n{\left(O_i-{\varnothing }_i\right)}^2}}$$

(3)

Let Ø_i denote the average measured discharge, while all the other variables keep their previously explained definitions. In Equation (3), the second term is used to denote the ratio between the mean square error (MSE) and the variance of the observed data. Therefore, if the value of E is equal to zero, it shows that the model output is not superior to the average observed stream flow throughout the entire analysis period. The accuracy of the model increases as the model efficiency approaches 1.

The Mean Square Error (MSE) is a metric that calculates the discrepancy between the estimated values produced by an estimator and the actual values of the quantity being estimated. A value of zero means that the estimator predicts observations of the parameter with perfect accuracy. It can be estimated by using Equation (4):

$$MSE={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left(S_i-O_i\right)$$

(4)

where n is the number of predictions.

The Root Mean Square Error (RMSE), sometimes referred to as the root-mean-square deviation, is a widely used metric in statistics that calculates the standard deviation of the discrepancies between the predicted values produced by a model and the observed values. The RMSE of a model prediction is defined as Equation (5):

$$RMSE=\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n\left(S_i-O_i\right)}$$

(5)

The Fourth Root Mean Quadruple Error (R4MS4E) is a metric that magnifies the impact of larger errors by utilizing the fourth power of the Root Mean Square Error (RMSE). It can be calculated using Equation (6):

$$R4MS4E=\sqrt[4]{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(S_i-O_i\right)}^4}$$

(6)

The Relative Volume Error is a metric that indicates the accuracy of a measurement in relation to the magnitude of the sample being measured. It can be estimated using Equation (7):

$$RVE={\displaystyle \frac{\frac{1}{n}\sum _{i=1}^n\left(S_i-O_i\right)}{\frac{1}{n}\sum _{i=1}^n\left(O_i\right)}}$$

(7)

The Standard Deviation Ratio (RSR) is a mathematical method that is like the RMSE method, but it considers the standard deviation of the observed values. It is characterized by an ideal value of zero, but it has the ability to vary between zero and infinite. It can be estimated using Equation (8):

$$RSR={\displaystyle \frac{\sqrt{\sum _{i=1}^n{\left(S_i-O_i\right)}^2}}{\sqrt{\sum _{i=1}^n{\left(O_i-\varnothing \right)}^2}}}$$

(8)

Equation (4) through Equation (8) values can vary between 0 and infinite, with an ideal value of zero for all criteria.

The Index of Agreement (IoAd) is similar to R², but it is more effective in handling variations in modeling and observed means and variances. With an ideal value of 1, it can vary from 0 to 1.

$$IoAd=1-{\displaystyle \frac{{\sum _{1=1}^n\left(S_i-O_i\right)}^2}{{\sum _{1=1}^n\left(\left|O_i-Ø\right|-\left|S_i-Ø\right|\right)}^2}}$$

(9)

2.3. Uncertainty and Sensitivity Analyses

Uncertainty analysis was conducted using a Monte Carlo (MC) approach [34]. The 10-year and 7-year periods (Ave and Este River, respectively) were used to perform multiple model simulation runs, where values for the selected model parameters were randomly chosen from assigned probability distributions.

Table 1. Range and distribution of uncertain of HSPF water quality parameters and their respective description [35,36,37,38].

Parameter	Description	Range	Distribution
FSTDEC	First order decay rate of bacteria (day−1)	(0.1, 5)	Uniform
WSQOP	Rate of surface runoff that will remove 90% of stored fecal coliform from pervious land use	(0.5, 1.0)	Uniform
WSQOP *	Rate of surface runoff that will remove 90% of stored quality constituent from pervious land use	(0.5, 2.4)	Uniform
KBOD20	BOD5 decay rate at 20 °C (h−1)	(0.0001, 1)	Uniform
KODSET	Rate of BOD5 settling (ft h−1)	(0, 1)	Uniform
BENOD	Benthal oxygen demand at 20 °C (mg m−2 h−1)	(0, 500)	Uniform
REAK	Empirical constant to calculate the reaeration coefficient (h−1)	(0.01, 2)	Uniform
KTAM20	Nitrification rate of ammonia at 20 °C (h−1)	(0.001, 1)	Uniform
KNO220	Nitrification rate of nitrites at 20 °C (h−1)	(0.001, 1)	Uniform
KNO320	Nitrate denitrification rate at 20 °C (h−1)	(0.001, 1)	Uniform
ACQOP **	Accumulation of fecal coliform on pervious land per day (CFU day−1)	(2 × 108, 2 × 1010, 2 × 1012)	Triangular
SQOLIM Factor	Factor, which is multiplied by ACQOP to obtain maximum accumulation of fecal coliform on pervious land	(2, 10)	Triangular
ACQOPNO3 **	Accumulation of nitrates on pervious land per day (lb ac−1 day−1)	(0.05, 30)	Triangular
ACQOPPO4 **	Accumulation of nitrates on pervious land per day (lb ac−1day−1)	(0.001, 1)	Triangular

* Constituents: nitrates, orthophosphates, and biochemical oxygen demand; ** for the months of January.

summarizes the water quality parameters that were uncertain in terms of knowledge, including their respective distributions [35,36,37,38]. In the context of this statement, the term “range” refers to the lower and upper limits for the distributions. These limits are specifically chosen to align with the typical minimum and maximum values for these parameters, as outlined in the HSPF user’s manual. Due to the possibility of certain parameter values varying up to infinity, the calibrated model observed the maximum value used for the distributions. In cases where the manual does not provide any additional information, parameters are assigned a uniform distribution. However, for accumulation and storage parameters, a triangular distribution is used, with the most probable value being the mode of the calibrated model’s parameter value. In situations that aim to capture the true variability of real-world scenarios, uniform distributions may not always provide the best representation. The reason for this is that they cannot consider the probabilistic nature and natural fluctuations that are observed in chemical reactions and environmental processes. The reason behind selecting uniform distributions as a representation of uncertainty in parameter values was based on their ability to clearly indicate the lack of precise information about the parameters. When detailed knowledge is not available, this method serves as a prudent and conservative initial step by providing an equal chance for all values within the given range. The ACQOP and SQOLIM values for the remaining months were determined by taking the January distribution and multiplying it with a monthly change factor derived from the calibrated model. This result was analyzed in 14 parameters.

In order to achieve a satisfactory approximation of model output uncertainty, a total of 10,000 model runs were conducted for the MC simulations. The calculation of the model uncertainty involved deriving quantiles of time series, which were then weighted by the Nash–Sutcliffe coefficient (E), deviation of volumes (D_v), and the coefficient of determination (R²). This calculation was specifically performed for a 95% confidence interval and applied to all quality parameters described [34]. The data used for model comparison in this study were generated through the execution of MC simulations using an R script [39] that was specifically written for this purpose. The R script successfully populated all the parameter subroutine tables in the HSPF user control input (UCI) file. As a result, there are now 10,000 UCI files representing the various parameter sets derived from the distributions.

In order to determine the significance of each parameter in the model, a multi-parametric sensitivity analysis (MPSA) was conducted on the parameters listed in Table 1. The MPSA carefully followed the procedure that was proposed by [40,41], which involved the implementation of a generalized sensitivity analysis [42]. The evaluation of the statistical errors mentioned previously will determine whether the 10,000 parameter sets are categorized as “acceptable” or “unacceptable”. In the case that the two distributions do not exhibit statistical differences, the parameter will be classified as insensitive; however, if there are statistical differences, the parameter will be deemed sensitive.

The HSPF model offers several advantages when it comes to water quality monitoring and planning. It not only allows for the simulation of comprehensive watershed processes but also enables the analysis of different management scenarios, supports calibration and validation efforts, conducts uncertainty analysis, integrates various data sources, and provides valuable insights for regulatory and policy development. With its ability to offer detailed and reliable predictions, HSPF plays a crucial role in assisting stakeholders in making well-informed decisions aimed at the protection and enhancement of water resources.

3. Results and Discussion

3.1. Model Calibration and Validation

In order to calibrate and validate the model, a visual comparison and evaluation of the simulated and observed daily and monthly flow and in-stream constituent’s concentration at the Ave and Este River stations were conducted. Statistical criteria were used for this evaluation, as shown in

Table 2. Statistical criteria results for calibration and validation at Ave and Este River station.

Parameters (Ave River)	Dv	E	R2	MSE	RMSE	R4MS4E	RVE	IOAD	RSR
Calibration
Q (daily)	10	0.54	0.67	1.31	0.66	1.36	0.09	0.90	0.68
Q (monthly)	10	0.69	0.74	0.80	0.40	0.68	0.09	0.92	0.55
T	0	0.70	0.72	7.99	2.83	4.05	0.00	0.92	0.55
FC	–13	0.71	0.72	8.0 × 107	8966	13327	–0.16	0.90	0.54
DO	–10	0.38	0.53	3.47	1.86	2.61	–0.11	0.82	0.79
BOD5	7	0.63	0.64	6.76	2.60	3.22	0.06	0.89	0.61
NO3	–1	0.60	0.68	3.02	1.74	2.16	–0.01	0.90	0.63
PO4	9	0.72	0.75	0.11	0.32	0.42	0.08	0.93	0.53
Validation
Q (daily)	–9	0.72	0.75	1.28	0.64	1.16	–0.10	0.93	0.53
Q (monthly)	–9	0.87	0.90	0.62	0.31	0.41	–0.10	0.97	0.37
T	–3	0.69	0.73	7.08	2.66	3.78	–0.04	0.92	0.56
FC	–12	0.33	0.34	3.6 × 107	6031	9839	–0.13	0.73	0.82
DO	–1	0.56	0.58	1.92	1.39	1.76	–0.01	0.87	0.66
BOD5	8	0.28	0.21	5.79	2.41	2.80	0.08	0.69	0.96
NO3	–8	0.37	0.46	9.34	3.06	3.96	–0.09	0.81	0.79
PO4	–13	0.63	0.70	0.12	0.35	0.46	–0.15	0.91	0.61
Parameters (Este River)	Dv	E	R2	MSE	RMSE	R4MS4E	RVE	IOAD	RSR
Calibration
Q (daily)	4	0.60	0.64	0.38	0.19	0.45	0.04	0.89	0.63
Q (monthly)	4	0.92	0.93	0.11	0.06	0.09	0.04	0.98	0.28
T	2	0.50	0.77	7.50	2.74	3.54	0.02	0.91	0.71
FC	–13	0.43	0.46	9.6 × 107	9815	16489	–0.15	0.81	0.75
DO	–1	0.42	0.67	1.02	1.01	1.18	–0.01	0.88	0.76
BOD5	0	0.43	0.62	1.68	1.30	1.72	0.00	0.88	0.75
NO3	–2	0.42	0.61	69.51	8.34	10.78	–0.02	0.87	0.76
PO4	–2	0.55	0.56	0.04	0.19	0.29	–0.02	0.86	0.67
Validation
Q (daily)	8	0.58	0.64	0.28	0.14	0.34	0.07	0.89	0.65
Q (monthly)	8	0.70	0.81	0.01	0.01	0.01	0.08	0.94	0.54
T	0	0.41	0.68	8.70	2.95	3.57	0.00	0.89	0.77
FC	–31	0.64	0.77	4.5 × 107	6690	9671	–0.45	0.86	0.60
DO	2	0.49	0.56	1.02	1.01	1.31	0.02	0.86	0.72
BOD5	1	0.45	0.54	2.04	1.43	1.75	0.01	0.85	0.74
NO3	–11	0.27	0.66	67.28	8.20	8.68	–0.12	0.86	0.86
PO4	–13	0.63	0.70	0.12	0.35	0.46	–0.15	0.91	0.61

. The Supplementary Data Material includes calibration and validation plots for all constituents analyzed in this study, which are presented in Figures S1–S6. Using statistical criteria provided a framework for evaluating and estimating the model’s performance. The results obtained from analyzing the deviation of volume values for all constituents were positive, with the highest deviation observed in fecal coliforms at both stations. The use of Nash–Sutcliffe values proved to be highly effective in calibrating and validating the results. Notably, the validation process for nitrates and BOD₅ at both stations resulted in a satisfactory approach, with the error values (E) being less than 0.50. In terms of the coefficient of determination, it is important to note that certain constituents, specifically FC, BOD₅, and NO₃, produced values below 0.50, which are unsatisfactory. This observation was made during the validation process at the Ave River station. In order to mitigate the impact of deviation volume, which occurs when positive and negative errors offset each other, we calculated the criteria of MSE, RMSE, and R4MS4E. Both criteria show good results with values close to zero. While both RMSE and R4MS4E provide valuable insights for interpretation, they are useful because the results are presented in the same units as the model and show a comprehensive alignment between predicted and observed data. Moreover, it is important to note that the magnitude of the mean square (ranging from two to four) directly correlates with the level of emphasis placed on larger events.

Although our model results meet the generally accepted criteria for model calibration and validation, we also acknowledge the complexity of water quality modeling and the potential for different interpretations of what constitutes “good” performance. For water quality modeling, slightly lower thresholds might still be acceptable due to the complexity and inherent variability of these parameters [21,31,32].

The results of the study suggest that the HSPF model is suitable for accurately simulating river discharge and in-stream water quality in the Ave watershed.

3.2. Uncertainty Analysis

The uncertainty analysis was conducted separately for both the calibration and validation periods. The calibration period, which spanned over 10 years, was analyzed for the Ave River station, while the validation period, which spanned over 6 years, was analyzed for the Este River station. The threshold for the likelihood measure, which is utilized to categorize the model parameter sets as acceptable, was established by evaluating the model output for every quality parameter. Only those parameter sets that satisfied the conditions of generating positive E values, D_v values between −30% and 30%, and R² values surpassing 0.50 were considered for inclusion. The parameter sets that were acceptable showed E values that were above 0.40. Figure 2 displays the uncertainty band for the water quality constituents, with a 95% confidence interval specifically addressing the Ave River. Similarly, Figure 3 presents the uncertainty band for the same constituents, but this time focusing on the Este River.

This approach resulted in the number of acceptable parameters sets shown in

Table 3. Behavioral parameter sets from the uncertainty analysis and respective contingency.

	FC	NO3	PO4	BOD5	DO
Parameter Sets
Ave	2909	643	876	2610	6243
Este	1210	594	1012	413	631
Contingency
Ave	29%	36%	48%	61%	58%
Este	27%	48%	27%	26%	28%

. Due to the limited time period of analysis at the Este River station, there was a decrease in the number of acceptable parameter sets. Among the various parameter sets observed, those resulting from the uncertainty analysis of nitrates and orthophosphorus were found to be the lowest but still within acceptable limits.

Table 3 provides details on the percentage of occurrences where the observed data falls within the 95% confidence interval (contingency). The connection between the defined acceptable threshold and the confidence interval size can be observed through this observation, with the additional recognition that the size of the sample also plays a role in determining the confidence interval. When comparing the Ave River station to the Este River station, it can be observed that the former exhibits a higher percentage of data falling within the confidence interval. The reason for this can be attributed to the fact that there is a larger amount of observed data available, which in turn leads to a model that is more accurately calibrated. During the process of calibrating oxygen governing reactions, it is important to take into consideration all parameters that impact nutrients, biochemical oxygen demand, and dissolved oxygen, as the calibrated model will rely on these factors. However, it is important to note that even if this goal is achieved successfully, there are still other factors to consider. The reason for this is that the analysis of other processes that require oxygen must be performed concurrently. Out of all the parameter sets that were acceptable, only 319 for the Ave River station and 231 for the Este River station were found to result in concurrent behavioral outcomes for all oxygen-involving processes, as defined by the threshold. By adjusting the R² threshold to 0.40 and the D_v limits to −35% and 35%, a total of 3536 behavioral parameter sets were obtained for the Ave River station, while for the Este River station, there were 2936 sets. Notably, the lowest observed E value was recorded at 0.12.

Besides the steps taken in our approach, enhanced calibration and validation, which accurately represent all relevant processes and Monte Carlo simulation analyses, in order to reduce uncertainty for all constituents, a few aspects should be considered. The first consideration is improving model input data quality; although we have a good representation of hydrological years, 10 years of data for the Ave River and 7 years of data for the Este River, more recorded data would provide more robust results. Exploring advanced techniques, such as a combination of real-time data assimilation with machine learning applications [43,44,45] or neural networks [46] to enhance predictive capabilities. A second consideration is engaging with local stakeholders to maintain an iterative approach to model improvement. The analysis of uncertainty in HSPF showed the model is not capable of accurately representing low values of FC at both stations.

3.3. Sensitivity Analysis

The results of the sensitivity analysis conducted on the FC output showed that the model is sensitive to two factors: the first order decay rate (FSTDEC) and the rate of surface runoff necessary to remove 90% of the stored FC (WSQOP). In addition, the results show that the maximum storage rate (SQOLIM) is a more reliable indicator compared to the monthly accumulation storage (ACQOP) as shown in Figure 4. The sensitivity of each parameter is represented by the extent of separation between both lines. Additionally, Figure 4 demonstrates that, as the amount of observed data increases, there is a decrease in sensitivity, leading to a better fit for the model. It is worth noting that both sensitivity analyses, namely the Ave River and the Este River, have confirmed that the same parameters remain the most sensitive in the model. Although these parameters show very high sensitivity, it is important to note that the concentrations of fecal coliform in surface waters are closely linked to oxygen-related processes. The presence and behavior of fecal coliforms in water bodies can affect multiple oxygen-related processes, and concurrently, they can also be influenced by these processes in different ways (i.e., the presence of fecal coliforms and other bacteria in the process of decomposing organic matter leads to a rise in the biochemical oxygen demand (BOD₅), which has a direct impact on the levels of dissolved oxygen (DO) in the water.

The sensitivity analysis revealed that the dissolved oxygen concentration at both stations showed a higher sensitivity towards benthal oxygen demand (BENOD) and nitrate denitrification rate (KNO320), while nitrates displayed a greater sensitivity towards the rate of nitrogen storage removal (WSQOP), nitrification rates (KNO320 and KNO220), and the biochemical oxygen demand rate (KBOD20). The cumulative frequencies of all parameters investigated in the context of orthophosphorus were similar. It is interesting to note that the surface runoff rate of phosphorus (WSQOP) and benthal oxygen demand (BENOD) played a significant role in the calibration of orthophosphates at both stations. At Este River station, the sensitivity analysis revealed that almost all the parameters analyzed impacted the sensitivity of biochemical oxygen demand parameters. However, the decay rate (KBOD20) and settling rate (KODSET) were significant. In contrast, no parameter displayed a significant sensitivity at the Ave River station. It is crucial to consider all parameters when calibrating the concentration of biochemical oxygen demand. Figure 5 shows the frequency distributions of the most sensitive parameters addressed here.

To ensure accurate calibration of the model, the governing process of oxygen involves considering multiple parameters [47]. Although MPSA does not display a top sensitive parameter, there are certain parameters that can be deemed as highly significant. According to the findings, the decay rate of biochemical oxygen demand (KBOD20), along with the rates of nitrification (KNO320 and KNO220) and benthal oxygen demand (BENOD), are crucial parameters that contribute to an effective calibration of the model. Despite not causing significant output variation, it is important to note that the accumulation rates (ACQOP) and maximum storage (SQOLIM) of nutrients might have been influenced by the rate of nutrient removal (WSQOP), which mimics the wash off of nutrients from the land. In addition, it is crucial to consider different parameters when simulating algae, such as the rates at which phytoplankton grow and settle. Due to the lack of observed data at all the stations, conducting a calibration and categorizing it as either acceptable or unacceptable became challenging.

Table 4. Summary of the most sensitive parameters of HSPF.

Constituent	Parameter
Fecal coliforms	FSTDEC	WSQOP
Biochemical oxygen demand *	KBOD20	KODSET
Dissolved oxygen	BENOD	KNO320
Nitrates	WSQOP	KNO320	KNO220	KBOD20
Orthophosphorus	WSQOP	BENOD

* only observed at Este River.

summarizes the most sensitive parameters.

In order to summarize the study’s findings, its main aim was to assess the uncertainty of model predictions when evaluating the river water quality of the Ave River watershed in Portugal through the use of the HSPF model. By utilizing a comprehensive two-step statistical evaluation framework, which includes conducting Monte Carlo simulations and multi-parametric sensitivity analyses, we were able to determine the crucial parameters that have a significant impact on water quality predictions. The results of our study showed that the concentrations of fecal coliform were found to be most influenced by parameters associated with the rate of decay and the efficiency of removing surface runoff. Among the various parameters measured, including dissolved oxygen (DO), biochemical oxygen demand (BOD₅), nitrate (NO₃), and phosphate (PO₄), it was observed that the benthal oxygen demand and nitrification/denitrification rates exhibited the highest sensitivity.

3.4. Implications for Water Resources Management

The implications of the findings from this study are of considerable significance to the field of water resource management. With implementing a comprehensive evaluation framework, encompassing sensitivity analysis and Monte Carlo uncertainty assessment, the HSPF model can provide a deeper insight into the critical parameters that play a significant role in water quality predictions.

In addition to the statistical analyses results, there are other sources of uncertainty that can affect the results. These uncertainties can stem from various factors, both natural and anthropogenic. Take, for example, the utilization of Monte Carlo uncertainty analysis in watershed water quality modeling, which underscores the importance of deliberating on which model parameters should be regarded as random variables and devising strategies to handle non-random parameters [48,49,50]. In addition, it is important to note that the complexity of river water quality, along with the influence of multiple parameters on the output, can introduce considerable uncertainties. These uncertainties are commonly addressed through a combination of monitoring, modeling, and sensitivity analysis. It is crucial to acknowledge that relying solely on statistical analysis may not provide enough evidence to draw definitive conclusions. The real-world applicability of water quality models may have limitations due to the uncertainty in the model outputs. The integration of management objectives into the design of the uncertainty analysis is crucial, and this is precisely why it is so important. By adopting this approach, the analysis becomes management-oriented, thereby providing valuable support for decision-making in water resource management policies [51,52]. By employing the Monte Carlo methodology, the assessment of model uncertainty is significantly improved as it offers a comprehensive probabilistic framework capable of quantifying both the variability and uncertainty associated with model inputs and parameters. Including sensitivity analysis, scenario evaluation, robust calibration, validation, and effective communication of uncertainty within this tool significantly enhances the process of making informed and reliable decisions in water quality management and planning.

This study provides valuable insights into the key processes that should be given priority in water quality management plans by identifying the parameters that have the greatest impact on fecal coliform concentrations and oxygen dynamics. By effectively managing these parameters, it is possible to achieve more accurate predictions and make better-informed decisions, which play a crucial role in safeguarding water resources [53]. The methodologies and results that have been presented in this study have the potential to apply to various other watersheds, increasing the overall generalizability and usefulness of this approach across different environmental contexts. Developing effective water quality management strategies depends on obtaining detailed insights, which are crucial when dealing with fluctuating environmental conditions and human-induced pressures. Water resource managers can make significant improvements in the sustainability and health of riverine ecosystems by utilizing these findings, as they can better predict and effectively address water quality issues. The importance of continuous research and improvement of hydrological models is highlighted by this study, as it addresses the ever-changing difficulties in managing water resources.

The effective management of water resources relies on the crucial role played by water quality modeling. These models have the capability to predict the impact of different pollutants, provide insights into the health of aquatic ecosystems, and assist in the development of strategies for sustainable water use. It is important to note that the accuracy of these models can be influenced by the uncertainty surrounding their parameters. By conducting parameter uncertainty analysis, it is possible to identify critical parameters and potential risks, which can assist in: (1) the design of pollution control measures [54], (2) the assessment of water quality helping to evaluate various risks, including the probability of surpassing regulatory standards and the potential consequences of extreme events [55], (3) informed choices by decision-makers considering a range of potential outcomes, ensuring that the management strategies they choose are able to withstand uncertainty and remain effective, (4) the ability to prioritize areas for data collection and research, leading to a reduction in uncertainty and an improvement in the reliability of the model, (5) emphasizing the various levels of uncertainty and showcasing the potential impacts of different management actions, this information plays a crucial role in shaping policies and regulations.

4. Conclusions

Due to the lack of data, uncertainty in water quality modeling is high. The complexity of the model approach, characterized by high parameterization, as well as the established likelihood, significantly influence the propagation of uncertainties, particularly when considering the limited quality data and the associated uncertainty. When there is a lack of good quality data available for calibration or when the data is scarce, the process of validating the model becomes particularly challenging. Despite the challenges, water quality models play a crucial role in helping managers effectively model and understand complex systems. Therefore, it is important to determine which parameters result in increased uncertainty. The identification of critical parameters that significantly affect water quality predictions enables management to prioritize their actions by focusing on monitoring and controlling these key drivers, specifically in low-data environments. This strategic approach aims to enhance data collection and accuracy specifically for the most influential parameters, leading to the creation of more reliable models. By conducting an uncertainty and sensitivity analysis, valuable information can be obtained that can improve budget allocation for target assessments and lead to the availability of additional resources for future management and monitoring. This, in turn, would result in more accurate quality data and ultimately reduce uncertainty in model results. By utilizing the results obtained from this study, it is possible to steer future efforts towards the collection of more information. This information would prove invaluable in better understanding and characterizing the parameters that are most sensitive and those that are most uncertain. Through the examination of parameter uncertainties and the effects they have on water quality predictions, the study has the potential to offer valuable insights that can inform the creation of more focused, streamlined, and impactful water quality management strategies. With these strategies, organizations, can achieve improved environmental outcomes, allocate resources more effectively, and build greater stakeholder confidence in their management practices. In summary, the intricate nature of modeling complex environmental systems is underscored by the uncertainty in parameterization within the hydrological simulation program FORTRAN. This analysis has revealed several noteworthy findings. Foremost among them is the criticality of conducting high sampling in order to ensure the representativeness of the collected samples, particularly for sites that have limited data availability. The difference between the Este River and Ave River stations serves as an example of how data scarcity can lead to parameter uncertainty and impact the acceptability of parameter sets. Additionally, it is worth mentioning that the model outputs are greatly affected by specific parameters, namely FSTDEC and WSQOP. This underscores the crucial need for a precise understanding and calibration of these parameters. In addition, the findings of this study highlight the importance of prioritizing kinetic governing equations in nutrient calibration, rather than solely focusing on nutrient accumulation and maximum nutrient storage. It is important to highlight the significance of implementing accurate calibration strategies that prioritize the dynamic elements of nutrient transport and transformation. Another point to consider is that, although the integrated model provides a comprehensive framework for understanding hydrological processes, its high parameterization presents difficulties in accurately describing the relevant parameters. This is clear in oxygen governing processes, where multiple parameters exhibit similar behaviors in modeling outputs, causing further refinement and validation. To effectively tackle these complexities, it is necessary to adopt a comprehensive approach that integrates advanced modeling techniques, ensures comprehensive data collection, and implements robust calibration strategies. This holistic approach is crucial for enhancing the accuracy and reliability of hydrological simulations. Sensitivity analysis is a useful tool that helps us understand the factors that contribute to these variations. Uncertainty analysis takes it a step further by providing a quantitative measure of potential outcomes within a range, providing valuable insights into the model’s behavior. By considering these analyses collectively, a well-rounded understanding of model reliability is acquired, enabling informed decision-making, even in situations where there is uncertainty. Researchers and practitioners can enhance the reliability and credibility of their predictions by integrating both uncertainty and sensitivity analyses into the hydrological modeling workflows. This can cause improved water resource management and a heightened sense of responsibility towards environmental conservation. It is crucial to emphasize that the results achieved in these investigations are subjective and strongly influenced by the specific attributes of the watersheds being examined. Nevertheless, these findings can be applied to other agricultural-driven watersheds.

Managing parameter uncertainty in water quality modeling is not only a challenge but also an invaluable tool that, when performed correctly, can enhance the robustness and knowledge of water resources management. Managers who have a thorough understanding and the ability to quantify uncertainty are able to make more informed decisions, effectively allocate resources, and develop resilient policies that can adapt to future changes and uncertainties.