Runoff Probability Prediction Model Based on Natural Gradient Boosting with Tree-Structured Parzen Estimator Optimization

Abstract

Accurate and reliable runoff prediction is critical for solving problems related to water resource planning and management. Deterministic runoff prediction methods cannot meet the needs of risk analysis and decision making. In this study, a runoff probability prediction model based on natural gradient boosting (NGboost) with tree-structured parzen estimator (TPE) optimization is proposed. The model obtains the probability distribution of the predicted runoff. The TPE algorithm was used for the hyperparameter optimization of the model to improve the prediction. The model was applied to the prediction of runoff on the monthly, weekly and daily scales at the Yichang and Pingshan stations in the upper Yangtze River. We also tested the prediction effectiveness of the models using exponential, normal and lognormal distributions for different flow characteristics and time scales. The results show that in terms of deterministic prediction, the proposed model improved in all indicators compared to the benchmark model. The root mean square error of the monthly runoff prediction was reduced by 9% on average and 7% on the daily scale. In probabilistic prediction, the proposed model can provide reliable probabilistic prediction on weekly and daily scales.

1. Introduction

Prevention of natural disasters [1,2] and the utilization of water resources [3,4,5] have been hot topics of research. As one of the most important forms of water resources, the prediction of runoff is a basis for research in the field of water resource research [6]. Especially in areas where precipitation data are scarce, it is of concern to obtain a reliable runoff prediction with few relevant data.

In order to obtain an accurate runoff prediction, many scholars have conducted extensive research. Current runoff forecasting methods can be divided into two main types. The first method starts with the physical mechanism of runoff formation and establishes hydrological models [7], such as the TANK model [8] and the Xinanjiang model [9]. This type of method has a clear physical meaning, but the model is complex and needs a large amount of relevant data to support it, which is difficult to apply in areas with few data. The second method analyzes historical data to predict runoff. With the dramatic increase in computer performance, a large number of machine learning methods have been introduced to runoff prediction to improve the accuracy and reliability of runoff prediction [10,11]. Among data-driven methods, a large number of neural network models [12] and decision tree algorithms [13] are applied to runoff prediction, in addition to earlier applied regression analysis methods [14]. In particular, extreme gradient boosting (XGboost) [15] as a representative gradient boosting tree method is widely used for data prediction in various disciplines and has performed well in data science competitions and in industry.

XGboost is a tree-based boosting algorithm that trains multiple weak classifiers and combines them linearly to form a strong classifier by changing the weight distribution of the training data. In addition, the method adds a regularization item to the loss function to prevent overfitting. When deriving the loss function, the method uses a second-order Taylor expansion, and the loss is calculated more accurately. This method was applied to runoff prediction and achieved a satisfactory result [16].

Until now, the predictions of all methods have been difficult to fully agree with the actual data. The results of deterministic prediction cannot be completely reliable, so the risks of prediction must be measured. In contrast to deterministic prediction, probabilistic prediction results can provide an interval of forecast values or a probability distribution of forecast values. There are various methods for probabilistic prediction of runoff. The Bayesian probabilistic hydrological forecasting method uses the results of deterministic prediction models to generate probabilistic predictions [17,18,19]. The error probability distribution method generates probabilistic predictions by assuming model errors [20]. The quantile regression method [21], on the other hand, considers only the relationship between the quantile and the impact factor. The NGboost method uses natural gradients to achieve probabilistic predictions [22]. Usually, more than one parameter of the probability distribution is predicted. Conventionally, a single-leaf node determines all parameters simultaneously, and different parameters construct mutually independent groups of decision trees [23]. This approach will cause the classification to fall into suboptimal solutions. Compared with traditional gradients, NGBoost uses natural gradients multiplied by the inverse of the Riemann measure for “pre-scaled”, which coordinates the relationship between multiple parameters and allows global line search for multi-parameter prediction of probability distributions, thus making the gradient boosting algorithm capable of probabilistic prediction. The method directly obtains the probability distribution of the forecast values, avoiding the complicated process of obtaining the deterministic prediction and then converting them to probabilistic prediction by other methods.

The NGboost method has been applied in various fields of prediction problems because of its accuracy and direct access to the probability distribution of the predicted values [24]. In the energy field, the method has been applied to the probabilistic prediction of wind power [25]. In biomedical sciences, the method is applied to the probabilistic prediction of treatment frequency [26]. In the field of meteorology, the method is applied to probabilistic prediction of temperature [27]. In environmental sciences, the method is applied to the probabilistic prediction of CO₂ emissions [28]. The application of the NGboost method in different fields shows that the method has broad applications in probabilistic prediction and can be applied in the probabilistic prediction of runoff.

Before building a prediction model using the NGboost method, there are some hyperparameters to be set. The selection of these hyperparameters significantly affects the accuracy and reliability of the model predictions. The main methods for selecting hyperparameters are grid search, random search, intelligent algorithms, Bayesian methods, etc. Grid search is an enumeration method that tries to find the optimum for all combinations within a given range of hyperparameter values, which requires a lot of computational resources and is therefore difficult to use when there are many parameters and a large range of values [29]. Random search attempts random combinations of parameters within the range of hyperparameter values; this method is fast, but it is difficult to guarantee the results. Intelligent algorithms are complex and computationally intensive to build models [30]. The Bayesian method, on the other hand, obtains a better combination of hyperparameters with a smaller number of trials by tracking the results of each evaluation [31]. The TPE algorithm [32] has performed well as a Bayesian method for a variety of model hyperparameter optimization applications [33], such as model hyperparameter optimization for the prediction of energy consumption of HVAC systems [34] and the prediction of steam generation data from nuclear power plants [35]. In this study, the TPE algorithm was used for the hyperparameter optimization of the model to improve the prediction [36].

2. Materials and Methods

2.1. Study Area and Data

Two stations in the upper Yangtze River were selected for the study. As the longest river in China and the third longest river in the world, the Yangtze River has a basin area of 1.8 million km², accounting for about one-fifth of China’s total land area. The Yichang station is the boundary between the upper and middle reaches of the Yangtze River, and the Pingshan station is the main control station for the upper reaches of the Yangtze River. The two stations have different characteristics in terms of runoff. The runoff data used for the study are monthly, weekly and daily runoff data from January 1940 to April 2021. The former 80% of the data is the training set and the latter 20% is the test set. Figure 1 shows the location of the Yichang and Pingshan stations in the Yangtze River basin.

Table 1. Basic statistical information of the runoff data from the Yichang station.

Time Scale	Mean 1	Minimum 1	Maximum 1	Standard Deviation 1	Autocorrelation 2
monthly	13,643	3058	52,168	9745	0.7554
weekly	13,706	2824	63,714	10,381	0.9038
daily	13,707	2470	79,881	10,665	0.9852

¹ The unit is m³/s. ² The value is the result of the first-order autocorrelation function.

and

Table 2. Basic statistical information of the runoff data from the Pingshan station.

Time scale	Mean 1	Minimum 1	Maximum 1	Standard Deviation 1	Autocorrelation 2
monthly	4499	900	19,448	3592	0.7516
weekly	4520	770	26,971	3793	0.9317
daily	4520	640	28,600	3847	0.9926

¹ The unit is m³/s. ² The value is the result of the first-order autocorrelation function.

show the basic statistical information of the runoff data at each scale for the Yichang and Pingshan stations, respectively.

2.2. Methods

2.2.1. Natural Gradient Boosting (NGboost)

Natural Gradient Boosting (NGBoost) is a method proposed by researchers at Stanford University that uses gradient boosting methods for probabilistic prediction. The method is based on the gradient boosting decision tree, which uses natural gradients instead of general gradients to overcome the influence brought by parameters on convergence [37,38]. Each base learner of the model fits the natural gradient, and after a combination of deflation and additivity, an integrated model is obtained to fit the parameters of the conditional distribution for the purpose of probabilistic prediction.

This study uses the Maximum Loss Expectancy (MLE) scoring rule [39,40] as the loss function during model training with the following equation:

$$ℒ(\theta ,y)=-\log P_{\theta }(y),$$

(1)

where $\theta$ is the parameter of the distribution. We assume that there are suitable scoring rules $\mathcal{S}$, predicted probability distribution $P$, true probability distribution $Q$ and observations $y$. The scores $\mathcal{S}\left(P,y\right)$, $\mathcal{S}\left(Q,y\right)$ should satisfy:

$$E_{y~Q}[\mathcal{S}(Q,y)]⩽E_{y~Q}[\mathcal{S}(P,y)]\forall P,Q.$$

(2)

The excess of the right-hand side of the inequality over the left-hand side can be interpreted as a measure of the difference between distributions Q and P, as the divergence induced by this scoring rule. The divergence is given by:

$$D_{\mathcal{S}}(Q\Vert P)=E_{y~Q}[\mathcal{S}(P,y)]-E_{y~Q}[\mathcal{S}(Q,y)].$$

(3)

For the MLE scoring rule, the derived divergence is the Kullback–Leibler divergence ( $D_{KL}$):

$$\begin{array}{cc}\hfill D_{ℒ}(Q\Vert P)& =E_{y~Q}[ℒ(P,y)]-E_{y~Q}[ℒ(Q,y)]\hfill \\ & =E_{y~Q}\left[\log \frac{Q(y)}{P(y)}\right]\hfill \\ & \doteq D_{KL}(Q\Vert P)\hfill \end{array}$$

(4)

The natural gradient was originally defined for the statistical manifold with the distance measure induced by $D_{KL}$. Duan et al. provide a more general treatment that makes it applicable to the divergence of some scoring rules. The generalized natural gradient is the direction of the steepest ascent in Riemannian space, which is invariant to parametrization and defined as:

$$\tilde{\nabla }ℒ(\theta ,y)\propto \underset{ϵ\to 0}{\lim }\underset{d:D_{ℒ}\left(P_{\theta }\Vert P_{\theta +d}\right)=ϵ}{\arg \max }ℒ(\theta +d,y).$$

(5)

By solving the corresponding optimization problem, the natural gradient can be obtained as:

$$\tilde{\nabla }ℒ(\theta ,y)\propto {ℐ}_{ℒ}{(\theta )}^{-1}\nabla ℒ(\theta ,y),$$

(6)

where ${ℐ}_{ℒ}(\theta )$ is the Fisher information from the observations on $P_{\theta }$, which is defined as:

$$\begin{array}{cc}\hfill {ℐ}_{ℒ}(\theta )& =E_{y~P_{\theta }}\left[{\nabla }_{\theta }ℒ(\theta ,y){\nabla }_{\theta }ℒ{(\theta ,y)}^T\right]\hfill \\ & =E_{y~P_{\theta }}\left[-{\nabla }_{\theta }^2ℒ(\theta ,y)\right]\hfill \end{array}$$

(7)

Gradient boosting is a supervised learning method. In this method, several base learners are combined and integrated using an additive approach. The model is learned sequentially, with the next base learner fitting the residuals of the currently integrated training target. The output of the fitted base learners is then scaled according to the learning rate and added to the integrated model. The NGboost method improves on the original method. For multi-parameter estimation, the original method using gradient segmentation based on one parameter may be suboptimal relative to the gradient of another parameter. In contrast to the conventional gradient, the natural gradient is multiplied by the inverse of the Riemann metric for “pre-scaled” to reconcile multiple parameters, thus allowing the algorithm to perform multi-parameter boosting. The flow chart of the NGBoost method for runoff probability prediction is shown in Figure 2.

2.2.2. Probability Distribution

A common probability distribution can usually be expressed by a probability distribution function. The parameters of the probability distribution function uniquely determine this probability distribution. The proposed model obtains the probability distribution of the predicted values by predicting the parameters of the probability distribution function. Different probability distributions have different parameters, so the type of probability distribution must be predetermined before building the prediction model. Three common forms of probability distributions, namely normal, lognormal and exponential, are tried in the study to investigate the prediction effects of each distribution function at different runoff characteristics and time scales. The probability density function of the normal distribution is determined by two parameters. Parameter $\mu$ is the mean, and parameter $\sigma$ is the standard deviation. The probability density function of the lognormal distribution is determined by two parameters. Parameter $\sigma$ is the standard deviation, and parameter $s$ is the shape parameter. The probability density function of the exponential distribution is determined by one parameter. Parameter $\sigma$ is the standard deviation.

2.2.3. Tree-Structured Parzen Estimator (TPE)

Tree-structured parzen estimator (TPE) is a method based on the Bayesian approach to tune the hyperparameters of models. The Gaussian process-based approach directly simulates $p(y|\theta )$, and the TPE strategy simulates $p(\theta |y)$ and $p(y)$ to indirectly obtain $p(y|\theta )$. The improvement in the model after choosing a new set of hyperparameters is given by:

$$I(\theta )=\max \left(y^{*}-y(\theta ),0\right),$$

(8)

where $\theta$ is the hyperparameter of the model, $y$ is the loss function of the model and $y^{*}$ is the threshold value of the loss function selected based on the data. The expectation of the improvement is:

$$\begin{array}{cc}\hfill E{I}_{y^{*}}(\theta )& ={\displaystyle {\int }_{-\infty }^{y^{*}}\left(y^{*}-y\right)}p(y\mid \theta )dy\hfill \\ & ={\displaystyle {\int }_{-\infty }^{y^{*}}\left(y^{*}-y\right)}\frac{p(\theta \mid y)p(y)}{p(\theta )}dy\hfill \end{array}$$

(9)

where $p(\theta |y)$ is defined as:

$$p(\theta \mid y)=\{\begin{array}{ll}l(\theta )\hfill & \mathrm{if}y<y^{*}\hfill \\ g(\theta )\hfill & \mathrm{if}y\ge y^{*}\hfill \end{array},$$

(10)

where $l(\theta )$ is the probability density formed by the set of loss values less than $y^{*}$, and $g(\theta )$ is the probability density formed by the remaining set. By constructing

$$\gamma =p\left(y<y^{*}\right)$$

(11)

and

$$p(\theta )={\displaystyle {\int }_{ℝ}p}(\theta \mid y)p(y)dy=\gamma l(\theta )+(1-\gamma )g(\theta ),$$

(12)

we can obtain:

$$\begin{array}{cc}\hfill {\displaystyle {\int }_{-\infty }^{y^{*}}\left(y^{*}-y\right)}p(\theta \mid y)p(y)dy& =\ell (x){\displaystyle {\int }_{-\infty }^{y^{*}}\left(y^{*}-y\right)}p(y)dy\hfill \\ & =\gamma y^{*}l(\theta )-l(\theta ){\displaystyle {\int }_{-\infty }^{y^{*}}p}(y)dy\hfill \end{array}$$

(13)

Furthermore, it can be introduced:

$$E{I}_{y^{*}}(x)=\frac{\gamma y^{*}l(\theta )-l(\theta ){\displaystyle {\int }_{-\infty }^{y^{*}}p}(y)dy}{{\gamma }^{l(\theta )}+(1-\gamma )g(x)}\propto {\left(\gamma +\frac{g(x)}{l(x)}(1-\gamma )\right)}^{-1}.$$

(14)

From Equation (14), it can be concluded that maximizing the improvement of the hyperparameters means the probability of $l(\theta )$ is high, while the probability of $g(\theta )$ is as low as possible. The TPE algorithm evaluates the hyperparameters by $g(\theta )/l(\theta )$ to find the hyperparameters ${\theta }^{*}$ with maximum EI. The flow chart of this algorithm is shown in Figure 3.

2.2.4. Performance Measures

To test the performance of the proposed model, some quantitative metrics are used to evaluate the prediction. For deterministic forecast results, three metrics are used: root mean square error (RMSE), mean relative error (MARE) and coefficient of determination (R²). The calculation methods are as follows.

$$\mathrm{RMSE}=\sqrt{\frac{1}{n}{\displaystyle \sum _{i=1}^n{\left(y_p^i-y_o^i\right)}^2}}.$$

(15)

$$\mathrm{MRE}=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left|\frac{y_p^i-y_o^i}{y_o^i}\right|}.$$

(16)

$${\mathrm{R}}^2=1-\frac{{\displaystyle \sum _{i=1}^n{\left(y_o^i-y_p^i\right)}^2}}{{\displaystyle \sum _{i=1}^n{\left(y_o^i-{\overline{y}}_o\right)}^2}},$$

(17)

where $n$ is the number of observations, $y_o^i$ is the observed runoff, ${\overline{y}}_o$ is the mean value of the observed runoff series and $y_p^i$ is the predicted runoff. To evaluate the prediction of runoff series with high autocorrelation, especially daily runoff series, the inertial root mean square error (IRMSE) [41,42] is introduced.

$$\mathrm{IRMSE}=\frac{\mathrm{RMSE}}{{\sigma }_{\Delta }},$$

(18)

where

$${\sigma }_{\Delta }=\sqrt{\frac{{\displaystyle \sum _{i=1}^n{\left({\Delta }_i-\overline{\Delta }\right)}^2}}{n}},$$

(19)

$${\Delta }_i=y_o^i-y_o^{i-1},$$

(20)

$$\overline{\Delta }=\frac{{\displaystyle \sum _{i=1}^n{\Delta }_i}}{n}.$$

(21)

where ${\Delta }_i$ is the first-order lagged observed series, $\overline{\Delta }$ is the mean value of ${\Delta }_i$ and ${\sigma }_{\Delta }$ is the standard deviation of ${\Delta }_i$.

The three metrics used to evaluate the probabilistic forecast results are interval coverage probability (ICP), interval normalized averaged width (INAW) and coverage width-based criterion (CWC). The metrics for the probabilistic forecast test are calculated as follows.

$$\mathrm{ICP}=\frac{1}{n}{\displaystyle \sum _{i=1}^n{\epsilon }_i}\left({\epsilon }_i=\{\begin{array}{ll}1,\hfill & y_o^i\in \left[y_{low},y_{up}\right]\hfill \\ 0,\hfill & y_o^i\notin \left[y_{low},y_{up}\right]\hfill \end{array}\right).$$

(22)

$$\mathrm{INAW}=\frac{1}{nR}{\displaystyle \sum _{i=1}^n\left(y_{up}-y_{low}\right)}.$$

(23)

$$\begin{array}{c}\mathrm{CWC}=\mathrm{INAW}\left(1+\gamma e^{-\eta (\mathrm{ICP}-\mu )}\right),\eta >0\\ \gamma =\{\begin{array}{l}0,\mathrm{ICP}⩾\mu \hfill \\ 1,\mathrm{ICP}<\mu \hfill \end{array}\end{array},$$

(24)

where ${\epsilon }_i$ is a Boolean variable with a value of 1 when the observation is within the prediction interval and 0 otherwise. $y_{up}$ and $y_{low}$ are the upper and lower bounds of the interval at the specified confidence level, respectively. $R=\max y_o-\min y_o$ denotes the range of the test set, which is used to normalize the evaluation metrics. $\eta$ and $\mu$ are the parameters that determine the degree of penalty. $\mu$ is determined by the confidence interval, and $\eta$ is a parameter greater than 0, which is taken as 1 in this study. ICP indicates the degree of coverage of the prediction interval on the observed values, and the value ranges from 0 to 1. Values closer to 1 indicate the better coverage of the prediction interval. However, when the prediction interval is too wide, although it can completely cover the observed values, it loses the value of probability prediction. INAW indicates the relative width of the prediction interval and takes a value between 0 and 1. Values closer to 0 indicate that when the relative width of the prediction interval is narrower, the better the prediction effect. CWC is a comprehensive index based on ICP and INAW, which reflects the overall effect of probability prediction.

To measure the prediction effectiveness of the models, both Support Vector Machine (SVM) and Extreme Gradient Boosting (XGboost) are used to train on the same data set and validate their prediction effectiveness on the test set. These two models are introduced as benchmark models for comparative evaluation of model prediction.

2.2.5. Runoff Prediction Model

In this study, the NGboost method is introduced to establish the runoff prediction model, which can obtain both deterministic and probabilistic prediction results of the predicted values. The input of the model is the historical runoff series. The number of timesteps of previous observations in the input is treated as one of the hyperparameters of the model, along with the other four hyperparameters in the NGBoost method, which are the depth of the decision tree base learner, the number of base learners, the learning rate and the percentage of subsample used in the model training. The hyperparameters are determined by the TPE algorithm optimization. The output of the model is the parameters of the probability density function of the assumed probability distribution. The output of the normal distribution is the mean $\mu$ and the standard deviation $\sigma$. The lognormal distribution is the standard deviation $\sigma$ and the shape parameter $s$. The exponential distribution is the standard deviation $\sigma$. The model corresponding to each time scale and each probability distribution is optimized by 100 rounds of TPE iterations according to the experimental and computational power limitations. The combination of hyperparameters with the best results is preferred to build the runoff prediction model for this situation. The proposed model is a single-step prediction model that predicts the probability distribution of runoff in the next timestep. The prediction effects are tested for normal, lognormal and exponential distributions. This provides guidance for the development of runoff prediction models at different time scales for different flow characteristics. Figure 4 shows the flow chart of the proposed model.

3. Results

3.1. Hyperparameter Tuning

The prediction model uses the TPE algorithm to preferably select the model hyperparameters. The hyperparameters of the model are the best combination of predictions after 100 rounds of iterations of the TPE algorithm. The optimal model hyperparameter combinations for the three distributions at each time scale at each station are shown in

Table 3. Optimal hyperparameters of the prediction model for each time scale of two stations.

Station	Time Scale	Distributions	Number of Timesteps	Base Learner Depth	Number of Learners	Learning Rate	Percent of Subsample
Yichang	Monthly	Normal	12	3	900	0.03722	0.6
		LogNormal	12	3	1100	0.01003	0.5
		Exponential	12	4	1100	0.00577	0.6
	Weekly	Normal	16	3	500	0.00568	0.5
		LogNormal	15	2	1200	0.00346	0.9
		Exponential	15	2	1400	0.00372	0.5
	Daily	Normal	16	3	1100	0.0068	0.7
		LogNormal	27	2	900	0.00864	1
		Exponential	23	2	700	0.01962	0.8
Pingshan	Monthly	Normal	12	5	800	0.01198	0.5
		LogNormal	12	5	900	0.05111	0.7
		Exponential	12	4	1400	0.01632	0.7
	Weekly	Normal	13	3	1200	0.00384	0.5
		LogNormal	13	2	800	0.01528	0.7
		Exponential	14	2	900	0.02597	0.5
	Daily	Normal	15	4	1500	0.01235	0.9
		LogNormal	15	2	700	0.01047	1
		Exponential	21	2	1100	0.0125	0.8

.

Runoff probability prediction models are developed separately according to each optimal combination of hyperparameters. Models are trained in the corresponding training sets, and prediction is validated in the test set.

3.2. Deterministic Predictions

Figure 5 and Figure 6 show the predicted and actual runoff processes of each model on the test set data at monthly, weekly and daily scales for the Yichang and Pingshan stations. The trends of predicted data and actual data of each model are in agreement. The predictions are improved from monthly and weekly to daily, in order. The results are consistent with the perception that the correlation of adjacent runoff data increases sequentially from month and week to day. It can also be seen that the model prediction errors for the extreme values are large at each scale.

As seen in

Table 4. Statistical results of RMSE, MARE, R2 and IRMSE on the test data set for two benchmark models, SVM and XGboost, and three NGboost prediction models with normal (NGboost-Normal), lognormal (NGboost-LogNormal) and exponential (NGboost-Exponential) distributions, under monthly, weekly and daily scales at the Yichang station. The best performing model for each metric is in bold.

Time Scale	Models	RMSE	MRE	R2	IRMSE
Monthly	SVM	8612.16	0.5501	−0.0625	0.9231
	Xgboost	4518.05	0.2036	0.7076	0.4843
	Normal	4138.10	0.1863	0.7547	0.4436
	LogNormal	4035.07	0.1703	0.7668	0.4325
	Exponential	4019.67	0.1747	0.7685	0.4309
Weekly	SVM	9050.41	0.5176	−0.0274	1.6681
	Xgboost	4387.76	0.1905	0.7585	0.8087
	Normal	3685.47	0.1488	0.8296	0.6793
	LogNormal	3654.54	0.1473	0.8325	0.6736
	Exponential	3632.26	0.1468	0.8345	0.6695
Daily	SVM	6958.66	0.2143	0.4358	1.9935
	Xgboost	2316.02	0.0895	0.9375	0.6635
	Normal	2103.43	0.0771	0.9484	0.6026
	LogNormal	2097.01	0.0774	0.9488	0.6007
	Exponential	2099.29	0.0777	0.9486	0.6014

, the NGboost model shows a significant improvement in all prediction effectiveness metrics compared to the two benchmark models. The metrics are closer between the three NGboost models using different distributions. In the monthly and weekly scale runoff prediction tests, the NGboost model with exponential distribution has the best prediction. According to the results of the monthly runoff prediction, the NGboost-Exponential model reduced RMSE by 11.03%, reduced MRE by 14.23% and improved R² by 8.61% compared to the XGboost model. On the weekly scale, RMSE and MRE decreased by 17.22% and 22.91%, respectively, and R² improved by 10.01%. In the daily runoff prediction tests, the NGboost model using lognormal distribution has the best prediction results. Compared to the XGboost benchmark model, the NGboost-LogNormal model has a 9.46% reduction in RMSE, a 13.56% reduction in MRE and a 1.2% improvement in R². For daily runoff with high autocorrelation, the IRMSE metrics of three NGBoost models are around 0.6, and the predictions are fair.

Table 5. Statistical results of the benchmark model and the NGboost prediction model with different distributions in the test data set at each scale at the Pingshan station. The best performing model for each metric is in bold.

Time Scale	Models	RMSE	MRE	R2	IRMSE
Monthly	SVM	3184.43	0.4586	−0.1542	0.9276
	Xgboost	1349.63	0.1934	0.7927	0.3931
	Normal	1232.74	0.1915	0.8270	0.3591
	LogNormal	1228.67	0.1810	0.8282	0.3579
	Exponential	1239.29	0.1832	0.8252	0.3610
Weekly	SVM	3172.36	0.3825	−0.0220	1.8276
	Xgboost	1205.08	0.1620	0.8525	0.6942
	Normal	1068.97	0.1405	0.8840	0.6158
	LogNormal	1058.50	0.1357	0.8862	0.6098
	Exponential	1063.74	0.1378	0.8851	0.6128
Daily	SVM	1596.71	0.1514	0.7511	2.0013
	Xgboost	503.49	0.0687	0.9753	0.6311
	Normal	467.98	0.0644	0.9786	0.5866
	LogNormal	475.35	0.0642	0.9779	0.5958
	Exponential	473.28	0.0641	0.9781	0.5932

shows the statistical results in the test data set at each scale at the Pingshan station. Similar to the Yichang station, the NGboost model significantly improved each metric compared to the predicted effects of the two benchmark models, while the metrics are closer between the three NGboost models using different distributions. In the monthly and weekly scale runoff prediction tests, the NGboost model using the lognormal distribution predicted the best results. According to the results of the monthly runoff prediction, the NGboost-LogNorm model has an 8.96% decrease in RMSE, a 6.43% decrease in MRE and a 4.48% increase in R² compared to the XGboost baseline model. On the weekly scale, RMSE and MRE were reduced by 12.16% and 16.26%, respectively, and R² was improved by 3.95%. In the daily runoff prediction tests, the NGboost model using a normal distribution has the best prediction results. Compared to the XGboost model, the NGboost-Normal model has a 7.05% reduction in RMSE, a 6.24% reduction in MRE and a 0.35% improvement in R². Regarding the daily runoff with high autocorrelation, the IRMSE metrics of three NGBoost models are about 0.59, and the predictions are fair.

3.3. Probabilistic Predictions

The data in

Table 6. Statistical results of ICP, INAW and CWC for the three NGboost prediction models with normal, lognormal and exponential distributions on the test data set at monthly, weekly and daily scales for Yichang station, respectively. The best performing model for each metric is in bold.

Time Scale	Distributions	ICP	INAW	CWC
Monthly	Normal	0.4388	0.0742	0.1919
	LogNormal	0.6582	0.0962	0.1897
	Exponential	1.0000	1.0108	1.0108
Weekly	Normal	0.9293	0.1649	0.1649
	LogNormal	0.8834	0.1562	0.3151
	Exponential	1.0000	0.6858	0.6858
Daily	Normal	0.7842	0.0428	0.0899
	LogNormal	0.7650	0.0419	0.0908
	Exponential	1.0000	0.4965	0.4965

are metrics of probabilistic prediction results for the three models at a confidence level of 90%. As can be seen from the metrics of each time scale, although the ICP of the exponential distribution model is 1, its prediction interval is too wide, and the INAW and CWC metrics far exceed the acceptable thresholds for poor prediction. In the monthly runoff probability prediction test, the NGboost model with lognormal distribution performs better, with a 50% improvement in ICP and a 1.14% decrease in CWC compared to the model with normal distribution. On the weekly and daily scales, the predictions using the normal distribution model are better than those using the lognormal distribution. On the weekly scale, compared to the NGboost-LogNormal model, the ICP of the NGboost-Normal model improved by 5.2% and the CWC decreased by 47.67%. On a daily scale, ICP improved by 2.51% and CWC decreased by 0.99% compared to the NGboost-LogNormal model.

Table 7. Statistic results of probabilistic metrics on for models at each time scale at the Pingshan station. The best performing model for each metric is in bold.

Time Scale	Distributions	ICP	INAW	CWC
Monthly	Normal	0.3214	0.0521	0.1451
	LogNormal	0.0204	0.0060	0.0205
	Exponential	1.0000	0.9307	0.9307
Weekly	Normal	0.7456	0.1315	0.2849
	LogNormal	0.6749	0.1300	0.2929
	Exponential	1.0000	0.7808	0.7808
Daily	Normal	0.6876	0.0374	0.0837
	LogNormal	0.7138	0.0403	0.0888
	Exponential	1.0000	0.6108	0.6108

, similar to Table 6, shows the statistics of the runoff probability prediction result metrics for each model in the Pingshan station test data set on different scales. In the monthly runoff probability prediction tests, both models with normal and lognormal distributions had ICP metrics that are significantly lower than the confidence level, and both predictions were poor. As in the case of the Yichang station, the prediction of the model with normal distribution is better than that of the model with lognormal distribution at weekly and daily scales. On the weekly scale, compared to the NGboost-LogNormal model, the ICP of the NGboost-Normal model improved by 10.47% and the CWC decreased by 2.73%. On the daily scale, INAW and CWC decreased by 7.05% and 5.7%, respectively, compared to the NGboost-LogNormal model.

In order to further compare the probability prediction performance of each prediction model on the test data, the 80%, 85% and 90% confidence interval runoff prediction processes are plotted based on the probability prediction results of three NGboost models with normal, lognormal and exponential distributions on monthly, weekly and daily scales, respectively, using the first year of the test data set, i.e., 2005, at Yichang station as an example. Figure 7, Figure 8 and Figure 9 show the runoff prediction process on the monthly, weekly and daily scales, respectively. As can be seen from the figure, the prediction interval of the model using the exponential distribution is too wide. The prediction intervals of the prediction models with normal and lognormal distributions can cover the test data better, especially in the case of weekly and daily time scales. The large prediction errors occur mainly in areas with large fluctuations in the runoff process and extreme points.

Overall, the evaluation metrics of the NGboost model prediction results on the test data set are significantly improved compared to the benchmark model. The optimization of model hyperparameters and the selection of probability distributions for different runoff characteristics and time scales are further discussed in the following sections of the paper.

4. Discussion

The above calculation results show that the different runoff characteristics of the stations have significant effects on the optimization of the model hyperparameters and the selection of the probability distribution.

Figure 10a shows the hyperparameter tuning results of the NGboost model with normal distribution under the monthly runoff at the Yichang station. As can be noticed from the figure, there are more dark lines when the number of timesteps of previous observations in the input is 10 or larger, especially 12, indicating that the models with a greater number of timesteps of previous observations perform better. The depth of the decision tree base learner has less effect on the model, and the model with a depth of 3 is slightly better. The models with the number of base learners of around 900 performed better. Models with learning rates between 0.02 and 0.04 performed better. The percentages of subsamples in the model training of the more effective models were mainly distributed at 50%, 60% and 70%.

Figure 10b plots the hyperparameter tuning results of the NGboost model with lognormal distribution under the weekly runoff of the Pingshan station. In contrast to the Yichang station, the depth of the decision tree base learner has a significant effect on the model in this condition, and the model with a depth of 2 performs better. The percentages of subsamples used in the model training of the more effective models were mainly distributed at 70%, 80% and 90%.

According to the tuning results of the hyperparameters of each model, there are some recommendations for the selection of the hyperparameters of the NGboost model for runoff prediction in similar runoff situations.

• In the range of values allowed by the experimental arithmetic, the greater the number of timesteps of previous observations in the input, the better the performance;
• The depth of the decision tree base learner needs to be optimally selected for different situations;
• The number of base learners can be selected between 800 and 1200;
• The learning rate between 0.001 and 0.03 is better; and
• The percentage of subsamples used in the model training has little effect on the prediction of the model.

The two stations selected for the study have different runoff characteristics. The average flow at the Yichang station exceeded 13,000 m³/s, with large flows but small variance and a relatively stable runoff process. In terms of deterministic prediction, the NGboost model using different distributions predicts closer results. The models with exponential and lognormal distributions predicted slightly better. The reason is that the runoff process at the Yichang station is stable and less random, in accordance with the characteristics of the skewed distribution. The average flow at the Pingshan station is 4500 m³/s, with a small flow but high variance and fluctuations in the runoff process. Due to the large randomness of runoff, the model with normal distribution predicts better.

The study also developed prediction models for each of the three different time scales of monthly, weekly and daily runoff prediction problems. For deterministic prediction, the model with exponential distribution predicts the best results on the monthly and weekly scales at the Yichang station. The model with lognormal distribution predicts better results on the daily scale. For the Pingshan station, the model with lognormal distribution predicted the best results on the monthly and weekly scales, while the model with normal distribution performed better on the daily scale. In terms of probabilistic prediction, all models perform poorly for the probabilistic prediction of runoff on the monthly scale due to the poor correlation of data on the adjacent time periods on the monthly scale, and there is room for improvement. Of the three time scales, the model predicts the best on weekly runoff data. On the monthly scale, the model with a lognormal distribution predicts the best. On the weekly and daily scales, the model with a normal distribution has the best prediction. As the time scale becomes smaller, the randomness of runoff becomes stronger, so more predicted runoff data conform to the characteristics of normal distribution.

5. Conclusions

Accurate runoff prediction data represent an essential foundation for other studies in the field of water resources. Many hydrological and data-driven models have been applied to predict runoff. These models often require a large amount of relevant data as input to the model. However, not all regions have complete data that meet the requirements of complex models. At the same time, an accurate prediction is difficult to obtain, and it is difficult for deterministic prediction results to meet needs such as risk assessment. In this study, the NGboost model was introduced into the runoff prediction. The model uses only the historical runoff series itself as the model input and can obtain the probability distribution of the data to be predicted. The study also used the TPE algorithm to optimize the hyperparameters of the NGboost model. The runoff probability prediction models are developed and optimized for three time scales of monthly, weekly and daily and three probability distributions of exponential, normal and lognormal distributions. The model was applied to the prediction of runoff at the Yichang and Pingshan stations in the upper reaches of the Yangtze River and achieved satisfactory prediction results. The results were obtained with the following process:

• Apply the NGboost model to the deterministic and probabilistic prediction of runoff at monthly, weekly and daily scales, and achieve better predictions;
• Use the TPE algorithm to optimize the prediction model hyperparameters, improve the model prediction effect and summarize some recommendations of prediction model hyperparameter tuning; and
• Analyze the prediction models with normal, lognormal and exponential distributions for different runoff characteristics and different time scales, and summarize the recommended distributions in different conditions.

It can be noted in the results that the NGBoost model had suitable performance in deterministic and probabilistic runoff prediction. However, due to the limitations of the model and the complexity of the actual runoff, there are still some questions that deserve further investigation.

• The prediction accuracy is not sufficient for high flow cases, especially at extreme values. In the prediction results of each time scale, it can be seen that there is more room to improve the model prediction accuracy in high flow cases. Some methods of extreme value analysis can be applied in subsequent work to deal specifically with the high flow case.
• More probability distribution functions can be introduced to participate in the test. In this study, three probability distributions—normal distribution, lognormal distribution and exponential distribution—were tested. Subsequent studies can introduce more distribution forms to test the prediction effect of the model.

In general, the NGboost runoff prediction model optimized by the TPE method can obtain better deterministic and probabilistic prediction results when only the runoff series itself is used as data input, which is superior compared to other models and worthy of further study.